Cloud computing +Web scraping + VPN - web-scraping

I was trying to scrape a website using vpn, but my country "Egypt" blocks VPN, after alot of search to find a cloud computing service that allows me to use vpn on my cloud server to help me in webscraping i found nothing, does any know other solutions?

Related

Forward request from compute engine (GCP) to on-premise API

I'm trying to forward requests from a compute engine instance to an internal API (its an on-premise API that communicates to the VM through Cloud VPN) but without luck, I tried to research how to accomplish this, but with no results.
Someone knows how to achieve it?. In short my goal is to access the on-premise API from my local computer for development purpose using the CE as a proxy.
I am open to create more resources or to change the CE instance if there its a better way to handling this.
My VM is a simple Debian 10 machine.
As John Hanley commented. You can accomplish this by doing SSH tunneling using the CE instance as the middle man.
A good tutorial to follow is this one, which is from the GCP community itself.

Google cloud engine connection with Bucket socket and instance

I have a bucket in Google Cloud which I have upload Angular template i.e. http://digitaldevnet.appspot.com
then I have VM instance IP where I have WordPress website i.e.
http://35.200.194.201
I found different tutorials where you can connect domain to Google Cloud hosting
but I want to connect appspot link i.e. http://digitaldevnet.appspot.com to WordPress site
can be connected and once we browse it should work as VM may be offline sometime.
Any recommendation and tutorial, please let me know
You would need to set up a Load balancer in order to direct your traffic between the GCE instance and the bucket, you can find the instructions for this over here.
Nevertheless, it would be a bit complicated as you would need to set up the configuration and some health checks so that the GCE instance is not accessible when it's down.
Hope you find this useful.

Best practice for setting up a ‘private’ organisation network in the cloud

I am to setup some infrastructure for my company on google cloud. Admittedly I don’t have much experience with network and infrastructure.
I want to create some web apps to be deployed on VMs but I only want employees to be able to access the sites. What is the best practice to achieve this?
I could add login feature on the web apps but I’d rather keep the access control centralised instead. Can I setup a VPC that employees can logon to? Or is VPN the best solution here (although VPN is for connecting cloud with on-premise, right?)
I don't have a lot of expertise about GCE and network, but I saw some solution using Cloud IAP. Maybe this is better than create VPNs. Look this link, perhaps this could help you: https://cloud.google.com/iap/docs/tutorial-gce

Setup VPN for database instance on Google Cloud SQL

I have a MySQL database instance on Google Cloud SQL. Currently it has over 10 authorized ip addresses since multiple teams are accessing it from various locations. I would like to know if I can setup a VPN to this database instance and authorize just this ip address, instead of 10 addresses.
If that's possible, I would also like to know how many user accounts I can create for one VPN. I could not understand the Google Cloud documentation about setting up VPN. Please provide links to websites/tutorials/documentation that can help me with setting up a VPN in Google Cloud Platform.
Thanks.
Using cloud VPN and authorize only its external IP will not work for two reasons:
1) You can not specify in cloud SQL a private network (for example, 10.x.x.x) as an authorized network. as documented here.
2) Packets will arrive to cloud SQL after decapsulation which means that Cloud SQL get them as they come from different source IPs.
If you want more secure connection without IP white-listing, maybe using cloud SQL proxy.

ASP.NET on cloud server

My website is running quite well and serving lacks of pages on daily basis. We want to add one more web server to share load on server at heavy traffic times. Instead, can I go with Amazon Elastic Compute Cloud (EC2) or any other cloud service as an alternative solution.
In cloud server environment, need I to install multiple instances as traffic increases or a single instance can scale based on the traffic?
I would suggest Windows Azure
You're right in that you would have multiple instances that scale to meet traffic. With the Azure platform you would design your application into Roles (chunks of functionality), where they make sense to create multiples of, like a page that displays store contents should be highly scalable, whereas the login portion may not. Windows Azure already runs services for Microsoft like Xbox Live and their BPOS offerings, plus there are great tools to develop for the Azure Cloud. You can read more about cloud development at MSDN.

Resources