I am trying to use a different proxy server to do webs scraping in R. I am using use_proxy function in R, but no luck.
Please find the snippet of my code below:
GET("http://had.co.nz", use_proxy("202.40.185.107", 8080), verbose())
It is spittinig out the following error:
Error in curl::curl_fetch_memory(url, handle = handle) :
Timeout was reached: [had.co.nz] Connection timed out after 10000 milliseconds
Can anyone help me how I can change my proxy server in R to avoid getting blocked by the owner of the website. I thought the above method would be the easiest but not working for me. I would very much appreciate it if any web scraping wizard could give me a better idea to do this or how to fix this issue.
Thanks in advance!
To use a proxy, you need to be able to connect to it. Are you sure you can connect to the proxy server 202.40.185.107:8080? You can try that easily by e.g. putting 202.40.185.107:8080 in your browser or trying to ping 202.40.185.107:8080 using command line.
You could try a different proxy. I found this one online and it is free. Just a word of caution - if you are using a proxy in order not to be blocked by the website owner, the proxy you would be using can be blocked by the website owner as well.
GET("http://had.co.nz", use_proxy("35.169.156.54", 3128), verbose())
Related
I know this isn't exactly a programming issue. However, I'm trying to have some fun with friends and host a Minecraft server. Whenever I try to portforward using "ngrok tcp 25565" which always worked for me, it says "internal server error
ERR_NGROK_802"
Can anyone please help me with this?
Ngrok is down. It's not just you, it's our whole team.
I am new to IIS, setting up servers and load balancer so please do not mind my ignorance.
I have an old website, say, www.mywebsite.com.
There are 3 IIS 6.1 web servers on which my ASP.NET code is hosted.
Lets call them web01, web02 and web03.
They are identical in configuration like OS, etc.
There is a load balancer which handles the requests coming in from the clients through www and routes it to the respective server based on whatever algorithm is put in place.
However, we were able to send requests directly to individual servers like this web01.mywebsite.com for all 3 servers.
I do not have access to the load balancer.
Recently there were some problems with the web02 server so the client took it out from the load balancer.
We then reinstalled IIS and now, when we are trying to access the server using web02.mywebsite.com, it is throwing 404 - Not Found error.
I am able to open the website from inside the server using www.mywebsite.com and I am able to see that the requests is catered by the web02 server.
However, from inside the server also web02.mywebsite.com is throwing 404 - Not Found error.
I am sure I am missing something in configuration at the IIS level but not able to figure out.
I tried searching on the internet but they all gave me generic solutions.
Please help.
Thanks in advance!
I was able to get this resolved on my own.
Strangely there was not a single answer or comment to this question.
I will try to answer so that others might also benefit.
There were 2 things that were required, out of which 1 was already present in my case:
In the hosts file, the mapping of IP address of the server to the actual name of the server should be present (This was there in my case)
In IIS, under bindings, I removed the host name i.e. www.mywebsite.com.
I was able to figure this through the posts here and here although they did not give the answer directly.
Hope this helps others.
I'm getting this error on one of the ASP pages of our site.
msxml3.dll error '80072efe'
The connection with the server was terminated abnormally
The error refers to this line in the ASP, which essentially is just posting some info back into a page after the user selects an option from the dropdown:
xmlHTTP.Send strFormData
I searched for a solution to the error and saw several responses which I didn't really understand. Some suggested adding SNAT to the server, some talked about using Server.Transfer and others talked about it being a Firewall issue. [I tried reconfiguring the Firewall to allow all incoming TCP connections to all ports. Didn't solve the issue]
I'm sort of inexperienced when it comes to server issues so I don't wanna make any changes in server configs or production DB unless there's a clear answer/solution to this. I'm running MS SQL Server Management Studio Express 2005.
Any help on this will be highly appreciated. Let me know if you need any other information.
Thanks.
How do I get around this error
**The Requested FTP command is not supported when using HTTP Proxy**
occur on the following code
ftpWebRequest.Method = WebRequestMethods.Ftp.DeleteFile.
It is urgent. Can anyone give suggestions for this problem
Your proxy might be blocking FTP delete keywords when filtering requests.
Can you check if you are able to do this activity when not in the proxy (try from a different network where you connect directly).
if yes, i dont think it is straighforward for you to do this activity from behind proxy.
Another solution is to try open source ftps libraries http://ftps.codeplex.com/
or
1. Create a WCF service on the FTP server which allows you to do all that you want.
2. From client side make call to the WCF service passing the command.
i created a Myservice.asmx and it's working fine all the methods i am ble to invoke,giving me the correct but when i adding my MyService.asmx to my wwebsite as areferences giving me the following error.
There was an erroe in downloading 'http://localhost:49211/Myservice.asmx'
unable to connect to the remote server No connection could be made because the targer machine actively refused '127.0.0.1.49211'
There was an erroe in downloading 'http://localhost:49211/Myservice.asmx/$metadata/'
aAny Suggestions to solve this....Error..
"actively refused" often means that the connection simply couldn't get through and is usually a firewall / router issue to forward and allow the necessary ports through.