How to distinguish between proxy and endpoint as source of connection timeout? - http

When I try to connect to a server (website) through a proxy, sometimes I get "Connect timeout" errors. How can I determine whether the failed connection was to the proxy or to the endpoint? I don't have access to logs on the proxy or endpoint.
My use case is evaluating proxies by accessing a proxy judge web page. It may be that the judge is overloaded. Or it may be that the proxy no longer works. If the judge is overloaded I want to back off, if the proxy is broken, I want to stop using it.

perl -MNet::HTTP -e 'Net::HTTP->new( Host => "HOSTNAME:PORT", Timeout => 5 ) && print "OK\n"'
This issues a CONNECT to the server at HOSTNAME:PORT and prints OK if the CONNECT succeeds.

Related

Nginx returning 499 error when browser closed before getting response for request

We are facing 499 error when we close the browser tab before getting response for the request. We are using nginx in k8s.
I have tried by configuring "proxy_ignore_client_abort: on" property in ingress configuration, still we are getting issue even after configuring the above property. Please suggest me way to fix this issue.
Firstly we are supposed to know that the nginx throw 499 if the client actively disconnected the connection. So it you may not pay much attention to it if everything is good.
Nginx could be the server to the user and the client to the backend server like the below:
from user->->nginx->server(tomcat).
In my case, I found that server like tomcat would abort the connection if it cannot handle too many requests in the accepted list.(or too slow to respond).
In tcp, the real server like tomcat would maintain 2 list. The first 1 is SYN list, and the 2nd is accepted list. Pls let me ellaborate it:
Clients firstly send syn to the server.
and the server put it into syn list and return SYN+ACK.
Client send the ACK to the server.
Finally the server established the connection after removing it from the syn list and put it into the accepted list.
In your case, if you close the tab before step2, I think you needn't do anything at all.
if you close the tab before the tab 4, you can refactor the interface of your server to be async to greatly enhance its responding speed.

What is the best way to redirect network requests?

I've written my own HTTP Server, but given certain criteria, I want to redirect some requests made to my server to another server running on the same machine. For example, I may want to redirect all requests to "/foo/*" to be handled by an apache server I also have running. What is the best way to do this?
The only way I can think of doing this is by running apache on a different port, and then making a completely new network request from my server to localhost:1234 (assuming apache is running on port 1234) with the same exact request headers and body, and then take the response and have my server send that back to the client.
That seems like a kind of hacky, roundabout way of accomplishing this though, and I'm sure this is a problem that is tackled by every major website. Is there a certain technology or protocol for doing this that I just haven't heard of?
Thanks a lot!
Edit: Just to be clear, the client should only make one network request for all this, rather than having my server return a 3xx response
HTTP runs over TCP. The Apache server can't just send the required response to a client who hasn't asked for it. The client has asked YOUR HTTP server for the data and so it must be the one to send a response. The client is probably behind a firewall and, as such, the Apache server can't even establish a TCP connection with it (incoming connections are usually blocked).
If your server takes the clients request, forwards it to the Apache server, gets the response from the Apache server and forwards it to the client, it's acting as a proxy server (a middleman). This won't be redirection.
The only sensible way to do this would be to have the client make two network requests.

Use JMeter HTTP Proxy to record JSON over HTTP request from not-a-browser client

I have a client program running locally in iPhone emulator and local server written in Java. Client talks to server with JSON over HTTP requests. Now I am trying to record a client session http requests with JMeter to use them as a base to load-test server.
The problem is client requests do not go throw JMeter proxy. Client gets 501 "Method not implemented" (it is not because of https, I am using http). There is nothing in JMeter log about the request, and obviously nothing gets recorded in JMeter and the request doesn't reach the server.
There is well-described steps to setup JMeter proxy to record request from a browser, but my client programm is not a browser. Though JSON over HTTP is widely adopted approach, I could not find anything on the web about recording such requests with JMeter. I understand I need to do on the client the same thing browser does when proxying request and what I've found about it is that I need to set Host header to server's host and port, but that did not work and I cannot see how is it related to 501 error client gets.
If someone can explain what should be done on client or how to configure JMeter to let it know where to proxy client's requests or link any manual explaining that, it would be great help. I've been searching for solution for a few hours already and had no luck. Please help.
You can see my question and the answer I got below:
Use Jmeter proxy to record HTTP calls from iOS simulator
In short:
With this tutorial you can record calls from your Android device:
http://blazemeter.com/blog/load-testing-mobile-apps-made-easy
For make same thing with iPhone, do the following steps:
Mac configuration:
system preferences -> Network -> Advanced.. -> Proxies -> check "Web Proxy (HTTP) ->in "Web Proxy Server" field, type your IP (http://www.wikihow.com/Find-Your-IP-Address-on-a-Mac), and choose available port (I using 8080) ->ok -> Apply
iPhone configuration:
Settings -> WiFi -> choose same wifi you use with your Mac -> press on it again to go to it's details -> scroll down ->In HTTP proxy, choose Manual -> server = your mac IP you found earlier -> port = the port you chosen (maybe 8080)
Now You can start recording all "iPhone network out" using jmeter recording controller

HTTP Connect via NTLM authenticating proxy server

I am trying to write Connect calls via a NTLM authenticating proxy server. I open a socket to the proxy server and send it a “CONNECT x.x.x.49:80 HTTP/1.1\r\n\r\n”. I expect this to fail as it is an authenticating proxy server but it fails and also closes my connection to the proxy server. I am using Wireshark to check packet values. I can see “Proxy-Connection: close\r\n”. If I do a GET it does not close the connection.
My question is if it is correct for a failed Connect call to close my socket connection?
I'm not 100% clear about your question - I would not expect the first CONNECT to fail with an NTLM proxy server, as it should have sent a Proxy-Authenticate: NTLM header back.
In any case, proxy server has the full right to close the client-side connection in a case of failure, so the answer to your question is "yes". It is odd that it does not do the same for GET, however proxies are more limited with what they can do as a response to CONNECT, and this one may decide that it can't support HTTPS for your request.

Checking for proxy connection

What do i need to send to a proxy server (http or socks) that will tell me if it is connected to the host i specified it to connect to.
Neither proxy protocol provides that functionality. There is no way to query a proxy to see what server it is connected to, or whether it is a still connected to a server. If you tell it to connect to a server, then you have to assume it is always connected to that server as long as you have a valid connection with the proxy. If the proxy loses its connection to the server, then it needs to close the connection with your client.

Resources