Use JMeter HTTP Proxy to record JSON over HTTP request from not-a-browser client - http

I have a client program running locally in iPhone emulator and local server written in Java. Client talks to server with JSON over HTTP requests. Now I am trying to record a client session http requests with JMeter to use them as a base to load-test server.
The problem is client requests do not go throw JMeter proxy. Client gets 501 "Method not implemented" (it is not because of https, I am using http). There is nothing in JMeter log about the request, and obviously nothing gets recorded in JMeter and the request doesn't reach the server.
There is well-described steps to setup JMeter proxy to record request from a browser, but my client programm is not a browser. Though JSON over HTTP is widely adopted approach, I could not find anything on the web about recording such requests with JMeter. I understand I need to do on the client the same thing browser does when proxying request and what I've found about it is that I need to set Host header to server's host and port, but that did not work and I cannot see how is it related to 501 error client gets.
If someone can explain what should be done on client or how to configure JMeter to let it know where to proxy client's requests or link any manual explaining that, it would be great help. I've been searching for solution for a few hours already and had no luck. Please help.

You can see my question and the answer I got below:
Use Jmeter proxy to record HTTP calls from iOS simulator
In short:
With this tutorial you can record calls from your Android device:
http://blazemeter.com/blog/load-testing-mobile-apps-made-easy
For make same thing with iPhone, do the following steps:
Mac configuration:
system preferences -> Network -> Advanced.. -> Proxies -> check "Web Proxy (HTTP) ->in "Web Proxy Server" field, type your IP (http://www.wikihow.com/Find-Your-IP-Address-on-a-Mac), and choose available port (I using 8080) ->ok -> Apply
iPhone configuration:
Settings -> WiFi -> choose same wifi you use with your Mac -> press on it again to go to it's details -> scroll down ->In HTTP proxy, choose Manual -> server = your mac IP you found earlier -> port = the port you chosen (maybe 8080)
Now You can start recording all "iPhone network out" using jmeter recording controller

Related

FiddlerCore: HTTP Tunnel Issue

I have an application, and I am trying to intercept its requests with FiddlerCore.
Using Fiddler, I see the requests as follows:
https://i.stack.imgur.com/bhUqK.png
(HTTP CONNECT tunnel into HTTPS request)
When using FiddlerCore, I am only seeing the initial HTTP CONNECT tunnel, and the application is not requesting. after that, stating:
Failed to connect to server.
This is the code I'm using in the BeforeRequest method:
if (oSession.HTTPMethodIs("CONNECT"))
{
oSession["x-replywithtunnel"] = "FakeTunnel";
return;
}
And the output from the FiddlerCore program (upon running the application):
https://i.stack.imgur.com/0Fc2q.png
The application is a Java application, which I had to create a keystore for using the FiddlerRoot certificate, and I am wondering if this is where the problem lies.
(Images as such since low reputation)
Solution: FiddlerCore uses a different SSL certificate than Fiddler itself.

Https communication on localhost in IIS using self-signed certificate

I have 2 sites running on the same machine, a client and an API.
Let's say the computer's IP is 10.10.10.10.
The API has a default page when you browse to it, the rest of the API is under 10.10.10.10/api.
The API has HTTP binding to port 80, and HTTPS binding to port 443.
The client has HTTP binding to port 8080, and HTTPS binding to port 64300.
Both HTTPS bindings use a self signed certificate I created via IIS manager.
Both sites have a HTTP to HTTPS redirect using "URL Rewrite".
When I try to browse either one of the apps, it works fine (gives the warning in the browser that you can skip).
When I do some action in the client which involves a HTTP request to the api using one of the following calls I get an error:
http://localhost/api/someMethod
http://localhost:80/api/someMethod
https://localhost/api/someMethod
https://localhost:443/api/someMethod
https://10.10.10.10/api/someMethod
The exception includes this error:
"The remote certificate is invalid according to the validation procedure"
I tried using the method described in this link (add the self-signed certificate to the Trusted Root Certificate Authorities folder) but it won't work.
Help please :D
found the answer.. posting if anyone else will get stuck on it.
It's pretty weird but the only thing that worked was to make the localhost http(s) request using the HOST NAME.
example:
https://the_name_of_the_computer:443/api/someMethod

Websockets situation - on port 80 or 443, websocket message doesn't go through

i'm having a problem with my app, on a certain situation.
We have a java server with jetty webserver embedded, and an air app on the client side.
It is working properly but on a single situation of a certain customer.
They have a private network that is not administrated by them (and has little chances of being changed as request). So, the only port they allow are 80 and 443.
The communications between the server and the client are through websockets and http.
The "online" check is made through http and, then, we use websockets to notify the client in order to start communication between them.
The thing is, in this situation, the "online" state works properly and any communication send by the client (forced), as it goes through http, gets to the server but, when the server communicates with the client, using websockets, it doesn't work.
We are using wireshark to check the communications: On a working setup, when the client app starts, a websocket is shown on wireshark, on the server side (registering the client on the server). And, after that, websockets that are only used from server to the client, don't show also.
What can be the problem? The port 80? (the same happens with 443 on that network).
Can it be a proxy/firewall that are blocking ws:// messages?
I've read somewhere that wss:// (encripted websockets) would work?
Thanks for your help.
Edit, so, I tried with https and wss communication and the same thing happens.. no websocket is set between the client and server (registering the client on the server).
This situation is happening for http on the customer network. On my test network, it works on http/ws but not with https/wss..
There are many firewalls and gateways out "in the wild" that do not understand the whole WebSocket HTTP/1.1 GET -> UPGRADE -> WebSocket mechanism.
There are several broken firewall implementations will attempt to interpret the WebSocket framing as improper content for HTTP/1.1 (which is a bad reading of the HTTP/1.1 spec) and start to muck with it.
The types of firewalls that inspect/filter/analyze the request/response contents are the ones that seem most susceptible.
I would check that the hardware (or software) that they are using to firewall their network is both compliant and upgraded to support WebSocket RFC-6455.

Proxy server basics

I'm learning about network programming. Specifically proxy servers. I've created a very rudimentary proxy server on my mobile phone. However I think there's some proxy server basics that I don't know that will help me create a more robust proxy server.
What I've done so far: server on my mobile device listens for requests from laptop. When server receives a request like www.google.com the web page contents are fetched and returned to the client on the laptop. The client then opens the page contents in a desktop browser.
I think the sending/receiving of requests can happen on a lower OSI model layer (perhaps transport). How can I create a more robust proxy server? (one that just sends and receives bytes and doesn't care/know about HTTP)
A proxy server runs at the same layer as the protocol being proxied. It seems you are talking about an HTTP proxy. HTTP runs over TCP, and so does an HTTP proxy.
Define 'more robust'. What have you done so far?
An HTTP proxy server is a pretty simple thing, unless it has elaborate logging, caching, etc. The basis of it is (1) something to recognize and action the GET/POST/PUT/CONNECT etc. commands and (2) thereafter just copying bytes in both directions simultaneously.

How do I monitor all incoming http requests?

I need to monitor my application from incoming http POST and GET requests originating from outside and sometimes inside the machine.
Is this possible?
Been using fiddler but this only does outgoing not incoming (from outside the machine) or have I configured it incorrectly?
This is for my web app that is meant to be receiving a POST from an external server.
What you need to do is configure Fiddler to work as a "reverse proxy"
There are instructions on 2 different ways you can do this on Fiddler's website. Here is a copy of the steps:
Step #0
Before either of the following options will work, you must enable other computers to connect to Fiddler. To do so, click Tools > Fiddler Options > Connections and tick the "Allow remote computers to connect" checkbox. Then close Fiddler.
Option #1: Configure Fiddler as a Reverse-Proxy
Fiddler can be configured so that any traffic sent to http://127.0.0.1:8888 is automatically sent to a different port on the same machine. To set this configuration:
Start REGEDIT
Create a new DWORD named ReverseProxyForPort inside HKCU\SOFTWARE\Microsoft\Fiddler2.
Set the DWORD to the local port you'd like to re-route inbound traffic to (generally port 80 for a standard HTTP server)
Restart Fiddler
Navigate your browser to http://127.0.0.1:8888
Option #2: Write a FiddlerScript rule
Alternatively, you can write a rule that does the same thing.
Say you're running a website on port 80 of a machine named WEBSERVER. You're connecting to the website using Internet Explorer Mobile Edition on a Windows SmartPhone device for which you cannot configure the web proxy. You want to capture the traffic from the phone and the server's response.
Start Fiddler on the WEBSERVER machine, running on the default port of 8888.
Click Tools | Fiddler Options, and ensure the "Allow remote clients to connect" checkbox is checked. Restart if needed.
Choose Rules | Customize Rules.
Inside the OnBeforeRequest handler, add a new line of code:
if (oSession.host.toLowerCase() == "webserver:8888") oSession.host = "webserver:80";
On the SmartPhone, navigate to http://webserver:8888
Requests from the SmartPhone will appear in Fiddler. The requests are forwarded from port 8888 to port 80 where the webserver is running. The responses are sent back through Fiddler to the SmartPhone, which has no idea that the content originally came from port 80.
I would install Microsoft Network Monitor, configure the tool so it would only see HTTP packets (filter the port) and start capturing packets.
You could download it here
Configure Fiddler as a 'reverse proxy' on Windows
(for Mac, see the link in Partizano's comment below)
I know there's already an answer suggesting this, however I want to provide the explanation and instructions for this that Telerik should have provided, and also cover some of the 'gotchas', so here goes:
What does it mean to configure Fiddler as a 'reverse proxy'?
By default, Fiddler only monitors outgoing requests from the machine on which you're running Fiddler
To monitor incoming requests, you need to configure Fiddler to work as a 'reverse proxy'
What this means is that you need to set Fiddler up as a 'proxy' that will intercept incoming http requests that are sent to a specific port (8888) on the machine where you want to listen to the incoming requests. Fiddler will then forward those requests to the web server on the same machine by sending them to the usual port for http requests (usually port 80 or 443 for https). It's actually very quick and easy to do!
The standard way to set this up with Fiddler is to get Fiddler to intercept all request sent to Port '8888' (since this won't normally be used by anything else, although you could just as easily use another port)
You then need to use the registry editor to get Fiddler to forward any http requests that Fiddler receives on port 8888, so that they're forwarded to the standard http port (port 80, port 443 for an https request, or another custom port that your web server is set to listen on)
NOTE: For this to work, any request you want to intercept must be sent to port 8888
You do this by appending :8888 to your hostname, for example like this for an MVC route:
http://myhostname:8888/controller/action
Walkthrough
Ensure Fiddler can accept remote http requests on port 8888:
Run Fiddler as administrator Go to Tools > Fiddler Options > Connections, and ensure that 'Allow remote computers to connect' is checked, and 'Fiddler listens on port' is set to 8888:
Configure Fiddler to forward requests received on port 8888 to port 80
Close Fiddler
Start REGEDIT
Create a new DWORD named ReverseProxyForPort inside HKEY_CURRENT_USER\SOFTWARE\Microsoft\Fiddler2.
Now set the DWORD value to the local port you'd like to re-route inbound traffic to (generally port 80 for a standard HTTP server)
To do this, right-click the DWORD you created and select 'Modify'. Ensure 'Base' is set to 'Decimal' and enter '80' (or another port) as the 'Value data':
Ensure that port 8888 is opened on the firewall
You must ensure that port 8888 is open to external requests (it won't be by default if your server is firewall-protected)
That's it! Fiddler should now be set up as a reverse proxy, to intercept all requests from port 8888 (so that you can view them in Fiddler), and it will then forward them to your web server to actually be handled.
Test a request
Restart Fiddler
To test that Fiddler is intercepting external requests, open a browser on the same machine where you've set up Fiddler as a reverse proxy. Navigate your browser to http://127.0.0.1:8888
This tests making a basic request to to port 8888
You should see the request intercepted by Fiddler
Now you can test a request from another machine, for example by making a request from a browser on another machine like this (where 'remoteHostname' is a hostname on the machine where you've set up Fiddler as a reverse proxy) :
http://remoteHostname:8888/controller/action
Alternatively, you can compose a request by using another instance of Fiddler on a remote machine, using a URL similar to the one above. This will allow you to make either a GET or a POST request.
IMPORTANT: Once you've finished viewing your request(s), go back to Tools > Fiddler Options > Connections and remove the 'Allow remote computers to connect' option, otherwise 3rd parties will be able to bounce traffic through your server
Guys found the perfect way to monitor ALL traffic that is flowing locally between requests from my machine to my machine:
Install Wireshark
When you need to capture traffic that is flowing from a localhost to a localhost then you will struggle to use wireshark as this only monitors incoming traffic on the network card. The way to do this is to add a route to windows that will force all traffic through a gateway and this be captured on the network interface.
To do this, add a route with <ip address> <gateway>:
cmd> route add 192.168.20.30 192.168.20.1
Then run a capture on wireshark (make sure you select the interface that has bytes flowing through it) Then filter.
The newly added routes will come up in black. (as they are local addresses)
Microsoft Message Analyzer is the successor of the Microsoft Network Monitor 3.4
If your http incoming traffic is going to your web server at 58000 port, start the Analyzer in Administrator mode and click new session:
use filter: tcp.Port = 58000 and HTTP
trace scenario: "Local Network Interfaces (Win 8 and earlier)" or "Local Network Interfaces (Win 8.1 and later)" depends on your OS
Parsing Level: Full
You might consider running Fiddler as a reverse proxy, you should be able to get clients to connect to Fiddler's address and then forward the requests from Fiddler to your application.
This will require either a bit of port manipulation or client config, depending on what's easier based on your requirements.
Details of how to do it are here: http://www.fiddler2.com/Fiddler/Help/ReverseProxy.asp
Use TcpView to see ports listening and connections. This will not give you the requests though.
In order to see requests, you need reverse of a proxy which I do not know of any such tools.
Use tracing to give you parts of the requests (first 1KB of the request).
Using Wireshark..
I have not tried this:
http://wiki.wireshark.org/CaptureSetup/Loopback
If that works, you could then filter for http/http contains GET/http contains POST traffic.
You might have to run two Wireshark instances, one capturing local, and one capturing remote. I'm not sure.
You can also try the HTTP Debugger, it has the built-in ability to display incoming HTTP requests and does not require any changes to the system configuration.

Resources