Is there any light weight tools which can filter and output http headers/responses in human readable form? Something like wireshark. I have tried tcpdump; but it is hard to read.
If you able to configure proxy settings on the embedded device you can (like in web browser) you can be very flexible. I like for example Fiddler tool http://www.fiddler2.com/fiddler2/. If you start fiddler tool on you "normal" computer with Windows 7/XP etc, fiddler runs as a proxy with a port (typically 8888). If you configure on the embedded device IP address with this port as a proxy, you will see or be able generate auto-responses etc. inside of fiddler of your computer.
So you can solve the problem without installation on embedded device any software.
I don’t want to write a long text here, but I recommend watching video http://microsoftpdc.com/Sessions/CL25, where Fiddler developer Eric Lawrence explains Fiddler’s features.
Related
We develop APPs, and sometimes we want to see the traffic of the APP.
Now we use Charles, AnyProxy, or Fiddler, but all these tools need manually settings on the Mobile Device, and not easily to share the traffic.
So I am thinking about a solution:
1. on the router run tcpdump, or make a port-traffic-mirror to a specific machine, so I can get all the traffic
2. there will be a program, decoding the traffic stream to pure http/https package, and show them on a webpage.
3. anyone who want to see the http/https(based on the APP, or may be protobuf) requests and responses. and can do some search or filter.
4. yes, these works like Wireshark, but how to make Wireshark run on a web?
I had Fiddler open recently with no filters on, and I was running a program from my command prompt that communicates with a server (it's a simple in-house proprietary program).
Interestingly, Fiddler was not picking up this communication, which is mostly made up of REST API calls. I did some research and found this quote, from here
With regard to why you're not seeing this traffic in Fiddler--
assuming you don't have any filters set, this suggests that whatever
mechanism you're using to send the HTTP request isn't adopting the
system's proxy settings. This means, for instance, that your code
would fail if run on a corporate computer that requires a proxy server
to reach the Internet.
However, I'm wondering why this would be the case. As far as I understand, my computer still needs to send data from my network card to the proxy server's network card. The traffic isn't bypassing my network card - it simply carries a different address. Why isn't Fiddler able to see this?
The reason is that Fiddler works not by sniffing your network card but by installing itself as the system proxy. If you click start and type "Internet Options" and choose the tab "Connections" and the button "LAN Settings" you'll see that localhost:8888 is your system proxy. Now, most well-behaved clients (e.g. IE and Chromer) respect and use the system proxy but some don't. In particular Java programs have their own ideas about which proxy they will use and you would have to set that separately. For other programs which are just hard-coded to make their own direct HTTP request you cannot AFAIK monitor them with Fiddler.
I am currently trying to debug a flex application that uses a vlc media player plugin to watch mpeg-2 videos.
I have run up Fiddler and I can see lots of calls from the flex application for resources it uses but when I play a video in the player, I do not see any requests for the video in Fiddler.
Any ideas how to get Fiddler to show the http traffic when a video is played?
Fiddler will show any HTTP/HTTPS traffic that is sent through it. For traffic not seen in Fiddler, there are two possibilities:
1> The traffic is non-HTTP/HTTPS (e.g. straight TCP/IP).
2> The traffic is HTTP/HTTPS but is not configured to go through the proxy.
If the traffic is non-HTTP/HTTPS, there's nothing you can do to get it to appear in Fiddler.
If the traffic is HTTP/HTTPS but isn't using the system's default proxy, you may be able to configure it to do so. For instance, I don't know if the VLC plugin respects the settings from the VLC client, but try clicking Tools > Preferences > Input & Codecs and put 127.0.0.1:8888 the box for the proxy URL.
Here is what I have to do :
Open a url in IE/Chrome/Firefox browser.
Do some UI user action (click/submit).
Record http request for a particular http call.
Here is what I did: console application + selenium RC to open url & do the user actions + Fiddler Core to capture that http traffic.
My code works fine when I'm running it in Firefox & IE, but don't know why Fiddler Core not able to capture any http traffic when running it against Chrome.
ISelenium selenium = new DefaultSelenium("localhost", 4444, "*googlechrome", "http://money.msn.com/");
selenium.Start();
selenium.Open("/");
selenium.WaitForPageToLoad("30000");
Another intersting thing, if I dont use Selenium RC & open Chrome using Process class then I can see fiddler core capturing that http request. But I need a UI automation tool to do the user actions.
Any help will be greatly appriciated.
Thanks.
I hope I do not misinterpret the problem -- I assume localhost traffic to be the thing that cannot be captured.
localhost traffic is routed via the local loopback interface (a 'virtual' interface that routes local traffic more efficiently), and this never reaches a network card. Most packet capture tools capture the packets on the network device driver layer and thus only capture packets that actually go throug physical network devices.
I use the following way to pass proxy and it is working for me.
ChromeOptions options = new ChromeOptions();
options.AddArgument(string.Format("--proxy-server=http={0}:{1};https={0}:{1}", proxyHost, port));
driver = new ChromeDriver(options);
Anyway, use ipv4.fiddler instead of localhost to ensure fiddler is able to capture the traffic.
For IE, if you are using IE9, it has been changed to allow proxy connections to localhost. It seems FF is also capable to do this. Check this URL:
Fiddler and the IE9 Release Candidate
http://blogs.msdn.com/b/fiddler/archive/2011/02/10/fiddler-is-better-with-internet-explorer-9.aspx
*IE9 RC introduces the ability to proxy loopback traffic. To do so, simply include the token <-loopback> (pronounced “minus-loopback”) in the proxy bypass list. When WinINET encounters this token in the bypass list, it will remove the loopback addresses (localhost, 127.0.0.1) from the list of hosts that bypass the proxy. Fiddler 2.3 and above automatically set this option when capturing traffic.
The FiddlerHook add-on automatically sets the equivalent version of this option for Firefox; for Opera, you can manually remove loopback addresses from the proxy-bypass list.*
I have some knowledge on my question but not exactly. What exactly programatically happening after http request?
Scott Hanselman said in one of his blog posts:
Describe, in as much detail as you think is relevant, as deeply as you can, what happens when I type "cnn.com" into a browser and press "Go".
My question is exactly this,
That's like asking "describe how to perform a coronary bypass". Yes, one can explain how, but one is better first studying medicine and learning about the basics, before starting with specific procedures. However, in bulletpoints:
Your browser will want to know the IP address of cnn.com. It doesn't do DNS lookups itself, but rather asks the operating system.
Your browser will connect to that IP address on port 80
Your browser will send a HTTP GET request
The webserver will reply with statuscode 200 and the body contents
Your browser will parse the HTML
In the HTML, other resources (images, scripts, css-files...) might be included, which the browser will also fetch.
After the browser is done, it will close the connection. If it doesn't, the webserver will.
Browser tries to resolve the name cnn.com into its ip address.
Browser TCP connects to cnn.com's ip address on the default HTTP port (80)
Browser sends a GET request to the server, asking for the / page
Browser says that it's trying to connect to "cnn.com" (cnn.com and bbc.com could be hosted on the same hosting company, with the same IP address)
Browser also says what's your browser, browser engine, browser version, operating system and the plug-ins that you have installed.
Server sends a header saying what's coming on your reply, the kind of data you're going to receive (in this case, HTML), and the size of the response if it's available.
Server closes the connection if there isn't any keep-alive instruction from the browser. Otherwise it will use this opened connection to ask for other things that might be needed (images within the page, for example.)
By the way, download and install Wireshark if you want to go deep and see what's really going on behind the curtains.