How can I get Wireshark to show me my local HTTP Traffic? - http

When I enter either this URI to call a REST method on my running Web API app:
http://SHANNON2:21608/api/inventory/sendXML/duckbill/platypus/someFileName // using Hostname of local machine
--or this one:
http://192.168.125.50:21608/api/inventory/sendXML/duckbill/platypus/someFileName // using IP Address of local machine
...in Postman (using HttpPost, and attaching an xml file) and run it, on examining it in Fiddler 2, I see this error in Fiddler's Inspectors.WebView pane:
Bad Request - Invalid Hostname
--------------------------------------------------------------------------------
HTTP Error 400. The request hostname is invalid.
Note: I get that error even though I do hit the breakpoint in the corresponding Controller REST method on the server.
Trying to examine further exactly what's happending (what HTTP is being sent), I run Wireshark and see only this for destination of my IP Address:
...and this with my IP Address as the source:
Why are there no HTTP Protocol entries? The HTTP is apparently being sent, because I am hitting the breakpoint I've set on the first line in the server Controller method:
[HttpPost]
[Route("api/inventory/sendXML/{userId}/{pwd}/{filename}")]
public async Task SendInventoryXML(String userId, String pwd, String fileName)
{
Task task = Request.Content.ReadAsStreamAsync().ContinueWith(t =>
{
var stream = t.Result;
using (FileStream fileStream = File.Create(String.Format(#"C:\HDP\{0}.xml", fileName), (int)stream.Length))
{
byte[] bytesInStream = new byte[stream.Length];
stream.Read(bytesInStream, 0, (int)bytesInStream.Length);
fileStream.Write(bytesInStream, 0, bytesInStream.Length);
}
});
}
So why is Wireshark being so secretive about showing me the HTTP sent? Is it turning a blind eye to that internal communication and, if so, how can I "unleash it" to show me that?
UPDATE
Aha / Voila! A combination of Postman, RawCap (http://www.netresec.com/?page=RawCap), and Wireshark seems to work. I promptly ran the newly-discovered RawCap from a command prompt with this verbiage:
RawCap.exe 192.168.125.50 captured.pcap
...then called the REST method from Postman, and opened the resulting file that RawCap created (captured.pcap in my case) in Wireshark, and it shows me some HTTP traffic:
Some more detail about the HTTP call can be seen here:
What I don't know is how to "shut down" RawCap gracefully.
UPDATE 2
RawCap/Wireshark tell me it's a Bad Request (which I already knew was the response):
...but I see nothing to give me a clue about why it's considered a bad request
UPDATE 3
When I try to call my REST method from Fiddler Composer with all the various forms of "local" (hostname, IP Address, "locohost"), I never reach the breakpoint in the server; I get err msgs prior to that (404 for locohost, 400 for all others):

Typically, Invalid Hostname means that the HTTP request's Host header contains a value that is not bound to any site on the web server. For instance, you're using an IP address as the host, but if your server is only configured to accept a machine name (e.g. mybox) then you will see a HTTP/400 from the server demanding that you supply a proper hostname.

Related

Fiddler HTTPS Decrypt causes HttpClient to throw "Task was canceled" exception when used with AutoResponder

My app calls an external API, to avoid flooding them with test requests I decided to use Fiddler's AutoResponder to catch the request and respond with a specific response.
The issue that I am running into is that the request is HTTPS, so Fiddler has to be configured to decrypt HTTPS traffic to trigger the AutoResponder rule, but then HttpClient throws a "Task is canceled" exception.
If I turn off HTTPS decryption, then the call goes out to the external API and returns a response, so I know that the code is working as expected.
I am assuming it could be that the AutoResponder's response is not encrypted, or something about the encryption is incorrect.
I have trusted the Fiddler certificate.
The issue was that the response data did not end with two line breaks.
Adding the line breaks made HttpClient read the response and continue processing as expected.

ESP32 how do I add a Host header with http.addHeader()

I'm debugging some DNS issues on a WiFi access point I'm setting up. I am connecting an ESP32 to my network so it can output what it's up to. I want to get my Public IP address as a test and I want to use http://ifconfig.me/ip to go and get it. This is superb when DNS is working, but as per standard HTTPS stuff, I can use the Host: name header in a call to the IP address to test connectivity to determine if it's just a DNS failure I'm seeing. If I set up the following in Postman, everything is super...
When I do the following on the ESP32, which should mimic the above, I get a 404 - which happens when the Host header is not sent or is incorrect.
...
String new_url = "http://34.117.59.81/ip";
String host = "ifconfig.me";
HTTPClient http;
http.begin(new_url.c_str());
http.addHeader("Host", host.c_str());
int httpResponseCode = http.GET();
...
What am I doing wrong? I've tried the addHeader() call either side of the begin()
I've tried collectHeaders() call, but that only keeps the response headers so I can't see what I'm actually sending.
On the positive note, the fact I'm getting a 404 does mean connectivity is fine, and the issue is my DNS as expected, but why can't I (allegedly) send the Host header?
The HTTPClient library adds the Host header for you automatically.
The library is written to silently ignore attempts to add your own Host field (along with Connection, UserAgent and Authorization):
// not allow set of Header handled by code
if(!name.equalsIgnoreCase(F("Connection")) &&
!name.equalsIgnoreCase(F("User-Agent")) &&
!name.equalsIgnoreCase(F("Host")) &&
!(name.equalsIgnoreCase(F("Authorization")) && _base64Authorization.length())){
String headerLine = name;
Unfortunately it doesn't currently offer a method to tell it to use a particular Host with an IP address that's not resolved from it.
You could copy the library into your project and just delete the check that stops it from using your Host field. Or (I hate suggesting this because I usually push people to use the HTTPClient library rather than roll their own) you could do your own simple HTTP implementation using WiFiClient.
It sounds like you understand this already but for others who may read this, HTTP allows one IP address to serve multiple domains (back in the old days it didn't but as we ran low on IP addresses this became critical). The server uses the Host field to decide which domain to direct traffic to. It's likely that the server you're connecting to is accepting the connection but without the Host field being set properly the request isn't making it to the correct server and is failing with a 404.

Invalid Metadata Requests on Google Compute Engine

I am running a Google Compute Engine instance. About every ten seconds, I get a request from a local link address (such as 169.254.169.254) requesting metadata from my instance. The request is on the computeMetadata path, suggesting that google is trying an [to get meta data from my instance].
Why am I receiving these requests? Do I have compute engine configured incorrectly? Right now my app returns a 404, should it do something else?
This is the full request:
010.240.059.243.48574-169.254.169.254.00080: GET /computeMetadata/v1beta1/instance/network-interfaces/0/public-endpoint-ips?alt=text&wait_for_change=true&timeout_sec=60&last_etag=NONE HTTP/1.1
Accept-Encoding: identity
Host: metadata
Connection: close
User-Agent: Python-urllib/2.7
The images provided by default on GCE will automatically configure themselves based on data returned by the metadata server.
This particular request is to find IPs that are forwarded to this instance as part of Load Balancing. Basically, the script at /usr/share/google/google_daemon/manage_addresses.py will continually wait for new IP addresses to be forwarded to this instance. Once it notices a new incoming IP (as indicated by the metadata server) it will configure the instances network stack to respond to that IP.
The question in my mind is: why are you seeing these? Are you doing something interesting to capture the requests sent to that address? These should be completely transparent to any application.

Use JMeter HTTP Proxy to record JSON over HTTP request from not-a-browser client

I have a client program running locally in iPhone emulator and local server written in Java. Client talks to server with JSON over HTTP requests. Now I am trying to record a client session http requests with JMeter to use them as a base to load-test server.
The problem is client requests do not go throw JMeter proxy. Client gets 501 "Method not implemented" (it is not because of https, I am using http). There is nothing in JMeter log about the request, and obviously nothing gets recorded in JMeter and the request doesn't reach the server.
There is well-described steps to setup JMeter proxy to record request from a browser, but my client programm is not a browser. Though JSON over HTTP is widely adopted approach, I could not find anything on the web about recording such requests with JMeter. I understand I need to do on the client the same thing browser does when proxying request and what I've found about it is that I need to set Host header to server's host and port, but that did not work and I cannot see how is it related to 501 error client gets.
If someone can explain what should be done on client or how to configure JMeter to let it know where to proxy client's requests or link any manual explaining that, it would be great help. I've been searching for solution for a few hours already and had no luck. Please help.
You can see my question and the answer I got below:
Use Jmeter proxy to record HTTP calls from iOS simulator
In short:
With this tutorial you can record calls from your Android device:
http://blazemeter.com/blog/load-testing-mobile-apps-made-easy
For make same thing with iPhone, do the following steps:
Mac configuration:
system preferences -> Network -> Advanced.. -> Proxies -> check "Web Proxy (HTTP) ->in "Web Proxy Server" field, type your IP (http://www.wikihow.com/Find-Your-IP-Address-on-a-Mac), and choose available port (I using 8080) ->ok -> Apply
iPhone configuration:
Settings -> WiFi -> choose same wifi you use with your Mac -> press on it again to go to it's details -> scroll down ->In HTTP proxy, choose Manual -> server = your mac IP you found earlier -> port = the port you chosen (maybe 8080)
Now You can start recording all "iPhone network out" using jmeter recording controller

BizTalk Send Adapter HTTP Post Response

I have a BizTalk 2010 project containing an orchestration that needs to make an HTTP Post and then examine the Status Code and Body of the Response to determine the next course of action.
I can configure the orchestration and port to make the HTTP Post, but I am unable to receive a response.
Should I be using a send / receive port or correlation?
What schema should I be using for the response (I believe the response is the standard http response: http://www.w3.org/Protocols/rfc2616/rfc2616-sec6.html#sec6).
If you are looking for a kind of notification ( not in content of the message) that the message has been successfully transmitted, you can set the logical send port property in the orchestration as follows:
"Delivery Notification" = Transmitted
And delivery failures can be handled using the Microsoft.XLANGs.BaseTypes.DeliveryFailureException
The Http Status Code should be available on the Response message as a Context Property, which you can access in an Expression shape.
statusCode = ResponseMessage(HTTP.ResponseStatusCode);
Your ResponseMessage should be of type System.Xml.XmlDocument, but as it won't be a real Xml Document, make sure the Request/Response port is configured to use the PassThruReceive pipeline on the Response side.

Resources