Writing an http sniffer (or any other application level sniffer) [closed] - http

Closed. This question needs to be more focused. It is not currently accepting answers.
Want to improve this question? Update the question so it focuses on one problem only by editing this post.
Closed 4 years ago.
Improve this question
I am trying my hands understanding PCAP libraries.
I am able to apply a filter and get the TCP payload at port 80. But what next ? How can I read the HTTP data - suppose I want to know the "User Agent" field value in the http header..how should I proceed ?
I have searched the website (and googled a lot too), and could find a related thread here :
writing a http sniffer. But this doesn't help me anywhere...
Thanks !!

First, you should know that PCAP give you packets, and will not reconstruct the TCP stream so you won't be able to read full HTTP TCP streams without first reconstructing the data.
Assuming all the data is available in one packet try and look at my answer for a similar question. All you need to do different is to parse the HTTP header and get the user agent.
If you don't limit yourself to C, and if you can use Windows, you can write a .NET application and use Pcap.Net to parse Ethernet, IPv4 and TCP perfectly.

Why don't you use a Wireshark Dissector?

There is already a good Pcap wrapper for .net called Pcap.Net - here it is
"Pcap.Net is a .NET wrapper for
WinPcap written in C++/CLI and C#. It
Features almost all WinPcap features
and includes a packet interpretation
framework."

Related

BLE Custom Scan Response [closed]

Closed. This question needs to be more focused. It is not currently accepting answers.
Want to improve this question? Update the question so it focuses on one problem only by editing this post.
Closed 1 year ago.
Improve this question
Working with ESP32 and IDF
I'm working in developing an application that advertises BLE data. I need to send custom data to each one of the different devices scanning. I'm able to write a custom manufacturer data. I can also write a custom scan response message but, can a different custom response message be sent to each individual remote device? If possible, how do I do it?
I have multiple iOS software for checking the BLE data but I have not been able to find one that displays both the Advertisement data and the scan response data in real-time. Does anyone know of one?
Thanks
Please don't ask two completely separate questions in one question. But anyway:
The Bluetooth specification has no way of specifying different scan response data for different scan request bluetooth addresses. The address field in the scan request is there only to make it possible to discard scan requests from devices that are not in the white list.
The iOS api does not differentiate advertisement packets from scan response packets, so I don't think any app can do this. But you should however be able to get a notification as soon as some data arrives. I guess nRF Connect does this.

how exactly does http.sys work [closed]

Closed. This question needs to be more focused. It is not currently accepting answers.
Want to improve this question? Update the question so it focuses on one problem only by editing this post.
Closed 4 years ago.
The community reviewed whether to reopen this question 10 months ago and left it closed:
Original close reason(s) were not resolved
Improve this question
I'm trying to get a deeper understanding of how IIS works.
http.sys i understand is one its major components. However, i have been having trouble finding easily digestible information about it. I couldn't get a good mental model going until i heard about the WSK, then i think it all fell into place.
From a lot of random googling a little experimentation this is my current high level understanding of why it exists and how it does it's stuff.
Why:
Port sharing, and higher performance caching.
How:
User mode processes use the WinSock api to open a socket listening on a port to gain access to the networking subsystem, e.g. tcp/ip. Kernal mode software like the http.sys driver uses Winsock Kernal Sockets (WSK) api to achieve the same end using the same pool of TCP port numbers as the WinSock api.
IIS, a web service or anything that wants to use http registers itself with http.sys using a unique url/port combination. http.sys opens up a socket on this port using WSK (if it hasn't already for another url/port combination with the same port) and listens.
When the transport layer (tcpip.sys) has reconstructed a load of ip packets back into an http request that a client sent it gives it to http.sys via the port in the request. Http.sys uses the url/port number to send it the the appropriate process which parses it however it pleases.
I know it seems like I'm answering my own question but I'm really not that sure of myself on this and would like some closure so i can get on with more interesting things.
Am i close?

Best protocol for embedded PIC microcontroller to communicate with server over the internet? [closed]

Closed. This question is opinion-based. It is not currently accepting answers.
Want to improve this question? Update the question so it can be answered with facts and citations by editing this post.
Closed 9 years ago.
Improve this question
Firstly, thank you for taking the time out to read this post.
I'm looking to develope a TCP/IP enabled device using the Microchip PIC18 or PIC32 family of embedded microcontrollers with Microchip's TCP/IP Stack. However, my knowledge of networking is pretty basic at the moment, thus the reason for this post.
Can anyone recommend the best protocol to use for my TPC/IP embedded device so that it can communicate with a server in a data centre? My intention is to have the embedded device located at a remote location somewhere over the internet, where the server can communicate with the device and download data such as thermometer probe readings to be stored in a database. I would also like the ability for the server in the data centre to be able to reconfigure settings and variables on the remote device should I need to.
My research on protocols so far has lead me to the following options:
SNMP v3 (version 3 due to encryption and authentication)
UDP (though I read this can be unreliable but is fast)
TCP (I'm not too clued up on this yet)
Can anyone offer me advice on the best route to go down? I'm not expecting a detailed answer from you, but I would really like an idea of what topics/protocols to look into and research.
My intent is to deploy many of these embedded devices over the internet where they all send their data back to the server.
I assume that the remote embedded device will have to connect to the server rather than vice versa as the server will have a static IP address or DNS name, whereas the remote device addresses will be unknown.
Any advice on this would be greatly appreciate. Please don't hesitate to ask if I've missed out any key information in this post.
Many thanks.
Rob
* UPDATE *
It was pointed out that I'm probably misusing the term Web Server, so I've amended my post to mention Server in a Data Centre instead. Thank you for pointing this out to me.
If the target is a Web server you don't have any choice. You have to use HTTP, which runs over TCP.
Or else you are misusing the term 'Web server'.
In many ways this depends on your specific requirements. TCP/IP is able to provide quite reliable connections because it provides a means to determine if the client is connected, when they connected and when they disconnected. UDP is connectionless, so the server opens a port and listens for data, but has no automatic connection management, so clients need to explicitly 'tell' the server when they have arrived or are going (this also means you will need to make your own timeout facility).
Also, if you have very limited memory/processing resources, it is worth bearing in mind that UDP is a less 'costly' protocol as it avoids a lot of the overheads TCP incurs due to its inbuilt connection management.
While these are all protocols, they really just handle the connections themselves. You will probably still need to create your own protocol for the management of the data itself. For instance, when you send data over either TCP or UDP, the bytes you send may not all arrive to the server at the same time. This means you need a way of validating each packet you receive to ensure you have it all. This is often achieved with a combination of a checksum and a byte representing the total size of the data sent.
You might also consider mqtt (http://mqtt.org). It is a lightweight messaging protocol. For encoding your messages, you might consider protobuf (https://code.google.com/p/protobuf/)

Would a custom IP-based protocol work? [closed]

Closed. This question is off-topic. It is not currently accepting answers.
Want to improve this question? Update the question so it's on-topic for Stack Overflow.
Closed 10 years ago.
Improve this question
Let's say somebody invented a new protocol I would put on top of IP. Would two computers from the other ends of the world be able to communicate with it, i.e. would routers forward the frames that aren't standard TCP/UDP/ICMP?
Yes, if it is build on top of IP then it would be routable over the internet. The IP protocol defines the header and payload. The header is used for routing. So you would be able to send custom IP-based protocol data from one computer to another over the internet.
However, both computers will need custom drivers to send, receive and understand the data.
I'm not sure why you'd bother though. If you're sending custom data, you're much better off writing an application level protocol on top of tcp or udp and take advantage of the networking layer built into all computers and operating systems already. It'll be easier to write, maintain, and debug.

Doubling Internet Connection Speed [closed]

Closed. This question is off-topic. It is not currently accepting answers.
Want to improve this question? Update the question so it's on-topic for Stack Overflow.
Closed 12 years ago.
Improve this question
Scenario: I have two ADSL modem that are connected to to different ISPs. Each has 256KBps Speed.
Question:Is it possible to have 512KBps speed?(I have one PC that can be host any OS)
Is any special appliance essential for doing that?
Thanks in Advance,
Ashkan.
This is called multihoming or load-balancing.
The simplest way to do this would be to buy a router with two WAN ports that supports load balancing. These are generally expensive.
Alternatively you can set up a computer with 3 network interfaces to do the routing for you.
Windows
Using regedit navigate to
HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\services\NetBT\Parameters
Create 32-bit dword value "RandomAdapter" with a value of 1
You will need to set the "metric" value of the different adapters if you find traffic is still favouring one connection over the other.
From my answer on serverfault, the
"route" command is used to set this metric. The basic syntax is;
route ADD <destination> <subnet mask> <gateway (vpn dhcp server)> <metrix> IF <interface number> -p
There's some tutorials floating around here and there.
Linux
See the answers on serverfault
RJFalconer is right, but you should know that if you do this, no single TCP connection will be able to get more than 256kBps. It's much like SMP in that manner.
You may also run into trouble with (web) applications and protocols that assume every user has a single IP address at any one time. If you can replace the two connections with a single faster one, that would be vastly preferable.

Resources