Is there any field/option/anything that I can put in a TCP packet (be it a syn or an ack or just plain data) that I can be sure will be returned by the other end intact?
For eg. I want to "tag" a particular connection (src, srcport, dst, dstport) with a number that I can always read from a packet belonging to that connection. That means I can identify the connection without using the 4-tuple (as given above).
Yes: it is called a Client protocol encapsulated in the TCP server protocol.
In other words: define the Client protocol to meet your needs. Don't try to "shove" extra bits in the TCP overhead.
There are of course the 'options' overhead in TCP but I doubt you'll find an easy way to access these... and in any case, you shouldn't.
You could possibly abuse the TCP Timestamp option for this. It does not seem like a great idea, though.
You can have a lookup table in your application where you associate your tag with the socket.
No, there isn't any facility for what you describe.
Typically what you would do if you're writing a socket application with multiple connections to other systems, is keep track of the socket handle that belongs to each remote system. When receiving data, you are using the socket handle (in some form, don't know which OS or language you're using) so you can take appropriate action based on whichever socket handle that is.
I've never seen a server application that keeps track of connections based on the 4-tuple of address/ports. That seems like way too much work.
On rereading your question, it seems like you may be asking this from the point of view of the TCP driver level. What sort of software are you writing here?
In UDP, destination IP and destination port number are used to demultiplex the packets, but in TCP destination IP, source IP, destination port number and source port numbers (4-tuple) all needed to distinguish between the connections why reasoning for this usage.
Related
Web games are forced to use tcp.
But with real time constraints tcp head of line blocking behavior is absurd when you don't care about old packets.
While I'm aware that there's definitely nothing that we can do on the client side, I'm wondering if there is a solution on the server side.
Indeed, on the server you get packets in order and miserably wait if misbehaving packet t+42 has been lost even though packets t+43, t+44 can already be nicely waiting in your receive buffer.
Since we are talking about local data, technically it should be possible to retrieve it..
So does anyone have an idea on how to perform that feat?
How to save this precious data from these pesky kernel space daemons?
TCP guarantees that the data arrives in order and re-transmits lost packets. TCP Man Page
Given this, there is only one way to achieve the results you want given your stated constraints, and that is to hack the TCP protocol at the server side (assuming you cannot control the Client WebSocket behavior). The simplest, relative term, would be to open a raw socket, implement your own simple TCP handshake (Syn-Ack when client Syns), then read and write from the socket managing your own TCP headers. Your custom implementation would need to keep track of received sequence numbers and acknowledge all of those you want the client to forget about.
You might be able to reduce effort by making this program a proxy to your original.
Example of TCP raw socket here.
I am designing a simple protocol on top of UDP and now I've realized that someone else can send a packet to the port I'm listening on. Such packet will obviusly be incorrect for my application (I'm not concerned about security now)
Is there the way of filtering these invalid messages? I was thinking about adding some fixed magic number at the beginning of each packet, but how large should it be? Is 16 bits enough?
I believe the typical solution is to require a handshake (that could include a "long enough" magic number) in the beginning of the session. Of course the "session" is something your protocol needs to keep track of, UDP does not have the concept. Keeping a list ip, port and last packet receive time for all current sessions should do it: then you can drop all packets from a peers that have not done the handshake beforehand. This not only prevents random unknown app traffic from breaking your application but will also prevent multiple legitimate peers from screwing up each others traffic.
Additionally you could add either a session id or an increasing packet sequence number (with allowances for missing packets) to packets if you need to ensure that the peer agrees with you on which session this is.
Probably. Java uses 32 for .class files (0xcafebabe). But you only have 534 bytes in practical UDP so you might need to save a couple.
Wireshark doesn't seem to be able to change the content of filtered packets in real time.
Does anyone know a symilar software which can change packet content that is filtered.
Finding something like this will really be a life saver
Thanks.
At least on Unices and -like where raw sockets are used, this is not possible, since the packet is copied to userspace and you only work on that copy. Furthermore, sending a packet back through the raw socket may be considered an "outgoing" packet so that it is, in fact, not reinjected to the input path where it should be. Raw sockets were — according to the Linux manpage — designed to implement new protocols, IOW, raw sockets are an "endpoint", not a "passthrough station".
For packet modification in the input path (passthrough-like), each OS has its own set of interfaces. In Linux (you were sort of unspecific as to which you target), that would be the nfqueue mechanism, usable through libnetfilter_queue. And of course, that is how wireshark, if it wanted to (I don't see it doing packet alteration last time I checked), would go about doing this.
Please give Burp Suite a try. It includes a repeater that let's you modify HTTP requests.
No wireshark won't let you change the contents of the packets and place them back on the line. However there are ways to change packets as they pass through the machine. Typically the host is setup with two nics bridged together. One nic is connected to one network and the other nic to the other network. Then as packets pass through this point the host can see them. Now you can use iptables/netfilter and write a module that changes data in the packet. For example you can write something that can remap source ip addresses. It's been a while since I've used netfilter/iptables, so I can't provide anymore details, but I have used it in a previous job to do some neat things with packets while they were inflight. It does mean you need a host machine sitting at network junction points though.
The documentation suggests that node.get("nextSibling") and node.get("previousSibling") are what you need.
Yes, it can.
You need to pass this option to the configure script before you build it:
--enable-packet-editor
Unfortunately I don't know much networks. I am writing a program that has two versions. A server version and a client version. Lets assume that the client versions are installed on, say 20 PCs that are connected to the server over ethernet. The client versions needs to CONSTANTLY get some data from the server. The data is kind of serial. I wanted to know a way to broadcast the data that gets updated every second and make it available to all the other PCs in the network. Could I use the HTTP Port for this?, like writing the data to an HTML page or something? or Is there a better port or method for doing this?
Any ideas will be greatly appreciated.
This sounds like a pretty straightforward application of TCP sockets. The server would be set up to "listen" on a particular port (you pick the port number, say 12345), and each client would make a TCP connection to the server on that port.
Whenever the server has data to send, it would send it once to each connected client. This could mean that the server sends the data up to 20 times on different sockets, but that's fine. The client would read the data from its connected socket to the server.
There are other alternatives, such as UDP or even UDP multicast, but these usually end up being a lot more complicated because UDP doesn't guarantee that packets always arrive at the destination (and they may even be duplicated or out of order). TCP ensures that the data you send either arrives complete in the correct order, or doesn't arrive at all (in that case the connection would be dropped).
An example of this sort of multiple TCP connection is VNC:
VNC is widely used in educational contexts, for example to allow a distributed group of students simultaneously to view a computer screen being manipulated by an instructor, or to allow the instructor to take control of the students' computers to provide assistance.
There are many ways. you can choose any of them but i think, document below will help you a lot.
Multicast over TCP/IP HOWTO:
http://www.ibiblio.org/pub/Linux/docs/howto/other-formats/html_single/Multicast-HOWTO.html#sect-trans-prots
We are trying to implement a proxy proof of concept but have encountered an interesting question: Since a single HTTP connection can, and indeed should, make multiple requests, and the HTTP transactions are sent via multiple packets due to TCP's magic, is it possible for a HTTP request to begin in the middle of a packet?
Bear in mind that this is not a theoretical question regarding possible optimization of the browser, but whether it actually happens in real life. It would be even better if someone could point me to a written reference on whether or not this is possible and if so how often it can occur.
Clarification update: We know that if we work in the HTTP layer alone we would not need to bother with this question, however we're trying to figure out if some advanced technique could be applied by working on the TCP layer first.
Assuming that you are talking about IP packets: Yes, it is possible that HTTP request starts middle of IP packet.
When you are using persistent HTTP connections, that is, using same TCP connection for several HTTP requests, it is fully possible that request boundary is middle of IP packet.
Also there is a TCP protocol between IP and HTTP. TCP contains also some headers so a IP packet may start with some TCP headers and rest of the packet consists of HTTP request.
HTTP request may also consist of several IP packets (in case of file uploads, transmission errors and following retransmissions etc).
However, I wonder why you are interested in packets if you are working at HTTP level. TCP should hide the IP packet details.
First of all, TCP is a stream based protocol and has no concept of packets. HTTP itself might have some kind of message or record delimiter, but TCP doesn't.
This page might be helpful: Structure of HTTP Transactions
From your question it sounds like you think that each read from a TCP socket is a "packet" of data. In reality, each read simply reads as many bytes as are in the buffer up to the maximum that you requested, without any concept of records or packets.
So for instance, lets say you read 2048 bytes from the socket, you could have the tail end of one transaction, followed by the beginning of a second response half way through the data you read, and only get the remainder of your second response on your next read from the socket.
If you're here in Jerusalem or near by maybe I could help you out.
Unless you are implementing your own TCP stack, you should not need to worry about the packets, but rather about the API that the TCP provides, in case of POSIX interfaces it would be the recv() or read(). So I treat the question then as "Can more than one HTTP requests come into a single read(), and can the HTTP request be split between multiple read() requests?" -- The answer to both would be "yes, it is possible".
An example of where this can happen is HTTP pipelining. This not frequent in real life (ironically, at least some of the browsers disable it by default because of "buggy proxies" :-) - but when it happens, can be a bit of a problem for the users to diagnose - especially if they have no access to the proxy.
One very notable place where it does happen by default apt-get in Debian-derived linux systems. Just install a Debian or Ubuntu server and try to use it through your proxy. You can do that by editing the /etc/apt/apt.conf.d/proxy file and placing the following there:
Acquire::http::Proxy "http://your.proxy.address:8080";
Depends of which abstraction layer of a packet you are talking about: there are many layers underneath HTTP.
HTTP --> TCP (byte stream) --> IP (packet) --> (possibly something else) Ethernet (frame) --> (possibly) some other transport
If you are talking about the IP layer, then yes the HTTP layer would start later on... Note that TCP presents a "byte stream interface" to its Client layer hence, no concept of packet here.
I think I understand where you are trying to go with this question.
If you don't use persistent HTTP connections, the HTTP GET request header is always the very first thing which is sent over the TCP connection, so we can be sure that the start of the HTTP GET request header does "not start in the middle of some TCP packet". But keep in mind that there may be one or more TCP packets without any user data, e.g. only a SYN, which may preceed the TCP packet with the start of the HTTP GET request header. And also keep in mind that the HTTP GET request header may not be contained in a single TCP packet.
If you do use persistent HTTP connections, the start of the HTTP GET request header for request number N+1 can start in the middle of a TCP packet, namely after the end of HTTP GET request body of request number N.
If you are asking these questions you are possibly "doing it wrong". As several other responders have already pointed out, in the vast majority of cases you should probably just be a TCP client and deal with a TCP stream of data and let the TCP code worry about the TCP packets. (Unless, of course, you are working on some special hardware which is looking at individual IP packets as they fly by and try to do some processing at the HTTP layer.)