TCP is stream to communicate and it has varying length. So in the application, how I can know whether the TCP ends or not?
In the transfom layer, The TCP packet header doesn't have a length field and its length is varying, how can the TCP layer know where is the end.
You design a protocol that runs on top of TCP that includes a length field (or a message terminator). You can look at existing protocols layered on top of TCP (such as DNS, HTTP, IRC, SMTP, SMB, and so on) to see how they do this.
To avoid pain, it helps to have a thorough understanding of several different protocols layered on top of TCP before attempting to design your own. There's a lot of subtle details you can easily get wrong.
If you look at this post, it gives a good answer.
Depending on what you are communicating with, or how you are communicating there will need to be some sort of character sequence that you look for to know that an individual message or transmission is done if you plan to leave the socket open.
n the transfom layer
I assume you mean 'transport layer'?
The TCP packet header doesn't have a length field and its length is varying
The IP header has a length field. Another one in the TCP header would be redundant.
how can the TCP layer know where is the end.
From the IP header length word, less the IP and TCP header sizes.
When a party of a TCP connection receives a FIN or RST signal it knows that the other side has stopped sending. At an API level you can call shutdown to give that signal. The other side will then get a zero length read and knows that nothing more will be coming.
Related
I'm writing a small application using libpcap, where I parse/analyze a TCP based application. I faced a situation, where application attempts to send a really large amount of data, say 64K, and TCP layer cuts it into a number of smaller segments.
Now, my question is -- how do I really tell that a TCP payload of the packet, read from pcap, is actually a chunk of a larger payload. So, in order to access original large payload, number of segments will need to be re-assembled.
TCP header has sequence field, but I don't fully understand how it can answer my question.
Also, IP header has total_length field, but it has nothing to do with TCP segmentation, it indicates IP payload size of the current packet.
I'd appreciate to get some hints. Thanks.
TCP can't help you here because it neither knows nor cares what PDUs are. You need to implement whatever protocol defines what a "large PDU" is. For example, if this is HTTP over TCP, implementing the HTTP protocol will tell you if the segment is part of a large PDU.
Because my question was - how do I tell that I have a small segment that has to be reassembled in a large packet.
That's what a message protocol is for. If, for example, the message protocol says that a PDU is a "series of characters not containing a newline character terminated by a newline character", then if you don't have a newline character, you know it's part of a larger PDU.
The concept of PDUs applies to message protocols, so if you're talking about PDUs, you must have a message protocol. The message protocol will tell you when you have an entire PDU. That's its purpose.
How can I understand packet lost in UDP?
For example in a chat application if a Sentence is not received how to understand it and warn client?
thanks to safeer I found this and I think it's correct:
If the higher-level protocol that's running on top of UDP (RTP, TFTP, etc.) has some sort of identification number or sequence number or block number that is unique per packet and that changes in a predictable way, then you could display that number in a custom column and manually look for missing numbers, but there is no way in the UDP protocol itself to identify missing packets.
It may be possible to somehow mix UDP and TCP.
Can anybody explain how does the receiver know if two nonconsecutive TCP segments belong to the same or different packets ? And how does it know if the next segment is the last segments in the packet ?
The receiver doesn't assemble TCP segments into packets, it assembles them into streams. The receiver knows the location, in the stream, of its received segment by its sequence number.
Is it possible that you are expecting the count result of the receiving application's read() system call to conform to the sending application's write() system call? If so, you will be disappointed. TCP streams are byte-wise, not packet-wise, streams. They neither preserve nor honor the boundaries of the sending system calls.
TCP does not deal with fragmentation. That's an IP problem. Packets arrive at the TCP level only when complete. IP uses special fields in the header that indicates whether the packet is fragmented or not, and, if yes, whether the fragment received is the last one or not.
You may take a look :
Transmission Control Protocol
Internet Protocol
is it possible to send multiple tcp or udp packets on a single ip packet? are there any specifications in the protocol that do not allow this.
if it is allowed by the protocol but is generally not done by tcp/udp implementations could you point me to the relevant portion in the linux source code that proves this.
are there any implementations of tcp/udp on some os that do send multiple packets on a single ip packet. (if it is allowed).
It is not possible.
The TCP seqment header does not describe its length. The length of the TCP payload is derived from the length of the IP packet(s) minus the length of the IP and TCP headers. So only one TCP segment per IP packet.
Conversely, however, a single TCP segment can be fragmented over several IP packets by IP fragmentation.
Tcp doesn't send packets: it is a continuous stream. You send messages.
Udp, being packet based, will only send one packet at a time.
The protocol itself does not allow it. It won't break, it just won't happen.
The suggestion to use tunneling is valid, but so is the warning.
You might want to try tunneling tcp over tcp, although it's generally considered a bad idea. Depending on your needs, your mileage may vary.
You may want to take a look at the Stream Control Transmission Protocol which allows multiple data streams across a single TCP connection.
EDIT - I wasn't aware that TCP doesn't have it's own header field so there would be no way of doing this without writing a custom TCP equivalent that contains this info. SCTP may still be of use though so I'll leave that link.
TCP is a public specification, why not just read it?
RFC4164 is the roadmap document, RFC793 is TCP itself, and RFC1122 contains some errata and shows how it fits together with the rest of the (IPv4) universe.
But in short, because the TCP header (RFC793 section 3.1) does not have a length field, TCP data extends from the end of the header padding to the end of the IP packet. There is nowhere to put another data segment in the packet.
You cannot pack several TCP packets into one IP packet - that is a restriction of specification as mentioned above. TCP is the closest API which is application-oriented. Or you want to program sending of raw IP messages? Just tell us, what problem do you want to solve. Think about how you organize the delivery of the messages from one application to another, or mention that you want to hook into TCP/IP stack. What I can suggest you:
Consider packing whatever you like into UDP packet. I am not sure, how easy is to initiate routing of "unpacked" TCP packages on remote side.
Consider using PPTP or similar tunnelling protocol.
im now developing a project using winpcap..as i have known packets being sniffed are usually fragmented packets.
how to reassemble this TCP segements?..any ideas, suggestion or tutorials available?..
this i assume to be the only way i can view the HTTP header...
thanks!..
tcp is a byte stream protocol.
the sequence of bytes sent by your http application is encapsulated in tcp data segments and the byte stream is recreated before the data is delivered to the application on the other side.
since you are accessing the tcp datasegments using winpcap, you need to go to the data portion of the segment. the header of tcp has a fixed length of 20 bytes + an optional part which you need to determine using the winpcap api.
the length of data part in the tcp segment is determined by subtracting the tcp header length (obtained from a field in the tcp segment) and the ip header length (from a field in the ip datagram that encapsulates the tcp segment) from the total length (obtained from another field in the ip datagram).
so now you have the total segment length and the length of the data part within the segment. so you know offset where the http request data starts.
the offset is
total length-length of data part
or
length of ip-header + length of tcp header
i have not used winpcap. so you will have to find out how to get these fields using the api.
also ip datagrams may be further fragmented but i am expecting that you are provided only reassembled datagrams using this api. you are good to go!
There is no such thing as a TCP fragment. The IP protocol has fragments. TCP is a stream protocol. You can assemble the stream to its intended order by following the sequence numbers of both sides. Every TCP Packet goes to the IP level and can be fragmented there. You can assemble each packet by collecting all of the fragments and following the fragment offset from the header.
All of the information you need is in the headers. The wikipedia articles are quite useful in explaining what each field is
http://en.wikipedia.org/wiki/TCP_header#Packet_structure
http://en.wikipedia.org/wiki/IPv4#Header
PcapPlusPlus offers this capability out-of-the-box for all major OS's (including Windows). Please check out the TcpReassembly example to see a working code and the API documentation to understand how to use the TCP reassembly feature
Depending on the whose traffic you're attempting to passively reassemble, you may run into some TCP obfuscation techniques designed to confuse people trying to do exactly what you're trying to do. Check out this paper on different operating system reassembly behaviors.
libtins provides classes to perform TCP stream reassembly in a very high level way, so you don't have to worry about TCP internals to do so.