I'm trying to do a capture and am seeing half the payload with '[!http]' half way through. Is there a way to make it show the whole payload?
I'm performing a REST call and want to see what the server is receiving:
<value>0x100000</value>
</attribute>
</greater-than-or-equals>
<less-than-or-equals>
<attribute id="0x129fa">
[!http]
16:34:06.549662 IP (tos 0x0, ttl 255, id 49564, offset 0, flags [DF], proto TCP (6), length 1301)
I'm using:
tcpdump -vvv -i ens192 tcp port 8080 and src 192.168.1.1
Any help would be appreciated.
Many Thanks
Frank
It is quite likely that the problem has less to do with what you are capturing and more to do with the payload being larger than a single packet.
When you run tcpdump, these days, the default is to capture packets whose length match the MTU of your interface (at least). You can override this, if you are unsure, by specifying a capture length of zero:
tcpdump -s 0 -w captureFile.cap
Again, this is likely not the problem here. It is more likely that the rest of the data is in the next TCP segment. Unfortunately, tcpdump is not the ideal tool for extracting session data. I would suggest that you look at Wireshark (or tshark) which will allow you to easily select a packet and then reassemble the data stream with all of the IP and TCP headers removed.
Related
Dear wireshark experts,
I got a tshark output image on ubuntu 16.04 after I ran the command curl www.baidu.com, as shown below. But I can't tell you the story from the output lines, because I can't find any mathematical relationships betweens the lines.
Could someone explain the meaning of each line, especially the field Ack and Win?
So appreciated in advance!
That's not related to tshark/wireshark, but generally to TCP protocol:
ACK (and SYN) are tcp flags being set during connection establishment
Win is also tcp parameter. see Window size
FrameNumber Timestamp Source-IP-Address -> Destination-IP-Address Protocol TCP-Source-Port TCP-Destination-Port [TCP-FLAGS] Protocol-Specific-Description
SYN/ACK/WIN are related to the TCP protocol which is used for reliable transmission on top of the unreliable IP network
Currently trying to setup iptables to allow for a client to connect to the server to listen to a stream of messages via TCP. The thing is, we want to block the client from sending any messages once connected (is OK if the client is DROP'ed in this case).
Is there a way to allow a client to connect and enforce a 1 way communication from the server to the client only?
Requires this to work purely within iptables (no software proxy-like solution).
The primary problem to be solved here is that we can't just shut down all the traffic from the client after the connection has been established because TCP is a positive acknowledgement protocol. If the server does not receive acks from the client, it will retransmit, and eventually time out. In what follows, I will assume that we are using IPV4.
So what we want to do is to allow the connection to be established, and then only allow acknowledgements from the client, that is, packets that contain no TCP payload.
Unfortunately, the length of the TCP payload is not explicitly represented in the TCP header. We can try to use the overall length in the IP header, but this is complicated by the fact that both the IP header and the TCP header include variable length options fields, so there are many possible overall lengths that ack with no payload.
Since IP options are rarely used and commonly filtered, let's simplify things by first dropping all packets that contain options in the IP header (if your firewall is not already doing so). The implications of doing so are discussed at length here.
To do this, we will drop all traffic to our server (here taken to be 10.2.3.4:1234) where the IP header length (bits 4-7 of byte 0 in the IP header) is not 5:
iptables -A INPUT -p tcp -d 10.2.3.4 --dport 1234 \
-m u32 --u32 "0>>24&0xF=6:0xF" -j DROP
This uses the iptables u32 module to grab 4 bytes from the packet starting at byte 0, right shift it 24 bits, mask the lower nibble, and then drop the packet if this is in the range 6-15. Note that 5 is actually the minimum size of an IP header.
The situation with TCP options is a bit more complicated. In establishing the connections, many different options may be used, e.g., to negotiate window scaling. However, once the connection is established, the only thing we need to worry about is TCP timestamps and selective acks. So let's let the connection be established:
iptables -A INPUT -p tcp -d 10.2.3.4 --dport 1234 \
--tcp-flags SYN SYN -j ACCEPT
Note that it is possible to send a payload in a SYN packet, so here we are not completely meeting your requirement. Most ordinary TCP implementations will not do so, although TCP fast open does. If you want to mitigate this, you could drop SYN packets that are fragments (that could reassemble to something very large) and limit the overall length of the non-fragmented SYN packets to something reasonable that would allow for the usual options to be present in the TCP three way handshake. Note that the above rule was added to the INPUT chain, which is processed after IP fragment reassembly.
OK, so we can get the TCP connection established, and the IP headers are restricted to 5 words (20 bytes).
However the TCP headers may contain selective acks, tcp timestamps, both, or neither. Let's start with TCP headers with no options. An ack with no options and no payload will consist of a 5 word IP header followed by a 5 word TCP header followed by no data. So the total length in the IP header would be 40. If the packet was a fragment, it could conceal a payload in a subsequent fragment, but since we are working with the INPUT chain which is processed after IP fragmentation reassembly, we won't have to worry about this.
iptables -A INPUT -p tcp -d 10.2.3.4 --dport 1234 \
-m u32 --u32 "32>>28=5 && 0&0xFFFF=40" -j ACCEPT
The IP header is 20 bytes and the data offset nibble is in byte 12, so taking the 4 bytes starting at byte 32=20+12, we shift the nibble down and compare it to five, and then make sure that the total length in bytes 2 and 3 of word 0 of the IP header is 40.
If there are TCP timestamps in the TCP header, then there will be an additional 12 bytes (3 words) in the TCP header. We can accept that in a similar manner:
iptables -A INPUT -p tcp -d 10.2.3.4 --dport 1234 \
-m u32 --u32 "32>>28=8 && 0&0xFFFF=52" -j ACCEPT
I'll leave it as an exercise for the reader to work out the other combinations. (Note that dealing with selective ack is several cases as there can be 1-4 selective ack blocks, or 1-3 selective ack blocks with timestamps.)
DISCLAIMER: I did not actually try this out, so my apologies if there is a typo or if I overlooked something. I believe the strategy to be sound, and if there is any error or omission, let me know and I will correct.
I have problem with identifying communication established by TCP.
I have to identify first completed communication, for example first complete http communication.
I have dump .pcap file with capture. I know that communication should start by three way handshake ( SYN, SYN - ACK, ACK ) and then closing of communication by double FIN flag from both side.
But I have a lot of communication in that dump file.
So here is the question. Which things i need to remember to match exact one communication ?
I thought about source IP, destination IP, protocol, maybe port but i am not sure.
Thank you for every advice.
And sorry for my english.
You stated that you need:
To identify a particular conversation
To identify the first completed conversation
You can identify a particular TCP or UDP conversation by filtering for
the 5-tuple of the connection:
Source IP
Source Port
Destination IP
Destination Port
Transport (TCP or UDP)
As Shane mentioned, this is protocol dependent e.g. ICMP does not have the concept of
ports like TCP and UDP do.
A libpcap filter like the following would work for TCP and UDP:
tcp and host 1.1.1.1 and port 53523 and dst ip 1.1.1.2 and port 80
Apply it with tcpdump:
$ tcpdump -nnr myfile.pcap 'tcp and host 1.1.1.1 and port 53523 and dst ip 1.1.1.2 and port 80'
To identify the first completed connection you will have to follow the timestamps.
Using a tool like Bro to read a PCAP would yield the answer as it will list each connection
attempt seen (complete or incomplete):
$ bro -r myfile.pcap
$ bro-cut -d < conn.log | head -1
2014-03-14T10:00:09-0500 CPnl844qkZabYchIL7 1.1.1.1 57596 1.1.1.2 80 tcp http 0.271392 248 7775 SF F ShADadfF 14 1240 20 16606 (empty) US US
Use the flag data for TCP to judge whether there was a successful handshake and tear down.
For other protocols you can make judgements based on byte counts, sent and received.
Identifying the first completed communication is highly protocol specific. You are on the right track with your filters. If your protocol is a commonly used one there are plug ins called protocol analyzers and filters that can locate "conversations" for you from a pcap data stream. If you know approximate start time and end time that would help narrow it down too.
Do long BPF filters slow down tcpdump?
I replay a packet trace where all the packets have ttl=k and wait for ICMP messages back. What I've been noticing is that if I use the following filter (on eth0):
(ip and ip[8]=$k and src host $myAddress) or (icmp and dst host $myAddress and icmp[0]=11)
...I always miss 20-30 packets among the sent packets, whereas if I just do:
ip
... and then do the exact above filtering offline on the capture file, I find all the packets I had sent.
Is this a known behaviour?
If tcpdump is not fast enough to pop out captured packets from the queue, the kernel could drop some of them.
Look at the "XXXX packets dropped by kernel" message at the end of the dump to see if effectively some of them is lost.
Ensure to add the -n option to the command line. This will avoid DNS resolving and it will speed up a little (depending on your network)
is it possible to send multiple tcp or udp packets on a single ip packet? are there any specifications in the protocol that do not allow this.
if it is allowed by the protocol but is generally not done by tcp/udp implementations could you point me to the relevant portion in the linux source code that proves this.
are there any implementations of tcp/udp on some os that do send multiple packets on a single ip packet. (if it is allowed).
It is not possible.
The TCP seqment header does not describe its length. The length of the TCP payload is derived from the length of the IP packet(s) minus the length of the IP and TCP headers. So only one TCP segment per IP packet.
Conversely, however, a single TCP segment can be fragmented over several IP packets by IP fragmentation.
Tcp doesn't send packets: it is a continuous stream. You send messages.
Udp, being packet based, will only send one packet at a time.
The protocol itself does not allow it. It won't break, it just won't happen.
The suggestion to use tunneling is valid, but so is the warning.
You might want to try tunneling tcp over tcp, although it's generally considered a bad idea. Depending on your needs, your mileage may vary.
You may want to take a look at the Stream Control Transmission Protocol which allows multiple data streams across a single TCP connection.
EDIT - I wasn't aware that TCP doesn't have it's own header field so there would be no way of doing this without writing a custom TCP equivalent that contains this info. SCTP may still be of use though so I'll leave that link.
TCP is a public specification, why not just read it?
RFC4164 is the roadmap document, RFC793 is TCP itself, and RFC1122 contains some errata and shows how it fits together with the rest of the (IPv4) universe.
But in short, because the TCP header (RFC793 section 3.1) does not have a length field, TCP data extends from the end of the header padding to the end of the IP packet. There is nowhere to put another data segment in the packet.
You cannot pack several TCP packets into one IP packet - that is a restriction of specification as mentioned above. TCP is the closest API which is application-oriented. Or you want to program sending of raw IP messages? Just tell us, what problem do you want to solve. Think about how you organize the delivery of the messages from one application to another, or mention that you want to hook into TCP/IP stack. What I can suggest you:
Consider packing whatever you like into UDP packet. I am not sure, how easy is to initiate routing of "unpacked" TCP packages on remote side.
Consider using PPTP or similar tunnelling protocol.