Here i am testing a telnet connection to a management-ip of device.
There is no telnet server running on 23 port. In fact no applications is listening on 23.Ideally the telnet connection should be refused when ever i tried to connect.But some times the connections is timing out instead of refusing.
Ideal Case:
$ telnet 10.145.65.46
Trying 10.145.65.46...
telnet: connect to address 10.145.65.46: Connection refused
Timedout case:
$ telnet 10.145.65.46
Trying 10.145.65.46...
Connected to 10.145.65.46.
Escape character is '^]'.
Connection closed by foreign host.
please find tcpdump on device which has above ip address in both cases.
Here
seadev01.olympus.f5net.com as telnet client
AdithyaVe1.com.telnet as telnet server
Ideal case:
3:02:02.950535 IP seadev01.olympus.f5net.com.39270 > AdithyaVe1.com.telnet: Flags [S], seq 853690436, win 14600, options [mss 1460,sackOK,TS val 2527350559 ecr 0,nop,wscale 7], length 0
23:02:02.950609 IP AdithyaVe1.com.telnet > seadev01.olympus.f5net.com.39270: Flags [R.], seq 0, ack 853690437, win 0, length 0
in above case the connection is straight away rejected by sending RST packet.
Timed-out case:
1 Packet - 23:02:12.284611 IP seadev01.olympus.f5net.com.39272 > AdithyaVe1.com.telnet: Flags [S], seq 2750626194, win 14600, options [mss 536], length 0
2 Packet - 23:02:12.284657 IP AdithyaVe1.com.telnet > seadev01.olympus.f5net.com.39272: Flags [R.], seq 0, ack 2750626195, win 0, length 0
3 Packet - 23:02:13.284453 IP seadev01.olympus.f5net.com.39272 > AdithyaVe1.com.telnet: Flags [S], seq 2750626194, win 14600, options [mss 536], length 0
4 Packet - 23:02:13.284515 IP AdithyaVe1.com.telnet > seadev01.olympus.f5net.com.39272: Flags [R.], seq 0, ack 1, win 0, length 0
5 Packet - 23:02:14.284836 IP seadev01.olympus.f5net.com.39272 > AdithyaVe1.com.telnet: Flags [S], seq 2750626194, win 14600, options [mss 536], length 0
6 Packet - 23:02:14.284887 IP AdithyaVe1.com.telnet > seadev01.olympus.f5net.com.39272: Flags [R.], seq 0, ack 1, win 0, length 0
7 Packet - 23:02:15.284560 IP seadev01.olympus.f5net.com.39272 > AdithyaVe1.com.telnet: Flags [R.], seq 1, ack 2802410946, win 0, length 0
In this case also device is sending RST packet but the connection is not
terminated.Suppose the RST(2nd)packet is dropped some where in the n/w. so telnet client is again sending SYN(3rd) packet but how the ack number is 1 in 4th packet. i am not understating why tcp is sending 6,7 packets to client.
How the connection is closed in 7th packet even though it has wrong ack seq number 2802410946 instead of 2750626195(SYN seq number(2750626194) +1)
Related
I'm using TCPSampler in Jmeter to do the performance test on fluentd's TCP input plugin.
But it always runs into 500 error:
Thread Name:Thread Group 1-1
Sample Start:2020-06-23 18:21:08 CST
Load time:2372
Connect Time:360
Latency:0
Size in bytes:0
Sent bytes:0
Headers size in bytes:0
Body size in bytes:0
Sample Count:1
Error Count:1
Data type ("text"|"bin"|""):text
Response code:500
Response message:org.apache.jmeter.protocol.tcp.sampler.ReadException: Error reading from server, bytes read: 0
SampleResult fields:
ContentType:
DataEncoding: UTF-8
My setting to TCP Sampler is simple:
Because fluentd returns nothing(0-length response) and there is no EOL can be set. Jmeter will hang up until timeout, then reports 500(verified with another simple TCP server with non-0 response).
How should I handle the 0-length response from the TCP server?
Appreciate!
========
Seen from the pcap of tcpdump on fluentd side, the package from jmeter has been 'ACK'-ed immediately.
# tcpdump -i ens160 -n tcp port 20001
reading from file from-jmeter.pcap, link-type EN10MB (Ethernet)
06:12:23.716615 IP BEI-L-00040107.olympus.****.com.50411 > telemetry-analytics.pdsea.****.com.microsan: Flags [S], seq 3419810629, win 65535, options [mss 1304,nop,wscale 6,nop,nop,TS val 1134449538 ecr 0,sackOK,eol], length 0
06:12:23.716709 IP telemetry-analytics.pdsea.****.com.microsan > BEI-L-00040107.olympus.****.com.50411: Flags [S.], seq 1061319631, ack 3419810630, win 28960, options [mss 1460,sackOK,TS val 1980947477 ecr 1134449538,nop,wscale 7], length 0
06:12:23.919891 IP BEI-L-00040107.olympus.****.com.50411 > telemetry-analytics.pdsea.****.com.microsan: Flags [.], ack 1, win 2059, options [nop,nop,TS val 1134449760 ecr 1980947477], length 0
06:12:23.920121 IP BEI-L-00040107.olympus.****.com.50411 > telemetry-analytics.pdsea.****.com.microsan: Flags [P.], seq 1:4, ack 1, win 2059, options [nop,nop,TS val 1134449761 ecr 1980947477], length 3
06:12:23.920147 IP telemetry-analytics.pdsea.****.com.microsan > BEI-L-00040107.olympus.****.com.50411: Flags [.], ack 4, win 227, options [nop,nop,TS val 1980947681 ecr 1134449761], length 0
06:12:25.994011 IP BEI-L-00040107.olympus.****.com.50411 > telemetry-analytics.pdsea.****.com.microsan: Flags [F.], seq 4, ack 1, win 2059, options [nop,nop,TS val 1134451735 ecr 1980947681], length 0
06:12:25.994402 IP telemetry-analytics.pdsea.****.com.microsan > BEI-L-00040107.olympus.****.com.50411: Flags [F.], seq 1, ack 5, win 227, options [nop,nop,TS val 1980949755 ecr 1134451735], length 0
06:12:26.220944 IP BEI-L-00040107.olympus.****.com.50411 > telemetry-analytics.pdsea.****.com.microsan: Flags [.], ack 2, win 2059, options [nop,nop,TS val 1134452013 ecr 1980949755], length 0
If you're basically waiting for an empty response and just want JMeter to stop marking the request as failed you can add a Response Assertion as a child of the TCP Sampler and configure it like:
This way JMeter will mark the sampler as passed given status code is 500.
More information: Response Assertions in JMeter 3.2 - New and Improved
I have an HTTP server and a client (I use curl for testing). The problem I have is that even the simplest, "ping" request takes 3-5 seconds to accomplish. I have run tcpdump on the server and it seems that after establishing TCP connection, the client (curl) waits 3-4 seconds for sending the HTTP request:
09:06:55.079376 IP client > server: Flags [S], seq 1639466412, win 64240, options [mss 1358,nop,wscale 8,nop,nop,sackOK], length 0
09:06:55.079457 IP server > client: Flags [S.], seq 2519785093, ack 1639466413, win 29200, options [mss 1460,nop,nop,sackOK,nop,wscale 7], length 0
09:06:55.087145 IP client > server: Flags [.], ack 1, win 1029, length 0
09:06:59.722060 IP client > server: Flags [P.], seq 1:83, ack 1, win 1029, length 82: HTTP: GET /ping HTTP/1.1
You can see the problem between 3 and 4 line of the tcpdump output.
Do you know what can cause this issue? Is this really on the client side or I should check also on the server side?
UPDATE
If I use telnet to connect to the server's port and send GET, the server replies immediately.
Flags [S] Syn
Flags [S.] - Syn Ack
Flags [.] - Placeholder, usually used for ACK.
Flags [P.] - Push Ack
Can you prepend time to your curl command? as shown below.
These hosts, is there latency between the two?
As a test on the same host (terminal):
terminal a $ python -m SimpleHTTPServer 8989
terminal b $ time curl -vvv localhost:8989
results look something like:
real 0m0.018s
user 0m0.006s
sys 0m0.005s
At this point its hard to say if this is client or server side, this depends on the data you are attempting to query. Can you share that query/curl command?
I am trying to get TCP. From TCP RFC 793 server and client select a random sequence numbers and after they increase it each time when it receives a new byte (it is wrong but just for example). To dump TCP packages I used tcpdump -n -i eth0 tcp:
listening on eth0, link-type EN10MB (Ethernet), capture size 96 bytes
04:32:20.732914 IP 10.10.0.2.43168 > 10.50.0.2.9: S 372254521:372254521(0)
win 5840 <mss 1460,sackOK,timestamp 3644068 0,nop,wscale 1>
04:32:20.766194 IP 10.50.0.2.9 > 10.10.0.2.43168: S 363863555:363863555(0)
ack 372254522 win 5792 <mss 536,sackOK,timestamp 3644074 3644068,nop,wscale 1>
04:32:20.766416 IP 10.10.0.2.43168 > 10.50.0.2.9: .
ack 1 win 2920 <nop,nop,timestamp 3644073 3644074>
04:32:25.502532 IP 10.10.0.2.43168 > 10.50.0.2.9: P 1:7(6)
ack 1 win 2920 <nop,nop,timestamp 3644548 3644074>
04:32:25.503272 IP 10.50.0.2.9 > 10.10.0.2.43168: .
ack 7 win 2896 <nop,nop,timestamp 3644548 3644548>
04:32:29.510131 IP 10.10.0.2.43168 > 10.50.0.2.9: F 7:7(0)
ack 1 win 2920 <nop,nop,timestamp 3644949 3644548>
04:32:29.513123 IP 10.50.0.2.9 > 10.10.0.2.43168: F 1:1(0)
ack 8 win 2896 <nop,nop,timestamp 3644949 3644949>
04:32:29.513356 IP 10.10.0.2.43168 > 10.50.0.2.9: .
ack 2 win 2920 <nop,nop,timestamp 3644949 3644949>
The first two packages look normally but from the third and so on it uses ack 1 instead of 363863556 and I can't get why?
It hasn't. You are running tcpdump without telling it that you want to see absolute sequence numbers (-S).
The default behavior for tcpdump is to translate sequence numbers to relative sequence numbers, which allow you to see how many bytes of data have transferred in either direction. In this specific case, you are seeing it go to 1 because, according to RFC-793, the SYN consumes one byte in the stream, so the proper response is SEQ+1. You will see the same thing happen going the other direction. (You will also find that the FIN consumes one byte). Following this, the ACK will increase by the number of bytes sent.
If you want to see the absolute sequence numbers, try again running tcpdump -n -i eth0 -S tcp
I have an ubuntu 12.04 server with nginx at port 80
There is only a single firewall rule, which relates to port mapping port 26 to 25
nginx is set to listen on port 80, initially in a fairly default manner but now with
listen x.x.x.x:80 backlog=5000;
nginx is not that loaded, about 50 requests a second says nginx_status
Active connections: 480
server accepts handled requests
84618 84618 143733
Reading: 0 Writing: 4 Waiting: 474
Some (a very few as a percentage) users complain that one of their computers (for example "it only happens at home") appears to have its SYN packets ignored. They can ping without loss. sometimes they get some responses to tcp requests. They can get responses on quiet ports for example the pop server. Generally however they experience long time outs. I have packet dumps from them that show this.
On my end, I can also see that some IP addresses are being ignored.
Here for example, multiple SYN packets from port 2010 to port 80 are not replied, while the server is fulfilling a prior connection on port 2031
02:21:46.950979 IP 72.38.0.37.2010 > 64.91.255.98.80: Flags [S], seq 3835139709, win 65535, options [mss 1460,nop,wscale 3,nop,nop,TS val 0 ecr 0,nop,nop,sackOK], length 0
02:21:49.887320 IP 72.38.0.37.2010 > 64.91.255.98.80: Flags [S], seq 3835139709, win 65535, options [mss 1460,nop,wscale 3,nop,nop,TS val 0 ecr 0,nop,nop,sackOK], length 0
02:21:55.923151 IP 72.38.0.37.2010 > 64.91.255.98.80: Flags [S], seq 3835139709, win 65535, options [mss 1460,nop,wscale 3,nop,nop,TS val 0 ecr 0,nop,nop,sackOK], length 0
02:22:24.950448 IP 72.38.0.37.2031 > 64.91.255.98.80: Flags [S], seq 4138069869, win 65535, options [mss 1460,nop,wscale 3,nop,nop,TS val 0 ecr 0,nop,nop,sackOK], length 0
02:22:24.950488 IP 64.91.255.98.80 > 72.38.0.37.2031: Flags [S.], seq 248034551, ack 4138069870, win 14480, options [mss 1460,sackOK,TS val 240617577 ecr 0,nop,wscale 7], length 0
02:22:24.982809 IP 72.38.0.37.2031 > 64.91.255.98.80: Flags [.], ack 1, win 50112, options [nop,nop,TS val 372774 ecr 240617577], length 0
02:22:24.982852 IP 72.38.0.37.2031 > 64.91.255.98.80: Flags [P.], seq 1:526, ack 1, win 50112, options [nop,nop,TS val 372774 ecr 240617577], length 525
02:22:24.982869 IP 64.91.255.98.80 > 72.38.0.37.2031: Flags [.], ack 526, win 122, options [nop,nop,TS val 240617585 ecr 372774], length 0
02:22:25.016783 IP 64.91.255.98.80 > 72.38.0.37.2031: Flags [P.], seq 1:265, ack 526, win 122, options [nop,nop,TS val 240617594 ecr 372774], length 264
02:22:25.190570 IP 72.38.0.37.2031 > 64.91.255.98.80: Flags [.], ack 265, win 50079, options [nop,nop,TS val 372777 ecr 240617594], length 0
02:22:45.017288 IP 64.91.255.98.80 > 72.38.0.37.2031: Flags [F.], seq 265, ack 526, win 122, options [nop,nop,TS val 240622594 ecr 372777], length 0
02:22:45.049437 IP 72.38.0.37.2031 > 64.91.255.98.80: Flags [.], ack 266, win 50079, options [nop,nop,TS val 372976 ecr 240622594], length 0
02:22:49.998299 IP 72.38.0.37.2031 > 64.91.255.98.80: Flags [R.], seq 526, ack 266, win 0, length 0
02:23:18.883263 IP 72.38.0.37.2059 > 64.91.255.98.80: Flags [S], seq 2419025537, win 65535, options [mss 1460,nop,wscale 3,nop,nop,TS val 0 ecr 0,nop,nop,sackOK], length 0
02:23:21.890861 IP 72.38.0.37.2059 > 64.91.255.98.80: Flags [S], seq 2419025537, win 65535, options [mss 1460,nop,wscale 3,nop,nop,TS val 0 ecr 0,nop,nop,sackOK], length 0
Much more simply, near the start of a 20 second period, here is a lone packet from one IP that is not paired with any other packets (to this host):
2:48:05.141703 IP 96.48.197.237.1275 > 64.91.255.98.80: Flags [S], seq 2682822499, win 65535, options [mss 1460,nop,wscale 2,nop,nop,TS val 0 ecr 0,nop,nop,sackOK], length 0
I wrote a perl script to watch a tcpdump and find / report the number of dangling SYNs, it finds a few every few seconds (as we go along the accumulated count of never-replied TCP SYN packets rises steadily). The rate of apparently unreplied SYNs is about 1 in 2500. When I ping those IPs, assuming they are pingable, there is no packet loss, no issue communicating with them.
Nothing useful (such as "sending syncookies") is in the kernel log.
nginx has
worker_processes 8
worker_connections 4096
keepalive is on, open_file_cache module is in use, but I'm struggling to see what other variables could silently ignore SYN packets but only and repeatably for specific IPs.
beyond the default ubuntu setup, sysctl.conf has
# increased
net.ipv4.tcp_fin_timeout = 10
net.ipv4.ip_local_port_range = 1024 65535
net.core.somaxconn = 1024
# default
net.ipv4.tcp_tw_reuse = 0
# default
net.netfilter.nf_conntrack_tcp_loose = 1
net.ipv4.netfilter.ip_conntrack_tcp_loose = 1
# reduced
net.netfilter.nf_conntrack_tcp_timeout_established = 86400
net.ipv4.tcp_ecn = 0
I've not experience this problem before, with the same audience, an earlier kernel nginx, different hardware (this is a virtual server). Different data center.
My "canary in a coal mine" reports that from their perspective they see the timeouts and lack of replies on their XP machine, but not if it goes via a linux machine setup as a proxy. So they are investigating that. However whatever their conclusion I'm not sure why I can sniff incoming SYN packets to port 80 with no subsequent reply packet sent out on the same interface.
Based on the information here
https://serverfault.com/questions/235965/why-would-a-server-not-send-a-syn-ack-packet-in-response-to-a-syn-packet
Turning off TCP timestamps on the server stopped SYNs getting dropped from Windows XP clients that send SYN packets with tsval set to zero, and the number of unreplied SYNs on the server went to zero and stayed there.
sysctl -w net.ipv4.tcp_timestamps = 0
My understanding is the behaviour of the XP stack when timestamps are enabled is well known as it has been discussed on the linux lists to do with ipv4, and at some point in time linux with tcp_timestamps enabled simply switched to non timestamp session with XP (or other buggy clients). It seems this behaviour has changed and now at least on busy ports, SYN packet with tsval 0 are dropped if tcp_timestamps is 1
Does your system have three way syn checking enabled? It sounds like it's checking for a three way handshake and some packets are not passing this check and even though they are not malicious, they are being dropped.
Check or post all of your configs and or check your router/firewall as this option is usually set by default.
When I type www.google.com at my browser address bar, what exactly happens technically and how entire stuff is loaded. Considering the same HTTP page is being loaded...
what is role of DNS server, IP address, MAC address, subnet mask, proxy setting, default gateway in this case.
Does it make any different if I am in different class of network?
You are asking about all things at once, it's a big concept.
Still in short.
When you type www.google.com (or any other site name) then the request goes to the DNS server which translates the URL into an IP address.
Read here more: http://en.wikipedia.org/wiki/Domain_Name_System
Then the request goes to server where the website is being hosted, the server which is providing hosting service for the website contains the website-stuff that has to be shown to the world.
Read about apache server: http://en.wikipedia.org/wiki/Apache_HTTP_Server
Subnet: http://en.wikipedia.org/wiki/Subnetwork
does it make any different if i am in different class of network?
No, it doesn't make any difference if you are in different class of network.
(Study about routers: http://en.wikipedia.org/wiki/Routers)
Points to help you out:
Every computer that belongs to a network -including yours has an ip address.
Every network has hosts under it. The network may be divided into subnets
The ip addressing is hierarchial.This helps in routing
IP addresses may be assigned manually or by the DHCP server
Manual-IP configuration
DHCP-Dynamic Host Configuration Protocol
All packets that are sent to your ip address come through your isp network - this includes switches and routers
Packets from other networks are forwarded by this ip address. Once they reach the nearest switch. Switches use your MAC address to send packets to your computer. The MAC address is obtained by ARP
The gateway address is the path through which packets are sent out of your network or your ISP's
Proxy servers are servers that allow connections through them
To understand more about how this works, download Wireshark:
Start the sniffer and then load google.com in your browser.
You will notice the following
The browser first sends a DNS request with the hostname to the DNS server of your ISP (or your network if any)- DNS finds out the ip address from the hostname.
The DNS server replies with the IP address of the server
The browser then sends the HTTP request
This is in the form e.g
GET /index.html HTTP/1.1
The server responds with the format
The data is sent to the user
Usually if a webpage is requested, it is in HTML format(with javascript,css,etc). This is then parsed and processed by the browser to get the webpage we see.
To test this :
ON LINUX,
type
telnet stackoverflow.com 80
in the terminal.
As soon as it gets connected,type the following (quickly before it gets disconnected):
GET /index.html HTTP/1.1 (enter)
Host: stackoverflow.com (enter)(enter)
to see the response
ON WINDOWS
Download the putty client and fill the host as stackoverflow.com, port as 80, and choose Connection type as Telnet.
As soon as it gets connected, repeat same steps as above to see the response.
The examples shown above illustrate how things work from Layer 7, but not Layer 3+ from your device's perspective. I would look at using tcpdump/wireshark to try and dump all of the network packets if you're interested in getting those kinds of low-level details. An example's provided below (run on FreeBSD).
Notes:
- Be sure to start wireshark/tcpdump before your web browser/client first so the packets get captured.
- Specify the right port when starting wireshark/tcpdump; filtering the connection via a DNS address might not work in all cases if the remote webserver has load balancing/failover setup.
Window with tcpdump:
# tcpdump -A tcp port 80
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on em0, link-type EN10MB (Ethernet), capture size 65535 bytes
capability mode sandbox enabled
01:48:17.917640 IP 10.0.2.15.50636 > nuq05s01-in-f20.1e100.net.http: Flags [P.], seq 1631738201:1631738227, ack 30720002, win 65535, length 26
E..BP.#.#...
...J}.t...PaBYY....P...F5..GET /index.html HTTP/1.1
01:48:17.918119 IP nuq05s01-in-f20.1e100.net.http > 10.0.2.15.50636: Flags [.], ack 26, win 65535, length 0
E..(.U..#..{J}.t
....P......aBYsP...'+........
01:48:18.072501 IP 10.0.2.15.50636 > nuq05s01-in-f20.1e100.net.http: Flags [P.], seq 26:28, ack 1, win 65535, length 2
E..*P.#.#...
...J}.t...PaBYs....P...F...
01:48:18.072662 IP nuq05s01-in-f20.1e100.net.http > 10.0.2.15.50636: Flags [.], ack 28, win 65535, length 0
E..(.X..#..xJ}.t
....P......aBYuP...')........
01:48:18.074353 IP nuq05s01-in-f20.1e100.net.http > 10.0.2.15.50636: Flags [P.], seq 1:687, ack 28, win 65535, length 686
E....Y..#...J}.t
....P......aBYuP...Z...HTTP/1.0 400 Bad request: request header 'Host' missing
Content-type: text/html; charset="iso-8859-1"
<html>
<body>
<h3> Request denied by WatchGuard HTTP proxy. </h3>
<b> Reason: </b> request header 'Host' missing <br>
<hr size="1" noshade>
<b> Method: </b> GET <br>
<b> Host: </b> 74.125.239.116 <br>
<b> Path: </b> /index.html <br>
<hr size="1" noshade>
</body>
<!-- PAD --></html>
01:48:18.074512 IP nuq05s01-in-f20.1e100.net.http > 10.0.2.15.50636: Flags [F.], seq 687, ack 28, win 65535, length 0
E..(.Z..#..vJ}.t
....P......aBYuP...$z........
01:48:18.074683 IP 10.0.2.15.50636 > nuq05s01-in-f20.1e100.net.http: Flags [.], ack 688, win 65014, length 0
E..(P.#.#...
...J}.t...PaBYu....P...F...
01:48:18.077023 IP 10.0.2.15.50636 > nuq05s01-in-f20.1e100.net.http: Flags [F.], seq 28, ack 688, win 65535, length 0
E..(P.#.#...
...J}.t...PaBYu....P...F...
01:48:18.077070 IP nuq05s01-in-f20.1e100.net.http > 10.0.2.15.50636: Flags [.], ack 29, win 65535, length 0
E..(.[..#..uJ}.t
....P......aBYvP...$y........
Window with telnet:
# telnet www.google.com 80
Trying 74.125.239.116...
Connected to www.google.com.
Escape character is '^]'.
GET /index.html HTTP/1.1
HTTP/1.0 400 Bad request: request header 'Host' missing
Content-type: text/html; charset="iso-8859-1"
<html>
<body>
<h3> Request denied by WatchGuard HTTP proxy. </h3>
<b> Reason: </b> request header 'Host' missing <br>
<hr size="1" noshade>
<b> Method: </b> GET <br>
<b> Host: </b> 74.125.239.116 <br>
<b> Path: </b> /index.html <br>
<hr size="1" noshade>
</body>
<!-- PAD --></html>
Connection closed by foreign host.