I have some problems with pinging hosts with big packet size.
I'm doing some research on latency between one configuration and another.
I have written bash/zsh script to make it simpler that pings 8.8.8.8 and then parses the output to get average ping value. Script below:
#!/bin/zsh
for SIZE in {100..65500..100}
do
ping 8.8.8.8 -c 5 -s $SIZE > tempfile
TEMP=$(cat tempfile | tail -1 | awk '{print $4}' | cut -d / -f 2)
echo "$SIZE - $TEMP" | tee -a results
done
rm tempfile
The problem is that with big packets (more than 30kB when the standard value is 64b) ping just doesn't work, I don't get any response just like the server was unavailable. Interestingly, when I was in another location, everything worked fine (Internet connection is a bit worse there), problems there started about 60kB but here they start about 30kB and to get one (out of ~300 to the end and bigger=worse) result I need to call this script ~10-20 times. It's really annoying because for smaller packets it's working like a charm (with the same server 8.8.8.8).
Anybody has an idea what causes it?
This behavior is related with IP Fragmentation, when you packet size is bigger that your maximum transmission unit (MTU) takes more that one packet and become pretty inefficient, take a look in https://en.wikipedia.org/wiki/IP_fragmentation
Related
I am using "netstat -s -P tcp" to calculate retransmission. The output shows that tcpRetransBytes is more than tcpOutDataBytes. Three servers show this-- all are up for about a month. Is there any bug ? Note that if I use the command "netstat -s -P tcp 6" then I see tcpRetransBytes is less than tcpOutDataBytes.
I have a feeling that tcpOutDataBytes counter is reset while tcpRetransBytes keeps accumulating since the last reboot.
I cannot show the output of the command at the moment. Any help will be appreciate, PLEASE
I got my answer. tcpOutDataBytes is kept in a 32-bit integer. This is why when the value goes above 4255151339 or there about it starts again from 0. It's really shame that it is happening in Solaris 10
I am running a tunnel like this:
socat TCP-LISTEN:9090,fork TCP:192.168.1.3:9090
I would like to run a script to execute code with the strings passing through the tunnel.
The script does not change the strings, only processes strings independently but allows passage without changing between both ends.
Is this possible?
You should be able to even alter the communication using this approach:
Prepare a helper script helper.sh which gets executed for each connection:
#!/bin/bash
./inFilter.sh | socat - TCP:192.168.1.3:9090 | ./outFilter.sh
And start listening by using:
socat TCP-LISTEN:9090,fork EXEC:"./helper.sh"
The scripts inFilter.sh and outFilter.sh are processing the client and the server parts of the communication.
Example inFilter.sh:
#!/bin/bash
while read -r l ; do echo "IN(${l})" ; done
Example outFilter.sh:
#!/bin/bash
while read -r l ; do echo "OUT(${l})" ; done
This method should work for line-based text communication (as almost everything is line-buffered).
To make it work for binary protocols wrapping all processes with stdbuf -i0 -o0 might help (see .e.g here) and getting rid of shell is probably a good idea in this case.
Good luck!
I'm trying to set up a pipeline of processing commands with unix pipes and FIFO:s (named pipes).
I also wanted to send/stream the output of the process to another compute node, which can start working on the stream of data as soon as it arrives, that is, I want to "pipe the stdout over to a process on another machine". How can I do that?
E.g. is it possible to set up a FIFO that will in the background write it's content over to a FIFO on the other compute node, or similar?
You can use netcat. In this minimal example, you can pipe the output of cat to netcat using:
cat local-filename.txt | netcat remote-hostname 1234
Where 1234 represents the TCP port that is going to be used. In the receiving side, you can use:
netcat -l 1234 > filename-on-remote-host.txt
where -l indicates you are setting up a server. This connection will be closed when the originating netcat process finishes. If you need it to keep going and waiting for the next connection, you can use the -k option:
netcat -kl 1234 | some-receiving-command
In any case you can use the abbreviated nc instead of the full netcat:
nc -kl 1234 | some-receiving-command
Yes it is possible, just use ssh for this purpose. The stdin of ssh is sent to the other host. You can use it for example to send data to a different server using tar:
tar cvzf - data | ssh otherhost 'cd /tmp; tar xvzf -'
I'm trying to plot the TCP congestion window and the slow start threshold using iperf and the tcp_probe module. I do exactly what is told here:
to obtain the data:
modprobe tcp_probe port=5001
chmod 444 /proc/net/tcpprobe
cat /proc/net/tcpprobe >/tmp/tcpprobe.out &
TCPCAP=$!
iperf -i 10 -t 100 -c receiver
kill $TCPCAP
Oops!
/tmp/tcpprobe.out is empty :(
This is Ubuntu 11.04 x86
and already tried the same on Ubuntu 11.04 x64
Any suggestions?
I was having the same problem. What worked for me was:
modprobe -r tcp_probe
sudo modprobe tcp_probe port=5002 full=1
sudo chmod 444 /proc/net/tcpprobe
cat /proc/net/tcpprobe > /tmp/tcpprobe.out &
TCPCAP=$!
iperf -c <servers IP address here> -p 5002 -t 100 -i 1
sudo kill $TCPCAP
See iperf parameters to check if those (-t 100 -i 1) are what you need by typing:
man iperf
I/O function in C standard library use buffer by default, usually 4k , so fread() only return when buffer full or EOF. You can use small buffer, 128 bytes, see:
dd if=/proc/net/tcpprobe ibs=128 obs=128
Now, message flush quickly.
By default the tcp_probe logs only when the cnwd changes, try modprobe tcp_probe ... full=1.
Linux source code referece: http://www.cs.fsu.edu/~baker/devices/lxr/http/source/linux/net/ipv4/tcp_probe.c#L47
I had similiar issue, the tcp_probe module outputs only in non-obvious time intervals. I've created a modified version of it that flushes on every received tcp segments. This slows down the system, but allows to better monitor short-lived connections like HTTP.
Find the source code to the module here.
another issue which causes no output is file permission of output file tcpprobe.out
when cat tcpprobe directly, it's able to see the output, but if redirecting the output the file, the output file size is 0, which reminds me that it's the permission issue...
A very late answer, but have been struggling with this issue myself. I was trying out the version Dyna provided, yet still got no output, regardless of the parameters used. In the end, I found that the order was the problem.
The way I was using tcp_probe was: install/activate the module, run some tcp application (I was running some tcp unit tests), then start the copy process for /proc/net/tcpprobe (as shown in the other answers) and then remove/stop the module. The correct way is to start the copy process (barring killing of the process) BEFORE you perform the tcp intensive activity. Keep the cat process running while you perform the tcp activity and only kill it afterwards.
A pretty humbling experience for me, as it took hours to figure this out. Hopefully, people find this useful.
I have some applications, and standard Unix tools sending their output to named-pipes in Solaris, however named pipes can only be read from the local storage (on Solaris), so I can't access them from over the network or place the pipes on an NFS storage for networked access to their output.
Which got me wondering if there was an analogous way to forward the output of command-line tools directly to sockets, say something like:
mksocket mysocket:12345
vmstat 1 > mysocket 2>&1
Netcat is great for this. Here's a page with some common examples.
Usage for your case might look something like this:
Server listens for a connection, then sends output to it:
server$ my_script | nc -l 7777
Remote client connects to server on port 7777, receives data, saves to a log file:
client$ nc server 7777 >> /var/log/archive
netcat (also known as nc) is exactly what you're looking for. It's getting to be reasonably standard, but not available on all systems.
socat seems to be a beefed-up version of netcat, with lots more features, but less commonly available.
On Linux, you can also use /dev/tcp/<host>/<port>. See the Advanced Bash-Scripting Guide for more information.
netcat will help establish a pipe over the network.
You may want to use one of:
ssh: secure (encrypted), already installed out-of-the-box on Solaris - but you have to set up a keypair for non-interactive sessions
e.g. vmstat 2>&1 | ssh -i private.key oss#remote.node "cat >vmstat.out"
netcat: simple to set up - but insecure and open to attacks
see http://www.debian-administration.org/articles/58 etc.
Everyone is on the right track with netcat. But I want to add that if you are piping into nc and expecting a response, you will need to use the -q <seconds> option. From the manual:
-q seconds
after EOF on stdin, wait the specified number of seconds and then quit. If seconds is negative, wait forever.
For instance, if you want to interact with your SSH Agent you can do something like this:
echo -en '\x00\x00\x00\x01\x0b' | nc -q 1 -U $SSH_AUTH_SOCK | strings
A more complete example is at https://gist.github.com/RichardBronosky/514dbbcd20a9ed77661fc3db9d1f93e4
* I stole this from https://ptspts.blogspot.com/2010/06/how-to-use-ssh-agent-programmatically.html