I have a lot of interfaces configured in my server, each of which connect to a specific nic card and have a separate routing table. These interfaces can be identified by "netstat -a" command.
Now, I want to execute the rsync command connecting only to specific interface. I have this requirement because each of the interfaces will go through a separate tunnel/path and I want a particular rsync command to sync files through a specified tunnel.
Specifically, I want a way to specify the interface name.
Thanks,
Mohan.
You can specify the address of the interface using --address=x.x.x.x on the command-line.
I don't think there is any way to specify the interface directly, but the ip command can tell you the address for an interface, so you could use something like this:
IP=$(ip -4 -br addr show eth0 | awk '{split($3,a,"/"); print a[1]}')
rsyncd ... --address=$IP
Edit For systems with the "real" iproute2 (anything not busybox-based, essentially), ip can produce JSON output which can be parsed a bit more sanely:
IP=$(ip -j -4 addr show wlo1 | jq .[0].addr_info[0].local)
rsyncd ... --address=$IP
I've written this little perl script to turn interface names to addresses, save it as iftoip (or similar)
#!/usr/bin/perl
use strict;
use warnings;
use IO::Interface::Simple;
use feature qw(say);
my $iface = shift;
my $if = IO::Interface::Simple->new($iface) or die "$!: $iface";
say $if->address;
exit 0;
You can do something similar with bash:
iftoip() {
ip addr show $1 | grep inet | awk '{print $2}' | cut -d'/' -f1
}
just add the above 3 lines to ~/.bashrc and start a new shell or source ~/.bashrc
Running it produces:
v#juno:~$ iftoip ens33
10.251.17.94
v#juno:~$ iftoip ens34
192.168.78.128
v#juno:~$ echo "IP=$(iftoip ens33)"
IP=10.251.17.94
v#juno:~$ iftoip ens35 #perl
No such device: ens35 at /home/v/bin/iftoip line 10.
or
v#juno:~$ iftoip ens35 #bash
Device "ens35" does not exist.
This has been tried using 2 interfaces, with different subnets and worked.
rsync -avzP -e 'ssh -b 10.100.16.X' /var/tmp/ent1 10.100.16.X:/var/tmp/;
rsync -avzP -e 'ssh -b 10.100.20.X' /var/tmp/ent2 10.100.20.X:/var/tmp/ ;
From client to server, over ssh use:
rsync -avP -e 'ssh -b x.x.x.x' tmp/ server:tmp/
Related
Docker creates these virtual ethernet interfaces veth[UNIQUE ID] listed in ifconfig. How can I find out which interface belongs to a specific docker container?
I want to listen to the tcp traffic.
To locate interface
In my case getting value from container was like (check eth0 to):
$ docker exec -it my-container cat /sys/class/net/eth1/iflink
123
And then:
$ ip ad | grep 123
123: vethd3234u4#if122: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue master docker_gwbridge state UP group default
Check with tcpdump -i vethd3234u4
Reference about mysterious iflink from http://lxr.free-electrons.com/source/Documentation/ABI/testing/sysfs-class-net:
150 What: /sys/class/net/<iface>/iflink
151 Date: April 2005
152 KernelVersion: 2.6.12
153 Contact: netdev#vger.kernel.org
154 Description:
155 Indicates the system-wide interface unique index identifier a
156 the interface is linked to. Format is decimal. This attribute is
157 used to resolve interfaces chaining, linking and stacking.
158 Physical interfaces have the same 'ifindex' and 'iflink' values.
Based on the provided answer (which worked for me), I made this simple bash script:
#!/bin/bash
export containers=$(sudo docker ps --format "{{.ID}}|{{.Names}}")
export interfaces=$(sudo ip ad);
for x in $containers
do
export name=$(echo "$x" |cut -d '|' -f 2);
export id=$(echo "$x"|cut -d '|' -f 1)
export ifaceNum="$(echo $(sudo docker exec -it "$id" cat /sys/class/net/eth0/iflink) | sed s/[^0-9]*//g):"
export ifaceStr=$( echo "$interfaces" | grep $ifaceNum | cut -d ':' -f 2 | cut -d '#' -f 1);
echo -e "$name: $ifaceStr";
done
My answer more like improvement on that important topic because it didn't help to "Find out which network interface belongs to docker container", but, as author noticed, he "want to listen to the tcp traffic" inside docker container - I'll try to help on that one during your troubleshooting of network.
Considering that veth network devices are about network namespaces, it is useful to know that we can execute program in another namespace via nsenter tool as follow (remember - you need a privileged permission (sudo/root) for doing that):
Get ID of any container you are interested in capture the traffic, for example it will be 78334270b8f8
Then we need to take PID of that containerized application (I assume you are running only 1 network-related process inside container and want to capture its traffic. Otherwise, that approach is hard to be suitable):
sudo docker inspect 78334270b8f8 | grep -i pid
For example, output for pid will be 111380 - that's ID of your containerized app, you can check also it via ps command: ps aux | grep 111380 just in curiosity.
Next step is to check what network interfaces you have inside your container:
sudo nsenter -t 111380 -n ifconfig
This command will return you list of network devices in network namespace of the containerized app (you should not have ifconfig tool on board of your container, only on your node/machine)
For example, you need to capture traffic on interface eth2 and filter it to tcp destination port 80 (it may vary of course) with this command:
sudo nsenter -t 111380 -n tcpdump -nni eth2 -w nginx_tcpdump_test.pcap 'tcp dst port 80'
Remember, that in this case you do not need tcpdump tool to be installed inside your container.
Then, after capturing packets, .pcap file will be available on your machine/node and to read it use any tool you prefer tcpdump -r nginx_tcpdump_test.pcap
approach's pros:
no need to have network tools inside container, only on docker node
no need to search for map between network devices in container and node
cons:
you need to have privileged user on node/machine to run nsenter tool
One-liner of the solution from #pbaranski
num=$(docker exec -i my-container cat /sys/class/net/eth0/iflink | tr -d '\r'); ip ad | grep -oE "^${num}: veth[^#]+" | awk '{print $2}'
If you need to find out on a container that does not include cat then try this tool: https://github.com/micahculpepper/dockerveth
You can also read the interface names via /proc/PID/net/igmp like (container name as argument 1):
#!/bin/bash
NAME=$1
PID=$(docker inspect $NAME --format "{{.State.Pid}}")
while read iface id; do
[[ "$iface" == lo ]] && continue
veth=$(ip -br addr | sed -nre "s/(veth.*)#if$id.*/\1/p")
echo -e "$NAME\t$iface\t$veth"
done < <(</proc/$PID/net/igmp awk '/^[0-9]+/{print $2 " " $1;}')
For Example,
I have two servers namely A and B. I want to use a grep command in either A or B, which will search in both A and B servers and display the match.
You could use parallel ssh (pssh) for that.
See this command:
parallel-ssh -P -v -l root -A -H "192.168.1.1 192.168.1.2 192.168.1.3" "hostname"
Where hostname is the command to execute on each of the hosts. -P means print the output of the command, -l root means login with the user root, -A ask for the password and -H provides the list of hosts.
The output might look similar to this:
192.168.140.193: hostname1
192.168.140.194: hostname2
192.168.140.195: hostname3
[1] 11:18:17 [SUCCESS] 192.168.140.193
[2] 11:18:17 [SUCCESS] 192.168.140.194
[3] 11:18:17 [SUCCESS] 192.168.140.195
For those without access to parallel-ssh, try this:
#!/bin/bash
remotehost='hostnameA'
if [ `hostname` == 'hostnameA' ]
then
remotehost='hostnameB'
fi
{
grep whatever
ssh $remotehost grep whatever
}
The first part figures out what host you're on and which it needs to ssh into. The second part performs the command on both hosts and groups the output together.
I have a command line application which I use and also have to pass my local ip address as an argument, like:
jekyll --url 'http://192.168.1.2:3000' --pygments --safe --server 3000 --auto
I would like to make the url argument get my ip automatically, since I am always on different networks and get different loal ip addresses.
so I can use this alias in my .bashrc
alias jkl="jekyll --url 'http://$IP:3000' --pygments --safe --server 3000 --auto"
where $IP would be my local ip adress acquired dynamically.
Is there any way to do it?
First, use double quotes instead of single quotes around your $IP variable or else it won't interpolate the value
#!/bin/bash
# tested on bash 4
while read -r line
do
case "$line" in
"inet "* )
line="${line/inet /}"
line="${line%% *}"
if [[ ! $line =~ ^(127|172) ]] ;then
IP="$line"
echo "IP: $IP"
fi
;;
esac
done < <(ifconfig)
echo jekyll --url "http://$IP:3000" --pygments --safe --server 3000 --auto
Note that you will have a few different IPs in the output. Choose the one that fits your requirement most.
A computer does not necessarily have "a local IP address", there are often several. For instance, you typically have the localhost address (127.0.0.1), and one or more "true" externally visible addresses. It's hard for an automated solution to know which one to pick.
One easy solution is perhaps to hard-code the "eth0" interface (or whatever the name is of your most typical interface).
On Linux, you could use something like this:
$ ifconfig | grep -A1 eth0 | cut -d: -f2 | cut -d ' ' -f1 | grep \\.
192.168.0.8
So to stuff this into a variable (assuming bash) you would use
MY_IP=$(ifconfig | grep -A1 eth0 | cut -d: -f2 | cut -d ' ' -f1 | grep \\.)
Note that this hard-codes the interface name as eth0.
I'm using plink to run a command on a Unix remote machine.
The command is:
ls -1trd testegrep.txt |tail -1 |xargs tail -f| grep 's';
The way I'm sending this command is by using a file with a set of commands like:
plink.exe -ssh -t -l user -pw pwd tst.url.pt -m commands.out
When I run the command this way the plink does not receive any input. It seems that is waiting for input.
But if I run:
plink.exe -ssh -t -l user -pw pwd tst.url.pt "ls -1trd testegrep.txt |tail -1 |xargs tail -f| grep 's';"
I get the expected result.
I'm not using the plink with a file with the command because I choose so. I'm using a test automation software that allows me to run tests on remote hosts and this is the way the tool works.
Any thoughts on what is going wrong?
I tested the command you provided and it worked without problems.
Maybe the problem is related to:
The server's host key is not cached in the registry.
The path to the file is not correct.
The file is empty.
include server hostkey
most importantly, you need to include the unix profile using the -m paramater
You can include all your commands in the same file where the profile is kept also.
$Output = ((plink.exe -hostkey hostkey -l UNAME -i SSHKEY -P 22 -ssh server -batch -m PROFILE) | ? {$_ -ne ""})
So I log into a Solaris box, try to start Apache, and find that there is already a process listening on port 80, and it's not Apache. Our boxes don't have lsof installed, so I can't query with that. I guess I could do:
pfiles `ls /proc` | less
and look for "port: 80", but if anyone has a better solution, I'm all ears! Even better if I can look for the listening process without being root. I'm open to both shell and C solutions; I wouldn't mind having a little custom executable to carry with me for the next time this comes up.
Updated: I'm talking about generic installs of solaris for which I am not the administrator (although I do have superuser access), so installing things from the freeware disk isn't an option. Obviously neither are using Linux-specific extensions to fuser, netstat, or other tools. So far running pfiles on all processes seems to be the best solution, unfortunately. If that remains the case, I'll probably post an answer with some slightly more efficient code that the clip above.
I found this script somewhere. I don't remember where, but it works for me:
#!/bin/ksh
line='---------------------------------------------'
pids=$(/usr/bin/ps -ef | sed 1d | awk '{print $2}')
if [ $# -eq 0 ]; then
read ans?"Enter port you would like to know pid for: "
else
ans=$1
fi
for f in $pids
do
/usr/proc/bin/pfiles $f 2>/dev/null | /usr/xpg4/bin/grep -q "port: $ans"
if [ $? -eq 0 ]; then
echo $line
echo "Port: $ans is being used by PID:\c"
/usr/bin/ps -ef -o pid -o args | egrep -v "grep|pfiles" | grep $f
fi
done
exit 0
Edit: Here is the original source:
[Solaris] Which process is bound to a given port ?
Here's a one-liner:
ps -ef| awk '{print $2}'| xargs -I '{}' sh -c 'echo examining process {}; pfiles {}| grep 80'
'echo examining process PID' will be printed before each search, so once you see an output referencing port 80, you'll know which process is holding the handle.
Alternatively use:
ps -ef| grep $USER|awk '{print $2}'| xargs -I '{}' sh -c 'echo examining process {}; pfiles {}| grep 80'
Since 'pfiles' might not like that you're trying to access other user's processes, unless you're root of course.
Mavroprovato's answer reports more than only the listening ports. Listening ports are sockets without a peer. The following Perl program reports only the listening ports. It works for me on SunOS 5.10.
#! /usr/bin/env perl
##
## Search the processes which are listening on the given port.
##
## For SunOS 5.10.
##
use strict;
use warnings;
die "Port missing" unless $#ARGV >= 0;
my $port = int($ARGV[0]);
die "Invalid port" unless $port > 0;
my #pids;
map { push #pids, $_ if $_ > 0; } map { int($_) } `ls /proc`;
foreach my $pid (#pids) {
open (PF, "pfiles $pid 2>/dev/null |")
|| warn "Can not read pfiles $pid";
$_ = <PF>;
my $fd;
my $type;
my $sockname;
my $peername;
my $report = sub {
if (defined $fd) {
if (defined $sockname && ! defined $peername) {
print "$pid $type $sockname\n"; } } };
while (<PF>) {
if (/^\s*(\d+):.*$/) {
&$report();
$fd = int ($1);
undef $type;
undef $sockname;
undef $peername; }
elsif (/(SOCK_DGRAM|SOCK_STREAM)/) { $type = $1; }
elsif (/sockname: AF_INET[6]? (.*) port: $port/) {
$sockname = $1; }
elsif (/peername: AF_INET/) { $peername = 1; } }
&$report();
close (PF); }
#!/usr/bin/bash
# This is a little script based on the "pfiles" solution that prints the PID and PORT.
pfiles `ls /proc` 2>/dev/null | awk "/^[^ \\t]/{smatch=\$0;next}/port:[ \\t]*${1}/{print smatch, \$0}{next}"
From Solaris 11.2 onwards you can indeed do this with the netstat command. Have a look here. The -u switch is what you are looking for.
If you are on a lower version of Solaris then - as others have pointed out - the Solaris way of doing this is some kind of script wrapper around pfiles command. Beware though that pfiles command halts the process for a split second in order to inspect it. For 99.9% of processes this is unimportant. Unfortunately we have a process that will give a core dump if it is hit with a pfiles command so we are a bit cautious about using the command. Your situation may be totally different if you are in the 99.9%, meaning you can safely use the pfiles command.
netstat on Solaris will not tell you this, nor will older versions of lsof, but if you download and build/install a newer version of lsof, this can tell you that.
$ lsof -v
lsof version information:
revision: 4.85
latest revision: ftp://lsof.itap.purdue.edu/pub/tools/unix/lsof/
latest FAQ: ftp://lsof.itap.purdue.edu/pub/tools/unix/lsof/FAQ
latest man page: ftp://lsof.itap.purdue.edu/pub/tools/unix/lsof/lsof_man
configuration info: 64 bit kernel
constructed: Fri Mar 7 10:32:54 GMT 2014
constructed by and on: user#hostname
compiler: gcc
compiler version: 3.4.3 (csl-sol210-3_4-branch+sol_rpath)
8<- - - - ***SNIP*** - - -
With this you can use the -i option:
$ lsof -i:22
COMMAND PID USER FD TYPE DEVICE SIZE/OFF NODE NAME
sshd 521 root 3u IPv6 0xffffffff89c67580 0t0 TCP *:ssh (LISTEN)
sshd 5090 root 3u IPv6 0xffffffffa8668580 0t322598 TCP host.domain.com:ssh->21.43.65.87:52364 (ESTABLISHED)
sshd 5091 johngh 4u IPv6 0xffffffffa8668580 0t322598 TCP host.domain.com:ssh->21.43.65.87:52364 (ESTABLISHED)
Which shows you exactly what you're asking for.
I had a problem yesterday with a crashed Jetty (Java) process, which only left 2 files in its /proc/[PID] directory (psinfo & usage).
pfiles failed to find the process (because the date it needed was not there)
lsof found it for me.
You might not want to, but your best bet is to grab the sunfreeware CD and install lsof.
Other than that, yes you can grovel around in /proc with a shell script.
I think the first answer is the best
I wrote my own shell script developing this idea :
#!/bin/sh
if [ $# -ne 1 ]
then
echo "Sintaxis:\n\t"
echo " $0 {port to search in process }"
exit
else
MYPORT=$1
for i in `ls /proc`
do
pfiles $i | grep port | grep "port: $MYPORT" > /dev/null
if [ $? -eq 0 ]
then
echo " Port $MYPORT founded in $i proccess !!!\n\n"
echo "Details\n\t"
pfiles $i | grep port | grep "port: $MYPORT"
echo "\n\t"
echo "Process detail: \n\t"
ps -ef | grep $i | grep -v grep
fi
done
fi
Most probly sun's administrative server..
It's usually bundled along with sun's directory and a few other webmin-ish stuff that is in the default installation
This is sort of an indirect approach, but you could see if a website loads on your web browser of choice from whatever is running on port 80. Or you could telnet to port 80 and see if you get a response that gives you a clue as to what is running on that port and you can go shut it down. Since port 80 is the default port for http traffic chances are there is some sort of http server running there by default, but there's no guarantee.
If you have access to netstat, that can do precisely that.