rsyslog to resend event from client after abnormal server crash - tcp

Rsyslog Server IP: 192.168.122.94
Rsyslog Client IP: 192.168.122.93
1) Done rsyslog server force reboot
root#rsyslogserver:~# reboot -f
Write failed: Broken pipe
2) After reboot I have sent an event from rsyslog client.
3) Server is running on port 1014 and client is configured to forward logs to server on 1014
3) Ran tcpdump on rsyslog server to listen the communication on port 1014. For the first time when we send event after force reboot, rsyslog client is not able to forward event to rsyslog server. Then after, rsyslog client is able to forward logs to rsyslog server.
root#rsyslogserver:~# tcpdump -i eth1 "src port 1014"
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on eth1, link-type EN10MB (Ethernet), capture size 65535 bytes
11:03:05.687971 IP 192.168.122.94.1014 > 192.168.122.93.40036: Flags [R], seq 3944299399, win 0, length 0
11:05:28.096264 IP 192.168.122.94.1014 > 192.168.122.93.52079: Flags [S.], seq 3014852900, ack 1286331701, win 14480, options [mss 1460,sackOK,TS val 4294939924 ecr 149156552,nop,wscale 6], length 0
11:05:28.096605 IP 192.168.122.94.1014 > 192.168.122.93.52079: Flags [.], ack 394, win 243, options [nop,nop,TS val 4294939924 ecr 149156552], length 0
Reason:
This seems general behavior of any TCP connection. If any System crashes or terminates abnormally and after that if we send any TCP request then it resets old pre-cash connection and establishes new connection.
This will not happen for normal reboot.
RefLink:
https://en.wikipedia.org/wiki/TCP_reset_attack (Section TCP resets)
But here my question is how to prevent loss of that event for the first time.
Will there be any configurations in rsyslog server/client side to prevent event loss.

Related

SSH connection tunneling refused

I have a virtual machine located inside a private network.
So firstly, I should come in server.com and then come in my-machine.
I want to make ssh-tunnel from my laptop to my-machine.
ssh -v -A -nNT -L 40000:127.0.0.1:40000 login#server.com ssh -v -nNT -L 40000:127.0.0.1:40000 my-machine &
Now I want to test ssh-tunnel with netcat.
I run at my-machine:
nc -l 40000
At my laptop:
~ ❯❯❯ nc 127.0.0.1 40000
But it gives me:
debug1: Connection to port 40000 forwarding to 127.0.0.1 port 40000 requested.
debug1: channel 2: new [direct-tcpip]
channel 2: open failed: connect failed: Connection refused
debug1: channel 2: free: direct-tcpip: listening port 40000 for 127.0.0.1 port 40000, connect from 127.0.0.1 port 49692 to 127.0.0.1 port 40000, nchannels 3
Why this happen and how to fix it? I expected that anything I type in my laptop console will appear in my-machine console.
What last string means? Especially 127.0.0.1 port 49692 why this port is used? I never type it.
debug1: channel 2: free: direct-tcpip: listening port 40000 for 127.0.0.1 port 40000, connect from 127.0.0.1 port 49692 to 127.0.0.1 port 40000, nchannels 3
Every TCP connection is point to point needs two pairs of IP addresses and ports. Reading all the message (not just the part you showed):
connect from 127.0.0.1 port 49692
So indeed you are connecting to the port 40000, but you are connecting from port 49692 (randomly assigned for your netcat or some of the forwarded steps).
How to fix this problem?
This double-jump forwarding does not work, because you need the second established before the first one.
Also you are using -N switch for the first command, which is preventing running the second ssh command.
But I would give it a shot with ProxyCommand, which will make you connect to the destination with single command directly from your host:
ssh -v -nNT -L 40000:127.0.0.1:40000 \
-oProxyCommand="ssh -W %h:%p login#server.com" my-machine &

OpenVPN: connection problems / ssh_exchange_identification: read: Connection reset by peer [closed]

Closed. This question does not meet Stack Overflow guidelines. It is not currently accepting answers.
This question does not appear to be about a specific programming problem, a software algorithm, or software tools primarily used by programmers. If you believe the question would be on-topic on another Stack Exchange site, you can leave a comment to explain where the question may be able to be answered.
Closed 5 years ago.
Improve this question
I've got issues with OpenVPN and SSH (and other services):
I can connect to the VPN without any error messages.
If I then want to connect to a computer inside the VPN via SSH i get the error: "ssh_exchange_identification: read: Connection reset by peer".
On which machine this problem occurs differs from day to day and connection to connection.
$ ssh root#storage -vvv
OpenSSH_6.9p1, OpenSSL 1.0.1k-fips 8 Jan 2015
debug1: Reading configuration data /etc/ssh/ssh_config
debug1: /etc/ssh/ssh_config line 56: Applying options for *
debug2: ssh_connect: needpriv 0
debug1: Connecting to storage [192.168.1.5] port 22.
debug1: Connection established.
debug1: identity file /home/ewald/.ssh/id_rsa type 1
debug1: key_load_public: No such file or directory
debug1: identity file /home/ewald/.ssh/id_rsa-cert type -1
debug1: key_load_public: No such file or directory
debug1: identity file /home/ewald/.ssh/id_dsa type -1
debug1: key_load_public: No such file or directory
debug1: identity file /home/ewald/.ssh/id_dsa-cert type -1
debug1: key_load_public: No such file or directory
debug1: identity file /home/ewald/.ssh/id_ecdsa type -1
debug1: key_load_public: No such file or directory
debug1: identity file /home/ewald/.ssh/id_ecdsa-cert type -1
debug1: key_load_public: No such file or directory
debug1: identity file /home/ewald/.ssh/id_ed25519 type -1
debug1: key_load_public: No such file or directory
debug1: identity file /home/ewald/.ssh/id_ed25519-cert type -1
debug1: Enabling compatibility mode for protocol 2.0
debug1: Local version string SSH-2.0-OpenSSH_6.9
ssh_exchange_identification: read: Connection reset by peer
If I can connect to a different machine inside the VPN, I can SSH to my target.
Also I tried to configure a Port Forwarding for port 22 on the Remote Router and conected from my client very often via this forwarding and it always worked, so I don't think the problem is the SSH server or client configuration.
So I tried many settings in the OpenVPN configuration (different link-mtus, tcp instead of udp, disabled lzo, ...), installed it on a different machine (default OpenVPN server is in an Ubuntu 12.04 VM. I tried to install it on bare metal too) and nothing changed.
Since the problem appeared the first time even the ISP was changed (from DSL to Fiber).
The problem exists with other client machines too: A macbook using viscosity (tunnelblick doesn't work either), several linux boxes: fedora, ubuntu, ...
Sometimes I also don't receive an answer from the HTTP-server (even when I can connect to it using SSH)
The HTTP-server itself is accessible from the internal network and via my SSH-tunnel (using it as SOCKS proxy), so the problem must be the OpenVPN connection.
Can anyone help me? How can i check, if there are problems with the connection itself?
TCP-dump:
$ sudo tcpdump -i tun0 dst port 22
15:56:17.689847 IP myclient.60920 > storage.ssh: Flags [S], seq 729708226, win 29200, options [mss 1460,sackOK,TS val 3108117687 ecr 0,nop,wscale 7], length 0
15:56:17.717556 IP myclient.60920 > storage.ssh: Flags [.], ack 2044707578, win 229, options [nop,nop,TS val 3108117714 ecr 1518717699], length 0
15:56:17.717949 IP myclient.60920 > storage.ssh: Flags [P.], seq 0:21, ack 1, win 229, options [nop,nop,TS val 3108117715 ecr 1518717699], length 21
15:56:17.945811 IP myclient.60920 > storage.ssh: Flags [P.], seq 0:21, ack 1, win 229, options [nop,nop,TS val 3108117943 ecr 1518717699], length 21
Maybe the POSTROUTING rule is missing, can you paste the result of this command (on the openvpn server) ?
iptables -L -t nat
If there is no rules you can try :
iptables -t nat -A POSTROUTING -j MASQUERADE
Also verify the ipv4 forwarding (on the openvpn server):
cat /proc/sys/net/ipv4/ip_forward
If the result is 0, you must add this line in '/etc/sysctl.conf':
net.ipv4.ip_forward = 1
Tell me if it helps

"No route to host" when I do a telnet [closed]

Closed. This question does not meet Stack Overflow guidelines. It is not currently accepting answers.
This question does not appear to be about a specific programming problem, a software algorithm, or software tools primarily used by programmers. If you believe the question would be on-topic on another Stack Exchange site, you can leave a comment to explain where the question may be able to be answered.
Closed 6 years ago.
Improve this question
I have two VM's in Azure with different public IP's:
10.10.1.9
10.10.1.6
When I do a telnet with following command from the server 10.10.1.6, I get the a error:
telnet 10.10.1.9 2181
Trying 10.10.1.9...
telnet: connect to address 10.10.1.9: No route to host
When I do a tcpdump in 10.10.1.9 side, I get the following log:
#tcpdump -i eth0 port 2181
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on eth0, link-type EN10MB (Ethernet), capture size 65535 bytes
07:55:35.530270 IP 10.10.1.6.55910 > 10.10.1.9.eforward: Flags [S], seq 1018543857, win 14600, options [mss 1418,sackOK,TS val 181360935 ecr 0,nop,wscale 7], length 0
Same time I also do a tcpdump on 10.10.1.6 side while i do a telnet from 10.10.1.6 to 10.10.1.9
tcpdump -i eth0 port 2181
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on eth0, link-type EN10MB (Ethernet), capture size 65535 bytes
07:55:57.970696 IP 10.10.1.6.55910 > 10.10.1.9.eforward: Flags [S], seq 1018543857, win 14600, options [mss 1460,sackOK,TS val 181360935 ecr 0,nop,wscale 7], length 0
**tcpdump on 10.10.1.9 with arp **
#tcpdump -i eth0 port 2181 or arp
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on eth0, link-type EN10MB (Ethernet), capture size 65535 bytes
08:00:18.356153 IP 10.10.1.6.55944 > 10.10.1.9.eforward: Flags [S], seq 3337054296, win 14600, options [mss 1418,sackOK,TS val 181643770 ecr 0,nop,wscale 7], length 0
08:00:42.294801 ARP, Request who-has 10.10.1.6 tell 10.10.1.9, length 28
08:00:42.295859 ARP, Reply 10.10.1.6 is-at 12:34:56:78:9a:bc (oui Unknown), length 28
tcpdump on 10.10.1.6
tcpdump -i eth0 port 2181 or arp
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on eth0, link-type EN10MB (Ethernet), capture size 65535 bytes
08:00:40.805565 IP 10.10.1.6.55944 > 10.10.1.9.eforward: Flags [S], seq 3337054296, win 14600, options [mss 1460,sackOK,TS val 181643770 ecr 0,nop,wscale 7], length 0
08:00:45.805204 ARP, Request who-has 10.10.1.9 tell 10.10.1.6, length 28
08:00:45.805721 ARP, Reply 10.10.1.9 is-at 12:34:56:78:9a:bc (oui Unknown), length 28
08:02:04.752283 ARP, Request who-has 10.10.1.9 tell 10.10.1.6, length 28
08:02:04.753141 ARP, Reply 10.10.1.9 is-at 12:34:56:78:9a:bc (oui Unknown), length 28
Sequence of run :
First I ran tcpdumps on both 10.10.1.9 and 10.10.1.10 and then tried doing telnet from 10.10.1.10.
arp -a on 10.10.1.9
#arp -a
? (10.10.1.7) at 12:34:56:78:9a:bc [ether] on eth0
? (10.10.1.4) at 12:34:56:78:9a:bc [ether] on eth0
? (10.10.1.1) at 12:34:56:78:9a:bc [ether] on eth0
? (10.10.1.8) at 12:34:56:78:9a:bc [ether] on eth0
? (10.10.1.10) at <incomplete> on eth0
? (10.10.1.11) at 12:34:56:78:9a:bc [ether] on eth0
? (10.10.1.6) at 12:34:56:78:9a:bc [ether] on eth0
? (10.10.1.5) at 12:34:56:78:9a:bc [ether] on eth0
arp -a on 10.10.1.6
#arp -a
? (10.10.1.1) at 12:34:56:78:9a:bc [ether] on eth0
? (10.10.1.10) at <incomplete> on eth0
? (10.10.1.9) at 12:34:56:78:9a:bc [ether] on eth0
Thanks in advance.
The tcpdump in 10.10.1.9 says that it received a packet from 10.10.1.10. but could not reply back...as a result we get "No route to host" on 10.10.1.10 side.
You should get "No route to host" if there is, in fact, no route from 10.10.1.10 to 10.10.1.9, not just because a packet sent from 10.10.1.10 to 10.10.1.9 didn't get a reply. I.e., you should only get "No route to host" if 10.10.1.10 couldn't send a packet to 10.10.1.9 in the first place!
Now, perhaps the OS running on 10.10.1.10 is being stupid and returning EHOSTUNREACH ("No route to host") rather than, for example, ETIMEDOUT ("Operation timed out") if it never gets a SYN+ACK back from the initial SYN.
Or perhaps there was a route from 10.10.1.10 to 10.10.1.9 during the time the
23:46:30.003480 IP 10.10.1.10.42946 > 10.10.1.9.eforward: Flags [S], seq 2823099523, win 14600, options [mss 1418,sackOK,TS val 74982205 ecr 0,nop,wscale 7], length 0
packet was sent, but 10.10.1.9 wasn't able to, or decided not to, respond to that initial SYN with a SYN+ACK, and when 10.10.1.10 retransmitted the SYN, it was no longer able to send packets to 10.10.1.9, and reported "No route to host".
If this is reproducible, I would suggest running tcpdump on both hosts, to see more details as to what happened. I would suggest running a command such as
tcpdump -i eth0 port 2181 or arp
so that, for example, if the problem is that the ARP entry for the other host timed out on one of the hosts, and a subsequent attempt to re-ARP for the other host's MAC address failed, that will show up. (I'm assuming here that there's no router in between 10.10.1.10 and 10.10.1.9, so that "No route to host" really means "No ARP entry for host".)
(Another possibility is that there's some sort of "packet filter"/firewall in place on one or the other host, handling some ports differently from others, so that connecting to port 22 is possible but connecting to port 2181 isn't possible.)

Meaning of *:* in lsof output

On Solaris (or Unix), running lsof gives me a bunch of lines like this:
java 25375 foo 8161u IPv4 0xfffffeb1f6f523c0 0t0 TCP *:* (IDLE)
But the *:* puzzles me - I was expecting to see something like
hostname1:port1->ipaddress:port2 (IDLE)
What does the *:* mean?
I will quote this from here:
If the Foreign Address is *:* (and, with TCP sockets, the state is
LISTEN), a socket is usually waiting for some remote host to send the
first data. Typical examples: sshd (waits for somebody to open an ssh
connection), apache (waits for somebody to request a web page), cupsd
(waits for somebody to send a print job), and dhclient (waits for the
DHCP server to send, for example, a lease renewal).

Netperf reporting zero throughput

I've installed netperf 2.6 in two sites and trying to run the netperf benchmark, but All I'm getting is zero Throughput... Does anyone knows how to use netperf properly? (I was following the official documentation)
I run this at a server:
./netserver -p xxxxx
the output is:
Starting netserver with host 'IN(6)ADDR_ANY' port 'xxxxx' and family AF_UNSPEC
In the other side I run:
./netperf -s 5 -H a.b.c.d -p xxxxx
The output is:
MIGRATED TCP STREAM TEST from 0.0.0.0 (0.0.0.0) port 0 AF_INET to a.b.c.d () port 0 AF_INET : demo
Recv Send Send
Socket Socket Message Elapsed
Size Size Size Time Throughput
bytes bytes bytes secs. 10^6bits/sec
87380 16384 16384 10.00 0.00
any ideas?
A netperf test has two "connections." The first is the "control connection" over which information about the test setup and result is exchanged. For the benchmarking itself a "data connection" is used. The control connection will use the control port you've specified with the global "-p" option. The data connection will by default use a port number chosen by the networking stack where the netserver runs.
Both have to be open through firewalls for a test to be successful.
If only the control port is open, you will see the test banners get displayed because the control connection is established. Since the data connection cannot be established, that will report zero.
You can specify an explicit port number for the data connection with a test-specific "-P" option. So, if you opened a second port number, 9992, you would start the netserver as before, and then your netperf command would become:
./netperf -s 5 -H a.b.c.d -p xxxxx -- -P ,9992
That comma is important. The test-specific -P option allows specifying both the local and remote port numbers for the data connection. The remote port number follows a comma.
terminal1:
$ sudo netserver -D -4 -L 0.0.0.0 -p 9991
Starting netserver with host '0.0.0.0' port '9991' and family AF_INET
terminal2:
$ sudo netperf -H 192.168.2.103 -l 60 -t TCP_STREAM
TCP STREAM TEST from 0.0.0.0 (0.0.0.0) port 0 AF_INET to 192.168.2.103 (192.168.2.103) port 0 AF_INET
Recv Send Send
Socket Socket Message Elapsed
Size Size Size Time Throughput
bytes bytes bytes secs. 10^6bits/sec
87380 524288 524288 60.02 89.66

Resources