I am trying to connect to a postgreSQL database (hosted on AWS RDS) via an SSH tunnel in R. So far, I have been able to connect using the following methods:
1.---------------------------
Opening the ssh tunnel in my terminal (MacOs) using
ssh -i {key file path} -f -N -L 5432:{db host}:5432 {ssh user}#{ssh host} -v
and then connecting to the database using
psql -hlocalhost -U{db user} -p5432 -dpostgres
2.---------------------------
Opening the ssh tunnel in my terminal and then running the following code in R to connect
conn <- dbConnect(
RPostgres::Postgres(),
dbname = db_name,
user = db_user,
password = db_password,
host = "127.0.0.1",
port = db_port
)
3.---------------------------
This is where the issue occurs. I'm able to connect by opening the ssh tunnel in R (in a background environment) with
tunnel_process <- callr::r_bg(
function(ssh_host, ssh_user, ssh_key, db_host, db_port) {
session <- ssh::ssh_connect(host = glue::glue("{ ssh_user }#{ ssh_host }"),
keyfile = ssh_key,
verbose = 3)
ssh::ssh_tunnel(session = session,
port = db_port,
target = glue::glue("{ db_host }:{ db_port }"))
},
args = list(ssh_host, ssh_user, ssh_key, db_host, db_port),
stdout = nullfile(),
stderr = nullfile()
)
But then I'm unable to use the same "dbConnect" code as above to connect. It only gives me the following error message
Error: could not connect to server: Connection refused
Is the server running on host "127.0.0.1" and accepting
TCP/IP connections on port 5432?
I am however able to connect directly from the terminal using the connection code in part 1. but only if I run psql -hlocalhost -U{db user} -p5432 -dpostgres, then re-run the ssh tunnel code in R, and only THEN enter my password in the terminal. I would appear that every time I try to connect, it closes the SSH tunnel, so I then have to re-launch it before submitting my password.
Question---------------------------
From what I just detailed, it would appear that:
a. My database is reachable since I can easily connect through the terminal
b. My R code works since I'm able to use it to both successfully open the SSH tunnel AND connect to the database. I'm just unable to use both together for some reason.
c. The tunnel I open through R breaks any time I try to connect to the database. This is not the case for the tunnel I open directly in the terminal.
Since I want to be able to do everything directly from R, does anybody here have any ideas on what may be causing the issue?
EDIT---------------------------
Here's the log I get in R when I try to connect to the database, just before it closes the tunnel:
> ssh::ssh_tunnel(session = session,
+ port = db_port,
+ target = glue::glue("{ db_host }:{ db_port }"))
\ Waiting for connetion on port 5432... client connected!
channel_open: Creating a channel 43 with 64000 window and 32768 max packet
ssh_socket_unbuffered_write: Enabling POLLOUT for socket
packet_send2: packet: wrote [len=124,padding=18,comp=105,payload=105]
channel_open: Sent a SSH_MSG_CHANNEL_OPEN type direct-tcpip for channel 43
ssh_packet_socket_callback: packet: read type 80 [len=492,padding=16,comp=475,payload=475]
ssh_packet_process: Dispatching handler for packet type 80
ssh_packet_global_request: Received SSH_MSG_GLOBAL_REQUEST packet
ssh_packet_global_request: UNKNOWN SSH_MSG_GLOBAL_REQUEST hostkeys-00#openssh.com 0
ssh_packet_process: Couldn't do anything with packet type 80
packet_send2: packet: wrote [len=12,padding=6,comp=5,payload=5]
ssh_socket_unbuffered_write: Enabling POLLOUT for socket
ssh_packet_socket_callback: packet: read type 91 [len=28,padding=10,comp=17,payload=17]
ssh_packet_process: Dispatching handler for packet type 91
ssh_packet_channel_open_conf: Received SSH2_MSG_CHANNEL_OPEN_CONFIRMATION
ssh_packet_channel_open_conf: Received a CHANNEL_OPEN_CONFIRMATION for channel 43:0
ssh_packet_channel_open_conf: Remote window : 2097152, maxpacket : 32768
| Tunneled -1 bytes...ssh_socket_unbuffered_write: Enabling POLLOUT for socket
packet_send2: packet: wrote [len=28,padding=10,comp=17,payload=17]
channel_write_common: channel_write wrote 8 bytes
| Tunneled 7 bytes...ssh_packet_socket_callback: packet: read type 94 [len=28,padding=17,comp=10,payload=10]
ssh_packet_process: Dispatching handler for packet type 94
channel_rcv_data: Channel receiving 1 bytes data in 0 (local win=64000 remote win=2097144)
channel_default_bufferize: placing 1 bytes into channel buffer (stderr=0)
channel_rcv_data: Channel windows are now (local win=63999 remote win=2097144)
ssh_socket_unbuffered_write: Enabling POLLOUT for socket
packet_send2: packet: wrote [len=28,padding=18,comp=9,payload=9]
grow_window: growing window (channel 43:0) to 1280000 bytes
ssh_channel_read_timeout: Read (1) buffered : 1 bytes. Window: 1280000
- Tunneled 8 bytes...ssh_socket_unbuffered_write: Enabling POLLOUT for socket
packet_send2: packet: wrote [len=316,padding=17,comp=298,payload=298]
channel_write_common: channel_write wrote 289 bytes
/ Tunneled 297 bytes...ssh_packet_socket_callback: packet: read type 94 [len=3964,padding=12,comp=3951,payload=3951]
ssh_packet_process: Dispatching handler for packet type 94
channel_rcv_data: Channel receiving 3942 bytes data in 0 (local win=1280000 remote win=2096855)
channel_default_bufferize: placing 3942 bytes into channel buffer (stderr=0)
channel_rcv_data: Channel windows are now (local win=1276058 remote win=2096855)
ssh_channel_read_timeout: Read (3942) buffered : 3942 bytes. Window: 1276058
\ Tunneled 4239 bytes...ssh_socket_unbuffered_write: Enabling POLLOUT for socket
packet_send2: packet: wrote [len=156,padding=8,comp=147,payload=147]
channel_write_common: channel_write wrote 138 bytes
- Tunneled 4377 bytes...ssh_packet_socket_callback: packet: read type 94 [len=76,padding=15,comp=60,payload=60]
ssh_packet_process: Dispatching handler for packet type 94
channel_rcv_data: Channel receiving 51 bytes data in 0 (local win=1276058 remote win=2096717)
channel_default_bufferize: placing 51 bytes into channel buffer (stderr=0)
channel_rcv_data: Channel windows are now (local win=1276007 remote win=2096717)
ssh_channel_read_timeout: Read (51) buffered : 51 bytes. Window: 1276007
| Tunneled 4428 bytes...ssh_socket_unbuffered_write: Enabling POLLOUT for socket
packet_send2: packet: wrote [len=140,padding=14,comp=125,payload=125]
channel_write_common: channel_write wrote 116 bytes
\ Tunneled 4544 bytes...ssh_packet_socket_callback: packet: read type 94 [len=60,padding=8,comp=51,payload=51]
ssh_packet_process: Dispatching handler for packet type 94
channel_rcv_data: Channel receiving 42 bytes data in 0 (local win=1276007 remote win=2096601)
channel_default_bufferize: placing 42 bytes into channel buffer (stderr=0)
channel_rcv_data: Channel windows are now (local win=1275965 remote win=2096601)
ssh_channel_read_timeout: Read (42) buffered : 42 bytes. Window: 1275965
/ Tunneled 4586 bytes...ssh_socket_unbuffered_write: Enabling POLLOUT for socket
packet_send2: packet: wrote [len=60,padding=19,comp=40,payload=40]
channel_write_common: channel_write wrote 31 bytes
- Tunneled 4617 bytes...packet_send2: packet: wrote [len=12,padding=6,comp=5,payload=5]
ssh_channel_send_eof: Sent a EOF on client channel (43:0)
ssh_socket_unbuffered_write: Enabling POLLOUT for socket
packet_send2: packet: wrote [len=12,padding=6,comp=5,payload=5]
ssh_channel_close: Sent a close on client channel (43:0)
ssh_socket_unbuffered_write: Enabling POLLOUT for socket
tunnel closed!
For reference, this is what the same log looks like when using the workaround detailed in 3. (re-running the ssh_tunnel right before submitting my password in the terminal):
> ssh::ssh_tunnel(session = session,
+ port = db_port,
+ target = glue::glue("{ db_host }:{ db_port }"))
\ Waiting for connetion on port 5432... client connected!
channel_open: Creating a channel 43 with 64000 window and 32768 max packet
ssh_socket_unbuffered_write: Enabling POLLOUT for socket
packet_send2: packet: wrote [len=124,padding=18,comp=105,payload=105]
channel_open: Sent a SSH_MSG_CHANNEL_OPEN type direct-tcpip for channel 43
ssh_packet_socket_callback: packet: read type 80 [len=492,padding=16,comp=475,payload=475]
ssh_packet_process: Dispatching handler for packet type 80
ssh_packet_global_request: Received SSH_MSG_GLOBAL_REQUEST packet
ssh_packet_global_request: UNKNOWN SSH_MSG_GLOBAL_REQUEST hostkeys-00#openssh.com 0
ssh_packet_process: Couldn't do anything with packet type 80
packet_send2: packet: wrote [len=12,padding=6,comp=5,payload=5]
ssh_socket_unbuffered_write: Enabling POLLOUT for socket
ssh_packet_socket_callback: packet: read type 91 [len=28,padding=10,comp=17,payload=17]
ssh_packet_process: Dispatching handler for packet type 91
ssh_packet_channel_open_conf: Received SSH2_MSG_CHANNEL_OPEN_CONFIRMATION
ssh_packet_channel_open_conf: Received a CHANNEL_OPEN_CONFIRMATION for channel 43:0
ssh_packet_channel_open_conf: Remote window : 2097152, maxpacket : 32768
| Tunneled -1 bytes...ssh_socket_unbuffered_write: Enabling POLLOUT for socket
packet_send2: packet: wrote [len=28,padding=10,comp=17,payload=17]
channel_write_common: channel_write wrote 8 bytes
| Tunneled 7 bytes...ssh_packet_socket_callback: packet: read type 94 [len=28,padding=17,comp=10,payload=10]
ssh_packet_process: Dispatching handler for packet type 94
channel_rcv_data: Channel receiving 1 bytes data in 0 (local win=64000 remote win=2097144)
channel_default_bufferize: placing 1 bytes into channel buffer (stderr=0)
channel_rcv_data: Channel windows are now (local win=63999 remote win=2097144)
ssh_socket_unbuffered_write: Enabling POLLOUT for socket
packet_send2: packet: wrote [len=28,padding=18,comp=9,payload=9]
grow_window: growing window (channel 43:0) to 1280000 bytes
ssh_channel_read_timeout: Read (1) buffered : 1 bytes. Window: 1280000
- Tunneled 8 bytes...ssh_socket_unbuffered_write: Enabling POLLOUT for socket
packet_send2: packet: wrote [len=316,padding=17,comp=298,payload=298]
channel_write_common: channel_write wrote 289 bytes
\ Tunneled 297 bytes...ssh_packet_socket_callback: packet: read type 94 [len=3964,padding=12,comp=3951,payload=3951]
ssh_packet_process: Dispatching handler for packet type 94
channel_rcv_data: Channel receiving 3942 bytes data in 0 (local win=1280000 remote win=2096855)
channel_default_bufferize: placing 3942 bytes into channel buffer (stderr=0)
channel_rcv_data: Channel windows are now (local win=1276058 remote win=2096855)
ssh_channel_read_timeout: Read (3942) buffered : 3942 bytes. Window: 1276058
/ Tunneled 4239 bytes...ssh_socket_unbuffered_write: Enabling POLLOUT for socket
packet_send2: packet: wrote [len=156,padding=8,comp=147,payload=147]
channel_write_common: channel_write wrote 138 bytes
| Tunneled 4377 bytes...ssh_packet_socket_callback: packet: read type 94 [len=76,padding=15,comp=60,payload=60]
ssh_packet_process: Dispatching handler for packet type 94
channel_rcv_data: Channel receiving 51 bytes data in 0 (local win=1276058 remote win=2096717)
channel_default_bufferize: placing 51 bytes into channel buffer (stderr=0)
channel_rcv_data: Channel windows are now (local win=1276007 remote win=2096717)
ssh_channel_read_timeout: Read (51) buffered : 51 bytes. Window: 1276007
- Tunneled 4428 bytes...ssh_socket_unbuffered_write: Enabling POLLOUT for socket
packet_send2: packet: wrote [len=140,padding=14,comp=125,payload=125]
channel_write_common: channel_write wrote 116 bytes
/ Tunneled 4544 bytes...ssh_packet_socket_callback: packet: read type 94 [len=60,padding=8,comp=51,payload=51]
ssh_packet_process: Dispatching handler for packet type 94
channel_rcv_data: Channel receiving 42 bytes data in 0 (local win=1276007 remote win=2096601)
channel_default_bufferize: placing 42 bytes into channel buffer (stderr=0)
channel_rcv_data: Channel windows are now (local win=1275965 remote win=2096601)
ssh_channel_read_timeout: Read (42) buffered : 42 bytes. Window: 1275965
\ Tunneled 4586 bytes...ssh_socket_unbuffered_write: Enabling POLLOUT for socket
packet_send2: packet: wrote [len=92,padding=12,comp=79,payload=79]
channel_write_common: channel_write wrote 70 bytes
- Tunneled 4656 bytes...ssh_packet_socket_callback: packet: read type 94 [len=380,padding=15,comp=364,payload=364]
ssh_packet_process: Dispatching handler for packet type 94
channel_rcv_data: Channel receiving 355 bytes data in 0 (local win=1275965 remote win=2096531)
channel_default_bufferize: placing 355 bytes into channel buffer (stderr=0)
channel_rcv_data: Channel windows are now (local win=1275610 remote win=2096531)
ssh_channel_read_timeout: Read (355) buffered : 355 bytes. Window: 1275610
| Tunneled 5011 bytes...
Finally, here's the log when running ssh_connect:
> session <- ssh::ssh_connect(host = glue::glue("{ ssh_user }#{ ssh_host }"),
+ keyfile = ssh_key,
+ verbose = 3)
ssh_pki_import_privkey_base64: Trying to decode privkey passphrase=false
ssh_connect: libssh 0.8.6 (c) 2003-2018 Aris Adamantiadis, Andreas Schneider and libssh contributors. Distributed under the LGPL, please refer to COPYING file for information about your rights, using threading threads_pthread
ssh_socket_connect: Nonblocking connection socket: 50
ssh_connect: Socket connecting, now waiting for the callbacks to work
ssh_connect: Actual timeout : 10000
ssh_socket_pollcallback: Received POLLOUT in connecting state
socket_callback_connected: Socket connection callback: 1 (0)
ssh_socket_unbuffered_write: Enabling POLLOUT for socket
callback_receive_banner: Received banner: SSH-2.0-OpenSSH_7.4
ssh_client_connection_callback: SSH server banner: SSH-2.0-OpenSSH_7.4
ssh_analyze_banner: Analyzing banner: SSH-2.0-OpenSSH_7.4
ssh_analyze_banner: We are talking to an OpenSSH client version: 7.4 (70400)
ssh_known_hosts_read_entries: Failed to open the known_hosts file '/etc/ssh/ssh_known_hosts': No such file or directory
ssh_client_select_hostkeys: Changing host key method to "ecdsa-sha2-nistp256,ssh-ed25519,ecdsa-sha2-nistp521,ecdsa-sha2-nistp384,ssh-rsa,ssh-dss"
ssh_socket_unbuffered_write: Enabling POLLOUT for socket
packet_send2: packet: wrote [len=644,padding=9,comp=634,payload=634]
ssh_packet_socket_callback: packet: read type 20 [len=1276,padding=10,comp=1265,payload=1265]
ssh_packet_process: Dispatching handler for packet type 20
ssh_kex_select_methods: Negotiated curve25519-sha256,ecdsa-sha2-nistp256,aes256-ctr,aes256-ctr,hmac-sha2-256,hmac-sha2-256,none,none,,
ssh_socket_unbuffered_write: Enabling POLLOUT for socket
packet_send2: packet: wrote [len=44,padding=6,comp=37,payload=37]
ssh_packet_socket_callback: packet: read type 31 [len=260,padding=11,comp=248,payload=248]
ssh_packet_process: Dispatching handler for packet type 31
ssh_packet_dh_reply: Received SSH_KEXDH_REPLY
ssh_socket_unbuffered_write: Enabling POLLOUT for socket
packet_send2: packet: wrote [len=12,padding=10,comp=1,payload=1]
ssh_client_curve25519_reply: SSH_MSG_NEWKEYS sent
ssh_packet_socket_callback: Processing 112 bytes left in socket buffer
ssh_packet_socket_callback: packet: read type 21 [len=12,padding=10,comp=1,payload=1]
ssh_packet_process: Dispatching handler for packet type 21
ssh_packet_newkeys: Received SSH_MSG_NEWKEYS
crypt_set_algorithms2: Set output algorithm to aes256-ctr
crypt_set_algorithms2: Set HMAC output algorithm to hmac-sha2-256
crypt_set_algorithms2: Set input algorithm to aes256-ctr
crypt_set_algorithms2: Set HMAC input algorithm to hmac-sha2-256
ssh_packet_newkeys: Signature verified and valid
ssh_packet_socket_callback: Processing 96 bytes left in socket buffer
ssh_packet_socket_callback: packet: read type 7 [len=60,padding=6,comp=53,payload=53]
ssh_packet_process: Dispatching handler for packet type 7
ssh_packet_ext_info: Received SSH_MSG_EXT_INFO
ssh_packet_ext_info: Follows 1 extensions
ssh_packet_ext_info: Extension: server-sig-algs=<rsa-sha2-256,rsa-sha2-512>
ssh_connect: current state : 7
packet_send2: packet: wrote [len=28,padding=10,comp=17,payload=17]
ssh_service_request: Sent SSH_MSG_SERVICE_REQUEST (service ssh-userauth)
ssh_socket_unbuffered_write: Enabling POLLOUT for socket
ssh_packet_socket_callback: packet: read type 6 [len=28,padding=10,comp=17,payload=17]
ssh_packet_process: Dispatching handler for packet type 6
ssh_packet_service_accept: Received SSH_MSG_SERVICE_ACCEPT
ssh_socket_unbuffered_write: Enabling POLLOUT for socket
packet_send2: packet: wrote [len=44,padding=4,comp=39,payload=39]
ssh_packet_socket_callback: packet: read type 51 [len=60,padding=15,comp=44,payload=44]
ssh_packet_process: Dispatching handler for packet type 51
ssh_packet_userauth_failure: Access denied for 'none'. Authentication that can continue: publickey,gssapi-keyex,gssapi-with-mic
ssh_packet_userauth_failure: Access denied for 'none'. Authentication that can continue: publickey,gssapi-keyex,gssapi-with-mic
ssh_key_algorithm_allowed: Checking rsa-sha2-512 with list <ssh-ed25519,ecdsa-sha2-nistp256,ecdsa-sha2-nistp384,ecdsa-sha2-nistp521,ssh-rsa,rsa-sha2-512,rsa-sha2-256,ssh-dss>
ssh_key_algorithm_allowed: Checking rsa-sha2-512 with list <ssh-ed25519,ecdsa-sha2-nistp256,ecdsa-sha2-nistp384,ecdsa-sha2-nistp521,ssh-rsa,rsa-sha2-512,rsa-sha2-256,ssh-dss>
ssh_key_algorithm_allowed: Checking rsa-sha2-512 with list <ssh-ed25519,ecdsa-sha2-nistp256,ecdsa-sha2-nistp384,ecdsa-sha2-nistp521,ssh-rsa,rsa-sha2-512,rsa-sha2-256,ssh-dss>
ssh_socket_unbuffered_write: Enabling POLLOUT for socket
packet_send2: packet: wrote [len=636,padding=11,comp=624,payload=624]
ssh_packet_socket_callback: packet: read type 52 [len=12,padding=10,comp=1,payload=1]
ssh_packet_process: Dispatching handler for packet type 52
ssh_packet_userauth_success: Authentication successful
I have this Python code that is supposed to use a certificate and connect to a server via the server's IP address (no FQDN available) but I get an error every time I try it:
ssl_transport_security.cc:222] LOOP - TLS client process_change_ciph - !!!!!!
ssl_transport_security.cc:222] LOOP - TLS client read_server_finishe - !!!!!!
ssl_transport_security.cc:222] LOOP - TLS client finish_client_hands - !!!!!!
ssl_transport_security.cc:222] LOOP - TLS client done - !!!!!!
ssl_transport_security.cc:222] HANDSHAKE DONE - TLS client done - !!!!!!
security_handshaker.cc:184] Security handshake failed: {"created":"#1582052112.923538253","description":"Peer name 172.18.0.14 is not in peer certificate","file":"src/core/lib/security/security_connector/ssl/ssl_security_connector.cc","file_line":55}
subchannel.cc:1003] Connect failed: {"created":"#1582052112.923538253","description":"Peer name 172.18.0.14 is not in peer certificate","file":"src/core/lib/security/security_connector/ssl/ssl_security_connector.cc","file_line":55}
subchannel.cc:940] Subchannel 0x55ad70542020: Retry immediately
subchannel.cc:967] Failed to connect to channel, retrying
From what I understand this could be due to me connecting to an ip address and not a FQDN but these are servers and I only have access to IP addresses. Any idea how to overcome this?
Python code used:
def get_secure_channel(host, port):
if os.environ.get('https_proxy'):
del os.environ['https_proxy']
if os.environ.get('http_proxy'):
del os.environ['http_proxy']
with open(os.path.join(settings.DJANGO_ROOT, '../grpc_proto/cert/server.crt'), 'rb') as f:
cert = f.read()
credentials = grpc.ssl_channel_credentials(root_certificates=cert)
return grpc.secure_channel('{}:{}'.format(host, port), credentials)
def reset_client(channel, ip_address):
stub = dnsadblock_pb2_grpc.DnsadblockServiceStub(channel)
return stub.ResetClient(dnsadblock_pb2.ResetClientRequest(ipAddress=ip_address))
channel = get_secure_channel(c.server.hostname, settings.GRPC_PORT)
rpc.reset_client(channel, c.ip_address)
https://support.dnsimple.com/articles/what-is-common-name/
In this case, the endpoint you are connecting are not matching the description of peer's certificate. Hence, the connection was rejected. You could either try to use one of the CN in your certificate to connect to the server. Or adding the endpoint to your certificate.
I was integrating a go grpc server and a Python client. To make it work I did the following.
When creating the server.csr file using this line
openssl req -new -sha256 -key certs/server.key -out certs/server.csr -config certs/certificate.conf
The certs/certificate.conf looked likes this:
[req]
default_bits = 4096
prompt = no
default_md = sha256
req_extensions = req_ext
distinguished_name = dn
[dn]
C = AU
ST = VIC
O = OCOM Software
CN = hub_proxy
[req_ext]
subjectAltName = #alt_names
[alt_names]
DNS.1 = hub_proxy
IP.1 = ::1
IP.2 = 127.0.0.1
Notice the CN = hub_proxy and the DNS.1 = hub_proxy
I am not sure which solved this but when I regenerated the certificates and keys with these settings.
And then connected to the server using the hostname (in the client) of hub_proxy:9000 (where 9000 is the port number) it worked.
I have a problem with a very basic usage of Scapy on Windows 7 (Python 3.6, Scapy 2.4.0). I'm also running Npcap 0.99r7 and Wireshark 2.6.2 on this sytem. The system does only have one wireless network interface plus the Npcap loopback interface.
I set up this very classic TCP server... :
import socket
host = '127.0.0.1'
port = 8089
s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
s.bind((host, port))
s.listen(1)
connection, address = s.accept()
while 1:
try :
data = connection.recv(1024)
except ConnectionAbortedError:
break
if data:
print('Received: %s' % (data.decode ('utf-8')))
connection.sendall('Data received'.encode())
connection.close()
s.close()
...and I set up this very classic TCP client:
import socket
host = '127.0.0.1'
port = 8089
s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
s.connect((host, port))
s.send('Hello, world!'.encode())
data = s.recv(1024)
print('Received: %s' % (data.decode('utf-8')))
s.close()
Both works fine. Wireshark does report the whole TCP traffic on the loopback interface.
Now, I'm running the server, and I try to run that piece of code that would just send a SYN to the server with Scapy :
from scapy.layers.inet import IP
from scapy.layers.inet import TCP
from scapy.sendrecv import *
dstHost='127.0.0.1'
dstPort = 8089
packet = IP(src='127.0.0.1', dst=dstHost)/TCP(dport=dstPort, flags='S')
response=sr1(packet, timeout=10)
response.display()
Python reports :
Begin emission:
..Finished sending 1 packets.
......Traceback (most recent call last):
File "R:/Documents/Projets/python/hacking/scan.py", line 46, in <module>
response.display()
AttributeError: 'NoneType' object has no attribute 'display'
Received 8 packets, got 0 answers, remaining 1 packets
Moreover, Wireshark does not see anything on the loopback interface. May somebody give an hint ?
Update 1
As suggested, I tried a more explicit code using sendp() and not send(), since we are talking layer 2 here:
route_add_loopback()
packet = Loopback()/IP(src='127.0.0.1', dst='127.0.0.1')/TCP(dport=8089, flags='S')
sendp(packet,iface='Npcap Loopback Adapter')
Unfortunately, Wireshark does not sniff the packet on either interfaces (the 'Intel(R) Centrino(R) Advanced-N 6235' and the 'Npcap Loopback Adapter').
Note that the call to route_add_loopback() is required, or show_interfaces() won't report the 'Npcap Loopback Adapter', which means that sendp() will fail. It is possible to restore the Scapy routing table by calling conf.route.resync () after route_add_loopback(), but the result is the same : Wireshark does not sniff the packet on either interface.
Should somebody find some Python piece of code running on Windows 7 that succesfully sends a simple TCP packet on the 'Npcap Loopback Adapter', he would be welcome...
The loopback interface is not a "regular" interface; this is particularly true for Windows.
You can check the route used by Scapy to send the packet by running: packet.route().
If the route displayed does not use the loopback interface, you can try to run (that's windows specific) route_add_loopback() and try again.
Another option would be to use srp1() instead of sr1(), and specify the loopback interface as iface= parameter.
I am setting up simple tcp connection routing using HAProxy acl's. The idea is to route connections depending on request content having two flavors: read and write requests.
For testing I made a simple tcp client/server setup using perl. Strangely enough about 10-40% of the ACL's fail and are sent to the default backend.
The ACL's should find the substring 'read' or 'write' and route accordingly, but this is not allways the case.
Sending a read/write request using nc (netcat) has the same effect.
I tested this configuration with mode=http and everything works as expected.
I also tested with reg, sub and bin, to no avail.
The example server setup is as follows:
HAProxy instance, listens on port 8000
Client (creates tcp connection to proxy and sends user input (read/write string) to server through port 8000, after which it closes the connection)
Server1 (write server), listens on port 8001
Server2 (read server), listens on port 8002
Server3 (default server), listens on port 8003
My HAProxy configuration file looks is:
global
log /dev/log local0 debug
#daemon
maxconn 32
defaults
log global
balance roundrobin
mode tcp
timeout connect 5000ms
timeout client 50000ms
timeout server 50000ms
frontend tcp-in
bind *:8000
tcp-request inspect-delay 3s
acl read req.payload(0,4) -m sub read
acl write req.payload(0,5) -m sub write
use_backend read_servers if read
use_backend write_server if write
default_backend testFault
backend write_server
server server1 127.0.0.1:8001 maxconn 32
backend read_servers
server server2 127.0.0.1:8002 maxconn 32
backend testFault
server server3 127.0.0.1:8003 maxconn 32
The client code (in perl):
use IO::Socket::INET;
# auto-flush on socket
#$| = 1;
print "connecting to the server\n";
while(<STDIN>){
# create a connecting socket
my $socket = new IO::Socket::INET (
PeerHost => 'localhost',
PeerPort => '8000',
Proto => 'tcp',
);
die "cannot connect to the server $!\n" unless $socket;
# data to send to a server
$req = $_;
chomp $req;
$size = $socket->send($req);
print "sent data of length $size\n";
# notify server that request has been sent
shutdown($socket, 1);
# receive a response of up to 1024 characters from server
$response = "";
$socket->recv($response, 1024);
print "received response: $response\n";
$socket->close();
}
The server (perl code):
use IO::Socket::INET;
if(!$ARGV[0]){
die("Usage; specify a port..");
}
# auto-flush on socket
$| = 1;
# creating a listening socket
my $socket = new IO::Socket::INET (
LocalHost => '0.0.0.0',
LocalPort => $ARGV[0],
Proto => 'tcp',
Listen => 5,
Reuse => 0
);
die "cannot create socket $!\n" unless $socket;
print "server waiting for client connection on port $ARGV[0]\n";
while(1){
# waiting for a new client connection
my $client_socket = $socket->accept();
# get information about a newly connected client
my $client_address = $client_socket->peerhost();
my $client_port = $client_socket->peerport();
print "connection from $client_address:$client_port\n";
# read up to 1024 characters from the connected client
my $data = "";
$client_socket->recv($data, 1024);
print "received data: $data\n";
# write response data to the connected client
$data = "ok";
$client_socket->send($data);
# notify client that response has been sent
shutdown($client_socket, 1);
$client_socket->close();
print "Connection closed..\n\n";
}
$socket->close();
Binary data in haproxy is tricky. Probably some bug, but the following worked for me on haproxy 1.7.9.
I am trying to build a thrift proxy server which can route to appropriate backend based on the user_id in the payload.
frontend thriftrouter
bind *:10090
mode tcp
option tcplog
log global
log-format "%ci:%cp [%t] %ft %b/%s %Tw/%Tc/%Tt %B %ts %ac/%fc/%bc/%sc/%rc %sq/%bq captured_user:%[capture.req.hdr(0)] req.len:%[capture.req.hdr(1)]"
tcp-request inspect-delay 100ms
tcp-request content capture req.payload(52,10) len 10
tcp-request content capture req.len len 10
tcp-request content accept if WAIT_END
acl acl_thrift_call req.payload(2,2) -m bin 0001 # Thrift CALL method
acl acl_magic_field_id req.payload(30,2) -m bin 270f # Magic field number 9999
# Define access control list for each user
acl acl_user_u1 req.payload(52,10) -m sub |user1|
acl acl_user_u2 req.payload(52,10) -m sub |user2|
# Route based on the user. No default backend so that one always has to set it
use_backend backend_1 if acl_user_u1 acl_magic_field_id acl_thrift_call
use_backend backend_2 if acl_user_u2 acl_magic_field_id acl_thrift_call
When matching binary data in acl, make sure you're looking at the right number of bytes, for substring to work properly. Or use the hex conversion method and match on hex bytes.
Dont I feel silly. Re-reading the HAProxy documentation I found the following directive (fetch method) that fixes the issue:
tcp-request content accept if WAIT_END
That solved the unexpected behaviour.
My issue is as follows: I want to implement a listen service using scapy to stimulate a honeypot (because honeypot uses a fake ip, so I can't use OS sockets) and I chose scapy.
I implemented a very simple TCP hand-shake procedure, however one thing frustrated me: one byte of the packet I use PSH to send is eaten.
For example I send "abc" out to a client, but the client's socket, for example netcat or wget, only receive "bc". Another example is "HTTP/1.1 200 OK" becomes "TTP/1.1 200 OK". I captured packet and wireshark can correctly recognize my hand-made packet as HTTP, but the client socket just lack 1 byte. I don't know why.
The code is as follows:
192.168.1.100 stands for server(my) ip addr,9999 is the port. For example, I run this python script on 192.168.1.100, then I use "nc 192.168.1.100 9999". I expect to get "abc", but I can only get "bc", but the packet seems no problem in Wireshark. it's so strange.
'''
Created on Jun 2, 2012
#author: root
'''
from scapy import all
from scapy.layers.inet import IP, ICMP, TCP
from scapy.packet import ls, Raw
from scapy.sendrecv import sniff, send
from scipy.signal.signaltools import lfilter
import scapy.all
HOSTADDR = "192.168.1.100"
TCPPORT = 9999 'port to listen for'
SEQ_NUM = 100
ADD_TEST = "abc"
def tcp_monitor_callback(pkt):
global SEQ_NUM
global TCPPORT
if(pkt.payload.payload.flags == 2):
'A syn situation, 2 for SYN'
print("tcp incoming connection")
ACK=TCP(sport=TCPPORT, dport=pkt.payload.payload.sport, flags="SA",ack=pkt.payload.payload.seq + 1,seq=0)
send(IP(src=pkt.payload.dst,dst=pkt.payload.src)/ACK)
if(pkt.payload.payload.flags & 8 !=0):
'accept push from client, 8 for PSH flag'
print("tcp push connection")
pushLen = len(pkt.payload.payload.load)
httpPart=TCP(sport=TCPPORT, dport=pkt.payload.payload.sport, flags="PA", ack=pkt.payload.payload.seq + pushLen)/Raw(load=ADD_TEST)
'PROBLEM HERE!!!! If I send out abc, the client socket only receive bc, one byte disappers!!!But the packet received by client is CORRECT'
send(IP(src=pkt.payload.dst,dst=pkt.payload.src)/httpPart)
if(pkt.payload.payload.flags & 1 !=0):
'accept fin from cilent'
print ("tcp fin connection")
FIN=TCP(sport=TCPPORT, dport=pkt.payload.payload.sport, flags="FA", ack=pkt.payload.payload.seq +1, seq = pkt.payload.payload.ack)
send(IP(src=pkt.payload.dst,dst=pkt.payload.src)/FIN)
def dispatcher_callback(pkt):
print "packet incoming"
global HOSTADDR
global TCPPORT
if(pkt.haslayer(TCP) and (pkt.payload.dst == HOSTADDR) and (pkt.payload.dport == TCPPORT)):
tcp_monitor_callback(pkt)
else:
return
if __name__ == '__main__':
print "HoneyPot listen Module Test"
scapy.all.conf.iface = "eth0"
sniff(filter=("(tcp dst port %s) and dst host %s") % (TCPPORT,HOSTADDR), prn=dispatcher_callback)
Some suggestions:
Sniff may append some payload to the end of the packet, so len(pkt.payload.payload.load) may not be the real payload length. You can use pkt[IP].len-40 (40 is the common header length of IP+TCP). You may also use -len(pkt[IP].options)-len(pkt[TCP].options) for more accurate results.
Usually the application layer above TCP uses line breaks ("\r\n") to separate commands, so you'd better change ADD_TEST to "abc\r\n"
If none of above methods work, you may upgrade to the latest netcat and try again.
I tested your code, you are missing sending proper tcp sequence
httpPart=TCP(sport=TCPPORT, dport=pkt.payload.payload.sport, flags="PA", ack=pkt.payload.payload.seq + pushLen, seq=pkt.payload.payload.ack)/Raw(load=ADD_TEST)
should fix the issue, you may have other packet length issue, but the eaten 1 byte is caused by missing proper tcp sequence