I am trying to connect to a postgreSQL database (hosted on AWS RDS) via an SSH tunnel in R. So far, I have been able to connect using the following methods:
1.---------------------------
Opening the ssh tunnel in my terminal (MacOs) using
ssh -i {key file path} -f -N -L 5432:{db host}:5432 {ssh user}#{ssh host} -v
and then connecting to the database using
psql -hlocalhost -U{db user} -p5432 -dpostgres
2.---------------------------
Opening the ssh tunnel in my terminal and then running the following code in R to connect
conn <- dbConnect(
RPostgres::Postgres(),
dbname = db_name,
user = db_user,
password = db_password,
host = "127.0.0.1",
port = db_port
)
3.---------------------------
This is where the issue occurs. I'm able to connect by opening the ssh tunnel in R (in a background environment) with
tunnel_process <- callr::r_bg(
function(ssh_host, ssh_user, ssh_key, db_host, db_port) {
session <- ssh::ssh_connect(host = glue::glue("{ ssh_user }#{ ssh_host }"),
keyfile = ssh_key,
verbose = 3)
ssh::ssh_tunnel(session = session,
port = db_port,
target = glue::glue("{ db_host }:{ db_port }"))
},
args = list(ssh_host, ssh_user, ssh_key, db_host, db_port),
stdout = nullfile(),
stderr = nullfile()
)
But then I'm unable to use the same "dbConnect" code as above to connect. It only gives me the following error message
Error: could not connect to server: Connection refused
Is the server running on host "127.0.0.1" and accepting
TCP/IP connections on port 5432?
I am however able to connect directly from the terminal using the connection code in part 1. but only if I run psql -hlocalhost -U{db user} -p5432 -dpostgres, then re-run the ssh tunnel code in R, and only THEN enter my password in the terminal. I would appear that every time I try to connect, it closes the SSH tunnel, so I then have to re-launch it before submitting my password.
Question---------------------------
From what I just detailed, it would appear that:
a. My database is reachable since I can easily connect through the terminal
b. My R code works since I'm able to use it to both successfully open the SSH tunnel AND connect to the database. I'm just unable to use both together for some reason.
c. The tunnel I open through R breaks any time I try to connect to the database. This is not the case for the tunnel I open directly in the terminal.
Since I want to be able to do everything directly from R, does anybody here have any ideas on what may be causing the issue?
EDIT---------------------------
Here's the log I get in R when I try to connect to the database, just before it closes the tunnel:
> ssh::ssh_tunnel(session = session,
+ port = db_port,
+ target = glue::glue("{ db_host }:{ db_port }"))
\ Waiting for connetion on port 5432... client connected!
channel_open: Creating a channel 43 with 64000 window and 32768 max packet
ssh_socket_unbuffered_write: Enabling POLLOUT for socket
packet_send2: packet: wrote [len=124,padding=18,comp=105,payload=105]
channel_open: Sent a SSH_MSG_CHANNEL_OPEN type direct-tcpip for channel 43
ssh_packet_socket_callback: packet: read type 80 [len=492,padding=16,comp=475,payload=475]
ssh_packet_process: Dispatching handler for packet type 80
ssh_packet_global_request: Received SSH_MSG_GLOBAL_REQUEST packet
ssh_packet_global_request: UNKNOWN SSH_MSG_GLOBAL_REQUEST hostkeys-00#openssh.com 0
ssh_packet_process: Couldn't do anything with packet type 80
packet_send2: packet: wrote [len=12,padding=6,comp=5,payload=5]
ssh_socket_unbuffered_write: Enabling POLLOUT for socket
ssh_packet_socket_callback: packet: read type 91 [len=28,padding=10,comp=17,payload=17]
ssh_packet_process: Dispatching handler for packet type 91
ssh_packet_channel_open_conf: Received SSH2_MSG_CHANNEL_OPEN_CONFIRMATION
ssh_packet_channel_open_conf: Received a CHANNEL_OPEN_CONFIRMATION for channel 43:0
ssh_packet_channel_open_conf: Remote window : 2097152, maxpacket : 32768
| Tunneled -1 bytes...ssh_socket_unbuffered_write: Enabling POLLOUT for socket
packet_send2: packet: wrote [len=28,padding=10,comp=17,payload=17]
channel_write_common: channel_write wrote 8 bytes
| Tunneled 7 bytes...ssh_packet_socket_callback: packet: read type 94 [len=28,padding=17,comp=10,payload=10]
ssh_packet_process: Dispatching handler for packet type 94
channel_rcv_data: Channel receiving 1 bytes data in 0 (local win=64000 remote win=2097144)
channel_default_bufferize: placing 1 bytes into channel buffer (stderr=0)
channel_rcv_data: Channel windows are now (local win=63999 remote win=2097144)
ssh_socket_unbuffered_write: Enabling POLLOUT for socket
packet_send2: packet: wrote [len=28,padding=18,comp=9,payload=9]
grow_window: growing window (channel 43:0) to 1280000 bytes
ssh_channel_read_timeout: Read (1) buffered : 1 bytes. Window: 1280000
- Tunneled 8 bytes...ssh_socket_unbuffered_write: Enabling POLLOUT for socket
packet_send2: packet: wrote [len=316,padding=17,comp=298,payload=298]
channel_write_common: channel_write wrote 289 bytes
/ Tunneled 297 bytes...ssh_packet_socket_callback: packet: read type 94 [len=3964,padding=12,comp=3951,payload=3951]
ssh_packet_process: Dispatching handler for packet type 94
channel_rcv_data: Channel receiving 3942 bytes data in 0 (local win=1280000 remote win=2096855)
channel_default_bufferize: placing 3942 bytes into channel buffer (stderr=0)
channel_rcv_data: Channel windows are now (local win=1276058 remote win=2096855)
ssh_channel_read_timeout: Read (3942) buffered : 3942 bytes. Window: 1276058
\ Tunneled 4239 bytes...ssh_socket_unbuffered_write: Enabling POLLOUT for socket
packet_send2: packet: wrote [len=156,padding=8,comp=147,payload=147]
channel_write_common: channel_write wrote 138 bytes
- Tunneled 4377 bytes...ssh_packet_socket_callback: packet: read type 94 [len=76,padding=15,comp=60,payload=60]
ssh_packet_process: Dispatching handler for packet type 94
channel_rcv_data: Channel receiving 51 bytes data in 0 (local win=1276058 remote win=2096717)
channel_default_bufferize: placing 51 bytes into channel buffer (stderr=0)
channel_rcv_data: Channel windows are now (local win=1276007 remote win=2096717)
ssh_channel_read_timeout: Read (51) buffered : 51 bytes. Window: 1276007
| Tunneled 4428 bytes...ssh_socket_unbuffered_write: Enabling POLLOUT for socket
packet_send2: packet: wrote [len=140,padding=14,comp=125,payload=125]
channel_write_common: channel_write wrote 116 bytes
\ Tunneled 4544 bytes...ssh_packet_socket_callback: packet: read type 94 [len=60,padding=8,comp=51,payload=51]
ssh_packet_process: Dispatching handler for packet type 94
channel_rcv_data: Channel receiving 42 bytes data in 0 (local win=1276007 remote win=2096601)
channel_default_bufferize: placing 42 bytes into channel buffer (stderr=0)
channel_rcv_data: Channel windows are now (local win=1275965 remote win=2096601)
ssh_channel_read_timeout: Read (42) buffered : 42 bytes. Window: 1275965
/ Tunneled 4586 bytes...ssh_socket_unbuffered_write: Enabling POLLOUT for socket
packet_send2: packet: wrote [len=60,padding=19,comp=40,payload=40]
channel_write_common: channel_write wrote 31 bytes
- Tunneled 4617 bytes...packet_send2: packet: wrote [len=12,padding=6,comp=5,payload=5]
ssh_channel_send_eof: Sent a EOF on client channel (43:0)
ssh_socket_unbuffered_write: Enabling POLLOUT for socket
packet_send2: packet: wrote [len=12,padding=6,comp=5,payload=5]
ssh_channel_close: Sent a close on client channel (43:0)
ssh_socket_unbuffered_write: Enabling POLLOUT for socket
tunnel closed!
For reference, this is what the same log looks like when using the workaround detailed in 3. (re-running the ssh_tunnel right before submitting my password in the terminal):
> ssh::ssh_tunnel(session = session,
+ port = db_port,
+ target = glue::glue("{ db_host }:{ db_port }"))
\ Waiting for connetion on port 5432... client connected!
channel_open: Creating a channel 43 with 64000 window and 32768 max packet
ssh_socket_unbuffered_write: Enabling POLLOUT for socket
packet_send2: packet: wrote [len=124,padding=18,comp=105,payload=105]
channel_open: Sent a SSH_MSG_CHANNEL_OPEN type direct-tcpip for channel 43
ssh_packet_socket_callback: packet: read type 80 [len=492,padding=16,comp=475,payload=475]
ssh_packet_process: Dispatching handler for packet type 80
ssh_packet_global_request: Received SSH_MSG_GLOBAL_REQUEST packet
ssh_packet_global_request: UNKNOWN SSH_MSG_GLOBAL_REQUEST hostkeys-00#openssh.com 0
ssh_packet_process: Couldn't do anything with packet type 80
packet_send2: packet: wrote [len=12,padding=6,comp=5,payload=5]
ssh_socket_unbuffered_write: Enabling POLLOUT for socket
ssh_packet_socket_callback: packet: read type 91 [len=28,padding=10,comp=17,payload=17]
ssh_packet_process: Dispatching handler for packet type 91
ssh_packet_channel_open_conf: Received SSH2_MSG_CHANNEL_OPEN_CONFIRMATION
ssh_packet_channel_open_conf: Received a CHANNEL_OPEN_CONFIRMATION for channel 43:0
ssh_packet_channel_open_conf: Remote window : 2097152, maxpacket : 32768
| Tunneled -1 bytes...ssh_socket_unbuffered_write: Enabling POLLOUT for socket
packet_send2: packet: wrote [len=28,padding=10,comp=17,payload=17]
channel_write_common: channel_write wrote 8 bytes
| Tunneled 7 bytes...ssh_packet_socket_callback: packet: read type 94 [len=28,padding=17,comp=10,payload=10]
ssh_packet_process: Dispatching handler for packet type 94
channel_rcv_data: Channel receiving 1 bytes data in 0 (local win=64000 remote win=2097144)
channel_default_bufferize: placing 1 bytes into channel buffer (stderr=0)
channel_rcv_data: Channel windows are now (local win=63999 remote win=2097144)
ssh_socket_unbuffered_write: Enabling POLLOUT for socket
packet_send2: packet: wrote [len=28,padding=18,comp=9,payload=9]
grow_window: growing window (channel 43:0) to 1280000 bytes
ssh_channel_read_timeout: Read (1) buffered : 1 bytes. Window: 1280000
- Tunneled 8 bytes...ssh_socket_unbuffered_write: Enabling POLLOUT for socket
packet_send2: packet: wrote [len=316,padding=17,comp=298,payload=298]
channel_write_common: channel_write wrote 289 bytes
\ Tunneled 297 bytes...ssh_packet_socket_callback: packet: read type 94 [len=3964,padding=12,comp=3951,payload=3951]
ssh_packet_process: Dispatching handler for packet type 94
channel_rcv_data: Channel receiving 3942 bytes data in 0 (local win=1280000 remote win=2096855)
channel_default_bufferize: placing 3942 bytes into channel buffer (stderr=0)
channel_rcv_data: Channel windows are now (local win=1276058 remote win=2096855)
ssh_channel_read_timeout: Read (3942) buffered : 3942 bytes. Window: 1276058
/ Tunneled 4239 bytes...ssh_socket_unbuffered_write: Enabling POLLOUT for socket
packet_send2: packet: wrote [len=156,padding=8,comp=147,payload=147]
channel_write_common: channel_write wrote 138 bytes
| Tunneled 4377 bytes...ssh_packet_socket_callback: packet: read type 94 [len=76,padding=15,comp=60,payload=60]
ssh_packet_process: Dispatching handler for packet type 94
channel_rcv_data: Channel receiving 51 bytes data in 0 (local win=1276058 remote win=2096717)
channel_default_bufferize: placing 51 bytes into channel buffer (stderr=0)
channel_rcv_data: Channel windows are now (local win=1276007 remote win=2096717)
ssh_channel_read_timeout: Read (51) buffered : 51 bytes. Window: 1276007
- Tunneled 4428 bytes...ssh_socket_unbuffered_write: Enabling POLLOUT for socket
packet_send2: packet: wrote [len=140,padding=14,comp=125,payload=125]
channel_write_common: channel_write wrote 116 bytes
/ Tunneled 4544 bytes...ssh_packet_socket_callback: packet: read type 94 [len=60,padding=8,comp=51,payload=51]
ssh_packet_process: Dispatching handler for packet type 94
channel_rcv_data: Channel receiving 42 bytes data in 0 (local win=1276007 remote win=2096601)
channel_default_bufferize: placing 42 bytes into channel buffer (stderr=0)
channel_rcv_data: Channel windows are now (local win=1275965 remote win=2096601)
ssh_channel_read_timeout: Read (42) buffered : 42 bytes. Window: 1275965
\ Tunneled 4586 bytes...ssh_socket_unbuffered_write: Enabling POLLOUT for socket
packet_send2: packet: wrote [len=92,padding=12,comp=79,payload=79]
channel_write_common: channel_write wrote 70 bytes
- Tunneled 4656 bytes...ssh_packet_socket_callback: packet: read type 94 [len=380,padding=15,comp=364,payload=364]
ssh_packet_process: Dispatching handler for packet type 94
channel_rcv_data: Channel receiving 355 bytes data in 0 (local win=1275965 remote win=2096531)
channel_default_bufferize: placing 355 bytes into channel buffer (stderr=0)
channel_rcv_data: Channel windows are now (local win=1275610 remote win=2096531)
ssh_channel_read_timeout: Read (355) buffered : 355 bytes. Window: 1275610
| Tunneled 5011 bytes...
Finally, here's the log when running ssh_connect:
> session <- ssh::ssh_connect(host = glue::glue("{ ssh_user }#{ ssh_host }"),
+ keyfile = ssh_key,
+ verbose = 3)
ssh_pki_import_privkey_base64: Trying to decode privkey passphrase=false
ssh_connect: libssh 0.8.6 (c) 2003-2018 Aris Adamantiadis, Andreas Schneider and libssh contributors. Distributed under the LGPL, please refer to COPYING file for information about your rights, using threading threads_pthread
ssh_socket_connect: Nonblocking connection socket: 50
ssh_connect: Socket connecting, now waiting for the callbacks to work
ssh_connect: Actual timeout : 10000
ssh_socket_pollcallback: Received POLLOUT in connecting state
socket_callback_connected: Socket connection callback: 1 (0)
ssh_socket_unbuffered_write: Enabling POLLOUT for socket
callback_receive_banner: Received banner: SSH-2.0-OpenSSH_7.4
ssh_client_connection_callback: SSH server banner: SSH-2.0-OpenSSH_7.4
ssh_analyze_banner: Analyzing banner: SSH-2.0-OpenSSH_7.4
ssh_analyze_banner: We are talking to an OpenSSH client version: 7.4 (70400)
ssh_known_hosts_read_entries: Failed to open the known_hosts file '/etc/ssh/ssh_known_hosts': No such file or directory
ssh_client_select_hostkeys: Changing host key method to "ecdsa-sha2-nistp256,ssh-ed25519,ecdsa-sha2-nistp521,ecdsa-sha2-nistp384,ssh-rsa,ssh-dss"
ssh_socket_unbuffered_write: Enabling POLLOUT for socket
packet_send2: packet: wrote [len=644,padding=9,comp=634,payload=634]
ssh_packet_socket_callback: packet: read type 20 [len=1276,padding=10,comp=1265,payload=1265]
ssh_packet_process: Dispatching handler for packet type 20
ssh_kex_select_methods: Negotiated curve25519-sha256,ecdsa-sha2-nistp256,aes256-ctr,aes256-ctr,hmac-sha2-256,hmac-sha2-256,none,none,,
ssh_socket_unbuffered_write: Enabling POLLOUT for socket
packet_send2: packet: wrote [len=44,padding=6,comp=37,payload=37]
ssh_packet_socket_callback: packet: read type 31 [len=260,padding=11,comp=248,payload=248]
ssh_packet_process: Dispatching handler for packet type 31
ssh_packet_dh_reply: Received SSH_KEXDH_REPLY
ssh_socket_unbuffered_write: Enabling POLLOUT for socket
packet_send2: packet: wrote [len=12,padding=10,comp=1,payload=1]
ssh_client_curve25519_reply: SSH_MSG_NEWKEYS sent
ssh_packet_socket_callback: Processing 112 bytes left in socket buffer
ssh_packet_socket_callback: packet: read type 21 [len=12,padding=10,comp=1,payload=1]
ssh_packet_process: Dispatching handler for packet type 21
ssh_packet_newkeys: Received SSH_MSG_NEWKEYS
crypt_set_algorithms2: Set output algorithm to aes256-ctr
crypt_set_algorithms2: Set HMAC output algorithm to hmac-sha2-256
crypt_set_algorithms2: Set input algorithm to aes256-ctr
crypt_set_algorithms2: Set HMAC input algorithm to hmac-sha2-256
ssh_packet_newkeys: Signature verified and valid
ssh_packet_socket_callback: Processing 96 bytes left in socket buffer
ssh_packet_socket_callback: packet: read type 7 [len=60,padding=6,comp=53,payload=53]
ssh_packet_process: Dispatching handler for packet type 7
ssh_packet_ext_info: Received SSH_MSG_EXT_INFO
ssh_packet_ext_info: Follows 1 extensions
ssh_packet_ext_info: Extension: server-sig-algs=<rsa-sha2-256,rsa-sha2-512>
ssh_connect: current state : 7
packet_send2: packet: wrote [len=28,padding=10,comp=17,payload=17]
ssh_service_request: Sent SSH_MSG_SERVICE_REQUEST (service ssh-userauth)
ssh_socket_unbuffered_write: Enabling POLLOUT for socket
ssh_packet_socket_callback: packet: read type 6 [len=28,padding=10,comp=17,payload=17]
ssh_packet_process: Dispatching handler for packet type 6
ssh_packet_service_accept: Received SSH_MSG_SERVICE_ACCEPT
ssh_socket_unbuffered_write: Enabling POLLOUT for socket
packet_send2: packet: wrote [len=44,padding=4,comp=39,payload=39]
ssh_packet_socket_callback: packet: read type 51 [len=60,padding=15,comp=44,payload=44]
ssh_packet_process: Dispatching handler for packet type 51
ssh_packet_userauth_failure: Access denied for 'none'. Authentication that can continue: publickey,gssapi-keyex,gssapi-with-mic
ssh_packet_userauth_failure: Access denied for 'none'. Authentication that can continue: publickey,gssapi-keyex,gssapi-with-mic
ssh_key_algorithm_allowed: Checking rsa-sha2-512 with list <ssh-ed25519,ecdsa-sha2-nistp256,ecdsa-sha2-nistp384,ecdsa-sha2-nistp521,ssh-rsa,rsa-sha2-512,rsa-sha2-256,ssh-dss>
ssh_key_algorithm_allowed: Checking rsa-sha2-512 with list <ssh-ed25519,ecdsa-sha2-nistp256,ecdsa-sha2-nistp384,ecdsa-sha2-nistp521,ssh-rsa,rsa-sha2-512,rsa-sha2-256,ssh-dss>
ssh_key_algorithm_allowed: Checking rsa-sha2-512 with list <ssh-ed25519,ecdsa-sha2-nistp256,ecdsa-sha2-nistp384,ecdsa-sha2-nistp521,ssh-rsa,rsa-sha2-512,rsa-sha2-256,ssh-dss>
ssh_socket_unbuffered_write: Enabling POLLOUT for socket
packet_send2: packet: wrote [len=636,padding=11,comp=624,payload=624]
ssh_packet_socket_callback: packet: read type 52 [len=12,padding=10,comp=1,payload=1]
ssh_packet_process: Dispatching handler for packet type 52
ssh_packet_userauth_success: Authentication successful
Currently using on OH1 Heart rate sensor (tiny.cc/mom03y)
I'm looking to subscribe to HR notification using bluepy. I have got the notifications working but the OH1 device disconnects after about 20-30secs in both bluepy and gatttool (remote user terminated) but not in bluetoothctl.
Looking for a reason why the connection stays alive in bluetoothctl and not in bluepy or gatttool, code and hcidump below, using bluez 5.50 and bluepy 1.30 on rasbian 4.14.
Bluepy
#packet count
packets = 0
class hrCallback(btle.DefaultDelegate):
def __init__(self):
btle.DefaultDelegate.__init__(self)
def handleNotification(self, cHandle, data):
global packets
packets += 1
print("packet: %s Handle: %s HR (bpm): %s " % (packets, cHandle, data[1]))
#connect to OH1
mac = "a0:9e:1a:4f:ef:8b"
oh1 = btle.Peripheral( mac )
oh1.setDelegate( hrCallback() )
#start hr notification
oh1.writeCharacteristic(38, b"\x01\x00", True)
#listen for notifications
while True:
try:
if oh1.waitForNotifications(1.0):
continue
except btle.BTLEDisconnectError:
pass
hcidump
> HCI Event: Command Complete (0x0e) plen 4
LE Set Scan Parameters (0x08|0x000b) ncmd 1
status 0x00
> HCI Event: Command Complete (0x0e) plen 4
LE Set Scan Enable (0x08|0x000c) ncmd 1
status 0x00
> HCI Event: Command Complete (0x0e) plen 4
LE Set Scan Enable (0x08|0x000c) ncmd 1
status 0x00
> HCI Event: Command Status (0x0f) plen 4
LE Create Connection (0x08|0x000d) status 0x00 ncmd 1
> HCI Event: Command Status (0x0f) plen 4
LE Read Remote Used Features (0x08|0x0016) status 0x00 ncmd 1
> HCI Event: Disconn Complete (0x05) plen 4
status 0x00 handle 64 reason 0x13
Reason: Remote User Terminated Connection
Which version of BlueZ are you using? several tools (e.g. gatttool, hcitool, hciconfig) was deprecated and replaced by bluetoothctl and btmgmt, and the recommendation from the BlueZ team is to use the new tools instead. Please have a look at the following link:-
Deprecated BlueZ Tools
The difference between the new tools and the old tools is that the old tools were able of interfacing directly with the kernel, whereas the new tools perform operations by interfacing with D-Bus instead.
Therefore, the recommendation is always to use bluetoothctl as old tools are not maintained and this is likely why you are seeing the issue.