R SSH Tunnel MySQL - r

I'm looking for a way that will allow me to use an SSH Tunnel to connect to a MySQL Server (as opposed to a file) within R; I'm assuming it'll require a combination of RCurl and RODBC, but I can't seem to get it to work properly.
I came across this post and this post that talk about utilizing SSH to connect to specific files or tables, but I'm hoping to use it as part of a Shiny app that will execute different SQL queries based on input from the user, which would require connecting into the server as opposed to specific files.
I'm assuming the code would look something along these lines x = scp("remote.ssh.host.com", "/home/dir/file.txt", "My.SCP.Passphrase", user="username"), but would I replace the "/home/dir/file.txt" piece with an odbcConnect() statement or replace it with the port number for the specific database I want to access?
Edit: The line I use for a regular odbcConnect() is odbcConnect(dsn, uid = "userid", pwd = "password"). Part of the problem is, I am developing it on Windows, but it will be deployed to a Linux server (handled be someone else) so I'm struggling to figure out what exactly will need to be used in my server.R code for connecting to the database.

Okay, so to test this on Windows, either grab Cygwin, or install OpenSSH so you can run ssh from the command line in Windows, like you would do in Linux.
Once you have ssh running on your Windows box, then try first making a tunnel through SSH. Run this from the command line:
ssh -f <server_user>#<server_ip> -L <unused_local_port>:localhost:<database_remote_port> -N
Obviously, replace everything in '<>' with the appropriate information. It will ask for the password, and remember that this isn't the database password, but the password to the server itself. Notably, the server_ip doesn't have to be the server with the database on it, just any server that is inside the proper subnet and that runs an SSH server, which is pretty much all Linux machines.
Now, setup an ODBC connection, except make the IP localhost, and the port unused_local_port. Now, try connecting to your new ODBC connection in R. If this works you're halfway there.
The next problem is the password, because you will have to enter a password to connect via SSH, but in R you won't be able to input it after a simple system command. So you have to setup some a public/private rsa key pair. Notably, this will make it so that anyone with access to your user/pass on your Windows box will now have automatic access to your server, so be careful. First, generate a SSH key:
ssh-keygen -t rsa
Don't make a passphrase, and save it in the default location. Now, create the directory for your public key on the remote host, and drop your public key in there.
# This creates a directory on the other machine if it wasn't already there. (Type in your password on the remote machine)
ssh <server_user>#<server_ip> mkdir -p .ssh
# This adds your public key to the list of accepted ones:
cat ~/.ssh/id_rsa.pub | ssh <server_user>#<server_ip> 'cat >> .ssh/authorized_keys'
Now try creating your tunnel again from the command line:
ssh -f <server_user>#<server_ip> -L <unused_local_port>:localhost:<database_remote_port> -N
If it doesn't ask you for the password, you have succeeded in creating your keypair. Now you are ready to run your ssh command from the command line. But before you do that, try and kill your ssh command, so you can make sure that R is actually creating the tunnel, and you aren't just reusing an old one. You can do it through Windows Task Manager (Ctrl+Alt+Esc), and just right click and End Process the ssh.exe.
So, just run:
system('ssh -f <server_user>#<server_ip> -L <unused_local_port>:localhost:<database_remote_port> -N')
And then connect to your new tunneled ODBC connection.

Related

How to perform a MySQL Dump to a remote datastore

I am trying to get the databases backed up from a local machine to a datastore. The local machine is a VM within VCenter and I need the DB to go onto one of it's datastores.
I know the command for MySQLDump is:
mysqldump -u (username) -p --all-databases > (backupfilename).sql.
What do I put in the second section to get it to connect and push the backup to the datastore as a file?
I tried the typical stuff you'd scp and rsync for but I'm not that versed in MariaDB, especially this version we have.
If you know login credentials and hostname (and whatever extra arguments you need) to connect to remote instance, you may replace > backupfilename.sql with | mysql -U user -P password -H host .... to pipe mysqldump directly or even use | tee somefile.sql | mysql .... to also have a local dump.
Mariadb vs mysql is almost inerchangeable, but dont bet your life on this assumption - check the outcome.
Nb: if remote server is only listening locally and not available via internet, you may use ssh port forwarding/socket forwarding to connect.

Password for a command run over SSH appearing in clear text

I have a laptop through which I connect to a server through SSH.
On the server, I typically have to run a command that authenticates me and fetches some credentials for the current session (let's call the command foo-creds)
Typically, if I perform an SSH first
$ ssh my-remote-server
and then in my SSH session run:
$ foo-creds
It prompts me for a password, and when I enter my password it appears blank (due to *nix based systems hiding the password for security reasons).
However, when I run the command as a part of SSH directly from my mac:
$ ssh my-remote-server foo-creds
It prompts me for a password but the password appears as plain-text (i.e. visible on the terminal).
Question: How do I make it work so that it appears as hidden even when executed as a part of the SSH command on my laptop?
Answer already exists at this SuperUser question.
$ ssh -t remote_server foo-creds
Simple explanation is that this forces allocation of a TTY to the ssh session which supports password hiding.
Thanks to #Martheen for providing the link!

How do I connect to a RDS MySQL instance from RStudio via a bastion host?

I would like to use RStudio for analysis of data on a MySQL instance. This is a AWS RDS MySql instance that is only accessible via a jump box / bastion host. I have the credentials necessary to connect to the jump box, and from the jump box to the RDS instance. What do I need to do be able to query this DB directly from within the RStudio console?
I can connect (using the Terminal tab in RStudio)to the jump box using:
ssh -p 22xx user#ip.add.re.ss
Then I can connect to RDS mysql using:
mysql -u username -p database -h hostname.us-east-1.rds.amazonaws.com
I can connect and do manual mysql commands from within RStudio terminal, but I don't seem to be able to do anything with the DB from the RStudio console.
Sorry for opening a 2yo thread, but for everyone dealing with this issue as I am - I found this thread and it looks like it works (connecting with MySQL via ssh from R Studio).
You should use something which is called port forwarding. Some details
are here
(https://help.ubuntu.com/community/SSH/OpenSSH/PortForwarding) For
example, say you wanted to connect from your laptop to
http://www.ubuntuforums.org using an SSH tunnel. You would use source
port number 8080 (the alternate http port), destination port 80 (the
http port), and destination server www.ubuntuforums.org. :
ssh -L 8080:www.ubuntuforums.org:80 Where should be
replaced by the name of your laptop.
This is done for whole computer so you dont need to do this from r
studio.
Offcourse you need to forward your port to 3036. But you need special
privilige on the server. Because on most hosting you can only connect
from localhost (for example from PHP)
Source: https://www.py4u.net/discuss/881859

How to do SSH key exchange between VMs via HEAT template (Not only between Control node and VMs)

I am newbie to openstack. I am creating a stack using HEAT template. In the yaml file I mentioned the key name as
parameters: # Common parameters
key_name: my-key-pair
After the stack is created, I am able to ssh to all VMs from my control node without password, successfully like this:
ssh -i /root/my-key-pair.pem
user#instanceip
My requirement here is, similarly I need to do ssh between the VMs also. Just like I did between ControlNode and VMs, I wanted to do ssh without password from VM1 to VM2.
If I copy the pem file to VM1, then I can do ssh without password from this VM1 to other VMS like
ssh -i /VM1-home/my-key-pair.pem
user#otherinstanceip
But, is there a way that this can be accomplished during stack creation itself? So that, immediately after stack creation via heat template, I can ssh from any instance to other instances?
Can someone help please.
Thank You,
Subeesh
You can do this without HEAT.
You should be able to make use of ssh agent forwarding.
steps:
Start the ssh-agent in the background.
eval "$(ssh-agent -s)"
Add your key to the agent
ssh-add /root/my-key-pair.pem
Then ssh into the first host, you should be able to jump between servers now.
The way of doing it with HEAT would be to place that pem file into the correct location on the created instances, this should be possible with the personality function
personality: {"/root/my-key-pair.pem": {get_file: "pathtopemfilelocaly"}}

Connecting to an external server / database using dplyr

I'm trying to connect to a database that is located in an external server using dplyr's
src_postgres(dbname = NULL, host = NULL, port = NULL, user = NULL,
password = NULL, ...)
So far so good, I've got all the parameters I need to connect to the database. The problem is that the server where the database is located requires an authentication too (username and password).
I tried creating a connection with ?pipe but seems like it only works when trying to extract files from a remote server.
Any clues?
Good news! I do this all the time and it's not hard :)
Two steps:
1. Create SSH key and put on remote server
from https://serverfault.com/posts/241593/edit
Generate ssh keys on your local machine:
$ ssh-keygen -t rsa -b 2048
And press Enter for empty passphrase to result in:
Generating public/private rsa key pair.
Enter file in which to save the key (/home/username/.ssh/id_rsa):
Enter passphrase (empty for no passphrase):
Enter same passphrase again:
Your identification has been saved in /home/username/.ssh/id_rsa.
Your public key has been saved in /home/username/.ssh/id_rsa.pub.
Copy your keys to the target server
ssh-copy-id id#server
Check that this worked with ssh 'id#server', and check folder .ssh/authorized_keys for the ssh keys.
You should know be able to log in with $ ssh id#server
2. Forward your database port to your local machine
You should know be able to use
ssh -fN id#server
to initiative an SSH connection and forward ports on to your local host.
You may need to adjust the -p parameter to ssh to select the correct port.
Once you can successfully forward the port, you should be able to use src_postres() from your local machine to access the remote database.
You can also start your R script with
system("ssh -fN id#server")
or put the command in your .Rprofile
3. (optional)
Also, maybe you don't want your id and server address in your scripts, say, if you were going to give them to a client, or put them on github.
Then, edit or create file (on local machine) .ssh/config with the following content:
Host my_ssh
User id
Hostname server
and then you can just use ssh -fN my_ssh
the best way to do this is by connecting via DBI and then using the open connection with dplyr. For example:
library(DBI)
con <- dbConnect(RPostgres::Postgres())
db_table <- tbl(con, "my_table")
db_table %>%
group_by(one_var) %>%
tally()

Resources