I have created the batch to transfer the file using SSH keys, I checked the public and private key mapping on both the servers and it's working fine.
My Windows batch code using SFTP command is as follows:
open sftp://sftp_user#ssh_dest_server -privatekey="D:\directory_full_path\private_key.ppk" -rawsettings TryAgent=0 AuthKI=0 AuthGSSAPI=0
CD "/XYZ_Directory/folder1"
Put "\\full_directory_path\FILE1.zip"
exit
When I execute the batch manually it's executing fine without any issue, but when I execute batch from SQL Job (using different user) then it's shows below error:
Searching for host...
Connecting to host...
Authenticating...
Continue connecting to an unknown server and add its host key to a cache?
The server's host key was not found in the cache. You have no guarantee that the server is the computer you think it is.
The server's RSA key details are:
Algorithm: ssh-rsa 2048
SHA-256: finger_print_key
MD5: zz:xx:yy:xx:yy:xx:yy:xx:yy:xx:yy:xx:yy:zz:zz:00
If you trust this host,
press Yes. To connect without adding host key to the cache,
press No. To abandon the connection press Cancel.
In scripting, you should use a -hostkey switch to configure the expected host key.
(Y)es, (N)o, C(a)ncel (10 s), (C)opy Key, (P)aste key: Cancel
Host key wasn't verified!
Host key fingerprint is ssh-rsa 2048 finger_print_key.
Authentication failed.
I already tried -hostkey WinSCP command but says "unknown command". Suggestions are most welcome.
Something I want to do like this link "WinSCP" through WinSCP command but inside my Windows batch automatically to verify the host.
To verify a host key in WinSCP script, add -hostkey switch to the open command:
open sftp://sftp_user#ssh_dest_server -hostkey=... -privatekey="D:\directory_full_path\private_key.ppk" -rawsettings TryAgent=0 AuthKI=0 AuthGSSAPI=0
See Verifying the host key ... in script in WinSCP documentation. It covers everything you need to know. In particular, where to get the host key value.
Also note that WinSCP GUI can generate a script template even with the -hostkey switch for you.
Also covered in My script works fine when executed manually, but fails or hangs when run by Windows Scheduler, SSIS or other automation service. What am I doing wrong?
Related
Our file transfer automation software is connecting to an external SFTP server to download files. It's connecting using Putty's command line tool PSFTP. When it kicks off at it's scheduled time, an error results. The error is:
Remote working directory is /directory
lcd: unable to change directory: Invalid Signature.
Using username "username".
Pre-authentication banner message from server:
| IM CCaaS FTP server
End of banner message from server
When I manually FTP to the server, I can log in with no issue. I then re-run the job and it completes successfully. I've googled this and found articles on key exchanges with similar errors, but this connection isn't using private/public keys. I cut over the application to a new Windows 2016 server on September 21st. There have been some successful transfers. I've also read the version of PSFTP may be the issue (0.73), but I would have expected consistent issues connecting. It seems sporadic. I'm not sure if the issue is on my end or at the destination. Any thoughts?
This is not SFTP problem. You have problem when changing a local working directory (using lcd command in the sftp client).
So this probably covers the problem:
System error 2148073478, extended error, or Invalid Signature error message on SMB connections in Windows Server 2012 or Windows 8.
I'm trying to connect to a database that is located in an external server using dplyr's
src_postgres(dbname = NULL, host = NULL, port = NULL, user = NULL,
password = NULL, ...)
So far so good, I've got all the parameters I need to connect to the database. The problem is that the server where the database is located requires an authentication too (username and password).
I tried creating a connection with ?pipe but seems like it only works when trying to extract files from a remote server.
Any clues?
Good news! I do this all the time and it's not hard :)
Two steps:
1. Create SSH key and put on remote server
from https://serverfault.com/posts/241593/edit
Generate ssh keys on your local machine:
$ ssh-keygen -t rsa -b 2048
And press Enter for empty passphrase to result in:
Generating public/private rsa key pair.
Enter file in which to save the key (/home/username/.ssh/id_rsa):
Enter passphrase (empty for no passphrase):
Enter same passphrase again:
Your identification has been saved in /home/username/.ssh/id_rsa.
Your public key has been saved in /home/username/.ssh/id_rsa.pub.
Copy your keys to the target server
ssh-copy-id id#server
Check that this worked with ssh 'id#server', and check folder .ssh/authorized_keys for the ssh keys.
You should know be able to log in with $ ssh id#server
2. Forward your database port to your local machine
You should know be able to use
ssh -fN id#server
to initiative an SSH connection and forward ports on to your local host.
You may need to adjust the -p parameter to ssh to select the correct port.
Once you can successfully forward the port, you should be able to use src_postres() from your local machine to access the remote database.
You can also start your R script with
system("ssh -fN id#server")
or put the command in your .Rprofile
3. (optional)
Also, maybe you don't want your id and server address in your scripts, say, if you were going to give them to a client, or put them on github.
Then, edit or create file (on local machine) .ssh/config with the following content:
Host my_ssh
User id
Hostname server
and then you can just use ssh -fN my_ssh
the best way to do this is by connecting via DBI and then using the open connection with dplyr. For example:
library(DBI)
con <- dbConnect(RPostgres::Postgres())
db_table <- tbl(con, "my_table")
db_table %>%
group_by(one_var) %>%
tally()
I'm looking for a way that will allow me to use an SSH Tunnel to connect to a MySQL Server (as opposed to a file) within R; I'm assuming it'll require a combination of RCurl and RODBC, but I can't seem to get it to work properly.
I came across this post and this post that talk about utilizing SSH to connect to specific files or tables, but I'm hoping to use it as part of a Shiny app that will execute different SQL queries based on input from the user, which would require connecting into the server as opposed to specific files.
I'm assuming the code would look something along these lines x = scp("remote.ssh.host.com", "/home/dir/file.txt", "My.SCP.Passphrase", user="username"), but would I replace the "/home/dir/file.txt" piece with an odbcConnect() statement or replace it with the port number for the specific database I want to access?
Edit: The line I use for a regular odbcConnect() is odbcConnect(dsn, uid = "userid", pwd = "password"). Part of the problem is, I am developing it on Windows, but it will be deployed to a Linux server (handled be someone else) so I'm struggling to figure out what exactly will need to be used in my server.R code for connecting to the database.
Okay, so to test this on Windows, either grab Cygwin, or install OpenSSH so you can run ssh from the command line in Windows, like you would do in Linux.
Once you have ssh running on your Windows box, then try first making a tunnel through SSH. Run this from the command line:
ssh -f <server_user>#<server_ip> -L <unused_local_port>:localhost:<database_remote_port> -N
Obviously, replace everything in '<>' with the appropriate information. It will ask for the password, and remember that this isn't the database password, but the password to the server itself. Notably, the server_ip doesn't have to be the server with the database on it, just any server that is inside the proper subnet and that runs an SSH server, which is pretty much all Linux machines.
Now, setup an ODBC connection, except make the IP localhost, and the port unused_local_port. Now, try connecting to your new ODBC connection in R. If this works you're halfway there.
The next problem is the password, because you will have to enter a password to connect via SSH, but in R you won't be able to input it after a simple system command. So you have to setup some a public/private rsa key pair. Notably, this will make it so that anyone with access to your user/pass on your Windows box will now have automatic access to your server, so be careful. First, generate a SSH key:
ssh-keygen -t rsa
Don't make a passphrase, and save it in the default location. Now, create the directory for your public key on the remote host, and drop your public key in there.
# This creates a directory on the other machine if it wasn't already there. (Type in your password on the remote machine)
ssh <server_user>#<server_ip> mkdir -p .ssh
# This adds your public key to the list of accepted ones:
cat ~/.ssh/id_rsa.pub | ssh <server_user>#<server_ip> 'cat >> .ssh/authorized_keys'
Now try creating your tunnel again from the command line:
ssh -f <server_user>#<server_ip> -L <unused_local_port>:localhost:<database_remote_port> -N
If it doesn't ask you for the password, you have succeeded in creating your keypair. Now you are ready to run your ssh command from the command line. But before you do that, try and kill your ssh command, so you can make sure that R is actually creating the tunnel, and you aren't just reusing an old one. You can do it through Windows Task Manager (Ctrl+Alt+Esc), and just right click and End Process the ssh.exe.
So, just run:
system('ssh -f <server_user>#<server_ip> -L <unused_local_port>:localhost:<database_remote_port> -N')
And then connect to your new tunneled ODBC connection.
I got the connection details of a SFTP server, connected to it with FileZilla, and then successfully downloaded a file from that SFTP.
The only details I had was host, port, user and pass.
Now I'm trying to connect to this same server trough WinSCP .NET assembly (C#)
using(Session session = new WinSCP.Session()) {
session.Open(new SessionOptions() {
Protocol = Protocol.,
HostName = "ftp.*********.be",
UserName ="*****",
Password ="*****"
});
TransferOptions transferOptions = new TransferOptions();
transferOptions.TransferMode = WinSCP.TransferMode.Binary;
TransferOperationResult transferResult;
transferResult = session.GetFiles("/downld/fileonserver.dbf",#"c:\testfolder\localfilename.dbf", false, transferOptions);
Whatever I try here it keeps asking for a key for SSH, but I don't have that key, I generated a 128 bit RSA key somewhere online and put it in the session options like:
SshHostKeyFingerprint = "ssh-rsa 1024 82:09:12:b4:93:92:3a:61:10:90:61:12:b4:XX:XX:XX"
But this just tells me that key is invalid.
I kind of figured out that I maybe need the public/private SSH key from the server to get this to work but I sadly don't have access to this server.
Since FileZilla can connect to it without me entering any KEYS, why can't my C# program do the same?
I'm not an expert when it comes to security related stuff, so please point me in the right direction. I found this thread but I don't have access to .ssh folder on the FTP server and I don't really get where they are going with this.
You are confusing the SSH server public host key verification with the client public key authentication. These are two completely different things. This first involves the public key of the server, while the latter involves your account public key.
Read about SSH Key Pairs to learn the difference.
FileZilla cannot connect without verifying the server's public host key either. On the first connection it always prompts you to accept the key. Once you do, it optionally caches the key and won't prompt you again, unless the key changes.
You have probably forgotten that you got this prompt before or someone else connected to the server before from your machine.
Any SSH (SFTP) client must do the same. You are losing any security had you not verified your server's host key.
You should get the host key fingerprint from your server administrator.
If you had not, you can see it on WinSCP Server and Protocol information dialog.
For details see WinSCP FAQ Where do I get SSH host key fingerprint to authorize the server?
I solved this by just copying the SSH key returned to my FileZilla client into my C# app. I don't know if this is the right thing to do, but at least it got my solution working now.
It was also an SSH-DSS key 2048 key instead of an SSH-RSA 1024, and that's why messing around with the keys kept failing I guess.
I'm trying to connect to a sftp but I'm getting an error:
Establishing sfpt connection failed: the host signature is invalid or the host key was not accepted.
Any idea how to make this work? I'm using filezilla and fugu just fine. It's just aptana that's not working.
Thanks!
I'm using Aptana Studio 3 and I was getting the same error in OS X (you didn't specify OS though). In my case it was objecting because I had spun up a new server so the remote server's ssh signature had changed, hence the "the host signature is invalid".
If you go to Preferences -> General -> Network Connections -> SSH2 there is a tab for Known Hosts which appears to be populated from the ~/.ssh/known_hosts file.
This is a list of remote host signatures and you should find your remote host in this list.
There is a Remove button here which will remove the host from the list but bringing up the dialog again shows it hasn't been removed at all!
It looks like the list is populated at startup and the Remove button is not functional. The only way I could get this to work was to:
Edit the ~/.ssh/known_hosts file outside of Aptana and delete the particular host key.
Restart Aptana