I am designing an integration service to go to an SFTP location, grab an encrypted file, copy it to another server on our network and decrypt the file. Then use SSIS tasks to load it into a database.
I am using an execute process task to call putty to grab the file, but I'd like to create a new directory on the remote server with the date I'm running the job and move the decrypted file into that directory.
using this commmand with mkdir is creating it on the SFTP server, not the remote server.
for /f "tokens=1* delims=" %%a in ('date /T') do set datestr=%%a
mkdir %datestr%
I'm new to using putty and encryption so any advice would be welcome.
Essentially I want to do this, but I have to create the date folder in the destination: (I'm not using all 1's for the ip :))
get encryptedFile.txt \\111.111.111.111\c$\%datestr%\decryptedFile.txt
I'm using this as my starting point - https://www.mssqltips.com/sqlservertip/3435/using-sftp-with-sql-server-integration-services/
Related
I have created the batch to transfer the file using SSH keys, I checked the public and private key mapping on both the servers and it's working fine.
My Windows batch code using SFTP command is as follows:
open sftp://sftp_user#ssh_dest_server -privatekey="D:\directory_full_path\private_key.ppk" -rawsettings TryAgent=0 AuthKI=0 AuthGSSAPI=0
CD "/XYZ_Directory/folder1"
Put "\\full_directory_path\FILE1.zip"
exit
When I execute the batch manually it's executing fine without any issue, but when I execute batch from SQL Job (using different user) then it's shows below error:
Searching for host...
Connecting to host...
Authenticating...
Continue connecting to an unknown server and add its host key to a cache?
The server's host key was not found in the cache. You have no guarantee that the server is the computer you think it is.
The server's RSA key details are:
Algorithm: ssh-rsa 2048
SHA-256: finger_print_key
MD5: zz:xx:yy:xx:yy:xx:yy:xx:yy:xx:yy:xx:yy:zz:zz:00
If you trust this host,
press Yes. To connect without adding host key to the cache,
press No. To abandon the connection press Cancel.
In scripting, you should use a -hostkey switch to configure the expected host key.
(Y)es, (N)o, C(a)ncel (10 s), (C)opy Key, (P)aste key: Cancel
Host key wasn't verified!
Host key fingerprint is ssh-rsa 2048 finger_print_key.
Authentication failed.
I already tried -hostkey WinSCP command but says "unknown command". Suggestions are most welcome.
Something I want to do like this link "WinSCP" through WinSCP command but inside my Windows batch automatically to verify the host.
To verify a host key in WinSCP script, add -hostkey switch to the open command:
open sftp://sftp_user#ssh_dest_server -hostkey=... -privatekey="D:\directory_full_path\private_key.ppk" -rawsettings TryAgent=0 AuthKI=0 AuthGSSAPI=0
See Verifying the host key ... in script in WinSCP documentation. It covers everything you need to know. In particular, where to get the host key value.
Also note that WinSCP GUI can generate a script template even with the -hostkey switch for you.
Also covered in My script works fine when executed manually, but fails or hangs when run by Windows Scheduler, SSIS or other automation service. What am I doing wrong?
I am seeking to set up a Release Pipeline in Azure DevOps Services that will deploy
an application to a Unix server, where it then executes some unix commands as part
of the deployment.
Would appreciate some guidance on which pipeline Task(s) I can set up to therefore
achieve the following objectives:
Connect to the Unix server.
Execute the required Unix commands.
By the way, the Agents are currently installed on Windows hosts but we are looking to
extend that to Unix servers in due course, so a solution that fits both setups would
be ideal, even though the former is the priority.
You can check out task SSH Deployment task.
Use this task to run shell commands or a script on a remote machine using SSH. This task enables you to connect to a remote machine using SSH and run commands or a script.
If you need to copy files to the remote linux server. You can check out Copy Files Over SSH task.
You probably need to create a SSH service connection. See steps here to create as service connection.
In the end, due to concerns raised about the install of private keys on the target server which is part of the SSH Deployment setup, we opted for the use of Deployment Groups which has enabled us to set up a persistent connection to our Linux server.
I am searching for a way to run a .sh Script located on MachineB on MachineA. So I don't want to run the script remotely. I want to run the script locally (the script is located on a remote machine).
I have tested this command
ssh pi#192.168.1.2 "/usr/local/bin/nas/script1.sh"
But with this the command get executed remotely.
Background info: MachineA is my Synology-NAS; MachineB is my RaspberryPI.
All my folders on the NAS are encrypted. If the NAS reboots, all folders are unmounted. I want to automate the mounting process, but don't want to share the keys on a USB-Media for security reasons. For that I have a script with the keys for all folders on my PI in an encrypted folder.
The PI is online 24/7. For the scenario of theft. The Raspberry will a) not be stolen, b) will power-off and the encrypted folder will be unmounted. So, the keys are safe.
Any help is appreciated.
As a starting point, try:
/bin/bash <<end
`ssh pi#192.168.1.2 "cat /usr/local/bin/nas/script1.sh"`
end
i have SSH access to my source server on digitalocean and ftp access on my azure server then need to move large directory of images from source server to remote server, how can i do it with FTP or mput or similar commands ?
i have only FTP access to remote server.
start with your source server, go to the directory where you have all the files. Start a FTP session to your remote server. After login type prompt
then issue mput *.*.
If you want to get files from target server, then you can use mget
is this what you are looking for?
start FTP connection as follow:
FTP
open destination ftp server
enter username &
password
cd destination folder
lcd source folder path
prompt
mput *.*
all the commands must be the above order.
I'm looking for a way that will allow me to use an SSH Tunnel to connect to a MySQL Server (as opposed to a file) within R; I'm assuming it'll require a combination of RCurl and RODBC, but I can't seem to get it to work properly.
I came across this post and this post that talk about utilizing SSH to connect to specific files or tables, but I'm hoping to use it as part of a Shiny app that will execute different SQL queries based on input from the user, which would require connecting into the server as opposed to specific files.
I'm assuming the code would look something along these lines x = scp("remote.ssh.host.com", "/home/dir/file.txt", "My.SCP.Passphrase", user="username"), but would I replace the "/home/dir/file.txt" piece with an odbcConnect() statement or replace it with the port number for the specific database I want to access?
Edit: The line I use for a regular odbcConnect() is odbcConnect(dsn, uid = "userid", pwd = "password"). Part of the problem is, I am developing it on Windows, but it will be deployed to a Linux server (handled be someone else) so I'm struggling to figure out what exactly will need to be used in my server.R code for connecting to the database.
Okay, so to test this on Windows, either grab Cygwin, or install OpenSSH so you can run ssh from the command line in Windows, like you would do in Linux.
Once you have ssh running on your Windows box, then try first making a tunnel through SSH. Run this from the command line:
ssh -f <server_user>#<server_ip> -L <unused_local_port>:localhost:<database_remote_port> -N
Obviously, replace everything in '<>' with the appropriate information. It will ask for the password, and remember that this isn't the database password, but the password to the server itself. Notably, the server_ip doesn't have to be the server with the database on it, just any server that is inside the proper subnet and that runs an SSH server, which is pretty much all Linux machines.
Now, setup an ODBC connection, except make the IP localhost, and the port unused_local_port. Now, try connecting to your new ODBC connection in R. If this works you're halfway there.
The next problem is the password, because you will have to enter a password to connect via SSH, but in R you won't be able to input it after a simple system command. So you have to setup some a public/private rsa key pair. Notably, this will make it so that anyone with access to your user/pass on your Windows box will now have automatic access to your server, so be careful. First, generate a SSH key:
ssh-keygen -t rsa
Don't make a passphrase, and save it in the default location. Now, create the directory for your public key on the remote host, and drop your public key in there.
# This creates a directory on the other machine if it wasn't already there. (Type in your password on the remote machine)
ssh <server_user>#<server_ip> mkdir -p .ssh
# This adds your public key to the list of accepted ones:
cat ~/.ssh/id_rsa.pub | ssh <server_user>#<server_ip> 'cat >> .ssh/authorized_keys'
Now try creating your tunnel again from the command line:
ssh -f <server_user>#<server_ip> -L <unused_local_port>:localhost:<database_remote_port> -N
If it doesn't ask you for the password, you have succeeded in creating your keypair. Now you are ready to run your ssh command from the command line. But before you do that, try and kill your ssh command, so you can make sure that R is actually creating the tunnel, and you aren't just reusing an old one. You can do it through Windows Task Manager (Ctrl+Alt+Esc), and just right click and End Process the ssh.exe.
So, just run:
system('ssh -f <server_user>#<server_ip> -L <unused_local_port>:localhost:<database_remote_port> -N')
And then connect to your new tunneled ODBC connection.