Password SSH authentication method in RCurl - r

I'm using the ftpUpload function in the RCurl package to upload files to an sftp file server. I'm having difficulty working out the authentication call.
Below is my call:
ftpUpload(what = "some-file.png",
to = "sftp://some-ftp-server.com:22/path/to/some-file.png",
verbose = TRUE,
userpwd = "my_userid:my_password")
As a result I get:
* About to connect() to some-ftp-server.com port 22 (#0)
* Trying some-ftp-server.com... * connected
* Connected to some-ftp-server.com (some ip address) port 22 (#0)
* SSH authentication methods available: publickey,password
* Using ssh public key file /home/.ssh/id_dsa.pub
* Using ssh private key file /home/.ssh/id_dsa
* SSH public key authentication failed: Unable to open public key file
* Authentication failure
* Closing connection #0
Error in function (type, msg, asError = TRUE) : Authentication failure
I wasn't the one to setup the sftp server, and I'm somewhat of an ssh noob -- apologies. What I do know is that I'm able to login using my_userid and my_password with Filezilla and that the server has an .htaccess and .htpasswd file.
I'm hoping that there is some way to authenticate using ftpUpload with just my userid and password. It seems that password is one of the two available methods, but I can't seem to get ftpUpload to understand that I'd like to use the later alone.
The .htpasswd file seems to contain my_userid:my_password, though the password portion is encrypted. I'm open to loading that in a certain place for ftpUpload to access, but I'm not sure how to point ftpUpload in the right directions.
Finally, I've tried playing around with and looking through the libcurl options listed here: http://www.omegahat.org/RCurl/philosophy.html and more fully explained here: http://curl.haxx.se/libcurl/c/curl_easy_setopt.html
Alas, no luck. Any help appreciated!

It seems a little late, but I am currently trying something similar (public key authentication, though) and got it working!
Here is what I did:
I did set up a public key authentication basically following the instructions at http://www.howtoforge.com/set-up-ssh-with-public-key-authentication-debian-etch
That is, I run on the server the command
ssh-keygen -t rsa -C "e#mail.com" -f "ir_rsa"
to generate a public and a private key. (Just as a sidenote, I first tried to use
puttygen on my local windows machine to create working keys but failed.)
Then I
add the public key to the authorized keys (still on the remote server)
mkdir ~/.ssh
chmod 700 ~/.ssh
cat id_rsa.pub >> ~/.ssh/authorized_keys
chmod 600 ~/.ssh/authorized_keys
and copy both the public and private key file on the local machine where
R is running. Then I can upload a file from R using
require(RCurl)
ftpUpload(what = "myfile.txt",
to = "sftp://myusername#my.host.com:22/path/to/myfile",
verbose = TRUE,
.opts = list(
ssh.public.keyfile = "path/to/local/pubkeyfile",
ssh.private.keyfile = "path/to/local/privatekeyfile"
)
)

Related

RCurl::scp() - Unable to exchange encryption keys

I currently try to read a file from a remote machine via RCurl::scp() on linux.
I have added my public (~/.ssh/id_rsa.pub in open-ssh format) and private (~/.ssh/id_rsa) keys and am able to ssh and scp to the remote machine from the terminal.
Unfortunatley I cannot yet transfer or read a file from within R.
RCurl::scp(
user = "root", host = "hostname",
path = "/path/file",
)
#> Error in function (type, msg, asError = TRUE) :
#> Failure establishing ssh session: -5, Unable to exchange encryption keys
My question may be related to Using SCP function from Rcurl in R

Problems connecting to SFTP with R and RCurl

I am attempting to connect to an SFTP site to pull data. It used to work, but for some reason, it stopped working a couple of weeks ago. The owners of the SFTP say nothing has changed on their end, and I can pull data easily without error using WinSCP.
protocol <- "sftp"
server <- "sftp.xxxx.net"
userpwd <- "user:password"
file <- "/public/bpus_dailytx.csv"
url <- paste0(protocol, "://", server, file)
data <- getURL(url = url, userpwd=userpwd, verbose = TRUE)
When I run this, I now get the following info:
* Trying xxx.xx.xx.xxx...
* Connected to sftp.xxxx.net (xxx.xx.xx.xxx) port 22 (#0)
* SSH MD5 fingerprint: 34rh3ie93hhr39hhdik3
* SSH authentication methods available: publickey,keyboard-interactive
* Using SSH public key file '(nil)'
* Using SSH private key file ''
* SSH public key authentication failed: Unable to extract public key from private key file: Unable to open private key file
* No identity would match
* Authentication failure
* Closing connection 0
Error in function (type, msg, asError = TRUE) : Authentication failure
It connects OK but then the authentication fails. Any ideas what could be going on here? Again, this code used to work but something has changed. What are some other ways I can attempt to pull the data other than this?
Edit: WinSCP screenshots:
It is difficult to say why your code stopped working because we do not have enough information about configurations (both on your machine and on the server) when it was working.
Because your keyfile was not in the proper format for RCurl, my leading hypothesis is that although the server folks said nothing changed on their end, I think they removed the password authentication option. That is because your code attempts password authentication only. If password authentication were still available, the one line in your output would look something like this:
SSH authentication methods available: publickey,password,keyboard-interactive
It is, as you noted, now:
SSH authentication methods available: publickey,keyboard-interactive
Therefore, the solution here was to convert your keyfile from PuTTY to OpenSSH format using PuTTYgen and then use the following RCurl code pointing to your new keyfile:
protocol <- "sftp"
server <- "sftp.xxxx.net"
file <- "/public/bpus_dailytx.csv"
url <- paste0(protocol, "://", server, file)
keypasswd <- "your_keypasswd"
ssh.private.keyfile = "your_path_to_keyfile"
username <- "your_username"
data <- getURL(url = url, keypasswd = keypasswd, ssh.private.keyfile = ssh.private.keyfile, username = username, verbose = TRUE)
And I will add a special thanks to #Tensibai for the assistance. It would have taken me way longer to arrive at this solution without their insight with the keyfile format.

How to HTTPS (SSL) with self-hosted ASP.NET Core 2 app (httpsys)

I wrote a little ASP.NET Core 2 application. It runs as a service, so no IIS. It runs on a PC with Windows 7 SP1.
var host = WebHost.CreateDefaultBuilder(args)
.UseContentRoot(pathToContentRoot)
.UseHttpSys(options =>
{
options.Authentication.Schemes = AuthenticationSchemes.None;
options.Authentication.AllowAnonymous = true;
options.MaxConnections = null;
options.MaxRequestBodySize = 30000000;
options.UrlPrefixes.Add("http://*:5050");
})
.UseStartup<Startup>()
.UseApplicationInsights()
.Build();
if (isService)
{
host.RunAsService();
}
else
{
host.Run();
}
As you can see, I want to listen on port 5050. This is working fine without SSL.
My question is, how can I enable https for my application? Again: No IIS, no Domain-Name (no internet connection). Communication is just inside the internal network, so I want to use a self-signed certificate.
I read the documentation (HTTP.sys documentation;Netsh Commands;New-SelfSignedCertificate), but there is always something different to my situation (they use Krestel, or it is for using IIS). Also, I dont know how to get the App-ID (needed for netsh) for my Application. I tryed this: StackOverflow Get GUID but it doesn't work.
var assembly = typeof(Program).Assembly;
// following line produces: System.IndexOutOfRangeException
var attribute = (GuidAttribute)assembly.GetCustomAttributes(typeof(GuidAttribute), true)[0];
var id = attribute.Value;
Console.WriteLine(id);
So I am a bit confused about all the possabilitys and different configurations. And the docs don't consider my specific case.
I created a certificate, and I guess I need to store it on the "my" Store. (Where is that? cert:\LocalMachine\My) And then I need to assign my Applicaion ID and Port to it.
But I have no idea how to do that exactly. Can anyone help?
So I solve the problem in the following way:
First, if you want to know your own GUID, you will get it with the following code:
var id = typeof(RuntimeEnvironment).GetTypeInfo().Assembly.GetCustomAttribute<GuidAttribute>().Value;
Create a SelfSigned Certificate
Now create a SelfSigned-Certificate (Skip this if you already got one, or purchased one)
Run the following OpenSSL command to generate your private key and public certificate. Answer the questions and enter the Common Name when prompted.
openssl req -newkey rsa:2048 -nodes -keyout key.pem -x509 -days 365 -out certificate.pem
Combine your key and certificate in a PKCS#12 (P12) bundle:
openssl pkcs12 -inkey key.pem -in certificate.pem -export -out certificate.p12
Install the certificate on the client:
For Windows 8 and higher:
Add Certificate to Windows Cert Store with PowerShell
PS C:> $certpwd = ConvertTo-SecureString -String "passwort" -Force –AsPlainText
PS C:> Import-PfxCertificate –FilePath D:\data\cert\certificate.p12 cert:\localMachine\my -Password $certpwd
Get Fingerprint (Hash) of certificate
PS C:\WINDOWS\system32> dir Cert:\LocalMachine\my
Install certificate (replace Hash, IP and Port with your values)
PS C:\WINDOWS\system32> $guid = [guid]::NewGuid()
PS C:\WINDOWS\system32> $certHash =
"A1D...B672E"
PS C:\WINDOWS\system32> $ip = "0.0.0.0"
PS C:\WINDOWS\system32> $port = "5050"
PS C:\WINDOWS\system32> "http add sslcert ipport=$($ip):$port
certhash=$certHash appid={$guid}" | netsh
You are done.
For Windows 7
Add Certificate to Windows Cert Store (note: use .pem file for this operation, because .p12 file seems to be not supported from certutil)
.\certutil.exe -addstore -enterprise -f "Root" C:\lwe\cert\certificate.pem
If his line throws the following error:
SSL Certificate add failed, Error 1312
A specified logon session does not exist. It may already have been terminated.
You have to do the steps manually (please insert the .p12 file when doing it manually, not .pem) :
Run mmc.exe
Go to File-> Add/Remove Snap-In
Choose the Certificates snap-in.
Select Computer Account
Navigate to: Certificates (Local Computer)\Personal\Certificates
Right click the Certificates folder and choose All Tasks -> Import.
Follow the wizard instructions to select the certificate. Be sure you check the export checkbox during wizard.
To get the hash of yor certificate, run the Internet Explorer, press Alt + X and go to Internet Options -> Content -> Certificates. Search your certificate and read the hash.
Now you can run the same commands as for Windows 8+:
Install certificate (replace Hash, IP and Port with your values)
PS C:\WINDOWS\system32> $guid = [guid]::NewGuid()
PS C:\WINDOWS\system32> $certHash =
"A1D...B672E"
PS C:\WINDOWS\system32> $ip = "0.0.0.0"
PS C:\WINDOWS\system32> $port = "5050"
PS C:\WINDOWS\system32> "http add sslcert ipport=$($ip):$port
certhash=$certHash appid={$guid}" | netsh
Edit your Code
After all, you have to set the UrlPrefixes to https. So in your Program.cs file you need to have:
var host = WebHost.CreateDefaultBuilder(args)
.UseContentRoot(pathToContentRoot)
.UseHttpSys(options =>
{
options.Authentication.Schemes = AuthenticationSchemes.None;
options.Authentication.AllowAnonymous = true;
options.MaxConnections = null;
options.MaxRequestBodySize = 30000000;
options.UrlPrefixes.Add("https://*:5050");
})
.UseStartup<Startup>()
.UseApplicationInsights()
.Build();

RCurl SSH public key authentication failed: Callback returned error

I need to get config file from the Linux server that is authenticated using ssh keys. I am stuck with "Callback returned error" message.
conf = scp(host="10.10.10.10", path="/home/admin/codebase/config.txt",
user="admin", keypasswd = "", verbose=TRUE,
key=c("C:/echinn/.ssh/my_public_key", "C:/echinn/.ssh/my_private_key"))
I get the following output
* Trying 10.10.10.10...
* Connected to 10.10.10.10 (10.10.10.10) port 22 (#0)
* SSH MD5 fingerprint: 8fa4562037d2f1e68c7ff419f9dc7656
* SSH authentication methods available: publickey,gssapi-keyex,gssapi-with-mic
* Using SSH public key file 'C:/echinn/.ssh/my_public_key'
* Using SSH private key file 'C:/echinn/.ssh/my_private_key'
* SSH public key authentication failed: Callback returned error
* Failure connecting to agent
* Authentication failure
* Closing connection 0
Error in function (type, msg, asError = TRUE) : Authentication failure
I also tried with "getURL()" but ended up with the same error. I am able to successfully connect using putty and WinSCP with (.ppk format of) the same public/private keys.
On the Linux server /var/log/secure I see the following for every execution
sshd[xxxx]: Connection closed by x.x.x.x [preauth]
From the R Documentation for the scp function, it looks like your key does not have a passphrase, so try setting keypasswd = NA or removing the argument altogether.

PSCP file from Windows to Linux using private/public keys

I can transfer file using PSCP:
C:\>pscp -pw <password> -r -p <path of the file> user#Server:<path file to stored>
But not using the public/private key.
Steps followed:
Generate public and private keys using PuTTYgen.
Copy the public key to authorized_keys of Remote Server
Save the private key to key.ppk in Windows server
Then
C:\>pscp -i privatekey pathofthefile user#server:pathfiletostored
It gives "Fatal: Network error: Connection refused"
Can someone please help?
Use the following code:
C:\>pscp -i "path\of\the\privatekey\privatekey.ppk" C:\temp\example_file.txt user#server:/path/file/to/be/stored
Note the quotes for the private key path and the private key should be in .ppk format.
The connection refused error may also be due to the wrong port. In that case, you need to mention the correct port by the following code :
C:\>pscp -i "path\of\the\privatekey\privatekey.ppk" -P 8022 C:\temp\example_file.txt user#server:path/file/to/be/stored
Note that 8022 is the port number and the P is uppercase. Hope this helps.

Resources