I've been playing with httr and RCurl for a while and am struggling with specifying some certificate information.
I have a Curl CLI command that works just fine to log me in to a website that uses self-signed certs for authentication (as well as uname/pwd), it looks something like this...
curl -k -q -v --cert "../certs/foobar.pem" https://signon.foobar.com/certlogin -d "username=foo&password=bar" -H "X-Application: Curltest"
This logs me in to foobar.com and returns a session token.
The same command works if I use the corresponding .crt cert file and add a key using the --key option.
I'm trying to do the same thing with RCurl or httr and keep getting an error that indicates that the cert is wrong. Bearing in mind that I know the cert is OK (as the CLI command works) I assume this is because I'm specifying the cert incorrectly in RCurl.
So, quick question...
How would you reproduce the CURL command given above in RCurl or httr?
BTW this is on Ubuntu - so should not be related to the issues lots of folks are having with Mac OS X 10.9 +
Many thanks
Eluxoso
(first time poster - be nice....!)
Here is my (probably very inelegant & certainly non-functioning) code
RCurl:
Library(RCurl)
url <- "https://signon.foobar.com/certlogin"
uname <- "foo"
pswd <-"bar"
headers <- list("X-Application" = "Curltest")
opts=curlOptions(verbose=TRUE,
httpheader=headers,
cainfo="../certs/foobar.pem",
ssl.verifypeer=FALSE)
postForm(url,
"username" = uname, "password"=pswd,
.opts=opts,
style='POST'
)
httr:
library(httr)
cafile="../certs/foobar.pem"
resp<-POST("https://signon.foobar.com/certlogin",
body = list(username="foo", password = "bar"),
add_headers("X-Application" = "Curltest" ),
verbose(),
encode = "form",
config(cainfo = cafile, ssl.verifypeer=FALSE))
content(resp,"text")
Having used a full pathname for the cert I still get the same server response, despite the CURL CLI command working.
In the HTTP requests I noticed this:
SSL certificate verify result: self signed certificate in certificate chain (19), continuing anyway.
Is that relevant/helpful to someone?..
I had this issue, and it was resolved by installing libcurl4-openssl-dev and updating the Rcurl package to the most recent version (1.95)
$ sudo apt-get install libcurl4-openssl-dev
$ R
> install.packages("RCurl")
Related
I want to use wget to download a whole folder from ftp (i know about -r), curl does not allow downloading folder in one request. I have the following syntax for curl working but can't work out the syntax for even downloading a single file via wget. The key things here are that ftp has auth and getting to ftp is via http proxy (with diff credentials).
This is the working curl command:
curl --proxy-anyauth --proxy-user NTADMIN\proxyuser:proxypass --proxy http://httpproxyhost:8080 --anyauth -u ftpuser:ftppass -v 'ftp://ftphost/file'
What is the equivalent in wget?
You can try to use canonical URI format:
curl --proxy http://proxyuser:proxypass#httpproxyhost:8080 -v 'ftp://ftpuser:ftppass#ftphost/file'
With wget you can use command like this:
ftp_proxy=http://proxyuser:proxypass#httpproxyhost:8080 wget ftp://ftpuser:ftppass#ftphost/file
All I'm trying to do is read all the repos and issues in my organizations private repos. I can from my Windows 7 cmd.exe execute
curl -u "user:pass" https://api.github.com/orgs/:org/repos
and I get back all of my repositories. I can pipe this to a file:
curl -u "user:pass" https://api.github.com/orgs/:org/repos > "C:\Users\Location\file.txt"
and this saves the JSON output. I can replicate this in R but in what seems like a terrible way.
fullRepos = system('curl -s -u "user:pass" -G https://api.github.com/orgs/:org/repos',
intern=T,show.output.on.console = F)
This captures the output (intern = T) and the -s gets rid of the progress bars so I can collapse the lines and turn it into a data frame. This gets back all the repositories, public and private.
I tried using RCurl to do the same thing but the code below only provides the public repositories. The httpheader is because otherwise it the API rejects my call.
RCurl::getURL(url="https://api.github.com/orgs/:org/repos",userpwd ="user:pass",
httpheader = c('User-Agent' = "A user agent"))
I also tried httr and it also only provides the public repositories.
httr::GET(url="https://api.github.com/orgs/:org/repos",userpwd="user:pass")
What am I doing wrong with RCurl and httr? I'd rather have a workflow that doesn't make a system command and then paste the lines together.
We can use the authenticate() helper function in httr to build the authentication header for us w/o having to manually create it. Also, verbose() can be used to debug HTTP issues:
httr::GET(url="https://api.github.com/orgs/:org/repos",
httr::authenticate("user", "pass"),
httr::verbose())
I am unable to download file's from svn repository into unix AIX system. I am using "curl" command in unix to download the file but it fails with below error
curl: (60) SSL certificate problem, verify that the CA cert is OK. Details:
error:14090086:SSL routines:SSL3_GET_SERVER_CERTIFICATE:certificate verify failed
More details here: http://curl.haxx.se/docs/sslcerts.html
command used :
curl -u username:password http://server.com/svn/trunk/test.file
Are there any settings in subversion repository which i need to change to make the curl command work.If yes, then how to change them.
I cannot use wget because it is not installed on our machine the alternative is curl.
Please, help me download subversion files into unix. your help will be greatly appreciated.
Thanks,
Sri
The short way to do it would be to add the --insecure option to the curl command. This tells cURL to ignore the fact that it can't verify the signer of the SSL cert used by your SVN server.
curl -u username:password --insecure https://server.com/svn/trunk/test.file
The error is happening for one of two reasons, either your SVN access is secured using a self-signed certificate from a CA not in the certificate chain for the OS, or the trusted certs cURL is using is outdated and doesn't have a certificate from the CA that signed your SVN SSL certificate.
You can either download the root certificate that signed your SSL cert and specify it like: curl --cacert /path/to/cert.pem. Otherwise, you'll need to determine how and where to install additional certificates to be trusted. This partly depends on whether or not cURL is using OpenSSL or NSS. This site has some guidance on how to do this for various operating systems.
Thanks for your response. I have changed the command to
curl -k http://server.com/svn/trunk/test.file -u username:password > test.file
and it worked.
Thanks,
Sri
I am trying to install Meteor.js on a VM (Ubuntu 12.04) created with Vagrant.
The install should be as simple as:
curl https://install.meteor.com | /bin/sh
However this fails with curl: (7) couldn't connect to host
I have isolated the failure to a request within that shell script to this URL:
https://warehouse.meteor.com/bootstrap/0.7.0.1/meteor-bootstrap-Linux_i686.tar.gz
When I changed it to use HTTP instead of HTTPS it works. However I am running into problems elsewhere where it needs to pull things from httpS://warehouse.meteor.com/...
I thought the problem was with https, but if I do:
curl https://google.com
I get the page no problem, so what could be the issue?
Per another Ubuntu/Meteor question, it appears that there's some kind of certificate error (Meteor's SSL CA may not be installed by default in Ubuntu?) that goes away when you:
sudo apt-get update && sudo apt-get upgrade
For me upgrade didn't solve the problem.
My solution was to download the script from install.meteor.com and replace TARBALL_URL from https to http and I ran the script manually.
I am in my Terminal and I want to send a POST request to a given URL. I have tested this with a REST client so I know that the parameters work.
So lets say I want to POST the following parameters:
username=tony
password=secret
To my URL: https://exmaple.com/login/
I tried the following curl command in my Terminal (I am using OSX Lion)
curl --data "username=tony&password=secret" http://exmaple.com/login/
I get an 500 Server Error back from the server so I am now thinking of something that could be different between the REST Client and the curl command.
Thanks for your help
Update: I am using a https service. Do I have to adjust my curl command to account for this?
Try this
curl -F username=tony -F password=secret http://exmaple.com/login/
-F (reference) should probably do the same as --data? Possible the problem is in the webapp.
Maybe the app you are hitting uses basic auth for authentication? Try this one:
curl --user name:password http://exmaple.com/login/