I´m trying to scrape a web page using a proxy but something isn´w working.
Here is an httr attemp to set the proxy options, below i try with RCurl.
I´ve read several answers regarding the subject but they don´t seem to be working.
Any suggestions?
### httr attempt
set_config(
use_proxy(url="proxy.xxx.com.ar", port=8080,
username = "xxxx\\xxxx", password = "xxxxx"),
override = TRUE
)
a <- GET("http://google.com/", verbose())
-> GET http://google.com/ HTTP/1.1
-> Proxy-Authorization: Basic dG1vdmlsZXNcbWFyYmVsOkFyYWNhbGFjYW5hMjM=
-> User-Agent: curl/7.19.7 Rcurl/1.95.4.1 httr/0.4.0.99
-> Host: google.com
-> Accept: */*
-> Accept-Encoding: gzip
-> Proxy-Connection: Keep-Alive
->
<- HTTP/1.1 407 Proxy Authentication Required
<- Server: pxsip02-srv.xxxxx.com.ar
<- Date: Mon, 11 Aug 2014 15:11:14 GMT
<- Content-Length: 309
<- Content-Type: text/html
<- Connection: Keep-Alive
<- Keep-Alive: timeout=60, max=8
<- Proxy-Authenticate: NTLM
<-
content(a)
<!DOCTYPE html PUBLIC "-//W3C//DTD HTML 4.0 Transitional//EN" "http://www.w3.org/TR/REC-html40/loose.dtd">
<html>
<head><title>Authentication Error</title></head>
<body>
<h1>Authentication Error</h1>There has been an error validating your user credentials. If the error persists,contact your network administrator.<br>Proxy authentication required<br><hr>
<br>Details: 407 Proxy Authentication Required</body>
</html>
### RCurl attempt
library("RCurl")
opts <- list(
proxy = "proxy.xxxxx.com.ar",
proxyusername = "xxxxxx\\xxxxx",
proxypassword = "xxxxxx",
proxyport = 8080,
capath = system.file("CurlSSL", "cacert.pem", package = "RCurl"),
verbose=TRUE, proxyauth=TRUE, useragent= "", header = TRUE
)
options( RCurlOptions = opts)
getURL("http://stackoverflow.com")
* About to connect() to proxy proxy.xxxxx.com.ar port 8080 (#0)
* Trying 10.167.195.11... * connected
* Connected to proxy.xxxxxx.com.ar (10.167.195.11) port 8080 (#0)
* Proxy auth using Basic with user 'xxxxxxx\xxxxx'
> GET http://stackoverflow.com HTTP/1.1
Proxy-Authorization: Basic VE1PVklMRVNcTUFSQkVMOkFyYWNhbGFjYW5hMjM=
Host: stackoverflow.com
Accept: */*
Proxy-Connection: Keep-Alive
[1] "HTTP/1.1 407 Proxy Authentication Required\r\nServer: pxsip02-srv.xxxx.com.ar\r\nDate: Mon, 11 Aug 2014 15:15:29 GMT\r\nContent-Length: 309\r\nContent-Type: text/html\r\nConnection: Keep-Alive\r\nKeep-Alive: timeout=60, max=8\r\nProxy-Authenticate: NTLM\r\n\r\n<html><head><title>Authentication Error</title></head><body><h1>Authentication Error</h1>There has been an error validating your user credentials. If the error persists,contact your network administrator.<br/>Proxy authentication required<br/><hr/><br/>Details: 407 Proxy Authentication Required</body></html>"
< HTTP/1.1 407 Proxy Authentication Required
< Server: pxsip02-srv.xxxxxxx.com.ar
< Date: Mon, 11 Aug 2014 15:15:29 GMT
< Content-Length: 309
< Content-Type: text/html
< Connection: Keep-Alive
< Keep-Alive: timeout=60, max=8
< Proxy-Authenticate: NTLM
<
* Connection #0 to host proxy.xxxxxx.com.ar left intact
Here is the update of the previous question. I add it as another answer so it´s easier to follow.
GET("http://google.com/",
config = list(
use_proxy(url="proxy.xxx.com.ar", port=8080,
username = "xxxx\\xxxx", password = "xxxxx",
proxyauth = 1)
)
)
The error messege:
Error in use_proxy(url = "proxy.xxxx.com.ar", port = 8080, username = "xxxxxx\\xxxx", :
unused argument (proxyauth = 1)
Related
I have created a web api using ASP.NET. The api only utilizes POST, receiving a json as a body.The web api works fine with Postman but I have problems using it with R.
A post with json using RStudio I get the error : Client error: (415) Unsupported Media Type.
Where is the error?
The verbose ( with code) from R studio is:
query = list(datarunID = "4",action = "CLIENTGETJOB",TreeRecordId = "73",result = "dfd")
> ListJSON = toJSON(query,pretty = FALSE,auto_unbox = TRUE)
> ListJSON
{"datarunID":"4","action":"CLIENTGETJOB","TreeRecordId":"73","result":"dfd"}
> url1 <- "http://localhost:29999/api/cycle"
> r <- POST(url = url1,body = ListJSON,encode = "json",verbose(data_out = TRUE, data_in = TRUE, info = TRUE) )
* Connected to localhost (::1) port 29999 (#3)
-> POST /api/cycle HTTP/1.1
-> Host: localhost:29999
-> User-Agent: libcurl/7.64.1 r-curl/4.3.3 httr/1.4.4
-> Accept-Encoding: deflate, gzip
-> Accept: application/json, text/xml, application/xml, */*
-> Content-Length: 76
>> {"datarunID":"4","action":"CLIENTGETJOB","TreeRecordId":"73","result":"dfd"}
* upload completely sent off: 76 out of 76 bytes
<- HTTP/1.1 415 Unsupported Media Type
<- Transfer-Encoding: chunked
<- Content-Type: application/problem+json; charset=utf-8
<- Server: Microsoft-IIS/10.0
<- X-Powered-By: ASP.NET
<- Date: Thu, xxxxx GMT
<< af
<< {"type":"https://tools.ietf.org/html/rfc7231#section-6.5.13","title":"Unsupported Media Type","status":415,"traceId":"00-67xxxxxxxxa537c45-00"}
The Json that works with Postman is :
{
"datarunID":"4",
"action":"CLIENTGETJOB",
"TreeRecordId" : "73",
"result" : "dfd"
}
In R I'm trying to get data from a ftp server using RCurl. The server has explicit encryption activated. Below one of the many attempts to make this work but I always get 530 Login or password incorrect! Still the username / password works when using another client like WinSCP. Any help is welcome!
library(RCurl)
# CURL_SSLVERSION_TLSv1_1 <- 5L
# CURL_SSLVERSION_TLSv1_2 <- 6L
opts <- curlOptions(
proxy = "http://my.proxy/",
proxyport = 8080,
dirlistonly = TRUE,
sslversion = 6L,
ftp.use.epsv = FALSE,
ssl.verifypeer=TRUE
)
dat <- getURL("ftp://myUser:myPaas#ftps.myServer.com/", .opts = opts )
Here some screenshots of the winscp setup (only German OS available)
Connection screen:
Protocol screen:
WinSCP connection string:
Here is the output having verbose = TRUE added
* Trying 10.x.x.x...
* Connected to 10.x.x.x (10.x.x.x) port 8080 (#0)
> GET ftp://User:Pass#ftp.myserver.com/ HTTP/1.1
Host: ftp.myserver.com:21
Accept: */*
Proxy-Connection: Keep-Alive
< HTTP/1.1 403 Forbidden
< Server: squid/3.5.20
< Mime-Version: 1.0
< Date: Thu, 15 Nov 2018 11:12:24 GMT
< Content-Type: text/html;charset=utf-8
< Content-Length: 3898
< X-Squid-Error: ERR_FTP_FORBIDDEN 530
< Vary: Accept-Language
< Content-Language: en
< WWW-Authenticate: Basic realm="FTP ftp.myserver.com"
< X-Cache: MISS from Proxy-xxxxxx
< X-Cache-Lookup: MISS from Proxy-xxxxx:8080
< Via: 1.1 Proxy-xxxxxx (squid/3.5.20)
< Connection: keep-alive
<
* Connection #0 to host 10.x.x.x left intact
Here is the WinSCP log:
. 2018-11-15 17:09:15.999 Verbinde mit ftp.MyServer.com ...
. 2018-11-15 17:09:15.999 HTTP proxy command: CONNECT ftp.MyServer.com:21 HTTP/1.1
. 2018-11-15 17:09:15.999 Host: ftp.MyServer.com:21
. 2018-11-15 17:09:16.014 Verbindung mit dem Proxy hergestellt, führe Handshakes aus ...
. 2018-11-15 17:09:16.039 HTTP proxy response: HTTP/1.1 200 Connection established
. 2018-11-15 17:09:16.039 HTTP proxy headers:
. 2018-11-15 17:09:16.039
. 2018-11-15 17:09:16.039 Verbunden mit ftp.MyServer.com, TLS Verbindung wird ausgehandelt...
< 2018-11-15 17:09:16.059 220 Welcome to FTP MyServer
> 2018-11-15 17:09:16.059 AUTH TLS
< 2018-11-15 17:09:16.093 234 Using authentication type TLS
. 2018-11-15 17:09:16.179 Verifying certificate for "MY Server" with fingerprint xx:yy:zz.............. and 18 failures
. 2018-11-15 17:09:16.179 Certificate common name "ftp.MyServer.com" matches hostname
. 2018-11-15 17:09:16.179 Certificate for "MY Server" matches cached fingerprint and failures
. 2018-11-15 17:09:16.179 Using TLSv1.2, cipher TLSv1/SSLv3: ECDHE-RSA-AES256-GCM-SHA384, 2048 bit RSA, ECDHE-RSA-AES256-GCM-SHA384 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(256) Mac=AEAD
. 2018-11-15 17:09:16.216 TLS Verbindung hergestellt. Warte auf die Willkommensnachricht...
> 2018-11-15 17:09:16.216 USER MyUser
< 2018-11-15 17:09:16.216 331 Password required for MyUser
> 2018-11-15 17:09:16.216 PASS **********
< 2018-11-15 17:09:16.236 230 Logged on
Here is the R console log with more info:
* Trying 10.101.0.32...
* Connected to 10.101.0.32 (10.x.x.x) port 8080 (#0)
> GET ftp://MyUser:#ftp.MyServer.com/ HTTP/1.1
Host: ftp.MyServer.com:21
Accept: */*
Proxy-Connection: Keep-Alive
< HTTP/1.1 401 Unauthorized
< Server: squid/3.5.20
< Mime-Version: 1.0
< Date: Fri, 16 Nov 2018 08:25:46 GMT
< Content-Type: text/html;charset=utf-8
< Content-Length: 3898
< X-Squid-Error: ERR_FTP_FORBIDDEN 530
< Vary: Accept-Language
< Content-Language: en
< WWW-Authenticate: Basic realm="FTP MyServer ftp.MyServer.com"
< X-Cache: MISS from Proxy-xxxx
< X-Cache-Lookup: MISS from Proxy-xxxx:8080
< Via: 1.1 Proxy-xxxx (squid/3.5.20)
< Connection: keep-alive
<
* Ignoring the response-body
* Connection #0 to host 10.x.x.x left intact
* Issue another request to this URL: 'ftp://ftp.MyServer.com'
* Found bundle for host ftp.MyServer.com: 0xafea17a
* Re-using existing connection! (#0) with host 10.x.x.x
* Connected to 10.x.x.x (10.x.x.x) port 8080 (#0)
* Server auth using Basic with user 'Nordex'
> GET ftp://MyUser:#ftp.MyServer.com/ HTTP/1.1
Authorization: Basic Tm9yZGV50g==
Host: ftp.MyServer.com:21
Accept: */*
Proxy-Connection: Keep-Alive
< HTTP/1.1 401 Unauthorized
< Server: squid/3.5.20
< Mime-Version: 1.0
< Date: Fri, 16 Nov 2018 08:25:46 GMT
< Content-Type: text/html;charset=utf-8
< Content-Length: 3947
< X-Squid-Error: ERR_FTP_FORBIDDEN 530
< Vary: Accept-Language
< Content-Language: en
* Authentication problem. Ignoring this.
< WWW-Authenticate: Basic realm="FTP MyServer ftp.MyServer.com"
< X-Cache: MISS from Proxy-xxxx
< X-Cache-Lookup: MISS from Proxy-xxxx:8080
< Via: 1.1 Proxy-xxxx (squid/3.5.20)
< Connection: keep-alive
<
* Connection #0 to host 10.x.x.x left intact
WinSCP sends CONNECT command to proxy:
. 2018-11-15 17:09:15.999 HTTP proxy command: CONNECT ftp.MyServer.com:21 HTTP/1.1
While RCurl sends GET command:
> GET ftp://User:Pass#ftp.myserver.com/ HTTP/1.1
In cURL, you use CURLOPT_HTTPPROXYTUNNEL option to make it use CONNECT:
make libcurl tunnel all operations through the HTTP proxy (set with CURLOPT_PROXY). There is a big difference between using a proxy and to tunnel through it.
Tunneling means that an HTTP CONNECT request is sent to the proxy, asking it to connect to a remote host on a specific port number and then the traffic is just passed through the proxy.
In RCurl, that corresponds to httpproxytunnel. So this should do:
opts <- curlOptions(
proxy = "http://my.proxy/",
proxyport = 8080,
httpproxytunnel = 1,
...
)
I am relatively new to API's and all that stuff, but I try to collect data through an API for http://my.wirelesstag.net through R with the RCurl package. Yesterday it worked just fine, but today I get an following error: Error: Internal Server Error. I suspect that the error occurs because I don't close the previous connection—maybe I am wrong. Anyway, any help to get the script working is appreciated.
My script for retrieving the data is as below (the example is unfortunately not a MWE as I keep the login information for myself):
library(RCurl)
library(jsonlite)
# Acccept cookies
curl <- getCurlHandle()
curlSetOpt(cookiejar = "/tmp/cookies.txt", curl = curl, verbose = T)
# Account information ----------------------------------------------
# Parameter list
prm <- list(baseUrl = "https://my.wirelesstag.net",
username = "user#user.com",
password = "password")
# Sign-in ----------------------------------------------------------
# Sign-in
invisible(
postForm(paste0(prm$baseUrl, "/ethAccount.asmx/SignIn"),
curl = curl,
.opts = list(
postfields = toJSON(list(
email = prm$username,
password = prm$password
)),
httpheader = c("Content-Type" = "application/json")
))
)
The entire log when executing postForm() is:
* Trying 130.211.169.89...
* TCP_NODELAY set
* Connected to my.wirelesstag.net (130.211.169.89) port 443 (#0)
* ALPN, offering http/1.1
* Cipher selection: ALL:!EXPORT:!EXPORT40:!EXPORT56:!aNULL:!LOW:!RC4:#STRENGTH
* successfully set certificate verify locations:
* CAfile: /etc/ssl/cert.pem
CApath: none
* SSL connection using TLSv1.2 / ECDHE-RSA-AES256-SHA384
* ALPN, server did not agree to a protocol
* Server certificate:
* subject: CN=my.wirelesstag.net
* start date: Jul 20 00:00:00 2017 GMT
* expire date: Oct 19 23:59:59 2019 GMT
* subjectAltName: host "my.wirelesstag.net" matched cert's "my.wirelesstag.net"
* issuer: C=US; O=GeoTrust Inc.; CN=RapidSSL SHA256 CA
* SSL certificate verify ok.
> POST /ethAccount.asmx/SignIn HTTP/1.1
Host: my.wirelesstag.net
Accept: */*
Content-Type: application/json
Content-Length: 58
* upload completely sent off: 58 out of 58 bytes
< HTTP/1.1 500 Internal Server Error
< Cache-Control: no-cache
< Pragma: no-cache
< Content-Type: application/json; charset=utf-8
< Expires: -1
< Server: Microsoft-IIS/8.5
< jsonerror: true
< Date: Thu, 01 Mar 2018 11:27:33 GMT
< Content-Length: 1019
<
* Connection #0 to host my.wirelesstag.net left intact
Error: Internal Server Error
Here, I want to attach a file to certain Policy/Claim Number with the below command in the intranet URL.
library(RCurl)
options(RCurlOptions = list(
cainfo = system.file("CurlSSL", "cacert.pem", package = "RCurl"),
followlocation = TRUE,
httpheader = c(
Accept = "text/xml",
Accept = "multipart/*",'Content-Type' = "text/xml; charset=utf-8",
SOAPAction = "INTRANET URL"
),
userpwd = "USERID:PASSWORD",
netrc = TRUE,
timeout = 100,
postfields = CONTAINS SOAP DATA,
writefunction = reader$update,
verbose = TRUE,
httpauth = AUTH_BASIC)
)
getForm(
uri = "INTRANET URL",
file = fileUpload(
filename = "C:\\Users\\Username\\Desktop\\image001.pdf",
contentType = "application/pdf"
),
upload = TRUE
)
When I run the above commands, I am getting the status as "HTTP/1.1 200 OK" but the response from the service says no attachment.
* Couldn't find host applications.test.intra.XXXXX.nl in the _netrc file; using defaults
* Trying 11.11.11.11... * Connected to URL (11.11.11.11) port 443 (#0)
* successfully set certificate verify locations:* CAfile: C:/Program Files/R/R-3.4.2/library/RCurl/CurlSSL/cacert.pem
CApath: none* SSL connection using TLSv1.0 / ROOT* Server certificate:
* subject: CN=applications-tst.XX.XXXXX.com* start date: DATE TIMESTAMP GMT
* expire date: DATE TIMESTAMP GMT* subjectAltName: applications.test.intra.XXXXX.XX matched
* issuer: DC=com; DC=XXXXX; DC=us; CN=XXXXX-SHA2-Issuing-us-XXXXXXXXXX-CA* SSL certificate verify ok.
* Server auth using Basic with user 'USERID'
> POST /QUALIFIER1/QUALIFIER2/QUALIFIER3?file=C%3A%5CUsers%5CUSERNAME%5CDesktop%5Cimage001.pdf&file=character%280%29&file=application%2Fpdf&upload=TRUE HTTP/1.1
Authorization: Basic XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX= Host: applications.test.intra.XXXXX.nl
Accept: text/xml Accept: multipart/* Content-Type: text/xml; charset=utf-8
SOAPAction: https://applications.test.intra.XXXXX.nl/QUALIFIER1/QUALIFIER2/QUALIFIER3 Content-Length: 2654 Expect: 100-continue
* Done waiting for 100-continue < HTTP/1.1 200 OK < content-language: en-GB
< content-type: text/xml; charset=utf-8 < date: Fri, 24 Nov 2017 11:33:00 GMT < p3p: CP="XXX XXX XXX XXX XXX XXX"
< transfer-encoding: chunked < cache-control: no-cache="set-cookie, set-cookie2" < expires: Thu, 01 Dec 1994 16:00:00 GMT
< x-powered-by: Servlet/3.0 < Set-Cookie: LtpaToken2=TOKEN; Path=/; Secure; HttpOnly
< Set-Cookie: BIGipServer~XX~isam-emp-applications_tst_11111=111111111.11111.1111; path=/; Httponly; Secure
< * Connection #0 to host applications.test.intra.XXXXX.nl left intact
I've been working on an R interface to a HTTP API with digest authentication and I've been running into a problem wherein the request will work absolutely fine on my non-Windows OSs, but I always get 401 status when running exactly the same code on Windows.
I'm currently trying to do it with RCurl, but the same thing was happening with httr when I tried that.
Also the API is unfotunately proprietary so I've had to change all the URLs, sorry.
On my non-Windows OSs I get the following behaviour:
rprompt> getURL('http://demo.someapi.net/some/url', userpwd="demo:demo", httpauth=1L, verbose=TRUE)
* Trying 195.224.16.34...
* Connected to demo.someapi.net (195.224.16.34) port 443 (#0)
* TLS 1.0 connection using TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA
* Server certificate: *.someapi.net
* Server certificate: RapidSSL SHA256 CA - G3
* Server certificate: GeoTrust Global CA
> GET /some/url HTTP/1.1
Host: demo.someapi.net
Accept: */*
< HTTP/1.1 401 Unauthorized
< Cache-Control: no-cache
< Content-Type: text/html
< Server: Microsoft-IIS/7.5
< WWW-Authenticate: Digest realm="company.product", nonce="MTEvMDcvMjAxNiAwODo0NTozMw", opaque="0000000000000000", stale=false, algorithm=MD5, qop="auth"
< X-Powered-By: ASP.NET
< Date: Mon, 11 Jul 2016 08:44:33 GMT
< Content-Length: 1293
<
* Ignoring the response-body
* Connection #0 to host demo.someapi.net left intact
* Issue another request to this URL: 'https://demo.someapi.net/some/url'
* Found bundle for host demo.someapi.net: 0x7f96c8d55af0
* Re-using existing connection! (#0) with host demo.someapi.net
* Connected to demo.someapi.net (195.224.16.34) port 443 (#0)
* Server auth using Digest with user 'demo'
> GET /some/url HTTP/1.1
Host: demo.someapi.net
Authorization: Digest username="demo", realm="company.product", nonce="MTEvMDcvMjAxNiAwODo0NTozMw", uri="/some/url", cnonce="YjRkMDQxYmM4MDFkYTMxOWZhNTViNGNmYTM5YzQyNGI=", nc=00000001, qop=auth, response="5d9643d083b2380f12d71855a98ceac3", opaque="0000000000000000", algorithm="MD5"
Accept: */*
< HTTP/1.1 200 OK
< Cache-Control: private
< Content-Length: 981
< Content-Type: application/json; charset=utf-8
< Server: Microsoft-IIS/7.5
< X-AspNet-Version: 4.0.30319
< X-Powered-By: ASP.NET
< Date: Mon, 11 Jul 2016 08:44:33 GMT
<
* Connection #0 to host demo.someapi.net left intact
and evertything works exactly as we expect it to. On Windows however we get this:
rprompt> getURL('http://demo.someapi.net/some/url', userpwd="demo:demo", httpauth=1L, verbose=TRUE)
* Trying 195.224.16.34...
* Connected to demo.someapi.net (195.224.16.34) port 443 (#0)
* successfully set certificate verify locations:
* CAfile: C:/Users/username/Documents/R/win-library/3.3/RCurl/etc/ca-bundle.crt
CApath: none
* SSL connection using TLSv1.0 / ECDHE-RSA-AES256-SHA
* Server certificate:
* subject: OU=GT56411961; OU=See www.rapidssl.com/resources/cps (c)15; OU=Domain Control Validated - RapidSSL(R); CN=*.someapi.net
* start date: 2015-01-26 09:31:11 GMT
* expire date: 2018-03-28 16:30:51 GMT
* subjectAltName: demo.someapi.net matched
* issuer: C=US; O=GeoTrust Inc.; CN=RapidSSL SHA256 CA - G3
* SSL certificate verify ok.
> GET /some/url HTTP/1.1
Host: demo.someapi.net
Accept: */*
< HTTP/1.1 401 Unauthorized
< Cache-Control: no-cache
< Content-Type: text/html
< Server: Microsoft-IIS/7.5
< WWW-Authenticate: Digest realm="company.product", nonce="MTEvMDcvMjAxNiAwODo1MjowOA", opaque="0000000000000000", stale=false, algorithm=MD5, qop="auth"
< X-Powered-By: ASP.NET
< Date: Mon, 11 Jul 2016 08:51:07 GMT
< Content-Length: 1293
<
* Ignoring the response-body
* Connection #0 to host demo.someapi.net left intact
* Issue another request to this URL: 'https://demo.someapi.net/some/url'
* Found bundle for host demo.someapi.net: 0xaa60b80
* Re-using existing connection! (#0) with host demo.someapi.net
* Connected to demo.someapi.net (195.224.16.34) port 443 (#0)
* Server auth using Digest with user 'demo'
> GET /some/url HTTP/1.1
Authorization: Digest username="demo",realm="",nonce="MTEvMDcvMjAxNiAwODo1MjowOA",uri="/some/url",cnonce="553f542ddef0e3c265e50539297bad81",nc=00000001,algorithm=MD5,response="1ec58793bb1d8142f09af112b905fa36",qop="auth",opaque="0000000000000000"
Host: demo.someapi.net
Accept: */*
< HTTP/1.1 401 Unauthorized
< Cache-Control: no-cache
< Content-Type: text/html
< Server: Microsoft-IIS/7.5
* Authentication problem. Ignoring this.
< WWW-Authenticate: Digest realm="company.product", nonce="MTEvMDcvMjAxNiAwODo1MjowOA", opaque="0000000000000000", stale=false, algorithm=MD5, qop="auth"
< X-Powered-By: ASP.NET
< Date: Mon, 11 Jul 2016 08:51:07 GMT
< Content-Length: 1293
<
* Connection #0 to host demo.someapi.net left intact
which just returns a 401 landing page HTML.
The issue seems to be that the realm field is empty, but I have no idea how to fix this or even how to work around it.
It should be noted that both .NET's webclient and Python's requests library handles things fine, but unfortunately this has to be done in R.
I'm happy to use any R packages that are needed to help solve this.
Thanks.
For anyone else who ends up with a similar problem, you can get around it by using httr and using your own handles.
make.request <- function (url, user, pass) {
handle <- httr::handle(url)
response <- GET(url=NULL, authenticate(user, pass, type="digest"), handle=handle)
# error checking and stuff...
}
I'm not sure what the issue is with RCurl though.