I am trying to use R to access a web page in our organization using httr GET.
However i get a message that "Access is denied due to invalid credentials".
I can do the desired action manually.
it seems that authorization is done automatically when i use internet explorer to get to the web site but the access is blocked when trying to do the same action through R.
This is how i'm trying to do it:
(I can't supply the exact address because it's an intranet address which can be used only inside the organization the same for the proxy address)
library(httr)
r <- GET(myurl, useproxy(myproxyid, 80), verbose())
-> GET http: //myurl
-> host: xxx
-> User-Agent : libcurl...
-> Accept-Encoding: gzip, deflate
-> Proxy-Connection: Keep-Alive
-> Accept: application/json, text/xml, application/xml, *.*
<- HTTP/1.1 401 Unauthorized
<- Content-Type: text/html
<- Server: Microsoft-IIS/8.5
<- WWW-Authenticate: Negotiate
<- WWW-Authenticate: NTLM
<- X-Powered-By: ASP.NET
r
Response [myurl]
Date
Status: 401
...
<title>401 - Unauthorized: Access is denied due to invalid credentials.</title>
....
I understand that i somehow have to send my credentials with my request.
is it possible to somehow use automatic authentication?
Thanks
Rafael
OK
I finally made it.
I had to supply authentication data to my get command like this:
library(httr)
r <- GET(myurl,
useproxy(myproxyid, 80),
verbose(),
authenticate(user = "myuserid", password = "mypassword", type = "ntlm"))
I hope it helps anybody
Thanks
Rafael
Related
Request to DailySalesReport service in development enviromant:
<ns6:DailySalesReportRQ Version="2.0.0"
xmlns:ns1="http://www.ebxml.org/namespaces/messageHeader"
xmlns:ns2="http://www.w3.org/1999/xlink"
xmlns:ns3="http://schemas.xmlsoap.org/soap/envelope/"
xmlns:ns4="http://schemas.xmlsoap.org/ws/2002/12/secext"
xmlns:ns6="http://webservices.sabre.com/sabreXML/2011/10"
xmlns:ns7="http://services.sabre.com/STL/v01"
xmlns:ns8="http://services.sabre.com/STL_Header/v120" xmlns:ns9="http://www.w3.org/2000/09/xmldsig#">
<ns6:SalesReport PseudoCityCode="5VYJ" StartDate="2020-02-13"/>
</ns6:DailySalesReportRQ>
Receive: ERR.SWS.HOST.ERROR_IN_RESPONSE with "TICKETING DATABASE ERROR"
What does the TICKET DATABASE ERROR mean and how can I fix it?
First of all u must check if the date u request have any doc issued.
Below the RAW request for the service="DailySalesReportLLSRQ" which i do now!
POST https://webservices.havail.sabre.com/websvc HTTP/1.1
Accept-Encoding: gzip,deflate
Content-Type: text/xml;charset=UTF-8
SOAPAction: "DailySalesReportLLSRQ"
Content-Length: 1503
Host: webservices.havail.sabre.com
Connection: Keep-Alive
User-Agent: Apache-HttpClient/4.1.1 (java 1.5)
Shared/IDL:IceSess/SessMgr:1.0.IDL/Common/!ICESMS/RESA!ICESMSLB/RES.LB!-2917635922269276274!1343176!0!1
soapui.gwaereo#cvccorp.com.br
https://webservices.havail.sabre.com/websvc
_
soapui.gwaereo#cvccorp.com.br
DailySalesReport
DailySalesReportLLSRQ
3912336541697520620
2020-02-21T15:02:49
https://developer.sabre.com/eticketcouponllsrq mentions:
Common errors
TICKETING DATABASE ERROR:
The ticket has been issued from PCC XXX and It can be viewed only from this PCC.
Please remove the conjunctive ticket number in your request and you should be able to get a good response.
Carrier does not seem to have the image of this ticket available.
Seems the service is orchestrated so it is impossible to identify whi PNR / ticket causes a trouble. You need to contact Sabre support and provide a request body + a date of error.
I have been using WHO package in R to get data from WHO database without problems for the last several weeks. Yesterday I found that I could no longer do it. I reproduced the error in different machines using different R versions, running from R itself and from Rstudio, Mac and Windows alike...
Example with 2 of the variables I want to request.
library(WHO)
socio_econ <- c("WHS7_143", "WHS9_95")
SECON <- lapply(socio_econ, function(t) get_data(t))
Here's the error:
Error in get_result(url) : Internal Server Error (HTTP 500).
NOT AN ANSWER JUST SHOWING A DEBUGGING METHOD FOR THE OP
httr::with_verbose(get_data("WHS7_143"))
-> GET /gho/athena/api/GHO/WHS7_143?format=json&profile=simple HTTP/1.1
-> Host: apps.who.int
-> User-Agent: libcurl/7.51.0 r-curl/2.0 httr/1.2.1
-> Accept-Encoding: gzip, deflate
-> Cookie: TS01ac0ef4=015dd60f3e63259629be28ff562fb98a7b99c500697d6a49e2671ad07b50034231788b7dd97944f7f6fd363c9ef2b32a1a34c37a22
-> Accept: application/json, text/xml, application/xml, */*
->
<- HTTP/1.1 500 Internal Server Error
<- Date: Mon, 01 May 2017 20:46:40 GMT
<- Content-Type: text/html;charset=utf-8
<- Content-Length: 1298
<- Via: 1.1 ghodata.who.int
<- Connection: close
<- Set-Cookie: TS01ac0ef4=015dd60f3e63259629be28ff562fb98a7b99c500697d6a49e2671ad07b50034231788b7dd97944f7f6fd363c9ef2b32a1a34c37a22; Path=/
<-
Error in get_result(url) : Internal Server Error (HTTP 500).
It's definitely something happening server-side, but it could be that the API changed and the pkg has not updated yet.
Hitting the inferred URL directly:
suggests it's an Java server error on their end.
Very strange issue. I am trying to connect to some API (inside organization) which require first to POST key & code to some url and then use that cookie to get the desired data (json).
Running the POST request returns status 200 which is good - but no cookie is returned.
Running the same request in Firefox using "httprequester" returns a cookie as expected and works fine.
url<-"https://some_url"
login <- list(
Key="some_key",
Code="some_code"
)
try_temp<-POST(url = url,body=login,encode="form",verbose())
Result is:
-> POST /api/Service/Login HTTP/1.1
-> Host: **************
-> User-Agent: libcurl/7.53.1 r-curl/2.5 httr/1.2.1
-> Accept-Encoding: gzip, deflate
-> Accept: application/json, text/xml, application/xml, */*
-> Content-Type: application/x-www-form-urlencoded
-> Content-Length: 43
->
>> Key=*****&Code=*******
<- HTTP/1.1 200 OK
<- Content-Type: text/html; charset="utf-8"
<- Content-Length: 6908
<- Connection: Close
<-
Thing is, that the same request works when down in browser.
GET request (after I know the cookie, I use GET in httr passing the cookie i've got. I get the same log as above.
BTW When instead of GET i use BROWSE - R opens the default browser and i see the expected data returned.
I suspect that some of the settings for R are not the same as for Firefox (or any other browser). We don't use PROXY but rather automatic configuration script.
Tnx
I'm trying to establish a connection using kerberos authentication. I think the question I have does not depend on the type of server (in my case it's a cognos tm1 server) nor the language (in my case R with use of the package httr (or RCurl)) since it's more a general http(s) thing.
I do not have much experience using kerberos. According to my understanding there is some negotiation between the client and server following the following steps (here get-requests). The only thin I need to pass is a username, no password is needed.
get(url) -> returning a "WWW-Authenticate: Kerberos" header telling this authmethod is supported.
get(url, header = Authentification: "Negotiate" + token) --> second request, this time with header information "Negotiate" plus token
Server returns some authentification details.
Received details can be sent in the header again and the requested data is sent back
httr (type = gssnegotiate) or curl (4 = CURLAUTH_NEGOTIATE) allow to enter negotiation types. I thought, this should do the negotiation process described above and return the requested data straight ahead. This does not seem to be the case:
library(httr)
httr::set_config(config( ssl_verifypeer = 0L))
httr::set_config(config( ssl_verifyhost = 0L))
GET(url, authenticate(user = "user", password = "", type = "gssnegotiate"), verbose = TRUE)
does not return the desired result. The log says:
-> GET /api/v1/Dimensions('Time')/Hierarchies('Time')/Subsets('Yesterday')/Elements HTTP/1.1
-> Host: myhostaddress.com:20049
-> User-Agent: libcurl/7.47.1 r-curl/1.2 httr/1.2.1
-> Accept-Encoding: gzip, deflate
-> Cookie: TM1SessionId=tbQcdXh4PsIHUQdkW_UyNQ
-> Accept: application/json, text/xml, application/xml, */*
->
<- HTTP/1.1 401 Unauthorized
<- Content-Type: text/plain
<- Content-Length: 0
<- Connection: keep-alive
<- OData-Version: 4.0
<- WWW-Authenticate: Kerberos
<-
* Connection #0 to host myhostaddress.com left intact
I tried the same using (R)curl
library(RCurl)
getURL(url, user = "username", userpwd="", httpauth = 4, verbose = TRUE, ssl.verifypeer = FALSE, ssl.verifyhost = FALSE)
Unfortunately, this wasn't successful as well:
< HTTP/1.1 401 Unauthorized
< Content-Type: text/plain
< Content-Length: 0
< Connection: keep-alive
< OData-Version: 4.0
< Set-Cookie: TM1SessionId=WMSrJHGTps0RIbmjCCaW5w; Path=/api/; HttpOnly; Secure
< WWW-Authenticate: Kerberos
Do you have any hints how I could get the desired data? I was also thinking about implementing the steps described above manually. By I'm stuck in step 2, because I do not have a token to send in the negotiation header (and do also not know where to get it from).
This won't work because the server requires WWW-Authenticate: Kerberos, but curl only talks SPNEGO. Modify your server to request WWW-Authenticate: Negotiate and it will work.
Note: no major browser supports pure Kerberos over HTTP, so don't expect any other library to do so.
On Windows, you can use
library(httr)
GET(url, authenticate(user=":", password="", type="gssnegotiate"), verbose = TRUE)
Or if no proxy is required for an internal website, state no proxy explictly as follows:
library(httr)
GET(url, use_proxy(""), authenticate(user=":", password="", type="gssnegotiate"), verbose = TRUE)
yI'm using the {httr} package to log into an internal web application (Theradoc on IIS7.5) in order to scrape some html (infection) data.
library(httr)
POST("http://ahdc372n2", authenticate("foo", "bar"), encode="multipart"), verbose())
The verbose console output says,
-> POST /theradoc/login/index.cfm HTTP/1.1
-> Authorization: Basic Y2xpbmludGVsbDowMWRFbmdsaXNo
-> User-Agent: curl/7.19.6 Rcurl/1.95.4.3 httr/0.4
-> Host: ahdc372n2.phs-sfalls.amck.net
-> Accept: */*
-> Accept-Encoding: gzip
-> Cookie: JSESSIONID=843052421c871dec2ac3a263b136d475a4a6
->
<- HTTP/1.1 411 Length Required
<- Content-Type: text/html; charset=us-ascii
<- Server: Microsoft-HTTPAPI/2.0
<- Date: Mon, 08 Sep 2014 15:53:02 GMT
<- Connection: close
<- Content-Length: 344
<-
* Closing connection #0
And ultimately I get an ">HTTP Error 411. The request must be chunked or have a content length."
I've reviewed this older post without and useful pointers.
Is there a way to force the Content-Length in the httr POST request?
UPDATE : Manually installing httr_0.5 from the zip archive seems to have solved the problem. Thank you hrbmstr.