I'm trying to make a GET call in R using httr and I keep getting an authorization 401 error.
R code:
testfunction2 <- function()
{
set_config(verbose())
locus_url <- "https://api.locusenergy.com/v3/clients/5599"
r <- GET(url = "https://api.locusenergy.com/v3/clients/5599",
query=list(authorization="Bearer c935845d8fc1124757e66ce04d2c75d0"),
Accept="application/json")
}
The results:
> print(testfunction2())
-> GET /v3/clients/5599
authorization=Bearer%20c935845d8fc1124757e66ce04d2c75d0 HTTP/1.1
-> User-Agent: libcurl/7.39.0 r-curl/0.9.1 httr/1.0.0
-> Host: api.locusenergy.com
-> Accept-Encoding: gzip, deflate
-> Cookie: AWSELB=D91FBFE1087EF6EBC125A126777051237474A8A060B6095B8E3C16151308453F8556B2A2E90CB2178F365FAA8AA8C29B124D15CA3EB859CFE615428E8D55C393ABB5B436BF
-> Accept: application/json, text/xml, application/xml, */*
->
<- HTTP/1.1 401 Unauthorized
<- Content-Type: application/json
<- Date: Sun, 16 Aug 2015 05:02:27 GMT
<- Server: Apache-Coyote/1.1
<- transfer-encoding: chunked
<- Connection: keep-alive
<-
I expect it to return a 200 code (rather than 401, that implies authorization error.)
I know the token is correct because it works if I use the Postman (google add-in) and Python. The token won't work for you because I changed it since I can't share it.
Python Code:
import http.client
conn = http.client.HTTPSConnection("api.locusenergy.com")
headers = {
'authorization': "Bearer 935845d8fc1124757e66ce04d2c75d0"
}
conn.request("GET", "/v3/clients/5599", headers=headers)
res = conn.getresponse()
data = res.read()
print(data)
results from Python
b'{"statusCode":200,"partnerId":4202,"tz":"US/Arizona","firstName":"xxx","lastName":"xxxx","email":"xxxx#aol.com","id":5599}'
So, again the question is what am I doing wrong in R or can you give me any hints? This won't be reproducible for you because the token expired and I can't share it.
Could it be the space in the authorization? authorization="Bearer 935845d8fc1124757e66ce04d2c75d0"? Are there any hints in the verbose output of the get call in R?
For reference, this is the site's API page:
https://developer.locusenergy.com/
The site requires OAUTH2 authentication to return the token. I didn't include that code but I verified the token works with Python and Postman.
Right now you are passing your authorization values in the query string of the httr code and not in the http header as you are doing in the python code. Instead use
GET(url = "https://api.locusenergy.com/v3/clients/5599",
accept_json(),
add_headers(Authorization="Bearer c935845d8fc1124757e66ce04d2c75d0")
)
Related
How to collect the HTTP response status for a script?
Below is a sample code which will start a server and allow two routes for interaction.
using Genie
import Genie.Router: route
import Genie.Renderer.Json: json
Genie.config.run_as_server = true
route("/try/", method=GET) do
(:message => "Welcome") |> json
end
route("/test/", method=POST) do
data = jsonpayload()
<body>
end
Genie.startup()
How to collect the response status like 200, 500 or others as a string variable?
Open connection to your server using HTTP and look for the status field:
julia> using HTTP
julia> response = HTTP.get("http://127.0.0.1:8000/try")
HTTP.Messages.Response:
"""
HTTP/1.1 200 OK
Content-Type: application/json; charset=utf-8
Server: Genie/1.18.1/Julia/1.6.1
Transfer-Encoding: chunked
{"message":"Welcome"}"""
julia> response.status
200
If you rather want to control the status yourself you can add on the server side:
route("/tryerror/", method=GET) do
Genie.Responses.setstatus(503)
end
And now let us test it for 503:
julia> response = HTTP.get("http://127.0.0.1:8000/tryerror")
ERROR: HTTP.ExceptionRequest.StatusError(503, "GET", "/tryerror", HTTP.Messages.Response:
"""
HTTP/1.1 503 Service Unavailable
Content-Type:
Server: Genie/1.18.1/Julia/1.6.1
Transfer-Encoding: chunked
""")
I'm trying to access JSON data (in R) from a REST API.
To authenticate myself, I need to use a POST method in https://dashboard.server.eu/login. The data that needs to be sent are email and password:
library(httr)
login <- list(
email = "my#email.com",
password = "mypass"
)
res <- POST("https://dashboard.server.eu/login", body = login, encode = "form", verbose())
When executing the above, I get this output:
-> POST /login HTTP/1.1
-> Host: dashboard.server.eu
-> User-Agent: libcurl/7.59.0 r-curl/3.3 httr/1.4.1
-> Accept-Encoding: gzip, deflate
-> Cookie: session=10kq9qv1udf0107F4C70RY14fsum41sq50
-> Accept: application/json, text/xml, application/xml, */*
-> Content-Type: application/x-www-form-urlencoded
-> Content-Length: 53
->
>> email=my%40email.com&password=mypass
<- HTTP/1.1 200 OK
<- access-control-allow-headers: Accept, Authorization, Content-Type, If-None-Match
<- access-control-allow-methods: HEAD, GET, POST, PUT, DELETE
<- cache-control: no-cache
<- content-encoding: gzip
<- content-type: application/json; charset=utf-8
<- date: Mon, 09 Mar 2020 14:58:31 GMT
<- set-cookie: session=10kq9qv1udf0107F4C70RY14fsum41sq50; HttpOnly; SameSite=Strict; Path=/
<- vary: origin,accept-encoding
<- x-microserv: NS4yNi4xODQuMjE3
<- x-poweredby: Poetry
<- Content-Length: 2346
<- Connection: keep-alive
The doc of the site says that, in case of success, a JSON res is returned and contains a string token in res.data._id.
I don't find it... even looking at every list (and sub-lists) of res.
How am I supposed to find the token?
Following the doc, and an example in AngularJS, I'm then supposed to do:
// Create JSON Object with your token
let authorizeObject = {
'Authorization': 'Session ' + token,
'content-type': 'application/json;charset=UTF-8',
'accept': 'application/json,text/plain',
};
// Create header from the previous JSON Object
let header = {'headers':authorizeObject};
// Use the header in your http request...
$http.get('https://dashboard.server.eu/', header)
Any hint on making this dream become true?
UPDATE -- With cURL, I could check that there is a _id key/value returned…
With the command:
curl -k -X POST "https://dashboard.server.eu/login" \
-d '{ "email" : "my#email.com", "password" : "mypass" }' \
-H "Content-Type: application/json"
I get the output:
{
"_id": "697v2on4ll0107F4C70RYhosfgtmhfug",
"isAuthenticated": true,
"user": {
"_id": "5dd57868d83cfc000ebbb273",
"firstName": "me",
"lastName": "Me",
...
So, the session token is indeed somewhere...
Does this help to help me?
Looking at the image of res in your question, the message is there, under content - it's just that the content is stored as a vector of raw bytes, which is why you didn't recognise it as json.
Since any file type can be sent by http, the contents in an httr response object are stored in raw format rather than a character string for various reasons - perhaps most importantly because many binary files will contain a 0x00 byte, which isn't allowed in a character string in R.
In your case, we can not only tell that res$content is text, but that it is your "missing" json. The first six bytes of res$content are shown in your image, and are 7b, 22, 5f, 69, 64, 22. We can convert these to a character string in R by doing:
rawToChar(as.raw(c(0x7b, 0x22, 0x5f, 0x69, 0x64, 0x22)))
[1] "{\"_id\""
This matches the first six characters of your expected json string.
Therefore if you do:
httr::content(res, "text")
or
rawToChar(res$content)
You will get your json as a character string.
Context
I want to access an API from R and have the correct credentials to do so.
The issue is that Authentication requires three arguments to be passed (Username, Password and Organisation).
I am not familiar with API calls from R and I do not know how to pass this "organisation" argument.
Example
An example of the API call to request a token is as follows (taken from the documentation):
curl -X POST "https://URL.com" \
-H "Content-Type: application/json" \
--data "{\"user_name\": \"<username>\" \
\"password\": \"<mypassword>\",\
\"organisation\": \"<organisation>\"}"
Using the crul package, I have tried:
require("crul")
z <- crul::HttpClient$new(url = "https://THEURL.com",
auth(user = "myuser",
pwd = "mypwd"))
z$get()
Which returns:
<crul response>
url: https://THEURL.com
request_headers:
User-Agent: libcurl/7.59.0 r-curl/3.3 crul/0.8.0
Accept-Encoding: gzip, deflate
Accept: application/json, text/xml, application/xml, */*
response_headers:
status: HTTP/1.1 403 Forbidden
content-type: application/json
content-length: 211
connection: keep-alive
x-amzn-errortype: IncompleteSignatureException
Im assuming that x-amzn-errortype is referring to the missing organisation authentication variable.
Questions
How can I pass the variable "Organisation" to the function?
Or is there a better way to do this?
I think it will be way easier for you to use the package httr when you deal with HTTP requests. I think your example request could be:
library(httr)
POST("https://THEURL.com",
body = list(user_name = "<username>", password = "<mypassword>", organisation = "<organisation>"),
encode = "json")
I'm just trying to get the response body when the post request fails using RCurl. When I'm running it through R, I only get the response error and it fails out.
Error: Unprocessable Entity
when I post the same request with the UI interface for testing, it gives
{
"reason": [
"Can not create Data with Name: DataTest. Data Name should be unique."
],
"singleReason": "Can not create Data with Name: DataTest. Data Name should be unique."
}
Any help on how to get response bodies on 4XX errors is appreciated. TIA.
Post request
postdata.json <- '{"name":"DataTest","description":"Test Payload","algorithm":{"name":"DataTest","version":"0.1.0"}}'
post.result <- httpPOST(url=SERVER, postfields = postdata.json, verbose = T,
httpheader=c(Authorization=access.token, 'Content-Type'='application/json', Accept='application/json'))
RStudio output
* About to connect() to SERVER port 80 (#0)
* Trying SERVER... * connected
* Connected to SERVER port 80 (#0)
> POST /api/test HTTP/1.1
Host: SERVER
Authorization: AUTHENTICATION
Content-Type: application/json
Accept: application/json
Content-Length: 171
< HTTP/1.1 422 Unprocessable Entity
< Content-Type: application/json;charset=UTF-8
< Date: Thu, 13 Nov 2014 16:31:42 GMT
< Server: Apache-Coyote/1.1
< Content-Length: 215
< Connection: keep-alive
<
* Connection #0 to host SERVER left intact
Show Traceback
Rerun with Debug
Error: Unprocessable Entity
n.b. I'm okay with it failing and returning an Error. That's expected. I'm just trying to get the response body associated with the error code.
Here's one approach with httr:
library(httr)
postdata.json <- '{"name":"DataTest","description":"Test Payload","algorithm":{"name":"DataTest","version":"0.1.0"}}'
res <- POST(SERVER,
body = postdata.json,
add_headers(
Authorization = access.token,
'Content-Type' = 'application/json',
Accept = 'application/json'
)
)
http_status(res)
content(res)
I was able to get the response body using RCurl by adding a basicTextGatherer() and a writefunction.
reader <- basicTextGatherer()
post.result <- httpPOST(url=SERVER, postfields = postdata.json, verbose = T,
writefunction = reader$update,
httpheader=c(Authorization=access.token,
'Content-Type'='application/json',
Accept='application/json')
)
return(body=reader$value())
but Hadley's solution is simpler to work with and doesn't error out.
In Windows cURL I can post a web request similar to this:
curl --dump-header cook.txt ^
--data "RURL=http=//www.example.com/r&user=bob&password=hello" ^
--user-agent "Mozilla/5.0" ^
http://www.example.com/login
With type cook.txt I get a response similar to this:
HTTP/1.1 302 Found
Date: Thu, ******
Server: Microsoft-IIS/6.0
SERVER: ******
X-Powered-By: ASP.NET
X-AspNet-Version: 1.1.4322
Location: ******
Set-Cookie: Cookie1=; domain=******; expires=****** ******
******
******
Cache-Control: private
Content-Type: text/html; charset=iso-8859-1
Content-Length: 189
I can manually read cookie lines like: Set-Cookie: AuthCode=ABC... (I could script this of course). So I can use AuthCode for subsequent requests.
I am trying do the same in R with RCurl and/or httr (still don't know which one is better for my task).
When I try:
library(httr)
POST("http://www.example.com/login",
body= list(RURL="http=//www.example.com/r",
user="bob", password="hello"),
user_agent("Mozilla/5.0"))
I get a response similar to this:
Response [http://www.example.com/error]
Status: 411
Content-type: text/html
<h1>Length Required</h1>
By and large I know about 411-error and I could try to fix the request; but I do not get it in cURL, so I am doing something wrong with the POST command.
Can you help me in translating my cURL command to RCurl and/or httr?
httr automatically preserves cookies across calls to the same site, as illustrated by these two calls to http://httpbin.org
GET("http://httpbin.org/cookies/set?a=1")
# Response [http://httpbin.org/cookies]
# Status: 200
# Content-type: application/json
# {
# "cookies": {
# "a": "1"
# }
# }
GET("http://httpbin.org/cookies")
# Response [http://httpbin.org/cookies]
# Status: 200
# Content-type: application/json
# {
# "cookies": {
# "a": "1"
# }
# }
Perhaps the problem is that you're sending your data as application/x-www-form-urlencoded, but the default in httr is multipart/form-data, so use multipart = FALSE in your POST call.
Based on Juba suggestion, here is a working RCurl template.
The code emulates a browser behaviour, as it:
retrieves cookies on a login screen and
reuses them on the following page requests containing the actual data.
### RCurl login and browse private pages ###
library("RCurl")
loginurl ="http=//www.*****"
mainurl ="http=//www.*****"
agent ="Mozilla/5.0"
#User account data and other login pars
pars=list(
RURL="http=//www.*****",
Username="*****",
Password="*****"
)
#RCurl pars
curl = getCurlHandle()
curlSetOpt(cookiejar="cookiesk.txt", useragent = agent, followlocation = TRUE, curl=curl)
#or simply
#curlSetOpt(cookiejar="", useragent = agent, followlocation = TRUE, curl=curl)
#post login form
web=postForm(loginurl, .params = pars, curl=curl)
#go to main url with real data
web=getURL(mainurl, curl=curl)
#parse/print content of web
#..... etc. etc.
#This has the side effect of saving cookie data to the cookiejar file
rm(curl)
gc()
Here is a way to create a post request, keep and reuse the resulting cookies with RCurl, for example to get web pages when authentication is required :
library(RCurl)
curl <- getCurlHandle()
curlSetOpt(cookiejar="/tmp/cookies.txt", curl=curl)
postForm("http://example.com/login", login="mylogin", passwd="mypasswd", curl=curl)
getURL("http://example.com/anotherpage", curl=curl)