I am trying to get data from an API with a POST request. The request works well with a direct shell command :
system(sprintf('curl POST -k --tlsv1.2 -v "https://api-gateway.inpi.fr/services/apidiffusion/api/marques/search" -H "X-XSRF-TOKEN: %s" -H \'accept: application/xml\' -H "Content-Type: application/json" -H "Cookie: XSRF-TOKEN=%s; access_token=%s; session_token=%s" -d \'%s\' > test.xml',token,token,access_token,refresh_token,json_request))
However, I would like to use httr for many reasons. I use the following code :
test <- httr::POST(
"https://api-gateway.inpi.fr/services/apidiffusion/api/marques/search",
httr::set_config(config(ssl_verifypeer = 0L)),
config = (add_headers(
"X-XSRF-TOKEN" = token,
"accept" = "application/xml",
"Content-Type" = "application/json",
"Cookie" = sprintf("XSRF-TOKEN=%s; access_token=%s; session_token=%s",token,access_token,refresh_token)
))
,set_cookies(`X-XSRF-TOKEN` = token,
`XSRF-TOKEN` = token,
access_token = access_token,
session_token = refresh_token)
,body = json_request
)
But this returns a 403 Forbidden error (my_token being the token I use) :
$error
[1] "access_denied"
$error_description
[1] "Invalid CSRF Token '*my_token*' was found on the request parameter '_csrf' or header 'X-XSRF-TOKEN'.
It seems like httr did not take into account my cookies because the token is different inside the test object I create :
> test2$cookies
domain flag path secure expiration name value
1 api-gateway.inpi.fr FALSE / FALSE <NA> XSRF-TOKEN *another_token*
Any idea ? I am sorry that I can't create a reproducible example for obvious security reasons.
Thank you !
The solution was wierd.
I had to rid off from httr, I used UNIX system commands instead, and it worked with the same request.
system(sprintf('curl POST -k --tlsv1.2 "https://api-gateway.inpi.fr/services/apidiffusion/api/marques/search" -H "X-XSRF-TOKEN: %s" -H \'accept: application/json\' -H "Content-Type: application/json" -H "Cookie: XSRF-TOKEN=%s; access_token=%s; session_token=%s" -d \'%s\' > %s/res.json',tokens$xsrf_token,tokens$xsrf_token,tokens$access_token,tokens$refresh_token,json_request,tempdir()))
It seems like httr tries to handle cookies by its own, so maybe that's what caused my problem.
Related
I want to create an R script to use instead of a curl request.
Using curl, I use a Google API which requires the --data option. I am not sure how to add this option to my curl request using the curl package.
The request that works using Git Bash is as follows:
API_KEY="[xyz]"
curl "https://chromeuxreport.googleapis.com/v1/records:queryHistoryRecord?key=$API_KEY" \
--header 'Content-Type: application/json' \
–-data '{"url": "https://www.example.com/"}'
This is my R code, which doesn't send the necessary URL to return anything good:
library(curl)
API_KEY = "xyz"
h <- new_handle()
handle_setheaders(h,
"Content-Type" = "application/json"
)
r <- curl_fetch_memory(url = paste0("https://chromeuxreport.googleapis.com/v1/records:queryHistoryRecord?key=",API_KEY), h)
So how can I add the --data option using the curl package?
You can use the lower-level curl if you really want to. In this case your original CURL command is performing a POST action with a JSON encoded body. The equivalent of that in the R curl library would be
library(curl)
API_KEY <- "xyz"
url <- paste0("https://chromeuxreport.googleapis.com/v1/records:queryHistoryRecord?key=", API_KEY)
payload <- charToRaw('{"url": "https://www.example.com/"}')
h <- new_handle()
handle_setopt(h, post=TRUE, postfields=payload)
handle_setheaders(h, "Content-Type" = "application/json")
r <- curl_fetch_memory(url = url, h)
cat(rawToChar(r$content))
I am trying to access the Amadeus travel API
To obtain a token, the given curl is:
curl "https://test.api.amadeus.com/v1/security/oauth2/token" \
-H "Content-Type: application/x-www-form-urlencoded" \
-d "grant_type=client_credentials&client_id={client_id}&client_secret={client_secret}"
My RScript Attempt is:
library("httr")
# Get Token
response <- POST("https://test.api.amadeus.com/v1/security/oauth2/token",
add_headers("Content-Type" = "application/x-www-form-urlencoded"),
body = list(
"grant_type" = "client_credentials",
"client_id" = API_KEY,
"client_secret" = API_SECRET),
encode = "json")
response
rsp_content <- content(response, as = "parsed", type = "application/json")
rsp_content
Resulting in the error:
Response [https://test.api.amadeus.com/v1/security/oauth2/token]
Date: 2021-07-23 00:59
Status: 400
Content-Type: application/json
Size: 217 B
{
"error":"invalid_request",
"error_description": "Mandatory grant_type form parameter missing",
"code": 38187,
"title": "Invalid parameters"
}
>
What is the correct way to call this API to obtain a token using R?
The curl -d option is used to send data in the same way an HTML form would. To match that format, use encode="form" rather than encode="json" in the call to POST().
How to resolve submission error:
curl: (6) Couldn't resolve host 'www-origin.coursera.org'
m = 15
error] submission with curl() was not successful
!! Submission failed: Grader sent no response
Function: submitWithConfiguration>validateResponse
FileName: C:\Users\admin\Desktop\ex7\lib\submitWithConfiguration.m
LineNumber: 158
This error is probably caused because your computer is not able to connect to the internet at the moment.
This is the function working behind collecting the data from the online grader.
function response = validateResponse(resp)
% test if the response is json or an HTML page
isJson = length(resp) > 0 && resp(1) == '{';
isHtml = findstr(lower(resp), '<html');
if (isJson)
response = resp;
elseif (isHtml)
% the response is html, so its probably an error message
printHTMLContents(resp);
error('Grader response is an HTML message');
else
error('Grader sent no response');
end
end
Now the statement: "Grader sent no response" is printed when the response is null.
And the response can be null when the computer is not connected.
Hope this is the reason behind your error, if not then let me know.
Matlab?
This steps may helpful:
open file: submitWithConfiguration.m ; and: goto line 131 and 134;
then change:
line131: json_command = sprintf('echo jsonBody=%s | curl -k -X POST -d #- %s', body, url);
line134: json_command = sprintf('echo ''jsonBody=%s'' | curl -k -X POST -d #- %s', body, url);
to:
line131: json_command = sprintf('echo jsonBody=%s | curl -k -X POST -s -d #- %s', body, url);
line134: json_command = sprintf('echo ''jsonBody=%s'' | curl -k -X POST -s -d #- %s', body, url);
(both add -s)
it looks like the code is fine and the problem is that your computer can't connect to the internet.
You can simply solve this by using a VPN.
Good luck!
What is the correct way of writing this Curl POST in R?
I would like to have R read the contents of a file as "values" in the post form.
curl -X POST https://api.priceapi.com/jobs \
-d "token=token" \
-d "country=country" \
-d "source=source" \
-d "currentness=currentness" \
-d "completeness=completeness" \
-d "key=key" \
-d 'values=<values>'
So far I have this-
library(RCurl)
library(RJSONIO)
url = "https://api.priceapi.com/jobs"
file.name = ".../output 1 .txt"
results = postForm(url, token="token",
country="country,
source="source",
currentness="currentness",
completeness="completeness,
key="key",
values=fileUpload(filename = file.name))
It returns "Error: Bad Request"
I also tried it using httr post request-
r = POST(url, body = list(token="token",
country="country,
source="source",
currentness="currentness",
completeness="completeness,
key="key",
values=upload_file(file.name)) )
Here upload_file is not uploading the contents of the file but, I am guessing it is passing the path to the file (as a string) into the "values" parmeter.
Naturally that does not return the correct results.
The result of the httr POST request is;
Response [https://api.priceapi.com/jobs]
Date: 2016-12-13 10:11
Status: 400
Content-Type: application/json; charset=utf-8
Size: 228 B
{
"success": false,
"reason": "parameter value invalid",
"parameter": "value",
"valid values": "An array or a string containing values separated by newline",
"comment": "Make sure the parameter 'value' has a valid value!"
I could solve this by using
file=readLines(".../output 1.txt")
inputValues <- paste(file,collapse="\n")
and then passing inputValues in the values parameter.
I would like to move the following curl call to Rcurl:
curl 'http://myserver.org/stream'
-H 'Authorization: Basic XXXXXXXX' -H 'Connection: keep-alive' --data-binary '{"limit": 20}'
-H 'Content-Type: application/json;charset=UTF-8'
This is one of my R tests:
library(RCurl)
url.opts <- list(httpheader = list(Authorization ="Basic XXXXXXXX",
Connection = "keep-alive",
"Content-Type" = "application/json;charset=UTF-8"))
getURLContent('http://myserver.org/stream', .opts=url.opts)
Now I am missing an attribute to add --data or --data-binary. How to add this option?
Thanks a lot
Markus
Thanks a lot #hadley.
This is my working solution:
library(httr)
user <- "test"
password <- "test123"
POST(url = 'http://myserver.org/stream',
config = c(authenticate(user, password, "basic"),
add_headers(Connection = "keep-alive"),
accept_json()),
body = "{'username':'test'}")