The webpage I am trying to access downloads an attachment but header status is always 500 which I have checked through postman.
I am using download.file with wget method in R to download that file but it stops and doesn't download since the header status is 500. How can I manage to download this file irrespective of the header status?
When I access this link in browser, the file is downloaded just fine.
Edit:
Here's how I use download.file function in R:
download.file(get(url), destfile=tmpFile, method = "wget");
I tried to pass the extra argument like:
download.file(get(url), destfile=tmpFile, method = "wget", extra =getOption("content-on-error"));
and
download.file(get(url), destfile=tmpFile, method = "wget", extra =getOption("--content-on-error=0"));
but it doesn't work
Here :
get(url) points to the link:(have replaced actual key with KEY)
https://app.adroll.com/api/v1/export/all_campaigns_report?advertisable=__KEY__&reports=AllCampaignsSummary,AllCampaignsChart,AllAds,AllCampaignsSites&start_date=01-01-2013&end_date=10-05-2016&format=csv¤cy=USD
message on trying to connect:
Resolving app.adroll.com... 52.11.56.178, 52.89.249.63
Connecting to app.adroll.com|52.11.56.178|:443... connected.
HTTP request sent, awaiting response... 500 Internal Server Error
2016-10-06 16:22:21 ERROR 500: Internal Server Error.
Related
I can access API data using "curl" on terminal but get error when i use R. Looking for some advice
Curl command:
curl -H "token:jQyrbzexeCEaWFIDBAwCWqbrkrVQTVhM" "https://www.ncdc.noaa.gov/cdo-web/api/v2/datasets"
R command:
books_key <- "&token=jQyrbzexeCEaWFIDBAwCWqbrkrVQTVhM"
url <- "https://www.ncdc.noaa.gov/cdo-web/api/v2/datasets"
req <- fromJSON(paste0(url, books_key))
Error in open.connection(con, "rb") : HTTP error 400.
Usually this error comes when there are spaces in URL (from other similar questions) but in my case there is no space in URL
Some info on using token from website https://www.ncdc.noaa.gov/cdo-web/webservices/v2#gettingStarted
Not a R issue because following example of another website works
movie_key <- "&api-key=b75da00e12d54774a2d362adddcc9bef"
url <- "http://api.nytimes.com/svc/movies/v2/reviews/dvd-picks.json?order=by-date"
req <- fromJSON(paste0(url, movie_key))
You're supposed to pass the token as a header not a query param, e.g. with the crul pkg
cli <- crul::HttpClient$new(
url = "https://www.ncdc.noaa.gov",
headers = list(token = "yourtoken"))
cli$get(path = "cdo-web/api/v2/datasets")
A quick look at your Curl and your R call - in Curl your passing the token as a header variable and in the R call it appears to be a query variable.
Also the NYT call has a query param in the URI so & would be appropriate in the R call.
The webpage I am trying to access downloads an attachment but header status is always 500 which I have checked through postman.
I am using download.file with wget method in R to download that file but it stops and doesn't download since the header status is 500. How can I manage to download this file irrespective of the header status?
When I access this link in browser, the file is downloaded just fine.
Edit:
Here's how I use download.file function in R:
download.file(get(url), destfile=tmpFile, method = "wget");
I tried to pass the extra argument like:
download.file(get(url), destfile=tmpFile, method = "wget", extra =getOption("content-on-error"));
and
download.file(get(url), destfile=tmpFile, method = "wget", extra =getOption("--content-on-error=0"));
but it doesn't work
Here :
get(url) points to the link:
https://app.adroll.com/api/v1/export/all_campaigns_report?advertisable=__KEY__&reports=AllCampaignsSummary,AllCampaignsChart,AllAds,AllCampaignsSites&start_date=01-01-2013&end_date=10-05-2016&format=csv¤cy=USD
message on trying to connect:
Resolving app.adroll.com... 52.11.56.178, 52.89.249.63
Connecting to app.adroll.com|52.11.56.178|:443... connected.
HTTP request sent, awaiting response... 500 Internal Server Error
2016-10-06 16:22:21 ERROR 500: Internal Server Error.
Hi I'm trying to write a script using R where I post data to a specific weblink and get the response back. I'm not really sure how to set curl headers for Content-type and Application.
this is what I have so far:
library("RCurl")
httpPOST("http://localhost", '{"title":"ello World!"}')
I get the following error:
Error in curlOptions(..., .opts = .opts) : unnamed curl option(s):
....
If use curl from command line the data does get posted.
Is there another library in R that does CURL and JSON posts better?
I am trying to use the getURL function of RCurl Package in order to access an ASP Webpage as:
my_url <- "http://www.my_site.org/my_site/main.asp?ID=11&REFID=33"
webpage <- getURL(my_url)
but I get an Object Moved redirection error message like:
"<head><title>Object moved</title></head>\n<body><h1>Object Moved</h1>
This object may be found here.</body>\n"
I followed various suggestions like using the curlEscape URL encoding function or by setting the CURLOPT_FOLLOWLOCATION and CCURLOPT_SSL_VERIFYHOST Parameters via the curlSetOpt Function as listed in the php ssl curl : object moved error link, but the later 2 were not recognized as valid RCurl options.
Any suggestions how to overcome the issue?
Use the followlocation curl option:
getURL(u,.opts=curlOptions(followlocation=TRUE))
with added cookiefile goodness - its supposed to be a file that doesnt exist, but I'm not sure how you can be sure of that:
w=getURL(u,.opts=curlOptions(followlocation=TRUE,cookiefile="nosuchfile"))
when ever i m trying to run the script i got an error of "wsdl error: HTTP ERROR: no data present after HTTP headers" can any bdy help me regarding it?
This means that while the response contains headers (for example, a HTTP status code and content type) there is no data - i.e. no XML in the file.