I have this curl command that I can call in bash
curl -X POST -H 'Content-Type: text/csv' --data-binary #data/data.csv https://some.url.com/invocations > data/churn_scored.jsonl
which posts a CSV file to the API endpoint and I redirect the results into a .jsonl file.
I can't find where I can specify a data file to POST to the endpoint like using curl's #.
What's the way to achieve the CURL post using R's curl package (or any other package)? The redirection of output I can figure out in another way.
This is a very helpful site in converting curl commands to other languages: https://curl.trillworks.com/#r
When plugging in your curl command there I got this.
require(httr)
headers = c(
`Content-Type` = 'text/csv'
)
data = upload_file('data/data.csv')
res <- httr::POST(url = 'https://some.url.com/invocations', httr::add_headers(.headers=headers), body = data)
To specific symbol #. From man curl:
--data-binary <data>
(HTTP) This posts data exactly as specified with no extra processing whatsoever.
If you start the data with the letter #, the rest should be a filename. Data is
posted in a similar manner as --data-ascii does, except that newlines are preserved
and conversions are never done.
If this option is used several times, the ones following the first will append data
as described in -d, --data.
It seems no need to worry about #
As it mentioned by #chinsoon12, httr is a good nice method to handle request:
-X or --request translates to VERB function POST(), which includes --data-binary
-H or --header translates to add_headers() but there are special functions for setting content type (see below)
So it looks like:
library(httr)
response <- POST(
url = "https://some.url.com/invocations",
body = upload_file(
path = path.expand("data/data.csv"),
type = 'text/csv'),
verbose()
)
# get response and write to you disk
Related
I used Plumber (https://github.com/trestletech/plumber) to create a web API for a R source code, which purpose is to process a json_string as an input, and converting it into a R dataframe.
R code:
#* #param data
#* #post /predict
jsonToDF <- function(data) {
library(jsonlite)
dat <- fromJSON(data)
}
As you can see from the code above, I'm using the library jsonlite and the fromJSON() method.
Then I exposed the service locally via:
library(plumber)
r <- plumb("code.R")
r$run(port=54321)
I used the curl request:
curl -v -i -X POST -H "Content-Type: application/json" --data "testData='{\"a\":\"b\"}'" http://localhost:54321/predict
to send a json_string ({"a":"b"}) but this is the error i got:
<simpleError: lexical error: invalid char in json text.
'{"a":"b"}'
(right here) ------^
I used R console to double check the validity of the input, and i did not get the same error message.
> z <- fromJSON('{"a":"b"}')
> z
$a
[1] "b"
Can anyone explain to me why it works in R console, but not as an input in the data field of the curl requests? Thank you very much!
I managed to get the desired input by changing "data='{\"a\":\"b\"}'" to "data={\"a\":\"b\"}". I simply removed the single quotation marks from the --data field of the curl command.
I think R automatically treats {\"a\":\"b\"} as a string after receiving it from curl, so there was no need for me to explicitly declare it as '{\"a\":\"b\"}' when sending it over curl.
I am trying to execute a CURL call by passing parameters.
However, it does not return the intended output.
Here is the code:
curl -X GET 'http://admin-app.prod.order-services.cp.glb.prod.walmart.com/order-services-admin/plutus-admin/published-transactions/count?fromDate=$from_date&toDate=$to_date'
Please let me know how I can pass $from_date and $to_date in the above URL.
I think you want to substitute the values of variables $from_date and $to_date in the link. Variables are not substituted in single quoted strings. Try replacing single quotes with double quotes:
curl -X GET "http://admin-app.prod.order-services.cp.glb.prod.walmart.com/order-services-admin/plutus-admin/published-transactions/count?fromDate=$from_date&toDate=$to_date"
I'm trying to use Postman for upload a WKT layer by the cURL code below.
curl -X POST
'https://gfe.cit.api.here.com/2/layers/upload.json?layer_id=123&app_id={APP_ID}&app_code={APP_CODE}'
-H 'content-type: multipart/form-data; boundary=----WebKitFormBoundary7MA4YWxkTrZu0gW'
-F 'zipfile=#C:\xampp\htdocs\here\southern.wkt.zip'`
but the response message is
{"issues":[{"message":"Multipart should contain exactly one part"}],"error_id":"2e1e3994-69d6-43bb-8224-2a869b5255ae","response_code":"400 Bad Request"}
Am I doing something wrong?
Make sure your WKT file doesn't contain additional indentation(spaces) avoid giving spaces and instead use equal tabs and give proper column names
refer this
I am using the Twitter REST API to retrieve data in JSON format. Twitter's developer page makes it easy by providing a command that can be pasted directly into the terminal and executed. The following command works in the terminal.
curl --get 'https://api.twitter.com/1.1/statuses/user_timeline.json' --data 'count=3200&screen_name=BernieSanders' --header 'Authorization: OAuth oauth_consumer_key="####", oauth_nonce="####", oauth_signature="####", oauth_signature_method="HMAC-SHA1", oauth_timestamp="1467719924", oauth_token="####", oauth_version="1.0"' --verbose
I am trying to get the JSON data into R, and would like to execute this same command in the R console. I have tried curlconverter using the exact same code, get an error that there is an unexpected symbol. However, the code is exactly the same. Is there a more suitable package for executing this code?
curlExample <- "curl --get 'https://api.twitter.com/1.1/statuses/user_timeline.json' --data 'count=3200&screen_name=BernieSanders' --header 'Authorization: OAuth oauth_consumer_key="####", oauth_nonce="####", oauth_signature="####", oauth_signature_method="HMAC-SHA1", oauth_timestamp="1467719924", oauth_token="####", oauth_version="1.0"' --verbose"
I think you are going to need to escape those quotation marks. Try this instead:
curlExample <- "curl --get 'https://api.twitter.com/1.1/statuses/user_timeline.json' --data 'count=3200&screen_name=BernieSanders' --header 'Authorization: OAuth oauth_consumer_key=\"####\", oauth_nonce=\"####\", oauth_signature=\"####\", oauth_signature_method=\"HMAC-SHA1\", oauth_timestamp=\"1467719924\", oauth_token=\"####\", oauth_version=\"1.0\"' --verbose"
That the quotes were not being escaped properly was actually evident in your question, because the Stack Overflow markup rendered the curlExample string in multiple colors.
I believe https://cran.r-project.org/web/packages/twitteR/twitteR.pdf is the way to do this in R.
But specifically in your case, you have unescaped " in your string.
I successfully downloaded a file from a remote server using cURL and HTTP, but the file includes all the HTML code.
Is there a function in cURL so that I can extract the values I want?
For example, I am getting:
...
<body>
Hello,Manu
</body>
...
But I only want Hello,Manu.
Thanks in advance,
Manu
try using DOMDocument or any other XML parser.
$doc= new DOMDocument();
$doc->loadHTML($html_content); // result from curl
$xpath= new DOMXPath($doc);
echo $xpath->query('//body')->item(0)->nodeValue;
alternatively for command line you can use
curl 'http://.................' | xpath '//body'