I am a bit puzzled because I can not get this to work from within R.
I want to execute the following Curl request from within R using RCurl:
curl -T 'temp.csv' -H 'Content-Type: text/csv' https://api.crowdflower.c.json?key={APIKEY}
That is, I want to transfer the file "temp.csv" to the server. This curl command itself works really well, however, I would like to send it from within R without having to use
system("curl -T 'temp.csv' -H 'Content-Type: text/csv' https://api.crowdflower.c.json?key={APIKEY}")
I have played around with the fileUpload() function, but it does not work.
Do you have any clues or hints for me how I can get this to work with RCurl? Maybe it is not really designed to be able to handle this as I see that most people use it only to automate forms or to scrape websites using R.
Thanks a lot for your help,
best Thiemo
Related
My command is:
curl -X POST --user "admin:admin" https://website.com/wp-json/wp/v2/posts/ -H "Content-Type: application/json" -d {"title":"test","content":"Content","status":"draft"}
The response I get:
{"code":"rest_invalid_json","message":"Invalid JSON body passed.","data":{"status":400,"json_error_code":4,"json_error_message":"Syntax error"}}
Thanks!
From my earlier comments, I have the feeling that this error is related to the -d possibly needing to have quotes around the data text.
So in your case, it might be worth trying to put single quotes around the -d data text because you are currently using double quotes in the json text.
Thus, your curl command might look like this:
curl -X POST --user "admin:admin" https://website.com/wp-json/wp/v2/posts/ -H "Content-Type: application/json" -d '{"title":"test","content":"Content","status":"draft"}'
This is based on how the curl docs appear to show that quotes should be used with -d related data text based on the example under the "POST (HTTP)" section of the curl documentation here. I think this is what I have done in the past, as well, but I don't have a specific personal example on me at the moment.
Update
For what it's worth, when I tried running this curl type of command on my own, I got the following error when I didn't use single quotes around the json text:
{"code":"rest_invalid_json","message":"Invalid JSON body passed.","data":{"status":400,"json_error_code":4,"json_error_message":"Syntax error"}}curl: (3) Port number ended with 'C' curl: (3) Port number ended with 'd'
When I ran the command with single quotes around the json data, like this: -d '{"title":"test","content":"Content","status":"draft"}', the command worked and a draft was added in WordPress. I also received back a json response that looks related to the new draft post.
Another thing to check might be the credentials used with the --user part. I ended up having to use an "Application Password" that I generated for a user in my WordPress account. This is different than my user's login password. If you haven't already tried using an "Application Password" in WordPress for this part, it might be worth trying to see if that fixes your issue. I found this when I:
Logged into my WordPress website
Clicked on the Users menu item
Clicked on a user
Scrolled down to the bottom of the user info page
Clicked on the Add New Application Password button
Ok, this was a windows specific problem, I guess most people out there are running unix/linux clients so this doesn't apply to them. I found the solution here: https://stackoverflow.com/a/7173011/15161479
The issue is with curl on windows, the quotes need to be escaped. I also did a couple other things like installing the "Application Passwords" plugin.
This is what my command looks like now
curl --user "user:application password" http://website.com/wp-json/wp/v2/posts/ -H "Content-Type: application/json" --data "{"""title""":"""test""","""content""":"""Content""","""status""":"""draft"""}"
Hope this helps some other people out there!
I have a mini program/server built on one of my computers (Machine1) and I am trying to create or overwrite a file through cURL on another computer (Machine2). So Machine2 is connected to Machine1. Ive been looking through cURL's documentation for command that will do this but have had no luck and as well on stack overflow.
https://curl.haxx.se/docs/manpage.html
I have also tried the examples on this SO post:
HTTP POST and GET using cURL in Linux
Any idea as to what the command might be through command prompt? (equivalent of a POST command). I have tried so far using -O, -K, -C and a multitude of others which have not worked.
In command line, all you need to do is using curl --form to simulate a multipart/form-data POST request:
curl --form "testfile=#thefilename.jpg" http://<Machine2>/<Path>
testfile is the field name used for form, if you don't care, just use any english word.
# is used here to make file thefilename.jpg get attached in the post as a file upload. Refer to curl man doc.
In server side, URL http://<Machine2>/<Path> should be listened. When curl send the previous POST request, server side program should get it, extract the attached file (thefilename.jpg), and save to disk.
In slack you can script slackbot to post messages to a channel like this:
curl --data "$msg" $'https://<yourteam>.slack.com/services/hooks/slackbot?token=<yourtoken>&channel=#random'
Now i'd like to mention a username as the first part of the message like msg="#joernhees hello self".
The problem with this is that if the --data argument of curl starts with an # sign it will interpret the string after the # as filename and post its content. Is there a way to make curl ignore the # sign and to send a literal # as the first char of a post request?
If you are on a new version of cURL you can also use the --data-raw option:
http://curl.haxx.se/docs/manpage.html#--data-raw
A word of warning is that looking my laptop it appears Yosemite ships with an older version of cURL.
In general if you're creating tools to post to Slack I'd recommend using an HTTP library in your script rather than calling out to a shell and invoking the curl command.
Actually i just found out i can do this (not sure it's the best option though):
curl --data '#-' $'https://<yourteam>.slack.com/services/hooks/slackbot?token=<yourtoken>&channel=#random' <<< "$msg"
The trick is to tell curl to read from stdin #- and then pass the message in via that.
Consider the following commands (2 examples)
curl -H "Accept:text/plain" http://rxnav.nlm.nih.gov/REST/rxcui/866350/allrelated
curl -i -L -H "Accept: application/xml" http://pub.orcid.org/search/orcid-bio?q=digital-object-ids:"10.1088/0031-9155/58/3/535"
The user is able to run those on win7 command line with curl.exe installed. The RCurl package has no vignette and it has a lot of commands to choose from. How do I get the text file and XML file output using RCurl?
I think you can use the following :
mytxt <- getURL("http://rxnav.nlm.nih.gov/REST/rxcui/866350/allrelated", httpheader=c(Accept="text/plain"))
And :
myxml <- getURL('http://pub.orcid.org/search/orcid-bio?q=digital-object-ids:"10.1088/0031-9155/58/3/535"', httpheader=c(Accept="application/xml"))
You will find many examples in the getURL help page.
The request for our service looks something like this:
GET http://[SERVICE]/Node:[id].Build?format=mime1,mime2,...,mimeN&template-id=[templateid]
Accept: multipart/mixed
Content-Type: application/json
body: json document
I am attempting to use ApacheBench to test benchmark this. Here is the call I am using:
ab -n 10 -c 2 -T 'application/json' -H 'Accept: multipart/mixed' 'http://phx5qa01c-02b0.stratus.phx.qa.ebay.com/.Build?format=text/html,text/plain&template-id=29b1468f-c8c3-db23-2f6f-74e112795540'
This call goes through, and results in an error since the expected json data is not there.Is there a way in ab to supply the necessary json along with this request. I see there are -p and -u commands to specify an input file, but those are for puts and posts.
I realize that this answer is six years late, but I think it is worth posting since I was banging my head against a wall on a very similar issue to this, in which I was trying to load test a URL which returned only JSON data, and my solution might help other readers encountering this issue. My issue was I kept specifying the -H option when I didn't need to. That kept making the server to send back an HTTP 406 response code (Not Acceptable) to my AB request. During most of my my troubleshooting, I had also kept -T 'application/json' in the AB request as well when I didn't need it. That is only used in conjunction with PUTs or POSTs (when using the -p switch). So I removed -H, and -T, and it worked. All that said, I see those two issues here. We need to be mindful that AB uses the GET method by default.
You are constraining AB by appending extra custom headers to the
request, by using the -H option: -H 'Accept: multipart/mixed', which
might make your target server think it's an invalid request and stop the
sequence right then and there. Just don't use -H unless you have a
really good reason why.
You are using the -T option: -T 'application/json' which
only works when you specify that in conjunction with -p and you
don't have -p anywhere in your command, which you don't want to use anyway since you are sending a GET and not a PUT or a POST.
So to fix this, simply remove both the -T and -H options, and it should work. Reminder to other readers: If on Windows, enclose the URL in double-quotes whenever it contains special characters like a "&", or a "?", as in this case.
ab -n 10 -c 2 'http://phx5qa01c-02b0.stratus.phx.qa.ebay.com/.Build?format=text/html,text/plain&template-id=29b1468f-c8c3-db23-2f6f-74e112795540'