RCurl JSON data to JIRA REST add issue - r

I'm trying to POST data to JIRA Project using R and I keep getting: Error Bad Request. At first I thought it must be the JSON format that I created. So I wrote the JSON to file and did a curl command from console (see below) and the POST worked just fine.
curl -D- -u fred:fred -X POST -d #sample.json -H "Content-Type: application/json" http://localhost:8090/rest/api/2/issue/
Which brings the issue to my R code. Can someone tell me what am I doing wrong with the RCurl postForm?
Source:
library(RJSONIO)
library(RCurl)
x <- list (
fields = list(
project = c(
c(key="TEST")
),
summary="The quick brown fox jumped over the lazy dog",
description = "silly old billy",
issuetype = c(name="Task")
)
)
curl.opts <- list(
userpwd = "fred:fred",
verbose = TRUE,
httpheader = c('Content-Type' = 'application/json',Accept = 'application/json'),
useragent = "RCurl"
)
postForm("http://jirahost:8080/jira/rest/api/2/issue/",
.params= c(data=toJSON(x)),
.opts = curl.opts,
style="POST"
)
rm(list=ls())
gc()
Here's the output of the response:
* About to connect() to jirahost port 80 (#0)
* Trying 10.102.42.58... * connected
* Connected to jirahost (10.102.42.58) port 80 (#0)
> POST /jira/rest/api/2/issue/ HTTP/1.1
User-Agent: RCurl
Host: jirahost
Content-Type: application/json
Accept: application/json
Content-Length: 337
< HTTP/1.1 400 Bad Request
< Date: Mon, 07 Apr 2014 19:44:08 GMT
< Server: Apache-Coyote/1.1
< X-AREQUESTID: 764x1525x1
< X-AUSERNAME: anonymous
< Cache-Control: no-cache, no-store, no-transform
< Content-Type: application/json;charset=UTF-8
< Set-Cookie: atlassian.xsrf.token=B2LW-L6Q7-15BO- MTQ3|bcf6e0a9786f879a7b8df47c8b41a916ab51da0a|lout; Path=/jira
< Connection: close
< Transfer-Encoding: chunked
<
* Closing connection #0
Error: Bad Request

You might find it easier to use httr which has been constructed with
the needs of modern APIs in mind, and tends to set better default
options. The equivalent httr code would be:
library(httr)
x <- list(
fields = list(
project = c(key = "TEST"),
summary = "The quick brown fox jumped over the lazy dog",
description = "silly old billy",
issuetype = c(name = "Task")
)
)
POST("http://jirahost:8080/jira/rest/api/2/issue/",
body = RJSONIO::toJSON(x),
authenticate("fred", "fred", "basic"),
add_headers("Content-Type" = "application/json"),
verbose()
)
If that doesn't work, you'll need to supply the output from a successful
verbose curl on the console, and a failed httr call in R.

Related

SOAP request in R - Content-length = 0

Can someone wise point out to me where i am going wring with this SOAP request in R? I can get a valid response from the server when sending request with Python but in R i get an empty body in the response (Content-length = 0).
Here is the example request: http://www.bom.gov.au/waterdata/wiski-web-public/GetCapabilities%20example%20request.xml
and the example response: http://www.bom.gov.au/waterdata/wiski-web-public/GetCapabilities%20example%20response.xml
library(RCurl)
headerFields =
c(Accept = "text/xml",
'Content-Type' = "text/xml; charset=utf-8",
SOAPAction = "")
TXbody = '<?xml version="1.0" encoding="UTF-8"?>
<soap12:Envelope xmlns:soap12="http://www.w3.org/2003/05/soap-envelope" xmlns:sos="http://www.opengis.net/sos/2.0" xmlns:wsa="http://www.w3.org/2005/08/addressing" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:ows="http://www.opengis.net/ows/1.1" xmlns:fes="http://www.opengis.net/fes/2.0" xmlns:gml="http://www.opengis.net/gml/3.2" xmlns:swes="http://www.opengis.net/swes/2.0" xsi:schemaLocation="http://www.w3.org/2003/05/soap-envelope http://www.w3.org/2003/05/soap-envelope/soap-envelope.xsd http://www.opengis.net/sos/2.0 http://schemas.opengis.net/sos/2.0/sos.xsd">
<soap12:Header>
<wsa:To>http://www.ogc.org/SOS</wsa:To>
<wsa:Action>
http://www.opengis.net/def/serviceOperation/sos/core/2.0/GetCapabilities
</wsa:Action>
<wsa:ReplyTo>
<wsa:Address>http://www.w3.org/2005/08/addressing/anonymous</wsa:Address>
</wsa:ReplyTo>
<wsa:MessageID>0</wsa:MessageID>
</soap12:Header>
<soap12:Body>
<sos:GetCapabilities service="SOS"/>
</soap12:Body>
</soap12:Envelope>'
h = basicTextGatherer()
R <- curlPerform(url = "http://www.bom.gov.au/waterdata/services?service=SOS",
httpheader = headerFields,
postfields = TXbody, verbose=TRUE,
writefunction = h$update)
RXbody <- h$value()
The response i get is:
Trying 104.99.8.39...
Connected to www.bom.gov.au (104.99.8.39) port 80 (#0)
POST /waterdata/services?service=SOS HTTP/1.1 Host: www.bom.gov.au Accept: text/xml Content-Type: text/xml; charset=utf-8 Content-Length:
1018
upload completely sent off: 1018 out of 1018 bytes < HTTP/1.1 200 OK < Access-Control-Allow-Origin: * < Content-Type: text/plain;
charset=UTF-8 < Server: Apache-Coyote/1.1 < X-UA-Compatible: IE=Edge <
Content-Length: 0 < Expires: Tue, 12 Mar 2019 08:11:53 GMT <
Cache-Control: max-age=0, no-cache, no-store < Pragma: no-cache <
Date: Tue, 12 Mar 2019 08:11:53 GMT < Connection: keep-alive <
Connection #0 to host www.bom.gov.au left intact
I have tried this: SOAP request in R
and this: SOAP request failure in R

Wrong oauth_signature

Please, can you help me to figure out what is wrong with the signature? I really, can't understand what is the problem. I have been searching for the answer for over a month. If you give me an example, of the similar working
request or help me to understand the problem I would appreciate it so much!
library(RCurl)
library(httr)
library(rvest)
cons_key<-"ae1dd19212a84c4299f4b157462d32d7"
shared_secret<-"c0bc9938ea754f4ba6e926865b347fae"
encoded_url<-URLencode("http://platform.fatsecret.com/rest/server.api",reserved = T,repeated = T)
encoded_string<-URLencode("method=GET&oauth_consumer_key=cons_key&oauth_nonce=asdas4&oauth_signature_method=HMAC-SHA1&oauth_timestamp=15138110238&oauth_version=1.0",reserved = T,repeated = T)
text_string<-paste0("GET&",encoded_url,"&",encoded_string)
signature_oauth<-sha1_hash(key = "ae1dd19212a84c4299f4b157462d32d7&",string = text_string)
getForm("http://platform.fatsecret.com/rest/server.api",
method="GET",
oauth_consumer_key=cons_key,
oauth_signature=signature_oauth,
oauth_signature_method="HMAC-SHA1",
oauth_timestamp="1513811023",
oauth_nonce="asdas4",
oauth_version="1.0",.opts = list(verbose = TRUE))
The answer is:
Trying 34.225.169.9...
* Connected to platform.fatsecret.com (34.225.169.9) port 80 (#0)
> GET /rest/server.api method=GET&oauth_consumer_key=ae1dd19212a84c4299f4b157462d32d7&oauth_signature=rB2IYo5tZ%2BAUIsk%2BjlP0ahtn%2Bhk%3D&oauth_signature_method=HMAC-SHA1&oauth_timestamp=1513811023&oauth_nonce=asdas4&oauth_version=1.0 HTTP/1.1
Host: platform.fatsecret.com
Accept: */*
< HTTP/1.1 200 OK
< Date: Wed, 20 Dec 2017 23:05:44 GMT
< Content-Type: text/xml; charset=utf-8
< Content-Length: 377
< Connection: keep-alive
< Cache-Control: private
< Server: Microsoft-IIS/8.5
< X-AspNet-Version: 4.0.30319
< X-Powered-By: ASP.NET
<
* Connection #0 to host platform.fatsecret.com left intact
[1] "<?xml version=\"1.0\" encoding=\"utf-8\" ?>\r\n<error
xmlns=\"http://platform.fatsecret.com/api/1.0/\" xmlns:xsi=\"http://www.w3.org/2001/XMLSchema-instance\" xsi:schemaLocation=\"http://platform.fatsecret.com/api/1.0/ http://platform.fatsecret.com/api/1.0/fatsecret.xsd\">\r\n\t<code>8</code>\r\n\t<message>Invalid signature: oauth_signature 'rB2IYo5tZ+AUIsk+jlP0ahtn+hk='</message>\r\n</error>\r\n"
attr(,"Content-Type")
charset
"text/xml" "utf-8"

convert curl command to Rcurl

How do I convert this command:
curl -v -u abcdefghij1234567890:X -H "Content-Type: application/json" -X GET 'https://domain.freshdesk.com/api/v2/tickets'
to
curl command in Rcurl?
The dev version of curlconverter (devtools::install_github("hrbrmstr/curlconverter") can convert curl command-line strings with authentication and verbose params now:
Copy your URL to the clipboard:
curl -v -u abcdefghij1234567890:X -H "Content-Type: application/json" -X GET 'https://domain.freshdesk.com/api/v2/tickets'
Then run:
library(curlconverter)
req <- make_req(straighten())[[1]]
The following will now be in your clipboard:
httr::VERB(verb = "GET", url = "https://domain.freshdesk.com/api/v2/tickets",
httr::authenticate(user = "abcdefghij1234567890",
password = "X"), httr::verbose(),
httr::add_headers(), encode = "json")
but req is now also a callable function. You can see that by doing:
req
## function ()
## httr::VERB(verb = "GET", url = "https://domain.freshdesk.com/api/v2/tickets",
## httr::authenticate(user = "abcdefghij1234567890", password = "X"),
## httr::verbose(), httr::add_headers(), encode = "json")
or by actually calling it:
req()
I usually reformat the function source to make it more readable:
httr::VERB(verb = "GET",
url = "https://domain.freshdesk.com/api/v2/tickets",
httr::authenticate(user = "abcdefghij1234567890", password = "X"),
httr::verbose(),
httr::add_headers(),
encode = "json")
and you can easily translate that to a plain GET call without namespacing:
GET(url = "https://domain.freshdesk.com/api/v2/tickets",
authenticate(user = "abcdefghij1234567890", password = "X"),
verbose(),
add_headers(),
encode = "json"))
We can validate it working with authenticated curl command-lines by a small substitution in your example:
curl_string <- 'curl -v -u abcdefghij1234567890:X -H "Content-Type: application/json" -X GET "https://httpbin.org/basic-auth/abcdefghij1234567890/X"'
make_req(straighten(curl_string))[[1]]()
## -> GET /basic-auth/abcdefghij1234567890/X HTTP/1.1
## -> Host: httpbin.org
## -> Authorization: Basic YWJjZGVmZ2hpajEyMzQ1Njc4OTA6WA==
## -> User-Agent: libcurl/7.43.0 r-curl/1.2 httr/1.2.1
## -> Accept-Encoding: gzip, deflate
## -> Accept: application/json, text/xml, application/xml, */*
## ->
## <- HTTP/1.1 200 OK
## <- Server: nginx
## <- Date: Tue, 30 Aug 2016 14:13:12 GMT
## <- Content-Type: application/json
## <- Content-Length: 63
## <- Connection: keep-alive
## <- Access-Control-Allow-Origin: *
## <- Access-Control-Allow-Credentials: true
## <-
## Response [https://httpbin.org/basic-auth/abcdefghij1234567890/X]
## Date: 2016-08-30 14:13
## Status: 200
## Content-Type: application/json
## Size: 63 B
## {
## "authenticated": true,
## "user": "abcdefghij1234567890"
## }
You can use httr to do this as follows:
require(httr)
GET('https://domain.freshdesk.com/api/v2/tickets',
verbose(),
authenticate("user", "passwd"),
content_type("application/json"))

equivalence of -d parameter in curl in httr package of R

I'm following the official manual of opencpu package in R. In chapter 4.3 Calling a function It uses curl to test API:
curl http://your.server.com/ocpu/library/stats/R/rnorm -d "n=10&mean=100"
and the sample output is:
/ocpu/tmp/x032a8fee/R/.val
/ocpu/tmp/x032a8fee/stdout
/ocpu/tmp/x032a8fee/source
/ocpu/tmp/x032a8fee/console
/ocpu/tmp/x032a8fee/info
I can use curl to get similar result, but when I try to send this http request using httr package in R, I don't know how to replicate the result. Here is what I tried:
resp <- POST(
url = "localhost/ocpu/library/stats/R/rnorm",
body= "n=10&mean=100"
)
resp
the output is:
Response [HTTP://localhost/ocpu/library/stats/R/rnorm]
Date: 2015-10-16 00:51
Status: 400
Content-Type: text/plain; charset=utf-8
Size: 30 B
No Content-Type header found.
I guess I don't understand what's the equivalence of curl -d parameter in httr, how can I get it correct?
Try this :)
library(httr)
library(jsonlite)
getFunctionEndPoint <- function(url, format) {
return(paste(url, format, sep = '/'))
}
resp <- POST(
url = getFunctionEndPoint(
url = "https://public.opencpu.org/ocpu/library/stats/R/rnorm",
format = "json"),
body = list(n = 10, mean = 100),
encode = 'json')
fromJSON(rawToChar(resp$content))

httr and TLS -- sometimes it works, sometimes it doesn't

I've been using httr to export data from REDCap databases into R for a few months now. We recently upgraded our R Studio Server to the most recent version (v0.98.1049) and upgraded to R 3.1.1 at the same time. After that upgrade, my httr::POST calls stopped working, sometimes. The error I keep getting is
Error in function (type, msg, asError = TRUE) :
GnuTLS recv error (-9): A TLS packet with unexpected length was received.
At first I thought it might be an SSL issue, but the error only occurs in certain databases, and in those databases, I can still download the data using RCurl. That is, this code will work
RCurl::postForm(uri=[URL],
.params=list(token=[TOKEN],
content='record',
format='csv'))
But this code will not
httr::POST(url=[URL],
body=list(token=[TOKEN],
content='record',
format='csv'))
Adding further to my confusion, even though I can't export data in projects where this error occurs, I can import data.
I'm out of ideas on where to start. I'd much appreciate any ideas on what might be going wrong here.
(I'd like to provide a reproducible example, but I'm afraid I'm working with healthcare data. sorry)
As requested, here's the verbose() output. It's a slightly different call, but produces the same error. (I used a call that wouldn't risk exposing confidential information)
> httr::POST(url=whi$url,
+ body=list(token=whi$token,
+ content='metadata',
+ format='csv'),
+ httr::verbose(data_in=TRUE, info=TRUE))
* Hostname was found in DNS cache
* Hostname in DNS cache was stale, zapped
* Trying 172.26.30.4...
* Connected to [URL] (172.26.30.4) port 443 (#7)
* found 153 certificates in /home/nutterb/R/x86_64-unknown-linux-gnu-library/3.1/httr/cacert.pem
* SSL re-using session ID
* server certificate verification OK
* common name: [URL] (matched)
* server certificate expiration date OK
* server certificate activation date OK
* certificate public key: RSA
* certificate version: #3
* subject: OU=Domain Control Validated,CN=[URL]
* start date: Thu, 10 Apr 2014 17:06:17 GMT
*
* expire date: Sat, 21 Mar 2015 16:35:07 GMT
*
* issuer: C=US,ST=Arizona,L=Scottsdale,O=Starfield Technologies\, Inc.,OU=http://certs.starfieldtech.com/repository/,CN=Starfield Secure Certificate Authority - G2
* compression: NULL
* cipher: AES-128-CBC
* MAC: SHA1
-> POST /redcap/api/ HTTP/1.1
-> User-Agent: curl/7.35.0 Rcurl/1.95.4.1 httr/0.5.0.9000
-> Host: [URL]
-> Accept-Encoding: gzip
-> accept: application/json, text/xml, */*
-> Content-Length: 374
-> Expect: 100-continue
-> Content-Type: multipart/form-data; boundary=------------------------05c968969cc362a9
->
<- HTTP/1.1 100 Continue
>> --------------------------05c968969cc362a9
>> Content-Disposition: form-data; name="token"
>>
>> [TOKEN]
>> --------------------------05c968969cc362a9
>> Content-Disposition: form-data; name="content"
>>
>> metadata
>> --------------------------05c968969cc362a9
>> Content-Disposition: form-data; name="format"
>>
>> csv
>> --------------------------05c968969cc362a9--
<- HTTP/1.1 200 OK
<- Date: Sat, 06 Sep 2014 09:55:43 GMT
<- Expires: 0
<- cache-control: no-store, no-cache, must-revalidate
<- Pragma: no-cache
<- Access-Control-Allow-Origin: *
<- Vary: Accept-Encoding
<- Content-Type: text/html; charset=utf-8
<- Connection: close
<- Content-Encoding: gzip
<-
<< NA
<< NA
<< NA
<< NA
<< NA
<< NA
<< NA
<< NA
<< NA
<< NA
<< NA
* GnuTLS recv error (-9): A TLS packet with unexpected length was received.
* Closing connection 7
Error in function (type, msg, asError = TRUE) :
GnuTLS recv error (-9): A TLS packet with unexpected length was received.
In addition: There were 11 warnings (use warnings() to see them)
> warnings()
Warning messages:
1: In strsplit(x, "\n", fixed = TRUE) : input string 1 is invalid in this locale
2: In strsplit(x, "\n", fixed = TRUE) : input string 1 is invalid in this locale
3: In strsplit(x, "\n", fixed = TRUE) : input string 1 is invalid in this locale
4: In strsplit(x, "\n", fixed = TRUE) : input string 1 is invalid in this locale
5: In strsplit(x, "\n", fixed = TRUE) : input string 1 is invalid in this locale
6: In strsplit(x, "\n", fixed = TRUE) : input string 1 is invalid in this locale
7: In strsplit(x, "\n", fixed = TRUE) : input string 1 is invalid in this locale
8: In strsplit(x, "\n", fixed = TRUE) : input string 1 is invalid in this locale
9: In strsplit(x, "\n", fixed = TRUE) : input string 1 is invalid in this locale
10: In strsplit(x, "\n", fixed = TRUE) : input string 1 is invalid in this locale
11: In strsplit(x, "\n", fixed = TRUE) : input string 1 is invalid in this locale
>

Resources