Connect to UberDuck API using Curl / R - r

I am trying to pass some text to the UberDuck API but I can't seem to get my credentials to process correctly.
What is the correct structure I need to pass? In the terminal I get:
matt#UNKNOWN:~> curl -u pub_vworsdasdssd3242ds3bfzc:pk_sa8f07saf-4e7c-dsfasd989ds7dfds5f2b6 'https://api.uberduck.ai/speak'
{"detail":"Method Not Allowed"}matt#UNKNOWN:~>
matt#UNKNOWN:~> \ --data-raw '{"speech":"This is just a test.","voice":"zwf"}'
API key and secret has been changed:
Curl:
curl -u pub_jutyrreufnrhzbfzc:pk_999999999-1f3b-4e7c-iu54-6c358525f2b6 'https://api.uberduck.ai/speak'
\ --data-raw '{"speech":"This is just a test.","voice":"zwf"}'
Code in R:
library(httr)
c("\n\nWe gon' fight for the people\nThat's why we came, that's why we came\nWe gon' fight for the poor\nGon' fight for the have-nots\nGonna make sure that they get a piece of the pie\nAnd we'll never give up, never give in\n'Til we see justice for every single one\n\nAnd it's not just a black and white issue\nIt's everybody's issue\nWe all need to stand up\nAnd demand that the government does right\nBy the people, all the people\nNot just the ones with the most power and money\n\nSo come on, join the fight\nIt's time to take a stand\nFor social justice, for equality\nFor human rights, for humanity\nAnd we'll never give up, never give in\n'Til we see justice for every single one")
url <- "https://api.uberduck.ai/speak"
singer <- "2pac"
payload <- list(voice = singer, pace = 1, speech = generated_song)
headers <- list("accept" = "application/json", "content-type" = "application/json", "authorization" = "Basic XXXXXXXXXXXXXXXXXXXXXXXXXXX")# this is just the encoded version of your Uberduck API credentials. You can have it directly on the Uberduck website
}
response <- POST(url, body = payload, add_headers(headers))
print(content(response))

Related

Plumber does not find request body

I am currently writing a webhook on a heroku server and for some reason I am having difficulty obtaining the values stored in the body of the request as it always seems to turn up as NULL. My app.R script looks like all those in the guides:
#app.R
library(plumber)
library(tidyverse)
port <- Sys.getenv('PORT')
pr <- plumb("plumber.R")
pr$run(
host = '0.0.0.0',
port = as.numeric(port)
)
My plumber.R file starts like this.
library(plumber)
library(tidyverse)
#* Log some information about the incoming request
#* #filter logger
function(req){
print('LOGGER')
print( req)
cat(as.character(Sys.time()), "-",
req$REQUEST_METHOD, req$PATH_INFO, "-",
req$HTTP_USER_AGENT, "#", req$REMOTE_ADDR, "\n")
plumber::forward()
}
#* #filter bodyExists?
function(req, res){
print( 'BODY FILTER')
print( req$body)
if (is.null(req$body)){
res$status <- 404
return(list(error="Body not found"))
} else {
plumber::forward()
}
}
Every time it receives a request the console shows out this:
And stops because there is no body. I have tried sending it requests from multiple sources where I know a body is being sent with the request but for some reason plumber does not find it when it reaches my api. I have been following using https://github.com/virtualstaticvoid/heroku-plumber-app as a template and I don't see what Im doing different from them.
Was supposed to use req$postBody rather than req$body
Plumber should be able to parse the body of the post request automatically unless the content type is incorrect or unspecified.
In my case, I had forgotten to specify the content type header of the request (I tested the API using curl from the command line). Once I set it to -H "Content-Type: application/json", plumber was able to parse the body correctly and I could see the payload in req$body as well as req$args.
If you decide to use req$postBody, like the OP did, note that as of plumber v1.0.0 it is recommended to use req$bodyRaw rather than req$postBody. For more information, see postBody in the plumber request object as well as plumber PR#665.

Scraping information from site that requires login with R (Maybe use API)

Given this URL requires the following login data:
Benutzername oder E-Mail -> User: testuserscrap#web.de
Passwort -> Password: testuserscrap
(The website is kind of fantasy football of the German Bundesliga.)
There exists a post where someone asks for help for the same website.
However, I do not want to retrieve information about certain players but about the actual team. In the browser, these steps are required:
Click on the red circled icon:
Leads to this page where I would like to retrieve all the names (of the players) in list 1 and 2:
Means I would like to have an output such as:
Diego Contento
Alfred Finnbogason
...
I am not sure which way might be the best one. According to the referred posts there seem to be an API. However, I cannot access the information with the code adapted from the referred post:
library(rvest)
library(jsonlite)
library(httr)
library(plyr)
library(dplyr)
url<-"https://kickbase.sky.de/"
page<-html_session(url)
page<-rvest:::request_POST(page,url="https://kickbase.sky.de/api/v1/user/login",
body=list("email"="testuserscrap#web.de",
"password"="testuserscrap",
"redirect_url"="https://www.kickbase.com/transfermarkt/kader"),
encode='json'
)
ck <- cookies(page)
player_page<-jump_to(ck$value,"https://api.kickbase.com/leagues/1420282/lineupex")
Unfortunately, I'm not such an expert in coding or webscraping. I tried many things but I do not come to a solution :/ Therefore, I would be really grateful if you have any advice or idea how I can retrieve the information.
Best :)
Wow, this was a tough question, but a very good learning experience for me. To solve this one I used the "curlconvertor" package, download available from GitHub using devtools package. See https://github.com/hrbrmstr/curlconverter, and other questions/answers posted here at stack overflow.
First login into the web page using your browser and navigate to the page in interest. Using the developer tools copy the 'cURL' address from the file of intereste. The cURL can be stripped of the nonessential parts, but I would need to determine the noncritical parts through trial and error.
Then use the straighten function, edit the userid and password (these were not saved with the cURL address), make the request, and then parse the return.
#cURL copied from network tab for the requested file
xcurl<-"curl 'https://api.kickbase.com/leagues/1420282/lineupex'
-XGET
-H 'Accept: */*'
-H 'Origin: https://kickbase.com'
-H 'Referer: https://kickbase.com/transfermarkt/kader'
-H 'Accept-Language: en-us'
-H 'Host: api.kickbase.com'
-H 'Authorization: Bearer XU3DGDZBxlHB0sjqG01yLhHihT2AacPeIeWOlY+u3nxz/iokfCjn8a9vaKeKFXwxJpcH/0FXOgGg3J2EfmUUDJ9uwjT+oxHZTGc1EuOxbG0i66fRBBm1RBT0Yd4ACRDQ9BCs8yb+/w9+gOPIyhM2Vio3DZemExATq22osCGeW6VzYmos/3F8MTDbKOAk8NPKQYr5xPSght26ayZ4/X21ag==' \
-H 'Accept-Encoding: br, gzip, deflate'
-H 'Connection: keep-alive'"
#See https://github.com/hrbrmstr/curlconverter, install from devtools
library(curlconverter)
library(dplyr)
my_ip<-straighten(xcurl)
#add password and user id
my_ip[[1]]$password<-"testuserscrap"
my_ip[[1]]$username<-"testuserscrap#web.de"
#Make page request
respone<-my_ip %>% make_req()
#retrieve the entire file
#jsonfile<-jsonlite::toJSON(content(respone[[1]](), as="parsed"), auto_unbox = TRUE, pretty=TRUE)
#retrieve only the player info from file and convert to data frame
dfs <- lapply(content(respone[[1]](), as="parsed")$players, data.frame)
#not every player has the same information thus bind_rows instead of rbind
players <- do.call(bind_rows, dfs)
players
in case you are still looking for access to the kickbase API I've written a small python library lately for it and just released it some days ago. Might still have some bugs but it serves my purpose and maybe you want to contribute to it. :)
https://github.com/kevinskyba/kickbase-api-python

Using HTTR POST to Create a Azure Devops Work Item

Im attempting to setup some R code to create a new work item task in Azure Devops. Im okay with a mostly empty work item to start with if thats okay to do (my example code is only trying to create a work item with a title).
I receive a 203 response but the work item doesn't appear in Devops.
Ive been following this documentation from Microsoft, I suspect that I might be formatting the body incorrectly.
https://learn.microsoft.com/en-us/rest/api/azure/devops/wit/work%20items/create?view=azure-devops-rest-5.1
Ive tried updating different fields and formatting the body differently with no success. I have attempted to create either a bug or feature work item but both return the same 203 response.
To validate that my token is working I can GET work item data by ID but the POST continues to return a 203.
require(httr)
require(jsonlite)
url <- 'https://dev.azure.com/{organization}/{project}/_apis/wit/workitems/$bug?api-version=5.1'
headers = c(
'Authorization' = sprintf('basic %s',token),
'Content-Type' = 'application/json-patch+json',
'Host' = 'dev.azure.com'
)
data <- toJSON(list('body'= list("op"= "add",
"path"= "/fields/System.AreaPath",
"value"= "Sample task")), auto_unbox = TRUE, pretty = TRUE)
res <- httr::POST(url,
httr::add_headers(.headers=headers),
httr::verbose(),
body = data)
Im expecting a 200 response (similar to the example in the link above) and a work item task in Azure DevOps Services when I navigate to the website.
Im not the best with R so please be detailed. Thank you in advanced!
The POST continues to return a 203.
The HTTP response code 203 means Non-Authoritative Information, it should caused by your token format is converted incorrectly.
If you wish to provide the personal access token through an HTTP
header, you must first convert it to a Base64 string.
Refer to this doc described, if you want to use VSTS rest api, you must convert your token to a Base64 string. But in your script, you did not have this script to achieve this convert.
So, please try with the follow script to convert the token to make the key conformant with the requirements(load the base64enc package first):
require(base64enc)
key <- token
keys <- charToRaw(paste0(key,":token"))
auth <- paste0("Basic ",base64encode(keys))
Hope this help you get 200 response code
I know this question is fairly old, but I cannot seem to find a good solution posted yet. So, I will add my solution in case others find themselves in this situation. Note, this did take some reading through other SO posts and trial-and-error.
Mengdi is correct that you do need to convert your token to a Base64 string.
Additionally, Daniel from this SO question pointed out that:
In my experience with doing this via other similar mechanisms, you have to include a leading colon on the PAT, before base64 encoding.
Mengdi came up big in another SO solution
Please try with adding [{ }] outside your request body.
From there, I just made slight modifications to your headers and data objects. Removed 'body' from your json, and made use of paste to add square brackets as well. I found that the Rcurl package made base64 encoding a breeze. Then I was able to successfully create a blank ticket (just titled) using the API! Hope this helps someone!
library(httr)
library(jsonlite)
library(RCurl)
#user and PAT for api
userid <- ''
token= 'whateveryourtokenis'
url <- 'https://dev.azure.com/{organization}/{project}/_apis/wit/workitems/$bug?api-version=5.1'
#create a combined user/pat
#user id can actually be a blank string
#RCurl's base64 seemed to work well
c_id <- RCurl::base64(txt = paste0(userid,
":",
devops_pat
),
mode = "character"
)
#headers for api call
headers <- c(
"Authorization" = paste("Basic",
c_id,
sep = " "
),
'Content-Type' = 'application/json-patch+json',
'Host' = 'dev.azure.com'
)
#body
data <- paste0("[",
toJSON(list( "op"= "add",
"path"= "/fields/System.Title",
"value"= "API test - please ignore"),
auto_unbox = TRUE,
pretty = TRUE
),
"]"
)
#make the call
res <- httr::POST(url,
httr::add_headers(.headers=headers),
httr::verbose(),
body = data
)
#check status
status <- res$status_code
#check content of response
check <- content(res)

curl POST statement to RCurl or httr

I have this working curl statement to post a file to Nokia's HERE batch geocoding service...
curl -X POST -H 'Content-Type: multipart/form-data;boundary=----------------------------4ebf00fbcf09' \
--data-binary #example.txt \
'http://batch.geocoder.cit.api.here.com/6.2/jobs?action=run&mailto=test#gmail.com&maxresults=1&language=es-ES&header=true&indelim=|&outdelim=|&outcols=displayLatitude,displayLongitude,houseNumber,street,district,city,postalCode,county,state,country,matchLevel,relevance&outputCombined=false&app_code=AJKnXv84fjrb0KIHawS0Tg&app_id=DemoAppId01082013GAL'
I have tried this:
library(RCurl)
url <- "http://batch.geocoder.cit.api.here.com/6.2/jobs? action=run&mailto=test#gmail.com&maxresults=1&language=es-ES&header=true&indelim=|&outdelim=|&outcols=displayLatitude,displayLongitude,houseNumber,street,district,city,postalCode,county,state,country,matchLevel,relevance&outputCombined=false&app_code=AJKnXv84fjrb0KIHawS0Tg&app_id=DemoAppId01082013GAL'"
postForm(url, file=fileUpload(filename="example.txt",
contentType="multipart/form-data;boundary=----------------------------4ebf00fbcf09"))
And this:
library(httr)
a <- POST(url, body=upload_file("example.txt", type="text/plain"),
config=c(add_headers("multipart/form-data;boundary=----------------------------4ebf00fbcf09")))
content(a)
Using this file as example.txt: https://gist.github.com/corynissen/4f30378f11a5e51ad9ad
Is there any way to do this property in R?
I'm not a Nokia developer, and I'm assuming those are not your real API creds. This should help you get further with httr:
url <- "http://batch.geocoder.cit.api.here.com/6.2/jobs"
a <- POST(url, encode="multipart", # this will set the header for you
body=list(file=upload_file("example.txt")), # this is how to upload files
query=list(
action="run",
mailto="test#example.com",
maxresults="1",
language="es-ES", # this will build the query string
header="true",
indelim="|",
outdelim="|",
outcols="displayLatitude,displayLongitude", # i shortened this for the example
outputCombined="false",
app_code="APPCODE",
app_id="APPID"),
verbose()) # this lets you verify what's going on
But, I can't be sure w/o registering (and no time to do that).
This is the solution based on hrbrmstr's solution
bod <- paste(readLines("example.txt", warn=F), collapse="\n")
a <- POST(url, encode="multipart", # this will set the header for you
body=bod, # this is how to upload files
query=list(
action="run",
mailto="test#gmail.com",
maxresults="1",
language="es-ES", # this will build the query string
header="true",
indelim="|",
outdelim="|",
outcols="displayLatitude,displayLongitude,houseNumber,street,district,city,postalCode,county,state,country,matchLevel,relevance", # i shortened this for the example
outputCombined="false",
app_code="AJKnXv84fjrb0KIHawS0Tg",
app_id="DemoAppId01082013GAL"),
#config=c(add_headers("multipart/form-data;boundary=----------------------------4ebf00fbcf09")),
verbose()) # this lets you verify what's going on
content(a)
The problem I had to get around was that the normal upload process strips line breaks... but I needed them in there for the API to work (--data-binary option in curl does this). To get around this, I insert the data as a string after reading it via readLines().

R API Connection (Localytics) with getURL and RCurl Error

I am trying to get data from the mobile analytics service Localytics via their API (https://api.localytics.com/docs#query). In particular I would like to translate the following cURL command in R:
curl --get 'https://api.localytics.com/v1/query' \
--user 'API_KEY:API_SECRET' \
--data 'app_id=APP_ID' \
--data 'metrics=users' \
--data 'dimensions=day' \
--data-urlencode 'conditions={"day":["between","2013-04-01","2013-04-07"]}'
My R code looks like this at the moment. APIKey and API secret are of course replaced by the actual keys. However, I receive an error stating that at least a dimension or a metric has to be specified.
object <- getURL('https://api.localytics.com/v1/query', userpwd = "API_Key:API_Secret", httpheader=list(app_id = "app_id=03343434353534",
metrics = "metrics=users",
dimensions = "dimensions=day",
conditions = toJSON('conditions={"day":["between","2014-07-01","2014-07-10"]}')), ssl.verifypeer = FALSE)
What changes would be necessary to get it to work.
Thanks in advance for helping me out,
Peter
This is particular easy with the dev version of httr:
library(httr)
r <- POST('https://api.localytics.com/v1/query',
body = list(
app_id = "APP_ID",
metrics = "users",
dimensions = "day",
conditions = list(
day = c("between", "2014-07-01", "2004-07-10")
)
),
encode = "json",
authenticate("API_key", "API_secret")
)
stop_for_status(r)
content(r)
(I converted the request to a POST and used json encoding for everything, as describe in the API docs).
If you want to see exactly what's being sent to the server, use the verbose() config.
It looks like getURL passes the parameters you quested as HTTP headers and not as querystring data as your curl call does. You should use getForm instead. Also, I wasn't sure from which library your toJSON function came form, but that's at least not the right syntax for the one from rsjon.
Anyway, here's a call from R which should produce the same HTTP call are your curl command
library(rjson)
library(RCurl)
object <- getForm('https://api.localytics.com/v1/query',
app_id = "APP_ID",
metrics = "users",
dimensions = "day",
conditions = toJSON(list(day=c("between","2014-07-01","2004-07-10"))),
.opts=curlOptions(
userpwd = "API_Key:API_Secret", httpauth = 1L)
)
I found that using the site http://requestb.in/ is very helpful in debugging these problems (and that's exactly what I used to create this solution). You can send requests to their site and they record the exact HTTP message that was sent so you can compare different methods.
The httpauth part was from this SO question which seemed to be required to trigger authentication for the test site; you may not need it for the "real" site.

Resources