I want to dynamically call the plumber API based on any number of input variables. I need to map the curl input to the input of a function name. For example if the function has an input hi then, curl -s --data 'hi=2' means that hi=2 should be passed as an input parameter to the function. This can be done directly in R with match.call() but it is failing while calling it through the plumber API.
Take the function
#' #post /API
#' #serializer unboxedJSON
tmp <- function(hi) {
out <- list(hi=hi)
out <- toJSON(out, pretty = TRUE, auto_unbox = TRUE)
return(out)
}
tmp(hi=2)
out: {hi:2}
Then
curl -s --data 'hi=10' http://127.0.0.1/8081/API
out: {\n \"hi\": \"2\"\n}
Everything looks good. However, take the function
#' #post /API
#' #serializer unboxedJSON
tmp <- function(...) {
out <- match.call() %>%
as.list() %>%
.[2:length(.)] # %>%
out <- toJSON(out, pretty = TRUE, auto_unbox = TRUE)
return(out)
}
tmp(hi=2)
out: {hi:2}
Then
curl -s --data 'hi=10' http://127.0.0.1/8081/API
out: {"error":"500 - Internal server error","message":"Error: No method asJSON S3 class: R6\n"}
In practice what I really want to do is load my ML model to predict a score with the plumber API. For example
model <- readRDS('model.rds') # Load model as a global variable
predict_score <- function(...) {
df_in <- match.call() %>%
as.list() %>%
.[2:length(.)] %>%
as.data.frame()
json_out <- list(
score_out = predict(model, df_in) %>%
toJSON(., pretty = T, auto_unbox = T)
return(json_out)
}
This function works as expected when running locally, but running through the API via curl -s --data 'var1=1&var2=2...etc' http://listen_address
I get the following error: {"error":"500 - Internal server error","message":"Error in as.data.frame.default(x[[i]], optional = TRUE): cannot coerce class \"c(\"PlumberResponse\", \"R6\")\" to a data.frame\n"}
Internally plumber match parameters in your request to the name of the parameters in your function. There are special arguments that you could use to explore all args in the request. If you have an argument named req, it will give you an environnement containing the entire request metadata, one of which is req$args. Which you could then parse. The first two args are self reference to special arguments req and res. They are environment and should not be serialized. I would not advise doing what is shown here in any production code as it opens up the api to abuse.
model <- readRDS('model.rds') # Load model as a global variable
#' #post /API
#' #serializer unboxedJSON
predict_score <- function(req) {
df_in <- as.data.frame(req$args[-(1:2)])
json_out <- list(
score_out = predict(model, df_in)
return(json_out)
}
But for your use case, what I would actually advise is having a single parameter named df_in. Here is how you would set that up.
model <- readRDS('model.rds') # Load model as a global variable
#' #post /API
#' #param df_in
#' #serializer unboxedJSON
predict_score <- function(df_in) {
json_out <- list(
score_out = predict(model, df_in)
return(json_out)
}
Then with curl
curl --header "Content-Type: application/json" \
--request POST \
--data '{"df_in":{"hi":2, "othercrap":4}}' \
http://listen_address
When the body of request starts with "{" plumber will parse the content of the body with jsonlite:fromJSON and use the name of the parsed objects to maps to parameters in your function.
Currently both CRAN and master branch on github do not handle this correctly via the swagger api but it will works just fine via curl or other direct calling method. Next plumber version will handle all that and more I believe.
See a similar answer to this of question here : https://github.com/rstudio/plumber/issues/512#issuecomment-605735332
Related
I am trying to wrap a function into REST API using plumber R Package. As a Input , function takes a shapefile and returns shapefile in terms of .shp format as well as GeoJSON format after the transformation. With the help of decorator now I need to enhance it into a web service.
R File:
#* Spatial Polygon Object
#* #param pathtoshpfile:character Path to Shapefile with Name
#* #param design:character one or two
#* #post /sprayermap
function(pathtoshpfile, design = c("one", "two")) {
# library
require(rgeos)
require(sp)
require(rgdal)
require(raster)
#Importing Shapefile
a_shape <- raster::shapefile(pathtoshpfile)
if (class(a_shape) == "SpatialPolygonsDataFrame") {
if (design == "one") {
a_shape <- tryCatch (
rgeos::gBuffer(a_shape, byid = TRUE, width = 0),
error = function(err) {
return(paste("sprayer map : ", err))
}
)
sprayer_map <- tryCatch (
aggregate(a_shape, "Rx"),
error = function(err) {
return(paste("sprayer map : ", err))
}
)
sprayer_map#data$Rx <- as.integer(sprayer_map#data$Rx)
raster::shapefile(sprayer_map, filename = "field_sprayer_map", overwrite =
TRUE)
rgdal::writeOGR(
sprayer_map,
dsn = "field_sprayer_map.GeoJSON",
layer = "geojson",
driver = "GeoJSON",
overwrite_layer = TRUE
)
return(paste0("SpatialPolygonsDataFrame"))
} else {
return(paste0("design two"))
}
} else {
return(paste0("Please provide spatial polygon object !"))
}
}
Plumber Part :
library(plumber)
# 'plumber.R' is the location of the file shown above
pr("plumber.R") %>%
pr_run(port=8000)
#############################
curl "http://127.0.0.1:8000/sprayermap?pathtoshpfile=/path/to/directory/test.shp&design=one"
Based on my limited understanding on creation of APIs , I created above script and got below error.
Error: unexpected string constant in "curl "http://127.0.0.1:8000/sprayermap?
Though plain R Function script works good
Looking for guidance how the decorators can be used to convert the above function into Restful APIs where Input and output both are shapefiles.
Test Shape File: Test Shape File
I believe your function is setup correctly. The following curl calls should help you out:
curl -X POST "http://127.0.0.1:8000/sprayermap?pathtoshpfile=../Downloads/jnk/test.shp&design=one"
(ie. insert -X POST in your existing call). Alternatively, via --data:
curl --data "pathtoshpfile=../Downloads/jnk/test.shp" "http://127.0.0.1:8000/sprayermap"
On my machine, both queries return
["SpatialPolygonsDataFrame"]
and field_sprayer_map is written to disk. For the record, this works with both the current plumber CRAN version 0.4.6:
p = plumb("../Downloads/jnk/plumber.R")
p$run(port=8000)
And the upcoming v1.0.0 release candidate:
pr("../Downloads/jnk/plumber.R") %>%
pr_run(port=8000)
I have the following code for R Plumber API server
library(jsonlite)
library(data.table)
#' Home endpoint
#' #get /
function(){
df <- data.table(msg = "Welcome")
toJSON(df)
}
It gives me ["[{\"msg\":\"Welcome\"}]"] result on API.
How to replace \" on " symbol to make it more human-friendly when working in a browser or Postman? Expected result is "msg":"Welcome".
Thanks!
plumber already jsonifies it, you are doubling it. Try this:
#' Home endpoint
#' #get /
function(){
df <- data.table::data.table(msg = "Welcome")
return(df)
}
And then in my console, I ran:
pr <- plumber::plumb("~/StackOverflow/4393334/60918243.R")
pr$run()
# Starting server to listen on port 5225
# Running the swagger UI at http://127.0.0.1:5225/__swagger__/
And then on my bash shell:
$ curl -s localhost:5225
[{"msg":"Welcome"}]
I am trying to run a Plumber API inline to receive an input, and once the proper input is received and a specified condition is met, the input is returned to the globalenv and the API closes itself such that the script can continue to run.
I've specified a condition within a #get endpoint that calls quit(), stop() etc, none of which successfully shut down the API.
I've attempted to run the API in parallel using future such that the parent script can close the Plumber API.
It appears that there isn't actually a method in the Plumber API class object to close the Plumber API, and the API can't be closed from within itself.
I've been through the extended documentation, SO, and the Github Issues in search of a solution. The only semi-relevant solution suggested is to use R.Utils::withTimeout to create a time-bounded timeout. However, this method is also unable to close the API.
A simple use case:
Main Script:
library(plumber)
code_api <- plumber::plumb("code.R")
code_api$run(port = 8000)
code.R
#' #get /<code>
function(code) {
print(code)
if (nchar(code) == 3) {
assign("code",code,envir = globalenv())
quit()}
return(code)
}
#' #get /exit
function(exit){
stop()
}
The input is successfully returned to the global environment, but the API does not shut down afterward, nor after calling the /exit endpoint.
Any ideas on how to accomplish this?
You could look at Iterative testing with plumber #Irène Steve's, Dec 23 2018 with:
trml <- rstudioapi::terminalCreate()
rstudioapi::terminalKill(trml)
excerpt of her article (2nd version of 3):
.state <- new.env(parent = emptyenv()) #create .state when package is first loaded
start_plumber <- function(path, port) {
trml <- rstudioapi::terminalCreate(show = FALSE)
rstudioapi::terminalSend(trml, "R\n")
Sys.sleep(2)
cmd <- sprintf('plumber::plumb("%s")$run(port = %s)\n', path, port)
rstudioapi::terminalSend(trml, cmd)
.state[["trml"]] <- trml #store terminal name
invisible(trml)
}
kill_plumber <- function() {
rstudioapi::terminalKill(.state[["trml"]]) #access terminal name
}
Running a Plumber in the terminal might work in some cases but as I needed access to the R session (for insertText) I had to come up with the different approach. While not ideal the following solution worked:
# plumber.R
#* Insert
#* #param msg The msg to insert to the cursor location
#* #post /insert
function(msg="") {
rstudioapi::insertText(paste0(msg))
stop_plumber(Sys.getpid())
}
.state <- new.env(parent = emptyenv()) #create .state when package is first loaded
stop_plumber <- function(pid) {
trml <- rstudioapi::terminalCreate(show = FALSE)
Sys.sleep(2) # Wait for the terminal to initialize
# Wait a bit for the Plumber to flash the buffers and then send a SIGINT to the R session process,
# to terminate the Plumber
cmd <- sprintf("sleep 2 && kill -SIGINT %s\n", pid)
rstudioapi::terminalSend(trml, cmd)
.state[["trml"]] <- trml # store terminal name
invisible(trml)
Sys.sleep(2) # Wait for the Plumber to terminate and then kill the terminal
rstudioapi::terminalKill(.state[["trml"]]) # access terminal name
}
Hi I try to translate this curl instruction using httr
curl -H "Authorization: Token f2210dacd9c6ccb8133606d94ff8e61d99b477fd" -F file=#test.txt -F filename=test.txt -F parent_dir=/ http://cloud.seafile.com:8082/upload-api/73c5d117-3bcf-48a0-aa2a-3f48d5274ae3
Without the -F parameter the instruction is :
httr::POST(
url = "http://cloud.seafile.com:8082/upload-api/73c5d117-3bcf-48a0-aa2a-3f48d5274ae3",
add_headers(Authorization = "Token f2210dacd9c6ccb8133606d94ff8e61d99b477fd")
)
)
I think I have to use the httr::upload_file function but I didn't manage to use this without error.
Do you have any idea how I can do that ?
Regards
Here is how to construct this curl request with httr package. I used httpbin.org to test the request sent.
You'll use POST filling the body with a list. encode argument controls how this list will be handle and by default it is the correct multipart you need.
res <- httr::POST(
url = "http://httpbin.org/post",
httr::add_headers(Authorization = "Token f2210dacd9c6ccb8133606d94ff8e61d99b477fd"),
# Add the file and metadata as body using a list. Default encode is ok
body = list(
file = httr::upload_file("test.txt"),
filename = "test.txt",
parent_dir = "/"
)
)
httr_ouput <- httr::content(res)
One way to check this is ok is to compare output with the curl command you know is working
out <- sys::exec_internal(
'curl -H "Authorization: Token f2210dacd9c6ccb8133606d94ff8e61d99b477fd" -F file=#test.txt -F filename=test.txt -F parent_dir=/ http://httpbin.org/post'
)
curl_output <- jsonlite::fromJSON(rawToChar(out$stdout))
#compare body
identical(curl_output$files, httr_ouput$files)
#> TRUE
identical(curl_output$form, httr_ouput$form)
#> TRUE
You can also do it with the crul package, another wrapper above curl; The logic is identical
con <- crul::HttpClient$new(
url = "http://httpbin.org/post"
)
crul_req <- con$post(
body = list(
file = crul::upload("test.txt"),
filename = "test.ext",
parent_dir = "/"
)
)
crul_output <- jsonlite::fromJSON(crul_req$parse())
I am trying to extract time series data from the BLS API using RCurl.
The BLS supplies the following sample code for command line extraction:
curl -i -X POST -H 'Content-Type: application/json'
-d '{"seriesid":["LEU0254555900", "APU0000701111"],
"startyear":"2002", "endyear":"2012"}'
http://api.bls.gov/publicAPI/v1/timeseries/data/
I have also confirmed that the specified files (i.e. series ids) are both present as the following both yield a JSON formatted object:
require(RCurl)
bls.content_test1 <- getURLContent("http://api.bls.gov/publicAPI/v1/timeseries/data/LEU0254555900")
bls.content_test2 <- getURLContent("http://api.bls.gov/publicAPI/v1/timeseries/data/APU0000701111")
Based on a variety of posts with the RCurl tag (and this post in particular), I have ported the command line script to the following chunk of code:
require(RJSONIO)
jsonbody <- toJSON(list("seriesid"=paste('"["CFU0000008000"', '[LEU0254555900"]"')
,"startyear"="2012"
,"endyear"="2013"))
httpheader <- c(Accept="application/json; charset=UTF-8",
"Content-Type"="application/json")
bls.content <- postForm("http://api.bls.gov/publicAPI/v1/timeseries/data/"
,.opts=list(httpheader=httpheader
,postfields=jsonbody))
Which yields:
[1] "{\"status\":\"REQUEST_FAILED\",\"responseTime\":0,\"message\":[\"Your request has failed, please check your input parameters and try your request again.\"],\"Results\":null}"
attr(,"Content-Type")
charset
"application/json" "UTF-8"
Does this appear to be a problem with my implementation of RCurl or does this appear to be a problem with the BLS API?
It's actually a problem with the way you are creating your json body. With your version if you do cat(jsonbody) you get
{
"seriesid": "\"[\"CFU0000008000\" [LEU0254555900\"]\"",
"startyear": "2012",
"endyear": "2013"
}
which has those extra escapes and brackets in there. It's not correct. Instead try
jsonbody <- toJSON(list("seriesid"=c("CFU0000008000", "LEU0254555900"),
startyear="2012",
endyear="2013"))
which gives
{
"seriesid": [ "CFU0000008000", "LEU0254555900" ],
"startyear": "2012",
"endyear": "2013"
}
which is valid JSON. Just changing that part and using the rest of the code as you had it gives me a REQUEST_SUCCEEDED message.
Here's an approach with httr:
library(httr)
library(jsonlite)
# Need unbox() to tell jsonlite() numbers are scalars, not vectors of length 1
body <- list(
seriesid = c("CFU0000008000", "LEU0254555900"),
startyear = unbox(2012),
endyear = unbox(2013)
)
r <- POST("http://api.bls.gov/publicAPI/v1/timeseries/data/", body = body, encode = "json")
stop_for_status(r)
# Need to specify type since site returns incorrect type of text/plain
str(content(r, type = "application/json"))