I want to do
curl -H "Authorization: Basic YOUR_API_KEY" -d '{"classifier_id":155, "value":"TEST"}' "https://www.machinelearningsite.com/language/classify"
I tried
h = getCurlHandle(header = TRUE, userpwd = YOUR_API_KEY, netrc = TRUE)
out <- getURL("https://www.machinelearningsite.com/language/classify?classifier_id=155&value=TEST", curl=h,ssl.verifypeer=FALSE)
but it says method not allowed
It's much easier to translate curl command-line arguments into httr calls:
library(httr)
result <- GET("https://www.machinelearningsite.com/language/classify",
add_headers(Authorization=sprintf("Basic %s", YOUR_API_KEY),
query=list(classifier_id=155, value="TEST")))
ideally, YOUR_API_KEY would be an environment variable, so you can change that to:
result <- GET("https://www.machinelearningsite.com/language/classify",
add_headers(Authorization=sprintf("Basic %s", Sys.getenv("YOUR_API_KEY")),
query=list(classifier_id=155, value="TEST")))
You can then do:
content(result)
To retrieve the actual data.
Related
I couldn't find any reference to access data from Delta using SparkR so I tried myself. So, fist I created a Dummy dataset in Python:
from pyspark.sql.types import StructType,StructField, StringType, IntegerType
data2 = [("James","","Smith","36636","M",2000),
("Robert","","Williams","42114","M",5000),
("Maria","Anne","Jones","39192","F",5000),
("Jen","Mary","Brown","","F",-1)
]
schema = StructType([ \
StructField("firstname",StringType(),True), \
StructField("middlename",StringType(),True), \
StructField("lastname",StringType(),True), \
StructField("id", StringType(), True), \
StructField("gender", StringType(), True), \
StructField("salary", IntegerType(), True) \
])
df = spark.createDataFrame(data=data2,schema=schema)
df.write \
.format("delta")\
.mode("overwrite")\
.option("userMetadata", "first-version") \
.save("/temp/customers")
You can modify this code to change the data and run again to simulate the change over time.
I can query in python using this:
df3 = spark.read \
.format("delta") \
.option("timestampAsOf", "2020-11-30 22:03:00") \
.load("/temp/customers")
df3.show(truncate=False)
But I don't know how to pass the option in Spark R, can you help me?
%r
library(SparkR)
teste_r <- read.df("/temp/customers", source="delta")
head(teste_r)
It works but returns only the current version.
timestampAsOf will work as a parameter in SparkR::read.df.
SparkR::read.df("/temp/customers", source = "delta", timestampAsOf = "2020-11-30 22:03:00")
This can be also done with SparkR::sql.
SparkR::sql('
SELECT *
FROM delta.`/temp/customers`
TIMESTAMP AS OF "2020-11-30 22:03:00"
')
Alternatively, to do it in sparklyr, use the timestamp parameter in spark_read_delta.
library(sparklyr)
sc <- spark_connect(method = "databricks")
spark_read_delta(sc, "/temp/customers", timestamp = "2020-11-30 22:03:00")
Does anybody know how to convert more complex cURL commands into httr:GET() requests. The issue I am having is that the API only requires a key in the form of a username <YOUR_API_KEY> but does not require any password.
$ curl https://api.goclimate.com/v1/flight_footprint \
-u YOUR_API_KEY: \
-d 'segments[0][origin]=ARN' \
-d 'segments[0][destination]=BCN' \
-d 'segments[1][origin]=BCN' \
-d 'segments[1][destination]=ARN' \
-d 'cabin_class=economy' \
-d 'currencies[]=SEK' \
-d 'currencies[]=USD' \
-G
Perhaps another package like Rcurl might be more appropriate?
Thanks!
Well, when you include the ":" with nothing after it, you are specifying a password as the empty string. So using httr that would be like
GET("https://api.goclimate.com/v1/flight_footprint",
authenticate("YOUR_API_KEY",""),
query=list(
"segments[0][origin]"="ARN",
"segments[0][destination]"="BCN",
"segments[1][origin]"="BCN",
"segments[1][destination]"="ARN",
"cabin_class"="ecomony",
"currencies[0]"="SEK",
"currencies[1]"="USD"))
Expanding the indexes of the paramters is kind of messy, you can write a helper function
query_expand <- function(x) {
expd <- function(name, value) {
do.call("c", unname(Map(function(name, value) {
if(is.list(value) && !is.null(names(value))) {
xx <- expd(paste0("[", names(value), "]"), value)
setNames(xx, paste0(name, names(xx)))
} else if(is.list(value)) {
xx <- expd(paste0("[",seq_along(value)-1,"]"), value)
setNames(xx, paste0(name, names(xx)))
} else if (length(value)>1) {
setNames(as.list(value), paste0(name, "[", seq_along(value)-1,"]"))
} else {
setNames(list(value), name)
}}, name, value)))
}
expd(names(x), x)
}
Then if you have your data nearly in an object
params <- list("segments" = list(
list(origin="ARN", destination="BCN"),
list(origin="BCN", destination=c("ARN"))
),
"cabin_class" = "ecomony",
"currencies" = c("SEK","USD"))
You could just use
GET("https://api.goclimate.com/v1/flight_footprint",
authenticate("YOUR_API_KEY",""),
query = query_expand(params))
I tried to do a API call to get data from a website called BirdEye. They have a developer website with instructions on API calls. However, I am having trouble converting curl code into R code using httr package.
Example:
curl -H "content-type: application/json" -H "accept:application/json" -X POST -d { "fromDate":"04/01/2013", "toDate":"05/3/2015", "updateFromDate":"04/01/2013", "updateToDate":"05/3/2015", "sources":["google","citysearch"], "ratings":[1,2,3,4,5,0],"searchStr":"Awesome","subBusinessIds":[145308764185002,145308769721320,145308702026610],"tags":["tag_1","tag_2"] } "https://api.birdeye.com/resources/v1/review/businessId/755009344_1?sindex=0&count=25&api_key=92bcd6e0-c102-43fd-8a67-1a7be5258451" -v
Here is what I did:
headers <- c('Accept' = 'application/json', 'Content-Type' = 'application/json')
body_content <- list('fromDate' = '2019-01-01', 'toDate' = '2019-05-01', 'sources' = 'google', 'ratings' = c(1, 2, 3, 4, 5, 0))
resp <- POST(url ="https://api.birdeye.com/resources/v1/review/businessId/xxxxxxxxxx?sindex=0&count=20&api_key=xxxxxxxxxxxxxxxxxxxxxxxxxxx",
body = body_content, content_type("application/json"), accept("application/json"))
But I got an error message below:
Unexpected character ('-' (code 45)) in numeric value: expected digit (0-9) to follow minus sign, for valid numeric value
Warning message:
In charToRaw(enc2utf8(val)) :
argument should be a character vector of length 1
all but the first element will be ignored
Could someone help with the issue? Appreciate it!
I have curl request that is working file, which is as follows:
curl -XGET "https://xxxx.com/xxx" -u "username:password"
How I want to do it using curl package in R
I have following code,
library(curl)
clCall <- new_handle(url = 'https://xxxx.com/xxx')
handle_setopt(clCall, customrequest = "XGET")
Now I don't know how to pass username and password in this curl request
You should set httpauth and userpwd options too:
h <- curl::new_handle()
curl::handle_setopt(
handle = h,
httpauth = 1,
userpwd = "user:passwd"
)
resp <- curl::curl_fetch_memory("https://httpbin.org/basic-auth/user/passwd", handle = h)
jsonlite::fromJSON(rawToChar(resp$content))
#> $authenticated
#> [1] TRUE
#>
#> $user
#> [1] "user"
I'm following the official manual of opencpu package in R. In chapter 4.3 Calling a function It uses curl to test API:
curl http://your.server.com/ocpu/library/stats/R/rnorm -d "n=10&mean=100"
and the sample output is:
/ocpu/tmp/x032a8fee/R/.val
/ocpu/tmp/x032a8fee/stdout
/ocpu/tmp/x032a8fee/source
/ocpu/tmp/x032a8fee/console
/ocpu/tmp/x032a8fee/info
I can use curl to get similar result, but when I try to send this http request using httr package in R, I don't know how to replicate the result. Here is what I tried:
resp <- POST(
url = "localhost/ocpu/library/stats/R/rnorm",
body= "n=10&mean=100"
)
resp
the output is:
Response [HTTP://localhost/ocpu/library/stats/R/rnorm]
Date: 2015-10-16 00:51
Status: 400
Content-Type: text/plain; charset=utf-8
Size: 30 B
No Content-Type header found.
I guess I don't understand what's the equivalence of curl -d parameter in httr, how can I get it correct?
Try this :)
library(httr)
library(jsonlite)
getFunctionEndPoint <- function(url, format) {
return(paste(url, format, sep = '/'))
}
resp <- POST(
url = getFunctionEndPoint(
url = "https://public.opencpu.org/ocpu/library/stats/R/rnorm",
format = "json"),
body = list(n = 10, mean = 100),
encode = 'json')
fromJSON(rawToChar(resp$content))