I would like to call my Philips Hue lights from R via the API and httr package. The problem is however that I can't get the body right. I'm sure the API works because GET calls work fine.
For example, the body in a PUT call to turn the lights on and off should look exactly like {"on":false}. The call looks like PUT(url = url), body = body1)
However, I cannot get this to work in the body section from the httr package. I already tried: body1 <- '{on:"false"}'
Which returns: "{on:\"false\"}", body2 <- list(on = "false") returns $on [1] "false" and body3 <- toJSON(body2) returns {"on":["false"]}.
As you can see none of the above options exactly desired return and they all produce extra punctuation marks. Any idea how I can get exactly {"on":false} in the body?
Unfortunately, I cannot provide you with a reproducible example because there is no public sandbox environment available and I don't want everyone to control my lights ;-) However the documentation can be found here.
If you are using toJSON from the jsonlite package, then you can do
library(jsonlite)
PUT("https://url", body=toJSON(list(on = unbox(FALSE))))
The unbox() will prevent the R vector from being wrapped in the brackets for a JSON array.
Related
I was learning about the package httr and webscraping based on an exercise from Dataquest and attempting to implement it in my own practice programs. My issue comes from trying to make a query within a function.
For example, the following code:
api_request <- function(base_url, loc){
url <- modify_url(paste(base_url),
path = paste(loc))
response <- GET(url)
return(response)
}
When I run the code, everything initially appears to run correctly. The status comes back with the code 200 and no errors or warnings show up. However, I cannot get the response to save to the global environment. I've tried this method as well as changing the return(response) to just response in the function as recommended by Dataquest, but it will not save to the global environment.
I can get this to work outside of a function, but I want to implement it inside of a function so that if any errors occur when making this query I can stop the function and not save a 404 link.
How can I get the query to return from the function so that I can reference it later on in the code?
I try to send a request to API, use RCurl library.
My code:
start = "2018-07-30"
end = "2018-08-15"
api_request <- paste("https://api-metrika.yandex.ru/stat/v1/data.csv?id=34904255&date1=",
start,
"&date2=",
end,
"&dimensions=ym:s:searchEngine&metrics=ym:s:visits&dimensions=ym:s:<attribution>SearchPhrase&filters=ym:s:<attribution>SearchPhrase!~'some|phrases|here'&limit=100000&oauth_token=OAuth_token_here", sep="")
s <- getURL(api_request)
And every time I try to do it I have the response "Error 400" or "Bad Request" if I use getUrlContent instead. When I just open this url in my browser - it works correctly.
I still couldn't find any solution for this problem, so if somebody knows something about it - please help me, kind man =)
There are several approaches you can use in case the URL is right. First you can add the following parameter to the getURL function. Setting the parameter followlocation equal TRUE allows to follow any "Location:" header that the server returns as part of the HTTP headers. Processing is recursive, PHP will follow any "Location:" header.
> s <- getURL(url1, .opts=curlOptions(followlocation = TRUE))
If this is not working an alternative way is to use the XML package by calling the htmlParse method instead of getURL
> library(XML)
> s <- htmlParse(api_request)
Another approach would be to use the httr package and call the GET function:
> library(httr)
> s <- GET(api_request)
I need to pass a custom header for a GET request using httr package in R, I checked the vignettes but can't get it right and looking for some help.
CusAuth <- c(database="database",user="login id",password="password")
GET(url,add_headers(.headers = CusAuth))
I had to include the vector member within quotes so it is sent as a single value to the header and this is working now.
CusAuth <- c('database="database",user="login id",password="password"')
This is not just a coding style question. If you know python (and I think also Ruby has something like this), you can have a docstring in a function, such that you can readily get that string by issuing a "help" command. e.g.:
def something(t=None):
'''Do something, perhaps to t
t : a thing
You may not want to do this
'''
if t is not None:
return t ** 2
else:
return 'Or maybe not'
Then help(something) returns the following:
Help on function something in module __main__:
something(t=None)
Do something, perhaps to t
t : a thing
You may not want to do this
The way things work in R, you can get the full text of the defined code snippet, so you could see comments (including those at the beginning of the function), but that can be a lot of scrolling and visual filtering. Is there any better way?
I recently wrote a package to do just this task. The docstring package allows one to write their documentation as roxygen style comments within the function they are documenting. For example one could do
square <- function(x){
#' Square a number
return(x^2)
}
and then to view the documentation either call the docstring function
docstring(square)
or use the built in ? support and do
?square
The comments can either be a single chunk like shown above or fully roxygen style to take advantage of some of the keywords provided
square <- function(x){
#' Square a number
#'
#' Calculates the square of the input
#'
#' #param x the input to be squared
return(x^2)
}
This is on CRAN now: https://cran.r-project.org/package=docstring so you can just install using
install.packages("docstring")
or if you want the latest development version you can install from github:
library(devtools)
install_github("dasonk/docstring")
You can add any attributes you like to R objects, including function. So something like
describe <- function(obj) attr(obj, "help")
foo <- function(t=NULL) ifelse(!is.null(t), t^2, "Or maybe not")
attr(foo, "help") <- "Here is the help message"
produces more or less the desired output
> foo(2)
[1] 4
> foo()
[1] "Or maybe not"
> describe(foo)
[1] "Here is the help message"
Sort-of -- look at the roxygen2 package on CRAN (vignette here). You write a declarative header, and among other things a help page is created for you when you 'roxygen-ize' your sources.
It may not be the easiest package to use, see here on SO for questions pertaining to it as well as its mailing list. But it probably is the closest match.
RStudio helps you to create documentation quite easily.
See their documentation for more information.
The new reference class system has something very similar to docstrings for documenting methods of a class. Here is an example:
Something <- setRefClass("Something",
methods=list(
something=function(t=NULL) {
"Do something, perhaps to t
t : a thing
You may not want to do this
"
if(!is.null(t))
t^2
else
"Or maybe not"
}
))
a <- Something$new()
a$something(2)
a$something()
Something$help("something") ## to see help page
I had another idea as I'm finally wrapping my head around the fact that "R is a (very poor) LISP". Specifically, you can get access to the source code (usually) using the deparse command. So, this function would be a start towards defining your own custom source-code parsing help function:
docstr <- function(my.fun) {
# Comments are not retained
# So, we can put extra stuff here we don't want
# our docstr users to see
'This is a docstring that will be printed with extra whitespace and quotes'
orig.code.ish <- deparse(my.fun)
print(orig.code.ish[3])
}
docstr(docstr)
The above illustrates that deparse really does deparse, and is different from what you'd print at the REPL prompt if you typed docstr: quotes are changed to (default) double-quotes, opening curly brace gets moved to the second line, and blank lines (including comments) are removed. This actually helps a lot if you want to design a robust function. Would be trivial to look for e.g., opening and closing quotes down through the first line that doesn't start with a quote.
Another way to do it would be to get the list of call objects that make up the body list with body(docstr). The string would be in body(docstr)[[2]]. I have to admit that I'm a bit out of my depth here, as I don't fully understand the return value of body, and don't know where to find documentation! In particular, note that body(docstr)[2] returns an object of type and mode 'call'.
This latter approach seems much more LISPy. I'd love to hear other's thoughts on the general idea, or have pointers to actual language reference materials for this!
As a way of exploring how to make a package in R for the Denver RUG, I decided that it would be a fun little project to write an R wrapper around the datasciencetoolkit API. The basic R tools come from the RCurl package as you might imagine. I am stuck on a seemingly simple problem and I'm hoping that somebody in this forum might be able to point me in the right direction. The basic problem is that I can't seem to use postForm() to pass an un-keyed string as part of the data option in curl, i.e. curl -d "string" "address_to_api".
For example, from the command line I might do
$ curl -d "Tim O'Reilly, Archbishop Huxley" "http://www.datasciencetoolkit.org/text2people"
with success. However, it seems that postForm() requires an explicit key when passing additional arguments into the POST request. I've looked through the datasciencetoolkit code and developer docs for a possible key, but can't seem to find anything.
As an aside, it's pretty straightforward to pass inputs via a GET request to other parts of the DSTK API. For example,
ip2coordinates <- function(ip) {
api <- "http://www.datasciencetoolkit.org/ip2coordinates/"
result <- getURL(paste(api, URLencode(ip), sep=""))
names(result) <- "ip"
return(result)
}
ip2coordinates('67.169.73.113')
will produce the desired results.
To be clear, I've read through the RCurl docs on DTL's omegahat site, the RCurl docs with the package, and the curl man page. However, I'm missing something fundamental with respect to curl (or perhaps .opts() in the postForm() function) and I can't seem to get it.
In python, I could basically make a 'raw' POST request using httplib.HTTPConnection -- is something like that available in R? I've looked at the simplePostToHost function in the httpRequest package as well and it just seemed to lock my R session (it seems to require a key as well).
FWIW, I'm using R 2.13.0 on Mac 10.6.7.
Any help is much appreciated. All of the code will soon be available on github if you're interested in playing around with the data science toolkit.
Cheers.
With httr, this is just:
library(httr)
r <- POST("http://www.datasciencetoolkit.org/text2people",
body = "Tim O'Reilly, Archbishop Huxley")
stop_for_status(r)
content(r, "parsed", "application/json")
Generally, in those cases where you're trying to POST something that isn't keyed, you can just assign a dummy key to that value. For example:
> postForm("http://www.datasciencetoolkit.org/text2people", a="Archbishop Huxley")
[1] "[{\"gender\":\"u\",\"first_name\":\"\",\"title\":\"archbishop\",\"surnames\":\"Huxley\",\"start_index\":44,\"end_index\":61,\"matched_string\":\"Archbishop Huxley\"},{\"gender\":\"u\",\"first_name\":\"\",\"title\":\"archbishop\",\"surnames\":\"Huxley\",\"start_index\":88,\"end_index\":105,\"matched_string\":\"Archbishop Huxley\"}]"
attr(,"Content-Type")
charset
"text/html" "utf-8"
Would work the same if I'd used b="Archbishop Huxley", etc.
Enjoy RCurl - it's probably my favorite R package. If you get adventurous, upgrading to ~ libcurl 7.21 exposes some new methods via curl (including SMTP, etc.).
From Duncan Temple Lang on the R-help list:
postForm() is using a different style (or specifically Content-Type) of submitting the form than the curl -d command.
Switching the style = 'POST' uses the same type, but at a quick guess, the parameter name 'a' is causing confusion
and the result is the empty JSON array - "[]".
A quick workaround is to use curlPerform() directly rather than postForm()
r = dynCurlReader()
curlPerform(postfields = 'Archbishop Huxley', url = 'http://www.datasciencetoolkit.org/text2people', verbose = TRUE,
post = 1L, writefunction = r$update)
r$value()
This yields
[1]
"[{\"gender\":\"u\",\"first_name\":\"\",\"title\":\"archbishop\",\"surnames\":\"Huxley\",\"start_index\":0,\"end_index\":17,\"matched_string\":\"Archbishop
Huxley\"}]"
and you can use fromJSON() to transform it into data in R.
I just wanted to point out that there must be an issue with passing a raw string via the postForm function. For example, if I use curl from the command line, I get the following:
$ curl -d "Archbishop Huxley" "http://www.datasciencetoolkit.org/text2people
[{"gender":"u","first_name":"","title":"archbishop","surnames":"Huxley","start_index":0,"end_index":17,"matched_string":"Archbishop Huxley"}]
and in R I get
> api <- "http://www.datasciencetoolkit.org/text2people"
> postForm(api, a="Archbishop Huxley")
[1] "[{\"gender\":\"u\",\"first_name\":\"\",\"title\":\"archbishop\",\"surnames\":\"Huxley\",\"start_index\":44,\"end_index\":61,\"matched_string\":\"Archbishop Huxley\"},{\"gender\":\"u\",\"first_name\":\"\",\"title\":\"archbishop\",\"surnames\":\"Huxley\",\"start_index\":88,\"end_index\":105,\"matched_string\":\"Archbishop Huxley\"}]"
attr(,"Content-Type")
charset
"text/html" "utf-8"
Note that it returns two elements in the JSON string and neither one matches on the start_index or end_index. Is this a problem with encoding or something?
The simplePostToHost function in the httpRequest package might do what you are looking for here.