I thought I'd try to write an R interface to Scribe (mature OAuth library for Java by Pablo Fernandez) as a way of refreshing myself on Java (not used it in 8 years), learning rJava and to make better use of the Twitter API. But mostly because it's Friday afternoon and I thought it'd be fun. :)
Unfortunately I haven't got very far...
I downloaded the .jar file for scribe and also commons-condec (its only dependency, which I subsequently unzipped). I've ran the code in Java using netbeans and it works fine using his twitter example.
I was OK for the first few lines of code by just following the rJava documenation:
# load R packages
library(rJava)
# Initialise
.jinit()
# Add class paths
d1 <- "C:/Users/Tony/Documents/R/java/scribe-1.1.0.jar"
d2 <- "C:/Users/Tony/Documents/R/java/commons-codec-1.4/"
.jaddClassPath(path=c(d1, d2))
But then scribe quick start guide says the following is needed:
// Java Code
OAuthService service = new ServiceBuilder()
.provider(TwitterApi.class)
.apiKey("6icbcAXyZx67r8uTAUM5Qw")
.apiSecret("SCCAdUUc6LXxiazxH3N0QfpNUvlUy84mZ2XZKiv39s")
.build();
I can't figure out how to rewrite that into rJava parlance. A little web searching suggests I should do it in parts, so first I did:
# Create object (back to R code again)
( service <- .jnew("org.scribe.builder.ServiceBuilder") )
[1] "Java-Object{org.scribe.builder.ServiceBuilder#58fe64b9}"
# Set up apiKey and apiSecret using "$" shortcut
service$apiKey("6icbcAXyZx67r8uTAUM5Qw")
service$apiSecret("SCCAdUUc6LXxiazxH3N0QfpNUvlUy84mZ2XZKiv39s")
Good so far. Then I need to figure out what return type is expected from the provider function:
# Inspect return type
.jmethods(service, "provider")
[1] "public org.scribe.builder.ServiceBuilder org.scribe.builder.ServiceBuilder.provider(java.lang.Class)"
It needs "java.lang.Class". This is where I get confused. What does that mean? I guess, looking at the source, it needs a return type of type "ServiceBuilder", but how to do that? This was my best guess after looking at ?.jcall (note: 'use.true.class = TRUE' didn't do anything):
> .jcall(obj = service, returnSig = "Lorg.scribe.builder.ServiceBuilder;", method = "org.scribe.builder.ServiceBuilder.provider", "org.scribe.builder.api.TwitterApi")
Error in .jcall(obj = service, returnSig = "Lorg.scribe.builder.ServiceBuilder;", :
method org.scribe.builder.ServiceBuilder.provider with signature (Ljava/lang/String;)Lorg.scribe.builder.ServiceBuilder; not found
Any ideas?
It looks to me like the provider method returns ServiceBuilder and takes a Class as a parameter.
In Java if you put the classname followed by .class that makes a class literal object in the code. If you instead load the class using reflection you can refer to a class by its string name. I'm not sure how this works in R but in Java the syntax is:
Class c = Class.forName("org.scribe.builder.api.TwitterApi");
This puts the class instance into the variable c. Then you could call the provider method:
service$provider(c);
Related
I have a bunch of functions and I'm trying to keep my workspace clean by defining them in an environment and attaching the environment. Some of the functions are S3 generics, and they don't seem to play well with this approach.
A minimum example of what I'm experiencing requires 4 files:
testfun.R
ttt.xxx <- function(object) print("x")
ttt <- function(object) UseMethod("ttt")
ttt2 <- function() {
yyy <- structure(1, class="xxx")
ttt(yyy)
}
In testfun.R I define an S3 generic ttt and a method ttt.xxx, I also define a function ttt2 calling the generic.
testenv.R
test_env <- new.env(parent=globalenv())
source("testfun.R", local=test_env)
attach(test_env)
In testenv.R I source testfun.R to an environment, which I attach.
test1.R
source("testfun.R")
ttt2()
xxx <- structure(1, class="xxx")
ttt(xxx)
test1.R sources testfun.R to the global environment. Both ttt2 and a direct function call work.
test2.R
source("testenv.R")
ttt2()
xxx <- structure(1, class="xxx")
ttt(xxx)
test2.R uses the "attach" approach. ttt2 still works (and prints "x" to the console), but the direct function call fails:
Error in UseMethod("ttt") :
no applicable method for 'ttt' applied to an object of class "xxx"
however, calling ttt and ttt.xxx without arguments show that they are known, ls(pos=2) shows they are on the search path, and sloop::s3_dispatch(ttt(xxx)) tells me it should work.
This questions is related to Confusion about UseMethod search mechanism and the link therein https://blog.thatbuthow.com/how-r-searches-and-finds-stuff/, but I cannot get my head around what is going on: why is it not working and how can I get this to work.
I've tried both R Studio and R in the shell.
UPDATE:
Based on the answers below I changed my testenv.R to:
test_env <- new.env(parent=globalenv())
source("testfun.R", local=test_env)
attach(test_env)
if (is.null(.__S3MethodsTable__.))
.__S3MethodsTable__. <- new.env(parent = baseenv())
for (func in grep(".", ls(envir = test_env), fixed = TRUE, value = TRUE))
.__S3MethodsTable__.[[func]] <- test_env[[func]]
rm(test_env, func)
... and this works (I am only using "." as an S3 dispatching separator).
It’s a little-known fact that you must use .S3method() to define methods for S3 generics inside custom environments (outside of packages).1 The reason almost nobody knows this is because it is not necessary in the global environment; but it is necessary everywhere else since R version 3.6.
There’s virtually no documentation of this change, just a technical blog post by Kurt Hornik about some of the background. Note that the blog post says the change was made in R 3.5.0; however, the actual effect you are observing — that S3 methods are no longer searched in attached environments — only started happening with R 3.6.0; before that, it was somehow not active yet.
… except just using .S3method will not fix your code, since your calling environment is the global environment. I do not understand the precise reason why this doesn’t work, and I suspect it’s due to a subtle bug in R’s S3 method lookup. In fact, using getS3method('ttt', 'xxx') does work, even though that should have the same behaviour as actual S3 method lookup.
I have found that the only way to make this work is to add the following to testenv.R:
if (is.null(.__S3MethodsTable__.)) {
.__S3MethodsTable__. <- new.env(parent = baseenv())
}
.__S3MethodsTable__.$ttt.xxx <- ttt.xxx
… in other words: supply .GlobalEnv manually with an S3 methods lookup table. Unfortunately this relies on an undocumented S3 implementation detail that might theoretically change in the future.
Alternatively, it “just works” if you use ‘box’ modules instead of source. That is, you can replace the entirety of your testenv.R by the following:
box::use(./testfun[...])
This code treats testfun.R as a local module and loads it, attaching all exported names (via the attach declaration [...]).
1 (and inside packages you need to use the equivalent S3method namespace declaration, though if you’re using ‘roxygen2’ then that’s taken care of for you)
First of all, my advice would be: don't try to reinvent R packages. They solve all the problems you say you are trying to solve, and others as well.
Secondly, I'll try to explain what went wrong in test2.R. It calls ttt on an xxx object, and ttt.xxx is on the search list, but is not found.
The problem is how the search for ttt.xxx happens. The search doesn't look for ttt.xxx in the search list, it looks for it in the environment from which ttt was called, then in an object called .__S3MethodsTable__.. I think there are two reasons for this:
First, it's a lot faster. It only needs to look in one or two places, and the table can be updated whenever a package is attached or detached, a relatively rare operation.
Second, it's more reliable. Each package has its own methods table, because two packages can use the same name for generics that have nothing to do with each other, or can use the same class names that are unrelated. So package code needs to be able to count on finding its own definitions first.
Since your call to ttt() happens at the top level, that's where R looks first for ttt.xxx(), but it's not there. Then it looks in the global .__S3MethodsTable__. (which is actually in the base environment), and it's not there either. So it fails.
There is a workaround that will make your code work. If you run
.__S3MethodsTable__. <- list2env(list(ttt.xxx = ttt.xxx))
as the last line of testenv.R, then you'll create a methods table in the global environment. (Normally there isn't one there, because that's user space, and R doesn't like putting things there unless the user asks for it.)
R will find that methods table, and will find the ttt.xxx method that it defines. I wouldn't be surprised if this breaks some other aspect of S3 dispatch, so I don't recommend doing it, but give it a try if you insist on reinventing the package system.
I'm trying to speed up my R code using future package by using mutlicore plan on Linux. In future definition I'm creating a java object and trying to pass it to .jcall(), But I'm getting a null value for java object in future. Could anyone please help me out to resolve this. Below is sample code-
library("future")
plan(multicore)
library(rJava)
.jinit()
# preprocess is a user defined function
my_value <- preprocess(a = value){
# some preprocessing task here
# time consuming statistical analysis here
return(lreturn) # return a list of 3 components
}
obj=.jnew("java.custom.class")
f <- future({
.jcall(obj, "V", "CustomJavaMethod", my_value)
})
Basically I'm dealing with large streaming data. In above code I'm sending the string of streaming data to user defined function for statistical analysis and returning the list of 3 components. Then want to send this list to custom java class [ java.custom.class ]for further processing using custom Java method [ CustomJavaMethod ].
Without using future my code is running fine. But I'm getting 12 streaming records in one minute and then my code is getting slow, observed delay in processing.
Currently I'm using Unix with 16 cores. After using future package my process is done fast. I have traced back my code, in .jcall something happens wrong.
Hope this clarifies my pain.
(Author of the future package here:)
Unfortunately, there are certain types of objects in R that cannot be sent to another R process for further processing. To clarify, this is a limitation to those type of objects - not to the parallel framework use (here the future framework). This simplest example of such an objects may be a file connection, e.g. con <- file("my-local-file.txt", open = "wb"). I've documented some examples in Section 'Non-exportable objects' of the 'Common Issues with Solutions' vignette (https://cran.r-project.org/web/packages/future/vignettes/future-4-issues.html).
As mentioned in the vignette, you can set an option (*) such that the future framework looks for these type of objects and gives an informative error before attempting to launch the future ("early stopping"). Here is your example with this check activated:
library("future")
plan(multisession)
## Assert that global objects can be sent back and forth between
## the main R process and background R processes ("workers")
options(future.globals.onReference = "error")
library("rJava")
.jinit()
end <- .jnew("java/lang/String", " World!")
f <- future({
start <- .jnew("java/lang/String", "Hello")
.jcall(start, "Ljava/lang/String;", "concat", end)
})
# Error in FALSE :
# Detected a non-exportable reference ('externalptr') in one of the
# globals ('end' of class 'jobjRef') used in the future expression
So, yes, your example actually works when using plan(multicore). The reason for that is that 'multicore' uses forked processes (available on Unix and macOS but not Windows). However, I would try my best to limit your software to parallelize only on "forkable" systems; if you can find an alternative approach I would aim for that. That way your code will also work on, say, a huge cloud cluster.
(*) The reason for these checks not being enabled by default is (a) it's still in beta testing, and (b) it comes with overhead because we basically need to scan for non-supported objects among all the globals. Whether these checks will be enabled by default in the future or not, will be discussed over at https://github.com/HenrikBengtsson/future.
The code in the question is calling unknown Method1 method, my_value is undefined, ... hard to know what you are really trying to achieve.
Take a look at the following example, maybe you can get inspiration from it:
library(future)
plan(multicore)
library(rJava)
.jinit()
end = .jnew("java/lang/String", " World!")
f <- future({
start = .jnew("java/lang/String", "Hello")
.jcall(start, "Ljava/lang/String;", "concat", end)
})
value(f)
[1] "Hello World!"
I'm new to R/web services.
I'm trying to use R's SSOAP package to get/post soap messages.
I want to generate functions definitions using the genSOAPCLientInterface by way of the processWSDL function.
when i run the processWSDL function, i get the following error message, which i dont know how to interpret:
Error in SOAPType(el, namespaceDefs=namespaceDefinitions):could not find function "SOAPType"
any help/pointers would be very helpful.
wsdl_doc<-xmlParse(paste0(wsdl_url, '?wsdl')
definitions<-processWSDL(wsdl_doc, verbose=T, useInternalNodes=T)
Error in SOAPType(el, namespaceDefs = namespaceDefinitions) :
could not find function "SOAPType"
SOAPType is a class used in SSOAP package, but it is declared in package XMLSchema. And for some reason it is not an exported class.
This workaround helped me to make the function processWSDL() working for me:
SOAPType = XMLSchema:::SOAPType
def <- processWSDL(doc)
Normally if a is a data.frame then one can autocomplete the column names by doing a$ tab. The chunked package has a nice feature where if you run
a <- chunked::read_csv_chunkwise("some.csv")
then when you type a[ then tab then it will show a list of variable via autocompletion even though a is not a data.frame.
I was trying to replicate this for my own code but I couldn't find any relevant resources after googling for "rstudio autocompletion" and various other searches.
I note that class(a) returns
[1] "chunkwise" "tbl"
I had a look at all the functions that belong to the S3 class "chunked" and I note that it has a method called tbl_vars, so I thought maybe that's what Rstudio uses to do the autocomplete.
So to test it out I tried
write.csv(data.frame(a = 1, b = 2), file = "test.csv",row.names = F)
tbl_vars.test_auto_complete <- function(fs) {
names(fread(fs$path))
}
test_auto_complete <- list(path = "test.csv")
class(test_auto_complete) <- "test_auto_complete"
tbl_vars(test_auto_complete)
[1] "a" "b"
But then when I type test_auto_complete tab the auto-complete doesn't show the variables that I want.
How can we give hints to Rstudio to make auto-completion work?
For objects that inherit from the tbl class, RStudio does indeed call tbl_vars() to populate completions. (This is an RStudio-specific autocompletion system feature.)
In your example, the object you're creating does not inherit from tbl, so this autocompletion pathway doesn't kick in.
However, this form of 'ad-hoc' S3 dispatch (where you define S3 methods directly as code like this) is not detected by RStudio, so you won't be able to verify this with test code like this. You'll have to explicitly define and register the S3 method in an R package.
Alternatively, you can try explicitly registering the S3 method with something like:
registerS3method("tbl_vars", "test_auto_complete", tbl_vars.test_auto_complete)
for inline testing.
I am trying to call a method in java class with rJava for a few days and I did not yet figure it out what I am doing wrong. Maybe someone here will have a some clues for me.
The situation looks like this:
I load library and initializing an object (that's works fine):
library(rJava)
.jinit('C:/javatemp/worker.jar')
jobject <- .jnew("worker.concrete")
I list methods and I get fine result:
.jmethods(jobject)
> [1] "public java.util.List worker.concrete.lookup(java.lang.CharSequence)"
I am preparing input structure which also works fine:
word <- .jnew("java/lang/String", "a word")
input = .jcast(word, "java/lang/CharSequence", check = TRUE)
However when I am trying to execute the method I get an error that such method does not exist...
out = .jcall(jobject,"Ljava/util/List","lookup",input)
> Error in .jcall(jobject, "Ljava/util/List", "lookup", input) :
method lookup with signature (Ljava/lang/CharSequence;)Ljava/util/List not found
Does anyone have an idea how to call such method?
Sorry for answering an old question, but this has bugged me as well for some time. The answer is: ;
The format of type specification for non-primitive return types is Lpackage/subpackage/Type; - it has to end with a semicolon. So in the example above, you would need:
out = .jcall(jobject,"Ljava/util/List;","lookup",input)