livecode is a R package for broadcasting code for live code demonstrations. I would like to use it for teaching R in online classrooms. The package can be installed from github and runs excellent on my computer and the result is very promising. Once installed from github:
remotes::install_github("rundel/livecode")
library(livecode)
You can start your web server via server = livecode::serve_file(). Next your browser pops up and shows you a syntax-hightlighted live broadcast of your coding. A benefit in comparison to a screenshare of code is that this broadcast delivers plain text which is scalable from the recipient. I tested it on my phone, worked great. No more scaling of RStudio in order to make people see your code. Furthermore text (the script) can be saved from browser.
https://www.rstudio.com/resources/rstudioconf-2020/livecode-broadcast-your-live-coding-sessions-from-and-to-rstudio/
My problem is, I cannot figure out how to configure my private work environment such that this broadcast can be received from outside my local network. serve_file can specify an ip and port. I tried to set it to my computer ip (found via ipconfig) and open port 80 (for http) in my router.
Can anybody support me in how to make a livecode broadcast to the global internet?
I contacted the package author and he suggested to follow this blogpost:
https://bitsandbricks.github.io/post/code-live-from-rstudio-and-share-it-with-the-world-in-real-time/
It suggests to use ngrok, a service that provides tunnels to localhost. This acutally worked for me for some seconds, killing my livecode package in the next moment. At the moment, I can no longer use livecode not even as a localhost. I assume this error may just be my fault. The blogpost provides an easy answer to my question.
I did some researches and video to figure out how to connect EEG devices which is Emotiv Insight in real-time to Microsoft Azure Machine Learning Studio.
I thought any ways to do it, perhaps I need to connect to other services before connecting to Azure studio.
My aim is making an app taken brainwave and use Azure studio to analyze it. Finally, data is saved to firebase and response to my app.
However, I am stucking to find a way to connect my EEG data to Azure. It is appreciated for anyone can help me.
Emotive use a service, running on the cortex process. You need a websocket to talk with it. Additionaly all communication to and from the service is using JSon objects. Then you need to transport that into Azure
how can I perform normal R-Code on a SQL Server without using the Microsoft rx-functions? I think the ComputeContext "RxInSqlServer" isn't the right one? But I couldn't find good Information about the other ComputeContext-options.
Is this possible with this Statement?
rxSetComputeContext(ComputeContext)
Or can I only use it to perform rx-functions? An other Option could be to set the Server Connection in RStudio or VisualStudio?
My Problem is: I want analyse data from hadoop via ODBC-Connection on the SQL Server, so I would like to use the performance of the remote SQL Server and not the data in SQL Server. And then I want analyse the hadoop-data with sparklyr.
Summary: I want to use the performance from the remote server and not the SQL Server data. So RStudio should run not local, it should perform and use the memory of the remote server.
Thanks!
The concept of a compute context in Microsoft R Server is, “Where will the computation be performed?”
When setting compute context, you are telling Microsoft R Server that computation will occur on either the local machine (with either “local” or “localpar” compute contexts), or, the script will be executed on a remote machine which has Microsoft R Server installed on it. Remote compute contexts are defined by creating a compute context object, and then setting the context to that object.
For SQL Server, you would create an RxInSqlServer() object, and then call rxSetComputeContext() on that object. For Hadoop, the object would be created via the RxHadoopMR() call.
In code, it would look something like:
CC <- RxHadoopMR( < context defined here > )
rxSetComputeContext(CC)
To see usage on defining a context, please see documentation (Enter "?RxHadoopMR" in the R Client, no quotes).
Any call to an "rx" function after this will be performed on the Hadoop cluster, with no data being transferred to the client; other than the results.
RxInSqlServer() would follow the same pattern.
Note: To perform any remote computation, Microsoft R Server must be installed on that machine.
If you wish to run a standard R function on a remote compute context, you must wrap that function in a call to rxExec(). rxExec() is desinged as an interface to parallelize any Open Source R function and allow for its execution on a remote context. Please see documentation (enter "?rxExec" in the R Client, no quotes) for usage.
For information on efficient parallelization, please see this blog: https://blogs.msdn.microsoft.com/microsoftrservertigerteam/2016/11/14/performance-optimization-when-using-rxexec-to-parallelize-algorithms/
You called out "without using the Microsoft rx-functions" and I am interpreting this as, "I would like to use Open Source R Algorithms on data in-SQL Server", with Microsoft R Server, you must use rxExec() as the interface to run Open Source R. If you want to use no rx functions at all, you will need to query the data to your local machine, and then use Open Source R. To interface with a remote context using Microsoft R Server, the bare minimum is using rxExec().
This is how you will be able to achieve the first part of your ask, "how can I perform normal R-Code on a SQL Server without using the Microsoft rx-functions? I think the ComputeContext "RxInSqlServer" isn't the right one?"
For your second ask, "My Problem is: I want analyse data from hadoop via ODBC-Connection on the SQL Server, so I would like to use the performance of the remote SQL Server and not the data in SQL Server. And then I want analyse the hadoop-data with sparklyr."
First, I'd like to comment that with the release of Microsoft R Server 9.1, you can use sparklyr in-line with an MRS Spark connection, for some examples, please see this blog: https://blogs.msdn.microsoft.com/microsoftrservertigerteam/2017/04/19/new-features-in-9-1-microsoft-r-server-with-sparklyr-interoperability/
Secondly, what you are trying to do is very involved. I can think of two ways that this is possible.
One is, if you have SQL Server PolyBase, you can configure SQL Server to make a virtual table referencing data in Hadoop, similar to Hive. After you have referenced your Hadoop data in SQl Server, you would use an RxInSqlServer() compute context on these tables. This would analyse the data in SQL Server, and return the results to the client.
Here is a detailed blog explaining an end-to-end setup on Cloudera and SQL Server: https://blogs.msdn.microsoft.com/microsoftrservertigerteam/2016/10/17/integrating-polybase-with-cloudera-using-active-directory-authentication/
The Second, which I would NOT recommend, is untested, hacky, and has the following prereqs:
1) Your Hadoop cluster must have OpenSSH installed and configured
2) Your SQL Server Machine must have the ability to SSH into your Hadoop Cluster
3) You must be able to place an SSH Key on your SQL Server machine in a directory which the R Services process has the ability to access
And I need to add another disclaimer here, there is No Guarantee of this working, and, likely, it will not work. The software was not designed to operate in this fashion.
You would then do the following:
On your client machine, you would define a custom function which contains the analysis that you wish to perform, this can be Open Source R Function, rx functions, or a mix.
In this custom function, before calling any other R or rx functions, you would define a RxHadoopMR compute context object which points to your cluster, referencing the SSH key in the directory on the SQL Server machine as if you were executing from that machine. (in the same way that you would define the RxHadoopMR object if you were to do a remote Hadoop operation from your client machine).
Within this custom function, immediately after RxHadoopMR() is defined, you would call rxSetComputeContext() on your defined RxHadoopMR() object
Still in this custom function, write the actual script which will operate on the data in Hadoop.
After this function is defined, you would define an RxInSqlServer() compute context object on the client machine.
You would set your compute context to RxInSqlServer()
Then you would call rxExec() with your custom function as an input.
What this will do is execute your custom function on the SQL Server machine, which would hopefully cause it to define its compute context as your Hadoop cluster, and pull the data over SSH for analysis on the SQL Server machine; returning the results to client.
With that said, this is not how Microsoft R Server was designed to be used, and if you wish to optimize performance, please use Option One and configure PolyBase.
You could say I am a fan of the Realm Mobile Platform. I'm using it and it seems to be working well.
However I am confused with how to operate it going to production. It seems to be deployed only to one server, and even the professional and enterprise editions are working on my single server.
Assuming Realm have thought of this (as Enterprise edition supports 'enterprise scaling) - how does this work if all clients point to my owned server URL?
Another question is how to monitor the load on that server.
Thanks!
The Professional Edition and the Enterprise Edition emit statsd compatible metrics which allow you to track the usage and load on each node in a Realm Object Server cluster. These metrics are also used internally inside the cluster in order to display statistics about the health of the cluster.
We are obviously still adding metrics as we understand more about our customer's use-cases, and fine-tuning the ones that we have.
With regards to the way the clustering works, we are currently implementing this according to an iterative process, where we add more and more features, and more and more resilience to the system with every passing day.
Basically, we have a logical load balancer process, which receives the incoming client connections, and then dispatches that to a node inside the cluster. This logical load balancer can be HA'd and LB'd itself as well, just like you would any regular WS connection handler. Handling many connections these days is easy. It's handling the quadratic merge algorithms that is expensive on the Realm Object Server, which is why the clustering is required for deployments at scale.
I am using vim-r-plugin to send commands from vim to a running R session (and get back info about object list and auto-completions).
My aim is to get communication between local vim and remote session of R. I manged to send commands to R with screen.vim plugin. However this type of communication was one-way only.
For a while I thought that this is not really possible (or at least not very easy to achieve) however I discovered one site: http://manuals.bioinformatics.ucr.edu/home/programming-in-r/vim-r
The author there mentions accessing remote R sessions from local vim multiple times:
"Flexible code sending options from local vim instances to R sessions on remote machines or among remote machines."
"The vim session can run on a local computer, while the R session can run on the same or a remote system."
However nowhere on that site is there any description telling how to achieve this exactly.
I also asked the same question directly on the gougle-group of vim-r-plugin, and the author replied with an option to run everything remotely: https://groups.google.com/forum/#!topic/vim-r-plugin/293VyyQntZ0 . I managed to do that, but it's not what I am after and I didn't want to bother him any further.
So my question: is it possible? If not directly - maybe there are work-arounds of not having to duplicate my vim configuration on all the remote servers I am using?
Reply after more than half of a year!
I tend to agree with others that it is a better idea to run everything remotely. However, this vimdoc may give what you want to achieve: