Callback style API on socket connection readable in R - r

I have an R script which consumes a feed periodically and then produces signals or actions, which need to be sent to a TCP server implemented in other language. I have handled the signal generation part in this fashion:
sendViaSocket <- function(object, socket) {
byteLen <- as.integer(object$byteLen())
writeBin(
byteLen, socket,
endian = "little",
useBytes = TRUE
)
writeBin(
object$toRaw(), socket,
useBytes = TRUE
)
}
readFromSocket <- function(socket) {
packetLen <- readBin (
socket, integer(), endian = "little"
)
rawVec <- readBin(
socket, "raw", n = packetLen
)
return (
parse (messages.MyMessage, rawVec)
)
}
getOHLC <- function() {
# feed reading function
...
}
generateSignal <- function (candleStick) {
# signal generating function
...
}
performAction <- function(message) {
# Action on server message
}
clientLoop <- function() {
socket <- socketConnection(
host = 'localhost',
port = 42000,
blocking = TRUE,
server = FALSE,
open = "a+b"
)
while(TRUE) {
candleStick <- getOHLC()
signal <- generateSignal(candleStick)
sendViaSocket(signal, socket)
# Reading any message from server
object <- readFromSocket(message)
Sys.sleep(10)
}
}
clientLoop()
I have deliberately set the socket connection as blocking as I want to read all the data available in the connection in one go, and send message in one go.
Currently I am reading any notification from client loop itself, but if the server does not send anything, the client loop is blocked.
What I want to do is have a callback when the socket has received any data, and then process that data in the callback. This will not block the client loop. Something along the following lines
onServerDataReceived <- function (serverMessage) {
# Perform related action on receiving server message
}
socket$onDataReceived(callback = onServerDataReceived)
Is there a way in which I can use a client socket connection with a callback when the server actually sends a message/ or do this in a reactive / asynchronous way?
Edit: As a bonus I also don't want to do while (TRUE) {...} and use an API or package that provides a better way to call the clientLoop() function periodically every n seconds (minimum interval is 30 seconds)

Related

How to read incomming messages using svSocket Server in R

I am using svSocket package in R to create a socket server. I have successfully created server using startSocketServer(...). I am able to connect my application to the server and send data from server to the application. But I am struggeling with reading of messages sent by application. I couldn't find any example for that on internet. I found only processSocket(...) example in documentation of vsSocket (see below) which describes the function that processes a command coming from the socket. But I want only read socket messages comming to the server in repeat block and print them on the screen for testing.
## Not run:
# ## A simple REPL (R eval/process loop) using basic features of processSocket()
# repl <- function ()
# {
# pars <- parSocket("repl", "", bare = FALSE) # Parameterize the loop
# cat("Enter R code, hit <CTRL-C> or <ESC> to exit\n> ") # First prompt
# repeat {
# entry <- readLines(n = 1) # Read a line of entry
# if (entry == "") entry <- "<<<esc>>>" # Exit from multiline mode
# cat(processSocket(entry, "repl", "")) # Process the entry
# }
# }
# repl()
# ## End(Not run)
Thx for your input.
EDIT:
Here more specific example of socket server creation and sending message:
require(svSocket)
#start server
svSocket::startSocketServer(
port = 9999,
server.name = "test_server",
procfun = processSocket,
secure = FALSE,
local = FALSE
)
#test calls
svSocket::getSocketClients(port = 9999) #ip and port of client connected
svSocket::getSocketClientsNames(port = 9999) #name of client connected
svSocket::getSocketServerName(port = 9999) #name of socket server given during creation
svSocket::getSocketServers() #server name and port
#send message to client
svSocket::sendSocketClients(
text = "send this message to the client",
sockets = svSocket::getSocketClientsNames(port = 9999),
serverport = 9999
)
... and response of the code above is:
> require(svSocket)
>
> #start server
> svSocket::startSocketServer(
+ port = 9999,
+ server.name = "test_server",
+ procfun = processSocket,
+ secure = FALSE,
+ local = FALSE
+ )
[1] TRUE
>
> #test calls
> svSocket::getSocketClients(port = 9999) #ip and port of client connected
sock0000000005C576B0
"192.168.2.1:55427"
> svSocket::getSocketClientsNames(port = 9999) #name of client connected
[1] "sock0000000005C576B0"
> svSocket::getSocketServerName(port = 9999) #name of socket server given during creation
[1] "test_server"
> svSocket::getSocketServers() #server name and port
test_server
9999
>
> #send message to client
> svSocket::sendSocketClients(
+ text = "send this message to the client",
+ sockets = svSocket::getSocketClientsNames(port = 9999),
+ serverport = 9999
+ )
>
What you can see is:
successfull creation of socket server
successfull connection of external client sock0000000005C576B0 (192.168.2.1:55427) to the server
successfull sending of message to the client (here no explizit output is provided in console, but the client reacts as awaited
what I am still not able to implement is to fetch client messages sent to the server. Could somebody provide me an example on that?
For interaction with the server from the client side, see ?evalServer.
Otherwise, it is your processSocket() function (either the default one, or a custom function you provide) that is the entry point triggered when the server got some data from one connected client. From there, you have two possibilities:
The simplest one is just to use the default processSocket() function. Besides some special code between <<<>>>, which is interpreted as special commands, the default version will evaluate R code on the server side. So, just call the function you want on the server. For instance, define f <- function(txt) paste("Fake process ", txt) on the server, and call evalServer(con, "f('some text')") on the client. Your custom f() function is executed on the server. Just take care that you need to double quote expressions that contain text here.
An alternate solution is to define your own processSocket() function to capture messages sent by the client to the server earlier. This is safer for a server that needs to process a limited number of message types without parsing and evaluating R code received from the client.
Now, the server is asynchronous, meaning that you still got the prompt available on the server, while it is listening to client(s) and processing their requests.

MOAR process ballooning while running Perl6 socket server

I have a socket server using IO::Socket::Async and Redis::Async for message publishing. Whenever there is a message received by the server, the script would translate the message and generate acknowledge message to be sent back to the sender so that the sender would send subsequent messages. Since translating the message is quite expensive, the script would run that portion under a 'start' method.
However, I noticed that the Moar process eating my RAM as the script is running. Any thought where should I look to solve this issue? Thanks!
https://pastebin.com/ySsQsMFH
use v6;
use Data::Dump;
use experimental :pack;
use JSON::Tiny;
use Redis::Async;
constant $SOCKET_PORT = 7000;
constant $SOCKET_ADDR = '0.0.0.0';
constant $REDIS_PORT = 6379;
constant $REDIS_ADDR = '127.0.0.1';
constant $REDIS_AUTH = 'xxxxxxxx';
constant $IDLING_PERIOD_MIN = 180 - 2; # 3 minutes - 2 secs
constant $CACHE_EXPIRE_IN = 86400; # 24h hours
# create socket
my $socket = IO::Socket::Async.listen($SOCKET_ADDR, $SOCKET_PORT);
# connnect to Redis ...
my $redis;
try {
my $error-code = "110";
$redis = Redis::Async.new("$SOCKET_ADDR:$SOCKET_PORT");
$redis.auth($REDIS_AUTH);
CATCH {
default {
say "Error $error-code ", .^name, ': Failed to initiate connection to Redis';
exit;
}
}
}
# react whenever there is connection
react {
whenever $socket -> $conn {
# do something when the connection wants to talk
whenever $conn.Supply(:bin) {
# only process if data length is either 108 or 116
if $_.decode('utf-8').chars == 108 or $_.decode('utf-8').chars == 116 {
say "Received --> "~$_.decode('utf-8');
my $ack = generateAck($_.decode('utf-8')); # generate ack based on received data
if $ack {
$conn.print: $ack;
}else{
say "No ack. Received data maybe corrupted. Closing connection";
$conn.close;
}
}
}
}
CATCH {
default {
say .^name, ': ', .Str;
say "handled in $?LINE";
}
}
}
### other subroutines down here ###
The issue was using the Async::Redis. Jonathon Stowe had fixed the Redis module so I'm using Redis module with no issue.

Testing RJDBC connections with time limit

How can I test if a RJDBC connection is working? With time limit?
Here is a toy code:
library("RJDBC")
testConnection <- function(){
tryCatch({
conn <- RJDBC::dbConnect(JDBC("com.mysql.jdbc.Driver", "driverPath"), "connString", "user", "password")
dbDisconnect(conn)
return("Connection: OK")
}
, error = function(e) {
return("Connection: Failed")
}
)
}
testConnection()
But if there is no network connection, I have to wait for 30 seconds to have the result.
I need to test many connections, so I would like to change the test limit to 2 seconds.
How can I set a evaluation time limit?
I tried setTimeLimit without success.

Connecting to socket.io socket with R

I am trying to connect to a socket.io socket in R using the R function socketConnection(). However, although I am able to set up the socket properly, I am not able to read data from it into R.
The javascript code I use to set up the server is:
var app = require('http').createServer(handler)
var io = require('socket.io')(app);
var fs = require('fs');
app.listen(8005);
function handler (req, res) {
fs.readFile(__dirname + '/index.html',
function (err, data) {
if (err) {
res.writeHead(500);
return res.end('Error loading index.html');
}
res.writeHead(200);
res.end(data);
});
}
io.on('connection', function (socket) {
setInterval(function() {
socket.emit('update', "test")
}, 1000);
});
The code for index.html is:
<script src="/socket.io/socket.io.js"></script>
<script>
var socket = io('http://localhost:8005');
socket.on('update', function (data) {
console.log(data);
});
</script>
I am able to verify that the socket is indeed working by going to localhost:8005 in my web browser and opening up the Developer's Console, where I can see "test" being logged. However, when I try to connect to the socket in R by:
sock <- socketConnection("localhost", 8005, blocking = FALSE, open = "r")
readLines(sock)
Every time I run readLines(sock) I get an empty character vector back. I have also confirmed I can read from other sockets in R by executing the following and indeed getting a response from readLines(sock):
sock <- socketConnection("rstudio.com", 6789, blocking = FALSE, open = "r")
readLines(sock)
Any help would be greatly appreciated. Thanks!
UPDATE (2015-09-01):
Thanks to the excellent help from Aaron Dufour, I was able to adapt this net server demo to stream data into R. The javascript code I used for the server was
var net = require('net');
var server = net.createServer(function(socket) {
setInterval(function() {
socket.write('Test\r\n');
socket.pipe(socket);
}, 1000)
});
server.listen(1337, '127.0.0.1');
And the R code was:
sock <- socketConnection("localhost", 1337, open = "r")
readLines(sock)
close(sock)
I did get the following warning warning: possible EventEmitter memory leak detected. 11 end listeners added. Use emitter.setMaxListeners() to increase limit. sometimes on the server side when I ran readLines(socket)
Also, when I ran close(socket) in R the server crashed and I get the following error: Error: This socket has been ended by the other party
With additional research, I think both the warning and error are preventable.
As I've described here, the socket.io protocol is much more than a WebSocket. Just opening a WebSocket to it won't work.
But socketConnection appears to not even be a WebSocket, but rather a raw socket. You're getting nothing back because the server is talking HTTP and you haven't finished sending an HTTP request.
You probably need a socket.io library for R, or to switch away from socket.io on the server.

Download large file with LuaSocket's HTTP module while keeping UI responsive

I would like to use LuaSocket's HTTP module to download a large file while displaying progress in the console and later on in a GUI. The UI must never block, not even when the server is unresponsive during the transfer. Additionally, creating a worker thread to handle the download is not an option.
Here's what I got so far:
local io = io
local ltn12 = require("ltn12")
local http = require("socket.http")
local fileurl = "http://www.example.com/big_file.zip"
local fileout_path = "big_file.zip"
local file_size = 0
local file_down = 0
-- counter filter used in ltn12
function counter(chunk)
if chunk == nil then
return nil
elseif chunk == "" then
return ""
else
file_down = file_down + #chunk
ui_update(file_size, file_down) -- update ui, run main ui loop etc.
return chunk -- return unmodified chunk
end
end
-- first request
-- determine file size
local r, c, h = http.request {
method = "HEAD",
url = fileurl
}
file_size = h["content-length"]
-- second request
-- download file
r, c, h = http.request {
method = "GET",
url = fileurl,
-- set our chain, count first then write to file
sink = ltn12.sink.chain(
counter,
ltn12.sink.file(io.open(fileout_path, "w"))
)
}
There are a few problems with the above, ignoring error checking and hard-coding:
It requires 2 HTTP requests when it is possible with only 1 (a normal GET request also sends content-length)
If the server is unresponsive, then the UI will also be unresponsive, as the filter only gets called when there is data to process.
How could I do this making sure the UI never blocks?
There is an example on non-preemptive multithreading in Programming in Lua that uses non-blocking luasocket calls and coroutines to do a multiple parallel downloads. It should be possible to apply the same logic to your process to avoid blocking. I can only add that you should consider calling this logic from IDLE event in your GUI (if there is such a thing) to avoid getting "attempt to yield across metamethod/c-call boundary" errors.

Resources