unable to create a cluster/plan to a remote server in R - r

Trying (still) to get a cluster running. I have two Linux (Ubuntu) computers, the server is "laptop". Pretty sure ssh works, with key authentication, I can ssh to "laptop" without a password from command line. ssh_exec_wait works, e.g.,
session <- ssh_connect("laptop")
a <- ssh_exec_wait(session, command = "whoami")
ssh_disconnect(session)
works. I've gone through the doc (A future for R), works fine until I get to cluster futures.
plan(cluster, workers = c("localhost"))
works fine, but
plan(cluster, workers = c("laptop"))
gives the error:
Warning: remote port forwarding failed for listen port 11968
List of 2
$ node_idx: int 1
$ node :List of 5
..$ con : 'sockconn' int 3
.. ..- attr(*, "conn_id")=<externalptr>
..$ host : chr "laptop"
.. ..- attr(*, "localhost")= logi FALSE
..$ rank : int 1
..$ rshlogfile : NULL
..$ session_info:List of 6
.. ..$ r :List of 15
.. .. ..$ platform : chr "x86_64-pc-linux-gnu"
.. .. ..$ arch : chr "x86_64"
.. .. ..$ os : chr "linux-gnu"
.. .. ..$ system : chr "x86_64, linux-gnu"
.. .. ..$ status : chr ""
.. .. ..$ major : chr "3"
.. .. ..$ minor : chr "6.3"
.. .. ..$ year : chr "2020"
.. .. ..$ month : chr "02"
.. .. ..$ day : chr "29"
.. .. ..$ svn rev : chr "77875"
.. .. ..$ language : chr "R"
.. .. ..$ version.string: chr "R version 3.6.3 (2020-02-29)"
.. .. ..$ nickname : chr "Holding the Windsock"
.. .. ..$ os.type : chr "unix"
.. ..$ system :List of 8
.. .. ..$ sysname : chr "Linux"
.. .. ..$ release : chr "5.3.0-46-generic"
.. .. ..$ version : chr "#38~18.04.1-Ubuntu SMP Tue Mar 31 04:17:56 UTC 2020"
.. .. ..$ nodename : chr "laptop"
.. .. ..$ machine : chr "x86_64"
.. .. ..$ login : chr "mark"
.. .. ..$ user : chr "mark"
.. .. ..$ effective_user: chr "mark"
.. ..$ libs : chr [1:4] "/home/mark/R/x86_64-pc-linux-gnu-library/3.6" "/usr/local/lib/R/site-library" "/usr/lib/R/site-library" "/usr/lib/R/library"
.. ..$ pkgs : NULL
.. ..$ pwd : chr "/home/mark"
.. ..$ process:List of 1
.. .. ..$ pid: int 2809
..- attr(*, "class")= chr "SOCKnode"
Error: Initialization of plan() failed, because the test future used for validation failed. The reason was: Unexpected result (of class ‘NULL’ != ‘FutureResult’) retrieved for ClusterFuture future (label = ‘future-plan-test’, expression = ‘NA’): . This suggests that the communication with ClusterFuture worker (‘SOCKnode’ #1) is out of sync.
I assume the first line is the real error message:
Warning: remote port forwarding failed for listen port 11968
Do I need to manually set the port?

In my experience, the warning
Warning: remote port forwarding failed for listen port 11968
is caused by a hanging R process on that port (it could be another process of course but it's likely that the future package created it). If you kill the process on laptop, the port should become available again.
You can check by manually running
ssh -R 11968:localhost:11968 laptop

Related

Get user detail from rstudio connect published shiny app into app.R code [duplicate]

I want to get User's IP, User's window ID and User's Server Access time
I can get User's IP with below code but Other things can't get....
getting User's IP code is
ip <- session$request$REMOTE_ADDR at server.R
How to get User's window ID and User's Server Access time ????
Long-story-short, there is a limit to what "you" (the app and therefore you, the dev) can know about the connection. I've put in similar requests for some more with RStudio professional support, and what I have below is the current capability. I hope (with no advertised timeline) that more will eventually be available.
I have RStudio Connect, which is technically a different product but similar underlying methods and I believe a lot of shared code.
Here's a shiny app that helps you "discover" things from session. I filter out most of the environment-like variables in session that are either packed full of functions or just not something useful. It's possible I've filtered out too much, you might want to play with it to see if there is more.
library(shiny)
ui <- bootstrapPage(
h3("Parsed query string"),
verbatimTextOutput("queryText"),
h3("URL components"),
verbatimTextOutput("sessionText"),
h3("EnvVars"),
verbatimTextOutput("envvarText")
)
server <- function(input, output, session) {
# Parse the GET query string
output$queryText <- renderText({
query <- parseQueryString(session$clientData$url_search)
# Return a string with key-value pairs
paste(names(query), query, sep = "=", collapse=", ")
})
# Return the components of the URL in a string:
output$sessionText <- renderText({
cls <- sapply(session, function(a) class(a)[1])
nms <- names(cls[ cls %in% c("list", "character", "numeric", "integer",
"NULL", "logical", "environment", "reactivevalues" ) ])
nms <- setdiff(nms, ".__enclos_env__")
paste(
capture.output(
str(
sapply(nms,
function(sessnm) {
if (inherits(session[[sessnm]], c("environment", "reactivevalues"))) {
sapply(names(session[[sessnm]]), function(nm) session[[sessnm]][[nm]], simplify = FALSE)
} else if (inherits(session[[sessnm]], c("character", "numeric", "integer"))) {
session[[sessnm]]
} else class(session[[sessnm]])
}, simplify = FALSE),
nchar.max = 1e5,
vec.len = 1e5
)
),
collapse = "\n"
)
})
# Dump the environment variables
output$envvarText <- renderText({
paste(
capture.output(
str(as.list(Sys.getenv()))
),
collapse = "\n"
)
})
}
shinyApp(ui, server)
Here's some sample output, cleansed for private information. (Context for this output: I changed the server's IP address to be 111.222.333.444, otherwise that should be the external IP address. Also, this is behind an nginx reverse proxy for a few reasons, so REMOTE_ADDR (for instance) is not very exciting ... but it normally might be for you.)
List of 10
$ groups : chr [1:45] "Admins" "Analytics" "Synology Desktop"
$ user : chr "r2evans"
$ userData : Named list()
$ singletons : chr(0)
$ request :List of 24
..$ HTTP_UPGRADE : chr "websocket"
..$ HTTP_CONNECTION : chr "Upgrade"
..$ HTTP_GUID : chr "_6433vf0"
..$ QUERY_STRING : chr ""
..$ httpuv.version :Classes 'package_version', 'numeric_version' hidden list of 1
.. ..$ : int [1:3] 1 5 2
..$ SERVER_NAME : chr "127.0.0.1"
..$ SCRIPT_NAME : chr ""
..$ SERVER_PORT : chr "40386"
..$ REMOTE_PORT : chr "52858"
..$ HTTP_SEC_WEBSOCKET_VERSION : chr "13"
..$ rook.input :Classes 'NullInputStream', 'R6' <NullInputStream>
Public:
close: function ()
read: function (l = -1L)
read_lines: function (n = -1L)
rewind: function ()
..$ PATH_INFO : chr "/websocket/"
..$ rook.version : chr "1.1-0"
..$ rook.errors :Classes 'ErrorStream', 'R6' <ErrorStream>
Public:
cat: function (..., sep = " ", fill = FALSE, labels = NULL)
flush: function ()
..$ HTTP_X_FORWARDED_FOR : chr "111.222.333.444:61928, ::1"
..$ HTTP_SHINY_SERVER_CREDENTIALS: chr "{\"user\":\"r2evans\",\"groups\":[\"Admins\",\"Analytics\",\"Synology Desktop\"]}"
..$ HTTP_SEC_WEBSOCKET_KEY : chr "NbavAciNEKYiS1AN/NGlVQ=="
..$ HTTP_SHINY_SHARED_SECRET : chr "65567ccb5db3d2990df150611eb022f4"
..$ REMOTE_ADDR : chr "127.0.0.1"
..$ HEADERS : Named chr [1:10] "Upgrade" "_6433vf0" "127.0.0.1:40386" "NbavAciNEKYiS1AN/NGlVQ==" "13" "{\"user\":\"r2evans\",\"groups\":[\"Admins\",\"Analytics\",\"Synology Desktop\"]}" "65567ccb5db3d2990df150611eb022f4" "websocket" "Go-http-client/1.1" "111.222.333.444!:61928, ::1"
.. ..- attr(*, "names")= chr [1:10] "connection" "guid" "host" "sec-websocket-key" "sec-websocket-version" "shiny-server-credentials" "shiny-shared-secret" "upgrade" "user-agent" "x-forwarded-for"
..$ rook.url_scheme : chr "http"
..$ REQUEST_METHOD : chr "GET"
..$ HTTP_USER_AGENT : chr "Go-http-client/1.1"
..$ HTTP_HOST : chr "127.0.0.1:40386"
$ closed : chr "logical"
$ token : chr "3a358fb6a0d42c688ea8693b67d448cb"
$ clientData :List of 13
..$ output_queryText_hidden : logi FALSE
..$ output_sessionText_hidden: logi FALSE
..$ output_envvarText_hidden : logi FALSE
..$ url_search : chr ""
..$ url_hash_initial : chr ""
..$ singletons : chr ""
..$ url_protocol : chr "https:"
..$ allowDataUriScheme : logi TRUE
..$ url_port : chr ""
..$ url_hostname : chr "server.mydomain.com"
..$ pixelratio : int 2
..$ url_hash : chr ""
..$ url_pathname : chr "/content/23/"
$ input : Named list()
$ progressStack:List of 10
..$ .__enclos_env__:<environment: 0x652f8e8>
..$ clone :function (deep = FALSE)
..$ as_list :function ()
..$ size :function ()
..$ peek :function ()
..$ pop :function ()
..$ push :function (..., .list = NULL)
..$ initialize :function (init = 20L)
..$ private :<environment: 0x652fbc0>
..$ self :<environment: 0x652f8e8>
And the environment variables:
List of 42
$ EDITOR : chr "vi"
$ HOME : chr "/home"
$ LANG : chr "en_US.UTF-8"
$ LD_LIBRARY_PATH : chr "/usr/lib/R/lib:/usr/lib/x86_64-linux-gnu:/usr/lib/jvm/default-java/jre/lib/amd64/server"
$ LN_S : chr "ln -s"
$ LOGNAME : chr "rstudio-connect"
$ MAKE : chr "make"
$ PAGER : chr "/usr/bin/pager"
$ PATH : chr "/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/snap/bin:/opt/rstudio-connect/ext/pandoc2"
$ PWD : chr "/opt/rstudio-connect/mnt/app"
$ R_ARCH : chr ""
$ R_BROWSER : chr "xdg-open"
$ R_BZIPCMD : chr "/bin/bzip2"
$ R_CONFIG_ACTIVE : chr "rsconnect"
$ R_DOC_DIR : chr "/usr/share/R/doc"
$ R_GZIPCMD : chr "/bin/gzip -n"
$ R_HOME : chr "/usr/lib/R"
$ R_INCLUDE_DIR : chr "/usr/share/R/include"
$ R_LIBS : chr "/opt/rstudio-connect/mnt/app/packrat/lib/x86_64-pc-linux-gnu/3.5.3:"
$ R_LIBS_SITE : chr "/usr/local/lib/R/site-library:/usr/lib/R/site-library:/usr/lib/R/library"
$ R_LIBS_USER : chr "~/R/x86_64-pc-linux-gnu-library/3.5"
$ R_PAPERSIZE : chr "letter"
$ R_PAPERSIZE_USER : chr "letter"
$ R_PDFVIEWER : chr "/usr/bin/xdg-open"
$ R_PLATFORM : chr "x86_64-pc-linux-gnu"
$ R_PRINTCMD : chr "/usr/bin/lpr"
$ R_RD4PDF : chr "times,inconsolata,hyper"
$ R_SESSION_TMPDIR : chr "/opt/rstudio-connect/mnt/tmp/Rtmp3WkNTY"
$ R_SHARE_DIR : chr "/usr/share/R/share"
$ RSTUDIO_PANDOC : chr "/opt/rstudio-connect/ext/pandoc2"
$ R_SYSTEM_ABI : chr "linux,gcc,gxx,gfortran,?"
$ R_TEXI2DVICMD : chr "/usr/bin/texi2dvi"
$ R_UNZIPCMD : chr "/usr/bin/unzip"
$ R_ZIPCMD : chr "/usr/bin/zip"
$ SED : chr "/bin/sed"
$ SHINY_PORT : chr "40386"
$ SHINY_SERVER_VERSION: chr "1.8.2-10"
$ SHLVL : chr "0"
$ TAR : chr "/bin/tar"
$ TMPDIR : chr "/opt/rstudio-connect/mnt/tmp"
$ USER : chr "rstudio-connect"
$ USERNAME : chr "rstudio-connect"
Ultimately, publish this app, visit it, and see what you find one your instance.
Not requested, but included for completeness: here's an equivalent plumber API hit:
library(jsonlite)
library(plumber)
#* #get /clientdata
function(req, res){
classes <- sapply(req, function(a) head(class(a), n=1))
classes <- classes[! classes %in% c("ErrorStream", "NullInputStream") ]
classes <- classes[ ! names(classes) %in% c("args", "httpuv.version") ]
lapply(setNames(nm=names(classes)), function(nm) req[[nm]])
}
With output:
{
"SERVER_NAME": [
"127.0.0.1"
],
"HTTP_X_FORWARDED_FOR": [
"111.222.333.555:54590, 127.0.0.1"
],
"HTTP_ACCEPT": [
"application/json"
],
"HTTP_DNT": [
"1"
],
"SERVER_PORT": [
"33632"
],
"rook.url_scheme": [
"http"
],
"REQUEST_METHOD": [
"GET"
],
"HTTP_X_ARR_LOG_ID": [
"e5bdaca6-dcd7-4ca0-87d5-0d54927244b5"
],
"rook.version": [
"1.1-0"
],
"HTTP_X_ARR_SSL": [
"..., CN=*.mydomain.com"
],
"postBody": [],
"HTTP_RSTUDIO_CONNECT_CREDENTIALS": [
"{\"user\":\"r2evans\",\"groups\":[\"Admins\",\"Analytics\",\"Synology Desktop\"]}"
],
"HTTP_USER_AGENT": [
"Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:77.0) Gecko/20100101 Firefox/77.0"
],
"SCRIPT_NAME": [
""
],
"cookies": [],
"HTTP_ACCEPT_LANGUAGE": [
"en-US,en;q=0.5"
],
"HTTP_PLUMBER_SHARED_SECRET": [
"..."
],
"HTTP_REFERER": [
"https://server.mydomain.com/content/28/__swagger__/"
],
"REMOTE_PORT": [
"48684"
],
"HTTP_ACCEPT_ENCODING": [
"gzip, deflate, br"
],
"HTTP_MAX_FORWARDS": [
"10"
],
"HTTP_X_ORIGINAL_URL": [
"/content/28/clientdata"
],
"HTTP_HOST": [
"127.0.0.1:33632"
],
"HEADERS": [
"application/json",
"gzip, deflate, br",
"en-US,en;q=0.5",
"1",
"127.0.0.1:33632",
"10",
"...",
"https://server.mydomain.com/content/28/__swagger__/",
"https://server.mydomain.com/content/28",
"{\"user\":\"r2evans\",\"groups\":[\"Admins\",\"Analytics\",\"Synology Desktop\"]}",
"Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:77.0) Gecko/20100101 Firefox/77.0",
"e5bdaca6-dcd7-4ca0-87d5-0d54927244b5",
"..., CN=*.mydomain.com",
"111.222.333.555:54590, 127.0.0.1",
"/content/28/clientdata",
"https://server.mydomain.com:443/content/28/clientdata"
],
"HTTP_X_RSC_REQUEST": [
"https://server.mydomain.com:443/content/28/clientdata"
],
"HTTP_RSTUDIO_CONNECT_APP_BASE_URL": [
"https://server.mydomain.com/content/28"
],
"PATH_INFO": [
"/clientdata"
],
"QUERY_STRING": [
""
],
"REMOTE_ADDR": [
"127.0.0.1"
]
}

R adding slots to existing S4 object

I am trying to follow this answer but getting lost with what to change for my example. Please can someone spell out specifically for the below?
I have the following S4 object called "classic":
> str(classic)
Formal class 'topGOresult' [package "topGO"] with 5 slots
..# description: chr "Custom Afol today"
..# score : Named num [1:7633] 0.5735 0.0883 0.8081 0.8005 0.4941 ...
.. ..- attr(*, "names")= chr [1:7633] "GO:0000002" "GO:0000003" "GO:0000018" "GO:0000027" ...
..# testName : chr "fisher"
..# algorithm : chr "classic"
.. ..- attr(*, "testClass")= chr "classicCount"
..# geneData : Named int [1:4] 9171 620 5 5361
.. ..- attr(*, "names")= chr [1:4] "Annotated" "Significant" "NodeSize" "SigTerms"
I am able to edit contents of existing slots, such as:
classic#description = "Custom Afol today"
However, I would like to add a new slot called "ontology" with character input of "BP". Is there a way to add a new slot to an existing object such as this?
Many thanks in advance for help.

First argument is not an open RODBC channel

I am connecting to an Oracle database via DNS (set up the connection on the tnsnames.ora with the name "database").
I am able to succesfully run the following code:
con <- DBI::dbConnect(odbc::odbc(),
"database",
UID = "user",
PWD = "password",
trusted_connection = TRUE)
I am also able to succesfully list all the tables in the database via:
list <- dbListTables(con)
View(list)
However, when I run:
results <- sqlQuery(channel = con, query = "select * from myschemaname.table")
I get the error:
Error in sqlQuery(channel = con, query = "select * from myschemaname.table") :
first argument is not an open RODBC channel
I have owner privileges and I am also able to update Excel Spreadsheets connected to this database via ODBC.
If it's of any use, here's the output of str(con)
str(con)
Formal class 'Oracle' [package ".GlobalEnv"] with 4 slots
..# ptr :<externalptr>
..# quote : chr "\""
..# info :List of 13
.. ..$ dbname : chr ""
.. ..$ dbms.name : chr "Oracle"
.. ..$ db.version : chr "11.02.0040"
.. ..$ username : chr "user"
.. ..$ host : chr ""
.. ..$ port : chr ""
.. ..$ sourcename : chr "database"
.. ..$ servername : chr "database"
.. ..$ drivername : chr "SQORA32.DLL"
.. ..$ odbc.version : chr "03.80.0000"
.. ..$ driver.version : chr "11.02.0001"
.. ..$ odbcdriver.version : chr "03.52"
.. ..$ supports.transactions: logi TRUE
.. ..- attr(*, "class")= chr [1:3] "Oracle" "driver_info" "list"
..# encoding: chr ""
You are using RODBC::sqlQuery() for a connection created with DBI::dbConnect(). Either use DBI::dbGetQuery() with DBI::dbConnect() or create a connection with RODBC::odbcConnect(), and use RODBC::sqlQuery().
Another context, but the same error:
If the table of your DB consists of numerics, you first of all have to load an object (nameofyourtable) of class xts:
>library(RODBC)
>library(DMwR)
>library(xts)
>data(nameofyourtable)
So I had something like this connecting to a SQL Server database. In the end I had to change the Trusted_Connection=TRUE to Trusted_Connection=Yes...

How to reference .. .. elements in list in R?

How do you access list elements prefaced with double dots .. ..?
I am trying to isolate the $attr part of the html to help in another question into a variable, and then place it another mime variable. x$parts$attr gives me NULL.
Thanks for your help.
EXAMPLE
library(gmailr)
test_email <- mime()
x <- html_body(mime = test_email, body = "I wish I had a nice body")
str(x)
List of 5
$ parts :List of 2
..$ : NULL
..$ :List of 4
.. ..$ parts : list()
.. ..$ header:List of 2
.. .. ..$ MIME-Version: chr "1.0"
.. .. ..$ Date : chr "Sun, 24 May 2015 09:44:59 GMT"
.. ..$ body : chr "I wish I had a nice body"
.. ..$ attr :List of 3 # << HOW do I get this?
.. .. ..$ content_type: chr "text/html"
.. .. ..$ charset : chr "utf-8"
.. .. ..$ encoding : chr "base64"
.. ..- attr(*, "class")= chr "mime"
Try this:
str(x$parts[[2]]$attr)
List of 3
$ content_type: chr "text/html"
$ charset : chr "utf-8"
$ encoding : chr "base64"

R understanding and access a nested object by reading from str

Hi I have a nested object which I think contains tables. I used str to see how it looks like,
> str(test)
Formal class 'CuffData' [package "cummeRbund"] with 5 slots
..# DB :Formal class 'SQLiteConnection' [package "RSQLite"] with 5 slots
.. .. ..# Id :<externalptr>
.. .. ..# dbname : chr "C:/temp/cuffData.db"
.. .. ..# loadable.extensions: logi TRUE
.. .. ..# flags : int 6
.. .. ..# vfs : chr ""
..# tables :List of 6
.. ..$ mainTable : chr "genes"
.. ..$ dataTable : chr "geneData"
.. ..$ expDiffTable : chr "geneExpDiffData"
.. ..$ featureTable : chr "geneFeatures"
.. ..$ countTable : chr "geneCount"
.. ..$ replicateTable: chr "geneReplicateData"
..# filters: list()
..# type : chr "genes"
..# idField: chr "gene_id"
however when I tried to to read the table mainTable all I get is this,
> test#tables$mainTable
[1] "genes"
is there any way I can read what is in this table?
Not sure if I'm even interpreting the str output correctly though. I'm a newbie at this and I'm also wondering if there is a good tutorial on how to interpret objects like these in R? Examples would be even better.
thanks you much in advance.
Ahdee

Resources