I'm trying to run an R script, which can be found here with the command
plot_trace.R -m ./log.model.csv ./log.trace.csv.
The two csv files can be found here.
This produces the error:
Error in `:=`(variable, as.character(variable)) :
Check that is.data.table(DT) == TRUE. Otherwise, := and `:=`(...) are defined for use in j, once only and in particular ways. See help(":=").
Calls: [ -> [.tbl_df -> check_names_df -> :=
Last but not least my sessionInfo() output:
R version 3.4.4 (2018-03-15)
Platform: x86_64-pc-linux-gnu (64-bit)
Running under: Ubuntu 18.04.3 LTS
Matrix products: default
BLAS: /usr/lib/x86_64-linux-gnu/blas/libblas.so.3.7.1
LAPACK: /usr/lib/x86_64-linux-gnu/lapack/liblapack.so.3.7.1
locale:
[1] LC_CTYPE=en_US.UTF-8 LC_NUMERIC=C
[3] LC_TIME=en_US.UTF-8 LC_COLLATE=en_US.UTF-8
[5] LC_MONETARY=en_US.UTF-8 LC_MESSAGES=en_US.UTF-8
[7] LC_PAPER=en_US.UTF-8 LC_NAME=C
[9] LC_ADDRESS=C LC_TELEPHONE=C
[11] LC_MEASUREMENT=en_US.UTF-8 LC_IDENTIFICATION=C
attached base packages:
[1] stats graphics grDevices utils datasets methods base
loaded via a namespace (and not attached):
[1] compiler_3.4.4
Thanks
UPDATE
I found two problems with this script:
The author mixes functions from tidyr and variables of class data.table. These functions will likely return data frames and using data.table syntax on these will cause errors. Unfortunately, some tidyr functions may return data.table when operating on one, but this seems to vary greatly depending on the version of the tidyr package.
The unnest function from tidyr has a new interface. The script won't work as is with tidyr version 1.0.0 or later.
Below are the lines I changed to fix the script, using R version 3.6.1 and tidyr version 1.0.0:
line 56: added trace.df <- as.data.table(trace.df)
line 64: changed to trace.df <- unnest(trace.df, cols)
line 65: added trace.df <- as.data.table(trace.df)
Original answer
I was able to run the script on the data you provided without error. Both
Rscript --vanilla plot_trace.R -m ./log.model.csv ./log.trace.csv
and
chmod +x plot_trace.R
./plot_trace.R -m ./log.model.csv ./log.trace.csv
produced the following plot:
Can you run the script step by step in RStudio to check the class of trace.df after each step?
It looks like the unnest call is creating something that is not a data table.
At line 56 try adding:
trace.df <- as.data.table(trace.df)
Related
I installed Microsoft-R-Open 3.4.0 on a Red Hat Linux Enterprise 7.3 machine following the instructions https://mran.microsoft.com/documents/rro/installation/ .
R starts up and seems to be doing fine at first sight. However, when I try to list the files in a directory with the command
files <- list.files(path="/home/username/directory_name/", pattern="*.Rda",, full.names=T, recursive=FALSE)
I get the error
translateCharUTF8' must be called on a CHARSXP
Execution halted
On my local windows machine the command works fine.
Googling this nothing really comes up except that the installation might be broken.
The strange thing is, that if I copy and paste the command into R and execute it, it does not work, but if I copy and paste it into R and change that command in a way that should not change its result but only add spaces or something like that it might run. E.g. changing it to
files <- list.files(path = "/home/username/directory_name/", pattern = "*.Rda",, full.names = T, recursive = FALSE)
might work, might not work and return the same error, or might execute but when I then type "files" that might return
[1]Error: 'getCharCE' must be called on a CHARSXP
When using R from the R-foundation (https://www.r-project.org/, installed via EPEL), I get the same error and behaviour.
The command sessionInfo() returns the following:
sessionInfo()
R version 3.3.2 (2016-10-31)
Platform: x86_64-redhat-linux-gnu (64-bit)
Running under: Red Hat Enterprise Linux Server 7.3 (Maipo)
locale:
[1] LC_CTYPE=en_US.UTF-8 LC_NUMERIC=C
[3] LC_TIME=en_US.UTF-8 LC_COLLATE=en_US.UTF-8
[5] LC_MONETARY=en_US.UTF-8 LC_MESSAGES=en_US.UTF-8
[7] LC_PAPER=en_US.UTF-8 LC_NAME=C
[9] LC_ADDRESS=C LC_TELEPHONE=C
[11] LC_MEASUREMENT=en_US.UTF-8 LC_IDENTIFICATION=C
attached base packages:
[1] stats graphics grDevices utils datasets methods base
uname -mrs gives me:
Linux 3.10.0-514.el7.x86_64 x86_64
Any help would be greatly appreciated,
best regards
Stefan
Your files have names containing accented characters. Changing them to pure-ASCII names should fix the problem.
For Example
unicode_and_raw_filename = paste0("/tmp/\u1234", as.raw("A"))
training_rows <<- read.csv(unicode_and_raw_filename, header = FALSE)
produces:
'translateCharUTF8' must be called on a CHARSXP, but got 'raw'
Looks like R internals has some spaghetti code in charset conversion for Latin2, ISO_8859-2, UTF-8, and CP1252: https://stat.ethz.ch/R-manual/R-devel/library/base/html/iconv.html
Since I updated R to the new version 3.4.0, the Rcpp plugin to enable C++11 does not seem to work properly. (Note that this is a different situation as reported here http://r.789695.n4.nabble.com/R-3-4-has-broken-C-11-support-td4732692.html)
I have C++ files requiring C++11 standard that I compile in R via Rcpp using the command Rcpp::sourceCpp(foo.cpp). The R default standard is C++98 but I could change that by adding the following command in my C++ file
// [[Rcpp::plugins("cpp11")]]
This does not work anymore with the new version of R 3.4.0, my compiler use the default standard (C++98) instead of C++11, despite the plugin. It used to work with previous version of R.
My solution for the moment is to set an environment variable for my R session:
Sys.setenv("PKG_CXXFLAGS"="-std=c++11")
Here is the configuration I use in R:
sessionInfo()
R version 3.4.0 (2017-04-21)
Platform: x86_64-suse-linux-gnu (64-bit)
Running under: openSUSE Leap 42.2
Matrix products: default
BLAS: /usr/lib64/R/lib/libRblas.so
LAPACK: /usr/lib64/R/lib/libRlapack.so
locale:
[1] LC_CTYPE=en_US.UTF-8 LC_NUMERIC=C LC_TIME=en_US.UTF-8 LC_COLLATE=en_US.UTF-8
[5] LC_MONETARY=en_US.UTF-8 LC_MESSAGES=en_US.UTF-8 LC_PAPER=en_US.UTF-8 LC_NAME=C
[9] LC_ADDRESS=C LC_TELEPHONE=C LC_MEASUREMENT=en_US.UTF-8 LC_IDENTIFICATION=C
attached base packages:
[1] stats graphics grDevices utils datasets methods base
loaded via a namespace (and not attached):
[1] compiler_3.4.0 rsconnect_0.4.3 tools_3.4.0
I use the Rcpp package version 0.12.1.0.
My compiler is the following: ~$ g++ --version | head -n1
g++ (SUSE Linux) 4.8.5
Am I missing something? Has anyone else encountered the same issue?
That was reported yesterday at the GitHub issue tracker in #683 and already closed in pull request #684 which has now been merged.
So just upgrade to the master branch, or set the env.var manually for now. It is an upstream R change that broke this with no provision for backwards compatibility.
According Dirk Eddelbuettel answer -- just insert
# R code
Sys.setenv("USE_CXX11" = "yes")
sourceCpp("code.cpp")
# C++ code
// [[Rcpp::plugins(cpp11)]]
// [[Rcpp::export]]
double myhypot(double x, double y) {
return std::hypot(x,y);
}
... and it really works !
The following code works fine in the R console (R 3.3.0):
m = system.file("external/pores_1.mtx", package = "Matrix")
x = Matrix::readMM(m)
I can put it in a script and it runs fine in R console as:
source("test.R")
However, when I execute it as Rscript --vanilla test.R or Rscript test.R, I get an error:
Error in validObject(.Object) :
invalid class “dgTMatrix” object: Not a valid 'Mnumeric' class object
Calls: <Anonymous> -> new -> initialize -> initialize -> validObject
Execution halted
I don't know if this is related to that specific function. I am guessing this has something to do with how Rscript works, but I used it with various other libraries and functions previously and have never seen anything like this. Any idea what is happening?
I can confirm ... I'm getting the exact same error when running a script containing a call to glmnet(). I was able to trace it back to the Matrix package, on which glmnet depends.
I back-rev'd my R version from v3.3.3 -> v3.3.2 and the error disappeared. I then checked the sessionInfo() between the two and discovered that the version of the Matrix package differed ... it is 1.2-8 (in v3.3.3) and 1.2-7.1 (in v3.3.2). To confirm, I simply replaced the "OK" version of Matrix (7.1) with the "broken" version and the error returned.
I can also confirm that explicitly loading the methods package via library(methods) resolves the error (somehow?), which explains the differing behavior between the console call and the Rscript call from the command line.
So, it appears we have 2 work-arounds:
library(methods)
back rev your version of Matrix to 1.2-7.1.
Neither is super satisfactory ... I'd just like to know what's going on with Matrix 1.2-8. Maybe it'll be bug-fixed in the next version.
If you're interested, here is my sessionInfo():
R version 3.3.3 (2017-03-06)
Platform: x86_64-pc-linux-gnu (64-bit)
Running under: Ubuntu 16.04.2 LTS
locale:
[1] LC_CTYPE=en_US.UTF-8 LC_NUMERIC=C
[3] LC_TIME=en_US.UTF-8 LC_COLLATE=en_US.UTF-8
[5] LC_MONETARY=en_US.UTF-8 LC_MESSAGES=en_US.UTF-8
[7] LC_PAPER=en_US.UTF-8 LC_NAME=C
[9] LC_ADDRESS=C LC_TELEPHONE=C
[11] LC_MEASUREMENT=en_US.UTF-8 LC_IDENTIFICATION=C
attached base packages:
[1] stats graphics grDevices utils datasets base
other attached packages:
[1] glmnet_2.0-5 foreach_1.4.3 Matrix_1.2-8
loaded via a namespace (and not attached):
[1] codetools_0.2-15 grid_3.3.3 iterators_1.0.8 methods_3.3.3
[5] lattice_0.20-35
Oh dear.
I'm pretty (not 100% !) sure that this problem should not apply in newer version of R and Matrix.
Still I would claim this is not a Matrix bug in a proper sense, rather either an 'Rscript' vs 'R' bug -- as #Stu Field mentions already; Rscript does not by default attach the methods package to the search() path, but regular R does.
OTOH, R CMD check Matrix nowadays should try to run Matrix without the methods package being in the search path, and hence the problem should not be present from Matrix 1.2-9 and newer, i.e., by default from R 3.4.0 and newer or then update the 'Matrix' package if you have an older version of R.
Again: Can you please confirm that the problem is gone with R 3.4.0 (which has Matrix 1.2-9 "along with it") ?
Here is a more useful example script, I called Rscript-tst.R.
Run as
Rscript --vanilla Rscript-tst.R > Rscript-tst.Rout 2>&1
or (as myself with many R versions installed) something like
`R-3.4.0 RHOME`/bin/Rscript --vanilla Rscript-tst.R > Rscript-tst.Rout_R340 2>&1
Rscript-tst.R :
options(echo = TRUE)# << even with "Rscript" or --slave ...
(m <- system.file("external/pores_1.mtx", package = "Matrix"))
packageDescription("Matrix")
## This *load*s the Matrix package but does not attach it to search()
str(Matrix::readMM)
sessionInfo()
x <- Matrix::readMM(m)
## used to fail because 'methods' was not "there" (in Rscript only)
I have a problem when running R scripts on a Unix cluster as a batch job. The issue is when trying to load libraries in the environment, R cannot find the library. I'll give you an example. I'll use a basic R script names sess.R:
print(.libPaths())
library("gtools")
print(sessionInfo())
If I just run this script from the command line using the command:
$ Rscript sess.R
I get the following output:
[1] "/usr/lib64/R/library" "/usr/share/R/library"
R version 3.2.3 (2015-12-10)
Platform: x86_64-redhat-linux-gnu (64-bit)
Running under: CentOS release 6.6 (Final)
locale:
[1] LC_CTYPE=en_US.UTF-8 LC_NUMERIC=C
[3] LC_TIME=en_US.UTF-8 LC_COLLATE=en_US.UTF-8
[5] LC_MONETARY=en_US.UTF-8 LC_MESSAGES=en_US.UTF-8
[7] LC_PAPER=en_US.UTF-8 LC_NAME=C
[9] LC_ADDRESS=C LC_TELEPHONE=C
[11] LC_MEASUREMENT=en_US.UTF-8 LC_IDENTIFICATION=C
attached base packages:
[1] stats graphics grDevices utils datasets base
other attached packages:
[1] gtools_3.5.0
The library "gtools" is correctly loaded, script is working. however if I write a simple batch job (I will include in the job a couple of optional parameters including an error and output file) like:
#!/bin/bash
#SBATCH --output=sess.out
#SBATCH --error=sess.err
Rscript sess.R
The job fails after a second. The two output files I get are of course sess.out and sess.err.
Sess.out contains the library directory:
[1] "/usr/lib64/R/library" "/usr/share/R/library"
which seem to be the same as when running Rscript from the command line, so no error there. However there is no printing of the Info Session since the script has been terminated. The sess.err file contains the following error:
Error in library("gtools") : there is no package called ‘gtools’
Execution halted
There, it seems like R cannot find gtools in this situation, even if the library path is the same...
Am I missing something? Is there an error I don't see somewhere? Or is it a problems of cluster settings?
I was using the spatsurv package in R and when compiling my code using R in the terminal on Debian GNU/Linux 8 (jessie), segmentation fault was returned and R was closed. After some debugging, I found out that the problem appears when using solve function. In addition, one of the authors of spatsurv package suggested me to test the same problem with each imported package. Then, after making that test, the problem was only with OpenStreetMap package.
library(OpenStreetMap)
x <- solve(diag(1, 1000))
# Segmentation fault
If I use the solve function before loading OpenStreetMap, I get the following:
x <- solve(diag(1, 1000))
library(OpenStreetMap)
x <- solve(diag(1, 1000))
# An irrecoverable stack overflow has occurred.
# Please check if any of your loaded .so files has enabled executable stack (see man page execstack(8))
# Error: segfault from C stack overflow
Information of my clean session is below:
R version 3.3.0 (2016-05-03)
Platform: x86_64-pc-linux-gnu (64-bit)
Running under: Debian GNU/Linux 8 (jessie)
locale:
[1] LC_CTYPE=en_GB.UTF-8 LC_NUMERIC=C
[3] LC_TIME=en_GB.UTF-8 LC_COLLATE=en_GB.UTF-8
[5] LC_MONETARY=en_GB.UTF-8 LC_MESSAGES=en_GB.UTF-8
[7] LC_PAPER=en_GB.UTF-8 LC_NAME=C
[9] LC_ADDRESS=C LC_TELEPHONE=C
[11] LC_MEASUREMENT=en_GB.UTF-8 LC_IDENTIFICATION=C
attached base packages:
[1] stats graphics grDevices utils datasets methods base
I would like to know how to solve this segmentation fault to be able to use spatsurv package.