furrr with rTorch in multisession - r

I want to use rTorch in a furrr "loop". A minimal example seems to be:
library(rTorch)
torch_it <- function(i) {
#.libPaths("/User/homes/mreichstein/.R_libs_macadamia4.0/")
#require(rTorch)
cat("torch is: "); print(torch)
out <- torch$tensor(1)
out
}
library(furrr)
plan(multisession, workers=8) # this works with sequential or multicore
result <- future_map(1:5, torch_it)
I get as output:
torch is: <environment: 0x556b57f07990>
attr(,"class")
[1] "python.builtin.module" "python.builtin.object"
Error in torch$tensor(1) : attempt to apply non-function
When I use multicore or sequential I get the expected output:
torch is: Module(torch)
torch is: Module(torch)
torch is: Module(torch)
torch is: Module(torch)
torch is: Module(torch)
... and result is the expected list of tensors.
Uncommenting the first lines in the function, assuming the new sessions "need this" did not help.
Update: I added torch <- import("torch") in the torch_it function. Then it runs without error, but the result I get is a list of empty pointers (?):
> result
[[1]]
<pointer: 0x0>
[[2]]
<pointer: 0x0>
[[3]]
<pointer: 0x0>
[[4]]
<pointer: 0x0>
[[5]]
<pointer: 0x0>
So, how do I properly link rTorch to each of the multisessions?
Thanks in advance!
Info:
> R.version
_
platform x86_64-redhat-linux-gnu
arch x86_64
os linux-gnu
system x86_64, linux-gnu
status
major 4
minor 0.3
year 2020
month 10
day 10
svn rev 79318
language R
version.string R version 4.0.3 (2020-10-10)
nickname Bunny-Wunnies Freak Out

Related

Format problem of the write.table in R, utf-8 original encode

There is a problem with the write.table. chr2 is correctly shown in the console but fail in the saved txt file.
> chr1 = chr2 = rep(NA, 8)
> for(i in 1:8){
+ chr1[i] = letters[i]
+ chr2[i] = intToUtf8(10240+2^(i-1))
+ }
>
> chr1
[1] "a" "b" "c" "d" "e" "f" "g" "h"
> chr2
[1] "⠁" "⠂" "⠄" "⠈" "⠐" "⠠" "⡀" "⢀"
>
write.table(cbind(chr1, chr2), file = "chr2.txt")
May I know how to get the txt file printing exactly the same way as the console? Thank you!
My OS and R version:
version
_
platform x86_64-w64-mingw32
arch x86_64
os mingw32
system x86_64, mingw32
status
major 4
minor 0.3
year 2020
month 10
day 10
svn rev 79318
language R
version.string R version 4.0.3 (2020-10-10)
nickname Bunny-Wunnies Freak Out
---------------- June-06 Update ----------------
In fact, what I really want to do is to save a "txt picture" like:
⣿⣿⣿⣿⣿⣿⣿⣿⣿⡿⠻⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿
⣿⣿⣿⣿⣿⣿⣿⣿⣿⣆⢀⠹⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⡟⢻⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿
⣿⣿⣿⣿⣿⣿⡟⠉⠻⣿⣦⢀⠘⣿⣿⣿⣿⣿⣿⡏⢸⣿⣿⣿⣿⣿⣿⣿⣿⣿⡇⢸⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿
⣿⣿⣿⣿⣿⣿⣿⣦⡀⠈⠻⣧⢀⠘⢿⡟⢠⣤⣌⡇⢀⣴⣅⣤⣄⠙⡟⢡⣤⣈⡇⢸⠟⢠⣾⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿
⣿⣿⣿⣿⡿⠉⠛⢿⣿⣄⢀⠈⢳⡀⢸⣧⣈⡉⠙⠇⢸⣿⠛⣉⣉⢀⠁⣾⣿⣿⡇⢀⡀⢻⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿
⣿⣿⣿⣿⣷⣦⣀⢀⠈⠙⠷⣄⢀⣽⣿⡟⠻⠿⠃⣠⠘⢿⢀⠿⠟⢀⣇⠘⠿⠛⡇⢸⣷⡀⢻⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿
⣿⣿⣿⡿⠛⠻⠿⣿⣦⣄⡀⠈⣿⣿⣿⣿⣶⣶⣾⣿⣷⣾⣷⣶⣾⣶⣿⣷⣶⣾⣷⣾⣿⣷⣾⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿
⡿⠿⣿⣧⣄⣀⢀⢀⢀⠉⠙⢺⣿⠿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⡿⠁⢀⡇⢀⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿
⡇⢀⣿⡿⠿⠿⠿⠿⠶⠶⠤⣼⣿⢀⣿⡿⠋⠉⠛⡏⠉⣿⡏⠉⡟⠋⠉⠻⡟⠉⠋⠉⠃⢀⠙⡇⢀⣿⠟⠉⠉⠻⠉⠹⣿⠉⢻⡟⠉
⡇⢀⣿⣇⣀⣀⣀⣀⣀⣀⣀⣿⣿⢀⣿⠁⢠⣶⢀⢰⢀⢹⢀⢸⢀⠐⠓⢀⠁⢀⣰⣶⡇⢀⣶⡇⢀⣿⢀⢰⡆⢀⠃⢀⡇⢀⠈⠇⢀
⡇⢀⠿⠿⠿⠿⠿⠿⠿⠿⠿⠿⠿⢀⣿⡀⠘⠿⢀⢸⡆⢀⢀⣿⢀⠰⡶⠶⡆⢀⣿⣿⡇⢀⣿⡇⢀⢿⢀⠸⠇⢀⣼⢀⢀⣸⢀⢀⣸
⣇⣀⣀⣀⣀⣀⣀⣀⣀⣀⣀⣀⣀⣀⣿⣷⣄⣀⣠⣿⣿⣀⣸⣿⣦⣀⣀⣠⣧⣀⣿⣿⣇⣀⣿⣷⣄⣀⣦⣀⣀⣴⣿⣆⣀⣿⣇⣀⣿
I have uploaded my code and the example picture to GitHub:
https://github.com/CarltonChen/TxtDotPic
Please have a look.
Unfortunately, coercing the two vectors to a data.frame to write it as a table already converts the characters to unicode references. You can see that when you use data.frame(chr1,chr2).
I frankly do not know how to write unicode symbols to tables, but if you are able to compose lines of a file (concatenate the values with '\t'), you can write them using writeLines() to a file with native encoding:
f = file("specialchar.txt",open="w",encoding="native.enc")
writeLines(paste(chr1,chr2,sep='\t'),con=f, useBytes=T)
close(f)
When I do this, I get the following result:

Why did R's sorting change data imported with load() after an upgrade from 3.5.2 to 4.0.0?

Short version. I load() data in a package. Previously, a test in a package passed, now it fails because the output of sort changed.
Here is a minimal reproducible example - for details see below:
y <- c("Schaffhausen", "Schwyz", "Seespital", "SRZ")
sort(y)
# OLD 3.5.2 [1] "Schaffhausen" "Schwyz" "Seespital" "SRZ"
# NEW 4.0.0 [1] "SRZ" "Schaffhausen" "Schwyz" "Seespital"
# Update 4.0.2 see comment:
# [1] "Schaffhausen" "Schwyz" "Seespital" "SRZ"
# From jay.sf's comment
sort.int(y, method="radix")
# [1] "SRZ" "Schaffhausen" "Schwyz" "Seespital"
sort.int(y, method="shell")
# [1] "Schaffhausen" "Schwyz" "Seespital" "SRZ"
# From Henrik's comment:
data.table::fsort(y)
# [1] "SRZ" "Schaffhausen" "Schwyz" "Seespital"
The only related reported change I found is
CHANGES IN R 4.0.0
NEW FEATURES
...
When loading data sets via read.table(), data() now uses LC_COLLATE=C to ensure locale-independent results for possible string-to-factor conversions.
But I am even not sure, if this could explain what I see.
As I want to minimize the number of imported packages and I would like to understand what's going on, I am not sure how to proceed. Do I miss something?
(A change to a sort.int with method radix would do the job, but still: Why did it change? Is that really better?
I just realized, that (thanks to Roland) sort calls in my case sort.int:
function (x, decreasing = FALSE, na.last = NA, ...)
{
if (is.object(x))
x[order(x, na.last = na.last, decreasing = decreasing)]
else sort.int(x, na.last = na.last, decreasing = decreasing,
...)
}
From ?sort.int:
The "auto" method selects "radix" for short (less than 2^31 elements) numeric vectors, integer vectors, logical vectors and factors; otherwise, "shell".)
And according to the docs, sort.int did not change from 4.0.0 to 4.0.2.
From ?data.table::setorder
data.table always reorders in "C-locale". As a consequence, the
ordering may be different to that obtained by base::order. In English
locales, for example, sorting is case-sensitive in C-locale. Thus,
sorting c("c", "a", "B") returns c("B", "a", "c") in data.table but
c("a", "B", "c") in base::order. Note this makes no difference in most
cases of data; both return identical results on ids where only
upper-case or lower-case letters are present ("AB123" < "AC234" is
true in both), or on country names and other proper nouns which are
consistently capitalized. For example, neither "America" < "Brazil"
nor "america" < "brazil" are affected since the first letter is
consistently capitalized.
Using C-locale makes the behaviour of sorting in data.table more
consistent across sessions and locales. The behaviour of base::order
depends on assumptions about the locale of the R session. In English
locales, "america" < "BRAZIL" is true by default but false if you
either type Sys.setlocale(locale="C") or the R session has been
started in a C locale for you – which can happen on servers/services
since the locale comes from the environment the R session was started
in. By contrast, "america" < "BRAZIL" is always FALSE in data.table
regardless of the way your R session was started.
(Related questions Language dependent sorting with R and Best practice: Should I try to change to UTF-8 as locale or is it safe to leave it as is?)
Details
R.version # old _
platform x86_64-w64-mingw32
arch x86_64
os mingw32
system x86_64, mingw32
status
major 3
minor 5.2
year 2018
month 12
day 20
svn rev 75870
language R
version.string R version 3.5.2 (2018-12-20)
nickname Eggshell Igloo
y <- c("Schaffhausen", "Schwyz", "Seespital", "SRZ")
sort(y)
# [1] "Schaffhausen" "Schwyz" "Seespital" "SRZ"
stringr::str_sort(y)
# [1] "Schaffhausen" "Schwyz" "Seespital" "SRZ"
stringr::str_sort(y, locale = "C")
# [1] "SRZ" "Schaffhausen" "Schwyz" "Seespital"
# =======
R.version # new after upgrade
platform x86_64-w64-mingw32
arch x86_64
os mingw32
system x86_64, mingw32
status
major 4
minor 0.0
year 2020
month 04
day 24
svn rev 78286
language R
version.string R version 4.0.0 (2020-04-24)
nickname Arbor Day
y <- c("Schaffhausen", "Schwyz", "Seespital", "SRZ")
sort(y)
# [1] "SRZ" "Schaffhausen" "Schwyz" "Seespital"
stringr::str_sort(y)
# [1] "Schaffhausen" "Schwyz" "Seespital" "SRZ"
stringr::str_sort(y, locale = "C")
#[1] "SRZ" "Schaffhausen" "Schwyz" "Seespital"
# ==== Test with new 4.0.2
R.version
platform x86_64-w64-mingw32
arch x86_64
os mingw32
system x86_64, mingw32
status
major 4
minor 0.2
year 2020
month 06
day 22
svn rev 78730
language R
version.string R version 4.0.2 (2020-06-22)
nickname Taking Off Again
y <- c("Schaffhausen", "Schwyz", "Seespital", "SRZ")
sort(y)
# [1] "Schaffhausen" "Schwyz" "Seespital" "SRZ"
stringr::str_sort(y)
# [1] "Schaffhausen" "Schwyz" "Seespital" "SRZ"
stringr::str_sort(y, locale = "C")
# [1] "SRZ" "Schaffhausen" "Schwyz" "Seespital"
In summary, it was a bug which has been removed in R version 4.0.1. As #Roland figured out.
From CRAN:
In R 4.0.0, sort.list(x) when is.object(x) was true, e.g., for x <-I(letters), was accidentally usingmethod = "radix". Consequently,
e.g., merge(<data.frame>) was much slower than previously; reported in
PR#17794.

Same seed, different OS, different random numbers in R

I was experiencing inconsistent results between two machines and a linux server, until I realized that fixing the seed was having different effects. I am running different R versions in all of them, all above 3.3.0. Here are the examples:
Linux 1
> set.seed(10); rnorm(1)
[1] -0.4463588
> version
_
platform x86_64-pc-linux-gnu
arch x86_64
os linux-gnu
system x86_64, linux-gnu
status
major 3
minor 3.0
year 2016
month 05
day 03
svn rev 70573
language R
version.string R version 3.3.0 (2016-05-03)
nickname Supposedly Educational
Linux 2
> set.seed(10); rnorm(1)
[1] 0.01874617
> version
_
platform x86_64-pc-linux-gnu
arch x86_64
os linux-gnu
system x86_64, linux-gnu
status
major 3
minor 4.2
year 2017
month 09
day 28
svn rev 73368
language R
version.string R version 3.4.2 (2017-09-28)
nickname Short Summer
Mac OS
> set.seed(10); rnorm(1)
[1] 0.01874617
> version
_
platform x86_64-apple-darwin15.6.0
arch x86_64
os darwin15.6.0
system x86_64, darwin15.6.0
status
major 3
minor 4.3
year 2017
month 11
day 30
svn rev 73796
language R
version.string R version 3.4.3 (2017-11-30)
nickname Kite-Eating Tree
Windows
> set.seed(10); rnorm(1)
[1] 0.01874617
> version
_
platform x86_64-w64-mingw32
arch x86_64
os mingw32
system x86_64, mingw32
status
major 3
minor 4.1
year 2017
month 06
day 30
svn rev 72865
language R
version.string R version 3.4.1 (2017-06-30)
nickname Single Candle
Linux gives a different random number generation from the same seed, thus making the result of a script run on it not fully reproducible (depending on the OS in which they are re-run, the results will agree or not). This is annoying.
I do not know what is happening here. Particularly:
(1) Is it an issue with R's versions or something more involved?
(2) How can this inconsistent behaviour be avoided? Any help is appreciated.
EDIT originated from #Jesse Tweedle answer (output in Linux 1 in a new session):
> set.seed(10); rnorm(1)
[1] -0.4463588
> set.seed(10); rnorm(1)
[1] -0.4463588
> set.seed(102); rnorm(1)
[1] 0.05752965
> set.seed(10, kind = "Mersenne-Twister"); rnorm(1)
[1] 0.01874617
> set.seed(10); rnorm(1)
[1] 0.01874617
> set.seed(102); rnorm(1)
[1] 0.1805229
From docs:
Random docs:
RNGversion can be used to set the random generators as they were in an earlier R version (for reproducibility).
So try this on all systems:
set.seed(10, kind = "Mersenne-Twister", normal.kind = "Inversion"); rnorm(1)
[1] 0.01874617

Error from CAPM.alpha in PerformanceAnalytics

When I try to run the example from page 22 of the PerformanceAnalytics reference, I get an error message. See below.
PS I am a beginner & this has never worked for me. Also, my underlying issue is that I'm getting exactly the same error when trying to use table.CAPM with my own data.
Thanks for any assistance.
> search()
[1] ".GlobalEnv" "package:PerformanceAnalytics"
[3] "package:xts" "package:zoo"
[5] "package:stats" "package:graphics"
[7] "package:grDevices" "package:utils"
[9] "package:datasets" "package:methods"
[11] "Autoloads" "package:base"
> version
_
platform x86_64-w64-mingw32
arch x86_64
os mingw32
system x86_64, mingw32
status
major 2
minor 15.2
year 2012
month 10
day 26
svn rev 61015
language R
version.string R version 2.15.2 (2012-10-26)
nickname Trick or Treat
> data(managers)
> CAPM.alpha(managers[,1,drop=FALSE], managers[,8,drop=FALSE], Rf=.035/12)
Error in lm.fit(x, y, offset = offset, singular.ok = singular.ok, ...) :
0 (non-NA) cases
>
The bug is not in your code, it is in the R package itself. It it is shown on the package validation check here and it can be reproduced with:
library(PerformanceAnalytics)
example(CAPM.alpha)
The error seems to be on line 40 of Return.excess.R. It should be replaced with:
xR = coredata(as.xts(R))-coredata(as.xts(Rf))
The easiest way of fixing this in practice is to run:
require(utils)
assignInNamespace(
"Return.excess",
function (R, Rf = 0)
{ # #author Peter Carl
# edited by orizon
# .. additional comments removed
R = checkData(R)
if(!is.null(dim(Rf))){
Rf = checkData(Rf)
indexseries=index(cbind(R,Rf))
columnname.Rf=colnames(Rf)
}
else {
indexseries=index(R)
columnname.Rf=Rf
Rf=xts(rep(Rf, length(indexseries)),order.by=indexseries)
}
return.excess <- function (R,Rf)
{
xR = coredata(as.xts(R))-coredata(as.xts(Rf)) #fixed
}
result = apply(R, MARGIN=2, FUN=return.excess, Rf=Rf)
colnames(result) = paste(colnames(R), ">", columnname.Rf)
result = reclass(result, R)
return(result)
},
"PerformanceAnalytics"
)
Then your original command works:
> data(managers)
> CAPM.alpha(managers[,1,drop=FALSE], managers[,8,drop=FALSE], Rf=.035/12)
[1] 0.005960609
Be aware that I have not verified that the function does what it purports to do.

Negative singular value using LAPACK svd

I have a 399 x 399 matrix for which the R svd() function (using LAPACK) gives me a negative singular value! This is not supposed to happen -- has anyone seen this before? It does not happen if I use the LINPACK option, so I guess this is a bug in the LAPACK svd.
ganymede: R --vanilla
R version 2.15.1 (2012-06-22) -- "Roasted Marshmallows"
Copyright (C) 2012 The R Foundation for Statistical Computing
ISBN 3-900051-07-0
Platform: x86_64-apple-darwin9.8.0/x86_64 (64-bit)
R is free software, etc ...
> load('A.dat')
> ls()
[1] "A"
> dim(A)
[1] 399 399
>
> L1 <- svd(A)
> any( L1$d < 0 )
[1] TRUE
> L1$d[1:4]
[1] 80.18833 68.93905 61.62659 57.62883
> L1$d[396:399]
[1] 3.777844e-15 3.582460e-15 3.175665e-15 -6.512578e+00
>
> L2 <- svd(A,LINPACK=TRUE)
> any( L2$d < 0 )
[1] FALSE
> L2$d[1:4]
[1] 80.18833 68.93905 61.62659 57.62883
> L2$d[396:399]
[1] 8.565532e-32 3.254162e-32 3.484425e-47 5.411232e-48
>
Per this bug report the problem can most likely be fixed by relinking against LAPACK 3.4.1. Upgrading the generic version 3.1.1 seems to be non-trivial and the procedure depends on the OS.

Resources