How to readRDS from stdin? - r

I try to use the following command to read an RDS file. But it doesn't work. My OS is Mac OS X.
$ lr -e "readRDS(file('stdin'))" < /tmp/x.rds
Error in readRDS(file("stdin")) : unknown input format
$ lr -p -e "readRDS('/dev/stdin')" < /tmp/x.rds
Error in readRDS("/dev/stdin") : error reading from connection
But this works.
$ lr -p -e "readRDS('/tmp/x.rds')"
x y
1 1 11
2 2 12
3 3 13
Does anybody know how to readRDS from stdin? Thanks.

It works for me (on linux, using littler 0.3.9 on R-devel) using '/dev/stdin' instead of 'stdin'; so try:
lr -p -e "print(readRDS('/dev/stdin'))" < /tmp/x.rds

Related

Executing Linux/Unix Command From Within R Using Variables

I'm trying to make a call from within R to execute BASH commands, to get my feet wet:
I wanted to simply capture a listing of my current files located in a specific directory through use of the "ls -al" command. The output would be sent to text file called a01_test.txt.
The directory I would like to capture the contents of is "C:\Users\user00\a01_TEST" which is referenced as "/mnt/c/Users/user00/a01_TEST/" from a WSL Ubuntu 20.04.5 LTS perspective.
The directory contains five (5) files: file_01.txt, file_02.txt ,..., file_05.txt.
FYI, I am running R (R version 4.2.0 (2022-04-22 ucrt)) via RStudio (2022.07.1 Build 554) on Windows 11 (Version 10.0.22000 Build 22000).
I tried:
PATH_UNIX <- "/mnt/c/Users/user00/a01_TEST/"
FILENAME_TEST <-"a01_test.txt"
paste0("system(\"bash -c \'ls -al ",PATH_UNIX," >",PATH_UNIX,FILENAME_TEST,"\'\")")
However that only returned a command prompt -- nothing else:
> paste0("system(\"bash -c \'ls -al ",PATH_UNIX," >",PATH_UNIX,FILENAME_TEST,"\'\")")
[1] "system(\"bash -c 'ls -al /mnt/c/Users/user00/a01_TEST/ >/mnt/c/Users/user00/a01_TEST/a01_test.txt'\")"
>
I thought one could test the code using:
cat(print(paste0("system(\"bash -c \'ls -al ",PATH_UNIX," >",PATH_UNIX,FILENAME_TEST,"\'\")")))
which resulted in:
> cat(print(paste0("system(\"bash -c \'ls -al ",PATH_UNIX," >",PATH_UNIX,FILENAME_TEST,"\'\")")))
[1] "system(\"bash -c 'ls -al /mnt/c/Users/user00/a01_TEST/ >/mnt/c/Users/user00/a01_TEST/a01_test.txt'\")"
system("bash -c 'ls -al /mnt/c/Users/user00/a01_TEST/ >/mnt/c/Users/user00/a01_TEST/a01_test.txt'")
If I do not use variables, such as, PATH_UNIX and FILENAME_TEST and code the entire path manually, I can create a text file (a01_test.txt) giving me the desired listing of the directory's contents:
system("bash -c 'ls -al /mnt/c/Users/user00/a01_TEST > /mnt/c/Users/user00/a01_TEST/a01_test.txt'")
which results in:
> system("bash -c 'ls -al /mnt/c/Users/user00/a01_TEST > /mnt/c/Users/user00/a01_TEST/a01_test.txt'")
[1] 0
>
giving me the file called "a01_test.txt" containing the directory's contents:
total 0
drwxrwxrwx 1 user00 user00 4096 Nov 3 2022 .
drwxrwxrwx 1 user00 user00 4096 Nov 3 05:07 ..
-rwxrwxrwx 1 user00 user00 0 Nov 3 2022 a01_test.txt
-rwxrwxrwx 1 user00 user00 0 Nov 3 05:26 file_01.txt
-rwxrwxrwx 1 user00 user00 0 Nov 3 05:26 file_02.txt
-rwxrwxrwx 1 user00 user00 0 Nov 3 05:26 file_03.txt
-rwxrwxrwx 1 user00 user00 0 Nov 3 05:26 file_04.txt
-rwxrwxrwx 1 user00 user00 0 Nov 3 05:26 file_05.txt
Any assistance to make use of the variables PATH_UNIX & FILENAME_TEST to make a call to Linux/Unix to obtain a directory listing would be appreciated.
sprintf (?sprintf for further details) is a convenient way to create format strings that can subsequently be passed to system:
PATH_UNIX <- '/mnt/c/Users/user00/a01_TEST/'
FILENAME_TEST <- 'a01_test.txt'
cmdstr <- sprintf('bash -c \'ls -al %s > %s\'', PATH_UNIX, FILENAME_TEST)
message('bash command string = ', cmdstr)
system(command = cmdstr)
Expanding on the solution provided by br00t, and doing some testing, one could also use the paste0() function:
# DESIRED CMD TO BE PASSED VIA BASH
cat(paste0("system(bash -c \'ls -al ",PATH_UNIX," >",PATH_UNIX,FILENAME_TEST,"\')"))
# OUTPUT:
# system(bash -c 'ls -al /mnt/c/Users/user00/a01_TEST/ >/mnt/c/Users/user00/a01_TEST/a01_test.txt')
# PLACE DESIRED CMD IN A VAR:
cmdstr_test <- paste0("bash -c \'ls -al ",PATH_UNIX," > ",PATH_UNIX,FILENAME_TEST,"\'")
# CHECK VAR:
message('bash command string = ', cmdstr_test)
# OUTPUT:
# bash command string = bash -c 'ls -al /mnt/c/Users/user00/a01_TEST/ > /mnt/c/Users/user00/a01_TEST/a01_test.txt'
# RUN COMMAND USING system() function:
system(command = cmdstr_test)
# OUTPUT (Will get "0", if successful)
> system(command = cmdstr_test)
[1] 0
>

Need help in solving an expression in shell

How to calculate X power Y in the unix shell script where value of Y is being supplied by a loop eg. Y=1 to 5. It means I would like to calculte (X^Y)
In bash you could do:
$ for i in {1..5}; do printf "$((2 ** $i))\n"; doneprintf "$((2 ** $i))\n"; done
2
4
8
16
32
Many shells however do not support raise to power operation and in
such situations you need to use bc:
$ for i in $(seq 5); do printf "%s\n" "$(echo "2 ^ $i" | bc)"; done
2
4
8
16
32

yosys fails at ABC pass (on counter.v demo)

I hope someone can help me with this...
This is my first encounter with yosys. For the start, I'm trying to run the very same demo as Clifford explained in his presentation. I downloaded the demo at the following location: https://github.com/cliffordwolf/yosys/tree/master/manual/PRESENTATION_Intro
yosys run beaks at the ABC pass with following message:
12. Executing ABC pass (technology mapping using ABC).
12.1. Extracting gate netlist of module `\counter' to `<abc-temp-dir>/input.blif'..
Extracted 6 gates and 12 wires to a netlist network with 4 inputs and 2 outputs.
12.1.1. Executing ABC.
Running ABC command: <yosys-exe-dir>/yosys-abc -s -f <abc-temp-dir>/abc.script 2>&1
ABC: ABC command line: "source <abc-temp-dir>/abc.script".
ABC:
ABC: + read_blif <abc-temp-dir>/input.blif
ABC: + read_lib -w /home/boris/Documents/Self Learning/yosys_synthesys/mycells.lib
ABC: usage: read_lib [-SG float] [-M num] [-dnvwh] <file>
ABC: reads Liberty library from file
ABC: -S float : the slew parameter used to generate the library [default = 0.00]
ABC: -G float : the gain parameter used to generate the library [default = 0.00]
ABC: -M num : skip gate classes whose size is less than this [default = 0]
ABC: -d : toggle dumping the parsed library into file "*_temp.lib" [default = no]
ABC: -n : toggle replacing gate/pin names by short strings [default = no]
ABC: -v : toggle writing verbose information [default = yes]
ABC: -v : toggle writing information about skipped gates [default = yes]
ABC: -h : prints the command summary
ABC: <file> : the name of a file to read
ABC: ** cmd error: aborting 'source <abc-temp-dir>/abc.script'
ERROR: Can't open ABC output file `/tmp/yosys-abc-KDGya6/output.blif'.
[boris#E7440 yosys_synthesys]$
I have had a look at the file location mentioned in the error statement above, there is no output.blif in there:
[boris#E7440 yosys_synthesys]$ ll /tmp/yosys-abc-KDGya6/
total 12K
-rw-rw-r--. 1 boris boris 542 Jul 5 11:21 abc.script
-rw-rw-r--. 1 boris boris 526 Jul 5 11:21 input.blif
-rw-rw-r--. 1 boris boris 852 Jul 5 11:21 stdcells.genlib
[boris#E7440 yosys_synthesys]$
Buy the way, here is some system/tools info that might be relevant for debugging:
Linux E7440.DELL 4.4.13-200.fc22.x86_64 #1 SMP Wed Jun 8 15:59:40 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux
Yosys 0.6+141 (git sha1 080f95f, gcc 5.3.1 -fPIC -Os)
UC Berkeley, ABC 1.01 (compiled Mar 8 2015 01:00:49)
The issue has been resolved...
Solution =
Changed rundir from:
/home/boris/Documents/Self Learning/yosys_synthesys/mycells.lib
to:
/home/boris/Documents/SelfLearning/yosys_synthesys/mycells.lib
Lesson learned =
ABC tool does not accept space characters in the path/file name.

Better string interpolation in R

I need to build up long command lines in R and pass them to system(). I find it is very inconvenient to use paste0/paste function, or even sprintf function to build each command line. Is there a simpler way to do like this:
Instead of this hard-to-read-and-too-many-quotes:
cmd <- paste("command", "-a", line$elem1, "-b", line$elem3, "-f", df$Colum5[4])
or:
cmd <- sprintf("command -a %s -b %s -f %s", line$elem1, line$elem3, df$Colum5[4])
Can I have this:
cmd <- buildcommand("command -a %line$elem1 -b %line$elem3 -f %df$Colum5[4]")
For a tidyverse solution see https://github.com/tidyverse/glue. Example
name="Foo Bar"
glue::glue("How do you do, {name}?")
With version 1.1.0 (CRAN release on 2016-08-19), the stringr package has gained a string interpolation function str_interp() which is an alternative to the gsubfn package.
# sample data
line <- list(elem1 = 10, elem3 = 30)
df <- data.frame(Colum5 = 1:4)
# do the string interpolation
stringr::str_interp("command -a ${line$elem1} -b ${line$elem3} -f ${df$Colum5[4]}")
#[1] "command -a 10 -b 30 -f 4"
This comes pretty close to what you are asking for. When any function f is prefaced with fn$, i.e. fn$f, character interpolation will be performed replacing ... with the result of running ... as an R expression.
library(gsubfn)
cmd <- fn$identity("command -a `line$elem1` -b `line$elem3` -f `df$Colum5[4]`")
Here is a self contained reproducible example:
library(gsubfn)
# test inputs
line <- list(elem1 = 10, elem3 = 30)
df <- data.frame(Colum5 = 1:4)
fn$identity("command -a `line$elem1` -b `line$elem3` -f `df$Colum5[4]`")
## [1] "command -a 10 -b 30 -f 4"
system
Since any function can be used we could operate directly on the system call like this. We have used echo here to make it executable but any command could be used.
exitcode <- fn$system("echo -a `line$elem1` -b `line$elem3` -f `df$Colum5[4]`")
## -a 10 -b 30 -f 4
Variation
This variation would also work. fn$f also performs substitution of $whatever with the value of variable whatever. See ?fn for details.
with(line, fn$identity("command -a $elem1 -b $elem3 -f `df$Colum5[4]`"))
## [1] "command -a 10 -b 30 -f 4"
Another option would be to use whisker.render from https://github.com/edwindj/whisker which is a {{Mustache}} implementation in R. Usage example:
require(dplyr); require(whisker)
bedFile="test.bed"
whisker.render("processing {{bedFile}}") %>% print
Not really a string interpolation solution, but still a very good option for the problem is to use the processx package instead of system() and then you don't need to quote anything.
library(GetoptLong)
str = qq("region = (#{region[1]}, #{region[2]}), value = #{value}, name = '#{name}'")
cat(str)
qqcat("region = (#{region[1]}, #{region[2]}), value = #{value}, name = '#{name}'")
https://cran.r-project.org/web/packages/GetoptLong/vignettes/variable_interpolation.html

Move top 1000 lines from text file to a new file using Unix shell commands

I wish to copy the top 1000 lines in a text file containing more than 50 million entries, to another new file, and also delete these lines from the original file.
Is there some way to do the same with a single shell command in Unix?
head -1000 input > output && sed -i '1,+999d' input
For example:
$ cat input
1
2
3
4
5
6
$ head -3 input > output && sed -i '1,+2d' input
$ cat input
4
5
6
$ cat output
1
2
3
head -1000 file.txt > first100lines.txt
tail --lines=+1001 file.txt > restoffile.txt
Out of curiosity, I found a box with a GNU version of sed (v4.1.5) and tested the (uncached) performance of two approaches suggested so far, using an 11M line text file:
$ wc -l input
11771722 input
$ time head -1000 input > output; time tail -n +1000 input > input.tmp; time cp input.tmp input; time rm input.tmp
real 0m1.165s
user 0m0.030s
sys 0m1.130s
real 0m1.256s
user 0m0.062s
sys 0m1.162s
real 0m4.433s
user 0m0.033s
sys 0m1.282s
real 0m6.897s
user 0m0.000s
sys 0m0.159s
$ time head -1000 input > output && time sed -i '1,+999d' input
real 0m0.121s
user 0m0.000s
sys 0m0.121s
real 0m26.944s
user 0m0.227s
sys 0m26.624s
This is the Linux I was working with:
$ uname -a
Linux hostname 2.6.18-128.1.1.el5 #1 SMP Mon Jan 26 13:58:24 EST 2009 x86_64 x86_64 x86_64 GNU/Linux
For this test, at least, it looks like sed is slower than the tail approach (27 sec vs ~14 sec).
This is a one-liner but uses four atomic commands:
head -1000 file.txt > newfile.txt; tail +1000 file.txt > file.txt.tmp; cp file.txt.tmp file.txt; rm file.txt.tmp
Perl approach:
perl -ne 'if($i<1000) { print; } else { print STDERR;}; $i++;' in 1> in.new 2> out && mv in.new in
Using pipe:
cat en-tl.100.en | head -10

Resources