How can I generate a script in Node.js and pipe to the shell?
E.g. I can create this file, e.g. hello.R, make it executable chmod +x hello.R and run it from the command line, ./hello.R:
#!/usr/bin/Rscript
hello <- function( name ) { return (sprintf( "Hello, %s", name ); })
cat(hello("World"));
What I'd like to do is to do the equivalent from Node. Specifically generate a more complex R script in memory (e.g. as a string using templating, etc.), execute it (using exec or spawn?), and read stdout.
But I can't quite figure out how to pipe a script to R. I tried this (among other things):
var rscript = [
hello <- function( name ) { return (sprintf( "Hello, %s", name ); })
cat(hello("World"));
].join('\n');
var exec = require('child_process').exec;
exec(rscript, { shell: '/usr/bin/R'}, function(err, stdout, stderr) {
if (err) throw(err);
console.log(stdout);
});
However, this fails as it seems neither /usr/bin/R nor /usr/bin/Rscript understand the -c switch:
Check the nodejs docs of child_process. You should be able to spawn an Rscript or R command just as you would do on the terminal, and send your commands over child.stdin.
var c = require('child_process');
var r = c.spawn("R","");
r.stdin.write(rscript);
/* now you should be able to read the results from r.stdout a/o r.stderr */
Related
i'm new to nf-core/nextflow and needless to say the documentation does not reflect what might be actually implemented. But i'm defining the basic pipeline below:
nextflow.enable.dsl=2
process RUNBLAST{
input:
val thr
path query
path db
path output
output:
path output
script:
"""
blastn -query ${query} -db ${db} -out ${output} -num_threads ${thr}
"""
}
workflow{
//println "I want to BLAST $params.query to $params.dbDir/$params.dbName using $params.threads CPUs and output it to $params.outdir"
RUNBLAST(params.threads,params.query,params.dbDir, params.output)
}
Then i'm executing the pipeline with
nextflow run main.nf --query test2.fa --dbDir blast/blastDB
Then i get the following error:
N E X T F L O W ~ version 22.10.6
Launching `main.nf` [dreamy_hugle] DSL2 - revision: c388cf8f31
Error executing process > 'RUNBLAST'
Error executing process > 'RUNBLAST'
Caused by:
Not a valid path value: 'test2.fa'
Tip: you can replicate the issue by changing to the process work dir and entering the command bash .command.run
I know test2.fa exists in the current directory:
(nfcore) MN:nf-core-basicblast jraygozagaray$ ls
CHANGELOG.md conf other.nf
CITATIONS.md docs pyproject.toml
CODE_OF_CONDUCT.md lib subworkflows
LICENSE main.nf test.fa
README.md modules test2.fa
assets modules.json work
bin nextflow.config workflows
blast nextflow_schema.json
I also tried with "file" instead of path but that is deprecated and raises other kind of errors.
It'll be helpful to know how to fix this to get myself started with the pipeline building process.
Shouldn't nextflow copy the file to the execution path?
Thanks
You get the above error because params.query is not actually a path value. It's probably just a simple String or GString. The solution is to instead supply a file object, for example:
workflow {
query = file(params.query)
BLAST( query, ... )
}
Note that a value channel is implicitly created by a process when it is invoked with a simple value, like the above file object. If you need to be able to BLAST multiple query files, you'll instead need a queue channel, which can be created using the fromPath factory method, for example:
params.query = "${baseDir}/data/*.fa"
params.db = "${baseDir}/blastdb/nt"
params.outdir = './results'
db_name = file(params.db).name
db_path = file(params.db).parent
process BLAST {
publishDir(
path: "{params.outdir}/blast",
mode: 'copy',
)
input:
tuple val(query_id), path(query)
path db
output:
tuple val(query_id), path("${query_id}.out")
"""
blastn \\
-num_threads ${task.cpus} \\
-query "${query}" \\
-db "${db}/${db_name}" \\
-out "${query_id}.out"
"""
}
workflow{
Channel
.fromPath( params.query )
.map { file -> tuple(file.baseName, file) }
.set { query_ch }
BLAST( query_ch, db_path )
}
Note that the usual way to specify the number of threads/cpus is using cpus directive, which can be configured using a process selector in your nextflow.config. For example:
process {
withName: BLAST {
cpus = 4
}
}
I tried to use Deno as a replacement for shell script, but got stuck.
I attempted to use Deno/Typescript to carry out the equivalent job as this:
docker run \
-d \
-v pgdata:/var/lib/postgresql/data \
--name pg \
-e POSTGRES_PASSWORD=123456 \
--rm \
-p 5432:5432 \
postgres
ts code looks like this:
function runCmd(s: string[]): Deno.Process {
return Deno.run({ cmd: s, stdout: "piped", stderr: "piped" })
}
function runPg() {
const cmd = [
"docker",
`run -d -v ${VOLUME}:/var/lib/postgresql/data --name pg -e POSTGRES_PASSWORD=${PASSWORD} --rm -p 5432:5432 postgres`
];
return runCmd(cmd);
}
add execution bit to this ts file and run it in terminal:
after this, I tried
function runPg() {
const cmd = [
"docker",
"run",
`-d -v ${VOLUME}:/var/lib/postgresql/data --name pg -e POSTGRES_PASSWORD=${PASSWORD} --rm -p 5432:5432 postgres`
];
return runCmd(cmd);
}
move out subcommand run from command options.
I got this:
I guess that Deno.run doesn't simply concatenate the passed-in string of command particles, but I cannot find enough information on this subject in order to fix the issue.
I haven't gone through the rust source code on this API, but I thought it's better to ask for help before trying the hard way.
You need to specify each part of the command as a separate string in the cmd array:
function runPg() {
const cmd = [
"docker",
"run",
"-d",
"-v",
`${VOLUME}:/var/lib/postgresql/data`,
"--name",
"pg",
"-e",
`POSTGRES_PASSWORD=${PASSWORD}`,
"--rm",
"-p",
"5432:5432",
"postgres"
];
return runCmd(cmd);
}
This will send run as the first argument to docker instead of sending run -d … as the first argument.
You can also build your command as a single string and then use split(" ") as long as no arguments contain spaces.
a follow up on my trial-n-error journey on this top.
While reading a book about unix shell programming, it points out a way to help the shell differentiate the space in identifier and the space as delimiter. When one tries to cat a file named a b (there is a space in between), the command should be cat a\ b or, using quotes, cat 'a b'.
This gives me an idea why my command does not work in Deno. See, each item in the cmd string list is an identifier, when I mix up delimiter-space with identifiers, the command is interpreted in the wrong way.
For example.
If I'd like to cat a file named a b. In Deno.run, I need to use { cmd: ["cat", "a b"] }.
If I'd like to cat two files named a and b. In Deno.run, I need to use { cmd: ["cat", "a", "b"] }.
Just remember that space in a command particle counts as a part of that term.
I use the following code to both output something to stdout, and pipe it to a program:
function example() {
local fd1
{
exec {fd1}>&1
{ echo hi >&$fd1 } | true
} always { exec {fd1}>&- }
}
I am wondering if I can safely drop always { exec {fd1}>&- }. fd1 goes out of scope after the function finishes anyways.
You need to keep always { exec {fd1}>&- }. If you get rid of that, the variable containing the file descriptor will go out of scope, but the file descriptor won't be closed, resulting in leaking it. You can see this by doing ls -l /proc/$$/fd before and after running your function without that line. Each run of the function will permanently add another FD to that list. Eventually, you'll run out of file descriptors and won't be able to open any new ones, which will break things.
When working with NodeJS, I can pass the arguments to a Node script like this:
$ node node-server.js arg1 arg2=arg2-val arg3
And can get the arguments like so:
// print process.argv
process.argv.forEach(function (val, index, array) {
console.log(index + ': ' + val);
});
//Output
0: node
1: /Users/umar/work/node/node-server.js
2: arg1
3: arg2=arg2-val
4: arg3
How to get the command-line arguments in Deno?
Some experts suggested me to solve the problem by answers to the question
Deno executable path ~ process.argv[0]:
Deno.execPath()
File URL of executed script ~ process.argv[1]:
Deno.mainModule
You can use path.fromFileUrl for conversions of URL to path string:
import { fromFileUrl } from "https://deno.land/std#0.55.0/path/mod.ts";
const modPath = fromFileUrl(import.meta.url)
Command-line arguments ~ process.argv.slice(2):
Deno.args
Example
deno run --allow-read test.ts -foo -bar=baz 42
Sample output (Windows):
Deno.execPath(): <scoop path>\apps\deno\current\deno.exe
import.meta.url: file:///C:/path/to/project/test.ts
as path: C:\path\to\project\test.ts
Deno.args: [ "-foo", "-bar=baz", "42" ]
To get your script’s CLI arguments in Deno, just use Deno.args:
> deno run ./code.ts foo bar
console.log(Deno.args); // ['foo', 'bar']
If you need something identical to Node's process.argv for compatibility reasons, use the official 'node' shim:
import process from 'https://deno.land/std#0.120.0/node/process.ts'
console.log(process.argv); // ['/path/to/deno', '/path/to/code.ts', 'foo', 'bar']
For illustrative purposes, if you wanted to manually construct a process.argv-style array (without using the official 'node' shim) you could do this:
import { fromFileUrl } from "https://deno.land/std#0.120.0/path/mod.ts";
const argv = [
Deno.execPath(),
fromFileUrl(Deno.mainModule),
...Deno.args,
]
console.log(argv); // ['/path/to/deno', '/path/to/code.ts', 'foo', 'bar']
I want to configure zsh to append the time each command started next to the line command was executed on. For example:
# before I press ENTER
$ ./script
# after I press enter
$ ./script [15:55:58]
Running script...
I came up with the following config (which also colors the timestamp yellow):
preexec () {
TIME=`date +"[%H:%M:%S] "`
echo -e "$1 %{$fg[yellow]%}\033[1A\033[1C${TIME}$reset_color"
}
But it breaks and prints { and % characters on basic commands such as cat and echo. It also breaks on password prompts (macOS terminal). For example with echo:
$ echo "hello" [15:55:58]
hello"hello" %{%}
How can I fix this config?
Thank you.
You inspired me and based on your script I wrote mine. I have tested this on zsh 5.4.smth.
preexec () {
local TIME=`date +"[%H:%M:%S] "`
local zero='%([BSUbfksu]|([FK]|){*})'
local PROMPTLEN=${#${(S%%)PROMPT//$~zero/}}
echo "\033[1A\033[$(($(echo -n $1 | wc -m)+$PROMPTLEN))C $fg[blue]${TIME}$reset_color"
}
In your ~/.zshrc file, put:
function preexec() {
timer=${timer:-$SECONDS}
}
function precmd() {
if [ $timer ]; then
timer_show=$(($SECONDS - $timer))
export RPROMPT="%F{cyan}${timer_show}s %F{$black%}"
unset timer
fi
}
And that should give you something like this: