i'm new to nf-core/nextflow and needless to say the documentation does not reflect what might be actually implemented. But i'm defining the basic pipeline below:
nextflow.enable.dsl=2
process RUNBLAST{
input:
val thr
path query
path db
path output
output:
path output
script:
"""
blastn -query ${query} -db ${db} -out ${output} -num_threads ${thr}
"""
}
workflow{
//println "I want to BLAST $params.query to $params.dbDir/$params.dbName using $params.threads CPUs and output it to $params.outdir"
RUNBLAST(params.threads,params.query,params.dbDir, params.output)
}
Then i'm executing the pipeline with
nextflow run main.nf --query test2.fa --dbDir blast/blastDB
Then i get the following error:
N E X T F L O W ~ version 22.10.6
Launching `main.nf` [dreamy_hugle] DSL2 - revision: c388cf8f31
Error executing process > 'RUNBLAST'
Error executing process > 'RUNBLAST'
Caused by:
Not a valid path value: 'test2.fa'
Tip: you can replicate the issue by changing to the process work dir and entering the command bash .command.run
I know test2.fa exists in the current directory:
(nfcore) MN:nf-core-basicblast jraygozagaray$ ls
CHANGELOG.md conf other.nf
CITATIONS.md docs pyproject.toml
CODE_OF_CONDUCT.md lib subworkflows
LICENSE main.nf test.fa
README.md modules test2.fa
assets modules.json work
bin nextflow.config workflows
blast nextflow_schema.json
I also tried with "file" instead of path but that is deprecated and raises other kind of errors.
It'll be helpful to know how to fix this to get myself started with the pipeline building process.
Shouldn't nextflow copy the file to the execution path?
Thanks
You get the above error because params.query is not actually a path value. It's probably just a simple String or GString. The solution is to instead supply a file object, for example:
workflow {
query = file(params.query)
BLAST( query, ... )
}
Note that a value channel is implicitly created by a process when it is invoked with a simple value, like the above file object. If you need to be able to BLAST multiple query files, you'll instead need a queue channel, which can be created using the fromPath factory method, for example:
params.query = "${baseDir}/data/*.fa"
params.db = "${baseDir}/blastdb/nt"
params.outdir = './results'
db_name = file(params.db).name
db_path = file(params.db).parent
process BLAST {
publishDir(
path: "{params.outdir}/blast",
mode: 'copy',
)
input:
tuple val(query_id), path(query)
path db
output:
tuple val(query_id), path("${query_id}.out")
"""
blastn \\
-num_threads ${task.cpus} \\
-query "${query}" \\
-db "${db}/${db_name}" \\
-out "${query_id}.out"
"""
}
workflow{
Channel
.fromPath( params.query )
.map { file -> tuple(file.baseName, file) }
.set { query_ch }
BLAST( query_ch, db_path )
}
Note that the usual way to specify the number of threads/cpus is using cpus directive, which can be configured using a process selector in your nextflow.config. For example:
process {
withName: BLAST {
cpus = 4
}
}
I have a simple PowerShell script in which I try to write some log data.
The script that's not working:
Add-Content -Path C:\temp\logg.txt -Value $file.FullName
This line I placed in the middle of the script and I never get any text in the logg.txt. The Script executes fine except this, it compiles some files to a zip file.
This is on a Windows server.
If I, however, run the same script on my local machine, Win7, it does work!?
So I think it must be something with the server but I can't figure it out. Am using -ExecutionPolicy Bypass when I run the script.
EDIT: more info.
The script is called from an ASP.NET webpage:
Process.Start("Powershell.exe", "-ExecutionPolicy Bypass -Command ""& {" & destinationFolderRoot.Replace("\\", "\") & "\scriptParameters.ps1}""")
BaseZipScript:
function New-Zip
{
param([string]$zipfilename)
Set-Content $zipfilename ("PK" + [char]5 + [char]6 + ("$([char]0)" * 18))
(dir $zipfilename).IsReadOnly = $false
}
function Add-Zip
{
param([string]$zipfilename)
if(-not (Test-Path($zipfilename)))
{
Set-Content $zipfilename ("PK" + [char]5 + [char]6 + ("$([char]0)" * 18))
(dir $zipfilename).IsReadOnly = $false
}
$shellApplication = New-Object -COM Shell.Application
$zipPackage = $shellApplication.NameSpace($zipfilename)
foreach($file in $input)
{
$zipPackage.CopyHere($file.FullName)
Add-Content -Path "C:\temp\logg.txt" -Value $file.FullName
Start-Sleep -Milliseconds 750
}
}
scriptParameters1:
. "C:\temp\multiDownload\baseZipScript.ps1"
New-Zip c:\temp\multiDownload\user379\2016-09-15_14.39\files_2016-09-15_14.39.zip
dir c:\temp\multiDownload\user379\2016-09-15_14.39\tempCopy\*.* -Recurse |
Add-Zip c:\temp\multiDownload\user379\2016-09-15_14.39\files_2016-09-15_14.39.zip
Start-sleep -milliseconds 250
exit
As I said earlier the script works on local machine and also on the server, e.g. the files are being zipped, except that on the server nothing is logged in logg.txt.
Though the logg.txt works and exists on the server because the webpage also log some info there and that is being written.
Update:
As the comments guessed it was as simple as write access to the file. Though, it is kind of wierd because all files are created from the webservice if they doesn't exists? Any how, it works now. Thx! : )
I'm trying to test out asynchronous functionality in PowerShell 3.
So I figured I would query the uptimes of some servers remotely, and wrote the following script:
function getServerUptimes() {
# Obtain credentials for logging into
# the remote machines...
$cred = Get-Credential
$compsHash = #{
"server1.domain.com" = $null;
"server2.domain.com" = $null;
}
# Define an async block to run...no blocking in this script...
$cmd = {
param($computer, $dasCred)
Write-Host "which: $($computer)"
$session = new-cimsession $computer -Credential $dasCred
return (get-CimInstance win32_operatingsystem -CimSession $session | Select PScomputername, LastBootuptime);
}
ForEach($comp in $compsHash.Keys) {
Write-Host "${comp}"
# Kick off an async process to create a cimSession
# on each of these machines...
Start-Job -ScriptBlock $cmd -ArgumentList $comp, $cred -Name "Get$($comp)"
}
$results = Get-Job | Wait-Job | Receive-Job
# Retrieve the job, so we can look at the output
ForEach($comp in $compHash.Keys) {
$dasJob = Get-Job -Name "Get$($comp)"
Write-Host $dasJob.output
}
}
However, I don't really seem to get back any output in the resulting $dasJob object, I returned the value in my scriptblock, where is it going?
You already have the output (not including Write-Host output, of course) in the variable $results. In PowerShell you retrieve job output via Receive-Job. The property Output seems to always be empty (not sure why).
What is the flow of execution of the time command in detail?
I have a user created function in PowerShell, which will compute the time for execution of the command in the following way.
It will open the new PowerShell window.
It will execute the command.
It will close the PowerShell window.
It will get the the different execution times using the GetProcessTimes function function.
Is the "time command" in Unix also calculated in the same way?
The Measure-Command cmdlet is your friend.
PS> Measure-Command -Expression {dir}
You could also get execution time from the command history (last executed command in this example):
$h = Get-History -Count 1
$h.EndExecutionTime - $h.StartExecutionTime
I've been doing this:
Time {npm --version ; node --version}
With this function, which you can put in your $profile file:
function Time([scriptblock]$scriptblock, $name)
{
<#
.SYNOPSIS
Run the given scriptblock, and say how long it took at the end.
.DESCRIPTION
.PARAMETER scriptBlock
A single computer name or an array of computer names. You mayalso provide IP addresses.
.PARAMETER name
Use this for long scriptBlocks to avoid quoting the entire script block in the final output line
.EXAMPLE
time { ls -recurse}
.EXAMPLE
time { ls -recurse} "All the things"
#>
if (!$stopWatch)
{
$script:stopWatch = new-object System.Diagnostics.StopWatch
}
$stopWatch.Reset()
$stopWatch.Start()
. $scriptblock
$stopWatch.Stop()
if ($name -eq $null) {
$name = "$scriptblock"
}
"Execution time: $($stopWatch.ElapsedMilliseconds) ms for $name"
}
Measure-Command works, but it swallows the stdout of the command being run. (Also see Timing a command's execution in PowerShell)
If you need to measure the time taken by something, you can follow this blog entry.
Basically, it suggest to use the .NET StopWatch class:
$sw = [System.Diagnostics.StopWatch]::startNew()
# The code you measure
$sw.Stop()
Write-Host $sw.Elapsed
How do I ask PowerShell where something is?
For instance, "which notepad" and it returns the directory where the notepad.exe is run from according to the current paths.
The very first alias I made once I started customizing my profile in PowerShell was 'which'.
New-Alias which get-command
To add this to your profile, type this:
"`nNew-Alias which get-command" | add-content $profile
The `n at the start of the last line is to ensure it will start as a new line.
Here is an actual *nix equivalent, i.e. it gives *nix-style output.
Get-Command <your command> | Select-Object -ExpandProperty Definition
Just replace with whatever you're looking for.
PS C:\> Get-Command notepad.exe | Select-Object -ExpandProperty Definition
C:\Windows\system32\notepad.exe
When you add it to your profile, you will want to use a function rather than an alias because you can't use aliases with pipes:
function which($name)
{
Get-Command $name | Select-Object -ExpandProperty Definition
}
Now, when you reload your profile you can do this:
PS C:\> which notepad
C:\Windows\system32\notepad.exe
I usually just type:
gcm notepad
or
gcm note*
gcm is the default alias for Get-Command.
On my system, gcm note* outputs:
[27] » gcm note*
CommandType Name Definition
----------- ---- ----------
Application notepad.exe C:\WINDOWS\notepad.exe
Application notepad.exe C:\WINDOWS\system32\notepad.exe
Application Notepad2.exe C:\Utils\Notepad2.exe
Application Notepad2.ini C:\Utils\Notepad2.ini
You get the directory and the command that matches what you're looking for.
Try this example:
(Get-Command notepad.exe).Path
My proposition for the Which function:
function which($cmd) { get-command $cmd | % { $_.Path } }
PS C:\> which devcon
C:\local\code\bin\devcon.exe
A quick-and-dirty match to Unix which is
New-Alias which where.exe
But it returns multiple lines if they exist so then it becomes
function which {where.exe command | select -first 1}
I like Get-Command | Format-List, or shorter, using aliases for the two and only for powershell.exe:
gcm powershell | fl
You can find aliases like this:
alias -definition Format-List
Tab completion works with gcm.
To have tab list all options at once:
set-psreadlineoption -editmode emacs
This seems to do what you want (I found it on http://huddledmasses.org/powershell-find-path/):
Function Find-Path($Path, [switch]$All = $false, [Microsoft.PowerShell.Commands.TestPathType]$type = "Any")
## You could comment out the function stuff and use it as a script instead, with this line:
#param($Path, [switch]$All = $false, [Microsoft.PowerShell.Commands.TestPathType]$type = "Any")
if($(Test-Path $Path -Type $type)) {
return $path
} else {
[string[]]$paths = #($pwd);
$paths += "$pwd;$env:path".split(";")
$paths = Join-Path $paths $(Split-Path $Path -leaf) | ? { Test-Path $_ -Type $type }
if($paths.Length -gt 0) {
if($All) {
return $paths;
} else {
return $paths[0]
}
}
}
throw "Couldn't find a matching path of type $type"
}
Set-Alias find Find-Path
Check this PowerShell Which.
The code provided there suggests this:
($Env:Path).Split(";") | Get-ChildItem -filter notepad.exe
Try the where command on Windows 2003 or later (or Windows 2000/XP if you've installed a Resource Kit).
BTW, this received more answers in other questions:
Is there an equivalent of 'which' on Windows?
PowerShell equivalent to Unix which command?
If you want a comamnd that both accepts input from pipeline or as paramater, you should try this:
function which($name) {
if ($name) { $input = $name }
Get-Command $input | Select-Object -ExpandProperty Path
}
copy-paste the command to your profile (notepad $profile).
Examples:
❯ echo clang.exe | which
C:\Program Files\LLVM\bin\clang.exe
❯ which clang.exe
C:\Program Files\LLVM\bin\clang.exe
I have this which advanced function in my PowerShell profile:
function which {
<#
.SYNOPSIS
Identifies the source of a PowerShell command.
.DESCRIPTION
Identifies the source of a PowerShell command. External commands (Applications) are identified by the path to the executable
(which must be in the system PATH); cmdlets and functions are identified as such and the name of the module they are defined in
provided; aliases are expanded and the source of the alias definition is returned.
.INPUTS
No inputs; you cannot pipe data to this function.
.OUTPUTS
.PARAMETER Name
The name of the command to be identified.
.EXAMPLE
PS C:\Users\Smith\Documents> which Get-Command
Get-Command: Cmdlet in module Microsoft.PowerShell.Core
(Identifies type and source of command)
.EXAMPLE
PS C:\Users\Smith\Documents> which notepad
C:\WINDOWS\SYSTEM32\notepad.exe
(Indicates the full path of the executable)
#>
param(
[String]$name
)
$cmd = Get-Command $name
$redirect = $null
switch ($cmd.CommandType) {
"Alias" { "{0}: Alias for ({1})" -f $cmd.Name, (. { which $cmd.Definition } ) }
"Application" { $cmd.Source }
"Cmdlet" { "{0}: {1} {2}" -f $cmd.Name, $cmd.CommandType, (. { if ($cmd.Source.Length) { "in module {0}" -f $cmd.Source} else { "from unspecified source" } } ) }
"Function" { "{0}: {1} {2}" -f $cmd.Name, $cmd.CommandType, (. { if ($cmd.Source.Length) { "in module {0}" -f $cmd.Source} else { "from unspecified source" } } ) }
"Workflow" { "{0}: {1} {2}" -f $cmd.Name, $cmd.CommandType, (. { if ($cmd.Source.Length) { "in module {0}" -f $cmd.Source} else { "from unspecified source" } } ) }
"ExternalScript" { $cmd.Source }
default { $cmd }
}
}
Use:
function Which([string] $cmd) {
$path = (($Env:Path).Split(";") | Select -uniq | Where { $_.Length } | Where { Test-Path $_ } | Get-ChildItem -filter $cmd).FullName
if ($path) { $path.ToString() }
}
# Check if Chocolatey is installed
if (Which('cinst.bat')) {
Write-Host "yes"
} else {
Write-Host "no"
}
Or this version, calling the original where command.
This version also works better, because it is not limited to bat files:
function which([string] $cmd) {
$where = iex $(Join-Path $env:SystemRoot "System32\where.exe $cmd 2>&1")
$first = $($where -split '[\r\n]')
if ($first.getType().BaseType.Name -eq 'Array') {
$first = $first[0]
}
if (Test-Path $first) {
$first
}
}
# Check if Curl is installed
if (which('curl')) {
echo 'yes'
} else {
echo 'no'
}
You can install the which command from https://goprogram.co.uk/software/commands, along with all of the other UNIX commands.
If you have scoop you can install a direct clone of which:
scoop install which
which notepad
There also always the option of using which. there are actually three ways to access which from Windows powershell, the first (not necessarily the best) wsl -e which command (this requires installation of windows subsystem for Linux and a running distro). B. gnuwin32 which is a port of several gnu binaries in .exe format as standle alone bundled lanunchers option three, install msys2 (cross compiler platform) if you go where it installed in /usr/bin you'll find many many gnu utils that are more up-to-date. most of them work as stand alone exe and can be copied from the bin folder to your home drive somewhere amd added to your PATH.
There also always the option of using which. there are actually three ways to access which from Windows powershell
The first, (though not the best) is wsl(windows subsystem for linux)
wsl -e which command
This requires installation of windows subsystem for Linux and a running distro.
Next is gnuwin32 which is a port of several gnu binaries in .exe format as standle alone bundled lanunchers
Third, install msys2 (cross compiler platform) if you go where it installed in /usr/bin you'll find many many gnu utils that are more up-to-date. most of them work as stand alone exe and can be copied from the bin folder to your home drive somewhere amd added to your PATH.