I want to log everything from the rsyslogd executable (daemon) to a file.
# /etc/rsyslog.conf
if $syslogtag isequal 'rsyslogd' then /tmp/foo.log
if $syslogtag isequal 'rsyslogd:' then /tmp/foo.log
if $syslogtag isequal ' rsyslogd' then /tmp/foo.log
if $syslogtag isequal ' rsyslogd:' then /tmp/foo.log
In a separate shell:
sv down /root/service/rsyslog/
sv up /root/service/rsyslog/
tail -F /tmp/foo.log
Result: bupkus
So I edit rsyslog.conf:
# /etc/rsyslog.conf
if $syslogtag contains 'rsyslogd' then /tmp/foo.log
Rinse and repeat; in a separate shell:
sv down /root/service/rsyslog/
sv up /root/service/rsyslog/
tail -F /tmp/foo.log
Tada: 20190424_195027 linuxbox info rsyslogd: [origin software="rsyslogd" swVersion="8.28.0" x-pid="27384" x-info="http://www.rsyslog.com"] start
What gives? I don't get why the attempts at using isequals fail.
The %syslogtag% looks like some variation of "rsyslogd" either with whitespace or a colon attached somewhere - but I think I've gone through every reasonable permutation thereof, so why is the isequal comparison failing?
You need to look for error messages during the parsing of your configuration (for example with rsyslogd -N1), as your lines are being ignored. The isequal operator is used with this sort of syntax:
:syslogtag, isequal, "rsyslogd:" ./output1
whereas the if..then syntax needs the == operator:
if $syslogtag == 'rsyslogd:' then ./output2
The contains operator works with both syntaxes.
Related
I use the snakemkae to create a pipeline to split bam by chr,but I met a problem,
Wildcards in input files cannot be determined from output files:
'OutputDir'
Can someone help me to figure it out ?
if config['ref'] == 'hg38':
ref_chr = []
for i in range(1,23):
ref_chr.append('chr'+str(i))
ref_chr.extend(['chrX','chrY'])
elif config['ref'] == 'b37':
ref_chr = []
for i in range(1,23):
ref_chr.append(str(i))
ref_chr.extend(['X','Y'])
rule all:
input:
expand(f"{OutputDir}/split/{name}.{{chr}}.bam",chr=ref_chr)
rule minimap2:
input:
TargetFastq
output:
Sortbam = "{OutputDir}/{name}.sorted.bam",
Sortbai = "{OutputDir}/{name}.sorted.bam.bai"
resources:
mem_mb = 40000
threads: nt
singularity:
OntSoftware
shell:
"""
minimap2 -ax map-ont -d {ref_mmi} --MD -t {nt} {ref_fasta} {input} | samtools sort -O BAM -o {output.Sortbam}
samtools index {output.Sortbam}
"""
rule split_bam:
input:
rules.minimap2.output.Sortbam
output:
splitBam = expand(f"{OutputDir}/split/{name}.{{chr}}.bam",chr=ref_chr),
splitBamBai = expand(f"{OutputDir}/split/{name}.{{chr}}.bam.bai",chr=ref_chr)
resources:
mem_mb = 30000
threads: nt
singularity:
OntSoftware
shell:
"""
samtools view -# {nt} -b {input} {chr} > {output.splitBam}
samtools index -# {nt} {output.splitBam}
"""
I change the wilcards {outputdir},but is dose not help.
expand(f"{OutputDir}/split/{name}.{{chr}}.bam",chr=ref_chr),
splitBamBai = expand(f"{OutputDir}/split/{name}.{{chr}}.bam.bai",chr=ref_chr),
A couple of comments on this lines...:
You escape chr by using double braces, {{chr}}. This means you don't want chr to be expanded, which I doubt it is correct. I suspect you want something like:
expand("{{OutputDir}}/split/{{name}}.{chr}.bam",chr=ref_chr),
The rule minimpa2 does not contain {chr} wildcard, hence the error you get.
As an aside, when you create a bam file and its index in the same rule, you can get the time stamp of the index file to be older than the bam file itself. This later can generate spurious warning from samtools/bcftools. See https://github.com/snakemake/snakemake/issues/1378 (not sure if it's been fixed).
I want to generate a text file containing the folowing lines:
http://example.com/file1.pdf
http://example.com/file2.pdf
http://example.com/file3.pdf
.
.
http://example.com/file1000.pdf
Can any one advise how to do it using unix command line, please?
Thank you
With an interating for loop
for (( i=1;i<=1000;i++ ));
do
echo "http://example.com/file$i.pdf";
done > newfile
With seq:
while read i;
do
echo "http://example.com/file$i.pdf";
done <<< $(seq 1000) > newfile
It is possible to create/run a python script file ato generate this. Using vim, nano, or any other terminal editor, create a python file as follows:
def genFile(fn, start, end):
with open(fn, "w+") as f:
f.writelines([f"http://example.com/file{str(i)}.pdf\n" for i in range(start, end+1)])
try:
fn = input("File Path: ") # can be relative
start = int(input("Start: ")) # inclusive
end = int(input("End: ")) # inclusive
genFile(fn, start, end)
except:
print("Invalid Input")
Once this is written to a file, let's call it script.py. We can run the following command to execute the script:
python script.py
Then, fill out the prompts for the file path, start, and end. This should result in all those lines printed in the file specified delimited by '\n'.
In julia, how do I check if the current is allowed to write to a folder?
I could do the python way, and just attempt to do it, and then fail fail and recover.
(In my case I can definitely recover, I have a list of locations to attempt to write to, as fallbacks. I expect the first few not to work (The first few are shared locations, so only computer admins are likely to have permission to writer there)
Python has also os.access function. Maybe Julia will have something similar in the future. Now we could borrow idea. :)
It is implemented in posixmodule.c (also functionality for windows!) so if you are on posix you could simply mimic:
julia> const R_OK = 4 # readability
julia> const W_OK = 2 # writability
julia> const X_OK = 1 # executability
julia> const F_OK = 4 # existence
julia> access(path, mode) = ccall(:access, Cint, (Cstring, Cint), path, mode) == 0;
Small test:
julia> access("/root", W_OK)
false
julia> access("/tmp", W_OK)
true
(for windows it could be just a little more complicated... But I could not test it now)
EDIT:
Thanks to Matt B. we could use libuv support in Julia which has to be portable (although slower on posix systems):
julia> function uv_access(path, mode)
local ret
req = Libc.malloc(Base._sizeof_uv_fs)
try
ret = ccall(:uv_fs_access, Int32, (Ptr{Void}, Ptr{Void}, Cstring, Int64, Ptr{Void}), Base.eventloop(), req, path, mode, C_NULL)
ccall(:uv_fs_req_cleanup, Void, (Ptr{Void},), req)
finally
Libc.free(req)
end
return ret, ret==0 ? "OK" : Base.struverror(ret)
end
julia> uv_access("/tmp", W_OK)
(0, "OK")
julia> uv_access("/root", W_OK)
(-13, "permission denied")
julia> uv_access("/nonexist", W_OK)
(-2, "no such file or directory")
Is the following sufficient:
julia> testdir(dirpath) = try (p,i) = mktemp(dirpath) ; rm(p) ; true catch false end
testdir (generic function with 1 method)
julia> testdir("/tmp")
true
julia> testdir("/root")
false
Returns true if dirpath is writable (by creating a temporary file inside a try-catch block). To find the first writable directory in a list, the following can be used:
julia> findfirst(testdir, ["/root","/tmp"])
2
Doing apropos("permissions"):
julia> apropos("permissions")
Base.Filesystem.gperm
Base.Filesystem.mkpath
Base.Filesystem.operm
Base.Filesystem.uperm
Base.Filesystem.mkdir
Base.Filesystem.chmod
shows a function called Base.Filesystem.uperm which seems to do exactly what you want it to:
help?> uperm
search: uperm supertype uppercase UpperTriangular isupper unescape_string unsafe_pointer_to_objref
uperm(file)
Gets the permissions of the owner of the file as a bitfield of
Value Description
––––– ––––––––––––––––––
01 Execute Permission
02 Write Permission
04 Read Permission
For allowed arguments, see stat.
Unfortunately it seems to be a bit buggy on my (old v7 nightly) build:
julia> uperm("/root")
0x07 # Uhhh I hope not?
I will update my build and raise a bug if one is not already present.
PS. In case it wasn't clear, I would expect to use this in combination with isdir to detect directory permissions specifically
I don't think that Dan Getz's answer will work on Windows because the temporary file created cannot be deleted while there is an open handle to it, but this amended version with a call to close does work:
function isfolderwritable(folder)
try
(p,i) = mktemp(folder)
close(i)
rm(p)
return(true)
catch
return(false)
end
end
I've given a look around about what puzzles me and I only found this:
Do some programs not accept process substitution for input files?
which is partially helping, but I really would like to understand the full story.
I noticed that some of my R scripts give different (ie. wrong) results when I use process substitution.
I tried to pinpoint the problem with a test case:
This script:
#!/usr/bin/Rscript
args <- commandArgs(TRUE)
file <-args[1]
cat(file)
cat("\n")
data <- read.table(file, header=F)
cat(mean(data$V1))
cat("\n")
with an input file generated in this way:
$ for i in `seq 1 10`; do echo $i >> p; done
$ for i in `seq 1 500`; do cat p >> test; done
leads me to this:
$ ./mean.R test
test
5.5
$ ./mean.R <(cat test)
/dev/fd/63
5.501476
Further tests reveal that some lines are lost...but I would like to understand why. Does read.table (scan gives the same results) uses seek?
Ps.
with a smaller test file (100) an error is reported:
$./mean.R <(cat test3)
/dev/fd/63
Error in read.table(file, header = F) : no lines available in input
Execution halted
Add #1: with a modified script that uses scan the results are the same.
I have written this general purpose function for opening a file connection in my own scripts:
OpenRead <- function(arg) {
if (arg %in% c("-", "/dev/stdin")) {
file("stdin", open = "r")
} else if (grepl("^/dev/fd/", arg)) {
fifo(arg, open = "r")
} else {
file(arg, open = "r")
}
}
In your code, replace file with file <- OpenRead(file) and it should handle all of the below:
./mean.R test
./mean.R <(cat test)
cat test | ./mean.R -
cat test | ./foo.R /dev/stdin
I'm new to shell scripting and need some help. I am trying to write a script to bounce some servers and I am having a few issues with my if statements. The First and Second one below is giving me a too many arguments error.
For the first one, I am the variable $jmsProcess is a ps -ef | grep command and I only want to go into the if-statement, if this returns some results. This is the same issue for the second one.
In the Third if-statement I want it to check if either of those variables have the value true but this gives me a
if[ [ false || false ] == true ]: command not found
Error.
#Check the JMS process has been killed
if [ $jmsProcess != null ] # SHOULD THIS BE NULL???
then
echo "JMS Process is still running"
$jmsRunning = "true"
fi
#Check the Bone process has been killed
if [ $boneProcess != null ] # SHOULD THIS BE NULL???
then
echo "B-One Process is still Running"
$boneRunning = "true"
fi
if[ [ $jmsRunning || $boneRunning ] == true ] # CHECK THIS FOR QUOTES
then
# $killProcess
fi
null is not a Bash keyword. If you want to check whether a variable is defined, you can do the following:
if [ "${var+defined}" = defined ]
To check whether it's empty or undefined:
if [ -z "${var-}" ]
We don't know how you're setting any of the variable values, (jmsProcess, boneProcess).
Try surrounding all var values like "$var" (with dbl-quotes) and see if the error messages change.
Also there are numerous syntax issues in code visible above. Hard to tell if it is an artifact of posting here, (The code block feature is pretty robust), so I'm going to comment on what I see wrong.
if[ [ false || false ] == true ]: command not found
There are a lot of issues here: false is an shell command. Try typing it on the command line and then do echo $?. you should see 1. true; echo $? will return 0. But the if statements continue or fall-over to the else block based on the last return code (with some special case exceptions).
Also, I'm guessing you're trying to make some sort of reg exp with [ false || false ] == true. Won't work. see below.
You can set status variables to have the value of false (or true) which will be evaluated correctly by the shell.
Also, if[ will give the 'command not found' msg. So by using vars that have the value false, you can do
Try
jmsRunning=false ; boneRunning=true
if [[ ${jmsRunning} || ${boneRunning} ]] ; then
echo both NOT running
else
echo at least 1 is running
fi
Change both to false to see the message change.
Also, null is just a string in a shell script, you probably mean "".
Finally, var assignments cannot have spaces surrounding the '=' sign AND do not use the '$' char at the front when it is on the left hand side of the statment, i.e.
boneRunning=true
I hope this helps.