Hoping someone kind can help me pls!
I have an file input.list:
/scratch/user/IFS/IFS001/IFS003.GATK.recal.bam
/scratch/user/IFS/IFS002/IFS002.GATK.recal.bam
/scratch/user/EGS/ZFXHG22/ZFXHG22.GATK.recal.bam
and I want to extract the bit before .GATK.recal.bam - I have found a solution for this:
sed 's/\.GATK\.recal\.bam.*//' input.list | sed 's#.*/##'
I now want to incorporate this into a while loop but it's not working... please can someone take a look and guide me where I'm doing wrong. My attempt is below:
while read -r line; do ID=${sed 's/\.GATK\.recal\.bam.*//' $line | sed 's#.*/##'}; sbatch script.sh $ID; done < input.list
Apologies for the easy Q...
You can use the output of the sed command as input for the loop:
sed 'COMMAND' input.file | while read -r id ; do
some_command "${id}"
done
Instead of the loop, also xargs could be used:
sed 'COMMAND' input.file | xargs -n1 some_command
ps: GNU sed supports to execute the result of a s operation as a command. I wouldn't recommend to use this in production, for portability reasons at least, but it's worth mention probably:
sed 's/\(.*\)\.GATK\.recal\.bam.*/sbatch script.sh \1/e' input.file
You can do this in straight up bash (If you're using that shell; ksh93 and zsh will be very similar) no sed required:
while read -r line; do
id="${line##*/}" # Remove everything up to the last / in the string
id="${id%.GATK.recal.bam}" # Remove the trailing suffix
sbatch script.sh "$id"
done < input.list
At the very least you can use a single sed call per line:
id=$(sed -e 's/\.GATK\.recal\.bam$//' -e 's#.*/##' <<<"$line")
or with plain sh
id=$(printf "%s\n" "$line" | sed -e 's/\.GATK\.recal\.bam$//' -e 's#.*/##')
Related
So I am new to SED and Unix and I would like to replace the following file:
1500:../someFile.C:111 error
1869:../anotherFile.C:222 error
1869:../anotherFile2.Cxx:333 error
//thousands of more lines with same structure
With the followig file
someFile.c
anotherFile.c
anotherFile2.Cxx
Basically, I just want to extract the filename from every line.
So far, I have read the documentation on sed and the second answer here. My best attempt was to use a regex as follows:
sed "s/.\*\/.:.*//g" myFile.txt
Lots of ways to do this.
Sure, you could use sed:
sed 's/^[^:]*://;s/:.*//;s#\.\./##' input.txt
sed 's%.*:\.\./\([^:]*\):.*%\1%' input.txt
Or you could use a series of grep -o instances in a pipe:
grep -o ':[^:]*:' input.txt | grep -o '[^:]\{1,\}' | grep -o '/.*' | grep -o '[^/]\{1,\}'
You could even use awk:
awk -F: '{sub(/\.\.\//,"",$2); print $2}' input.txt
But the simplest way would probably be to use cut:
cut -d: -f2 input.txt | cut -d/ -f2
You can capture the substring between last / and following : and replace the whole string with the captured string(\1).
sed 's#.*/\([^:]\+\).*#\1#g' myFile.txt
someFile.C
anotherFile.C
anotherFile2.Cxx
OR , with little less escaping, sed with -r flag.
sed -r 's#.*/([^:]+).*#\1#g' myFile.txt
Or if you want to use grep,this will only work if your grep supports -P flag which will enable PCRE:
grep -oP '.*/\K[^:]+' myFile.txt
someFile.C
anotherFile.C
anotherFile2.Cxx
I want to do the following substitution in a text file:
the original string: "---a---"
after substitution : "---\a---"
and I run the following command:
sed -r -e "s/-(a)-/-\\\1-/g" test.txt
but it doesn't give the right result. What command args should I use?
Remember that backslashes are significant in Bash's double-quoted strings as well as in sed itself. Either use single quotes:
sed -r -e 's/-(a)-/-\\\1-/g' test.txt
Or escape the backslashes again:
sed -r -e "s/-(a)-/-\\\\\\1-/g" test.txt
If you echo the strings, you'll see what's happening:
$ echo "s/-(a)-/-\\\1-/g"
s/-(a)-/-\\1-/g
$ echo 's/-(a)-/-\\\1-/g'
s/-(a)-/-\\\1-/g
$ echo "s/-(a)-/-\\\\\\1-/g"
s/-(a)-/-\\\1-/g
The first one (your original) just looks like a literal backslash followed by a literal 1 to sed.
Try sed -r -e "s/-(a)-/-\\\\\\1-/g" or sed -r -e 's/-(a)-/-\\\1-/g'
The problem is that \ is captured by bash if you use double quotes.
With " you will have to do something like:
[jaypal:~/Temp] echo "---a---" | sed -r "s/-(a)-/-\\\\\1-/g"
---\a---
You have to replace 1 with a
sed -r -e "s/-(a)-/-\\\a-/g" test.txt
so many answers ......
Kaizen ~/so_test
$ echo "---a---" | sed -n 's/a/\\a/p'
---\a---
since you have a text file the following should work :
sed -i 's/a/\\a/g' filename.txt ;
does this help ?
I am having a log file a.log and i need to extract a piece of information from it.
To locate the start and end line numbers of the pattern i am using the following.
start=$(sed -n '/1112/=' file9 | head -1)
end=$(sed -n '/true/=' file9 | head -1)
i need to use the variables (start,end) in the following command:
sed -n '16q;12,15p' orig-data-file > new-file
so that the above command appears something like:
sed -n '($end+1)q;$start,$end'p orig-data-file > new-file
I am unable to replace the line numbers with the variables. Please suggest the correct syntax.
Thanks,
Rosy
When I realized how to do it, I was looking for anyway to get line number into a file containing the requested info, and display the file from that line to EOF.
So, this was my way.
with
PATTERN="pattern"
INPUT_FILE="file1"
OUTPUT_FILE="file2"
line number of first match of $PATTERN into $INPUT_FILE can be retrieved with
LINE=`grep -n ${PATTERN} ${INPUT_FILE} | awk -F':' '{ print $1 }' | head -n 1`
and the outfile will be the text from that $LINE to EOF. This way:
sed -n ${LINE},\$p ${INPUT_FILE} > ${OUTPUT_FILE}
The point here, is the way how can variables be used with command sed -n:
first witout using variables
sed -n 'N,$p' <file name>
using variables
LINE=<N>; sed -n ${LINE},\$p <file name>
Remove the single quotes thus. Single quotes turn off the shell parsing of the string. You need shell parsing to do the variable string replacements.
sed -n '('$end'+1)q;'$start','$end''p orig-data-file > new-file
Im trying to print only lines that do not start with a letter from the file "main"
Ive tried sed -n '/^[a-z]/ /!w' main
and it gives me "w': event not found"
With sed as requested:
sed '/^[[:alpha:]]/d' main
or
sed -n '/^[^[:alpha:]]/p' main
or
sed -n '/^[[:alpha:]]/!p' main
Note: you could use [a-z] inplace of [[:alpha:]] but I prefer the latter because it is safe to use across different locales
there are many other ways to print lines
sed -n '/^[^a-zA-Z]/p' main
sed -n '/^[^a-z]/Ip' main
awk 'BEGIN{IGNORECASE=1}!/^[a-z]/' main
grep -vi "^[a-z]" main
ruby -ne 'print unless /^[a-z]/i' main
shell
while read -r line
do
case "$line" in
[^a-zA-Z]*) echo $line;;
esac
done < main
grep -v '^[a-z]' main
will do it.
is there a way to search and replace a string using single unix command grep recusrsively in multiple directories?
i know it can be done by using the combination of find with other utilities like sed perl etc.but is there a way where we can use only grep for doing this on unix command line?
I don't think that only grep would work here; involving sed and other utilities will be much more easier, than just grep
one way, if you have GNU find and bash shell
find /path -type f -iname "*.txt" | while read -r FILE
do
while read -r LINE
do
case "$LINE" in
*WORD_TO_SEARCH* ) LINE=${LINE//WORD_TO_SEARCH/REPLACE};;
esac
echo "$LINE" >> temp
done < "$FILE"
mv temp "$FILE"
done