I have a file which has ^# in it and I am unable to remove it using sed or replace command in python. I can see ^# only when I open the file in vi editor. Please suggest. Below is what i tried using sed.
sed 's/^#/?/g' filename
Tested on Linux, not sure if syntax varies elsewhere, try
$ printf 'abc\0baz\n' | cat -v
abc^#baz
$ printf 'abc\0baz\n' | tr -d '\0' | cat -v
abcbaz
tr will delete all ASCII NUL characters from input.. cat -v is used here to highlight non-printing characters
for file input, use tr -d '\0' <filename
GNU sed (and possibly few other implementations) allow to use hex value to represent a character
$ printf 'abc\0baz\n' | sed 's/\x00//g' | cat -v
abcbaz
so, for in-place editing, use sed -i 's/\x00//g' filename (See also: sed in-place flag that works both on Mac (BSD) and Linux )
Related
I have the below output from unix:
$ diff -y --suppress-common-lines backup.txt newfile.txt
> `jjj' int,
i need only jjj : int as output.
tried the below didnt work as expected:
$ diff -y --suppress-common-lines backup.txt newfile.txt | grep -i '>' |tr -d '[>]' |sed 's/,//g'
suggesting to try gawk script:
diff -y --suppress-common-lines backup.txt newfile.txt | gawk '{print $1 ":" $2}' FPAT="[[:alnum:]]+"
The most common reasons for this not working are:
Your file is encoded as a non ASCII file, most commonly in UTF-8.
(Save the text files as ASCII.)
You are running this in a command shell with colors.
(Colors are actually ANSI characters and messes up sed.)
You have encoded your file with a different EOL than used in your *nix OS (\n), such as \r\n (Windows) or \r (MacOS).
There are hidden TAB (\t) characters in the file.
After you have fixed the above, try this:
diff -Ewy -r --suppress-common-lines -aB -W 512 file.txt file2.txt | tr -d '[>]'
Hoping someone kind can help me pls!
I have an file input.list:
/scratch/user/IFS/IFS001/IFS003.GATK.recal.bam
/scratch/user/IFS/IFS002/IFS002.GATK.recal.bam
/scratch/user/EGS/ZFXHG22/ZFXHG22.GATK.recal.bam
and I want to extract the bit before .GATK.recal.bam - I have found a solution for this:
sed 's/\.GATK\.recal\.bam.*//' input.list | sed 's#.*/##'
I now want to incorporate this into a while loop but it's not working... please can someone take a look and guide me where I'm doing wrong. My attempt is below:
while read -r line; do ID=${sed 's/\.GATK\.recal\.bam.*//' $line | sed 's#.*/##'}; sbatch script.sh $ID; done < input.list
Apologies for the easy Q...
You can use the output of the sed command as input for the loop:
sed 'COMMAND' input.file | while read -r id ; do
some_command "${id}"
done
Instead of the loop, also xargs could be used:
sed 'COMMAND' input.file | xargs -n1 some_command
ps: GNU sed supports to execute the result of a s operation as a command. I wouldn't recommend to use this in production, for portability reasons at least, but it's worth mention probably:
sed 's/\(.*\)\.GATK\.recal\.bam.*/sbatch script.sh \1/e' input.file
You can do this in straight up bash (If you're using that shell; ksh93 and zsh will be very similar) no sed required:
while read -r line; do
id="${line##*/}" # Remove everything up to the last / in the string
id="${id%.GATK.recal.bam}" # Remove the trailing suffix
sbatch script.sh "$id"
done < input.list
At the very least you can use a single sed call per line:
id=$(sed -e 's/\.GATK\.recal\.bam$//' -e 's#.*/##' <<<"$line")
or with plain sh
id=$(printf "%s\n" "$line" | sed -e 's/\.GATK\.recal\.bam$//' -e 's#.*/##')
I need to modify an xml file using Sed to replace the line
url="jdbc:oracle:thin:#//ttpdbscan.axel.net:1521/axel.telco.net"
with
url="jdbc:oracle:thin:#//ttpdbscan.axeltelecom.net:1598/axelPRD.telco.net"
I have stored the lines like this
ACTUAL_DB=$(sed -n 's#^.*url="\(.*\).*"#\1#p' $FILE.xml)
and
NEW_DB="jdbc:oracle:thin:#//ttpdbscan.axeltelecom.net:1598/axelPRD.telco.net"
And the replacing method is this one
sed -i "s#$ACTUAL_DB#$NEW_DB#g" $File.xml
The problem is that when I run the script the file stays the same.
I have echoed the variables and all of them return the correct values.
Assuming the file you have is File.xml (if it is not a variable), you may use
sed -i "s#${ACTUAL_DB}#${NEW_DB}#g" File.xml
Try also with other delimiters:
sed -i "s~${ACTUAL_DB}~${NEW_DB}~g" File.xml
If your sed does not support -i use
sed "s~${ACTUAL_DB}~${NEW_DB}~g" File.xml 1<> File.xml
See sed edit file in place
So I saved the output into another file and found out that the string had an extra space
so it looked like this
ACTUAL_DB= "jdbc:oracle:thin:#//ttpdbscan.axel.net:1521/axel.telco.net "
I removed the extra space with
"$(echo -e "${ACTUAL_DB}" | tr -d '[:space:]')"
And now the sed is working as intended
I want to do the following substitution in a text file:
the original string: "---a---"
after substitution : "---\a---"
and I run the following command:
sed -r -e "s/-(a)-/-\\\1-/g" test.txt
but it doesn't give the right result. What command args should I use?
Remember that backslashes are significant in Bash's double-quoted strings as well as in sed itself. Either use single quotes:
sed -r -e 's/-(a)-/-\\\1-/g' test.txt
Or escape the backslashes again:
sed -r -e "s/-(a)-/-\\\\\\1-/g" test.txt
If you echo the strings, you'll see what's happening:
$ echo "s/-(a)-/-\\\1-/g"
s/-(a)-/-\\1-/g
$ echo 's/-(a)-/-\\\1-/g'
s/-(a)-/-\\\1-/g
$ echo "s/-(a)-/-\\\\\\1-/g"
s/-(a)-/-\\\1-/g
The first one (your original) just looks like a literal backslash followed by a literal 1 to sed.
Try sed -r -e "s/-(a)-/-\\\\\\1-/g" or sed -r -e 's/-(a)-/-\\\1-/g'
The problem is that \ is captured by bash if you use double quotes.
With " you will have to do something like:
[jaypal:~/Temp] echo "---a---" | sed -r "s/-(a)-/-\\\\\1-/g"
---\a---
You have to replace 1 with a
sed -r -e "s/-(a)-/-\\\a-/g" test.txt
so many answers ......
Kaizen ~/so_test
$ echo "---a---" | sed -n 's/a/\\a/p'
---\a---
since you have a text file the following should work :
sed -i 's/a/\\a/g' filename.txt ;
does this help ?
I am having a log file a.log and i need to extract a piece of information from it.
To locate the start and end line numbers of the pattern i am using the following.
start=$(sed -n '/1112/=' file9 | head -1)
end=$(sed -n '/true/=' file9 | head -1)
i need to use the variables (start,end) in the following command:
sed -n '16q;12,15p' orig-data-file > new-file
so that the above command appears something like:
sed -n '($end+1)q;$start,$end'p orig-data-file > new-file
I am unable to replace the line numbers with the variables. Please suggest the correct syntax.
Thanks,
Rosy
When I realized how to do it, I was looking for anyway to get line number into a file containing the requested info, and display the file from that line to EOF.
So, this was my way.
with
PATTERN="pattern"
INPUT_FILE="file1"
OUTPUT_FILE="file2"
line number of first match of $PATTERN into $INPUT_FILE can be retrieved with
LINE=`grep -n ${PATTERN} ${INPUT_FILE} | awk -F':' '{ print $1 }' | head -n 1`
and the outfile will be the text from that $LINE to EOF. This way:
sed -n ${LINE},\$p ${INPUT_FILE} > ${OUTPUT_FILE}
The point here, is the way how can variables be used with command sed -n:
first witout using variables
sed -n 'N,$p' <file name>
using variables
LINE=<N>; sed -n ${LINE},\$p <file name>
Remove the single quotes thus. Single quotes turn off the shell parsing of the string. You need shell parsing to do the variable string replacements.
sed -n '('$end'+1)q;'$start','$end''p orig-data-file > new-file