I have tried following commands, but they don't work. sed isn't installed and hence doesn't work. Same goes for dos2unix.
awk 'sub(/^M/,"")' finename
cat finename | sed 's/^M//’ > finename
awk '{sub(/^M/,"")}1' finename > finename
tr -d $'\r' < finename
tr -d '\015' < finename > finename
awk 'sub(/^M/,"");1' finename
This command worked :
tr -d '\r' < filename > new_file
The easiest way is the dos2unix way. In case it's not installed, you might try this:
sudo apt install dos2unix
In case the installation of dos2unix is not permitted, you might try the following command:
sed 's/\r//' input > output
If sed is not working too, you might go for the following awk solution:
awk '{sub(/^M/,"")}1' input > output
(Afterwards you just rename output back to input)
if the file hasn't been pre-mangled by cat into caret notation of "^M" for \r, one could try
{m,n,g}awk 3 ORS= RS='\r'
if it has already been mangled,
gawk -Pe/-ce NF=NF OFS= FS='[\\^]M' # these 2 gawk modes act up;
# switching to FS instead of RS
gawk/nawk 6 ORS= RS='\\^M'
mawk 9 ORS= RS='\^M'
Related
I have the below output from unix:
$ diff -y --suppress-common-lines backup.txt newfile.txt
> `jjj' int,
i need only jjj : int as output.
tried the below didnt work as expected:
$ diff -y --suppress-common-lines backup.txt newfile.txt | grep -i '>' |tr -d '[>]' |sed 's/,//g'
suggesting to try gawk script:
diff -y --suppress-common-lines backup.txt newfile.txt | gawk '{print $1 ":" $2}' FPAT="[[:alnum:]]+"
The most common reasons for this not working are:
Your file is encoded as a non ASCII file, most commonly in UTF-8.
(Save the text files as ASCII.)
You are running this in a command shell with colors.
(Colors are actually ANSI characters and messes up sed.)
You have encoded your file with a different EOL than used in your *nix OS (\n), such as \r\n (Windows) or \r (MacOS).
There are hidden TAB (\t) characters in the file.
After you have fixed the above, try this:
diff -Ewy -r --suppress-common-lines -aB -W 512 file.txt file2.txt | tr -d '[>]'
So I am new to SED and Unix and I would like to replace the following file:
1500:../someFile.C:111 error
1869:../anotherFile.C:222 error
1869:../anotherFile2.Cxx:333 error
//thousands of more lines with same structure
With the followig file
someFile.c
anotherFile.c
anotherFile2.Cxx
Basically, I just want to extract the filename from every line.
So far, I have read the documentation on sed and the second answer here. My best attempt was to use a regex as follows:
sed "s/.\*\/.:.*//g" myFile.txt
Lots of ways to do this.
Sure, you could use sed:
sed 's/^[^:]*://;s/:.*//;s#\.\./##' input.txt
sed 's%.*:\.\./\([^:]*\):.*%\1%' input.txt
Or you could use a series of grep -o instances in a pipe:
grep -o ':[^:]*:' input.txt | grep -o '[^:]\{1,\}' | grep -o '/.*' | grep -o '[^/]\{1,\}'
You could even use awk:
awk -F: '{sub(/\.\.\//,"",$2); print $2}' input.txt
But the simplest way would probably be to use cut:
cut -d: -f2 input.txt | cut -d/ -f2
You can capture the substring between last / and following : and replace the whole string with the captured string(\1).
sed 's#.*/\([^:]\+\).*#\1#g' myFile.txt
someFile.C
anotherFile.C
anotherFile2.Cxx
OR , with little less escaping, sed with -r flag.
sed -r 's#.*/([^:]+).*#\1#g' myFile.txt
Or if you want to use grep,this will only work if your grep supports -P flag which will enable PCRE:
grep -oP '.*/\K[^:]+' myFile.txt
someFile.C
anotherFile.C
anotherFile2.Cxx
I have a file which has ^# in it and I am unable to remove it using sed or replace command in python. I can see ^# only when I open the file in vi editor. Please suggest. Below is what i tried using sed.
sed 's/^#/?/g' filename
Tested on Linux, not sure if syntax varies elsewhere, try
$ printf 'abc\0baz\n' | cat -v
abc^#baz
$ printf 'abc\0baz\n' | tr -d '\0' | cat -v
abcbaz
tr will delete all ASCII NUL characters from input.. cat -v is used here to highlight non-printing characters
for file input, use tr -d '\0' <filename
GNU sed (and possibly few other implementations) allow to use hex value to represent a character
$ printf 'abc\0baz\n' | sed 's/\x00//g' | cat -v
abcbaz
so, for in-place editing, use sed -i 's/\x00//g' filename (See also: sed in-place flag that works both on Mac (BSD) and Linux )
I'd like to join lines that end with a | with the next line.
Example:
abcd|
ef
123456|
78|
90
Desired output:
abcdef
1234567890
This might work for you (GNU sed):
sed ':a;N;s/|\s*\n//;ta;P;D' file
With GNU sed:
$ sed ':a;/| *$/{N;ba};s/| *\n//g' infile
abcdef
1234567890
This does the following:
:a # Label to branch to
/| *$/{ # If the line ends with a pipe and optional spaces
N # Append next line to pattern space
ba # Branch to label
}
s/| *\n//g # Remove all pipes followed by optional spaces and a newline
To get this working under BSD sed, the command has to be split up after labels and branching commands:
sed -e ':a' -e '/| *$/{N;ba' -e '};s/| *\n//g' infile
With GNU awk for multi-char RS:
$ awk -v RS='[|]\\s*\\n' -v ORS= '1' file
abcdef
1234567890
With other awks one way would be:
$ awk 'sub(/\|[[:blank:]]*$/,"") {s = s $0; next} {print s $0; s=""}' file
abcdef
1234567890
If your last input line can end with | you'd need to tell us how to handle that and add sample input/output including that case to your question.
You can try this sed,
sed ':loop; /| *$/{N;s/\n//g}; t loop' yourfile | sed 's/ *| *//g'
awk to the rescue! The following script removes pipe characters and newlines when they occur.
$ awk '{c=sub(/\|/,"")?"":"\n"; $1=$1; printf "%s%s",$0,c}' file
abcdef
1234567890
I started with this
awk -F '|' 'NF>1 {printf "%s", $1} NF==1' file
But then I thought that made too many assumptions about how many fields you might have. So I ended up with
awk -F '|' '{if (NF>1 && $NF ~ /^[[:blank:]]*$/) {NF--; printf "%s", $0} else print}' file
which is much sturdier but less terse.
Considering the input text is stored in a file name infile
You can use this GNU sed command:
sed ':begin;$!N;s/\n//;s/\s//g;s/|//g' infile
awk '{sub(/\|/,"")}{printf "%s",$1}/f/{print "\r"}END {print ""}' file
abcdef
1234567890
I want to do the following substitution in a text file:
the original string: "---a---"
after substitution : "---\a---"
and I run the following command:
sed -r -e "s/-(a)-/-\\\1-/g" test.txt
but it doesn't give the right result. What command args should I use?
Remember that backslashes are significant in Bash's double-quoted strings as well as in sed itself. Either use single quotes:
sed -r -e 's/-(a)-/-\\\1-/g' test.txt
Or escape the backslashes again:
sed -r -e "s/-(a)-/-\\\\\\1-/g" test.txt
If you echo the strings, you'll see what's happening:
$ echo "s/-(a)-/-\\\1-/g"
s/-(a)-/-\\1-/g
$ echo 's/-(a)-/-\\\1-/g'
s/-(a)-/-\\\1-/g
$ echo "s/-(a)-/-\\\\\\1-/g"
s/-(a)-/-\\\1-/g
The first one (your original) just looks like a literal backslash followed by a literal 1 to sed.
Try sed -r -e "s/-(a)-/-\\\\\\1-/g" or sed -r -e 's/-(a)-/-\\\1-/g'
The problem is that \ is captured by bash if you use double quotes.
With " you will have to do something like:
[jaypal:~/Temp] echo "---a---" | sed -r "s/-(a)-/-\\\\\1-/g"
---\a---
You have to replace 1 with a
sed -r -e "s/-(a)-/-\\\a-/g" test.txt
so many answers ......
Kaizen ~/so_test
$ echo "---a---" | sed -n 's/a/\\a/p'
---\a---
since you have a text file the following should work :
sed -i 's/a/\\a/g' filename.txt ;
does this help ?