Can anyone help me to understand the below sed command?
These are the values I am using:
InsertPoint - 2
TOT - 15
Count- it is the csv file, input to this command.
sed -e ''"${InsertPoint}"'s/^[^,]*,//' -e ''"${InsertPoint}"'s/$/, '"${TOT}"'/' ${Count}
I need to know, what they are replacing with what?
There are two substitution commands here, applied on line 2 of your file. The first one removes first field, the second one adds a field at the end of the line with value 15.
Basic substitution command with sed have the syntax s/old text/replacement text/ where s can be preceded with a line number to apply the command to, so:
'"${InsertPoint}"': at line 2(value of ${InsertPoint})
s/^[^,]*,// removes first field by replacing from the start of line(^) any number of non comma characters([^,]*) followed by comma(,) with nothing(//)
'"${InsertPoint}"': at line 2
s/$/, '"${TOT}"'/: adds a new field by replacing end of line $ with 15(value of , ${TOT})
sed command is applied on ${Count} value file
Related
This post could count as duplicate , but i have not found any relevant answer in previous threads. I have a large (6 GB) text file and i wish to remove every 3rd and 4th line in a set of 4 lines . For example , the following
line1
line2
line3
line4
line5
line6
line7
line8
needs to be converted to this
line1
line2
line5
line6
Is there any vim script / command to remove those lines ? It could be also in multiple passes . 1 pass to delete the 3rd lines (in a set of 4 (line1,line2,line3,line4)) and another pass to delete again the 3rd lines (previously 4th ones , in a set of 3 (line1,line2,line3)) .
The commands :g/^/+1 d3 is close to what i want but it also removes the second lines .
If you have GNU sed, you can filter the buffer through this pipeline:
sed -e '0~4d' | sed '0~3d'
The first sed deletes every 4th line, the second deletes every 3rd line.
This has the desired effect.
To pipe the current buffer through this command, enter this in command mode:
%!sed -e '0~4d' | sed '0~3d'
The % selects the range of lines to pass to a command (% means all lines, the entire buffer), and !cmd is the command to pipe through.
To perform this outside of vim, run these two steps:
sed -ie '0~4d' file
sed -ie '0~3d' file
This will modify the file, in two steps.
Alternatively you can also use Awk.
awk 'NR%4==3||NR%4==0{next;}1' file.txt > output.txt
To do this via Vim:
%!awk 'NR\%4==3||NR\%4==0{next;}1'
UPDATE: It is a bad approach for large files, it needs ~3 sec for a 6MB file to perform a substitution.
This approach works in vim. Using regular expression, you find 4 lines and substitute them with first two lines of these 4. Works for a long file as well. Doesn't work for last 1–3 lines if there is a remainder of division of total lines number by 4.
:%s#\(^.*\n^.*\)\n^.*\n^.*\n#\1\r#g
Explanation:
:%s — substitute in the whole file, # used as a delimiter
\(^.*\n^.*\) — \(\) select two lines that will be used later as \1; \n stands for linebreak; ^ for the beginning of the line; .* for any symbol repeated as much times as possible before the linebreak
\n — linebreak after the second line
^.*\n^.*\n — next two lines to be deleted
\1\r — substitute for lines with first two lines and add a linebreak \r
g — apply to the whole file
I want to get specific lines using sed, where the first line and the last line to get are stored in variables.
Here is an example, I want to get all the lines between the first line (here line number 5) and the last line (here number 8), and then I use grep to search a specific word.
firstLine=5
lastLine=8
sedResult="$(sed -n "$firstLine,$lastLine p" text.txt | grep word -aIi)"
But I am having errors. The errors look like this :
sed: -e expression #1, char 5: unexpected `,'
what is the proper way to use variables as line numbers ?
Your initial expression has broken double quotes usage. It should be:
firstLine=5
lastLine=8
sedResult=$(sed -n "$firstLine,$lastLine p" text.txt)
I can find my lines with this pattern, but in some case the info is on the line after the match. How can I also get the line following my match line?
sed -n '/SQL3227W Record token/p' /log/PLAN_2015-08-16*.MSG >ERRORS.txt
Firstly, this looks like a job for grep:
grep -A 1 'SQL3227W Record token' /log/PLAN_2015-08-16*.MSG >ERRORS.txt
(-A 1 means to print an additional 1 line After the match).
Secondly, if you're using GNU sed, you can use a second address of +1 thus:
sed -n '/SQL3227W Record token/,+1p' /log/PLAN_2015-08-16*.MSG >ERRORS.txt
Otherwise, (if you really must use non-Gnu sed), then each time you match, append the following line to your pattern space. Delete the first line, before continuing loop (in case the second line is also a match).
Untested code:
#!/bin/sed -nf
/SQL3227W Record token/{
N
P
D
}
sed is for simple substitutions on individual lines, that is all. For anything even slightly more interesting just use awk:
awk '/SQL3227W Record token/{c=2} c&&c--' file
See Printing with sed or awk a line following a matching pattern for other related idioms.
I have a file where a few lines end with tux. How do I add " to the end of any line that ends in words like this or This?
You could visit this site for more examples and help about using sed in overall. Also check it's "Regular expressions" tab or search the web for something like "unix anchor characters".
For this actual problem, these are the relevant parts of the site:
Sed has the ability to specify which lines are to be examined and/or modified, by specifying addresses before the command. I will just describe the simplest version for now - the /PATTERN/ address. When used, only lines that match the pattern are given the command after the address. Briefly, when used with the /p flag, matching lines are printed twice:
sed '/PATTERN/p' file
And of course PATTERN is any regular expression.
According to these, you could use a sed command like this to get the lines ending with "this" or "This" in your file, or "tux" if you meant that:
$ sed '/[tT]his$/p' yourfile
or
$ sed '/tux$/p' yourfile
For putting the double quotes at the end of these lines, you also need to understand:
$ has a special meaning (end of the input line) as an anchor character in regular expressions
... and the character "$" is the end anchor. The expression "A$" will match all lines that end with the capital A. If the anchor characters are not used at the proper end of the pattern, then they no longer act as anchors. The "$" is only an anchor if it is the last character.
how to use sed for substitution of characters (see the linked page)
Sed has several commands, but most people only learn the substitute command: s. The substitute command changes all occurrences of the regular expression into a new value. A simple example is changing "day" in the "old" file to "night" in the "new" file:
$ sed 's/day/night/' newfile
Or another way (for UNIX beginners),
$ sed 's/day/night/' old >new
and for those who want to test this:
$ echo day | sed 's/day/night/'
This will output "night".
After these you can construct your own sed command, knowing that you can use this two parts together in one command like this:
$ sed '/[pP]atternAtTheEndOfLine$/s/$/patternToAddToEndOfTheLine/' yourfile
I am new to Unix.
I am using a sed command to remove junk from a .txt file in Unix.
This is the command that i used--
sed -e 's/[^ -~]//g' final.txt > file1_now
but here i am facing a problem the junks are getting removed, but in case my data contains a '-' that is also removed. I dont want that.
Appreciate your help.
Thanks,
Binayak
Try doing this :
sed -e 's/[^ ~-]//g' final.txt > file1_now
The - character must be the latest (or the first) in your character class, because the meaning is different in other cases : it means a range like in [a-z]
The - character is treated as a literal character if it is the last or the first (after the ^) character within the brackets: [abc-], [-abc].
http://en.wikipedia.org/wiki/Regular_expression