Julia is ignoring quotes when I use run(), e.g.:
run(`cat file.txt | sed "s/blah/hi/"`)
ignores the quotes, which are required.
\"
does not work...
EDIT: The error is with the pipeline:
cat: |: No such file or directory
cat: sed: No such file or directory
cat: s/blah/hi/: No such file or directory
ERROR: failed process: Process(`cat file.txt | sed s/blah/hi/`, ProcessExited(1)) [1]
in pipeline_error at process.jl:502
in run at ./process.jl:479
The | does not create a pipe inside of Julia backtick syntax. Instead, you are invoking the cat program with four arguments:
file.txt
|
sed
s/blah/hi/
Since these files are unlikely to all exist, cat terminates with an error. Note that sed does not require quotes around the last argument. In fact, if it did get quotes then it wouldn't do what you want at all since the program would be a single string literal. It's the shell that sees the double quotes and passes their contents to sed as a single argument. In this case, since there are no spaces or other characters that are special to most shells between the quotes, it makes no difference. To accomplish what you want you can do this:
run(`cat file.txt` |> `sed "s/blah/hi/"`)
The double quotes are optional as they are in the shell since there are no spaces or other special characters inside of the argument to sed.
Related
Let's say that you would like to remove comments from a datafile using one of two methods:
cat file.dat | sed -e "s/\#.*//"
cat file.dat | grep -v "#"
How do these individual methods work, and what is the difference between them? Would it also be possible for a person to write the clean data to a new file, while avoiding any possible warnings or error messages to end up in that datafile? If so, how would you go about doing this?
How do these individual methods work, and what is the difference
between them?
Yes, they work same though sed and grep are 2 different commands. Your sed command simply substitutes all those lines which having # with NULL. On other hand grep will simply skip or ignore those lines which will skip lines which have # in it.
You could get more information on these by man page as follows:
man grep:
-v, --invert-match
Invert the sense of matching, to select non-matching lines. (-v is specified by POSIX.)
man sed:
s/regexp/replacement/
Attempt to match regexp against the pattern space. If successful, replace that portion matched with replacement. The
replacement may
contain the special character & to refer to that portion of the pattern space which matched, and the special escapes \1
through \9 to
refer to the corresponding matching sub-expressions in the regexp.
Would it also be possible for a person to write the clean data to a
new file, while avoiding any possible warnings or error messages to
end up in that datafile?
yes, we could re-direct the errors by using 2>/dev/null in both the commands.
If so, how would you go about doing this?
You could try like 2>/dev/null 1>output_file
Explanation of sed command: Adding explanation of sed command too now. This is only for understanding purposes and no need to use cat and then use sed you could use sed -e "s/\#.*//" Input_file instead.
sed -e " ##Initiating sed command here with adding the script to the commands to be executed
s/ ##using s for substitution of regexp following it.
\#.* ##telling sed to match a line if it has # till everything here.
//" ##If match found for above regexp then substitute it with NULL.
That grep -v will lose all the lines that have # on them, for example:
$ cat file
first
# second
thi # rd
so
$ grep -v "#" file
first
will drop off all lines with # on it which is not favorable. Rather you should:
$ grep -o "^[^#]*" file
first
thi
like that sed command does but this way you won't get empty lines. man grep:
-o, --only-matching
Print only the matched (non-empty) parts of a matching line,
with each such part on a separate output line.
When I run the command
sed -i.backup "/proxySettings['nginxCacheSize']/s/$/ . " inactive=5h"/"
I get the error
sed: -e expression #1, char 40: unterminated `s' command
Any idea how to fix it?
First, let's leave the -i flag out of this for now, and simplify the command a little:
sed "/'foo']/s/$/ "bar"/"
You have both single quotes and double quotes inside your expression, and sed is getting confused about where the command begins and ends. As written, it appears to be this:
sed "/'foo']/s/$/ "
(with some gibberish after it), which clearly has an unterminated s command. We could use single quotes instead of double quotes to demark the command, but then we'd have the same problem with the single quotes around foof. So we escape those internal double quotes with backslashes:
sed "/'foo']/s/$/ \"bar\"/"
I'm trying to replace a every reference of \' with ' in a file
I've used variations of: sed -e s/\'/"\'"/g file.txt
But they always replace every.single.(single).quote
Any help would be greatly appreciated.
Not sure it's the best solution,I could do it like this:
sed "s/[\]'/\"\'\"/g" file.txt
(putting the backslash character in a character range so it doesn't interfere with the following quote, and protect with double quotes)
Or just extending your syntax, without quotes but using almost the same trick:
sed -e s/[\\]\'/"\'"/g file.txt
An approach trying to conserve as much of the "single-quotedness" of the sed command as possible:
sed 's/\\'"'"'/\'/g'
Just escaping \ with \\ and getting a single quote into the command with '"'"': the first single quote ends the command so far, then we have a double-quoted single quote ("'"), and finally an opening single quote for the rest of the command.
Alternatively, double quoting the whole command and escaping both the backslash and single quote:
sed "s/\\\'/\'/g"
The correct syntax is:
$ echo "foo'bar" | sed 's/'\''/\'/'
foo'bar
Every script (sed, awk, whatever) should always be enclosed in single quotes and you just us other single quotes to stop/restart the script delimiters break out to shell for the minimal portion of the script that's absolutely necessary, in this case long enough to use \'. You need to break out to shell to specify that ' because per shell rules no script enclosed in 's can contain a ', not even if you try to escape it.
echo "foo'bar" | gawk '{gsub(/\47/,"\\'")}1'
foo'bar
The tricky part here is to replace a single quote with ampersand.
First in order to make the single quote manageable use its octal
code here \47 and then escaping ampersand by two back slash. And all of sudden
it becomes feasible :)
I have a text file of million lines to precess on UNIX as below:
"item"
"item"
"item"
"item"
And I use sed -i "s/$/,/g" filename > new_file to add comma at the end of each line.
What I expected is this way:
["item",
"item",
"item",
"item"]
Now, I am just using Vim to edit manually. Is there anyway to add brackets at the beginning and ending automatically with removing the comma at last line? So that, I could write a bash script to process these text files neatly.
Thanks!
sed -e '1s/^/[/' -e 's/$/,/' -e '$s/,$/]/' file_name > new_file
The only funny bit is replacing the comma added to the last line with the close square bracket.
Also note that using -i means there will be no output to standard output. Either use -i or use I/O redirection but not both. (And if you're a portability nut — like me — note that Mac OS X or BSD sed supports -i but requires a suffix for the backup. It will quite happily use -e as the suffix, if there's a -e after the -i, or use the sed script if you don't specify a -e — but then it complains about the file name not being a valid sed script).
I run below sed command
sed -i s/abc=.*$/abc=def ghi/g hpq_sf_attach_wf_param.txt
and it gave me error:
sed: -e expression #1, char 17: unterminated `s' command
I noticed it is due to space in between of def and ghi.
Any idea how to fix it?
You need to use quoting to protect special characters, including spaces, $, and *.
sed -i 's/abc=.*$/abc=def ghi/g' hpq_sf_attach_wf_param.txt
So geekosaur had it right. The the reason you had the problem though is because it needs to be double quotes for the wildcards because with single quotes it takes them as literal characters, not for the meaning you want.
sed -i "s/abc=.*$/abc=def ghi/g" hpq_sf_attach_wf_param.txt
Also if the space between "def" and "ghi" gives you problems, adding a "\" should help making it read it as a literal space.
sed -i "s/abc=.*$/abc=def\ ghi/g" hpq_sf_attach_wf_param.txt