I have nw executable file...
sed -i 's/udev\.so\.0/udev.so.1/g' nw
the Node-webkit application binary.
I am running ubuntu 14.04Lts i tried to open nw with ./nw. i doesn't open.
After that i typed in the above command. It started working.
I am too curious to know what that command did to my executable.
I know sed is for matching file patters with regular expression. how can it operate on .exe (application/x-executable) file.
Please anyone explain.
The sed command that you've used searches the nw executable file (like an ordinary text file imagine) for any occurrence of the string udev.so.0 and substitutes it with udev.so.1. The backslashes \. before the dots are just for escaping the . character, which is a special character for sed (it means any character - a wildcard like * is on windows systems). The g character at the end of your command denotes also to continue searching a specific line, even if a first occurrence has been already found. This way the whole file - nw here - is being searched and replaced.
Related
I am having trouble trying to add environment variables to a REnviron file that have special characters. This is on a Debian machine with the file located at /usr/lib/R/etc/Renviron. If my value has a &, I get a weird error when installing packages (although the package installs fine):
REnviron file: TEST_KEY=HEY&X&THERE
Command: install.packages(futures)
Error:
/usr/lib/R/bin/Rcmd: 468: /usr/lib/R/etc/Renviron: THERE: not found
/usr/lib/R/bin/Rcmd: 468: /usr/lib/R/etc/Renviron: X: not found
Which seems like it's because & is a special character. I can fix this by putting quotes around the value like this: TEST_KEY="HEY&X&THERE". However at that point I can't figure out how to handle when a value itself has a " in it. For example if I wanted the value to be HEY&"&THERE I am not sure how to format that (a backlash in front of the quote didn't work). I tried "HEY&\"&THERE", but that left the \ in the string once loaded into R. Which leads me to my broader question:
How can I ensure that anything that satisfies linux environment variable styling rules works in an REnviron file?
Update: this seems to be a Debian specific issue. You can recreate it using the debian:bullseye-slim docker image, installing R, then editing the Renviron to have a & in it.
Okay I spent an hour looking into this and I think there is the answer.
In both Ubuntu and Debian (and maybe other systems too), the Renviron file gets executed within bash. So what you're typing in the file is exactly bash commands. You can see in lines 39-40 of RCmd the commands:
. "${R_HOME}/etc${R_ARCH}/Renviron"
export `sed 's/^ *#.*//; s/^\([^=]*\)=.*/\1/' "${R_HOME}/etc${R_ARCH}/Renviron"`
The first line runs the Renviron file in the shell, the second then exports the variable names based on lines that have a = in them.
So in our case the way to handle this is to put double quotations around all the values, and any double-quote within the string should get a \ before it. The reason why I didn't realize the solution before I posted the question is that I didn't use cat() when printing my text in R, which removes the leading \. So: "HEY&\"&THERE" would be the right way to do it.
To recap:
The Renviron file is executed on the shell
To handle special characters in strings you use the same logic you would in the OS (so double quotes with \ to escape actual double quotes).
We have requirement where i need to replace part of param value in our configuration file.
Example
key1=123-456
I need to replace the value after hyphen with new value.
I got command which is being used in other projects but i am not sure how it works.
Command
[test]$ cat test_sed_key_value.txt
key1=123-456
[test]$ sed -i -e '/key1/ s/-.*$/-789/' test_sed_key_value.txt
[test]$
[test]$ cat test_sed_key_value.txt
key1=123-789
[test]$
It will be helpful if some one can explain how the above command or is there a simpler way to do this using sed.
Here is a list of parts of that commandline, each followed by a short explanation:
sed
which tool to use
-i
flag: apply the effect directly to the processed file (whithout creating a copy of the input file)
-e
expression parameter: the sed code to apply follows
/key1/
"address": only process lines on which this regex applies, i.e. those containing the text "key1"
s/replacethis/withthis/
command: do a search-and-replace, "replacethis" and "withthis" are the next to explanations
-.*$
regex: (what is actually in the commandline instead of "replacethis") a regular expression representing a "minus" followed by anything, in any number, until the end of the line
-789
literal: (what is actually in the commandline instead of "withthis") simply that string "-789"
test_sed_key_value.txt
file parameter: process this file
I cannot think of any way to do this simpler. The shown command already uses some assumptions on the formatting of the input file.
I'd add to Yunnosch's answer that here the "replacethis" is a regexp:
-.*$
See here for an overview of the syntax of sed's regular expressions by Gnu.
Asterisk means a repetition of the previous thing, dot means any character, so .* means a sequence of characters.
$ is the end of the line.
You might want to be a bit more restrictive, since here you'd lose something in a line like this one for instance:
key1=123-456, key2=abc-def
replacing it by:
key1=123-789
removing completely the key2 part (since the .* takes all characters after the first dash until end of line).
So depending on the format of your values, you might prefer something like
-[0-9]*
(without the $), meaning a sequence of numbers after the -
or
-[0-9a-zA-Z_]
meaning a sequence of numbers or letters or underscore after the -
I have come across unix sed command usage and not able to understand what it does. Could you please help me to understand the usage ? If possible please share some reference to understand such usages of sed command.
sed -i '/^export JAVA_HOME/ s:.*:export JAVA_HOME=/usr/java/default\nexport HADOOP_PREFIX=/usr/local/hadoop\nexport HADOOP_HOME=/usr/local/hadoop\n:' $HADOOP_PREFIX/etc/hadoop/hadoop-env.sh
The command is simple, though it assumes GNU sed because of the way it uses the -i option; for macOS Sierra and related systems, you'd need to use -i '' in place of just -i.
Overall, it corresponds to:
sed -i '/Pattern/ s:.*:Replacement:' file
where:
-i means overwrite each input file with its edited output without creating a backup copy.
/Pattern/ is ^export JAVA_HOME; a line starting with the word export and then JAVA_HOME separated by a single space.
s:.*:Replacement: is a substitute command, using : instead of the more conventional / (often s/.*/Replacement/) as the pattern delimiter. This is done because the replacement text contains slashes. The .* matches the whole line. The rest of the material is written in place of the original export JAVA_HOME line. The \n sequence expands to a newline, so it actually produces a number of lines in the output.
file is $HADOOP_PREFIX/etc/hadoop/hadoop-env.sh
As others have pointed out, this is a sed command invocation. The command is short for "Stream EDitor" and is quite useful for modifying files programaticallly. Your best bet is to read the man pages (man sed, but I've broken down your particular command here for instructive purposes:
sed # The command
-i # Edit file in place (no backup)
'/^export JAVA_HOME/ # For every line that begins with 'export JAVA_HOME'...
s: # substitue...
.*: # the entire line with...
export JAVA_HOME=/usr/java/default
export HADOOP_PREFIX=/usr/local/hadoop
export HADOOP_HOME=/usr/local/hadoop
:' # End of command
$HADOOP_PREFIX/etc/hadoop/hadoop-env.sh # Run on the following file
Points of interest:
Commands can be limited to a particular address range or scope. Here, the scope was a search.
The substitue command can be delimited by almost any character (usually it is /, but in this case, : was chosen to prevent escaping of the / in the filepaths
The sed expression was enclosed in ' to prevent shell expansion of variables. Although no expansions would have taken place in this scenario, it is fairly common to see the expression wrapped in ' to eliminate the possibility.
i am facing hard time in removing the special characters from the csv file .
My process like this in my output table i have some data like this
Col1
BC,BS/APP
Like this i have another 10 columns where there is a chance of getting the special characters when i tried with patindex i'm able to remove only first special character and for removing the other characters i need to use while loop which is taking hard time to do that .
So i tried to remove the special characters after bcping the data to the csv file below is the bcp command i am using
bcp_with_error_check tempdb..STT_IM166_WEB_MWE out temp.dat -SSVR -UUSR -PPWD -c -b1000 -t'","'
sed -e 's/,"0/,="0/g;s/,"1/,="1/g;s/,"2/,="2/g;s/,"3/,="3/g;s/,"4/,="4/g;s/,"5/,="5/g;s/,"6/,="6/g;s/,"7/,="7/g;s/,"8/,="8/g;s/,"9/,="9/g'temp.dat > temp1.dat
sed -e 's/$/"/g' temp1.dat > temp2.dat
sed -e 's/^/="/g' temp3.dat >>Filename.csv
My problem is since it is CSV file if i remove comma (,) considering as special character it is disturbing the file layout .
i can replace comma alone in data base but i am not getting the command to exclude the comma alone and remove other charachters . Please help me out i am in very need of this command
I'm not clear what you're really after, but at the very least you can shrink your first sed command by a factor of 10:
sed -e 's/,"\([0-9]\)/,="\1/g' temp.dat > temp1.dat
The pattern looks for comma, double quote and a digit (and remembers what the digit is); it is replaced by comma, equals, double quote and the remembered digit.
Unless you have a reason for the different temporary files, you can collapse the three sed commands into one with:
sed -e 's/,"\([0-9]\)/,="\1/g' -e 's/$/"/g' -e 's/^/="/g' temp.dat >>Filename.csv
And if bcp_with_error_check will write to standard output if you omit the out temp.dat arguments, then you don't need any temporary files (which is generally a good idea). Note that if two people innocently ran this command at the same time in the same directory, they'd be trampling over each other's temporary files (or running into problems because they couldn't). With no temporary files, you've only got the final file name, Filename.csv to worry about.
However, that does not address your main question — it just improves your scripting.
I am new to Unix.
I am using a sed command to remove junk from a .txt file in Unix.
This is the command that i used--
sed -e 's/[^ -~]//g' final.txt > file1_now
but here i am facing a problem the junks are getting removed, but in case my data contains a '-' that is also removed. I dont want that.
Appreciate your help.
Thanks,
Binayak
Try doing this :
sed -e 's/[^ ~-]//g' final.txt > file1_now
The - character must be the latest (or the first) in your character class, because the meaning is different in other cases : it means a range like in [a-z]
The - character is treated as a literal character if it is the last or the first (after the ^) character within the brackets: [abc-], [-abc].
http://en.wikipedia.org/wiki/Regular_expression