I'm making a Shiny app that constructs a bash script to run on a cluster (basically just a txt file). One of the user inputs is a curl command (provided by the database where the files are stored) that they can copy/paste into a textInput field in the app. When run on the cluster, it will download the file for further processing. However, the curl command they provide contains several single backslashes. Example:
curl --cookie jgi_session=/api/sessions/ec32f2d578304a9e62b4646ae2bec6d4 --output download.20210731.211924.zip -d "{\"ids\":[\"5d94dc9fc0d65a87debccfd3\"]}" -H "Content-Type: application/json" https://files.jgi.doe.gov/filedownload/
It works fine if I paste this directly into a script or if I manually add in double backslashes, but I want to keep this as user friendly as possible. Every other post I've seen about this just says to use double backslashes, but I'd rather do this automatically if at all possible. So any ideas? I'm open to alternate solutions, less work for the user the better.
Your code is picking up the curl line as escaped characters. When you write to file, those escaped characters get converted to their actual character (i.e \" gets converted to literal ".
To avoid, replace special escaped characters by the character sequence that literally created the escape sequence. So to build \" in the final written string, you have to produce \\" as escaped character sequence (which is what the output of a print commmand should show).
Once way to achieve this for this particular character sequence is
escapedString = gsub('\"', '\\"', curlString)
Note that, in terms of string interpretation, \" is a single character (converting to "), while \\" is a sequence of two characters: an escaped \ and a literal ", converting to \" when written, which is the desired output.
Related
We have requirement where i need to replace part of param value in our configuration file.
Example
key1=123-456
I need to replace the value after hyphen with new value.
I got command which is being used in other projects but i am not sure how it works.
Command
[test]$ cat test_sed_key_value.txt
key1=123-456
[test]$ sed -i -e '/key1/ s/-.*$/-789/' test_sed_key_value.txt
[test]$
[test]$ cat test_sed_key_value.txt
key1=123-789
[test]$
It will be helpful if some one can explain how the above command or is there a simpler way to do this using sed.
Here is a list of parts of that commandline, each followed by a short explanation:
sed
which tool to use
-i
flag: apply the effect directly to the processed file (whithout creating a copy of the input file)
-e
expression parameter: the sed code to apply follows
/key1/
"address": only process lines on which this regex applies, i.e. those containing the text "key1"
s/replacethis/withthis/
command: do a search-and-replace, "replacethis" and "withthis" are the next to explanations
-.*$
regex: (what is actually in the commandline instead of "replacethis") a regular expression representing a "minus" followed by anything, in any number, until the end of the line
-789
literal: (what is actually in the commandline instead of "withthis") simply that string "-789"
test_sed_key_value.txt
file parameter: process this file
I cannot think of any way to do this simpler. The shown command already uses some assumptions on the formatting of the input file.
I'd add to Yunnosch's answer that here the "replacethis" is a regexp:
-.*$
See here for an overview of the syntax of sed's regular expressions by Gnu.
Asterisk means a repetition of the previous thing, dot means any character, so .* means a sequence of characters.
$ is the end of the line.
You might want to be a bit more restrictive, since here you'd lose something in a line like this one for instance:
key1=123-456, key2=abc-def
replacing it by:
key1=123-789
removing completely the key2 part (since the .* takes all characters after the first dash until end of line).
So depending on the format of your values, you might prefer something like
-[0-9]*
(without the $), meaning a sequence of numbers after the -
or
-[0-9a-zA-Z_]
meaning a sequence of numbers or letters or underscore after the -
I have the following variables in Unix Korn Shell
host=nyc43ksj
qry_dir='\test\mydoc\mds'
fullpath="\\$host\$qry_dir"
echo "$fullpath"
When I execute the above, I get output such as \nyc43ksj\qrydir.
It looks like the backslashes are used as escape characters.
I tried changing fullpath as follows:
fullpath="\\$host\\$qry_dir"
echo "$fullpath"
This time I get \nyc43ksj\test\mydoc\mds. However, the two backslashes at the beginning are not display as two backslashes. How can get the fullpath as \\nyc43ksj\test\mydoc\mds (two backslashes at the beginning).
In a string, the \ (backslash) character acts as an escape character (as you mention), and the second backslash instructs the shell to put in an actual backslash, as opposed to some special character.
If you want to have two actual backslash characters in the string in sequence, you will need to put \\\\ in the string, so:
fullpath="\\\\$host\\$qry_dir"
I'm trying to run this terminal command from within R using system(mess):
mess <- "sed -i -e '62i\ \\\usepackage[margin=2cm]{geometry}' intro-spatial-rl.tex"
But it keeps failing with the following error:
Error: '\u' used without hex digits in character string starting ""sed -i -e '62i\ \\\u"
I've seen paste used for system commands also, but this fails also.
Could use a different regex program, but thought this may be useful to others and improve my understanding of how R deals with characters. Thank you!
Your problem is the unequal number of \ in your escape sequence.
R sees two escape sequences here: \\ and \u. The second one is invalid and gives an error. You probably want to escape the second backslash as well, yielding \\\\. Likewise, you probably meant to escape the previous \ in \ as well, leaving you with \\ .
All that being said, would replace the sed invocation completely by R code in this instance. The way I understand it you just want to insert a line of text. That’s easy in R (although it’s not clear what your input and output here is).
I have a plain text file where newline character in not "\n" but a special character.
Now I want to sort this file.
Is there a direct way to specify custom new-line character while using unix sort command?
I don't want to use a script for this as far as possible?
Please note the data in text file have \n, \r\n, and \t characters(the reason for such data is application specific so please don't comment on that).
The sample data is as below:
1111\n1111<Ctrl+A>
2222\t2222<Ctrl+A>
3333333<Ctrl+A>
Here Ctrl+A is the newline character.
Use perl -001e 'print sort <>' to do this:
prompt$ cat -tv /tmp/a
2222^I2222^A3333333^A1111
1111^A
prompt$ perl -001e 'print sort <>' /tmp/a | cat -tv
1111
1111^A2222^I2222^A3333333^Aprompt$
That works because character 001 (octal 1) is control-A ("\cA"), which is your record terminator in this dataset.
You can also use the code point in hex using -0xHHHHH. Note that it must be a single code point, not a string, using this shortcut. There are ways of doing it for strings and even regexes that involve infinitessimally more code.
I'm working on a project where we are dealing with importing/exporting data from database tables using ksh scripts and Perl scripts. We have an existing process to export data from a table to a file and it is then imported into another system.
Here's the catch - the export process dumps out pipe delimited files while the system that is doing the import expects files delimited by the ASCII group separator character which is decimal 29, hexidecimal 1d, or octal 35. It shows up in vi as ^] Right now, I'm converting the delimiter via a Perl script. What I'd like to do is tell our export process to just use the delimiter we are expecting. Something like:
export_table.ksh -d '\035'
The problem is I can't figure out how to pass this character to the export script.
I've tried all kinds of combinations of single quotes, double quotes, backslashes, and the octal and hex version of this character.
I'm on Solaris 10 using ksh and/or Perl.
have you tried:
$'\x29'
actually try this for ]:
echo $'\x5d'
and for ^
echo $'\x5e'
so you just need to do:
export_table.ksh -d $'\x5e\x5d'
In bash(1), one can prefix a character with ^v to enter that character verbatim. Perhaps ksh(1) does the same?