I am trying to execute a CURL call by passing parameters.
However, it does not return the intended output.
Here is the code:
curl -X GET 'http://admin-app.prod.order-services.cp.glb.prod.walmart.com/order-services-admin/plutus-admin/published-transactions/count?fromDate=$from_date&toDate=$to_date'
Please let me know how I can pass $from_date and $to_date in the above URL.
I think you want to substitute the values of variables $from_date and $to_date in the link. Variables are not substituted in single quoted strings. Try replacing single quotes with double quotes:
curl -X GET "http://admin-app.prod.order-services.cp.glb.prod.walmart.com/order-services-admin/plutus-admin/published-transactions/count?fromDate=$from_date&toDate=$to_date"
Related
If I use the zsh shell and execute the following command I get
$zsh
$echo '$_GET["test"]'
preexec: bad math expression: operand expected at `"test"'
$echo '$_GET[]'
preexec: invalid subscript
In bash I get what I expect:
$bash
$echo '$_GET["test"]'
$_GET["test"]
I assume that zsh is trying to expand the $_GET variable. How can I avoid this? I always expected this to only happen within double quotes anyhow.
[update]
I found the following three lines in the .zshrc:
# Display last command interminal
echo -en "\e]2;Parrot Terminal\a"
preexec () { print -Pn "\e]0;$1 - Parrot Terminal\a" }
After commenting them out everything seems to work as expected.
What I understand is that preexec is executed after a command in the terminal has been submitted but before it is executed. The $1 is the command that one submitted.
I still do not understand the purpose of the two lines but is it because of the double quotes in the preexec print statement that the variables are expanded?
The combination of print -P together with the expansion of $1 is killing you. With this, you first get a "normal" expansion of $1, yielding something like "\e]0;echo '$_GET["test"]'...". Now -P causes print to do a prompt expansion on this string, which means that it has to expand $_GET["test"] as well. This causes the error.
I suggest to remove the -P, in particular since you don't have any characters in your string which would benefit from prompt expansion.
I'm making a Shiny app that constructs a bash script to run on a cluster (basically just a txt file). One of the user inputs is a curl command (provided by the database where the files are stored) that they can copy/paste into a textInput field in the app. When run on the cluster, it will download the file for further processing. However, the curl command they provide contains several single backslashes. Example:
curl --cookie jgi_session=/api/sessions/ec32f2d578304a9e62b4646ae2bec6d4 --output download.20210731.211924.zip -d "{\"ids\":[\"5d94dc9fc0d65a87debccfd3\"]}" -H "Content-Type: application/json" https://files.jgi.doe.gov/filedownload/
It works fine if I paste this directly into a script or if I manually add in double backslashes, but I want to keep this as user friendly as possible. Every other post I've seen about this just says to use double backslashes, but I'd rather do this automatically if at all possible. So any ideas? I'm open to alternate solutions, less work for the user the better.
Your code is picking up the curl line as escaped characters. When you write to file, those escaped characters get converted to their actual character (i.e \" gets converted to literal ".
To avoid, replace special escaped characters by the character sequence that literally created the escape sequence. So to build \" in the final written string, you have to produce \\" as escaped character sequence (which is what the output of a print commmand should show).
Once way to achieve this for this particular character sequence is
escapedString = gsub('\"', '\\"', curlString)
Note that, in terms of string interpretation, \" is a single character (converting to "), while \\" is a sequence of two characters: an escaped \ and a literal ", converting to \" when written, which is the desired output.
I have a json output, representing a linux command in one of it's values:
... ,"proc.cmdline":"sh -c pgrep -fl \"unicorn.* worker\[.*?\]\"", ...
In some cases, the command contains a backslash, so the outputing json will contain a backslash too.
I need to parse the output with jq, but it fails with an error:
parse error: Invalid escape at line 1, column 373
It refers to this: \[
However, this is a part of the command, so it is expected to be there.
If a manually edit the line, converting \[ to \\[, then it passes. However the resulting output contains both backslashes:
...
"proc.cmdline": "sh -c pgrep -fl \"unicorn.* worker\\[.*?\\]\"",
...
Now, I can't be there to manually edit every time. This output is produced automatically by another software, and I need to parse it with jq every time it comes in.
Also, even if I was able to edit every \[ to \\[, (like by using something like sed) the output becomes a lie, the second \ is fake.
Any ideas on how to work around this?
EDIT: here is the full json for reference (received raw by the output of the program I'm using (falco)):
{"priority":"Debug","rule":"Run shell untrusted","time":"2019-05-15T07:32:36.597411997Z", "output_fields": {"evt.time":1557905556597411997,"proc.aname[2]":"gitlab-mon","proc.aname[3]":"runsv","proc.aname[4]":"runsvdir","proc.aname[5]":"wrapper","proc.aname[6]":"docker-containe","proc.aname[7]":"docker-containe","proc.cmdline":"sh -c pgrep -fl \"unicorn.* worker\[.*?\]\"","proc.name":"sh","proc.pcmdline":"reactor.rb:249 ","proc.pname":"reactor.rb:249","user.name":null}}
JSON standard is quite explicit about which characters have to be escaped, and [ is not one of them (though reverse solidus - \ is). So it's your script / software generating JSON violates the JSON standard - you can validate it on any of well-known online JSON validators, e.g., like this one: https://jsoncompare.com/#!/simple/ - it will produce the error too.
If you cannot enhance/fix your script generating that JSON, then you'd need to ensure you double quote those non-compliant quotations before passing to JSON processor: e.g.
... | sed -E 's/\\([][])/\\\\\1/g' | ...
You'll need to fix whatever is generating that "json" string. Use something that produces compliant json.
If that's not an option for you, then you will have to modify it so that it is valid json. Fortunately jq can handle that. Read it in raw, fix the string then parse it.
Assuming we just need to fix the \[ and \] sequence:
$ ... | jq -R 'gsub("\\\\(?<c>[[\\]])"; "\\\\\(.c)") | fromjson | "your filter"'
Remember, "sh -c pgrep -fl \"unicorn.* worker\\[.*?\\]\"" is a string with escapes... it represents the value:
sh -c pgrep -fl "unicorn.* worker\[.*?\]"
So it's absolutely correct to have "both backslashes."
I am using the Twitter REST API to retrieve data in JSON format. Twitter's developer page makes it easy by providing a command that can be pasted directly into the terminal and executed. The following command works in the terminal.
curl --get 'https://api.twitter.com/1.1/statuses/user_timeline.json' --data 'count=3200&screen_name=BernieSanders' --header 'Authorization: OAuth oauth_consumer_key="####", oauth_nonce="####", oauth_signature="####", oauth_signature_method="HMAC-SHA1", oauth_timestamp="1467719924", oauth_token="####", oauth_version="1.0"' --verbose
I am trying to get the JSON data into R, and would like to execute this same command in the R console. I have tried curlconverter using the exact same code, get an error that there is an unexpected symbol. However, the code is exactly the same. Is there a more suitable package for executing this code?
curlExample <- "curl --get 'https://api.twitter.com/1.1/statuses/user_timeline.json' --data 'count=3200&screen_name=BernieSanders' --header 'Authorization: OAuth oauth_consumer_key="####", oauth_nonce="####", oauth_signature="####", oauth_signature_method="HMAC-SHA1", oauth_timestamp="1467719924", oauth_token="####", oauth_version="1.0"' --verbose"
I think you are going to need to escape those quotation marks. Try this instead:
curlExample <- "curl --get 'https://api.twitter.com/1.1/statuses/user_timeline.json' --data 'count=3200&screen_name=BernieSanders' --header 'Authorization: OAuth oauth_consumer_key=\"####\", oauth_nonce=\"####\", oauth_signature=\"####\", oauth_signature_method=\"HMAC-SHA1\", oauth_timestamp=\"1467719924\", oauth_token=\"####\", oauth_version=\"1.0\"' --verbose"
That the quotes were not being escaped properly was actually evident in your question, because the Stack Overflow markup rendered the curlExample string in multiple colors.
I believe https://cran.r-project.org/web/packages/twitteR/twitteR.pdf is the way to do this in R.
But specifically in your case, you have unescaped " in your string.
I tried this but it does not seem to work.
Please help thanks
TEST_STRING= test
echo Starting awk command
awk -v testString=$TEST_STRING'
BEGIN {
}
{
print testString
}
END {}
' file2
There are two problems here: You aren't actually assigning to TEST_STRING, and you're passing the program code in the same argument as the variable value. Both of these are caused by whitespace and quoting being in the wrong places.
TEST_STRING= test
...does not assign a value to TEST_STRING. Instead, it runs the command test, with an environment variable named TEST_STRING set to an empty value.
Perhaps instead you want
TEST_STRING=test
or
TEST_STRING=' test'
...if the whitespace is intentional.
Second, passing a variable to awk with -v, the right-hand side should be double-quoted, and there must be unquoted whitespace between that value and the program to be passed to awk (or other values). That is to say:
awk -v testString=$TEST_STRING' BEGIN
...will, if TEST_STRING contains no whitespace, pass the BEGIN as part of the value of testString, not as a separate argument!
awk -v testString="$TEST_STRING" 'BEGIN
...on, the other hand, ensures that the value of TEST_STRING is passed as part of the same argument as testString=, even if it contains whitespace -- and ensures that the BEGIN is passed as part of a separate argument.