httr GET request with large query list - r

I have translated this cURL command into a httr:GET() request in R. However, I'm still getting a 400 error as an invalide request. I double checked using the normal cURL command in a git terminal and it worked perfectly.
Where is the problem in my translation of cURL command to R httr:GET query? Do I need to account for the -G at the end of the cURL in my GET request?
$ curl https://api.goclimate.com/v1/flight_footprint \
-u YOUR_API_KEY: \
-d 'segments[0][origin]=ARN' \
-d 'segments[0][destination]=BCN' \
-d 'segments[1][origin]=BCN' \
-d 'segments[1][destination]=ARN' \
-d 'cabin_class=economy' \
-d 'currencies[]=SEK' \
-d 'currencies[]=USD' \
-G
In R:
r <- GET("https://api.goclimate.com/v1/flight_footprint/",
authenticate(api.key,""),
query = list(
"segments[0][origin]"="ARN",
"segments[0][destination]"="BCN",
"segments[1][origin]"="BCN",
"segments[1][destination]"="ARN",
"cabin_class"="ecomony",
"currencies[]"="SEK",
"currencies[]"="USD"))

Related

Exclude Path for GoAccess

I am trying to exclude some Paths in GoAccess to get an better result for my WordPress Installation.
So I am doing this:
#!/bin/sh
set -x
zcat -f /var/log/nginx/access-blog.log* > /var/www/serverstats/work/access-parsed.log | grep -Ev '/wp-config.php|/xmlrpc.php|/wp-json|/adminer|/robots.txt|/app-ads.txt|/ads.txt|/wp-login.php|//feed|/?author=|/wp-content|/wp-admin|/rss|/api/v1|/wp-cron.php' /var/www/serverstats/work/access-parsed.log |goaccess \
--log-file=/var/www/serverstats/work/access-parsed.log \
--log-format=COMBINED \
--exclude-ip=0.0.0.0 \
--geoip-database=/var/www/serverstats/GeoLite2-City.mmdb \
--ignore-crawlers \
--hide-referer=*.dasnetzundich.de \
--hide-referer=dasnetzundich.de \
--browsers-file=/var/www/serverstats/crawler.list \
--anonymize-ip \
--persist \
--db-path=/var/www/serverstats/db \
--real-os \
--output=/var/www/serverstats/index.html
rm /var/www/serverstats/work/access-parsed.log
set +x
but that wont work. Can anyone help me to get it working?
Thanks
Lars
I tried these Scripts and it wont exclude the path like /xmlrpc.php and wp.login.php.
It would be amazing if I can filter the output from GoAccess.

Using grep to remove from verbose call and saving in a shell script function

I have a curl put request that I can use some unix command to see the stdout and stderr from.
If I add the line 2>&1 | grep -v "Authorization" At the end of my curl request, I can see the verbose output from my curl in my CLI minus a line that starts with "Authorization"
I tried creating a function with the above command, but when I call that function at the end of my curl request (like below) it no longer removes the "Authorization" line.
function remove_item_from_verbose {
2>&1 | grep -v "Authorization"
}
echo -e "\n +++ Creating '$TENANT' tenant:\n"
# Create tenant
curl -L -X PUT "http://localhost:$HOST_PORT/admin/v2/tenants/$TENANT" \
--verbose \
--header 'Content-Type: application/json' \
--header "Authorization: Bearer $AUTHORIZATION" \
--data-raw "{\"allowedClusters\": [\"$CLUSTER\"]}" remove_item_from_verbose
AFAIK functions do work like that. What you are trying to so is add extra details to your command. You can do it like that:
#!/bin/bash
remove_item_from_verbose="2>&1 | grep -v \"Authorization\""
echo -e "\n +++ Creating '$TENANT' tenant:\n"
# Create tenant
eval curl -L -X PUT "http://localhost:$HOST_PORT/admin/v2/tenants/$TENANT" \
--verbose \
--header 'Content-Type: application/json' \
--header "Authorization: Bearer $AUTHORIZATION" \
--data-raw "{\"allowedClusters\": [\"$CLUSTER\"]}" "$remove_item_from_verbose"
Like this, $remove_item_from_verbose becomes a suffix of your command.
Note the addition of eval in front of curl, it is required to have this suffix treated as part of the command.

Protoc does not export the TS file version of *_grpc_pb.js?

I am new to setting up the gRPC web based client side. Our backend is already up and running on Go with gRPC. I am testing out what it's like converting the .proto file into TS. I am successfully able to generate some of the files, however, I am missing the TypeScript "Service" file.
I pretty much followed the instructions from the grpc_tools_node_protoc_ts site.
Setup a script to generate files for 1) the service and 2) the client model:
PROTOC_GEN_TS_PATH="./node_modules/.bin/protoc-gen-ts"
GRPC_TOOLS_NODE_PROTOC_PLUGIN="./node_modules/.bin/grpc_tools_node_protoc_plugin"
GRPC_TOOLS_NODE_PROTOC="./node_modules/.bin/grpc_tools_node_protoc"
OUT_DIR="./_protos_/proto/"
# JavaScript code generating
${GRPC_TOOLS_NODE_PROTOC} \
--plugin=protoc-gen-grpc="${GRPC_TOOLS_NODE_PROTOC_PLUGIN}" \
--js_out=import_style=commonjs,binary:"${OUT_DIR}" \
--grpc_out="${OUT_DIR}" \
-I "${OUT_DIR}" \
"${OUT_DIR}"/*.proto
${GRPC_TOOLS_NODE_PROTOC} \
--plugin=protoc-gen-ts="${PROTOC_GEN_TS_PATH}" \
--ts_out="${OUT_DIR}" \
-I "${OUT_DIR}" \
"${OUT_DIR}"/*.proto
What I get on the output is missing the *_grpc_pb.d.ts. I am under the impression I need this? 🤷🏻‍♂️
I have also tried adding the service option to the flag:
--ts_out="service=grpc-web:${OUT_DIR}" \
This now generates a *_pb_service.d.ts output file, still without the *_grpc_pb.d.ts file. I was reading the docs more and am thinking this service=grpc-web is actually the option I need since we're not running a node server.
Does this seem right? This is what I have now:
# Note the ts_out flag "service=grpc-node":
# This does generate the *_grpc_pb.d.ts but not the service files
protoc \
--plugin="protoc-gen-ts=${PROTOC_GEN_TS_PATH}" \
--plugin=protoc-gen-grpc=${GRPC_TOOLS_NODE_PROTOC_PLUGIN} \
--js_out="import_style=commonjs,binary:${OUT_DIR}" \
--ts_out="service=grpc-node:${OUT_DIR}" \
--grpc_out="${OUT_DIR}" \
-I "${OUT_DIR}" \
"${OUT_DIR}"/*.proto
# Note the ts_out flag "service=grpc-web":
# This does generate the service files, but not the *_grpc_pb.d.ts file
protoc \
--plugin="protoc-gen-ts=${PROTOC_GEN_TS_PATH}" \
--plugin=protoc-gen-grpc=${GRPC_TOOLS_NODE_PROTOC_PLUGIN} \
--js_out="import_style=commonjs,binary:${OUT_DIR}" \
--ts_out="service=grpc-web:${OUT_DIR}" \
--grpc_out="${OUT_DIR}" \
-I "${OUT_DIR}" \
"${OUT_DIR}"/*.proto

Phabricator API - Create a Project is not working

curl -k -s -X POST "https://phabricator_URL.eu/api/project.edit" \
-d api.token=api-myapitoken \
-d members.add=TestMember \
-d name=TestProject \
-d objectIdentifier= | jq -r
ERROR receive:
{
"result": null,
"error_code": "ERR-CONDUIT-CALL",
"error_info": "API Method \"project.edit\" does not define these parameters: 'members.add', 'name'."
}
Link Used:
https://secure.phabricator.com/conduit/method/project.edit/
You have to specify members.add as part of a list of transactions, and the users to add must also be a list:
curl -k -s -X POST "https://phabricator_URL.eu/api/project.edit" \
-d api.token=api-myapitoken \
-d transactions[0][type]=name
-d transactions[0][value]=TestProject \
-d transactions[1][type]=members.add \
-d transactions[1][value][0]=PHID-USER-5555 \
-d objectIdentifier= | jq -r
If you submit the API call you want to make through the conduit doc page, it will display the curl call needed to perform those same actions.

How to place a file on salt master via salt-api

I want to place a file a file on salt-master via salt-api. I have configured salt-api using rest cherrypy and configured a custom hook for it. I wanted to explore the use-case where we can transfer the file first to salt-master and secondly distribute it to minions. I'm able to achieve the second part but not been able to post data file to the API.
Here is one way to do it using file.write execution module.
First login and save the token to a cookie file (I had to change eauth to ldap, auto didn't work for some reason):
curl -sSk http://localhost:8000/login \
-c ~/cookies.txt \
-H 'Accept: application/x-yaml' \
-d username=USERNAME\
-d password=PASSWORD \
-d eauth=auto
Now run a job to create a file on the salt-master (assuming your salt-master is also running a salt-minion):
curl -sSk http://localhost:8000 \
-b ~/cookies.txt \
-H 'Accept: application/x-yaml' \
-d client=local \
-d tgt='saltmaster' \
-d fun=file.write \
-d arg='/tmp/somefile.txt' \
-d arg='This is some example text
with newlines
A
B
C'
Note that the spacing used in your command will affect how the lines will show up in the file, with the example above is gives the most aesthetically pleasing result.

Resources