How to access config values when triggering Airflow DAG externally? - airflow

According to https://airflow.apache.org/api.html I can trigger an Airflow DAG like so:
curl -X POST \
http://localhost:8080/api/experimental/dags/<DAG_ID>/dag_runs \
-H 'Cache-Control: no-cache' \
-H 'Content-Type: application/json' \
-d '{"conf":"{\"key\":\"value\"}"}'
This seems to work ok for me, but I cannot figure out how to access the key/value stuff in the conf object being passed in.
I tried this:
something = dag.params.get("key", "unknown")
But it doesn't seem to work.
Does anyone know how to do this?

Related

How to write function in .zsh profile that returns cURL response?

At work, I run a command daily to return a specific token for access to our local repos. It's not a big deal, but I was trying to figure out a faster way of generating this token. Usually, I paste it into VSCode and run a Send Request via the REST Client Extension.
I'm trying to create a function in my .zsh profile that can then be run in my command line instead. Here's what I currently have:
generateToken() {
response=curl -H "Content-Type: application/json" -H "service-token: xxxx" -X POST -d '{"user": "me.myself#xxxxxxx.com"}' <the-url.com>
echo "$(response)"
}
Then I save, quit, and run eval ${generateToken}, but nothing happens. Any suggestions on the best way to get this going?
Thanks to #thatotherguy in the comments, I was able to create the following:
function generateToken() {
echo "Generating token..."
response=$(curl -H "Content-Type: application/json" -H "service-token: xxxxxxx" -X POST -d '{"user": "me.myself#mycompany.com"}' https://api.my.company/token)
echo "${response}"
}
If you're not doing anything to the output, you don't need to capture it and echo it. You can just run the command. It will write its output to the same place that echo would if you captured and re-echoed the output.
generateToken() {
echo "Generating token..."
curl -H "Content-Type: application/json" -H "service-token: xxxxxxx" \
-X POST -d '{"user": "me.myself#mycompany.com"}' \
https://api.my.company/token
}

Why can't the exported cURLS be imported back to POSTMAN?

The latest update of the POSTMAN (7.16) doesn't seem to understand the cURL when importing them.
Here is a cURL exported by POSTMAN itself via the "Code" option.
curl -L -X POST 'https://example.com/example/' \
-H 'Content-Type: application/json' \
--data-raw '{
"requestFields":{
"address":"123, Main Street",
"ping":false
}
}'
But when this cURLis is tried to be imported to POSTMAN again, it doesn't populate the Body area of the request.
Is there a workaround?

Is it possible in Firebase to setCustomClaims with curl?

I am trying to set setCustomClaims on a user
with curl. Is it possible?
So far I got this
curl 'https://identitytoolkit.googleapis.com/v1/accounts:setCustomClaims?key=[API_KEY]' \
-H 'Content-Type: application/json' \
--data-binary '{"email":"[user#example.com]","password":"[PASSWORD]"}'

Pass parameters to Airflow Experimental REST api when creating dag run

Looks like Airflow has an experimental REST api that allow users to create dag runs with https POST request. This is awesome.
Is there a way to pass parameters via HTTP to the create dag run? Judging from the official docs, found here, it would seem the answer is "no" but I'm hoping I'm wrong.
I had the same issue. "conf" value must be in string
curl -X POST \
http://localhost:8080/api/experimental/dags/<DAG_ID>/dag_runs \
-H 'Cache-Control: no-cache' \
-H 'Content-Type: application/json' \
-d '{"conf":"{\"key\":\"value\"}"}'
Judging from the source code, it would appear as though parameters can be passed into the dag run.
If the body of the http request contains json, and that json contains a top level key conf the value of the conf key will be passed as configuration to trigger_dag. More on how this works can be found here.
Note the value of the conf key must be a string, e.g.
curl -X POST \
http://localhost:8080/api/experimental/dags/<DAG_ID>/dag_runs \
-H 'Cache-Control: no-cache' \
-H 'Content-Type: application/json' \
-d '{"conf":"{\"key\":\"value\"}"}'
This is no longer true with the stable REST API.
You can do something like -
curl --location --request POST 'localhost:8080/api/v1/dags/unpublished/dagRuns' \
--header 'Content-Type: application/json' \
--header 'Authorization: Basic YWRtaW46YWRtaW4=' \
--data-raw '{
"dag_run_id": "dag_run_1",
"conf": {
"key": "value"
}
}'
I understand that the question is asked for experimental API but this question is the top search result for airflow REST API.

How can I use the Pingdom API to pause and resume checks from bash?

I'm writing a quick and dirty deployment script and would like to disable and reenable a Pingdom check as part of it. How do I do that using something like cURL?
To pause a check:
curl -X PUT -u 'your#email:yourpassword' -H 'Content-Type: application/json' -H 'App-Key: yourapplicationkey' -d 'paused=true' https://api.pingdom.com/api/2.0/checks/checkid
To resume a check:
curl -X PUT -u 'your#email:yourpassword' -H 'Content-Type: application/json' -H 'App-Key: yourapplicationkey' -d 'paused=false' https://api.pingdom.com/api/2.0/checks/checkid
Replace your#email with your pingdom email.
Replace yourpassword with your pingdom password.
Replace yourapplicationkey with a generated key from the "Sharing" section in your account.
Replace checkid with the numeric ID you see in the browser URL when you click on your check in the Pingdom UI.
You can also use modern way - just API key instead of using also email/password.
First, generate your own API key in https://my.pingdom.com/app/api-tokens and then you can use curl commands like for pausing:
curl -X PUT \
https://api.pingdom.com/api/3.1/checks \
-H 'Authorization:Bearer YOURAPIKEY' \
-d 'paused=true&checkids=777'
or for resuming:
curl -X PUT \
https://api.pingdom.com/api/3.1/checks \
-H 'Authorization:Bearer YOURAPIKEY' \
-d 'paused=false&checkids=777'
Replace YOURAPIKEY with your real API key and 777 with valid check ID.
checkids can be also omitted, then all checks will be modified.

Resources