How do I trigger an Airflow DAG via the REST API? - airflow

The 1.10.0 documentation says I should be able to make a POST against /api/experimental/dags//dag_runs to trigger a DAG run, but instead when I do this, I receive an error:
<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 3.2 Final//EN">
<title>400 Bad Request</title>
<h1>Bad Request</h1>
<p>The browser (or proxy) sent a request that this server could not understand.</p>

To make this work, I figured out that I needed to send an empty JSON string in the body:
curl -X POST \
http://airflow.dyn.fa.disney.com/api/experimental/dags/people_data/dag_runs \
-H 'Cache-Control: no-cache' \
-d '{}'

With the stable API REST/Airflow 2.x, I used:
From CLI:
curl -X POST "http://localhost:8080/api/v1/dags/taskflow_api/dagRuns"
-H "accept: application/json" -H "Content-Type: application/json" \
-d "{}" \
--user "username:password"
With library request https://docs.python-requests.org/en/latest/:
import requests
import json
headers = {
'accept': 'application/json',
'Content-Type': 'application/json'
}
auth = ('username', 'password')
body = {
"conf": {}
}
response = requests.post(url=url,
headers=headers,
auth=auth,
data=json.dumps(body)
)
Note: to check which authentication backend is currently set, see this link https://airflow.apache.org/docs/apache-airflow/stable/security/api.html
I hope it is very useful
Regards

Related

POST request with httr fails while shell request works

I am trying to get data from an API with a POST request. The request works well with a direct shell command :
system(sprintf('curl POST -k --tlsv1.2 -v "https://api-gateway.inpi.fr/services/apidiffusion/api/marques/search" -H "X-XSRF-TOKEN: %s" -H \'accept: application/xml\' -H "Content-Type: application/json" -H "Cookie: XSRF-TOKEN=%s; access_token=%s; session_token=%s" -d \'%s\' > test.xml',token,token,access_token,refresh_token,json_request))
However, I would like to use httr for many reasons. I use the following code :
test <- httr::POST(
"https://api-gateway.inpi.fr/services/apidiffusion/api/marques/search",
httr::set_config(config(ssl_verifypeer = 0L)),
config = (add_headers(
"X-XSRF-TOKEN" = token,
"accept" = "application/xml",
"Content-Type" = "application/json",
"Cookie" = sprintf("XSRF-TOKEN=%s; access_token=%s; session_token=%s",token,access_token,refresh_token)
))
,set_cookies(`X-XSRF-TOKEN` = token,
`XSRF-TOKEN` = token,
access_token = access_token,
session_token = refresh_token)
,body = json_request
)
But this returns a 403 Forbidden error (my_token being the token I use) :
$error
[1] "access_denied"
$error_description
[1] "Invalid CSRF Token '*my_token*' was found on the request parameter '_csrf' or header 'X-XSRF-TOKEN'.
It seems like httr did not take into account my cookies because the token is different inside the test object I create :
> test2$cookies
domain flag path secure expiration name value
1 api-gateway.inpi.fr FALSE / FALSE <NA> XSRF-TOKEN *another_token*
Any idea ? I am sorry that I can't create a reproducible example for obvious security reasons.
Thank you !
The solution was wierd.
I had to rid off from httr, I used UNIX system commands instead, and it worked with the same request.
system(sprintf('curl POST -k --tlsv1.2 "https://api-gateway.inpi.fr/services/apidiffusion/api/marques/search" -H "X-XSRF-TOKEN: %s" -H \'accept: application/json\' -H "Content-Type: application/json" -H "Cookie: XSRF-TOKEN=%s; access_token=%s; session_token=%s" -d \'%s\' > %s/res.json',tokens$xsrf_token,tokens$xsrf_token,tokens$access_token,tokens$refresh_token,json_request,tempdir()))
It seems like httr tries to handle cookies by its own, so maybe that's what caused my problem.

Change project key with bitbucket rest api

I am trying to write a script to move repos in a project to another project but I am getting a 400 error whenever I try.
My python requests line looks like:
url = 'https://bitbucketserver.com/rest/api/1.0/projects/example1/repos/repo1'
token = 'TokenString'
response = requests.put(url, headers={'Content-Type': 'application/json', 'Authorization': 'Bearer' + token}, data={'project': {'key': 'NEW_PROJECT'}}, verify=False)
I get a response 400 that says 'Unexpected character ('p' (code112)): expected a valid value (number, string, array, object, true, false, or null) at [Source: com.atlassian.stash.internal.web.util.web.CountingServletInputStream#7ccd7631; line 1, column 2]
I'm not sure where my syntax is wrong
Not python, but work for me via curl:
curl -u 'USER:PASSWORD' --request PUT \
--url 'https://stash.vsegda.da/rest/api/1.0/projects/OLD_PROJECT/repos/REPO_TO_MOVE' \
--header 'Accept: application/json' \
--header 'Content-Type: application/json' \
--data '{
"project": {"key":"NEW_PROJECT"}
}'
Maybe can someone help.

Why is Firebase Cloud Messaging web client side messaging.getToken() yield a different token sometimes? (Same browser, same tab)

I've found that the FCM web client side occasionally delivers a different token to me.
My code is entirely based on the official docs.
// Get registration token. Initially this makes a network call, once retrieved
// subsequent calls to getToken will return from cache.
messaging.getToken({ vapidKey: '<YOUR_PUBLIC_VAPID_KEY_HERE>' }).then((currentToken) => {
if (currentToken) {
// Send the token to your server and update the UI if necessary
// Occasionally, I received a different token here.
} else {
// Show permission request UI
console.log('No registration token available. Request permission to generate one.');
// ...
}
}).catch((err) => {
console.log('An error occurred while retrieving token. ', err);
// ...
});
Occasionally, I receive a different currentToken; this occurs regularly!
I did not change my web browser (Chrome), nor did I change the vapidKey, and I continue to use the same tab for localhost development.
It seemed to have changed every couple of hours.
And if the token changes, my service worker will record an error in the console, as well as a 404 API request failure.
The console log error is as follows:
FirebaseError: Messaging: A problem occurred while unsubscribing the user from FCM: FirebaseError: Messaging: A problem occurred while unsubscribing the user from FCM: Requested entity was not found. (messaging/token-unsubscribe-failed). (messaging/token-unsubscribe-failed).
at _callee8$ (eval at <anonymous> (app.js:2982), <anonymous>:572:45)
at tryCatch (eval at <anonymous> (app.js:6025), <anonymous>:62:40)
at Generator.invoke [as _invoke] (eval at <anonymous> (app.js:6025), <anonymous>:296:22)
at Generator.prototype.<computed> [as next] (eval at <anonymous> (app.js:6025), <anonymous>:114:21)
at step (eval at <anonymous> (app.js:636), <anonymous>:17:30)
at eval (eval at <anonymous> (app.js:636), <anonymous>:28:13)
And here is the failed API request I believe FCM was attempting to make; I did not make the request, and I copied it as curl for greater visibility.
curl 'https://fcmregistrations.googleapis.com/v1/projects/chatisfy-d2721/registrations/cD0VOZLBLdaVymfaUbQyE4:APA91bHj2qCU02_Sib6gEPw3VuPTDkjpj0ZVpgmWYaaHESpTjpH-uwY5JX5mn_W7YhJ1AOMp4dNnwpUffs7SQkBs1UYGGie0o4u_i-OjYY5Q5uRSl3pZQoRGVzwNXxe0lDrQIHD4SN5A' \
-X 'DELETE' \
-H 'authority: fcmregistrations.googleapis.com' \
-H 'sec-ch-ua: " Not A;Brand";v="99", "Chromium";v="96", "Google Chrome";v="96"' \
-H 'sec-ch-ua-mobile: ?0' \
-H 'user-agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/96.0.4664.110 Safari/537.36' \
-H 'content-type: application/json' \
-H 'accept: application/json' \
-H 'x-goog-api-key: AIzaSyDFP12b-P9JwiDvJuqsWVz6k2Z8ww6_2-E' \
-H 'x-goog-firebase-installations-auth: FIS eyJhbGciOiJFUzI1NiIsInR5cCI6IkpXVCJ9.eyJhcHBJZCI6IjE6NTI4MTcxMTk2MzYxOndlYjowMWQyMzQ4ODNiYWQ3NWU5MmYxMjE4IiwiZXhwIjoxNjQwMjUwMDc2LCJmaWQiOiJjRDBWT1pMQkxkYVZ5bWZhVWJReUU0IiwicHJvamVjdE51bWJlciI6NTI4MTcxMTk2MzYxfQ.AB2LPV8wRQIhAL4F96JV_fSn2LHzpBiYDWnOVcpA7zBT35lvWz0WqS8fAiAlC28Un0hO2uD6_DuRPnHdqnO_5wIr-byID127niFXRg' \
-H 'sec-ch-ua-platform: "macOS"' \
-H 'origin: http://localhost:8080' \
-H 'x-client-data: CJO2yQEIpbbJAQjBtskBCKmdygEInvnLAQjmhMwBCLWFzAEIy4nMAQjSj8wBGI6eywE=' \
-H 'sec-fetch-site: cross-site' \
-H 'sec-fetch-mode: cors' \
-H 'sec-fetch-dest: empty' \
-H 'referer: http://localhost:8080/' \
-H 'accept-language: en-US,en;q=0.9,zh-TW;q=0.8,zh;q=0.7,zh-CN;q=0.6' \
--compressed
And here is the response to the aforementioned request:
{
"error": {
"code": 404,
"message": "Requested entity was not found.",
"status": "NOT_FOUND"
}
}
What is causing this?
Some security models expire and reissue tokens at intervals to prevent long-term token validity. Or the token may contain data that updates based on request time.

Amazon - Can't request performance report using Advertising API

I'm getting an error using amazon advertising API. I'm currently trying to request performance report using https://advertising-api.amazon.com/v1/campaigns/report.
But the server reply Cannot consume content type
here is my request header and body.
End point : https://advertising-api.amazon.com/v1/campaigns/report
Method Type: POST
Header :
{
Authorization: 'Bearer xxxxxx',
Amazon-Advertising-API-Scope: '11111111111',
Content-Type: 'application/json'
}
Body :
{
campaignType:'sponsoredProducts',
reportDate:'20180320',
metrics:'impressions,clicks'
}
I think I did everything correctly as API document but it says
{
"code": "415",
"details": "Cannot consume content type"
}
Please help me.
Try this way
curl -X POST \
https://advertising-api.amazon.com/v1/campaigns/report \
-H 'Amazon-Advertising-API-Scope: REPLACE_YOUR_PROFILE_ID' \
-H 'Authorization: REPLACE_YOUR_ACCESS_TOKEN' \
-H 'Cache-Control: no-cache' \
-H 'Content-Type: application/json' \
-H 'Host: advertising-api.amazon.com' \
-H 'cache-control: no-cache' \
-d '{
"campaignType": "sponsoredProducts",
"metrics": "impressions,clicks",
"reportDate": "20181101"
}
And you will get a response like
{
"reportId": "amzn1.clicksAPI.v1.p1.......",
"recordType": "campaign",
"status": "IN_PROGRESS",
"statusDetails": "Report is being generated."
}
You can put this curl command in Postman also.
I think your Body may be missing a parameter. When I successfully make a similar POST I need my body to have at least what you have written as well as the segment type. Try adding this to your body:
{
campaignType:'sponsoredProducts',
reportDate:'20180320',
metrics:'impressions,clicks'
segment:'query'
}
Just copy the body from the documentation and paste it in the raw area (of postman) and choose JSON format. For me it works fine.

How do I POST XML data with curl

I want to post XML data with cURL. I don't care about forms like said in How do I make a post request with curl.
I want to post XML content to some webservice using cURL command line interface. Something like:
curl -H "text/xml" -d "<XmlContainer xmlns='sads'..." http://myapiurl.com/service.svc/
The above sample however cannot be processed by the service.
Reference example in C#:
WebRequest req = HttpWebRequest.Create("http://myapiurl.com/service.svc/");
req.Method = "POST";
req.ContentType = "text/xml";
using(Stream s = req.GetRequestStream())
{
using (StreamWriter sw = new StreamWriter(s))
sw.Write(myXMLcontent);
}
using (Stream s = req.GetResponse().GetResponseStream())
{
using (StreamReader sr = new StreamReader(s))
MessageBox.Show(sr.ReadToEnd());
}
-H "text/xml" isn't a valid header. You need to provide the full header:
-H "Content-Type: text/xml"
I prefer the following command-line options:
cat req.xml | curl -X POST -H 'Content-type: text/xml' -d #- http://www.example.com
or
curl -X POST -H 'Content-type: text/xml' -d #req.xml http://www.example.com
or
curl -X POST -H 'Content-type: text/xml' -d '<XML>data</XML>' http://www.example.com
It is simpler to use a file (req.xml in my case) with content you want to send -- like this:
curl -H "Content-Type: text/xml" -d #req.xml -X POST http://localhost/asdf
You should consider using type 'application/xml', too (differences explained here)
Alternatively, without needing making curl actually read the file, you can use cat to spit the file into the stdout and make curl to read from stdout like this:
cat req.xml | curl -H "Content-Type: text/xml" -d #- -X POST http://localhost/asdf
Both examples should produce identical service output.
Have you tried url-encoding the data ? cURL can take care of that for you :
curl -H "Content-type: text/xml" --data-urlencode "<XmlContainer xmlns='sads'..." http://myapiurl.com/service.svc/
You can try the following solution:
curl -v -X POST -d #payload.xml https://<API Path> -k -H "Content-Type: application/xml;charset=utf-8"

Resources