REST API Authentication in Apache Airflow - HttpOperator - airflow

DAG - Image
I want to make a pipeline which will login the user on API, and and perform some operations using the API. I am exploiting my REST API using SimpleHttpOperator, the API is build on FLASK and some end-points require user to be logged in. I am unable to maintain session on the Airflow DAG so that I can call those end-points after calling the login route.

It will not work. Each task runs as different process or on different workers based on your executor type. So your secret_route task will not have any clue about the cookies received in login_task.
Assuming your API uses session based authentication, this is how your API's login and sessions work in a browser on a high level:
Browser sends login credentials to server
Server creates a session and send session ID to browser in cookie response header
Browser stores the session ID as cookie and sends the cookie to server in subsequent requests (for example, request to secret route)
Now if you want to achieve this with the DAG structure you proposed, then you will have to store the cookie and use it to when calling secret route.

Related

Symfony Security Deletes my Session Cookie Errantly

I have a Symfony 4.4 application, using symfony/security-bundle. The application accepts SAML connections. When a SAML connection comes in for a user that doesn't exist yet, Symfony...
Stores some data from the SAML request in the session with Symfony\Component\HttpFoundation\Session\SessionInterface::set()
Redirects the user to a registration page, which renders an Angular app that makes several concurrent API calls to help populate the page.
When the user submits the registration form, Angular makes another API call, which registers the user with a combination of form data and data stored in the session with SessionInterface::get().
The problem: Often, one of the API requests from step 2 will have header Set-cookie: PHPSESSID=deleted. This tells the browser to delete the cookie, throwing away the key to the session. As a result, the API call from step 3 has no way of retrieving the information from the session.
I'm storing session data in Redis, using Symfony's Symfony\Component\HttpFoundation\Session\Storage\Handler\RedisSessionHandler. I have confirmed the session data is in Redis, even after the session cookie gets deleted.
Temporary solution: Adding a new firewall with security: false, covering just the endpoints in step 2, prevented the endpoints from deleting the session cookie. So it seems that the security component is to blame.
How can I save data in the session for an unauthenticated user, and ensure the session cookie isn't removed before I need to retrieve the session data?

Outgoing HTTP requests involved in App Check token verification

I'm trying to verify Firebase App Check tokens on my custom backend. Everything is fine so far, but there's one thing I'm not sure about: My backend is hosted on a private network therefore I need to know if grabbing key sets from https://firebaseappcheck.googleapis.com/v1/jwks is the only necessary outgoing HTTP request involved in the verification process.
The answer is "Yes" according to https://github.com/firebase/firebase-admin-node/issues/2014#issuecomment-1353472578
Obtaining the Firebase App Check public JSON Web Key Set is the only outgoing request for the app check verify token API. The SDK might make other http calls during the initialization based on your environment (obtaining the service account or credentials etc.) or if you use other APIs in combination with the App check token verification.
The rest of the JWT verification happens all offline. We also cache the public keys (JWKS) for up to 6 hours, so if your environment doesn't lose its state then the outgoing request to fetch the keys should not happen (if the keys are cached) every time you call the API.

Use Firebase auth cookie in third server

I have a web application that deployed on Firebase Hosting and uses Firebase Authentication.
I also have a backend server, deployed on Google App Engine, that serve this app.
How can I get the Firebase-UID cookie in the backend server to validate the user?
I don't want to enforce the app to add the cookie content as a parameter for each request.
When I used the Google Identity Toolkit (merged into the Firebase Authentication since) it took a pretty long time to get an authentication token check done, so I didn't want to do it for each and every backend request.
So what I did was to perform it (typically) once per user session and, upon successful verification, create a unique memcache entry for the user and place its key in the user session info. So for subsequent requests for the same session I'd only check if the session info contains the memcache key and, if so, check if the corresponding memcache entry exists - a memcache key lookup is a lot faster than a token verification. The only thing needed to complete the picture is deletion of the memcache entry whenever the user logs out. If you want you can also enforce a token recheck after a certain amount of time - simply by setting the memcache entry's expiration time.
Note: the memcache entry can disappear anytime, which would require another token verification even if the user didn't log out - so multiple times per session. But in my case it was a rare enough occurrence.

How does ASP.NET WebAPI using IIS store my users authentication state?

I have an asp.net Web Api 2 / Identity 2 application that requires a user to be authenticated. The authentication works but I notice that when I restart my local development machine and try to access a method that requires authentication then I get a failure.
As my application is unchanged from the asp.net sample then I think it uses cookies to store user data on the client. Where and how does the Server or IIS store information on which users have authenticated? Does it do this just the once or on every HTTP? Is there a difference between my using Token or cookie authentication in the way that the authentication and also authorization is checked on the server?
I think you are misunderstanding how authentication works with ASP.Net. As an example, let me show you some cookie details for a site of mine that uses Identity (note the token is actually in the cookie, the two are not mutually exclusive concepts):
Name __RequestVerificationToken
Value afeILhaIvRr56jXXXXXXXXXXX
Host site.azurewebsites.net
Path /
Expires At end of session
Note that the cookie, by default, expires at the end of your session. That means when you restart your development machine, your cookie is expired and your token is no longer valid.
In particular I have read that with token authentication then there is no need for continual re-authentication every time a request is made to the server
You need to understand that HTTP is a stateless protocol. Each request happens in a vacuum, and therefore you need to pass some data back to the server so that it can tell that the person who authenticated with Request A is really the initiator of Request B. Almost always, that piece of data is from a cookie. So, every request does indeed re-authenticate, and typically with a token in a cookie.
The only piece of data about your session that is stored on the client is the cookie (unless you are doing something atypical). The rest is on the server. How it is stored can vary:
Inproc: Easiest to setup, sessions are stored in process. So when your server or app pool is restarted, that data disappears
State Server Mode: Sessions are stored in process, but outside of the ASP.Net worker process, so the application can be restarted without losing session data
SQL Server: Unsurprisingly, this stores data in a database. Very resilient, but more work to setup. Also your best option if you are on a web farm.
ref: http://msdn.microsoft.com/en-us/library/vstudio/ms178586(v=vs.100).aspx
Expanding on the great answer by Chris, I would like to add that there are two possible models here. In forms authentication (which is the default membership type for asp.net) the cookie can either store authentication information and then it's called a ticket or the information can be stored in session, with the cookie being a simple identifier for "reconnecting" the authenticated session with the requesting client on each subsequent request.
This "reconnecting" happens in the Application_AuthenticateRequest method of the global.asax. If you are using the default forms authentication storage, i.e. an SQL DB created for you by the framework, the reconnection will be done automatically. If you are using a custom authentication store (like accessing active directory yourself or a custom users table structure) you can override the the method and reconnect the authenticated session using your own implementation. In any case, the authentication data is populated in the User.Identity object's different properties. From that point, if you use the [Authorize] attribute, the framework accesses the object to check if the user is indeed authenticated and authorized.
I any case, the authentication information is tied to both the cookie and the session. Assuming your session is InProc, like Chris said, when the session is lost (through timeout, app pool recycle or restart of the dev machine) the server-side of the session is lost and your authentication / session cookie is replaced by a new one on the next request.
EDIT: Ohh... and one more side comment. Make sure you distinguish between authentication and authorization. The client is not re-authenticated on each request. Authentication is the process of providing your credentials and being identified by the server. Authorization is, now that the server has verified who you are, on each request it checks if you are authorized to access the resource you are requesting.
The server doesn't store information about who's authenticated and who isn't. Depending on your authentication mechanism (forms, tokens?), typically, when a user logs in, the server will return some form of authentication token that the client should pass back to the server on each API call.
Without knowing more about your configuration, it's difficult to explain why when you restart your server you have to re-authenticate, it sounds like the authentication token generated by the server is invalidated on restart.
Where and how does the Server or IIS store information on which users have authenticated?
IIS does not store state based on cookie authentication. Everything is determined based on the request. Either a request has the correct encrypted information, or it doesn't. If you look at a default Forms authentication in ASP.NET, you will find a cookie called .ADUAUTH ... this cookie has all the information to authenticate the request. If the cookie is half expired, it will be reset, but that's all IIS does.
Does it do this just the once or on every HTTP?
Every HTTP request is unique, so yes, per HTTP request.
Is there a difference between my using Token or cookie authentication in the way that the authentication and also authorization is checked on the server?
It's always checked on the server: To find out more, check out: How ASP.NET Security Works: http://msdn.microsoft.com/en-us/library/ks310b8y.ASPX
I think my answer could be a little contradicting to all of the above.. But I think If I understand right..
IIS stores inside the memory space of the ASP.NET worker process, i.e the session data in the RAM.
The storing of authentication state depends on the authentication model you are using. For example: if you are using the Federated authentication, through ADFS, then when a user loads your web page he is required to sign in providing his credentials. The ADFS then sets the authentication token which is stored in the session data, the session id is stored as cookies in user's browser. The server has the mapping of Session Id to its session data.
Now the user is authenticated. The ADFS looks for authentication token to mark a user as authenticated.
When you restart the server, the session data is lost as the data is stored in RAM.
There are ways to handle this, there are 3 types of session storage:
1. InProc (Stored in memory space of ASP .NET Worker process - RAM)
2. State Server (Stored out side of ASP .NET worker process, like on cloud Azure storage)
3. SQL Server session storage (Stored in SQL server)
I think you are adopting 1, because of which you encounter the problem.
In cases 2 and 3, the session is not lost when you restart the server.
Several things --
Token based authentication is not really authentication. It is just issuing you a unique token (can be a guid, unique string, etc) and then associating it with something (like your IP address) and saving that association server side (in a database?). Now whenever you use that token, from the client app, the server checks the association already stored and serves or denies or request.
In many ways, it is very similar to using Cookies to maintain authentication. Only, token-auth was designed more for web services operation than for UIs.
In short: Out of the box, the membership provider will run it's authentication method and upon success, it will create an auth ticket/token/cookie that will be stored from the site. In addition to this, there is a session cookie that is stored with the site as well. When you make a page request, it'll pull these things and use them to determine whether or not you are already authenticated. If it finds the ticket and sees that it is still good, it'll allow access.
When you restart your local environment, the session and it's information is destroyed which is why you have to log in again.
There is an entire pipeline in the framework that makes all of this stuff happen (having to do with authentication, authorization, and identity) and there are number of ok articles on the interwebs explaining this, but imo, they're almost all incomplete or hard to follow. If you want a great soup-to-nuts explanation, PluralSight.com has some training videos that will deconstruct and explain the entire pipeline for you. Understanding the pipeline can help you implement your own custom authentication, and I highly recommend it.

Workflow of JWT authentication

I'm tasked with creating a service-oriented ecosystem for a client. The whole thing is going to be REST based and built in ASP.NET, but my question is technology-agnostic. We want to have a centralized authentication service that issues JWT tokens and claims that are trusted by the other services in the environment.
My issue is this - what's the first thing that a web client (browser) requests? All of the diagrams I've seen (I'll try to add a couple of example links) make it seems as if the client needs to be self-aware and realize that they're going to need a token before they make the first request to the functional REST service, which seems, well, janky to me.
The way I want it to work is that they just attempt to access the secured resource, but there's no auth token with the request my REST service challenge them for user/password, but then delegate the authentication to my auth service. So:
Browser requests restricted resource on REST service
REST service returns 401
Browser gathers credentials, sends to same web service
REST service connects to the authentication service, passing along the Auth header from the client's request
Auth service creates the JWT token and returns it to the REST service
REST service validates the JWT and replaces the Auth header with the JWT token
JWT token is persisted for subsequent requests, up to expy setting
...am I completely off about this? Does the web client need to know that there's a separate auth service involved and make one request there to get their JWT, and then a second request for the REST resource passing the JWT? That seems clunky to me, I hope that's not the idea.
Also, another n00b question - is the JWT token automagically kept by the web clients and re-sent with every request so I don't have to go through the auth service step each time? Is that what the expiration setting is for?
TIA.
See figure 1 here for an example of what I mean: http://msdn.microsoft.com/en-us/library/hh446531.aspx
Starting with your last question will make the rest of the answers clearer:
"...is the JWT token automagically kept by the web clients and re-sent with every request.." - The idea is to issue JWT once, send it to the client so client can save it and send it on each subsequent request. This way your front-end app will send username and password just once and then use JWT for authentication. You will have to store the JWT using browser storage (local or session) or cookies (common fallback for older browsers).
"...Does the web client need to know that there's a separate auth service involved..." - You will need to send the username and password to a service in order to have the JWT issued. You could implement it with just one request, but you need to send credentials to the service (provided by the user), receive JWT as part of response and store it (as above). It might be easier to do it on a separate request, depending on requirements and implementation.

Resources