I have created DAG file that lists the S3 files. This Dag file is working fine when there is no multi-factor authentication(MFA) enabled. After that, I have enabled the AWS account MFA then the DAG file is showing the "Access denied" exception. Can anyone know how to handle multi-factor authentication in Airflow while listing S3 files
MFA tokens on AWS are only valid for 1 hour (or maybe 6 hours at max) so using MFA tokens for any kind of programatic access is never really going to work without a lot of manual intervention (i.e. re-populating the "token" field in the S3 connection string in this context.)
So in short: you can't really handle this in Airflow or any other long running service.
Related
I am using the Airflow EcsOperator to run AWS ECS Tasks. As part of this, I am using a custom fluentbit container that is setup to log the container logs to Cloudwatch and AWS Open Search. The logging to both destinations work fine. However, I noticed that the Cloudwatch log groups are getting generated in the format {awslogs_stream_prefix}-{ecs_task_id}. Braces are added to just show the two parts separately, the actual prefix is of the form "ecsprime-generator-container-firelens-977be157d3be4614a84537cd081152d7" where the string starting with 977 is the Task Id. Unfortunately, Airflow code that reads Cloudwatch logs expects the log group name to be in the format {awslogs_stream_prefix}/{ecs_task_id}. Due to this, I am not able to have the Airflow EcsOperator display the corresponding Cloudwatch logs.
Are there any workarounds to address this?
I am managing a Google Cloud Composer environment which runs Airflow for a data engineering team. I have recently been asked to troubleshoot one of the dags they run which is failing with this error : [12:41:18,119] {credentials_utils.py:23} WARNING - [redacted-name] connection ID not available, falling back to Google default credentials
The job is basically a data pipeline which reads from various sources and stores data into GBQ. The odd part is that they have a strictly similar Dag running for a different project and it works perfectly.
I have recreated the .json credentials for the service account behind the connection as well as the connection itself in Airflow. I have sanitized the code to see if there was any hidden spaces or so.
My knowledge of Airflow is limited and I have not been able to find any similar issue in my research, any one have encountered this before?
So the DE team came back to me saying it was actually a deployment issue where an internal module involved in service account authentication was being utilized inside another DAG running in stage environment, rendering it impossible to proceed to credential fetch from the connection ID.
I've a project using firebase realtime database for over 2 years now... it is a big project with many rules and over 1GB data and around 10k users dayly
I was checking the firebase rules monitor and i notice something unwanted
In my happy world there would be ZERO denies and zero errors. The denies I'm monitoring on my own, but I dont know any way to monitore the errors or debug them
I understand that errors come from firebase trying to evaluate some specific rule and it throwing an exception... BUT HOW CAN I DETERMINE WHICH RULE WAS IT? so i can fix the error...
when dealing with the client side firebase only return PERMISSION DENIED without any further explanation so it is impossible to me guess from the client side if this was a normal permission denied or anything related to a rule error [making it impossible to log or debug]
does anyone know ANY WAY to determine which rules are having errors?
================UPDATE===================
I've pasted the rules in this bin
https://ghostbin.co/paste/3bj823
This normally isn't possible with the limitations firebase gives us.
However, a sound solution is to have a catch on your realtime requests that pushes a log object to realtime db in a separate bucket or a dedicated node. Since realtime db writes are fundamentally free besides storage, this creates an environment where you can debug the users' request, their app state, and any data they may be trying to send.
example:
realtimeReference.push({
user: uid,
path: "path/that/failed",
payload: Json.stringify(payload),
meta: {additional:data,etc:etc},
timestamp: firebase.database.ServerValue.TIMESTAMP
})
To ensure that your app isn't always pushing data and to avoid constantly updating your app, you can use global variables with Remote Config:
https://firebase.google.com/docs/remote-config
If you are using cloud functions, you could automatically save all errors into a separate log file with the full trace of the requests chains
With the code below I have been connected to the telegram using telethon for about one year!
client = TelegramClient(
SESSION,
API_ID,
API_HASH
)
client.connect()
# the rest of my code
Yesterday when I want to connect to my session with the code I got this error:
telethon.errors.rpcerrorlist.AuthKeyDuplicatedError: An auth key with the same ID was already generated
The error does not tell anymore and in the doc there is no more description about this error message. How can I retrieve my session?
one of the ways to produce this error is to run 2 instances of the same session in separate locations simultaneously(or at least with a small difference in their connection time to TG)
if you have access to those phone numbers(or any other active sessions of them e.g. Android app), you can do the below procedure, otherwise as far as I know, I afraid that you can't do anything for other compromised clients that you don't have access to their numbers(or any other active sessions in other applications):
First, make sure that no Telethon session instances are running on your server(s)(at least for those you have access to). Second, you can kill that session manually(in "active session" part of telegram applications e.g. Android). third, delete ".session" files from your server. fourth, sign in to your client in the server again.
I have been configuring AWS CodeDeploy for a few days and my first deployment is failing. The error message I get reads "The overall deployment failed because too many individual instances failed deployment, too few healthy instances are available for deployment, or some instances in your deployment group are experiencing problems."
To get more detailed info I have installed the AWS CodeDeploy agent on the Windows instance and it appears not to be working. All what I manage to read in the code-deploy-agent-log.txt file are the repetitive lines.
2016-05-31 16:05:24 DEBUG [codedeploy-agent(4872)]: InstanceAgent::Plugins::CodeDeployPlugin::CommandPoller: Sleeping 90 seconds.
2016-05-31 16:06:55 DEBUG [codedeploy-agent(4872)]: InstanceAgent::Plugins::CodeDeployPlugin::CommandPoller: Calling PollHostCommand:
2016-05-31 16:06:55 INFO [codedeploy-agent(4872)]: Version file found in C:/ProgramData/Amazon/CodeDeploy/.version.
2016-05-31 16:06:55 ERROR [codedeploy-agent(4872)]: InstanceAgent::Plugins::CodeDeployPlugin::CommandPoller: Missing credentials - please check if this instance was started with an IAM instance profile
My question is how can I get more information about the error message I am getting over the deployments. Which credentials am I missing (specifying incorrectly) that cause to the error message in the log file?
I think you are missing service-role-arn while creating your deployment group. The service role ARN allows AWS CodeDeploy to act on the user's behalf when interacting with AWS services. The service role ARN is of the code deploy role that you may have created it earlier.
In addition please make sure that your deployment policy is set to CodeDeployDefault.OneAtATime. This is to avoid taking all instances down if you push incorrect or failing build.
I tried Suken Shah's steps it didn't solve for me. What solved for me are:
1)Creating an IamInstanceProfile say Webserver.
2)Adding AWSCodeDeployRole to the IamInstanceProfile Webserver.
3)Adding the following to AWSCodeDeployRole's Trust Relationship: "codedeploy.amazonaws.com", "ec2.amazonaws.com", "codedeploy.MY_REGION.amazonaws.com"
4)Rebooting the ec2
Make sure the role you use for EC2 has 'AWSCodeDeployRole' policy and trust relationship has 'ec2.amazonaws.com' service. if you need to change the role then restart the EC2