how to check the output of airflow test? - airflow

Question

I tried the airflow tutorial DAG, and it works with scheduler, I can see the logs generated by scheduled run. But if I use command line test, I didn't see the output:
airflow test my_tutorial_2 templated 2018-09-08
[2018-09-10 15:41:43,121] {__init__.py:51} INFO - Using executor SequentialExecutor
[2018-09-10 15:41:43,281] {models.py:258} INFO - Filling up the DagBag from /Users/xiang/Documents/BigData/airflow/dags
[2018-09-10 15:41:43,338] {example_kubernetes_operator.py:54} WARNING - Could not import KubernetesPodOperator: No module named 'kubernetes'
[2018-09-10 15:41:43,339] {example_kubernetes_operator.py:55} WARNING - Install kubernetes dependencies with: pip install airflow['kubernetes']
That is all the output, and my output is not there.
The airflow version is:
▶ pip list
Package Version
---------------- ---------
alembic 0.8.10
apache-airflow 1.10.0

Answer 1

If you use Ariflow v1.10, you can set the propagate attribute of taskinstance logger to True, then the log record will be propagated to root logger, which use console handler, and printed to sys.stdout.
Add ti.log.propagate = True
after line 589 to site-packages/airflow/bin/cli.py could do this trick.

Answer 2

I've since found that whilst setting 'console' as a handler for the airflow.task logger allows you to see the output of 'airflow test' commands, it also seems to cause 'airflow run' commands to enter an infinite loop and run out of memory. I would therefore only do this in an environment where you only want to run 'airflow test' commands
Why it does this I don't know yet, and whether there's a way to accomplish this question without breaking 'airflow run' is unclear to me
The default logging config for Airflow 1.10.0 has the following loggers available:
'loggers': {
'airflow.processor': {
'handlers': ['processor'],
'level': LOG_LEVEL,
'propagate': False,
},
'airflow.task': {
'handlers': ['task'],
'level': LOG_LEVEL,
'propagate': False,
},
'flask_appbuilder': {
'handler': ['console'],
'level': FAB_LOG_LEVEL,
'propagate': True,
}
},
and the airflow.task logger (which is the logger used when running your task) uses the 'task' handler:
'handlers': {
'console': {
'class': 'airflow.utils.log.logging_mixin.RedirectStdHandler',
'formatter': 'airflow',
'stream': 'sys.stdout'
},
'task': {
'class': 'airflow.utils.log.file_task_handler.FileTaskHandler',
'formatter': 'airflow',
'base_log_folder': os.path.expanduser(BASE_LOG_FOLDER),
'filename_template': FILENAME_TEMPLATE,
},
'processor': {
'class': 'airflow.utils.log.file_processor_handler.FileProcessorHandler',
'formatter': 'airflow',
'base_log_folder': os.path.expanduser(PROCESSOR_LOG_FOLDER),
'filename_template': PROCESSOR_FILENAME_TEMPLATE,
},
},
which (unless changed) will only write the output of the task to a log file. If you want to see the output in stdout as well, then you need add the console handler to the list of handlers used by the airflow.task logger:
'airflow.task': {
'handlers': ['task', 'console'],
'level': LOG_LEVEL,
'propagate': False,
},
This can be done by either setting up a custom logging configuration class, which overrides the default configuration, or by editing the default settings file
wherever_you_installed_airflow/site-packages/airflow/config_templates/airflow_local_settings.py

Answer 3

I ran into this problem as well with AirFlow 1.10.0. As Louis Genasi mentioned, airflow run would go into a death spiral with default settings and console handler. I suspect there may be a bug with the default logging class in 1.10.0.
I got around the issue by changing the logging handler to Python's logging.StreamHandler (which appears to be the default in Airflow < 1.10.0):
'handlers': {
'console': {
'class': 'logging.StreamHandler',
'formatter': 'airflow',
'stream': 'ext://sys.stdout'
},
'loggers': {
'airflow.processor': {
'handlers': ['console'],
'level': LOG_LEVEL,
'propagate': False,
},
'airflow.task': {
'handlers': ['console'],
'level': LOG_LEVEL,
'propagate': False,
},
'flask_appbuilder': {
'handler': ['console'],
'level': FAB_LOG_LEVEL,
'propagate': True,
}
},
'root': {
'handlers': ['console'],
'level': LOG_LEVEL,
}