What determines the layer a task is in, in an airflow DAG? - airflow

Question

If we look at an example DAG in Airflow we see (Graph View):
What determines the positions of tasks also_run_this and this_will_skip? Notice these 2 tasks don't have any connecting lines prior to themselves, which means they could be placed on the same layer (first vertical set of tasks) as runme_0, runme_1 and runme_2 (using my obviously incorrect assumptions about the DAG).
Is it their runtime that places them in the same layer as run_after_loop?
I am looking at the detailed tasks data for this DAG and I don't see anything that distinguishes also_run_this and this_will_skip from runme_0 in terms of position:
Here is runme_0:
{
"class_ref": {
"class_name": "BashOperator",
"module_path": "airflow.operators.bash"
},
"depends_on_past": false,
"downstream_task_ids": ["run_after_loop"],
"end_date": null,
"execution_timeout": null,
"extra_links": [],
"owner": "airflow",
"pool": "default_pool",
"pool_slots": 1,
"priority_weight": 1,
"queue": "default",
"retries": 0,
"retry_delay": {
"__type": "TimeDelta",
"days": 0,
"microseconds": 0,
"seconds": 300
},
"retry_exponential_backoff": false,
"start_date": "2021-06-17T00:00:00+00:00",
"task_id": "runme_0",
"template_fields": ["bash_command", "env"],
"trigger_rule": "all_success",
"ui_color": "#f0ede4",
"ui_fgcolor": "#000",
"wait_for_downstream": false,
"weight_rule": "downstream"
}
And here is also_run_this:
{
"class_ref": {
"class_name": "BashOperator",
"module_path": "airflow.operators.bash"
},
"depends_on_past": false,
"downstream_task_ids": ["run_this_last"],
"end_date": null,
"execution_timeout": null,
"extra_links": [],
"owner": "airflow",
"pool": "default_pool",
"pool_slots": 1,
"priority_weight": 1,
"queue": "default",
"retries": 0,
"retry_delay": {
"__type": "TimeDelta",
"days": 0,
"microseconds": 0,
"seconds": 300
},
"retry_exponential_backoff": false,
"start_date": "2021-06-17T00:00:00+00:00",
"task_id": "also_run_this",
"template_fields": ["bash_command", "env"],
"trigger_rule": "all_success",
"ui_color": "#f0ede4",
"ui_fgcolor": "#000",
"wait_for_downstream": false,
"weight_rule": "downstream"
}
It would make sense if the same layer was based on parallelism (all tasks in the same vertical layer run in parallel) but this would require some thresholding of the run times, and I don't see any such data available in the DAG or TASK information.
In fact, looking at the Tree View, it appears to show runme_0, runme_1, runme_2, also_run_this and this_will_skip all running at the same time:
As per #bruno-uy 's comment, it appears the Graph View has a UI "problem." Definitely not very intuitive.

Answer 1

After checking that runme_0, runme_1, runme_2, also_run_this and this_will_skip all running at the same time, we can say that this is a UI "problem" that they're not shown in the same "layer". Airflow doesn't have the "layer" concept, so basically they don't assure the tasks starting at the same time are aligned vertically.
Could be a good improvement for Airflow, or just add another diagram as you mentioned Sankey.