Airflow not picking up "start_date" from DAG - airflow

Question

I'm registering a new DAG like so:
from airflow import DAG
from airflow.operators.python_operator import PythonOperator
from airflow.operators.bash_operator import BashOperator
from airflow.utils.dates import days_ago
from airflow.hooks.base_hook import BaseHook
from datetime import datetime, timedelta, timezone
import pendulum
local_tz = pendulum.timezone("UTC")
default_args = {
'owner': 'me',
'depends_on_past': False,
'start_date': datetime(2020, 6, 19, 9, 37, 35, tzinfo=local_tz),
'email': ["blah#blah.com"],
'email_on_failure': False,
'email_on_retry': False,
'retries': 1,
'retry_delay': timedelta(minutes=15)
}
dag = DAG(
dag_id="some_id",
default_args=default_args,
description= "Some description",
schedule_interval="#once"
)
def to_be_executed_py():
print("I did it, ma!")
with dag:
t1 = PythonOperator(
task_id="some_id",
python_callable=to_be_executed_py)
I want this to run once and only once at the time given in start_date. After uploading the DAG (using S3), I'm not seeing the "start_date" in details. Instead, I see in details (under default_args):
{'owner': 'me',
'depends_on_past': False,
'start_date': datetime.datetime(2020, 6, 19, 9, 37, 35, tzinfo=<TimezoneInfo [UTC, GMT, +00:00:00, STD]>),
'email': ['bleh#blah.com'],
'email_on_failure': False,
'email_on_retry': False,
'retries': 1,
'retry_delay': datetime.timedelta(0, 900)}
Am I doing something wrong here? Am I making the right assumption that this should execute at the given start_time? I've looked all over for similar use cases, but not many are setting their start_date to include a time.
UPDATE
Currently, the DAG is running immediately upon being unpaused. Definitely not picking up the start time. All the resources I've found online don't have an answer that works here.

Answer 1

Figured out the issue. It was twofold. One, a translator we were using was a 12 hour clock. As this was in the evening, it was setting it to be in the past (causing Airflow to play catchup).
Secondly, we didn't need the timezone. Plus, we weren't setting the dag in the task. So the code should read as the following:
from airflow import DAG
from airflow.operators.python_operator import PythonOperator
from airflow.operators.bash_operator import BashOperator
from airflow.utils.dates import days_ago
from airflow.hooks.base_hook import BaseHook
from datetime import datetime, timedelta, timezone
default_args = {
'owner': 'me',
'depends_on_past': False,
'start_date': datetime(2020, 6, 19, 21, 37, 35),
'email': ["blah#blah.com"],
'email_on_failure': False,
'email_on_retry': False,
'retries': 1,
'retry_delay': timedelta(minutes=15)
}
dag = DAG(
dag_id="some_id",
default_args=default_args,
description= "Some description",
schedule_interval="#once"
)
def to_be_executed_py(ds, **kwargs):
print("I did it, ma!")
with dag:
t1 = PythonOperator(
dag=dag,
provide_context=True,
task_id="some_id",
python_callable=to_be_executed_py)
With those changes, everything runs at the given time, once and only once.

Answer 2

With a combination of schedule_interval='#daily', ShortCircuitOperator and airflow variables you could do a workaround; DAG runs every day and checks if today is in the list of the days that you entered as an airflow variable. If so, proceed and runs downstream tasks and if not, it skips the downstream tasks and waits until subsequent execution on tomorrow.
here is the DAG definition:
import airflow.utils.helpers
from airflow.models import DAG, Variable
from airflow.operators.dummy_operator import DummyOperator
from airflow.operators.python_operator import ShortCircuitOperator
args = {
'owner': 'airflow',
'start_date': airflow.utils.dates.days_ago(2)
}
dag = DAG(
dag_id='run_on_release_day',
default_args=args,
schedule_interval='#daily'
)
def check_release_date(**context):
release_dates = Variable.get('release_dates')
print(context, release_dates)
return context['ds'] in release_dates
cond = ShortCircuitOperator(
task_id='condition',
python_callable=check_release_date,
dag=dag,
provide_context=True,
)
tasks = [DummyOperator(task_id='task_' + str(i), dag=dag) for i in [1, 2]]
airflow.utils.helpers.chain(cond, *tasks)