Task Scheduling In Airflow Is Not Working

I am scheduling dag in airflow for 10 minutes is not doing anything. here is my dags code: from airflow import DAG from airflow.operators.bash_operator import BashOperator from da

Solution 1:

Airflow is an ETL/data pipelining tool. This means its meant to execute things over already "gone by" periods. E.g. using:

• Task parameter 'start_date': datetime(2018,1,4)
• Default dag parameter schedule_interval='@daily'

Means that the DAG won't run until the whole schedule interval unit (one day) has gone through since the start date; thus on Airflow server time equal to datetime(2018,1,5).

Since you have a start_date of datetime.now() with a @daily inteval (which again is the default), the aforementioned condition is never fulfilled (refer to the official FAQ).

You can change the start_date parameter to, e.g. yesterday using timedelta for a relative start_date earlier than today (although this is not recommended). I would advise using 'start_date': datetime(2018,1,1) and adding a scheduler_interval='@once' to the DAG parameters for test purposes. This should get your DAG to run.