PrepPilotPractise free

Orchestration (Airflow) interview questions

DAGs, scheduling, retries, sensors, best practices

16 questions · DAG · updated November 2026

Practise these as flashcards. One or two lines each; the ones you miss come back first.

Start free

Core questions

What is a DAG in Airflow and how do you define task dependencies?

A DAG is a Directed Acyclic Graph that represents the workflow structure in Airflow. You define tasks as nodes and set dependencies using >> or << operators, or methods like set_upstream. This ensures tasks run in the correct order.

What is the purpose of the schedule_interval parameter in a DAG?

schedule_interval sets how often the DAG should run, using cron syntax or a timedelta. It determines when Airflow creates new DAG runs and what data interval each run covers.

How can you make a task retry on failure in Airflow?

You can set the retries parameter on the task to specify how many times it should retry, and retry_delay to set the wait time between attempts. This helps handle transient errors automatically.

Why should you avoid passing large data via XComs?

XComs are stored in Airflow's metadata database and are intended for small pieces of data. Passing large data can slow down the system and bloat the database, so it's better to use external storage for big data.

Intermediate questions

Explain the difference between catchup and backfill in Airflow.

Catchup is a DAG-level setting that tells Airflow whether to run all missed intervals since the DAG's start date when it's enabled. Backfill refers to the actual process of running those historical DAG runs, either automatically (if catchup is True) or manually triggered.

How do you implement alerting for task failures or SLA misses in Airflow?

You can configure email or Slack alerts by setting email, email_on_failure, or on_failure_callback in the DAG or task. For SLAs, you can set the sla parameter and configure sla_miss_callback to trigger alerts when a task exceeds its expected duration.

What are sensors in Airflow and what problem do deferrable operators solve?

Sensors are special tasks that wait for an external event, like a file or a database record. Traditional sensors block a worker slot while waiting, but deferrable operators (introduced in Airflow 2.2) use asynchronous triggers, freeing up resources and improving scalability.

How would you pass data from one task to another in Airflow?

I use XComs to pass small pieces of data, like IDs or status flags, between tasks. For larger data, I write it to external storage (like S3 or a database) in one task and read it in the next, passing only references via XCom.

How do you ensure a task is idempotent in Airflow?

I design tasks so that running them multiple times with the same inputs doesn't cause duplicate or inconsistent results. For example, I use partitioned outputs, upserts, or overwrite operations to make tasks safe to retry.

What are some best practices for structuring Airflow DAGs?

I use clear and consistent task naming, modularize reusable logic into Python functions or custom operators, avoid deeply nested dependencies, and document the DAG's purpose and schedule. This makes DAGs easier to maintain and debug.

Describe how Airflow handles missed DAG runs when a DAG is paused for several days and then resumed.

If catchup=True, Airflow will schedule DAG runs for all missed intervals since the last run. If catchup=False, it will only schedule the latest interval, skipping the missed ones. This behavior prevents accidental overload from backfilling unless explicitly desired.

Senior questions

How would you debug a DAG that is stuck in a running state due to a sensor task?

I would check the sensor's configuration, especially the poke_interval and timeout settings, and see if it's using the traditional or deferrable mode. If it's blocking a worker slot, I might switch to a deferrable sensor or adjust the mode to 'reschedule'. I'd also check logs and the external system the sensor is waiting on.

What trade-offs would you consider when choosing between Airflow, Dagster, and Prefect for a new project?

See the answer in the app, free

How would you design a DAG to handle late-arriving data without causing duplicate processing?

See the answer in the app, free

How do you handle dependency management and versioning of DAG code in a production environment?

See the answer in the app, free

What are the operational risks of enabling catchup on a high-frequency DAG, and how would you mitigate them?

See the answer in the app, free

More Data Engineering topics