---
title: "Airflow | Skills We Assign For | Azendo"
description: "Airflow for pipeline orchestration — idempotency, backfills and failure handling, and the Azendo roles assigned for data work."
url: "https://azendo.co/skills/airflow/"
---

[Skills](https://azendo.co/skills/) Data pipelines 

# Airflow.

Apache Airflow schedules and orchestrates data workflows defined as code, tracking dependencies between tasks and making failures visible and re-runnable. It is the scheduler for a data platform rather than a tool for moving data itself.

## Where Airflow fits on a long engagement.

The reason to orchestrate rather than schedule is dependency and recovery. When a job fails at three in the morning, what matters is whether the failure is visible, whether downstream tasks were stopped, and whether the run can be repeated safely. Cron answers none of those questions.

Tasks that are not idempotent are where Airflow estates break. If re-running a task double-counts rows, the platform cannot recover automatically and every failure becomes a manual investigation. Designing for re-run is the discipline that makes the rest of it work.

## What an assigned team does with Airflow.

Orchestration is where a data platform is either operable or not. The measure is what happens at three in the morning when a task fails: whether anyone finds out, whether downstream work stopped, and whether the fix is a re-run or an investigation.

Getting that right is design work done once and maintained continuously. Where the pipelines exist to feed models rather than reports, the same capacity is usually assigned alongside [MLOps engineering](https://azendo.co/services/mlops-engineering/) on one agreement.

## What we use Airflow for.

* Pipelines that recover themselves Idempotent tasks and retries, so an overnight failure does not become a morning of manual repair.
* Backfilling after a change Reprocessing history deliberately when a transformation changes, rather than leaving inconsistent data behind.
* Replacing scattered cron jobs Scheduled scripts across several machines consolidated into one place with visible dependencies.

## How Airflow capacity is assigned.

Orchestration work is assigned under [managed data services](https://azendo.co/services/data-engineering/), and the monitoring around it matters as much as the pipelines themselves.

## Roles we assign Airflow for

* [Data Engineer Data engineering](https://azendo.co/services/data-engineering/data-engineer/)
* [ETL Developer Data engineering](https://azendo.co/services/data-engineering/etl-developer/)
* [MLOps Engineer MLOps engineering](https://azendo.co/services/mlops-engineering/mlops-engineer/)

## Service lines it sits in

* [Data engineering](https://azendo.co/services/data-engineering/)
* [MLOps engineering](https://azendo.co/services/mlops-engineering/)

Capacity is agreed as a committed monthly capacity across a discipline, not per skill.

## Related in data pipelines

* [dbt — skill we assign for](https://azendo.co/skills/dbt/)
* [Fivetran — skill we assign for](https://azendo.co/skills/fivetran/)
* [Databricks — skill we assign for](https://azendo.co/skills/databricks/)
* [Spark — skill we assign for](https://azendo.co/skills/spark/)
* [Flink — skill we assign for](https://azendo.co/skills/flink/)
* [Dagster — skill we assign for](https://azendo.co/skills/dagster/)
* [Prefect — skill we assign for](https://azendo.co/skills/prefect/)
* [Airbyte — skill we assign for](https://azendo.co/skills/airbyte/)

## Tell us what your roadmap needs Airflow for.

A service delivery manager replies with the disciplines we would assign, the monthly capacity and what the first month looks like.

[All skills we assign for](https://azendo.co/skills/)