---
title: "incremental loads | Skills We Assign For | Azendo"
description: "Incremental loading patterns — watermarks, late data and correctness, plus the Azendo roles assigned for it."
url: "https://azendo.co/skills/incremental-loads/"
---

[Skills](https://azendo.co/skills/) Data pipelines 

# incremental loads.

An incremental load moves only data that has changed since the last run, rather than reprocessing everything. It is what keeps pipeline cost and runtime proportionate to change rather than to total volume.

## Where incremental loads fits on a long engagement.

Full reloads are simple and correct, which is why they survive longer than they should. They stop being viable when the runtime exceeds the window available, and the move to incremental is usually made under pressure — which is exactly when the correctness details get skipped.

Those details are where incremental loads go wrong. A watermark on an updated-at column misses rows updated during the run; late-arriving data lands before the watermark and is never picked up; and a failed run leaves the watermark in a state that silently skips a range. Each produces missing data rather than an error, which is the worst failure mode available.

## What an assigned team does with incremental loads.

Because these failures are silent, reconciliation is not optional. A periodic full comparison against the source is what turns "we think the pipeline is correct" into evidence, and it is routinely absent.

Building that in from the start is part of what assigned data capacity is responsible for, agreed as part of [how the monthly fee is built](https://azendo.co/pricing/).

## What we use incremental loads for.

* Runtime proportional to change Loads that move what is new, so cost does not grow with total history.
* Late-arriving data handled Overlap windows and reprocessing, so rows behind the watermark are not lost.
* Reconciliation that proves correctness Periodic comparison against source, because silent gaps produce no error.

## How incremental loads capacity is assigned.

Pipeline design is assigned under [managed data services](https://azendo.co/services/data-engineering/), with reconciliation built in rather than added after a discrepancy is found.

## Roles we assign incremental loads for

* [ETL Developer Data engineering](https://azendo.co/services/data-engineering/etl-developer/)

## Service lines it sits in

* [Data engineering](https://azendo.co/services/data-engineering/)

Capacity is agreed as a committed monthly capacity across a discipline, not per skill.

## Related in data pipelines

* [Spark — skill we assign for](https://azendo.co/skills/spark/)
* [Flink — skill we assign for](https://azendo.co/skills/flink/)
* [Dagster — skill we assign for](https://azendo.co/skills/dagster/)
* [Prefect — skill we assign for](https://azendo.co/skills/prefect/)
* [Airbyte — skill we assign for](https://azendo.co/skills/airbyte/)
* [Informatica — skill we assign for](https://azendo.co/skills/informatica/)
* [Talend — skill we assign for](https://azendo.co/skills/talend/)
* [SSIS — skill we assign for](https://azendo.co/skills/ssis/)

## Tell us what your roadmap needs incremental loads for.

A service delivery manager replies with the disciplines we would assign, the monthly capacity and what the first month looks like.

[All skills we assign for](https://azendo.co/skills/)