What is an ETL pipeline?

An ETL pipeline extracts data from a source, transforms it into a clean, structured shape, then loads it into a destination like a warehouse or lakehouse. The three steps run in that order: transform happens before the data ever lands, which is the opposite of the newer ELT pattern.

The three steps

• Extract: Pull data out of a source system, a database, an API, or a file.
• Transform: Clean, reshape, validate, and aggregate that data outside the destination, often on a separate processing server.
• Load: Write the finished, transformed data into the destination, ready to query.

Why teams built pipelines this way

ETL grew up in an era when warehouse storage and compute were both expensive. Transforming data before loading kept the warehouse lean: it only ever held clean, ready-to-query data, never the messy raw version. The tradeoff is that the raw data usually isn't kept anywhere, so reprocessing a mistake means going back to the source.

How OptimaFlo differs

OptimaFlo runs ELT, not ETL. Raw data lands in the Raw layer exactly as the source sent it, zero transforms, before anything gets reshaped. The Analytics Engineer role then writes SQL that transforms that already-landed data into the Clean layer, and every change is user-approved and version controlled before it runs. See ETL vs ELT for the full comparison.

Related terms

• Data pipeline: The broader term for any automated system that moves and processes data, of which ETL is one pattern.
• ETL vs ELT: A direct comparison of the two orderings, transform-then-load versus load-then-transform.
• Data transformation: The T step in ETL, reshaping data into a usable form.
• Medallion architecture: The Raw, Clean, and Ready layers OptimaFlo's ELT pipelines build instead.