General
292 views
Data Pipeline
Quick Definition
Series of processing steps moving data from source to destination
Full Definition
A series of data processing steps that move data from source to destination.
Examples
ETL, Apache Airflow, Kafka Streams, data transformation
Related Terms
etl
data-engineering
analytics
More General Terms
Trunk Based Development
Branching model integrating code to main branch frequently
CI/CD
Automating build test and deployment pipelines
Event Sourcing
Storing all state changes as immutable event sequences
Scalability
Ability to handle increased load by adding resources
SOLID
Five design principles for maintainable OOP software
Clean Code
Writing readable maintainable code