Data Jargon 101: The Vocabulary of Modern Data Architecture

Before exploring the major data architectures, it’s essential to speak the same language. Modern data systems are filled with jargon — and understanding these terms precisely will make the difference between choosing the right architecture and getting lost in the acronym soup.

Schema & Storage Models

Schema-on-Read vs Schema-on-Write

Schema-on-Read means you store raw, unstructured data first and define its shape only when you query it. This maximises flexibility — you can ingest data without knowing how you’ll use it. The trade-off is that queries may be slower and errors surface late.

Schema-on-Write enforces a defined schema at ingestion. Data that doesn’t conform is rejected. This produces clean, reliable data but requires upfront modelling and makes schema evolution harder.

Storage Orientations

Row-Oriented Storage stores records row by row. Every column of a record lives adjacent on disk. This is ideal for OLTP workloads (inserts, updates, point reads) but inefficient for analytics that scan only a few columns across millions of rows.

Row store: [id=1, name="Alice", age=30] [id=2, name="Bob", age=25] ...

Columnar Storage stores each column’s values together. Analytical queries that touch only 3 of 50 columns read 3/50th of the data. Compression is far more effective because similar values (e.g. all ages) live together.

Column store: [id: 1,2,3,...] [name: "Alice","Bob",...] [age: 30,25,...]

In-Memory Storage holds data in RAM rather than disk, achieving ultra-low latency for analytics. Used in systems like Redis, Apache Ignite, and SAP HANA.

Processing Patterns

ETL vs ELT

ETL (Extract, Transform, Load) was the standard approach for decades. Data is cleansed and shaped in a staging area before being loaded into the warehouse. It works well when the destination has limited compute, but creates bottlenecks and inflexibility.

ELT (Extract, Load, Transform) flips the order — raw data lands first in a powerful cloud data store, then is transformed using that store’s compute. This is the modern lakehouse pattern, enabling iterative, code-driven transformations with full raw data retained for reprocessing.

Batch vs Streaming

Batch Processing operates on bounded datasets at scheduled intervals (hourly, daily). It’s simple to implement, easy to debug, and efficient for large volumes. Tools: Apache Spark, Hadoop MapReduce, Azure Data Factory.

Streaming Processing operates on unbounded, continuously arriving data with low latency (seconds or sub-second). It’s essential for real-time dashboards, fraud detection, and event-driven architectures. Tools: Apache Kafka, Apache Flink, Azure Event Hubs + Stream Analytics.

Most modern architectures support both — a Lambda Architecture runs separate batch and streaming paths, while a Kappa Architecture handles everything through a unified streaming pipeline.

Access & Governance Concepts

OLTP vs OLAP

OLTP systems (PostgreSQL, MySQL, SQL Server) power your applications — order entries, user logins, inventory updates. They need fast, consistent, transactional writes.

OLAP systems (Snowflake, BigQuery, Redshift, Synapse) power your dashboards and reports. They need fast reads across billions of rows, typically with columnar storage and massively parallel execution.

Data Virtualisation

Data Virtualisation provides a unified, virtual view of data without physically moving or copying it. Queries are pushed down to source systems and results are assembled at query time. Tools: Denodo, Dremio, Trino (PrestoSQL). Useful for federated queries across heterogeneous sources.

Metadata & Data Catalog

A Data Catalog is a searchable inventory of all your data assets — tables, schemas, pipelines, dashboards, ML models — enriched with:

• Technical metadata: column types, row counts, partitioning
• Business metadata: descriptions, owners, business terms
• Data lineage: origin to consumption path
• Data quality scores: completeness, freshness, accuracy

Tools: Apache Atlas, Amundsen, DataHub, Microsoft Purview, Alation.

Transaction & Consistency

ACID Transactions

ACID is the gold standard for reliable data mutations:

Traditional relational databases are fully ACID. Modern table formats (Delta Lake, Iceberg, Hudi) bring ACID transactions to data lakes at scale.

Idempotent Processing

An idempotent operation produces the same result regardless of how many times it is executed. This is critical for exactly-once semantics in pipelines: if a job fails and restarts, re-running it won’t corrupt data or produce duplicates. Achieved via upserts, unique keys, or write-ahead logs.

Advanced Concepts

Data Lineage

Data lineage tracks the complete lifecycle of data — from its origin (source system, file, API) through every transformation and join, to its final consumption (report, ML model, dashboard). It enables:

• Auditability: “Where did this number come from?”
• Impact analysis: “If I change this table, what breaks downstream?”
• Regulatory compliance: GDPR, HIPAA data traceability

Data Contract

A Data Contract is a formal agreement between data producers (engineering teams) and consumers (analysts, scientists) defining:

• Schema: field names, types, required fields
• SLA: freshness guarantees (e.g. “updated within 1 hour of source event”)
• Quality rules: null rates, value ranges
• Versioning: how breaking changes will be communicated

Medallion Architecture

The Medallion (or Multi-hop) Architecture organises data into progressive quality tiers:

Bronze  →  Silver  →  Gold
(Raw)      (Cleaned)   (Curated/Aggregated)

• Bronze: Raw, immutable ingested data — exactly as it arrived
• Silver: Cleaned, validated, deduped — ready for analysis
• Gold: Aggregated, business-ready datasets — powering dashboards and ML

Polyglot Persistence

Polyglot Persistence means using multiple storage technologies within a single system, each chosen for its optimal fit with a specific workload:

Quick Reference Glossary