Enterprise engineering

How do data engineering, analytics and databases differ (SQL vs NoSQL, OLTP vs OLAP)?

  • Relational (SQL) — structured data, joins and ACID transactions: the default for business systems (PostgreSQL, MySQL, SQL Server)
  • NoSQL — document (MongoDB), key-value (Redis, DynamoDB), wide-column and graph stores; choose for a specific access pattern or scale need, not by fashion
  • OLTP — many small, concurrent transactions (the application database); OLAP — large analytical scans (a data warehouse such as BigQuery or Snowflake)
  • Data modelling — normalise for integrity, denormalise deliberately for read performance; index for real query patterns
  • ETL/ELT pipelines move data from operational systems into warehouses; keep them idempotent, observable and tested
  • Data quality & governance — ownership, lineage, access control and retention

Keep analytics workloads off the primary transactional database.

All guides