Skip to content
Writing

Data Engineering Blog

Deep dives on building production data pipelines, cloud warehouses and analytics systems: the decisions, trade-offs and numbers behind real engineering work.

Data Engineering

AWS DMS Limitations: What It Won't Migrate for You

AWS DMS copies your table data, not your stored procedures, indexes, triggers or validation. A practical guide to what Database Migration Service leaves you to do.

AWS DMSDatabase MigrationAWSData Engineering
Data Engineering

Interacting with AWS: Console, CLI, SDKs & API Access

Learn the differences between the AWS Console, CLI and SDKs, how AWS control plane requests are authenticated, and which interface belongs in automation vs manual ops.

AWSControl PlaneCLISDK
Data Engineering

AWS Global Infrastructure: Regions & Availability Zones Explained

Learn how AWS Regions, Availability Zones and service scope affect latency, cost, compliance and resiliency. Choose the right AWS location with confidence.

AWS InfrastructureRegionsAvailability ZonesCloud Architecture
Security

AWS Shared Responsibility Model: The Practical Blueprint

A concise guide to the AWS Shared Responsibility Model what AWS secures, what you must secure, and practical controls for cloud teams to reduce risk.

AWSSecurityShared ResponsibilityCompliance
Data Engineering

AWS Athena Query Optimization: Scan Less, Pay Less

AWS Athena query optimization comes down to one thing: scanning less data. How partitioning, Parquet, bucketing and projection cut what you scan, and the bill.

Amazon AthenaQuery OptimizationCost OptimizationPartitioning
Data Engineering

AWS Glue Cost Optimization: Stop Overpaying for Your Batch ETL

A practical guide to right-sizing DPUs, using Glue Flex and scanning less S3 data, to cut your Glue bill by 50% without touching business logic.

AWS GlueCost OptimizationData EngineeringETL
Data Engineering

Cloud Data Warehouse Migration: Snowflake vs Redshift vs BigQuery

A cloud data warehouse migration guide to Snowflake vs Redshift vs BigQuery vs Databricks: how to choose on cost, lock-in and performance, and how to de-risk the move.

Data WarehouseSnowflakeRedshiftBigQuery
AI Engineering

AI Cost Optimization: How We Cut a Document AI Bill by 99%

A practical AI cost optimization guide built on a real case study: how task-based model routing cut one document platform's AI spend by 99% and trims most LLM bills 30 to 50%.

AI CostsLLMCost OptimizationAI Strategy
Data Engineering

Dagster vs Airflow vs Prefect for ETL in 2026

Dagster vs Airflow vs Prefect for ETL in 2026, compared by the one question they disagree on: what a pipeline is actually made of. Field notes from a 50M-records/day build.

DagsterAirflowPrefectETL
Taking on new projects · Outside IR35

Have a data pipeline or warehouse problem worth solving?

From messy source data to analytics-ready warehouses that cut cost. Let's scope it. I reply within one business day.