Deep dives on building production data pipelines, cloud warehouses and analytics systems: the decisions, trade-offs and numbers behind real engineering work.
Mask PII in an S3 data lake at the ingest boundary: deterministic hashing that keeps joins working, Glue detection, Lake Formation filters, KMS and erasure.
Why LLM features fail in production when the demo worked: messy real inputs, queues that starve each other, and failures that never raise an alarm.
Fix your AWS bill in-house, buy an automated cost tool, or hire someone? A straight comparison of what each option reaches, and when each is wasted money.
AWS data engineer contract rates in the UK: the median is £513 a day. What sits behind it, what moves you up the range, and what outside IR35 really changes.
Hit AWS Lambda's 15-minute timeout? Durable functions did not raise it. Which of the three fixes you need depends on whether your job computes or waits.
AWS Lambda deployment limits explained: layers don't raise the 250 MB cap. What actually counts, what to delete first, and when a container image wins instead.
How to manage multiple projects as a developer: WIP limits, project-day blocking, landing notes, and pointing AI at re-entry cost instead of writing more code.
How to schedule a Python script to run automatically: cron on Mac and Linux, Task Scheduler on Windows, the schedule library, and the gotchas that break them.
New to data engineering? Build your first ETL pipeline in Python: pull data from an API, clean it with pandas, and load it into a database, the right way.
How to scale a data pipeline with RabbitMQ: work queues, a worker pool, durability and backpressure. The pattern that took one pipeline past 2M records a day.
AWS DMS copies your table data, not your stored procedures, indexes, triggers or validation. A practical guide to what DMS leaves you to do yourself.
The differences between the AWS Console, CLI and SDKs: how control plane requests are authenticated, and which interface belongs in automation vs manual ops.
Learn how AWS Regions, Availability Zones and service scope affect latency, cost, compliance and resiliency. Choose the right AWS location with confidence.
A concise guide to the AWS Shared Responsibility Model what AWS secures, what you must secure, and practical controls for cloud teams to reduce risk.
AWS Athena query optimization comes down to one thing: scanning less data. How partitioning, Parquet, bucketing and projection cut what you scan, and the bill.
A practical guide to right-sizing DPUs, using Glue Flex and scanning less S3 data, to cut your Glue bill by 50% without touching business logic.
A cloud data warehouse migration guide: how to choose between Snowflake, Redshift, BigQuery and Databricks on cost and lock-in, and how to de-risk the move.
A practical AI cost optimization guide from a real build: how task-based model routing cut one platform's AI spend by 99%, and trims most LLM bills 30 to 50%.
Dagster vs Airflow vs Prefect for ETL in 2026, compared on the one thing they disagree on: what a pipeline is made of. Notes from a 50M-records/day build.
Occasional write-ups from real AWS data work: what the bill actually did, what broke, and how long it took. No roundups, no thought leadership. Unsubscribe whenever.
From messy source data to analytics-ready warehouses that cut cost. Let's scope it. I reply within one business day.