How to Mask PII on Ingest Into an S3 Data Lake
Mask PII in an S3 data lake at the ingest boundary: deterministic hashing that keeps joins working, Glue detection, Lake Formation filters, KMS and erasure.
Re-platforming a tier-1 banking EDW onto a cloud-native AWS warehouse
Led the migration of a major retail bank’s on-premises Enterprise Data Warehouse from Sybase to Amazon Redshift on AWS, re-engineering 800+ stored procedures and automating data-quality testing to lift processing throughput by 40%.
The bank’s on-premises Enterprise Data Warehouse (EDW) could no longer keep up. Hard ceilings on scalability and performance slowed reporting, ran up maintenance cost, and left the business behind on growing data volumes and regulatory demand.
Data-processing throughput
Operational efficiency
Data-validation accuracy
A phased, zero-data-loss migration from Sybase to Amazon Redshift on AWS. It re-architected 800+ stored procedures, decoupled compute from storage, and wrapped the pipeline in automated data-quality tests so correctness was checked against source, not assumed.
Re-engineered 800+ Sybase/PL-SQL stored procedures into Redshift-optimised SQL with tuned distribution and sort keys.
Staged raw extracts through AWS S3, then bulk-loaded into Redshift for elastic, pay-as-you-grow compute.
Automated data-testing harness validated every table against source, locking in 95% validation accuracy.
Decoupled storage and compute to absorb peak reporting loads without over-provisioning hardware.
Legacy on-prem source
Python + SQL staging
800+ procs → Redshift SQL
Redshift COPY
95% validation accuracy
Gave the bank elastic cloud scale for its analytics workload.
Turned an 800-procedure migration into a repeatable, test-driven process.
Cut manual validation effort while raising confidence in regulatory reporting.
Mask PII in an S3 data lake at the ingest boundary: deterministic hashing that keeps joins working, Glue detection, Lake Formation filters, KMS and erasure.
AWS DMS copies your table data, not your stored procedures, indexes, triggers or validation. A practical guide to what DMS leaves you to do yourself.
AWS Athena query optimization comes down to one thing: scanning less data. How partitioning, Parquet, bucketing and projection cut what you scan, and the bill.
Whether that is ongoing technical ownership, one independent read on a problem, or someone to run the project. Let's scope it — I reply within one business day.