Securing a university master database while modernising reporting
Moved a South-African university’s reporting stack off QlikView querying a live Oracle master DB onto Power BI fed by purpose-built data marts and an orchestrated ETL pipeline.
The BI team queried the Oracle master database directly for QlikView reports. That model was insecure, slow on raw data, prone to integrity issues, and limited by QlikView’s poor integration with other tools.
Database security
Dashboard load time
Operational cost
Added staging and reporting data marts in MySQL, an orchestrated ETL pipeline that cleans and shapes data into reporting views, and Power BI dashboards sourced from those views. This decoupled reporting from the master DB.
Staging DB pulls raw data from the Oracle master; reporting DB holds refined, report-ready views.
SQLMesh + Dagster ETL performs quality checks, handles missing data and builds views.
Power BI dashboards consume the reporting views with rich filters and sorting.
Modular stages make the system easy for BI engineers to monitor and troubleshoot.
Source of truth
MySQL raw extract
SQLMesh + Dagster
Refined views
Dynamic dashboards
Secured the master database from unnecessary access and excessive read/write load.
Standardised and orchestrated ETL with minimal human intervention.
Gave stakeholders fast, reliable dashboards on cleaned, governed data.
Fix your AWS bill in-house, buy an automated cost tool, or hire someone? A straight comparison of what each option reaches, and when each is wasted money.
Dagster vs Airflow vs Prefect for ETL in 2026, compared on the one thing they disagree on: what a pipeline is made of. Notes from a 50M-records/day build.
Whether that is ongoing technical ownership, one independent read on a problem, or someone to run the project. Let's scope it — I reply within one business day.