Skip to content

← All work

CS-04 · 2026

Migrating 909 reporting connections to a governed warehouse

Hundreds of live dashboards were reading files through a fragile path. I led the move of every one onto governed warehouse views, then proved the cutover was complete.

909
Reporting data sources migrated
Measured, from the project runbook
28
Data-prep flows migrated
Measured
0
Left to migrate at verified cutover
Data sources and flows
From fragile files to a verified cutover
  1. Scope the estate909 data sources and 28 data-prep flows
  2. Wrap source tablesGoverned views in the warehouse
  3. Repoint every sourceAway from the document-store path
  4. Verify the cutoverZero left
  5. Weekly automationKeeps the estate on the governed path

The order matters. Governed views come first so every source has somewhere safe to point, and the cutover is only declared once a check confirms it.

The problem

Hundreds of reporting data sources were reading their data from files held in a document store, rather than from the governed warehouse. That path was fragile and unmonitored, and every dashboard built on it inherited both weaknesses.

The goal was simple to state and hard to do safely: move every one of them onto the governed warehouse, without the business noticing anything except that reporting got more reliable.

What I did

  • Scoped the whole estate first. The work covered 909 data sources and 28 data-prep flows, so I treated it as an inventory problem before it was a migration problem.
  • Wrapped the source tables in governed views. Each source needed somewhere stable to point. The views gave every dashboard a single, versioned interface to the warehouse instead of a file path.
  • Repointed every data source to that governed warehouse interface.
  • Verified the cutover rather than declaring it. The migration was only called done when zero data sources and zero flows were left to migrate, and a data-return check confirmed the new connections returned data (7 passed, 1 raised a warning, 0 failed).
  • Stood up a weekly automation that keeps the estate on the governed path now that the cutover is complete.

My role

I led the migration end to end: scoping, the governed-view layer, the repointing, the verification and the ongoing automation.

Architecture and trade-offs

  • Views before repointing. Creating the governed views first meant each connection could be moved once, directly to its final destination, instead of being touched twice.
  • A defined end state. “Zero left to migrate” is a number you can check. Without it, a migration of this size never quite finishes, and nobody can say how much of the estate is still exposed.
  • Automation after cutover. Migrations decay. Without something running on a schedule, new sources drift back onto the old path. The weekly automation is what turns a one-off project into a standing guarantee.

Impact

  • 909 data sources and 28 data-prep flows now read through the governed warehouse interface. Measured, from the project runbook.
  • A verified cutover, with none left on the old path. Measured.
  • Reporting connections that were previously unmonitored are now covered by a weekly automated check. Measured: the automation has been running since late summer 2026.

The hardest part

Moving live reporting connections at this scale without breaking dashboards that people were using that same day. Any single repoint is easy. The difficulty is doing hundreds of them while keeping every dashboard correct, and being able to show afterwards that none were missed.

Skills shown

BI platform engineering, migration at scale, automation, data governance, verification and cutover discipline.