When the platform can't keep up with the mission

The Situation

A platform that had outgrown its original design.

The charity managed membership services for tens of thousands of civil servants. The systems holding that data together had grown organically over time — layered, manual, and increasingly fragile. Nobody could be certain where errors originated. Data arrived through manual file uploads and email processes with no automated validation, no deduplication, and no audit trail.

The consequences were real: duplicate member records, unresolved data integrity issues, slow processing, no visibility into partner data, and a dependency on external support for tasks that the platform wasn't yet equipped to handle automatically. The team was spending significant time managing a system rather than serving members.

• Manual file uploads and email-based data imports with no automation
• No deduplication — duplicate records accumulating with no reliable way to resolve them
• No partner integration — external organisations couldn't connect directly
• Limited error visibility — issues were difficult to trace back to their source
• Reliance on external support for day-to-day operational tasks
• Legacy SSIS workloads adding infrastructure overhead and cost

What We Built

A five-layer ingestion architecture designed for reliability, security, and scale.

We designed and built a bespoke end-to-end data ingestion system on the m-hance NFP platform — five layers, each with a defined role, each auditable independently. Power Automate handled orchestration and trigger logic. Azure Service Bus provided reliable message queuing with guaranteed delivery. Azure Function Apps, built on the Dynamics SDK, handled the processing logic with full error capture. SQL Server acted as a staging layer for validation and transformation. Dataverse CRM received clean, validated, deduplicated data.

Custom deduplication logic was built directly into the Power Automate pipeline — handling high-volume data flows and preventing integrity issues before they reached the CRM layer. Webhook authentication and security controls were implemented, identifying and remediating vulnerabilities in the existing production system that hadn't previously been visible.

All data imports were automated via APIs, eliminating manual file handling entirely. SSIS workloads were migrated to Azure Functions, removing legacy infrastructure and cutting running costs. A model-driven app was built for trustee use — covering verification workflows, consent logic, and document automation via DocsCorePack.

Deployment and release processes were established using Power Platform native tooling — giving the internal team the ability to manage and release changes themselves as needed, without relying on external support or specialist pipeline knowledge to keep the platform moving.

The Outcome

A platform the team owns. Not one they depend on others to run.

Every manual data import was eliminated. The platform now handles ingestion, deduplication, validation, and delivery automatically — end to end, without human intervention in the data flow. The error rate dropped because errors are now caught at the layer they occur, logged, and surfaced rather than silently corrupting downstream data.

Infrastructure costs fell by 25% through the migration of legacy workloads to Azure Functions. The trustee app gave the governance layer the tools it needed to operate independently. A security review of the production environment identified and resolved areas of exposure.

Over more than two years as the sole technical authority on this system, full documentation and knowledge transfer was maintained throughout — not as an afterthought at project close, but as a continuous discipline. The organisation understood what they had, why it was built the way it was, and how to run it.