Microsoft Fabric Data Quality: Turn Purview Scores into an Owned Remediation System

Executive summary

Many enterprises can measure data quality but still cannot control it. A dashboard shows that customer identifiers are incomplete, duplicate orders are rising, or finance data is late. Yet the affected data continues into semantic models, reports, operational workflows, and AI systems because nobody has defined what must stop, what may proceed with a warning, who owns correction, or how recovery will be verified.

Microsoft Fabric and Microsoft Purview can address different parts of this problem. Fabric can enforce technical quality conditions inside the data path. Microsoft Purview Unified Catalog can organize data into governed products, apply quality rules and business-specific thresholds, alert stewards, report trends, and publish failed records into organization-managed storage. The enterprise design challenge is to connect those capabilities into one operating loop.

This article presents that loop. Its central recommendation is simple: enforce critical invariants before trusted data is published, use Purview to govern quality across products and domains, and make every exception enter an owned, timed, auditable remediation process. A quality score is evidence. It is not, by itself, a control.

The enterprise problem: quality is visible, but bad data still ships

Data quality programs often begin with profiling and dashboards. That is useful, but it leaves five questions unanswered:

1. Which defects make a data product unsafe to release?
2. Which defects are tolerable for one use case but unacceptable for another?
3. Who decides whether to repair, quarantine, waive, or reject affected records?
4. How do teams trace an issue from a source record to the business outputs it affected?
5. How do they prove that the correction worked and prevent recurrence?

Without explicit answers, a high-level score can hide consequential failures. A customer table may score 98% overall while the missing 2% contains the highest-risk accounts. A freshness average may look healthy while one regulated report is using yesterday’s balance. A uniqueness rule may identify duplicate transactions, but an alert sent to an unowned mailbox changes nothing.

The problem matters more as enterprises reuse the same Fabric data products across Power BI, operational analytics, machine learning, and generative AI. Reuse increases value, but it also increases the blast radius of a defect. The governing question is therefore not “Do we have quality checks?” It is “Can we prevent unfit data from becoming trusted, route defects to an accountable owner, and restore service within a defined objective?”

Representative enterprise scenario

The following scenario is representative, not a YuniQ customer case and not a claim of achieved results.

A multi-region services organization is building a customer operations data product in Microsoft Fabric. It combines CRM accounts, service cases, billing transactions, contact-centre events, and third-party reference data. Executives use its Power BI semantic model for service-level reporting; operations teams use it to prioritize unresolved cases; a planned AI assistant will use the same curated data to answer account questions.

The platform team has added null checks to several notebooks. The governance team has a catalog. Report developers reconcile totals before major releases. Nonetheless:

• customer identifiers fail to match across systems;
• late billing extracts distort revenue and backlog measures;
• valid codes differ between regions;
• duplicate service events inflate contact volumes;
• failed records are logged, but no team owns their correction; and
• developers sometimes bypass the curated layer during urgent analysis.

The organization does not primarily need more checks. It needs a data-quality control plane that joins business policy, technical enforcement, exception operations, and evidence.

Root causes and consequences

Rules are written at the wrong level

Technical teams often define rules column by column without linking them to a business decision. “Customer ID cannot be null” is useful. “A service case used for priority routing must resolve to exactly one active customer as of the event time” is a stronger contract because it states the context and consequence.

Microsoft Purview Unified Catalog supports governance domains, data products, critical data elements, and quality rules that can be applied across the environment. It reports scores at asset, data-product, and governance-domain levels. That hierarchy is most valuable when enterprises use it to translate business obligations into reusable controls, rather than merely cataloguing tables.

One threshold is applied to every field

A uniform threshold creates false confidence. Microsoft explicitly recommends thresholds aligned to business criticality and supports rule-level and asset-level thresholds. A descriptive field may tolerate some missing values; a financial transaction identifier may not. Purview can also alert named recipients when a rule or asset falls below its configured threshold.

Detection is separated from the data path

A scheduled scan can discover a problem after a Gold table and its semantic model have already been refreshed. Conversely, a pipeline assertion can stop a load but provide little estate-wide context. Enterprises need both: in-path enforcement for release-critical conditions and independent assurance for product health, stewardship, and trend analysis.

Exception logs are not an operating queue

A log record has no inherent owner, due date, severity, or disposition. Without those elements, the same defect recurs and teams normalize failure. Exception handling must be designed like an operational service, including triage, assignment, remediation, validation, closure, and problem management.

Quality controls ignore access risk

Failed rows frequently contain the very data an organization is trying to govern: patient identifiers, payment details, employee records, or commercially sensitive attributes. Publishing exceptions into a broadly accessible lakehouse can turn a quality initiative into a security incident. Microsoft notes that organizations own role-based access control for error rows stored in their managed storage .

Why common approaches fall short

“Profile everything and fix the red scores”

Profiling is an excellent discovery technique, not a prioritization model. It can generate more issues than stewards can resolve and treats business-critical and low-value assets alike. Start with a small set of decisions, data products, and critical data elements whose failure has a clear operational, financial, regulatory, or customer consequence.

“Fail every pipeline on every defect”

This protects quality at the cost of availability. A single malformed optional field can stop an entire product even when unaffected records remain valuable. Controls need explicit severity and disposition: reject the release, quarantine records, publish with a warning, or observe only.

“Drop invalid rows and continue”

Dropping can protect downstream consumers, but silent exclusion can alter totals and hide source-system problems. Fabric materialized lake views support FAIL and DROP actions for constraint violations; FAIL stops refresh, while DROP continues and exposes the dropped-row count in lineage. If both appear, FAIL takes precedence. Use DROP only when the source record remains recoverable, the exclusion is visible, reconciliation tolerances are defined, and remediation has another evidence path.

“A Purview alert is a release gate”

An alert makes a threshold actionable for a person, but it should not be assumed to block a Fabric publication automatically. Microsoft also documents that configured threshold metadata does not publish to a Fabric Lakehouse or ADLS Gen2 for self-service analytics. If a hard release gate is required, implement it explicitly in the Fabric delivery path and test its failure behavior.

“The catalog team owns quality”

The governance team can define standards and monitor compliance, but it cannot correct every operational source. Business data owners decide fitness; source owners correct upstream processes; data engineers implement controls; stewards manage definitions and exceptions; product owners accept or reject residual risk.

A practical Microsoft Fabric solution

The target architecture separates four responsibilities: preserve, enforce, assure, and remediate.

Operational sources
       |
       v
Bronze: preserved source data + ingestion metadata
       |
       | structural checks, reconciliation, recoverability
       v
Silver: conformed data + enforceable Fabric quality gates
       |                  \
       |                   -> restricted exception evidence
       v
Gold: governed data products and semantic models
       |
       v
Reports, operations, analytics and AI

Across the layers:
Microsoft Purview Data Map -> metadata and lineage
Unified Catalog            -> domains, products, critical elements and ownership
Purview Data Quality       -> profiling, rules, thresholds, scans, alerts and trends
Remediation workflow       -> triage, correction, validation, closure and prevention

1. Define a quality contract for each important data product

Create a compact contract before creating hundreds of rules. For each data product, record:

Purview provides out-of-the-box rules across six quality dimensions : completeness, consistency, conformity, accuracy, freshness, and uniqueness. It also supports custom rules, profiling, scheduled scans, and job monitoring. The contract supplies the business meaning those features cannot infer on their own.

2. Preserve recoverable Bronze data

Bronze should preserve received source data and operational metadata such as source system, extraction time, batch or event identifier, source file, and ingestion run. Avoid “cleaning” away evidence before validation. Restrict access because Bronze and error data often contain sensitive values.

Perform lightweight ingestion checks here:

• Was the expected source delivered?
• Is the file or message readable?
• Does the observed schema meet the ingestion contract?
• Are record counts and control totals within expected bounds? and
• Can every row be traced to a source batch or event?

These checks protect recoverability and support replay. They do not declare the data fit for business use.

3. Enforce critical invariants in Silver

Silver is the conformance boundary. Standardize identifiers, timestamps, codes, reference relationships, and deduplication logic. Place release-critical tests as close as possible to the transformation that creates the trusted field.

For suitable Lakehouse transformations, Fabric materialized lake views can express Bronze-to-Silver-to-Gold logic declaratively, persist the result as Delta tables, track dependencies, orchestrate refresh order, and expose quality violations in monitoring and lineage. A simplified example is:

CREATE OR REPLACE MATERIALIZED LAKE VIEW silver.service_cases (
    CONSTRAINT case_id_required
        CHECK (case_id IS NOT NULL) ON MISMATCH FAIL,
    CONSTRAINT valid_priority
        CHECK (priority IN ('LOW', 'MEDIUM', 'HIGH', 'CRITICAL')) ON MISMATCH FAIL
)
AS
SELECT
    TRIM(case_id) AS case_id,
    TRIM(customer_id) AS customer_id,
    UPPER(TRIM(priority)) AS priority,
    opened_at,
    status,
    source_run_id
FROM bronze.service_cases;

Use this pattern selectively. Materialized lake views fit SQL-expressible transformations and deterministic constraints. Microsoft recommends other approaches for non-SQL logic, API calls, machine-learning inference, or subsecond streaming scenarios. The PySpark authoring option is documented as preview and currently performs full refreshes, so production adoption should be assessed against current regional availability and workload requirements.

For pipelines, notebooks, Warehouse transformations, and streaming paths that do not use materialized lake views, implement equivalent checks in the workload’s native execution path. Standardize the result schema so every check emits the same operational fields: product, asset, rule, severity, run, observed value, expected condition, failed count, disposition, owner, and timestamp.

4. Publish only an explicitly approved Gold version

Gold is a business interface, not just another transformation layer. A publish decision should evaluate:

• all mandatory Silver gates;
• record and value reconciliation against the previous trusted version or source controls;
• freshness and completeness objectives;
• downstream impact of a partial or delayed release;
• approved waivers with owner and expiry; and
• semantic-model validation for affected measures and relationships.

Keep the last known good Gold version available when a new version fails, where the business semantics permit it. Clearly expose its “data as of” time so continuity does not become hidden staleness. Do not refresh downstream semantic models merely because upstream compute completed; refresh them after the data-product publication decision succeeds.

5. Use Purview for independent assurance and federated ownership

Register and scan the Fabric tenant into Microsoft Purview Data Map , then associate the relevant Lakehouse tables with governed data products. Microsoft documents full, incremental, and scoped Fabric scans, while noting limitations: for non-Power BI Fabric items, scan support includes item-level metadata and lineage; Lakehouse tables and files can provide sub-item metadata, but sub-item lineage is not supported. Private-link combinations and some Parquet encodings also have documented constraints that should be tested in the target environment.

Purview data-quality support currently includes Fabric Lakehouse, filesystem shortcuts, and mirrored databases, with Delta, Parquet, and Iceberg formats listed for profiling and quality scans .

Use Purview to:

• model governance domains and data products around business accountability;
• map critical data elements such as Customer ID across differently named fields;
• define reusable rules and thresholds by criticality;
• schedule scans at a cadence that matches the data’s freshness objective;
• alert accountable stewards when a threshold is breached;
• review product and domain trends rather than isolated run results; and
• give executives a view of persistent risk and remediation performance.

Purview roles should follow least privilege . The Data Quality Steward role can manage rules, scanning, schedules, monitoring, thresholds, and alerts. Reader roles have narrower access, and Microsoft recommends minimizing high-privilege administration. Assign roles to groups rather than individuals where possible, and maintain at least two accountable owners for critical governance domains.

6. Turn failed records into a controlled remediation queue

Purview can publish data-quality error records into a Fabric Lakehouse or ADLS Gen2 so teams can investigate and remediate them. The output includes rule and governance identifiers that can support reporting and workflow correlation. Microsoft documents a limit of 100,000 published failed records per rule per run, so this store should be treated as investigation evidence, not a guaranteed full reject archive.

Build a workflow around that evidence:

1. Detect: create an exception with rule, asset, product, run, severity, count, and sample evidence.
2. Classify: identify source defect, transformation defect, reference-data defect, timing issue, or rule defect.
3. Assign: route to the named source owner, data-product team, or steward.
4. Contain: stop publication, retain last known good data, quarantine affected records, or publish an approved warning.
5. Correct: repair at the earliest authoritative point. Avoid editing Gold outputs directly.
6. Replay: reprocess from recoverable Bronze data or the authoritative source.
7. Validate: rerun the failed rule plus reconciliations and downstream regression checks.
8. Close: record cause, disposition, evidence, impacted consumers, and recurrence prevention.

Store exceptions in a separate, restricted operations workspace. Apply sensitivity and retention policies appropriate to the underlying fields. Where regions have separate residency requirements, Microsoft supports region-specific error-record folders and states that Purview does not copy these records between Azure regions.

A phased implementation roadmap

Phase 1: Select one consequential product and baseline it

Choose a product connected to a meaningful decision, with a reachable source owner and measurable pain. Inventory its sources, consumers, critical elements, known incidents, manual reconciliations, refresh timing, and current quality logic. Profile representative periods, including month-end, peak events, and late-arriving data.

Deliverables: product quality contract, owner and steward matrix, baseline defect and incident measures, initial security and cost model, and a ranked rule backlog.

Phase 2: Implement critical Fabric gates

Add source controls, recoverable Bronze storage, standardized Silver transformations, release-critical checks, reconciliation, and an explicit Gold publish decision. Prove failure paths deliberately: missing source, duplicate key, invalid reference, late delivery, and partial load.

Do not proceed until the team can demonstrate that a failed critical control prevents untrusted publication, preserves evidence, alerts an owner, and supports replay.

Phase 3: Establish Purview product governance

Register and scan the relevant Fabric scope, curate the governed product, map critical elements, assign least-privilege roles, and configure rules and thresholds. Schedule scans according to the contract rather than a universal cadence. Enable error-record publication only after its workspace permissions, region, retention, and masking requirements have been approved.

Phase 4: Operationalize remediation

Integrate alerts with the organization’s work-management or incident process. Add severity, service targets, escalation, waiver expiry, and root-cause categories. Build a view that connects defects to affected products and business services. Run simulations with business and engineering owners.

Phase 5: Scale by reusable control pattern

Expand to adjacent products using templates for rule naming, evidence schema, ownership, severity, exception disposition, and metrics. Centralize standards while leaving product owners accountable for fitness. Review thresholds quarterly and after material source or business changes.

Governance, security, adoption, and cost considerations

Governance

Maintain a versioned rule register. Every rule should have a business purpose, owner, severity, enforcement point, threshold, effective date, and test evidence. Separate rule defects from data defects; a failing check can indicate either. Waivers must be time-bound and approved by the person accountable for the affected decision.

Security and privacy

Scanning and exception publishing introduce identities, permissions, network paths, and additional data copies. Use managed identities or service principals appropriate to the documented scenario, isolate operations storage, and test that readers cannot access failed-row values unless their role requires it. Minimize retained columns in exception evidence and avoid distributing raw samples in email alerts or tickets.

Adoption and operating model

Business ownership must be real enough to make trade-offs. Train product owners to interpret fitness, trends, and residual risk rather than SQL. Train stewards to distinguish a rule change from a source correction. Give engineers a standard way to implement and emit checks, so governance does not become a collection of one-off controls.

Cost and performance

Quality scans consume compute and can contend with source workloads. Align schedules to risk and change frequency, use incremental scans where supported, and run large checks outside peak periods. Microsoft states that Unified Catalog charges for unique governed assets per day and that data-quality and health actions consume data governance processing units per run. Cost varies by rule type, data volume, source, and performance tier.

Control cost by governing high-value assets first, avoiding duplicate rules, matching scan frequency to the product service objective, using incremental scanning where supported, tracking consumption by governance domain, and accounting for Fabric capacity used by transformations, replay, and exception analytics.

Success metrics

Avoid making the aggregate quality score the primary outcome. Use a balanced set of operational and business measures:

Set targets only after measuring a representative baseline. A target without a baseline invites arbitrary thresholds and misleading improvement claims.

How YuniQ can help

YuniQ’s Microsoft Fabric consulting practice aligns directly with this operating model. Its Fabric Strategy & Readiness service includes current-state diagnostics, capacity planning, workspace topology, and governance and operating-model design. Its Data Integration & Engineering capability includes Fabric pipelines, Lakehouse and Warehouse implementation, Spark and Delta frameworks, and data-quality observability and exception handling. Its Governance, Security & DevOps service covers Purview integration, lineage and ownership tracking, least-privilege design, environment separation, CI/CD, and monitoring.

For implementation, YuniQ describes a phased delivery framework spanning discovery, architecture, a focused proof of value, iterative build, automated reconciliation and access testing, production launch, and ongoing optimization. That creates a practical engagement path for a quality-control pilot: select one high-value product, prove the gates and remediation loop with real data, validate capacity and permissions, then scale reusable patterns across domains.

Practical next steps

Follow these nine practical steps to baseline and control your data quality:

1. Select one data product whose failure changes a consequential business decision.
2. Name its business owner, technical owner, steward, source owners, and exception responders.
3. Identify no more than five critical elements and ten initial rules.
4. Classify each rule as stop, quarantine, warn, or observe.
5. Map where each rule is enforced in Fabric and where it is independently measured in Purview.
6. Design exception storage as sensitive operational data, including region, access, retention, and record limits.
7. Test at least five failure scenarios and one replay before production launch.
8. Baseline detection, containment, correction, recurrence, and cost metrics.
9. Review the pilot after one operating cycle and scale only the patterns that proved useful.

The key design decision is not which quality score to display. It is where the organization draws the boundary between raw, usable, and trusted data, and whether every breach of that boundary has an owner and a tested recovery path.