0
Data monitoring for AI and regulatory reporting: What to check, thresholds, and alerts
An insurer’s data feeds two consumers that do not forgive errors: AI models and supervisory reports. A wrong value rarely shows itself. It ends up as a mispriced policy, a biased decision, or a reported figure that has to be corrected later. This guide is written for architects, data owners, and IT leads in insurance. It explains what data monitoring covers, which rules require it, and where in the data flow to check. It then sets out a concrete monitoring plan: which checks to run, how to set thresholds, and who receives each alert. One trade-off runs through every choice in that plan. More checks and tighter thresholds catch more problems, but they also raise more false alarms, and a monitoring system that people learn to ignore protects nothing.
What is data monitoring?
Data monitoring is the systematic process of observing, tracking, and analyzing data to ensure its accuracy, quality, and proper integration within an organization’s systems. Acceldata describes it as analyzing and evaluating data to ensure the availability of high-quality data that meets business purposes and standards, covering aspects such as uniqueness, accuracy, completeness, and conformity.
The term covers more than one activity. Secoda lists seven types of data monitoring: real-time monitoring, compliance monitoring, anomaly detection, performance monitoring, data quality monitoring, data security monitoring, and data lineage monitoring. Data quality monitoring is the narrower discipline; lakeFS defines it as the practice of measuring and reporting changes in data quality dimensions and concerns.
Two meanings, two owners
Search results for “data monitoring” mix two different jobs. One is monitoring the quality of data: whether values are complete, correct, and fit for use. The other is monitoring access to data and databases for security incidents.
In an insurer, the two have different legal bases and different owners. Security detection belongs to ICT risk management: DORA requires financial entities to have mechanisms to promptly detect anomalous activities, including ICT network performance issues and ICT-related incidents (Article 10). It is usually owned by IT security. This article is about the first job: the quality of the data that models and reports depend on. For the security side, see what cyber security means and why it matters.
Why data monitoring matters for an insurer
Acceldata notes that monitoring data regularly reduces the risk of incorrect analytics and data-related errors. For an insurer, regulation turns that benefit into an obligation, from several directions at once.
Technical provisions and Solvency II reporting
Solvency II requires insurers to have internal processes and procedures that ensure the appropriateness, completeness, and accuracy of the data used to calculate technical provisions (Article 82). The delegated rules spell out what that means: complete data has enough history to show the characteristics and trends of the risks; accurate data is free from material errors and recorded promptly and consistently over time; appropriate data is consistent with the purposes for which it is used (Delegated Regulation 2015/35, Article 19).
High-risk AI systems
For high-risk AI systems, the AI Act requires training, validation, and testing data to be relevant, sufficiently representative, and, to the best extent possible, free of errors and complete in view of the intended purpose (Article 10). It also requires automatic logging (Article 12) and a post-market monitoring system run by the provider (Article 72). In insurance, AI systems for risk assessment and pricing of natural persons in life and health insurance are high-risk. These obligations apply from December 2, 2027.
Other AI systems
For AI that is not high-risk, EIOPA’s Opinion of August 2025 includes data governance and record-keeping among the supervisory expectations under Solvency II and the Insurance Distribution Directive (IDD).
Personal data
GDPR requires personal data to be accurate and, where necessary, kept up to date (Article 5(1)(d)).
Enterprise context
In a regulated insurer, monitoring has a second function beyond quality assurance: it produces the evidence that a control existed and worked. An alert is a record that a check ran at a specific point and either passed or failed. That changes design priorities. A monitoring setup built only for operational speed may discard alerts once resolved; one built for a regulated environment keeps the alert history, documents which rule or model triggered each alert, and can produce that record on request.
Whether a specific log falls under a retention or audit-trail obligation is a question for the compliance function, not an architectural assumption.
Where in the data flow to monitor
lakeFS describes three places where data quality problems arise: at data ingestion, inside the data systems and pipelines that transform the data, and in downstream systems such as machine-learning training pipelines or analytics tools. In an insurer, the three places look like this:
- Extracts from policy administration and claims systems, payment and accounting feeds, and data from brokers and external providers. Problems here are missing records, wrong formats, and late deliveries.
- Transformations into the data warehouse, actuarial data marts, and model feature stores. Problems here are broken joins, wrong mappings, and transformation errors that silently change values.
- Models, regulatory reports, and management reporting. Problems here are distribution shifts in model inputs, outputs that drift away from their baseline, and report figures that no longer reconcile with the source.
A check at ingestion is cheapest but cannot see errors introduced later. A check downstream sees the result but cannot tell where the error came from. Insurers therefore need checks at all three places, with lineage that connects them.
The table below turns the three places into monitoring types by stage (input, process, and output) and adds two operating modes that can apply at any stage: real-time monitoring, which Secoda describes as continuous tracking as it happens, and high-dimensional monitoring for data with many related variables, such as model features.
| Monitoring type | Description | Example use case | When to use | Limitations | Best for |
|---|---|---|---|---|---|
| Input data monitoring | Checks data where it enters, before any transformation (lakeFS: data ingestion) | Mandatory fields in claims records arriving from the claims system | Sources that are external or change without notice | Cannot see errors introduced later in the pipeline | Catching bad records before they spread |
| Process/transformation monitoring | Checks the transformation steps themselves (lakeFS: data systems and pipelines) | Product-code mapping into the actuarial data mart | Complex or frequently changed transformations | Needs instrumentation inside the pipeline | Finding where an error was introduced |
| Output data monitoring | Checks data as it reaches consumers (lakeFS: downstream systems) | Reconciling reserve totals in a report with the source | Before every report and every model run | Finds problems only after processing | Protecting reports and models |
| Real-time monitoring | Continuous tracking of data and system performance as it happens | Volume of incoming claims notifications during a storm | When a delay has direct financial or customer cost | Higher infrastructure cost and more alerts | Time-critical streams |
| High-dimensional monitoring | Monitoring many related variables together | Joint shift in the features of an underwriting model | When the risk lies in combinations of variables | Thresholds are harder to tune; more false alarms | Model inputs |
Types of data monitoring and where each fits
The table above shows where in the data flow to place checks. The next question is which kinds of monitoring to run. Secoda groups them into seven types, which map onto insurance needs as follows. The final column says which job each type is best for.
| Monitoring type | What it does | Insurance example | Best for |
|---|---|---|---|
| Real-time monitoring | Continuous tracking of data and system performance as it happens | Payment and claims-notification streams | Processes where delay costs money or customers |
| Compliance monitoring | Checks that data practices align with regulations and internal policies | Reporting-calendar and data-retention checks | Evidence for supervisors and auditors |
| Anomaly detection | Identifies data points or patterns that deviate from the norm | Sudden change in claim frequency in one region | Unknown problems that no rule anticipated |
| Performance monitoring | Tracks the efficiency and effectiveness of data systems | Nightly batch duration before reporting deadlines | Keeping pipelines within their time windows |
| Data quality monitoring | Evaluates data against criteria such as accuracy, completeness, and consistency | Mandatory fields in policy data for technical provisions | Models and regulatory reports |
| Data security monitoring | Protects sensitive information from unauthorized access or alteration | Access to health data in underwriting files | ICT risk under DORA, owned by IT security |
| Data lineage monitoring | Tracks the origins, transformations, and movements of data | Tracing a report figure back to source systems | Audit and root-cause analysis |
Most insurers need data quality, lineage, and compliance monitoring for reporting, and add anomaly detection for model inputs and outputs. Real-time monitoring is worth its cost only where a delay has a direct financial or customer impact.
Dimensions and metrics
lakeFS lists six data quality dimensions: accuracy, completeness, consistency, validity, uniqueness, and integrity. It also names practical metrics to monitor: error ratio, empty values, duplicate record rate, data transformation errors, address validity percentage, volume of dark data, and data time-to-value.
These metrics translate directly into checks. Empty values in mandatory fields measure completeness. Duplicate policy or claim numbers measure uniqueness. Codes outside the reference lists measure validity. Transformation errors and failed reconciliations measure integrity and consistency across systems. The next section turns them into a plan.
What to check, thresholds, and alerts: An insurer’s data monitoring plan
The table below is a baseline plan for the data behind AI models and supervisory reports. The thresholds are starting points, not standards: calibrate each one on your own history before you switch alerts on, and record why you chose it.
| Check | Applies to | Starting threshold | Alert goes to | On failure |
|---|---|---|---|---|
| Record count per run compared with previous runs | Ingestion, all feeds | Outside the range observed over the last twelve runs | Data owner of the feed | Hold the load until confirmed |
| Empty values in mandatory fields (e.g., date of birth, sum insured, line of business) | Policy and claims data for reports and models | Any empty value in a field the report or model requires | Data steward of the source system | Block the field from the report or model run |
| Duplicate policy or claim identifiers | Ingestion and data warehouse | Zero tolerance | Data owner | Block and correct at source |
| Codes outside reference lists (product, line of business, currency) | Pipelines | Zero tolerance for reporting fields | Data steward | Block until mapped |
| Reconciliation of totals (premiums, claims paid, reserves) with the source and the general ledger | Downstream, before each report | Tolerance agreed with the actuarial and finance functions | Actuarial function and finance | Report not released until explained |
| Transformation errors | Pipelines | Any error in a reporting or model pipeline | Pipeline owner (IT) | Rerun after fix; log the cause |
| Distribution of model input features compared with the training baseline | Model inputs | A drift statistic outside the band set from historical variation | Model owner | Review before the next scoring run |
| Distribution of model outputs (scores, referral rates) | Model outputs | Change beyond the agreed band since the last review | Model owner and business process owner | Human review of affected decisions |
| Data arrival time | All feeds before a reporting deadline | Later than the buffer the reporting calendar needs | Reporting coordinator | Escalate under the reporting plan |
| Lineage coverage of report figures | Regulatory reports | Any reported figure without documented lineage | Data governance lead | Document before submission |
Three rules make the plan work
First, every check has one named owner, not a team; an alert without an owner is the same as no alert. Second, every alert has a defined consequence (hold, block, review, or escalate) so that no one has to work out under deadline pressure what an alert means. Third, every alert and its resolution is logged, because that log is the evidence described in the Enterprise context section above. The owners in the table follow the control matrix in Artificial intelligence governance for insurers.
When the model comes from a vendor
Many insurers do not build their pricing or underwriting models; they buy them or use them inside a vendor platform. The monitoring duty does not move to the vendor with the model. For high-risk AI systems, from December 2, 2027, the AI Act splits it between the two parties:
- The provider (the vendor) must run a post-market monitoring system (Article 72) and build in automatic logging (Article 12).
- The deployer (the insurer) must ensure that input data is relevant and sufficiently representative for the intended purpose, to the extent it controls that data (Article 26(4)). It must monitor the operation of the system on the basis of the provider’s instructions for use (Article 26(5)) and keep the logs the system generates, to the extent they are under its control, for at least six months, unless Union or national law provides otherwise (Article 26(6)).
Insurers have a specific position here. For deployers that are financial institutions subject to internal-governance requirements under Union financial services law, such as insurers under Solvency II, the AI Act treats the monitoring obligation as fulfilled by complying with those governance rules (Article 26(5)). Such deployers keep the logs as part of the documentation they maintain under that law (Article 26(6)). The other deployer duties under Article 26 still apply. An insurer that puts its own name on a vendor’s high-risk system, substantially modifies it, or changes the intended purpose of a non-high-risk system so that it becomes high-risk becomes its provider (Article 25(1)).
In practice, the insurer owns the checks on everything it feeds into the model (the ingestion and pipeline rows of the monitoring plan) and the review of outputs in its own portfolio. Two questions belong in every vendor contract for an AI model: which monitoring signals and logs the vendor provides, and how the insurer is told when the vendor’s own monitoring finds a problem. Without answers, the insurer cannot show a supervisor that the system is under control.
Implementation steps for data monitoring
Acceldata’s recommended strategies include setting clear data objectives, choosing the right data sources, establishing realistic benchmarks, setting regular alerts, and performing regular audits. In an insurer, they translate into six steps.
Implementation checklist:
- Define objectives: Decide what the monitoring must protect: specific reports, specific models, specific quality dimensions. Without this, later steps default to monitoring what is easy to instrument rather than what matters.
- Identify data sources: Catalog every system, pipeline stage, and feed that supplies the protected reports and models, with an owner for each.
- Select checks and methods: Choose the checks from the plan above for each source. Use explicit rules where the expected value is known; lakeFS lists data quality rules among its monitoring techniques. Use statistical comparison against history where the expected value moves over time.
- Design the alert workflow: Specify how alerts are generated, routed, and escalated, who acts on each one, and what happens if nobody does.
- Integrate with the stack: Connect checks to the existing pipelines, warehouse, and orchestration tools, without duplicating checks that already run elsewhere.
- Validate and iterate: Test the setup against known past incidents to confirm it would have caught them, then tune thresholds to keep false alarms manageable.
For model inputs, step 3 carries extra weight. Single-field rules do not catch a shift in the combination of features a model relies on, so model-input monitoring needs statistical comparison with the training baseline in addition to rules.
Methodology
Scope: design of data monitoring for the data behind AI models and supervisory reports in an insurer: the checks, thresholds, owners, and alert workflow.
What this does not address: selection of specific commercial tools, configuration of named platforms, and security monitoring of databases, which belongs to ICT risk management under DORA.
Prerequisites: familiarity with the organization’s data flows and reporting obligations. This article does not replace a compliance review.
Source: monitoring types from Secoda; quality dimensions, metrics, and techniques from lakeFS; strategies from Acceldata; legal requirements from Solvency II and its delegated rules, the AI Act, GDPR, DORA, and EIOPA’s Opinion. The monitoring plan is the author’s synthesis; its thresholds are starting points to calibrate, not published benchmarks.
Practical examples
The following scenarios are illustrative patterns, not case studies from a named organization.
Scenario 1: A broken mapping before quarterly reporting
A product-code mapping changes in the policy system, and a group of policies lands in the wrong line of business. The validity check flags codes outside the reference list during the pipeline run. The data steward maps the new code, and the reconciliation check confirms that totals per line of business match the source before the report is released.
Scenario 2: Drift in a pricing model’s inputs
A change in how a sales channel records customer age shifts the age distribution feeding a pricing model. The input-distribution check flags the shift against the training baseline. The model owner holds the next scoring run, the source is corrected, and the affected quotes are reviewed, which is the human review the AI governance controls require.
Scenario 3: A late feed before a deadline
A claims extract arrives later than the reporting buffer allows. The arrival-time check escalates to the reporting coordinator early enough to use the contingency in the reporting plan, instead of discovering the gap on the day of submission.
Limitations
Data monitoring, whatever types and methods are chosen, has limits that persist in a good implementation.
- False positives and alert fatigue. Statistical checks flag deviation, not confirmed error, so a threshold tuned to catch real problems also produces false alarms. Teams flooded with them start ignoring alerts, which erodes monitoring without any visible failure. Calibrating thresholds on history and reviewing them regularly is the only remedy.
- Rules catch only what someone anticipated. Rule-based checks are explicit and easy to audit, but each rule covers a failure mode someone foresaw, and the rule set needs maintenance as systems change.
- Compliance scope is a legal determination. A monitoring system can keep alert history, but whether that history satisfies a specific obligation under Solvency II, GDPR, or DORA is for compliance and legal to confirm.
- Monitoring detects; it does not correct. Acceldata describes the tool sending an alert so that the error can be checked and corrected. The correction is a separate step that depends on people and processes outside the monitoring system.
Tools and platforms
Tool selection is out of scope here. Choosing a monitoring platform depends on the existing stack, the licensing model, and the features you need for the checks above. Use the monitoring plan as the requirements list when you evaluate tools: a tool that cannot assign an owner, define a consequence, and keep an alert history for each check does not meet a regulated insurer’s needs.
Frequently asked questions about data monitoring
What are the main types of data monitoring?
Secoda lists seven: real-time monitoring, compliance monitoring, anomaly detection, performance monitoring, data quality monitoring, data security monitoring, and data lineage monitoring. For AI models and regulatory reports, data quality, lineage, compliance, and anomaly monitoring matter most.
What is the difference between data monitoring and database security monitoring?
Data quality monitoring checks whether data is complete, correct, and fit for use. Database security monitoring detects unauthorized access and attacks. In a financial institution, security detection falls under DORA, which requires mechanisms to promptly detect anomalous activities (Article 10). Security detection is usually owned by IT security, while data quality is owned by data owners and the actuarial and finance functions.
How does data quality monitoring fit into an overall data monitoring strategy?
It is one of the monitoring types, focused on quality dimensions: accuracy, completeness, consistency, validity, uniqueness, and integrity. A complete strategy combines it with lineage, compliance, and anomaly monitoring.
Which regulations require insurers to monitor data quality?
Solvency II requires processes that ensure the appropriateness, completeness, and accuracy of the data used for technical provisions (Article 82). The AI Act sets data quality requirements for high-risk AI systems from December 2, 2027 (Article 10). EIOPA’s Opinion covers data governance for other AI systems, and GDPR requires personal data to be accurate (Article 5(1)(d)).
How should alert thresholds be set?
Start from your own history rather than generic numbers: measure how much each metric varies over recent runs and set the threshold outside that normal range. Use zero tolerance only where any error is unacceptable, such as duplicate identifiers or invalid codes in reporting fields. Record the reasoning for each threshold and review it after each reporting cycle.
What are the key steps to ensure auditability in data monitoring workflows?
Keep a record of each check, what triggered each alert, who acted on it, and how it was resolved. Maintain lineage from each reported figure back to its source. Confirm with compliance which records fall under retention or audit-trail obligations.
Which metrics or signals indicate that a data monitoring strategy is effective?
Useful internal signals of an effective strategy are a falling time from defect to detection, a stable or falling share of false alarms, alerts that are acted on rather than left open, and fewer corrections after reports are submitted.
Next steps and further reading
For the owners of the controls behind each alert, see Artificial intelligence governance for insurers. To check whether your data is ready for an AI project at all, use the scorecard in AI readiness assessment for insurers. For agent-based systems that read and write insurance data, see Multi-agent systems in insurance. For more detail on quality metrics and techniques, lakeFS’s guide covers them in depth.
At msg life Slovakia, I work as a Data Scientist with experience in developing insurance systems. I focus on the design and compliance of agentic AI systems in insurance and write about how the regulatory requirements of the EU AI Act translate into real-world architecture. I'm the author of the video series S Michalom o AI on the msg life Slovakia YouTube channel, where I explore various aspects of agentic programming.
Get to know us
See who translates insurance into code at msg life Slovakia.