AI examinations will test the operating record

The NAIC says its Big Data and Artificial Intelligence Working Group has been developing an AI systems evaluation tool for market conduct, financial analysis and examination contexts. As of March 2026, the tool was being piloted by twelve states.

For MGAs, examination readiness should begin with the carrier relationship. A delegated workflow can influence insurer decisions and customer outcomes even when the technology is purchased and operated by the MGA.

Retain the operating record

Governance becomes credible through contemporaneous evidence.

• Approved use case, accountable owner and risk classification.
• Data lineage and material transformations.
• Validation method, sample and limitations.
• Human review thresholds and override history.
• Performance monitoring, incidents and change approvals.

Contracting is part of governance

Vendor contracts should support access to the information needed for oversight, incident response and regulatory inquiry. A promise of accuracy without audit access is not a control.

The most difficult evidence may sit with the vendor

An MGA can document its approval and monitoring process yet remain unable to explain training data, material changes, performance by segment or the origin of a consequential output. Contractual access determines whether governance survives a real inquiry.

Procurement should therefore treat explainability, incident notice, change control and evidence retention as operating requirements rather than legal boilerplate.

The counterpoint: exhaustive transparency may be unavailable

Some useful systems will remain technically complex or commercially protected. Governance cannot depend on complete access to every internal detail; it needs sufficient evidence to assess the decision risk, validate outcomes and stop use when confidence falls.

MGA Index expects examination readiness to converge on proportional assurance: deeper access for systems with greater influence on eligibility, pricing, claims and customer outcomes.

• Define minimum evidence before vendor selection.
• Retain the versions used for consequential decisions.
• Test the ability to suspend a workflow without disrupting underwriting continuity.

The requested evidence is becoming foreseeable

The NAIC says its developing evaluation tool addresses AI use, governance, risk mitigation, higher-risk models and input data. Its third-party working group is separately considering oversight of external data and models used in property and casualty pricing and underwriting. Together, these efforts point toward examination of operating evidence rather than principles alone.

An MGA should be able to identify affected decisions, accountable owners, validation, changes, incidents and material overrides—even when the carrier is the regulated entity and a vendor supplies the model.

The countercase: one inventory can create false assurance

A complete model list does not show where AI actually influences work. Embedded features, vendor updates and employee tools can alter information presented to underwriters without appearing as formal models. Inventory should begin with consequential decisions and trace the systems supporting them.

Readiness is the ability to reconstruct use and response, not the existence of a policy binder.