Champion challenger governance

How to study this topic

Champion challenger governance lets a bank compare the approved production model with controlled alternatives without allowing experimental models to quietly influence customer, capital, fraud, compliance or reporting outcomes before approval. Study this as a banking governance chapter. The important question is not whether AI can produce a score, explanation or document pack. The important question is whether the bank can prove the model is suitable, lawful, monitored, limited and accountable for model comparison, controlled experimentation, performance challenge, risk appetite review, approval evidence and safe model replacement.

The topic belongs to banking champion challenger model governance. Keep the focus on credit, compliance, model-risk governance, fair lending, customer outcome, validation evidence and approval control. Do not drift into generic AI productivity language. In banking, the model output matters only when the control context around it is strong enough.

A strong learner should be able to explain this topic to a credit-risk manager, fair lending specialist, compliance analyst, model validator, product owner, developer, tester, auditor and governance committee member. Each person should understand what evidence is needed, what the model can do, what it cannot prove and where human accountability remains.

Plain language meaning

In plain language, champion challenger governance is about preventing AI from being treated as trusted banking evidence before the bank has tested the model, checked the law, reviewed the customer impact and agreed the approval boundary. A model can be technically impressive and still be unsuitable for a regulated credit or compliance use.

The practical discipline is to separate prediction, explanation, compliance evidence and final action. Prediction estimates an outcome. Explanation describes model drivers. Compliance evidence proves the bank followed the right control process. Final action affects a customer, report, control, case or policy position. Mixing those four layers is where many model-governance failures begin.

A good bank does not approve AI because it sounds modern. It approves a controlled use of a specific model for a specific population, purpose, product, jurisdiction and decision boundary. That approval should be visible in the model inventory, validation pack, committee record and monitoring process.

Where it sits in the bank

This topic normally sits across credit risk, compliance, fair lending, legal, model risk management, product, data, technology, operations and internal audit. The exact operating model differs by bank, but ownership must not be vague. Someone must own the model, someone must validate it, someone must approve use and someone must monitor the outcome.

The population in scope is production credit models, candidate challenger models, fraud models, AML models, provisioning models, pricing models, scorecard replacements, model owners, validators, technology teams and governance committees. Testing must reflect the population actually affected by the model. Clean demonstration examples are not enough. Banking populations include missing data, thin files, local policy exceptions, manual overrides, legacy records, vulnerable customers, new products, rejected applicants and changing economic conditions.

The output may support application scoring, limit setting, pricing, referral, adverse action notices, compliance review, model approval, documentation review, audit evidence or regulatory response. The risk level changes when the same model moves from research to decision support, then from decision support to automated action.

Evidence and source material

Relevant evidence includes champion model, challenger model, business objective, test population, parallel run output, holdout sample, performance metric, fairness result, stability result, override record, customer-impact analysis, validation finding, approval condition, and deployment recommendation. Evidence must be current, traceable and fit for the question being asked. A policy document, model metric, explanation output, validation report or approval note is useful only when the bank can show source, version, owner, date, limitation and approved use.

For credit and fair lending topics, evidence also needs customer-outcome context. A model that performs well at portfolio level can still create unacceptable outcomes for a subgroup, a product segment, a pricing path or a manual referral population. Aggregate accuracy does not remove the need for segment-level challenge.

For model approval topics, evidence must connect the business purpose to the technical method. A validator should be able to see why the model was built, what data it uses, how it was tested, what limitations remain, what risks were accepted and how those risks will be monitored after release.

Control expectations

Controls should include experiment approval, traffic restriction, no-customer-impact rule, parallel-run governance, metric definition, fairness comparison, statistical significance review, validation challenge, rollback plan, approval committee, implementation control, and post-deployment monitoring. These controls make the difference between a useful model and an uncontrolled model. The bank should know what must be documented before use, what must be validated independently, what must be approved by governance and what must be monitored in production.

Control design should be proportionate to materiality. A low-risk internal research tool does not need the same approval pack as a model that affects credit access, pricing, limits, regulatory reporting or customer communication. But once the model can influence a material banking outcome, casual governance is not enough.

Controls should also define the response when something is wrong. That may mean restricting use, forcing manual review, changing thresholds, updating reason codes, remediating documentation, retraining users, opening an issue, notifying a committee or stopping the model until the gap is closed.

How AI and ML can be adopted

Useful AI adoption includes comparing champion and challenger outcomes, detecting performance deterioration, identifying segment-level trade-offs, summarising validation differences, monitoring challenger stability, and preparing approval evidence. These are support uses first. AI can help find weak documentation, monitor patterns, summarise validation packs, detect proxy risk, compare outcomes and prepare governance material. It should not quietly replace the bank's legal, compliance, validation or approval judgement.

A sensible adoption path starts with controlled analysis and documentation support, then moves into validated decision support, then into restricted automation only when governance, monitoring, fallback and accountability are mature. The higher the customer or regulatory impact, the stronger the approval boundary must be.

The bank should write the model's role in operational language: score, explain, classify, recommend, refer, approve, decline, price, notify, document, monitor or escalate. Each verb carries a different control burden. If the bank cannot name the verb precisely, it cannot govern the use precisely.

Validation and challenge

Validation should review concept, data, methodology, assumptions, implementation, outcome quality, limitations, fairness, explainability, operational use and monitoring design. For banking AI, validation is not a final signature at the end. It is a structured challenge to whether the model is fit for the stated purpose.

Effective challenge means the validator can question the developer, the business owner, the data source, the training sample, the feature logic, the testing design, the reason-code mapping, the customer impact and the proposed monitoring thresholds. Challenge should be documented, answered and closed with evidence.

A model may pass technical accuracy tests and still need restrictions. It may be acceptable for analyst prioritisation but not for automatic decline. It may be acceptable for one product but not another. It may be acceptable in one jurisdiction but not another. Validation should make those boundaries visible.

Customer, compliance and conduct impact

The direct wrong outcome is a challenger model is treated as better because it wins one metric, while hidden customer harm, fairness weakness, operational fragility or approval gaps are missed. That is why this topic should be studied as customer-impact control, not only model governance theory. Credit AI can affect access, price, limit, explanation, complaint handling and trust. Compliance AI can affect evidence, escalation and regulatory position.

A bank must ask who is affected when the model is wrong. Is a sustainable applicant declined? Is an unaffordable customer approved? Is a protected group disadvantaged? Is a reason code inaccurate? Is a reviewer over-trusting the explanation? Is a governance committee approving a model without seeing a key limitation?

Conduct risk appears when the model creates pressure, exclusion, opacity, delay, poor explanation or weak remediation. The bank should treat those outcomes as control issues, not as cosmetic issues in the user interface or documentation wording.

Diagram walkthrough

The diagram follows five control steps: Champion model, Challenger test, Controlled comparison, Governance review, and Approved replacement. Read it left to right. It starts with the model or regulatory use case, moves through testing and governance, and ends with accountable use and retained evidence.

Each box is a bank control point. The implementation should name the owner, input, rule, evidence, review point and limitation at every step. If one box cannot be explained clearly, the model is not ready for high-trust banking use.

Bank-ready checklist

Before using this topic in production, ask whether the model purpose is clear, the legal classification is understood, the population is defined, the data is governed, the validation is independent, the customer impact is tested and the approval boundary is documented.

Then ask whether the monitoring thresholds, override process, adverse action reason logic, issue management, change control, audit pack and retirement criteria are in place. Banking AI governance is only strong when the bank knows what happens after approval, not just before approval.

If the answers are strong, the model can support banking work with discipline. If the answers are weak, the model may still produce a result, but the bank should not treat that result as controlled evidence for customer, regulatory or financial impact.

Source anchors for accurate study

Federal Reserve SR 26-2, dated 17 April 2026, supersedes SR 11-7 and SR 21-8 and attaches revised model-risk guidance for banking organisations.

The 2026 interagency model-risk guidance focuses on model development and use, validation and monitoring, governance and controls, and vendor or third-party model products.

The revised model-risk guidance says model risk depends on inherent risk, exposure, purpose and use, and that practices should be tailored to the bank's risk profile and model usage.

The EU AI Act treats AI systems used to evaluate the creditworthiness of natural persons or establish a credit score as high-risk, except where the system is used for financial fraud detection.

The EU AI Act high-risk framework includes controls around risk management, data governance, technical documentation, record keeping, transparency to deployers, human oversight, accuracy, robustness and cybersecurity.

ECOA and Regulation B require creditors to provide specific and accurate reasons for adverse action; using a complex algorithm does not remove that obligation.

U.S. Regulation B, 12 CFR 1002.9, requires specific principal reasons for adverse action in covered credit decisions, including when a creditor uses an AI model. CFPB Circular 2022-03 was withdrawn on 12 May 2025; do not cite it as current guidance. Primary sources: https://www.consumerfinance.gov/rules-policy/regulations/1002/9 and https://www.consumerfinance.gov/compliance/guidance/withdrawn-guidance/.

Federal Reserve public remarks on AI in the financial system emphasise that AI is not exempt from existing laws and risk-management expectations, including fair lending, privacy, cybersecurity, third-party risk and model risk.

NIST AI RMF describes trustworthy AI through characteristics including valid and reliable, safe, secure and resilient, accountable and transparent, explainable and interpretable, privacy-enhanced, and fair with harmful bias managed.

Banking practice note: legal classification

For champion challenger governance, legal classification is not optional detail. It decides whether the bank can explain the model, defend the use and protect the customer. A model may be fast and accurate in a narrow test, but banking approval depends on whether the result is suitable for model comparison, controlled experimentation, performance challenge, risk appetite review, approval evidence and safe model replacement.

The practical test is to trace one item from champion model through experiment approval and into the business use. If the team cannot show that path without guesswork, the model pack is not mature enough for serious banking reliance.

AI adoption should strengthen governance here. It should make weak evidence easier to see, proxy risk easier to challenge, documentation gaps easier to find and unstable outcomes easier to monitor. It should not become a way to hide uncertainty behind dashboards, explanations or committee slides.

A good implementation records model owner, model version, source data, feature list, intended use, population, limitation, validation result, approval condition, user action, override decision, monitoring result and issue history. That record is what makes the topic useful to risk, compliance, technology, audit and business owners.

Banking practice note: model purpose

For champion challenger governance, model purpose is not optional detail. It decides whether the bank can explain the model, defend the use and protect the customer. A model may be fast and accurate in a narrow test, but banking approval depends on whether the result is suitable for model comparison, controlled experimentation, performance challenge, risk appetite review, approval evidence and safe model replacement.

The practical test is to trace one item from challenger model through traffic restriction and into the business use. If the team cannot show that path without guesswork, the model pack is not mature enough for serious banking reliance.

Banking practice note: customer population

For champion challenger governance, customer population is not optional detail. It decides whether the bank can explain the model, defend the use and protect the customer. A model may be fast and accurate in a narrow test, but banking approval depends on whether the result is suitable for model comparison, controlled experimentation, performance challenge, risk appetite review, approval evidence and safe model replacement.

The practical test is to trace one item from business objective through no-customer-impact rule and into the business use. If the team cannot show that path without guesswork, the model pack is not mature enough for serious banking reliance.

Banking practice note: source authority

For champion challenger governance, source authority is not optional detail. It decides whether the bank can explain the model, defend the use and protect the customer. A model may be fast and accurate in a narrow test, but banking approval depends on whether the result is suitable for model comparison, controlled experimentation, performance challenge, risk appetite review, approval evidence and safe model replacement.

The practical test is to trace one item from test population through parallel-run governance and into the business use. If the team cannot show that path without guesswork, the model pack is not mature enough for serious banking reliance.

Banking practice note: training data

For champion challenger governance, training data is not optional detail. It decides whether the bank can explain the model, defend the use and protect the customer. A model may be fast and accurate in a narrow test, but banking approval depends on whether the result is suitable for model comparison, controlled experimentation, performance challenge, risk appetite review, approval evidence and safe model replacement.

The practical test is to trace one item from parallel run output through metric definition and into the business use. If the team cannot show that path without guesswork, the model pack is not mature enough for serious banking reliance.

Banking practice note: feature governance

For champion challenger governance, feature governance is not optional detail. It decides whether the bank can explain the model, defend the use and protect the customer. A model may be fast and accurate in a narrow test, but banking approval depends on whether the result is suitable for model comparison, controlled experimentation, performance challenge, risk appetite review, approval evidence and safe model replacement.

The practical test is to trace one item from holdout sample through fairness comparison and into the business use. If the team cannot show that path without guesswork, the model pack is not mature enough for serious banking reliance.

Banking practice note: protected-class testing

For champion challenger governance, protected-class testing is not optional detail. It decides whether the bank can explain the model, defend the use and protect the customer. A model may be fast and accurate in a narrow test, but banking approval depends on whether the result is suitable for model comparison, controlled experimentation, performance challenge, risk appetite review, approval evidence and safe model replacement.

The practical test is to trace one item from performance metric through statistical significance review and into the business use. If the team cannot show that path without guesswork, the model pack is not mature enough for serious banking reliance.

Banking practice note: proxy-variable review

For champion challenger governance, proxy-variable review is not optional detail. It decides whether the bank can explain the model, defend the use and protect the customer. A model may be fast and accurate in a narrow test, but banking approval depends on whether the result is suitable for model comparison, controlled experimentation, performance challenge, risk appetite review, approval evidence and safe model replacement.

The practical test is to trace one item from fairness result through validation challenge and into the business use. If the team cannot show that path without guesswork, the model pack is not mature enough for serious banking reliance.

Banking practice note: adverse action reasons

For champion challenger governance, adverse action reasons is not optional detail. It decides whether the bank can explain the model, defend the use and protect the customer. A model may be fast and accurate in a narrow test, but banking approval depends on whether the result is suitable for model comparison, controlled experimentation, performance challenge, risk appetite review, approval evidence and safe model replacement.

The practical test is to trace one item from stability result through rollback plan and into the business use. If the team cannot show that path without guesswork, the model pack is not mature enough for serious banking reliance.

Banking practice note: reason-code mapping

For champion challenger governance, reason-code mapping is not optional detail. It decides whether the bank can explain the model, defend the use and protect the customer. A model may be fast and accurate in a narrow test, but banking approval depends on whether the result is suitable for model comparison, controlled experimentation, performance challenge, risk appetite review, approval evidence and safe model replacement.

The practical test is to trace one item from override record through approval committee and into the business use. If the team cannot show that path without guesswork, the model pack is not mature enough for serious banking reliance.

Banking practice note: explainability limits

For champion challenger governance, explainability limits is not optional detail. It decides whether the bank can explain the model, defend the use and protect the customer. A model may be fast and accurate in a narrow test, but banking approval depends on whether the result is suitable for model comparison, controlled experimentation, performance challenge, risk appetite review, approval evidence and safe model replacement.

The practical test is to trace one item from customer-impact analysis through implementation control and into the business use. If the team cannot show that path without guesswork, the model pack is not mature enough for serious banking reliance.

Banking practice note: independent validation

For champion challenger governance, independent validation is not optional detail. It decides whether the bank can explain the model, defend the use and protect the customer. A model may be fast and accurate in a narrow test, but banking approval depends on whether the result is suitable for model comparison, controlled experimentation, performance challenge, risk appetite review, approval evidence and safe model replacement.

The practical test is to trace one item from validation finding through post-deployment monitoring and into the business use. If the team cannot show that path without guesswork, the model pack is not mature enough for serious banking reliance.

Banking practice note: effective challenge

For champion challenger governance, effective challenge is not optional detail. It decides whether the bank can explain the model, defend the use and protect the customer. A model may be fast and accurate in a narrow test, but banking approval depends on whether the result is suitable for model comparison, controlled experimentation, performance challenge, risk appetite review, approval evidence and safe model replacement.

The practical test is to trace one item from approval condition through experiment approval and into the business use. If the team cannot show that path without guesswork, the model pack is not mature enough for serious banking reliance.

Banking practice note: approval committee

For champion challenger governance, approval committee is not optional detail. It decides whether the bank can explain the model, defend the use and protect the customer. A model may be fast and accurate in a narrow test, but banking approval depends on whether the result is suitable for model comparison, controlled experimentation, performance challenge, risk appetite review, approval evidence and safe model replacement.

The practical test is to trace one item from deployment recommendation through traffic restriction and into the business use. If the team cannot show that path without guesswork, the model pack is not mature enough for serious banking reliance.

Banking practice note: documentation quality

For champion challenger governance, documentation quality is not optional detail. It decides whether the bank can explain the model, defend the use and protect the customer. A model may be fast and accurate in a narrow test, but banking approval depends on whether the result is suitable for model comparison, controlled experimentation, performance challenge, risk appetite review, approval evidence and safe model replacement.

The practical test is to trace one item from champion model through no-customer-impact rule and into the business use. If the team cannot show that path without guesswork, the model pack is not mature enough for serious banking reliance.

Banking practice note: implementation evidence

For champion challenger governance, implementation evidence is not optional detail. It decides whether the bank can explain the model, defend the use and protect the customer. A model may be fast and accurate in a narrow test, but banking approval depends on whether the result is suitable for model comparison, controlled experimentation, performance challenge, risk appetite review, approval evidence and safe model replacement.

The practical test is to trace one item from challenger model through parallel-run governance and into the business use. If the team cannot show that path without guesswork, the model pack is not mature enough for serious banking reliance.

Banking practice note: monitoring threshold

For champion challenger governance, monitoring threshold is not optional detail. It decides whether the bank can explain the model, defend the use and protect the customer. A model may be fast and accurate in a narrow test, but banking approval depends on whether the result is suitable for model comparison, controlled experimentation, performance challenge, risk appetite review, approval evidence and safe model replacement.

The practical test is to trace one item from business objective through metric definition and into the business use. If the team cannot show that path without guesswork, the model pack is not mature enough for serious banking reliance.

Banking practice note: override review

For champion challenger governance, override review is not optional detail. It decides whether the bank can explain the model, defend the use and protect the customer. A model may be fast and accurate in a narrow test, but banking approval depends on whether the result is suitable for model comparison, controlled experimentation, performance challenge, risk appetite review, approval evidence and safe model replacement.

The practical test is to trace one item from test population through fairness comparison and into the business use. If the team cannot show that path without guesswork, the model pack is not mature enough for serious banking reliance.

Banking practice note: complaint feedback

For champion challenger governance, complaint feedback is not optional detail. It decides whether the bank can explain the model, defend the use and protect the customer. A model may be fast and accurate in a narrow test, but banking approval depends on whether the result is suitable for model comparison, controlled experimentation, performance challenge, risk appetite review, approval evidence and safe model replacement.

The practical test is to trace one item from parallel run output through statistical significance review and into the business use. If the team cannot show that path without guesswork, the model pack is not mature enough for serious banking reliance.

Banking practice note: customer harm

For champion challenger governance, customer harm is not optional detail. It decides whether the bank can explain the model, defend the use and protect the customer. A model may be fast and accurate in a narrow test, but banking approval depends on whether the result is suitable for model comparison, controlled experimentation, performance challenge, risk appetite review, approval evidence and safe model replacement.

The practical test is to trace one item from holdout sample through validation challenge and into the business use. If the team cannot show that path without guesswork, the model pack is not mature enough for serious banking reliance.

Banking practice note: regulatory evidence

For champion challenger governance, regulatory evidence is not optional detail. It decides whether the bank can explain the model, defend the use and protect the customer. A model may be fast and accurate in a narrow test, but banking approval depends on whether the result is suitable for model comparison, controlled experimentation, performance challenge, risk appetite review, approval evidence and safe model replacement.

The practical test is to trace one item from performance metric through rollback plan and into the business use. If the team cannot show that path without guesswork, the model pack is not mature enough for serious banking reliance.

Banking practice note: third-party dependency

For champion challenger governance, third-party dependency is not optional detail. It decides whether the bank can explain the model, defend the use and protect the customer. A model may be fast and accurate in a narrow test, but banking approval depends on whether the result is suitable for model comparison, controlled experimentation, performance challenge, risk appetite review, approval evidence and safe model replacement.

The practical test is to trace one item from fairness result through approval committee and into the business use. If the team cannot show that path without guesswork, the model pack is not mature enough for serious banking reliance.

Banking practice note: change control

For champion challenger governance, change control is not optional detail. It decides whether the bank can explain the model, defend the use and protect the customer. A model may be fast and accurate in a narrow test, but banking approval depends on whether the result is suitable for model comparison, controlled experimentation, performance challenge, risk appetite review, approval evidence and safe model replacement.

The practical test is to trace one item from stability result through implementation control and into the business use. If the team cannot show that path without guesswork, the model pack is not mature enough for serious banking reliance.

Banking practice note: incident response

For champion challenger governance, incident response is not optional detail. It decides whether the bank can explain the model, defend the use and protect the customer. A model may be fast and accurate in a narrow test, but banking approval depends on whether the result is suitable for model comparison, controlled experimentation, performance challenge, risk appetite review, approval evidence and safe model replacement.

The practical test is to trace one item from override record through post-deployment monitoring and into the business use. If the team cannot show that path without guesswork, the model pack is not mature enough for serious banking reliance.

Banking practice note: retirement criteria

For champion challenger governance, retirement criteria is not optional detail. It decides whether the bank can explain the model, defend the use and protect the customer. A model may be fast and accurate in a narrow test, but banking approval depends on whether the result is suitable for model comparison, controlled experimentation, performance challenge, risk appetite review, approval evidence and safe model replacement.

The practical test is to trace one item from customer-impact analysis through experiment approval and into the business use. If the team cannot show that path without guesswork, the model pack is not mature enough for serious banking reliance.

Banking practice note: legal classification

The practical test is to trace one item from validation finding through traffic restriction and into the business use. If the team cannot show that path without guesswork, the model pack is not mature enough for serious banking reliance.

Banking practice note: model purpose

The practical test is to trace one item from approval condition through no-customer-impact rule and into the business use. If the team cannot show that path without guesswork, the model pack is not mature enough for serious banking reliance.

Banking practice note: customer population

The practical test is to trace one item from deployment recommendation through parallel-run governance and into the business use. If the team cannot show that path without guesswork, the model pack is not mature enough for serious banking reliance.

Banking practice note: source authority

The practical test is to trace one item from champion model through metric definition and into the business use. If the team cannot show that path without guesswork, the model pack is not mature enough for serious banking reliance.

Banking practice note: training data

The practical test is to trace one item from challenger model through fairness comparison and into the business use. If the team cannot show that path without guesswork, the model pack is not mature enough for serious banking reliance.

Banking practice note: feature governance

The practical test is to trace one item from business objective through statistical significance review and into the business use. If the team cannot show that path without guesswork, the model pack is not mature enough for serious banking reliance.

Banking practice note: protected-class testing

The practical test is to trace one item from test population through validation challenge and into the business use. If the team cannot show that path without guesswork, the model pack is not mature enough for serious banking reliance.

Banking practice note: proxy-variable review

The practical test is to trace one item from parallel run output through rollback plan and into the business use. If the team cannot show that path without guesswork, the model pack is not mature enough for serious banking reliance.

Banking practice note: adverse action reasons

The practical test is to trace one item from holdout sample through approval committee and into the business use. If the team cannot show that path without guesswork, the model pack is not mature enough for serious banking reliance.

Banking practice note: reason-code mapping

The practical test is to trace one item from performance metric through implementation control and into the business use. If the team cannot show that path without guesswork, the model pack is not mature enough for serious banking reliance.

Banking practice note: explainability limits

The practical test is to trace one item from fairness result through post-deployment monitoring and into the business use. If the team cannot show that path without guesswork, the model pack is not mature enough for serious banking reliance.

Banking practice note: independent validation

The practical test is to trace one item from stability result through experiment approval and into the business use. If the team cannot show that path without guesswork, the model pack is not mature enough for serious banking reliance.

Banking practice note: effective challenge

The practical test is to trace one item from override record through traffic restriction and into the business use. If the team cannot show that path without guesswork, the model pack is not mature enough for serious banking reliance.

Banking practice note: approval committee

The practical test is to trace one item from customer-impact analysis through no-customer-impact rule and into the business use. If the team cannot show that path without guesswork, the model pack is not mature enough for serious banking reliance.

Banking practice note: documentation quality

The practical test is to trace one item from validation finding through parallel-run governance and into the business use. If the team cannot show that path without guesswork, the model pack is not mature enough for serious banking reliance.

Banking practice note: implementation evidence

The practical test is to trace one item from approval condition through metric definition and into the business use. If the team cannot show that path without guesswork, the model pack is not mature enough for serious banking reliance.

Banking practice note: monitoring threshold

The practical test is to trace one item from deployment recommendation through fairness comparison and into the business use. If the team cannot show that path without guesswork, the model pack is not mature enough for serious banking reliance.

Compare on the same decision clock

Run a candidate fraud model in shadow beside the approved champion on the same eligible payments and point-in-time features. Compare coverage, latency, calibration where labels mature, action simulations and subgroup effects. A challenger evaluated on corrected future-complete data has an unfair advantage. The actual outcome still reflects champion policy, especially for held payments whose settled loss is not observed.

Promotion requires an approved experiment scope, stop criteria, capacity plan and rollback owner. A 5% traffic test should retain transaction IDs, assigned arm, score versions, policy actions and final states. Check false holds and investigations as well as confirmed fraud. If an upstream feature changes during the test, pause interpretation and separate cohorts. Document why the challenger is better for the bank's use, not only for a ranking metric.