How models are trained using banking data

Plain language meaning

How models are trained using banking data explains how approved historical records, labels, features, outcomes and validation samples become a model, and why the bank must control relevance, representativeness, leakage, bias, documentation and validation before production use.

This topic is about controlled training using banking data. It is not about feeding every available customer record into a model or confusing experimental accuracy with production readiness.

In a real bank, this topic cannot be handled as a loose data-science or technology idea. It affects customer outcomes, fraud and AML control, operational queues, service continuity, privacy, security, model governance, audit replay, management reporting and regulatory confidence. AI should improve speed and quality, but the bank must still prove source data, permitted use, approved logic, human accountability, fallback handling and retained evidence.

Where it sits in the banking AI journey

This card belongs to AI Data and Model Operations. The working flow is Approved history, Label and feature design, Training and validation, Model approval, and Production candidate.

Read the flow as a bank operating model. Each stage needs a source system, a data owner, a timing rule, a quality gate, a model or rule boundary, an exception path, a customer-impact view, a fallback option, a monitoring requirement and a retained record. That is what separates useful AI adoption from uncontrolled automation.

Banking data and evidence

The important data points are historical event, outcome label, feature value, training window, validation sample, holdout result, bias test, and model artefact. These items matter because they can influence risk scoring, operational repair, fraud action, AML triage, customer treatment, reporting, model monitoring and management decisions.

The evidence pack should include training data record, label definition, feature list, validation report, bias analysis, model card, and approval pack. A strong bank can replay the journey from source data to transformed input, AI output, rule result, human action, system outcome and monitoring result. A weak bank only knows that a process ran and hopes the process was right.

Controls that make AI adoption safe

The core controls are training-data approval, label definition, leakage test, sample representativeness, independent validation, documentation, and approval workflow. These controls keep the topic anchored to banking purpose, approved policy, data governance, model-risk expectations, operational resilience, customer fairness, privacy, security and auditability.

The practical design should define what AI may recommend, what it must never decide alone, which deterministic rule remains authoritative, who owns thresholds and overrides, how degraded service is handled, how customer harm is detected and what evidence is retained. Without that control design, faster AI can simply make weak processes fail faster.

Architecture and data-operation lens

Banking AI depends on the architecture around it. Storage, streams, feature definitions, training sets, model versions, thresholds, feedback labels and rollback paths must be governed before the bank relies on AI output. The model is only one part of the control chain.

A bank-grade design connects channels, source systems, core records, payment hubs where relevant, fraud systems, AML platforms, case tools, data platforms, feature stores, model-serving endpoints, policy engines, audit logs and management dashboards. It also records degraded operation, recovery actions and lessons learned.

Regulatory and governance lens

Federal Reserve SR 26-2, dated 17 April 2026, gives revised model-risk guidance for traditional models and non-generative AI models used by banking organisations, including development, validation, monitoring, change control and governance.

The Federal Reserve's 2026 model-risk guidance states that generative and agentic AI are outside that guidance, while broader bank risk-management and governance practices still need to control tools and processes not covered by the guidance.

NIST AI RMF 1.0 uses Govern, Map, Measure and Manage functions for AI risk management, and NIST AI 600-1 adds generative-AI risk actions for source grounding, content provenance, data protection, cybersecurity and human oversight.

FFIEC Architecture, Infrastructure and Operations guidance expects financial-institution technology environments to be governed, resilient, secure, monitored, documented and aligned to business risk, including emerging technologies such as artificial intelligence and machine learning.

BCBS 239 remains current for effective risk data aggregation and risk reporting, and the Basel Committee's January 2026 newsletter reiterates the importance of accurate, comprehensive and timely data capabilities in banks.

FFIEC BSA/AML examination guidance expects suspicious activity monitoring systems and supporting technology to be risk-based, explainable by management, independently tested where appropriate and aligned to the bank's risk profile.

FinCEN's 12 June 2026 Section 314(b) materials clarify information sharing for possible terrorist activity, money laundering and fraud-related specified unlawful activity within the statutory safe-harbor framework for participating financial institutions.

OFAC's Framework for Compliance Commitments describes sanctions compliance programme components including management commitment, risk assessment, internal controls, testing and auditing, and training.

Diagram walkthrough

Read the diagram from left to right as Approved history, Label and feature design, Training and validation, Model approval, and Production candidate. It is a banking control map. The point is to show how data, AI or ML output, rules, human action, operational routing and audit evidence should connect.

Use it as a 30-minute study method. For each box, ask which system creates the data, which definition is used, which model or rule acts, what can go wrong, who can override it, how a fallback works, which customer or regulatory impact exists and what record proves the final state.

Most important mistake to avoid

The common failure is celebrating model accuracy while ignoring whether the training data was lawful, relevant, time-correct, representative, free from leakage and aligned to the banking decision that the model will support.

The correction is disciplined scope. Keep the chapter anchored to banking purpose, prove the data path, make ownership visible, test failure behaviour, record the evidence and make the final outcome explainable without relying on memory, assumptions or developer-only knowledge.

A training cohort with an observable outcome

A bank wants to train a credit-risk model on applications decided in a defined year. It first specifies the target, such as a stated default event within twelve months, and determines which accounts have a full observation window. Treating recent accounts without twelve months of history as non-defaults would bias the labels. The development record should state eligible population, exclusions, outcome source, observation dates and policy changes that affected who was approved.

Features must be reconstructed as they stood at each application decision. A later collections note or account closure is evidence of an outcome, not a valid input to an earlier application. Split data by time and, where appropriate, entity so linked records do not leak across development and test sets. Compare the model with a relevant incumbent on the same population, then test discrimination, calibration, segment behavior and operational use. A model with a better aggregate metric may still fail a small but important population.

The training artifact should identify the data snapshot, code, feature definitions, parameters, validation result and approval. A later refresh is a new version requiring its own comparison and change control. A business owner also needs an answer to what happens when an applicant has missing data or falls outside the training population. The revised U.S. Federal Reserve SR 26-2 describes risk-based model governance for covered banking organisations; applicability to a specific institution and use must be checked rather than assumed.

Start with the decision to be supported

Training a banking model is not simply feeding historical tables to an algorithm. The team must define the decision, eligible population, observation time, outcome, permitted data, baseline policy and how the model will be used. A payment fraud model scores an instruction before release. A credit model may estimate future default risk at application. An AML model may prioritize existing alerts for review. These targets have different labels, time horizons, selection effects and consequences.

Write the intended use in operational terms. Which product and channel are in scope? Does the model recommend, rank, refer or trigger an automated action under policy? What is the latency budget? What evidence can be used at decision time? Which cases are excluded or sent to humans? A model trained on consumer card purchases should not be assumed valid for corporate cross-border payments merely because both are called transactions.

The training data should reflect this intended population. Count eligible records by period and segment, document exclusions and compare to live traffic. A convenience extract of only completed cases may omit failed payments, referred applications or unresolved alerts. The resulting model can look accurate while being evaluated on a population shaped by prior decisions.

Define the target carefully

Fraud labels can come from confirmed investigations, chargebacks or customer reports. Those sources differ in delay and certainty. A disputed card charge is not always confirmed fraud; a blocked transaction may never have an observable downstream loss. Choose a label definition and maturation window, preserve revisions and distinguish unresolved cases. Report performance only on cohorts whose outcomes had time to develop.

Credit default needs a facility, horizon and event definition. Delinquency, write-off, restructuring and cure may be treated differently by product and policy. An applicant rejected by the bank does not produce a repayment outcome on the proposed loan, creating selection bias. Treating unobserved applicants as good or bad without justification can distort training. A credit risk owner should approve label and population definitions.

An AML case closure is not necessarily a ground-truth statement that activity was lawful. Investigators may close for insufficient evidence or limited capacity. A model trained to replicate historical dispositions can learn past review patterns and workload. Define whether the target is investigator priority, useful case outcome or another measurable goal, and acknowledge imperfect labels. Do not equate a model score with a legal determination.

Build point-in-time features

For every training row, choose an observation timestamp and construct features from information available then. A payment at 10:00 cannot use a case disposition from 15:00. A credit application cannot use future repayments on the requested loan. A current customer master may contain a corrected relationship learned after the original decision. Historical joins should respect effective and recorded time, with explicit source cutoffs.

Features need documented units, windows and eligible events. A seven-day transfer count may include accepted instructions but exclude technical retries; a monthly income estimate may exclude transfers between the customer's own accounts. A bureau attribute has observation and receipt dates. Missingness states should distinguish no history, delayed feed, failed match and true zero. Training and serving must implement equivalent definitions under realistic data availability.

Leakage can be subtle. A source code might be assigned only after an investigator confirms fraud. An account status might change after collections begins. A policy-assistant evaluation set might contain the answer in a document added after the question's historical date. Investigate unusually powerful features and hand-replay sample rows. Good validation metrics do not excuse impossible timing.

Assemble datasets and splits

Version the source extract, transformations, feature definitions, label rules, cohort and quality checks. Reconcile row counts before and after joins; a one-to-many customer mapping can duplicate applicants and overweight them. Measure missingness and coverage by product, channel and customer group. Preserve the dataset manifest so an independent reviewer can reproduce the experiment.

Split by time and entity where appropriate. Randomly dividing transactions from the same customer or fraud ring between training and validation can overstate generalization. A later-period holdout tests whether the model survives shifts in behavior and policy. Keep final test data separate from repeated tuning decisions. For rare events, report uncertainty and confidence intervals rather than relying on a single flattering metric.

Outcome maturation affects the split. A payment from last week may not yet have a resolved dispute. A credit application from six months ago may not have a full one-year default horizon. Exclude or handle immature rows according to a predeclared rule. Otherwise a model can appear to improve because newer outcomes are missing.

Banking data can change with economic conditions and operational policies. Use validation periods that include relevant shifts if available, and stress scenarios where historical coverage is limited. A model trained during calm conditions may miscalibrate in a downturn. A fraud pattern can evolve adversarially. Explain limitations of the available history rather than claiming universal performance.

Baselines and model choice

Compare the candidate to a meaningful baseline: current rules, existing model, human queue or a simpler statistical model. Report what changes in final actions, not just a score metric. A complex model may improve area under a curve while increasing false holds or failing under source outages. A simpler model with stable inputs and clear monitoring can be preferable for a particular decision.

Choose an algorithm appropriate to data, volume, latency and explanation requirements. Tree models can handle tabular banking features; sequence and graph methods may capture temporal or network patterns but add data and validation complexity. Generative models can summarize or retrieve policy information but require source grounding and review. The algorithm is one component of a controlled decision system.

Hyperparameter tuning should use a validation design that respects time and entity boundaries. Record search space, data versions and selection criteria. Repeatedly adjusting on the final test set makes it no longer an independent test. Consider class imbalance, but do not confuse resampling for training with the true prevalence at deployment. Calibrate probabilities on representative data.

Evaluate the operating point

For fraud, precision and recall depend on the threshold and underlying prevalence. A high recall can overwhelm analysts or block customers. Examine losses caught, false holds, time to review and outcomes by transaction value and segment. Delayed confirmations and prevented transactions make true performance uncertain. Report what is observed and what remains unobservable.

For credit, assess discrimination and calibration, approval or referral rates, expected losses and errors across relevant groups and time. A rank-order metric alone does not show whether predicted default probabilities are well calibrated or whether policy outcomes are fair. Include affordability and other rules in evaluation. A model score is not the final credit decision.

For AML prioritization, measure whether analysts see useful cases earlier, workload effects, missed high-severity cases and time to disposition. Historical case labels can reflect prior prioritization. A reduction in reviewed false positives may be useful, but it must not hide unreviewed risk or mandatory alerts. Validate with domain experts and controlled sampling of low-ranked cases.

Fairness and privacy

Assess performance and decision effects by relevant customer segments under appropriate legal and privacy governance. Missing source data may be concentrated in thin-file, branch or new-to-bank populations. A model can amplify prior policy patterns even if protected attributes are not in its input. Review proxies, error rates, referral burden and availability of meaningful human review.

Use only data authorized for the purpose and restrict training extracts. A feature that improves offline prediction may be inappropriate because of privacy, contractual constraints or poor explainability. Minimize raw data copies, log access and set retention. A model trained on sensitive case narratives needs a specific risk assessment for memorization and downstream disclosure.

Human labels can be inconsistent. Investigators, underwriters and case workers may apply different standards over time. Sample labels for quality, document policy changes and avoid treating every override as ground truth. An override may reflect new information or policy rather than a model error. Feed corrections into the right source or label process.

Validation and independent challenge

Validation should review conceptual soundness, data provenance, feature timing, methodology, implementation, outcomes and limitations. Reproduce key metrics on the exact dataset and artifact proposed for use. Challenge leakage, segmentation, stability and adverse scenarios. Test serving inputs and fallback behavior, not only notebook predictions. Record findings, owners and conditions of approval.

Backtesting uses historical data, but history may not represent the new policy. A fraud model deployed with more holds changes which transactions settle and receive labels. A credit policy changes which applicants become borrowers. A model that ranks AML cases changes investigation effort. Evaluation should state these selection effects and plan post-deployment monitoring or carefully governed experiments where appropriate.

Document what the model should not do. A fraud score should not override mandatory screening. An AML rank should not auto-clear required cases. A generated assistant answer should not become an unreviewed binding policy interpretation. Limits should be enforced in orchestration and monitored, not left only in a model card.

Deployment

Register the approved model artifact with dataset, code, feature versions, validation report and owner. Test the exact serving path against representative inputs, including missing, stale and malformed features. Shadow scoring can compare a candidate with the current model without changing customer actions. A staged rollout can limit exposure, but it needs clear monitoring and rollback criteria.

Record model response and policy action separately. A score alone does not determine whether a payment was held or a credit application declined. Keep input validity, model version, threshold version, fallback state, human review and final outcome linked to the business decision. An independent reviewer should be able to reconstruct an ordinary and a failure case.

Deployment may change feature definitions or source mappings. Compare online values with point-in-time offline calculations on the same decision IDs. A green API response is insufficient if the live feature distribution differs from training because of a source migration. Block or limit use under the approved policy when critical inputs are invalid.

Monitoring and retraining

Monitor source completeness, feature freshness, missingness, score distribution, calibration as labels mature, action rates, overrides and customer or business outcomes. Segment by product, channel and relevant groups. A distribution shift can reflect real behavior, source defect or policy change. Investigate the cause before retraining. Retraining on corrupted data can institutionalize the defect.

Define triggers and owner for review: a source mapping change, sustained calibration drift, material segment harm, new product or economic regime. A candidate retrained model is a new artifact requiring testing and approval, not an automatic replacement. Keep the old version and compatible feature state for rollback where appropriate. Retain decisions made under each version.

Feedback loops can bias future labels. If a model holds more transactions, fewer suspicious payments settle and generate chargebacks. If a credit model rejects a group, repayment outcomes become sparse for that group. Monitor eligible, scored, acted-on and observable populations. Use domain review and robust evaluation methods rather than interpreting a lower observed loss rate as proof of improvement.

Fraud training example

Build a cohort of card authorizations with stable transaction IDs and decision timestamps. Calculate pre-authorization features such as recent distinct merchant count, amount deviation and device-change status from sources available then. Define a later confirmed-fraud outcome with a maturation period. Preserve declined authorizations as a separate population with limited outcome observability. Split by time and customer or linked fraud entity to reduce leakage.

Compare current rules, a simple baseline and the candidate at several workload levels. Evaluate confirmed losses caught, legitimate authorizations interrupted, analyst capacity and segment effects. Test live feature availability under a delayed event stream. A model with slightly better offline recall but frequent stale-input fallbacks may not improve actual banking outcomes.

Credit training example

Build application-time rows with verified identity, bureau response as received, and cash-flow features from a defined lookback. Document joint accounts, irregular income, disputed bureau data and missing source states. Define default over an appropriate horizon for funded facilities and acknowledge the unobserved outcome of rejected applicants. Use later-period validation and examine calibration and policy outcomes by product and segment.

If a bureau adapter changes a field meaning, compare old and new feature values and decisions before rollout. A model trained on the old semantics may not be valid on the new feed. The underwriting policy and human review remain distinct from the predicted probability. Retain original application and corrected-source evidence for disputes.

Reproducibility exercise

Select one training row, trace its source records and timestamps, calculate its features by hand, identify its later label and explain why that label was mature. Then locate the same definitions in the serving path and compare a real production request. Repeat with a rejected application or blocked payment, where the outcome is not observed in the same way. Document the selection limitation.

A training process is ready for banking use when the data, target and decision timeline are coherent; the evaluation reflects actual operations and uncertainty; the artifact and inputs are versioned; and the bank can detect and correct failures after deployment. A high offline metric is one piece of that evidence.

Sampling and imbalance

Fraud and some compliance outcomes are rare. Training on every negative record may be costly, so a team may sample negatives or weight classes. Record the sampling design and restore real-world prevalence when evaluating probabilities and operational thresholds. A model can rank cases well on an artificially balanced test but have poor precision in production. Report metrics on an untouched representative holdout in addition to any sampled training set.

Sampling must preserve important slices. High-value cross-border transfers, new accounts and particular channels may be rare but material. Stratified sampling can ensure enough examples for analysis, yet the reported aggregate must reflect the real population. Avoid drawing multiple near-identical transactions from the same fraud campaign into both train and test. Group linked entities or episodes where relevant.

If confirmed outcomes are delayed, recent negative examples may be mislabeled because investigations are open. Use a mature observation window, a censored or unresolved category, or another justified method. Document how many records were excluded and whether the exclusion changes population mix. A lower apparent fraud rate in the latest month can simply mean labels have not arrived.

Calibration and thresholds

Many operational policies interpret a model score as a probability or risk ranking. Test calibration: among comparable cases assigned an estimated 10 percent risk, is the observed mature outcome approximately consistent, subject to selection and uncertainty? Examine calibration by period and segment. A well-ranked model can still be poorly calibrated, especially after prevalence changes.

Threshold choice should be evaluated with the policy owner using actual costs, capacity and customer effects. For fraud, raising the hold threshold may reduce false holds but miss losses; lowering it can overwhelm investigators. For credit, referral capacity and affordability rules influence final decisions. For AML, ranking may determine review order without clearing alerts. Keep threshold and model versions separate so an action-rate change can be traced to the right cause.

Do not optimize a threshold against an after-the-fact label as if every past action had the same observable outcome. Declined payments and rejected loans have different outcome paths. Simulate with assumptions and bounds, and where lawful and appropriate use controlled monitoring to learn from decisions. Report the limitation to business owners instead of presenting a single precise benefit figure.

Explainability and reason generation

The bank may need to explain individual outcomes and challenge a model's behavior. Choose methods appropriate to the model and use case, and validate them against actual inputs. A feature contribution is not automatically a causal reason. Correlated variables can make local explanations unstable. A generated natural-language reason can be persuasive but inaccurate if it is not tied to the final policy path.

For a credit decision, distinguish model factors from deterministic affordability or eligibility rules and underwriter judgment. A decline caused by an independent policy rule should not be attributed to the model's highest feature contribution. For a fraud hold, a risk score may combine velocity and device behavior while a separate screening hold remains mandatory. The decision record should preserve the actual action reason.

Test explanations on corrected inputs, missing values and model versions. An independent reviewer should be able to reproduce the output or identify its limitations. Do not expose another customer's confidential network information in a customer-facing explanation. Maintain a protected detailed record and an appropriate communication route.

Stress and robustness tests

Stress test economic and operational shifts that the training set may underrepresent: a downturn, rapid payment-product adoption, changed fraud tactics, new customer cohort, source migration or vendor outage. Evaluate how missing or stale features affect the model and decision policy. A model that returns a number for malformed input has not necessarily handled it safely.

Probe sensitivity to plausible input errors. If a small change in reported income or account age causes a large decision change, inspect calibration, policy and data validation around that boundary. For fraud, test repeated technical retries, split payments and a burst against one beneficiary. For text models, test OCR errors, conflicting versions and unsupported answers. Record failure cases and the approved action when the model abstains.

Measure uncertainty in estimated performance. Rare losses can yield wide confidence intervals. A stress period may have few mature labels. State what the test can and cannot support. Validation should not turn an absence of observed failures in a small sample into proof of safety.

Experiment governance

Keep a structured experiment record: question, dataset and code versions, feature set, model configuration, split, metrics, thresholds, segment results, limitations and decision. Separate exploration from the final validation evidence. If many candidates were tried, disclose how the final one was selected and avoid claiming that the best observed test score is an unbiased estimate.

Reproducibility includes random seeds but is broader than them. Data extracts can change as sources are corrected; library and accelerator behavior can change; nondeterministic training can produce small differences. Preserve sufficient artifacts and environment information to reproduce material conclusions. An independent validator should be able to run key checks and compare outputs within stated tolerance.

Document failed experiments where they reveal limitations. If a graph feature was dropped because entity resolution was weak, retain that finding. If a model performed poorly for new customers, it should not be silently deployed to them later through a channel expansion. A model inventory should carry scope and exclusions into production controls.

Human feedback quality

Underwriter overrides, fraud case outcomes and AML analyst dispositions can inform improvement, but they are not interchangeable labels. Record reason and supporting evidence. An analyst may override because a document arrived after the original score, because policy demanded a referral or because the model relied on stale data. Only some of these indicate model error. Review a sample of feedback with domain owners before using it for retraining.

Reviewer workload influences labels. A large queue can lead to shorter investigations or different closure codes. A model that sends a particular segment to review more often can create more observed adverse labels there. Track review intensity and time alongside outcome. Avoid using an operational disposition as if it were a neutral truth independent of prior model decisions.

Design a feedback correction route. If the source income field is wrong, fix source data and affected features. If the label taxonomy is inconsistent, clarify it and revise datasets. If the policy threshold is inappropriate, change policy through governance. Retraining a model is only one possible response.

Post-release cohort analysis

Group decisions by model and policy version, product, channel and observation period. Report eligible population, scored population, fallbacks, final actions and mature outcomes. Compare with the approved baseline, accounting for differences in traffic. A rise in fraud catches may accompany more false holds; a lower observed default rate may reflect fewer approvals. Include both benefits and burdens.

Track performance over time with label maturity. A recent cohort can show source quality, score distribution and referral rates while confirmed losses remain incomplete. Revisit it when outcomes arrive. A monitoring cadence should reflect the decision's consequence and outcome lag, not a generic monthly dashboard ritual.

If a material issue appears, identify the affected decisions from artifact and feature versions. Determine whether a source defect, model calibration change or policy rule caused it. Invoke the approved fallback or rollback if needed, preserve original records and review customer impact. This closes the training lifecycle: historical learning must connect to accountable live operations.

Review a candidate end to end

Ask the team to pick a randomly selected live decision from the rollout period. Rebuild its features from source records available at the time, identify its model artifact and policy version, and verify the final business action. Then find the training cohort and validation evidence supporting that type of request. If the request belongs to an excluded segment or uses a new feature meaning, the original validation may not cover it.

Repeat with a failed feature lookup and an overridden case. Show how those paths enter monitoring and later evaluation without turning a missing score into zero or an override into an automatic ground-truth label. A model is operationally trained only when its data definitions, validation and live decision evidence form a coherent chain.

A compact training approval record

The approval record should state the target population, decision time, allowed inputs, label definition, maturation horizon, dataset version, split method, baseline and candidate metrics, calibration, segment analysis, stress results and expected business effects. It should identify validation findings and the policy that turns a score into an action. It should also name the owner, monitoring thresholds, fallback, rollback and review date. A link to a notebook without those conclusions is not enough for an operating decision.

Attach examples of an ordinary case, a high-risk case, an excluded case, an input failure and a human override. For each, record what the model actually returned and what the bank actually did. The examples make scope and control boundaries visible. They also become regression cases when a source, feature or policy changes. Preserve the input validity state and timestamp for each example so a reviewer can distinguish a genuine zero from a missing source. Record whether the outcome was mature enough for evaluation.

Before approval, ask a reviewer to challenge one impressive metric. Could it be explained by leakage, duplicate customers, late outcomes, historical policy selection or a concentrated segment? Calculate a simpler baseline and inspect the confusion or ranking behavior at the intended workload. If the result remains useful under that challenge, the model has earned stronger evidence than a headline score. If not, revise the data and evaluation before exposing customers to it. Document the exact holdout cohort and label-maturity cutoff so the challenge can be repeated independently. Include unresolved outcomes and the share of intended live traffic outside the training population. Both limit how broadly the validation result applies.

Label maturity test

Build a fraud training cohort of payment authorizations from January. Features use only events available before authorization; confirmed fraud outcomes are observed through a declared later date. Do not label a February payment legitimate merely because its investigation remains open. Keep held transactions as a separate selectively observed population. Split by time and linked customer or attack episode to limit near-duplicate leakage.

Compare candidate and current policy on representative traffic with real prevalence and analyst capacity. Record the source extract, feature definitions, label version and maturity cutoff. An independent reviewer should hand-replay a high-score case, a low-score confirmed fraud and an unscored fallback. If a feature was populated after the original decision, remove it and rerun validation. A headline accuracy number cannot compensate for timing leakage or missing outcome coverage. Check that a rejected loan application is not labeled a good repayment outcome and that a blocked payment is not automatically called fraud. Report the mature, unresolved and unobservable populations separately. A candidate trained on selected prior decisions needs its limitation documented before a new threshold expands its use.