Test implementation first, assess conceptual and data limitations separately, and compare challengers on frozen inputs with predeclared review criteria. Preserve disagreement and restrict interpretation where evidence does not support the intended use.
Define the claim being validated
Write a short intended-use statement before reviewing model performance. Identify the instruments, scenario types, horizons, and decisions the output is meant to inform. A linear approximation for simple holdings may be adequate for an educational sensitivity exercise but unsuitable for a portfolio dominated by contingent payoffs. Validation should assess the stated claim rather than reward a method for producing a number in every case.
The Federal Reserve's revised 2026 model-risk guidance discusses model purpose, materiality, effective challenge, validation, and monitoring. It is banking guidance, not a certification program for this illustrative tool. Its useful conceptual distinction is that even a sound model can be misused. In this guide, review intensity follows the potential consequence of an incorrect interpretation, without claiming that particular supervisory obligations apply.
Separate implementation evidence from economic evidence
Implementation checks establish that formulas, units, signs, data handling, and aggregation behave as specified. Use small fixtures with independently calculable results, including zero shocks, missing inputs, short positions where supported, and currency conversions. Confirm that unsupported inputs are rejected or visibly excluded. A passing test is evidence about those behaviors, not evidence that an estimated sensitivity or scenario narrative is economically reliable.
Economic validation examines why the representation should work for the intended holdings and conditions. Inspect calibration data, proxy suitability, stability across states, and sensitivity to assumptions. Distinguish independent observations from repeated calculations on the same sample. Running more software tests does not create additional market evidence, and duplicating a calculation through a different interface does not create an independent economic challenger.
Hypothetical benchmark: expose approximation error
Consider invented zero-coupon claims that each pay USD 100, one after one year and the other after two years. At an initial annual yield of zero, each is worth USD 100. Holding the valuation date fixed, change the yield to 5%. Exact discounted values are 100 divided by 1.05 and 100 divided by 1.05 squared, approximately USD 95.238095 and USD 90.702948.
A duration-only expansion at the initial zero yield gives USD 95 and USD 90 because the initial modified durations are one and two. The absolute pricing differences are approximately USD 0.238095 and USD 0.702948. These hypothetical fixtures quantify approximation error under explicitly known cash flows. They do not validate an investment strategy, forecast market yields, or establish accuracy for instruments with optionality or uncertain cash flows.
| Hypothetical claim under 5% yield | Exact discounted value | Duration-only value | Absolute difference |
|---|---|---|---|
| USD 100 in one year | USD 95.238095 | USD 95.000000 | USD 0.238095 |
| USD 100 in two years | USD 90.702948 | USD 90.000000 | USD 0.702948 |
Design a challenger that can disagree meaningfully
A useful challenger changes a consequential assumption or method while preserving a comparable question. Examples include direct cash-flow repricing against a duration approximation, an alternative factor mapping, or a different but justified liquidity horizon. Specify in advance what disagreement will trigger investigation. Avoid selecting a challenger only after seeing which method produces the preferred loss total or the most reassuring portfolio ranking.
Record shared dependencies. Two methods using the same incomplete holdings and the same stale prices can agree for the same wrong reason. Independence is a matter of evidence and design, not just a different software package. For each comparison, state what is independent, what is held common, and which failure modes both methods still share despite their different mathematical implementations.
Investigate differences at the right level
Compare results by instrument group and scenario component before examining only the total. Offsetting errors can produce agreement at portfolio level while leaving individual positions badly misrepresented. Trace differences to inputs, formulas, calibration, or contract treatment. Preserve signed differences and their materiality. Do not average incompatible methods merely to reduce the appearance of disagreement, since the average may have no defensible interpretation.
Choose acceptance criteria that reflect intended use and the scale of the exposure. An absolute tolerance can be useful for reconciliation, while a relative tolerance can become misleading near zero. Document both where necessary. If the method remains unsuitable for a material position, narrow the claimed coverage or mark the result as incomplete. A prominent limitation is preferable to a falsely comprehensive total.
Review scenarios and ongoing relevance
The Basel Committee's stress-testing principles place methodology and documentation within a broader framework of objectives and oversight. For an illustrative portfolio analysis, the reviewer can apply that idea by checking whether the scenario set addresses the stated vulnerabilities. A mathematically correct engine can still miss the relevant risk if all scenarios preserve favorable offsets or omit the factor most important to a concentrated holding.
Monitoring should identify changes that weaken previous evidence: new instruments, missing data, revised mappings, larger shock sizes, or a different use of the output. Retain the scope of each review and the conditions that require another one. A validation note from an earlier version should not silently cover a newly introduced product or a shift from educational illustration to consequential decision support.
Conclude with a scoped disposition
The review outcome should distinguish verified implementation, supported use, unresolved limitations, and prohibited interpretations. Identify who owns each issue and what evidence would close it. A challenger disagreement can remain unresolved while the tool is usable for a narrower purpose, but that boundary must be explicit. Avoid a single green status that conceals material exclusions or dependence on an untested assumption.
Preserve inputs, versions, test results, and the reasoning behind the conclusion. The final question is whether another reviewer could reproduce both the calculation and the decision about its use. This guide describes analytical discipline, not a regulatory approval process. Passing its hypothetical fixtures does not certify a bank stress engine, demonstrate future portfolio resilience, or provide current investment advice.
Review checkpoints
- Correct arithmetic does not establish economic suitability.
- A challenger should test a consequential assumption with clear independence boundaries.
- Validation conclusions must state scope, limitations, and conditions for review.
Source notes
Continue the analysis
Reading exposures that change as markets move
Recognize where delta-based estimates fail, distinguish payoff diagrams from valuation, and review nonlinear positions under joint shocks.
Coverage and Missing Positions in a Stress Test
Measure what a stress calculation includes, expose unsupported holdings, and interpret partial losses without silently treating missing risk as zero.
Keep stress-report revisions explainable and reproducible
Assign ownership, preserve report versions, bridge changed results, and prevent corrections or assumption changes from rewriting the record.
First edition, 16 October 2026. Hypothetical values are teaching inputs, not observed portfolio or provider results.