VaR Backtesting, Stress Testing and Model Limitations - Article

Introduction

Value at Risk converts market risk into a simple statement such as:

The portfolio's one-day 99% VaR is ₹X.

This is useful, but it immediately creates another question:

Can the model be trusted?

A VaR model may give an exact-looking number even when:

• market volatility changes sharply;

• correlations break down;

• historical data are unrepresentative;

• liquidity disappears;

• the portfolio contains complex options; or

• the model itself is incorrectly specified.

Banks therefore should not use VaR mechanically.

Three concepts must be understood together:

VaR → Backtesting → Stress Testing

• VaR estimates risk under specified model assumptions.

• Backtesting checks how the model performed against realised outcomes.

• Stress testing asks what could happen under severe conditions that normal VaR may not capture adequately.

The current 2026 IIBF CAIIB BFM syllabus includes Risk Measures: VaR and Duration under Module C – Treasury Risk Management.

RBI's market-risk guidance also requires regular backtesting and rigorous stress testing where internal market-risk models are used.

Learning Objectives

After studying this topic, you should be able to:

• explain the purpose of VaR backtesting;

• understand what a VaR exception means;

• distinguish actual P&L from hypothetical P&L backtesting;

• interpret excessive VaR exceptions;

• explain the purpose of stress testing;

• distinguish sensitivity, historical, hypothetical, and reverse stress tests;

• identify major limitations of VaR;

• understand model risk and validation; and

• explain why VaR, backtesting, and stress testing must be used together.

Why VaR Needs Validation

VaR is produced by a model.

Every model makes assumptions about matters such as:

• volatility;

• correlations;

• distribution of returns;

• holding period;

• market liquidity;

• historical behaviour; and

• valuation of instruments.

If those assumptions are poor, the VaR number can be misleading.

Therefore, risk management should not ask only:

“What is today's VaR?”

It should also ask:

• How accurate has the model been?

• How often have losses exceeded VaR?

• Are the data and assumptions still appropriate?

• What happens under extreme market shocks?

• Are important risks missing from the model?

What Is VaR Backtesting?

Backtesting compares the risk predicted by the VaR model with the profit or loss that subsequently occurred.

RBI describes backtesting as the routine comparison of daily profits and losses with model-generated risk measures to assess the quality and accuracy of the risk-measurement system.

The basic process is:

Calculate VaR

↓

Observe Subsequent Trading Outcome

↓

Compare Actual Loss with VaR

↓

Count Exceptions

↓

Assess Model Performance

What Is a VaR Exception?

A VaR exception occurs when the observed loss exceeds the VaR estimate for the corresponding period.

Suppose a bank uses:

1-day 99% VaR

If the day's loss is greater than the VaR amount, that day is a:

VaR exception

or

backtesting exception.

This does not automatically mean the model is defective.

At 99% confidence, some exceptions are expected.

Conceptually:

99% coverage → approximately 1% of observations may fall outside the VaR threshold

RBI's traditional VaR backtesting framework explains that a 99% daily risk measure is expected to cover approximately 99% of trading outcomes.

Backtesting Example Logic

Assume a treasury uses one-day VaR.

At the start of each trading day:

1. VaR is calculated.

2. The day's market outcome occurs.

3. The resulting P&L is determined.

4. The loss is compared with the VaR estimate.

5. Any loss exceeding VaR is recorded as an exception.

Over a sufficiently long observation period, risk management examines whether the number and pattern of exceptions are broadly consistent with the model's intended confidence level.

No detailed numerical computation is required to understand the concept.

Why One-Day VaR Is Commonly Used for Backtesting

A longer holding-period VaR is harder to compare directly with daily realised trading results because portfolio positions often change from day to day.

Traditional RBI internal-model guidance therefore describes backtesting using one-day risk measures to facilitate comparison with daily trading outcomes.

This allows repeated observations:

Day 1 VaR vs Day 1 P&L

Day 2 VaR vs Day 2 P&L

and so on.

Actual P&L Backtesting

One approach compares VaR with the bank's actual daily trading profit or loss.

Actual P&L may reflect:

• market movement;

• intraday trading;

• new transactions;

• closed positions;

• fees;

• spreads;

• other trading effects.

This tells management how the risk measure compares with the real trading outcome.

However, actual P&L may contain effects that were not present in the portfolio when the previous day's VaR was calculated.

This can make interpretation more difficult.

Hypothetical P&L Backtesting

A second approach assumes that the previous day's positions remain unchanged and asks:

What would today's profit or loss have been purely because market prices moved?

This is often called hypothetical P&L.

RBI's backtesting guidance calls for comparison using both hypothetical changes in portfolio value and actual trading outcomes in its internal-model framework for Primary Dealers.

Why Hypothetical P&L Is Useful

It gives a cleaner comparison because both:

• VaR; and

• hypothetical P&L

are based on essentially the same starting portfolio.

It helps isolate whether the model captured the market risk already present at the time VaR was estimated.

Actual vs Hypothetical P&L

What Do Too Many Exceptions Mean?

Repeated exceptions can indicate several possibilities.

VaR Is Too Low

The model may be systematically underestimating risk.

Volatility Estimate Is Stale

Recent market volatility may be much higher than the historical estimate used.

Correlation Assumptions Are Wrong

Assets believed to provide diversification may suddenly move together.

Important Risk Factor Is Missing

The model may fail to capture:

• basis risk;

• volatility risk;

• spread risk;

• option risk; or

• another material factor.

Valuation Problem

Complex positions may be priced incorrectly.

Poor Data

Market data may be:

• incomplete;

• stale;

• incorrect; or

• inconsistent.

Actual P&L Contains Other Effects

An exception using actual P&L can sometimes arise from:

• intraday trading;

• fee effects; or

• positions not represented in the original VaR calculation.

Therefore, an exception should be analysed, not merely counted.

Too Few Exceptions Can Also Be Informative

A model that produces almost no exceptions may appear excellent.

But it may also be excessively conservative.

If VaR is consistently far larger than realistic exposure, the bank may:

• allocate too much risk capital internally;

• impose unnecessarily tight limits; or

• misjudge the relative riskiness of businesses.

The objective is therefore not:

Zero exceptions at any cost

but:

A model whose behaviour is appropriate for its stated confidence level and risk-management purpose.

Backtesting Is Not Stress Testing

These two concepts are often confused.

Backtesting

Looks backward.

It asks:

Did the model's risk forecasts perform reasonably compared with actual outcomes?

Stress Testing

Looks at severe conditions.

It asks:

What happens to our current portfolio if markets move much more severely than normal?

Therefore:

Backtesting = Model-performance check

Stress Testing = Vulnerability analysis

What Is Stress Testing?

Stress testing evaluates the financial effect of severe but plausible adverse conditions.

RBI describes stress testing as an important forward-looking risk-management tool that supplements other risk measures and helps assess potential losses, capital needs, risk concentrations, and contingency actions.

For treasury market risk, stress scenarios may involve:

• major interest-rate shocks;

• sharp bond-yield movements;

• sudden currency depreciation;

• extreme equity-market decline;

• widening credit spreads;

• volatility spikes;

• market-liquidity collapse; or

• several shocks occurring together.

Why Stress Testing Is Needed if VaR Already Exists

VaR can be weak precisely when markets become abnormal.

Consider a portfolio that has experienced low volatility for several years.

Its historical VaR may therefore appear small.

Then a sudden financial shock occurs.

Market movements may be far greater than those represented in the recent data.

Stress testing asks:

What if today's calm market assumptions suddenly fail?

RBI specifically states that stress testing helps overcome limitations of models and historical data and should feed into risk tolerance, capital and liquidity planning, and contingency actions.

Sensitivity Testing

A sensitivity test changes one important risk factor while broadly keeping others unchanged.

Examples:

• interest rates increase sharply;

• INR depreciates by a specified percentage;

• credit spreads widen;

• equity prices fall;

• implied volatility rises.

This answers:

How sensitive is the portfolio to this particular risk factor?

Sensitivity testing is relatively simple and helps identify concentrated exposures.

Historical Scenario Testing

A historical stress test applies market movements similar to a previous crisis or period of extreme stress to today's portfolio.

Possible scenarios may draw on events such as:

• an earlier financial crisis;

• a severe interest-rate shock;

• an extreme currency movement;

• a pandemic-related market disruption; or

• another period of significant market dislocation.

The key idea is:

Past Crisis Movement + Current Portfolio

RBI's market-risk stress-testing guidance specifically discusses testing current portfolios against historical periods of significant market disturbance, including both large price movements and reduced liquidity.

Hypothetical Scenario Testing

Historical crises are useful, but the next crisis may be different.

A bank can therefore create a hypothetical scenario.

For example:

• policy rates rise sharply;

• INR simultaneously depreciates;

• bond yields rise;

• credit spreads widen;

• market liquidity declines.

A hypothetical scenario can combine shocks that have not previously occurred together in exactly the same way.

This is particularly valuable for exposures created by new products or current market conditions.

Combined Stress Scenarios

Risk factors often become more strongly connected during a crisis.

Therefore, stress tests should not always examine them independently.

For example:

Interest rates rise

credit spreads widen

market liquidity falls

counterparties demand additional collateral

The total loss and liquidity pressure may be much larger than the result of a single-factor test.

RBI guidance calls for stress scenarios covering institution-specific and market-wide stresses, including combinations of adverse conditions.

Reverse Stress Testing

Normal stress testing starts with a scenario and calculates the resulting loss.

Reverse stress testing starts with failure or severe vulnerability and works backward.

It asks:

What combination of events could make our business model or treasury position unviable?

Examples of target outcomes could include:

• capital falling below an internal threshold;

• liquidity becoming critically strained;

• trading loss exceeding risk appetite;

• collateral requirement becoming impossible to meet.

Reverse stress testing is valuable because management may discover vulnerabilities that ordinary predefined scenarios fail to reveal.

Stress Testing Should Lead to Action

Stress testing is not useful if management simply produces a report and files it.

If a test reveals excessive vulnerability, possible actions can include:

• reducing positions;

• hedging;

• lowering limits;

• diversifying concentrations;

• holding additional liquidity;

• increasing collateral resources;

• revising contingency plans; or

• strengthening capital planning.

RBI's market-risk guidance states that stress-test results should be reviewed by senior management and reflected in policies and limits; identified vulnerabilities should lead to appropriate risk-reduction measures.

Major Limitation 1: VaR Does Not Show Maximum Loss

If 99% VaR is ₹X, losses can still exceed ₹X.

VaR does not normally tell you how large the loss will be after crossing the VaR threshold.

This is perhaps its most important limitation.

Two portfolios may have the same VaR but very different losses in the extreme tail.

Major Limitation 2: Dependence on Historical Data

Many VaR methods depend heavily on historical observations.

If the historical period is unusually calm:

estimated volatility may be low

↓

VaR may be low

But future markets need not resemble recent history.

The model may therefore underestimate risk immediately before a major regime change.

Major Limitation 3: Normal-Distribution Assumption

Simple parametric VaR often assumes returns behave approximately according to a normal distribution.

Real financial returns can display:

• fat tails;

• skewness; and

• more extreme movements than a normal model predicts.

This means extreme losses can occur more often than the simplified distribution suggests.

Major Limitation 4: Correlations Can Change

Diversification benefits often depend on correlations among positions.

During normal markets:

• Asset A may move differently from Asset B.

During stress:

• both may fall together.

If correlations increase sharply during a crisis, the portfolio may be less diversified exactly when diversification is most needed.

Major Limitation 5: Liquidity Risk May Be Understated

VaR may assume that positions can be:

• valued reliably; and

• sold or hedged within the stated holding period.

In stressed markets:

• bid-ask spreads can widen;

• trading may become thin;

• prices may gap;

• positions may take much longer to exit.

A ten-day statistical holding period does not guarantee that every large position can actually be liquidated efficiently within ten days.

Major Limitation 6: Non-Linear Products

Options and other complex derivatives have non-linear payoffs.

A simple linear or delta-based VaR model may fail to capture adequately:

• gamma risk;

• volatility risk;

• changing hedge ratios; and

• extreme option behaviour.

More sophisticated full-revaluation or simulation approaches may be needed.

Major Limitation 7: Model Risk

Model risk is the risk of loss or poor decisions arising from:

• incorrect model design;

• inappropriate assumptions;

• incorrect implementation;

• bad data;

• coding errors; or

• misuse of model output.

A sophisticated model can therefore still be dangerous if:

• its assumptions are inappropriate; or

• users treat its output as unquestionably correct.

RBI's model-control guidance emphasises independent validation of model structure, data, valuation methods, volatility assumptions, and backtesting results.

Major Limitation 8: Risk Factors May Be Missing

A model may capture:

• general interest-rate risk;

but fail to capture adequately:

• basis risk;

• spread risk;

• volatility risk;

• concentration risk; or

• another portfolio-specific exposure.

A model can therefore be mathematically correct but economically incomplete.

Major Limitation 9: VaR Can Encourage False Precision

A result such as:

₹127.43 crore

can appear extremely accurate.

But the decimal precision does not remove uncertainty about:

• data;

• assumptions;

• liquidity;

• future volatility;

• correlations; or

• extreme events.

VaR should therefore be treated as a decision-support measure, not an exact forecast of tomorrow's loss.

Major Limitation 10: VaR Is Not a Complete Risk Measure

Traditional VaR primarily focuses on market-value risk.

It does not by itself fully measure:

• credit risk;

• settlement risk;

• operational risk;

• funding liquidity risk;

• legal risk;

• model risk; or

• concentration risk.

Treasury therefore needs multiple risk measures and controls.

VaR vs Stress Testing

VaR vs Backtesting vs Stress Testing

These functions should operate together.

Model Validation

A robust validation process should examine:

• whether all major risk factors are captured;

• quality and independence of data;

• valuation methods;

• volatility and correlation assumptions;

• model calculations;

• sensitivity to assumptions;

• backtesting performance;

• stress-test behaviour;

• model changes; and

• limitations communicated to users.

Validation should be sufficiently independent of the people who design or use the trading model.

RBI guidance explicitly requires validation of significant changes in risk measurement, verification of model assumptions and data quality, and frequent backtesting.

Role of Independent Risk Control

Front Office uses risk measures to manage trading positions.

But independent risk control should:

• calculate or independently verify VaR;

• monitor limits;

• analyse exceptions;

• perform backtesting;

• conduct stress tests;

• challenge model assumptions;

• report weaknesses; and

• escalate material exposures.

The trader whose performance depends on the position should not be the sole judge of whether the risk model is reliable.

Current Basel Context: Expected Shortfall

Traditional market-risk models relied heavily on VaR.

The revised Basel market-risk framework moved the principal internal-model tail-risk measure from 99% VaR to 97.5% Expected Shortfall (ES).

Why?

Expected Shortfall looks beyond the threshold and asks about the average loss in the tail.

This addresses one important weakness of VaR:

VaR identifies the threshold but does not measure the severity of losses beyond it.

However, VaR has not disappeared from risk management.

The current Basel framework still uses VaR-based backtesting at trading-desk level as part of internal-model eligibility, alongside other tests such as profit-and-loss attribution.

RBI Regulatory Context

India's current Basel III capital framework for scheduled commercial banks continues to prescribe the applicable regulatory methodology for market-risk capital rather than allowing a basic treasury VaR calculation to substitute automatically for the regulatory capital requirement. RBI's April 1, 2025 Basel III Master Circular consolidates the current prudential capital framework.

Therefore:

Internal Treasury VaR ≠ automatically Regulatory Market-Risk Capital

This distinction is important both practically and for exams.

Practical Treasury Example

Suppose a treasury's VaR has remained low for several months.

Suddenly:

• policy-rate expectations change sharply;

• bond yields rise;

• INR depreciates;

• credit spreads widen; and

• market liquidity falls.

VaR

The previous day's VaR gives a normal statistical risk estimate based on its model.

Backtesting

If the actual loss exceeds VaR, an exception is recorded and investigated.

Stress Testing

The bank examines whether similar or even more severe combined shocks could create:

• unacceptable trading losses;

• margin calls;

• liquidity pressure;

• limit breaches; or

• capital stress.

Management Response

The bank may:

• hedge;

• reduce positions;

• change limits;

• increase liquidity; or

• revise model assumptions.

This shows why VaR alone is insufficient.

CAIIB Exam Focus

Backtesting compares model-generated VaR with subsequent trading outcomes.

A loss greater than VaR is a VaR exception.

Some exceptions are expected because VaR is not a maximum-loss measure.

Excessive exceptions may indicate model or assumption weaknesses.

Backtesting may use actual P&L and hypothetical P&L.

Hypothetical P&L generally keeps the starting portfolio unchanged.

Stress testing examines severe but plausible conditions.

Backtesting looks at model performance; stress testing examines vulnerability.

Sensitivity testing changes selected risk factors.

Historical stress testing applies past severe market movements to the current portfolio.

Hypothetical stress testing creates severe scenarios that need not have occurred historically.

Reverse stress testing starts from a severe failure outcome and asks what circumstances could cause it.

Stress-test results should influence limits, hedging, liquidity, capital planning, and contingency actions.

VaR does not measure maximum possible loss.

VaR may underestimate fat tails and changing correlations.

Historical data may not represent future crises.

Illiquidity can make statistical holding-period assumptions unrealistic.

Complex derivatives can expose weaknesses in simple linear VaR models.

Model validation should be independent and should test data, assumptions, implementation, valuation, and backtesting performance.

Current Basel internal-model market-risk standards use Expected Shortfall as the principal tail-risk measure rather than traditional VaR.

Common Exam Traps

Trap 1: A VaR Exception Proves the Model Is Wrong

Incorrect.

Some exceptions are expected at any confidence level below 100%.

Trap 2: A 99% VaR Should Never Be Exceeded

Incorrect.

The remaining tail probability allows exceptions.

Trap 3: Backtesting Predicts Future Crisis Loss

Incorrect.

Backtesting primarily evaluates how the model performed against realised outcomes.

Trap 4: Stress Testing and Backtesting Are the Same

Incorrect.

Backtesting validates model forecasts.

Stress testing examines severe adverse scenarios.

Trap 5: Actual and Hypothetical P&L Are Identical

Incorrect.

Actual P&L may include new trades and intraday activity, while hypothetical P&L generally preserves the starting portfolio.

Trap 6: Zero VaR Exceptions Always Means the Best Model

Not necessarily.

The model may be excessively conservative.

Trap 7: Historical VaR Automatically Captures the Next Crisis

Incorrect.

Future stress can differ radically from historical experience.

Trap 8: Diversification Correlations Are Constant

Incorrect.

Correlations can rise sharply during market stress.

Trap 9: VaR Measures Loss after the VaR Threshold

Incorrect.

Traditional VaR identifies the threshold, not the average severity of losses beyond it.

Trap 10: VaR Fully Captures Liquidity Risk

Incorrect.

Actual liquidation may take longer and cost more during stressed markets.

Trap 11: More Decimal Places Make VaR More Accurate

Incorrect.

Model accuracy depends on assumptions, data, methodology, and validation—not numerical formatting.

Trap 12: Stress Tests Need Only Historical Scenarios

Incorrect.

Useful programmes also employ hypothetical and portfolio-specific scenarios.

Trap 13: Stress-Test Results Need Not Affect Treasury Decisions

Incorrect.

Identified vulnerabilities should influence risk management.

Trap 14: Expected Shortfall Means VaR Is No Longer Useful

Incorrect.

VaR remains useful for treasury risk measurement and backtesting even though the revised Basel internal-model framework uses ES as its principal tail-risk capital measure.

Memory Technique

Remember V-B-S:

• V – VaR estimates

• B – Backtesting checks

• S – Stress testing challenges

For backtesting:

Forecast → Outcome → Exception → Investigate

For stress testing:

Shock → Revalue → Measure Loss → Take Action

And remember the central CAIIB principle:

VaR tells you the model's estimate; backtesting tells you how the model behaved; stress testing tells you what may happen when normal assumptions fail.