The Case for Wise Financial Modeling
Rodney N. Sullivan, CFA

The financial crisis has led many to suggest that financial models are broken and should be relegated to the dustbins of finance. Rodney Sullivan discusses how financial models can bring important value to understanding market dynamics, but only when used properly.

What is the source of criticism of financial models?

Critics suggest that financial models were ineffective during the financial crisis, resulting in portfolio underperformance and unexpected losses to investors. For instance, various innovative products found to fuel the crisis, such as those introduced by certain banks, were largely made possible through modeling techniques.

Recent events thus highlight the need for a close look at the role and effectiveness of financial models and the expertise of modelers.

How are financial models used?

Financial modeling is often tackled with the full force of mathematical technology and applied to activities such as hedging, risk management, product innovation, and alpha hunting. In short, it is a competency that can be applied in a variety of ways.

What are the limitations of models?

Perhaps most important among the limitations of modeling techniques is the inability to effectively capture change or the unexpected. Models can pose serious risks to those who rely too heavily on them, and because investors possess the free will to make decisions, the future direction of markets is highly uncertain. In short, no single model can contain all the necessary information to capture the uncertain path of outcomes—including the extreme events that frequent markets.

Compounding this challenge, markets jump, sometimes abruptly producing fat tails in the distribution of returns. Such shocks in financial markets severely hamper the effectiveness of model-driven decision making.

For example, risk models, such as Gaussian VaR (value at risk), often fail to predict the duration and magnitude of extreme losses because they are ill suited to that task. They do not admit the extreme outcomes during periods of turmoil.

How should investors adapt their approach to modeling?

A stubborn disregard for the dynamic reality of market relationships will be accompanied by poor outcomes. A reliance on the persistence of historical relationships is dangerous.

Viewed in this context, modeling applications and techniques are merely tools to help investors accomplish their tasks; they are not solutions in themselves and do not manage themselves. An improperly modeled callable bond, for instance, may give a grossly incorrect duration estimate, thus misrepresenting the bond’s interest rate risk. 

Those depending on financial models have a responsibility to be keenly aware of the limits, boundaries, and risks of such techniques. Effectiveness requires understanding completely the limitations of the model and knowing when to turn away from the model and seek input elsewhere. This task becomes more challenging each day as finance grows increasingly complex.

How does one know the limits of a financial model?

One guide to knowing the limits of a model rests in understanding its underlying assumptions and the implications of those assumptions being incorrect. A lack of technical expertise among the individuals implementing and relying on a model is no excuse for ignorance of the model’s limits. In one such example, rating agency models dangerously had no ability to accept a negative change in real estate prices.

Those who seek an interdisciplinary, multidimensional approach to modeling the dynamics of markets and the macroeconomy are more likely to prove successful.

Can you describe the practical application of such an approach?

Proper use of models is a matter of competence. For instance, as a consequence of the reality of extreme events, markets are more appropriately modeled using heavy-tailed (i.e., fat-tailed) distributions. Models built on the back of Gaussian distributions are doomed to failure.

Given the dynamic complexity of markets, investors should use a network of modeling approaches to test the range of possible outcomes and to gauge the status of things. This idea is akin to an automobile dashboard with a series of gauges that help the driver recognize when things are going astray. I, along with others, am continuing to publish research demonstrating various metrics for use in such a dashboard.

Altogether, it is imperative for managers to monitor potential fundamental changes in the macro-environment on an ongoing basis. Expertise points toward a useful framework for monitoring risk as well as for conducting sensitivity and scenario analyses. Such tools offer a genuine barometer for understanding dynamics across our global financial system and monitoring the evolution of markets across time.

Rodney Sullivan, CFA, is head of publications at CFA Institute and editor of the Financial Analysts Journal.