Risk Management

We design independent model validation solutions that give banks and financial service companies a unique perspective on their modeling tools.

More

Request more information

Use our information request form to request more information about our Services or Solutions.

More

Methodology Overview

Montana Analytics has invested significant time learning and optimizing our analytical methodology. We utilize the best-in-class analytics and are always improving and developing new techniques.

Risk Modeling Methodology

Our risk modeling methodology has been crafted over years of efforts that include the fundamental study of consumer credit, credit utilization, borrower behavior, and loan analysis. We began to utilize the "Three C's" of credit: Character, Capacity and Capital - back in the 1980s.

We have analytical experience across the lending spectrum from automated underwriting and loan decisioning to Billion dollar portfolio risk exposure analysis and optimal allocation of Economic Capital.

Our background is rich with experience in analytical model development and the techniques needed to examine and validate complex models.

We've developed experience with a number of loan asset types and understand the unique aspects of residential mortgages, ARM loans, payment option arms, Negative amortization features and more.

Our tools and methods center on financial engineering and statistical analysis, including the use of multinomial logistic regression models to determine probabilities and discrete-time hazards modeling. We utilize competing-risks methods and examine conditional time-to-event modeling including survival analysis if appropriate.

Our modeling methods combined with our rich data warehouse allows us to seek out and quantify relationships between and across "layered-risks" inherent in mortgages. We utilize rigorous techniques to examine, estimate and test numerous outcomes. We employ backtesting and out-of-sample analysis to establish solid models and stable results.

Model Validation Background

Our model validation experience literally grew out of a focus on analytical model development, testing and implementation. Following years of advanced risk modeling, we began side-by-side with the "new" model validation industry as we tested and validated internal corporate models in 2001-2002. Since then, we have seen the industry develop into a more thorough and mature solution – and we promoted more rigorous methods each step of the way. We've been embedded with Enterprise Risk Management groups as they enhanced oversight of risk models following increased audit scrutiny stemming from Sarbanes-Oxley 404. This resulted in more focus on model risk management surrounding key financial accounting items.

We have direct model validation experience within the financial services and banking system including a close alignment with the OCC Advisory Bulletins.

We also have direct model validation experience within the FHLBank system including a close alignment with the FHFA Advisory Bulletins. We have performed numerous validations of internally-developed models. We have also performed a number of independent validation efforts on leading vendor models including asset valuation, pre-trade analytics, interest rate risk models and MBS portfolio analysis models. MBS models include prepayment, delinquency, default, loss and loss exposure and PD, LGD and EAD Basel models.

Model Validation Methodology

Our Model Validation Methodology involves a solid framework of validation standards developed from rigorous model development experience, industry practices and regulatory requirements. This framework is a comprehensive sequence of technical and analytical tests focused on the core elements of model validation including testing inputs, mathematical processing and outputs.

Recognizing the importance of proactively managing model risk, the OCC and FHFA regulatory bulletins address the key components and issues involved in model validation. Subsequently, many industry participants and other regulators have adopted these bulletins as standard validation guidelines.

Our methodology creates repeatable and transparent testing steps and documented analysis worthy of regulatory scrutiny.

Our model validation methodology is based upon this industry defined guidance and, in addition, our process is enhanced from industry experience and is extended by including a formal Test Case design to establish pre-determined expectations specific to each validation step. This methodology creates repeatable and transparent testing steps and documented analysis worthy of regulatory scrutiny and is the basis for our examination of complex financial models.

Our methodology examines the following components:

  • Theory and Methodology
  • Assumption Management
  • Data Quality
  • Internal Controls
  • Regulatory Compliance
  • Statistical Sampling and Model design
  • Mathematical Testing and Code Review
  • Benchmarking and Backtesting

In each area, we produce comprehensive and transparent results and documentation that is typically utilized for internal audit and external regulatory reviews. We always invite critical assessment of our work product – so we may improve and produce superior solutions in the future.

Complex Models

Some financial models are multi-layered and very complex. External "vendor" models often are both complex and not completely transparent due to the nature of proprietary vendor confidentiality.

Validation of modeling assumptions can take many forms, including:

  • A review for reasonableness
  • Backtesting when feasible (comparing actual model results against original modeling assumptions)
  • An approval process for material changes to assumptions.

Processing (calculations) contains theory, program code and mathematical functionality. Validation of model theory will entail a review of model design and methodology relative to its intended purpose, and often requires specialty skills or an independent review.

Validating results often include benchmarks to comparable models, market prices and qualitative review by subject-matter experts.

Validation of model calculations is often completed through model replication or reverse engineering, when feasible. In some cases, a code-level review could be completed on segments of the model, if not the entire model. In either case, best-practice organizations compare model results with the results of a well-validated, pre-existing benchmark model when possible.

For vendor models, the regulators recommend the same validation principles should be applied. Outputs consist of model results and related reports. Validating results often include benchmarks to comparable models, market prices and qualitative review by subject-matter experts. Differences should be reconciled by verifying that all computations were executed correctly, and determining the reasons for departures in projected outcomes. The extent of the variances will determine whether corrective action is required for the model assumptions to better align results with reality.