Reimagining Model Risk Management in Light of COVID-19

The COVID-19 pandemic sent shockwaves through global banking systems. Fortunately, the industry was better prepared to pivot compared to another challenge in recent memory: the 2007-08 global financial crisis. This shift comes despite banks now making more decisions than ever before about credit and other customer matters based on statistical and machine learning (ML) models.

Model risk management (MRM), re-defined by SR 11-7 guidance, has been in existence for more than 10 years, and there are clear benefits to taking a principled approach to managing model risk. However, COVID-19 presented some additional learnings for the banking and financial services industry. MRM processes should adjust to overcome new and complex market scenarios, increasing uncertainty and talent scarcity. Financial institutions that embrace technology across the model lifecycle are well-equipped to meet the moment.

 

Recent trends expose limitations to classic MRM

From heightened expectations of convenience to a prioritization of worker demands, the COVID-19 pandemic forced firms in all industries, including banking, to quickly update operations to meet shifting needs. When it comes to the MRM process, key learnings coming out of 2020-21 are as follows:

• Surge in number of new models or re-build of failed models.
• Increased model complexity due to rapid development of new AI-/ML-based models.
• Need for faster time-to-market for new models due to rapidly evolving market scenarios.
• Need for an end-to-end model risk management paradigm covering data, infrastructure and operations versus just model validation.
• High incidence of management overriding model outcomes due to worsening conditions and increasing uncertainty.
• Recruiting challenges due to the ongoing talent shortage.

 

Embracing technology across the model lifecycle can help

The standard of MRM work must rise commensurately with growing model inventory, diversity, complexity and ecosystems. Financial institutions need a more comprehensive MRM approach beyond validation. That would strategically prioritize the redevelopment and adjustment of models based on concept and data drift evaluations while remaining efficient and largely process dependent. Organizations with manual model operations have been hit the worst by being reactive to economic crisis and the ensuing talent shortage during the COVID-19 era.

The traditional model validation should evolve in the wake of digitization leading to optimal confluence of technology and governance oversight. Now and in the future, model risks will be harder to digest with dated toolsets because evolving model complexities are intensified by the demand for a shorter model lifecycle.

Consequently, the role of model risk managers is likely to change, partly to account for more unconventional risks arising from this confluence. Embracing technology to bring more rigor, insight and timeliness across the model lifecycle can help financial institutions navigate the current climate.

The goal of a model risk manager should be to propagate an adequate operating model to facilitate the quicker and more efficient production of the models right through to retraining. To a large extent, this approach can eliminate management’s need to override existing model outcomes by having a freshly trained model in production more quickly. Fast-evolving data can build improved confidence in future outcomes.  
 

Technology that aligns with the modern MRM imperative

A standardized, technology-driven set of principles such as Machine Learning Model Operations (MLOps) can provide the foundation for the use of advanced analytical and digital tools for progressive automation across the model lifecycle. MLOps essentially enables organizations to track, version, audit, certify and reuse every asset in the model lifecycle. It also provides orchestration services to streamline managing this lifecycle.

Key MLOps benefits for MRM are as follows:

• Automation of end-to-end (E2E) MRM workflows: This is driven by automation of development, validation, testing, monitoring and documentation. Steps are completed while other process redundancies are minimized to enable faster production. MLOps promotes an automated E2E model lifecycle.
• Cost pressures: The set of best practices should balance cost, quality and efficiency in model risk management. MLOps helps balance this trinity.
• Reduced people dependency: Automation, reproducibility and documentation facilitate business continuity in times of talent shortage.

AI has also increased the importance and complexity of data management for MRM frameworks since AI needs higher volume and velocity of data to train. Organizations need to be able to apply rigorous data management frameworks with clearly defined model-related data governance and ownership structures. MLOps similarly stresses the need for data versioning and audit.  
 

Moving forward with automated agility

The risks exposed during the pandemic have accelerated reliance on digital infrastructure and the data-driven environment. Due to these factors, automation and agility are beginning to migrate into mainstream model operations. Organizations would be wise to expedite building their MLOps capabilities by experimenting and moving quickly to create and execute a sensible MLOps implementation strategy. This will aid in emerging from the current crisis stronger, more assured of risk exposure and better prepared for the future.

BDO’s next article sets out to understand MLOps in the context of guidance from the Financial Accounting Standards Board (FASB) in ASU 2016-13 regarding current expected credit loss (CECL) models. Early CECL adopters from January 1, 2020, found themselves amid widespread business disruption driven by the COVID-19 pandemic. This coincidence was an immediate test of the CECL methodology and underlying MLOps principles during an economic downturn and recovery, albeit an unusual one. Pandemics and similar events stress the need for a more fluid and technology-driven approach to building agile MRM in the future.

BDO’s Valuation & Capital Market Analysis practice can help businesses evaluate and adjust their MRM processes to meet their evolving needs. 
 

---