Luis de Guindos: The evolution of stress-testing in Europe

Keynote speech by Mr Luis de Guindos, Vice-President of the European Central Bank, at the annual US-EU Symposium organised by the Program on International Financial Systems, Frankfurt am Main, 4 September 2019.

The views expressed in this speech are those of the speaker and not the view of the BIS.

Introduction

During the financial crisis, stress tests were primarily used to identify capital shortfalls in the banking sector and to enhance market discipline. This was achieved through the publication of consistent and granular data on a bank-by-bank basis, with banks judged to have either "passed" or "failed" the test.

Since the crisis, the way in which stress tests are used has evolved, both in Europe and abroad. They have become a key part of the supervisory and financial stability toolkit for assessing risk profiles and performance under adverse macroeconomic conditions.

In the United States, the regular Dodd-Frank Act stress tests and the Comprehensive Capital Analysis and Review are now an integral part of the process for determining the capital requirements of large US banks, influencing their capital plans and their dividend distribution policies.

In Europe, the biennial EU-wide stress tests coordinated by the European Banking Authority (EBA) are important inputs for the ECB's Supervisory Review and Evaluation Process (SREP).¹ In other words, rather than finishing with a pass or fail assessment, stress tests now provide a starting point, both for discussions between banks and supervisors and for macroprudential policymakers. The ECB also conducts a parallel exercise covering the banks under its supervision that are not included in the EBA exercise.

While this evolution in stress tests has occurred on both sides of the Atlantic - albeit with a few remaining differences in respective approaches - in my remarks today I will take a European perspective. In particular, I will focus on two key facets of this evolution - the more integral use of the top-down review process and the advantages of adding a macroprudential perspective.

Current stress test methodology in Europe

The EU-wide stress tests involve significant input from the banks, following a so-called constrained bottom-up approach. Under this approach, banks generate their stress test projections using their own models. These projections are based on a macro-financial scenario that is the same for all banks and are subject to a predefined methodology prescribed by the EBA.

The constrained bottom-up approach has several advantages. By treating all participating banks equally, it ensures a level playing field and provides results that are comparable across banks. This is an important element in fostering transparency and market discipline.

Ideally, a bottom-up approach should also support banks' own risk management capacity by forcing them to consider how severe adverse circumstances may affect their solvency. It should strengthen their ability to detect vulnerabilities and encourage them to develop their own internal stress-testing models.

But the constrained bottom-up approach is not without drawbacks.

The static balance sheet assumption limits the realism of the exercise as it does not account for how banks would respond under stressful situations. Certain caps and floors in the stress test methodology may compromise the realism of the stress test outcome.

The approach also provides banks with substantial leeway to materially underestimate their vulnerability to adverse circumstances, to "game" the exercise, in other words.²

Using top-down models to benchmark results

As a consequence, European supervisors conduct a thorough quality assurance of banks' bottom-up stress test results to ensure their outcomes are credible. Notably, this includes confronting banks with independent model-based estimates - a so-called top-down model challenge. This process generally leads to individual banks revising their stress test outcomes before publication.

We believe there is substantial value in this extensive supervisory scrutiny³ and in the improved market discipline stemming from the European stress test framework.

Indeed, discussions about the long-term strategy for the stress tests in Europe are underway at both the ECB/SSM and EBA levels.⁴ Proposed changes aim to improve the realism of stress test outcomes while retaining their reliability and credibility.

I do not want to pre-empt those discussions. But let me focus on one key aspect, namely the importance of top-down models in the context of the European prudential stress tests.

I believe the top-down model challenge, which has so far taken the form of a dialogue between supervisors and banks, could play a greater role in disciplining banks and reducing the incentives for them to systematically underestimate their vulnerabilities. This could be achieved by publishing the top-down view of supervisory stress tests.

The mere possibility of this information being made public should by itself encourage banks to produce more credible and sufficiently prudent stress test projections. In fact, doing so would provide market participants with a benchmark against which to judge each individual bank's result.

The exact form of publication would need to be considered. For example results could be published in aggregate form. Alternatively, results could be published individually or in ranges, such as grouping banks into solvency categories that could be compared with bottom-up, bank-by-bank results.

There is also a range of model-based results that could be produced and published. A simple exercise would be the equivalent of the bottom-up view using a static balance sheet assumption. At the other end of the spectrum, more comprehensive exercises could inform macroprudential analysis and incorporate amplification effects caused by interbank contagion or feedback loops between the real economy and the financial sector.

Macroprudential stress testing

This leads me to the second point I would like to emphasise today, the advantages of taking a macroprudential perspective with stress testing.

The exercises coordinated by the EBA, using the constrained bottom-up approach, are microprudential in nature. They aim to assess individual banks' resilience to adverse financial and economic circumstances. They inform and help determine bank-specific capital requirements, in particular the components of Pillar 2 capital.

But it is also possible to carry out a macroprudential exercise that instead focuses on the system-wide impact of adverse shocks on the banking sector as a whole. Macroprudential stress testing is characterised by a number of key elements.

First, it applies a top-down perspective to model the impact of a stress scenario on banks and the wider economy. A top-down approach helps ensure that all banks in the system are treated impartially and consistently. This approach is necessary in macroprudential stress tests to be able to model the aggregate implications of banks' dynamic responses to stress.

Second, macroprudential stress tests operate under the assumption of dynamic balance sheets. This enables the model to account for any amplification effects that banks' reactions to the initial shock may entail at the aggregate level. As an example, we estimate that the responses of banks to the adverse scenario in the 2018 stress test would exacerbate the fall in GDP by a further 1.6% beyond the initial shock. That deeper economic contraction would itself further deplete banks' capital, something that would not be picked up under the static assumptions of the microprudential exercise.⁵

Third, macroprudential stress testing typically aims to capture a comprehensive view of systemic risks affecting the banking sector and beyond.

Just as microprudential stress tests can help banks improve their internal risk management processes, macroprudential exercises provide benefits to supervisors and regulators. Developing the models and infrastructure for these tests enables supervisors and regulators to analyse a range of scenarios, helping them to better understand the links between banks, other parts of the financial sector and the wider economy. This also means that they are able to see how the banking sector reacts to a number of alternative macroprudential policy paths, helping them to ensure that the macroprudential measures currently in place remain appropriate and adequate.

There are benefits, too, from carrying out both microprudential and macroprudential exercises simultaneously. A combined exercise would help banks, market participants and regulators to assess the system-wide consequences of banks' reactions to situations of stress, which would complement the information retrieved from the microprudential stress tests.

With that in mind, it may well be worth regularly publishing the results of the top-down macroprudential stress tests, possibly alongside the results of the microprudential exercise.⁶ This simultaneous publication would provide all stakeholders with a more solid and comprehensive overview of the banking sector's resilience.

Conclusion

Let me conclude.

Over the past decade, system-wide stress tests have been established as an important tool for monitoring financial stability. They promote transparency in the banking sector, improve market discipline, foster banks' own risk management capacity and, importantly, inform prudential decisions of both a microprudential and macroprudential nature.

The way stress tests are implemented has evolved since the financial crisis, and we continue to discuss the long-term strategy for their use with stakeholders. Our experiences have shown the additional benefits of supervisors including a top-down view in their discussions, and of considering the wider, macroprudential dimensions of stress testing.

---

¹ ECB Banking Supervision uses both the qualitative results (quality and timeliness of banks' submissions) and the quantitative results (capital depletion and banks' resilience under the adverse scenario) as input for the SREP, to inform the setting of Pillar 2 capital requirements and Pillar 2 capital guidance, respectively.

² For empirical evidence of the "gaming" behaviour of banks participating in the stress test, see: Philippon, T., Pessarossi, P. and Camara, B. (2017), "Backtesting European Stress Tests", Working Paper Series, No 23083, National Bureau of Economic Research, January; Niepmann, F. and Stebunovs, V. (2018), "Modeling Your Stress Away", International Finance Discussion Papers, No 1232, Board of Governors of the Federal Reserve System, July; Quagliariello, M. (2019), "Are stress tests beauty contests? (And what we can do about it)", Staff Paper Series, No 4, European Banking Authority, July; and Kok, C., Müller, C. and Pancaro, C. (2019a), "The disciplinary effect of supervisory scrutiny on bank risk taking: evidence from the EU-wide stress test", Macroprudential Bulletin, No 9, European Central Bank, forthcoming.

³ In fact, recent empirical evidence suggests that the comprehensive intrusion associated with the ECB stress test quality assurance process has a disciplinary effect on banks' risk taking after the stress tests (see Kok et al. (2019a), ibid.; and Kok, C., Müller, C. and Pancaro, C. (2019b), "The disciplinary effect of supervisory scrutiny in the EU-wide stress test", Working Paper Series, European Central Bank, forthcoming). For related evidence in a US context, see: Acharya, V.V., Berger, A.N. and Roman, R.A. (2018), "Lending implications of U.S. bank stress tests: Costs or benefits?", Journal of Financial Intermediation, Vol. 34(C), pp. 58-90; and Hirtle, B., Kovner, A. and Plosser, M. (2019), "The Impact of Supervision on Bank Performance", Staff Reports, No 768, Federal Reserve Bank of New York.

⁴ See also Enria, A. (2018), "What have we learnt from EU-wide stress tests", speech delivered at the Banca Naţională a României, 15 November.

⁵ See Budnik, K. (2019), "A bird's-eye view of the resilience of the European banking system: results from the new macroprudential stress test framework", Macroprudential Bulletin, Issue 7, European Central Bank, March.

⁶ In addition to both the 2016 and 2018 EU-wide stress test exercises, albeit not in parallel with them, the ECB published macroprudential stress test analyses for euro area banks, see Dees, S. et al. (2016), "Macroprudential effects of systemic bank stress", Macroprudential Bulletin, Issue 2, European Central Bank, October; and Budnik, K. (2019), ibid.