James Proudman: Managing machines - the governance of artificial intelligence

Speech by Mr James Proudman, Executive Director of UK Deposit Takers Supervision of the Bank of England, at the FCA Conference on Governance in Banking, London, 4 June 2019.

The views expressed in this speech are those of the speaker and not the view of the BIS.

Introduction

Consider the well-known story of one Big Tech company's attempt to use artificial intelligence to improve the efficiency of its staff recruitment. The machine learning system reviewed job applicants' CVs with the aim of automating the search for top talent. The company's experimental hiring tool used artificial intelligence to give job candidates scores ranging from one to five stars. Within a year, the company realised its new system was not rating candidates for software developer jobs and other technical posts in a gender-neutral way. That is because the computer models were trained to vet applicants by observing patterns in CVs submitted to the company over a 10-year period. Most came from men, a reflection of male dominance across the technology industry. In effect, the company's system taught itself that male candidates were preferable. It penalised CVs that included the word "women's" as in "women's chess club captain". And it downgraded graduates of two all-women's colleges.

The story is a clear example of how artificial intelligence can produce bad outcomes for all concerned. It also offers a case study for exploring the root causes that lead to bad outcomes - and so in turn offers insights for boards on how to govern the introduction of artificial intelligence. The art of managing technology is an increasingly important strategic issue facing boards, financial services companies included. And since it is a mantra amongst banking regulators that governance failings are the root cause of almost all prudential failures, this is also a topic of increased concern to prudential regulators.