Fintech: AI regulation must be grounded in human rights
Artificial intelligence is already transforming the financial sector, from automating customer services roles to assisting with fraud detection. While the technology offers opportunities for efficiency and growth, it raises urgent questions for regulators and businesses about governance and the mitigation of risks.
The UK has decided to take a hands-off approach when it comes to regulating the use of AI in financial services. The Financial Conduct Authority (FCA) has decided to rely on its existing frameworks, including the Consumer Duty and the Senior Managers and Certification Regime. These aim to strengthen market integrity and hold businesses and their individual leaders accountable for harms to customers.
Subhajit Basu, professor of Law and Technology at the University of Leeds, says that in principle the UK regulator’s approach is defensible because most of the potential harms of AI – discrimination, exclusion and mis-selling – aren’t new. The difficulty is the lack of guidance the FCA has provided about how existing rules apply. ‘The interpretive work has been pushed onto firms, and firms have said fairly consistently they can’t do it with confidence,’ he says. ‘You have the odd spectacle of industry asking for more regulatory specificity and the regulator declining’.
That said, Basu believes additional regulation is necessary because existing laws have failed to prevent the harms that algorithmic systems have already produced. ‘The Post Office Horizon scandal is the case everyone in this country now understands, and its lesson is about what happens when a system's outputs can't be effectively contested and there are no tamper-proof records to analyse’.
The Post Office Horizon scandal is what happens when a system's outputs can't be effectively contested and there are no tamper-proof records to analyse
Subhajit Basu
Professor of Law and Technology, University of Leeds
Matt Hancock is a partner and member of the Litigation Practice at Greenberg Traurig in London. He says there is a ‘huge risk’ of AI resulting in increased regulatory issues for businesses stemming from misleading financial advice and exposure to financial crime. ‘My general sense is that it will be something that forms a significant part of our caseload in years to come,’ he says. Hancock also expects AI to increasingly feature in the day-to-day supervisory interactions between the FCA and businesses with the regulator likely to require an explanation of exactly how AI was used and its sufficiency for the task.
Hancock says that in the short term it would be helpful for the FCA to release guidance for companies on how existing rules apply and look to introducing new regulation in the long term once the risks are more fully understood. ‘I think the FCA recognises that, if it tries to regulate something today, by the time they actually do, it’ll be a year at least.’ He says. ‘In the context of AI, a year is a very long time in terms of how and where the tech is evolving.’
Josh Hogan is an officer of the IBA Banking and Financial Law Committee and a partner at McCann FitzGerald in Dublin. He says the main challenge for multinational businesses is responding to the different regulatory approaches to AI in the UK, EU and US. The EU introduced the AI Act in 2024. The legislation includes a risk-based system that treats some of the most common use cases for AI in financial services – such as credit scoring – as a high-risk activity. ‘The big challenge is going to be the distillation of the core principles and standards, building that within institutions and then fine-tuning for the local legal and regulatory requirements of each jurisdiction where the institutions are operating,’ he says.
In July, the AI Omnibus entered into force across the EU. The regulation delayed the implementation of some rules in the AI Act related to high-risk systems, including the evaluation of creditworthiness, until December 2027. The EU said changes to the original legislation were necessary to support innovation and allow companies an extended timeline to test AI systems and achieve compliance.
The Central Bank of Ireland’s approach to AI regulation is based on four core principles: strategic alignment, accountability, explainability and proportionate governance. Hogan says these safeguards are essential to ensure AI outputs are properly stress tested. ‘It is not enough to have somebody essentially rubber-stamping decisions made by AI. They need to be able to properly test it and that means keeping the human ability to actually do the task that AI is doing.’
Basu is clear that tech companies must be held accountable on claims they can improve financial services for customers and the detection of financial crime. ‘The promise that AI makes our lives better remains marketing rhetoric until we have robust evidence of how much these systems actually serve people in specific contexts, and at what cost,’ he says. ‘Regulation in the public interest insists on real-world evidence not just about benefits, but about the nature, extent and distribution of adverse impacts.’
Basu says governments should ground AI regulation in international human rights frameworks rather than a set of aspirations decided by the tech industry. This includes ensuring that people whose legal rights or significant interests are adversely affected by AI decision-making can contest the output and access redress. Businesses should also be required to keep records and logs of their AI use to make meaningful human oversight possible. ‘And it means recognising that where even the developers can't explain why a black-box system produced a given output, that opacity is a fundamental limit on how far the system can be trusted.’
Header image: Dan Talson/Adobe Stock