Author(s): Puneet Matai
This whitepaper focuses on the critical concept of AI Governance in the Banking, Financial Services, and Insurance (BFSI) sector. It highlights the need for regulations and governance frameworks to mitigate risks associated with accelerated adoption of AI in BFSI, including unethical bias, potential privacy breaches and threats to market stability. The paper advocates for the adoption of Explainable AI (XAI) frameworks to enhance transparency and prevent algorithmic bias. It also examines the evolving global landscape of AI governance, citing examples like the EU AI Act and Singapore’s Model AI Governance Framework. In conclusion, the paper suggests implementing a robust AI governance framework for the financial services industry to address legal, ethical, and regulatory challenges and promote responsible innovation.
The advent of Artificial Intelligence (AI) has marked a transformative epoch, fundamentally altering the operational dynamics across diverse industries. It has transformed the way tasks are performed, decisions are made, and services are delivered. Isn’t it fascinating how Artificial Intelligence has been making its way into various industries and transforming the way things work? From streamlining processes and improving decision-making to path-breaking innovative solutions, it has proven its worth time and again. AI not only plays a significant role but has also become an integral part of the transformation journeys of various organizations across industries. In healthcare, it augments medical diagnostics, while in manufacturing, AI-enabled sensors predict equipment failure with ultra-high precision. The automotive sector uses AI to power self-driving technologies, and the financial sector relies on AI algorithms to predict financial risks and market trends. However, the rapid adoption of artificial intelligence in highly regulated industries like Healthcare and Financial Services, also calls for stricter governance ensuring trustworthy and responsible innovation. This whitepaper navigates through the concept of AI governance, its need, challenges and barriers in firm-wide implementation, and future roadmap in the banking and financial services industry. It also highlights the need to address regulatory challenges and ethical considerations to establish a foundation for sustainable innovation. AI governance is essential not only from a regulatory standpoint but also to ensure the ethical deployment of AI technologies through policies and frameworks.
AI Governance refers to the set of policies that institutions put in place to ensure the ethical use of AI. The goal of AI governance is to manage the risks associated with AI applications and their effectiveness. Beyond effectiveness, it addresses risk management, regulatory compliance, and AI ethics. The policy paper, “A Framework for U.S AI Governance” proposes leveraging existing government entities to regulate AI tools in relevant industries (finance, healthcare, transportation etc) [1]. This includes considerations for improving the performance, reliability, and impact of AI technologies in achieving their intended goals.
Deloitte’s survey on data concerns for AI implementation as presented in Figure 1 indicate important challenges and concerns related to the use of AI.
Figure 1: Respondents’ (n=91) analysis for the issues on AI use and data structure. Image Credits: Deloitte [2].
The results show that there is a significant focus on controlling the quality of data (46%). This highlights the importance of ensuring that the data used to train and operate AI is accurate and reliable. The fact that 38% of respondents agreed that there is no governance structure for analytics, illustrates the need for regulations in managing data, protecting privacy, and overseeing the deployment of AI systems. The shared concerns about data quality and governance structure indicate a wider need for comprehensive AI regulations.
Advanced technology and finance offer transformative opportunities but also pose challenges related to ethics, accountability, and risk management. According to FinTech magazine, Singapore has been strategically investing in an engineering talent pool for developing AI applications since 2019 [3]. The International Association of Privacy Professionals (IAPP) has closely monitored these developments since the introduction of ChatGPT in November 2022 [4]. Organizations are prioritizing AI, as revealed in the IAPP-EY Annual Governance Report 2023. A significant 33% of respondents now designate AI governance as a strategic priority [5]. The IAPP report also shows that 60% of respondents indicated that they either already have teams or are planning to create a specific team to manage AI-related matters. By adopting clear guidelines and committing to fair and equitable uses of AI in the BFSI sector, trust can be cultivated.
The integration of AI into the BFSI sector has reached a pivotal stage, for example: BlackRock, a global investment management company implemented Machine Learning (ML) and AI algorithms in its investment process [6]. The application employs advanced analytics to analyse massive datasets, market trends, and economic indicators. BlackRock’s Whitepaper on “Artificial Intelligence and Machine Learning in Asset Management” also urges the implementation of regulatory approaches:
Overall, the importance of AI governance in the financial services sector is aligned with the strategic investments and global prioritization of responsible AI. Insights from the IAPP-EY Report and Black Rock’s advancements highlight the sector’s evolution towards ethical and regulated AI integration.
Financial institutions make use of data analytics in various domains, like in the case of credit and lending services, it is to evaluate the creditworthiness of potential clients. With the help of AI/ML, they can predict and assign risk scores to clients who are likely to default. Research by SP Global indicates that financial services (18% market share) are among the biggest users of machine learning [7]. It is crucial to establish governance that ensures the ethical design of AI algorithms to ensure unbiased scoring while protecting customer privacy. Let’s examine some of the key business domains within financial services to comprehend the current AI landscape.
Figure 2: Business Functions in Financial Services
Banks are employing AI for fraud detection, customer service through chatbots and personalized financial advice. Barclays employ an AI tool dedicated to fraud detection [7]. It implements real-time monitoring of merchant payment transactions.
AI in investment banking enhances data analysis, automates trading, and optimizes portfolio management. The Bank of America’s platform Glass for research analysis assists in recognising alerts for increases/decreases of short-term interest rates and adjusting trading portfolio positions [8].
The Central Bank of India (Reserve Bank of India) has affirmed to introduction of a system powered by AI to complete transactions through user-initiated voice commands. This system is set to benefit the 300 million active users of India’s Unified Payment Interface (UPI) [9].
Advanced analytics and AI can power credit rating agencies for precise score calculations. According to the survey, over 40% of organizations still rely on rule-based systems, and 29% of organizations have embraced AI and ML for credit risk management [10].
Analysis of vast amounts of data, trends, and investment opportunities is crucial for wealth management. As per a survey by Accenture around 84% of the respondents believe that AI will transform the industry and 85% have the perception that AI in wealth management is currently in the hype [11].
The rapid adoption of AI across all domains has also led to concerns around effecting quick but potentially flawed decisions. The lack of proper governance controls can put data at a higher risk of being vulnerable to unintended bias driving inaccurate and unwarranted outcomes. For example, banks use AI for fraud detection, but there is a risk of detecting false positives or negatives if the model parameters aren’t chosen consciously to avoid demographics and other variables that might introduce inherent bias.
The escalating integration of Artificial Intelligence in the Banking, Financial Services, and Insurance (BFSI) sector has yielded significant benefits; however, it also presents intrinsic risks, necessitating the need for a robust governance framework to oversee the model design and implementation. For instance, online fraud, specifically synthetic fraud, is now common and experts predict that it will continue to increase as AI becomes more prevalent. A recent survey of 500 fraud and risk professionals revealed that there are growing concerns in the financial industry about the growing number of fake online customers [12]. Inaccurate fraud detection in banking services may pose a significant threat, which may lead to financial as well as reputational loss.
The occurrence of false positives or negatives may trigger unnecessary security alerts or overlook actual instances of fraud. To address this, implementing AI governance in banking services becomes imperative, ensuring regular audits and continuous monitoring of algorithms. For example, adopting diverse datasets during model training can be a potential solution with the right governance controls in place, reducing bias, and enhancing the accuracy of fraud detection.
The growing risk associated with AI in automated trading decisions also poses a potential threat to financial market stability. High- frequency trading (HFT) caused market volatility and a flash crash in May 2010 due to unregulated rapid trading [13]. Flawed AI analysis, synthetic media, and voice impersonation scams can also mislead customers when making decisions. The New York Times recently reported on a disturbing case in Florida where scammers used AI-based software to mimic an investor’s voice during a phone call with a bank teller, impersonating a legitimate customer [14]. This is just one in a million scenarios highlighting potential risks of AI implementation in the absence of proper governance.
In credit and lending services, the issue of unfair lending practices leading to discrimination highlights the necessity for regulatory enforcement for AI. The First American Financial Corp website suffered a critical security lapse in May 2019, which resulted in over 885 million compromised credit card applications [15]. This data leak was not caused by any external hacker but instead was due to an internal error in facilitating sensitive data access. It is essential to implement proper governance when integrating AI to avoid cybersecurity threats such as ransomware attacks, identity theft, and malware injections. Failure to do so can lead to severe consequences.
Hence, responsible and ethical use of AI is crucial in achieving the optimal balance between innovation and risk management in financial services.
Explainable Artificial Intelligence (XAI) is a set of processes and methods designed to enhance the transparency and interpretability of machine learning algorithms. The primary goal of XAI is to enable human users who may not have a deep understanding of complex AI models.
Figure 3: XAI Framework [16].
An extensive survey within financial institutions indicates a substantial consensus with 77% expressing anticipation that AI will have a highly impactful role towards an innovative future [17]. Financial institutions are likely to engage in strategic planning to implement the benefit of Explainable AI towards service delivery. Acknowledging AI in finance and accountancy requires a level of explainability for optimal model performance, legal use, and compliance. The Whitepaper by the Association of Chartered Certified Accountants (ACCA) suggests that 51% of survey respondents lack awareness of the XAI Framework [18]. This offers insights to policymakers in emphasizing the significance of explainability in AI applications.
These policies can act as guidelines to prevent any misuse of AI and to promote the ethical and responsible adoption of AI in organisations’ transformational journeys. However, the existing set of principles governing the use of AI tools in various industries is not comprehensive enough to cater to its diverse needs and applications. Taking heed from enterprise data governance frameworks like DAMA and DCAM, it is paramount to design a robust and scalable AI governance framework that can help effectively regulate the use of AI across all industries and ensure that it is implemented safely, ethically, and fairly.
The road ahead for AI governance in financial services is poised to fortify responsible, ethical, and efficient AI deployment. With global initiatives underscoring the importance of robust AI governance frameworks to address ethical dilemmas; regulations are rapidly evolving.
Currently, the regulatory landscape is increasingly gravitating towards risk-based oversight of AI technologies, rigorously examining AI applications to pre-empt potential societal or individual harm. Anticipated regulations, including the AI Disclosure Act of 2023 and updates to the EU AI Act, are in the pipeline, promising to steer AI governance towards a more secure future. This proactive regulatory approach signals a bright prospect for intelligent technologies within the financial sector, ensuring that innovation progresses hand-in-hand with ethical considerations and public trust
To conclude, the establishment of a comprehensive AI governance framework is not just necessary-it’s imperative. Such a framework will address the intricate legal, ethical, and regulatory challenges emerging from AI’s evolution. Global regulations underscore the necessity for responsible AI practices to mitigate risks and ensure strict compliance. Ongoing amendments to and introduction of new AI acts, along with existing data protection laws, reflect a global consensus on the ethical deployment of AI.
However, the accelerated adoption of AI in financial services also unveils significant risks, ranging from online fraud to potential market disturbances and biases in lending practices. These concerns make the call for an expansive AI governance framework unignorable. Proposed solutions and best practices, such as Explainable AI (XAI) frameworks and stricter regulations, are steps towards addressing these issues. AI-driven initiatives must align with global trends that champion innovation while conscientiously considering ethical impacts.
To sum up, integrating AI and ML into the BFSI realm offers remarkable prospects for enhancing efficiency, mitigating risks, and driving innovation forward. Yet, this progressive transformation introduces inherent challenges that can only be navigated by implementing a robust AI governance framework and policy. Effective governance is not only a regulatory obligation but also a pivotal factor in fostering responsible innovation. It’s incumbent upon policymakers to weave these technological advances into holistic policies that balance innovation with ethical AI utilization.
