Your Customer-facing AI Systems Will Make Mistakes. How Can You Reduce the Risks?
For banks, the risk of AI providing misinformation could have significant consequences for business. Experts recommend taking a few key steps to enhance your team’s AI proficiency and mitigate potential risks.
By Caroline Hroncich, Contributor at The Financial Brand
If you’ve kept up with the news over the past year, you’ve probably come across warnings about open-source generative AI tools like ChatGPT pulling data from unreliable sources. In the public domain, this fuels growing fears about misinformation spreading in critical areas like politics and healthcare.
But the risks aren’t limited to widely accessible AI. While enterprise AI systems may offer more safeguards than public models, leaders still face a major concern: the potential for an AI-powered chatbot to "hallucinate" — a term experts use to describe AI generating misleading or outright false information.
These situations are already happening. An Air Canada chatbot mistakenly promised a customer a refund under an incorrect policy. When the airline later denied the refund, it blamed the chatbot, arguing in court that the AI was "a separate legal entity responsible for its own actions," CBS reported.
Banks have understandably been more cautious with AI chatbots that interact with customers, especially compared to industries with lower stakes, like airlines or the cruise industry. Currently, most chatbots in financial services operate within strict guardrails, functioning more like a FAQ tool than a ChatGPT. That’s a good thing, says Amas Tenumah, global leader of service transformation & AI innovation at Slalom. But that’s poised to shift as AI technology continues to evolve.
"We’re going to see more and more contacts with the customer," Tenumah says. "This is going to be a relief valve. It will allow bankers to build more relationships with customers because they can use these tools to automate things like chat and make agents more efficient. They can have more human to human conversations because they can use automation for minutiae."
As AI tools become common in banking, it’s more important than ever for leaders to understand how these systems interact with customers, pull information and respond to queries. Beyond functionality, executives must also navigate the ethical, legal and governance challenges that come with AI adoption. There’s a lot to consider, especially for smaller institutions like community banks and credit unions, which often depend on third-party vendors rather than developing AI technology in-house.
The good news is executives are becoming more savvy about generative AI, especially as the hype cools and the reputational risks become clearer, says Bill Waid, chief product and technology officer at FICO. But there’s still room for additional education.
"What does it mean to be ethical? What does it mean to have a policy that is enforceable and you can defend its reputation?" he says. "I think more and more executives are starting to ask those questions."
So, how can bank marketers reduce the risk of AI-generated errors or the spread of misinformation? Even without a technical background, industry experts suggest a few essential steps to enhance your AI literacy and oversight.
Establish Guidelines for AI Use
For bank marketers, the stakes of using AI for creative or marketing functions are lower than in other areas that require detailed regulatory reporting, Waid says. But it’s still important for marketers to have transparency into how the AI tools they are using source information.
If you haven’t already, consider establishing clear policies on AI ethics and usage guidelines to ensure your team leverages these tools responsibly and effectively. Companies like IBM have established AI ethics boards to address the complex issues surrounding AI. If you’re working with a vendor, inquire about how the tool uses AI, along with its standards and policies. Effective strategies should address issues like bias, transparency, and accountability.
"The first caution that we offer is buyer be wary of what is in the market today," Waid says. "Because there’s a lot of hype and a lot of push that if you don’t have generative AI, or AI in general, in your solution, then you’re not prime. But the reality is there are so many techniques associated with AI. It’s not new, we’ve been doing this stuff since the mid 80s. It’s about applying the right principles and the right use cases."
Given the rapid evolution of AI technology, it’s inevitable that policies will need to adapt over time. Make use of your internal technical expertise and company policies, while also staying informed about how both your in-house systems and third-party vendors are implementing AI.
"You can call it responsible, ethical, explainable, but are you catching those biases properly in the execution phase," Waid says. "There are techniques to make sure that those ethical issues are addressed across the board."
Learn more:
- As Banks Embrace AI, Soft Skills Matter More Than Ever
- Why AI Agents May Be the Tech to Finally Deliver on Personalization
- When It Comes to AI, ‘Wait-and-See’ May Mean ‘Wait-and-Die’
Introduce Human Oversight
The most reliable way to ensure the accuracy of AI-generated content is to have a human review and verify it for errors — particularly when the end product is customer facing, Tenumah says.
"In financial services for a while, the bot will be riding in the passenger’s seat. It’ll execute the event and it’ll say ‘hey, human employee I think this is the order, is that okay,’ and the human will modify or say yes," he says. "This is an industry where the stakes are too high."
Leaders should integrate steps for human oversight into the team’s standard practices and policies. Educate team members on how to effectively double-check for errors and ensure they’re aware of the key challenges and limitations of AI tools.
It’s also beneficial to promote general AI literacy, such as understanding the differences between natural language processing and generative AI. This will empower them to become more informed users of AI tools and create a safeguard to catch any misinformation that might slip through the cracks.
Human oversight will be increasingly crucial as banks experiment with emerging AI technologies, such as voice tools, which Tenumah expects to see significant growth in 2025 and beyond. Many large banks are already developing or have launched their own customer support voice tools. For instance, Bank of America’s virtual assistant, Erica, leverages natural language processing — an advanced form of AI — to assist customers with their financial inquiries.
"Voice is richer," he says. "There will be more interactions with customers because now that you can automate parts of the interactions."
Keep Generative AI In-House
For now, experts advise being cautious about rushing to implement a customer-facing AI chatbot just because you want to stay competitive. Instead, think about how you can take steps to make your own team more efficient by using AI internally, Tenumah says.
"My advice is don’t jump into that deep end of the pool just yet, because of all of these risks," he says. "I think the low hanging fruit today is thinking about how to utilize this technology to help humans do their jobs faster."
AI offers immense potential across various industries, but business leaders should focus on the specific problem they aim to solve with the tool, rather than getting swept up in the general hype surrounding the technology, Waid says.
"It gives me information extremely fast and I don’t have to work at it. If you can improve the accuracy, it simplifies the human machine interface, and that is one of the most transformative elements that comes from gen AI," he says.
