For Financial Institutions, Effective AI Implementation Requires a Scalable Foundation

An emerging lesson of AI adoption is that progress is rarely linear. Whether for an individual enterprise or an industry, it comes in bursts, and each new milestone requires internal and external users to recalibrate their trust.

Few sectors feel this as keenly as small to midsized financial institutions. Operating under strict regulatory oversight and tight margins, banks and credit unions face both rising competitive pressure to deploy AI applications and persistent resistance from the customers they serve and the stakeholders responsible for risk and regulatory oversight.

Key insight: Current research reflects this tension. On one hand, implementation is steadily rising. In a 2026 survey of small and midsized financial institutions by Cornerstone Advisors, 59% of credit unions and 49% of banks said they’d already deployed generative AI into production, up from single digits in 2024. Agentic applications are farther behind, but also picking up momentum: 17% of credit unions and 7% of banks have invested in or deployed agentic AI, while about 25% plan to invest in it in 2026.

At the same time, deployment continues to bump up against trust. According to a January 2026 consumer survey by MX, a technology firm that helps financial institutions harness the value of their customer data, only 32% of consumers trust AI to help manage their finances, and half distrust it. Their experience as users may be a driver: Of the 34% of consumers who said they have used an AI chatbot in financial services, more than half found it not very helpful or not helpful at all. This is in line with a 2026 J.D. Power study of national bank and credit card app satisfaction: Only 28% use their app’s virtual assistant, and satisfaction drops sharply when they turn to it for more sensitive issues such as problem resolution, payment disputes, or fraud reporting.

Reality check: Smaller financial institutions, caught between well-resourced national and global players and agile fintechs, might think they need to either spend heavily or “move fast and break things” to scale up AI use and remain competitive. But a more sustainable path is to focus on the fundamentals.

In an interview with The Financial Brand, Elias Hiller, Product Leader for AI at MX, offered perspective on how institutions deploying AI can build in both the functionality and trust they need at the outset, and preserve both as their capabilities become more sophisticated.

Data Readiness is Job #1

Bank and credit union technology teams will be tempted to devote heavy scrutiny to which model they should use, fretting over various providers’ reputations and token efficiency. “It’s much more important for financial institutions to focus their energy on the readiness of their data,” said Hiller. This is critical for both simple generative AI and more-complex agentic use cases.

Financial institution data typically exists in silos with inconsistent naming conventions and taxonomies. According to Hiller, improving it requires cleansing (to remove errors and duplicates), normalization (to bring data from different sources into consistent formats), and categorization (to sort transactions, for example, into meaningful groupings that enable specified outcomes).

The resulting data architecture can be configured as “context graphs,” similar to Google’s Knowledge Graph or Facebook’s social graph. An institution’s graph holds its FAQs, product information, and brand guidance; each accountholder’s graph holds financial data, call-center history, and signals such as propensity to buy or need support. Each can be structured on its own, but bringing the two together into shared context is essential to AI readiness, Hiller said.

Key insight: With data cleansed and organized, AI models are less likely to produce hallucinations or inaccurate insights, not just because the chances of error are reduced but because applications and prompts can be better orchestrated across deterministic or probabilistic paths. By laying this groundwork, institutions also enable effective governance, because it gives them the ability to define and control access to sensitive categories of accountholder data and transactions, and associate them with relevant rules and policies.

Connectivity Across Systems

Connectivity must also be part of an institution’s AI foundation. Organizations that build AI-native applications, or design their systems with AI in mind, typically implement API-first architectures, Hiller said. This ensures that AI agents can reach the data sources they’re permitted to use through governed interfaces, and that real-time data can be securely delivered to model endpoints.

But few, if any, banks or credit unions are built that way. Many run core systems that were put in place decades ago, before real-time API-driven access was a design goal. Their data is spread across core, lending, card, and digital banking systems that come from different vendors and were put in place at different times.

As a result, Hiller said, financial institutions must bridge their legacy systems, adding an integration layer, sometimes called middleware or an API layer. This layer sits in front of legacy systems, makes their data available through APIs, and shields the core from direct AI access.

Key insight: Intentionally designing for connectivity also addresses a persistent accountholder pain point: An institution’s inability to “recognize” a customer or member and reflect a complete and up-to-the-minute view of their products and transactions across all channels.

Advanced use cases may also require permissioned access to external data, such as information held by another of an accountholder’s financial providers. Accountholder permission is typically granted through open banking connections or a data aggregation provider, and they should be able to see and revoke that permission. Once data is permissioned, open standards such as the Model Context Protocol (MCP) give AI systems a consistent way to connect to it. MCP also works in the other direction, letting institutions decide whether and how to make their own data available to external AI platforms.

“Imagine you have all these different signals and inputs coming in,” Hiller said. “How do you refine this to what you actually need to make to improve the consumer’s experience and create a better relationship with you as a bank?” By establishing governed connections that let LLMs and agents operate fluidly across the right data sets, an institution’s AI tools can gain relevance and context, without requiring it to overhaul existing systems and architectures. Each internal or external data set adds to the institution’s picture of the accountholder and to the value the accountholder derives from the institution’s applications.

The Elements of Trust

In thinking about AI trust and governance, institutions can start with three questions.

• Is the model accurate and fair?
• Can we impose hard controls on its activities, including through human intervention?
• And is every output explainable and auditable?

The answers to these questions are critical to maintaining the trust and confidence of both the internal and the external users of an institution’s AI applications.

Model validation. According to Hiller, AI models must undergo rigorous testing for accuracy and algorithmic bias, particularly when they play a role in lending, risk assessment, or financial advice. Accuracy improves with clean data and the right balance of deterministic and probabilistic functions, but it will never reach 100%. Bias is harder to control. A model can learn patterns that disadvantage protected groups — even if race, sex, or age aren’t specifically present in the data — because other data may act as a proxy for them and lead to unintended consequences.

“When it comes to algorithm bias, I think we’re not there yet as an industry,” Hiller said. He advises institutions to keep eligibility decisions within existing underwriting and compliance processes. AI may help a customer understand their debt and explore options such as consolidation, but the institution’s established processes determine whether a customer ultimately qualifies.

Guardrails and human oversight. Guardrails embedded in the system constrain what the model can say and do. These include baseline guardrails (e.g., blocking harmful language and dangerous topics) and domain guardrails (e.g., prohibiting investment advice). When the system hits a guardrail, it should stop or offer a next-best action, such as referral to an advisory line, rather than leave the customer at a dead end. Guardrails should be precise rather than broad – every question an assistant declines unnecessarily costs a little of the trust it was meant to build.

Human-in-the-loop safeguards are a further line of defense, set according to the institution’s policies and risk tolerance and weighed against the model’s demonstrated accuracy. Hiller recommends keeping a human in the loop for each type of AI task until the model’s accuracy is sufficient for the task’s risk profile. The human-in-the-loop principle also extends to the customer, whose requirement to approve or confirm AI actions should increase in line with each transaction’s risk, Hiller said.

Such guardrails are present in traditional financial services applications, but implementing them for generative or agentic AI is more complex because these applications are dynamic, often determining and sequencing next steps for themselves. Institutions need controls that can govern those decisions at runtime, not just within a predefined workflow.

Audit trail. “Traceability and auditability are not an afterthought but belong in the design from day one,” Hiller said. Regulators and institutions expect AI decisioning to be transparent. The fact that generative models are probabilistic makes definitive audit trails more critical to show what happened — from the data inputs used, to the tools and processes invoked and the guardrails applied, to the model and configuration in use.

Transparency can also enhance the customer or member experience, showing the sources and basis for each output they receive. The API layer also supports audit: When data moves through defined interfaces, the institution is better able to track the data AI relies on to generate recommendations and execute transactions.

The Challenge Ahead

As banks and credit unions advance their customer-facing AI capabilities, their AI foundation will play an increasingly pivotal role. Whether an institution is launching its first general-purpose conversational assistant or moving toward agentic applications, the technology’s scope and usefulness will depend on the data, connectivity, governance, and controls it has put in place.

To be sure, many institutions aren’t ready. “They know the experience they want to deliver, but everything that would support that experience is not where it needs to be,” Hiller said.

Launching a virtual assistant may seem like a simple first step, but a chatbot that declines to answer too many questions or stops short when asked to take what seems like a basic action may undermine the trust the institution is trying to build. “A customer who hits a wall may not come back,” Hiller said. “They’ll just think, ‘I’m not going to use this tool because I tried it once and it didn’t work.’”

But with the right foundation in place, institutions can expand AI capabilities in step with both customer expectations and their own ability to support them.

Bottom line: Bank and credit union strategists looking to move their institution to the next level can start by asking themselves some critical questions:

• What are the essential data repositories that support a 360-degree view of our customers or members?
• Do these sources share a common taxonomy across accountholders, products, and transactions?
• Are they accessible through a common interface? Are our governance policies and practices centrally managed, or do they live in functional silos?

In addressing these questions, an institution can establish the parameters of the work that lies ahead — what it will take to deploy AI in a way that progressively improves the accountholder experience while also ensuring that trust is retained, and deepened, along the way.