A new NVIDIA survey reveals that 91% of financial institutions are either assessing or deploying AI in production, marking a significant shift from experimentation to implementation. Malcolm deMayo, global vice president of financial services, leads NVIDIA’s engagement with financial institutions in their AI transformation journey. On a recent episode of the Banking Transformed podcast, deMayo sat down with host Jim Marous to discuss the rapid evolution of AI in banking.
Q: What are the most significant changes AI has brought to financial services in the past year?
Malcolm deMayo: One of the biggest changes with generative AI is that the programming language is our language. So, instead of less than 2% of the world’s population being involved in developing solutions, you now have virtually everyone engaged.
Banks have to be very careful. AI is not new to them. This is a decades-old journey. So, they have to move cautiously in terms of bringing in new technology but when you think about the virtually hundreds of processes in banks and the fact that the language of banking is our language, the impact or the potential for impact of AI is enormous.
Q: How is NVIDIA helping financial institutions improve their AI initiatives?
deMayo: Our journey in financial services is not new. We began working with financial services firms 16 or 17 years ago. What we do at NVIDIA is build an accelerated compute platform, which has taken 30 years to build. It’s very different from hardware accelerators, and there are lots of them on the market.
We build a full stack. And that full stack solution allows us to accelerate a variety of workloads like high-performance computing. These are workloads you find very commonly in financial services. Things like options trading, where they’re looking to discover the price, they’re looking to do risk assessments and using back-testing to develop algorithmic trading strategies.
Current State of AI Implementation
Q: Your study showed that 43% of institutions are using generative AI. What applications are driving this adoption?
deMayo: Well, a big area is customer service in the contact center. The opportunity to improve the productivity of a call center agent is massive. Most of these people have to toggle between five or six systems. They’re reading texts that are unscannable, and this is after someone’s waited anywhere from 10 to 20 minutes to speak to them.
So, we’re building solutions today that allow AI to listen in on the question and scan ahead. You think about our accelerated computing platform, which you have with every node of our platform that gives you four research assistants who can read a thousand books a second.
Q: How are organizations balancing AI innovation with regulatory compliance?
deMayo: So, it’s a great question. Regulations exist to protect the consumer or protect our society from financial crime. And so, banks, I think, don’t get enough credit for the work they do for society.
Inside a financial firm, there is constant balance. Large financial institutions want to protect their customers and remain the trusted house bank or bank of choice.
How AI is Transforming Banking Operations
Q: How are financial institutions using AI to improve back-office operations?
deMayo: Goldman Sachs talks very much about how they’re using generative AI for code generation and that 40% of the code being generated is being accepted by their developers. If you think about when you have thousands of developers, 40% of their work is now automatically generated by AI. They’re so much more productive.
Another example is in customer service. You’ve probably read about Klarna over in Sweden using OpenAI’s technology, API to be able to handle two thirds of their calls in a single month using generative AI, reducing the average time to air and resolution from 11 minutes to two minutes.
Q: What role does talent acquisition play in AI implementation?
deMayo: The most prevalent challenge — and this dates back to the beginning of computer science, is data. But interestingly, I’ll give you a sneak preview of this year’s survey: talent acquisition has just trumped data at the top.
So, there is a fierce competition to bring in data science and data management engineers. And that’s going to continue. And they’re going to have to train their employees, retrain their employees, and also augment with new recruits and partnerships with universities.
Democratizing Data Access and Insights
Q: How is AI changing the way financial institutions handle and distribute data internally?
deMayo: When you think about the difference between just search and generative AI, you ask a question on search, you get a thousand documents back that you have to read. You ask a very precise question to generative AI, it summarizes those thousand documents into what you tell it to do.
Now, impose that model on your own corporate data, and think about all of the different data repositories across the bank. You now have the opportunity to rethink how you price for customers.
Q: What advantages does this democratization of data bring to organizations?
deMayo: If a customer is a single product customer, they might get a different rate than a customer across your products. And you now have the ability to understand that quickly in real-time as opposed to there have just been so many barriers in the data retrieval era that we both grew up in — the store and retrieve. At the end of the day today, it’s now take that data and generate insights ideas, and suggestions.
AI Factory: The New Infrastructure Model
Q: Can you explain the concept of an AI factory and why it’s important?
deMayo: These new applications don’t run on yesterday’s CPU servers well, so they need to move to what we’re calling an AI factory. And an AI factory is something that we’ve built with the Bank of New York and a number of other institutions, where they have created an AI hub. That AI hub is a standard, essentially massive computing system where various groups, such as wealth managers, settlements, treasurers, and everyone can build, experiment, and progress AIs into production.
Q: How does NVIDIA’s three-layer platform approach support this new model?
deMayo: What we’ve built is very simple: a three-layer cake. Its hardware, its software, and its development frameworks are available — you can access this through the server vendors, through the cloud providers, and also through colors.
Q: Your survey indicated that 97% of institutions plan to increase infrastructure spending. What’s driving this trend?
deMayo: Well, I mean, in fact, it’s really basic. They need to build the AI factory. They can do it in their data center, or they can do it in cloud, or they can do it in a colo. We’re agnostic as to where they do this, but they need to do it. You need to have the right infrastructure.
In 2012, we built Titan, a supercomputer that took up a tennis court at 17,000 GPUs, consumed close to 10 megawatts of power, and generated 20 petaflops of AI performance. Fast forward to 2024, Blackwell fits in the palm of your hand, consumes three quarters of a kilowatt of energy. Tennis court to the palm of your hand, and generates 20 petaflops of AI performance.
Building for Scale and Security
Q: How are institutions addressing the technical challenges of AI implementation?
deMayo: The second layer of this accelerated compute platform is an operating system we call NVIDIA AI Enterprise. And it does three things. First, it abstracts the developer from the hardware so they can program in Python, Scala, SQL, whatever they’re used to programming in as they build their AI models.
The second thing it does, is we’ve optimized it to run on the platform so you get the best performance, which means you can do more with less. And then the third thing it does is it’s hardened for enterprise. So, you don’t have to worry about security patches. We take care of all of that.
Q: What emerging AI applications excite you the most?
deMayo: I think the idea is that it’s just the constant innovation and the fast pace; we’re immersed in it. It’s just incredible to be part of. The fact that we have the opportunity to truly rethink how things are done, it’s very, very likely that the first application will look a lot like the way we do things today.
We’ll just sort of repave the road and optimize the workflow the way they are and then quickly realize, “Wow, we can rethink this and do this smarter.”
Q: How will AI change the customer experience in banking?
deMayo: We had another brand tell us that 7 out of 10 of the questions they get in their contacts doesn’t ask for advice. They’re problems that need to be solved, but not advice questions, not financial advice questions. When you hear these things, you realize just how impactful AI can be.
Think about NVIDIA, in 2022, our revenues were 29 billion. The next year it was 60 billion. We added less than 5% headcount. Think about that operating leverage. Some of that is attributed to our ecosystem, some of that is attributed to the fact that we’re leveraging AI to do the R&D work that normally would’ve required hiring more people.
For a longer version of this conversation, listen to “MindShift: Unlocking Innovation and Humanity in the Age of AI”, a podcast with Jim Marous, available here. This Q&A has been edited and condensed for clarity.
Justin Estes is an award-winning writer, strategist, and financial marketing expert with expertise in banking, investments, and fintech. His clients include the NYSE, Franklin Templeton, Credit Karma, Citi and, UBS, and his work has appeared in Forbes, Barrons and ThinkAdvisor as well as The Financial Brand.