Financial marketers often talk about Generative Artificial Intelligence (GenAI) as if it’s going to be the solution to every marketing task, big or small. After six months of hands-on experimentation, Ally Financial has strong hopes and the evidence to back them up.
Andrea Brimmer, chief marketing and public relations officer at Ally, has been one of the leaders of the company’s exploration of GenAI’s potential. But she’s also a realist with a sense of humor.
“We just saw our first creative presentation from an agency where all the storyboards were done via GenAI,” says Brimmer. “The agency people said, ‘If you see a person on the storyboard with 11 fingers, don’t freak out. It will be because it was all done by GenAI.'”
Brimmer generally favors financial marketers adopting Gen AI carefully and where appropriate, and has been extensively involved in Ally’s corporate experiments with the technology. But she also has spent a long career learning to tap human ideas inside and outside her organizations.
“While AI can be exceptionally creative it will never replace the need for a great creative mind to be thinking about a brand problem or a brand opportunity – at least within my tenure, I hope,” she says.
Sathish Muthukrishnan, Ally Financial’s chief information, data and digital officer, and Andrea Brimmer, chief marketing and public relations officer, worked together to experiment with GenAI assistance for marketing functions. (Zack Flannick, not shown, directed the experiment for the marketing function.)
Brimmer and her marketing staff spent the second half of 2023 experimenting with OpenAI’s ChatGPT. One of the most important lessons was learning how much craft goes into the “prompts” — orders given to GenAI to produce something — and how much revision of those prompts is necessary to help the AI to come through with useful results.
“I think some people have this perception that ‘I just tell ChatGPT something and it goes ahead and does it,'” says Brimmer. Her team learned during the experiment how far that expectation is from reality.
Sathish Muthukrishnan, Ally’s chief information, data and digital officer, has overall charge of the company’s move into Generative AI. He suggests that it’s helpful to think of ChatGPT and its ilk in terms of raw material, rather than an end product. He uses this analogy:
“Think of it as stone. How do you, the person chiseling away, make that stone into a beautiful statue? You need to give it eyes, ears, nose, mouth, hands, legs for it to become the perfect likeness.”
He explains that many of the ingredients actually come from within the organization, in the form of data, talent, and the willingness to invest time in experimentation.
Anyone who thinks of GenAI as a kind of vending machine where you instantly get perfect answers “will be proved wrong,” Muthukrishnan says.
In a joint interview with The Financial Brand Brimmer and Muthukrishnan discuss the early realities — and results — with GenAI at Ally. (An earlier article discussed the effort just as it was starting out.)
The Tech Foundation Ally Started with for GenAI Experimentation
The bottom line? Using a large language model, a selected group of Ally marketers reduced the time needed to product creative campaigns and content by as much as two to three weeks, and saved 34% of the time that the same work would have taken without Generative AI.
The major savings came in the time taken for research, first drafts and other steps – totaling roughly 3,000 hours of human work. Already Brimmer sees the potential for GenAI to save on staffing and that it will influence the types of people she hires in the future.
At the same time, the team was cognitive of attendant risks: Muthukrishnan says that Ally took concerns about maintaining the privacy of customer data seriously and built a system that insulated that data from misuse. The company has been migrating its operations to cloud computing over the last decade. (Ability to operate in the cloud is essential to adopting generative artificial intelligence on an enterprise basis, according to Muthukrishnan.)
Ally implemented GenAI on what a layman would think of as an “arm’s length basis.” The company conducted the experiment on “Ally.ai,” an in-house GenAI platform between outside providers of GenAI and the banking company’s own systems. At present, the LLM used for the marketing experiment (and for most of Ally’s GenAI efforts) is OpenAI software provided though partnership with Microsoft.
Ally is also experimenting with Amazon’s Bedrock GenAI software and Muthukrishnan says the company could later also work with Google’s Bard product.
“Ally.ai has the ability to seamlessly shift between these and other technologies coming up,” says Muthukrishnan. Ally.ai is similar to a railroad switching point, where a train (a project) can be routed to one of several possible tracks.
In the future, as more work in marketing and other parts of the company goes through GenAI, that analogy could get more complicated. As each provider’s offerings develop, divergent strengths could emerge. Muthukrishnan says it’s also conceivable that a project could be submitted to multiple LLMs. The best result among the LLMs could be selected for implementation or even a blend of what all produced could be chosen.
Right now, however, Muthukrishnan says the offerings are roughly similar.
“It’s like picking a bottle of water. Each has a different label, but it’s all still water. Right now, people are longing for and thirsty for water. When things get a little bit more sophisticated, then you might have still or sparkling or flavored water. “
— Sathish Muthukrishnan, Ally Financial
Read more:
- How OpenAI’s Turmoil Could Impact Banking’s Use of Generative AI
- After the OpenAI Drama, Should Banks Hit the Pause Button on Generative AI?
How the Ally Built a Moat Between GenAI and the Rest of the Company Proceeded in Marketing
The “arm’s length” nature of Ally.ai required adherence to three principles that the organization set out early on. Muthukrishnan says these principles formed a moat between the company and external GenAI:
- External technology will not “learn” from Ally data. The LLM is able to use the data while it is performing work for Ally, but it is instructed to “forget” it when the session is over. Past work can be captured for reuse, but Ally.ai holds that data, rather than ChatGPT.
- People must be involved. “There will always be a human in the middle to understand the answers the technology provides and to continue giving feedback to refine it,” says Muthukrishnan.
- Test use cases internally first. Any material deemed ready for prime time will be exposed to internal “customers” first, before any external user sees it.
All personally identifiable information is stripped out before data is shared with the LLM, and Muthukrishnan says Ally is also contractually assured that none of its data will be used to train OpenAI’s own models.
At the same time, the controls couldn’t hobble the experiment. Brimmer, for example, wanted the team trying the technology to be representative. Marketing team members fresh out of school were involved, as well as veteran marketers in their 50s.
“I wanted GenAI spread out across a lot of different functions on our team. Also, I wanted it spread amongst a variety of age groups, so it wasn’t just the younger kids on the team who would be using it. I wanted all groups to have an opportunity to be part of the experiment, to dabble and get comfortable.”
— Andrea Brimmer, Ally Financial
Before testing began in earnest, Ally subjected the technology to the same risk-management process it applies to any new technology – including reviews for compliance, legal, information processing security, brand risk and more. Muthukrishnan says such reviews usually take six to eight months. This effort was fast-tracked and the risk review completed in under four weeks.
“Everybody understood the urgency behind this, and the need to stay ahead of the competition in the industry,” Muthukrishnan says.
Certain basic guardrails were part of each submission to the LLM. Technologically, these are called “system prompts.” They include an explanation to the GenAI that the organization is a financial services institution, that it is customer-focused, and that it prefers to take a personal approach to marketing. Examples of past marketing efforts were also submitted. Muthukrishnan says this process was refined by having marketing team members sitting side by side with AI programmers.
All this was fed to the AI before a user submitted an actual prompt for a specific project. (“It’s so you don’t have to start from zero again,” explains Muthukrishnan.)
Read more: Is ChatGPT Ready for Banking? Yes and No
The Human Side of the Equation: Where the Prompts Come From
“The intention of the experiment was to be able to move GenAI into real use across the team,” says Brimmer. “It was intended to help us get smarter about how to use the prompts.”
Thanks to the training, Ally says it experienced an 81% “prompt accuracy rate.” This means that in four out of five efforts users eventually received the right type of content back from the Ally.ai platform.
Among the tasks that the software addressed best: summarizing of research, brainstorming, analyzing information and creating drafts of content. Examples of the latter include first drafts of blog posts, ad copy, video scripts and social media posts.
In a commentary on a blog post experiment, Ally observed: “While Ally.ai output required editing and the finesse of Ally’s content writers, and it still went through Ally’s well-established regulatory review processes, having a quick first draft done in 15 minutes helped reduce the total time need to create and edit the article from four hours to just one.”
Read more: ChatGPT Will Become ‘ChatOMG!’ in 2024, Forrester Predicts
Moving Ally.ai into Live Marketing Implementation in 2024
At this point, Brimmer believes that Ally.ai can serve as a helpful assistant that can enable creative types to work faster. She explains that the technology will not be “shoved down anybody’s throat,” but also not just made available to anyone. Muthukrishnan says of 14 people selected for the experiment, only 11 ultimately received access to Ally.ai.
“Right now, we’re identifying people across the marketing team who can benefit the most from having GenAI as part of their martech stack,” says Brimmer. “We’ll roll that out in the first quarter of next year.”
Among early users will be blog posters in the marketing function’s content studio. Brimmer also sees a role for GenAI in marketing tasks that are mostly pulling together data for dissemination, such as producing periodic rate sheets for Ally’s auto dealer network.
“We’ll see where it will deliver the most value and the most efficiency, creating the greatest opportunity for quality,” says Brimmer. “We’ll step it out from there.”
Marketing staffers will have to make a case for tapping Ally.ai for a given project and will have to justify continuing use of the technology, according to Muthukrishnan. People who want to use it for a project will have access for 30 days to test it with their concept. At the end of that time the would-be user must produce a white paper to be vetted within Ally, including at its central AI council.
“You’ll only get 30 days to prove out the value. If you don’t, then you have to leave the Ally.ai platform. You’re out of the sandbox. You’ll have to come up with a different use case.”
— Sathish Muthukrishnan, Ally Financial
To get staff throughout Ally thinking about the technology, Muthukrishnan has published an AI playbook internally, detailing the kinds of projects that could be assisted by GenAI. Muthukrishnan says more than 185 use cases are under consideration at present.
An interesting one: Investor relations. Muthukrishnan says the technology would be used to analyze press releases, earnings calls and other performance-related communications from other banking companies. This is intended to help IR staff understand how the industry is doing, to help them prepare Ally’s own earnings communications package.
Meanwhile, Brimmer says that the experiment has already influenced what she tells college audiences when she speaks about marketing careers. Since ChatGPT came on the scene, she says, she’s frequently asked, “Will I have a job when I get out?”
She says she tells students to learn all they can now about how to use GenAI. The more they know, the more employable they will be. She’ll be hiring for such skills.
She also adds, to the students: “You might actually teach us some things.”