As AI Agents Take on Tasks in the Real World, New Risks Emerge
By Driss Temsamani, managing director, head of digital at Citi; author, The Agentic Bank
Simple Subscribe
Subscribe Now!
We spent two years marveling at AI that could write, reason, and pass the bar exam. Now it’s learning to walk, grasp, and build.
Over the past few years researching AI’s role in banking, treasury, and finance, I’ve watched intelligence evolve from machine learning into an agentic operational force. Since our Netscape moment in 2022 with ChatGPT, another shift is underway, one that matters more than fluent conversation ever could: AI is moving from language models to physical intelligence. This mirrors what I witnessed in financial infrastructure modernization. The moment intelligence stops merely recommending or executing and instead steps into the world, learning from its own experience, everything changes.
For decades, we evaluated AI through Alan Turing’s lens. In 1950, he reframed the question: Could a machine converse convincingly enough to be considered intelligent? That framing proved extraordinarily powerful. It gave us game-playing systems, expert advisors, and eventually generative models capable of reasoning across domains with remarkable fluency.
But embedded in that success was an unexamined assumption: that once intelligence could speak and reason, it was largely complete.
It wasn’t.
This brings us to a paradox that quietly challenged those assumptions. In 1988, Hans Moravec observed something counterintuitive. Machines could outperform adults on formal reasoning tasks yet struggled to replicate the motor skills of a one-year-old. Abstract reasoning was comparatively easy to formalize. The truly difficult part was everything humans do without conscious thought: perception, balance, navigation, adaptation to messy reality.
That paradox explains why AI progress appeared uneven for decades. Machines mastered chess and translation but repeatedly failed in robotics. High-level cognition proved tractable. Low-level sensorimotor intelligence remained brutally hard.
What has changed is not the insight. It’s the environment around it.
Sensors are now ubiquitous. Compute has moved to the edge. Industrial systems are instrumented and connected. The physical world, while still governed by immutable laws, has become increasingly observable.
Moravec’s paradox is no longer an explanation for stagnation. It’s a roadmap for breakthrough.
The New Risks
This is where Physical AI becomes fundamentally different. It isn’t smarter automation. It’s intelligence that can perceive its environment, reason about what it perceives, and act directly through physical systems, without waiting for human mediation.
That distinction matters because digital intelligence hit a natural plateau. Language-centric AI flourished in a forgiving environment. Words can be rewritten. Recommendations can be ignored. Errors are cheap and reversible.
The physical world operates under different rules. Action introduces irreversibility. A manufacturing defect propagates downstream at scale. A control system error causes damage. Mistakes aren’t just incorrect outputs, they are costly events.
As complexity increased across supply chains, factories, energy grids, and transportation networks, human-mediated decision loops became the bottleneck. The time between sensing a problem and acting on it stretched from minutes to hours, sometimes days. Intelligence confined to dashboards simply cannot keep pace with systems that operate continuously at machine speed.
What this means: Physical AI collapses the separation between thinking and doing. It operates through closed feedback loops: sensors capture real-time signals, models interpret them under physical constraints, control systems translate decisions into motion, and outcomes are continuously learned from and incorporated into future behavior.
Performance is no longer measured in eloquence. It’s measured in throughput, uptime, defect rates, and resilience.
But perception and motion alone don’t create autonomy. A system that can sense and act, yet cannot sustain intent, remains reactive. This is where Agentic AI becomes essential.
Agentic AI maintains goals over time. It sequences actions, coordinates resources, and adapts as conditions change. Rather than asking “what is the next command?”, it asks “what outcome am I responsible for, and how do current conditions affect my ability to achieve it?”
When combined with Physical AI, this creates closed-loop autonomy across time. Physical AI provides the ability to sense and act. Agentic AI provides the logic that governs when to act, how to sequence actions, and when to escalate decisions.
I’ve seen this transition firsthand in financial systems, from periodic batch processing to continuous real-time settlement, from human-approved exceptions to algorithmic resolution within defined risk envelopes. The pattern is consistent across industries. Intelligence moves from episodic oversight to continuous governance.
The Economic Impact
The implications extend beyond technology. The most consequential impact is economic.
For most of industrial history, productivity has been tightly coupled to human availability. Growth required hiring, training, and retaining people. Physical AI breaks that coupling.
When intelligence becomes embedded in machines and infrastructure, productivity scales through assets rather than headcount. Output increases not because more people are added, but because systems operate more continuously, precisely, and with less downtime.
This doesn’t eliminate the need for human judgment. It elevates where that judgment matters most. The question shifts from how many people to where human expertise creates the most value.
The traditional wage bill is partially replaced by a depreciation bill. Fixed investment in autonomous systems substitutes for variable labor costs. Capital expenditure rises upfront, but operating costs stabilize over time. Productivity becomes more predictable and less sensitive to labor volatility.
This is not just efficiency. It’s a redefinition of how growth is achieved.
Competitive advantage shifts away from generic software toward domain-specific intelligence. Organizations with decades of operational data embedded in their assets develop structural advantages. Physical AI trained on long histories of real-world behavior accumulates depth that cannot be replicated quickly. These data moats reflect knowledge of how systems behave under stress, variability, and edge conditions.
Physical AI has taken longer to emerge precisely because it operates under constraints that cannot be abstracted away. Gravity doesn’t negotiate. Materials fatigue. Sensors drift. The physical world imposes hard limits.
Far from being a weakness, this grounding gives Physical AI its strategic weight. It forces intelligence to become reliable, accountable, and precise. Unlike language models trained on broadly similar data across domains, Physical AI is deeply contextual. A refinery, a factory floor, a power grid, and a logistics hub each impose distinct realities. Value accumulates over time through exposure to real systems.
Which brings us to the real challenge. As intelligence becomes both physical and agentic, the challenge facing leaders is no longer primarily technological. It’s institutional.
When systems can perceive, decide, and execute autonomously, intelligence becomes embedded in the operational fabric of the organization. Decisions are continuous, not episodic. Oversight shifts from direct control to boundary-setting.
Leaders must define objectives, constraints, and acceptable risk envelopes within which autonomous systems operate. Performance, resilience, and trust increasingly depend on the quality of those boundaries. Governance becomes architectural rather than procedural.
The bottom line: Accountability doesn’t disappear. It moves upstream into system design, training data, simulation assumptions, and escalation rules.
Turing gave us a way to recognize intelligence through conversation. Moravec reminded us that intelligence must be embodied. Physical AI, guided by agency, is where those ideas converge and become operational reality.
As intelligence leaves the screen and enters the world, it stops being something organizations experiment with. It becomes something they are built around.
The defining challenge of the coming decade won’t be building intelligence. It will be integrating intelligence into physical, economic, and regulatory systems in ways that are resilient, accountable, and aligned with long-term objectives.
We’re not just automating tasks. We’re redesigning the relationship between intelligence and action in the physical world.
