AI usage within large enterprise organizations is now ubiquitous. Yet only 2% of C-suite executives in the Forbes Research 2026 AI Survey describe AI as transformative to their business. After several years of deploying tools that help employees work more efficiently, organizations are shifting their focus to agentic systems that can work independently. But many leaders are finding it harder to scale autonomous systems than expected.
The models are only part of the equation. Architecture, integration, governance and keeping security and trust intact as autonomy scales are the true bottlenecks.
This was the focus of a Forbes CIO dinner I hosted in Chicago in partnership with AWS and Salesforce, bringing together top technology leaders for a conversation on why autonomous AI initiatives stall, and what the leaders moving past that are doing differently.
“In the last six months, CEOs are leaning in much deeper, prioritizing AI and then trying to figure out how do they change the culture of their companies to embrace it.”
Chris Schaaf, CxO enterprise technologist and advisor at AWS, echoed the trends we’re seeing in our research, noting that truly material impact — where AI drives 25-50% of how a company operates and goes to market — has yet to occur at most organizations.
What has changed is who is paying attention. CIOs led the first wave of AI adoption, but as autonomous systems begin to reshape workflows and revenue, the rest of the C-suite and the board are becoming more involved. “In the last six months, CEOs are leaning in much deeper, prioritizing AI and then trying to figure out how do they change the culture of their companies to embrace it,” said Schaaf.
Chris Schaaf, CxO Enterprise Technologist And Advisor, AWS
Donte Tatum for Forbes
That shift in leadership attention changes the conversation. AI’s early era was about efficiency. The leaders now leaning in are focused on something bigger: growth, faster decision-making and competitive resilience. Business outcomes are the measure of this wave, not productivity gains.
The Process Problem
Why are so many autonomous initiatives stuck? Mark Wanish, CIO and chief architect at Salesforce, shared that cultures resistant to change are the largest challenge. In many cases, people are still doing the work they have always done, even with the latest AI tools available to them.
The companies that are still stuck in pilot mode are trying to layer AI onto existing processes, and that approach can only go so far. Schaaf compared the moment to the enterprise resource planning wave of the 1990s, when companies bought new systems, but didn’t reengineer processes that would have made them valuable. They took what they already had and tried to make the new technology fit. Most of the value was not realized. He thinks we are at a similar tipping point now.
Unless the company is a startup, generative AI and autonomous systems often run on legacy, disparate systems. That creates interoperability challenges not just at the architecture level but also in the data layer, and 97% of respondents in our AI survey cite integration with existing systems and processes as a top challenge over the next two years. Those moving beyond pilots are rebuilding their operations around agents and autonomous systems.
The Data Question Returns
One surprise of the evening: the ‘is our data ready?’ conversation is back. It dominated the early days of generative AI, went quiet as the conversation shifted to use cases, and has now returned because agents make decisions based on data. Wanish pushed back on the instinct to “fix your data first,” arguing that pursuing perfect data is a competitive disadvantage, and instead stressed the importance of context.
Mark Wanish, CIO And Chief Architect, Salesforce
Donte Tatum for Forbes
“You need to figure out what is the right data, then you need to add context to that data — what’s the relationship to this business process, what’s the relationship to the other data that’s relevant … once you can take that and align it to business, that’s how you align it to an outcome.”
The relationship between the data and the business process is what matters — agents reason on context, not on data alone. Without it, said Schaaf, you spend lots of money on tokens with disappointing results.
Accountability At Scale
What about when an agent makes a mistake? Who is accountable? When this technology was in the early stages, it was the CIO and the technology teams. They built it, and they owned it. Today, it is the responsibility of the business or technology leader who deployed the agent.
“Autonomous agents are digital employees. They require the same oversight as human employees, and accountability for their actions belongs to the manager responsible for them.”
Ideal agentic systems are partnerships across the enterprise, with security and governance in place and clearly defined business goals. The accountability is joint, but the lines of ownership are clear. Wanish was direct about where ownership sits: “Autonomous agents are digital employees. They require the same oversight as human employees, and accountability for their actions belongs to the manager responsible for them.”
One of the most fascinating areas of debate is how AI agents should be managed — like software, or like employees? The answer, increasingly, is the latter. Over half of respondents in the Forbes Research 2026 AI Survey (59%) say their organization is beginning to manage AI agents like employees, with identity controls, role assignments and performance monitoring. Enterprises are building management structures around them as digital workers — and the accountability follows.
A Leadership Challenge
At the conclusion of the conversation, I asked both panelists for the one thing they wanted to take into work the next day. Schaaf’s answer was simple: do not do anything AI-related unless it is tied to a business outcome. Wanish offered a different kind of challenge: stop redesigning processes the way you would have a decade ago. Use agents to help define what the new process should look like.
The organizations that successfully scale AI over the next several years may be the ones most capable of reshaping leadership structures, workflows, governance and accountability fast enough to operationalize autonomy responsibly and deliver measurable results. The autonomy trap is not really a technology problem. It’s a leadership challenge.
Leave a comment