Agentic artificial intelligence is fundamentally reshaping how enterprises approach their technology stacks, data strategies, and operational costs. Over the past year, the maturation of agentic AI has forced IT leaders to rethink assumptions about infrastructure, GPU requirements, and the economics of AI consumption.

Speaking at the Dell Technologies World conference in Las Vegas, Dell's global chief technology officer, John Roese, noted that the era of one-shot AI tasks, such as chatbots, is giving way to systems that can handle complex objectives autonomously. "We have shifted our assumption in that the use of AI is no longer a one-shot task like a chatbot," said Roese. "It's about handing objectives to the AI system, and that's what agents do today." He pointed to Google's redesign of its search engine, where users provide an objective and the system orchestrates multiple agents to build a full page of results.

The GPU myth and enterprise inference

During the initial AI boom, there was a rush to secure large numbers of graphics processing units (GPUs) for model training. Roese, however, argued that enterprise requirements differ vastly from those of hyperscalers. "The myth out there is that enterprises need thousands of GPUs," he said. "Our biggest workload inside of Dell only sits on 16 GPUs and supports 40,000 people. You don't need thousands of GPUs in an enterprise, because for each workload, agent or project, you only need a handful of GPUs, sometimes half a GPU."

This is because enterprise AI is overwhelmingly focused on inference rather than training. For agents, no training is required, only inference. However, the architecture for inference workloads is evolving. Earlier chatbot deployments placed a light load on central processing units (CPUs), but agentic AI uses external tools, communication protocols, and knowledge graphs—components that do not naturally reside in the GPU. Roese noted that the balance shifts: "When you move to agentic, it's almost balanced. The number of CPUs and GPUs are very similar, about maybe for every two GPUs you have a CPU. You don't just build an AI infrastructure with a pile of GPUs—you build it with GPUs and traditional CPU compute."

Air-gapped frontier models and edge deployment

Enterprises now benefit from evolving model deployment options. A year ago, the most capable frontier models were only accessible through cloud APIs. Today, hyperscalers enable top-tier models to run on-premise via services like Google Distributed Cloud. Roese highlighted that private models can be deployed in multiple topologies: "You can consume it in a virtual private cloud or your datacentre, and you can air-gap it from everything else. We didn't have any of those options, except the API one, a year ago."

At the same time, AI is moving to the edge in a structured manner. Agentic frameworks such as OpenClaw run natively on devices and AI PCs. Roese described this as "incredibly powerful, and not a fad," enabling agents to operate locally without constant cloud connectivity.

Re-architecting the data layer

The performance demands of agentic AI require a fundamental rethinking of data infrastructure. Roese warned that traditional storage systems bolted onto compute clusters are insufficient. Organizations must build knowledge and context layers that include vector databases, graph databases, and data annotation tools, integrated deeply into compute. The bottleneck, according to Roese, is data delivery speed: "One of the performance bottlenecks is you can't get data fast enough to the GPUs to do the work. The GPUs you're paying for are sitting idle, waiting for data." To address this, Dell's AI data platform interfaces with Nvidia's Cuda-X, running data layer services at GPU speed and reducing latency.

Mastering tokenomics and model routing

As model deployment options multiply with varying pricing, IT leaders must manage AI consumption costs. Roese noted that while cost per token declines, overall AI spending will not decrease—"there's no path where it becomes cheaper to do AI." Enterprises must treat AI as an arbitrage game. For example, in specification-driven development, an agentic framework spawning dozens of coding tasks can be optimized by routing complex planning tasks to expensive frontier models and routine coding to smaller, on-premise open-source models where only energy costs apply. "Building a piece of software and doing spec-driven development might have four or five different economic paths to ultimately get to the best overall economic efficiency," said Roese. Mastering model routing will be a competitive differentiator, lowering product development costs.

The human element

The most challenging aspect of operationalizing agentic AI is adapting the workforce. Roese described human jobs as "containers of work" that include hygiene, productivity, coordination, and expert tasks. Agents cannot perform entire jobs, but they can execute specific types of work within these containers. Dell audited 6,400 jobs across its business to assess AI's impact. "The first thing we realised is every single job in the company is going to change," said Roese. "I'm taking work out of the job and removing stuff from the container. If the container is now only half full, do I need half the number of people, or do I expand that by half? Am I able to do more expert work?"

The impact is so profound that change management has become a key remit of IT leadership. Roese noted that for the past four months, he had spent 50% of his time dealing with human dynamics. "AI has ceased being a technology and an ROI discussion. It's now very much an organisational and human dynamic discussion. You simply can't use these things unless you fully understand how you're going to adapt the human population around them."

As agentic AI continues to evolve, enterprises are learning that infrastructure, data, and workforce strategies must be redesigned in tandem. The new playbook requires a clear-eyed view of real GPU needs, a willingness to deploy models across cloud and edge, a data architecture that can keep up with GPU speeds, a cost optimization strategy through intelligent model routing, and a deep commitment to managing workforce transitions. These elements together define how organizations will harness the full potential of agentic AI in the coming years.

Source:ComputerWeekly.com News