Companies confront Gen AI cost pressures

Business leaders are shifting their focus from Gen AI deployment to the economics of running AI agents at scale, according to a McKinsey report. After the first two years of adoption centered on access, experimentation and deployment, the next phase is being shaped by financial sustainability and return on investment.

Enterprises are moving beyond efforts to reduce AI costs and are increasingly expected to prove measurable business value. CFOs and CIOs are seeking evidence that AI spending is generating tangible returns, but many companies still lack systems to track the business impact of AI-driven decisions.

McKinsey identified six major factors behind agentic AI operating expenses, including long-lived context, response refinement, autonomous system variability, advanced reasoning for simple tasks, agent orchestration and information structure. Agentic AI tasks can use nearly 1,000 times more tokens than conventional code reasoning or chat-based AI tasks, making per-token pricing less useful as a measure of enterprise costs.

The report also said nearly 60 per cent of operating costs are spent on verifying and refining responses. It noted that About 60 per cent of an agentic task’s costs are tied to refining answers, while prompt design, context length, formatting, language and data structure can all affect token consumption.