Enterprise AI Spending: Where IDC’s $632 Billion Goes

The numbers are staggering. In August 2024, IDC published its Worldwide AI and Generative AI Spending Guide, projecting that worldwide spending on AI — including AI-centric systems, software, and services — will reach $632 billion by 2028, growing at a compound annual growth rate of 29.0% over the 2024–2028 forecast period. Grand View Research separately valued the global AI market at $390.91 billion in 2025, projecting it to hit $3.5 trillion by 2033 at a CAGR of 30.6%. The Stanford HAI AI Index 2025 report documented that US private AI investment alone reached $109.1 billion in 2024 — twelve times China's $9.3 billion in the same period.

Enterprise generative AI spending specifically hit $13.8 billion in 2024, a sixfold increase from $2.3 billion in 2023, according to Menlo Ventures' 2024 State of Generative AI in the Enterprise report. Every indicator points in the same direction: organizations are pouring unprecedented capital into AI.

But here is the number that should concern every executive approving an AI budget:McKinsey's Global Survey on AI found that only 6% of companies capture disproportionate value from their AI investments. The rest are spending heavily and getting incrementally better chatbots, mildly improved internal search, and dashboards nobody opens after the first month.

The question is not whether to invest in AI. That debate ended years ago. The question is whether your investment is going to the right places.

Where the Money Actually Goes

When an enterprise allocates its AI budget, the spending typically falls into four categories. Understanding how these categories consume capital reveals why most organizations struggle to extract value proportional to their investment.

Category 1: Model API Costs and Licensing

The most visible line item is model access. Organizations pay for API calls to OpenAI, Anthropic, Google, or other providers. They license enterprise tiers for higher rate limits, longer context windows, and service level agreements. Some invest in fine-tuning runs or host open-weight models on their own infrastructure. This category is growing rapidly but is often the smallest portion of total AI spending — typically 10–20% of the total cost of ownership for an enterprise AI deployment.

Category 2: Infrastructure and Compute

GPU compute for training and inference dominates infrastructure costs. Whether organizations are running workloads on AWS, Azure, GCP, or on-premises clusters, the compute bills are substantial. Nvidia's data center revenue reached $47.5 billion in fiscal year 2024, up from $15.0 billion in fiscal year 2023 — a direct reflection of enterprise infrastructure spending. Vector databases, embedding pipelines, retrieval-augmented generation infrastructure, and the storage behind them add another significant layer.

Category 3: People and Professional Services

Hiring ML engineers, data scientists, AI platform engineers, and prompt engineers has become one of the most expensive talent markets in technology. Many organizations supplement internal teams with consulting engagements from firms like McKinsey, Deloitte, Accenture, and specialized AI consultancies. In our experience, people costs frequently represent the largest share of total AI program spending.

Category 4: Integration, Governance, and Operations

This is the category that determines whether AI spending generates value — and it is consistently the most underfunded. Integration work connects AI capabilities to actual business processes. Governance ensures those capabilities operate within regulatory and organizational boundaries. Operations keeps them running reliably in production. In general, most organizations underinvest significantly in this category relative to model and infrastructure spending. The organizations that McKinsey identified as capturing disproportionate value invest far more heavily here.

The Governance Gap

Deloitte's 2026 State of AI in the Enterprise report — surveying 3,235 leaders across 24 countries — found that only 21% have mature governance frameworks for autonomous AI agents. This is not a minor gap. It is the gap that explains why 42% of companies in the S&P Global Market Intelligence 2025 survey abandoned most of their AI initiatives.

Without governance, AI projects follow a predictable arc: a team builds an impressive prototype, leadership gets excited, the project goes to compliance review, compliance asks questions nobody can answer (what decisions did the agent make? who approved them? can you reproduce the decision chain?), and the project is shelved. The investment in model APIs, compute, and engineering time is written off.

The tragedy is that governance is not expensive relative to the rest of the AI stack. The tools exist. The frameworks exist. But organizations treat governance as something to add before launch rather than something to build from the foundation. By the time they realize it is missing, they have already spent the budget.

Where Spending Should Go Instead

Based on the patterns we see in organizations that successfully extract value from AI, here is how enterprise AI budgets should be restructured:

Invest in the Abstraction Layer, Not Just the Model

The model landscape changes every quarter. New providers emerge, existing providers change pricing, open-weight models close the performance gap with proprietary ones. Organizations that bet their entire architecture on a single provider face switching costs measured in months and millions. Our platform's SmartModelRouter turns model selection from a locked-in strategic bet into an operational parameter, routing requests across multiple model families and provider integrations based on cost ceilings, latency targets, and capability requirements. An intelligence control gives teams direct control over the cost-vs-quality tradeoff per workflow: route high-volume classification to efficient open-source models, reserve frontier reasoning for tasks where quality justifies the cost. With automatic failover across providers, a pricing change is a configuration update, not a rewrite.

Fund Operations as a First-Class Category

AI in production is a distributed system. It needs monitoring, alerting, cost management, and incident response — the same operational rigor applied to any production service. The organizations succeeding with AI are the ones that fund AI operations teams, build or adopt observability tooling, and treat token budgets with the same discipline as cloud compute budgets. This is not glamorous work. It is the work that determines whether AI investments compound or evaporate.

Build Governance Before Building Features

Every dollar spent on AI features without governance infrastructure is a dollar at risk of being written off when the project hits compliance review. Audit trails, human-in-the-loop approval workflows, access controls, and data lineage tracking should be the first investments, not the last. They are cheaper to build early and catastrophically expensive to retrofit.

The $632 Billion Question

IDC's $632 billion projection is not wrong. The money will be spent. The question is whether it will be spent in ways that generate proportional value — or whether most of it will fund expensive experiments that never reach production.

The pattern is already clear. Organizations that invest heavily in models and compute but underinvest in governance, operations, and integration end up with impressive demos and abandoned projects. Organizations that allocate budget to the full stack — from model access through production operations — end up with AI capabilities that actually transform their businesses.

The 6% that McKinsey identified are not spending more than everyone else. They are spending differently. They are building the operational foundation first and adding capabilities on top of it, rather than building capabilities and hoping the foundation will materialize.

Our platform helps organizations get this sequence right. The SmartModelRouter optimizes per-token costs with intelligent routing and the Intelligence Slider. Multi-Cloud Management provides unified provisioning across AWS, Azure, and GCP, so infrastructure spending stays competitive across providers. And production-grade governance — Audit System, RBAC, DLP Scanner — is built in from day one, so the compliance review that kills most projects never becomes a budget-consuming crisis. The models are the easy part. The hard part is everything around them. And that is exactly where the 6% invest their marginal dollar.