Enterprise AI budget failures are almost never caused by choosing the wrong vendor. They are caused by failing to model costs accurately before committing to enterprise volumes. Token-based AI pricing introduces cost dynamics that are structurally different from traditional software licensing, and the gap between initial projections and actual spend routinely surprises even financially sophisticated procurement teams.
This guide explains how AI pricing works at the mechanical level, identifies the modelling errors that cause projections to fail, and provides the framework for building cost models that hold up in production. For contract and negotiation context, see the Enterprise AI Procurement Guide and our OpenAI Enterprise pricing benchmarks.
How Token Pricing Works: The Mechanics
A token is approximately 3–4 characters of text, or roughly 0.75 words. A 1,000-word document contains approximately 1,300–1,400 tokens. AI models are priced per million tokens processed, with separate rates for input tokens (text you send to the model) and output tokens (text the model generates in response).
The critical asymmetry: output tokens cost 3–5× more than input tokens across all frontier AI providers. For GPT-4o, the input rate is $2.50/M tokens and the output rate is $10.00/M tokens — a 4:1 ratio. For Claude 3.5 Sonnet, the ratio is similar. This asymmetry is not a pricing quirk; it reflects genuine computational cost differences in how inference works. But it fundamentally changes cost modelling because most initial estimates focus on the volume of text being sent to the model, not the volume of text being generated.
| Model | Input (per 1M tokens) | Output (per 1M tokens) | Output/Input Ratio |
|---|---|---|---|
| GPT-4o | $2.50 | $10.00 | 4.0× |
| GPT-4o mini | $0.15 | $0.60 | 4.0× |
| o1 | $15.00 | $60.00 | 4.0× |
| Claude 3.5 Sonnet | $3.00 | $15.00 | 5.0× |
| Gemini 1.5 Pro | $1.25 | $5.00 | 4.0× |
| AWS Bedrock (Claude) | $3.00 | $15.00 | 5.0× |
The Five Most Common Cost Modelling Errors
Our AI practice has reviewed more than 80 enterprise AI cost models over the past three years. The same modelling errors appear repeatedly, causing systematic underestimation of actual costs.
Error 1: Ignoring System Prompts and Context
In production AI deployments, every API call includes not just the user's query but also a system prompt that sets context, persona, and instructions for the model. System prompts typically run 500–2,000 tokens and are charged as input tokens on every API call. A production customer service AI making 100,000 calls per day with a 1,500-token system prompt incurs 150 million additional input tokens daily — purely from context that never changes and therefore appears "free" in naive cost models. At $2.50/M tokens, this adds $375/day, $137,000/year, before processing a single user query.
Error 2: Underestimating Output Volume
Most organisations model their AI use cases by volume of input — "we process 50,000 documents per day" — without adequately modelling the output generated per input. Output length varies dramatically by use case: a summarisation task might generate 200 tokens per 2,000-token input document; a customer email response might generate 500 tokens per 150-token query; a code generation task might generate 2,000 tokens per 100-token specification. Getting the input/output ratio wrong by 2× doubles your cost model error at a 4:1 pricing ratio.
Error 3: Assuming Uniform Model Selection
Enterprise AI deployments typically use multiple model tiers: a lower-cost model for simple, high-volume tasks and a frontier model for complex, lower-volume tasks. Cost models that assume all workloads run on the same model either overestimate (if they assume frontier model pricing for everything) or underestimate costs through "model sprawl" — where individual teams and use cases gradually migrate to more capable and more expensive models without budget reapproval.
Error 4: Ignoring Retry and Error Token Consumption
Production AI systems routinely encounter timeout errors, rate limit errors, and quality validation failures that trigger retries. Each retry consumes tokens. A system with a 3% retry rate adds 3% to token consumption — small in isolation, but significant at scale. Safety system refusals — where the model declines to process a request — still consume input tokens in most API billing models. These overhead costs are rarely modelled in initial projections.
Error 5: Not Accounting for Context Window Growth
Multi-turn conversational AI systems accumulate context in their context window as a conversation progresses. A customer service interaction that involves 10 turns of conversation charges input tokens for the full conversation history on each turn — meaning turn 10 includes 9 previous turns of context as input. Cost models that treat each turn as an independent transaction with only the new message as input systematically underestimate costs by 3–5× for conversational use cases.
From the Advisory Desk: The most significant AI cost surprise we see in practice occurs in retrieval-augmented generation (RAG) deployments. RAG systems augment user queries with retrieved document chunks from a knowledge base, inserting those chunks into the context window as input tokens. A RAG implementation that retrieves 5 document chunks of 500 tokens each adds 2,500 tokens to every query's input — on top of the system prompt and the user's question. Enterprise RAG systems processing 500,000 queries per month can easily consume 2–3 billion tokens monthly in retrieved context alone. This is almost never in the initial cost model.
Building an Accurate AI Cost Model
A commercially defensible AI cost model requires four inputs that most organisations do not have at the start of a deployment: actual system prompt length, realistic input/output token ratios by use case, conversation turn distribution for multi-turn systems, and RAG context volume for knowledge-augmented systems. The only way to obtain these inputs accurately is from a pilot deployment at meaningful scale.
The model framework has five components. First, measure baseline consumption from a 30-day pilot processing representative workloads on the intended model. Record input tokens, output tokens, system prompt tokens, context tokens, and retry volume separately. Second, apply a scale factor to extrapolate from pilot volume to production volume, accounting for seasonal peaks and adoption growth. Third, model the model tier split — what percentage of queries will use lower-cost models versus frontier models. Fourth, add infrastructure overhead — retry rate (typically 2–5% in production), safety refusals (0.5–2% depending on use case), and monitoring/evaluation API calls. Fifth, apply your negotiated enterprise rate and calculate committed volume at the 80th percentile of your modelled range to avoid overspend risk.
Comparative AI Pricing: Choosing the Right Model for Cost
Model selection has a more significant impact on AI costs than pricing negotiation. Choosing GPT-4o mini over GPT-4o for appropriate use cases reduces costs by approximately 94% at list prices — no negotiation required. The challenge is identifying which use cases genuinely require frontier model capabilities and which can be served adequately by smaller, cheaper models.
The use case framework for model selection considers three dimensions: reasoning complexity (does this task require multi-step reasoning, or is it primarily pattern matching?), quality sensitivity (is a small quality regression acceptable, or does the use case require consistently high-quality outputs?), and volume scale (does high transaction volume make model cost material to total cost of ownership?). High reasoning complexity, high quality sensitivity, low volume favours frontier models. Low reasoning complexity, moderate quality sensitivity, high volume strongly favours smaller models. Our GPT vs Claude vs Gemini enterprise comparison provides a systematic comparison across quality and cost dimensions.
The Batch API Opportunity
All major AI providers offer batch processing APIs that process large volumes of requests asynchronously at 50% of real-time API pricing. Batch API is appropriate for any use case that does not require immediate response — document processing, overnight analysis, bulk classification, periodic reporting, and similar workloads. Organisations that identify 40–60% of their AI workload as batch-eligible can reduce their effective AI cost by 20–30% without any additional negotiation.
Embedding Costs: The Hidden Infrastructure Expense
Retrieval-augmented generation and semantic search systems require embedding models that convert text into numerical vectors for storage and retrieval. Embedding costs are billed separately from inference costs and are easy to overlook. A knowledge base of 10 million document pages, embedded at $0.13/M tokens with an average page length of 300 tokens, incurs initial embedding costs of $390 — trivial. However, if that knowledge base is re-embedded monthly to incorporate updates, the recurring annual cost is $4,680. For knowledge bases with frequent updates, embedding costs can become material relative to inference costs and must be modelled explicitly.
For the full picture on AI total cost of ownership — including infrastructure, personnel, and integration costs beyond API charges — see our enterprise AI total cost guide. For the procurement strategy context, see the AI Procurement Guide and our AI procurement advisory service. For cross-cluster context, our cloud contract negotiation guide addresses how AI costs fit within broader cloud commercial frameworks.