What is a token in Azure OpenAI?

A token is a chunk of text, roughly three to four characters of English, so a token is usually a bit shorter than a word. Both the prompt you send and the completion you get back are counted, which means the conversation history you resend on every turn is billed on every turn.

Is Provisioned Throughput cheaper than pay-as-you-go?

Only above a sustained utilisation threshold. Because provisioned capacity bills for time rather than usage, idle capacity costs exactly as much as saturated capacity. It wins on high, steady, predictable volume and on latency guarantees; it loses badly on spiky or exploratory workloads.

Why is my Azure OpenAI bill higher than my estimate?

The usual causes are output verbosity, prompt and context bloat from resending chat history or stuffing retrieved documents, silent retries, and agentic loops that make many model calls per user action. Each is invisible in a per-request estimate and compounds in production.

Azure OpenAI Pricing Explained: What You're Actually Billed For

July 28, 2026 · 9 min read

Azure OpenAI bills surprise people for a specific reason: the thing you are charged for is not the thing you count. You count requests, users, or features. Azure counts tokens — and the relationship between a request and its token cost is far looser than most estimates assume.

This is a structural explainer: what the meter actually measures, how the three deployment models differ, and which cost drivers reliably turn a modest estimate into an uncomfortable invoice.

On specific prices

Per-token rates vary by model, region, and deployment type, and they change often enough that any figure written into a blog post is wrong within a quarter. This article deliberately explains the model rather than quoting rates. For current numbers, use the official Azure OpenAI pricing page and the Azure Pricing Calculator — then validate against your own consumption data, which is the only source that reflects what you are really doing.

What a token is, and why it matters

A token is a chunk of text — roughly three to four characters of English, so slightly shorter than an average word. Both directions are metered: the prompt you send (input tokens) and the completion you receive (output tokens).

Two consequences follow, and they cause most of the surprises:

Input and output are priced differently. Output is normally the more expensive side. A change that makes responses longer costs more than a change that makes prompts longer.
Conversation history is re-billed every turn. A chat interface that resends the whole thread pays for that thread again on each message. Cost per turn grows as the conversation does, even if every user message is short.

The three deployment models

This is the single biggest fork in Azure OpenAI cost behaviour, and picking wrong is expensive in both directions.

Deployment	You pay for	Cost at zero traffic	Fits
Standard (pay-as-you-go)	Tokens consumed	Nothing	Spiky, exploratory, or early-stage workloads
Provisioned Throughput (PTU)	Reserved capacity over time	Full rate	High, steady volume; latency guarantees
Batch	Tokens, at a discount	Nothing	Asynchronous bulk work tolerant of delay

The critical asymmetry: provisioned capacity bills for time, not usage. An idle PTU deployment costs exactly what a saturated one costs. Teams migrate to provisioned throughput for predictable latency, then discover the economics only work above a sustained utilisation threshold. Below it, pay-as-you-go was cheaper the whole time.

The reverse mistake is just as common: running a heavy, steady production workload on Standard and paying a premium per token that a reservation would have absorbed.

The cost drivers that actually blow up bills

None of these show up in a per-request estimate. All of them compound in production.

Driver	Why it costs	What helps
Output verbosity	Output tokens usually cost more than input, and models pad when unconstrained	Set max_tokens, ask for structured or bounded output, stop rewarding long answers
Context bloat	Resent chat history and stuffed retrieval context are billed on every single turn	Summarise or window the history; retrieve fewer, better chunks
Retries	A retried call is a fully billed call, and failures cluster exactly when traffic spikes	Cap retries, use backoff, alert on retry rate rather than absorbing it
Agentic loops	One user action can become dozens of model calls, each carrying full context	Bound iterations, checkpoint, and track cost per completed task, not per call
Embeddings at scale	Cheap per token, but re-embedding an entire corpus on every change is not	Embed incrementally; only re-embed what actually changed
Idle provisioned capacity	Provisioned throughput bills for time — an unused deployment bills at full rate	Right-size units to real utilisation, or move spiky workloads to Standard

Agentic workloads deserve particular caution — the multiplier between “one user request” and “model calls made” is where estimates break down most violently. We covered that failure mode in detail in Your Azure AI Bill Is About to Explode.

What the token price does not include

The model meter is rarely the whole AI bill. Budget for the surrounding infrastructure, which is billed under entirely separate meters and is easy to omit from a business case:

Vector search and retrieval — the index behind a RAG system is its own ongoing service cost
Storage — documents, embeddings, and chat transcripts
Content safety and moderation where metered separately
Fine-tuning — a training charge, plus a hosting charge for as long as the customised model exists
The application tier — compute, networking, and egress serving the thing

A fine-tuned model that nobody calls still costs money to keep deployed. So does a provisioned deployment left over from a proof of concept. These are the AI equivalent of an unattached disk, and they are just as easy to forget.

Estimating before you commit

The arithmetic is simple; the honesty about inputs is the hard part.

monthly cost  =  requests/month
              x  (avg input tokens  x  input rate
                + avg output tokens x  output rate)
              x  (1 + retry rate)
              x  calls per user action

The last two multipliers are the ones people leave out, and they are frequently where the real money is. A workload averaging three model calls per user action with a ten percent retry rate costs more than three times a naive single-call estimate.

Measure average input and output tokens from real traffic rather than guessing. Prompts in production are almost always longer than the ones in the prototype.

Attributing the spend once it is running

One practical trap when you go to allocate AI costs: a deployment name is an arbitrary label, not a model identity. Deployments are frequently named after whatever model they originally served and then re-pointed at a newer one, so a deployment called after a two-year-old model may be serving something entirely current at a completely different rate. Attribute cost by the deployment's actual configuration, not by inference from its name — and audit that mapping periodically.

For per-team or per-feature attribution you need usage captured at the call site, tagged with something meaningful to the business. The billing data alone will tell you what a deployment cost; it will not tell you which product line caused it. CostBeacon's AI spend tracking and unit economics exist for exactly this gap.

Frequently asked questions

How is Azure OpenAI priced?

Per token on Standard deployments, with input and output billed at different rates and output normally dearer. Provisioned Throughput reserves capacity and bills for time regardless of usage. Batch trades latency for a discount.

Is Provisioned Throughput cheaper?

Only above a sustained utilisation threshold, because idle reserved capacity bills at full rate. It wins on steady, high volume and on latency guarantees; it loses on spiky workloads.

Why is my bill higher than my estimate?

Almost always output verbosity, context bloat from resent history or stuffed retrieval, silent retries, or agentic loops multiplying calls per user action — none of which appear in a per-request calculation.

Does an idle deployment cost anything?

On Standard, no. On Provisioned Throughput, yes — at the full rate. A fine-tuned model also carries a hosting charge for as long as it stays deployed, whether or not anything calls it.

See what your Azure AI spend is actually doing

CostBeacon tracks Azure OpenAI consumption alongside the rest of your Azure bill, and attributes it to the teams and features causing it.

Join the waitlist →