LLM Token Cost Explained: How Usage Turns Into a Bill

Hosted language models are billed by tokens, and most cost surprises trace to how tokens accumulate rather than to the price list. Every request re-sends its full context, output is priced higher than input, and model tier multiplies everything. Understanding the mechanics turns the bill from a mystery into a model you can forecast and control. This guide explains it, drawing on FISTA Solutions' AI enablement practice. The conceptual foundation is in what are tokens in ai and reduction techniques in llm api cost optimization.

What is a token?

A token is the unit of text a language model reads and writes: commonly a word fragment, a short whole word, a number chunk, or punctuation. English text runs roughly three-quarters of a word per token on average, so a thousand tokens is about seven hundred fifty words, though the ratio varies with vocabulary, formatting, and language. Code, other languages, and structured data often use more tokens per unit of meaning. Providers publish tokenizers and counting tools; measure rather than assume.

How are tokens priced?

Token type	What it is	Relative price	Lever
Input tokens	Everything sent to the model: system prompt, context, history, user message	Base	Context size, prompt length
Output tokens	Everything the model generates	Higher than input	Output length and format
Cached input tokens	Repeated prefix reused from a recent request	Lower than input	Prompt structure, cache hit rate
Tool and function calls	Definitions sent and calls returned, counted as tokens	Input and output	Number and size of tools
Images and audio	Converted to token equivalents or priced per unit	Varies	Media size and resolution

Prices differ by model tier by large multiples, and providers change them; verify current pricing pages when estimating.

What makes up the input tokens of a request?

The system prompt and instructions, tool definitions, retrieved documents or context, conversation history, and the user's message. In retrieval systems, context often dominates. In multi-turn conversations, history grows each turn and is re-sent every time, so cost per turn rises through a conversation unless history is summarized or trimmed. Context limits are in what is a context window.

What drives output tokens?

Answer length, verbosity of style, restated inputs, explanations, and format. Structured outputs with tight schemas produce fewer tokens than prose; explicit length limits and concise instructions reduce generation. Reasoning-heavy modes on some models generate additional internal tokens that are billed; check how each model accounts for them. Format control is in what is structured output.

How do you calculate cost per request?

Cost per request equals input tokens times input price, plus output tokens times output price, with cached tokens priced at the discounted rate, plus any tool, image, or audio charges. Total cost equals cost per request times requests, summed across features and models. Build the estimate from measured token counts on representative requests, not from word-count guesses.

What is a worked illustration?

Take a retrieval-augmented assistant. Each request sends a system prompt, several retrieved passages, a few turns of history, and the user's question, and returns a moderate-length answer. Input tokens might be several thousand per request, dominated by retrieved passages; output tokens a few hundred. If the system prompt is cached, that portion is charged at the lower rate. Cost per request follows from multiplying each token category by its current price. If retrieval is tightened to return fewer, more relevant passages and answers are constrained in length, tokens per request can fall considerably, reducing cost per request by a similar proportion at unchanged volume. Retrieval trade-offs are in rag vs long context.

Why do bills grow faster than traffic?

Because tokens per request creep: prompts accumulate instructions, retrieval returns more, conversations run longer, outputs become verbose, agents retry or loop, and teams switch to larger models for safety. None of these show up in request counts. Tracking tokens per request by feature reveals them. Attribution design is in how to build an ai cost dashboard.

How do model tiers change the picture?

Price per token differs by multiples between small, mid, and frontier models. Routing most requests to smaller models and reserving frontier models for hard cases changes the blended price per token dramatically, provided evaluation confirms quality. Routing is in what is an llm router and the comparison of hosted versus self-hosted economics in llm api vs self-hosted llm.

How does caching change cost?

Cached input tokens are billed at a discount when a request repeats a recent prefix. Structuring prompts with stable content first, system prompt, tool definitions, shared documents, and variable content last raises hit rates. For high-volume applications with large stable prompts, this is one of the largest single savings. Mechanics are in what is prompt caching.

How should you forecast token spend?

Measure tokens per request by feature on real traffic, project request volume with adoption curves, apply current prices per model, include growth in context and history, add buffers for spikes, and re-forecast monthly against actuals. Treat the forecast as a living model tied to your dashboard. Budget process is in the ai budget planning guide and the full production picture in cost of running llms in production.

How FISTA Solutions manages token cost

FISTA Solutions instruments token attribution per feature from the first deployment, designs prompts and retrieval for caching and minimal context, constrains outputs, routes by complexity, and gives clients dashboards that show tokens and cost per request over time. The AI enablement practice delivers the cost architecture, AI agents are built to these patterns, and forward deployed engineers work with client finance and engineering teams on forecasts. The record behind the approach is 150+ projects with 47% efficiency gains for clients.

To forecast and control token spend, message FISTA on WhatsApp, or read ai inference cost for how the same mechanics apply when you host models yourself.