Practical use and limits
Use it for: Export a representative set of request usage, classify input, cached input, output, retries, and tools, then compare p50 and p95 unit cost with provider cost data before setting production budgets.
Limits: Provider models, token accounting, prices, tiers, cache rules, billing currencies, and discounts change. The calculator is a planning model, not a quote or a substitute for the invoice.
Start with a unit cost, not a monthly guess#
A useful cost forecast begins with one completed unit of work: one support answer, document extraction, coding task, search response, or agent run. Record the input tokens, cached portion of input, output tokens, model and service tier for that unit. Multiply each token category by its current per-million rate, then add separately billed features. This creates a per-request cost that can be multiplied by traffic. Starting from a round monthly budget without a unit cost makes it difficult to explain overruns or compare two architectures.
Use the three-part token formula#
For a basic text request, variable cost equals uncached input tokens divided by one million times the input rate, plus cached input tokens divided by one million times the cached-input rate, plus output tokens divided by one million times the output rate. Keep prices in the provider's billing currency until the end; apply an explicit exchange-rate assumption only for planning. The site's LLM API cost calculator implements this formula without built-in model presets, because names, rates, tiers, long-context rules, and promotions can change.
Measure tokens instead of estimating from characters#
Tokenization varies by model, language, structured data, images, audio, tool definitions, and conversation history. A words-to-tokens shortcut can be useful for a sketch but should not approve a production budget. Use the provider's count endpoint, tokenizer, response usage fields, or organization usage export. Collect the complete request—including system instructions, retrieved passages, schemas, and prior messages—and the complete output. Reasoning or thought tokens may also appear in usage data and may be billed or reported under model-specific rules.
Build a distribution, not one average prompt#
Capture a representative test set and report at least median and high-percentile input, output, latency, and retries. Customer messages vary; retrieval can return different document lengths; agents can call tools several times; safety or formatting retries create extra completions. Multiply traffic by a weighted mix of request classes instead of forcing every workflow into one average. A p50 estimate describes the common case, while p95 and a traffic-spike scenario expose the budget needed to survive unusually long or repeated work.
Model caching as an observed hit rate#
Prompt caching can reduce the price of eligible repeated input, but a discount does not mean every repeated-looking prompt becomes a cache hit. Providers apply model, prefix, minimum-token, duration, routing, and storage rules that can change. Structure stable instructions and reusable context as a consistent prefix, then read cached-token usage from responses or usage exports. In a forecast, enter the observed cached share and test a zero-cache case. Also account for cache-write or storage charges where the provider lists them.
Add costs that tokens do not capture#
The simple formula may exclude web search, code execution, file retrieval, embeddings, vector storage, image or audio tokens, cache storage, batch or priority tiers, regional processing, dedicated capacity, taxes, and currency conversion. Retries and failed calls can consume billable work even when the user never sees a useful answer. List each non-token item beside the base estimate with its unit, rate, and evidence source. Keeping these lines separate prevents a provider-specific feature from silently contaminating the core model comparison.
Turn traffic into a monthly budget#
Monthly variable cost is the per-request cost multiplied by successful and unsuccessful attempts across active days. Add fixed platform costs, then compare the total with a budget and an explicit safety margin. Model at least half-volume, expected-volume, and double-volume cases. If demand is uncertain, set hard project budgets, rate limits, maximum output tokens, task timeouts, and alerts outside the model. A budget is only effective when the application can degrade, queue, switch models, or stop before an uncontrolled bill.
Reconcile forecast, usage, and invoice#
After a limited launch, aggregate usage by project, model, endpoint, service tier, and day. Compare counted requests and token categories with application traces, then compare provider cost data with the invoice. OpenAI's usage documentation notes that usage views and cost data may differ slightly and recommends cost data or the billing dashboard for financial reconciliation. Record the explanation for any gap—credits, cached tokens, batch pricing, tool charges, failed calls, tax, rounding, or a missing project tag—and update the forecast version.
Optimize the dominant cost without lowering quality blindly#
First identify whether input, output, retries, tools, or traffic is the largest driver. Shorten repeated context only after verifying answer quality; raise cache reuse by stabilizing eligible prefixes; cap output when the task has a natural stopping point; route simple work to a suitable lower-cost model; batch delay-tolerant jobs when supported; and reduce retries by validating inputs and schemas. Re-run a fixed evaluation set after every change. The cheapest request is not useful if it fails the task, creates manual repair work, or increases safety risk.
Frequently asked questions
How do I know the token count before sending a request?
Use the provider's tokenizer or count endpoint for the exact model and request format. For a production forecast, also record the usage returned by completed responses because tool definitions, retrieved context, and generated output affect the final count.
Should reasoning tokens be included?
Yes when the provider reports or bills them. Their treatment varies by model, so use the current official pricing and usage schema instead of assuming they are free or identical to visible output tokens.
Can prompt caching guarantee a lower bill?
No. Savings depend on eligibility, prefix stability, actual cache hits, the cached-input rate, and possible write or storage charges. Compare an observed-cache scenario with a zero-cache fallback.
Why might the invoice differ from the calculator?
The calculator models entered token rates, request volume, and one fixed-cost line. Tools, storage, multimodal usage, tiers, retries, credits, tax, rounding, and contract terms can change the invoice.