What Is a Token in AI, and Why Does It Decide What I Pay?

Why does my AI bill climb even when my questions look short?

Your bill climbs because AI tools charge by the token, not by the question. A short question can still carry a long history, a big system prompt, and attached files behind it. Every one of those tokens counts. So the visible question is tiny, but the real input the model reads is often huge.

I see this trip up founders who expect a flat monthly fee. AI usage does not work that way at the API level. It works like a metered utility. The more text goes in and out, the more you pay. Once you can see the meter, you can control it.

Tokens are the unit on that meter. So let me explain what a token is and how it turns into real money.

What is a token in AI?

A token is a small piece of text that a model reads as one unit. It is usually part of a word, a whole short word, or a punctuation mark. Models do not see letters or full words the way we do. They break text into tokens first, then work with those pieces.

As a rough guide, one hundred tokens land near seventy five words of plain English. Longer or unusual words split into more tokens. Code, symbols, and other languages often use more tokens per character too. So the token count for a page of text is always a bit higher than the word count.

This matters because tokens are the thing you are billed for. Not words, not messages, not minutes. When I estimate the cost of an automation, I count tokens, not sentences. If you want the full picture of how tokens fill a model's memory, I cover that in my piece on what a context window is and how it changes your prompts.

How do tokens turn into a bill?

Tokens turn into a bill in two parts. You pay for input tokens, which is everything you send the model, and you pay for output tokens, which is everything it writes back. Output usually costs more per token than input. So a chatty reply can cost more than the question that triggered it.

Input covers your prompt, the chat history, system instructions, and any files. Output covers the full response, including any long explanations you did not really need. This split is why a request with a small question but a giant attached document still runs up a real charge.

Once you understand the two sides, you can attack cost from both. You can trim what you send, and you can ask for shorter replies. In my work, doing both together often cuts a workflow's spend without hurting the result at all.

How much do the main models cost per token?

Prices are set per million tokens, and they vary a lot by model. Anthropic lists Claude Opus 4.8 at five dollars per million input tokens and twenty five dollars per million output tokens, per its published pricing. Claude Sonnet 5 sits lower at three dollars and fifteen dollars, per Anthropic, with an introductory rate of two and ten dollars through the end of August 2026.

Smaller models cost much less. Claude Haiku 4.5 runs one dollar per million input tokens and five dollars per million output tokens, per Anthropic's pricing. At the top end, the more capable Claude Fable 5 is priced at ten dollars and fifty dollars per million, per Anthropic.

Those gaps are the point. The same task on Haiku can cost a small fraction of what it costs on a top model. So the model you pick is one of the biggest cost levers you have. I match the model to the difficulty of the task, not to habit.

Why do long prompts cost more than I expect?

Long prompts cost more because you pay for every token you send on every single request. AI tools are stateless, so the model does not remember the last message on its own. To keep context, you resend the whole history each time. That history grows, and so does the input bill.

This surprises people who build chat flows. Turn one is cheap. By turn twenty, each new question drags along nineteen prior turns as input. The visible message is still one line, but the token count behind it keeps rising. I have watched a simple bot get expensive purely from bloated history.

The fix is to manage the history on purpose. I summarize old turns into a short recap and drop the raw text. I strip out material the current step does not need. Leaner input means a lower bill and, often, a sharper answer. Model settings matter too, which is why I wrote about the temperature setting for website copy.

How does prompt caching cut my token costs?

Prompt caching cuts costs by letting you reuse a stable block of text at a steep discount. When you send the same large preamble again and again, the model can read it from cache instead of processing it fresh. Anthropic prices cache reads at roughly one tenth of the normal input rate, per its documentation.

There is a small catch. Writing to the cache the first time costs a bit more than a normal read, around one and a quarter times the input rate for the short lived cache, per Anthropic. So caching pays off when you reuse the same prefix many times, not when the text changes on every call.

I use caching for the fixed parts of a workflow, like a long brand guide or a set of rules that never changes. Those get cached once and reused cheaply. The parts that change, like the actual user question, stay outside the cache. That layout keeps most of a big prompt on the discount rate.

When should I use batch processing to save money?

Use batch processing when the work is not urgent and can wait a bit. With Anthropic's Batch API, you send many requests at once and get results back within a day, and you pay half the standard token price, per Anthropic's documentation. For bulk jobs, that fifty percent cut is real money.

This fits jobs that run in the background rather than in a live chat. Think of rewriting hundreds of product descriptions, tagging a big content library, or generating draft meta descriptions across a whole site. None of those need an instant answer, so trading speed for a lower price is an easy call.

I lean on batching for exactly these large, patient tasks. A live support bot would not use it, since users want a fast reply. But a nightly content run can. Sorting your work into urgent and patient buckets is a simple way to spend less on the same output.

How do I pick the right model for the price?

Pick the smallest model that can do the job well, then move up only if quality falls short. A cheap, fast model like Claude Haiku 4.5 handles sorting, tagging, and simple rewrites at a low rate. A stronger model earns its higher price only on harder reasoning, nuanced writing, or complex code.

In practice, I test a task on a cheaper model first. If the output holds up, I keep it there and pocket the savings. If it stumbles, I step up to a mid or top model for that task alone. There is no rule that every job in a workflow must use the same model.

You can even split one workflow across models. A cheap model can draft or classify, and a stronger model can polish the final piece. Building prompts you can reuse across those steps helps a lot, which is why I keep a prompt library for my content workflow.

What should I do to keep my AI costs under control?

Start by treating tokens as your spend, and watch both what you send and what you get back. Trim the history and files you feed in. Ask for shorter replies. Cache the parts that repeat. Batch the jobs that can wait. And match each task to the cheapest model that clears the bar.

These moves stack. On their own each one shaves a little, but together they can turn a scary AI bill into a predictable line item. Most of the waste I find comes from sending too much, asking for too much, and using a bigger model than the task needs.

If your AI costs feel unpredictable, I am happy to look at your setup and find the leaks. I help founders and marketers build automations that stay accurate and stay affordable. Let's chat, and I will show you where your tokens are going.