What a token actually is
A token is the unit a language model reads, generates, and gets billed in. It sits somewhere between a character and a word. Language models don't process raw text one letter at a time or one word at a time; they run a tokenizer first, which breaks your text into a sequence of sub-word pieces drawn from a fixed vocabulary the model was trained on. Common English words like "the" or "and" are usually a single token. Longer, rarer, or made-up words often split into two or three token pieces. Punctuation, whitespace, and non-English scripts tokenize differently again, a block of code or a sentence in a language the tokenizer wasn't heavily trained on can use noticeably more tokens per character than plain English prose. As a rough working rule, one token is about 4 characters, or roughly ¾ of a word, for typical English text, but that's an average, not a guarantee for any specific sentence.
Why the same text counts differently on different models
Every model family trains its own tokenizer on its own vocabulary, so the same paragraph produces a different token count depending on which model reads it. OpenAI's newer models (the GPT-4o/4.1/5-era and the o-series) share one encoding, called o200k_base; OpenAI's older GPT-4, GPT-3.5, and embedding models use an earlier encoding, cl100k_base. Switch the model selector above between, say, GPT-5.5 and GPT-4, and you'll usually see the token count shift slightly on identical text. That's not a bug; it's two different vocabularies making different splitting decisions. Other providers' models will count differently again, for the same underlying reason, even before you get into the exact-versus-estimate distinction below.
Exact counts vs. estimates, and why we label them
This tool runs two genuinely different methods depending on the model you pick, and it's honest about which one you're getting. For OpenAI models, it uses gpt-tokenizer, a pure-JavaScript implementation of the same byte-pair encodings (o200k_base and cl100k_base) that OpenAI's own tiktoken library uses. That's an exact count, marked with a green "exact" badge, computed entirely in your browser with no network call. For every other provider (Claude, Gemini, Llama, Mistral, DeepSeek, and Grok) there's no equivalent public, client-side tokenizer to bundle for free. Anthropic deprecated the local tokenizer it used to publish. Google's Gemini tokenizer is SentencePiece-based and only exact via a network API call, which this tool deliberately avoids because it would mean sending your text to a server. So those models fall back to a character-based heuristic, and the badge switches to an honest orange "estimate," with a tooltip explaining exactly why, right where you'd otherwise assume precision you're not actually getting.
That heuristic isn't one flat number applied blindly across every provider, either. Most estimate-lane models split at roughly 4 characters per token, and that's the default here. But Anthropic's newer tokenizer (Opus 4.7 and later, Fable 5, Sonnet 5) is measurably denser: it produces about 30% more tokens for identical text than the older one, so this tool estimates those three Claude models at roughly 3.1 characters per token instead of 4, rather than quietly underselling how many tokens your prompt actually costs. Claude Haiku 4.5 isn't confirmed to use the newer tokenizer, so it stays on the standard 4-char default until that's verified either way. Competitors that quietly present a heuristic as if it were exact, or apply one ratio to every non-OpenAI model regardless of vendor, are giving you a false sense of precision; we'd rather show you a number that says "estimate" and get the estimate itself right, per model, than round every provider down to the same guess.
Reading the context-window meter
Underneath the token count, a meter shows what percentage of your selected model's context window your text would use. A model's context window is the maximum number of tokens it can hold in a single conversation or request, prompt plus any expected response. Watching this meter matters most once you're pasting something substantial: a long document, a big codebase file, or an entire chat history you're re-sending. If the meter passes 100%, the text as written won't fit in that model's window at all, and you'll need to trim it, chunk it, or switch to a model with more headroom.
Cutting your token count
The fastest wins are usually mechanical: strip repeated boilerplate and redundant instructions from a system prompt, remove unused few-shot examples, and avoid pasting the same large block of context into every message when your integration could reuse it instead. Beyond that, tokenization itself rewards plain, common-word English over dense jargon, unusual formatting, or heavily nested code, the same idea expressed simply will often tokenize shorter than the same idea expressed densely. Once you know your token count, the natural next question is what it costs to actually run: the AI token cost calculator takes the exact numbers from this page (use the "See what this costs to run" link above to hand them over automatically) and turns them into a per-call and per-month dollar estimate across every provider.
