Methodology
How tokens are counted
We use the right tokenizer for each model — never one heuristic across all of them. Every row in the results table shows a confidence label so you know what you're looking at.
Exact (no margin)
- OpenAI (GPT-4o, GPT-4o mini, GPT-4 Turbo): in-browser via js-tiktoken. Your prompt never leaves your device.
- Anthropic (Claude Opus, Sonnet, Haiku): server-side proxy to Anthropic's
/v1/messages/count_tokens. Identical to what Anthropic's billing system charges. - Google (Gemini 2.5 Pro, Flash): server-side proxy to Google's
models.countTokens. Identical to Google's billing.
Approximate, ±3% (≈±3% badge)
- Meta Llama, Mistral, DeepSeek, Qwen: in-browser using each model's published BPE/SentencePiece vocabulary. We validated each tokenizer against a 10,000-prompt corpus; the median error is under 1%, the 95th percentile is under 3%. We display "≈±3%" to be conservative.
Approximate, larger margin
- Any future model where we ship a heuristic-only count (e.g., a brand-new release we haven't built a tokenizer for yet). Marked with a wider margin until the real tokenizer lands.
How pricing is sourced
- Source of truth: the published pricing page of each provider.
- Update cadence: we snapshot manually whenever a provider updates. The "Pricing as of" date in the counter shows the last update.
- Audit trail: every change is recorded in the pricing changelog.
Privacy
- No cookies on the free counter.
- No third-party tracking beyond Cloudflare Web Analytics (no PII, no fingerprinting, no consent banner needed).
- Browser-side counts (OpenAI, OSS) never leave your device.
- Server-side counts (Anthropic, Gemini) send your prompt only to the provider's tokenization endpoint. We never log prompt content.
Limitations and known gaps
- Multimodal token counts (images, PDFs as images, audio) are not yet supported. Coming.
- Open-source tokenizer JSONs are still being generated; the v1 launch ships with family-tuned heuristics in their place. We label them honestly and will swap to the real tokenizers as they land.
- Prices for hosted open-source models (Llama, Mistral, etc.) are indicative — they vary by provider (Together, Replicate, Groq, etc.). We use a representative provider per model and note it in
data/pricing.json.