Aggregators Detect, Providers Settle: What We Learned Keeping 45 Model Prices Honest

tokencheat tracks list prices for 45 models across 13 providers, each verified against the provider's own pricing page. To catch drift, we also reconcile daily against models.dev — an MIT-licensed open database that publishes prices in the same USD-per-million unit we use.

The first full reconciliation disagreed with us on six models. That could have meant our catalog was stale. We checked the two most consequential against primary sources:

Qwen3.7 Plus. The aggregator said $0.50 input / $3.00 output. Alibaba's own pricing page gives the international (Singapore) tier as $0.40 / $1.60 — exactly our number. Alibaba publishes six regional price sets, each tiered by context length; a single scalar price for that model is meaningless without naming both. The aggregator had quoted a different region or the long-context tier.

Claude Sonnet 4.5's context window. The aggregator said 1M tokens. Anthropic's documentation scopes the full 1M window to "Claude 4.6 and later." Sonnet 4.5 is earlier. Our 200K stood.

Both times, auto-syncing from the aggregator would have shipped a regression into a tool people use for budget decisions.

The rule, and the system built on it

An aggregator may detect divergence. Only the provider's page may settle it. Our drift report flags disagreements; a human resolves each one against the primary source; and what gets recorded is the provider page, dated. Nothing auto-applies.

The same discipline produced our pricing changelog: a dated, append-only log of observed changes, where every entry says whether our own catalog agrees — the difference between "we are behind" and "the source is noisy" — and links the provider page that settles it. History begins at the first snapshot we actually observed. We do not backfill, because a dated record we did not observe on that date is not evidence.

None of this is exotic engineering. It is mostly the willingness to say "we don't know" in a column where a plausible number would look better. That turns out to be rare enough to be a feature.