The cheap tier disagrees with the expensive tier

Everyone comparing AI models compares brands: ChatGPT versus Claude versus Gemini versus Grok. That's what we do every day at modelsagree.com — re-polling all four on hundreds of “best X” questions and publishing where they agree. But there's a quieter variable hiding under the brand name: the tier picker. Claude ships as Haiku, Sonnet and Opus. Gemini ships as Flash and Pro. Grok ships as Fast and Expert. Most users never touch the default. So we asked: if you hold the brand constant and change only the tier, does “the best vector database” stay the same?

It does not.

The experiment

We took ten live categories from our AI-tooling leaderboards — vector databases, AI coding assistants, LLM observability, RAG frameworks, GPU clouds, text-to-speech APIs, LLM gateways, agent frameworks, eval tools, and frontier API providers — and sent the identical ranking prompt (the same one our site uses, verbatim, top-5 with reasons) to every tier of each family we could access as a normal subscriber:

• Claude: Haiku → Sonnet → Opus
• ChatGPT: GPT-5.5 → GPT-5.6 (OpenAI blocks the mini tiers on consumer accounts — so for GPT this measures generations, not sizes)
• Gemini: 3.5 Flash → 3.1 Pro
• Grok: Fast → Expert (grok.com's own reasoning tiers)

Same day, same wording, one answer per tier. Then we compared each family's tiers against each other.

The scoreboard

Read that again: within a single brand, switching the tier changes the #1 recommendation roughly half the time. And the top-5 lists behind those #1s only overlap by about 50–65% (Jaccard) between tiers of the same family.

Across all nine tiers, zero of the ten questions produced a unanimous winner.

Who do they crown in their own backyard?

The single most revealing question was the most self-referential one: which frontier-model API should a developer build on? Each tier had to rank its own maker against its rivals. Here's every answer:

There is no clean story here — and that's the finding. The folk theory says “models shill for their maker.” The tier data says self-preference is tier-dependent and inconsistent in direction:

• Claude gets humbler as it gets bigger. Haiku and Sonnet crown Anthropic; Opus — the flagship — hands the crown to OpenAI.
• Gemini gets prouder as it gets bigger. Flash crowns OpenAI; Pro crowns Google — the only tier in the whole grid that picked its own maker while its sibling tier picked a rival.
• ChatGPT flipped between generations. GPT-5.5 crowns OpenAI; GPT-5.6 crowns Anthropic.
• Grok never crowns itself. Both Fast and Expert hand the crown to Anthropic — the only family whose tiers agree on this question, and they agree on a rival.

If you were auditing one of these families for bias by testing one tier, you'd walk away with whichever conclusion that tier happened to hold.

The budget tier buys budget tools

A second pattern surfaced that we didn't go looking for. On “best GPU cloud for training,” every flagship tier — Sonnet, Opus, GPT-5.5, GPT-5.6, Gemini Pro, and both Grok modes — said CoreWeave. The two cheapest tiers in the grid, Claude Haiku and Gemini Flash, both said Lambda — the value-priced alternative. On “best vector database,” Gemini Flash skipped Pinecone (the big tiers' favorite) for pgvector, the free Postgres extension.

It's a hypothesis, not a law — Grok's Expert mode also picked pgvector, so the free-tool instinct isn't exclusive to cheap tiers. But the GPU-cloud split is clean: every budget tier picked the budget cloud, and every flagship picked the flagship cloud. Whether that's training-data skew, a shallower read of the question, or something like taste — the cheap models pattern-matched to the cheap stack.

Why this matters if you sell software

AI answers are becoming a real acquisition channel — we watch it happen in our own traffic. The industry is starting to monitor “what does ChatGPT say about us?” the way it once monitored Google rank. But almost all of that monitoring hits the flagship API tier.

Free users get the cheap tier. The cheap tier gives different answers. Your “AI visibility” is not one number.

If Gemini Pro recommends you but Flash recommends your competitor, then the millions of users on the free tier — the default tier — are hearing your competitor's name. In our ten-question sample, that exact split happened to LangSmith (Pro's pick for observability) versus Langfuse (Flash's pick), to Pinecone versus pgvector, to CoreWeave versus Lambda, to LiteLLM versus Portkey. The recommendation your future customers hear depends on a dropdown they never open.

That's also why modelsagree.com tracks the consensus across four families rather than trusting any single model's opinion: the disagreement is the signal. When all four families — and now, all their tiers — still agree on something (ElevenLabs held #1 in text-to-speech on seven of nine tiers; Braintrust on six of nine in evals; LangSmith on six of nine in observability), that consensus is far harder to dismiss.