How I Cut My LLM API Costs Without Losing Quality

TL;DR

• Prompt caching is the single biggest win: repeated context can cost up to 90% less after the first call.
• The Batch API knocks about 50% off any job that doesn't need an answer this second.
• Most bills are bloated by one habit: sending every request to a flagship model when a small one would pass.
• Frontier model prices span roughly two orders of magnitude, so the "cheapest model" changes with your workload, not the rate card.
• A unified gateway makes switching between models a config change instead of a rewrite.

I looked at my API bill one month and it was four times what I expected. Nothing had changed in the product. What changed was that I'd stopped paying attention to which model handled what, and I was caching nothing. Here's what actually moved the number down, in the order that gave me the most savings per hour of effort.

Why is my LLM API bill so high?

Almost always it's one of two things: you're paying full price for the same tokens over and over, or you're using an expensive model for work a cheap one could do.

The second one is sneakier. You pick a flagship model while prototyping because you want it to just work, and then that choice quietly ships to production and runs on every request forever. One report I trust puts the spread between the cheapest and most expensive frontier output pricing at around $18 per million tokens. That's not a rounding error. When two teams build similar apps and see a 10x cost difference, it's usually here, not in the rate card.

So before optimizing anything clever, ask a boring question: does this specific call need the smart model, or did I just never change it?

How much can prompt caching actually save?

Up to 90% on the cached portion of a request, and it's the first thing I'd turn on. Most providers now let you mark the stable part of your prompt — the system instructions, the few-shot examples, the long document you keep referencing — so you pay full price to process it once and a fraction of that on every call after.

The catch nobody mentions: caching only helps if your prompt is structured so the stable part comes first and doesn't change. If you interpolate a timestamp or a user ID at the top of your system prompt, you bust the cache on every single call and pay full freight. Put the volatile stuff at the end.

For my workload this was the biggest line-item drop, bigger than switching models. If you only do one thing from this post, do this.

Does the Batch API really cut costs in half?

Yes, roughly 50% off, and the only price is patience. If a job doesn't need its answer in the next few seconds — nightly summaries, bulk classification, generating embeddings over a backlog, evaluating a test set — send it through the Batch API instead of the real-time endpoint.

The mental model that helped me: split every workload into "a human is waiting" versus "a machine will read this later." Everything in the second bucket is a batch candidate. I moved my entire eval pipeline over and the per-run cost halved with zero code changes beyond the endpoint.

When should I use a cheaper model instead of a flagship?

More often than you think. The honest test is to route a task to a small model and check whether the output actually gets worse in a way your users would notice. For classification, extraction, short rewrites, and routing decisions, a small model is usually indistinguishable — and small tiers can run roughly 6x cheaper than their full-size siblings.

Here's the tactic that works better than picking one model: use a cheap model as the default and escalate to an expensive one only when the cheap one fails a check. Draft with the small model, verify with the big one, or let the small model handle 80% of traffic and pass the hard 20% up. You keep flagship quality where it matters and stop paying for it where it doesn't.

How do I stop surprise bills from runaway loops?

Set a hard monthly cap in your provider dashboard and log usage per feature, not just per day. The daily total tells you the bill went up; it doesn't tell you which feature did it. Log the model, input tokens, output tokens, and the feature that triggered each call, and the culprit becomes obvious in an afternoon.

The scariest bills I've seen came from agent loops that retried on failure without a ceiling. One bad prompt, one flaky tool call, and a loop hammers the API hundreds of times before anyone notices. Cap the iterations and cap the spend. Both.

How do I compare prices across GPT, Claude, and Gemini?

The problem is that "cheapest" depends on your workload, and testing that means signing up for three providers, managing three keys, and rewriting your client for each SDK. That friction is why most people just stick with whatever they started on and overpay.

This is the gap a unified gateway fills. You hit one OpenAI-compatible endpoint with one key and can point the same request at a different model by changing a string. Gateways like OpenRouter pioneered this — worth knowing they pass provider token prices through and add a separate platform fee (a 5.5% credit-purchase fee on their published schedule), rather than marking up the tokens.

I run this through velokey because a single key across 100+ models let me actually A/B a cheap model against a flagship on real traffic instead of guessing, and set up failover so a provider outage doesn't take my app down. That's the part that closed the loop on everything above: caching and batching cut the bill, but being able to switch models in a config file is what let me right-size without a rewrite every time.

Whatever you use, the point stands: if changing models costs you a day of engineering, you won't do it, and you'll overpay by default.

Frequently asked questions

What's the fastest way to cut an OpenAI API bill today?

Turn on prompt caching for your system prompt and long context. It's the single biggest lever, often up to 90% off the repeated portion, and it usually takes an hour to wire up. Do this before touching model selection or anything more involved.

Does a smaller model always mean worse output?

No. For classification, extraction, routing, and short rewrites, small models are often indistinguishable from flagships to your users. The reliable approach is to default to the cheap model and escalate to an expensive one only when a quality check fails, so you pay for premium quality only where it shows.

Do unified API gateways add a markup?

It depends on the gateway. The common model is pass-through token pricing plus a separate platform fee — OpenRouter, for example, charges a 5.5% credit-purchase fee rather than marking up tokens, per its published schedule. Always check whether the fee is on tokens or on credit purchases, because that changes the math at scale.

Is prompt caching worth it for low-volume apps?

Often no. Caching shines when the same stable context is reused across many calls. If you make a handful of one-off requests a day with no shared context, the cache expires between calls and there's nothing to save. Focus on right-sizing your model instead.

How do I know which model is cheapest for my use case?

You have to test on your real traffic, because effective cost depends on your input-to-output ratio and cache hit rate more than the rate card. Two teams with the same rate card can see 10x different bills. Run the same workload through a few models and measure, rather than trusting a pricing table.

Can I switch models without rewriting my code?

Yes, if you use an OpenAI-compatible endpoint or a gateway that exposes one. Then switching models is changing a string, not swapping SDKs. This is the whole reason gateways exist — they turn a migration into a config change, which is what makes right-sizing practical instead of theoretical.