Fixation LogoFixation

Case Studies

How We Cut a Client’s AI Costs by 6x

AI costs are out of control, but there's a solution: run evals to find a cheaper model.

A 50-person SaaS company was spending $10,000/month on AI inference. Six months later, they’d cut that to $1,600—same work, better accuracy.

The Bill That Quietly Becomes a Problem

The AI invoice looked fine at first. A document processing SaaS serving insurance brokers was using a frontier model to extract structured data from policy documents. The feature worked well, customers liked it, and $10,000/month seemed reasonable compared to hiring a manual review team.

Then leadership wanted to expand. More document types, more fields extracted, more customers processing more volume. The projection showed $50,000/month within a year. That “reasonable” invoice was now blocking the product roadmap.

The CTO knew cheaper models existed. He tried one, and it broke things—not catastrophically, but enough that support started getting confused calls. He rolled back within three hours. After that, nobody wanted to touch it.

This is where most companies get stuck. The expensive model becomes permanent because nobody can prove a cheaper option will work. Every conversation ends the same way: “We can’t risk it.”

Why They Couldn’t Fix It Themselves

They tried the obvious approaches first.

Prompt optimization. They made the instructions clearer and reduced token counts where possible. This cut costs by maybe 15%, but they were still on the most expensive tier.

Cheaper model with crossed fingers. They picked a well-reviewed alternative and tested it on five documents, which looked good in manual testing. But after pushing to production, customers noticed a clear degradation in quality, so it had to be rolled back.

Offshore team bid. They got a quote to “rebuild with optimized AI infrastructure.” The timeline was six months, the cost was $80,000, and there was no guarantee on the final monthly spend.

The real problem wasn’t technical competence—their team was strong. The problem was they needed a method to prove quality before changing production, and building that kind of evaluation system is a completely different skillset than building the original feature.

What Fixation Did Differently

The engagement started by measuring what “good” actually looks like.

The team had been treating their production model as a black box. Inputs go in, outputs come out, and if customers don’t complain, it must be working. That approach created too much risk, and downtime or degraded results would be unacceptable to customers.

The first step was capturing 500 real production documents and their accepted outputs. These became the reference set—not theoretical benchmarks or academic datasets, but the actual work their system was already doing successfully.

From there, Fixation built an automated evaluation suite. For each document, the system could run it through any model, compare the structured output to the known-good reference, and score it on field accuracy, format compliance, and edge case handling. The whole evaluation ran in under 10 minutes.

This changed the conversation immediately. Instead of “does this feel right?” leadership could see a dashboard: Model A scores 94% on production cases, Model B scores 96%, Model C scores 91% but costs one-sixth as much.

Testing began on cheaper models using the evaluation suite. Most models failed the tests, but there were a few standouts which showed promise. One smaller model hit 89% accuracy—not good enough on its own, but it demonstrated that the path existed.

The next phase involved adjusting the prompts systematically. Run the eval, identify failure patterns, revise instructions, run the eval again. After a dozen iterations, the cheaper model hit 95% accuracy. That was close enough to justify a controlled test.

Weekly demos with the CTO showed concrete progress. Not “trust us, it’ll work,” but actual numbers: here’s the score, here’s what failed, here’s what the team is fixing next week. That visibility gave leadership the confidence to move forward.

The rollout used feature flags. Traffic started at 5% through the new model, with close monitoring of quality signals and error rates. When nothing broke, it expanded to 20%, then 50%, then 100%.

The entire process took three weeks from kickoff to full production traffic on the cheaper model.

The Results

Monthly AI cost dropped from $10,000 to $1,600—a 6x reduction. Same document volume, better accuracy scores because the prompts had been optimized during testing, and zero customer complaints about the change because quality actually improved.

That’s $100,800 back in their pocket every year.

The client owns the evaluation system. They have the code, the test cases, the scoring logic, and the knowledge to maintain it. When a new model launches, they can test it themselves in an afternoon.

Their CTO told us: “Before, we were hostages. Any change meant gambling with customer trust. Now we test changes like we test any other code. The evaluation suite made AI infrastructure feel like engineering again instead of guesswork.”

Six months later, they shipped three new AI features using the same approach: capture reference cases, build tests, optimize for cost, roll out gradually. Their total AI spend stayed under $2,500/month while supporting five times the feature surface area.

What This Means for Your Business

If your AI bill keeps climbing and you’re afraid to change anything, you’re not stuck because the technology is inherently expensive. You’re stuck because nobody gave you a way to measure quality independent of the vendor.

Once you have that measurement system, model changes become routine engineering decisions instead of risks you can’t quantify. You can negotiate with providers from a position of strength. You can test alternatives when pricing changes. You can build new features without checking your budget first.

Fixation builds these systems for clients who need AI infrastructure that scales without vendor lock-in. Evaluation-first development, clean architecture that keeps models replaceable, and code ownership that stays with you.

If this sounds like the approach your business needs, let’s talk.

Ready to transform your business?

Let’s discuss how we can help you achieve similar results.

Ready to own your software? Let’s talk about your roadmap.