Startups Shift to Cheaper Open AI Models as Costs Surge

Overview

A growing wave of artificial intelligence startups is turning away from proprietary models offered by US tech giants like OpenAI and Anthropic, opting instead for cheaper, open-weight alternatives. This strategic pivot is driven by soaring compute costs and the desire for greater technological independence.

Companies that once relied entirely on powerful but expensive closed models are now building custom AI solutions using open-source frameworks, including highly capable options from Chinese developers. The shift represents a significant challenge to the business models of the dominant AI labs as they prepare for upcoming initial public offerings.

"It made absolutely no sense a year ago. It's starting to make a lot of sense now." — Karim Atiyeh, Co-CEO of Ramp

This trend is reshaping the financial dynamics of the AI application layer. While open-weight models allow startups to retain more value and control their data pipelines, industry veterans warn that the move is not a one-size-fits-all solution.

The Cost Crisis Driving the Shift

For many AI-native companies, the economics of relying on external model providers have become untenable. High-growth usage spikes often lead to exponential increases in inference costs, severely compressing margins.

Legal tech firm Harvey, valued at $15.6 billion, recently experienced a dramatic swing in its financial health. After a March update spurred customer adoption of its AI agents, its gross margins collapsed from approximately 50% at the start of the year to negative 50% by June. The company reported a twenty-fold increase in AI token usage, a volume that proved unsustainable under the pricing structures of leading US providers.

Similarly, Uber found itself burning through its entire annual AI budget by April after encouraging engineers to maximize their use of Anthropic’s Claude Code. These cases highlight the risks of unoptimized token usage when pricing scales with consumption.

As proprietary prices remain high, many firms view custom model development not as a luxury but as a necessity for survival. Investors increasingly expect startups to optimize their cost structures through fine-tuning or full model training.

The Rise of Open-Weight Alternatives

The solution for many startups has been to embrace open-weight AI models. These models allow companies to deploy AI infrastructure on their own servers, paying only for compute rather than steep per-token licensing fees.

Harvey has already begun this transition. In August, the legal startup launched a new model powered by Moonshot AI’s Kimi K3, a Chinese-made open-weight model. According to insiders, Kimi K3 can perform competitively with Anthropic’s most advanced offerings at a fraction of the cost. This switch, combined with other efficiency tweaks, has reportedly restored Harvey’s gross margins to positive territory.

The trend extends across multiple sectors:

• Healthtech: Abridge recently announced the development of a custom foundation model for clinical settings, built on Nvidia’s open models.
• Customer Support: Decagon revealed that it now routes 80% of customer queries through its own models rather than third-party APIs.
• Fintech: Startups like Ramp and Rogo are exploring the training of bespoke models for the first time. Ramp’s co-CEO noted that the startup’s recent $750 million raise made these efforts financially viable.
• Coding: Companies such as Cursor and Cognition were among the early adopters of bespoke model strategies.

China’s rapid advancement in open-weight technology has been a critical enabler. Models from developers like Moonshot AI provide performance levels that were previously exclusive to US-led labs, offering a viable and affordable alternative for global startups.

Investor Perspectives and Strategic Risks

The migration toward open models is backed by prominent venture capital firms, including Sequoia Capital and General Catalyst. These investors see the trend as a way for portfolio companies to secure their long-term viability while reducing dependency on a handful of dominant US providers.

Dr. Lan Xuezhao, founder of Basis Set, argues that the ability to build custom models is becoming a prerequisite for funding. She stated that companies failing to optimize their AI costs by developing their own models may find themselves unable to raise capital in the current climate.

However, not all investors are convinced that building proprietary models is the right path for every company. Matt Kraning, a partner at Menlo Ventures, which backs Anthropic, cautioned that custom model development requires specialized talent and significant upfront investment.

"In most cases, it tends to be a lot of cosplay." — Matt Kraning, Partner at Menlo Ventures

Kraning suggested that for many firms, claiming to have their own model is more of a marketing tactic than a sustainable technical strategy. He emphasized that the question of whether to build versus buy is often misplaced, arguing that operational efficiency matters more than ownership of the underlying architecture.

Competitive Tension with AI Giants

The shift toward open-weight models poses a direct threat to the revenue streams of OpenAI and Anthropic. Both companies are gearing up for highly anticipated initial public offerings (IPOs), and investor enthusiasm relies partly on demonstrating that their proprietary models remain the gold standard.

OpenAI and Anthropic have responded to cost concerns by releasing lower-cost model tiers. However, they have also tightened their enterprise pricing structures, adding usage-based fees on top of subscription costs. This dual approach aims to capture more value from high-volume customers but has further incentivized enterprises to seek alternatives.

Furthermore, both giants are aggressively expanding into vertical-specific applications. Anthropic has recently launched pilot programs and plugins in legal, finance, and healthcare sectors, directly competing with the very startups that are moving away from its API.

Internal discussions at Anthropic have reportedly touched on how startups are building their own models. Slides referenced companies like Rogo and OpenEvidence, urging clients to find ways to "ride the curve" of value rather than abandon the platform entirely. The situation remains complex, as even companies building custom models often still rely on cutting-edge proprietary systems for specific, high-complexity tasks.

Despite these tensions, the momentum behind open-weight AI appears strong. As Chinese models continue to improve and compute costs remain a pressing concern, more startups are likely to follow Harvey’s lead, prioritizing cost control and independence over exclusive reliance on US-based AI giants.