[ad_1]
When OpenAI went down in December, one among TrueFoundry’s prospects confronted a disaster that had nothing to do with chatbots or content material era. The corporate makes use of massive language fashions to assist refill prescriptions. Each second of downtime meant hundreds of {dollars} in misplaced income — and sufferers who couldn’t entry their medicines on time.
TrueFoundry, an enterprise AI infrastructure firm, introduced Wednesday a brand new product known as TrueFailover designed to stop precisely that state of affairs. The system mechanically detects when AI suppliers expertise outages, slowdowns, or high quality degradation, then seamlessly reroutes site visitors to backup fashions and areas earlier than customers discover something went incorrect.
"The problem is that within the AI world, failover is now not that straightforward," stated Nikunj Bajaj, co-founder and chief govt of TrueFoundry, in an unique interview with VentureBeat. "Whenever you transfer from one mannequin to a different, you even have to contemplate issues like output high quality, latency, and whether or not the immediate even works the identical manner. In lots of instances, the immediate must be adjusted in real-time to stop outcomes from degrading. That isn’t one thing most groups are set as much as handle manually."
The announcement arrives at a pivotal second for enterprise AI adoption. Firms have moved far past experimentation. AI now powers prescription refills at pharmacies, generates gross sales proposals, assists software program builders, and handles buyer assist inquiries. When these methods fail, the implications ripple via complete organizations.
Why enterprise AI methods stay dangerously depending on single suppliers
Massive language fashions from OpenAI, Anthropic, Google, and different suppliers have turn out to be important infrastructure for hundreds of companies. However in contrast to conventional cloud companies from Amazon Internet Providers or Microsoft Azure — which supply strong uptime ensures backed by a long time of operational expertise — AI suppliers function complicated, resource-intensive methods that stay liable to surprising failures.
"Main LLM suppliers expertise outages, slowdowns, or latency spikes each few weeks or months, and we recurrently see the downstream impression on companies that depend on a single supplier," Bajaj instructed VentureBeat.
The December OpenAI outage that affected TrueFoundry's pharmacy buyer illustrates the stakes. "At their scale, even seconds of downtime can translate into hundreds of {dollars} in misplaced income," Bajaj defined. "Past the financial impression, there may be additionally a human consequence when sufferers can’t entry prescriptions on time. As a result of this buyer had our failover answer in place, they had been capable of reroute requests to a different mannequin supplier inside minutes of detecting the outage. With out that setup, restoration would seemingly have taken hours."
The issue extends past full outages. Partial failures — the place a mannequin slows down or produces lower-quality responses with out going absolutely offline — can quietly destroy consumer expertise and violate service-level agreements. These "gradual however technically up" situations usually show extra damaging than dramatic crashes as a result of they evade conventional monitoring methods whereas steadily eroding efficiency.
Contained in the expertise that retains AI purposes on-line when suppliers fail
TrueFailover operates as a resilience layer on high of TrueFoundry's AI Gateway, which already processes greater than 10 billion requests per thirty days for Fortune 1000 corporations. The system weaves collectively a number of interconnected capabilities right into a unified security web for enterprise AI.
At its core, the product allows multi-model failover by permitting enterprises to outline major and backup fashions throughout suppliers. If OpenAI turns into unavailable, site visitors mechanically shifts to Anthropic, Google's Gemini, Mistral, or self-hosted options. The routing occurs transparently, with out requiring software groups to rewrite code or manually intervene.
The system extends this safety throughout geographic boundaries via multi-region and multi-cloud resilience. By distributing AI endpoints throughout zones and cloud suppliers, health-based routing can detect issues in particular areas and divert site visitors to wholesome options. What would in any other case turn out to be a world incident transforms into an invisible infrastructure adjustment that customers by no means understand.
Maybe most critically, TrueFailover employs degradation-aware routing that repeatedly screens latency, error charges, and high quality indicators. "We have a look at a mix of indicators that collectively point out when a mannequin's efficiency is beginning to degrade," Bajaj defined. "Massive language fashions are shared assets. Suppliers run the identical mannequin occasion throughout many shoppers, so when demand spikes for one consumer or workload, it could possibly have an effect on everybody else utilizing that mannequin."
The system watches for rising response instances, rising error charges, and patterns suggesting instability. "Individually, none of those indicators inform the total story," Bajaj stated. "However taken collectively, they permit us to detect early indicators {that a} mannequin is slowing down or changing into unreliable. These indicators feed into an AI-driven system that may resolve when and how you can reroute site visitors earlier than customers expertise a noticeable drop in high quality."
Strategic caching rounds out the safety by shielding suppliers from sudden site visitors spikes and stopping rate-limit cascades throughout high-demand intervals. This permits methods to soak up demand surges and supplier limits with out brownouts or throttling surprises.
The method represents a basic shift in how enterprises ought to take into consideration AI reliability. "TrueFailover is designed to deal with that complexity mechanically," Bajaj stated. "It repeatedly screens how fashions behave throughout many shoppers and use instances, appears to be like for early warning indicators like rising latency, and takes motion earlier than issues break. Most particular person enterprises should not have that form of visibility as a result of they’re solely capable of see their very own methods."
The engineering problem of switching fashions with out sacrificing output high quality
One of many thorniest challenges in AI failover entails sustaining constant output high quality when switching between fashions. A immediate optimized for GPT-5 might produce totally different outcomes on Claude or Gemini. TrueFoundry addresses this via a number of mechanisms that steadiness pace in opposition to precision.
"Some groups depend on the truth that massive fashions have turn out to be adequate that small variations in prompts don’t materially have an effect on the output," Bajaj defined. "In these instances, switching from one supplier to a different can occur with some seen impression — that's not best, however some groups select to do it."
Extra subtle implementations keep provider-specific prompts for a similar software. "When site visitors shifts from one mannequin to a different, the immediate shifts with it," Bajaj stated. "In that case, failover isn’t just switching fashions. It’s switching to a configuration that has already been examined."
TrueFailover automates this course of. The system dynamically routes requests and adjusts prompts based mostly on which mannequin handles the question, retaining high quality inside acceptable ranges with out guide intervention. The important thing, Bajaj emphasised, is that "failover is deliberate, not reactive. The logic, prompts, and guardrails are outlined forward of time, which is why finish customers usually don’t discover when a swap occurs."
Importantly, many failover situations don’t require altering suppliers in any respect. "It may be routing site visitors from the identical mannequin in a single area to a different area, reminiscent of from the East Coast to the West Coast, the place no immediate adjustments are required," Bajaj famous. This geographic flexibility gives a primary line of protection earlier than extra complicated cross-provider switches turn out to be mandatory.
How regulated industries can use AI failover with out compromising compliance
For enterprises in healthcare, monetary companies, and different regulated sectors, the prospect of AI site visitors mechanically routing to totally different suppliers raises speedy compliance considerations. Affected person information can’t merely circulation to whichever mannequin occurs to be out there. Monetary information require strict controls over the place they journey. TrueFoundry constructed specific guardrails to handle these constraints.
"TrueFailover won’t ever route information to a mannequin or supplier that an enterprise has not explicitly authorized," Bajaj stated. "Every little thing is managed via an admin configuration layer the place groups set clear guardrails upfront."
Enterprises outline precisely which fashions qualify for failover, which suppliers can obtain site visitors, and even which areas or mannequin classes — reminiscent of closed-source versus open-source — are acceptable. As soon as these guidelines take impact, TrueFailover operates solely inside them.
"If a mannequin shouldn’t be on the authorized listing, it’s merely not an choice for routing," Bajaj emphasised. "There isn’t a state of affairs the place site visitors is mechanically despatched someplace surprising. The thought is to offer groups full management over compliance and information boundaries, whereas nonetheless permitting the system to reply shortly when one thing goes incorrect. That manner, reliability improves with out compromising safety or regulatory necessities."
This design displays classes realized from TrueFoundry's current enterprise deployments. A Fortune 50 healthcare firm already makes use of the platform to deal with greater than 500 million IVR calls yearly via an agentic AI system. That buyer required the power to run workloads throughout each cloud and on-premise infrastructure whereas sustaining strict information residency controls — precisely the form of hybrid surroundings the place failover insurance policies should be exactly outlined.
The place computerized failover can’t assist and what enterprises should plan for
TrueFoundry acknowledges that TrueFailover can’t remedy each reliability downside. The system operates throughout the guardrails enterprises configure, and people configurations decide what safety is feasible.
"If a workforce permits failover from a big, high-capacity mannequin to a a lot smaller mannequin with out adjusting prompts or expectations, TrueFailover can’t assure the identical output high quality," Bajaj defined. "The system can route site visitors, nevertheless it can’t make a smaller mannequin behave like a bigger one with out applicable configuration."
Infrastructure constraints additionally restrict safety. If an enterprise hosts its personal fashions and all of them run on the identical GPU cluster, TrueFailover can’t assist when that infrastructure fails. "When there is no such thing as a alternate infrastructure out there, there may be nothing to fail over to," Bajaj stated.
The query of simultaneous multi-provider failures often surfaces in enterprise threat discussions. Bajaj argues this state of affairs, whereas theoretically potential, hardly ever matches actuality. "In observe, 'happening' normally doesn’t imply a whole supplier is offline throughout all fashions and areas," he defined. "What occurs much more usually is a slowdown or disruption in a selected mannequin or area due to site visitors spikes or capability points."
When that happens, failover can occur at a number of ranges — from on-premise to cloud, cloud to on-premise, one area to a different, one mannequin to a different, and even throughout the similar supplier earlier than switching suppliers fully. "That alone makes it impossible that every little thing fails without delay," Bajaj stated. "The important thing level is that reliability is constructed on layers of redundancy. The extra suppliers, areas, and fashions which can be included within the guardrails, the smaller the prospect that customers expertise an entire outage."
A startup that constructed its platform inside Fortune 500 AI deployments
TrueFoundry has established itself as infrastructure for a few of the world's largest AI deployments, offering essential context for its failover ambitions. The corporate raised $19 million in Collection A funding in February 2025, led by Intel Capital with participation from Eniac Ventures, Peak XV Companions, and Leap Capital. Angel buyers together with Gokul Rajaram and Mohit Aron additionally joined the spherical, bringing complete funding to $21 million.
The San Francisco-based firm was based in 2021 by Bajaj and co-founders Abhishek Choudhary and Anuraag Gutgutia, all former Meta engineers who met as classmates at IIT Kharagpur. Initially targeted on accelerating machine studying deployments, TrueFoundry pivoted to assist generative AI capabilities because the expertise went mainstream in 2023.
The corporate's buyer roster demonstrates enterprise-scale adoption that few AI infrastructure startups can match. Nvidia employs TrueFoundry to construct multi-agent methods that optimize GPU cluster utilization throughout information facilities worldwide — a use case the place even small enhancements in utilization translate into substantial enterprise impression given the insatiable demand for GPU capability. Undertake AI routes greater than 15 million requests and 40 billion enter tokens via TrueFoundry's AI Gateway to energy its enterprise agentic workflows.
Gaming firm Video games 24×7 serves machine studying fashions to greater than 100 million customers via the platform at scales exceeding 200 requests per second. Digital adoption platform Whatfix migrated to a microservices structure on TrueFoundry, decreasing its launch cycle sixfold and reducing testing time by 40 %.
TrueFoundry at present studies greater than 30 paid prospects worldwide and has indicated it exceeded $1.5 million in annual recurring income final yr whereas quadrupling its buyer base. The corporate manages greater than 1,000 clusters for machine studying workloads throughout its shopper base.
TrueFailover will likely be supplied as an add-on module on high of the prevailing TrueFoundry AI Gateway and platform, with pricing following a usage-based mannequin tied to site visitors quantity together with the variety of customers, fashions, suppliers, and areas concerned. An early entry program for design companions opens within the coming weeks.
Why conventional cloud uptime ensures might by no means apply to AI suppliers
Enterprise expertise patrons have lengthy demanded uptime commitments from infrastructure suppliers. Amazon Internet Providers, Microsoft Azure, and Google Cloud all provide service-level agreements with monetary penalties for failures. Will AI suppliers finally face comparable expectations?
Bajaj sees basic constraints that make conventional SLAs tough to attain within the present era of AI infrastructure. "Most foundational LLMs right this moment function as shared assets, which is what allows the usual pricing you see publicly marketed," he defined. "Suppliers do provide greater uptime commitments, however that normally means devoted capability or reserved infrastructure, and the price will increase considerably."
Even with substantial budgets, enterprises face utilization quotas that create surprising publicity. "If site visitors spikes past these limits, requests can nonetheless spill again into shared infrastructure," Bajaj stated. "That makes it laborious to attain the form of laborious ensures enterprises are used to with cloud suppliers."
The economics of working massive language fashions create further boundaries which will persist for years. "LLMs are nonetheless extraordinarily complicated and costly to run. They require huge infrastructure and power, and we don’t anticipate a near-term future the place most corporations run a number of, absolutely devoted mannequin cases simply to ensure uptime."
This actuality drives demand for options like TrueFailover that present resilience no matter what particular person suppliers can promise. "Enterprises are realizing that reliability can’t come from the mannequin supplier alone," Bajaj stated. "It requires further layers of safety to deal with the realities of how these methods function right this moment."
The brand new calculus for corporations that constructed AI into essential enterprise processes
The timing of TrueFoundry's announcement displays a basic shift in how enterprises use AI — and what they stand to lose when it fails. What started as inside experimentation has developed into customer-facing purposes the place disruptions instantly have an effect on income and repute.
"Many enterprises experimented with Gen AI and agentic methods up to now, and manufacturing use instances had been largely internal-facing," Bajaj noticed. "There was no speedy impression on their high line or the general public notion of the enterprise."
That period has ended. "Now that these enterprises have launched public-facing purposes, the place each the highest line and public notion could be impacted if an outage happens, the stakes are a lot greater than they had been even six months in the past. That's why we’re seeing increasingly consideration on this now."
For corporations which have woven AI into essential enterprise processes — from prescription refills to buyer assist to gross sales operations — the calculus has modified fully. The query is now not which mannequin performs finest on benchmarks or which supplier affords probably the most compelling options. The query that now retains expertise leaders awake is way less complicated and much more pressing: what occurs when the AI disappears on the worst potential second?
Someplace, a pharmacist is filling a prescription. A buyer assist agent is resolving a grievance. A gross sales workforce is producing a proposal for a deal that closes tomorrow. All of them depend upon AI methods that depend upon suppliers that, regardless of their scale and class, nonetheless go darkish with out warning.
TrueFoundry is betting that enterprises can pay handsomely to make sure these moments of darkness by no means attain the individuals who matter most — their prospects.
[ad_2]
