Enhancing AI Agent Reliability: Lessons From Autonomous Vehicle Development

While AI agents are a major focus, their reliability in complex tasks often falls short of expectations. Startup Plurai proposes a solution inspired by methods from autonomous driving to enable agents to operate more efficiently and with less oversight.

Enhancing AI Agent Reliability: Lessons From Autonomous Vehicle Development

Artificial intelligence in the form of so-called AI agents is currently very popular and promising, yet their practical reliability in solving complex tasks often falls short of expectations. Recent performance tests have shown that even the most advanced models still require significant human oversight to achieve the desired accuracy and reliability. For instance, in a simulated corporate environment, they achieved a success rate of around 75-80%.

Startup Plurai aims to significantly boost this reliability using proven methods that have contributed to the development of autonomous vehicles. This primarily involves extensive simulations with synthetic data derived from a company's specific digital systems. This approach is designed to prepare models for handling complex and unpredictable edge cases. Plurai's founders, Ilan Kadar and Elad Levi, have extensive experience developing autonomous driving technologies, where they previously participated in simulations and evaluation of complex vehicle control software. They see a parallel between the development of autonomous systems and current AI agents: it's easy to create a demonstration, but difficult to achieve a reliable and cost-effective solution. Therefore, they are focusing on building a robust simulation layer.

To realize its ambitions, Plurai secured initial investments totaling $13 million, underscoring investors' confidence in their innovative approach.

Similar to autonomous driving systems, AI agents are probabilistic systems that operate in not entirely predictable environments. They often encounter issues, such as ignoring corporate budget limits when booking travel, or struggling to combine information from multiple databases even when they reliably handle a single one. Long interactions with chatbots can overload agents with context, leading to erroneous decisions.

Plurai addresses these shortcomings by creating a digital twin of a company's electronic systems. This includes both corporate knowledge bases and the tools the agent will access in real-world operation. Subsequently, all possible agent interaction flows are mapped, and this graph is used to prepare the model, including evaluation and corrections for potential edge cases. This platform is designed to work seamlessly with existing agent development tools.

Kadar emphasizes that Plurai's key value lies in using collected data to subsequently train smaller, open-source models tailored to specific customer needs. Internal tests have shown that models developed with this approach can outperform some comparable advanced models at significantly lower costs. The process involves breaking down complex tasks into smaller parts through simulations, training a smaller model, and then verifying its quality, which is comparable to or higher than large, general-purpose models, but with potential cost savings of up to tenfold.

Dominik Medal

Planning a new website or app?

Tell me what you have in mind — I will take a look and advise on the best way forward.

Reah Miyara, an AI researcher who supported Plurai as a private investor, sees the company's competitive advantage in its deep understanding of failure modes in a production environment and its ability to generate synthetic data for fine-tuning models. At a time when companies are striving to optimize their AI expenditures, Plurai's approach can be a significant benefit, as it is clear that general-purpose models are often not optimally adapted to specific business data, and their long-term operational costs are unsustainable.

Plurai believes that the parallels between AI agents in the physical and digital worlds are predictive. While general, highly powerful AI models give robots fascinating capabilities, they are still not reliable enough for widespread deployment. In contrast, robots powered by AI models created for specific tasks are already operating in the field, serving customers, and collecting valuable real-world data. Kadar adds that even the smartest and most advanced AI models will never know the specifics and internal policies of a particular enterprise as thoroughly as custom-trained models.

Dominik Medal

Let's talk about your project

Tell me what you need and together we will work out the best way forward.

Stop scrolling, call me

+420 735 505 585