Agentic workforce platform DataRobot recently announced the launch of a first-of-its-kind open-source framework, syftr, which identifies performant agentic workflows for commercial use.
Optimizing discovery and implementation without sacrificing cost
The framework empowers AI practitioners to programmatically discover and implement ideal combinations of components, parameters, tools, and strategies for agentic use cases, optimized for accuracy, processing speed, and cost.
“Practitioners and developers are navigating a constantly evolving AI ecosystem– on the order of 10²³ possible agentic architecture combinations– where the most obvious approaches often fall flat,” said Venky Veeraraghavan, chief product officer at DataRobot. “Our mission is to cut through that noise and guide developers to the parameters that actually work for commercial use cases and production environments. With syftr, we’re changing that paradigm to make agentic AI useful, performant, and customizable for enterprises. For the first time, practitioners and developers can truly evaluate the full landscape of AI technologies against company data and implement use cases that balance accuracy, speed, and cost. Now with syftr, they can confidently and quickly implement agentic pipelines and take the guesswork out of manual experimentation.”
The syftr framework addresses challenges related to agentic AI systems, practitioners, and developers to ensure that their agentic workflows are optimally performant for specific use cases based on model quality, cost, and desired behavior.
These challenges are addressed through a multi-objective approach that simulates possible configurations to identify the optimal AI workflows for enterprise data, optimizing for task accuracy, latency, and cost.
Key features of the syftr framework for AI developers
The framework increased accuracy by 25 percent overall while reducing costs by 37 percent at baseline accuracy levels in testing against six industry RAG benchmarks.
The key features for AI practitioners include:
- Discovering optimal agent pipeline patterns, components, and parameters with Multi-Objective Search: Practitioners can leverage an approach using Pareto efficiency to rapidly generate and evaluate different workflow strategies, parameters, models, and components to find a configuration with optimal accuracy, cost, and latency.
- Running computational efficiently with minimized costs: The Bayesian optimization early stopping mechanism can expedite search with the Pareto pruner technique to compare new subflows to a baseline benchmark, removing any new flows that don’t show promise by meeting or exceeding the baseline to reduce compute time and cost by 80 percent.
- Evaluating and implementing the latest techniques and technologies: A component-agnostic setup enables the evaluation of any module, flow, embedding model, or LLM, ensuring that the most recent technologies are considered for optimization based on contributions from DataRobot engineers and the open-source community. Further, an agent pipeline code generator implements and finetunes AI workflows by copying the generated production-ready LlamaIndex code.
“At today’s scale and pace of innovation, it’s impossible for developers to manually evaluate every new technique, tool, and LLM update,” said Debadeepta Dey, distinguished researcher at DataRobot. “And while there are many benchmarks to evaluate model capabilities and performance in isolation, models are rarely used in a vacuum, particularly in the enterprise. Now, syftr is breaking down these barriers for the first time and enabling AI teams to explore large-scale workflow search spaces and deliver AI agents faster than ever before.”
DataRobots has announced that the syftr framework is now available as a permissively licensed open-source project, featuring industry benchmark datasets and a DataRobot training dataset, which is accessible for free on Huggingface. The enterprise version will be available in fall 2025.
DataRobot has continued to explore solutions to help entities address challenges with AI and to improve efficiency. Read more about the company’s recently launched federal AI application suite to assist government agencies and meet security, compliance, and operational standards.