Channel Insider recently spoke with Joao Moura, CEO of CrewAI, to discuss balancing AI and human oversight, as well as how to establish confidence and trustworthiness in AI systems.
Discussions around AI agents still vary from everything to nothing
When it comes to AI agents, there’s a lot of discussion in channel circles about what AI can and cannot do.
There are plenty of LinkedIn posts out there from professionals about AI adoptability that start with “AI won’t do X” or “AI doesn’t do X.” And plenty that say AI will solve all your organization’s gaps.
Upwork’s CTO recently said that AI agents “aren’t that good,” following new research suggesting that AI performs best when paired with human collaboration. There’s no doubt that AI presents plenty of challenges, including how it should be utilized and the breadth of its capabilities.
Why Moura thinks the challenge actually lies in whether humans can trust AI
Moura says AI itself isn’t the real challenge, but rather our inability to trust what it can already do.
“AI agents are only truly autonomous when human trust is built into their core. The biggest barrier to adoption right now isn’t capability, it’s production-grade confidence,” said Moura. “To earn that confidence, we need a mature ecosystem with agent operation principles, compliance frameworks, and runtime transparency. Smarter models won’t make agents more reliable; trustworthy systems will. The future belongs to AI we can understand, measure, and hold accountable.”
“Human-in-the-loop is something that many of these companies are using, especially for mission-critical stuff right now,” Moura explains. “I would say that at the end of the day, the benefits are still massive. I can tell that some of our customers, for example, have use cases where they’re not able to remove everyone. But it went from having 20-something people to now two doing the same process, and everyone else got relocated to other things.”
Why human oversight is still critical for agentic AI success
Moura agrees that having humans review and getting the correct outputs is “definitely the way to go.” When it comes to balancing human-in-the-loop and automation workflows, Moura says there are two axes to consider.
He says that there’s complexity and then there’s precision – high and low complexity and high and low precision. With these four quadrants, CrewAI is seeing that complexity is becoming less of a limiting factor. However, on the precision side, you still want to involve humans, especially in approval processes, to maintain trust in agents.
“There’s a few patterns that we have seen that can help along the way,” Moura explains. “There’s this concept of guardrails now that you can add along the process to make sure that as your agents do work, they’re not deviating too much and, in that way, you’re getting to a better final outcome.”
Moura explains that having people review things at the end is still critical.
“There’s a few use cases that we are seeing where the agents go fully autonomous and actually make these decisions, but I think a lot of the companies don’t depend on that to actually extract true value,” he said.
Moura’s other advice for AI deployments includes vendor optionality and ongoing education and training
In terms of addressing trust gaps and building production-grade confidence into AI solutions, multi-agent orchestration and a solid infrastructure have emerged as essential. Moura emphasizes education, strategic playbooks, and maintaining optionality to avoid vendor lock-in.
Moura also says that a pattern is going to begin emerging: companies starting to build their own custom orchestration for the underlying agents across many different platforms.
“A lot of our customers partner with us not only because they want the product itself, but they really like the training. And training here is twofold, with the technical aspect, but then there’s also an element of the non-technical adoption in the company,” Moura said. “So, I think given how early this industry is, and despite the fast adoption being so early, the penetration is very little so far.”
Additionally, Moura shared insights on a new partnership model with a GSI, where CrewAI is used internally to enhance services. This BPO-like approach streamlines scalability and agreements for service providers.
A recent report found that there’s mounting burnout and risk anxiety around AI adoption. Read more about the ISC2 research and shifting cybersecurity priorities heading into 2026.