Hack the Box: AI Boosts Productivity, Creates Skills Risk

AI is helping cybersecurity teams move faster than ever, but new research suggests the speed boost could come at the cost of long-term workforce risks.

A new benchmark report from Hack The Box suggests that teams using AI can significantly outperform human-only cybersecurity teams, completing tasks faster and solving more challenges during simulated security competitions.

AI-assisted teams outperform human-only competitors in CTF benchmark

The findings come from the company’s AI-Augmented vs Human-Only Cybersecurity Performance Benchmark Report, which analyzed results from its NeuroGrid Capture The Flag (CTF) event. 

The competition included 1,078 teams with 120 using agentic AI tools and 958 human-only teams, competing across 36 cybersecurity challenges spanning nine technical domains.

According to the report, AI-assisted teams completed tasks much faster, producing up to 4.1x more output among elite teams and 1.4x more output across all teams within the same timeframe. 

AI-augmented teams also improved challenge completion rates, achieving a 27% solve rate compared with 16% for top human-only teams.

“AI can raise the bar of cybersecurity performance, but it does not eliminate the need for human expertise,” said Haris Pylarinos, founder and CEO of Hack The Box. 

“Our findings show measurable productivity gains, but also predictable failure patterns. Security leaders must build and test human-in-the-loop workflows that are proven under pressure, and develop the AI and cybersecurity skills needed to unlock benefits safely as models evolve.”

Mid-career cybersecurity analysts see the largest AI productivity gains

The benchmark revealed that AI’s impact varies significantly by team experience level.

Mid-career cybersecurity professionals saw the largest productivity jump, particularly when working on medium-difficulty problems. 

In those cases, AI-augmented teams performed 3.89x better, suggesting a practical sweet spot where automation and human expertise combine effectively.

At the highest skill tier, however, the gap between AI-assisted and human-only teams narrowed. 

While AI-augmented elite teams completed challenges 312% faster, their solve-rate advantage dropped from 3.2x overall to 1.7x among the top 5%.

Researchers warn AI could weaken the cybersecurity talent pipeline

One of the most significant concerns highlighted in the report is what researchers describe as a “productivity illusion” among entry-level practitioners.

Less-experienced teams sometimes relied heavily on AI assistance but struggled to verify results or todirect automated agents effectively. 

In some cases, lower-performing AI-assisted teams were 12.5% slower, becoming stuck in inefficient loops without proper oversight.

The findings suggest that if AI absorbs many routine and mid-tier tasks, traditionally where analysts gain experience, organizations could unintentionally weaken the development pipeline for future senior cybersecurity experts.

Report outlines how CISOs should deploy AI in security teams

The data suggests that the “sweet spot” for AI investment isn’t at the bottom or the very top, but in the middle. 

The report recommends that CISOs deploy AI tooling to mid-career teams first to maximize ROI while redesigning entry-level roles to focus on “agentic orchestration” rather than just rote tasks.

The urgency is real. With breakout times for attackers now measured in minutes and sometimes seconds, the defensive window is shrinking. 

As the report concludes, the goal is not just to adopt AI, but to build “AI-fluent” teams that can govern these digital agents under pressure.

Hack The Box plans to present deeper findings from the study at RSAC 2026 on March 26, where researchers will further discuss how AI-augmented security teams are reshaping modern cyber defense operations.