Humans missed 1 in 3 threats approving AI agent commands across 40k game runs
Results from AI agent permission game: which attacks beat human reviewers, and which safe commands got blocked instead.
Summary
A recent study reveals that human reviewers overlooked one-third of threats while approving AI agent commands during extensive gameplay simulations. The findings highlight specific attack strategies that succeeded against human judgment, as well as instances where benign commands were mistakenly blocked.
Comments (243 of ) ⌛