Marcus On AI

A new study from researchers from Stanford, MIT CSAIL, Carnegie Mellon, ITU Copenhagen, and NVIDIA and Elloe AI Labs, examining 847 autonomous agent deployments drawn from healthcare, finance, customer service and code-generation, showed that 91% were vulnerable to subtle but dangerous tool-chaining attacks

Additionally, in the new study 89.4% of agents showed drift relative to their goals after about 30 steps in their process, and 94% of agents with some form of memory-augmentation were vulnerable to poisoning attacks.

Importantly the new paper shows that agents are in many ways much more vulnerable than pure (“stateless”) LLMs