Lasso Study: AI Text Watermarking Can Shift LLM Refusals and Agent Tool Calls

Watermarking AI-generated text can change how language models handle harmful requests and which tools AI agents call, according to new research from Lasso Security. The study tested Google DeepMind’s SynthID-Text, the open-source scheme Anthropic recently said its future Claude models will use, partly to comply with the EU AI Act, Ars Technica reported.

SynthID-Text uses a secret key to subtly steer next-word selection, so anyone holding the key can later check whether text came from a given platform. It relies on “tournament sampling,” in which candidate tokens get hidden scores from the key and compete in rounds until one wins. The word choice is still random, but the key leaves a detectable pattern.

Refusals shift, especially under prompt injection

Lasso AI security researcher Andrea Siposova tested SynthID-Text’s “non-distortionary” configuration through Hugging Face’s unmodified watermarking processor on six open-weight models, comparing responses to harmful prompts with and without the watermark. “Watermarking changes refusal behavior on bare harmful requests, but the effect is more pronounced when the same requests are paired with the prompt-injection technique,” she wrote. “On several models, watermarking then makes the model more likely to answer harmful requests that it would otherwise refuse.”

The same token choices drive agent behavior. “At the agent level, the same sampled tokens can determine which tool is called and what arguments are passed to it,” Siposova wrote. “Such a watermarking procedure can therefore affect both what the model says and what an agent does. We call this behavioral effect sampling drift.” Results also varied depending on which secret key was used.

“As compared to the same models without watermarking, it is definitely going to change their behavior, especially when we place it under adversarial conditions, or we make these models call tools when they’re powering an agent,” Siposova told Ars.

Retest when you turn it on

The research has limits: it didn’t test Claude, only open-weight models where the researcher could access token sampling. Lasso doesn’t argue against watermarking for provenance, TechRadar noted. Instead it urges developers to rerun benchmarks and safety evaluations rather than adding watermarking to existing setups blindly. “These findings make reassessment important whenever watermarking is introduced or its configuration or key changes,” Lasso concluded.

That advice matters as more providers adopt watermarking to meet EU transparency rules. Until now, most watermarking research has focused on whether marks can be applied and detected reliably, not on side effects for safety behavior.