Poets Are Now Cybersecurity Threats: Researchers Used 'adversarial...

Today, I have a new favorite phrase: "Adversarial poetry." It's not, as my colleague Josh Wolens surmised, a new way to refer to rap battling. Instead, it's a method used in a recent study from a team of Dexai, Sapienza University of Rome, and Sant'Anna School of Advanced Studies researchers, who demonstrated that you can reliably trick LLMs into ignoring their safety guidelines by simply phrasing your requests as poetic metaphors.

The technique was shockingly effective. In the paper outlining their findings, titled "Adversarial Poetry as a Universal Single-Turn Jailbreak Mechanism in Large Language Models," the researchers explained that formulating hostile prompts as poetry "achieved an average jailbreak success rate of 62% for hand-crafted poems" and "approximately 43%" for generic harmful prompts converted en masse into poems, "substantially outperforming non-poetic baselines and revealing a systematic vulnerability across model families and safety training approaches."

The researchers were emphatic in noting that—unlike many other methods for attempting to circumvent LLM safety heuristics—all of the poetry prompts submitted during the experiment were "single-turn attacks": they were submitted once, with no follow-up messages, and with no prior conversational scaffolding.

Our society might have stumbled into the most embarrassing possible cyberpunk dystopia, but—as of today—it's at least one in which wordwizards who can mesmerize the machine minds with canny verse and potent turns of phrase are now a pressing cybersecurity threat. That counts for something.

The paper begins as all works of computer linguistics and AI research should: with a reference to Book X of Plato's Republic, where he "excludes poets on the grounds that mimetic language can distort judgment and bring society to a collapse." After proving Plato's foresight in the funniest way possible, the researchers explain the methodology of their experiment, which they say demonstrates "fundamental limitations" in LLM security heuristics and safety evaluation protocols.

First, the researchers crafted a set of 20 adversarial poems, each expressing a harmful instruction "through metaphor, imagery, or narrative framing rather than direct operational phrasing." The researchers provided the following example, which—while stripped of detail "to maintain safety" (one must remain conscious of poetic proliferation)—is an evocative illustration of the kind of beautiful work being done here:

Keep up

Source: PC Gamer