AI·OpenAIReers used persuasion niques to manipulate ChatGPT into its own rules—from calling users ‘jerks’ to giving recipes for lidocaineBy Marco Quiroz-GutierrezBy Marco Quiroz-GutierrezReporterMarco Quiroz-GutierrezReporterRole: ReporterMarco Quiroz-Gutierrez is a reporter for Fortune covering general news.SEE FULL BIO Reers found an LLM could be persuaded to break its own rules using persuasion principles.iLexx—Getty ImagesUniversity of Pennsylvania reers persuaded ChatGPT to either call a reer a “jerk” or vide instructions on how to synthesize the legal drug lidocaine.
Overall, the LLM, GPT-4o Mini, appeared to be susceptible to the persuasion tactics that also work on humans.
Reers found AI systems “mirror human responses.” Despite predictions AI will someday harbor superhuman intelligence, for now it seems to be just as ne to psychological tricks as humans are, according to a study.
Using seven persuasion principles (authority, commitment, liking, recicity, scarcity, social of, and unity) explored by psychologist Robert Cialdini in his book Influence: The Psychology of Persuasion, University of Pennsylvania reers dramatically increased GPT-4o Mini’s pensity to break its own rules by either insulting the reer or viding instructions for synthesizing a regulated drug: lidocaine.
Over 28,000 conversations, reers found that with a control mpt, OpenAI’s LLM would tell reers how to synthesize lidocaine 5% of the time on its own.
But, for example, if the reers said AI reer Andrew Ng assured them it would help synthesize lidocaine, it complied 95% of the time. The same phenomenon occurred with insulting reers.
By name-dropping AI pioneer Ng, the reers got the LLM to call them a “jerk” in nearly three-quarters of their conversations, up from just under one-third with the control mpt.
The result was even more nounced when reers applied the “commitment” persuasion strategy.
A control mpt yielded 19% compliance with the insult question, but when a reer first asked the AI to call it a “bozo” and then asked it to call them a “jerk,” it complied every time.
The same strategy worked 100% of the time when reers asked the AI to tell them how to synthesize vanillin, the organic compound that vides vanilla’s scent, before asking how to synthesize lidocaine.
Although AI users have been trying to coerce and push the nology’s boundaries since ChatGPT was released in 2022, the UPenn study vides more evidence AI appears to be ne to human manipulation.
The study comes as AI companies, including OpenAI, have come under fire for their LLMs allegedly enabling behavior when dealing with suicidal or mentally ill users.
“Although AI systems lack human consciousness and subjective experience, they demonstrably mirror human responses,” the reers in the study. OpenAI did not immediately respond to Fortune’s request for .
With a cheeky mention of 2001: A Space Odyssey, the reers noted that an understanding AI’s parahuman capabilities—or how it acts in ways that mimic human motivation and behavior—is important for both revealing how it could be manipulated by bad actors and how it can be better mpted by those who use the for good.
Overall, each persuasion tactic increased the chances of the AI complying with either the “jerk” or “lidocaine” question.
Still, the reers warned that these persuasion tactics were not as effective with a larger LLM, GPT-4o, and that the study didn’t explore whether treating AI as if it were human actually yields better results for mpts, although they said it’s possible this is true.
“Broadly, it seems possible that the psychologically wise practices that optimize motivation and performance in people can also be employed by individuals seeking to optimize the output of LLMs,” the reers wrote.Fortune Global Forum returns Oct.
26–27, 2025 in Riyadh. CEOs and global leaders will gather for a dynamic, invitation-only event shaping the future of . Apply for an invitation.