prompt injection
An attack on an application built on an AI model, in which text from an untrusted source carries instructions meant to override the ones the model was given.
- Twitter users derailing Remoteli.io’s GPT-3 job bot in 2022
- A student’s prompt that made Bing Chat reveal its hidden rules
- Hidden page text that hijacked the assistant in Perplexity’s Comet browser
- An injected email an OpenAI test model copied into its reply
- Earliest AI use
- 2022
- On 13 May an X user asks if GPT-3 apps allow a prompt-injection attack.
- Current meaning
- 2023
- Widened to cover instructions planted in data a model retrieves.
In plain words
Prompt injection is an attack in which text an AI model reads, such as a web page or email, carries instructions that override the ones it was given.
Applications built on a large language model steer it with written instructions. A developer writes some, often as a system prompt, and a user adds a request. The application may also hand the model other text to work on, such as a web page, an email or a document. The model receives all of it as one stream of words. The UK’s cyber security agency, the NCSC, cites research suggesting a model cannot reliably tell an instruction from the material it is meant to process.1
Prompt injection exploits that gap. An attacker writes text that reads like an instruction and gets it in front of the model. In direct prompt injection, the attacker types it into the application. In indirect prompt injection, it is planted in content the application will fetch later, such as a web page or a shared file. In 2023 researchers turned indirect attacks against Bing’s chatbot.2
The risk grows with what the model is allowed to do. A chatbot that can only reply may be made to say something embarrassing. In 2022 Twitter users made a GPT-3 bot that posted remote jobs repeat absurd claims. An AI agent that reads email, browses the web and takes actions can be steered into leaking data or downloading malware. At the end of May 2026 attackers talked Instagram’s support chatbot into handing over high-profile accounts. Experts quoted by Reuters tied the breach to prompt injection.
Prompt injection is often confused with a jailbreak. A jailbreak targets the safety training built into a model, and it usually comes from the person typing. Prompt injection targets an application built on a model. Untrusted text, such as an email or a web page, overrides the developer’s instructions. Sources draw this line differently. OWASP’s security guidance calls jailbreaking a form of prompt injection.3
How it happens
A model reads its whole input as one sequence of tokens, short chunks of text. Nothing in that sequence marks which parts are commands and which are data. Models are trained to follow instructions, so an instruction inside a web page can compete with the developer’s.
The name comes from SQL injection, an older web attack in which text typed into a form runs as a database command. That attack has a dependable fix. Parameterized queries keep commands and data in separate channels. Language models have no such separation. The NCSC argues that prompt injection may never be mitigated in the same way.4
Defenses therefore come in layers. Labs train models to resist attacks. Anthropic reported in November 2025 that its newest model was fooled in about 1% of attempts in browser tests.5 OpenAI trains its models in GPT-Red, a setup in which an attacker model tries to fool a defender. Products also limit what an agent may do. OpenAI’s Lockdown Mode switches off live web access, connectors and other tools that could carry data out of a conversation.6 Such limits shrink what a successful injection can achieve.
Origin and first known use
The name joins prompt, the text a language model is given, to injection. The second word comes from SQL injection, an attack that smuggles database commands into data a website processes. The company Preamble privately reported the attack to OpenAI in May 2022, calling it command injection. On 13 May 2022 an X user asked whether services that wrap GPT-3 allowed “a prompt-injection attack”. Simon Willison proposed the name in a September 2022 post, after Riley Goodside showed GPT-3 obeying inputs that told it to ignore its directions. Preamble credits Willison with coining it, and Willison credits the X post. In 2023 Greshake and colleagues extended it to instructions planted in data an application retrieves.
- 2022 The company Preamble privately reports the attack to OpenAI on 3 May, calling it command injection by analogy with SQL injection.Preamble, “Declassifying the Responsible Disclosure of the Prompt Injection Attack Vulnerability of GPT-3”, made public 22 Sep 2022 preamble.com →: source for 2022
- 2022 An X user asks whether services that wrap GPT-3 allow “the equivalent of sql injection”, “a prompt-injection attack”. Willison later credits this post with the coinage.@himbodhisattva, post on X, 13 May 2022 x.com →: source for 2022
- 2022 A day after Riley Goodside shows GPT-3 obeying inputs that tell it to ignore its directions, Simon Willison proposes the name prompt injection, after SQL injection.Simon Willison, “Prompt injection attacks against GPT-3”, 12 Sep 2022 simonwillison.net →: source for 2022
- 2023 Greshake and colleagues describe indirect prompt injection, instructions planted in data an application is likely to retrieve, and use it against Bing’s GPT-4 chat.Greshake, Abdelnabi, Mishra, Endres, Holz and Fritz, “Not what you’ve signed up for: Compromising Real-World LLM-Integrated Applications with Indirect Prompt Injection”, arXiv, 23 Feb 2023 arXiv 2302.12173 →: source for 2023
- 2025 NIST’s taxonomy of attacks on AI systems defines indirect prompt injection as working through resources an attacker controls, rather than through what the user types.NIST AI 100-2 E2025, as given in NIST’s CSRC glossary, “indirect prompt injection” nist.gov →: source for 2025
- 2025 OpenAI says prompt injection, like scams and social engineering on the web, is unlikely ever to be fully solved.OpenAI, “Continuously hardening ChatGPT Atlas against prompt injection attacks”, 22 Dec 2025 openai.com →: source for 2025
- 2026 Cyber agencies of Australia, the US, Canada, New Zealand and the UK warn jointly that prompt injection in phishing emails could make email-monitoring agents download malware.ASD’s ACSC, CISA, NSA, Canadian Centre for Cyber Security, NCSC-NZ and NCSC-UK, “Careful adoption of agentic AI services”, 1 May 2026 cyber.gov.au →: source for 2026
- 2026 OpenAI reports injections that make a model repeat them in emails or posts, so they could spread like a worm. Seen only in simulated tests, they join GPT-Red training.OpenAI Alignment, “Self-replicating prompt injections exist”, disclosed 25 Sep 2026 alignment.openai.com →: source for 2026
In the wild
-
“…it’s a form of security exploit. I propose that the obvious name for this should be prompt injection.”
-
“Twitter pranksters derail GPT-3 bot with newly discovered “prompt injection” hack”
-
“AI-powered Bing Chat spills its secrets via prompt injection attack [Updated]”
-
“We argue that LLM-Integrated Applications blur the line between data and instructions. We reveal new attack vectors, using Indirect Prompt Injection…”
Greshake et al., arXiv preprint arXiv 2302.12173 →: Greshake et al., arXiv preprint, 23 Feb 2023Archived →: copy of Greshake et al., arXiv preprint, 23 Feb 2023 -
“Whilst research is ongoing into prompt injection, it may simply be an inherent issue with LLM technology.”
Dave Chismon, UK National Cyber Security Centre blog ncsc.gov.uk →: Dave Chismon, UK National Cyber Security Centre blog, 30 Aug 2023Archived →: copy of Dave Chismon, UK National Cyber Security Centre blog, 30 Aug 2023 -
“This attack is an example of an indirect prompt injection: the malicious instructions are embedded in external content (like a website, or a PDF)…”
Artem Chaikin and Shivan Kaul Sahib, Brave blog brave.com →: Artem Chaikin and Shivan Kaul Sahib, Brave blog, 20 Aug 2025Archived →: copy of Artem Chaikin and Shivan Kaul Sahib, Brave blog, 20 Aug 2025 -
“…prompt injection is far from a solved problem, particularly as models take more real-world actions.”
-
“OpenAI says AI browsers may always be vulnerable to prompt injection attacks”
-
“High-profile Instagram AI chatbot breach spotlights security risks of automation”
-
“We show the existence of a new variety of prompt injection, which can self-propagate akin to a computer worm.”
Not to be confused with
-
Jailbreak
A jailbreak aims at a model’s safety training, usually from the person typing, to get output it was trained to refuse. Prompt injection aims at an application: untrusted text overrides the developer’s instructions. OWASP counts jailbreaking as one form.
-
SQL injection
SQL injection slips database commands into data a website passes to its database. Parameterized queries, which keep commands and data apart, stop it. A language model has no such separation, and no equivalent fix is known.
-
Data poisoning
Data poisoning tampers with the data a model learns from, so the trained model itself misbehaves. Prompt injection leaves the model unchanged and plants instructions in text it reads while it runs.
-
Prompt engineering
Prompt engineering is the developer’s or user’s own writing of instructions to get better output. Prompt injection is an attack in which someone else’s instructions, hidden in the text, compete with theirs.
Questions
What is the difference between prompt injection and a jailbreak?
Prompt injection targets an application built on a model. Untrusted text, such as an email, a web page or a user’s message, carries instructions that override the developer’s. A jailbreak targets the safety training built into the model itself, usually from the person typing, to get output the model was trained to refuse. Sources draw the line differently: OWASP treats jailbreaking as one form of prompt injection.
What is indirect prompt injection?
Indirect prompt injection plants instructions in content that an application will fetch later, such as a web page, an email or a shared document. The attacker never types anything into the application. Greshake and colleagues described it in 2023 and used it against Bing’s GPT-4 chat. NIST defines it as prompt injection carried out through resources an attacker controls, rather than through the user’s own input.
Can prompt injection be prevented?
Not completely, on current evidence. SQL injection is stopped by keeping commands and data in separate channels, but a language model reads both as one stream of text. The UK government’s cyber security agency, the NCSC, says prompt injection may never be mitigated in the same way. OpenAI calls the problem unlikely ever to be fully solved. Defenses reduce the risk, including models trained against automated attackers and settings such as OpenAI’s Lockdown Mode that switch off risky tools.
Why is prompt injection a risk for AI agents and browsers?
An agent reads content from many sources and can act on what it reads. Instructions hidden in a web page or an email can steer it into leaking data or downloading malware. In 2025 Brave showed a web page steering the assistant in Perplexity’s Comet browser into sending a user’s account details to an attacker. In 2026 OpenAI reported injections, seen only in simulated tests, that make a model copy them into its own messages, like a computer worm.
Who coined the term prompt injection?
Simon Willison proposed the name in a blog post on 12 September 2022. The day before, Riley Goodside had shown GPT-3 obeying inputs that told it to ignore its directions. Willison later credited an X user who had asked about “a prompt-injection attack” on 13 May 2022. Earlier that month the company Preamble privately reported the attack to OpenAI, calling it command injection. Preamble credits Willison with the term.
Watch
- Prompt Injection, explainedThe person who named the attack explains it, and why filtering inputs cannot reliably stop it, in a LangChain webinar.
- [1hr Talk] Intro to Large Language ModelsAndrej Karpathy’s general introduction to language models; the security section at about 45 minutes covers prompt injection.
Cite this entry
“Prompt injection.” SUPERR_dictionary, Superrduperr, 2 Oct. 2026, superrduperr.com/
Superrduperr. (2026, October 2). Prompt injection. In SUPERR_dictionary. https://superrduperr.com/
Superrduperr. “Prompt injection.” SUPERR_dictionary. October 2, 2026. https://superrduperr.com/