Definition
Rather than attempting one prompt that defeats a control outright, the attacker advances in steps: establishing context, obtaining a small concession, then treating that concession as precedent for the next.
Why per-message filtering struggles
Controls that evaluate each message in isolation see nothing objectionable, because no individual step is objectionable. The bypass exists in the accumulated conversation state, which is exactly what a single-message classifier does not evaluate.
What determines whether it matters
A model producing text outside policy is a quality problem. It becomes a security problem when the surrounding application grants that output authority: to reveal restricted context, to reach data, or to influence an action. The consequence is a property of the system, not of the sentence.
- User
- Adversarial Prompt Sequence
- Control Bypass
- Restricted Behavior
Each hop validated by successful exploitation
Tags
- jailbreak
- prompt injection
- control bypass