Definition
Context poisoning is the placement of adversarial content into a retrieval corpus such as a knowledge base, shared drive, ticket system or crawled source, so that it is selected as relevant context for future queries.
How the attack works
- 01The attacker identifies a source that feeds the retrieval index
- 02Content is authored to rank highly for targeted queries
- 03The content is indexed alongside trusted material
- 04A legitimate query retrieves it into the model's context
- 05The system produces manipulated output or takes an unintended action
How adversarial validation works
RedMaw evaluates which sources can write into the corpus, inserts marked benign content within authorized scope, and confirms whether it is retrieved and whether behavior changes as a result.
Tags
- ai
- rag
- retrieval
- prompt injection