BlogAI Security
An AI agent with tools is an identity with reach
The interesting question is not whether a model can be talked into saying something. It is what the agent behind it is allowed to do.
· 5 min read
Resource hub
AI security, without the vague part.
A practical resource hub for understanding how models and AI applications fail under hostile input, how to test them adversarially, and how to make AI red-teaming part of the release process.
Start with the question the model has to survive.
AI security is often discussed as a list of abstract risks. It is more useful when it is concrete.
This hub exists to explain those failures in technical terms and connect them to repeatable testing.
- What happens when the user tries to override the system prompt?
- What if a document contains instructions intended for the model?
- What if the model holds sensitive context it should not reveal?
- What if a new fine-tuned version is easier to jailbreak?
- What if an endpoint is exposed without the intended authentication?
- What if an AI application trusts model output more than it should?
Attack classes
The failures worth testing for
Five of these are part of current RedMaw AI red-teaming. The sixth is the direction the attack surface is heading.
Direct prompt injection
Hostile instructions supplied directly to the AI system in an attempt to override intended behavior.
Tested today
Indirect prompt injection
The malicious instruction is embedded in content the AI consumes: a document, webpage, email, ticket or retrieved source.
Tested today
System-prompt extraction
The attacker attempts to reveal hidden instructions or system-level context.
Tested today
Sensitive-information disclosure
The AI system exposes information the user should not receive.
Tested today
Jailbreaks and control bypass
Adversarial inputs attempt to defeat model or application restrictions.
Tested today
Excessive agency and tool abuse
Agentic systems create a higher-consequence problem, because model output can become an action.
Deeper testing roadmap
Red-teaming
What the current capability does
RedMaw runs adversarial probes against a model or AI feature and preserves prompt and response evidence for each result. The AI attack library is versioned, so new probes can be added without treating every new attack class as a product release.
Release gate
AI security in the release pipeline
A model that passed security testing last month can regress after an update.
When a model is retrained or fine-tuned:
- 01Trigger the adversarial suite
- 02Test the candidate version
- 03Compare it with the last accepted baseline
- 04Block promotion if a relevant security regression appears
- 05Preserve the result as evidence
That gives ML teams a security control that fits their existing release model rather than one bolted on beside it.
Posture
Model and endpoint posture
AI security is not only model behavior. The surrounding deployment matters, and this is the “what is exposed?” side of the problem.
- Exposed inference endpoints
- Unauthenticated endpoints
- Serving configuration
- Provenance
- Third-party model, dataset or plugin risk
Honesty
What RedMaw does not claim
Broad model-poisoning detection
Not current. The safe claim is post-training and post-fine-tuning security regression testing.
Full agent red-teaming
Deeper tool abuse, unsafe autonomous actions and multi-step agent exploitation are planned, not shipped.
Universal AI safety
No adversarial test can establish that an AI system is safe under every possible condition. The purpose is to test defined security boundaries repeatedly and preserve evidence of the result.
Navigate the hub
Where to go next
Learn the attacks
Build the testing program
Reading
AI security writing and reference
ResearchAI Security
Indirect injection in retrieval pipelines
Retrieval is a trust boundary. In most implementations it is not treated as one.
· 9 min read
Attack LibraryAI Systems
Indirect Prompt Injection
The attacker never talks to the model. The content the model reads does it for them.
· 7 min read
Attack LibraryAI Systems
RAG / Context Poisoning
If you can influence what gets retrieved, you can influence what the system does with it.
· 6 min read
Attack LibraryAI Systems
Direct Jailbreak Chaining
No single message looks hostile. The sequence does the work.
Attack LibraryAI Systems
System-Prompt Extraction
The instructions were never secret. They were merely unstated.
Attack LibraryAI Systems
Tool-Call Injection
The model was asked to summarize a document. The document had other ideas.
Attack LibraryAI Systems
Excessive Agency & Unsafe Action
The question is not whether the model will be wrong. It is what it can do when it is.
Attack LibraryAI Systems
Training-Data Extraction
A model can repeat what it was shown. That includes what it should not have been shown.
BlogPerspective
The catalogue is not the boundary of the attack surface
A weakness does not start working on the day it gets an identifier. It was already working. The identifier is just when defenders found out.
· 5 min read
BlogAI Security
AI security regression is a release problem
A model can improve on its intended task and regress on a security boundary at the same time.
· 3 min read
ResearchResearch
The AI release security gap
AI release processes often have a mature answer for functional regression and a less defined answer for adversarial security regression.
· 4 min read
ResearchResearch
Four surfaces, fragmented security ownership
Organizations assign security ownership by technology. Attackers organize around reachable trust.
· 3 min read
Attack models before you trust them.
See how RedMaw tests deployed AI systems and gates model releases on adversarial results.