Multilingual Jailbreaking in Agentic AI
Jailbreaking beyond the user prompt: injecting harmful instructions through tool outputs, documents, and skills in multilingual agentic AI.
About the project
The basic idea is that, most jailbreaking focuses on the user prompt, but with the agentic nature of the current AI solution, there are multiple components that can be used to inject harmful instruction that goes beyond user prompt, exposing a real security risk beyond the direct user of the LLMs.
What mentees potentially work on
- Set up a corporate scenario in which an attacker injects instructions into internal documents
- Focus on tool outputs: embed instructions into an instruction document, add gibberish to obfuscate the model, and inject instructions into a skill document