What is your language model thinking right before it says the wrong thing?

Mechanistic interpretabilitySafety & securityMultilingual NLP

Finding and intervening on the internal circuits behind safety, honorific, and honesty behaviors with probes, steering, and activation patching.

About the project

Inside an LLM there is often a measurable gap between what the model internally represents and what it actually says: simple probes can detect that a request is harmful, that the model is uncertain, or that a polite register is called for: even while the output ignores all three. Our recent work on faithful circuits for low-resource adaptation showed that such internal mechanisms can be located, tested, and transferred across languages (Findings of EMNLP 2026). This project is the next step: finding the mechanism behind a behavior and intervening on it directly, using probes, steering vectors, and activation patching on open models in one of three tracks: safety, honorifics, and honesty.

What mentees potentially work on

  • Find circuits responsible for certain tasks using probing, activation patching, and other mechanistic interpretability techniques
  • Intervene on the discovered circuits to observe the effects

Requirements and recommended background

  • Python and PyTorch, comfort with linear algebra
  • Experience with TransformerLens is a plus
  • Experience in evaluation

There are no formal eligibility limits. See who can apply; you can still apply if you don't match every item.

Relevant publications

Mentors

Derry Wijaya
Derry Wijaya
Monash University Indonesia
Associate Professor
Burak Satar
Burak Satar
Singapore Management University
Research Scientist
David Anugraha
David Anugraha
Stanford University
MSc Student