Mitigating Many-Shot Jailbreak Attacks in Large Language Models with One-Shot Safety Demonstrations

Many-shot jailbreaking (MSJ) poses a significant threat to safety-aligned language models by using numerous harmful question-answer demonstrations to bypass safety guardrails. Research indicates that this attack works through activation drift, where the internal representation of a query moves progressively away from the model's safety-aligned region as more harmful examples are added to the context. This phenomenon can be interpreted as implicit malicious fine-tuning that occurs during the inference process itself. To address this vulnerability, the researchers suggest appending a single, fixed safety demonstration at inference time. This approach induces a counteracting safety-oriented update in the model's internal state, restoring the expected refusal behavior. The proposed method is particularly valuable for production environments because it does not require modifying model parameters or having white-box access to the underlying architecture. Software engineers can implement this as a prompt-level defense to improve robustness against sophisticated prompt injection and jailbreaking attempts. Evaluation shows that this lightweight intervention significantly improves model resistance to long-context attacks without the computational overhead of retraining or the complexity of white-box defenses.