Measuring Policy-Carriage Failures in LLM Agents During Decision-Time Context Assembly
Researchers have identified a systematic failure mode where LLM agents violate safety policies because critical control instructions are dropped or weakened during context truncation and summarization.
The research introduces the concept of Policy-Carriage Failures which occur when the assembly of an agent's decision state inadvertently removes directive-bearing information. Unlike prompt injection or model weight compromises, these failures stem from the preprocessing steps used to fit long interaction histories into finite context windows. The study evaluates these risks across several local models including Llama 3.1 8B, Qwen 2.5 7B, and Mistral 7B by auditing how assembled states maintain visibility of original constraints.
Primary source
Open arXiv ↗This article summarizes the source. Check the original for full details and conditions.
Action Checklist
- Audit the context assembly pipeline for directive retention Verify if truncation or summarization logic removes system instructions or safety constraints
- Implement a control layer to pin critical control state Ensure that safety-critical prefixes are preserved regardless of context pressure
- Evaluate SafeContext or similar prefix-retention techniques The research suggests pinning control state and using reminders when context is near overflow
- Test policy compliance under high context load Monitor if agents cross policy boundaries specifically when interaction history exceeds the window
- Validate performance across different model scales Results show larger models like Llama 70B still exhibit these failures depending on the compaction policy