RealtimeBench and TORSI Framework Address LLM Agent Delays in Dynamic Environments
A new research paper introduces RealtimeBench to evaluate LLM agents under real-time constraints where environment state updates continuously during inference. The authors also propose TORSI, a two-timescale framework that allows slow teacher models to generate reusable instructions for faster student models.
Large language model agents often struggle in dynamic environments because inference and tool calls introduce latency, causing delayed actions to fail even if they are logically correct. To measure this challenge, researchers introduced RealtimeBench, a benchmark featuring 180 tasks across six game environments where time continues to elapse during model generation. Evaluations demonstrated that slower, higher-capacity models like GPT-6.1-sol outperform faster models when time is paused, but lose their advantage when environments run in real time. To bridge this gap without sacrificing execution speed during live operations, the paper introduces Two-Timescale Offline Recursive Self-Improvement. In this setup, a slower teacher model analyzes failure logs in paused environments to draft execution guidelines between attempts. A faster student model then executes actions in real time using these guidelines, completely offline from the teacher during gameplay. Across testing environments, two rounds of TORSI training raised the fast student model's average score from 13.51 to 24.19 on a 100-point scale across all six game types. The generated guidelines also improved performance for three other fast models without requiring additional tuning. Engineers considering real-time agent deployments should evaluate the reproduction conditions and task dependencies before applying these asynchronous instruction strategies to production workloads.
Primary source
Open arXiv ↗This article summarizes the source. Check the original for full details and conditions.
Comparison
| Aspect | Before / Alternative | After / This |
|---|---|---|
| Evaluation Environment | Paused state during agent inference | Continuous live clock during agent inference |
| Model Architecture | Single slow, powerful model handling real-time decisions directly | Asynchronous two-timescale setup with slow teacher and fast student |
Action Checklist
- Evaluate inference latency impact on target interactive tasks using continuous environments Determine if slow reasoning steps lead to state drift before actions complete
- Implement an offline teacher-student instruction loop for real-time workflows Have a high-capability model analyze execution logs to write static runtime instructions for faster runtime models
