Real-Time Reasoning Agents in Evolving Environments
3Georgia Institute of Technology 4Stanford University *Co-leading authors
We create three real-time games, Freeway, Snake and Overcooked, to study the challenge of real-time reasoning. In these games, agents need to deal with dynamic environments smartly and timely to achieve high rewards. Experiments show that under cognitive load and time pressure, AgileThinker (Ours), which engages two LLMs with both System 1 and 2 reasoning, greatly outperforms agents engaging only one LLM. Here scores are normalized to [0, 1] for each game and then taken an average.
Abstract
Agents in the real world need to make not only logical but also timely judgments, which demands continuous awareness of the dynamic environment where hazards emerge, opportunities arise, and other agents act - all while the agent's own reasoning is still unfolding. Despite significant advances in reasoning capabilities of language models, existing approaches fail to account for this dynamic nature. We introduce real-time reasoning as a new problem formulation for bringing reasoning capabilities to agents operating in evolving environments and build a Real-Time Reasoning Gym to demonstrate it. We study two paradigms for deploying reasoning language models in agents: System 1 Agent, which employs reasoning models with bounded computation for rapid responses, and System 2 Agent, which allows extended computation for complex problems. Our experiments reveal that even state-of-the-art language models struggle with making logical and timely judgment via either of the two paradigms. To address this limitation, we propose AgileThinker, a parallel architecture that simultaneously engages both reasoning systems. This approach demonstrates superior performance as task difficulty and time pressure increase, managing the trade-off between reasoning depth and response latency. Our work establishes real-time reasoning as a critical frontier for developing practical reasoning agents and provides a foundation for future research in temporally-constrained artificial intelligence systems, highlighting a path toward real-time capable language agents.
Agent Thinking Process Comparison
Choose Your Settings
Using the rlr-gym Python Package
Installation
Install the Real-Time Reasoning Gym package using pip:
pip install rlr-gym
Requires Python 3.8+ and PyTorch
Quick Start
Get started with a simple example:
import rlr_gym
# Create environment
env = rlr_gym.make('Freeway-v0')
obs = env.reset()
Available Environments
Freeway
Navigate through dynamic traffic with real-time decision making.
rlr_gym.make('Freeway-v0')
Snake
Strategic planning for food collection while avoiding obstacles.
rlr_gym.make('Snake-v0')
Overcooked
Cooperative cooking with coordination and task prioritization.
rlr_gym.make('Overcooked-v0')
Complete Example
import rlr_gym
from rlr_gym.agents import AgileThinker
# Initialize environment and agent
env = rlr_gym.make('Freeway-v0', cognitive_load='medium')
agent = AgileThinker(model_name='gpt-4')
# Run episode
obs = env.reset()
total_reward = 0
for step in range(1000):
action = agent.act(obs)
obs, reward, done, info = env.step(action)
total_reward += reward
if done:
break
print(f"Total reward: {total_reward}")
BibTeX
@article{wen2024realtime,
title={Real-Time Reasoning Agents in Evolving Environments},
author={Wen, Yule and Ye, Yixin and Zhang, Yanzhe and Yang, Diyi and Zhu, Hao},
journal={International Conference on Learning Representations},
year={2025},
url={https://bleaves.github.io/real-time-reasoning/}
}