GitHub - wenyl22/RealtimeGym (original) (raw)

⚡🧠🏋️ Realtime Reasoning Gym

Realtime Reasoning Gym is a specialized evaluation framework for testing how well language agents can reason and make decisions under real-time constraints. Unlike traditional OpenAI Gym environments where agents have unlimited thinking time, Realtime Reasoning Gym enforces strict time budgets (measured in seconds) or token budgets (measured in LLM decoding tokens) to simulate real-world pressure.

Furthermore, each environment in the gym offers multiple cognitive load levels that vary the intellectual complexity of tasks, enabling a comprehensive assessment of whether agents can balance decision quality and speed when facing different levels of cognitive load and time pressure.

Upper: We create three real-time games, Freeway, Snake, and Overcooked, to study the challenge of real-time reasoning. Lower: Under cognitive load and time pressure, AgileThinker (Ours), which engages both reactive and planning reasoning paradigms, consistently outperforms agents that engage only one of them. Scores are averaged across different games.

You can see our post, paper, website for more detailed demonstrations and explanations.

• Quick Start
• Real-Time Reasoning Challenge
  • Cognitive Load Control
  • Time Pressure Control
• Built-in LLM Agents
• Add a New Environment
• API Reference
• More Examples

Quick Start

1. Install Realtime Reasoning Gym:
   git clone https://github.com/wenyl22/RealtimeGym.git
   cd RealtimeGym

Install in development mode

pip install -e .

Install development dependencies

pip install -e ".[dev]" 2. Set up API keys in .env, a template is provided in .env.example.

The Real-Time Reasoning Challenge

In typical OpenAI Gym environments, agents have unlimited time to think:

Traditional approach - unbounded thinking time

obs, done = env.reset() while not done: action = agent.act(obs) # Can take minutes! obs, reward, done, info = env.step(action)

RealtimeGym introduces explicit constraints on the agent's thinking time:

Real-time approach - bounded thinking time

obs, done = env.reset() while not done: agent.observe(obs) # Fast observation agent.think(timeout=8192) # Bounded thinking (tokens or seconds) action = agent.act() or DEFAULT_ACTION # Default if a decision isn't made in time obs, done, reward, reset = env.step(action)

Real-time reasoning requires agents to balance correctness and timeliness.

Cognitive Load Control

Realtime Reasoning Gym includes three real-time games with increasing cognitive loads:

Create any environment:

env, real_seed, renderer = realtimegym.make('Freeway-v2', seed=0, render=False)

Note:

seed != real_seed, because real_seed embeds the cognitive load level.

renderer = None if render=False, otherwise call renderer.render(env) to visualize the game state.

See examples/basic_renderer.py for details.

Time Pressure Control

Realtime Reasoning Gym supports two time constraint types:

Token Budget

agent.think(timeout=8192) # Environment evolves after 8192 decoding steps

• Best for: LLM-based agents
• Measures: Token count from API calls
• Advantage: Platform-independent, reproducible

Time Budget

agent.think(timeout=5.0) # Environment evolves after 5 seconds

• Best for: Real-time deployment scenarios
• Measures: Wall-clock time
• Advantage: Direct real-world applicability

Set via --time_unit token or --time_unit seconds.

Built-in LLM Agents

Real-time Reasoning Gym's built-in agents all implement the BaseAgent interface:

from .base import BaseAgent

class MyAgent(BaseAgent): def init( self, prompts: Any, # prompt: dynamically loaded module, mapping game state to text file: str, # log file path time_unit: str, # 'token' or 'seconds' **kwargs, ) -> None: super().init(prompts, file, time_unit) # Initialize internal state here

def think(self, timeout: int): -> None:
    """Process information and plan action within budget.

    Args:
        timeout: Token count (time_unit='token') or seconds (time_unit='seconds')
    """
    # Decision making within timeout budget
    # store chosen action here
    self.action = ...

RealtimeGym provides three ready-to-use agent types:

Evaluation

Evaluate agents using the command-line interface:

Detailed configuration

agile_eval --time_unit token
--time_pressure 8192
--internal_budget 4096
--game freeway
--cognitive_load E
--mode agile
--reactive-model-config configs/deepseek-v3.2-reactive.yaml
--planning-model-config configs/deepseek-v3.2-planning.yaml
--seed_num 1 --repeat_times 1

Using more compact configurations

agile_eval --time_unit token
--settings freeway_H_8192_agile_4096
--reactive-model-config configs/deepseek-v3.2-reactive.yaml
--planning-model-config configs/deepseek-v3.2-planning.yaml
--seed_num 8 --repeat_times 1

Add a New Environment

To create a custom environment:

1. Create a new file in src/realtimegym/environments/ inheriting from BaseEnvironment:
   from .base import BaseEnvironment
   class MyGameEnv(BaseEnvironment):
   def init(self):
   super().init()
   def set_seed(self, seed: int):
   # Set game random seed
   # In our implementation, cognitive load level is embedded in seed
   pass
   def reset(self):
   # Initialize game state
   return observation, done
   def step(self, action):
   # Process action and update state
   return observation, done, reward, reset
   def state_string(self):
   # Return human-readable state in text
   return state_string
   def state_builder(self):
   # Return detailed state dict
   return state_dict
2. Register your environment in src/realtimegym/__init__.py:
3. (Optional) To evaluate built-in LLM agents, create prompts in src/realtimegym/prompts/mygame.py following existing patterns. Specifically, you need to implement a function:
   def state_to_description(observation: dict, mode: str) -> str | dict:

   Return different descriptions based on agent mode

   if mode == "reactive":
   return text_description_for_reactive_agent
   elif mode == "planning":
   return text_description_for_planning_agent
   elif mode == "agile":
   return {
   "planning": text_description_for_planning_agent,
   "reactive": text_description_for_reactive_agent
   }

API Reference

Environment API

Environment creation

env, seed, renderer = realtimegym.make(env_id, seed=0, render=False)

Environment interaction

obs, done = env.reset() obs, done, reward, reset = env.step(action)

Environment Observation Structure

{ 'state_string': str, # Human-readable game state 'game_turn': int, # Current turn number 'state': dict # Detailed game-specific state }

Agent Configuration

Agents accept YAML configuration files specifying model parameters:

model: "gpt-4o-mini" api_key: "your-api-key" inference_parameters: temperature: 0.7 max_tokens: 1000

Testing

Run the comprehensive test suite:

All tests

pytest

Specific test categories

pytest tests/test_environments.py pytest tests/test_agents.py

Examples

Explore the examples/ directory:

Basic usage patterns

python examples/basic_usage.py

Basic usage with rendering

python examples/basic_renderer.py

Compare all environments

python examples/all_environments.py

Custom agent implementation

python examples/custom_agent.py

Cognitive load level analysis

python examples/difficulty_levels.py

Citation

If you use Realtime Reasoning Gym in your research, please cite our work:

@software{realtimegym2025, title={Realtime Reasoning Gym: A Real-Time Learning Environment for Language Agents}, author={Yule Wen, Yixin Ye, Yanzhe Zhang, Diyi Yang and Hao Zhu}, year={2025}, url={https://github.com/wenyl22/RealtimeGym}, note={A framework for evaluating language agents under real-time constraints} }

Acknowledgements

RealtimeGym builds upon the excellent open-source project Overcooked. We thank the original authors for their contributions.

Links

• 🏠 Homepage: https://github.com/wenyl22/RealtimeGym
• 📖 Documentation: https://bleaves.github.io/real-time-reasoning/
• 🐛 Issues: https://github.com/wenyl22/RealtimeGym/issues
• 💬 Discussions: https://github.com/wenyl22/RealtimeGym/discussions

RealtimeGym - Advancing real-time reasoning capabilities in language agents ⚡