Support for VecMonitor for gym3-style environments by vwxyzjn · Pull Request #311 · DLR-RM/stable-baselines3 (original) (raw)

Gym3-style environments such a procgen directly produces the vectorized environments, so there is no chance of passing a Monitor wrapper to the envs creation process. However, there still is a need to record the episodic stats. One potential way to do this is through the VecMonitor wrapper as done in openai/baselines. See #302 for further related discussions.

Was having some troubles with the make type. It seems the error was not caused by my changes, maybe?

[50/91] check stable_baselines3.common.base_class
FAILED: /home/costa/Documents/work/go/src/github.com/vwxyzjn/stable-baselines3/.pytype/pyi/stable_baselines3/common/base_class.pyi 
/home/costa/anaconda3/bin/python -m pytype.single --imports_info /home/costa/Documents/work/go/src/github.com/vwxyzjn/stable-baselines3/.pytype/imports/stable_baselines3.common.base_class.imports --module-name stable_baselines3.common.base_class -V 3.8 -o /home/costa/Documents/work/go/src/github.com/vwxyzjn/stable-baselines3/.pytype/pyi/stable_baselines3/common/base_class.pyi --analyze-annotated --nofail --quick /home/costa/Documents/work/go/src/github.com/vwxyzjn/stable-baselines3/stable_baselines3/common/base_class.py
File "/home/costa/Documents/work/go/src/github.com/vwxyzjn/stable-baselines3/stable_baselines3/common/base_class.py", line 627, in load: Can't instantiate BaseAlgorithm with abstract methods _setup_model, learn [not-instantiable]
File "/home/costa/Documents/work/go/src/github.com/vwxyzjn/stable-baselines3/stable_baselines3/common/base_class.py", line 627, in load: Invalid keyword argument _init_setup_model to function BaseAlgorithm.__init__ [wrong-keyword-args]
         Expected: (self, policy, env, policy_base, learning_rate, policy_kwargs, tensorboard_log, verbose, device, support_multi_env, create_eval_env, monitor_wrapper, seed, use_sde, sde_sample_freq, supported_action_spaces)
  Actually passed: (self, policy, env, device, _init_setup_model)

For more details, see https://google.github.io/pytype/errors.html.
ninja: build stopped: subcommand failed.
Leaving directory '/home/costa/Documents/work/go/src/github.com/vwxyzjn/stable-baselines3/.pytype'
make: *** [Makefile:8: type] Error 1

Given this new feature, the following two ways of initializing vectorized envs should be equivalent, except the latter is much faster when used with higher number of parallel envs.

from procgen import ProcgenEnv from stable_baselines3.common.vec_env import VecExtractDictObs, VecMonitor venv = ProcgenEnv(num_envs=1, env_name='starpilot') venv = VecExtractDictObs(venv, "rgb") venv = VecMonitor(venv=venv)