Bump stable-baselines3 from 2.2.1 to 2.4.1

[dependabot]

Bumps stable-baselines3 from 2.2.1 to 2.4.1.

Release notes

Sourced from stable-baselines3's releases.

Stable-Baselines3 v2.4.1: Fix for VecVideoRecorder

Bug Fixes

• Fixed a bug introduced in v2.4.0 where the VecVideoRecorder would override videos

Full Changelog: DLR-RM/stable-baselines3@v2.4.0...v2.4.1

Stable-Baselines3 v2.3.0: New defaults hyperparameters for DDPG, TD3 and DQN

[!WARNING] Because of weights_only=True, this release breaks loading of policies when using PyTorch 1.13. Please upgrade to PyTorch >= 2.0 or upgrade SB3 version (we reverted the change in SB3 2.3.2)

SB3 Contrib (more algorithms): https://github.com/Stable-Baselines-Team/stable-baselines3-contrib RL Zoo3 (training framework): https://github.com/DLR-RM/rl-baselines3-zoo Stable-Baselines Jax (SBX): https://github.com/araffin/sbx

To upgrade:

pip install stable_baselines3 sb3_contrib --upgrade

or simply (rl zoo depends on SB3 and SB3 contrib):

pip install rl_zoo3 --upgrade

Breaking Changes:

• The defaults hyperparameters of TD3 and DDPG have been changed to be more consistent with SAC

  # SB3 < 2.3.0 default hyperparameters
  # model = TD3("MlpPolicy", env, train_freq=(1, "episode"), gradient_steps=-1, batch_size=100)
  # SB3 >= 2.3.0:
  model = TD3("MlpPolicy", env, train_freq=1, gradient_steps=1, batch_size=256)

[!NOTE] Two inconsistencies remain: the default network architecture for TD3/DDPG is [400, 300] instead of [256, 256] for SAC (for backward compatibility reasons, see report on the influence of the network size ) and the default learning rate is 1e-3 instead of 3e-4 for SAC (for performance reasons, see W&B report on the influence of the lr )

• The default learning_starts parameter of DQN have been changed to be consistent with the other offpolicy algorithms

  # SB3 < 2.3.0 default hyperparameters, 50_000 corresponded to Atari defaults hyperparameters
  # model = DQN("MlpPolicy", env, learning_starts=50_000)
</tr></table> 

... (truncated)

Commits

• 9555dd9 Fix tests and warnings when running locally with a GPU (#2069)
• 593b2d9 Ignore A2C warnings too
• c0eab56 Sort file list for consistent results
• c09a9d7 Ignore new errors
• 48a8389 Fix mypy error
• 5b19e37 Fix video recorder and add test (#2063)
• 020ee42 Release 2.4.0 (#2040)
• e4f4f12 Add note about SAC ent coeff optimization (#2037)
• 8f0b488 Update Gymnasium to v1.0.0 (#1837)
• dd3d0ac Update readme and clarify planned features (#2030)
• Additional commits viewable in compare view

Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.

---

Dependabot commands and options

You can trigger Dependabot actions by commenting on this PR:

• @dependabot rebase will rebase this PR
• @dependabot recreate will recreate this PR, overwriting any edits that have been made to it
• @dependabot merge will merge this PR after your CI passes on it
• @dependabot squash and merge will squash and merge this PR after your CI passes on it
• @dependabot cancel merge will cancel a previously requested merge and block automerging
• @dependabot reopen will reopen this PR if it is closed
• @dependabot close will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
• @dependabot show <dependency name> ignore conditions will show all of the ignore conditions of the specified dependency
• @dependabot ignore this major version will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
• @dependabot ignore this minor version will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
• @dependabot ignore this dependency will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)

[dependabot]

Superseded by #508.