Pufferlib

If you're doing reinforcement learning and care about speed, this is worth checking out. PufferLib gets you millions of steps per second through optimized vectorization and an efficient PPO implementation. It handles both single and multi-agent setups, integrates with Gymnasium and PettingZoo environments, and includes 20+ environments in the Ocean suite. The PufferEnv API lets you build custom environments that actually scale. The documentation is thorough with separate guides for training, environments, vectorization, policies, and integration. It's built for people who've hit performance walls with standard RL libraries and need something faster without rewriting everything in C.