Pufferlib

If you're doing reinforcement learning and hitting performance walls, this is worth checking out. PufferLib gets you 2-10x speedups over standard implementations through optimized vectorization and parallel environment simulation, hitting millions of steps per second. It's built around an efficient PPO implementation and works with all the usual frameworks (Gymnasium, PettingZoo, Atari, Procgen). The multi-agent support is native rather than bolted on, which matters if you're working with those environments. Fair warning: the learning curve is steeper than stable-baselines3, so this makes sense when you actually need the speed or are scaling up experiments. The documentation covers training loops, custom environments, and distributed setups across the reference files.