Vaex

If you're hitting memory limits with pandas on multi-gigabyte datasets, this is the out-of-core alternative that actually delivers. It uses lazy evaluation and memory mapping to let you run aggregations and filters on billions of rows at laptop scale, with a familiar DataFrame API. The workflow is straightforward: load your massive CSV or Parquet file, create virtual columns for feature engineering without copying data, run your groupbys and stats, then export to HDF5 or Arrow for instant future access. Real win is interactive exploration of datasets that would normally require a cluster. Just know you're trading pandas' ecosystem breadth for raw performance on big tabular data.