Dask

When your pandas DataFrames or NumPy arrays stop fitting in RAM, this is the escape hatch. It scales familiar Python code from gigabytes on a laptop to terabytes on a cluster without rewriting everything. The lazy execution model means you build up operations and compute only when needed. Real talk: chunking strategy matters more than you'd think, and the distributed scheduler's dashboard is legitimately helpful for debugging. Best for parallelizing file processing or scaling existing analytics code. If you're still comfortably in memory, polars is faster. If you need true out-of-core on a single machine, consider vaex. This shines when you're already hitting memory limits with pandas and need a relatively smooth migration path.

Install to Claude Code

npx -y skills add k-dense-ai/scientific-agent-skills --skill dask --agent claude-code