Tensorrt Llm

This is NVIDIA's optimization library for squeezing maximum performance out of LLMs on their GPUs. If you're deploying on A100s or H100s and need serious throughput (the docs claim 24,000+ tokens per second on Llama 3), this is the toolkit. It handles quantization down to FP4 and scales across multiple GPUs, but you're trading setup simplicity for raw speed. The skill points out that vLLM might be better if you want something more straightforward or aren't locked into NVIDIA hardware. With 27.7K GitHub stars and 353 installs, it's clearly production-grade stuff, though you'll need to be comfortable with TensorRT compilation and GPU-specific optimization work.