Model Quantization

This handles the full pipeline for quantizing AI models down to 4-bit or 8-bit formats, specifically targeting llama.cpp GGUF conversion for running LLMs on consumer hardware. You get test-first benchmarking that tracks perplexity degradation, memory footprint verification, and quality thresholds before you ship a quantized model. The workflow is built around the reality that Q5_K_M is usually the sweet spot for a 7B model, getting you from 14GB down to under 5GB while keeping quality loss under 10%. Honest take: the structured approach to measuring tradeoffs is more valuable than the quantization itself, since llama.cpp does the heavy lifting anyway.