• Lock MCG and MUL1 multipliers, no longer flag as experimental
• Switch to MCG codebook by default to new models (use --codebook 3inst for previous default)
• Add more calibration data
• Increase default calibration size to 250 rows (use --cal_rows 100 for previous default)
• Fix quantized cache for bsz > 1
• Fix kernel selection on A100
• A few more TP-related fixes

Full Changelog: v0.0.8...v0.0.9

• New GEMM kernel tuning scheme
• Fix banned strings regression
• Fix some memory leaks
• Fix potential stack overflow in cache defrag

Full Changelog: v0.0.7...v0.0.8

• Support SeedOssForCausalLM
• Support ApertusForCausalLM
• Support Qwen3NextForCausalLM¹
• Reduced CPU overhead
• Fix support for non-AVX2 CPUs
• Optimized GEMM kernels
• Faster quantization, especially on Blackwell
• Quant optimizer utils
• Much lower overhead from quantized cache
• Tensor split option for MoE layers with large experts
• Add recurrent model support to generator
• Generator now allows allocating pages on the fly
• Many more improvements and bugfixes

¹ Qwen3-Next currently requires Triton and Flash Linear Attention. causal-conv1d is recommended but not required. Triton-free implementation is in the works for v0.0.8.

Full Changelog: v0.0.6...v0.0.7

• Add tensor-parallel mode
• Add support for Arcee achitecture
• Add support for GLM4 achitecture (GLM4.5, GLM4.5-Air)
• Fix CPU bottleneck in model loader
• Reduce VRAM usage during quantization
• Fused MoE routing kernels
• Various bugfixes
• QoL improvements

Full Changelog: v0.0.5...v0.0.6

• Add filter interface
• Add Formatron support (JSON/KBNF grammar etc.)
• Support Ernie 4.5 architecture
• Support SmolLM3 architecture
• Support Exaone4 architecture
• Improve Cohere2 support (fixes support for Command-A)
• Fix compatibility with certain Pixtral preprocessors
• Fix excessive virtual memory usage with large generator queues on Windows
• Add facilities for manually mixing/optimizing quants
• Add MMLU eval script
• Various other fixes, improvements and optimizations

Full Changelog: v0.0.4...v0.0.5

• Vision support
• Support MiMoForCausalLM (no MTP)
• Support Gemma3ForConditionalGeneration
• Support Gemma3ForCausalLM
• Support Mistral3ForConditionalGeneration (Mistral 3.1)
• Support Dots1ForCausalLM (dots.llm1)
• Add Transformers integration
• Faster tokenizer initialization
• Various fixes

Full Changelog: v0.0.3...v0.0.4

• Support Mixtral architecture
• Support Gemma3 architecture (text only)
• Fixed support for Command-R+
• Return probs and top probs from generator
• Defrag for generator
• Many bugfixes
• Many performance improvements
• A bunch of new eval and test functionality

Full Changelog: v0.0.2...v0.0.3

• Adds Qwen3MoE support
• Various bugfixes and optimizations

Full Changelog: v0.0.1...v0.0.2