日本語のみで絞り込む
Extreme weight + KV cache compression for LLMs on Apple Silicon (MLX implementation of Google's TurboQuant) - manjunathshiva/turboquant-mlx.
Reproduction of KV-Cache quantization from TurboQuant (Google, 2025) (Paper) on Apple Silicon using MLX. Result: Up to 5.5x KV-Cache compression.
TurboQuant introduces a specific bit-packing and dequantization logic that deviates from the current grouped quantization (Q4_0, Q4_1, etc.) implemented in MLX.
A standalone, pip-installable TurboQuant adapter for mlx-lm on Apple silicon, with a custom Metal kernel for non-uniform quantization.
The first complete MLX implementation of TurboQuant — model weights AND KV cache. Run Qwen2.5-32B on a 16 GB MacBook Air. License: MIT ...
TurboQuant MLX is an advanced implementation of near-optimal distortion-rate KV cache compression algorithms tailored specifically for the Apple MLX framework.
This repo packages the current best TurboQuant runtime recipe we have measured for mlx-community/Qwen3.5-9B-MLX-4bit : TurboQuant 3-bit KV-cache compression ( ...
2026/3/25 -The closed-source thing is a fair concern but the underlying TurboQuant method is well-documented in the Google paper -- anyone can reimplement ...
V-only TurboQuant cache. Works with stock mlx-lm (no fork needed). Keeps K in fp16, compresses V with PolarQuant 3-bit.
2026/3/29 -Tools for evaluation are already available. The mlx-optiq package [7] provides the MLX implementation on PyPI. The turboquant_plus repository [9] ...