日本語のみで絞り込む
TurboQuant: Near-optimal KV cache quantization for LLM inference (3-bit keys, 2-bit values) with Triton kernels + vLLM integration - 0xSero/turboquant.
This is an independent open-source implementation of the TurboQuant algorithm. All credit for the algorithm design, theoretical analysis, and original research ...
LLM inference in C/C++. Contribute to TheTom/llama-cpp-turboquant development by creating an account on GitHub.
3日前 -TurboQuant Pro compresses each vector by the metric its downstream consumer actually uses — retrieval recall for indexes, attention/generation ...
TurboQuant is a Rust library for research-grade vector quantization of LLM KV caches. It now includes three benchmark/evaluation paths: synthetic : model-shaped ...
2026/3/25 -TurboQuant changes that calculus a bit because it targets a different bottleneck -- the hot path during inference rather than the cold storage ...
TurboQuant compresses LLM KV caches 4-7x at inference time using random rotation + optimal scalar quantization, with near-zero quality loss. No training, no ...
TurboQuant is a compression method that achieves a high reduction in model size with zero accuracy loss, making it ideal for supporting both key-value (KV) ...
This repository is the research home: the Python reference implementation, the validation papers, and the benchmark data. To run TurboQuant in an inference ...
TurboQuant is an online (training-free) vector quantization algorithm that achieves near-optimal distortion rates. Its key insight: applying a random orthogonal ...