日本語のみで絞り込む
LLM inference in C/C++. Contribute to TheTom/llama-cpp-turboquant development by creating an account on GitHub.
This repository is the research home: the Python reference implementation, the validation papers, and the benchmark data. To run TurboQuant in an inference ...
Working on LLM inference systems, KV cache compression, and kernel-level optimizations (TurboQuant). 606 followers · 3 following.
2026/3/30 -the community implementation is at https://github.com/TheTom/llama-cpp-turboquant — it's a llama.cpp fork you build from source. not merged ...
Get up and running with TurboQuant+ KV cache compression in under 5 minutes. Prerequisites. A GGUF model (download from HuggingFace); CMake 3.14+; One of ...
A from-scratch PyTorch implementation of TurboQuant (ICLR 2026), Google's vector quantization algorithm for compressing LLM key-value caches.
Turboquant's main selling point (at least, as far as I'm aware) is giving the ability to fit in a bigger context window (kv cache) onto a smaller memory space, ...
2026/3/29 -TurboQuant works on any existing model. Llama, Mistral, Gemma, Qwen. No calibration data required. No fine-tuning. No vendor-specific adaptation ...
I built TurboQuant+ (https://github.com/TheTom/llama-cpp-turboquant), the llama.cpp implementation of this paper with extensions: asymmetric K/V compression ...