4日前 -Fork of llama.cpp, started in June 2024 and last synced in August 2024. It adds SOTA quantization types and often better performance.
3日前 -This is similar to the Apple Silicon benchmark thread, but for Vulkan! We'll be testing the Llama 2 7B model like the other thread to keep things consistent ...
2日前 -The main goal of llama.cpp is to enable LLM (and VLM) inference with minimal setup and state-of-the-art performance on a wide range of hardware - locally ...
2日前 -llama.cpp fork for significantly improved performance on Ampere generation Nvidia cards (especially RTX 3090 / 3090 Ti): example: 110+ tok/s over a ...
2日前 -... Llama.cpp repo as well. Step 1: Clone llama.cpp from GitHub. git clone https://github.com/ggerganov/llama.cpp. Step 2: Move into the llama.cpp folder and ...
6日前 -llama-halo-hybrid is designed to run larger (120B+) mixture-of-experts models on an AMD Strix Halo (and soon Gorgon Halo) with an added Radeon card as an ...
2日前 -By hand, without the installer. Tested with ROCm 7.2.4 on gfx1100 (RX 7900 XT / XTX). git clone https://github.com/freriadoc/Avalon ...
4日前 -Step 1: Clone llama.cpp from GitHub. git clone https://github.com/ggerganov/llama.cpp. Step 2: Move into the llama.
3日前 -LocalAI supports 60+ backends including llama.cpp, vLLM, SGLang, transformers, whisper.cpp, diffusers, MLX, MLX-VLM, and many more. Hardware acceleration is ...