Intro: llama.cpp running Gemma 4 on my own PC

Day 7 of 30 and what llama.cpp is

What you need, and no installer

Step 1: Find the right download

Step 2: The second zip, the CUDA DLLs

Step 3: Check your graphics card with --list-devices

Step 4: First model from Hugging Face with -hf

First chat with llama-cli

Step 5: Reuse your LM Studio models

Step 6: llama-server and the port gotcha

Step 7: The built-in web UI

Ask about a screenshot

Step 8: The OpenAI-compatible API with curl.exe

Step 9: Read the real speed

Step 10: No graphics card, the CPU build

Three quick tips

ChatGPT Plus vs llama.cpp

Next: Codex CLI

Intro to Llama CPP

What is Llama CPP?

Getting Started with Llama CPP

Llama CPP Extensibility & Community

GGML & Whisper CPP Overview

The Llama CPP Manifesto

Compiling Llama CPP for Your Machine

Package Binaries vs Compiling

Llama CPP Build Files

Running a "Hello World" Test

Using Configuration Files

Testing Llama CPP Server

Quantization Providers

Running Whisper CPP

Locate llama.cpp GitHub repository

Clone repository using Git

Check for Visual Studio Build Tools

Install Visual Studio 2019 via winget

Open Visual Studio installer and enable “Desktop development with C++”

Install CMake using winget (winget install Kitware.CMake)

Install and Explain vcpkg and why it’s needed

Install Zlib, Curl, and OpenSSL with vcpkg

Vital CUDA Prepration & File Movement

Open x64 Native Tools Command Prompt

Create and enter build directory (mkdir build && cd build)

Run CMake configuration command (sets up build system)

Run CMake build command to compile all binaries

Verify successful build (check /bin/Release folder)

Overview of built executables (CLI, GGUF, TTS, etc.)

Run llama-simple-chat example (llama-simple-chat.exe -m model.gguf)

Demo with pirate “valley girl” prompt

Recap of all steps: Visual Studio, CMake, vcpkg, Zlib/Curl/OpenSSL, CUDA fix, build

Outro and membership plug

Installing llama.cpp

Installing LLM

Testing server within installation directory

Editing .bashrc file

Testing server outside of installation directory.

Why run LLMs locally?

The problem llama.cpp solves

Repository and project overview

Homebrew installation

Faster builds with parallel jobs

Hardware backends

GGUF models

Downloading models

Quantization

Importance-matrix quantization

Using llama-cli

Performance factors

Multi-GPU inference

Health checks

Chat completions

Using the OpenAI Python client

Chat templates

Structured JSON output

Tool calling

Vision model requirements

Embeddings and reranking

Limitations and compatibility

Licensing

Diagnosing slow inference

Final checklist and next steps

Why run llms locally?

What is llama.cpp?

llama.cpp vs ollama vs vllm vs lmstudio

Tour of the llama.cpp repo

How to build / install llama.cpp

How to run llms locally with llama.cpp

How to benchmark llms

Structured outputs with grammars and json-schema

Memory mapping (no-mmap, mlock)

How to create a gguf model with llama.cpp

How to quantize an llm

How to use a lora adapter

How to merge lora with base model

How to use llama.cpp bindings to build applications

Outro

Local AI on Windows: the result first

Why Llama Windows matters

The real repo and docs

Install in one click

Models panel and downloads

Chat in the web UI (Gemma 4 reasoning)

Tray panel, Alt+Space, deep links

It's an OpenAI-compatible server

CLI + vision, tools, JSON, code tests

Real numbers: tokens/s, prefill vs decode

Honest limits on 8GB

Wrap up

Motivation

Install Git

Download the Remote Repository of Llama Cpp

Verify Where Are the Executables

Execute the Model

Download and Install Microsoft Visual Studio C + +

Install Cuda Toolkit

Download Git

Create a Workspace Folder

Download the Model

Install cmake

Clone llama.cpp repo

Build llama.cp (DDGML_METAL)

Install Huggingface CLI

Download a Model from Huggingface

Configure Our Model and Run llama-server

Explore llama-server Web U

Introducción a llama.cpp

Cómo descargar llama.cpp desde GitHub

Configuración de CUDA, Vulkan y CPU en Windows

Uso de llama-server y API compatible con OpenAI

Cómo buscar y descargar modelos GGUF en Hugging Face

Configuración de modelos con visión multimodal

Ejecución de modelos mediante comandos en CMD

Prueba de la interfaz de chat local

Automatización de la carga con archivos .BAT

Ajustes avanzados de rendimiento y contexto

Optimización con KV cache, MTP y speculative decoding

Conexión con herramientas externas y API de OpenAI

Ejecución de modelos grandes combinando GPU, CPU y RAM

Conclusiones y próximos tutoriales

Download Pre-Built Binaries from Releases

Cmake Download

Add Cmake to the Path Environment Variable

Installing Visual Studio

Installing Visual Studio 2019

Basic Usage

Introduction to LLM Deployment Series

Overview of LLAMA CPP

Installing LLAMA CPP

Setting Up the LLAMA CPP Server

Making Requests to the Server

Practical Examples and Demonstrations

Advanced Server Options

Using OpenAI Client with LLAMA CPP

Concurrent Requests with Python

Conclusion and Next Steps