Intro: llama.cpp running Gemma 4 on my own PC

Day 7 of 30 and what llama.cpp is

What you need, and no installer

Step 1: Find the right download

Step 2: The second zip, the CUDA DLLs

Step 3: Check your graphics card with --list-devices

Step 4: First model from Hugging Face with -hf

First chat with llama-cli

Step 5: Reuse your LM Studio models

Step 6: llama-server and the port gotcha

Step 7: The built-in web UI

Ask about a screenshot

Step 8: The OpenAI-compatible API with curl.exe

Step 9: Read the real speed

Step 10: No graphics card, the CPU build

Three quick tips

ChatGPT Plus vs llama.cpp

Next: Codex CLI

Why run LLMs locally?

The problem llama.cpp solves

Repository and project overview

Homebrew installation

Faster builds with parallel jobs

Hardware backends

GGUF models

Downloading models

Quantization

Importance-matrix quantization

Using llama-cli

Performance factors

Multi-GPU inference

Health checks

Chat completions

Using the OpenAI Python client

Chat templates

Structured JSON output

Tool calling

Vision model requirements

Embeddings and reranking

Limitations and compatibility

Licensing

Diagnosing slow inference

Final checklist and next steps

llama.cpp in the FlyEnv Plugin Market

Manage llama.cpp runtimes side by side

Local GGUF models

The GGUF library: typed files and sizes

Per-model settings

Draft-safe settings and starting the service

llama.cpp running locally with the selected model

Locate llama.cpp GitHub repository

Clone repository using Git

Check for Visual Studio Build Tools

Install Visual Studio 2019 via winget

Open Visual Studio installer and enable “Desktop development with C++”

Install CMake using winget (winget install Kitware.CMake)

Install and Explain vcpkg and why it’s needed

Install Zlib, Curl, and OpenSSL with vcpkg

Vital CUDA Prepration & File Movement

Open x64 Native Tools Command Prompt

Create and enter build directory (mkdir build && cd build)

Run CMake configuration command (sets up build system)

Run CMake build command to compile all binaries

Verify successful build (check /bin/Release folder)

Overview of built executables (CLI, GGUF, TTS, etc.)

Run llama-simple-chat example (llama-simple-chat.exe -m model.gguf)

Demo with pirate “valley girl” prompt

Recap of all steps: Visual Studio, CMake, vcpkg, Zlib/Curl/OpenSSL, CUDA fix, build

Outro and membership plug

Installing llama.cpp

Installing LLM

Testing server within installation directory

Editing .bashrc file

Testing server outside of installation directory.

Why run llms locally?

What is llama.cpp?

llama.cpp vs ollama vs vllm vs lmstudio

Tour of the llama.cpp repo

How to build / install llama.cpp

How to run llms locally with llama.cpp

How to benchmark llms

Structured outputs with grammars and json-schema

Memory mapping (no-mmap, mlock)

How to create a gguf model with llama.cpp

How to quantize an llm

How to use a lora adapter

How to merge lora with base model

How to use llama.cpp bindings to build applications

Outro

Intro to Llama CPP

What is Llama CPP?

Getting Started with Llama CPP

Llama CPP Extensibility & Community

GGML & Whisper CPP Overview

The Llama CPP Manifesto

Compiling Llama CPP for Your Machine

Package Binaries vs Compiling

Llama CPP Build Files

Running a "Hello World" Test

Using Configuration Files

Testing Llama CPP Server

Quantization Providers

Running Whisper CPP

Introducción a llama.cpp

Cómo descargar llama.cpp desde GitHub

Configuración de CUDA, Vulkan y CPU en Windows

Uso de llama-server y API compatible con OpenAI

Cómo buscar y descargar modelos GGUF en Hugging Face

Configuración de modelos con visión multimodal

Ejecución de modelos mediante comandos en CMD

Prueba de la interfaz de chat local

Automatización de la carga con archivos .BAT

Ajustes avanzados de rendimiento y contexto

Optimización con KV cache, MTP y speculative decoding

Conexión con herramientas externas y API de OpenAI

Ejecución de modelos grandes combinando GPU, CPU y RAM

Conclusiones y próximos tutoriales

Download and Install Microsoft Visual Studio C + +

Install Cuda Toolkit

Download Git

Create a Workspace Folder

Download the Model