llama.cpp

Overview

Inference of Meta's LLaMA model (and others) in pure C/C++

Requirements

        The image for this container is several gigabytes. If you receive a "no space left on device" warning during installation, please increase the vDisk size in your Docker settings.
        This container expects a "model.gguf" file to be present in the model storage path.
        If you are using an Nvidia GPU, add "--gpus all" to the Extra Parameters field under Advanced.

Runtime arguments

Web UI: http://[IP]:[PORT:8000]/
Network: bridge
Privileged: false

Template configuration

WebUIPorttcp

Container Port: 8000

Target: 8000
Default: 8000
Value: 8000

Model Storage PathPathrw

Storage for model

Target: /models
Default: /mnt/user/appdata/llama_cpp/model
Value: /mnt/user/appdata/llama_cpp/model

Links

Template Support GHCR Project

Details

Repository

ghcr.io/ggml-org/llama.cpp:full

Registry

https://registry.hub.docker.com/r/ghcr.io/ggml-org/llama.cpp

Last Updated2026-06-01

First Seen2026-02-24

Run llama.cpp on Unraid.

llama.cpp is listed in Community Apps for Unraid OS. Explore Unraid to build a flexible home server, NAS, or homelab.

Explore Unraid OS