llama.cpp
Docker app from grtgbln's Repository
Inference of Meta's LLaMA model (and others) in pure C/C++
Requirements
The image for this container is several gigabytes. If you receive a "no space left on device" warning during installation, please increase the vDisk size in your Docker settings.
This container expects a "model.gguf" file to be present in the model storage path.
If you are using an Nvidia GPU, add "--gpus all" to the Extra Parameters field under Advanced.
Overview
Inference of Meta's LLaMA model (and others) in pure C/C++
Readme
View on GitHubllama.cpp
LLM inference in C/C++
License: MIT Release Nightly Server Docker Winget
v1.0.0 / ggml / ops / maintainer PRs / dev stats / lib llama API / llama-server REST API
Description
The main goal of llama.cpp is to enable LLM (and MLLM) inference with minimal setup and state-of-the-art performance on a wide range of hardware - locally and in the cloud.
The llama.cpp project is built on top of the ggml library.
Quick start
A few options to get llama.cpp installed on your machine:
# curl
curl -LsSf https://llama.app/install.sh | sh
# powershell
irm https://llama.app/install.ps1 | iex
- Visit https://llama.app and follow the instructions
- Run with Docker - see our Docker documentation
- Download pre-built binaries from the releases page
- Build from source by cloning this repository - check out our build guide
Once installed:
# Download and run a model directly from Hugging Face
llama cli -hf ggml-org/Qwen3.5-0.8B-GGUF
# Launch OpenAI-compatible API server
llama serve -hf ggml-org/Qwen3.5-0.8B-GGUF
|
|
|
Categories
Related apps
Explore more like this
Explore allLinks
Details
Repository
ghcr.io/ggml-org/llama.cpp:fullLast Updated2026-11-22
First Seen2026-02-24
Runtime arguments
- Web UI
http://[IP]:[PORT:8000]/- Network
bridge- Privileged
- false
Template configuration
WebUIPorttcp
Container Port: 8000
- Target
- 8000
- Default
- 8000
- Value
- 8000
Model Storage PathPathrw
Storage for model
- Target
- /models
- Default
- /mnt/user/appdata/llama_cpp/model
- Value
- /mnt/user/appdata/llama_cpp/model