llama.cpp

llama.cpp

Docker app from grtgbln's Repository

Inference of Meta's LLaMA model (and others) in pure C/C++

Requirements


        The image for this container is several gigabytes. If you receive a "no space left on device" warning during installation, please increase the vDisk size in your Docker settings.
        This container expects a "model.gguf" file to be present in the model storage path.
        If you are using an Nvidia GPU, add "--gpus all" to the Extra Parameters field under Advanced.
    
Overview
Inference of Meta's LLaMA model (and others) in pure C/C++

llama.cpp

llama.cpp

Description

The main goal of llama.cpp is to enable LLM (and MLLM) inference with minimal setup and state-of-the-art performance on a wide range of hardware - locally and in the cloud.

The llama.cpp project is built on top of the ggml library.

Quick start

A few options to get llama.cpp installed on your machine:

# curl
curl -LsSf https://llama.app/install.sh | sh

# powershell
irm https://llama.app/install.ps1 | iex

Once installed:

# Download and run a model directly from Hugging Face
llama cli -hf ggml-org/Qwen3.5-0.8B-GGUF

# Launch OpenAI-compatible API server
llama serve -hf ggml-org/Qwen3.5-0.8B-GGUF
VLM session with `llama cli` VLM session with llama cli Llama Chat, the built-in chat app, against `llama serve` running Qwen 3.6 Llama Chat, the built-in chat app, against llama serve

Documentation

Tools

Development

Contributing

  • Contributors can open PRs
  • Collaborators will be invited based on contributions
  • Maintainers can push to branches in the llama.cpp repo and merge PRs into the master branch
  • Any help with managing issues, PRs and projects is very appreciated!
  • Read the CONTRIBUTING.md for more information

Acknowledgements

  • yhirose/cpp-httplib - Single-header HTTP server, used by llama-server - MIT license
  • nothings/stb - Single-header image format decoder, used by multimodal subsystem - Public domain
  • nlohmann/json - Single-header JSON library, used by various tools/examples - MIT License
  • mackron/miniaudio - Single-header audio format decoder, used by multimodal subsystem - Public domain
  • sheredom/subprocess.h - Single-header process launching solution for C and C++ - Public domain

Categories

Related apps

Links

Details

Repository
ghcr.io/ggml-org/llama.cpp:full
Last Updated2026-11-22
First Seen2026-02-24

Runtime arguments

Web UI
http://[IP]:[PORT:8000]/
Network
bridge
Privileged
false

Template configuration

WebUIPorttcp

Container Port: 8000

Target
8000
Default
8000
Value
8000
Model Storage PathPathrw

Storage for model

Target
/models
Default
/mnt/user/appdata/llama_cpp/model
Value
/mnt/user/appdata/llama_cpp/model