OpenChat-Cuda
OpenChat-Cuda
Docker app from el_pino's Repository
Overview
A self-hosted, offline, ChatGPT-like chatbot with open source LLM support. 100% private, with no data leaving your device.
Please note that this version requires an NVIDIA GPU with the Unraid NVIDIA-DRIVER plugin.
Runtime arguments
- Web UI
http://[IP]:[PORT:3000]/- Network
bridge- Shell
sh- Privileged
- false
- Extra Params
--gpus all
Template configuration
Local Model PathVariable
The local model path
- Target
- MODEL
- Default
- /models/llama-2-7b-chat.bin
Model Download URLVariable
GGML Model Binary.
- Target
- MODEL_DOWNLOAD_URL
- Default
- https://huggingface.co/TheBloke/Llama-2-7B-Chat-GGML/resolve/main/llama-2-7b-chat.ggmlv3.q4_0.bin
Model DirectoryPathrw
The local model directory to use as a cache
- Target
- /models
- Default
- /mnt/user/appdata/models
Web UIPorttcp
Chat UI Port
- Target
- 3000
- Default
- 3000
API PortPorttcp
HTTP API Port
- Target
- 8000
- Default
- 8000
Number Of GPU LayersVariable
Layers to offload to GPU. Update this number if server fails to load.
- Target
- N_GPU_LAYERS
- Value
- 12
Details
Repository
ghcr.io/edgar971/open-chat-cuda:v1.0.6Last Updated2026-06-01
First Seen2023-09-06
Run OpenChat-Cuda on Unraid.
OpenChat-Cuda is listed in Community Apps for Unraid OS. Explore Unraid to build a flexible home server, NAS, or homelab.