Strata
Docker app from MadeOfCard's Repository
Strata runs large mixture-of-experts models on an NVIDIA GPU plus system RAM: Qwen3.8-Flash-Next and its variants, the smaller Qwen3.6-35B-A3B, and DeepSeek-V4.1-Flash.
Requirements
Nvidia Driver plugin on driver 580 or newer (575+ with the cuda12 tag). DeepSeek needs the cuda13 tag and an RTX 30 card or newer. Do not set a memory limit on the container: Strata plans from the server's whole RAM. Do not pin CPUs: the engine sets its own thread affinity.
Overview
Strata runs large mixture-of-experts models on an NVIDIA GPU plus system RAM: Qwen3.8-Flash-Next and its variants, the smaller Qwen3.6-35B-A3B, and DeepSeek-V4.1-Flash. It serves an OpenAI- and Anthropic-compatible API, a chat page and a live monitor. This template runs Strata's official image.
Requirements: the Nvidia Driver plugin on driver 580 or newer and an RTX 20/30/40/50 card. For a GTX 10, P100 or V100, or a driver from 575 to 579, change the Repository tag to latest-cuda12. You also need 32 GB of RAM or more (more RAM allows a larger, better model) and room for the model on an SSD pool.
Choosing a model: leave Model and Model family empty and Strata picks one for this server, and the log says which and why. To choose yourself, set both. The model names are compression levels: the number is roughly the bits kept per weight, so a smaller number means a smaller download and less RAM, at some cost in quality. RAM below is the server's total RAM.
Qwen3.8-Flash-Next, the main model, for chat, code and tools (Model family qwen):
- Q2_0: the fastest, good quality. 66 GB download, 48 GB RAM.
- IQ2_XS: a little better, nearly as fast. 68 GB, 48 GB RAM.
- IQ3_XXS: great quality, slower. 76 GB, 60 GB RAM.
- IQ3_S: the best quality, matches the full model; the slowest. 84 GB, 64 GB RAM.
Other versions of Qwen3.8 (set the family shown):
- swift, with the same four sizes as qwen: a fine-tune by UkisAI that thinks for less time before it answers.
- coder, Model IQ1_M: for code only, with half the model removed. 58 GB, 32 GB RAM.
- gyro, Model Gyro-S: the least memory (about 28 GB of RAM and graphics memory together), lower quality. 59 GB. Gyro-M: about IQ3_XXS quality in 35 GB, 16-21% slower than Q2_0. 92 GB.
- unsloth, more bits for servers with a lot of RAM (the RAM figure keeps the whole model in memory): UD-IQ4_XS 94 GB, 72 GB RAM; UD-Q4_K_XL 111 GB, 88 GB RAM; UD-Q5_K_XL 158 GB, 108 GB RAM; UD-Q6_K_XL 169 GB, 118 GB RAM; Q8_0 188 GB, 136 GB RAM (the reference quality).
- nvfp4: NVIDIA's 4-bit format, fastest on RTX 50 cards. NVFP4 (the best of the three), NVFP4-NVIDIA or NVFP4-QAD. 102-128 GB, 80 GB RAM.
Qwen3.6-35B-A3B, a smaller model for servers with less RAM (Model family qwen36):
- UD-Q4_K_XL: 23 GB download, 32 GB RAM.
- UD-Q6_K_XL: closer to the full model. 33 GB, 44 GB RAM.
- Q8_0: the closest to the full model. 38 GB, 52 GB RAM.
DeepSeek-V4.1-Flash, the largest model (Model family deepseek41). Each download includes a 111 GB lookup table and an 8 GB helper model:
- ist-das-2.25bpw: the best quality, the best for code and agents. 274 GB.
- ist-das-2.00bpw: the middle choice. 257 GB.
- ist-das-1.75bpw: the smallest, noticeably weaker at writing prose. 239 GB.
- ista-das-coder: for code only, with half the model removed. 191 GB.
DeepSeek needs the latest-cuda13 tag, an RTX 30/40/50 card with 12 GB or more, 46 GB of RAM or more (64 GB is the tested size) and an NVMe pool. Expect about 5-10 tokens per second on a 12 GB card. Its own engine is downloaded into the Data folder on the first start. It is checked at 8k context and offers up to 1m.
First start: downloads the model into the Data folder and prepares it, which takes a while; watch the log. A stopped download resumes where it left off. Later starts go straight to serving.
API key: always on. Leave it empty and the first start makes one and keeps it in the Data folder; show it with: docker exec Strata strata launch --show-key --preset docker. Clients send it as Authorization: Bearer KEY.
Changing settings: edit them and Apply. The new values replace the saved ones on the next start; the model files and tuning stay in the Data folder.
Template and Unraid problems: the Support link. Strata itself: the Project link.
Readme
View on GitHubStrata for Unraid
An Unraid Community Applications template for Strata, which runs large mixture-of-experts models on an NVIDIA GPU plus system RAM: Qwen3.8-Flash-Next and its variants, the smaller Qwen3.6-35B-A3B, and DeepSeek-V4.1-Flash. It serves an OpenAI- and Anthropic-compatible API, a chat page and a live monitor.
The template runs Strata's official image, ghcr.io/niko1221/strata:latest-cuda13. Report template or Unraid problems in this repository's issues, and problems with Strata itself to the Strata project.
Install
Search for Strata in the Apps tab. Before installing you need:
- the Nvidia Driver plugin on driver 580 or newer, and an RTX 20/30/40/50 card. For a GTX 10, P100 or V100, or a driver from 575 to 579, change the Repository tag to
latest-cuda12. - 32 GB of RAM or more; more RAM allows a larger, better model (Strata's model guide)
- room for the model on an SSD pool, mapped as a direct pool path (
/mnt/cache/...), not/mnt/user
Choosing a model
Leave Model and Model family empty and Strata picks one for your server; the log says which and why. To choose yourself, set both.
The model names are compression levels. The number is roughly the bits kept per weight: a smaller number means a smaller download and less RAM, at some cost in quality. RAM in the tables is the server's total RAM.
Qwen3.8-Flash-Next, the main model, for chat, code and tools. Model family qwen:
| Model | What it is | Download | RAM |
|---|---|---|---|
Q2_0 |
the fastest, good quality | 66 GB | 48 GB |
IQ2_XS |
a little better, nearly as fast | 68 GB | 48 GB |
IQ3_XXS |
great quality, slower | 76 GB | 60 GB |
IQ3_S |
the best quality, matches the full model; the slowest | 84 GB | 64 GB |
Other versions of Qwen3.8:
| Model family | Model | What it is | Download | RAM |
|---|---|---|---|---|
swift |
the four qwen sizes |
UkisAI's fine-tune that thinks for less time before it answers | as qwen |
as qwen |
coder |
IQ1_M |
for code only, with half the model removed | 58 GB | 32 GB |
gyro |
Gyro-S |
the least memory, lower quality | 59 GB | about 28 GB of RAM and graphics memory together |
gyro |
Gyro-M |
about IQ3_XXS quality in less memory, 16–21% slower than Q2_0 |
92 GB | about 35 GB together |
unsloth |
UD-IQ4_XS |
4-bit, better than IQ3_S |
94 GB | 72 GB |
unsloth |
UD-Q4_K_XL |
4-bit | 111 GB | 88 GB |
unsloth |
UD-Q5_K_XL |
5-bit, close to Q8_0 |
158 GB | 108 GB |
unsloth |
UD-Q6_K_XL |
6-bit | 169 GB | 118 GB |
unsloth |
Q8_0 |
8-bit, the reference quality | 188 GB | 136 GB |
nvfp4 |
NVFP4, NVFP4-NVIDIA, NVFP4-QAD |
NVIDIA's 4-bit format, fastest on RTX 50 cards; NVFP4 has the best quality of the three |
102–128 GB | 80 GB |
The unsloth RAM figures keep the whole model in memory. With less RAM they still run, reading part of the model from the SSD, which is much slower.
Qwen3.6-35B-A3B, a smaller model for servers with less RAM. Model family qwen36:
| Model | What it is | Download | RAM |
|---|---|---|---|
UD-Q4_K_XL |
4-bit, the smallest | 23 GB | 32 GB |
UD-Q6_K_XL |
6-bit, closer to the full model | 33 GB | 44 GB |
Q8_0 |
8-bit, the closest to the full model | 38 GB | 52 GB |
DeepSeek-V4.1-Flash, the largest model. Model family deepseek41. Each download includes a 111 GB lookup table and an 8 GB helper model:
| Model | What it is | Download |
|---|---|---|
ist-das-2.25bpw |
the best quality, the best for code and agents | 274 GB |
ist-das-2.00bpw |
the middle choice | 257 GB |
ist-das-1.75bpw |
the smallest, noticeably weaker at writing prose | 239 GB |
ista-das-coder |
for code only, with half the model removed | 191 GB |
DeepSeek needs:
- the
latest-cuda13tag (DeepSeek's engine has no CUDA 12 build) and an RTX 30/40/50 card with 12 GB or more - 46 GB of RAM or more; 64 GB is the size Strata tested
- an NVMe pool for the Data folder: the model reads its lookup table from the disk as it writes
Expect about 5–10 tokens per second on a 12 GB card. DeepSeek uses its own engine, which the first start downloads into the Data folder. It is checked at 8k context and offers up to 1m. Strata's model guide and DeepSeek guide have the measurements.
First start
The first start downloads the model and prepares it. Later starts go straight to serving. The template's Overview explains each setting, and Strata's Docker guide covers them in full.
An API key is always on. If you leave it empty, the first start makes one; show it with:
docker exec Strata strata launch --show-key --preset docker
Updates
Strata is in rapid development and changes often, so expect this template to change along with it: settings get added, renamed or removed as new releases come out. Installed apps keep their saved copy of the template, so a new setting only appears after you reinstall or add it by hand.
The template follows latest-cuda13, which Strata's maintainer moves to each new release when its image is published. Unraid shows the update on the Docker tab. To stay on one release, change the tag to a version such as 0.2.3-cuda13.
Credits and licence
Strata and its image are by Niko1221 and the Strata contributors, under the MIT licence. The icon is Strata's own. This repository is MIT licensed; see LICENSE.
Categories
Related apps
Explore more like this
Explore allLinks
Details
ghcr.io/niko1221/strata:latest-cuda13Runtime arguments
- Web UI
http://[IP]:[PORT:8080]/- Network
bridge- Shell
bash- Privileged
- false
- Extra Params
--runtime=nvidia --ulimit memlock=-1:-1 --shm-size=2g --stop-timeout 120
Template configuration
Chat page, monitor and the API (/v1 for OpenAI clients, the root URL for Anthropic clients).
- Target
- 8080
- Default
- 8080
- Value
- 8080
The model, the prepared pack, presets, tuning and the API key: tens of GB for Qwen, 191 to 274 GB for DeepSeek; the log names the size. Use a direct pool path on an NVMe drive (/mnt/cache/... or /mnt/POOLNAME/...), not /mnt/user: the engine reads the model files while it runs.
- Target
- /data
- Default
- /mnt/cache/appdata/strata
- Value
- /mnt/cache/appdata/strata
The cards the container sees: a UUID (GPU-xxxxxxxx-...) from the Nvidia Driver plugin page or nvidia-smi -L, several separated by commas, or all. With exactly two RTX 30 or two RTX 50 class cards, a supported Qwen3.8 file and a context up to 128k, Strata uses its all-cards split engine by itself.
- Target
- NVIDIA_VISIBLE_DEVICES
- Default
- all
- Value
- all
Empty: Strata picks for this server, and the log says which and why. Otherwise set Model family too. A smaller number in the name means a smaller download and less RAM, at some cost in quality. Qwen3.8 (qwen): Q2_0 is the fastest (48 GB RAM), then IQ2_XS, IQ3_XXS, and IQ3_S the best (64 GB RAM). Qwen3.6 (qwen36), a smaller model: UD-Q4_K_XL for 32 GB of RAM, UD-Q6_K_XL, Q8_0. DeepSeek (deepseek41), the largest: ist-das-2.25bpw the best, ist-das-2.00bpw, ist-das-1.75bpw the smallest, ista-das-coder for code. The Overview lists every option with its download size and RAM.
- Target
- MODEL
Which model, or which version of it. Empty: Strata's pick. qwen: Qwen3.8-Flash-Next, the main model. swift: Qwen3.8 tuned to think for less time. coder: Qwen3.8 for code only, in less RAM. gyro: Qwen3.8 in the least memory, slower. unsloth: Qwen3.8 at 4 to 8 bits, for servers with a lot of RAM. nvfp4: Qwen3.8 in NVIDIA's 4-bit format, fastest on RTX 50. qwen36: Qwen3.6-35B-A3B, a smaller model for 32-52 GB of RAM. deepseek41: DeepSeek-V4.1-Flash, the largest (cuda13 tag, 191-274 GB of disk).
- Target
- FAMILY
How much text it remembers at once. Longer costs memory. Qwen3.8 goes up to 256k. DeepSeek is checked at 8k and offers up to 1m, but nothing past 256k has been tested and a very long prompt is read slowly.
- Target
- CONTEXT
- Default
- 8k|32k|64k|128k|256k|512k|1m
- Value
- 32k
on: the model reads pictures (about 0.9 GB more to download). gpu or cpu choose where the picture reader runs. DeepSeek reads pictures with its own engine; off turns that off.
- Target
- VISION
- Default
- off|on|gpu|cpu
- Value
- off
16 characters or more (openssl rand -hex 24 makes one). Empty: the first start makes one and keeps it in the Data folder.
- Target
- API_KEY
Measures the best engine settings for this server, once, on the next start, and keeps them. thorough takes longer than quick; how long depends on the hardware.
- Target
- TUNE
- Default
- skip|quick|thorough
- Value
- skip
1 tunes again on the next start even if this preset is already tuned, for example after an update. Set it back to 0 afterwards.
- Target
- RETUNE
- Default
- 0|1
- Value
- 0
1 lets the tuner also try settings that can change answers very slightly. 0 keeps only settings that leave answers the same.
- Target
- TUNE_ALLOW_SMALL
- Default
- 0|1
- Value
- 0
Requests answered at the same time, 1 to 8. Empty: fitted to the card. With 2 or more, Strata uses the MTP drafter, because its default DFlash2 drafter answers one request at a time. DeepSeek answers one at a time unless you set this.
- Target
- PARALLEL
Conversation memory format: int8 (default), q4_0 (half the memory), k8v4, fp16.
- Target
- KV
- Default
- int8|q4_0|k8v4|fp16
- Value
- int8
GGUF files you already have, as a path inside the container. Add a Path mapping for their folder (read-only is fine), e.g. /models, and enter that here; set Model too. Nothing is downloaded for the model itself.
- Target
- MODEL_PATH
When model files from the older image are still in /data/models, the start stops instead of downloading the model again (tens of GB) and the log says where they are. Use them with Existing model files, or set 1 to download anyway.
- Target
- ALLOW_DOWNLOAD
- Default
- 0|1
- Value
- 0
Several cards: a role for each, e.g. 1=main,0=helper (main, helper or off; cards you do not name are off). Empty: the engine's choice. See docs/MULTI_GPU.md in the Strata repository.
- Target
- GPU_ROLES
Advanced engine settings, space separated KEY=VALUE, like strata launch --set. For DeepSeek, dspark=on|off|auto sets its DSpark drafter (default auto). docs/SETTINGS.md in the Strata repository lists every setting.
- Target
- STRATA_SET