Strata

Strata

Docker app from MadeOfCard's Repository

Strata runs large mixture-of-experts models on an NVIDIA GPU plus system RAM: Qwen3.8-Flash-Next and its variants, the smaller Qwen3.6-35B-A3B, and DeepSeek-V4.1-Flash.

Requirements

Nvidia Driver plugin on driver 580 or newer (575+ with the cuda12 tag). DeepSeek needs the cuda13 tag and an RTX 30 card or newer. Do not set a memory limit on the container: Strata plans from the server's whole RAM. Do not pin CPUs: the engine sets its own thread affinity.

Overview

Strata runs large mixture-of-experts models on an NVIDIA GPU plus system RAM: Qwen3.8-Flash-Next and its variants, the smaller Qwen3.6-35B-A3B, and DeepSeek-V4.1-Flash. It serves an OpenAI- and Anthropic-compatible API, a chat page and a live monitor. This template runs Strata's official image.

Requirements: the Nvidia Driver plugin on driver 580 or newer and an RTX 20/30/40/50 card. For a GTX 10, P100 or V100, or a driver from 575 to 579, change the Repository tag to latest-cuda12. You also need 32 GB of RAM or more (more RAM allows a larger, better model) and room for the model on an SSD pool.

Choosing a model: leave Model and Model family empty and Strata picks one for this server, and the log says which and why. To choose yourself, set both. The model names are compression levels: the number is roughly the bits kept per weight, so a smaller number means a smaller download and less RAM, at some cost in quality. RAM below is the server's total RAM.

Qwen3.8-Flash-Next, the main model, for chat, code and tools (Model family qwen):

  • Q2_0: the fastest, good quality. 66 GB download, 48 GB RAM.
  • IQ2_XS: a little better, nearly as fast. 68 GB, 48 GB RAM.
  • IQ3_XXS: great quality, slower. 76 GB, 60 GB RAM.
  • IQ3_S: the best quality, matches the full model; the slowest. 84 GB, 64 GB RAM.

Other versions of Qwen3.8 (set the family shown):

  • swift, with the same four sizes as qwen: a fine-tune by UkisAI that thinks for less time before it answers.
  • coder, Model IQ1_M: for code only, with half the model removed. 58 GB, 32 GB RAM.
  • gyro, Model Gyro-S: the least memory (about 28 GB of RAM and graphics memory together), lower quality. 59 GB. Gyro-M: about IQ3_XXS quality in 35 GB, 16-21% slower than Q2_0. 92 GB.
  • unsloth, more bits for servers with a lot of RAM (the RAM figure keeps the whole model in memory): UD-IQ4_XS 94 GB, 72 GB RAM; UD-Q4_K_XL 111 GB, 88 GB RAM; UD-Q5_K_XL 158 GB, 108 GB RAM; UD-Q6_K_XL 169 GB, 118 GB RAM; Q8_0 188 GB, 136 GB RAM (the reference quality).
  • nvfp4: NVIDIA's 4-bit format, fastest on RTX 50 cards. NVFP4 (the best of the three), NVFP4-NVIDIA or NVFP4-QAD. 102-128 GB, 80 GB RAM.

Qwen3.6-35B-A3B, a smaller model for servers with less RAM (Model family qwen36):

  • UD-Q4_K_XL: 23 GB download, 32 GB RAM.
  • UD-Q6_K_XL: closer to the full model. 33 GB, 44 GB RAM.
  • Q8_0: the closest to the full model. 38 GB, 52 GB RAM.

DeepSeek-V4.1-Flash, the largest model (Model family deepseek41). Each download includes a 111 GB lookup table and an 8 GB helper model:

  • ist-das-2.25bpw: the best quality, the best for code and agents. 274 GB.
  • ist-das-2.00bpw: the middle choice. 257 GB.
  • ist-das-1.75bpw: the smallest, noticeably weaker at writing prose. 239 GB.
  • ista-das-coder: for code only, with half the model removed. 191 GB.
    DeepSeek needs the latest-cuda13 tag, an RTX 30/40/50 card with 12 GB or more, 46 GB of RAM or more (64 GB is the tested size) and an NVMe pool. Expect about 5-10 tokens per second on a 12 GB card. Its own engine is downloaded into the Data folder on the first start. It is checked at 8k context and offers up to 1m.

First start: downloads the model into the Data folder and prepares it, which takes a while; watch the log. A stopped download resumes where it left off. Later starts go straight to serving.

API key: always on. Leave it empty and the first start makes one and keeps it in the Data folder; show it with: docker exec Strata strata launch --show-key --preset docker. Clients send it as Authorization: Bearer KEY.

Changing settings: edit them and Apply. The new values replace the saved ones on the next start; the model files and tuning stay in the Data folder.

Template and Unraid problems: the Support link. Strata itself: the Project link.

Strata for Unraid

An Unraid Community Applications template for Strata, which runs large mixture-of-experts models on an NVIDIA GPU plus system RAM: Qwen3.8-Flash-Next and its variants, the smaller Qwen3.6-35B-A3B, and DeepSeek-V4.1-Flash. It serves an OpenAI- and Anthropic-compatible API, a chat page and a live monitor.

The template runs Strata's official image, ghcr.io/niko1221/strata:latest-cuda13. Report template or Unraid problems in this repository's issues, and problems with Strata itself to the Strata project.

Install

Search for Strata in the Apps tab. Before installing you need:

  • the Nvidia Driver plugin on driver 580 or newer, and an RTX 20/30/40/50 card. For a GTX 10, P100 or V100, or a driver from 575 to 579, change the Repository tag to latest-cuda12.
  • 32 GB of RAM or more; more RAM allows a larger, better model (Strata's model guide)
  • room for the model on an SSD pool, mapped as a direct pool path (/mnt/cache/...), not /mnt/user

Choosing a model

Leave Model and Model family empty and Strata picks one for your server; the log says which and why. To choose yourself, set both.

The model names are compression levels. The number is roughly the bits kept per weight: a smaller number means a smaller download and less RAM, at some cost in quality. RAM in the tables is the server's total RAM.

Qwen3.8-Flash-Next, the main model, for chat, code and tools. Model family qwen:

Model What it is Download RAM
Q2_0 the fastest, good quality 66 GB 48 GB
IQ2_XS a little better, nearly as fast 68 GB 48 GB
IQ3_XXS great quality, slower 76 GB 60 GB
IQ3_S the best quality, matches the full model; the slowest 84 GB 64 GB

Other versions of Qwen3.8:

Model family Model What it is Download RAM
swift the four qwen sizes UkisAI's fine-tune that thinks for less time before it answers as qwen as qwen
coder IQ1_M for code only, with half the model removed 58 GB 32 GB
gyro Gyro-S the least memory, lower quality 59 GB about 28 GB of RAM and graphics memory together
gyro Gyro-M about IQ3_XXS quality in less memory, 16–21% slower than Q2_0 92 GB about 35 GB together
unsloth UD-IQ4_XS 4-bit, better than IQ3_S 94 GB 72 GB
unsloth UD-Q4_K_XL 4-bit 111 GB 88 GB
unsloth UD-Q5_K_XL 5-bit, close to Q8_0 158 GB 108 GB
unsloth UD-Q6_K_XL 6-bit 169 GB 118 GB
unsloth Q8_0 8-bit, the reference quality 188 GB 136 GB
nvfp4 NVFP4, NVFP4-NVIDIA, NVFP4-QAD NVIDIA's 4-bit format, fastest on RTX 50 cards; NVFP4 has the best quality of the three 102–128 GB 80 GB

The unsloth RAM figures keep the whole model in memory. With less RAM they still run, reading part of the model from the SSD, which is much slower.

Qwen3.6-35B-A3B, a smaller model for servers with less RAM. Model family qwen36:

Model What it is Download RAM
UD-Q4_K_XL 4-bit, the smallest 23 GB 32 GB
UD-Q6_K_XL 6-bit, closer to the full model 33 GB 44 GB
Q8_0 8-bit, the closest to the full model 38 GB 52 GB

DeepSeek-V4.1-Flash, the largest model. Model family deepseek41. Each download includes a 111 GB lookup table and an 8 GB helper model:

Model What it is Download
ist-das-2.25bpw the best quality, the best for code and agents 274 GB
ist-das-2.00bpw the middle choice 257 GB
ist-das-1.75bpw the smallest, noticeably weaker at writing prose 239 GB
ista-das-coder for code only, with half the model removed 191 GB

DeepSeek needs:

  • the latest-cuda13 tag (DeepSeek's engine has no CUDA 12 build) and an RTX 30/40/50 card with 12 GB or more
  • 46 GB of RAM or more; 64 GB is the size Strata tested
  • an NVMe pool for the Data folder: the model reads its lookup table from the disk as it writes

Expect about 5–10 tokens per second on a 12 GB card. DeepSeek uses its own engine, which the first start downloads into the Data folder. It is checked at 8k context and offers up to 1m. Strata's model guide and DeepSeek guide have the measurements.

First start

The first start downloads the model and prepares it. Later starts go straight to serving. The template's Overview explains each setting, and Strata's Docker guide covers them in full.

An API key is always on. If you leave it empty, the first start makes one; show it with:

docker exec Strata strata launch --show-key --preset docker

Updates

Strata is in rapid development and changes often, so expect this template to change along with it: settings get added, renamed or removed as new releases come out. Installed apps keep their saved copy of the template, so a new setting only appears after you reinstall or add it by hand.

The template follows latest-cuda13, which Strata's maintainer moves to each new release when its image is published. Unraid shows the update on the Docker tab. To stay on one release, change the tag to a version such as 0.2.3-cuda13.

Credits and licence

Strata and its image are by Niko1221 and the Strata contributors, under the MIT licence. The icon is Strata's own. This repository is MIT licensed; see LICENSE.

Categories

Related apps

Explore more like this

Explore all

Links

Details

Repository
ghcr.io/niko1221/strata:latest-cuda13
Last Updated2026-11-19
First Seen2026-10-26

Runtime arguments

Web UI
http://[IP]:[PORT:8080]/
Network
bridge
Shell
bash
Privileged
false
Extra Params
--runtime=nvidia --ulimit memlock=-1:-1 --shm-size=2g --stop-timeout 120

Template configuration

WebUI / API portPorttcp

Chat page, monitor and the API (/v1 for OpenAI clients, the root URL for Anthropic clients).

Target
8080
Default
8080
Value
8080
DataPathrw

The model, the prepared pack, presets, tuning and the API key: tens of GB for Qwen, 191 to 274 GB for DeepSeek; the log names the size. Use a direct pool path on an NVMe drive (/mnt/cache/... or /mnt/POOLNAME/...), not /mnt/user: the engine reads the model files while it runs.

Target
/data
Default
/mnt/cache/appdata/strata
Value
/mnt/cache/appdata/strata
GPUVariable

The cards the container sees: a UUID (GPU-xxxxxxxx-...) from the Nvidia Driver plugin page or nvidia-smi -L, several separated by commas, or all. With exactly two RTX 30 or two RTX 50 class cards, a supported Qwen3.8 file and a context up to 128k, Strata uses its all-cards split engine by itself.

Target
NVIDIA_VISIBLE_DEVICES
Default
all
Value
all
ModelVariable

Empty: Strata picks for this server, and the log says which and why. Otherwise set Model family too. A smaller number in the name means a smaller download and less RAM, at some cost in quality. Qwen3.8 (qwen): Q2_0 is the fastest (48 GB RAM), then IQ2_XS, IQ3_XXS, and IQ3_S the best (64 GB RAM). Qwen3.6 (qwen36), a smaller model: UD-Q4_K_XL for 32 GB of RAM, UD-Q6_K_XL, Q8_0. DeepSeek (deepseek41), the largest: ist-das-2.25bpw the best, ist-das-2.00bpw, ist-das-1.75bpw the smallest, ista-das-coder for code. The Overview lists every option with its download size and RAM.

Target
MODEL
Model familyVariable

Which model, or which version of it. Empty: Strata's pick. qwen: Qwen3.8-Flash-Next, the main model. swift: Qwen3.8 tuned to think for less time. coder: Qwen3.8 for code only, in less RAM. gyro: Qwen3.8 in the least memory, slower. unsloth: Qwen3.8 at 4 to 8 bits, for servers with a lot of RAM. nvfp4: Qwen3.8 in NVIDIA's 4-bit format, fastest on RTX 50. qwen36: Qwen3.6-35B-A3B, a smaller model for 32-52 GB of RAM. deepseek41: DeepSeek-V4.1-Flash, the largest (cuda13 tag, 191-274 GB of disk).

Target
FAMILY
Context lengthVariable

How much text it remembers at once. Longer costs memory. Qwen3.8 goes up to 256k. DeepSeek is checked at 8k and offers up to 1m, but nothing past 256k has been tested and a very long prompt is read slowly.

Target
CONTEXT
Default
8k|32k|64k|128k|256k|512k|1m
Value
32k
VisionVariable

on: the model reads pictures (about 0.9 GB more to download). gpu or cpu choose where the picture reader runs. DeepSeek reads pictures with its own engine; off turns that off.

Target
VISION
Default
off|on|gpu|cpu
Value
off
API keyVariable

16 characters or more (openssl rand -hex 24 makes one). Empty: the first start makes one and keeps it in the Data folder.

Target
API_KEY
TuningVariable

Measures the best engine settings for this server, once, on the next start, and keeps them. thorough takes longer than quick; how long depends on the hardware.

Target
TUNE
Default
skip|quick|thorough
Value
skip
Tune againVariable

1 tunes again on the next start even if this preset is already tuned, for example after an update. Set it back to 0 afterwards.

Target
RETUNE
Default
0|1
Value
0
Allow tuning to change answersVariable

1 lets the tuner also try settings that can change answers very slightly. 0 keeps only settings that leave answers the same.

Target
TUNE_ALLOW_SMALL
Default
0|1
Value
0
Parallel requestsVariable

Requests answered at the same time, 1 to 8. Empty: fitted to the card. With 2 or more, Strata uses the MTP drafter, because its default DFlash2 drafter answers one request at a time. DeepSeek answers one at a time unless you set this.

Target
PARALLEL
KV cache typeVariable

Conversation memory format: int8 (default), q4_0 (half the memory), k8v4, fp16.

Target
KV
Default
int8|q4_0|k8v4|fp16
Value
int8
Existing model filesVariable

GGUF files you already have, as a path inside the container. Add a Path mapping for their folder (read-only is fine), e.g. /models, and enter that here; set Model too. Nothing is downloaded for the model itself.

Target
MODEL_PATH
Download despite old model filesVariable

When model files from the older image are still in /data/models, the start stops instead of downloading the model again (tens of GB) and the log says where they are. Use them with Existing model files, or set 1 to download anyway.

Target
ALLOW_DOWNLOAD
Default
0|1
Value
0
GPU rolesVariable

Several cards: a role for each, e.g. 1=main,0=helper (main, helper or off; cards you do not name are off). Empty: the engine's choice. See docs/MULTI_GPU.md in the Strata repository.

Target
GPU_ROLES
Engine settingsVariable

Advanced engine settings, space separated KEY=VALUE, like strata launch --set. For DeepSeek, dspark=on|off|auto sets its DSpark drafter (default auto). docs/SETTINGS.md in the Strata repository lists every setting.

Target
STRATA_SET