Stable Diffusion WebUI Forge - Neo

^{[ Neo | Classic ]}

Stable Diffusion WebUI Forge is a platform on top of the original Stable Diffusion WebUI by AUTOMATIC1111, to make development easier, optimize resource management, speed up inference, and study experimental features.
The name "Forge" is inspired by "Minecraft Forge". This project aims to become the Forge of Stable Diffusion WebUI.

- lllyasviel
^{(paraphrased)}

"Neo" mainly serves as an continuation for the "latest" version of Forge, which was built on Gradio 4.40.0 before lllyasviel became too busy... Additionally, this fork is focused on optimization and usability, with the main goal of being able to run the latest popular models via an easy-to-use GUI.

[!Tip] How to Install

Features [Oct.]

Most base features of the original Automatic1111 Webui should still function

New Features

• Support Moss-Image
• Support Qwen-Image-2.1
• Support Krea 2
• Support Krea 2 Identity Edit
  • require specific LoRA
• Support Anima 2B / Anima 2.9B / Anima 3.8B
  • automatically map LoRA / Controllllite for 2.9B & 3.8B
  • qwen35_4b adapter requires Extension
• Support Anima Edit
  • require specific LoRA
• Support Flux.2-Klein
  • 4B / 9B (not FLUX.2-Dev)

[!Note] To toggle between img2img and reference (edit) for Qwen-Image-2.1 / Krea 2 / Anima / Flux.2-Klein, refer to Settings/Stable Diffusion

• Support Ernie-Image
• Support PiD 1.5
  • sdxl / qwen / flux1 / flux2 (not PixelDiT)
  • use PiD Integrated to automatically upscale after generation
• Support Z-Image
• Support Wan 2.2
  • 14B (not 5B)
  • use Refiner to achieve High Noise / Low Noise switching

[!Important] To export a video, you need to have FFmpeg installed

• Support Mugen
  • display the Shift slider for xl preset in Settings/Presets/XL
• Support advanced SDXL models

[!Note]

• v-prediction: state_dict must include "v_pred"
• Zero Terminal SNR: state_dict must include "ztsnr"
• Rectified Flow: the model must include "rectified" in its path (e.g. file name or folder name)

• Support Qwen-Image / Qwen-Image-Edit

[!Note] To be detected as an Edit model, the model must include both "qwen" and "edit" in its path (e.g. file name or folder name)

• Support Flux Kontext

[!Note] To be detected as a Kontext model, the model must include "kontext" in its path (e.g. file name or folder name)

• Implement ImageStitch Integrated
  • support multi-image inputs for Edit models
  • support FirstLastFrameToVideo for Wan 2.2
• Support Nunchaku (SVDQ) Models (Deprecated)
  • flux-dev, flux-krea, flux-kontext, qwen-image, qwen-image-edit, z-image-turbo
  • only Flux and Qwen support LoRA currently
  • see Commandline
• Support Lumina-Image-2.0
• Support Chroma1-HD
• Support MixedPrecision Models
  • fp4mixed / fp8mixed / mxfp8 / nvfp4 / fp8_scaled / int8_convrot / convrot_w4a4 / asym_w4a8_int8 / w6a8_int8
• Support Flux.2-Small-Decoder & Qwen2D VAE

[!Tip] Check out Download Models for where to get each model and the accompanying modules

[!Tip] Check out Inference References for how to use each model and the recommended parameters

• Rewrite Preset System
  • now save the checkpoint/module selection and parameters per each Preset

[!Note] This overrides the UI Defaults for the controlled parameters

• Enforce Resolution Steps
  • dimensions must be multiples of 64 by default
  • adjust in Settings/System
• Support uv package manager
  • drastically speed up installation
  • require manually installing uv
  • see Commandline
• Support SageAttention, FlashAttention, ComfyKitchenAttention, fp16_accumulation
  • see Commandline
• Implement Radial Attention
  • speed up Wan 2.2
  • require manually installing SpargeAttn
• Implement Sol-Attn
  • speed up inference
• Implement fast state_dict switching for Refiner
  • enable in Settings/Refiner
• Implement RescaleCFG
  • reduce burnt colors; mainly for v-pred checkpoints
  • enable in Settings/UI Alternatives
• Implement MaHiRo
  • alternative CFG calculation; improve prompt adherence
  • enable in Settings/UI Alternatives
• Implement DeGrid
  • remove VAE artifacts
• Implement Spectrum
  • speed up inference
• Implement Epsilon Scaling
  • enable in Settings/Stable Diffusion
• Implement torch.compile
  • speed up inference after compilation
  • require triton
• Implement alternative Prompt Box layouts
• Implement tiled Conv2d for VAE
  • reduce memory usage; reduce speed
  • see Commandline
• Implement full precision calculation for Mask blur blending
  • enable in Settings/img2img
• Support TAESD / TAEHV live preview for all models
• Support video previews for ExtraNetworks
• Support loading upscalers in half precision
  • speed up; reduce quality
  • enable in Settings/Upscaling
• Support running tile composition on GPU
  • enable in Settings/Upscaling
• Support (short) videos in Extras tab
• Add support for .avif, .heif, and .jxl image formats
• Automatically determine the optimal row count for X/Y/Z Plot
• Update LLLite Controlnet
  • SDXL / Anima
  • support MultiDiffusion
• Support Union Controlnet
  • SDXL / Chenkin
• Support Region Controlnet
  • Anima
  • use the Region mode to create color masks
• Implement LoRA Control
  • require on-the-fly LoRA
  • require syntax <lora:name:[w1@t1, w2@t2, ...]>
  • see comment for example

Removed Features

• SD2
• SD3
• Forge Spaces
• Hypernetworks
• CLIP Interrogator
• Deepbooru Interrogator
• Textual Inversion Training
• Some built-in Extensions
• Some built-in Scripts
• Some Samplers & Schedulers
• Some Compatibility Settings
• Stealth Infotext
• bitsandbytes Support
• Nunchaku Support
• GGUF Support

Optimizations

• [Comfy] Rewrite the Backend (memory_management.py, ModelPatcher, attention.py, etc.)
• No longer git clone any repository on fresh install
• No longer create the UI twice when using --api
• No longer install open-clip
• Fix memory leak when switching checkpoints
• Restore the ability to drag-and-drop images onto gr.Image that already contains image
• Speed up launch time
• Speed up model loading
• Improve timer logs
• Remove unused cmd_args
• Remove unused args_parser
• Remove unused shared_options
• Remove legacy codes
• Fix some typos
• Fix automatic Tiled VAE fallback
• Fix Tiling for SD1 and SDXL
• Pad conditioning for SDXL
• Remove duplicated upscaler codes
• Update spandrel
  • support new upscaler architectures

[!Note] Put every upscaler (.pth / .safetensors) inside the ESRGAN folder

[!Tip] Check out OpenModelDB for where to get upscalers

• Improve ForgeCanvas
  • brush adjustments
  • customization
  • deobfuscate
  • eraser
  • hotkeys
  • mobile friendly
• Optimize upscaler logics
• Optimize certain operations in Spandrel
• Optimize certain operations for VAE
• Speed up model loading
• Improve color correction
• Improve memory management
• Unload all models on OutOfMemory
• Update the implementation for X/Y/Z Plot
• Update the implementation for Soft Inpainting
• Update the implementation for MultiDiffusion
• Update the implementation for uni_pc and LCM samplers
• Update the implementation of LoRAs
• Streamline Model Refresh
  • clicking refresh in Quicksettings will now also update Hires. fix, Refiner, and X/Y/Z Plot dropdowns
• Revamp settings
  • improve formatting
  • update descriptions
• Check for Extension updates in parallel
• Move embeddings folder into models folder
• Infotext Rewrite
  • allow switching Models and Modules
  • save emphasis properly
  • correct default values
• ControlNet Rewrite
  • change Units to gr.Tab
  • improve masks & buttons
  • remove multi-inputs, as they are "misleading"
• Checkpoint Merger Rewrite
  • update ui
  • support modern models
• Refiner Rewrite
  • update ui
  • improve tooltips
  • implement Refiner CFG Scale
• Improve inference speed
• Improve non-Nvidia support
• Lint & Format
• Update Pillow
  • faster image processing
• Update protobuf
  • faster insightface loading
• Update to latest PyTorch
  • torch==2.14.0+cu130

[!Note] If your GPU does not support the latest PyTorch, manually install older version of PyTorch

• Update some packages to newer versions
• Update recommended Python to 3.13.15
• many more... :tm:

Commandline

These flags can be added after the set COMMANDLINE_ARGS= line in the webui-user.bat (in the same line ; separate each flag with space)

[!Tip] Use python launch.py --help to see all available flags

• --xformers: Install the xformers package to speed up generation

[!Warning] xformers does not support RTX 50s

• --port: Specify a server port to use
  • defaults to 7860
• --api: Enable API access

by. Neo

• --cuda-malloc: Improve memory allocation
• --cuda-stream: Enable async weight offloading
• --pin-shared-memory: Improve RAM utilization
• --expandable-segments: Enable experimental PyTorch allocator (may prevent OutOfMemory errors on certain platforms)

• --uv: Replace the python -m pip calls with uv pip to massively speed up package installation
  • require uv to be installed first (see Extra Installations)
• --uv-symlink: Same as above; but additionally pass --link-mode symlink to the commands
  • significantly reduce installation size (~7 GB to ~100 MB)
• --uv-local-cache: Same as above; but additionally set UV_CACHE_DIR to a .uv-cache folder within WebUI directory
  • speed up installation on non-default drive (i.e. not C: on Windows)
  • allow clean uninstallation by simply deleting the WebUI directory

[!Important] symlink means it will directly access the packages from the cache folder instead of copying the packages over ; refrain from clearing the cache when using this option

• --model-ref: Points to a central models folder that contains all your models
  • said folder should contain subfolders like Stable-diffusion, Lora, VAE, ESRGAN, etc.

[!Important] This simply replaces the models folder rather than adding on top of it

• --forge-ref-a1111-home: Point to an Automatic1111 installation to load its models folders

  • i.e. Stable-diffusion, text_encoder, etc.

• --forge-ref-comfy-home: Point to a ComfyUI installation to load its models folders

  • i.e. diffusion_models, clip, etc.

• --forge-ref-comfy-yaml: Point to the ComfyUI extra_model_paths.yaml to load its configurations

  • i.e. base_path, checkpoints, etc.

• --sage: Install the sageattention package to speed up generation
  • will also attempt to install triton automatically
• --flash: Install the flash_attn package to speed up generation
• --nunchaku: Install the nunchaku package to inference SVDQ models

• --fast-fp8: Use the torch._scaled_mm function when the model type is float8_e4m3fn
• --fast-fp16: Enable the allow_fp16_accumulation option
• --autotune: Enable the torch.backends.cudnn.benchmark option
  • this is slower in my experience...
• --tiled-conv2d: Replace Conv2d ops with tiled variants
  • has greater reduction for SD1 and SDXL VAE; less for Wan VAE
  • 64 / 128 / 256 / 512

• --use-ck-attention: Use the Comfy-Kitchen attention to speed up generation
  • override other attention packages
• --enable-triton-backend: Enable the use of Triton backend in comfy-kitchen
  • might speed up inference
• --pynvml: Use the NVIDIA Management Library to determine the actual amount of free VRAM
  • prevent system freeze when running other programs

Installation

1. Install git

2. Clone the Repo

   git clone https://github.com/Haoming02/sd-webui-forge-classic sd-webui-forge-neo --branch neo
   

3. Setup Python

1. (Optional) Configure Commandline
2. Launch the WebUI via webui-user.bat
3. During the first launch, it will automatically install all the requirements
4. Once the installation is finished, the WebUI will start in a browser automatically

[!Tip]

• For AMD, refer to CS1o 's Guide
• For Linux and macOS, refer to Wiki
• For Docker (Nvidia / AMD), refer to Docker

[!Tip] Check out Extra Installations for how to install git, uv, and FFmpeg

Attention Functions

[!Note] The --xformers, --flash, and --sage args are only responsible for installing the packages, not whether its respective attention is used (this also means you can remove them once the packages are successfully installed)

[!Caution] Do not just blindly install all of them
Nowadays the native PyTorch scaled_dot_product_attention is usually as fast, and also more stable

Forge Neo tries to import the packages and automatically choose the first available attention function in the following order:

1. SageAttention
2. FlashAttention
3. xformers
4. PyTorch
5. Basic

[!Tip] To skip a specific attention, add the respective disable arg such as --disable-sage

Issues & Requests

• Issues about removed features will simply be ignored
• Issues that is obviously user-error will simply be ignored
• Issues regarding AMD GPU will simply be ignored
• Issues running non-official models will simply be ignored
  • do not just randomly download every single finetune/quant you find
• Issues about 3rd-party Extensions will simply be ignored
  • extension should support the UI, not the other way around
• Issues caused by StabilityMatrix will simply be ignored
  • only open an Issue if you can reproduce it on a clean install following the official Installation instruction

[!Caution]

• If you post NSFW images/videos, you will immediately be banned
  • the sole discretion is on me ; if you are unsure, just generate cats and dogs...

[!Tip] Check out the Wiki & FAQ

Special thanks to AUTOMATIC1111, lllyasviel, and comfyanonymous, kijai, city96,
along with the rest of the contributors,
for their invaluable efforts in the open-source image generation community

_{Buy me a Coffee ☕~
PayPal me 💳~}