Nexus Orchestrator

A self-hosted orchestration layer that intelligently routes each request to the best local or cloud model.

Overview

Nexus is more than a chat interface — it's an orchestration layer for your local and cloud LLMs. A lightweight router model analyzes each prompt's intent and dispatches it to the most capable model for the job: local Ollama for privacy-sensitive or everyday tasks, cloud APIs (Gemini, OpenAI-compatible) for heavier reasoning or specialized work.

Key Features

• Intelligent Intent Routing — A small router model classifies each prompt (CODING, REASONING, CREATIVE, VISION, DOCUMENT, GENERAL, FAST, SECURITY) and selects the right model automatically.
• Hybrid Orchestration — Mix local and cloud models per category. Switch seamlessly without changing your workflow.
• Multi-User Support — Multiple users with per-user provider config, category mappings, conversations, and projects. Admin manages users from the System tab.
• Vision & Document Support — Upload images and documents directly in the chat. Vision models receive them in the correct format automatically.
• Web Search via Tool Calling — Connect a self-hosted SearXNG instance so the LLM can search the web when it needs current information. Enable globally or per-chat with a single click.
• Thinking Toggle — For Ollama models that support it, Nexus sends think: true via the native API and streams reasoning live as it generates. Models like DeepSeek R1 that natively emit <think> tags are also supported. Reasoning appears in a collapsible purple section above the response. Toggle globally (System tab) or per-chat (Brain icon). Models that don't support thinking fall back silently.
• Privacy First — Point the router at a local Ollama model and your prompts never leave your network.
• Structured Logging — JSON logs in production (pino), pretty-printed in dev. Control verbosity with LOG_LEVEL.
• Unraid Ready — Includes a community template for one-click Unraid deployment.
• Security Hardened — CORS, security headers, rate limiting, session management, body size limits, and input validation all configured out of the box.

Origin

Nexus started as a vibe coding session — no grand plan, just a frustration with constantly switching between chat interfaces depending on which model I wanted to use that day. The idea was simple: what if something just decided for me?

What began as a single-file prototype grew into something I actually use. Along the way it picked up proper component architecture, structured logging, vision support, Docker packaging. Most of the core ideas came from just running it, hitting its rough edges, and fixing them.

It's still evolving.

Quick Start

Docker (Recommended)

docker run -d \
  --name nexus-orchestrator \
  -p 3000:3000 \
  -v $(pwd)/data:/app/data \
  -e ADMIN_API_KEY=your_secure_admin_key \
  -e LOCAL_URL=http://host.docker.internal:11434 \
  -e ROUTER_MODEL="REPLACE_YOUR_MODEL_NAME" \
  pikkonmg/nexus-orchestrator

Then open http://localhost:3000 and log in with username admin and your ADMIN_API_KEY as the password.

Docker Compose

services:
  nexus:
    image: pikkonmg/nexus-orchestrator
    ports:
      - "3000:3000"
    volumes:
      - ./data:/app/data
    environment:
      - ADMIN_API_KEY=your_secure_admin_key
      - LOCAL_URL=http://host.docker.internal:11434
      - ROUTER_MODEL="REPLACE_YOUR_MODEL_NAME"
      # Optional: add cloud provider for hybrid routing
      # - CLOUD_URL=https://your-cloud-provider.com/v1
      # - CLOUD_API_KEY=your_api_key
    restart: unless-stopped

Unraid

1. Copy template_unraid.xml to your flash drive: /boot/config/plugins/dockerMan/templates-user/
2. Add the container from your Unraid Docker tab.

Environment Variables

Privacy Configuration

Nexus has no hardcoded cloud defaults — all provider URLs and models start empty. For 100% local operation:

1. Set LOCAL_URL to your Ollama instance
2. Set ROUTER_MODEL to a small local model (e.g. gemma3:4b, qwen2.5:3b)
3. Assign local models to each category in the UI → Models tab

Your prompts will never leave your network.

Remote Access

If Nexus runs in the cloud but your Ollama is local, expose it via a tunnel:

# Cloudflare
cloudflared tunnel --url http://localhost:11434

# Ngrok
ngrok http 11434

Then set LOCAL_URL to the tunnel address.

Configuration Guide

Local Model / API Provider

This is your local LLM backend — usually Ollama running on your home server or PC.

• Provider URL — The address Nexus uses to reach your local models. For Ollama this is typically http://192.168.1.x:11434. Don't use localhost — inside Docker that refers to the container itself, not your machine.
• API Key — Leave blank for Ollama. Only needed if your local provider requires authentication (e.g. Open WebUI with auth enabled).

Nexus auto-detects Ollama and switches to its native /api/chat endpoint automatically.

Cloud Model / API Provider

An optional OpenAI-compatible cloud API for heavier tasks or models you don't run locally.

• Provider URL — The base URL of your cloud provider (e.g. https://api.openai.com/v1, https://openrouter.ai/api/v1).
• API Key — Your API key for that provider.

If you leave this blank, the CLOUD provider option in Category Mappings will show a warning and fall back to local.

Intent Router (Orchestrator)

The router is a small model that reads each prompt and decides which category and model should handle it. It runs before every chat message.

• Model — The model used for routing. A small fast model works well here — you don't need a large model, just one that can return clean JSON. Good options: gemma3:4b, qwen2.5:3b, gpt-4.1-mini.
• Router URL — Leave blank to use your Local Provider. Set a custom URL if you want to use a different endpoint just for routing (e.g. a paid cloud router while keeping chat local).
• Router Key — API key for the router endpoint if required.

The router model must be capable of returning valid JSON. If it wraps its response in markdown code blocks that's fine — Nexus strips them automatically.

Discovered Models

This section shows all models Nexus finds at your Local Provider URL. Nexus queries your provider on the Models tab and lists everything available. Click any model to add it to a category pool.

If your models aren't showing up, check that your Local Provider URL is correct and the connection shows as Online in the header.

Category Mappings

Categories are how Nexus decides which model handles which type of request. Each category has a model pool and a provider (Local or Cloud).

How to set up a category:

1. Go to the Models tab
2. Find the category you want to configure
3. Set the provider to Local or Cloud
4. Click models from the Discovered Models list to add them to the pool, or type a custom model name
5. The first model in the pool is used by default. If it fails, Nexus automatically tries the next one.

You can add custom categories using the input at the bottom of the Models tab.

Web Search (SearXNG)

Connect a self-hosted SearXNG instance to give your LLMs tool-calling access to the web. The model decides when to search — it won't search on every message.

• SearXNG URL — The address of your SearXNG instance (e.g. http://192.168.1.50:8080). Must be accessible from the Nexus container.
• Always On — Include the search tool in every request. Otherwise, use the 🌐 globe button in the chat input to enable it per-chat.

Requires a model that supports tool calling. Recommended: llama3.1, llama3.2, qwen2.5, mistral-nemo. FAST category always skips search.

When a search fires mid-response, the routing indicator switches to > Searching the Web... with a blue globe. Completed messages show a Web Search: <query> badge.

System Settings

Router Result Caching

By default, every prompt makes a fresh call to the router model for intent classification. If you're using a paid cloud router (OpenAI, OpenRouter, etc.), you can enable caching to avoid redundant API calls.

How to enable:

1. Go to the System tab
2. Toggle Router Result Caching on

When enabled, identical routing prompts reuse the cached decision for 5 minutes. After that the cache expires and the router model is called again. The cache is in-memory only and clears on server restart.

When to use it: Paid cloud routers where each call costs money. When to skip it: Local routers (Ollama) where the call is fast and free.

Show Model Thinking

Some reasoning models (DeepSeek R1, QwQ, etc.) embed their chain-of-thought inside <think>...</think> tags in the response content. Nexus parses these tags client-side and displays the reasoning in a collapsible purple section above the response.

Global toggle:

1. Go to the System tab
2. Toggle Show Model Thinking on or off (default: on)

Per-chat toggle: Click the Brain icon (🧠) in the chat input bar to override the global setting for the current session.

When thinking is visible, the section auto-collapses once the model's actual response starts streaming. Click to expand it again at any time. When thinking is hidden, the <think> tags are stripped and only the final response is shown.

Building from Source

git clone https://github.com/FaqFirebase/Nexus-Orchestrator.git
cd nexus-orchestrator
npm install
cp .env.example .env   # fill in your keys
npm run dev            # starts server + Vite HMR on :3000

To build the Docker image:

docker build -t nexus-orchestrator:latest .

Roadmap

Planned

• URL fetch/browse tool — Companion to web search; lets the LLM fetch and read the content of a specific URL directly
• Ollama backend abort — Investigate stopping Ollama generation server-side when client disconnects (current TCP disconnect does not propagate through Docker networking)

See ROADMAP.md for the full history of completed features.

Changelog

v1.1.9 — Thinking toggle for reasoning models (DeepSeek R1, QwQ, etc.). Client-side <think> tag parsing with collapsible display above responses. Global default (System tab) and per-chat override (Brain icon). Docker image reduced from 127 MB to ~86 MB via dependency cleanup and build-stage pruning. Added CONTRIBUTING.md, SECURITY.md, and GitHub issue templates.

v1.1.8 — Copy button on code blocks (hover to reveal, 2-second "Copied" feedback). FAST category routing tightened — now restricted to greetings and micro-interactions only; factual questions route to GENERAL. Security hardening: CORS spec compliance, cloud metadata SSRF blocking, security headers (CSP, HSTS, X-Frame-Options, etc.), rate limiting on password change, session memory leak fix, per-user session cap, reduced body size limits, trust proxy for Caddy, admin settings schema validation, password complexity requirements, and cookie parser hardening.

v1.1.7 — Collapsible settings sections with persistent state (all Models tab sections collapse/expand and remember their state across refreshes). Active tab persists on page refresh. Discovered Models redesigned as a provider-grouped collapsible list with active router highlighting and size-tiered colour coding. Mixed content fix for HTTPS deployments.

See CHANGELOG.md for the full release history.