Scanbutler

Turn scanned paper into searchable PDFs, one per document, named after their content. It runs as a Docker container that watches up to three folders.

Stacks. Drop a scan of a whole pile of paper, with hundreds of pages and no separator sheets. The container finds where each document begins from the content alone and splits the scan:

stacks/inbox/Household/stack-01.pdf  (500 pages)
        ↓
stacks/output/Household/Blood count 2026-09-30.pdf
stacks/output/Household/CT report chest 2026-09-30.pdf
stacks/output/Household/Electricity bill 2026-08-14.pdf
...

Scanner. Point a document scanner that saves to a network share at the second inbox. Each file there is one document. It gets the same OCR and naming, but is never split:

scanner/inbox/20261002_141503.pdf
        ↓
scanner/output/Insurance renewal notice 2026-09-28.pdf

This suits Fujitsu (now Ricoh) ScanSnap scanners such as the iX1600 well. When they scan straight to a network folder (SMB), they save image-only PDFs: the OCR is done by the ScanSnap software on a computer. Point the scanner at this inbox, and the files come out searchable and named without any PC running.

Paperless (optional). Files dropped here only get the Tesseract text layer and are then uploaded to Paperless-ngx. Paperless, or an AI tagger working with it such as Zettelrobbe, takes care of the title, tags and correspondent:

paperless/inbox/scan.pdf  →  text layer  →  Paperless-ngx document #1234

Titles are written in the language of each document unless you set TITLE_LANGUAGE.

Under the hood:

• ocrmypdf with Tesseract adds a fresh, invisible text layer, so every output PDF is searchable.
• Mistral OCR reads stacks through Mistral's batch API, at half the regular price. Its structured text makes splitting more reliable. Scanner files are read from the Tesseract layer instead, which costs nothing and adds no waiting time.
• A Mistral chat model decides where documents begin and names them.

Contents

• System requirements
• Quick start
• How it works
• Reviewing and correcting splits
• Configuration
• Languages
• Queue webhook (Home Assistant)
• Spending limit and paused processing
• Choosing the text source
• Paperless-ngx input
• Rate limits and cost
• Privacy
• Unraid
• Development
• Contributing

System requirements

The container adapts to its memory. With OCRMYPDF_JOBS=auto, the default, it runs as many OCR pages in parallel as its memory limit allows: 1 GB as a base plus 0.75 GB per page. It never runs more than one page per CPU the container may use, and Docker's --cpus and --cpuset-cpus limits are respected. Less memory therefore makes processing slower, never makes it fail.

All inputs share that budget, and scanner and Paperless files go first:

• A file gets one job per page, as many as are free. A two-page scan takes two jobs and leaves the rest to others.
• When jobs are scarce, waiting scanner and Paperless files are served before stacks. The same goes for requests to Mistral.
• Stacks get their text layer in pieces of 20 pages. A scan therefore waits for one piece at most, even while a 500-page stack is running.
• The scanner and Paperless inputs work on several files at once.

Beyond a certain point, more memory no longer helps: the CPUs become the limit. Full speed needs about 1 GB + 0.75 GB × CPU cores, so 5.5 GB for 6 cores, or 7 GB for 8. The startup log shows the outcome: "memory_gb":6.0,"cpus":6,"ocr_jobs_total":6.

Measured with a worst-case file: four A4 pages, each holding a colour image at 1550 dpi. The images are downsampled to 600 dpi first, see OCRMYPDF_MAX_IMAGE_DPI.

For scale: 156 ordinary scanned pages took about 4.5 minutes on a 12-thread desktop CPU. Expect a low-power NAS CPU to be several times slower. That is fine for a folder watcher, but a 500-page stack may then take hours.

Synology and other NAS systems: in Container Manager, set the memory limit under Resources to at least 1 GB, or 4 GB if available. NAS models with 32-bit ARM CPUs (armv7) are not supported; check your model's CPU architecture. Map /data to a shared folder and /config to a folder for the container's own data, and set PUID/PGID to a user that may write there, as with any linuxserver.io container.

Quick start

You need Docker and a Mistral API key.

git clone https://github.com/Tom-Joad/scanbutler.git
cd scanbutler
cp .env.example .env        # set MISTRAL_API_KEY, and DATA_PATH, PUID, PGID, TZ if needed
docker compose up -d --build
docker compose logs -f

The prebuilt image ghcr.io/tom-joad/scanbutler can replace the local build, see docker-compose.yml.

On first start, the container creates this layout under DATA_PATH:

stacks/     inbox/  output/  archive/  failed/
scanner/    inbox/  output/  archive/  failed/
paperless/  inbox/          archive/  failed/    (only with PAPERLESS_URL set)
paperless-2/ inbox/         archive/  failed/    (only with PAPERLESS_2_TOKEN set)

Its own data (plans, caches, review files) goes to CONFIG_PATH, mounted at /config.

Put PDFs into stacks/inbox/ or scanner/inbox/, optionally in sub-folders. Sub-folders are mirrored in output/. A file is picked up once it has not changed for STABLE_SECONDS (60 s by default), so copying a large scan over the network is safe.

To process a single file once, outside the inboxes:

docker compose exec scanbutler scanbutler process /data/some.pdf --profile stacks --folder "Household"

Commands run with docker exec act as the abc user, so their files get the same owner as everything else.

How it works

1. OCR (stacks by default). Pages go to Mistral OCR in chunks of 50 through the batch API. A job takes minutes instead of seconds and costs half as much. Job ids and results are stored as soon as they exist, so an interrupted run never pays for a page twice. Uploads and results are deleted from Mistral's file storage afterwards. A chunk that fails inside a batch is retried directly. OCR_MODE=direct skips the batch API.

   Scanner files skip this step by default. Their text comes from the Tesseract layer that step 5 adds anyway. Either input can be switched with STACKS_TEXT_SOURCE / SCANNER_TEXT_SOURCE; see Choosing the text source.

2. Blank pages. A page is dropped when its image shows almost no ink, typically the back of a duplex scan. The check uses the image itself, not the OCR text, because OCR models sometimes invent whole paragraphs on an empty page.

3. Boundaries (stacks only). A chat model reads overlapping windows of 12 pages and decides, page by page, whether a new document starts. It looks at letterheads, salutations, headings, dates, layout changes and sentences that run across pages. Each page's verdict comes from the window where it had the most context on both sides. Explicit "page k of n" markers override the model. A second copy of a document becomes a file of its own, for example ... (2).pdf.

4. Naming. Each document gets a short title of 1 to 6 words, naming its type and the detail that sets it apart, plus the date it is about. For reports and lab results, that is the examination or sampling date. For letters, it is the issue date.

5. Text layer. ocrmypdf adds Tesseract's text, choosing the mode per file:

   • Pure scans (no text at all): --force-ocr. Pages are deskewed, the image Tesseract reads is cleaned, and poor scans are upsampled to 300 dpi.
   • Tagged PDFs, which carry a logical structure tree: kept as they are. These are born-digital files such as office documents or online bank statements. Their text is the original, and re-OCR would discard the structure.
   • Other PDFs that already contain text, such as scans that carry the scanner's own OCR: --redo-ocr. An old OCR layer is replaced, while real digital text stays as it is. Forcing OCR would turn such pages into pictures of themselves, many times the size. There is no image cleaning in this mode: ocrmypdf rasterizes at the resolution of the sharpest image on a page, and cleaning a page with a high-resolution logo at that size can exhaust the memory.

   Images sharper than 600 dpi, such as a high-resolution logo, are first downsampled with Ghostscript. Text and vector graphics stay untouched. Without this, ocrmypdf would rasterize the whole page at the image's resolution. Every mode also runs within limits, so one odd page can't exhaust the server. Tesseract sees at most OCRMYPDF_MAX_OCR_MPIXELS per page, and each page and each run has a time limit. The visible page is never downsampled.

   If a mode fails, a simpler one is tried before the file counts as failed. The last resort OCRs only pages without text and does no image processing. The log and .error.txt show ocrmypdf's actual error message. The image ships Tesseract's tessdata_best models for German and English. They are slower than the defaults, but hold up much better on poor scans. Other languages are downloaded on first use, see Languages.

6. Output. Each document's pages are cut from the searchable scan into <input>/output/<sub-folder>/<title> <date>.pdf. The PDF's title and subject metadata are set as well.

The original file then moves to archive/. A file that fails moves to failed/ together with an .error.txt. Move it back into inbox/ to retry: OCR that was already paid for is reused.

To check that a PDF really has a text layer, open it and search for a word with Ctrl+F, or select the text. In the document's font list, the invisible layer shows up as GlyphLessFont.

Reviewing and correcting splits

Without separator sheets, splitting cannot be perfect. Every input file gets a work directory, work/<input>/<sub-folder>/<file name>-<hash>/. It holds:

• review.md: every document with its page range. A document is flagged ⚠ when it starts mid-document (for example on "page 3 of 5"), when it is a single, nearly empty page, or when the model was unsure.
• decisions.json: the model's verdict and reason for every page.
• plan.json: the split plan, meant to be edited by hand. Page numbers are 1-based ranges such as "4-6, 9".

To fix a split, edit plan.json: change pages, or merge and split entries. Set title to "" to have the title and date generated again. Then run:

docker compose exec scanbutler scanbutler rebuild "stacks/Household/stack-01-1a2b3c4d"

The files listed in written_files are replaced. Nothing is OCR'd again.

Configuration

All settings are environment variables. .env.example lists them with comments.

Container (linuxserver.io conventions)

The image is built on linuxserver.io's Debian base and behaves like their containers: it starts as root, gives the abc user the IDs below and runs Scanbutler as abc.

Don't run the container with --user or --init: the base image's init system (s6-overlay) has to start as root and as process 1. Docker mods (DOCKER_MODS) work as with any linuxserver.io image.

Mistral

Folders and inputs

Paperless-ngx input

Naming

Splitting and blank pages

Text layer

Throughput, webhook and logging

Languages

deu, eng and osd (page orientation) are built into the image. Any other language in OCRMYPDF_LANGUAGES is downloaded at startup, once:

• Models come from tessdata_best at the pinned tag 4.1.0. Every download is checked against a SHA-256 list shipped in the image; a file that doesn't match is discarded.
• They are kept in /config/tessdata and survive image updates. Delete a file there to download it again.
• Codes are Tesseract's three-letter ones: fra, ita, spa, pol, tur, chi_sim and so on, 124 in total. A misspelt code stops the container with a hint, e.g. unknown language 'ger' (did you mean 'deu'?).
• Without internet access, a language that isn't downloaded yet is left out with a warning, and the rest keeps working; the built-in languages always do. A local mirror can stand in via TESSDATA_URL.

The start of the log says what is in use:

{"event":"languages ready","languages":"deu+eng+fra","downloaded":["fra"],"cached":[],"missing":[]}

More languages make OCR slower; list only the ones your documents use.

Queue webhook (Home Assistant)

With QUEUE_WEBHOOK_URL set, the container POSTs the queue state as JSON. It sends whenever a number changes, and again every QUEUE_WEBHOOK_HEARTBEAT_SECONDS, so the receiver catches up after a restart. The payload holds counts only, never file names:

{
  "queued": 3, "waiting": 2, "processing": 1, "failed": 0,
  "paused": false, "pause_reason": null, "paused_since": null,
  "profiles": {
    "stacks":  {"waiting": 1, "processing": 1, "failed": 0},
    "scanner": {"waiting": 1, "processing": 0, "failed": 0}
  }
}

• queued is waiting + processing.
• processing can be more than 1: the scanner and Paperless inputs work on several files at once.
• failed counts the PDFs in the failed/ folders.
• profiles has one entry per enabled input; paperless appears only when PAPERLESS_URL is set, paperless-2 only when PAPERLESS_2_TOKEN is.
• paused, pause_reason and paused_since are always present. The last two are null unless processing is paused. pause_reason is Mistral's raw error text, up to 300 characters. paused_since is an ISO 8601 timestamp in UTC.

Every send and every failure appears in the container log, with the error text but never the URL. An unchanged error is repeated at most once per heartbeat interval. If the receiver is unreachable, processing carries on.

For Home Assistant, a trigger-based template entity reads the webhook. Pick a long random webhook_id, because anyone who knows it can post to it:

template:
  - triggers:
      - trigger: webhook
        webhook_id: scan-splitter-queue-CHANGE-ME
        allowed_methods: [POST]
        local_only: true
    sensor:
      - name: Scan-Splitter queue
        unique_id: scan_splitter_queue
        state: "{{ trigger.json.queued }}"
        unit_of_measurement: files
        attributes:
          waiting: "{{ trigger.json.waiting }}"
          processing: "{{ trigger.json.processing }}"
          stacks_waiting: "{{ trigger.json.profiles.stacks.waiting | default(0) }}"
          scanner_waiting: "{{ trigger.json.profiles.scanner.waiting | default(0) }}"
      - name: Scan-Splitter failed
        unique_id: scan_splitter_failed
        state: "{{ trigger.json.failed }}"
        unit_of_measurement: files
    binary_sensor:
      - name: Scan-Splitter paused
        unique_id: scan_splitter_paused
        state: "{{ trigger.json.paused | default(false) }}"
        attributes:
          reason: "{{ trigger.json.pause_reason }}"
          since: "{{ trigger.json.paused_since }}"

Then set QUEUE_WEBHOOK_URL=http://<home-assistant>:8123/api/webhook/scan-splitter-queue-CHANGE-ME.

Spending limit and paused processing

Mistral can refuse an account, for example because its spending limit is reached, its quota is used up or its API key is rejected. In that case, processing pauses instead of moving file after file to failed/.

A refusal is an HTTP 401, 402 or 403, or a 429 that is not the ordinary per-second rate limit. Mistral does not document how a reached spending limit is answered, so this errs on the side of pausing.

While paused:

• Files stay in their inboxes, including the one that hit the limit.
• The log shows processing paused with Mistral's error text, and the webhook reports "paused": true.
• Every PAUSE_RETRY_MINUTES, one file is tried as a probe. Once it goes through, processing resumes on its own and the log shows processing resumed. That happens, for example, after you raise the limit or a new billing month starts.

Choosing the text source

Splitting and naming read the text of each page. Per input, it can come from Mistral OCR (mistral) or from the Tesseract text layer (tesseract). The Tesseract layer is added to every output PDF either way. By default, stacks use mistral and scanner files use tesseract.

A comparison on one test stack used the same text layer for both sources. The stack held 80 real documents (156 pages of mostly clean office and medical scans):

The remaining misses of both runs were debatable cases. One example is two X-ray views of the same examination that had been filed as two documents.

In short: Mistral OCR splits somewhat better. For naming alone, as with scanner files, the difference was not measurable. That is why the defaults are what they are. If your scanner files are handwritten or of poor quality, SCANNER_TEXT_SOURCE=mistral may be worth the cost.

Paperless-ngx input

Set PAPERLESS_URL and PAPERLESS_TOKEN to enable a third inbox, paperless/inbox/. It is meant for documents that Paperless-ngx should name and tag itself, for example with an AI tagger such as Zettelrobbe. For each file:

1. ocrmypdf adds the Tesseract text layer, with the same settings as for the other inputs. By default, no paid service is involved; see Content from Mistral OCR for the option.
2. The PDF is uploaded through POST /api/documents/post_document/, keeping its file name and adding PAPERLESS_TAGS, if set.
3. The container follows Paperless's consumption task until a document has been created. Only then does the original move to archive/, and the work copy is deleted.

Content from Mistral OCR

By default, the content field Paperless shows and searches comes from the Tesseract text layer. In that text, tables lose their structure. Set PAPERLESS_TEXT_SOURCE=mistral, and Mistral OCR reads each file before the upload, through the batch API. Right after Paperless confirms the new document, its content is replaced with Mistral's Markdown text. A lab report then reads:

Laborbefund vom 30.09.2026

| Parameter | Ergebnis | Einheit | Referenz |
| --- | --- | --- | --- |
| Leukozyten | 6,2 | /nl | 3,9 - 10,2 |

The same page in Tesseract's text reads Leukozyten 6,2 /nl 3,9 - 10,2, one line per row, with no columns.

• The PDF's own text layer stays Tesseract's, because only that one has word positions for search and selection in the file.
• Cost: Mistral OCR at batch price, about $1 per 500 pages. No chat model is involved.
• An AI tagger such as Zettelrobbe should see the better text. The content is replaced a few seconds after the document appears, and taggers usually poll less often. If yours reacts instantly, it may read Tesseract's text first.
• If replacing the content fails, the document stays in Paperless with Tesseract's text. A warning is logged, and the file is not uploaded again.
• A Mistral pause (see Spending limit) holds this input too, because it uses Mistral. With the default tesseract, the input keeps uploading during a pause.

Failure handling:

• Paperless unreachable, or token rejected. The file stays in the inbox, and the input retries after 5 minutes. A file that is still being consumed is not uploaded a second time after a restart: the task id is stored.
• Paperless rejects the document, for example as a duplicate. The file moves to failed/ with Paperless's message in the .error.txt.
• The same original dropped in twice. The file moves to failed/ and names the Paperless document it already became. Paperless's own duplicate check cannot catch this, because the text layer makes every upload a slightly different file. The container therefore keeps a register of uploaded originals in work/paperless/uploaded.json. Remove an entry there to upload that file again.

Paperless decides by itself whether to run its own OCR. With the default PAPERLESS_OCR_MODE=auto, it keeps the text layer added here. With redo or force, it replaces it.

Sub-folders in paperless/inbox/ are allowed and mirrored in archive/, but Paperless itself doesn't see them. Use PAPERLESS_TAGS, or Paperless workflows, to sort documents.

Tested with Paperless-ngx 3.2. The task format of version 2 is supported as well.

A second Paperless user

Paperless makes the uploading user the owner of a document. When several people share one Paperless instance, each should own the documents scanned for them; otherwise permissions that limit someone to their own documents don't work.

Set PAPERLESS_2_TOKEN to the API token of a second Paperless user, and a second inbox, paperless-2/inbox/, uploads as that user. Point a second scanner profile, or a second network folder, at it.

PAPERLESS_MAX_WAIT_MINUTES applies to both. The second input works exactly like the first, with its own duplicate register in work/paperless-2/uploaded.json: the same scan may go to both users. Every call it makes, uploads and content replacement included, uses its own token.

Shared tags, correspondents and document types

Paperless gives every new tag, correspondent and document type an owner, the user who created it, and other users only see those that have no owner or are shared with them. On an instance with several users, what one of them, or an AI tagger, creates is therefore invisible to everyone else. Paperless has no setting that makes new ones ownerless.

Set PAPERLESS_SHARE_TAGS=true, PAPERLESS_SHARE_CORRESPONDENTS=true and PAPERLESS_SHARE_DOCUMENT_TYPES=true, each on its own, and the container takes care of it. Every PAPERLESS_SHARE_TAGS_MINUTES (default 1), it removes the owner and any explicit permissions from every object of those kinds that has an owner. Every user with the global permissions for that kind may then see and change it.

• The token needs permission to change other users' objects, in practice a superuser's. With a weaker one, tags could not be shared (or correspondents …, document types …) is logged once with HTTP 403, and the other kinds and the uploads carry on. The same happens while Paperless is unreachable.
• Shared correspondents show every user who writes to whom: a bank, an employer, a doctor. Switch them on only where all users may see that.
• PAPERLESS_SHARE_TAGS_READONLY (tags only) takes comma-separated tag names, for example ai-processed, a marker an AI tagger uses to find work. Those tags keep their owner, every other user may see them, and nobody else may change them. A read-only tag without an owner is left alone with a warning: give it one in Paperless.
• Paperless doesn't allow two ownerless objects of a kind with the same name, e.g. a correspondent that you and an AI tagger both created. Such an object stays as it is, and the log names it once as correspondents not shared with its id and the ownerless one it clashes with (same_name_as); all others are shared. Merge the two in Paperless, and it is shared in the next round.
• A check with nothing to do costs one list request per kind. The log says how many were shared, with their ids (tags shared, correspondents shared, document types shared), never their names.
• Storage paths and custom fields keep their owners.

Rate limits and cost

Mistral limits requests per second and tokens per minute. The limits depend on the model and the account tier, and they are not published. Look yours up in Mistral's console under API › Limits and set MISTRAL_MAX_RPS a little below the requests-per-second limit of your MISTRAL_LLM_MODEL. If a request still runs into the limit, all workers pause together and retry. Requests for scanner and Paperless files take the next free slot, ahead of those for a stack.

Splitting needs one request per 6 pages, and naming one request per document. A 500-page stack holding about 200 documents takes roughly 280 requests, which is about 20 minutes at 0.25 requests per second. A scanner file needs a single request and, with the default text source, no OCR. Every answer is cached in the work directory.

At the prices published in October 2026, a 500-page stack costs about US$1.60: about $1.00 for batch OCR and about $0.60 for the chat model. Check Mistral's pricing for current figures.

Privacy

Every page is sent to Mistral's API, first for OCR, then for splitting and naming. Only use this for documents you are entitled to process that way. If the documents belong to someone else, get their consent first, especially for health or financial records.

• The work directories hold the full OCR text of every input file and a searchable copy of it. They are deleted WORK_RETENTION_DAYS (30) days after the file was processed; until then, rebuild can re-cut it. Folders of failed files stay until you delete them.
• Logs contain file names, page numbers and counts, never document text. The file names are there on purpose: the log has to show which file is being worked on. Keep this in mind before sharing a log, and redact the names.
• The Paperless input keeps no copy once a document is confirmed in Paperless. Only the register of uploaded originals remains: checksum, file name, document id and date.
• The queue webhook sends counts only.
• Languages beyond the built-in ones are downloaded from GitHub (or TESSDATA_URL) at startup. The request names only the language file.

Unraid

A Docker template and step-by-step instructions are in unraid/.

Development

There is no local Python setup to maintain: the image contains everything, and the tests run inside it. The image ships without pip, so the test run installs it first with ensurepip.

docker build -t scanbutler:dev .
docker run --rm --entrypoint sh -v "$PWD:/src" -w /src scanbutler:dev \
  -c "python3 -m ensurepip >/dev/null && python3 -m pip install -q pytest && python3 -m pytest -q"

The tests replace Mistral with a fake and need no API key. A smoke test checks the built image itself (watcher, health check, OCR models, file owners), also without a key:

tests/smoke.sh scanbutler:dev

CI runs both together with pip-audit, shellcheck and gitleaks on every push. Images are built only for version tags (v*), for linux/amd64 and linux/arm64. Each image is signed with cosign and ships an SBOM and provenance.

See CHANGELOG.md for the release history and SECURITY.md for reporting vulnerabilities.

Contributing

Bug reports, ideas and pull requests are welcome. Please never attach real documents, OCR text or unredacted file names to an issue. The log lines and settings the issue form asks for are almost always enough. If a problem only shows with one particular file, describe it (pages, scanner, what is special about it) instead of sharing it.

For pull requests: keep the tests passing (see Development), add a test for new behaviour, and add a line to the changelog.

Disclaimer

This is an independent project, not affiliated with or endorsed by Mistral AI, Paperless-ngx or any scanner manufacturer. Product names belong to their owners.

Splitting and naming are automated and can be wrong. Check review.md and the results before relying on them, for example before discarding paper originals. The software comes without warranty; see the license.

License

MIT