fclones-scheduler

fclones-scheduler

Docker app from cplieger's Repository

Runs the fclones duplicate finder on a schedule and gets back the disk space that duplicate files take.

Requirements

Start with the default action, group, which only reports. link, remove and dedupe modify the files under the scan path, and remove deletes them.
Hardlinks cannot span disks. For the link action, scan a single disk path such as /mnt/disk1/Media rather than /mnt/user/Media, or duplicates that live on two disks fail the run.
The container reports unhealthy until its first scan completes, and on a large share that can take hours. Later scans reuse the hash cache in /cache and are much faster.
Overview
Runs the fclones duplicate finder on a schedule and gets back the disk space that duplicate files take. Scans your share every 3 hours by default, or when your own scheduler asks. Replaces each extra copy with a hardlink or a copy-on-write clone, deletes it, or only reports it, and logs each duplicate it found and the space it freed. The default action only reports, so switch to link, remove or dedupe once a report has shown you what it would touch. No web UI, and no open port unless you turn on metrics.

docker-fclones-scheduler

Image Size Platforms base: Distroless Mutation SBOM

docker-fclones-scheduler runs the fclones duplicate finder on a schedule, in a rootless container, and replaces the copies it finds with hardlinks or removes them. It has no web interface and writes its results to its log.

What it does

docker-fclones-scheduler gets back the disk space that duplicate files take, with no cron job to write:

  • Scans your folders on the interval you set, 3 hours by default, or when your scheduler asks.
  • Replaces each extra copy with a hardlink or a copy-on-write clone, deletes it, or only reports it.
  • Logs each duplicate it found, up to 500 pairs per scan, and the space it freed.
  • Shows in its Docker health status when a scan or its action fails, until the next good scan.

Who it is for

docker-fclones-scheduler is built for a media library or a file share on an always-on Linux machine with Docker, where the same file can land in several folders. It checks every setting before the first scan and, by default, refuses fclones options that run commands. You need a folder the container's user can write to, and hardlinks need the copies on one filesystem, in one volume.

Two other projects suit a different setup:

  • Consider fclones-gui if you want to pick in a desktop window which copies to remove, with the fclones author's interactive frontend.
  • Consider Krokiet if you also want to find similar images, similar videos or music duplicates, in a desktop app.

docker-fclones-scheduler is free software under the Apache-2.0 license.

Quick start

The image is on GitHub Container Registry and on Docker Hub as cplieger/docker-fclones-scheduler, for amd64 and arm64. This is the compose.yaml in this repository.

services:
  fclones:
    image: ghcr.io/cplieger/docker-fclones-scheduler:latest
    container_name: fclones
    restart: unless-stopped
    # Run "sudo mkdir -p /opt/appdata/fclones && sudo chown 1000:1000 /opt/appdata/fclones"
    # before the first start, or the container exits. If .env sets PUID and PGID, use those numbers.
    user: "${PUID:-1000}:${PGID:-1000}"

    environment:
      SCAN_INTERVAL: "1h"  # or 30m, 12h. "off" waits for an outside trigger and "0" scans once
      FCLONES_SCAN_PATHS: "/scandir"  # must match a volume target below
      FCLONES_ARGS: "--rf-over 1"  # report files that have more than one copy
      FCLONES_ACTION: "link"  # group only reports, link replaces copies with hardlinks, remove deletes them, dedupe uses reflinks
      FCLONES_ACTION_ARGS: "--priority bottom"  # keep the first file of each group, the one the log names keeper, and replace the others

    volumes:
      - "/path/to/media:/scandir"  # link, remove and dedupe change files here, so this user must be able to write to it
      - "/opt/appdata/fclones:/cache"  # the container exits at start if this user cannot write here
  1. Create the cache folder and give it to user 1000 with sudo mkdir -p /opt/appdata/fclones && sudo chown 1000:1000 /opt/appdata/fclones. If .env sets PUID and PGID, use those numbers.
  2. Replace /path/to/media with the folder to deduplicate. For link, remove and dedupe, that same user must be able to write to it.
  3. For a first run that changes nothing, set FCLONES_ACTION to "group". The log then lists the duplicates and the space you would get back.
  4. Run docker compose up -d.

Run docker logs fclones. You should see a scan complete line with groups= and duplicate_files= counts. If you see cache directory verification failed, the cache folder does not belong to the container's user, so repeat step 1.

On Unraid, open the Apps tab, search for fclones-scheduler and click Install.

Configuration reference

Settings are environment variables, read once at start, so recreate the container after a change. Configuration covers the three scheduling modes, extra fclones options and a slow first scan.

Variable Description Default
SCAN_INTERVAL Time between scans, such as 30m, 1h or 12h. off waits for an outside trigger, and 0 scans once and exits 3h
FCLONES_SCAN_PATHS Folders inside the container to scan, separated by spaces. Each must be a volume or a folder inside one /scandir
FCLONES_ARGS Extra fclones options for the scan, as flags and their values. --cache, -f and --format are refused (unset)
FCLONES_ACTION group only reports, link makes hardlinks, remove deletes, dedupe makes copy-on-write clones group
FCLONES_ACTION_ARGS Extra fclones options for the action, as flags and their values (unset)
ALLOW_UNSAFE_ARGS true allows --transform, --in-place and --no-copy, which are refused otherwise false
SCAN_TIMEOUT Longest time the scan and the action may each run before the run is stopped and fails. 0 means no limit 12h
LOG_LEVEL debug, info, warn or error. Any other value means info info
METRICS_LISTEN_ADDR Address for a Prometheus metrics page and a health check, for example :9469. Empty opens no port (unset)
Mount Description
/scandir The folder to scan, matching FCLONES_SCAN_PATHS. link, remove and dedupe change files here, so the container's user must be able to write to it
/cache The fclones hash cache and the record of the last scan. The container exits at start when its user cannot write here

The image publishes no port. With METRICS_LISTEN_ADDR set, publish the port it names, such as 9469:9469.

Security

The image opens no port unless you set METRICS_LISTEN_ADDR, which serves only /metrics and /healthz, with no login, so keep that port on your network. Another program can start a scan with docker exec fclones /app/wrapper scan. That command talks to the app through a socket file inside the container, which only the container's own user can open. The container runs as the user that user: names, on a distroless base with no shell. Without a user: line it runs as UID 65532.

FCLONES_ACTION must be one of the four actions. The fclones options that run a command or change a file in place are refused unless ALLOW_UNSAFE_ARGS is true. Leave it false unless you need one of them, such as --transform. Options reach fclones as a list, with no shell to expand them. Security has a hardened compose example and what the image contains.

Troubleshooting

The container is unhealthy when the last scan failed, when its action failed or changed none of the duplicates the scan found, or when fclones' report was unreadable. It is also unhealthy when it could not write to /cache at start. It turns healthy again after the next good scan, with no restart.

With the built-in schedule, the container also turns unhealthy after 30 hours without a good scan, counted from its last good scan or its start, whichever is later. The 30 hours are twice SCAN_INTERVAL plus twice SCAN_TIMEOUT at the defaults, and How it works has the full rules. An unhealthy container notifies nobody. Knowing the app is alive says what does.

  • The container restarts in a loop with cache directory verification failed. The cache folder does not belong to the container's user. Repeat step 1 of the quick start.
  • A run fails with action reclaimed nothing. With the shipped alert rules loaded, FclonesRunFailed fires too. The container's user cannot write to the scanned folder, or, for dedupe, the filesystem has no reflink support.
  • The container exits with positional argument not allowed. Give each pattern its own flag, as in --name '*.mp4' --name '*.mkv'.
  • The container shows unhealthy during a long first scan. Raise start_period, as Configuration shows.

Monitoring

docker-fclones-scheduler logs one run finished record per run and can serve a Prometheus metrics page when METRICS_LISTEN_ADDR is set. The same alerts ship for Prometheus in alerts/promql.yaml and for Loki in alerts/logql.yaml. Load one of the two files. A Grafana dashboard of each run and the space it freed ships as grafana-dashboard.json and needs Grafana 13.2 or newer. The image has a Docker healthcheck, and Knowing the app is alive says what it proves and who is told.

Documentation

  • Configuration covers the scheduling modes, extra fclones options and a slow first scan.
  • How it works explains the schedule, the two phases of a run and the health rules.
  • Monitoring and alerts lists the metrics, the run finished record, both rule files and the dashboard, and says who tells you the schedule stopped.
  • Security has the hardened compose example and what the image contains.

Credits

This project packages fclones (MIT) into a container image. All credit for finding and removing duplicate files goes to the fclones maintainers.

Contributing

See CONTRIBUTING.md.

Disclaimer

This project is built with care and follows security best practices, but it is intended for personal / self-hosted use. No guarantees of fitness for production environments. Use at your own risk.

This project was built with AI assistance. The human maintainer defines architecture, supervises implementation, and makes all final decisions.

License

Apache-2.0. See LICENSE. The image carries the license text of every bundled component under /usr/share/licenses/.

The image packages fclones (MIT) at the version pinned by FCLONES_REF in the Dockerfile: amd64 downloads and unpacks the upstream release tarball https://github.com/pkolaczk/fclones/releases/download/<tag>/fclones-<version>-linux-musl-x86_64.tar.gz, arm64 builds that same tag from source at a pinned commit. The build applies no patches, so this repository's Dockerfile plus the upstream sources it names is the complete recipe for the fclones binary the image ships. The license texts of the Rust crates compiled into the amd64 binary are kept under licenses/crates/ in this repository, regenerated by scripts/vendor-crate-licenses.sh.

Categories

Download Statistics

30,196
Total Downloads
3,477
This Month
3,477
Avg / Month

Total Downloads Over Time

Loading chart...

Related apps

Explore more like this

Explore all

Links

Details

Repository
cplieger/docker-fclones-scheduler:latest
Last Updated2026-11-13
First Seen2026-09-17

Runtime arguments

Network
bridge
Privileged
false
Extra Params
--user 99:100 --health-start-period=6h

Template configuration

Scan pathPathrw

The share or folder to scan for duplicates, for example /mnt/user/Media. link, remove and dedupe need it read-write

Target
/scandir
CachePathrw

The fclones hash cache and the last-scan record, which keep rescans fast and the schedule stable across restarts

Target
/cache
Default
/mnt/user/appdata/fclones-scheduler
Value
/mnt/user/appdata/fclones-scheduler
ActionVariable

What to do with duplicates: group reports only, link replaces them with hardlinks, remove deletes them and dedupe makes copy-on-write clones. Anything but group modifies your files

Target
FCLONES_ACTION
Default
group
Value
group
Scan intervalVariable

Time between scans as a Go duration such as 3h, 12h or 24h. off turns the schedule off so you trigger scans yourself, and 0 runs one scan and exits

Target
SCAN_INTERVAL
Default
3h
Value
3h
Scan pathsVariable

Container paths to scan, separated by spaces. Each needs its own path mapping. Leave at /scandir unless you add more mappings

Target
FCLONES_SCAN_PATHS
Default
/scandir
Value
/scandir
Scan argumentsVariable

Extra flags for the fclones group phase, the scan, such as --name '*.mkv' --name '*.mp4' or --rf-over 1. Flags and their values only

Target
FCLONES_ARGS
Action argumentsVariable

Extra flags for the action phase, such as --priority bottom. --dry-run previews the action without changing files

Target
FCLONES_ACTION_ARGS
Scan timeoutVariable

Timeout for each phase as a Go duration. Raise it when a first scan of a large share exceeds 12h. 0 turns it off

Target
SCAN_TIMEOUT
Default
12h
Value
12h
Log levelVariable

Minimum log level: debug, info, warn or error

Target
LOG_LEVEL
Default
info
Value
info
Metrics addressVariable

Leave empty to open no port. Set to :9469 to serve Prometheus metrics at /metrics and a health check at /healthz, both with no login, then add a Port mapping for 9469 so your scraper or uptime prober can reach it at the server's IP

Target
METRICS_LISTEN_ADDR
Allow unsafe argumentsVariable

Set to true to allow the --transform, --in-place and --no-copy fclones flags, which are refused by default

Target
ALLOW_UNSAFE_ARGS
Default
false
Value
false