Relative-path persistence for a multi-disk model library — the AI Models Manager design notes

The personal-library problem

If you spend any time generating images with diffusion models on Windows, the models/ folder turns into archaeology surprisingly fast. The pattern repeats across every install: an SSD where the system lives and that’s almost full; a fast NVMe dedicated to AI workloads; an HDD for the overflow. Fifteen sub-folders organised by model type — checkpoints, loras, embeddings, vae, controlnet, upscale_models, ipadapter, clip_vision, and so on. Hundreds of .safetensors files with names like lora_v3_final_FINAL_real.safetensors that meant something six months ago and mean nothing now.

The data exists to make this organised — the platforms that host the models (CivitAI in particular) expose rich metadata, sample images with full generation prompts, version histories, license information. But the data sits over there, behind a browser, while your library sits over here, on disk. Bridging the two without coupling the tool to a particular generator front-end is exactly the gap AI Models Manager fills.

The shape of the app is four top-level tabs (Welcome / Run / Settings / More Tools) with the Run tab subdividing into eight workflow panes: Discover (scan CivitAI by filters, batch-grab metadata), Gallery (browse the images you’ve downloaded, click-through to full metadata), Users (organised by creator), Logs (the real-time log pane), Models (every version and file of the archived models, sent to the cart), Workflows (the generation workflows CivitAI also hosts, shown as cards), Cart (a serial download queue with pause/resume), and Import/Export (the multi-disk collection workflow).

The path-conversion trick that makes it portable

Three kinds of record get persisted to disk as JSON under the data root (the app’s own folder):

• archived_models.json — one entry per model the user has archived, with the metadata snapshot from the CivitAI API plus references to local artifacts. The galleries themselves live in Downloads\Galleries\<type>\<base>\<model>\, one _meta.json per image.
• Downloads\Users\<creator>.json — one file per CivitAI creator the user has grabbed content from, indexing the images and metadata stored locally.
• external_collections.json — collections exported from another disk and imported here. The user sees “you already have this LoRA on the other drive” without having to plug that drive in.

The naive implementation stores absolute paths in those JSONs — “image lives at D:/IA/data/images/foo.png”. It works fine until the user moves D:/IA/data to E:/AI/data (drive reorganisation, NAS migration, switching to a faster SSD). At that point every absolute path in every JSON points at a non-existent location and the library appears empty.

The PathConverter service rewrites every absolute path through two operations:

• ToRelative(absolutePath) strips the current DataPath prefix at serialisation time. The stored value becomes something like .\Downloads\Galleries\…\foo.png — independent of where the root lives on disk today.
• ToAbsolute(relativePath) prepends the current DataPath at deserialisation time. If the user moved the root, the new value is the new absolute path. If the root didn’t move, the value is identical to what was stored before.

One subtlety: there’s a long tail of older JSON files written before the PathConverter existed, with full absolute paths still inside. The ToAbsolute function detects whether the input is already absolute (drive letter or backslash root) and returns it unchanged if that path still exists. If it no longer does, it re-anchors it on its Downloads\, models_cache\ or Users\ segment under the current root — so old entries survive a move too, and get quietly upgraded the first time the user re-saves them.

The root itself is never a setting: at every launch the app resolves it as its own folder (or the folder just above, if the data is there), and Settings only shows it. Moving the library from a 1 TB SSD to a 4 TB SSD means moving the app’s folder — no index to rebuild, nothing to configure. The app doesn’t even ask — it just works.

Cursor pagination, and why the offset alternative wouldn’t cut it

The CivitAI API uses cursor-based pagination: each response includes a metadata.nextCursor token, and the caller passes that token back as a query parameter on the next request. There is no “page 1, page 2, page 3” numbering.

This is the right design for a feed that moves in real time. New models get added to the platform every few minutes; if the API used offset pagination (“give me items 100-199 of the most-downloaded LoRAs”), the same offset would return different items between two requests, and a long scan would either skip new items or return the same one twice depending on which side of the cursor the new arrival landed on. Cursor pagination is stable: the cursor pins a position in the feed, and following it gives you a coherent, monotonic walk.

AI Models Manager’s Discover scans in runs of N pages (Batches) and keeps the cursor of the last completed run in _lastCursor. The Continue button fetches the next N pages from there instead of re-fetching from the top; New Scan starts over. Rate limits and server errors are retried with a back-off of 10 to 60 seconds before a scan gives up. For a long walk through the LoRAs, that’s the difference between “I have to restart from the beginning every time” and “I can go a few pages at a time.”

Thumbnails — async load, disk cache, lazy

A model gallery with 50 000 thumbnails is unusable if every image is fetched synchronously over the network every time the user scrolls. The AsyncImage control wraps WPF’s image binding with a three-step protocol:

1. Compute a key from the image URL. Look in thumbs_cache\<key>.<ext> on disk.
2. If the cached file exists, decode it directly into the bound BitmapImage — instant, no network.
3. If not, fetch the source URL asynchronously, store the bytes in the cache as served, then decode them at the width the view needs and bind the image.

None of this blocks the UI thread. The decode uses BitmapImage.CacheOption = OnLoad so the file handle closes immediately after read (the file can be deleted by the user without WPF complaining). The cache itself lives under thumbs_cache\ in the app’s folder; Clear Cache in Discover empties it. Its size depends on the preview sizes the CDN serves, so it is worth clearing after a long browsing spree.

The deliberately manual composition root

The app instantiates ten services and ten ViewModels at startup. The standard pattern for a project this size is to wire them through a dependency-injection container — Microsoft.Extensions.DependencyInjection or one of the older third-party options. AI Models Manager doesn’t. App.OnStartup is about a hundred and fifteen lines of explicit new statements, in a deliberate order:

ConfigManager → AppConfig → LogService → PathConverter → DataService
  → FolderStatsCache → CivitAIApi → DownloadService → ThumbnailService
  → ImageCache → HuggingFaceApi → QueueManager
  → (splash, thumbs_cache, base-model list synced from CivitAI, 6 s max)
  → 8 sub-VMs → MainViewModel → ShellViewModel → MainWindow

The trade-off is honest. With a DI container, adding a new service is a one-line registration in a setup method. Without, it’s an edit to App.OnStartup in the right position. For a small app — ten ViewModels, ten services — the cost of that edit is negligible compared to the benefits of the explicit version:

• When something fails at startup, you see exactly which line in App.OnStartup threw, in a single global try/catch that MessageBox’es the stack trace and exits cleanly. No “registration order” mystery to debug. After startup, a dispatcher-level handler logs any unexpected error, tells the user and keeps the app alive.
• No assembly scanning, no reflection-based dependency graph walk: the time startup takes is spent loading the user’s collection, behind a splash screen.
• The composition is self-documenting. New contributors can read the file top-to-bottom and see the dependency order.

This pattern doesn’t scale to enterprise-size projects. For a personal-use desktop tool with ten services, it’s the right choice.

Folder stats cache — the 50 000-image performance trap

Counting the files in a folder and adding up their sizes is an O(n) walk. For a folder of fifty thousand images, that’s a few seconds on a slow disk — long enough that switching tabs in the Gallery view, which triggers a re-count of each source, becomes noticeably slow. Worse, the count happens every time the user switches tabs, so the cumulative time across a session is unbounded.

The FolderStatsCache persists per-folder stats (image, video and model-file counts, plus the thumbnails to show) to disk between sessions and only invalidates an entry when the folder’s last-write time changes. The first time a folder is touched, the cost is paid; every subsequent access is instant. The cache is written every 60 seconds when something changed, and once more at MainWindow.Closed — not on every change, which keeps the disk-write rate down on long sessions.

This is the kind of optimisation that doesn’t appear in any feature list, but it’s the difference between “Gallery tab feels snappy” and “Gallery tab freezes for three seconds every time I click it.”

HuggingFace as a mirror, tried first

Many model files published on CivitAI also exist on the HuggingFace Hub, whose storage is fast for large files. When the user has saved a HuggingFace token, the cart first looks for a mirror of each .safetensors file on the Hub, by exact file name among the most-downloaded matching repositories. If one is found, the file comes from there; if not — or if that download fails — the app uses the CivitAI link, and the partial file from the Hub is discarded first so that bytes from two different sources are never mixed.

Either way, a download resumes: the partial .tmp file is kept when a download is cut, paused, stopped or interrupted by closing the app, the transfer continues with an HTTP range request, and a resumed file is checked against its SHA-256 before it is accepted.

Cross-tab navigation by Action delegates

The eight sub-tabs of the Run view need to navigate between each other: from the Gallery, clicking a creator name jumps to Users; from a Users gallery, clicking a model name jumps to Models; from Discover, clicking a model card jumps to Models with that model selected; from Import/Export, importing a collection refreshes the Discover view.

Patterns at hand: an event-aggregator/mediator service; a single ViewModel that owns the navigation state and is referenced by every sub-VM; or direct method calls via Action delegates wired at composition time. AI Models Manager picks the third:

ModelsViewModel.NavigateToGalleryAction = modelId => { ...; SelectedTabIndex = 1; };
DiscoverViewModel.NavigateToModelsAction = modelId => { ...; SelectedTabIndex = 4; };
GalleryViewModel.NavigateToUserGalleryAction = username => { ...; SelectedTabIndex = 2; };
UsersGalleryViewModel.NavigateToModelsAction = modelName => { ...; SelectedTabIndex = 4; };
ImportExportViewModel.OnExternalModelsChanged = () => DiscoverViewModel.ReloadExternalModels();

The MainViewModel composes the sub-VMs and assigns the delegates at construction time. Each navigation is a direct method call; the IDE’s “Find All References” shows the entire navigation graph; adding a new navigation is one assignment in MainViewModel’s constructor. No event subscription state to forget to unsubscribe, no mediator with magic strings.

The Workflows tab, added since, wires in the same way (WorkflowsViewModel.NavigateToGalleryAction), and a delete in Discover refreshes Models and Workflows through one more delegate. For a graph of about nine edges, the trade is still correct. For a more complex graph it’s probably time for a different pattern; for this one, the explicit version reads better.

Supporting the platforms the app relies on

The app deliberately shows two buttons at the top right of its window: “Support Civitai.com” and “Support Hugging Face”, opening each platform’s membership page. The platforms’ public APIs and CDN bandwidth aren’t a free resource — they cost real money to run. A personal-use library tool that relies on them should make the supporting-the-source path easy to find. It’s a small UI choice with a clear intention.

Credits and license

AI Models Manager is original ExpSoft C# / WPF code:

• .NET 8 + WPF — Microsoft, MIT.
• Newtonsoft.Json — James Newton-King, MIT.
• Microsoft.Web.WebView2 — Microsoft, distribution permitted.
• FFmpeg — LGPL-2.1+ / GPL-2.0+. Optional, user-supplied through the FFmpeg Path setting. Used to generate thumbnails for video samples produced by recent video-generation models.

External services the app talks to (with the user’s own access tokens):

• CivitAI — public REST API for browsing and downloading models. The platform’s rate limits and acceptable-use policy apply.
• HuggingFace Hub — public REST API and CDN for mirror downloads.
• nicolas-riquier.com — the public ExpSoft version list, read to tell you when a new version is out.

The model weights downloaded through the app remain under the licenses of their respective authors on CivitAI or HuggingFace. AI Models Manager bundles no model weights and is purely a client / browser / downloader.

Take-aways

• For any tool that persists references to local files, store the references as paths relative to a movable root. The user’s ability to reorganise their storage without rebuilding indexes is worth the small abstraction.
• Cursor pagination is the right choice for feed-style APIs. Keeping the last cursor on the client lets a Continue button fetch the next pages instead of starting over.
• Disk-cache async-loaded thumbnails. For libraries past a few thousand items, the difference is between “smooth” and “unusable.”
• For small desktop apps (single-digit count of services), a deliberately manual composition root in App.OnStartup beats a DI container on readability and failure diagnostics.
• Make the supporting-the-source link prominent. Tools that rely on free public APIs should make it easy to fund those APIs.