The worst moment in moving a creative archive is not the waiting. It’s the moment the import stops at file 31,000 of 40,000 and you have no idea which 31,000 made it.
Everyone who generates at volume eventually faces the same afternoon: years of output sitting in nested folders on an external drive, and a decision about how to get it into something that can actually find it again. That import is the least glamorous part of adopting any catalogue, and it is where most attempts quietly fail.
Not because importing is hard. Because importing is long, and long operations meet reality — a dropped connection, a sleeping laptop, a file that turns out to be corrupt. When a long operation has no memory, every interruption costs you the whole run.
Smart Import now remembers.
Why Big Imports Break
A first import is rarely one clean batch. It is tens of thousands of files of mixed vintage: ComfyUI PNGs with workflows embedded, Midjourney downloads named by the platform, videos from three different tools, and a scattering of files that were never going to parse. Some fraction will fail for perfectly ordinary reasons.
The old failure mode was all-or-nothing. If the run broke, your options were to start again — and risk importing everything that had already landed a second time — or to work out the boundary by hand and import the remainder yourself. Both are bad, and the second is worse, because a partially duplicated library is harder to fix than an empty one.
The problem is not the failure. Failures are normal. The problem is that the import had no record of itself.
What Changed
Smart Import now creates a session record for each import. That record is what makes the difference between a crash and a pause:
- Re-drop the same files and the import picks up where it stopped. The session recognises what has already landed, so re-importing the same folder does not produce a second copy of anything.
- Failed files retry on their own, and you can retry individual files on demand rather than re-running the batch.
- Per-folder rules are remembered, so the structural decisions you made on the way in — how a given folder should be treated — survive the interruption too.
What an interrupted import does now
The practical effect is that a big import stops being an event you have to supervise. You can start it, lose the connection, come back, and drop the same folder again. The catalogue works out what is new. And once the back catalogue is in, a synced folder can pick up new work as it lands.
Duplicates Are the Real Cost
It is worth being precise about why duplicate handling matters more than speed here.
A slow import costs you an afternoon. A duplicated import costs you the trustworthiness of the library.
It costs it lastingly, because from that point on every search returns pairs, collections have ambiguous membership, and counts are wrong. The whole reason to have a catalogue is to be able to ask it a question and believe the answer.
This is the “after creation” problem in miniature. Generating the images was never the hard part. Making them findable afterwards — and keeping them findable as the library grows past the point where you can hold it in your head — is the part that needs infrastructure.
Bringing in an Archive
If you are importing a back catalogue for the first time, a few things make it go better:
- Import in folders that mean something. Folder structure is a signal about how the work is organised, and per-folder rules let you act on it.
- Let the failures fail. Files that do not parse on the first pass are visible afterwards and can be retried individually. Stopping the run to investigate one file is rarely worth it.
- Re-drop rather than reconstruct. If a run is interrupted, the fastest recovery is to drop the same folder again. Working out the boundary by hand is the slower and riskier path.
Key Takeaways
- Large imports fail for ordinary reasons; the design question is what happens next.
- Smart Import keeps a session record, so re-dropping the same files resumes rather than duplicates.
- Individual failed files retry without re-running the batch.
- Per-folder import rules persist across interruptions.
- Duplicate prevention protects something more valuable than time: whether you can trust what the library tells you.