Preservation
Verification isn’t a button, it’s a campaign
Verifying an archive means recomputing file checksums and comparing them to the ones recorded on entry. The operation is simple; what needs method is running it on a real collection, where reading 15 TB from an external drive takes hours.
Hence the notion of a campaign: a scope, a date, a recorded result. A verification with no written trace is worthless, because in a year you won’t know what was checked.
Exhaustive or sampled
Exhaustive: every file. The only way to state the whole collection is sound. Expensive in time and disk reads. Reasonable once a year, or after a move, a hardware change, an electrical incident.
Sampled: a random draw, say 5% of the collection, prioritising the batches checked longest ago. Proves nothing about a given file, but detects systemic degradation — a disk starting to fail shows up statistically. Quarterly or monthly.
The combination that works: frequent sampling for early detection, annual exhaustive for the claim.
Three verdicts, one of which gets forgotten
- Verified. Recomputed checksum identical.
- Failed. Different checksum. The file has changed.
- Unverifiable. The file is unreadable, missing, the volume isn’t mounted, the read was interrupted.
The third case is the problematic one, because many tools treat it as a simple “retry later” and keep no record. That’s a mistake: a file you couldn’t read is a signal, often the first sign of a dying disk. It must appear in the report, distinct from both a success and a failure.
Corollary: only update the last-checked date after a success. Otherwise a failure gets erased by the next attempt.
The decision tree when a checksum no longer matches
First, overwrite nothing. The reflex to “restore from backup” before diagnosis can propagate the problem in the other direction.
- Was the file legitimately modified? Writing metadata into the file changes its bytes, hence its checksum. If your tool writes internally, the checksum should have been updated at the same moment. Test this hypothesis first: it’s the most common cause of false positives.
- Otherwise, compare the copies. Compute the checksum on every available copy. The one matching the original checksum is sound.
- If a copy is sound, restore from it, re-verify, and log the incident with its date and the medium involved.
- If none matches, you have propagated corruption. Find the least damaged copy, document the loss, and go back through your dated versions if you have them.
- In every case, note the source medium. Two failures on the same disk within six months point at the disk, not at chance.
In practice
- Schedule campaigns instead of deciding on them: what depends on remembering doesn’t happen.
- Alternate frequent sampling with an annual exhaustive pass.
- Record all three verdicts, including “unverifiable”.
- Only update the timestamp after a success.
- Diagnose before restoring.
- Track failures by medium, not only by file.
Obscura Flow runs fixity audits in campaigns, explicitly distinguishes failure from unverifiable, fails safe on a computation error rather than concluding success, and only updates the fixity timestamp after a successful audit.