Backup and restore

What a snapshot holds, where backups go, how often and how many are kept, why one was taken, and how a restore is requested and applied at the next start.

Smith snapshots what it keeps on this machine into a dated folder, prunes old ones, and restores one. The backup block of desk.json sets the folder, the interval and the count; those keys are on the desk.json page and are not listed again here: backup, backup.dir, backup.every_hours and backup.keep. Windows behaviour is from code only.

01what is backed up#

backup.Parts is the one list of what smith keeps. A part with an empty path is left out; one missing on disk is recorded as absent.

part	path	copied as
`tickets-files`	`<data dir>/tickets`	tree
`tickets`	`<data dir>/tickets/tickets.db`	sqlite
`memory`	`conductor.memory_dir`, else `<data dir>/memory`	tree
`memory-projects`	`conductor.project_memory_dir`, else `<data dir>/memory-projects`	tree
`assets`	`<data dir>/assets`	tree
`conversations`	`<project>/.smith/desk/conversations`	tree
`desk.json`	`<project>/.smith/desk/desk.json`	file
`harness.json`	the harness config path, when there is one	file
`projects`	`<project>/.smith/desk/projects`	tree
`roles`	the user roles folder, `<lib>/roles`	tree
`runs`	`<data dir>/runs`	tree

A sqlite database is only ever copied with VACUUM INTO, never raw, because a live db's -wal and -shm files would be torn. The tree is listed before the db on purpose: a restore goes in list order, and restoring the tree moves the whole folder aside, so the db is then copied into place.

02defaults#

From newBackupRunner:

Folder: ~/smith-backups, outside the data dir, when backup.dir is empty. A mounted share works.
Interval: every 24 h when backup.every_hours is unset or 0. A negative value turns the schedule off. Backups before a sync restart and by hand still happen.
Kept: 7 scheduled snapshots when backup.keep is unset or not above 0.
Snapshots that were not scheduled are kept apart from those, 3 of them (DefaultKeepOther; OthersKept).

A fact test pins these (internal/cli/kb_backup_facts_test.go).

03why a snapshot was taken#

Each manifest records a reason:

scheduled: the interval's own snapshot. An older snapshot with no reason counts as scheduled.
before sync restart: taken before a sync restarts smith.
by hand: the settings press.

Pruning keeps the newest keep scheduled snapshots and, separately, the newest 3 of the others (Prune).

04status.json#

<dir>/status.json holds the last backup's time, result, reason, snapshot name and size, plus the last restore's result (Status). It is written through a temp file and renamed. A failed backup is tried again after an hour when that is sooner than the interval (retryAfter). An overdue backup, as when smith was off at the time, is taken at once when it starts.

Each snapshot is a folder named by its UTC time, with a manifest.json that lists the parts, each file's size and its sha256.

05restore#

A restore is requested, then applied at the next start, before any store opens.

RequestRestore writes restore.json in the backups folder naming the snapshot. A snapshot taken by a newer smith than the one running is refused and nothing is written (CheckVersion). It only refuses when both versions are releases like 1.2.3; an empty, dev or git-sha version is accepted.
At start, applyPendingRestore calls backup.ApplyPending before any store is opened. The request is removed first, so a restore that fails is not tried at every start.
Restore checks the snapshot's files against its manifest before anything moves. Each part's current copy moves aside to <path>.pre-restore-<stamp>. If anything fails, everything is put back; when all succeed the aside copies are removed.
The result is kept in status.json under restore. A failed restore has put everything back, and the start goes on.

On a fresh install the setup page can pick the folder and restore from it: see restore from a backup.

06the tickets-db copy#

The tickets sqlite db also has its own snapshot runner. Its folder is <backups dir>/tickets-db, and its interval and count are the backup block's (newTicketSnapshotRunner). A pending tickets restore is applied before any store opens, with its own message (applyPendingTicketsRestore). Nothing opens a sqlite store in the desk yet, so the db file is usually absent (a comment in the code; whether that is still true is not checked). Before an update the shell asks for a snapshot of it too: see updates.

07windows rename retries#

On Windows a rename can fail briefly with access denied or a sharing violation while an antivirus scanner or the indexer holds a handle. renameRetrying then retries with a bounded backoff, starting at 50 ms and doubling to a 500 ms cap, within a total sleep budget of 2 s, and returns the first error if the budget runs out (; transient errors). Elsewhere it makes one attempt (rename_other.go).