Recover missing file provenance with Commonlib 0.1.28

docs/specs_conflict_resolution.md

@@ -145,7 +145,7 @@ path -> { revision, observedStorageMtime? }
 
 `revision` identifies the exact database revision most recently saved from or reflected in this device's Vault. It is the base for subsequent local edits, rather than a certificate that the current file still contains those bytes. `observedStorageMtime` is the raw local modification time of the saved snapshot or the file observed after reflection. It is not rounded, combined with another device's value, or used as proof of branch identity. No content hash is persisted.
 
-The record changes only after a successful database-to-Vault reflection or Vault-to-database write. Reading a file does not change it. The recorded revision remains authoritative even if the user edits the file to bytes which equal another branch; otherwise content equality could silently move the edit to a branch which was not displayed.
+The record changes after a successful database-to-Vault reflection or Vault-to-database write. An equal-time scan may also fill a missing record while the current database revision uniquely matches the file's bytes and no conflict leaf exists. It rereads storage and the current database revision before recording; it does not change file bytes or create a database revision. Ordinary file reads do not change the record. An existing recorded revision remains authoritative even if the user edits the file to bytes which equal another branch; otherwise content equality could silently move the edit to a branch which was not displayed.
 
 Saving and reflection for the same Metadata document run one at a time, including the final provenance update. An ordinary save holds one captured file body and its base until the database write completes. An edit made while that save is running belongs to the next operation; the save does not reread the file to prove that it remained unchanged. Different files retain their existing concurrency limits, and the handler acquires the lock before loading a file body from storage. Conflict checking runs after the lock is released so that an immediate resolution can safely call the file handler again. The host queues count document-lock waiters against their concurrency limits, so a burst for one document can temporarily delay unrelated files.
 
@@ -252,7 +252,7 @@ The first decision has already changed the ordinary revision tree. On restart, L
 
 ### The device-local record is missing
 
-A local-database reset removes revision provenance. When an ordinary save or incoming reflection examines the file, exactly one matching current non-deleted leaf can reconstruct the record. Multiple current matches prevent duplicate storage but leave branch identity unproved. A match only in past history is insufficient; differing current content is preserved as an independent branch. An unchanged-time scan alone does not guarantee that a record is created.
+A local-database reset removes revision provenance. When an ordinary save or incoming reflection examines the file, exactly one matching current non-deleted leaf can reconstruct the record. An equal-time scan can fill a missing record only while the file bytes match the sole current non-deleted revision, the revision tree has no conflict, and the file and database remain unchanged during verification. A failed provenance read is not treated as a missing record. Multiple current matches prevent duplicate storage but leave branch identity unproved. A match only in past history is insufficient; differing current content is preserved as an independent branch. Once the local database has advanced beyond an unrecorded file, the scan cannot infer its earlier origin from historical byte equality.
 
 If no current non-deleted leaf contains the file bytes, an ordinary save retains them as another independent branch. An unproven deletion leaves all existing branches intact. A cross-path rename stores the target but leaves every unproven source branch for review. The result can require an extra decision, but it does not discard data by guessing the winner.
 
@@ -279,7 +279,7 @@ Do not:
 
 LiveSync also exercises three and four independently editing devices through real CouchDB, using the installed Commonlib package and the CLI conflict-resolution command dispatcher. These tests check unchanged losing files before and after resolution, genuine edits on a losing branch, missing provenance, compacted bases, independent-root deduplication, and propagation of the selected result. See the [multiple-device regression procedure and coverage boundaries](../test/README.md#multiple-device-conflict-regression-tests).
 
-Commonlib owns the real-PouchDB and injected-boundary tests for revision ancestry, content preservation, provenance, independent branches, and repeated file events. LiveSync owns persistent host composition and actual Obsidian restart coverage. The focused `test:e2e:obsidian:stale-file-restart` scenario advances the local DB while old Vault bytes remain, persists pending file events, and restarts the same isolated profile. It requires an unchanged recorded file to reflect the DB without a new revision, an unknown file to remain on an independent branch alongside the DB content, and repeated processing after provenance loss to leave those branches unchanged. It uses real local storage and startup processing; transport replication and mobile lifecycle coverage are separate.
+Commonlib owns the real-PouchDB and injected-boundary tests for revision ancestry, content preservation, provenance, independent branches, and repeated file events. LiveSync owns persistent host composition and actual Obsidian restart coverage. The focused `test:e2e:obsidian:stale-file-restart` scenario advances the local DB while old Vault bytes remain, persists pending file events, and restarts the same isolated profile. It requires an unchanged recorded file to reflect the DB without a new revision, an unknown file to remain on an independent branch alongside the DB content, and repeated processing after provenance loss to leave those branches unchanged. It also removes the record for a file whose bytes still match the current database revision, checks that start-up restores the record without a new database revision, and then checks that a later incoming revision reflects without a conflict. It uses real local storage and startup processing; transport replication and mobile lifecycle coverage are separate.
 
 LiveSync's optional real-Obsidian two-Vault checks have two scopes. `E2E_OBSIDIAN_INCLUDE_MARKDOWN_CONFLICT=true` resolves and edits a Markdown conflict, propagates it to a Vault which still displays the deleted losing content, and requires one current result to remain. `E2E_OBSIDIAN_INCLUDE_CONFLICT_OPERATIONS=true` edits, deletes, case-renames, and cross-path-renames files while conflicts remain active; it verifies the parent revision of each resulting branch, replicates those exact trees, and confirms that the other conflict branches remain intact.
 

docs/troubleshooting.md

@@ -47,6 +47,12 @@ Do not switch to P2P or reset the database as the first response. Check:
 
 If the remote is healthy but one device's local database is not, use [Reset Synchronisation on This Device](recovery.md#reset-synchronisation-on-this-device) only after backing up unsynchronised local files.
 
+## An unchanged file appears as a conflict after restart
+
+If the log says `Preserved unsynchronised local changes as a conflict`, keep both versions available until you have checked their contents. The message means that LiveSync could not prove that the Vault file was unchanged; it does not establish that you edited the file. Use **Inspect conflicts and file/database differences** in Hatch to review the current branches before selecting a version.
+
+At start-up, LiveSync can restore a missing device-local revision record when the file still exactly matches the current local database revision. If a newer revision has already arrived, that match may no longer exist. LiveSync then preserves the old file for review because it cannot distinguish an unchanged copy from an intentional edit back to older content. A new release cannot resolve a conflict which was already created merely by recognising a historical content match. If the problem recurs, include the first relevant log entries, plug-in versions on both devices, and a redacted full report with the issue steps.
+
 ## Synchronisation is paused for compatibility review
 
 A compatibility review is separate from the Change Log. It can appear after an internal database or settings-format change, or when a configured Vault is copied, restored, or opened in a new Obsidian profile without its device-local acknowledgement.

package-lock.json

@@ -23,7 +23,7 @@
                 "@smithy/types": "^4.14.3",
                 "@smithy/util-retry": "^4.4.5",
                 "@vrtmrz/browser-ui-kit": "0.1.0",
-                "@vrtmrz/livesync-commonlib": "0.1.27",
+                "@vrtmrz/livesync-commonlib": "0.1.28",
                 "@vrtmrz/obsidian-plugin-kit": "0.1.4",
                 "@vrtmrz/ui-interactions": "0.1.2",
                 "diff-match-patch": "^1.0.5",
@@ -4567,9 +4567,9 @@
             }
         },
         "node_modules/@vrtmrz/livesync-commonlib": {
-            "version": "0.1.27",
-            "resolved": "https://registry.npmjs.org/@vrtmrz/livesync-commonlib/-/livesync-commonlib-0.1.27.tgz",
-            "integrity": "sha512-n/AIqXPGN9Z7zmq0QSKnlE5uY5hTZgXW6VM6NJERR0/ggx3TTv1b3WsSbiLpCeU7Mu6KbPqC2BSjr4jCGvSdCA==",
+            "version": "0.1.28",
+            "resolved": "https://registry.npmjs.org/@vrtmrz/livesync-commonlib/-/livesync-commonlib-0.1.28.tgz",
+            "integrity": "sha512-3NXswTtOU+fE4KRxBWHfkMx9r2Eu9IFH9N/ibj75C5eLh1tFPH7n87bGib6La7Twz9yCDIJRBsBUc2tx81rxOQ==",
             "license": "MIT",
             "dependencies": {
                 "@aws-sdk/client-s3": "^3.808.0",

package.json

@@ -183,7 +183,7 @@
         "@smithy/types": "^4.14.3",
         "@smithy/util-retry": "^4.4.5",
         "@vrtmrz/browser-ui-kit": "0.1.0",
-        "@vrtmrz/livesync-commonlib": "0.1.27",
+        "@vrtmrz/livesync-commonlib": "0.1.28",
         "@vrtmrz/obsidian-plugin-kit": "0.1.4",
         "@vrtmrz/ui-interactions": "0.1.2",
         "diff-match-patch": "^1.0.5",

test/README.md

@@ -122,4 +122,4 @@ Each simulated device owns a separate real PouchDB database, `LiveSyncLocalDB` m
 
 This is a service integration test, not a CLI subprocess or Obsidian runtime test. The file and provenance stores are in-memory fixtures; automatic conflict callbacks are observed without running interactive or automatic merge policies. PouchDB revision creation, chunk storage and retrieval, local compaction, CouchDB replication, the CLI command dispatcher, and its resolution operations are real.
 
-Use the CLI and real-Obsidian procedures linked above for argument parsing, persistent host stores, file watchers, and dialogues. In particular, the real-Obsidian `stale-file-restart` scenario exercises persisted pending events and restart, and `folder-batch` exercises bulk Vault rename and deletion. These scenarios do not simulate a mobile operating system suspending the application.
+Use the CLI and real-Obsidian procedures linked above for argument parsing, persistent host stores, file watchers, and dialogues. In particular, the real-Obsidian `stale-file-restart` scenario exercises persisted pending events and restart. It checks that start-up records an unchanged file whose provenance is missing, then accepts a newer database revision without a conflict, while an unknown historical file remains protected. The `folder-batch` scenario exercises bulk Vault rename and deletion. These scenarios do not simulate a mobile operating system suspending the application.

test/e2e-obsidian/README.md

@@ -84,7 +84,7 @@ The wrapper accepts only maintained real-Obsidian scenario names; run it with `-
 
 `folder-batch` needs no remote service. It creates 24 notes in nested folders, renames and deletes the parent through the Obsidian Vault API, and checks descendant events, content, Chunks, deletion markers, and provenance. A note outside the parent must remain writable.
 
-`stale-file-restart` needs no remote service. It advances the local database while old Vault bytes remain, persists pending storage events, and restarts the same isolated Vault and profile. It checks that an unchanged file with exact provenance receives the newer database content without creating a revision, that unknown-origin content is preserved on a fresh independent branch, and that losing provenance and processing the file again does not duplicate or automatically merge that branch. The database advance and pending snapshot are controlled fixtures; startup processing, persistence, file reflection, and conflict checking run in real Obsidian. The scenario does not simulate a mobile operating system suspending the application.
+`stale-file-restart` needs no remote service. It advances the local database while old Vault bytes remain, persists pending storage events, and restarts the same isolated Vault and profile. It checks that an unchanged file with exact provenance receives the newer database content without creating a revision, that unknown-origin content is preserved on a fresh independent branch, and that losing provenance and processing the file again does not duplicate or automatically merge that branch. A third file begins with no provenance while its bytes still match the current database revision; start-up must record that revision without creating a new one, and a later incoming revision must reflect without a conflict. The database advances and pending snapshot are controlled fixtures; start-up processing, persistence, file reflection, and conflict checking run in real Obsidian. The scenario does not simulate a mobile operating system suspending the application.
 
 The principal entry points are:
 

test/e2e-obsidian/scripts/stale-file-restart.ts

@@ -10,6 +10,7 @@ import { startObsidianLiveSyncSession, type ObsidianLiveSyncSession } from "../r
 import { createTemporaryVault } from "../runner/vault.ts";
 
 const paths = ["stale-known.md", "stale-unknown.md"];
+const recoveredPath = "stale-recovered.md";
 const oldContent = "# Note\nKeep\n\nTail\n\nFooter\n";
 const newContent = oldContent.replace(
     "Footer\n",
@@ -26,7 +27,7 @@ async function readState(cliBinary: string, env: NodeJS.ProcessEnv): Promise<Fil
             const core=app.plugins.plugins['obsidian-livesync'].core;
             const store=core.services.keyValueDB.openSimpleStore('file-reflection-provenance-v1');
             const states=[];
-            for(const path of ${JSON.stringify(paths)}){
+            for(const path of ${JSON.stringify([...paths, recoveredPath])}){
                 const meta=await core.localDatabase.getDBEntryMeta(path,{conflicts:true},true);
                 const branches=[];
                 for(const rev of [meta._rev,...(meta._conflicts??[])]){
@@ -82,16 +83,16 @@ async function main(): Promise<void> {
         await evalObsidianJson(
             cliBinary,
             `(async()=>{
-            for(const path of ${JSON.stringify(paths)}) await app.vault.create(path,${JSON.stringify(oldContent)});
+            for(const path of ${JSON.stringify([...paths, recoveredPath])}) await app.vault.create(path,${JSON.stringify(oldContent)});
             return JSON.stringify(true);
         })()`,
             session.cliEnv
         );
-        for (const path of paths) await waitForLocalDatabaseEntry(cliBinary, session.cliEnv, path);
+        for (const path of [...paths, recoveredPath]) await waitForLocalDatabaseEntry(cliBinary, session.cliEnv, path);
 
         // Drain real Vault events before creating a persisted pending-event fixture.
         // The DB advances without reflecting it in the Vault, as on an offline device.
-        const fixture = await evalObsidianJson<{ current: string[]; original: string[] }>(
+        const fixture = await evalObsidianJson<{ current: string[]; original: string[]; recovered: string }>(
             cliBinary,
             `(async()=>{
             const core=app.plugins.plugins['obsidian-livesync'].core;
@@ -110,8 +111,11 @@ async function main(): Promise<void> {
                 else await store.delete(path);
                 snapshot.push({type:'CHANGED',key:'CHANGED-'+path,args:{file}});
             }
+            const recovered=${JSON.stringify(recoveredPath)};
+            const recoveredMeta=await core.localDatabase.getDBEntryMeta(recovered,{},true);
+            await store.delete(recovered);
             await core.kvDB.set('storage-event-manager-snapshot',snapshot);
-            return JSON.stringify({current,original});
+            return JSON.stringify({current,original,recovered:recoveredMeta._rev});
         })()`,
             session.cliEnv
         );
@@ -120,7 +124,7 @@ async function main(): Promise<void> {
 
         session = await startObsidianLiveSyncSession({ binary, cliBinary, vault });
         await waitForLiveSyncCoreReady(cliBinary, session.cliEnv);
-        const [known, unknown] = await readState(cliBinary, session.cliEnv);
+        const [known, unknown, recovered] = await readState(cliBinary, session.cliEnv);
         assertEqual(known.rev, fixture.current[0], "An unchanged stale file created a revision during restart.");
         assertEqual(known.branches.length, 1, "An unchanged stale file created a conflict.");
         assertEqual(known.content, newContent, "The newer DB content was not reflected after suppressing the save.");
@@ -134,6 +138,32 @@ async function main(): Promise<void> {
         if (!unknown.branches.some((branch) => branch.content === newContent)) {
             throw new Error("The remote additions were lost.");
         }
+        assertEqual(recovered.rev, fixture.recovered, "Unchanged recovery added a database revision.");
+        assertEqual(recovered.branches.length, 1, "Unchanged recovery created a conflict.");
+        assertEqual(recovered.content, oldContent, "Unchanged recovery modified the Vault file.");
+        assertEqual(recovered.provenance, fixture.recovered, "Start-up did not recover missing file provenance.");
+
+        const receivedRevision = await evalObsidianJson<string>(
+            cliBinary,
+            `(async()=>{
+                const core=app.plugins.plugins['obsidian-livesync'].core;
+                const path=${JSON.stringify(recoveredPath)};
+                const meta=await core.localDatabase.getDBEntryMeta(path,{},true);
+                const file=await core.storageAccess.getFileStub(path);
+                const data=new Blob([${JSON.stringify(newContent)}],{type:'text/plain'});
+                const result=await core.localDatabase.putDBEntry({...meta,data,mtime:file.stat.mtime+60000,
+                    size:data.size,children:[]},false,meta._rev);
+                if(!result?.ok) throw new Error('Could not advance '+path);
+                if(!await core.fileHandler.dbToStorage(path,path,false)) throw new Error('Could not reflect '+path);
+                return JSON.stringify(result.rev);
+            })()`,
+            session.cliEnv
+        );
+        const [, , received] = await readState(cliBinary, session.cliEnv);
+        assertEqual(received.rev, receivedRevision, "Incoming revision was not retained.");
+        assertEqual(received.branches.length, 1, "Incoming revision created a conflict after recovery.");
+        assertEqual(received.content, newContent, "Incoming revision was not reflected after recovery.");
+        assertEqual(received.provenance, receivedRevision, "Incoming reflection did not record its revision.");
 
         await evalObsidianJson(
             cliBinary,
@@ -174,7 +204,7 @@ async function main(): Promise<void> {
         assertEqual(resolved.branches.length, 1, "The explicit newer-file option did not resolve the conflict.");
         assertEqual(resolved.content, newContent, "The newer-file option did not reflect the newer DB version.");
         console.log(
-            "Stale-file restart: known content reflected; unknown content preserved without duplicate branches; explicit newer-file resolution retained."
+            "Stale-file restart: missing unchanged provenance recovered; later update reflected; unknown content preserved; explicit newer-file resolution retained."
         );
     } finally {
         if (session) await session.app.stop();

updates.md

@@ -12,6 +12,12 @@ Earlier releases remain available in the 1.0 release history, the 1.0 preview hi
 
 ## Unreleased
 
+### Synchronisation
+
+#### Fixed
+
+- On start-up, an unchanged file with a missing local revision record can be recognised before newer content arrives, avoiding an unnecessary conflict. Files with actual local edits still require conflict review. (#1207)
+
 ## 1.0.30
 
 18th September, 2026