Compare commits

..
Author SHA1 Message Date
vorotamoroz f449292792 Keep clean installation compatible with npm 10 2026-09-17 15:04:28 +00:00
vorotamoroz dee5b69689 Use compatible timers for TURN credential requests
Use compatGlobal and CompatTimeoutHandle for request deadlines. Enforce the community no-global-this rule for integrations and make violations fail quiet lint checks.
2026-09-17 14:26:08 +00:00
vorotamoroz 1044dca94f Merge main into stale-file protection integration 2026-09-17 12:46:32 +00:00
vorotamoroz 63c2811d47 Use published Commonlib 0.1.26 for stale-file protection 2026-09-17 12:45:34 +00:00
vorotamoroz 00abaf2668 Reject direct global access in community lint checks
Make no-global-this an error so quiet validation detects violations of Obsidian window compatibility rules.
2026-09-17 11:06:53 +00:00
vorotamoroz ef99cd8499 Cover stale-file recovery and bulk folder operations
Document exact file revision provenance and add real Obsidian scenarios for stale files after restart, parent-folder renames, and parent-folder deletion. Characterise replication queue delays during repeated writes to one document.

Companion to Commonlib c0a84a4. Validation used its packed artefact; the local file dependency remains uncommitted until a published Commonlib version is available.
2026-09-17 11:06:53 +00:00
13 changed files with 468 additions and 38 deletions
+3
View File
@@ -121,6 +121,9 @@ jobs:
node-version: '24.x'
cache: 'npm'
- name: Verify clean installation with npm 10
run: npx --yes npm@10.9.4 ci --ignore-scripts --no-audit --no-fund
- name: Install dependencies
run: npm ci
+2 -2
View File
@@ -197,9 +197,9 @@ Markdown conflict auto-merge should behave like a conservative three-way merge.
When in doubt, prefer the safer outcome: preserve data, keep the conflict visible, and ask the user rather than silently discarding content or choosing one side.
The detailed contract is documented in [Conflict resolution and revision provenance](docs/specs_conflict_resolution.md). Determine the merge base by intersecting the exact `available` revision IDs from both leaf histories and selecting the nearest shared revision. Do not infer ancestry from revision generation numbers. When a remote resolution reaches a Vault which still contains the exact content of a deleted losing branch, treat that content as known synchronised history so the resolution can be reflected without recreating the conflict.
The detailed contract is documented in [Conflict resolution and revision provenance](docs/specs_conflict_resolution.md). Determine the merge base by intersecting the exact `available` revision IDs from both leaf histories and selecting the nearest shared revision. Do not infer ancestry from revision generation numbers. An unchanged file is recognised by comparing its bytes with its exact device-local file-reflection provenance, including when that revision belongs to a deleted losing branch.
File operations made while a conflict is active must use the device-local file-reflection provenance injected into `ServiceFileHandlerBase`. Treat its exact revision as authoritative; use byte equality only to reconstruct a missing record when exactly one available revision matches. If branch identity remains unknown, preserve data and leave the conflict visible. Do not hide key-value database readiness behind an implicit wait: maintained hosts open it through the sequential settings lifecycle before file events or replication begin.
Ordinary file saves and incoming reflection use that provenance even before a conflict exists. An unchanged stale file must not become a child of the current winner; a genuine edit extends the recorded revision. Without a readable recorded base, compare only current live leaves to avoid duplicate content. Otherwise, preserve the file as a fresh independent root under the same document ID, leaving ancestry unknown. Historical byte equality cannot distinguish an unchanged file from an intentional revert. Explicit reconciliation, deletion, and rename retain their separate contracts. Do not hide key-value database readiness behind an implicit wait: maintained hosts open it through the sequential settings lifecycle before file events or replication begin.
- If one side deletes a line and the other side leaves that same line unchanged, treat it as a safe deletion. The deleted line must not be reintroduced into the merged result.
- If one side inserts new content in a different region while the other side deletes an unchanged old region, preserve the insertion and the deletion.
+25 -15
View File
@@ -28,24 +28,27 @@ The modifiers defined under [Revision](glossary.md#revision) describe independen
| The Vault displays conflict leaf `C` | `W` | `C` | `C` |
| The database advances before Vault reflection | new winner `W2` | previous revision `R`, while the Vault is unchanged | `R` |
| A local edit of displayed revision `R` is pending | independent | none, or a coincidental content match | `R`, as the branch which the edit must extend |
| Provenance is missing and exactly one revision fits | independent | `M` | none, then `M` after safe reconstruction |
| Provenance is missing and exactly one current non-deleted leaf fits | independent | `M` | none, then `M` after safe reconstruction |
| Provenance is missing and several revisions fit | independent | every matching revision | none |
| A logical-deletion winner agrees with an absent file | deleted winner `D` | `D`, and possibly other logical-deletion revisions | none; an absent file retains no displayed provenance |
At most one revision is the winner, more than one revision can be Vault-matching, and at most one revision can be displayed for a path on one device. A displayed revision may stop matching the Vault while a local edit is pending, but its branch identity remains authoritative until that edit is stored or the relationship is safely reconstructed.
## Implemented 1.0 guarantees
## File saving and reflection guarantees
- Automatic text and structured-data merge uses the nearest `available` revision ID which is present in both leaf histories.
- Missing or compacted history stops conservative automatic merge instead of guessing a base.
- A receiving Vault file which exactly matches any available revision in the document tree is treated as previously synchronised content. This includes an ancestor below a deleted losing leaf.
- A receiving Vault file whose bytes do not match any available revision is preserved as an unsynchronised local change.
- A Vault file which still matches its exact recorded revision is unchanged. An ordinary save does not append those stale bytes to a newer database revision; a newer, unconflicted database result is reflected through the existing file-reflection path.
- A file which differs from its readable recorded revision is an edit of that revision, even if its bytes match another historical revision. Saving and incoming overwrite protection use the same rule.
- Without a readable recorded revision, current non-deleted leaves are checked for duplicate content. If none matches, the file is preserved as a fresh independent root under the same document ID. Its unknown ancestry cannot supply a three-way merge base.
- File bytes, rather than path, size, modification time, or revision generation, determine whether content is known.
- Three or more current versions are reviewed one pair at a time in a deterministic order, with each completed pair committed before the next pair is read.
- Each device records the exact revision most recently reflected in each Vault file. An edit, deletion, or case-only rename made while a conflict is active extends that displayed branch rather than the deterministic database winner.
- Each device records the exact revision most recently reflected in each Vault file. An ordinary edit extends that displayed branch even before a conflict exists. Conflict-time deletion and case-only rename retain their separate displayed-branch contracts.
- A cross-path rename stores the target before logically deleting only the displayed source branch.
The all-branch history check prevents a resolved conflict from being recreated merely because the receiving Vault still contains the known losing version. If the user has edited that version again, its bytes differ and the overwrite guard preserves it.
The recorded revision can belong to a deleted losing branch. If its readable body still matches the Vault, the propagated resolution can be reflected without recreating the conflict. A historical byte match without that record does not establish that the file is unchanged: it may be an intentional revert. Existing Vaults can lack records, so an upgrade, reset, or unavailable old body can expose additional conflicts requiring review.
The explicit **Always overwrite with a newer file** option retains its existing modification-time policy. An independent branch prevents an inferred three-way merge; it does not disable the user's selected conflict-resolution option. Metadata and Chunks retain their existing format, and matching chunks can be shared between branches.
## Resolution patterns
@@ -55,8 +58,10 @@ The all-branch history check prevents a resolved conflict from being recreated m
| Text or structured data has an available shared base and non-overlapping changes | Perform a conservative three-way merge. |
| One side deletes content which the other leaves unchanged | Preserve the deletion. |
| One side deletes content which the other modifies | Ask the user. |
| A receiving file matches a revision available anywhere in the tree | Apply the propagated database result. |
| A receiving file matches no available revision | Preserve it and ask the user. |
| A receiving file matches its exact readable recorded revision | Apply the propagated database result under the existing conflict policy. |
| A receiving file differs from its readable recorded revision | Preserve the edit as a child of that exact revision. |
| Provenance is unknown and no current non-deleted leaf matches the file | Preserve a fresh independent branch for conflict resolution. |
| Provenance is unknown and current non-deleted leaves already hold the file bytes | Avoid duplicate storage; infer provenance only for a unique match. |
| A required body or shared ancestor is missing or compacted | Ask the user. |
| Binary contents differ | Prefer an explicit user selection; semantic merge is unavailable. |
@@ -138,13 +143,17 @@ LiveSync composes Commonlib's injected `FileReflectionProvenance` with its local
path -> { revision, observedStorageMtime? }
```
`revision` identifies the exact database revision which most recently produced the displayed Vault file. `observedStorageMtime` is the raw local modification time observed after reflection. It is not rounded, combined with another device's value, or used as proof of branch identity. No content hash is persisted.
`revision` identifies the exact database revision most recently saved from or reflected in this device's Vault. It is the base for subsequent local edits, rather than a certificate that the current file still contains those bytes. `observedStorageMtime` is the raw local modification time of the saved snapshot or the file observed after reflection. It is not rounded, combined with another device's value, or used as proof of branch identity. No content hash is persisted.
The record changes only after a successful database-to-Vault reflection or Vault-to-database write. Reading a file does not change it. The recorded revision remains authoritative even if the user edits the file to bytes which equal another branch; otherwise content equality could silently move the edit to a branch which was not displayed.
Saving and reflection for the same Metadata document run one at a time, including the final provenance update. An ordinary save holds one captured file body and its base until the database write completes. An edit made while that save is running belongs to the next operation; the save does not reread the file to prove that it remained unchanged. Different files retain their existing concurrency limits, and the handler acquires the lock before loading a file body from storage. Conflict checking runs after the lock is released so that an immediate resolution can safely call the file handler again. The host queues count document-lock waiters against their concurrency limits, so a burst for one document can temporarily delay unrelated files.
The common lock does not stop Obsidian edits, external filesystem writes, or replication into the database. Incoming overwrite and deletion protection still checks current storage. Pending events restored at startup retain bounded rechecks because they run before file watching begins and cannot rely on another change notification.
LiveSync creates the namespaced store handle during service composition, before the key-value database is open. The sequential `onSettingLoaded` lifecycle opens that database before Vault scanning, watching, or replication starts. Store operations do not wait for implicit readiness: a lifecycle violation fails promptly, avoiding an indefinite or self-referential initialisation wait. Local database reset is a transient unavailable boundary, after which scanning reconstructs derived state.
When no record exists, LiveSync may reconstruct the displayed revision only if the current Vault bytes match exactly one available revision body. No match, or identical content in multiple revisions, cannot prove branch identity.
For ordinary saves and incoming reflection, a missing or unreadable recorded base permits reconstruction only from exactly one matching current non-deleted leaf. Matching several current leaves avoids duplicate storage but does not identify a displayed branch. No current match creates an independent branch, even when an older ancestor has the same bytes. Deletion and rename retain their existing provenance-recovery contracts.
## Operations while a conflict exists
@@ -231,7 +240,7 @@ If the user renames `draft.md` to `published.md`, LiveSync stores `published.md`
### A remote resolution reaches a device which still shows the losing content
Android may resolve a conflict and continue editing while Mac still shows the losing revision. When Mac receives the resolved tree, LiveSync searches every available branch and recognises Mac's unchanged bytes as content which was already synchronised below the deleted losing leaf. It can apply Android's resolution without asking Mac to resolve the same unchanged conflict again.
Android may resolve a conflict and continue editing while Mac still shows the losing revision. When Mac receives the resolved tree, LiveSync compares Mac's bytes with the exact revision recorded for its Vault. If that body remains readable and matches, it can apply Android's resolution without asking Mac to resolve the same unchanged conflict again.
If the user edited the file on Mac before the resolution arrived, the bytes no longer match that historical revision. LiveSync preserves the Mac edit as an unsynchronised conflict instead of overwriting it.
@@ -243,15 +252,15 @@ The first decision has already changed the ordinary revision tree. On restart, L
### The device-local record is missing
A local-database reset removes revision provenance. On the next scan, if the Vault file matches exactly one available revision, LiveSync can reconstruct which branch was displayed and continue from it. If the bytes match multiple revisions, or no available revision, the branch remains unproved.
A local-database reset removes revision provenance. When an ordinary save or incoming reflection examines the file, exactly one matching current non-deleted leaf can reconstruct the record. Multiple current matches prevent duplicate storage but leave branch identity unproved. A match only in past history is insufficient; differing current content is preserved as an independent branch. An unchanged-time scan alone does not guarantee that a record is created.
In that unproved state, an edit is retained as another manual-resolution branch. A deletion leaves all existing branches intact. A cross-path rename stores the target but leaves every source branch for review. The result can require an extra decision, but it does not discard data by guessing the winner.
If no current non-deleted leaf contains the file bytes, an ordinary save retains them as another independent branch. An unproven deletion leaves all existing branches intact. A cross-path rename stores the target but leaves every unproven source branch for review. The result can require an extra decision, but it does not discard data by guessing the winner.
### Start-up or reset overlaps a provenance operation
LiveSync creates the provenance handle during composition, then opens its backing store during the sequential settings lifecycle before starting scans, watchers, or replication. If the store cannot open, start-up stops rather than leaving file processing waiting indefinitely.
During reset, the store can be temporarily unavailable. A racing provenance lookup fails promptly and follows the same conservative missing-record behaviour. After reopen, scanning can reconstruct a record when one exact revision body matches the Vault file.
During reset, the store can be temporarily unavailable. A racing provenance lookup fails promptly and follows the same conservative missing-record behaviour. After reopen, ordinary saving or reflection can reconstruct a record from a unique matching current non-deleted leaf.
## Unsafe shortcuts
@@ -260,6 +269,7 @@ Do not:
- infer a common ancestor from generation numbers alone;
- assume that the PouchDB winner is the version currently displayed in the Vault;
- replace recorded displayed provenance merely because current bytes match another branch;
- classify a file as unchanged solely because it matches an ancestor somewhere in history;
- discard local content when revision-history lookup fails;
- infer revision identity from path, size, modification time, or content hash without a revision ID;
- select the newest modification time unless the user has explicitly chosen that destructive policy; or
@@ -267,7 +277,7 @@ Do not:
## Verification
Commonlib's real-PouchDB and injected-boundary unit tests cover unequal branch lengths, exact shared ancestry, deterministic ordering of multiple current leaves, a sensible stage followed by reconstruction of a manual pair, content below a deleted losing leaf, recorded and reconstructed branch identity, ambiguous matches, conflict-time editing, missing-body preservation when parent metadata is available, refusal to invent a parent for a generation-one revision, logical deletion, case-only rename, cross-path rename, and safe unproven fallbacks.
Commonlib owns the real-PouchDB and injected-boundary tests for revision ancestry, content preservation, provenance, independent branches, and repeated file events. LiveSync owns persistent host composition and actual Obsidian restart coverage. The focused `test:e2e:obsidian:stale-file-restart` scenario advances the local DB while old Vault bytes remain, persists pending file events, and restarts the same isolated profile. It requires an unchanged recorded file to reflect the DB without a new revision, an unknown file to remain on an independent branch alongside the DB content, and repeated processing after provenance loss to leave those branches unchanged. It uses real local storage and startup processing; transport replication and mobile lifecycle coverage are separate.
LiveSync's optional real-Obsidian two-Vault checks have two scopes. `E2E_OBSIDIAN_INCLUDE_MARKDOWN_CONFLICT=true` resolves and edits a Markdown conflict, propagates it to a Vault which still displays the deleted losing content, and requires one current result to remain. `E2E_OBSIDIAN_INCLUDE_CONFLICT_OPERATIONS=true` edits, deletes, case-renames, and cross-path-renames files while conflicts remain active; it verifies the parent revision of each resulting branch, replicates those exact trees, and confirms that the other conflict branches remain intact.
+2 -7
View File
@@ -52,6 +52,8 @@ export default defineConfig(
"obsidianmd/rule-custom-message": "off",
"no-console": "warn",
"obsidianmd/no-unsupported-api": "error",
// Reject direct globalThis access even when routine checks use --quiet.
"obsidianmd/no-global-this": "error",
// Keep legacy type-safety debt visible while reserving errors for directory-review blockers.
"@typescript-eslint/no-unsafe-argument": "warn",
"@typescript-eslint/no-unsafe-assignment": "warn",
@@ -63,13 +65,6 @@ export default defineConfig(
"@typescript-eslint/no-unnecessary-type-assertion": "warn",
},
},
{
files: ["src/integrations/**/*.ts"],
rules: {
// External-service integrations also run in Node and do not own window UI.
"obsidianmd/no-global-this": "off",
},
},
{
files: ["src/apps/**/*.{ts,js,mjs}"],
rules: {
+4 -4
View File
@@ -23,7 +23,7 @@
"@smithy/types": "^4.14.3",
"@smithy/util-retry": "^4.4.5",
"@vrtmrz/browser-ui-kit": "0.1.0",
"@vrtmrz/livesync-commonlib": "0.1.25",
"@vrtmrz/livesync-commonlib": "0.1.26",
"@vrtmrz/obsidian-plugin-kit": "0.1.4",
"@vrtmrz/ui-interactions": "0.1.2",
"diff-match-patch": "^1.0.5",
@@ -4567,9 +4567,9 @@
}
},
"node_modules/@vrtmrz/livesync-commonlib": {
"version": "0.1.25",
"resolved": "https://registry.npmjs.org/@vrtmrz/livesync-commonlib/-/livesync-commonlib-0.1.25.tgz",
"integrity": "sha512-uWlzcXi32EvrEx6OgKsSEuNQsY+PQDHPY3eq4Xd9W9lHKu2LNh5n1f2z+bMsfZJh1AGMuTkjjFW26X2Bvv/iog==",
"version": "0.1.26",
"resolved": "https://registry.npmjs.org/@vrtmrz/livesync-commonlib/-/livesync-commonlib-0.1.26.tgz",
"integrity": "sha512-AVJky976PP1M+g18im6tJ1eKSq82uN73vkS98WWZWvMJ1UDz6O8YRVWa9dkFakAVXecdxxxLkiD45g5HTRnn6Q==",
"license": "MIT",
"dependencies": {
"@aws-sdk/client-s3": "^3.808.0",
+3 -1
View File
@@ -78,6 +78,8 @@
"test:e2e:obsidian:p2p-connection-check:services": "npm run test:e2e:obsidian:p2p-connection-check -- --manage-p2p",
"test:e2e:obsidian:partial-startup-file-failure": "tsx test/e2e-obsidian/scripts/partial-startup-file-failure.ts",
"test:e2e:obsidian:startup-scan": "tsx test/e2e-obsidian/scripts/startup-scan.ts",
"test:e2e:obsidian:stale-file-restart": "tsx test/e2e-obsidian/scripts/stale-file-restart.ts",
"test:e2e:obsidian:folder-batch": "tsx test/e2e-obsidian/scripts/folder-batch.ts",
"test:e2e:obsidian:setup-uri-workflow": "tsx test/e2e-obsidian/scripts/setup-uri-workflow.ts",
"test:e2e:obsidian:two-vault-sync": "tsx test/e2e-obsidian/scripts/two-vault-sync.ts",
"test:e2e:obsidian:security-seed-reconnect": "tsx test/e2e-obsidian/scripts/security-seed-reconnect.ts",
@@ -181,7 +183,7 @@
"@smithy/types": "^4.14.3",
"@smithy/util-retry": "^4.4.5",
"@vrtmrz/browser-ui-kit": "0.1.0",
"@vrtmrz/livesync-commonlib": "0.1.25",
"@vrtmrz/livesync-commonlib": "0.1.26",
"@vrtmrz/obsidian-plugin-kit": "0.1.4",
"@vrtmrz/ui-interactions": "0.1.2",
"diff-match-patch": "^1.0.5",
@@ -4,6 +4,7 @@ import {
type CloudflareTurnConfiguration,
validateCloudflareTurnConfiguration,
} from "./settings";
import { compatGlobal, type CompatTimeoutHandle } from "@vrtmrz/livesync-commonlib/compat/common/coreEnvFunctions";
/** Fetch-compatible function supplied by the host composition. */
export type CloudflareTurnFetch = (input: string | Request, init?: RequestInit) => Promise<Response>;
@@ -283,9 +284,9 @@ export async function acquireCloudflareTurnCredentials(
requestController.abort();
throw abortError();
}
let timeoutId: ReturnType<typeof setTimeout> | undefined;
let timeoutId: CompatTimeoutHandle | undefined;
const deadline = new Promise<never>((_resolve, reject) => {
timeoutId = globalThis.setTimeout(() => {
timeoutId = compatGlobal.setTimeout(() => {
timedOut = true;
requestController.abort();
reject(credentialFailure("unavailable", true));
@@ -293,7 +294,7 @@ export async function acquireCloudflareTurnCredentials(
});
const cleanup = () => {
if (timeoutId !== undefined) globalThis.clearTimeout(timeoutId);
if (timeoutId !== undefined) compatGlobal.clearTimeout(timeoutId);
signal.removeEventListener("abort", onAbort);
};
@@ -83,6 +83,40 @@ function setup(options: SetupOptions = {}) {
}
describe("ReplicateResultProcessor", () => {
it("resumes another document after in-flight updates to one document fill the application slots", async () => {
const hotGate = promiseWithResolvers<boolean>();
const { processor, processSynchroniseResult } = setup({
processSynchroniseResult: async (entry) => {
if ((entry as { _id: string })._id === "hot-queue") return await hotGate.promise;
return true;
},
});
try {
for (let index = 1; index <= 10; index++) {
// A queued duplicate is coalesced; a new notification for a document
// already being processed can occupy another application slot.
processor.enqueueAll([note("hot-queue")]);
await vi.waitFor(() => expect(processor["_processingChanges"]).toHaveLength(index));
}
processor.enqueueAll([note("unrelated-queue")]);
await vi.waitFor(() => {
expect(processor["_semaphore"].waiting).toBeGreaterThan(0);
expect(processSynchroniseResult).toHaveBeenCalledTimes(1);
});
expect(processor["_queuedChanges"].map((entry) => entry._id)).toEqual(["unrelated-queue"]);
} finally {
hotGate.resolve(true);
await vi.waitFor(() => {
expect(processor["_processingChanges"]).toHaveLength(0);
expect(processor["_queuedChanges"]).toHaveLength(0);
});
}
expect(processSynchroniseResult).toHaveBeenCalledTimes(11);
expect(processSynchroniseResult.mock.calls.some(([entry]) =>
(entry as { _id: string })._id === "unrelated-queue"
)).toBe(true);
});
it("suspends result application while the application is not ready", () => {
const { isReady, processor } = setup({ applicationReady: false });
+6
View File
@@ -75,11 +75,17 @@ After changing plug-in source, use the focused wrapper rather than invoking a sc
```bash
npm run test:e2e:obsidian:focused -- settings-ui
npm run test:e2e:obsidian:focused -- two-vault-sync
npm run test:e2e:obsidian:focused -- stale-file-restart
npm run test:e2e:obsidian:focused -- folder-batch
npm run test:e2e:obsidian:focused -- security-seed-reconnect
```
The wrapper accepts only maintained real-Obsidian scenario names; run it with `--help` for the current list. It deliberately does not manage CouchDB, Object Storage, or the P2P signalling relay. Start the required fixture first, or use the complete service-managed suite.
`folder-batch` needs no remote service. It creates 24 notes in nested folders, renames and deletes the parent through the Obsidian Vault API, and checks descendant events, content, Chunks, deletion markers, and provenance. A note outside the parent must remain writable.
`stale-file-restart` needs no remote service. It advances the local database while old Vault bytes remain, persists pending storage events, and restarts the same isolated Vault and profile. It checks that an unchanged file with exact provenance receives the newer database content without creating a revision, that unknown-origin content is preserved on a fresh independent branch, and that losing provenance and processing the file again does not duplicate or automatically merge that branch. The database advance and pending snapshot are controlled fixtures; startup processing, persistence, file reflection, and conflict checking run in real Obsidian. The scenario does not simulate a mobile operating system suspending the application.
The principal entry points are:
```bash
+167
View File
@@ -0,0 +1,167 @@
import { evalObsidianJson } from "../runner/cli.ts";
import { discoverObsidianCli, requireObsidianBinary } from "../runner/environment.ts";
import { createE2eObsidianDeviceLocalState, waitForLiveSyncCoreReady } from "../runner/liveSyncWorkflow.ts";
import { startObsidianLiveSyncSession, type ObsidianLiveSyncSession } from "../runner/session.ts";
import { createTemporaryVault } from "../runner/vault.ts";
process.env.E2E_OBSIDIAN_CLI_TIMEOUT_MS ??= "60000";
const originalRoot = "batch/original";
const renamedRoot = "batch/renamed";
const outsidePath = "batch/outside.md";
const folders = ["alpha", "alpha/deep", "beta"];
const notes = Array.from({ length: 24 }, (_, index) => ({
relativePath: `${folders[index % folders.length]}/note-${index}.md`,
body: `# Descendant ${index}\n\nThis body must survive a parent folder rename.\n`,
}));
async function main(): Promise<void> {
const binary = requireObsidianBinary();
const cli = discoverObsidianCli();
if (!cli.binary) throw new Error(`Could not find obsidian-cli. Checked: ${cli.checked.join(", ")}`);
const cliBinary = cli.binary;
const vault = await createTemporaryVault("obsidian-livesync-folder-batch-");
let session: ObsidianLiveSyncSession | undefined;
try {
session = await startObsidianLiveSyncSession({
binary,
cliBinary,
vault,
pluginData: {
doctorProcessedVersion: "1.0.0",
isConfigured: true,
liveSync: false,
remoteType: "",
couchDB_URI: "http://127.0.0.1:5984",
couchDB_DBNAME: "folder-batch",
notifyThresholdOfRemoteStorageSize: -1,
periodicReplication: false,
syncOnStart: false,
syncOnSave: false,
syncOnFileOpen: false,
syncOnEditorSave: false,
syncAfterMerge: false,
useEden: false,
},
localStorageEntries: createE2eObsidianDeviceLocalState(vault.name),
});
await waitForLiveSyncCoreReady(cliBinary, session.cliEnv);
const result = await evalObsidianJson<{ descendants: number; renamed: number; deleted: number }>(
cliBinary,
`(async()=>{
const core=app.plugins.plugins['obsidian-livesync'].core;
const provenance=core.services.keyValueDB.openSimpleStore('file-reflection-provenance-v1');
const notes=${JSON.stringify(notes)};
const originalRoot=${JSON.stringify(originalRoot)};
const renamedRoot=${JSON.stringify(renamedRoot)};
const outsidePath=${JSON.stringify(outsidePath)};
const renamed=new Set(), deleted=new Set();
const refs=[
app.vault.on('rename',(file,oldPath)=>{
if(file.stat) renamed.add(oldPath+' -> '+file.path);
}),
app.vault.on('delete',(file)=>{if(file.stat) deleted.add(file.path);}),
];
const meta=(path)=>core.localDatabase.getDBEntryMeta(path,{conflicts:true},true);
const isDeleted=(entry)=>entry && (entry.deleted || entry._deleted);
const getContent=(entry)=>Array.isArray(entry.data)?entry.data.join(''):entry.data;
async function liveErrors(path,body){
const errors=[];
const file=app.vault.getAbstractFileByPath(path);
const entry=await meta(path);
if(!file?.stat || file.path!==path || await app.vault.read(file)!==body)
errors.push('Vault content: '+path);
if(!entry || isDeleted(entry) || entry.path!==path || !entry.children.length){
errors.push('DB metadata: '+path);
}else{
const loaded=await core.localDatabase.getDBEntry(path,{rev:entry._rev},false,true,true);
if(!loaded || getContent(loaded)!==body) errors.push('DB content: '+path);
if(entry._conflicts?.length) errors.push('Unexpected conflict: '+path);
if((await provenance.get(path))?.revision!==entry._rev)
errors.push('Provenance: '+path);
}
return errors;
}
async function deletedErrors(path){
const errors=[];
const entry=await meta(path);
if(app.vault.getAbstractFileByPath(path)) errors.push('File remains: '+path);
if(!isDeleted(entry)) errors.push('Missing tombstone: '+path);
if(entry?._conflicts?.length) errors.push('Deletion conflict: '+path);
if(await provenance.get(path)) errors.push('Old provenance remains: '+path);
return errors;
}
async function waitFor(phase,check){
const deadline=Date.now()+20000;
let errors=[];
do{
await core.services.fileProcessing.commitPendingFileEvents();
errors=await check();
if(!errors.length) return;
await new Promise(resolve=>setTimeout(resolve,50));
}while(Date.now()<deadline);
throw new Error(phase+': '+errors.slice(0,8).join('; '));
}
const liveBatch=(root)=>Promise.all(notes.map(note=>
liveErrors(root+'/'+note.relativePath,note.body))).then(results=>results.flat());
const deletedBatch=(root)=>Promise.all(notes.map(note=>
deletedErrors(root+'/'+note.relativePath))).then(results=>results.flat());
try{
await app.vault.createFolder('batch');
await app.vault.createFolder(originalRoot);
for(const folder of ${JSON.stringify(folders)})
await app.vault.createFolder(originalRoot+'/'+folder);
await Promise.all(notes.map(note=>app.vault.create(originalRoot+'/'+note.relativePath,note.body)));
await app.vault.create(outsidePath,'Outside note');
await waitFor('Initial batch',async()=>[
...await liveBatch(originalRoot), ...await liveErrors(outsidePath,'Outside note'),
]);
const originalIds=await Promise.all(notes.map(async note=>(await meta(originalRoot+'/'+note.relativePath))._id));
// Rename the parent once: Obsidian must emit every descendant event.
await app.vault.rename(app.vault.getAbstractFileByPath(originalRoot),renamedRoot);
await waitFor('Renamed batch',async()=>[
...await liveBatch(renamedRoot), ...await deletedBatch(originalRoot),
...await liveErrors(outsidePath,'Outside note'),
]);
for(const [index,note] of notes.entries()){
const from=originalRoot+'/'+note.relativePath, to=renamedRoot+'/'+note.relativePath;
if(!renamed.has(from+' -> '+to)) throw new Error('Missing descendant rename: '+from);
if((await meta(to))._id===originalIds[index]) throw new Error('Rename reused the source ID: '+to);
}
// Delete the parent once, without synthesising individual file events.
await app.vault.delete(app.vault.getAbstractFileByPath(renamedRoot),true);
await waitFor('Deleted batch',async()=>[
...await deletedBatch(renamedRoot), ...await deletedBatch(originalRoot),
...await liveErrors(outsidePath,'Outside note'),
]);
for(const note of notes){
const path=renamedRoot+'/'+note.relativePath;
if(!deleted.has(path)) throw new Error('Missing descendant deletion: '+path);
}
if(app.vault.getAbstractFileByPath(renamedRoot)) throw new Error('Deleted folder remains');
await app.vault.modify(app.vault.getAbstractFileByPath(outsidePath),'Outside note updated');
await waitFor('Outside update',()=>liveErrors(outsidePath,'Outside note updated'));
return JSON.stringify({descendants:notes.length,renamed:renamed.size,deleted:deleted.size});
}finally{
for(const ref of refs) app.vault.offref(ref);
}
})()`,
session.cliEnv
);
console.log(
`Folder batch: ${result.descendants} descendants persisted, renamed, and deleted; ` +
`${result.renamed} rename and ${result.deleted} delete events observed; outside note remained writable.`
);
} finally {
if (session) await session.app.stop();
await vault.dispose();
}
}
main().catch((error: unknown) => {
console.error(error instanceof Error ? error.stack : error);
process.exitCode = 1;
});
+2
View File
@@ -25,6 +25,8 @@ const focusedScenarios = new Set([
"p2p-setup-uri-workflow",
"partial-startup-file-failure",
"startup-scan",
"stale-file-restart",
"folder-batch",
"setup-uri-workflow",
"two-vault-sync",
"security-seed-reconnect",
@@ -0,0 +1,188 @@
import { evalObsidianJson } from "../runner/cli.ts";
import { discoverObsidianCli, requireObsidianBinary } from "../runner/environment.ts";
import {
assertEqual,
createE2eObsidianDeviceLocalState,
waitForLiveSyncCoreReady,
waitForLocalDatabaseEntry,
} from "../runner/liveSyncWorkflow.ts";
import { startObsidianLiveSyncSession, type ObsidianLiveSyncSession } from "../runner/session.ts";
import { createTemporaryVault } from "../runner/vault.ts";
const paths = ["stale-known.md", "stale-unknown.md"];
const oldContent = "# Note\nKeep\n\nTail\n\nFooter\n";
const newContent = oldContent.replace(
"Footer\n",
Array.from({ length: 50 }, (_, index) => `Remote addition ${index}\n`).join("") + "Footer\n"
);
type Branch = { rev: string; content: string; history: string[] };
type FileState = { path: string; content: string; rev: string; branches: Branch[]; provenance: string | null };
async function readState(cliBinary: string, env: NodeJS.ProcessEnv): Promise<FileState[]> {
return await evalObsidianJson<FileState[]>(
cliBinary,
`(async()=>{
const core=app.plugins.plugins['obsidian-livesync'].core;
const store=core.services.keyValueDB.openSimpleStore('file-reflection-provenance-v1');
const states=[];
for(const path of ${JSON.stringify(paths)}){
const meta=await core.localDatabase.getDBEntryMeta(path,{conflicts:true},true);
const branches=[];
for(const rev of [meta._rev,...(meta._conflicts??[])]){
const entry=await core.localDatabase.getDBEntry(path,{rev,revs:true},false,true,true);
const raw=await core.localDatabase.getRaw(meta._id,{rev,revs:true});
branches.push({rev,content:Array.isArray(entry.data)?entry.data.join(''):entry.data,
history:raw._revisions.ids.map((id,i)=>(raw._revisions.start-i)+'-'+id)});
}
const file=app.vault.getAbstractFileByPath(path);
states.push({path,content:await app.vault.read(file),rev:meta._rev,branches,
provenance:(await store.get(path))?.revision??null});
}
return JSON.stringify(states);
})()`,
env
);
}
async function main(): Promise<void> {
const binary = requireObsidianBinary();
const cli = discoverObsidianCli();
if (!cli.binary) throw new Error(`Could not find obsidian-cli. Checked: ${cli.checked.join(", ")}`);
const cliBinary = cli.binary;
const vault = await createTemporaryVault("obsidian-livesync-stale-file-");
let session: ObsidianLiveSyncSession | undefined;
try {
session = await startObsidianLiveSyncSession({
binary,
cliBinary,
vault,
pluginData: {
doctorProcessedVersion: "1.0.0",
isConfigured: true,
liveSync: false,
remoteType: "",
couchDB_URI: "http://127.0.0.1:5984",
couchDB_DBNAME: "stale-file-restart",
notifyThresholdOfRemoteStorageSize: -1,
periodicReplication: false,
syncAfterMerge: false,
syncOnEditorSave: false,
syncOnFileOpen: false,
syncOnSave: false,
syncOnStart: false,
disableMarkdownAutoMerge: false,
resolveConflictsByNewerFile: false,
checkConflictOnlyOnOpen: true,
showMergeDialogOnlyOnActive: true,
},
localStorageEntries: createE2eObsidianDeviceLocalState(vault.name),
});
await waitForLiveSyncCoreReady(cliBinary, session.cliEnv);
await evalObsidianJson(
cliBinary,
`(async()=>{
for(const path of ${JSON.stringify(paths)}) await app.vault.create(path,${JSON.stringify(oldContent)});
return JSON.stringify(true);
})()`,
session.cliEnv
);
for (const path of paths) await waitForLocalDatabaseEntry(cliBinary, session.cliEnv, path);
// Drain real Vault events before creating a persisted pending-event fixture.
// The DB advances without reflecting it in the Vault, as on an offline device.
const fixture = await evalObsidianJson<{ current: string[]; original: string[] }>(
cliBinary,
`(async()=>{
const core=app.plugins.plugins['obsidian-livesync'].core;
const store=core.services.keyValueDB.openSimpleStore('file-reflection-provenance-v1');
await core.services.fileProcessing.commitPendingFileEvents();
const snapshot=[], current=[], original=[];
for(const [index,path] of ${JSON.stringify(paths)}.entries()){
const meta=await core.localDatabase.getDBEntryMeta(path,{},true);
const file=await core.storageAccess.getFileStub(path);
const data=new Blob([${JSON.stringify(newContent)}],{type:'text/plain'});
const result=await core.localDatabase.putDBEntry({...meta,data,mtime:file.stat.mtime+60000,
size:data.size,children:[]},false,meta._rev);
if(!result?.ok) throw new Error('Could not advance '+path);
current.push(result.rev); original.push(meta._rev);
if(index===0) await store.set(path,{revision:meta._rev,observedStorageMtime:file.stat.mtime});
else await store.delete(path);
snapshot.push({type:'CHANGED',key:'CHANGED-'+path,args:{file}});
}
await core.kvDB.set('storage-event-manager-snapshot',snapshot);
return JSON.stringify({current,original});
})()`,
session.cliEnv
);
await session.app.stop();
session = undefined;
session = await startObsidianLiveSyncSession({ binary, cliBinary, vault });
await waitForLiveSyncCoreReady(cliBinary, session.cliEnv);
const [known, unknown] = await readState(cliBinary, session.cliEnv);
assertEqual(known.rev, fixture.current[0], "An unchanged stale file created a revision during restart.");
assertEqual(known.branches.length, 1, "An unchanged stale file created a conflict.");
assertEqual(known.content, newContent, "The newer DB content was not reflected after suppressing the save.");
assertEqual(known.provenance, fixture.current[0], "The reflected revision was not recorded.");
assertEqual(unknown.branches.length, 2, "Unknown local content was not preserved as a conflict.");
assertEqual(unknown.content, oldContent, "Unknown local content was overwritten.");
const independent = unknown.branches.find((branch) => branch.content === oldContent);
if (!independent) throw new Error("The old local content is missing from the current branches.");
assertEqual(independent.history.length, 1, "Unknown content was attached to an inferred ancestor.");
if (independent.rev === fixture.original[1]) throw new Error("The historical root was reused.");
if (!unknown.branches.some((branch) => branch.content === newContent)) {
throw new Error("The remote additions were lost.");
}
await evalObsidianJson(
cliBinary,
`(async()=>{
const core=app.plugins.plugins['obsidian-livesync'].core;
const path=${JSON.stringify(paths[1])};
await core.services.keyValueDB.openSimpleStore('file-reflection-provenance-v1').delete(path);
if(!await core.fileHandler.storeFileToDB(path)) throw new Error('Repeated save failed');
await app.workspace.getLeaf(false).openFile(app.vault.getAbstractFileByPath(${JSON.stringify(paths[0])}));
await core.services.conflict.resolve(path);
return JSON.stringify(true);
})()`,
session.cliEnv
);
const [, repeated] = await readState(cliBinary, session.cliEnv);
assertEqual(
repeated.branches
.map((branch) => branch.rev)
.sort()
.join(","),
unknown.branches
.map((branch) => branch.rev)
.sort()
.join(","),
"Losing provenance and reprocessing added or auto-merged a branch."
);
await evalObsidianJson(
cliBinary,
`(async()=>{
const core=app.plugins.plugins['obsidian-livesync'].core;
core.settings.resolveConflictsByNewerFile=true;
await core.services.conflict.resolve(${JSON.stringify(paths[1])});
return JSON.stringify(true);
})()`,
session.cliEnv
);
const [, resolved] = await readState(cliBinary, session.cliEnv);
assertEqual(resolved.branches.length, 1, "The explicit newer-file option did not resolve the conflict.");
assertEqual(resolved.content, newContent, "The newer-file option did not reflect the newer DB version.");
console.log(
"Stale-file restart: known content reflected; unknown content preserved without duplicate branches; explicit newer-file resolution retained."
);
} finally {
if (session) await session.app.stop();
await vault.dispose();
}
}
main().catch((error: unknown) => {
console.error(error instanceof Error ? error.stack : error);
process.exitCode = 1;
});
+28 -6
View File
@@ -997,14 +997,26 @@ async function runMarkdownAutoMerge(
session = await startConfiguredSession(context, vaultA, conflictOverrides);
const baseOnA = await waitForLocalDatabaseEntry(context.cliBinary, session.cliEnv, conflictPath);
await storeFileRevision(context.cliBinary, session.cliEnv, conflictPath, left, baseOnA.rev);
await writeVaultFile(vaultA.path, conflictPath, left);
await writeNoteViaObsidian(context.cliBinary, session.cliEnv, conflictPath, left);
const storedLeft = await waitForConflictBranch(
context.cliBinary,
session.cliEnv,
conflictPath,
(branch) => branch.content === left
);
assertEqual(storedLeft.parentRev, baseOnA.rev, "Vault A's edit did not extend its displayed base.");
await pushLocalChanges(context.cliBinary, session.cliEnv);
await stopTrackedSession(context, session);
session = await startConfiguredSession(context, vaultB, conflictOverrides);
await storeFileRevision(context.cliBinary, session.cliEnv, conflictPath, right, baseOnB.rev);
await writeVaultFile(vaultB.path, conflictPath, right);
await writeNoteViaObsidian(context.cliBinary, session.cliEnv, conflictPath, right);
const storedRight = await waitForConflictBranch(
context.cliBinary,
session.cliEnv,
conflictPath,
(branch) => branch.content === right
);
assertEqual(storedRight.parentRev, baseOnB.rev, "Vault B's edit did not extend its displayed base.");
await pushLocalChanges(context.cliBinary, session.cliEnv);
const conflict = await waitForFileConflict(context.cliBinary, session.cliEnv, conflictPath);
const leftBranch = conflict.branches.find((branch) => branch.content === left);
@@ -1028,8 +1040,18 @@ async function runMarkdownAutoMerge(
);
const afterResolution = `${merged.trimEnd()}\n\nPost-resolution edit on B.\n`;
await storeFileRevision(context.cliBinary, session.cliEnv, conflictPath, afterResolution, mergedRev);
await writeVaultFile(vaultB.path, conflictPath, afterResolution);
await writeNoteViaObsidian(context.cliBinary, session.cliEnv, conflictPath, afterResolution);
const storedAfterResolution = await waitForConflictBranch(
context.cliBinary,
session.cliEnv,
conflictPath,
(branch) => branch.content === afterResolution
);
assertEqual(
storedAfterResolution.parentRev,
mergedRev,
"The post-resolution edit did not extend the merged revision."
);
await pushLocalChanges(context.cliBinary, session.cliEnv);
await stopTrackedSession(context, session);