Compare commits

..
5 Commits
Author SHA1 Message Date
vorotamoroz 0789e47c17 Merge pull request #1173 from vrtmrz/fix/live-vault-reflection-failure-notice
Warn immediately when live Vault reflection fails
2026-09-05 22:08:20 +09:00
vorotamoroz bbbd6fb174 Merge pull request #1174 from vrtmrz/fix/prevent-stale-file-deletions
Prevent stale file deletions after parent case changes
2026-09-05 17:02:50 +09:00
vorotamoroz 14a133588d Use Commonlib 0.1.23 for stale deletion protection 2026-09-05 06:19:47 +00:00
vorotamoroz 7110b9eebf Add parent case deletion regression coverage 2026-09-05 03:38:12 +00:00
vorotamoroz e018cab039 Warn when live Vault reflection fails 2026-09-05 02:10:52 +00:00
14 changed files with 538 additions and 48 deletions
+5
View File
@@ -133,6 +133,11 @@ The [Project glossary](docs/glossary.md#developer-and-design-terms) defines the
stable developer and design vocabulary used in this section. The guidance
below describes how those boundaries are applied.
For file-event admission versus physical Vault writes, see
[File events and storage writes](docs/tech_info.md#file-events-and-storage-writes)
and its linked Commonlib contract. Keep regression coverage for those two
directions separate when changing deletion handling.
### Service composition and legacy Modules
The application is composed from Services, ServiceModules, serviceFeatures, add-ons, and a legacy Module layer:
+20
View File
@@ -11,6 +11,26 @@
Note: The figure is drawn as single-directional, between two devices for demonstration purposes. Everything actually occurs bi-directionally between many devices at the same time.
## File events and storage writes
File events describe changes observed in the Vault. Commonlib filters and
serialises those events before updating file Metadata in the local database.
A queued `DELETE` therefore requests a database change; it is not itself an
instruction to delete the physical file. A rename out of the selected files
can also become a database deletion while the destination remains on disk.
The opposite direction starts with database Metadata. Replicated changes and
full scans can call the database-to-storage handler, which writes or removes
Vault files subject to its conflict and content-preservation rules. Preventing
a stale file event from deleting Metadata and applying a valid replicated
deletion are separate decisions.
Commonlib's [Storage events and database-to-storage reflection](https://github.com/vrtmrz/livesync-commonlib/blob/main/docs/storage-events-and-reflection.md)
documents the event boundary, the deletion revalidation introduced in
Commonlib 0.1.23, and its limits. In particular, deletion protection does not
promise full support for external folder case changes or convergence of path
spelling.
## Current technical references
- [Database Data Structures](datastructure.md) describes current Metadata and
+1 -1
View File
@@ -1,7 +1,7 @@
{
"id": "obsidian-livesync",
"name": "Self-hosted LiveSync",
"version": "1.0.25",
"version": "1.0.24",
"minAppVersion": "1.7.2",
"description": "Community implementation of self-hosted livesync. Reflect your vault changes to some other devices immediately. Please make sure to disable other synchronize solutions to avoid content corruption or duplication.",
"author": "vorotamoroz",
+12 -12
View File
@@ -1,12 +1,12 @@
{
"name": "obsidian-livesync",
"version": "1.0.25",
"version": "1.0.24",
"lockfileVersion": 3,
"requires": true,
"packages": {
"": {
"name": "obsidian-livesync",
"version": "1.0.25",
"version": "1.0.24",
"license": "MIT",
"workspaces": [
"src/apps/cli",
@@ -23,7 +23,7 @@
"@smithy/types": "^4.14.3",
"@smithy/util-retry": "^4.4.5",
"@vrtmrz/browser-ui-kit": "0.1.0",
"@vrtmrz/livesync-commonlib": "0.1.22",
"@vrtmrz/livesync-commonlib": "0.1.23",
"@vrtmrz/obsidian-plugin-kit": "0.1.4",
"@vrtmrz/ui-interactions": "0.1.2",
"diff-match-patch": "^1.0.5",
@@ -4620,9 +4620,9 @@
}
},
"node_modules/@vrtmrz/livesync-commonlib": {
"version": "0.1.22",
"resolved": "https://registry.npmjs.org/@vrtmrz/livesync-commonlib/-/livesync-commonlib-0.1.22.tgz",
"integrity": "sha512-8TsFo6xgEO/uZzkQ4TE3yydUyK8pCbuMm0C4DC/8KhG8z06N6hQQmwR7bV+a3Zgt9A5tXPImTFJWTMUIxJYV2g==",
"version": "0.1.23",
"resolved": "https://registry.npmjs.org/@vrtmrz/livesync-commonlib/-/livesync-commonlib-0.1.23.tgz",
"integrity": "sha512-hsaz2N04qNqM9HL0B+d5G/do1T0fe6Y4gVK3IueXvEnM6HM+3Jp17mdjbLn93FUOxkUVMcn4M+zIwPuppryVbw==",
"license": "MIT",
"dependencies": {
"@aws-sdk/client-s3": "^3.808.0",
@@ -12937,11 +12937,11 @@
},
"src/apps/cli": {
"name": "self-hosted-livesync-cli",
"version": "1.0.25-cli",
"version": "1.0.24-cli",
"dependencies": {
"chokidar": "^4.0.0",
"minimatch": "^10.2.5",
"octagonal-wheels": "^0.1.54",
"octagonal-wheels": "^0.1.53",
"pouchdb-adapter-http": "^9.0.0",
"pouchdb-adapter-leveldb": "^9.0.0",
"pouchdb-core": "^9.0.0",
@@ -12962,9 +12962,9 @@
},
"src/apps/webapp": {
"name": "livesync-webapp",
"version": "1.0.25-webapp",
"version": "1.0.24-webapp",
"dependencies": {
"octagonal-wheels": "^0.1.54"
"octagonal-wheels": "^0.1.53"
},
"devDependencies": {
"@sveltejs/vite-plugin-svelte": "^7.1.2",
@@ -12974,9 +12974,9 @@
}
},
"src/apps/webpeer": {
"version": "1.0.25-webpeer",
"version": "1.0.24-webpeer",
"dependencies": {
"octagonal-wheels": "^0.1.54"
"octagonal-wheels": "^0.1.53"
},
"devDependencies": {
"@sveltejs/vite-plugin-svelte": "^7.1.2",
+2 -2
View File
@@ -1,6 +1,6 @@
{
"name": "obsidian-livesync",
"version": "1.0.25",
"version": "1.0.24",
"description": "Reflect your vault changes to some other devices immediately. Please make sure to disable other synchronize solutions to avoid content corruption or duplication.",
"main": "main.js",
"type": "module",
@@ -178,7 +178,7 @@
"@smithy/types": "^4.14.3",
"@smithy/util-retry": "^4.4.5",
"@vrtmrz/browser-ui-kit": "0.1.0",
"@vrtmrz/livesync-commonlib": "0.1.22",
"@vrtmrz/livesync-commonlib": "0.1.23",
"@vrtmrz/obsidian-plugin-kit": "0.1.4",
"@vrtmrz/ui-interactions": "0.1.2",
"diff-match-patch": "^1.0.5",
+2 -2
View File
@@ -1,7 +1,7 @@
{
"name": "self-hosted-livesync-cli",
"private": true,
"version": "1.0.25-cli",
"version": "1.0.24-cli",
"main": "dist/index.cjs",
"type": "module",
"scripts": {
@@ -37,7 +37,7 @@
"dependencies": {
"chokidar": "^4.0.0",
"minimatch": "^10.2.5",
"octagonal-wheels": "^0.1.54",
"octagonal-wheels": "^0.1.53",
"pouchdb-adapter-http": "^9.0.0",
"pouchdb-adapter-leveldb": "^9.0.0",
"pouchdb-core": "^9.0.0",
+2 -2
View File
@@ -1,7 +1,7 @@
{
"name": "livesync-webapp",
"private": true,
"version": "1.0.25-webapp",
"version": "1.0.24-webapp",
"type": "module",
"description": "Browser-based Self-hosted LiveSync using FileSystem API",
"scripts": {
@@ -15,7 +15,7 @@
"test:browser": "deno test -A --no-check --frozen --config ../../../test/browser-apps/deno.json --lock ../../../test/browser-apps/deno.lock ../../../test/browser-apps/webapp/browser-smoke.test.ts"
},
"dependencies": {
"octagonal-wheels": "^0.1.54"
"octagonal-wheels": "^0.1.53"
},
"devDependencies": {
"@sveltejs/vite-plugin-svelte": "^7.1.2",
+2 -2
View File
@@ -1,7 +1,7 @@
{
"name": "webpeer",
"private": true,
"version": "1.0.25-webpeer",
"version": "1.0.24-webpeer",
"type": "module",
"scripts": {
"dev": "vite",
@@ -15,7 +15,7 @@
"test:browser": "deno test -A --no-check --frozen --config ../../../test/browser-apps/deno.json --lock ../../../test/browser-apps/deno.lock ../../../test/browser-apps/webpeer/browser-smoke.test.ts"
},
"dependencies": {
"octagonal-wheels": "^0.1.54"
"octagonal-wheels": "^0.1.53"
},
"devDependencies": {
"eslint-plugin-svelte": "^3.19.0",
@@ -25,6 +25,7 @@ import type { LiveSyncBaseCore } from "@/LiveSyncBaseCore";
import { isNotFoundError } from "@vrtmrz/livesync-commonlib/compat/common/utils.doc";
import type PouchDB from "pouchdb-core";
import { promiseWithResolvers, type PromiseWithResolvers } from "octagonal-wheels/promises";
import { $msg } from "@/common/translation";
const KV_KEY_REPLICATION_RESULT_PROCESSOR_SNAPSHOT = "replicationResultProcessorSnapshot";
const REPROCESS_BATCH_SIZE = 100;
@@ -79,6 +80,14 @@ export class ReplicateResultProcessor {
private logError(e: unknown) {
Logger(e, LOG_LEVEL_VERBOSE);
}
private reportVaultReflectionFailure(entry: MetaEntry, cause?: unknown) {
this.log(
`Live replication could not reflect ${this.getPath(entry)} from the local database to the Vault; this path remains eligible for a later Vault scan.`,
LOG_LEVEL_VERBOSE
);
if (cause !== undefined) this.logError(cause);
Logger($msg("Ui.Common.SomeFilesCouldNotBeSynchronised"), LOG_LEVEL_NOTICE);
}
constructor(private readonly context: ReplicateResultProcessorContext) {}
private get localDatabase() {
@@ -510,8 +519,16 @@ export class ReplicateResultProcessor {
this.log(`Processed by other processor: ${docNote}`, LOG_LEVEL_DEBUG);
} else if (this.services.vault.isValidPath(this.getPath(doc))) {
// Apply to storage if the path is valid
await this.applyToStorage(doc as MetaEntry);
this.log(`Processed: ${docNote}`, LOG_LEVEL_DEBUG);
try {
const reflected = await this.applyToStorage(doc as MetaEntry);
if (!reflected) {
this.reportVaultReflectionFailure(doc as MetaEntry);
return;
}
this.log(`Processed: ${docNote}`, LOG_LEVEL_DEBUG);
} catch (error) {
this.reportVaultReflectionFailure(doc as MetaEntry, error);
}
} else {
// Should process, but have an invalid path
this.log(`Unprocessed (Invalid path): ${docNote}`, LOG_LEVEL_VERBOSE);
@@ -525,9 +542,10 @@ export class ReplicateResultProcessor {
* @returns
*/
protected applyToStorage(entry: MetaEntry) {
return this.withCounting(async () => {
await this.services.replication.processSynchroniseResult(entry);
}, this.services.replication.storageApplyingCount);
return this.withCounting(
() => this.services.replication.processSynchroniseResult(entry),
this.services.replication.storageApplyingCount
);
}
/**
@@ -2,6 +2,13 @@ import { promiseWithResolvers } from "octagonal-wheels/promises";
import { reactiveSource } from "octagonal-wheels/dataobject/reactive";
import { describe, expect, it, vi } from "vitest";
import { VER, type EntryDoc } from "@vrtmrz/livesync-commonlib/compat/common/types";
import {
defaultLogger,
LOG_LEVEL_DEBUG,
LOG_LEVEL_NOTICE,
LOG_LEVEL_VERBOSE,
setGlobalLogFunction,
} from "octagonal-wheels/common/logger";
import { ReplicateResultProcessor } from "./ReplicateResultProcessor";
function note(id: string): PouchDB.Core.ExistingDocument<EntryDoc> {
@@ -21,12 +28,12 @@ function note(id: string): PouchDB.Core.ExistingDocument<EntryDoc> {
type SetupOptions = {
applicationReady?: boolean;
processSynchroniseResult?: (entry: unknown) => Promise<void>;
processSynchroniseResult?: (entry: unknown) => Promise<boolean>;
setSnapshot?: (key: string, value: unknown) => Promise<unknown>;
};
function setup(options: SetupOptions = {}) {
const processSynchroniseResult = vi.fn(options.processSynchroniseResult ?? (async () => undefined));
const processSynchroniseResult = vi.fn(options.processSynchroniseResult ?? (async () => true));
const setSnapshot = vi.fn(options.setSnapshot ?? (async () => undefined));
const runBoundedLocalApplicationActivity = vi.fn(async (task: () => Promise<void>) => await task());
const onCloseActiveReplication = vi.fn(async () => true);
@@ -120,7 +127,7 @@ describe("ReplicateResultProcessor", () => {
});
it("keeps one local application activity until every replicated document has been applied", async () => {
const applying = promiseWithResolvers<void>();
const applying = promiseWithResolvers<boolean>();
let activityFinished = false;
const { processor, processSynchroniseResult, runBoundedLocalApplicationActivity } = setup({
processSynchroniseResult: async () => applying.promise,
@@ -139,7 +146,7 @@ describe("ReplicateResultProcessor", () => {
});
expect(activityFinished).toBe(false);
applying.resolve();
applying.resolve(true);
await vi.waitFor(() => expect(activityFinished).toBe(true));
});
@@ -160,7 +167,7 @@ describe("ReplicateResultProcessor", () => {
});
it("releases and reacquires local application activity around processing suspension", async () => {
const applying = promiseWithResolvers<void>();
const applying = promiseWithResolvers<boolean>();
let completedActivities = 0;
const { processor, processSynchroniseResult, runBoundedLocalApplicationActivity } = setup({
processSynchroniseResult: async () => applying.promise,
@@ -178,7 +185,47 @@ describe("ReplicateResultProcessor", () => {
processor.resume();
await vi.waitFor(() => expect(runBoundedLocalApplicationActivity).toHaveBeenCalledTimes(2));
applying.resolve();
applying.resolve(true);
await vi.waitFor(() => expect(completedActivities).toBe(2));
});
it.each([
["returns false", async () => false, undefined],
["throws", async () => Promise.reject(new Error("File name too long")), "File name too long"],
])("reports when Vault reflection %s", async (_description, processSynchroniseResult, errorMessage) => {
const log = vi.fn((_message: unknown, _level?: number) => undefined);
setGlobalLogFunction(log);
try {
const { processor } = setup({ processSynchroniseResult });
processor.enqueueAll([note("unreflectable")]);
await vi.waitFor(() =>
expect(log).toHaveBeenCalledWith(
"Not all files could be synchronised. Check the affected files. Generate a report to review the detailed log.",
LOG_LEVEL_NOTICE,
undefined
)
);
expect(log).toHaveBeenCalledWith(
"[ReplicateResultProcessor] Live replication could not reflect unreflectable.md from the local database to the Vault; this path remains eligible for a later Vault scan.",
LOG_LEVEL_VERBOSE,
undefined
);
if (errorMessage !== undefined) {
expect(log).toHaveBeenCalledWith(
expect.objectContaining({ message: errorMessage }),
LOG_LEVEL_VERBOSE,
undefined
);
}
expect(log).not.toHaveBeenCalledWith(
expect.stringContaining("Processed: unreflectable.md"),
LOG_LEVEL_DEBUG,
undefined
);
} finally {
setGlobalLogFunction(defaultLogger);
}
});
});
+6 -1
View File
@@ -170,7 +170,11 @@ LIVESYNC_CLI_COMMAND="docker run --rm --network host --user $(id -u):$(id -g) --
`test:e2e:obsidian:setup-uri-workflow` runs the repository's public Commonlib-backed CouchDB provisioning and Setup URI tools against the local CouchDB fixture. It configures a new, empty Vault in the first real Obsidian session through the visible onboarding wizard and uses Rebuild. After that device is working, it generates a new Setup URI through the registered command; the second real Obsidian Vault uses that URI for Fetch instead of reusing the initial Setup URI produced by the provisioning tool. The workflow verifies ordinary notes from the first device to the second and back again, independently enables Hidden File Sync on each device, and verifies a snippet. The retained Setup URI screenshots show only encrypted URIs and visually masked Setup URI passphrases; plaintext credentials are not captured. Files prefixed with `guide-` capture the relevant dialogue, settings panel, or workspace leaf without transient Notices. Public documentation copies selected images only after visual inspection; the E2E run does not overwrite repository documentation assets.
`test:e2e:obsidian:two-vault-sync` runs a two-vault note synchronisation workflow. It verifies note creation, update, ordinary rename, a case-only file name change within the same directory, deletion, and a separate encrypted round-trip with Path Obfuscation enabled. Its target-filter scenario confirms that one Vault receives and checkpoints a remote document without reflecting it, restarts with the same profile and filter, and then reflects the stored document after the filter is broadened through the settings service. Directory case changes deliberately remain outside this scenario because they require directory-aware rename handling. The optional Markdown conflict check can be enabled with `E2E_OBSIDIAN_INCLUDE_MARKDOWN_CONFLICT=true`. It creates divergent revisions in two separate Vaults, performs a conservative merge on one Vault, edits that result again, and requires the other Vault to replace its known deleted losing revision without recreating the conflict. The separate `E2E_OBSIDIAN_INCLUDE_CONFLICT_OPERATIONS=true` check keeps four conflicts active while one Vault edits, deletes, performs a case-only rename, and performs a cross-path rename. It asserts that each operation extends the revision displayed on that device, replicates the exact resulting revision tree, and preserves the other conflict branch. During focused development, `E2E_OBSIDIAN_ONLY_CONFLICT_OPERATIONS=true` runs that self-contained scope without the ordinary, target-filter, or encrypted scenarios. Both conflict checks remain outside the default local suite.
`test:e2e:obsidian:two-vault-sync` runs a two-vault note synchronisation workflow. It verifies note creation, update, ordinary rename, a case-only file name change within the same directory, deletion, and a separate encrypted round-trip with Path Obfuscation enabled. Its target-filter scenario confirms that one Vault receives and checkpoints a remote document without reflecting it, restarts with the same profile and filter, and then reflects the stored document after the filter is broadened through the settings service. Directory case changes deliberately remain outside the ordinary workflow because they require directory-aware rename handling.
During focused development, `E2E_OBSIDIAN_ONLY_PARENT_CASE_DELETION=true` runs an Issue #1168 check which renames `parent/test3` to `parent/Test3` through external `node:fs/promises.rename` while Vault A is open, and verifies that the note content, Metadata, and Chunk references are not logically deleted locally, remotely, or after restart. It accepts either case spelling on Vault B, so it does not provide directory rename support or exact case convergence between devices. The natural Obsidian event sequence and resulting database state are evidence for the selected build; an existing-version reproduction result must be reported separately from fixed-version safety evidence.
The optional Markdown conflict check can be enabled with `E2E_OBSIDIAN_INCLUDE_MARKDOWN_CONFLICT=true`. It creates divergent revisions in two separate Vaults, performs a conservative merge on one Vault, edits that result again, and requires the other Vault to replace its known deleted losing revision without recreating the conflict. The separate `E2E_OBSIDIAN_INCLUDE_CONFLICT_OPERATIONS=true` check keeps four conflicts active while one Vault edits, deletes, performs a case-only rename, and performs a cross-path rename. It asserts that each operation extends the revision displayed on that device, replicates the exact resulting revision tree, and preserves the other conflict branch. During focused development, `E2E_OBSIDIAN_ONLY_CONFLICT_OPERATIONS=true` runs that self-contained scope without the ordinary, target-filter, or encrypted scenarios. Both conflict checks remain outside the default local suite.
`test:e2e:obsidian:security-seed-reconnect` is a focused CouchDB release-acceptance workflow. Device A first recognises an initial remote Security Seed, stops automatic replication while remaining open, and creates an unsent note. The runner replaces only the Security Seed in the managed remote synchronisation-parameter fixture. Device A must retain its deliberately stale cached value until the next one-shot synchronisation, refresh it before sending, and upload an HKDF-encrypted payload which uses the replacement value. A fresh device B must decrypt that note and send an encrypted note back; the original device A then receives the return journey with its Vault and isolated profile preserved. Desktop Obsidian may enforce a single application instance, so the two device sessions run sequentially after the same-process stale-cache assertion has completed.
@@ -258,6 +262,7 @@ Useful environment variables:
- `E2E_OBSIDIAN_FILE_TIMEOUT_MS`: timeout for waiting until a note created through Obsidian's vault API is reflected to disk.
- `E2E_OBSIDIAN_CORE_READY_TIMEOUT_MS`: timeout for waiting until Self-hosted LiveSync reports that its core lifecycle and local database are ready.
- `E2E_OBSIDIAN_LOCAL_DB_TIMEOUT_MS`: timeout for waiting until a file appears in Self-hosted LiveSync's local database.
- `E2E_OBSIDIAN_ONLY_PARENT_CASE_DELETION=true`: run only the focused external parent-directory case-rename protection check in `two-vault-sync`.
- `E2E_OBSIDIAN_COUCHDB_TIMEOUT_MS`: timeout for waiting until CouchDB contains uploaded E2E documents.
- `E2E_OBSIDIAN_REMOTE_ACTIVITY_TIMEOUT_MS`: timeout for an observed remote activity to enter or leave its status boundary; default is 30 seconds.
- `E2E_OBSIDIAN_DIAGNOSTICS_DIR`: directory for screenshots and status snapshots, including the Security Seed reconnect stages; default is `/tmp/obsidian-livesync-e2e`.
+395 -4
View File
@@ -1,10 +1,11 @@
import { mkdir, readFile, rm, writeFile } from "node:fs/promises";
import { mkdir, readFile, rename as renameFilesystemPath, rm, writeFile } from "node:fs/promises";
import { dirname, join } from "node:path";
import { evalObsidianJson } from "../runner/cli.ts";
import {
assertCouchDbReachable,
createCouchDbDatabase,
deleteCouchDbDatabase,
fetchAllCouchDbDocs,
loadCouchDbConfig,
makeUniqueDatabaseName,
waitForCouchDbDocs,
@@ -27,6 +28,7 @@ import {
} from "../runner/liveSyncWorkflow.ts";
import { startObsidianLiveSyncSession, type ObsidianLiveSyncSession } from "../runner/session.ts";
import { createTemporaryVault, type TemporaryVault } from "../runner/vault.ts";
import { captureObsidianPage } from "../runner/ui.ts";
process.env.E2E_OBSIDIAN_CLI_TIMEOUT_MS ??= "30000";
process.env.E2E_OBSIDIAN_COUCHDB_TIMEOUT_MS ??= "20000";
@@ -47,6 +49,11 @@ const conflictRenameFromPath = "E2E/two-vault/conflict-operations/rename-source.
const conflictRenameToPath = "E2E/two-vault/conflict-operations/renamed/rename-target.md";
const targetMismatchPath = "E2E/two-vault/target-mismatch.md";
const encryptedPath = "E2E/two-vault/encrypted.md";
const parentCaseRenameFromDirectoryPath = "E2E/two-vault/parent/test3";
const parentCaseRenameToDirectoryPath = "E2E/two-vault/parent/Test3";
const parentCaseRenameFromPath = `${parentCaseRenameFromDirectoryPath}/note.md`;
const parentCaseRenameToPath = `${parentCaseRenameToDirectoryPath}/note.md`;
const parentCaseEventObserverKey = "__livesyncE2eParentCaseEventObserver";
type RunnerContext = {
binary: string;
@@ -68,6 +75,34 @@ type FileConflictState = {
}[];
};
type ParentCaseVaultEvent = {
type: "create" | "delete" | "rename";
path: string;
oldPath: string | null;
};
type ParentCaseMetadataState = {
id: string;
found: boolean;
rev: string | null;
path: string | null;
deleted: boolean;
children: string[];
contentMatches: boolean;
childrenMatch: boolean;
chunksPresent: boolean;
chunkReferenceCount: number;
availableChunkCount: number;
};
type ParentCaseRemoteMetadataState = {
id: string;
rev: string | null;
path: string | null;
deleted: boolean;
children: string[];
};
async function writeVaultFile(vaultPath: string, path: string, content: string): Promise<void> {
const fullPath = join(vaultPath, path);
await mkdir(dirname(fullPath), { recursive: true });
@@ -94,6 +129,57 @@ async function pathExists(vaultPath: string, path: string): Promise<boolean> {
}
}
async function installParentCaseEventObserver(
cliBinary: string,
env: NodeJS.ProcessEnv,
observedPaths: readonly string[]
): Promise<string> {
return await evalObsidianJson<string>(
cliBinary,
[
"(async()=>{",
`const key=${JSON.stringify(parentCaseEventObserverKey)};`,
`const observedPaths=${JSON.stringify(observedPaths)};`,
"const previous=globalThis[key];",
"if(previous){for(const ref of previous.refs??[]) app.vault.offref(ref);}",
"const events=[];",
"const record=(type,file,oldPath)=>{",
" const path=typeof file?.path==='string'?file.path:'';",
" const previousPath=typeof oldPath==='string'?oldPath:null;",
" if(!observedPaths.includes(path)&&(!previousPath||!observedPaths.includes(previousPath))) return;",
" globalThis[key].lastEventAt=Date.now();",
" if(events.length<32) events.push({type,path,oldPath:previousPath});",
"};",
"const refs=[",
" app.vault.on('create',(file)=>record('create',file)),",
" app.vault.on('delete',(file)=>record('delete',file)),",
" app.vault.on('rename',(file,oldPath)=>record('rename',file,oldPath)),",
"];",
"globalThis[key]={events,refs,lastEventAt:Date.now()};",
"return JSON.stringify(app.plugins.plugins['obsidian-livesync'].core.services.API.getAppVersion());",
"})()",
].join(""),
env
);
}
async function takeParentCaseEventEvidence(cliBinary: string, env: NodeJS.ProcessEnv): Promise<ParentCaseVaultEvent[]> {
return await evalObsidianJson<ParentCaseVaultEvent[]>(
cliBinary,
[
"(async()=>{",
`const key=${JSON.stringify(parentCaseEventObserverKey)};`,
"const observer=globalThis[key];",
"if(!observer) return JSON.stringify([]);",
"const events=Array.isArray(observer.events)?observer.events.slice(0,32):[];",
"try{for(const ref of observer.refs??[]) app.vault.offref(ref);}finally{delete globalThis[key];}",
"return JSON.stringify(events);",
"})()",
].join(""),
env
);
}
async function stopTrackedSession(context: RunnerContext, session: ObsidianLiveSyncSession): Promise<void> {
if (!context.activeSessions.has(session)) return;
await session.app.stop();
@@ -141,6 +227,165 @@ async function waitForPathDeleted(
throw new Error(`Timed out waiting for deleted file: ${join(vaultPath, path)}`);
}
async function waitForExactObsidianPath(
cliBinary: string,
env: NodeJS.ProcessEnv,
path: string,
oldPath: string,
timeoutMs = Number(process.env.E2E_OBSIDIAN_FILE_TIMEOUT_MS ?? 10000)
): Promise<void> {
await evalObsidianJson<unknown>(
cliBinary,
[
"(async()=>{",
`const expectedPath=${JSON.stringify(path)};`,
`const oldPath=${JSON.stringify(oldPath)};`,
`const observerKey=${JSON.stringify(parentCaseEventObserverKey)};`,
`const timeoutMs=${JSON.stringify(timeoutMs)};`,
"const deadline=Date.now()+timeoutMs;",
"let observedPath=null;",
"while(Date.now()<deadline){",
" const files=app.vault.getFiles();",
" const file=files.find((candidate)=>candidate.path===expectedPath);",
" observedPath=typeof file?.path==='string'?file.path:null;",
" const observer=globalThis[observerKey];",
" if(observedPath===expectedPath&&!files.some((candidate)=>candidate.path===oldPath)&&observer?.events.length>0&&Date.now()-observer.lastEventAt>=500) return JSON.stringify({path:observedPath});",
" await new Promise((resolve)=>setTimeout(resolve,100));",
"}",
"throw new Error(`Timed out waiting for Obsidian to recognise the exact path: ${JSON.stringify({expectedPath,observedPath})}`);",
"})()",
].join(""),
env
);
}
async function waitForEitherPathContent(
vaultPath: string,
paths: readonly string[],
expectedContent: string,
timeoutMs = Number(process.env.E2E_OBSIDIAN_FILE_TIMEOUT_MS ?? 10000)
): Promise<{ path: string }> {
const deadline = Date.now() + timeoutMs;
let lastPath: string | null = null;
let contentMatched = false;
while (Date.now() < deadline) {
for (const path of paths) {
if (!(await pathExists(vaultPath, path))) continue;
lastPath = path;
contentMatched = (await readVaultFile(vaultPath, path)) === expectedContent;
if (contentMatched) return { path };
}
await new Promise((resolve) => setTimeout(resolve, 250));
}
throw new Error(
`Timed out waiting for content at either case variant: ${JSON.stringify({
paths,
lastPath,
contentMatched,
})}`
);
}
async function waitForParentCaseMetadata(
cliBinary: string,
env: NodeJS.ProcessEnv,
id: string,
expectedPath: string,
expectedContent: string,
expectedChildren: readonly string[],
expectedRevision?: string
): Promise<ParentCaseMetadataState> {
const timeoutMs = Number(process.env.E2E_OBSIDIAN_LOCAL_DB_TIMEOUT_MS ?? 15000);
return await evalObsidianJson<ParentCaseMetadataState>(
cliBinary,
[
"(async()=>{",
`const id=${JSON.stringify(id)};`,
`const expectedPath=${JSON.stringify(expectedPath)};`,
`const expectedContent=${JSON.stringify(expectedContent)};`,
`const expectedChildren=${JSON.stringify(expectedChildren)};`,
`const expectedRevision=${JSON.stringify(expectedRevision ?? null)};`,
`const timeoutMs=${JSON.stringify(timeoutMs)};`,
"const core=app.plugins.plugins['obsidian-livesync'].core;",
"const deadline=Date.now()+timeoutMs;",
"let state={id,found:false,rev:null,path:null,deleted:false,children:[],contentMatches:false,childrenMatch:false,chunksPresent:false,chunkReferenceCount:0,availableChunkCount:0};",
"while(Date.now()<deadline){",
" await core.services.fileProcessing.commitPendingFileEvents();",
" const raw=await core.localDatabase.getRaw(id,{revs_info:true}).catch(()=>null);",
" const row=((await core.localDatabase.allDocsRaw({keys:[id],include_docs:true})).rows??[])[0];",
" const rawDoc=raw??row?.doc??null;",
" const deleted=Boolean(raw?.deleted||raw?._deleted||row?.value?.deleted||row?.doc?.deleted||row?.doc?._deleted);",
" const children=Array.isArray(rawDoc?.children)?rawDoc.children:[];",
" const rev=rawDoc?._rev??row?.value?.rev??null;",
" if(deleted) throw new Error(`Parent case rename marked Metadata as deleted (deleted or _deleted): ${JSON.stringify({id,rev,path:rawDoc?.path??null})}`);",
" if(rawDoc){",
" const loaded=await core.localDatabase.getDBEntry(expectedPath,{rev},false,true,true).catch(()=>false);",
" const content=loaded===false?'':Array.isArray(loaded.data)?loaded.data.join(''):typeof loaded.data==='string'?loaded.data:'';",
" const chunkRows=children.length===0?{rows:[]}:await core.localDatabase.allDocsRaw({keys:children,include_docs:true});",
" const availableChunkCount=chunkRows.rows.filter((chunkRow)=>Boolean(chunkRow.doc)&&!Boolean(chunkRow.value?.deleted)&&!Boolean(chunkRow.doc?.deleted)&&!Boolean(chunkRow.doc?._deleted)).length;",
" state={id,found:true,rev,path:rawDoc?.path??null,deleted:false,children,contentMatches:content===expectedContent,childrenMatch:children.length===expectedChildren.length&&children.every((child,index)=>child===expectedChildren[index]),chunksPresent:availableChunkCount===children.length&&children.length===expectedChildren.length,chunkReferenceCount:children.length,availableChunkCount};",
" if(state.contentMatches&&state.childrenMatch&&state.chunksPresent&&(!expectedRevision||state.rev===expectedRevision)) return JSON.stringify(state);",
" }",
" await new Promise((resolve)=>setTimeout(resolve,250));",
"}",
"throw new Error(`Timed out waiting for parent case Metadata and Chunks: ${JSON.stringify(state)}`);",
"})()",
].join(""),
env
);
}
async function waitForParentCaseRemoteMetadata(
context: RunnerContext,
entry: LocalDatabaseEntry
): Promise<ParentCaseRemoteMetadataState> {
const timeoutMs = Number(process.env.E2E_OBSIDIAN_COUCHDB_TIMEOUT_MS ?? 15000);
const deadline = Date.now() + timeoutMs;
let lastState: ParentCaseRemoteMetadataState | null = null;
while (Date.now() < deadline) {
const response = await fetchAllCouchDbDocs(context.couchDb, context.dbName);
const row = response.rows.find((candidate) => candidate.id === entry.id);
const doc = row?.doc;
const deleted = Boolean(row?.value.deleted || doc?.deleted || doc?._deleted);
lastState = {
id: entry.id,
rev: row?.value.rev ?? doc?._rev ?? null,
path: doc?.path ?? null,
deleted,
children: Array.isArray(doc?.children) ? doc.children : [],
};
if (deleted) {
throw new Error(
`Parent case rename uploaded deleted remote Metadata: ${JSON.stringify({
id: entry.id,
rev: lastState.rev,
path: lastState.path,
})}`
);
}
if (
doc &&
lastState.children.length === entry.children.length &&
lastState.children.every(
(child, index) =>
child === entry.children[index] &&
response.rows.some(
(chunk) =>
chunk.id === child &&
chunk.doc &&
!chunk.value.deleted &&
!chunk.doc.deleted &&
!chunk.doc._deleted
)
)
) {
return lastState;
}
await new Promise((resolve) => setTimeout(resolve, 500));
}
throw new Error(`Timed out waiting for non-deleted remote Metadata: ${JSON.stringify(lastState)}`);
}
async function writeNoteViaObsidian(cliBinary: string, env: NodeJS.ProcessEnv, path: string, content: string) {
await evalObsidianJson<unknown>(
cliBinary,
@@ -559,6 +804,145 @@ async function runCaseOnlyRename(
console.log("Two-vault case-only note rename round-tripped without a tombstone.");
}
async function runParentCaseDeletionProtection(
context: RunnerContext,
vaultA: TemporaryVault,
vaultB: TemporaryVault
): Promise<void> {
const fileContent = "# Parent case rename\n\nThe document must remain live after an external parent rename.\n";
const parentCaseOverrides = {
handleFilenameCaseSensitive: false,
batchSave: false,
};
const observedPaths = [
parentCaseRenameFromDirectoryPath,
parentCaseRenameToDirectoryPath,
parentCaseRenameFromPath,
parentCaseRenameToPath,
];
let session: ObsidianLiveSyncSession | undefined;
let observerInstalled = false;
let obsidianVersion: string | undefined;
let observedEvents: ParentCaseVaultEvent[] = [];
let localMetadataEvidence: ParentCaseMetadataState | undefined;
let remoteMetadataEvidence: ParentCaseRemoteMetadataState | undefined;
let restartedMetadataEvidence: ParentCaseMetadataState | undefined;
try {
session = await startConfiguredSession(context, vaultA, parentCaseOverrides);
await writeNoteViaObsidian(context.cliBinary, session.cliEnv, parentCaseRenameFromPath, fileContent);
const initialEntry = await uploadNote(context, session, parentCaseRenameFromPath);
if (initialEntry.children.length === 0) {
throw new Error(`Parent case fixture did not retain a Chunk reference: ${initialEntry.id}`);
}
await stopTrackedSession(context, session);
session = undefined;
session = await startConfiguredSession(context, vaultB, parentCaseOverrides);
await syncAndApply(context, session);
await waitForPathContent(vaultB.path, parentCaseRenameFromPath, (content) => content === fileContent);
await stopTrackedSession(context, session);
session = undefined;
session = await startConfiguredSession(context, vaultA, parentCaseOverrides);
await waitForLocalDatabaseEntry(context.cliBinary, session.cliEnv, parentCaseRenameFromPath);
obsidianVersion = await installParentCaseEventObserver(context.cliBinary, session.cliEnv, observedPaths);
observerInstalled = true;
await renameFilesystemPath(
join(vaultA.path, parentCaseRenameFromDirectoryPath),
join(vaultA.path, parentCaseRenameToDirectoryPath)
);
await waitForExactObsidianPath(
context.cliBinary,
session.cliEnv,
parentCaseRenameToPath,
parentCaseRenameFromPath
);
localMetadataEvidence = await waitForParentCaseMetadata(
context.cliBinary,
session.cliEnv,
initialEntry.id,
parentCaseRenameToPath,
fileContent,
initialEntry.children
);
await pushLocalChanges(context.cliBinary, session.cliEnv);
const remoteMetadata = await waitForParentCaseRemoteMetadata(context, initialEntry);
remoteMetadataEvidence = remoteMetadata;
observedEvents = await takeParentCaseEventEvidence(context.cliBinary, session.cliEnv);
observerInstalled = false;
await stopTrackedSession(context, session);
session = undefined;
session = await startConfiguredSession(context, vaultB, parentCaseOverrides);
await syncAndApply(context, session);
await waitForParentCaseMetadata(
context.cliBinary,
session.cliEnv,
initialEntry.id,
parentCaseRenameToPath,
fileContent,
initialEntry.children,
remoteMetadata.rev ?? undefined
);
await waitForEitherPathContent(vaultB.path, [parentCaseRenameFromPath, parentCaseRenameToPath], fileContent);
await stopTrackedSession(context, session);
session = undefined;
session = await startConfiguredSession(context, vaultA, parentCaseOverrides);
await syncAndApply(context, session);
await waitForEitherPathContent(vaultA.path, [parentCaseRenameFromPath, parentCaseRenameToPath], fileContent);
restartedMetadataEvidence = await waitForParentCaseMetadata(
context.cliBinary,
session.cliEnv,
initialEntry.id,
parentCaseRenameToPath,
fileContent,
initialEntry.children,
remoteMetadata.rev ?? undefined
);
await stopTrackedSession(context, session);
session = undefined;
} catch (error) {
if (session) {
await captureObsidianPage(session.remoteDebuggingPort, "parent-case-deletion-failure.png", async () => {})
.then((path) => console.error(`Parent case failure screenshot: ${path}`))
.catch((captureError: unknown) => {
console.warn(captureError instanceof Error ? captureError.message : captureError);
});
}
throw error;
} finally {
if (observerInstalled && session) {
try {
observedEvents = await takeParentCaseEventEvidence(context.cliBinary, session.cliEnv);
} catch (error) {
console.warn(
`Could not collect parent case rename event evidence: ${
error instanceof Error ? error.message : String(error)
}`
);
}
}
try {
if (session) await stopTrackedSession(context, session);
} finally {
console.log(
`Parent case rename evidence: ${JSON.stringify({
obsidianVersion,
events: observedEvents,
localMetadata: localMetadataEvidence ?? null,
remoteMetadata: remoteMetadataEvidence ?? null,
restartedMetadata: restartedMetadataEvidence ?? null,
})}`
);
}
}
console.log("External parent case rename preserved the note Metadata, Chunks, and content.");
}
async function runEncryptedRoundTrip(
context: RunnerContext,
vaultA: TemporaryVault,
@@ -1002,8 +1386,12 @@ async function main(): Promise<void> {
console.log(`Temporary CouchDB database: ${dbName}`);
console.log(`Temporary encrypted CouchDB database: ${encryptedDbName}`);
const onlyParentCaseDeletion = process.env.E2E_OBSIDIAN_ONLY_PARENT_CASE_DELETION === "true";
if (onlyParentCaseDeletion) {
await runParentCaseDeletionProtection(context, vaultA, vaultB);
}
const onlyConflictOperations = process.env.E2E_OBSIDIAN_ONLY_CONFLICT_OPERATIONS === "true";
if (!onlyConflictOperations) {
if (!onlyParentCaseDeletion && !onlyConflictOperations) {
await runCreateUpdateDelete(context, vaultA, vaultB);
await runRename(context, vaultA, vaultB);
await runCaseOnlyRename(context, vaultA, vaultB);
@@ -1011,10 +1399,13 @@ async function main(): Promise<void> {
await runMarkdownAutoMerge(context, vaultA, vaultB);
}
}
if (onlyConflictOperations || process.env.E2E_OBSIDIAN_INCLUDE_CONFLICT_OPERATIONS === "true") {
if (
!onlyParentCaseDeletion &&
(onlyConflictOperations || process.env.E2E_OBSIDIAN_INCLUDE_CONFLICT_OPERATIONS === "true")
) {
await runConflictTimeStorageOperations(context, vaultA, vaultB);
}
if (!onlyConflictOperations) {
if (!onlyParentCaseDeletion && !onlyConflictOperations) {
await runTargetMismatch(context, vaultA, vaultB);
await runEncryptedRoundTrip(encryptedContext, encryptedVaultA, encryptedVaultB);
}
+14 -9
View File
@@ -12,20 +12,25 @@ Earlier releases remain available in the 1.0 release history, the 1.0 preview hi
## Unreleased
## 1.0.25
### Synchronisation and storage
4th September, 2026
#### Fixed
### Fixed
- Watcher events from an external case-only parent-folder rename no longer let a stale deletion remove a note's Metadata while the file still exists. Moves out of the selected files retain the existing deletion behaviour; this fix does not add general folder-rename handling or path-case convergence. (#1168)
- Conflict resolution dialogues now close when the same file is resolved elsewhere or the plug-in unloads. Requests for different files are shown one at a time, while a newer request for the same file replaces the stale dialogue.
- An individual file-processing failure during ordinary start-up no longer keeps the entire application unready. A start-up notice asks the user to check the affected files and generate a report for details; each path is recorded in verbose logs and remains eligible for retry, while explicit Fetch and Rebuild operations retain strict completion.
- When a replicated file cannot be written to the Vault, LiveSync now warns immediately instead of appearing to have processed it successfully. The affected path is recorded in the report and remains eligible for a later scan. (#1164)
- Replication readiness diagnostics now state that application initialisation is incomplete instead of reporting only 'Not ready'. Database-preparation failures show a short notice, with the failed stage available in verbose logs.
- A file that cannot be processed during start-up no longer prevents every other file from synchronising. LiveSync completes ordinary start-up, warns about the affected files, and allows them to be retried during a later scan. Fetch and Rebuild still stop rather than treating an incomplete operation as successful. (#1164)
- When local database initialisation cannot finish, LiveSync no longer leaves the failure at 'Not ready'. It states that synchronisation is unavailable and directs you to generate a report, where the failed preparation stage is recorded. (#1164)
#### Improved
### Improved
- Start-up now keeps unconfigured Vaults on the onboarding path without running configured-only checks or accepting Config Doctor and incomplete-document repair requests. Returning a configured Vault to an unconfigured state also retires those requests for the current plug-in process, so completing setup admits them only after the requested restart.
- The active-file warning now concisely identifies file or folder names longer than 255 UTF-8 bytes as an Android and Linux compatibility risk, without rejecting or changing the path.
- New or unconfigured Vaults no longer run Config Doctor or incomplete-document repair before setup. If a configured Vault is returned to an unconfigured state, those checks remain paused until the requested restart. (#1161)
- Conflict dialogue clean-up is now more robust when repeated checks overlap, a waiting conflict is resolved elsewhere, or the plug-in unloads. (#1162)
- When the active file's path contains a file or folder name longer than 255 UTF-8 bytes, LiveSync now warns that the path may not work on some Android and Linux file systems. The path is neither renamed nor rejected, so you can decide how to handle the compatibility risk. (#1164)
### Testing
- An optional real-Obsidian regression checks for stale deletions after an external parent-directory case change, including Metadata, Chunks, a second Vault, and restart. Developer documentation now distinguishes file-event handling from database-to-storage reflection.
- Start-up migrations, integrity checks, Config Doctor, basic commands, and the Obsidian replication ribbon now have focused regression tests for their service composition. Real Obsidian checks cover unconfigured onboarding, configured start-up scanning and individual file failures, Config Doctor detection and layout, command registration, and the established ribbon icon.
## 1.0.24
+1 -2
View File
@@ -36,6 +36,5 @@
"1.0.21": "1.7.2",
"1.0.22": "1.7.2",
"1.0.23": "1.7.2",
"1.0.24": "1.7.2",
"1.0.25": "1.7.2"
"1.0.24": "1.7.2"
}