Verify NFC P2P migration and Journal resend in Obsidian

This commit is contained in:
vorotamoroz committed 2026-11-02 17:20:39 +00:00
1 parent 74954d9b51
commit 538fc52903
6 files changed
+1182 -26

No files matched your search

+26 -5
View File
@@ -365,13 +365,34 @@ physical NFD names and verifying edits after restart and deletion propagation.
The CLI Compose P2P suite also passed with the published package, including
legacy and NFC identity, plain and encrypted NFC transfer, logical rename and
deletion, automatic hosting, adapter replacement, and relay disconnection.
The earlier Journal resend and conflict-ancestry checks were not repeated with
the published package.
Additional real Obsidian cases with the published package passed legacy-to-NFC
P2P migration with plain IDs and independently encrypted IDs. They seed legacy
NFD Metadata IDs using real Chunks, since Obsidian's file cache normalises
names. The checks covered source Rebuild and restart, Standard Fetch on the
receiving device, removal of obsolete active IDs, exact content, and retention
of physical NFD names. Both cases also passed transfer of a Unicode note,
logical rename with matching new IDs, and deletion after restart. An
incompatible P2P Fetch reset the local database before a peer was selected.
Its physical note retained its exact bytes, and a Fetch with matching settings
completed recovery. This check does not establish preservation of the old
database's revision history.
Real Obsidian Journal cases also passed remote-only Rebuild after deleting
their isolated Object Storage prefix while leaving the source database and
populated transfer checkpoint intact. With both plain IDs and independently
encrypted IDs, a fresh receiving database and its subsequent restart retained
the exact bodies of both conflict leaves, Chunk references, revision IDs, and
each leaf's `_revisions`. The source retained the readable body of its common
ancestor. Fresh receivers retained the ancestor ID without its historical
body; ancestry IDs alone do not establish that an older body is readable. The
[conflict policy](../specs_conflict_resolution.md#file-saving-and-reflection-guarantees)
continues to require an available shared base for conservative automatic merge.
The settings, first-join identity inspection, legacy Setup URI defaults, and
initial CLI Fetch persistence are also covered by unit tests. Logical rename
and deletion in the P2P extension have CLI runtime coverage; the real Obsidian
P2P cases above do not separately exercise those operations.
initial CLI Fetch persistence are also covered by unit tests. The additional
migration cases call the running plug-in's production Rebuild and Fetch
methods with real peer selection; reservation through the settings dialogue
has separate coverage.
These checks do not extend NFC support to other hosts or establish native iOS
or macOS filesystem behaviour.
+1
View File
@@ -79,6 +79,7 @@
"test:e2e:obsidian:couchdb-manual-setup-workflow": "tsx test/e2e-obsidian/scripts/couchdb-manual-setup-workflow.ts",
"test:e2e:obsidian:cli-to-obsidian-sync": "tsx test/e2e-obsidian/scripts/cli-to-obsidian-sync.ts",
"test:e2e:obsidian:minio-upload": "tsx test/e2e-obsidian/scripts/minio-upload.ts",
"test:e2e:obsidian:journal-nfc-resend": "tsx test/e2e-obsidian/scripts/journal-nfc-resend.ts",
"test:e2e:obsidian:object-storage-setup-uri-workflow": "tsx test/e2e-obsidian/scripts/object-storage-setup-uri-workflow.ts",
"test:e2e:obsidian:object-storage-compatible-setup-uri-workflow": "tsx test/e2e-obsidian/scripts/object-storage-setup-uri-workflow.ts --compatible",
"test:e2e:obsidian:object-storage-qr-workflow": "tsx test/e2e-obsidian/scripts/object-storage-setup-uri-workflow.ts --qr",
+6
View File
@@ -230,6 +230,8 @@ Set `E2E_LIVESYNC_TARGET_ARTIFACT_ROOT` to the directory containing the exact re
Set `E2E_OBSIDIAN_INDEPENDENT_IDS=true` to run the same upload with E2EE, Path Obfuscation, and a separately derived ID key. The scenario verifies the local document and Chunk ID shapes before the Journal transfer.
Set `E2E_OBSIDIAN_CUSTOM_HTTP_HANDLER=true` when the local Object Storage fixture does not allow browser requests from Obsidian's renderer.
`test:e2e:obsidian:journal-nfc-resend` uses the same Object Storage variables with NFC path identity. Run `npm run test:e2e:obsidian:focused -- journal-nfc-resend --case all` for the plain and E2EE/Path Obfuscation cases with an independent ID key, or select `plain` or `encrypted`. It creates two conflict leaves with real Chunks with an exact shared ancestor, completes the initial upload, and deletes only its unique remote prefix while retaining the source database and populated transfer checkpoint. Rebuilding the remote must resend enough data for a fresh Obsidian Standard Fetch to recover both exact leaf bodies, Chunk references, revision IDs, and each leaf's `_revisions`. The source must retain its readable ancestor body. The receiver records whether that older body is available; ancestry IDs alone do not make a historical body readable. A full receiver restart must retain the same leaves, history, ancestor availability, and one NFC-equivalent physical control file. The restart consumes the saved settings and device state.
`test:e2e:obsidian:object-storage-setup-uri-workflow` uses the public Commonlib-backed tool to generate the initial Setup URI for a unique Object Storage prefix, completes visible initialisation on the first device, and then asks that working real Obsidian device to create a new Setup URI through the registered command. A second real Obsidian device imports only the device-generated URI. The workflow verifies the A-to-B note through explicit replication, then verifies that the B-to-A note arrives through `syncOnStart` after restarting the first device, without requesting manual replication. It captures the documented onboarding choices, and removes the Object Storage prefix only after both sessions have stopped. The test requires the current version marker and absence of a compatibility pause after Fetch and after restarting the same Vault, without accepting a review automatically.
`test:e2e:obsidian:object-storage-qr-workflow` runs the same Object Storage round trip with QR settings on the second device. It takes the first device's settings, assigns a distinct database suffix in the QR fixture, encodes them with Commonlib's QR encoder, passes the payload to the real QR decoding entry point, and selects **Join this device** in the visible dialogue. Unlike the Setup URI, the QR payload includes a database suffix; explicitly choosing one makes the namespace change independent of Obsidian's initial defaults. Before Fetch begins, the scenario verifies that the imported namespace has its current compatibility marker without a pause. Fetch then selects the receiving device's own suffix when resetting the local database. The test verifies the marker and absence of a pause again after Fetch and after a natural restart. It covers the QR settings and setup flow, without requiring a camera or exercising operating-system URI dispatch.
@@ -246,6 +248,10 @@ The P2P Setup URI workflow also schedules another Fetch and injects an exception
`test:e2e:obsidian:p2p-nfc` runs the plain case and the E2EE case using Path Obfuscation and an independent ID key, serially, in an isolated Compose project with a local signalling relay and two real Obsidian processes at a time. Each case seeds an NFD-named source file, fetches it by its NFC spelling, and compares the real local Metadata ID, exact body, and referenced Chunks on both peers. It edits the note from the receiving peer, verifies that the Metadata ID remains stable and that the original revision body remains readable, then fully stops and relaunches the same Vault to verify the ID, updated body, original revision, and a further transfer after restart. A fresh path version 0 peer must show the visible P2P compatibility refusal; the test verifies that it receives no source Metadata and retains its local sentinel. Run `npm run test:e2e:obsidian:p2p-nfc -- --case plain` or `-- --case encrypted` to select one case for debugging. The host wrapper builds the plug-in, uses the existing extracted Obsidian directory and its `obsidian-cli`, mounts the repository read-only, and tears down its unique Compose project after the run. It does not download Obsidian or npm dependencies.
Each P2P case also transfers another Unicode note after restart, renames it on the receiving peer, compares the new Metadata IDs and exact content, and verifies deletion on both peers.
Add `-- --migration` to start with legacy databases and exercise source Rebuild followed by receiving-device Standard Fetch. Because Obsidian's file cache normalises names, the scenario seeds valid legacy NFD Metadata IDs using the real stored Chunks before reconstruction. It verifies replacement of those IDs, physical NFD-name retention, ordinary legacy-peer refusal, and successful migration after a source restart. It also attempts Fetch with mismatched settings, verifies the local database is reset before peer refusal while the physical note remains intact, then retries with NFC and verifies convergence. The migration phase calls the running plug-in's production Rebuild and Fetch methods and uses real peer selection; reservation through the settings dialogue has separate coverage.
`test:e2e:obsidian:startup-scan` starts from a CouchDB fixture using current settings with its device-local compatibility marker already acknowledged, stops Obsidian, writes a note directly into the Vault, restarts the same isolated Vault and profile without rewriting its plug-in data, and verifies from CouchDB that the start-up scan picked up the offline file. Onboarding remains covered by `onboarding-invitation`; this scenario owns the ordinary configured restart and start-up scan.
`test:e2e:obsidian:partial-startup-file-failure` is a focused Linux release-acceptance scenario for an ordinary configured restart. It stores one valid database-only note and one database-only note whose path component is 258 UTF-8 bytes, then restarts the same isolated Vault and profile. On a Linux test Vault which enforces the conventional 255-byte component limit, the scenario requires the valid file to be reflected, the application to become ready, the partial-failure Notice to appear, and the failed path to remain readable and eligible for a later scan with its exact path in the verbose log. It remains outside `local-suite` because the failure fixture is deliberately platform-specific.
@@ -0,0 +1,843 @@
import { randomBytes } from "node:crypto";
import { createServer } from "node:net";
import { readFile, readdir } from "node:fs/promises";
import { join } from "node:path";
import { deriveIdKey } from "@vrtmrz/livesync-commonlib/settings";
import { evalObsidianJson } from "../runner/cli.ts";
import { discoverObsidianCli, requireObsidianBinary } from "../runner/environment.ts";
import {
assertEqual,
configureObjectStorage,
createE2eObjectStoragePluginData,
createE2eObsidianDeviceLocalState,
prepareRemote,
pushLocalChanges,
waitForLiveSyncCoreReady,
waitForLocalDatabaseEntry,
type LocalDatabaseEntry,
} from "../runner/liveSyncWorkflow.ts";
import {
deleteObjectStoragePrefix,
ensureObjectStorageBucket,
listObjectStorageObjects,
loadObjectStorageConfig,
makeUniqueBucketPrefix,
readObjectStorageJson,
type ObjectStorageConfig,
} from "../runner/objectStorage.ts";
import { startObsidianLiveSyncSession, type ObsidianLiveSyncSession } from "../runner/session.ts";
import { createTemporaryVault, type TemporaryVault } from "../runner/vault.ts";
import { withObsidianPage } from "../runner/ui.ts";
process.env.E2E_OBSIDIAN_CLI_TIMEOUT_MS ??= "60000";
const timeoutMs = Number(process.env.E2E_OBSIDIAN_JOURNAL_NFC_TIMEOUT_MS ?? 60000);
const objectTimeoutMs = Number(process.env.E2E_OBSIDIAN_OBJECT_STORAGE_TIMEOUT_MS ?? 30000);
const uiTimeoutMs = Number(process.env.E2E_OBSIDIAN_JOURNAL_NFC_UI_TIMEOUT_MS ?? 10000);
const relativeFolder = "E2E/journal-nfc";
const folderNfd = "Cafe\u0301";
const conflictNameNfd = "conflict-e\u0301.md";
const conflictPathNfd = `${relativeFolder}/${folderNfd}/${conflictNameNfd}`;
const conflictPathNfc = conflictPathNfd.normalize("NFC");
const controlNameNfd = "ordinary-e\u0301.md";
const controlPathNfd = `${relativeFolder}/${folderNfd}/${controlNameNfd}`;
const controlContent = "# NFC control\n\nThis ordinary, non-conflicted note keeps one NFC-equivalent path.\n";
const baseContent = [
"# Journal resend conflict fixture",
"",
"The common ancestor contains this exact sentence and enough bytes for real Chunks.",
"The source file begins from this baseline before either device makes a change.",
"Shared value: the original value remains here.",
"",
].join("\n");
const sourceLeafContent = [
"# Journal resend conflict fixture",
"",
"The common ancestor contains this exact sentence and enough bytes for real Chunks.",
"The source file begins from this baseline before either device makes a change.",
"Shared value: the source device replaced this line with its own edit.",
"",
].join("\n");
const competingLeafContent = [
"# Journal resend conflict fixture",
"",
"The common ancestor contains this exact sentence and enough bytes for real Chunks.",
"The source file begins from this baseline before either device makes a change.",
"Shared value: another device replaced this line differently.",
"",
].join("\n");
type ScenarioCase = { name: "plain" | "encrypted"; independentIds: boolean };
type RevisionSnapshot = {
rev: string;
revisionTree: { start: number; ids: string[] };
revisions: string[];
body: string;
children: string[];
chunksPresent: boolean;
};
type ConflictTreeSnapshot = {
id: string;
path: string;
leafRevs: string[];
leaves: RevisionSnapshot[];
ancestor?: RevisionSnapshot;
ancestorRev?: string;
ancestorBodyAvailable: boolean;
};
type TransferCheckpoint = {
lastLocalSeq: number | string | undefined;
journalEpoch: string;
knownIDs: string[];
sentIDs: string[];
receivedFiles: string[];
sentFiles: string[];
};
type ScenarioSettings = Record<string, unknown>;
function selectedCases(args: readonly string[]): ScenarioCase[] {
let requested = process.env.E2E_OBSIDIAN_INDEPENDENT_IDS === "true" ? "encrypted" : "plain";
for (let index = 0; index < args.length; index += 1) {
const argument = args[index];
if (argument === "--help" || argument === "-h") {
console.log("Usage: journal-nfc-resend.ts [--case all|plain|encrypted|independent-ids]");
process.exit(0);
}
if (argument === "--case") {
const value = args[index + 1];
if (!value) throw new Error("--case requires all, plain, encrypted, or independent-ids.");
requested = value;
index += 1;
continue;
}
if (argument.startsWith("--case=")) {
requested = argument.slice("--case=".length);
continue;
}
throw new Error(`Unknown Journal NFC resend argument: ${argument}`);
}
const cases: ScenarioCase[] = [
{ name: "plain", independentIds: false },
{ name: "encrypted", independentIds: true },
];
if (requested === "all") return cases;
if (requested === "independent-ids") requested = "encrypted";
const selected = cases.filter((candidate) => candidate.name === requested);
if (selected.length === 0) throw new Error(`Unsupported Journal NFC resend case: ${requested}`);
return selected;
}
async function allocatePort(excluded: ReadonlySet<number>): Promise<number> {
for (let attempt = 0; attempt < 10; attempt += 1) {
const server = createServer();
const port = await new Promise<number>((resolve, reject) => {
server.once("error", reject);
server.listen(0, "127.0.0.1", () => {
const address = server.address();
if (!address || typeof address === "string") {
reject(new Error("Could not allocate an Obsidian debugging port."));
return;
}
resolve(address.port);
});
});
await new Promise<void>((resolve, reject) => server.close((error) => (error ? reject(error) : resolve())));
if (!excluded.has(port)) return port;
}
throw new Error("Could not allocate distinct Obsidian debugging ports.");
}
async function createVaultNote(
cliBinary: string,
environment: NodeJS.ProcessEnv,
relativePath: string,
content: string
): Promise<void> {
await evalObsidianJson<boolean>(
cliBinary,
[
"(async()=>{",
"const path=" + JSON.stringify(relativePath.normalize("NFC")) + ";",
"const content=" + JSON.stringify(content) + ";",
"const parent=path.slice(0,path.lastIndexOf('/'));",
"if(!(await app.vault.adapter.exists(parent)))await app.vault.createFolder(parent);",
"await app.vault.create(path,content);",
"await app.plugins.plugins['obsidian-livesync'].core.services.fileProcessing.commitPendingFileEvents();",
"return JSON.stringify(true);",
"})()",
].join(""),
environment,
timeoutMs
);
}
async function assertSinglePhysicalUnicodeFile(
vaultPath: string,
relativePath: string,
expectedContent: string
): Promise<void> {
const segments = relativePath.split("/");
const stableParent = join(vaultPath, ...segments.slice(0, -2));
const expectedFolder = segments.at(-2)!.normalize("NFC");
const expectedFilename = segments.at(-1)!.normalize("NFC");
const physicalFolders = (await readdir(stableParent, { withFileTypes: true })).filter(
(entry) => entry.isDirectory() && entry.name.normalize("NFC") === expectedFolder
);
assertEqual(physicalFolders.length, 1, "NFC-equivalent folder spellings created duplicate physical folders.");
const actualFolder = join(stableParent, physicalFolders[0].name);
const matchingFiles = (await readdir(actualFolder)).filter((name) => name.normalize("NFC") === expectedFilename);
assertEqual(matchingFiles.length, 1, "NFC-equivalent file spellings created duplicate physical files.");
assertEqual(
await readFile(join(actualFolder, matchingFiles[0]), "utf8"),
expectedContent,
"The ordinary NFC control file has unexpected bytes."
);
}
async function waitForSinglePhysicalUnicodeFile(
vaultPath: string,
relativePath: string,
expectedContent: string
): Promise<void> {
const deadline = Date.now() + timeoutMs;
let lastError: unknown;
while (Date.now() < deadline) {
try {
await assertSinglePhysicalUnicodeFile(vaultPath, relativePath, expectedContent);
return;
} catch (error) {
lastError = error;
await new Promise((resolve) => setTimeout(resolve, 250));
}
}
throw new Error(
`Timed out waiting for one NFC-equivalent physical file: ${lastError instanceof Error ? lastError.message : String(lastError)}`
);
}
async function makeScenarioSettings(scenario: ScenarioCase): Promise<ScenarioSettings> {
return {
pathNormalizationVersion: 1,
...(scenario.independentIds
? {
encrypt: true,
usePathObfuscation: true,
passphrase: randomBytes(32).toString("base64url"),
idDerivationVersion: 1,
idDerivationKey: await deriveIdKey(randomBytes(32).toString("base64url")),
}
: {}),
};
}
async function startSession(
binary: string,
cliBinary: string,
vault: TemporaryVault,
port: number,
storage: ObjectStorageConfig & { bucketPrefix: string },
overrides: ScenarioSettings,
configure: boolean
): Promise<ObsidianLiveSyncSession> {
const session = await startObsidianLiveSyncSession({
binary,
cliBinary,
vault,
startupGraceMs: Number(process.env.E2E_OBSIDIAN_STARTUP_GRACE_MS ?? 1000),
env: { E2E_OBSIDIAN_REMOTE_DEBUGGING_PORT: String(port) },
...(configure
? {
pluginData: createE2eObjectStoragePluginData(storage, { pathNormalizationVersion: 1 }),
localStorageEntries: createE2eObsidianDeviceLocalState(vault.name),
}
: {}),
});
try {
await waitForLiveSyncCoreReady(cliBinary, session.cliEnv, timeoutMs);
if (configure) {
const configured = await configureObjectStorage(cliBinary, session.cliEnv, storage, overrides);
assertEqual(configured.isConfigured, true, "Self-hosted LiveSync was not marked as configured.");
assertEqual(configured.remoteType, "MINIO", "The configured remote was not Object Storage.");
assertEqual(configured.bucket, storage.bucket, "The Object Storage bucket changed during configuration.");
assertEqual(
configured.bucketPrefix,
storage.bucketPrefix,
"The Object Storage prefix changed during configuration."
);
assertEqual(configured.liveSync, false, "LiveSync should remain disabled during the one-shot workflow.");
await waitForLiveSyncCoreReady(cliBinary, session.cliEnv, timeoutMs);
}
return session;
} catch (error) {
await session.app.stop().catch((stopError: unknown) => {
console.warn(stopError instanceof Error ? stopError.message : stopError);
});
throw error;
}
}
function readTreeSource(path: string, expectedBaseRev: string, requireAncestorBody: boolean): string {
return [
"(async()=>{",
`const path=${JSON.stringify(path)};`,
`const expectedBaseRev=${JSON.stringify(expectedBaseRev)};`,
`const requireAncestorBody=${JSON.stringify(requireAncestorBody)};`,
"const core=app.plugins.plugins['obsidian-livesync'].core;",
"const db=core.localDatabase;",
"const readRevision=async(rev)=>{",
"const metadata=await db.getDBEntryMeta(path,{rev,revs:true},true);",
"const entry=await db.getDBEntry(path,{rev},false,true,true);",
"if(!metadata||!entry) throw new Error('Could not read exact Metadata revision '+rev);",
"const tree=metadata._revisions;",
"if(!tree||!Number.isInteger(tree.start)||!Array.isArray(tree.ids)||tree.ids.length===0) throw new Error('Revision ancestry is unavailable for '+rev);",
"const revisionTree={start:tree.start,ids:[...tree.ids]};",
"const revisions=revisionTree.ids.map((id,index)=>`${revisionTree.start-index}-${id}`);",
"const data=entry.data;",
"const body=Array.isArray(data)?data.join(''):typeof data==='string'?data:data instanceof Blob?await data.text():null;",
"if(typeof body!=='string') throw new Error('Could not decode exact revision body '+rev);",
"const children=Array.isArray(metadata.children)?[...metadata.children]:Array.isArray(entry.children)?[...entry.children]:[];",
"const rows=children.length?await db.allDocsRaw({keys:children,include_docs:true}):{rows:[]};",
"const available=rows.rows.filter(row=>Boolean(row.doc)&&!Boolean(row.value?.deleted)&&!Boolean(row.doc?.deleted)&&!Boolean(row.doc?._deleted)).length;",
"const chunksPresent=children.length>0&&available===children.length;",
"if(!chunksPresent) throw new Error('Exact Metadata revision has missing referenced Chunks '+rev);",
"return {rev:entry._rev,revisionTree,revisions,body,children,chunksPresent};",
"};",
"const current=await db.getDBEntryMeta(path,{conflicts:true},true);",
"if(!current?._rev) throw new Error('Conflict Metadata is missing: '+path);",
"const leafRevs=[current._rev,...(current._conflicts??[])].sort();",
"const leaves=[];",
"for(const rev of leafRevs) leaves.push(await readRevision(rev));",
"let ancestor;let ancestorRev;let ancestorBodyAvailable=false;",
"if(leaves.length>=2){",
"const shared=leaves[0].revisions.filter(rev=>leaves.slice(1).every(leaf=>leaf.revisions.includes(rev)));",
"if(shared.length===0) throw new Error('Conflict leaves have no exact shared revision ID.');",
"ancestorRev=shared[0];",
"if(ancestorRev!==expectedBaseRev) throw new Error('Nearest common revision did not match the seeded baseline.');",
"const metadata=await db.getDBEntryMeta(path,{rev:ancestorRev,revs:true},true).catch(error=>{if(error?.status===404||error?.name==='not_found')return false;throw error;});",
"if(metadata){ancestor=await readRevision(ancestorRev);ancestorBodyAvailable=true;}",
"if(requireAncestorBody&&!ancestorBodyAvailable)throw new Error('The source lost its available common ancestor body.');",
"}",
"return JSON.stringify({id:current._id,path:current.path,leafRevs,leaves,ancestor,ancestorRev,ancestorBodyAvailable});",
"})()",
].join("");
}
async function readConflictTree(
cliBinary: string,
env: NodeJS.ProcessEnv,
path: string,
expectedBaseRev: string,
requireAncestorBody = true
): Promise<ConflictTreeSnapshot> {
return await evalObsidianJson<ConflictTreeSnapshot>(
cliBinary,
readTreeSource(path, expectedBaseRev, requireAncestorBody),
env,
timeoutMs
);
}
async function waitForConflictTree(
cliBinary: string,
env: NodeJS.ProcessEnv,
path: string,
expectedBaseRev: string,
expectedBodies: readonly string[],
requireAncestorBody = true
): Promise<ConflictTreeSnapshot> {
const deadline = Date.now() + timeoutMs;
let last: ConflictTreeSnapshot | undefined;
while (Date.now() < deadline) {
try {
last = await readConflictTree(cliBinary, env, path, expectedBaseRev, requireAncestorBody);
const bodies = last.leaves.map((leaf) => leaf.body).sort();
if (last.leafRevs.length === 2 && JSON.stringify(bodies) === JSON.stringify([...expectedBodies].sort())) {
return last;
}
} catch (error) {
if (Date.now() + 250 >= deadline) throw error;
}
await new Promise((resolve) => setTimeout(resolve, 250));
}
throw new Error(`Timed out waiting for both exact conflict leaves: ${JSON.stringify(last)}`);
}
function assertSameConflictTree(actual: ConflictTreeSnapshot, expected: ConflictTreeSnapshot, label: string): void {
assertEqual(actual.id, expected.id, `${label}: Metadata ID changed.`);
assertEqual(
JSON.stringify(actual.leafRevs),
JSON.stringify(expected.leafRevs),
`${label}: leaf revision IDs changed.`
);
assertEqual(
JSON.stringify(
actual.leaves.map(({ rev, revisionTree, body, children, chunksPresent }) => ({
rev,
revisionTree,
body,
children,
chunksPresent,
}))
),
JSON.stringify(
expected.leaves.map(({ rev, revisionTree, body, children, chunksPresent }) => ({
rev,
revisionTree,
body,
children,
chunksPresent,
}))
),
`${label}: a leaf body, exact _revisions, or referenced Chunk changed.`
);
assertEqual(actual.ancestorRev, expected.ancestorRev, `${label}: the exact common ancestor changed.`);
if (!actual.ancestorBodyAvailable) return;
assertEqual(
JSON.stringify(
actual.ancestor && {
revisionTree: actual.ancestor.revisionTree,
revisions: actual.ancestor.revisions,
body: actual.ancestor.body,
children: actual.ancestor.children,
chunksPresent: actual.ancestor.chunksPresent,
}
),
JSON.stringify(
expected.ancestor && {
revisionTree: expected.ancestor.revisionTree,
revisions: expected.ancestor.revisions,
body: expected.ancestor.body,
children: expected.ancestor.children,
chunksPresent: expected.ancestor.chunksPresent,
}
),
`${label}: the common ancestor body or its Chunks changed.`
);
}
async function storeSourceLeaf(
cliBinary: string,
env: NodeJS.ProcessEnv,
path: string,
content: string,
baseRev: string
): Promise<void> {
await evalObsidianJson<unknown>(
cliBinary,
[
"(async()=>{",
`const path=${JSON.stringify(path)};`,
`const content=${JSON.stringify(content)};`,
"const core=app.plugins.plugins['obsidian-livesync'].core;",
"const file=app.vault.getFiles().find(candidate=>candidate.path.normalize('NFC')===path.normalize('NFC'));",
"if(!file) throw new Error('Could not find the NFC-equivalent source file '+path);",
"await app.vault.modify(file,content);",
"await core.services.fileProcessing.commitPendingFileEvents();",
"const deadline=Date.now()+20000;",
"const sleep=(ms)=>new Promise(resolve=>setTimeout(resolve,ms));",
"let entry=false;",
"let stored=false;",
"while(Date.now()<deadline){",
"entry=await core.localDatabase.getDBEntry(path,undefined,false,true,true).catch(()=>false);",
"const data=entry?.data;",
"const body=Array.isArray(data)?data.join(''):typeof data==='string'?data:data instanceof Blob?await data.text():null;",
`if(entry&&entry._rev!==${JSON.stringify(baseRev)}&&body===content){stored=true;break;}`,
"await sleep(250);",
"}",
"if(!stored) throw new Error('The real Obsidian source edit did not create a child revision with the expected body.');",
"const children=Array.isArray(entry.children)?entry.children:[];",
"if(children.length===0) throw new Error('The real Obsidian source revision has no Chunks.');",
"return JSON.stringify(true);",
"})()",
].join(""),
env,
timeoutMs
);
}
async function storeCompetingLeaf(
cliBinary: string,
env: NodeJS.ProcessEnv,
path: string,
content: string,
baseRev: string
): Promise<void> {
await evalObsidianJson<unknown>(
cliBinary,
[
"(async()=>{",
`const path=${JSON.stringify(path)};`,
`const content=${JSON.stringify(content)};`,
`const baseRev=${JSON.stringify(baseRev)};`,
"const core=app.plugins.plugins['obsidian-livesync'].core;",
"const id=await core.services.path.path2id(path);",
"const blob=new Blob([content],{type:'text/plain'});",
"const now=Date.now();",
"const result=await core.localDatabase.putDBEntry({",
"_id:id,path,data:blob,ctime:now,mtime:now,size:blob.size,children:[],datatype:'plain',type:'plain',eden:{},",
"},false,baseRev);",
"if(!result?.ok) throw new Error('Could not create the sibling conflict leaf from its exact baseline.');",
"const entry=await core.localDatabase.getDBEntry(path,{rev:result.rev},false,true,true);",
"if(!entry||!Array.isArray(entry.children)||entry.children.length===0) throw new Error('The sibling conflict leaf has no real Chunks.');",
"return JSON.stringify(true);",
"})()",
].join(""),
env,
timeoutMs
);
}
async function readTransferCheckpoint(cliBinary: string, env: NodeJS.ProcessEnv): Promise<TransferCheckpoint> {
return await evalObsidianJson<TransferCheckpoint>(
cliBinary,
[
"(async()=>{",
"const core=app.plugins.plugins['obsidian-livesync'].core;",
"const replicator=core.services.replicator.getActiveReplicator();",
"const client=replicator?.client;",
"if(!client||typeof client.getCheckpointInfo!=='function') throw new Error('The active Object Storage replicator did not expose its Journal checkpoint.');",
"const info=await client.getCheckpointInfo();",
"const values=value=>[...(value??[])].sort();",
"return JSON.stringify({lastLocalSeq:info.lastLocalSeq,journalEpoch:info.journalEpoch,knownIDs:values(info.knownIDs),sentIDs:values(info.sentIDs),receivedFiles:values(info.receivedFiles),sentFiles:values(info.sentFiles)});",
"})()",
].join(""),
env,
timeoutMs
);
}
function assertCheckpointPopulated(checkpoint: TransferCheckpoint, tree: ConflictTreeSnapshot): void {
if (checkpoint.sentFiles.length === 0 || checkpoint.sentIDs.length === 0) {
throw new Error("The initial Journal transfer did not populate its persistent send checkpoint.");
}
const localSequence = checkpoint.lastLocalSeq;
const sequenceIsPopulated =
typeof localSequence === "number"
? Number.isFinite(localSequence) && localSequence > 0
: typeof localSequence === "string" && localSequence.trim() !== "" && localSequence.trim() !== "0";
assertEqual(
sequenceIsPopulated,
true,
"The initial Journal transfer did not advance its local sequence checkpoint."
);
const sentIds = new Set(checkpoint.sentIDs);
const expectedSentIds = tree.leaves.map((leaf) => `${tree.id}-${leaf.rev}`);
for (const expectedId of expectedSentIds) {
if (!sentIds.has(expectedId))
throw new Error(`The Journal checkpoint does not record sent revision ${expectedId}.`);
}
}
function assertIndependentIdShape(tree: ConflictTreeSnapshot): void {
if (!/^f:[0-9a-f]{64}$/u.test(tree.id)) {
throw new Error(`The encrypted case did not use an independent Metadata ID: ${tree.id}`);
}
for (const revision of [...tree.leaves, ...(tree.ancestor ? [tree.ancestor] : [])]) {
if (revision.children.some((id) => !/^h:\+[0-9a-f]{64}$/u.test(id))) {
throw new Error("The encrypted case did not use independent Chunk IDs.");
}
}
}
async function waitForJournalObjects(
storage: ObjectStorageConfig,
prefix: string,
operationLabel: string
): Promise<string[]> {
const milestoneKey = `${prefix}_00000000-milestone.json`;
const deadline = Date.now() + objectTimeoutMs;
let keys: string[] = [];
while (Date.now() < deadline) {
keys = (await listObjectStorageObjects(storage, prefix)).flatMap((object) => (object.Key ? [object.Key] : []));
if (keys.includes(milestoneKey) && keys.some((key) => key.endsWith("-docs.jsonl.gz"))) return keys;
await new Promise((resolve) => setTimeout(resolve, 300));
}
throw new Error(
`${operationLabel} did not create both the Journal milestone and document objects: ${keys.join(", ")}`
);
}
async function postponeConflictIfOpen(session: ObsidianLiveSyncSession): Promise<void> {
await withObsidianPage(session.remoteDebuggingPort, async (page) => {
const dialogue = page.locator(".modal-container:visible").filter({ hasText: "Conflicting changes" });
if ((await dialogue.count()) === 0) return;
const notNow = dialogue.getByRole("button", { name: "Not now", exact: true });
if ((await notNow.count()) > 0) await notNow.click({ timeout: uiTimeoutMs });
});
}
async function rebuildRemote(cliBinary: string, env: NodeJS.ProcessEnv, port: number): Promise<void> {
await evalObsidianJson<unknown>(
cliBinary,
[
"(()=>{",
"const rebuilder=app.plugins.plugins['obsidian-livesync'].core.rebuilder;",
"const state={done:false,error:null};",
"globalThis.__journalNfcRemoteRebuild=state;",
"void Promise.resolve().then(()=>rebuilder.$rebuildRemote()).then(()=>{state.done=true;},error=>{state.error=String(error?.stack??error);});",
"return JSON.stringify(true);",
"})()",
].join(""),
env,
timeoutMs
);
await withObsidianPage(port, async (page) => {
const title = "All optional features are disabled";
const deadline = Date.now() + timeoutMs;
let acknowledged = false;
while (Date.now() < deadline) {
const state = await evalObsidianJson<{ done: boolean; error: string | null }>(
cliBinary,
"JSON.stringify(globalThis.__journalNfcRemoteRebuild)",
env,
timeoutMs
);
if (state.error) throw new Error(state.error);
if (state.done) {
if (!acknowledged) {
throw new Error(`Remote-only Rebuild finished without showing the '${title}' dialogue.`);
}
return;
}
if (!acknowledged) {
for (const candidate of page.context().pages()) {
const dialogue = candidate
.locator(".modal-container")
.filter({ has: candidate.locator(".modal-title").filter({ hasText: title }) })
.last();
if (!(await dialogue.isVisible().catch(() => false))) continue;
await dialogue.getByRole("button", { name: "OK", exact: true }).click({ timeout: uiTimeoutMs });
await dialogue.waitFor({ state: "hidden", timeout: uiTimeoutMs });
acknowledged = true;
break;
}
}
await new Promise((resolve) => setTimeout(resolve, 250));
}
const state = await evalObsidianJson<{ done: boolean; error: string | null }>(
cliBinary,
"JSON.stringify(globalThis.__journalNfcRemoteRebuild)",
env,
timeoutMs
);
if (state.error) throw new Error(state.error);
throw new Error("Timed out waiting for remote-only Rebuild and its optional-features dialogue.");
});
}
async function fetchIntoFreshVault(cliBinary: string, env: NodeJS.ProcessEnv, path: string): Promise<void> {
await evalObsidianJson<unknown>(
cliBinary,
[
"(async()=>{",
`const path=${JSON.stringify(path)};`,
"const core=app.plugins.plugins['obsidian-livesync'].core;",
"const existing=await core.localDatabase.getDBEntry(path,undefined,false,true,true).catch(()=>false);",
"if(existing) throw new Error('The fresh receiving database already contains the conflict Metadata.');",
"await core.rebuilder.$fetchLocal(false,true);",
"return JSON.stringify(true);",
"})()",
].join(""),
env,
timeoutMs
);
}
async function runCase(
binary: string,
cliBinary: string,
objectStorage: ObjectStorageConfig,
scenario: ScenarioCase
): Promise<void> {
const bucketPrefix = makeUniqueBucketPrefix(`journal-nfc-resend-${scenario.name}`);
const storage = { ...objectStorage, bucketPrefix };
const overrides = await makeScenarioSettings(scenario);
const sourceVault = await createTemporaryVault(`obsidian-livesync-journal-nfc-source-${scenario.name}-`);
let receiverVault: TemporaryVault | undefined;
let sourceSession: ObsidianLiveSyncSession | undefined;
let receiverSession: ObsidianLiveSyncSession | undefined;
try {
receiverVault = await createTemporaryVault(`obsidian-livesync-journal-nfc-receiver-${scenario.name}-`);
const sourcePort = await allocatePort(new Set());
const receiverPort = await allocatePort(new Set([sourcePort]));
assertEqual(sourcePort === receiverPort, false, "Source and receiver Obsidian debugging ports must differ.");
console.log(`Journal NFC resend case: ${scenario.name}`);
console.log(`Temporary source and receiver Vaults: ${sourceVault.name}, ${receiverVault.name}`);
console.log(`Temporary Object Storage prefix: ${bucketPrefix}`);
sourceSession = await startSession(binary, cliBinary, sourceVault, sourcePort, storage, overrides, true);
await prepareRemote(cliBinary, sourceSession.cliEnv);
await createVaultNote(cliBinary, sourceSession.cliEnv, conflictPathNfd, baseContent);
await createVaultNote(cliBinary, sourceSession.cliEnv, controlPathNfd, controlContent);
await assertSinglePhysicalUnicodeFile(sourceVault.path, controlPathNfd, controlContent);
const initialEntry: LocalDatabaseEntry = await waitForLocalDatabaseEntry(
cliBinary,
sourceSession.cliEnv,
conflictPathNfc,
{
timeoutMs,
}
);
if (!initialEntry.rev || initialEntry.children.length === 0) {
throw new Error("The source baseline was not saved as a Chunk-backed Metadata revision.");
}
await storeSourceLeaf(cliBinary, sourceSession.cliEnv, conflictPathNfc, sourceLeafContent, initialEntry.rev);
await storeCompetingLeaf(
cliBinary,
sourceSession.cliEnv,
conflictPathNfc,
competingLeafContent,
initialEntry.rev
);
const sourceTree = await waitForConflictTree(
cliBinary,
sourceSession.cliEnv,
conflictPathNfc,
initialEntry.rev,
[sourceLeafContent, competingLeafContent]
);
if (!sourceTree.ancestor || sourceTree.ancestor.rev !== initialEntry.rev) {
throw new Error(
"The two source leaves did not retain the exact available baseline as their nearest common ancestor."
);
}
assertEqual(
sourceTree.ancestor.body,
baseContent,
"The source conflict ancestor did not preserve the exact seeded baseline body."
);
await postponeConflictIfOpen(sourceSession);
if (scenario.independentIds) {
assertIndependentIdShape(sourceTree);
} else {
assertEqual(
sourceTree.id,
conflictPathNfc.toLowerCase(),
"The plain case did not derive its Metadata ID from the NFC path."
);
}
await pushLocalChanges(cliBinary, sourceSession.cliEnv);
const initialKeys = await waitForJournalObjects(objectStorage, bucketPrefix, "Initial Journal upload");
const initialCheckpoint = await readTransferCheckpoint(cliBinary, sourceSession.cliEnv);
assertCheckpointPopulated(initialCheckpoint, sourceTree);
await readObjectStorageJson<Record<string, unknown>>(objectStorage, `${bucketPrefix}_00000000-milestone.json`);
await postponeConflictIfOpen(sourceSession);
// Removing this fixture prefix leaves the source PouchDB tree and its device-local transfer checkpoint intact.
await deleteObjectStoragePrefix(objectStorage, bucketPrefix);
assertEqual(
(await listObjectStorageObjects(objectStorage, bucketPrefix)).length,
0,
"The test's isolated Object Storage prefix was not empty after deletion."
);
assertEqual(
JSON.stringify(await readTransferCheckpoint(cliBinary, sourceSession.cliEnv)),
JSON.stringify(initialCheckpoint),
"Deleting the remote fixture prefix changed the source transfer checkpoint."
);
assertSameConflictTree(
await readConflictTree(cliBinary, sourceSession.cliEnv, conflictPathNfc, initialEntry.rev),
sourceTree,
"Source before remote-only Rebuild"
);
await rebuildRemote(cliBinary, sourceSession.cliEnv, sourceSession.remoteDebuggingPort);
const rebuiltKeys = await waitForJournalObjects(objectStorage, bucketPrefix, "Remote-only Rebuild");
await postponeConflictIfOpen(sourceSession);
assertSameConflictTree(
await readConflictTree(cliBinary, sourceSession.cliEnv, conflictPathNfc, initialEntry.rev),
sourceTree,
"Source after remote-only Rebuild"
);
if (scenario.independentIds) assertIndependentIdShape(sourceTree);
await sourceSession.app.stop();
sourceSession = undefined;
receiverSession = await startSession(binary, cliBinary, receiverVault, receiverPort, storage, overrides, true);
await fetchIntoFreshVault(cliBinary, receiverSession.cliEnv, conflictPathNfc);
await waitForLiveSyncCoreReady(cliBinary, receiverSession.cliEnv, timeoutMs);
await postponeConflictIfOpen(receiverSession);
const receivedTree = await waitForConflictTree(
cliBinary,
receiverSession.cliEnv,
conflictPathNfc,
initialEntry.rev,
[sourceLeafContent, competingLeafContent],
false
);
await postponeConflictIfOpen(receiverSession);
assertSameConflictTree(receivedTree, sourceTree, "Fresh receiver after Standard Fetch");
await waitForSinglePhysicalUnicodeFile(receiverVault.path, controlPathNfd, controlContent);
await receiverSession.app.stop();
receiverSession = undefined;
const restartPort = await allocatePort(new Set([sourcePort, receiverPort]));
receiverSession = await startSession(binary, cliBinary, receiverVault, restartPort, storage, overrides, false);
await waitForLiveSyncCoreReady(cliBinary, receiverSession.cliEnv, timeoutMs);
await postponeConflictIfOpen(receiverSession);
const restartedTree = await waitForConflictTree(
cliBinary,
receiverSession.cliEnv,
conflictPathNfc,
initialEntry.rev,
[sourceLeafContent, competingLeafContent],
false
);
await postponeConflictIfOpen(receiverSession);
assertSameConflictTree(restartedTree, sourceTree, "Receiver after full Obsidian restart");
assertEqual(
restartedTree.ancestorBodyAvailable,
receivedTree.ancestorBodyAvailable,
"Restart changed the common ancestor body's availability."
);
await waitForSinglePhysicalUnicodeFile(receiverVault.path, controlPathNfd, controlContent);
const initialJournalObjects = initialKeys.filter((key) => key.endsWith("-docs.jsonl.gz")).length;
const rebuiltJournalObjects = rebuiltKeys.filter((key) => key.endsWith("-docs.jsonl.gz")).length;
console.log(
`Journal NFC resend ${scenario.name}: source checkpoint survived external prefix deletion, ${rebuiltJournalObjects} rebuilt Journal object(s), exact conflict leaves and ancestry fetched, and restart verified (initial objects: ${initialJournalObjects}; receiver ancestor body available: ${receivedTree.ancestorBodyAvailable}).`
);
console.log(
"The history assertion covers this deliberately uncompacted fixture and does not claim arbitrary historical body availability."
);
} finally {
if (receiverSession) {
await receiverSession.app.stop().catch((error: unknown) => {
console.warn(error instanceof Error ? error.message : error);
});
}
if (sourceSession) {
await sourceSession.app.stop().catch((error: unknown) => {
console.warn(error instanceof Error ? error.message : error);
});
}
await sourceVault.dispose().catch((error: unknown) => {
console.warn(error instanceof Error ? error.message : error);
});
if (receiverVault) {
await receiverVault.dispose().catch((error: unknown) => {
console.warn(error instanceof Error ? error.message : error);
});
}
await deleteObjectStoragePrefix(objectStorage, bucketPrefix).catch((error: unknown) => {
console.warn(error instanceof Error ? error.message : error);
});
}
}
async function main(): Promise<void> {
const scenarios = selectedCases(process.argv.slice(2));
const binary = requireObsidianBinary();
const cli = discoverObsidianCli();
if (!cli.binary) throw new Error(`Could not find obsidian-cli. Checked paths: ${cli.checked.join(", ")}`);
const objectStorage = await loadObjectStorageConfig();
await ensureObjectStorageBucket(objectStorage);
for (const scenario of scenarios) await runCase(binary, cli.binary, objectStorage, scenario);
}
main().catch((error: unknown) => {
console.error(error instanceof Error ? error.stack : error);
process.exit(1);
});
+305 -21
View File
@@ -1,11 +1,12 @@
import { randomBytes } from "node:crypto";
import { access, mkdir, readFile, writeFile } from "node:fs/promises";
import { access, mkdir, readFile, readdir, writeFile } from "node:fs/promises";
import { connect } from "node:net";
import { dirname, join } from "node:path";
import type { ObsidianLiveSyncSettings } from "@vrtmrz/livesync-commonlib/compat/common/types";
import { encodeTimeBoundSetupURI } from "@vrtmrz/livesync-commonlib/setup-uri";
import { createNewVaultSettings, P2P_DEFAULT_SETTINGS, PREFERRED_BASE } from "@vrtmrz/livesync-commonlib/settings";
import { upsertRemoteConfigurationInPlace } from "@vrtmrz/livesync-commonlib/remote-configurations";
import { path2id_base } from "@vrtmrz/livesync-commonlib/compat/string_and_binary/path";
import { evalObsidianJson } from "../runner/cli.ts";
import { discoverObsidianCli, requireObsidianBinary } from "../runner/environment.ts";
import {
@@ -73,9 +74,10 @@ function selectedCases(arguments_: readonly string[]): CaseDefinition[] {
for (let index = 0; index < arguments_.length; index += 1) {
const argument = arguments_[index];
if (argument === "--help" || argument === "-h") {
console.log("Usage: p2p-nfc-sync.ts [--case all|plain|encrypted]");
console.log("Usage: p2p-nfc-sync.ts [--case all|plain|encrypted] [--migration]");
process.exit(0);
}
if (argument === "--migration") continue;
if (argument === "--case") {
const value = arguments_[index + 1];
if (!value) throw new Error("--case requires all, plain, or encrypted.");
@@ -242,16 +244,20 @@ async function assertSourceNameIsNfd(vaultPath: string, expectedContent: string)
}
async function readVaultFileState(vaultPath: string, requestedPath: string): Promise<VaultFileState> {
const candidates = [...new Set([requestedPath, requestedPath.normalize("NFD")])];
for (const candidate of candidates) {
try {
const content = await readFile(join(vaultPath, candidate), "utf8");
return { found: true, path: candidate, content };
} catch (error) {
if ((error as NodeJS.ErrnoException).code !== "ENOENT") throw error;
try {
let physicalPath = "";
for (const component of requestedPath.split("/")) {
const names = await readdir(join(vaultPath, physicalPath));
const matches = names.filter((name) => name.normalize("NFC") === component.normalize("NFC"));
if (matches.length > 1) throw new Error("Ambiguous physical names at " + join(physicalPath, component));
if (matches.length === 0) return { found: false, path: null, content: null };
physicalPath = join(physicalPath, matches[0]);
}
return { found: true, path: physicalPath, content: await readFile(join(vaultPath, physicalPath), "utf8") };
} catch (error) {
if ((error as NodeJS.ErrnoException).code !== "ENOENT") throw error;
return { found: false, path: null, content: null };
}
return { found: false, path: null, content: null };
}
async function waitForVaultContent(
@@ -297,7 +303,8 @@ async function readEntryState(
cliBinary: string,
environment: NodeJS.ProcessEnv,
path: string,
revision?: string
revision?: string,
documentId?: string
): Promise<EntryState> {
return await evalObsidianJson<EntryState>(
cliBinary,
@@ -305,8 +312,9 @@ async function readEntryState(
"(async()=>{",
"const path=" + JSON.stringify(path) + ";",
"const revision=" + JSON.stringify(revision ?? null) + ";",
"const documentId=" + JSON.stringify(documentId ?? null) + ";",
"const core=app.plugins.plugins['obsidian-livesync'].core;",
"const entry=await core.localDatabase.getDBEntry(path,revision?{rev:revision}:undefined,false,true,true);",
"const entry=documentId?await core.localDatabase.getDBEntryFromMeta(await core.localDatabase.getRaw(documentId,revision?{rev:revision}:undefined),false,true):await core.localDatabase.getDBEntry(path,revision?{rev:revision}:undefined,false,true,true);",
"if(!entry||!entry._id||!entry._rev) throw new Error('Metadata was not available at '+path);",
"const children=Array.isArray(entry.children)?entry.children:[];",
"const chunkRows=children.length?await core.localDatabase.allDocsRaw({keys:children,include_docs:true}):{rows:[]};",
@@ -577,7 +585,150 @@ async function readMetadataIds(
);
}
async function runCase(definition: CaseDefinition): Promise<void> {
async function seedLegacyNfdMetadata(
context: RunnerContext,
session: ObsidianLiveSyncSession,
shared: SharedSettings,
currentId: string
): Promise<EntryState> {
// Obsidian's file cache normalises names even in legacy mode. Seed the
// legacy ID of an NFD-writing host while retaining real Chunk documents.
const legacyId = await path2id_base(
pathNfd,
shared.encrypted ? shared.e2eePassphrase : false,
true,
shared.encrypted ? shared.idKey : undefined
);
if (legacyId === currentId) throw new Error("The legacy NFD fixture did not change the Metadata ID.");
await evalObsidianJson<boolean>(
context.cliBinary,
[
"(async()=>{",
"const db=app.plugins.plugins['obsidian-livesync'].core.localDatabase.localDatabase;",
"const original=await db.get(" + JSON.stringify(currentId) + ");",
"const legacy={...original,_id:" + JSON.stringify(legacyId) + ",path:" + JSON.stringify(pathNfd) + "};",
"delete legacy._rev;delete legacy._revisions;delete legacy._conflicts;",
"await db.put(legacy);await db.remove(original);",
"return JSON.stringify(true);",
"})()",
].join(""),
session.cliEnv
);
const legacy = await readEntryState(context.cliBinary, session.cliEnv, pathNfd, undefined, legacyId);
assertEqual(legacy.content, sourceContent, "The legacy NFD fixture lost its Chunk-backed body.");
assertEqual(legacy.chunksPresent, true, "The legacy NFD fixture lost its Chunk documents.");
return legacy;
}
async function startReconstruction(
cliBinary: string,
environment: NodeJS.ProcessEnv,
operation: "rebuild" | "fetch",
pathNormalizationVersion: 0 | 1 = 1
): Promise<void> {
await evalObsidianJson<boolean>(
cliBinary,
[
"(()=>{",
"const core=app.plugins.plugins['obsidian-livesync'].core;",
"globalThis.__p2pNfcReconstruction={done:false,error:null};",
"void(async()=>{",
"await core.services.setting.applyPartial({pathNormalizationVersion:" + pathNormalizationVersion + "});",
"await core.services.setting.saveSettingData();",
"if(core.settings.pathNormalizationVersion!==" +
pathNormalizationVersion +
")throw new Error('The test did not apply the selected identity: '+core.settings.pathNormalizationVersion);",
operation === "rebuild"
? "await core.rebuilder.$rebuildEverything();"
: "await core.rebuilder.$fetchLocal(false,true);",
"})().then(()=>{globalThis.__p2pNfcReconstruction.done=true;},error=>{globalThis.__p2pNfcReconstruction.error=String(error?.stack??error);});",
"return JSON.stringify(true);",
"})()",
].join(""),
environment
);
}
async function waitForReconstruction(cliBinary: string, environment: NodeJS.ProcessEnv): Promise<void> {
const deadline = Date.now() + syncTimeoutMs;
while (Date.now() < deadline) {
const state = await evalObsidianJson<{ done: boolean; error: string | null }>(
cliBinary,
"JSON.stringify(globalThis.__p2pNfcReconstruction)",
environment
);
if (state.error) throw new Error(state.error);
if (state.done) return;
await new Promise((resolve) => setTimeout(resolve, 250));
}
throw new Error("The P2P reconstruction did not finish.");
}
async function waitForVaultAbsence(vaultPath: string, path: string): Promise<void> {
const deadline = Date.now() + syncTimeoutMs;
while (Date.now() < deadline) {
if (!(await readVaultFileState(vaultPath, path)).found) return;
await new Promise((resolve) => setTimeout(resolve, 250));
}
throw new Error("The retired path is still present: " + path);
}
async function verifyRenameAndDeletion(
context: RunnerContext,
sessionA: ObsidianLiveSyncSession,
sessionB: ObsidianLiveSyncSession,
vaultA: TemporaryVault,
vaultB: TemporaryVault,
portA: number,
portB: number
): Promise<void> {
const oldPath = pathNfc.replace(/[^/]+$/, "a\u0301rbol.md").normalize("NFC");
const newPath = pathNfc.replace(/[^/]+$/, "renamed-e\u0301.md").normalize("NFC");
const content = "# Rename and deletion\n\nPreserve these exact bytes through the P2P rename.\n";
await evalObsidianJson<boolean>(
context.cliBinary,
`(async()=>{const source=app.vault.getFiles().find(file=>file.path.normalize('NFC')===${JSON.stringify(pathNfc)});if(!source)throw new Error('Missing source note');await app.vault.create(source.path.replace(/[^/]+$/,'a\\u0301rbol.md'),${JSON.stringify(content)});return JSON.stringify(true);})()`,
sessionA.cliEnv
);
const before = await writeEntryStateWhenReady(context.cliBinary, sessionA.cliEnv, oldPath, content);
await replicateAndWait(portA, portB, vaultB.path, oldPath, content);
const received = await writeEntryStateWhenReady(context.cliBinary, sessionB.cliEnv, oldPath, content);
assertEqual(received.id, before.id, "The new Unicode note has different peer IDs.");
await evalObsidianJson<boolean>(
context.cliBinary,
`(async()=>{const file=app.vault.getFiles().find(file=>file.path.normalize('NFC')===${JSON.stringify(oldPath)});if(!file)throw new Error('Missing rename source');await app.fileManager.renameFile(file,file.path.replace(/[^/]+$/,'renamed-e\\u0301.md'));return JSON.stringify(true);})()`,
sessionB.cliEnv
);
const renamed = await writeEntryStateWhenReady(context.cliBinary, sessionB.cliEnv, newPath, content);
if (renamed.id === before.id) throw new Error("The cross-path rename retained the old Metadata ID.");
await replicateAndWait(portB, portA, vaultA.path, newPath, content);
const renamedAtSource = await writeEntryStateWhenReady(context.cliBinary, sessionA.cliEnv, newPath, content);
assertEqual(renamedAtSource.id, renamed.id, "The renamed note has different peer IDs.");
await waitForVaultAbsence(vaultA.path, oldPath);
await waitForVaultAbsence(vaultB.path, oldPath);
await evalObsidianJson<boolean>(
context.cliBinary,
`(async()=>{const file=app.vault.getFiles().find(file=>file.path.normalize('NFC')===${JSON.stringify(newPath)});if(!file)throw new Error('Missing deletion source');await app.vault.delete(file);return JSON.stringify(true);})()`,
sessionA.cliEnv
);
let finished = false;
const acceptor = acceptConnectionRequests([portA, portB], () => finished);
try {
const deadline = Date.now() + syncTimeoutMs;
while ((await readVaultFileState(vaultB.path, newPath)).found && Date.now() < deadline) {
await replicateFromStatusPane(portB);
await new Promise((resolve) => setTimeout(resolve, 500));
}
await waitForVaultAbsence(vaultA.path, newPath);
await waitForVaultAbsence(vaultB.path, newPath);
} finally {
finished = true;
await acceptor;
}
console.log("P2P NFC rename and deletion: canonical IDs, exact content, and both physical removals verified.");
}
async function runCase(definition: CaseDefinition, migration: boolean): Promise<void> {
if (pathNfd === pathNfc) throw new Error("The test fixture path did not produce distinct NFC and NFD spellings.");
const binary = requireObsidianBinary();
const cli = discoverObsidianCli();
@@ -593,7 +744,9 @@ async function runCase(definition: CaseDefinition): Promise<void> {
relay,
roomId: "real-obsidian-p2p-nfc-" + randomBytes(12).toString("hex"),
};
const artifact = await createSetupArtifact(shared, 1);
const initialVersion = migration ? 0 : 1;
const initialPath = migration ? pathNfd : pathNfc;
const artifact = await createSetupArtifact(shared, initialVersion);
const incompatibleArtifact = await createSetupArtifact(shared, 0);
const captures: SetupCaptureNames = {
scenario: "p2p-nfc-" + definition.name,
@@ -610,6 +763,7 @@ async function runCase(definition: CaseDefinition): Promise<void> {
const vaultB = await createTemporaryVault("obsidian-livesync-p2p-nfc-receiver-");
vaults.push(vaultB);
await writeVaultFile(vaultA.path, pathNfd, sourceContent);
if (migration) await writeVaultFile(vaultB.path, pathNfd, sourceContent);
console.log("P2P NFC case: " + definition.name);
console.log("Temporary source and receiving Vaults: " + vaultA.name + ", " + vaultB.name);
@@ -623,9 +777,14 @@ async function runCase(definition: CaseDefinition): Promise<void> {
const firstState = await finishInitialisation(portA, context.cliBinary, sessionA.cliEnv);
await resumeCompatibilityReviewIfShown(portA);
assertEqual(firstState.p2pEnabled, true, "The first device did not enable P2P.");
await assertCaseSettings(context.cliBinary, sessionA.cliEnv, definition, shared.idKey, 1);
await assertCaseSettings(context.cliBinary, sessionA.cliEnv, definition, shared.idKey, initialVersion);
await waitForVaultContent(vaultA.path, pathNfc, sourceContent);
const sourceEntry = await writeEntryStateWhenReady(context.cliBinary, sessionA.cliEnv, pathNfc, sourceContent);
let sourceEntry = await writeEntryStateWhenReady(
context.cliBinary,
sessionA.cliEnv,
initialPath,
sourceContent
);
assertEqual(sourceEntry.path.normalize("NFC"), pathNfc, "The source Metadata path is not NFC-equivalent.");
await assertSourceNameIsNfd(vaultA.path, sourceContent);
@@ -635,16 +794,16 @@ async function runCase(definition: CaseDefinition): Promise<void> {
await captureAndStartInitialisation(portB, "existing", captures);
await confirmFastFetch(portB, captures);
await fetchFromFirstPeer(portA, portB);
await waitForLocalDatabaseEntry(context.cliBinary, sessionB.cliEnv, pathNfc, { timeoutMs: syncTimeoutMs });
await waitForLocalDatabaseEntry(context.cliBinary, sessionB.cliEnv, initialPath, { timeoutMs: syncTimeoutMs });
const secondState = await finishInitialisation(portB, context.cliBinary, sessionB.cliEnv);
await resumeCompatibilityReviewIfShown(portB);
assertEqual(secondState.p2pEnabled, true, "The second device did not enable P2P.");
await assertCaseSettings(context.cliBinary, sessionB.cliEnv, definition, shared.idKey, 1);
await assertCaseSettings(context.cliBinary, sessionB.cliEnv, definition, shared.idKey, initialVersion);
await waitForVaultContent(vaultB.path, pathNfc, sourceContent);
const receiverEntry = await writeEntryStateWhenReady(
let receiverEntry = await writeEntryStateWhenReady(
context.cliBinary,
sessionB.cliEnv,
pathNfc,
initialPath,
sourceContent
);
assertEqual(
@@ -654,6 +813,129 @@ async function runCase(definition: CaseDefinition): Promise<void> {
);
assertEqual(receiverEntry.content, sourceContent, "The NFC peer did not receive the exact source body.");
if (migration) {
sourceEntry = await seedLegacyNfdMetadata(context, sessionA, shared, sourceEntry.id);
receiverEntry = await seedLegacyNfdMetadata(context, sessionB, shared, receiverEntry.id);
assertEqual(receiverEntry.id, sourceEntry.id, "The legacy NFD peers have different IDs.");
const legacyId = sourceEntry.id;
await assertSourceNameIsNfd(vaultB.path, sourceContent);
await stopSession(context, sessionB);
await startReconstruction(context.cliBinary, sessionA.cliEnv, "rebuild");
await acknowledgeDisabledOptionalFeatures(portA, captures);
await waitForReconstruction(context.cliBinary, sessionA.cliEnv);
sourceEntry = await writeEntryStateWhenReady(context.cliBinary, sessionA.cliEnv, pathNfc, sourceContent);
if (sourceEntry.id === legacyId)
throw new Error(
"P2P Rebuild did not replace the legacy Metadata ID: " +
JSON.stringify({
legacyId,
sourceEntry,
settings: await readCurrentSettings(context.cliBinary, sessionA.cliEnv, shared.idKey),
})
);
assertEqual(
(await readMetadataIds(context.cliBinary, sessionA.cliEnv)).ids.includes(legacyId),
false,
"The rebuilt source retained active legacy Metadata."
);
await assertSourceNameIsNfd(vaultA.path, sourceContent);
await stopSession(context, sessionA);
sessionA = await startSession(context, vaultA, portA);
await waitForLiveSyncCoreReady(context.cliBinary, sessionA.cliEnv);
await assertCaseSettings(context.cliBinary, sessionA.cliEnv, definition, shared.idKey, 1);
sessionB = await startSession(context, vaultB, portB);
await waitForLiveSyncCoreReady(context.cliBinary, sessionB.cliEnv);
await assertCaseSettings(context.cliBinary, sessionB.cliEnv, definition, shared.idKey, 0);
await openP2PStatus(portA);
await openP2PStatus(portB);
await waitForDetectedPeer(portB);
let finished = false;
const acceptor = acceptConnectionRequests([portA, portB], () => finished);
try {
await replicateFromStatusPane(portB);
await waitForTweakMismatchNotice(portB, sessionB);
} finally {
finished = true;
await acceptor;
}
const refused = await readEntryState(context.cliBinary, sessionB.cliEnv, pathNfd, undefined, legacyId);
assertEqual(refused.id, receiverEntry.id, "The refused legacy receiver changed its Metadata ID.");
assertEqual(refused.rev, receiverEntry.rev, "The refused legacy receiver changed its revision.");
assertEqual(refused.content, sourceContent, "The refused legacy receiver changed its content.");
await withObsidianPage(portB, async (page) => {
await page.locator(".notice").evaluateAll((notices) => notices.forEach((notice) => notice.remove()));
});
await startReconstruction(context.cliBinary, sessionB.cliEnv, "fetch", 0);
await withObsidianPage(portB, async (page) => {
await modalByTitle(page, "P2P Rebuild").locator(".peer-item").first().waitFor({
state: "visible",
timeout: uiTimeoutMs,
});
});
finished = false;
const fetchAcceptor = acceptConnectionRequests([portA, portB], () => finished);
try {
await withObsidianPage(portB, async (page) => {
await modalByTitle(page, "P2P Rebuild")
.locator(".peer-item")
.first()
.getByRole("button", { name: "Sync", exact: true })
.click({ timeout: uiTimeoutMs });
});
await waitForTweakMismatchNotice(portB, sessionB);
await withObsidianPage(portB, async (page) => {
const modal = modalByTitle(page, "P2P Rebuild");
if (await modal.isVisible()) {
await modal.getByRole("button", { name: "Skip and close" }).click({ timeout: uiTimeoutMs });
}
});
} finally {
finished = true;
await fetchAcceptor;
}
let fetchFailure = "";
try {
await waitForReconstruction(context.cliBinary, sessionB.cliEnv);
} catch (error) {
fetchFailure = error instanceof Error ? error.message : String(error);
}
if (!fetchFailure.includes("The first Standard Fetch pass did not complete.")) {
throw new Error("The incompatible P2P Fetch did not report its expected failure: " + fetchFailure);
}
const refusedFetchMetadata = await readMetadataIds(context.cliBinary, sessionB.cliEnv);
assertEqual(
refusedFetchMetadata.ids.includes(legacyId),
false,
"P2P Fetch did not replace the old local database before peer selection."
);
assertEqual(
refusedFetchMetadata.ids.includes(sourceEntry.id),
false,
"The incompatible Fetch imported source Metadata."
);
await assertSourceNameIsNfd(vaultB.path, sourceContent);
console.log(
"P2P incompatible Fetch: local database reset preceded refusal; physical NFD note and exact bytes retained for retry."
);
await startReconstruction(context.cliBinary, sessionB.cliEnv, "fetch");
await fetchFromFirstPeer(portA, portB);
await waitForReconstruction(context.cliBinary, sessionB.cliEnv);
await waitForLiveSyncCoreReady(context.cliBinary, sessionB.cliEnv);
const migrated = await writeEntryStateWhenReady(context.cliBinary, sessionB.cliEnv, pathNfc, sourceContent);
assertEqual(migrated.id, sourceEntry.id, "P2P Fetch did not converge to the rebuilt source ID.");
assertEqual(
(await readMetadataIds(context.cliBinary, sessionB.cliEnv)).ids.includes(legacyId),
false,
"The migrated receiver retained active legacy Metadata."
);
await assertSourceNameIsNfd(vaultB.path, sourceContent);
console.log(
"P2P legacy-to-NFC migration: source Rebuild and restart, legacy refusal, receiver Fetch, matching IDs, exact content, and NFD physical names verified."
);
}
console.log("P2P NFC initial Fetch: matching IDs and exact content verified.");
await reconnectP2PStatus(portA);
await reconnectP2PStatus(portB);
@@ -712,6 +994,7 @@ async function runCase(definition: CaseDefinition): Promise<void> {
await assertSourceNameIsNfd(vaultA.path, restartedContent);
console.log("P2P NFC full restart: persisted data and another transfer verified.");
await verifyRenameAndDeletion(context, sessionA, sessionB, vaultA, vaultB, portA, portB);
await stopSession(context, sessionB);
sessionB = undefined;
const vaultMode0 = await createTemporaryVault("obsidian-livesync-p2p-nfc-mode0-");
@@ -798,7 +1081,8 @@ async function runCase(definition: CaseDefinition): Promise<void> {
async function main(): Promise<void> {
const cases = selectedCases(process.argv.slice(2));
for (const definition of cases) await runCase(definition);
const migration = process.argv.includes("--migration");
for (const definition of cases) await runCase(definition, migration);
}
main().catch((error: unknown) => {
+1
View File
@@ -30,6 +30,7 @@ const focusedScenarios = new Set([
"couchdb-manual-setup-workflow",
"cli-to-obsidian-sync",
"minio-upload",
"journal-nfc-resend",
"object-storage-setup-uri-workflow",
"object-storage-compatible-setup-uri-workflow",
"object-storage-qr-workflow",