Merge internal Metadata integration for stacked Setup URI validation

This commit is contained in:
vorotamoroz
2026-09-29 06:38:27 +00:00
26 changed files with 1167 additions and 44 deletions
+5 -1
View File
@@ -137,7 +137,7 @@ The mobile pass uses Obsidian's `app.emulateMobile(true)`, a 390 by 844 CSS-pixe
The same workflow checks the two remote-activity status boundaries. It first holds a real CouchDB request at the selected fetch implementation and confirms that `🌐N` is visible while `📲` is absent. It then holds the real one-shot replication immediately before its replicator call, confirms that `📲` is visible while no physical request is active, releases it, and requires the finite and bounded activity counts to return to zero, the request and response counts to balance, and both indicators to disappear. Finally, it creates a remote-only chunk, holds the real on-demand fetch immediately before its remote call, makes the same logical active and idle assertions, and verifies that the fetched chunk is written into the local database. These gates make the active states deterministic without replacing the remote request or operation.
`test:e2e:obsidian:couchdb-manual-setup-workflow` follows the visible onboarding path for the first device when no Setup URI is available. It enters end-to-end encryption and CouchDB details, runs the read-only `Check server requirements` step, requires the prepared fixture to pass without applying a server fix, and lets the onboarding connection test create the named database. After Rebuild completes on the first device, it creates an ordinary note, asks that working device to generate a Setup URI for a second device, completes Fetch there, and verifies a bidirectional note round-trip. The workflow captures each decision point and the expanded server-check result; password controls remain visually masked.
`test:e2e:obsidian:couchdb-manual-setup-workflow` follows the visible onboarding path for the first device when no Setup URI is available. It enters end-to-end encryption and CouchDB details, runs the read-only `Check server requirements` step, requires the prepared fixture to pass without applying a server fix, and lets the onboarding connection test create the named database. After Rebuild completes on the first device, it creates an ordinary note, asks that working device to generate a Setup URI for a second device, completes Fetch there, and verifies a bidirectional note round-trip. The workflow captures each decision point and the expanded server-check result; password controls remain visually masked. It uses an E2EE passphrase beginning with `%`, confirms that the saved settings do not contain it in plain text, and checks that Obsidian restores it after restarting with the first Vault.
If this status workflow fails while Obsidian is running, it writes a full-page screenshot and a JSON snapshot of the status text and counters under `/tmp/obsidian-livesync-e2e`. The dialogue-mount workflow leaves desktop and mobile screenshots for both representative Svelte routes, the Hidden File Sync workflow captures the successfully displayed JSON Resolve dialogue before selecting an option, and the Security Seed reconnect workflow captures each significant application state. The suite therefore records representative evidence without capturing every interaction. Set `E2E_OBSIDIAN_DIAGNOSTICS_DIR` to use another directory.
@@ -202,6 +202,10 @@ This proves in real Obsidian the plug-in behaviour shared by supported platforms
`test:e2e:obsidian:customisation-sync` runs a two-vault Customisation Sync workflow. It scans a real snippet CSS file, config JSON file, and sample plug-in fixture into per-file Customisation Sync data, synchronises the entries through CouchDB, applies them on the second vault, verifies the resulting `.obsidian` files, propagates a snippet update, and verifies deletion of the source-vault snippet sync data without confusing it with the target vault's own applied copy.
`test:e2e:obsidian:remote-feature-change` starts real Obsidian with continuous CouchDB replication, then changes the remote version document from generation 12 to generation 13 with an unknown feature. It waits for the control document to reach the local database and the active Replicator to retire, checks that another replication is refused, and verifies that an already accepted Vault note remains intact. After restarting the same Vault, the current remote declaration still blocks finite replication and the actual continuous connection attempt. No KV feature history is involved. It is a focused test outside `test:e2e:obsidian:local-suite`; recovery with a future compatible client remains a separate validation boundary.
`test:e2e:obsidian:internal-metadata-migration` enables internal Metadata encryption through the settings UI without Rebuild. It checks unchanged plaintext and rewritten encrypted Hidden File Sync and Customisation Sync Metadata in CouchDB, stable document IDs, mismatch rejection on a second device, and file restoration after aligning settings. It then turns the preference OFF, runs Fast Fetch, and compares content loaded from both Metadata representations and their Chunks while retaining the remote feature declaration. These focused tests use the local CouchDB fixture and are outside `test:e2e:obsidian:local-suite`.
`test:e2e:obsidian:setting-markdown-export` enables setting Markdown export, waits for the generated Markdown file in the vault, and verifies that credentials are omitted when `writeCredentialsForSettingSync=false`.
`test:e2e:obsidian:upgrade-from-stable` is the release-acceptance upgrade workflow. It installs the exact published 0.25.83 artefacts into an isolated Vault, verifies their pinned SHA-256 values, and then replaces only the plug-in artefacts with the current target while retaining the same Vault and isolated Obsidian profile. The first run downloads the old release into the ignored `_testdata/releases` cache; every later run verifies the cached bytes before use.
@@ -210,6 +210,7 @@ async function configureLiveSyncCli(
encrypt: true,
passphrase: e2eePassphrase,
usePathObfuscation: true,
encryptInternalMetadata: true,
doctorProcessedVersion: "0.25.27",
isConfigured: true,
});
@@ -323,6 +324,7 @@ async function main(): Promise<void> {
encrypt: true,
passphrase: e2eePassphrase,
usePathObfuscation: true,
encryptInternalMetadata: true,
E2EEAlgorithm: "v2",
}
),
@@ -39,6 +39,7 @@ process.env.E2E_OBSIDIAN_CLI_TIMEOUT_MS ??= "90000";
process.env.E2E_OBSIDIAN_COUCHDB_TIMEOUT_MS ??= "30000";
const uiTimeoutMs = Number(process.env.E2E_OBSIDIAN_SETUP_URI_TIMEOUT_MS ?? 30000);
const e2eePassphrase = `%${randomBytes(24).toString("base64url")}`;
const notePath = "E2E/manual-couchdb/from-first-device.md";
const noteContent = "# Manual CouchDB setup\n\nThis note was sent by the manually configured first device.\n";
const returnNotePath = "E2E/manual-couchdb/from-second-device.md";
@@ -147,8 +148,7 @@ async function enterManualCouchDBSettings(port: number, couchDb: CouchDbConfig,
.locator('input[type="checkbox"]')
.first()
.check({ timeout: uiTimeoutMs });
const passphraseValue = randomBytes(24).toString("base64url");
await passphraseInput.fill(passphraseValue);
await passphraseInput.fill(e2eePassphrase);
const passwordToggle = encryption.locator("button.sls-password-toggle");
await passwordToggle.click({ timeout: uiTimeoutMs });
assertEqual(
@@ -158,7 +158,7 @@ async function enterManualCouchDBSettings(port: number, couchDb: CouchDbConfig,
);
assertEqual(
await passphraseInput.inputValue(),
passphraseValue,
e2eePassphrase,
"Toggling visibility changed the passphrase value."
);
await passwordToggle.click({ timeout: uiTimeoutMs });
@@ -169,7 +169,7 @@ async function enterManualCouchDBSettings(port: number, couchDb: CouchDbConfig,
);
assertEqual(
await passphraseInput.inputValue(),
passphraseValue,
e2eePassphrase,
"Re-masking the passphrase changed its value."
);
});
@@ -301,6 +301,23 @@ async function assertPersistedE2EE(vault: TemporaryVault): Promise<void> {
if (typeof persisted.encryptedPassphrase !== "string" || persisted.encryptedPassphrase.length === 0) {
throw new Error("Manual CouchDB setup did not persist an encrypted E2EE passphrase.");
}
if (JSON.stringify(persisted).includes(e2eePassphrase)) {
throw new Error("Manual CouchDB setup persisted the E2EE passphrase in plain text.");
}
}
async function assertRestoredE2EEPassphrase(session: ObsidianLiveSyncSession, cliBinary: string): Promise<void> {
const restored = await evalObsidianJson<boolean>(
cliBinary,
[
"(()=>{",
"const settings=app.plugins.plugins['obsidian-livesync'].core.services.setting.currentSettings();",
`return JSON.stringify(settings.passphrase === ${JSON.stringify(e2eePassphrase)});`,
"})()",
].join(""),
session.cliEnv
);
assertEqual(restored, true, "The E2EE passphrase was not restored after Obsidian restarted.");
}
async function setRemotePreferredE2EEDisabled(context: RunnerContext): Promise<void> {
@@ -454,6 +471,7 @@ async function main(): Promise<void> {
session = await startUnconfiguredSession(context, vaultA);
try {
await assertRestoredE2EEPassphrase(session, context.cliBinary);
await scheduleRemoteOverwrite(session.remoteDebuggingPort);
screenshots.push(await confirmRebuild(session.remoteDebuggingPort, e2eeRebuildCaptures));
screenshots.push(
@@ -1,10 +1,13 @@
import { mkdir, readFile, rm, writeFile } from "node:fs/promises";
import { dirname, join } from "node:path";
import { VERSIONING_DOCID } from "@vrtmrz/livesync-commonlib/compat/common/types";
import { ENCRYPTED_INTERNAL_METADATA_FEATURE, REMOTE_FEATURE_GENERATION } from "@vrtmrz/livesync-commonlib/replication";
import { evalObsidianJson } from "../runner/cli.ts";
import {
assertCouchDbReachable,
createCouchDbDatabase,
deleteCouchDbDatabase,
fetchCouchDbDocument,
loadCouchDbConfig,
makeUniqueDatabaseName,
waitForCouchDbDocs,
@@ -159,6 +162,10 @@ async function startConfiguredSession(
dbName: context.dbName,
};
const customisationSettings = {
encrypt: true,
passphrase: "internal-metadata-e2e-secret",
usePathObfuscation: true,
encryptInternalMetadata: true,
deviceAndVaultName: deviceName,
usePluginSync: true,
usePluginSyncV2: true,
@@ -486,6 +493,18 @@ async function main(): Promise<void> {
(target) => ids.has(target.id) && target.children.every((childId) => ids.has(childId))
);
});
for (const target of [entry, configEntry, ...pluginEntries]) {
const remoteEntry = await fetchCouchDbDocument(context.couchDb, context.dbName, target.id);
if (!remoteEntry.path?.startsWith("/\\:") || remoteEntry.children?.length !== 0 ||
remoteEntry.ctime !== 0 || remoteEntry.mtime !== 0 || remoteEntry.size !== 0) {
throw new Error(`Customisation Sync Metadata was not encrypted for ${target.id}.`);
}
}
const versionInfo = await fetchCouchDbDocument(context.couchDb, context.dbName, VERSIONING_DOCID);
if (versionInfo.version !== REMOTE_FEATURE_GENERATION ||
!(versionInfo.used_features as unknown[] | undefined)?.includes(ENCRYPTED_INTERNAL_METADATA_FEATURE)) {
throw new Error("The remote feature list does not declare encrypted internal Metadata.");
}
await session.app.stop();
session = await startConfiguredSession(context, vaultB, targetDeviceName);
@@ -1,5 +1,7 @@
import { mkdir, readFile, rm, writeFile } from "node:fs/promises";
import { dirname, join } from "node:path";
import { VERSIONING_DOCID } from "@vrtmrz/livesync-commonlib/compat/common/types";
import { ENCRYPTED_INTERNAL_METADATA_FEATURE, REMOTE_FEATURE_GENERATION } from "@vrtmrz/livesync-commonlib/replication";
import {
assertLocatorHasMinimumTouchTarget,
assertLocatorWithinSafeArea,
@@ -11,6 +13,7 @@ import {
assertCouchDbReachable,
createCouchDbDatabase,
deleteCouchDbDatabase,
fetchCouchDbDocument,
loadCouchDbConfig,
makeUniqueDatabaseName,
waitForCouchDbDocs,
@@ -324,6 +327,10 @@ async function startConfiguredSession(
dbName: context.dbName,
};
const hiddenFileSettings = {
encrypt: true,
passphrase: "internal-metadata-e2e-secret",
usePathObfuscation: true,
encryptInternalMetadata: true,
syncInternalFiles: true,
syncInternalFilesBeforeReplication: true,
watchInternalFileChanges: false,
@@ -360,6 +367,23 @@ async function uploadHiddenFile(
const ids = new Set(docs.map((doc) => doc._id));
return ids.has(entry.id) && entry.children.every((childId) => ids.has(childId));
});
const remoteEntry = await fetchCouchDbDocument(context.couchDb, context.dbName, entry.id);
if (
!remoteEntry.path?.startsWith("/\\:") ||
remoteEntry.children?.length !== 0 ||
remoteEntry.ctime !== 0 ||
remoteEntry.mtime !== 0 ||
remoteEntry.size !== 0
) {
throw new Error(`Hidden File Sync Metadata was not encrypted for ${entry.id}.`);
}
const versionInfo = await fetchCouchDbDocument(context.couchDb, context.dbName, VERSIONING_DOCID);
if (
versionInfo.version !== REMOTE_FEATURE_GENERATION ||
!(versionInfo.used_features as unknown[] | undefined)?.includes(ENCRYPTED_INTERNAL_METADATA_FEATURE)
) {
throw new Error("The remote feature list does not declare encrypted internal Metadata.");
}
return entry;
}
@@ -383,6 +407,29 @@ async function runCreateRoundTrip(
await writeVaultFile(vaultA.path, snippetPath, snippetContent);
let session = await startConfiguredSession(context, vaultA);
const entry = await uploadHiddenFile(context, session, snippetPath);
await evalObsidianJson(
context.cliBinary,
[
"(async()=>{",
"const rebuilder=app.plugins.plugins['obsidian-livesync'].core.rebuilder;",
"const inform=rebuilder.informOptionalFeatures;",
"rebuilder.informOptionalFeatures=async()=>{};",
"try{await rebuilder.$rebuildRemote();}finally{rebuilder.informOptionalFeatures=inform;}",
"return JSON.stringify(true);",
"})()",
].join(""),
session.cliEnv
);
const rebuiltVersion = await fetchCouchDbDocument(context.couchDb, context.dbName, VERSIONING_DOCID);
const rebuiltEntry = await fetchCouchDbDocument(context.couchDb, context.dbName, entry.id);
if (
rebuiltVersion.version !== REMOTE_FEATURE_GENERATION ||
!(rebuiltVersion.used_features as unknown[] | undefined)?.includes(ENCRYPTED_INTERNAL_METADATA_FEATURE) ||
!rebuiltEntry.path?.startsWith("/\\:")
) {
throw new Error("Remote Rebuild did not declare and encrypt internal Metadata for its accepted writer.");
}
console.log("Remote Rebuild declared the feature and preserved encrypted Hidden File Sync Metadata.");
await session.app.stop();
session = await startConfiguredSession(context, vaultB);
@@ -571,11 +618,14 @@ async function runInitialisationNoticeGrouping(context: RunnerContext, vault: Te
await withObsidianPage(port, async (page) => {
const deadline = Date.now() + timeoutMs;
while ((await page.locator(".notice:visible").count()) > 0 && Date.now() < deadline) {
await page.locator(".notice:visible").first().click({
force: true,
position: { x: 2, y: 2 },
timeout: timeoutMs,
});
await page
.locator(".notice:visible")
.first()
.click({
force: true,
position: { x: 2, y: 2 },
timeout: timeoutMs,
});
}
assertEqual(
await page.locator(".notice:visible").count(),
@@ -707,17 +757,15 @@ async function runInitialisationNoticeGrouping(context: RunnerContext, vault: Te
const result = await withObsidianPage(port, async (page) => {
await page.evaluate((stateKey) => {
const state = (globalThis as unknown as Record<
string,
{ releasePreparation?: () => void } | undefined
>)[stateKey];
const state = (
globalThis as unknown as Record<string, { releasePreparation?: () => void } | undefined>
)[stateKey];
state?.releasePreparation?.();
}, hiddenFileInitialisationStateKey);
await page.waitForFunction(
(stateKey) =>
(globalThis as unknown as Record<string, { reachedInitialisation?: boolean } | undefined>)[
stateKey
]?.reachedInitialisation === true,
(globalThis as unknown as Record<string, { reachedInitialisation?: boolean } | undefined>)[stateKey]
?.reachedInitialisation === true,
hiddenFileInitialisationStateKey,
{ timeout: timeoutMs }
);
@@ -0,0 +1,285 @@
import { mkdir, readFile, writeFile } from "node:fs/promises";
import { dirname, join } from "node:path";
import { VERSIONING_DOCID, type LoadedEntry } from "@vrtmrz/livesync-commonlib/compat/common/types";
import { readContent } from "@vrtmrz/livesync-commonlib/compat/common/utils";
import { ENCRYPTED_INTERNAL_METADATA_FEATURE } from "@vrtmrz/livesync-commonlib/replication";
import { evalObsidianJson } from "../runner/cli.ts";
import {
assertCouchDbReachable,
createCouchDbDatabase,
deleteCouchDbDatabase,
fetchCouchDbDocument,
loadCouchDbConfig,
makeUniqueDatabaseName,
} from "../runner/couchdb.ts";
import { discoverObsidianCli, requireObsidianBinary } from "../runner/environment.ts";
import {
assertEqual,
configureCouchDb,
createE2eCouchDbPluginData,
createE2eObsidianDeviceLocalState,
prepareRemote,
pushLocalChanges,
waitForLiveSyncCoreReady,
} from "../runner/liveSyncWorkflow.ts";
import { startObsidianLiveSyncSession, type ObsidianLiveSyncSession } from "../runner/session.ts";
import { openLiveSyncSettings, waitForVisibleObsidianDialogue, withObsidianPage } from "../runner/ui.ts";
import { createTemporaryVault, type TemporaryVault } from "../runner/vault.ts";
process.env.E2E_OBSIDIAN_CLI_TIMEOUT_MS ??= "60000";
const hiddenPaths = [".metadata-migration/retained.json", ".metadata-migration/rewritten.json"];
const customPaths = [".obsidian/snippets/retained-metadata.css", ".obsidian/snippets/rewritten-metadata.css"];
const paths = [...hiddenPaths, ...customPaths];
const initialContent = "/* Metadata migration fixture */\n";
const updatedContent = "/* Updated after enabling internal Metadata encryption */\n";
const optionSettings = {
encrypt: true,
passphrase: "internal-metadata-migration-secret",
usePathObfuscation: true,
encryptInternalMetadata: false,
syncInternalFiles: true,
syncInternalFilesBeforeReplication: false,
watchInternalFileChanges: false,
syncInternalFilesTargetPatterns: "^\\.metadata-migration(?:/|$)",
usePluginSync: true,
usePluginSyncV2: true,
autoSweepPlugins: false,
autoSweepPluginsPeriodic: false,
autoAcceptCompatibleTweak: false,
};
type Entry = { id: string; path: string };
async function main(): Promise<void> {
const binary = requireObsidianBinary();
const cliBinary = discoverObsidianCli().binary;
if (!cliBinary) throw new Error("The Obsidian CLI is unavailable.");
const couchDb = await loadCouchDbConfig();
const dbName = makeUniqueDatabaseName(couchDb.dbPrefix, "internal-metadata-migration");
const connection = { ...couchDb, dbName };
const source = await createTemporaryVault();
const target = await createTemporaryVault();
let session: ObsidianLiveSyncSession | undefined;
const evaluate = async <T>(body: string): Promise<T> => {
if (!session) throw new Error("No active Obsidian session.");
return await evalObsidianJson<T>(
cliBinary,
`(async()=>{const core=app.plugins.plugins['obsidian-livesync'].core;${body}})()`,
session.cliEnv
);
};
const start = async (vault: TemporaryVault, device: string) => {
const settings = { ...optionSettings, deviceAndVaultName: device };
session = await startObsidianLiveSyncSession({
binary,
cliBinary,
vault,
pluginData: createE2eCouchDbPluginData(connection, settings),
localStorageEntries: createE2eObsidianDeviceLocalState(vault.name),
});
await waitForLiveSyncCoreReady(cliBinary, session.cliEnv);
await configureCouchDb(cliBinary, session.cliEnv, connection, settings);
await evaluate(`core.services.setting.setDeviceAndVaultName(${JSON.stringify(device)});
await core.services.setting.saveSettingData(); return JSON.stringify(true);`);
await prepareRemote(cliBinary, session.cliEnv);
};
const store = async (customisations: string[]) => {
return await evaluate<Entry[]>(`
await core.getAddOn('HiddenFileSync').scanAllStorageChanges(true);
const config=core.getAddOn('ConfigSync');
for(const path of ${JSON.stringify(customisations)}){
await config.storeCustomizationFiles(path,core.services.setting.getDeviceAndVaultName());
}
const rows=(await core.localDatabase.allDocsRaw({include_docs:true})).rows;
const entries=${JSON.stringify(paths)}.map(path=>rows.map(row=>row.doc).find(doc=>
doc?.path==='i:'+path || doc?.path?.startsWith('ix:migration-source/') && doc.path.endsWith('%'+path.split('/').pop())));
if(entries.some(entry=>!entry)) throw new Error('Missing internal Metadata fixtures: '+JSON.stringify({entries,paths:rows.map(row=>row.doc?.path).filter(Boolean)}));
return JSON.stringify(entries.map(doc=>({id:doc._id,path:doc.path})));`);
};
const preferCurrentSettings = async () => {
await evaluate(`await core.services.replicator.getActiveReplicator()
.setPreferredRemoteTweakSettings(core.services.setting.currentSettings()); return JSON.stringify(true);`);
};
const applyAndCheckFiles = async () => {
await evaluate(`
await core.getAddOn('HiddenFileSync').scanAllDatabaseChanges(true);
const config=core.getAddOn('ConfigSync');
const rows=(await core.localDatabase.allDocsRaw({include_docs:true})).rows;
for(const path of ${JSON.stringify(customPaths)}){
const entry=rows.map(row=>row.doc).find(doc=>doc?.path?.startsWith('ix:migration-source/') && doc.path.endsWith('%'+path.split('/').pop()));
if(!entry) throw new Error('Missing Customisation Sync Metadata');
const display=config.createPluginDataFromV2(entry.path);
await display.setFile(await config.createPluginDataExFileV2(entry.path));
if(!(await config.applyDataV2(display))) throw new Error('Could not apply Customisation Sync data');
}
return JSON.stringify(true);`);
for (const path of paths) {
assertEqual(
await readFile(join(target.path, path), "utf8"),
path.includes("rewritten") ? updatedContent : initialContent,
`Unexpected restored content: ${path}`
);
}
};
const assertDeclaration = async () => {
const version = await fetchCouchDbDocument(couchDb, dbName, VERSIONING_DOCID);
assertEqual(version.version, 13, "The feature generation was not retained.");
assertEqual(
(version.used_features as string[]).includes(ENCRYPTED_INTERNAL_METADATA_FEATURE),
true,
"The encrypted internal Metadata declaration was not retained."
);
};
try {
await assertCouchDbReachable(couchDb);
await createCouchDbDatabase(couchDb, dbName);
for (const path of paths) {
await mkdir(dirname(join(source.path, path)), { recursive: true });
await writeFile(join(source.path, path), initialContent);
}
await start(source, "migration-source");
const entries = await store(customPaths);
await pushLocalChanges(cliBinary, session!.cliEnv);
const originals = await Promise.all(entries.map((entry) => fetchCouchDbDocument(couchDb, dbName, entry.id)));
for (let index = 0; index < entries.length; index++) {
assertEqual(originals[index].path, entries[index].path, "OFF unexpectedly encrypted internal Metadata.");
}
assertEqual(
(await fetchCouchDbDocument(couchDb, dbName, VERSIONING_DOCID)).version,
12,
"The original database was not generation 12."
);
await withObsidianPage(session!.remoteDebuggingPort, async (page) => {
const navigator = await openLiveSyncSettings(page);
const remotePage = await navigator.openPage("Remote Configuration");
await remotePage
.locator(".setting-item")
.filter({
has: navigator.page.getByText("Configure E2EE", { exact: true }),
})
.getByRole("button", { name: "Configure", exact: true })
.click();
const dialog = await waitForVisibleObsidianDialogue(navigator.page, "End-to-End Encryption");
await dialog.getByLabel("Encrypt internal file Properties", { exact: true }).check();
await dialog.getByRole("button", { name: "Proceed", exact: true }).click();
const warning = await waitForVisibleObsidianDialogue(navigator.page, "Encrypt internal file Properties");
await warning
.getByRole("button", {
name: "Enable without rebuilding — update every other device first",
exact: true,
})
.click();
});
assertEqual(
await evaluate(`app.setting.close(); return JSON.stringify(core.settings.encryptInternalMetadata);`),
true,
"The setting dialogue did not enable encryption."
);
for (let index = 0; index < entries.length; index++) {
assertEqual(
(await fetchCouchDbDocument(couchDb, dbName, entries[index].id))._rev,
originals[index]._rev,
"Enabling without rebuilding rewrote an existing document."
);
}
await preferCurrentSettings();
for (const path of [hiddenPaths[1], customPaths[1]]) await writeFile(join(source.path, path), updatedContent);
const rewritten = await store([customPaths[1]]);
assertEqual(
JSON.stringify(rewritten),
JSON.stringify(entries),
"Enabling encryption changed document IDs or paths."
);
await pushLocalChanges(cliBinary, session!.cliEnv);
for (let index = 0; index < entries.length; index++) {
const raw = await fetchCouchDbDocument(couchDb, dbName, entries[index].id);
if (index % 2 === 0) {
assertEqual(raw._rev, originals[index]._rev, "An untouched document was rewritten.");
assertEqual(raw.path, entries[index].path, "An untouched document lost its plaintext Metadata.");
} else {
assertEqual(raw.path?.startsWith("/\\:"), true, "Updated Metadata was not encrypted.");
assertEqual(
JSON.stringify([raw.ctime, raw.mtime, raw.size, raw.children]),
"[0,0,0,[]]",
"Updated Metadata exposed file properties."
);
}
}
await assertDeclaration();
console.log(
"The settings UI enabled encryption without Rebuild; unchanged and encrypted Metadata coexist with stable IDs."
);
await session!.app.stop();
session = undefined;
await start(target, "migration-target");
const rejected = evaluate<boolean>(`return JSON.stringify(await core.services.replication.replicate(true));`);
await withObsidianPage(session!.remoteDebuggingPort, async (page) => {
const dialog = await waitForVisibleObsidianDialogue(page, "Configuration Mismatch Detected");
await dialog
.getByText("Encrypt internal file Properties", { exact: false })
.first()
.waitFor({ state: "visible" });
await dialog.getByRole("button", { name: "Dismiss", exact: true }).click();
});
assertEqual(await rejected, false, "Mismatched settings admitted replication.");
assertEqual(
await evaluate(`const rows=(await core.localDatabase.allDocsRaw({include_docs:true})).rows;
return JSON.stringify(rows.some(row=>${JSON.stringify(entries.map((entry) => entry.id))}.includes(row.id)));`),
false,
"The mismatched device received internal Metadata."
);
await evaluate(`await core.services.setting.applyPartial({encryptInternalMetadata:true},true);
return JSON.stringify(true);`);
await pushLocalChanges(cliBinary, session!.cliEnv);
await applyAndCheckFiles();
console.log("A second device rejected the mismatch, then restored both formats after setting alignment.");
await evaluate(`await core.services.setting.applyPartial({encryptInternalMetadata:false},true);
return JSON.stringify(true);`);
await preferCurrentSettings();
await evaluate(`await core.rebuilder.$fetchLocalDBFast(true);
await core.services.setting.applyPartial(${JSON.stringify(optionSettings)},true);
return JSON.stringify(true);`);
const fetchedEntries = await evaluate<LoadedEntry[]>(`
const entries=[];
for(const path of ${JSON.stringify(entries.map((entry) => entry.path))}){
const entry=await core.localDatabase.getDBEntry(path,undefined,false,true);
if(!entry || entry.deleted || entry._deleted) throw new Error('Could not read fetched Metadata: '+path);
const file=path.startsWith('ix:')
? await core.getAddOn('ConfigSync').createPluginDataExFileV2(path,entry) : entry;
if(!file) throw new Error('Could not decode fetched Customisation Sync content: '+path);
entries.push(file);
}
return JSON.stringify(entries);`);
for (let index = 0; index < fetchedEntries.length; index++) {
const expected = index % 2 === 0 ? initialContent : updatedContent;
const content = readContent(fetchedEntries[index]);
const text = typeof content === "string" ? content : new TextDecoder().decode(content);
assertEqual(
text,
expected,
`OFF did not read internal file content after Fast Fetch: ${entries[index].path}`
);
}
await applyAndCheckFiles();
await assertDeclaration();
console.log(
"Fast Fetch and database content reads accept both formats with the option OFF; the remote declaration remains."
);
} finally {
await session?.app.stop();
await source.dispose();
await target.dispose();
await deleteCouchDbDatabase(couchDb, dbName);
}
}
main().catch((error: unknown) => {
console.error(error instanceof Error ? error.stack : error);
process.exitCode = 1;
});
@@ -0,0 +1,218 @@
import { mkdir, readFile, writeFile } from "node:fs/promises";
import { dirname, join } from "node:path";
import { VERSIONING_DOCID } from "@vrtmrz/livesync-commonlib/compat/common/types";
import { evalObsidianJson } from "../runner/cli.ts";
import {
assertCouchDbReachable,
createCouchDbDatabase,
deleteCouchDbDatabase,
fetchCouchDbDocument,
loadCouchDbConfig,
makeUniqueDatabaseName,
putCouchDbDocument,
} from "../runner/couchdb.ts";
import { discoverObsidianCli, requireObsidianBinary } from "../runner/environment.ts";
import {
assertE2eCompatibilityMarker,
configureCouchDb,
createE2eCouchDbPluginData,
createE2eObsidianDeviceLocalState,
prepareRemote,
pushLocalChanges,
waitForLiveSyncCoreReady,
waitForLocalDatabaseEntry,
} from "../runner/liveSyncWorkflow.ts";
import { startObsidianLiveSyncSession, type ObsidianLiveSyncSession } from "../runner/session.ts";
import { createTemporaryVault } from "../runner/vault.ts";
const acceptedPath = "E2E/remote-feature/accepted.md";
const acceptedContent = "Accepted before the remote feature changed.\n";
const unknownFeature = "future-format-v7";
type FeatureState = {
version: number | null;
features: string[];
hasActiveReplicator: boolean;
};
async function readFeatureState(cliBinary: string, env: NodeJS.ProcessEnv): Promise<FeatureState> {
return await evalObsidianJson<FeatureState>(
cliBinary,
[
"(async()=>{",
"const core=app.plugins.plugins['obsidian-livesync'].core;",
`const id=${JSON.stringify(VERSIONING_DOCID)};`,
"const info=await core.localDatabase.getRaw(id).catch(()=>null);",
"return JSON.stringify({",
"version:typeof info?.version==='number'?info.version:null,",
"features:Array.isArray(info?.used_features)?info.used_features:[],",
"hasActiveReplicator:!!core.services.replicator.getActiveReplicator(),",
"});",
"})()",
].join(""),
env
);
}
async function waitForState(
cliBinary: string,
env: NodeJS.ProcessEnv,
predicate: (state: FeatureState) => boolean,
description: string
): Promise<FeatureState> {
const deadline = Date.now() + 20_000;
let state = await readFeatureState(cliBinary, env);
while (!predicate(state) && Date.now() < deadline) {
await new Promise((resolve) => setTimeout(resolve, 250));
state = await readFeatureState(cliBinary, env);
}
if (!predicate(state)) throw new Error(`Timed out waiting for ${description}: ${JSON.stringify(state)}`);
return state;
}
async function main(): Promise<void> {
const binary = requireObsidianBinary();
const cli = discoverObsidianCli();
if (!cli.binary) throw new Error(`Could not find obsidian-cli. Checked paths: ${cli.checked.join(", ")}`);
const couchDb = await loadCouchDbConfig();
const dbName = makeUniqueDatabaseName(couchDb.dbPrefix, "remote-feature-change");
const vault = await createTemporaryVault();
let session: ObsidianLiveSyncSession | undefined;
try {
await assertCouchDbReachable(couchDb);
await createCouchDbDatabase(couchDb, dbName);
const couchDbSettings = {
uri: couchDb.uri,
username: couchDb.username,
password: couchDb.password,
dbName,
};
const settings = {
encrypt: false,
usePathObfuscation: false,
encryptInternalMetadata: false,
liveSync: false,
};
session = await startObsidianLiveSyncSession({
binary,
cliBinary: cli.binary,
vault,
startupGraceMs: Number(process.env.E2E_OBSIDIAN_STARTUP_GRACE_MS ?? 1000),
pluginData: createE2eCouchDbPluginData(couchDbSettings, settings),
localStorageEntries: createE2eObsidianDeviceLocalState(vault.name),
});
await waitForLiveSyncCoreReady(cli.binary, session.cliEnv);
await assertE2eCompatibilityMarker(cli.binary, session.cliEnv);
await configureCouchDb(cli.binary, session.cliEnv, couchDbSettings, settings);
await prepareRemote(cli.binary, session.cliEnv);
const fullPath = join(vault.path, acceptedPath);
await mkdir(dirname(fullPath), { recursive: true });
await writeFile(fullPath, acceptedContent, "utf-8");
await waitForLocalDatabaseEntry(cli.binary, session.cliEnv, acceptedPath);
await pushLocalChanges(cli.binary, session.cliEnv);
const initialVersion = await fetchCouchDbDocument(couchDb, dbName, VERSIONING_DOCID);
if (initialVersion.version !== 12 || "used_features" in initialVersion) {
throw new Error(
`An inactive feature unexpectedly changed the remote contract: ${JSON.stringify(initialVersion)}`
);
}
const start = await evalObsidianJson<{ status: string }>(
cli.binary,
[
"(async()=>{",
"const core=app.plugins.plugins['obsidian-livesync'].core;",
"await core.services.setting.applyExternalSettings({liveSync:true},true);",
"await core.services.control.applySettings();",
"const result=await core.services.replication.startContinuous({trigger:'daemon',interaction:{kind:'forbidden'}});",
"return JSON.stringify(result);",
"})()",
].join(""),
session.cliEnv
);
if (start.status !== "completed")
throw new Error(`Continuous replication did not start: ${JSON.stringify(start)}`);
await waitForState(cli.binary, session.cliEnv, (state) => state.hasActiveReplicator, "an active Replicator");
await putCouchDbDocument(couchDb, dbName, {
...initialVersion,
version: 13,
used_features: [unknownFeature],
});
const observed = await waitForState(
cli.binary,
session.cliEnv,
(state) => state.version === 13 && state.features.includes(unknownFeature) && !state.hasActiveReplicator,
"the live feature change and Replicator retirement"
);
const replicated = await evalObsidianJson<boolean>(
cli.binary,
"(async()=>JSON.stringify(!!(await app.plugins.plugins['obsidian-livesync'].core.services.replication.replicate(true))))()",
session.cliEnv
);
if (replicated) throw new Error("An unknown remote feature was admitted for another replication.");
const acceptedAfterStop = await readFile(fullPath, "utf-8");
if (acceptedAfterStop !== acceptedContent)
throw new Error("Previously accepted Vault content changed on stop.");
await session.app.stop();
session = undefined;
session = await startObsidianLiveSyncSession({ binary, cliBinary: cli.binary, vault });
await waitForLiveSyncCoreReady(cli.binary, session.cliEnv);
const afterRestart = await waitForState(
cli.binary,
session.cliEnv,
(state) => state.version === 13 && state.features.includes(unknownFeature),
"the remote feature requirement after restart"
);
const replicatedAfterRestart = await evalObsidianJson<boolean>(
cli.binary,
"(async()=>JSON.stringify(!!(await app.plugins.plugins['obsidian-livesync'].core.services.replication.replicate(true))))()",
session.cliEnv
);
const continuousAfterRestart = await evalObsidianJson<{ requestStatus: string; connected: boolean }>(
cli.binary,
[
"(async()=>{",
"const core=app.plugins.plugins['obsidian-livesync'].core;",
"const replicator=core.services.replicator.getActiveReplicator();",
"const original=replicator.openContinuousReplication;",
"let completion;",
"replicator.openContinuousReplication=function(...args){completion=original.apply(this,args);return completion;};",
"try{",
"const result=await core.services.replication.startContinuous({trigger:'daemon',interaction:{kind:'forbidden'}});",
"if(!completion) throw new Error('The continuous provider did not attempt its remote check');",
"return JSON.stringify({requestStatus:result.status,connected:await completion});",
"}finally{replicator.openContinuousReplication=original;}",
"})()",
].join(""),
session.cliEnv
);
if (replicatedAfterRestart || continuousAfterRestart.connected !== false)
throw new Error(
`Replication resumed after restart despite an unknown remote feature: ${JSON.stringify({ afterRestart, replicatedAfterRestart, continuousAfterRestart })}`
);
if ((await readFile(fullPath, "utf-8")) !== acceptedContent)
throw new Error("Previously accepted Vault content changed after restart.");
console.log(
`Active feature change retired the Replicator; the remote declaration refused synchronisation after restart: ${JSON.stringify({ observed, afterRestart })}`
);
} finally {
await session?.app.stop();
await vault.dispose();
if (process.env.E2E_OBSIDIAN_KEEP_COUCHDB !== "true") {
await deleteCouchDbDatabase(couchDb, dbName).catch((error: unknown) => {
console.warn(error instanceof Error ? error.message : error);
});
}
}
}
main().catch((error: unknown) => {
console.error(error instanceof Error ? error.stack : error);
process.exit(1);
});