Merge internal Metadata integration for stacked Setup URI validation

This commit is contained in:
vorotamoroz
2026-09-29 06:38:27 +00:00
26 changed files with 1167 additions and 44 deletions
@@ -57,6 +57,8 @@ Keep configured-state inference separate from new-Vault initialisation. If an ex
- On resume, clear `versionUpFlash` and persist that fail-closed change before recording the current `VER` as acknowledged. If saving fails, restore the gate. Reapply settings only after the marker has advanced so that the previously configured synchronisation behaviour can resume without reconstruction.
- Preserve the original legacy review message as a structured reason when no more specific database or settings-schema reason is available. Escape it before including it in Markdown UI.
- Continue to reject a remote version document which is newer than the running implementation. That receiver-side check is independent of the local upgrade review.
- From remote generation 13, assess the `used_features` list in that document as a separate compatibility dimension. A client must recognise every listed feature before it interprets the database or runs maintenance which depends on Metadata. Declare a feature before writing its representation, and retain the declaration while older data may depend on it. An unknown identifier is reported as text without requiring a descriptive label in that client.
- Do not advance the device-local `VER` acknowledgement merely because a remote feature is introduced. The remote generation and its feature list govern remote admission; `VER` remains the local compatibility review gate. Connecting to a generation-12 database does not promote it solely because the client understands generation 13.
### Onboarding activation and initialisation
@@ -90,7 +92,7 @@ Keep configured-state inference separate from new-Vault initialisation. If an ex
- Accepted new-device and existing-device setup cannot enable ordinary processing before the selected Rebuild or Fetch has been reserved.
- An older installation cannot dismiss evidence that a newer implementation or settings schema has already been used on the device.
- The Obsidian-specific dialogue depends only on a host-neutral compatibility result and the injected confirmation capability. Commonlib remains responsible for settings migration, device-local storage, and the replication gate.
- A future incompatible database change must increment `VER`, provide an actionable review message, verify the remote version negotiation, and test both the pending and acknowledged states. A major SemVer increase without those changes has no database-compatibility effect.
- A future local database change which requires compatibility review must increment `VER`, provide an actionable review message, and test both the pending and acknowledged states. A new remote representation must declare its feature before use and verify remote admission independently. A major SemVer increase alone has no database-compatibility effect.
## Verification
@@ -0,0 +1,163 @@
---
date: 2026-09-27
commonlib-version: "0.1.30"
self-hosted-livesync-version: "1.0.32"
status: unreleased
---
# Internal Metadata encryption and remote feature changes
This document defines the LiveSync integration of Commonlib's remote feature
contract and encrypted Metadata for Hidden File Sync and Customisation Sync.
It describes unreleased behaviour being implemented in this branch.
Commonlib's companion `docs/remote-feature-compatibility.md` is the
source of truth for the wire document, identifiers, validation, and shared
assessment. This document owns the application behaviour, settings, Doctor
recommendation, and verification of the Obsidian and CLI integrations.
## Scope and settings
Add `encryptInternalMetadata` to the shared encryption settings. A genuinely new
Vault or CLI configuration defaults to true. Existing stored settings and old
Setup URI or QR imports complete an absent value as false. Ordinary partial
setting updates retain the current value.
The preference applies to CouchDB with E2EE V2 and Property Encryption enabled.
Show the preference as unavailable and explain its prerequisites when they are
absent. Keep Journal and P2P's existing
transport protection and avoid unrelated setting mismatches for those remotes.
Use the existing HKDF Metadata representation to protect path, creation and
modification times, size, and Chunk references for obfuscated internal entries.
Keep the `i:`, `ix:`, and supported legacy `ps:` document IDs, path conversion,
and content Chunk representation. Read encrypted Metadata independently of the
write preference, including after that preference is disabled.
The protection leaves document IDs, namespaces, revisions, deletion state,
document counts, and ciphertext lengths visible. It does not encrypt device or
Vault names stored in separate participant records.
## Enabling the preference
Changing the preference does not automatically reconstruct a database or gather
all devices' data. It affects subsequent Metadata writes. Unchanged documents
and old revisions can retain plaintext; mixed plaintext and encrypted Metadata
are a supported transition state.
Strongly recommend the existing manual remote Rebuild workflow when the person
wants existing Metadata protected as well. The person prepares the authoritative
data for that workflow. Describe this distinction in the setting, Doctor reason,
and operational documentation. Do not advertise complete historical protection
merely because the preference is enabled.
Copy the preference with the other encryption settings when preparing a remote
profile. Recreate a connection when its effective encryption settings change.
Use the existing Tweak assessment and manual mismatch resolution; do not change
the remote's shared policy silently when importing or loading settings.
## Doctor
Use Commonlib's existing conditional recommendation rules. Recommend true when
the selected CouchDB settings have E2EE V2 and Property Encryption enabled and
the new preference is false. Do not require Hidden File Sync or Customisation
Sync to be active before offering the recommendation.
Retain the existing E2EE V2 recommendation for a legacy algorithm. After that
change, ensure the newly applicable Metadata recommendation is not hidden by a
premature `doctorProcessedVersion` update. Advance the Doctor rule revision so
an older completed consultation does not suppress this new recommendation.
Apply the preference only when the person accepts the recommendation. Include
the existing-data limitation, the manual Rebuild recommendation, and the need
for compatible clients in the explanation. Do not set `requireRebuild` or
`requireRebuildLocal` for this rule: the current host wrapper can schedule those
operations and restart. `recommendRebuild` currently exists only as an unused
rule field, so setting it alone does not display an explanation.
## Admission and received version documents
The remote version document is the source of feature requirements. Commonlib
checks it before replication, Fast Fetch, and direct access. The milestone keeps
the existing Tweak comparison and Rebuild lock. An accepted writer declares the
feature before using it, including the writer admitted to a locked rebuilt
remote; an unaccepted device remains blocked by that lock.
Retain the existing received-version path through `parseSynchroniseResult`,
`enqueueAll`, and `processIfNonDocumentChange`. Replace its numeric comparison
with the shared assessment so unknown names at the same generation are also
reported. Known features, reordered lists, and ordinary revision updates do not
retire the connection. Unsupported or malformed control documents request
retirement through the existing Replicator owner and display the reason.
The callback must not await retirement of the operation which delivered it.
This is an admission check and a best-effort stop for exceptional changes during
an active connection. It does not fence every queued file application, roll back
accepted writes, or guarantee an atomic change across live devices. Feature
changes are an infrequent administrative operation: update all devices first,
then enable the preference and use the recommended manual Rebuild. Rebuild
locks the remote using the existing workflow; changing this preference alone
does not lock it. The action to proceed without rebuilding explicitly reminds
the person to update every other device, including currently connected devices.
## Persistence and recovery boundaries
Do not retain a second feature list, highest generation, or rejection flag in
KV storage. Do not add compatibility checks to pending-work snapshot recovery
or make that recovery a new prerequisite for application readiness. Preserve
the existing queue and startup behaviour. A later attempt checks the current
remote declaration, including after restart. Declared features remain on the
remote when the write preference is disabled because older data can still use
them; manually shortening that declaration is not a supported migration.
After updating clients, use normal reconnection and the existing Hatch
inspection or Fetch workflow if reconciliation is needed. This feature does not
repair unrelated KV inconsistencies or the existing readiness queue behaviour.
Garbage Collection V3 is a beta manual operation which begins with an ordinary
bidirectional synchronisation. That admission checks the remote feature
contract; no additional per-step GC checks are introduced. The separate
cleaned-remote recovery path checks the local version document before its
first Chunk-reference count because it does not start with that synchronisation.
Use the same Commonlib assessment at the CLI, Fast Fetch, and direct-access
boundaries. The Obsidian result processor is one consumer, not the only place
which determines compatibility. Keep unrelated Vaults and databases operational.
Fast Fetch checks the remote declaration before opening or resetting the local
database, both for a fresh Fetch and for checkpoint resumption.
## Verification and documentation
Keep focused tests for settings defaults and imports, the Doctor condition
matrix, acceptance and dismissal, connection replacement, and absence of an
automatic Rebuild, Fetch, or restart for this rule.
Keep unit tests for known and unknown feature notifications, generic identifier
presentation, retirement without a circular wait, and the unchanged snapshot
behaviour after KV failure or obsolete snapshot fields. The previous batch
fences, physical-database tracking, and persistent rejection tests are outside
this design; they must not imply an atomic live migration guarantee.
Use real Obsidian Hidden File Sync and Customisation Sync scenarios to inspect
raw CouchDB Metadata and restore content in another Vault. Check the admitted
writer on a locked remote, unknown-feature rejection before and during
replication, and remote-based rejection after restart. Retain the encrypted
CLI-to-Obsidian interoperability scenario. A future client upgrade that adds
support for an unknown feature is a separate validation boundary.
Also exercise enabling the preference through the settings UI without Rebuild:
retain unchanged plaintext Metadata, encrypt rewritten entries with stable IDs,
reject a second device's mismatched preference, and restore both representations
after alignment. With the preference subsequently OFF, verify that Fast Fetch
still decodes encrypted Metadata and preserves the remote feature declaration.
Keep the primary-language settings and troubleshooting guides, the
database-compatibility ADR, and Unreleased notes aligned with this behaviour.
Keep the detailed shared protocol in Commonlib and link to it after publication;
do not maintain another copy of its wire schema here. Translations are a separate
change. Update tested-version evidence when the implementation and its
validation have been accepted.
Related application contracts: [Replicator architecture](replicator_architecture.md),
[Tweak compatibility](tweak_compatibility.md), and
[database compatibility](../adr/2026_07_release_notes_and_database_compatibility.md).
+8
View File
@@ -245,6 +245,14 @@ Setting key: usePathObfuscation
In default, the path of the file is not obfuscated to improve the performance. If you enable this, the path of the file will be obfuscated. This is useful when you want to hide the path of the file.
#### Encrypt internal file Properties
Setting key: encryptInternalMetadata
For CouchDB, this encrypts paths, times, sizes, and Chunk references in the Metadata used by Hidden File Sync and Customisation Sync. It requires E2EE V2 and **Property Encryption**. New Vaults enable the preference by default, but it has no effect until those prerequisites are enabled. Existing Vaults and older Setup URIs and QR codes keep it disabled unless you enable it.
Enabling the preference protects future Metadata writes. Existing Metadata and earlier revisions can remain readable in the remote database. If you want to protect existing Metadata too, prepare the authoritative data, update every device to a compatible version, and manually Rebuild the remote database. LiveSync does not gather data or start a Rebuild when you change this preference. The action to enable it without rebuilding explicitly reminds you to update every other synchronising device first, including devices currently running LiveSync. Plaintext and encrypted Metadata can coexist during the transition. Document IDs, revision information, document counts, and ciphertext lengths remain visible.
#### Encryption Algorithm
Setting key: E2EEAlgorithm
+6
View File
@@ -96,6 +96,8 @@ Current releases automatically align compatible settings which control how new c
A missing legacy file-name case setting means case-insensitive handling. It matches an explicit disabled setting and does not require a rebuild for that difference. An explicitly enabled setting can use different document IDs and still requires a compatibility decision against either value. Other configuration differences shown in the dialogue must still be resolved.
If the mismatch names **Encrypt internal file Properties**, update every device before accepting that preference. It affects subsequent Metadata writes for Hidden File Sync and Customisation Sync; it does not automatically protect existing Metadata. A manual remote Rebuild is strongly recommended if you need to protect existing paths, times, sizes, and Chunk references.
The `Sync now` command keeps routine replication progress quiet so that it is convenient to assign to a keyboard shortcut; assign one in Obsidian if that suits your workflow. A quiet command may still open this dialogue when a mismatch or another decision requires your attention.
The available actions depend on when the mismatch is found:
@@ -109,6 +111,10 @@ The available actions depend on when the mismatch is found:
Historic defect notices and renamed controls are retained in the [0.25 release history](releases/0.25.md) and [legacy release history](releases/legacy.md), rather than in the current troubleshooting path.
## The remote database uses an unknown feature
When a notice identifies an unknown feature, update this device and every other client of the same CouchDB database, including the CLI. The notice includes the feature identifier even if this version has no descriptive name for it. New synchronisation is refused, and receiving an unsupported requirement stops active replication, because an older client may not interpret the Metadata and its Chunk references correctly. Already queued file changes are not rolled back. The cleaned-remote recovery path also checks compatibility before counting Chunk references. Do not remove the feature name from the remote version document to bypass the check. After updating, reconnect and review any pending file changes before running Garbage Collection.
## Setup and settings questions
### Share a configuration with another device
+2
View File
@@ -85,6 +85,8 @@
"test:e2e:obsidian:security-seed-reconnect": "tsx test/e2e-obsidian/scripts/security-seed-reconnect.ts",
"test:e2e:obsidian:hidden-file-snippet-sync": "tsx test/e2e-obsidian/scripts/hidden-file-snippet-sync.ts",
"test:e2e:obsidian:customisation-sync": "tsx test/e2e-obsidian/scripts/customisation-sync.ts",
"test:e2e:obsidian:remote-feature-change": "tsx test/e2e-obsidian/scripts/remote-feature-change.ts",
"test:e2e:obsidian:internal-metadata-migration": "tsx test/e2e-obsidian/scripts/internal-metadata-migration.ts",
"test:e2e:obsidian:setting-markdown-export": "tsx test/e2e-obsidian/scripts/setting-markdown-export.ts",
"test:e2e:obsidian:upgrade-from-stable": "tsx test/e2e-obsidian/scripts/upgrade-from-stable.ts",
"test:e2e:obsidian:local-suite": "tsx test/e2e-obsidian/scripts/local-suite.ts",
@@ -1,4 +1,5 @@
import type { RemoteDBSettings } from "@vrtmrz/livesync-commonlib/compat/common/types";
import { usesEncryptedInternalMetadata } from "@vrtmrz/livesync-commonlib/replication";
type EndpointProjection = readonly [kind: "url" | "invalid-url", value: string];
@@ -77,6 +78,7 @@ export function getCouchDBReplicatorConfigurationIdentity(settings: RemoteDBSett
settings.useRequestAPI,
settings.disableRequestURI,
projectRemoteSecurity(settings),
usesEncryptedInternalMetadata(settings),
settings.enableCompression,
]);
}
@@ -67,6 +67,22 @@ describe("active Replicator configuration identity", () => {
);
});
it("recreates the CouchDB connection when internal Metadata encryption becomes effective", () => {
const active = configuredSettings({ usePathObfuscation: true, encryptInternalMetadata: false });
const enabled = { ...active, encryptInternalMetadata: true };
expect(getCouchDBReplicatorConfigurationIdentity(enabled)).not.toBe(
getCouchDBReplicatorConfigurationIdentity(active)
);
const inactive = { ...active, usePathObfuscation: false };
expect(getCouchDBReplicatorConfigurationIdentity({ ...inactive, encryptInternalMetadata: true })).toBe(
getCouchDBReplicatorConfigurationIdentity(inactive)
);
expect(getObjectStorageReplicatorConfigurationIdentity(enabled)).toBe(
getObjectStorageReplicatorConfigurationIdentity(active)
);
});
it("projects only the active CouchDB authentication mode", () => {
const basic = configuredSettings({ useJWT: false, jwtKey: "inactive-a" });
expect(getCouchDBReplicatorConfigurationIdentity({ ...basic, jwtKey: "inactive-b" })).toBe(
@@ -569,6 +569,7 @@ export class ObsidianLiveSyncSettingTab extends PluginSettingTab {
}
}
// Internal Metadata encryption affects future Metadata writes and is not a rebuild requirement.
isNeedRebuildLocal() {
return this.isSomeDirty([
"useIndexedDBAdapter",
@@ -116,7 +116,16 @@ export function paneRemoteConfig(
.onClick(async () => {
const setupManager = this.core.getModule(SetupManager);
const originalSettings = getSettingsFromEditingSettings(this.editingSettings);
await setupManager.onlyE2EEConfiguration(UserMode.Update, originalSettings);
const applied = await setupManager.onlyE2EEConfiguration(UserMode.Update, originalSettings);
if (applied) {
this.editingSettings.encryptInternalMetadata =
this.core.settings.encryptInternalMetadata;
if (this.initialSettings) {
this.initialSettings.encryptInternalMetadata =
this.core.settings.encryptInternalMetadata;
}
this.requestUpdate();
}
updateE2EESummary();
})
.setButtonText("Configure")
@@ -243,6 +252,7 @@ export function paneRemoteConfig(
...DEFAULT_SETTINGS,
encrypt: this.editingSettings.encrypt,
usePathObfuscation: this.editingSettings.usePathObfuscation,
encryptInternalMetadata: this.editingSettings.encryptInternalMetadata,
passphrase: this.editingSettings.passphrase,
configPassphraseStore: this.editingSettings.configPassphraseStore,
});
@@ -2,6 +2,7 @@ import { afterEach, describe, expect, it, vi } from "vitest";
const runtime = vi.hoisted(() => ({
buttonClasses: [] as string[],
clickHandlers: [] as Array<() => Promise<void> | void>,
panels: [] as Array<{ destroy: ReturnType<typeof vi.fn> }>,
settingClasses: [] as string[],
}));
@@ -51,7 +52,8 @@ vi.mock("./LiveSyncSetting.ts", () => ({
setDestructive() {
return this;
},
onClick() {
onClick(callback: () => Promise<void> | void) {
runtime.clickHandlers.push(callback);
return this;
},
setButtonText() {
@@ -97,6 +99,7 @@ vi.mock("@vrtmrz/livesync-commonlib/compat/common/ConnectionString", () => ({
},
}));
vi.mock("@/modules/features/SetupWizard/dialogs/SetupRemote.svelte", () => ({ default: {} }));
vi.mock("@/modules/features/SetupWizard/dialogs/SetupRemoteE2EE.svelte", () => ({ default: {} }));
vi.mock("@/modules/features/SetupWizard/dialogs/SetupRemoteCouchDB.svelte", () => ({ default: {} }));
vi.mock("@/modules/features/SetupWizard/dialogs/SetupRemoteBucket.svelte", () => ({ default: {} }));
vi.mock("@/modules/features/SetupWizard/dialogs/SetupRemoteP2P.svelte", () => ({ default: {} }));
@@ -114,6 +117,7 @@ function createPanelElement(): HTMLElement {
afterEach(() => {
runtime.buttonClasses.length = 0;
runtime.clickHandlers.length = 0;
runtime.panels.length = 0;
runtime.settingClasses.length = 0;
vi.clearAllMocks();
@@ -148,4 +152,46 @@ describe("paneRemoteConfig", () => {
expect(runtime.panels[0].destroy).toHaveBeenCalledOnce();
});
it("applies an internal Metadata preference change without scheduling setup initialisation", async () => {
const originalSettings = {
encrypt: true,
passphrase: "passphrase",
E2EEAlgorithm: "v2",
usePathObfuscation: true,
encryptInternalMetadata: false,
remoteConfigurations: {},
};
const setupManager = {
onlyE2EEConfiguration: vi.fn(async () => {
host.core.settings.encryptInternalMetadata = true;
return true;
}),
};
const host = {
editingSettings: { ...originalSettings },
initialSettings: { ...originalSettings },
core: {
settings: { ...originalSettings },
getModule: vi.fn(() => setupManager),
},
lifetimeComponent: { register: vi.fn() },
requestUpdate: vi.fn(),
};
const addPanel = vi.fn((_parent: HTMLElement, heading: string) => ({
then(callback: (paneEl: HTMLElement) => void) {
if (heading === "E2EE Configuration") {
callback(createPanelElement());
}
},
}));
paneRemoteConfig.call(host as never, {} as HTMLElement, { addPanel } as never);
await runtime.clickHandlers[0]();
expect(setupManager.onlyE2EEConfiguration).toHaveBeenCalledOnce();
expect(host.editingSettings.encryptInternalMetadata).toBe(true);
expect(host.initialSettings.encryptInternalMetadata).toBe(true);
expect(host.requestUpdate).toHaveBeenCalledOnce();
});
});
+25
View File
@@ -336,6 +336,31 @@ export class SetupManager extends AbstractModule {
this._log("E2EE configuration cancelled.", LOG_LEVEL_NOTICE);
return false;
}
const onlyInternalMetadataPreferenceChanged =
currentSetting.encryptInternalMetadata !== e2eeConf.encryptInternalMetadata &&
currentSetting.encrypt === e2eeConf.encrypt &&
currentSetting.passphrase === e2eeConf.passphrase &&
currentSetting.E2EEAlgorithm === e2eeConf.E2EEAlgorithm &&
currentSetting.usePathObfuscation === e2eeConf.usePathObfuscation;
if (userMode === UserMode.Update && onlyInternalMetadataPreferenceChanged) {
if (e2eeConf.encryptInternalMetadata && currentSetting.remoteType === REMOTE_COUCHDB) {
const proceed = "Enable without rebuilding — update every other device first";
const choice = await this.core.confirm.askSelectStringDialogue(
"A manual remote Rebuild is strongly recommended to protect existing file properties. " +
"Before continuing without rebuilding, update every other synchronising device to a version " +
"which supports this option, including devices currently running LiveSync. " +
"Existing properties remain unchanged until they are rewritten or rebuilt.",
[proceed, "Cancel"],
{ title: "Encrypt internal file Properties", defaultAction: "Cancel" }
);
if (choice !== proceed) return false;
}
await this.services.setting.applyPartial(
{ encryptInternalMetadata: e2eeConf.encryptInternalMetadata },
true
);
return true;
}
const newSetting = {
...currentSetting,
...e2eeConf,
@@ -659,3 +659,23 @@ describe("SetupManager", () => {
expect(setting.currentSettings().P2P_ActiveRemoteConfigurationId).toBe("existing");
});
});
describe("internal Metadata configuration", () => {
it.each([true, false])(
"applies the preference only after accepting the no-Rebuild warning (%s)",
async (accept) => {
const { manager, setting, dialogManager, core } = createSetupManager();
const current = { ...setting.settings, encryptInternalMetadata: false, remoteType: REMOTE_COUCHDB };
dialogManager.openWithExplicitCancel.mockResolvedValue({ ...current, encryptInternalMetadata: true });
const ask = vi.fn(async (_message: string, choices: string[]) => (accept ? choices[0] : "Cancel"));
core.confirm = { askSelectStringDialogue: ask };
const apply = vi.spyOn(setting, "applyPartial").mockResolvedValue(undefined);
await expect(manager.onlyE2EEConfiguration(UserMode.Update, current)).resolves.toBe(accept);
expect(ask.mock.calls[0][1][0]).toContain("update every other device first");
expect(ask.mock.calls[0][0]).toContain("currently running LiveSync");
expect(apply).toHaveBeenCalledTimes(accept ? 1 : 0);
expect(core.rebuilder.scheduleRebuild).not.toHaveBeenCalled();
expect(core.rebuilder.scheduleFetch).not.toHaveBeenCalled();
}
);
});
@@ -26,7 +26,8 @@
passphrase: "",
E2EEAlgorithm: DEFAULT_SETTINGS.E2EEAlgorithm,
usePathObfuscation: true,
} as EncryptionSettings;
encryptInternalMetadata: true,
};
let encryptionSettings = $state<EncryptionSettings>({ ...default_encryption });
@@ -42,6 +43,11 @@
if (!encryptionSettings.encrypt) return true;
return encryptionSettings.passphrase.trim().length >= 1;
});
let canEncryptInternalMetadata = $derived(
encryptionSettings.encrypt &&
encryptionSettings.E2EEAlgorithm === E2EEAlgorithms.V2 &&
encryptionSettings.usePathObfuscation
);
function commit() {
setResult(pickEncryptionSettings(encryptionSettings));
@@ -87,6 +93,24 @@
</InfoNote>
{/if}
<InputRow label="Encrypt internal file Properties">
<input
type="checkbox"
bind:checked={encryptionSettings.encryptInternalMetadata}
disabled={!canEncryptInternalMetadata}
/>
</InputRow>
<InfoNote>
This option encrypts file properties used by Hidden File Sync and Customisation Sync.
<br />
It applies only to CouchDB and requires End-to-End Encryption, the V2 algorithm, and Property Encryption
(Obfuscate Properties). The remote type is selected later in this setup wizard.
<br />
It protects properties written after the option is enabled; existing properties are not rewritten. A manual remote
Rebuild is strongly recommended to protect existing properties. Update every other synchronising device to a compatible
version before enabling this option, including devices currently running LiveSync.
</InfoNote>
<ExtraItems title={translateMessage("Advanced")}>
<InputRow label={translateMessage("Encryption Algorithm")}>
<select bind:value={encryptionSettings.E2EEAlgorithm} disabled={!encryptionSettings.encrypt}>
@@ -1,6 +1,7 @@
import { assessRemoteFeatureDocument, describeRemoteFeatureRejection } from "@vrtmrz/livesync-commonlib/replication";
import {
SYNCINFO_ID,
VER,
VERSIONING_DOCID,
type AnyEntry,
type EntryDoc,
type EntryLeaf,
@@ -274,13 +275,14 @@ export class ReplicateResultProcessor {
this.log(`Processed chunk: ${shortenId(change._id)}`, LOG_LEVEL_DEBUG);
return true;
}
if (change.type == "versioninfo") {
if (change._id === VERSIONING_DOCID || change.type === "versioninfo") {
this.log(`Version info document received: ${change._id}`, LOG_LEVEL_VERBOSE);
if (change.version > VER) {
const assessment = assessRemoteFeatureDocument(change);
if (assessment.status !== "supported" && assessment.status !== "older-generation") {
// Fence and retire the active publication through its owner.
this.context.requestActiveReplicatorRetirement();
this.log(
`Remote database updated to incompatible version. update your Self-hosted LiveSync plugin.`,
`${describeRemoteFeatureRejection(assessment)} Update Self-hosted LiveSync before synchronising.`,
LOG_LEVEL_NOTICE
);
}
@@ -1,7 +1,12 @@
import { promiseWithResolvers } from "octagonal-wheels/promises";
import { reactiveSource } from "octagonal-wheels/dataobject/reactive";
import { describe, expect, it, vi } from "vitest";
import { VER, type EntryDoc, type FilePathWithPrefix } from "@vrtmrz/livesync-commonlib/compat/common/types";
import {
VERSIONING_DOCID,
type EntryDoc,
type FilePathWithPrefix,
} from "@vrtmrz/livesync-commonlib/compat/common/types";
import { ENCRYPTED_INTERNAL_METADATA_FEATURE, REMOTE_FEATURE_GENERATION } from "@vrtmrz/livesync-commonlib/replication";
import {
isValidFilenameInAndroid,
isValidFilenameInWidows,
@@ -37,6 +42,10 @@ type SetupOptions = {
isValidPath?: (path: string) => boolean;
processSynchroniseResult?: (entry: unknown) => Promise<boolean>;
setSnapshot?: (key: string, value: unknown) => Promise<unknown>;
getSnapshot?: (key: string) => Promise<unknown>;
localVersionInfo?: unknown;
isTargetFile?: (path: string) => Promise<boolean>;
databaseId?: string;
};
function setup(options: SetupOptions = {}) {
@@ -47,6 +56,9 @@ function setup(options: SetupOptions = {}) {
const isReady = vi.fn(() => options.applicationReady ?? true);
const isValidPath = vi.fn(options.isValidPath ?? (() => true));
const getDBEntryFromMeta = vi.fn(async (entry: object) => ({ ...entry, data: "x" }));
const localPhysicalDatabase = {
...(options.databaseId ? { id: vi.fn(async () => options.databaseId) } : {}),
} as PouchDB.Database<EntryDoc>;
const core = {
services: {
appLifecycle: { isReady, isSuspended: () => false },
@@ -62,14 +74,21 @@ function setup(options: SetupOptions = {}) {
},
replicator: { onCloseActiveReplication, runBoundedLocalApplicationActivity },
vault: {
isTargetFile: vi.fn(async () => true),
isTargetFile: vi.fn(options.isTargetFile ?? (async () => true)),
isFileSizeTooLarge: vi.fn(() => false),
isValidPath,
},
},
kvDB: { set: setSnapshot },
kvDB: { set: setSnapshot, get: vi.fn(options.getSnapshot ?? (async () => undefined)) },
localDatabase: {
getRaw: vi.fn(async (id: string) => ({ _id: id, _rev: "1-test" })),
localDatabase: localPhysicalDatabase,
getRaw: vi.fn(async (id: string) => {
if (id === VERSIONING_DOCID) {
if (options.localVersionInfo === undefined) throw { status: 404 };
return options.localVersionInfo;
}
return { _id: id, _rev: "1-test" };
}),
getDBEntryFromMeta,
},
};
@@ -88,6 +107,9 @@ function setup(options: SetupOptions = {}) {
} as never);
return {
getDBEntryFromMeta,
isTargetFile: core.services.vault.isTargetFile,
localPhysicalDatabase,
localDatabase: core.localDatabase,
isReady,
isValidPath,
onCloseActiveReplication,
@@ -98,6 +120,34 @@ function setup(options: SetupOptions = {}) {
}
describe("ReplicateResultProcessor", () => {
it("does not add a permanent application block when snapshot recovery fails", async () => {
const { processor } = setup({
getSnapshot: async () => {
throw new Error("KV unavailable");
},
});
await expect(processor.restoreFromSnapshotOnce()).rejects.toThrow("KV unavailable");
expect(processor.isSuspended).toBe(false);
});
it("restores pending notes without retaining a past feature rejection in KV", async () => {
const { processor, processSynchroniseResult, onCloseActiveReplication } = setup({
databaseId: "same-database",
getSnapshot: async () => ({
databaseId: "same-database",
invalidControlObserved: true,
observedFeatures: ["future-format-v7"],
observedGeneration: 14,
queued: [note("recovered-note")],
processing: [],
}),
});
await processor.restoreFromSnapshotOnce();
expect(processor.isSuspended).toBe(false);
await vi.waitFor(() => expect(processSynchroniseResult).toHaveBeenCalledOnce());
expect(onCloseActiveReplication).not.toHaveBeenCalled();
});
it.each([
["Windows", isValidFilenameInWidows],
["Android", isValidFilenameInAndroid],
@@ -145,9 +195,9 @@ describe("ReplicateResultProcessor", () => {
});
}
expect(processSynchroniseResult).toHaveBeenCalledTimes(11);
expect(processSynchroniseResult.mock.calls.some(([entry]) =>
(entry as { _id: string })._id === "unrelated-queue"
)).toBe(true);
expect(
processSynchroniseResult.mock.calls.some(([entry]) => (entry as { _id: string })._id === "unrelated-queue")
).toBe(true);
});
it("suspends result application while the application is not ready", () => {
@@ -199,10 +249,10 @@ describe("ReplicateResultProcessor", () => {
it("retires active ownership when a newer remote version is observed", async () => {
const { onCloseActiveReplication, processor } = setup();
const versionInfo = {
_id: "versioninfo",
_id: VERSIONING_DOCID,
_rev: "1-test",
type: "versioninfo",
version: VER + 1,
version: REMOTE_FEATURE_GENERATION + 1,
} as unknown as PouchDB.Core.ExistingDocument<EntryDoc>;
processor.enqueueAll([versionInfo]);
@@ -210,6 +260,64 @@ describe("ReplicateResultProcessor", () => {
await vi.waitFor(() => expect(onCloseActiveReplication).toHaveBeenCalledOnce());
});
it("continues applying documents after restoring a legacy local version document", async () => {
const { onCloseActiveReplication, processor, processSynchroniseResult } = setup({
localVersionInfo: {
_id: VERSIONING_DOCID,
type: "versioninfo",
version: 11,
},
});
await processor.restoreFromSnapshotOnce();
processor.enqueueAll([note("legacy-database-note")]);
await vi.waitFor(() => expect(processSynchroniseResult).toHaveBeenCalledOnce());
expect(processor.isSuspended).toBe(false);
expect(onCloseActiveReplication).not.toHaveBeenCalled();
});
it("continues when a newly received feature is supported", async () => {
const { onCloseActiveReplication, processor, processSynchroniseResult } = setup();
const versionInfo = {
_id: VERSIONING_DOCID,
_rev: "2-supported",
type: "versioninfo",
version: REMOTE_FEATURE_GENERATION,
used_features: [ENCRYPTED_INTERNAL_METADATA_FEATURE],
} as PouchDB.Core.ExistingDocument<EntryDoc>;
processor.enqueueAll([versionInfo, note("supported-update")]);
await vi.waitFor(() => expect(processSynchroniseResult).toHaveBeenCalledOnce());
expect(onCloseActiveReplication).not.toHaveBeenCalled();
});
it("reports unknown feature identifiers and requests Replicator retirement", () => {
const logger = vi.fn();
setGlobalLogFunction(logger);
try {
const { processor, onCloseActiveReplication } = setup();
processor.enqueueAll([
{
_id: VERSIONING_DOCID,
_rev: "1-unknown",
type: "versioninfo",
version: REMOTE_FEATURE_GENERATION,
used_features: ["future-format-v7"],
} as PouchDB.Core.ExistingDocument<EntryDoc>,
]);
expect(onCloseActiveReplication).toHaveBeenCalledOnce();
expect(logger).toHaveBeenCalledWith(
expect.stringContaining("future-format-v7"),
LOG_LEVEL_NOTICE,
undefined
);
} finally {
setGlobalLogFunction(defaultLogger);
}
});
it("scans normal-file metadata without loading chunk documents and requeues it", async () => {
const documents = [
{ _id: "first", _rev: "1-a", type: "plain", path: "first.md" },
@@ -1,4 +1,8 @@
import type { ObsidianLiveSyncSettings } from "@vrtmrz/livesync-commonlib/compat/common/types";
import {
VERSIONING_DOCID,
type EntryDoc,
type ObsidianLiveSyncSettings,
} from "@vrtmrz/livesync-commonlib/compat/common/types";
import { assessTweakCompatibility } from "@vrtmrz/livesync-commonlib/settings";
import { LOG_LEVEL_INFO, LOG_LEVEL_NOTICE, Logger } from "octagonal-wheels/common/logger";
import { skipIfDuplicated } from "octagonal-wheels/concurrency/lock";
@@ -7,12 +11,27 @@ import { LiveSyncCouchDBReplicator } from "@vrtmrz/livesync-commonlib/compat/rep
import {
CENTRAL_COMPATIBILITY_REJECTION_REASONS,
REPLICATION_PROGRESS_PRESENTATIONS,
assessRemoteFeatureDocument,
describeRemoteFeatureRejection,
type ReplicatorInstance,
type ReplicationFailureRequest,
} from "@vrtmrz/livesync-commonlib/replication";
import { $msg } from "@/common/translation";
import { usesLegacyIndexedDBAdapter } from "@/common/compatibilitySettings";
import type { LiveSyncBaseCore } from "@/LiveSyncBaseCore";
import type PouchDB from "pouchdb-core";
async function canInterpretCleanupDatabase(db: PouchDB.Database<EntryDoc>): Promise<boolean> {
try {
const assessment = assessRemoteFeatureDocument(await db.get(VERSIONING_DOCID));
if (assessment.status === "supported" || assessment.status === "older-generation") return true;
Logger(`Database cleanup cancelled: ${describeRemoteFeatureRejection(assessment)}`, LOG_LEVEL_NOTICE);
} catch (error) {
Logger("Database cleanup cancelled: feature compatibility could not be checked.", LOG_LEVEL_NOTICE);
Logger(error, LOG_LEVEL_INFO);
}
return false;
}
type CentralCompatibilityRecoveryServices = Pick<
LiveSyncBaseCore["services"],
@@ -60,6 +79,7 @@ export function createCentralCompatibilityRecovery(context: CentralCompatibility
) {
Logger("The remote database has been cleaned.", showProgress ? LOG_LEVEL_NOTICE : LOG_LEVEL_INFO);
await skipIfDuplicated("cleanup", async () => {
if (!(await canInterpretCleanupDatabase(context.getLocalDatabase().localDatabase))) return;
const count = await purgeUnreferencedChunks(context.getLocalDatabase().localDatabase, true);
const message = `The remote database has been cleaned up.
To synchronize, this device must be also cleaned up. ${count} chunk(s) will be erased from this device.
@@ -1,5 +1,5 @@
import { describe, expect, it, vi } from "vitest";
import type { ObsidianLiveSyncSettings } from "@vrtmrz/livesync-commonlib/compat/common/types";
import { VERSIONING_DOCID, type ObsidianLiveSyncSettings } from "@vrtmrz/livesync-commonlib/compat/common/types";
import { assessTweakCompatibility } from "@vrtmrz/livesync-commonlib/settings";
import { defaultLogger, LOG_LEVEL_INFO, LOG_LEVEL_NOTICE, setGlobalLogFunction } from "octagonal-wheels/common/logger";
import {
@@ -24,6 +24,49 @@ import { LiveSyncCouchDBReplicator } from "@vrtmrz/livesync-commonlib/compat/rep
import { createCentralCompatibilityRecovery } from "./centralCompatibilityRecovery";
describe("central compatibility recovery", () => {
it("does not count chunks for cleanup when local feature requirements are unknown", async () => {
chunkMocks.purgeUnreferencedChunks.mockClear();
const confirmWithMessage = vi.fn(async () => "Dismiss");
const recovery = createCentralCompatibilityRecovery({
confirm: { confirmWithMessage },
getLocalDatabase: () => ({
localDatabase: {
get: vi.fn(async (id: string) => ({
_id: id,
type: "versioninfo",
version: 13,
used_features: ["future-format-v7"],
})),
},
}),
services: { replicator: {} },
} as never);
await recovery.reconcileCleanedRemote(true, {} as ObsidianLiveSyncSettings, {} as never);
expect(chunkMocks.purgeUnreferencedChunks).not.toHaveBeenCalled();
expect(confirmWithMessage).not.toHaveBeenCalled();
});
it("allows cleanup counting for a legacy local version document", async () => {
chunkMocks.purgeUnreferencedChunks.mockClear();
const confirmWithMessage = vi.fn(async () => "Dismiss");
const recovery = createCentralCompatibilityRecovery({
confirm: { confirmWithMessage },
getLocalDatabase: () => ({
localDatabase: {
get: vi.fn(async (id: string) => ({ _id: id, type: "versioninfo", version: 11 })),
},
}),
services: { replicator: {} },
} as never);
await recovery.reconcileCleanedRemote(true, {} as ObsidianLiveSyncSettings, {} as never);
expect(chunkMocks.purgeUnreferencedChunks).toHaveBeenCalledWith(expect.anything(), true);
expect(confirmWithMessage).toHaveBeenCalledOnce();
});
it("passes the failed attempt's exact tweak assessment to mismatch resolution", async () => {
const setting = { customChunkSize: 0 };
const preferredTweakValue = { customChunkSize: 60 };
@@ -292,7 +335,9 @@ describe("central compatibility recovery", () => {
});
const runFiniteReplicationActivity = vi.fn(async (task: () => unknown) => await task());
const openOneShotReplication = vi.fn(async () => true);
const remoteDatabase = { close: vi.fn(async () => undefined) };
const remoteDatabase = {
close: vi.fn(async () => undefined),
};
const close = vi.fn(async () => undefined);
const activeReplicator = Object.assign(new LiveSyncCouchDBReplicator({} as never), {
connectRemoteCouchDBWithSetting: vi.fn(async () => ({ db: remoteDatabase, close })),
@@ -303,7 +348,12 @@ describe("central compatibility recovery", () => {
const runWithActiveReplicatorContext = vi.fn(async (task: (context: unknown) => unknown) =>
task(expectedContext)
);
const localDatabase = { localDatabase: {}, clearCaches: vi.fn() };
const localDatabase = {
localDatabase: {
get: vi.fn(async () => ({ _id: VERSIONING_DOCID, type: "versioninfo", version: 12 })),
},
clearCaches: vi.fn(),
};
const getLocalDatabase = vi.fn(() => localDatabase);
const recovery = createCentralCompatibilityRecovery({
confirm: { confirmWithMessage: vi.fn(async () => "Cleanup") },
@@ -335,7 +385,7 @@ describe("central compatibility recovery", () => {
activityFinished.mock.invocationCallOrder[0]
);
expect(chunkMocks.balanceChunkPurgedDBs).toHaveBeenCalledOnce();
expect(getLocalDatabase).toHaveBeenCalledTimes(2);
expect(getLocalDatabase).toHaveBeenCalled();
expect(close).toHaveBeenCalledOnce();
expect(close.mock.invocationCallOrder[0]).toBeLessThan(activityFinished.mock.invocationCallOrder[0]);
});
@@ -1,6 +1,7 @@
import { describe, expect, it, vi } from "vitest";
import { createServiceContext } from "@vrtmrz/livesync-commonlib/context";
import { VER, type EntryDoc } from "@vrtmrz/livesync-commonlib/compat/common/types";
import { VERSIONING_DOCID, type EntryDoc } from "@vrtmrz/livesync-commonlib/compat/common/types";
import { REMOTE_FEATURE_GENERATION } from "@vrtmrz/livesync-commonlib/replication";
import { promiseWithResolvers } from "octagonal-wheels/promises";
import { useReplicationFeature } from "./index";
@@ -19,8 +20,14 @@ type SetupOptions = {
};
function setup(options: SetupOptions = {}) {
const defaultLocalDatabase = {
localDatabase: {},
getRaw: vi.fn(async () => {
throw { status: 404 };
}),
};
const {
getLocalDatabase = () => ({}),
getLocalDatabase = () => defaultLocalDatabase,
keyValueDB = {
kvDB: {
get: vi.fn(async () => undefined),
@@ -39,7 +46,7 @@ function setup(options: SetupOptions = {}) {
API: { isMobile: vi.fn(() => false), isOnline: true },
appLifecycle: {
getUnresolvedMessages: { addHandler: vi.fn() },
isReady: true,
isReady: vi.fn(() => true),
isSuspended: vi.fn(() => false),
onSettingLoaded: { addHandler: vi.fn() },
},
@@ -48,6 +55,7 @@ function setup(options: SetupOptions = {}) {
keyValueDB,
path: { getPath: vi.fn((entry: { path: string }) => entry.path) },
replication: {
replicationResultCount: { value: 0 },
onBeforeReplicate: {
addHandler: vi.fn((handler: BooleanHandler, priority = 0) => {
beforeReplicateHandlers.set(priority, handler);
@@ -165,10 +173,10 @@ describe("replication serviceFeature composition", () => {
const onCloseActiveReplication = vi.fn(() => retirement.promise);
const harness = setup({ onCloseActiveReplication });
const versionInfo = {
_id: "versioninfo",
_id: VERSIONING_DOCID,
_rev: "1-test",
type: "versioninfo",
version: VER + 1,
version: REMOTE_FEATURE_GENERATION + 1,
} as unknown as PouchDB.Core.ExistingDocument<EntryDoc>;
expect(harness.parseHandler).toBeDefined();
+5 -1
View File
@@ -137,7 +137,7 @@ The mobile pass uses Obsidian's `app.emulateMobile(true)`, a 390 by 844 CSS-pixe
The same workflow checks the two remote-activity status boundaries. It first holds a real CouchDB request at the selected fetch implementation and confirms that `🌐N` is visible while `📲` is absent. It then holds the real one-shot replication immediately before its replicator call, confirms that `📲` is visible while no physical request is active, releases it, and requires the finite and bounded activity counts to return to zero, the request and response counts to balance, and both indicators to disappear. Finally, it creates a remote-only chunk, holds the real on-demand fetch immediately before its remote call, makes the same logical active and idle assertions, and verifies that the fetched chunk is written into the local database. These gates make the active states deterministic without replacing the remote request or operation.
`test:e2e:obsidian:couchdb-manual-setup-workflow` follows the visible onboarding path for the first device when no Setup URI is available. It enters end-to-end encryption and CouchDB details, runs the read-only `Check server requirements` step, requires the prepared fixture to pass without applying a server fix, and lets the onboarding connection test create the named database. After Rebuild completes on the first device, it creates an ordinary note, asks that working device to generate a Setup URI for a second device, completes Fetch there, and verifies a bidirectional note round-trip. The workflow captures each decision point and the expanded server-check result; password controls remain visually masked.
`test:e2e:obsidian:couchdb-manual-setup-workflow` follows the visible onboarding path for the first device when no Setup URI is available. It enters end-to-end encryption and CouchDB details, runs the read-only `Check server requirements` step, requires the prepared fixture to pass without applying a server fix, and lets the onboarding connection test create the named database. After Rebuild completes on the first device, it creates an ordinary note, asks that working device to generate a Setup URI for a second device, completes Fetch there, and verifies a bidirectional note round-trip. The workflow captures each decision point and the expanded server-check result; password controls remain visually masked. It uses an E2EE passphrase beginning with `%`, confirms that the saved settings do not contain it in plain text, and checks that Obsidian restores it after restarting with the first Vault.
If this status workflow fails while Obsidian is running, it writes a full-page screenshot and a JSON snapshot of the status text and counters under `/tmp/obsidian-livesync-e2e`. The dialogue-mount workflow leaves desktop and mobile screenshots for both representative Svelte routes, the Hidden File Sync workflow captures the successfully displayed JSON Resolve dialogue before selecting an option, and the Security Seed reconnect workflow captures each significant application state. The suite therefore records representative evidence without capturing every interaction. Set `E2E_OBSIDIAN_DIAGNOSTICS_DIR` to use another directory.
@@ -202,6 +202,10 @@ This proves in real Obsidian the plug-in behaviour shared by supported platforms
`test:e2e:obsidian:customisation-sync` runs a two-vault Customisation Sync workflow. It scans a real snippet CSS file, config JSON file, and sample plug-in fixture into per-file Customisation Sync data, synchronises the entries through CouchDB, applies them on the second vault, verifies the resulting `.obsidian` files, propagates a snippet update, and verifies deletion of the source-vault snippet sync data without confusing it with the target vault's own applied copy.
`test:e2e:obsidian:remote-feature-change` starts real Obsidian with continuous CouchDB replication, then changes the remote version document from generation 12 to generation 13 with an unknown feature. It waits for the control document to reach the local database and the active Replicator to retire, checks that another replication is refused, and verifies that an already accepted Vault note remains intact. After restarting the same Vault, the current remote declaration still blocks finite replication and the actual continuous connection attempt. No KV feature history is involved. It is a focused test outside `test:e2e:obsidian:local-suite`; recovery with a future compatible client remains a separate validation boundary.
`test:e2e:obsidian:internal-metadata-migration` enables internal Metadata encryption through the settings UI without Rebuild. It checks unchanged plaintext and rewritten encrypted Hidden File Sync and Customisation Sync Metadata in CouchDB, stable document IDs, mismatch rejection on a second device, and file restoration after aligning settings. It then turns the preference OFF, runs Fast Fetch, and compares content loaded from both Metadata representations and their Chunks while retaining the remote feature declaration. These focused tests use the local CouchDB fixture and are outside `test:e2e:obsidian:local-suite`.
`test:e2e:obsidian:setting-markdown-export` enables setting Markdown export, waits for the generated Markdown file in the vault, and verifies that credentials are omitted when `writeCredentialsForSettingSync=false`.
`test:e2e:obsidian:upgrade-from-stable` is the release-acceptance upgrade workflow. It installs the exact published 0.25.83 artefacts into an isolated Vault, verifies their pinned SHA-256 values, and then replaces only the plug-in artefacts with the current target while retaining the same Vault and isolated Obsidian profile. The first run downloads the old release into the ignored `_testdata/releases` cache; every later run verifies the cached bytes before use.
@@ -210,6 +210,7 @@ async function configureLiveSyncCli(
encrypt: true,
passphrase: e2eePassphrase,
usePathObfuscation: true,
encryptInternalMetadata: true,
doctorProcessedVersion: "0.25.27",
isConfigured: true,
});
@@ -323,6 +324,7 @@ async function main(): Promise<void> {
encrypt: true,
passphrase: e2eePassphrase,
usePathObfuscation: true,
encryptInternalMetadata: true,
E2EEAlgorithm: "v2",
}
),
@@ -39,6 +39,7 @@ process.env.E2E_OBSIDIAN_CLI_TIMEOUT_MS ??= "90000";
process.env.E2E_OBSIDIAN_COUCHDB_TIMEOUT_MS ??= "30000";
const uiTimeoutMs = Number(process.env.E2E_OBSIDIAN_SETUP_URI_TIMEOUT_MS ?? 30000);
const e2eePassphrase = `%${randomBytes(24).toString("base64url")}`;
const notePath = "E2E/manual-couchdb/from-first-device.md";
const noteContent = "# Manual CouchDB setup\n\nThis note was sent by the manually configured first device.\n";
const returnNotePath = "E2E/manual-couchdb/from-second-device.md";
@@ -147,8 +148,7 @@ async function enterManualCouchDBSettings(port: number, couchDb: CouchDbConfig,
.locator('input[type="checkbox"]')
.first()
.check({ timeout: uiTimeoutMs });
const passphraseValue = randomBytes(24).toString("base64url");
await passphraseInput.fill(passphraseValue);
await passphraseInput.fill(e2eePassphrase);
const passwordToggle = encryption.locator("button.sls-password-toggle");
await passwordToggle.click({ timeout: uiTimeoutMs });
assertEqual(
@@ -158,7 +158,7 @@ async function enterManualCouchDBSettings(port: number, couchDb: CouchDbConfig,
);
assertEqual(
await passphraseInput.inputValue(),
passphraseValue,
e2eePassphrase,
"Toggling visibility changed the passphrase value."
);
await passwordToggle.click({ timeout: uiTimeoutMs });
@@ -169,7 +169,7 @@ async function enterManualCouchDBSettings(port: number, couchDb: CouchDbConfig,
);
assertEqual(
await passphraseInput.inputValue(),
passphraseValue,
e2eePassphrase,
"Re-masking the passphrase changed its value."
);
});
@@ -301,6 +301,23 @@ async function assertPersistedE2EE(vault: TemporaryVault): Promise<void> {
if (typeof persisted.encryptedPassphrase !== "string" || persisted.encryptedPassphrase.length === 0) {
throw new Error("Manual CouchDB setup did not persist an encrypted E2EE passphrase.");
}
if (JSON.stringify(persisted).includes(e2eePassphrase)) {
throw new Error("Manual CouchDB setup persisted the E2EE passphrase in plain text.");
}
}
async function assertRestoredE2EEPassphrase(session: ObsidianLiveSyncSession, cliBinary: string): Promise<void> {
const restored = await evalObsidianJson<boolean>(
cliBinary,
[
"(()=>{",
"const settings=app.plugins.plugins['obsidian-livesync'].core.services.setting.currentSettings();",
`return JSON.stringify(settings.passphrase === ${JSON.stringify(e2eePassphrase)});`,
"})()",
].join(""),
session.cliEnv
);
assertEqual(restored, true, "The E2EE passphrase was not restored after Obsidian restarted.");
}
async function setRemotePreferredE2EEDisabled(context: RunnerContext): Promise<void> {
@@ -454,6 +471,7 @@ async function main(): Promise<void> {
session = await startUnconfiguredSession(context, vaultA);
try {
await assertRestoredE2EEPassphrase(session, context.cliBinary);
await scheduleRemoteOverwrite(session.remoteDebuggingPort);
screenshots.push(await confirmRebuild(session.remoteDebuggingPort, e2eeRebuildCaptures));
screenshots.push(
@@ -1,10 +1,13 @@
import { mkdir, readFile, rm, writeFile } from "node:fs/promises";
import { dirname, join } from "node:path";
import { VERSIONING_DOCID } from "@vrtmrz/livesync-commonlib/compat/common/types";
import { ENCRYPTED_INTERNAL_METADATA_FEATURE, REMOTE_FEATURE_GENERATION } from "@vrtmrz/livesync-commonlib/replication";
import { evalObsidianJson } from "../runner/cli.ts";
import {
assertCouchDbReachable,
createCouchDbDatabase,
deleteCouchDbDatabase,
fetchCouchDbDocument,
loadCouchDbConfig,
makeUniqueDatabaseName,
waitForCouchDbDocs,
@@ -159,6 +162,10 @@ async function startConfiguredSession(
dbName: context.dbName,
};
const customisationSettings = {
encrypt: true,
passphrase: "internal-metadata-e2e-secret",
usePathObfuscation: true,
encryptInternalMetadata: true,
deviceAndVaultName: deviceName,
usePluginSync: true,
usePluginSyncV2: true,
@@ -486,6 +493,18 @@ async function main(): Promise<void> {
(target) => ids.has(target.id) && target.children.every((childId) => ids.has(childId))
);
});
for (const target of [entry, configEntry, ...pluginEntries]) {
const remoteEntry = await fetchCouchDbDocument(context.couchDb, context.dbName, target.id);
if (!remoteEntry.path?.startsWith("/\\:") || remoteEntry.children?.length !== 0 ||
remoteEntry.ctime !== 0 || remoteEntry.mtime !== 0 || remoteEntry.size !== 0) {
throw new Error(`Customisation Sync Metadata was not encrypted for ${target.id}.`);
}
}
const versionInfo = await fetchCouchDbDocument(context.couchDb, context.dbName, VERSIONING_DOCID);
if (versionInfo.version !== REMOTE_FEATURE_GENERATION ||
!(versionInfo.used_features as unknown[] | undefined)?.includes(ENCRYPTED_INTERNAL_METADATA_FEATURE)) {
throw new Error("The remote feature list does not declare encrypted internal Metadata.");
}
await session.app.stop();
session = await startConfiguredSession(context, vaultB, targetDeviceName);
@@ -1,5 +1,7 @@
import { mkdir, readFile, rm, writeFile } from "node:fs/promises";
import { dirname, join } from "node:path";
import { VERSIONING_DOCID } from "@vrtmrz/livesync-commonlib/compat/common/types";
import { ENCRYPTED_INTERNAL_METADATA_FEATURE, REMOTE_FEATURE_GENERATION } from "@vrtmrz/livesync-commonlib/replication";
import {
assertLocatorHasMinimumTouchTarget,
assertLocatorWithinSafeArea,
@@ -11,6 +13,7 @@ import {
assertCouchDbReachable,
createCouchDbDatabase,
deleteCouchDbDatabase,
fetchCouchDbDocument,
loadCouchDbConfig,
makeUniqueDatabaseName,
waitForCouchDbDocs,
@@ -324,6 +327,10 @@ async function startConfiguredSession(
dbName: context.dbName,
};
const hiddenFileSettings = {
encrypt: true,
passphrase: "internal-metadata-e2e-secret",
usePathObfuscation: true,
encryptInternalMetadata: true,
syncInternalFiles: true,
syncInternalFilesBeforeReplication: true,
watchInternalFileChanges: false,
@@ -360,6 +367,23 @@ async function uploadHiddenFile(
const ids = new Set(docs.map((doc) => doc._id));
return ids.has(entry.id) && entry.children.every((childId) => ids.has(childId));
});
const remoteEntry = await fetchCouchDbDocument(context.couchDb, context.dbName, entry.id);
if (
!remoteEntry.path?.startsWith("/\\:") ||
remoteEntry.children?.length !== 0 ||
remoteEntry.ctime !== 0 ||
remoteEntry.mtime !== 0 ||
remoteEntry.size !== 0
) {
throw new Error(`Hidden File Sync Metadata was not encrypted for ${entry.id}.`);
}
const versionInfo = await fetchCouchDbDocument(context.couchDb, context.dbName, VERSIONING_DOCID);
if (
versionInfo.version !== REMOTE_FEATURE_GENERATION ||
!(versionInfo.used_features as unknown[] | undefined)?.includes(ENCRYPTED_INTERNAL_METADATA_FEATURE)
) {
throw new Error("The remote feature list does not declare encrypted internal Metadata.");
}
return entry;
}
@@ -383,6 +407,29 @@ async function runCreateRoundTrip(
await writeVaultFile(vaultA.path, snippetPath, snippetContent);
let session = await startConfiguredSession(context, vaultA);
const entry = await uploadHiddenFile(context, session, snippetPath);
await evalObsidianJson(
context.cliBinary,
[
"(async()=>{",
"const rebuilder=app.plugins.plugins['obsidian-livesync'].core.rebuilder;",
"const inform=rebuilder.informOptionalFeatures;",
"rebuilder.informOptionalFeatures=async()=>{};",
"try{await rebuilder.$rebuildRemote();}finally{rebuilder.informOptionalFeatures=inform;}",
"return JSON.stringify(true);",
"})()",
].join(""),
session.cliEnv
);
const rebuiltVersion = await fetchCouchDbDocument(context.couchDb, context.dbName, VERSIONING_DOCID);
const rebuiltEntry = await fetchCouchDbDocument(context.couchDb, context.dbName, entry.id);
if (
rebuiltVersion.version !== REMOTE_FEATURE_GENERATION ||
!(rebuiltVersion.used_features as unknown[] | undefined)?.includes(ENCRYPTED_INTERNAL_METADATA_FEATURE) ||
!rebuiltEntry.path?.startsWith("/\\:")
) {
throw new Error("Remote Rebuild did not declare and encrypt internal Metadata for its accepted writer.");
}
console.log("Remote Rebuild declared the feature and preserved encrypted Hidden File Sync Metadata.");
await session.app.stop();
session = await startConfiguredSession(context, vaultB);
@@ -571,11 +618,14 @@ async function runInitialisationNoticeGrouping(context: RunnerContext, vault: Te
await withObsidianPage(port, async (page) => {
const deadline = Date.now() + timeoutMs;
while ((await page.locator(".notice:visible").count()) > 0 && Date.now() < deadline) {
await page.locator(".notice:visible").first().click({
force: true,
position: { x: 2, y: 2 },
timeout: timeoutMs,
});
await page
.locator(".notice:visible")
.first()
.click({
force: true,
position: { x: 2, y: 2 },
timeout: timeoutMs,
});
}
assertEqual(
await page.locator(".notice:visible").count(),
@@ -707,17 +757,15 @@ async function runInitialisationNoticeGrouping(context: RunnerContext, vault: Te
const result = await withObsidianPage(port, async (page) => {
await page.evaluate((stateKey) => {
const state = (globalThis as unknown as Record<
string,
{ releasePreparation?: () => void } | undefined
>)[stateKey];
const state = (
globalThis as unknown as Record<string, { releasePreparation?: () => void } | undefined>
)[stateKey];
state?.releasePreparation?.();
}, hiddenFileInitialisationStateKey);
await page.waitForFunction(
(stateKey) =>
(globalThis as unknown as Record<string, { reachedInitialisation?: boolean } | undefined>)[
stateKey
]?.reachedInitialisation === true,
(globalThis as unknown as Record<string, { reachedInitialisation?: boolean } | undefined>)[stateKey]
?.reachedInitialisation === true,
hiddenFileInitialisationStateKey,
{ timeout: timeoutMs }
);
@@ -0,0 +1,285 @@
import { mkdir, readFile, writeFile } from "node:fs/promises";
import { dirname, join } from "node:path";
import { VERSIONING_DOCID, type LoadedEntry } from "@vrtmrz/livesync-commonlib/compat/common/types";
import { readContent } from "@vrtmrz/livesync-commonlib/compat/common/utils";
import { ENCRYPTED_INTERNAL_METADATA_FEATURE } from "@vrtmrz/livesync-commonlib/replication";
import { evalObsidianJson } from "../runner/cli.ts";
import {
assertCouchDbReachable,
createCouchDbDatabase,
deleteCouchDbDatabase,
fetchCouchDbDocument,
loadCouchDbConfig,
makeUniqueDatabaseName,
} from "../runner/couchdb.ts";
import { discoverObsidianCli, requireObsidianBinary } from "../runner/environment.ts";
import {
assertEqual,
configureCouchDb,
createE2eCouchDbPluginData,
createE2eObsidianDeviceLocalState,
prepareRemote,
pushLocalChanges,
waitForLiveSyncCoreReady,
} from "../runner/liveSyncWorkflow.ts";
import { startObsidianLiveSyncSession, type ObsidianLiveSyncSession } from "../runner/session.ts";
import { openLiveSyncSettings, waitForVisibleObsidianDialogue, withObsidianPage } from "../runner/ui.ts";
import { createTemporaryVault, type TemporaryVault } from "../runner/vault.ts";
process.env.E2E_OBSIDIAN_CLI_TIMEOUT_MS ??= "60000";
const hiddenPaths = [".metadata-migration/retained.json", ".metadata-migration/rewritten.json"];
const customPaths = [".obsidian/snippets/retained-metadata.css", ".obsidian/snippets/rewritten-metadata.css"];
const paths = [...hiddenPaths, ...customPaths];
const initialContent = "/* Metadata migration fixture */\n";
const updatedContent = "/* Updated after enabling internal Metadata encryption */\n";
const optionSettings = {
encrypt: true,
passphrase: "internal-metadata-migration-secret",
usePathObfuscation: true,
encryptInternalMetadata: false,
syncInternalFiles: true,
syncInternalFilesBeforeReplication: false,
watchInternalFileChanges: false,
syncInternalFilesTargetPatterns: "^\\.metadata-migration(?:/|$)",
usePluginSync: true,
usePluginSyncV2: true,
autoSweepPlugins: false,
autoSweepPluginsPeriodic: false,
autoAcceptCompatibleTweak: false,
};
type Entry = { id: string; path: string };
async function main(): Promise<void> {
const binary = requireObsidianBinary();
const cliBinary = discoverObsidianCli().binary;
if (!cliBinary) throw new Error("The Obsidian CLI is unavailable.");
const couchDb = await loadCouchDbConfig();
const dbName = makeUniqueDatabaseName(couchDb.dbPrefix, "internal-metadata-migration");
const connection = { ...couchDb, dbName };
const source = await createTemporaryVault();
const target = await createTemporaryVault();
let session: ObsidianLiveSyncSession | undefined;
const evaluate = async <T>(body: string): Promise<T> => {
if (!session) throw new Error("No active Obsidian session.");
return await evalObsidianJson<T>(
cliBinary,
`(async()=>{const core=app.plugins.plugins['obsidian-livesync'].core;${body}})()`,
session.cliEnv
);
};
const start = async (vault: TemporaryVault, device: string) => {
const settings = { ...optionSettings, deviceAndVaultName: device };
session = await startObsidianLiveSyncSession({
binary,
cliBinary,
vault,
pluginData: createE2eCouchDbPluginData(connection, settings),
localStorageEntries: createE2eObsidianDeviceLocalState(vault.name),
});
await waitForLiveSyncCoreReady(cliBinary, session.cliEnv);
await configureCouchDb(cliBinary, session.cliEnv, connection, settings);
await evaluate(`core.services.setting.setDeviceAndVaultName(${JSON.stringify(device)});
await core.services.setting.saveSettingData(); return JSON.stringify(true);`);
await prepareRemote(cliBinary, session.cliEnv);
};
const store = async (customisations: string[]) => {
return await evaluate<Entry[]>(`
await core.getAddOn('HiddenFileSync').scanAllStorageChanges(true);
const config=core.getAddOn('ConfigSync');
for(const path of ${JSON.stringify(customisations)}){
await config.storeCustomizationFiles(path,core.services.setting.getDeviceAndVaultName());
}
const rows=(await core.localDatabase.allDocsRaw({include_docs:true})).rows;
const entries=${JSON.stringify(paths)}.map(path=>rows.map(row=>row.doc).find(doc=>
doc?.path==='i:'+path || doc?.path?.startsWith('ix:migration-source/') && doc.path.endsWith('%'+path.split('/').pop())));
if(entries.some(entry=>!entry)) throw new Error('Missing internal Metadata fixtures: '+JSON.stringify({entries,paths:rows.map(row=>row.doc?.path).filter(Boolean)}));
return JSON.stringify(entries.map(doc=>({id:doc._id,path:doc.path})));`);
};
const preferCurrentSettings = async () => {
await evaluate(`await core.services.replicator.getActiveReplicator()
.setPreferredRemoteTweakSettings(core.services.setting.currentSettings()); return JSON.stringify(true);`);
};
const applyAndCheckFiles = async () => {
await evaluate(`
await core.getAddOn('HiddenFileSync').scanAllDatabaseChanges(true);
const config=core.getAddOn('ConfigSync');
const rows=(await core.localDatabase.allDocsRaw({include_docs:true})).rows;
for(const path of ${JSON.stringify(customPaths)}){
const entry=rows.map(row=>row.doc).find(doc=>doc?.path?.startsWith('ix:migration-source/') && doc.path.endsWith('%'+path.split('/').pop()));
if(!entry) throw new Error('Missing Customisation Sync Metadata');
const display=config.createPluginDataFromV2(entry.path);
await display.setFile(await config.createPluginDataExFileV2(entry.path));
if(!(await config.applyDataV2(display))) throw new Error('Could not apply Customisation Sync data');
}
return JSON.stringify(true);`);
for (const path of paths) {
assertEqual(
await readFile(join(target.path, path), "utf8"),
path.includes("rewritten") ? updatedContent : initialContent,
`Unexpected restored content: ${path}`
);
}
};
const assertDeclaration = async () => {
const version = await fetchCouchDbDocument(couchDb, dbName, VERSIONING_DOCID);
assertEqual(version.version, 13, "The feature generation was not retained.");
assertEqual(
(version.used_features as string[]).includes(ENCRYPTED_INTERNAL_METADATA_FEATURE),
true,
"The encrypted internal Metadata declaration was not retained."
);
};
try {
await assertCouchDbReachable(couchDb);
await createCouchDbDatabase(couchDb, dbName);
for (const path of paths) {
await mkdir(dirname(join(source.path, path)), { recursive: true });
await writeFile(join(source.path, path), initialContent);
}
await start(source, "migration-source");
const entries = await store(customPaths);
await pushLocalChanges(cliBinary, session!.cliEnv);
const originals = await Promise.all(entries.map((entry) => fetchCouchDbDocument(couchDb, dbName, entry.id)));
for (let index = 0; index < entries.length; index++) {
assertEqual(originals[index].path, entries[index].path, "OFF unexpectedly encrypted internal Metadata.");
}
assertEqual(
(await fetchCouchDbDocument(couchDb, dbName, VERSIONING_DOCID)).version,
12,
"The original database was not generation 12."
);
await withObsidianPage(session!.remoteDebuggingPort, async (page) => {
const navigator = await openLiveSyncSettings(page);
const remotePage = await navigator.openPage("Remote Configuration");
await remotePage
.locator(".setting-item")
.filter({
has: navigator.page.getByText("Configure E2EE", { exact: true }),
})
.getByRole("button", { name: "Configure", exact: true })
.click();
const dialog = await waitForVisibleObsidianDialogue(navigator.page, "End-to-End Encryption");
await dialog.getByLabel("Encrypt internal file Properties", { exact: true }).check();
await dialog.getByRole("button", { name: "Proceed", exact: true }).click();
const warning = await waitForVisibleObsidianDialogue(navigator.page, "Encrypt internal file Properties");
await warning
.getByRole("button", {
name: "Enable without rebuilding — update every other device first",
exact: true,
})
.click();
});
assertEqual(
await evaluate(`app.setting.close(); return JSON.stringify(core.settings.encryptInternalMetadata);`),
true,
"The setting dialogue did not enable encryption."
);
for (let index = 0; index < entries.length; index++) {
assertEqual(
(await fetchCouchDbDocument(couchDb, dbName, entries[index].id))._rev,
originals[index]._rev,
"Enabling without rebuilding rewrote an existing document."
);
}
await preferCurrentSettings();
for (const path of [hiddenPaths[1], customPaths[1]]) await writeFile(join(source.path, path), updatedContent);
const rewritten = await store([customPaths[1]]);
assertEqual(
JSON.stringify(rewritten),
JSON.stringify(entries),
"Enabling encryption changed document IDs or paths."
);
await pushLocalChanges(cliBinary, session!.cliEnv);
for (let index = 0; index < entries.length; index++) {
const raw = await fetchCouchDbDocument(couchDb, dbName, entries[index].id);
if (index % 2 === 0) {
assertEqual(raw._rev, originals[index]._rev, "An untouched document was rewritten.");
assertEqual(raw.path, entries[index].path, "An untouched document lost its plaintext Metadata.");
} else {
assertEqual(raw.path?.startsWith("/\\:"), true, "Updated Metadata was not encrypted.");
assertEqual(
JSON.stringify([raw.ctime, raw.mtime, raw.size, raw.children]),
"[0,0,0,[]]",
"Updated Metadata exposed file properties."
);
}
}
await assertDeclaration();
console.log(
"The settings UI enabled encryption without Rebuild; unchanged and encrypted Metadata coexist with stable IDs."
);
await session!.app.stop();
session = undefined;
await start(target, "migration-target");
const rejected = evaluate<boolean>(`return JSON.stringify(await core.services.replication.replicate(true));`);
await withObsidianPage(session!.remoteDebuggingPort, async (page) => {
const dialog = await waitForVisibleObsidianDialogue(page, "Configuration Mismatch Detected");
await dialog
.getByText("Encrypt internal file Properties", { exact: false })
.first()
.waitFor({ state: "visible" });
await dialog.getByRole("button", { name: "Dismiss", exact: true }).click();
});
assertEqual(await rejected, false, "Mismatched settings admitted replication.");
assertEqual(
await evaluate(`const rows=(await core.localDatabase.allDocsRaw({include_docs:true})).rows;
return JSON.stringify(rows.some(row=>${JSON.stringify(entries.map((entry) => entry.id))}.includes(row.id)));`),
false,
"The mismatched device received internal Metadata."
);
await evaluate(`await core.services.setting.applyPartial({encryptInternalMetadata:true},true);
return JSON.stringify(true);`);
await pushLocalChanges(cliBinary, session!.cliEnv);
await applyAndCheckFiles();
console.log("A second device rejected the mismatch, then restored both formats after setting alignment.");
await evaluate(`await core.services.setting.applyPartial({encryptInternalMetadata:false},true);
return JSON.stringify(true);`);
await preferCurrentSettings();
await evaluate(`await core.rebuilder.$fetchLocalDBFast(true);
await core.services.setting.applyPartial(${JSON.stringify(optionSettings)},true);
return JSON.stringify(true);`);
const fetchedEntries = await evaluate<LoadedEntry[]>(`
const entries=[];
for(const path of ${JSON.stringify(entries.map((entry) => entry.path))}){
const entry=await core.localDatabase.getDBEntry(path,undefined,false,true);
if(!entry || entry.deleted || entry._deleted) throw new Error('Could not read fetched Metadata: '+path);
const file=path.startsWith('ix:')
? await core.getAddOn('ConfigSync').createPluginDataExFileV2(path,entry) : entry;
if(!file) throw new Error('Could not decode fetched Customisation Sync content: '+path);
entries.push(file);
}
return JSON.stringify(entries);`);
for (let index = 0; index < fetchedEntries.length; index++) {
const expected = index % 2 === 0 ? initialContent : updatedContent;
const content = readContent(fetchedEntries[index]);
const text = typeof content === "string" ? content : new TextDecoder().decode(content);
assertEqual(
text,
expected,
`OFF did not read internal file content after Fast Fetch: ${entries[index].path}`
);
}
await applyAndCheckFiles();
await assertDeclaration();
console.log(
"Fast Fetch and database content reads accept both formats with the option OFF; the remote declaration remains."
);
} finally {
await session?.app.stop();
await source.dispose();
await target.dispose();
await deleteCouchDbDatabase(couchDb, dbName);
}
}
main().catch((error: unknown) => {
console.error(error instanceof Error ? error.stack : error);
process.exitCode = 1;
});
@@ -0,0 +1,218 @@
import { mkdir, readFile, writeFile } from "node:fs/promises";
import { dirname, join } from "node:path";
import { VERSIONING_DOCID } from "@vrtmrz/livesync-commonlib/compat/common/types";
import { evalObsidianJson } from "../runner/cli.ts";
import {
assertCouchDbReachable,
createCouchDbDatabase,
deleteCouchDbDatabase,
fetchCouchDbDocument,
loadCouchDbConfig,
makeUniqueDatabaseName,
putCouchDbDocument,
} from "../runner/couchdb.ts";
import { discoverObsidianCli, requireObsidianBinary } from "../runner/environment.ts";
import {
assertE2eCompatibilityMarker,
configureCouchDb,
createE2eCouchDbPluginData,
createE2eObsidianDeviceLocalState,
prepareRemote,
pushLocalChanges,
waitForLiveSyncCoreReady,
waitForLocalDatabaseEntry,
} from "../runner/liveSyncWorkflow.ts";
import { startObsidianLiveSyncSession, type ObsidianLiveSyncSession } from "../runner/session.ts";
import { createTemporaryVault } from "../runner/vault.ts";
const acceptedPath = "E2E/remote-feature/accepted.md";
const acceptedContent = "Accepted before the remote feature changed.\n";
const unknownFeature = "future-format-v7";
type FeatureState = {
version: number | null;
features: string[];
hasActiveReplicator: boolean;
};
async function readFeatureState(cliBinary: string, env: NodeJS.ProcessEnv): Promise<FeatureState> {
return await evalObsidianJson<FeatureState>(
cliBinary,
[
"(async()=>{",
"const core=app.plugins.plugins['obsidian-livesync'].core;",
`const id=${JSON.stringify(VERSIONING_DOCID)};`,
"const info=await core.localDatabase.getRaw(id).catch(()=>null);",
"return JSON.stringify({",
"version:typeof info?.version==='number'?info.version:null,",
"features:Array.isArray(info?.used_features)?info.used_features:[],",
"hasActiveReplicator:!!core.services.replicator.getActiveReplicator(),",
"});",
"})()",
].join(""),
env
);
}
async function waitForState(
cliBinary: string,
env: NodeJS.ProcessEnv,
predicate: (state: FeatureState) => boolean,
description: string
): Promise<FeatureState> {
const deadline = Date.now() + 20_000;
let state = await readFeatureState(cliBinary, env);
while (!predicate(state) && Date.now() < deadline) {
await new Promise((resolve) => setTimeout(resolve, 250));
state = await readFeatureState(cliBinary, env);
}
if (!predicate(state)) throw new Error(`Timed out waiting for ${description}: ${JSON.stringify(state)}`);
return state;
}
async function main(): Promise<void> {
const binary = requireObsidianBinary();
const cli = discoverObsidianCli();
if (!cli.binary) throw new Error(`Could not find obsidian-cli. Checked paths: ${cli.checked.join(", ")}`);
const couchDb = await loadCouchDbConfig();
const dbName = makeUniqueDatabaseName(couchDb.dbPrefix, "remote-feature-change");
const vault = await createTemporaryVault();
let session: ObsidianLiveSyncSession | undefined;
try {
await assertCouchDbReachable(couchDb);
await createCouchDbDatabase(couchDb, dbName);
const couchDbSettings = {
uri: couchDb.uri,
username: couchDb.username,
password: couchDb.password,
dbName,
};
const settings = {
encrypt: false,
usePathObfuscation: false,
encryptInternalMetadata: false,
liveSync: false,
};
session = await startObsidianLiveSyncSession({
binary,
cliBinary: cli.binary,
vault,
startupGraceMs: Number(process.env.E2E_OBSIDIAN_STARTUP_GRACE_MS ?? 1000),
pluginData: createE2eCouchDbPluginData(couchDbSettings, settings),
localStorageEntries: createE2eObsidianDeviceLocalState(vault.name),
});
await waitForLiveSyncCoreReady(cli.binary, session.cliEnv);
await assertE2eCompatibilityMarker(cli.binary, session.cliEnv);
await configureCouchDb(cli.binary, session.cliEnv, couchDbSettings, settings);
await prepareRemote(cli.binary, session.cliEnv);
const fullPath = join(vault.path, acceptedPath);
await mkdir(dirname(fullPath), { recursive: true });
await writeFile(fullPath, acceptedContent, "utf-8");
await waitForLocalDatabaseEntry(cli.binary, session.cliEnv, acceptedPath);
await pushLocalChanges(cli.binary, session.cliEnv);
const initialVersion = await fetchCouchDbDocument(couchDb, dbName, VERSIONING_DOCID);
if (initialVersion.version !== 12 || "used_features" in initialVersion) {
throw new Error(
`An inactive feature unexpectedly changed the remote contract: ${JSON.stringify(initialVersion)}`
);
}
const start = await evalObsidianJson<{ status: string }>(
cli.binary,
[
"(async()=>{",
"const core=app.plugins.plugins['obsidian-livesync'].core;",
"await core.services.setting.applyExternalSettings({liveSync:true},true);",
"await core.services.control.applySettings();",
"const result=await core.services.replication.startContinuous({trigger:'daemon',interaction:{kind:'forbidden'}});",
"return JSON.stringify(result);",
"})()",
].join(""),
session.cliEnv
);
if (start.status !== "completed")
throw new Error(`Continuous replication did not start: ${JSON.stringify(start)}`);
await waitForState(cli.binary, session.cliEnv, (state) => state.hasActiveReplicator, "an active Replicator");
await putCouchDbDocument(couchDb, dbName, {
...initialVersion,
version: 13,
used_features: [unknownFeature],
});
const observed = await waitForState(
cli.binary,
session.cliEnv,
(state) => state.version === 13 && state.features.includes(unknownFeature) && !state.hasActiveReplicator,
"the live feature change and Replicator retirement"
);
const replicated = await evalObsidianJson<boolean>(
cli.binary,
"(async()=>JSON.stringify(!!(await app.plugins.plugins['obsidian-livesync'].core.services.replication.replicate(true))))()",
session.cliEnv
);
if (replicated) throw new Error("An unknown remote feature was admitted for another replication.");
const acceptedAfterStop = await readFile(fullPath, "utf-8");
if (acceptedAfterStop !== acceptedContent)
throw new Error("Previously accepted Vault content changed on stop.");
await session.app.stop();
session = undefined;
session = await startObsidianLiveSyncSession({ binary, cliBinary: cli.binary, vault });
await waitForLiveSyncCoreReady(cli.binary, session.cliEnv);
const afterRestart = await waitForState(
cli.binary,
session.cliEnv,
(state) => state.version === 13 && state.features.includes(unknownFeature),
"the remote feature requirement after restart"
);
const replicatedAfterRestart = await evalObsidianJson<boolean>(
cli.binary,
"(async()=>JSON.stringify(!!(await app.plugins.plugins['obsidian-livesync'].core.services.replication.replicate(true))))()",
session.cliEnv
);
const continuousAfterRestart = await evalObsidianJson<{ requestStatus: string; connected: boolean }>(
cli.binary,
[
"(async()=>{",
"const core=app.plugins.plugins['obsidian-livesync'].core;",
"const replicator=core.services.replicator.getActiveReplicator();",
"const original=replicator.openContinuousReplication;",
"let completion;",
"replicator.openContinuousReplication=function(...args){completion=original.apply(this,args);return completion;};",
"try{",
"const result=await core.services.replication.startContinuous({trigger:'daemon',interaction:{kind:'forbidden'}});",
"if(!completion) throw new Error('The continuous provider did not attempt its remote check');",
"return JSON.stringify({requestStatus:result.status,connected:await completion});",
"}finally{replicator.openContinuousReplication=original;}",
"})()",
].join(""),
session.cliEnv
);
if (replicatedAfterRestart || continuousAfterRestart.connected !== false)
throw new Error(
`Replication resumed after restart despite an unknown remote feature: ${JSON.stringify({ afterRestart, replicatedAfterRestart, continuousAfterRestart })}`
);
if ((await readFile(fullPath, "utf-8")) !== acceptedContent)
throw new Error("Previously accepted Vault content changed after restart.");
console.log(
`Active feature change retired the Replicator; the remote declaration refused synchronisation after restart: ${JSON.stringify({ observed, afterRestart })}`
);
} finally {
await session?.app.stop();
await vault.dispose();
if (process.env.E2E_OBSIDIAN_KEEP_COUCHDB !== "true") {
await deleteCouchDbDatabase(couchDb, dbName).catch((error: unknown) => {
console.warn(error instanceof Error ? error.message : error);
});
}
}
}
main().catch((error: unknown) => {
console.error(error instanceof Error ? error.stack : error);
process.exit(1);
});
+16
View File
@@ -12,6 +12,22 @@ Earlier releases remain available in the 1.0 release history, the 1.0 preview hi
## Unreleased
### Privacy and compatibility
#### New Feature
- We can now keep the file properties used by Hidden File Sync and Customisation Sync private in CouchDB.
- **Encrypt internal file Properties** extends E2EE V2 and Property Encryption to their paths, times, sizes, and Chunk references.
- Existing configurations keep this preference disabled. New Vaults enable it for use when the required encryption settings are active.
- Update every synchronising device before enabling it. It protects future writes; a manual remote Rebuild is strongly recommended to protect existing properties.
- We can now see which unsupported feature prevents a client from synchronising with CouchDB.
- Clients check the features required by the remote before transferring data or resetting the local database for Fast Fetch. Receiving an unsupported requirement also stops active replication.
#### Fixed
- We can now keep using an E2EE passphrase beginning with `%` after restarting Obsidian. (#1221)
- LiveSync encrypts it before saving the settings. If an earlier version saved it in plain text, re-enter the passphrase used to encrypt the existing data after updating. Treat that passphrase as exposed if the affected `data.json` was shared.
## 1.0.32
27th September, 2026