Refactor optional file synchronisation ownership

This commit is contained in:
vorotamoroz
2026-09-04 05:19:47 +00:00
parent 826413bf84
commit 8449f9d3f5
53 changed files with 6574 additions and 2156 deletions
@@ -20,12 +20,13 @@ one writer for raw events, scheduled scans, and automatic database-to-local
reflection. The constructor-name add-on lookup and the `ConfigSync` and
`HiddenFileSync` add-on identities have been retired.
The Customisation Sync private context owns its catalogue, progress, manifest,
queue, and periodic state. The Hidden File Sync private context owns its
processed-state caches, reconciliation and conflict queues, activity counts,
and periodic state. Each receives live settings and database projections,
focused storage, path, and exact-revision capabilities, and explicit host
effects rather than `LiveSyncCore`.
The Customisation Sync private context coordinates its snapshot operations,
scan queues, periodic work, and focused transient-state owners. The Hidden File
Sync private context coordinates reconciliation, periodic work, and the
lifetimes of focused processed-state, change-processing, and
conflict-resolution owners. Each receives live settings and database
projections, focused storage, path, and exact-revision capabilities, and
explicit host effects rather than `LiveSyncCore`.
The corresponding implemented topology is documented in
[Optional-file synchronisation architecture](../design_docs/optional_file_sync_architecture.md).
@@ -105,7 +106,7 @@ The Customisation Sync runtime will own:
operations; and
- the state required to serialise and report those operations.
The Hidden File Sync runtime will own:
The Hidden File Sync runtime and its composed focused owners will own:
- the device-local processed-state records;
- one-file storage and database transfer, including exact-revision repair;
@@ -194,15 +195,44 @@ caches, processed-state records, recent-event records, locks, semaphores,
queues, periodic processors, Notices, and event subscriptions must be created
and disposed by their feature context or a focused resource owner.
The contexts are orchestration roots, rather than containers for every mutable
detail. `HiddenFileSyncProcessedState` owns the three device-local maps, their
autosave initialisation, exact key formats, retained known mtime, reset rules,
and settlement effects on `IPathService`. Database write and extraction
operations receive this capability through one narrow `processedState` port;
they do not receive bundles of individual state callbacks or implement state
keys themselves.
`CustomisationSyncCatalogueState` separately owns the transient catalogue
rows, manifest lookup and mtime cache, their reactive stores, and catalogue
update progress. A small recent-event deduplicator owns raw-event admission
history. These Customisation Sync owners are deliberately not implementations
of a shared Hidden File Sync state abstraction: the former state is a derived,
in-memory projection, while Hidden File Sync markers are persisted operational
reconciliation state with different identity, invalidation, and deletion
rules.
`HiddenFileSyncChangeProcessor` owns storage and database change processing,
the bounded semaphore, same-path event serialisation, activity counts, and the
inherited order in which processed-state markers and transfer results settle.
This boundary keeps event concurrency and settlement directly testable without
giving the processor full scan, initialisation, notification, or host
responsibilities.
The joint composition may return several views backed by those contexts:
- a Customisation Sync catalogue and operation view for its dialogue;
- a Hidden File Sync initialisation view for settings workflows; and
- a Hidden File Sync repair view for the Hatch pane.
- a Hidden File Sync initialisation view for settings workflows;
- a Hidden File Sync repair view for the Hatch pane;
- immutable semantic handler views for registration by the joint composition;
and
- explicitly internal testing views for maintained real-Obsidian workflows.
Several views over one context do not create several owners. Views expose
stable application data and operations rather than PouchDB entries, queue
objects, mutable settings records, or the complete core.
stable application data and named operations rather than PouchDB entries,
queue objects, mutable settings records, dependency objects, or the complete
core. The testing views also avoid exposing context instances or writable
internal state; time-sensitive E2E work uses a scoped operation interceptor.
Obsidian commands, ribbon actions, dialogues, Notices, plug-in reloads, and
restart scheduling will remain in host-owned composition. The UI will receive
@@ -231,10 +261,10 @@ serviceFeature. `LiveSyncBaseCore.getAddOn()` and its constructor-name lookup
have been removed.
Production code consumes focused views and does not use the broad context
surface. Maintained real-Obsidian E2E workflows use an explicitly internal
test view exposed by the composed feature. This is a transitional test seam,
not a production service locator, and it should be narrowed as those workflows
move to public operations or commands.
surface. Maintained real-Obsidian E2E workflows use explicitly internal,
immutable test views exposed by the composed feature. These are transitional
test seams, not production service locators, and they should be narrowed as
those workflows move to public operations or commands.
The retirement was gated on:
@@ -328,11 +358,13 @@ to the non-owner.
- Replace the complete-core dependency with narrow dependencies.
The private context, path module, codec module, focused presentation view, and
resource teardown are implemented. Catalogue, enumeration, migration, scan,
and manifest state is owned per context instance. The context accepts only
narrow, live projections and explicit effects; an Obsidian adapter at the
composition edge owns dialogues, Notices, plug-in reload, restart, lifecycle,
Vault access, and compatibility scan telemetry.
resource teardown are implemented. A focused catalogue-state owner holds the
rows, manifests, manifest mtime cache, reactive stores, and update progress,
while a bounded deduplicator owns recent raw-event keys. Enumeration and scan
queues remain with the orchestrating context. The context accepts only narrow,
live projections and explicit effects; an Obsidian adapter at the composition
edge owns dialogues, Notices, plug-in reload, restart, lifecycle, Vault access,
and compatibility scan telemetry.
### Stage 5: extract the Hidden File Sync runtime — implemented
@@ -343,16 +375,26 @@ Vault access, and compatibility scan telemetry.
The private context, focused initialisation, repair, and command views,
host-owned command registration, and processor, cache, subscription, and
Notice teardown are implemented. The context owns transfer, reconciliation,
conflict, and processed-state behaviour through narrow live dependencies.
An Obsidian adapter owns JSON conflict dialogues, progress presentation,
grouped Notices, plug-in reload, restart scheduling, Vault enumeration, and
compatibility activity publication.
Notice teardown are implemented. A focused processed-state owner holds all
three persisted maps, their key and mtime rules, reset operations, and
cross-side settlement. Database write and extraction operations consume one
narrow state port. A focused change processor owns storage and database event
processing, bounded concurrency, per-path serialisation, activity publication,
and compatibility settlement order. The context retains scan, initialisation,
notification, and reconciliation orchestration. A focused conflict-resolution
owner owns pending-path admission,
the parallel classification and serial interaction queues, automatic merge,
newer-revision selection, interactive JSON application, settlement, and queue
disposal. An Obsidian adapter owns JSON conflict dialogue instances, progress
presentation, grouped Notices, plug-in reload, restart scheduling, Vault
enumeration, and compatibility activity publication.
### Stage 6: move synchronisation composition — implemented
- Register the overlapping Service handlers once through the joint
serviceFeature.
- Consume immutable semantic handler views rather than exposing registry-style
methods on either context.
- Preserve the characterised lifecycle callback order and Commonlib
aggregation semantics through focused tests.
@@ -438,11 +480,13 @@ migration without first establishing narrow dependencies.
- The composition root gains several focused views but does not gain another
runtime service locator.
- The legacy add-on identity and constructor-name lookup are removed.
- The two contexts remain sizeable because each owns one cohesive persisted
synchronisation model, but their dependency surfaces are explicit and do
not include the complete core. Further extraction should follow a concrete
behavioural boundary rather than create additional serviceFeatures for
private operations.
- The two contexts coordinate separate synchronisation workflows, but their
dependency surfaces are explicit and do not include the complete core.
Customisation Sync delegates its derived catalogue and recent-event state,
while Hidden File Sync delegates processed-state, change-processing, and
conflict lifecycles, to focused owners. Further extraction should follow a
concrete behavioural boundary rather than create additional serviceFeatures
for private operations.
## References
@@ -31,6 +31,9 @@ Obsidian composition (`main.ts`)
| +--> pure local-path and document routing policy
| |
| +--> `CustomisationSyncContext`
| | +-- `CustomisationSyncCatalogueState`
| | +-- recent-event deduplicator
| | +-- immutable service-handler and testing views
| | ^
| | +-- narrow dependencies from
| | `customisationSyncObsidianAdapter`
@@ -39,6 +42,20 @@ Obsidian composition (`main.ts`)
| ^
| +-- narrow dependencies from
| `hiddenFileSyncObsidianAdapter`
| |
| +--> `HiddenFileSyncProcessedState`
| | +-- three autosaved reconciliation maps
| | +-- key, mtime, reset, and settlement rules
| |
| +--> `HiddenFileSyncChangeProcessor`
| | +-- storage and database change processing
| | +-- per-path serialisation and activity state
| |
| +--> `HiddenFileSyncConflictResolution`
| | +-- pending paths and two-stage conflict queue
| | +-- automatic and interactive JSON resolution
| |
| +-- immutable service-handler, command, repair, and testing views
|
+--> `useCustomisationSyncUI`
| +-- catalogue and operation view
@@ -58,22 +75,37 @@ application core.
## Ownership
| Owner | Owns | Does not own |
| --- | --- | --- |
| `useOptionalFileSync` | Construction of both contexts, handler registration and removal, local-owner selection, namespace dispatch, compatibility callback order, and context disposal order. | Persisted feature state, synchronisation algorithms, commands, dialogues, or Notices. |
| `CustomisationSyncContext` | The `ix:` codec and path rules, catalogue, manifest cache, scan queues, migration progress, snapshot storage and application, and periodic scan state. | Obsidian dialogues, plug-in lifecycle APIs, ribbon actions, or handler registration. |
| `HiddenFileSyncContext` | The `i:` transfer rules, device-local processed-state caches, reconciliation, exact-revision repair, conflict queues, notification batching, activity counts, and periodic scan state. | Obsidian conflict dialogues, grouped Notices, plug-in lifecycle APIs, or handler registration. |
| Customisation Sync Obsidian adapter | Obsidian conflict selection, Notice presentation, plug-in reload, restart requests, Vault enumeration, progress telemetry, and platform-derived fallback device names. | Catalogue state, routing, or persisted document operations. |
| Hidden File Sync Obsidian adapter | JSON conflict dialogue lifetime, progress presentation, grouped change Notices, plug-in reload actions, restart scheduling, Vault enumeration, and compatibility activity publication. | Transfer, reconciliation, processed-state, or conflict decisions. |
| `useCustomisationSyncUI` | Command, ribbon, dialogue, open-request subscription, and their unload teardown. | Synchronisation state or Hidden File Sync initialisation behaviour. |
| `useHiddenFileSyncCommands` | Hidden File Sync command registration, setting-change subscription, and their unload teardown. | Synchronisation state or command implementation. |
| Owner | Owns | Does not own |
| ------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | --------------------------------------------------------------------------------------------------------------------- |
| `useOptionalFileSync` | Construction of both contexts, handler registration and removal, local-owner selection, namespace dispatch, compatibility callback order, and context disposal order. | Persisted feature state, synchronisation algorithms, commands, dialogues, or Notices. |
| `CustomisationSyncContext` | The `ix:` codec and path rules, scan queues, snapshot storage and application, periodic scan state, and the lifetimes of its transient-state owners. | Catalogue mutations, recent-event history, Obsidian dialogues, ribbon actions, or handler registration. |
| `CustomisationSyncCatalogueState` | Transient catalogue rows, manifest lookup and mtime cache, reactive catalogue and manifest stores, and catalogue update progress. | Database or storage I/O, scan scheduling, routing, or persistence. |
| Customisation Sync event deduplicator | The bounded, newest-first keys used to admit raw configuration events once. | Scheduling, path ownership, catalogue state, or persisted data. |
| `HiddenFileSyncContext` | The `i:` scan and reconciliation workflow, exact-revision repair composition, notification batching, periodic scan state, and focused-owner lifetimes. | Change-event serialisation, processed-state representation, conflict queue state, Obsidian dialogues, or service handler registration. |
| `HiddenFileSyncProcessedState` | Three device-local autosaved maps, exact storage and database keys, retained known mtime, reset behaviour, and storage/database settlement effects. | File transfer, scan scheduling, conflict handling, or presentation. |
| `HiddenFileSyncChangeProcessor` | Storage and database change processing, same-path event serialisation, bounded concurrency, activity counts, and the inherited event-consumption and settlement order. | Full scans, initialisation policy, notification presentation, or conflict interaction. |
| `HiddenFileSyncConflictResolution` | Conflict admission and deduplication, pending paths, the parallel classification and serial interaction queues, automatic merge, newer-revision policy, interactive JSON resolution, and conflict settlement. | Obsidian dialogue instances, general Hidden File Sync scans, processed-state caches, or Service handler registration. |
| Customisation Sync Obsidian adapter | Obsidian conflict selection, Notice presentation, plug-in reload, restart requests, Vault enumeration, progress telemetry, and platform-derived fallback device names. | Catalogue state, routing, or persisted document operations. |
| Hidden File Sync Obsidian adapter | JSON conflict dialogue lifetime, progress presentation, grouped change Notices, plug-in reload actions, restart scheduling, Vault enumeration, and compatibility activity publication. | Transfer, reconciliation, processed-state, or conflict decisions. |
| `useCustomisationSyncUI` | Command, ribbon, dialogue, open-request subscription, and their unload teardown. | Synchronisation state or Hidden File Sync initialisation behaviour. |
| `useHiddenFileSyncCommands` | Hidden File Sync command registration, setting-change subscription, and their unload teardown. | Synchronisation state or command implementation. |
The two domain contexts remain sizeable because they each own one cohesive
persisted synchronisation model. Their private operations are not additional
serviceFeatures: they do not independently register host integration or have
separate application lifetimes. Extract a further ordinary module or focused
state owner when a concrete invariant, replacement lifecycle, or independently
testable operation justifies that boundary.
The two domain contexts coordinate one cohesive synchronisation workflow each.
Their private operations and focused owners are not additional serviceFeatures:
they do not independently register host integration or have separate
application lifetimes. `HiddenFileSyncChangeProcessor` is a focused resource
owner because its semaphore, per-path serialisation, activity counters, and
event settlement form one independently testable lifecycle.
`HiddenFileSyncConflictResolution` is another focused resource owner because
conflict admission, pending-path identity, two serialisation stages, and
disposal form a separate lifecycle.
The two state owners intentionally do not implement a common generic state
contract. Customisation Sync projects transient catalogue and presentation
state from `ix:` documents. Hidden File Sync persists operational markers used
for reconciliation, with distinct path identity, deletion, reset, and retained
mtime rules. Their common boundary is lifecycle ownership by a context, rather
than interchangeable state semantics.
## Routing and handler contracts
@@ -85,21 +117,24 @@ static policy selects Hidden File Sync.
The maintained local ownership is:
| Path mode | Owner |
| --- | --- |
| Customisation path in Selective or Flagged Selective mode | Customisation Sync |
| Customisation path in Automatic mode, with Hidden File Sync enabled | Hidden File Sync |
| Customisation path in Ignore mode | Neither context |
| Other eligible hidden path | Hidden File Sync |
| Disabled, excluded, ignored, or ordinary Vault path | Neither context |
| Path mode | Owner |
| ------------------------------------------------------------------- | ------------------ |
| Customisation path in Selective or Flagged Selective mode | Customisation Sync |
| Customisation path in Automatic mode, with Hidden File Sync enabled | Hidden File Sync |
| Customisation path in Ignore mode | Neither context |
| Other eligible hidden path | Hidden File Sync |
| Disabled, excluded, ignored, or ordinary Vault path | Neither context |
Local ownership is distinct from persisted-document recognition. The
composition dispatches `ps:` and `ix:` conflict documents to Customisation
Sync and `i:` conflict documents to Hidden File Sync. Existing documents remain
recognisable after a local mode changes.
Sync and sends `i:` conflict documents through the Hidden File Sync semantic
handler view. Existing documents remain recognisable after a local mode
changes.
The composition adapts the contexts to the existing Commonlib handler
contracts:
Each context exposes an immutable semantic handler view. The composition
registers these operations and adapts them to the existing Commonlib handler
contracts; registry aggregation names and binding concerns do not leak back
into either context:
- raw optional-file events are offered to exactly one selected owner;
- a selected handler which skips or fails does not fall through to the other
@@ -148,23 +183,35 @@ owners:
operations to the Hatch pane.
Production consumers receive these views directly. `OptionalFileSyncFeature`
also exposes an explicitly internal `testing` view for maintained real-Obsidian
contract tests. That test seam is not a production service locator and should
not be used by application features.
also exposes immutable, explicitly internal testing views for maintained
real-Obsidian contract tests. They provide named operations, including a scoped
rebuild interceptor, without exposing context instances, dependency objects,
queues, or writable stores. These test seams are not production service
locators and should not be used by application features.
## Lifecycle and disposal
The composition is created after the Service Hub and required ServiceModules
exist, and before lifecycle-driven feature work begins. Each context creates
its own queues, caches, semaphores, activity state, and periodic processor.
its own periodic processor and focused resource owners.
`CustomisationSyncContext` creates one catalogue-state owner and one
recent-event deduplicator. `HiddenFileSyncContext` creates one processed-state
owner before composing database write and extraction operations around its
narrow port, then creates one change processor and one conflict-resolution
owner. The change processor owns its semaphore and activity state.
Full conflict scans admit discovered paths into the same queue without
suspending it, so ordinary database conflict notifications continue during a
slow scan. After enumeration, the operation waits for both classification and
interaction stages to drain.
On application unload, `useOptionalFileSync` first removes every Service
handler registration. It then disposes Customisation Sync followed by Hidden
File Sync, preserving the former compatibility order. Disposal disables
periodic admission, terminates queues, clears transient caches and pending
sets, cancels scheduled notification work, resets compatibility telemetry, and
hides owned Notices. The two presentation serviceFeatures independently remove
their commands, event subscriptions, ribbon state, and dialogue instances.
periodic admission, disposes the conflict-resolution owner, terminates queues,
clears transient caches and pending sets, cancels scheduled notification work,
resets compatibility telemetry, and hides owned Notices. The two presentation
serviceFeatures independently remove their commands, event subscriptions,
ribbon state, and dialogue instances.
## Persisted compatibility
@@ -184,9 +231,11 @@ appropriate, a migration decision.
## Verification boundaries
Focused unit tests cover routing, handler aggregation, context state
isolation, teardown, initial cache selection, exact-revision repair, conflict
dialogue adaptation, grouped Notices, and compatibility activity publication.
Focused unit tests cover routing, semantic handler views, context state
isolation, teardown, initial cache selection, exact-revision repair, change
event serialisation and settlement, conflict queue admission, revision
selection, automatic and interactive merge effect ordering, conflict dialogue
adaptation, grouped Notices, and compatibility activity publication.
The boundary test prevents either domain context from regaining core or
Obsidian dependencies.
@@ -0,0 +1,152 @@
import type { PluginManifest } from "@/deps.ts";
import type { FilePathWithPrefix } from "@vrtmrz/livesync-commonlib/compat/common/types";
import { isObjectDifferent } from "@vrtmrz/livesync-commonlib/compat/common/utils";
import { writable } from "svelte/store";
import { PluginDataExDisplayV2 } from "./customisationSyncModel.ts";
import type { IPluginDataExDisplay } from "./customisationSyncView.ts";
/**
* Owns the transient catalogue projection used by Customisation Sync.
*
* The database and storage operations remain in the context. This owner only
* coordinates the in-memory rows, their reactive publications, manifest
* lookup, and the update counter which is derived from those operations.
*/
export class CustomisationSyncCatalogueState {
private catalogueRows: IPluginDataExDisplay[] = [];
private readonly manifestByKey = new Map<string, PluginManifest>();
private readonly loadedManifestMTimeByKey = new Map<string, number>();
private activeUpdateCount = 0;
readonly catalogue = writable<IPluginDataExDisplay[]>([]);
readonly migrationProgress = writable(0);
readonly manifests = writable(this.manifestByKey);
/** The current manifest lookup passed to V2 display rows. */
get manifestLookup(): ReadonlyMap<string, PluginManifest> {
return this.manifestByKey;
}
/** The current loaded-manifest cache, exposed read-only for diagnostics. */
get loadedManifestMTime(): ReadonlyMap<string, number> {
return this.loadedManifestMTimeByKey;
}
/** Returns the authoritative row for a V1 document path, when present. */
findPlugin(documentPath: FilePathWithPrefix | string): IPluginDataExDisplay | undefined {
return this.catalogueRows.find((entry) => entry.documentPath == documentPath);
}
/** Returns every row matching a document path, preserving legacy duplicates. */
findPlugins(documentPath: FilePathWithPrefix | string): readonly IPluginDataExDisplay[] {
return this.catalogueRows.filter((entry) => entry.documentPath == documentPath);
}
/** Replaces a V1 row and publishes it immediately, preserving legacy order. */
replacePlugin(plugin: IPluginDataExDisplay): void {
const newList = this.catalogueRows.filter((entry) => entry.documentPath != plugin.documentPath);
newList.push(plugin);
this.catalogueRows = newList;
this.catalogue.set(newList);
}
/**
* Replaces a V2 row without publishing it. V2 callers publish through the
* existing delayed task after a cohesive row update has completed.
*/
private replaceV2Plugin(plugin: PluginDataExDisplayV2): void {
const newList = this.catalogueRows.filter((entry) => entry.documentPath != plugin.documentPath);
newList.push(plugin);
this.catalogueRows = newList;
}
/** Applies one loaded or removed V2 file and replaces its catalogue row. */
async updateV2Plugin(
plugin: PluginDataExDisplayV2,
file: Parameters<PluginDataExDisplayV2["setFile"]>[0] | false,
missingFilePath: string
): Promise<void> {
if (file) {
await plugin.setFile(file);
} else {
plugin.deleteFile(missingFilePath);
}
this.replaceV2Plugin(plugin);
}
/** Publishes the current V2 row set when the legacy delayed task fires. */
publishCatalogue(): void {
this.catalogue.set(this.catalogueRows);
}
/**
* Clears rows and loaded manifest mtimes for an explicit reload. The
* manifest map intentionally survives this narrower refresh.
*/
clearForReload(): void {
this.catalogueRows = [];
this.loadedManifestMTimeByKey.clear();
this.catalogue.set(this.catalogueRows);
}
/** Clears only the catalogue rows for a disabled refresh. */
clearForDisabledRefresh(): void {
this.catalogueRows = [];
this.catalogue.set(this.catalogueRows);
}
/** Begins one catalogue update and publishes its progress count. */
beginUpdate(): void {
this.activeUpdateCount++;
this.migrationProgress.set(this.activeUpdateCount);
}
/** Ends one catalogue update and publishes its progress count. */
endUpdate(): void {
this.activeUpdateCount--;
this.migrationProgress.set(this.activeUpdateCount);
}
/**
* Applies a manifest according to the inherited cache rules. A manifest is
* parsed only when no manifest has previously been accepted for the key;
* failed parses still record their mtime, while a later mtime never
* replaces a successfully parsed first manifest.
*/
processManifest(
confKey: string,
mtime: number,
parseManifest: () => PluginManifest,
onParseError: (error: unknown) => void = () => undefined
): void {
let publishCatalogue = false;
if (this.loadedManifestMTimeByKey.get(confKey) != mtime && this.manifestByKey.get(confKey) == undefined) {
try {
this.setManifest(confKey, parseManifest());
this.applyLoadedManifest(confKey);
publishCatalogue = true;
} catch (error) {
onParseError(error);
}
this.loadedManifestMTimeByKey.set(confKey, mtime);
} else {
this.applyLoadedManifest(confKey);
publishCatalogue = true;
}
if (publishCatalogue) this.catalogue.set(this.catalogueRows);
}
private setManifest(key: string, manifest: PluginManifest): void {
const old = this.manifestByKey.get(key);
if (old && !isObjectDifferent(manifest, old)) return;
this.manifestByKey.set(key, manifest);
this.manifests.set(this.manifestByKey);
}
private applyLoadedManifest(confKey: string): void {
this.catalogueRows
.filter((entry) => entry instanceof PluginDataExDisplayV2 && entry.confKey == confKey)
.forEach((entry) => (entry as PluginDataExDisplayV2).applyLoadedManifest());
}
}
@@ -0,0 +1,127 @@
import { get } from "svelte/store";
import { describe, expect, it, vi } from "vitest";
import type { PluginManifest } from "@/deps.ts";
import type { FilePathWithPrefix } from "@vrtmrz/livesync-commonlib/compat/common/types";
import { CustomisationSyncCatalogueState } from "./customisationSyncCatalogueState.ts";
import { PluginDataExDisplayV2 } from "./customisationSyncModel.ts";
import type { IPluginDataExDisplay, LoadedEntryPluginDataExFile } from "./customisationSyncView.ts";
function display(documentPath = "ix:device-a/PLUGIN_MAIN/example.md"): IPluginDataExDisplay {
return {
documentPath: documentPath as FilePathWithPrefix,
category: "PLUGIN_MAIN",
name: "example",
term: "device-a",
files: [],
mtime: 1,
};
}
function file(filename: string, mtime: number): LoadedEntryPluginDataExFile {
return {
path: `ix:device-a/PLUGIN_MAIN/example%${filename}` as FilePathWithPrefix,
filename,
mtime,
data: [filename],
size: filename.length,
} as LoadedEntryPluginDataExFile;
}
describe("Customisation Sync catalogue state", () => {
it("publishes V1 replacement and keeps V2 replacement delayed", async () => {
const state = new CustomisationSyncCatalogueState();
const setCatalogue = vi.spyOn(state.catalogue, "set");
const row = display();
state.replacePlugin(row);
expect(get(state.catalogue)).toEqual([row]);
expect(setCatalogue).toHaveBeenCalledOnce();
const v2 = new PluginDataExDisplayV2(
{
...display(),
files: [file("main.js", 1)],
},
state.manifestLookup
);
await state.updateV2Plugin(v2, file("main.js", 2), "main.js");
expect(get(state.catalogue)).toEqual([row]);
expect(state.findPlugin(row.documentPath)).toBe(v2);
state.publishCatalogue();
expect(get(state.catalogue)).toEqual([v2]);
});
it("retains the first parsed manifest and records failed mtimes", () => {
const state = new CustomisationSyncCatalogueState();
const first = { name: "First", version: "1.0.0" } as PluginManifest;
const parseManifest = vi.fn(() => first);
state.processManifest("device-a/plugins/example", 20, parseManifest);
state.processManifest(
"device-a/plugins/example",
30,
() => ({ name: "Second", version: "2.0.0" }) as PluginManifest
);
expect(state.manifestLookup.get("device-a/plugins/example")).toBe(first);
expect(state.loadedManifestMTime.get("device-a/plugins/example")).toBe(20);
expect(parseManifest).toHaveBeenCalledOnce();
const failedState = new CustomisationSyncCatalogueState();
const onParseError = vi.fn();
const failure = new SyntaxError("invalid");
failedState.processManifest(
"device-a/plugins/failure",
40,
() => {
throw failure;
},
onParseError
);
failedState.processManifest("device-a/plugins/failure", 40, () => first, onParseError);
expect(onParseError).toHaveBeenCalledWith(failure);
expect(failedState.loadedManifestMTime.get("device-a/plugins/failure")).toBe(40);
expect(failedState.manifestLookup.has("device-a/plugins/failure")).toBe(false);
});
it("clears rows and loaded mtimes on reload while retaining manifest lookup", () => {
const state = new CustomisationSyncCatalogueState();
const key = "device-a/plugins/example";
state.processManifest(key, 20, () => ({ name: "Example" }) as PluginManifest);
state.replacePlugin(display());
state.clearForReload();
expect(get(state.catalogue)).toEqual([]);
expect(state.loadedManifestMTime.size).toBe(0);
expect(state.manifestLookup.get(key)).toEqual({ name: "Example" });
expect(get(state.catalogue)).toEqual([]);
});
it("keeps manifest caches through the narrower disabled refresh", () => {
const state = new CustomisationSyncCatalogueState();
const key = "device-a/plugins/example";
state.processManifest(key, 20, () => ({ name: "Example" }) as PluginManifest);
state.replacePlugin(display());
state.clearForDisabledRefresh();
expect(get(state.catalogue)).toEqual([]);
expect(state.loadedManifestMTime.get(key)).toBe(20);
expect(state.manifestLookup.has(key)).toBe(true);
});
it("tracks V2 updates through migration progress", () => {
const state = new CustomisationSyncCatalogueState();
state.beginUpdate();
state.beginUpdate();
expect(get(state.migrationProgress)).toBe(2);
state.endUpdate();
state.endUpdate();
expect(get(state.migrationProgress)).toBe(0);
});
});
@@ -26,29 +26,42 @@ vi.mock("@/common/translation", () => ({
vi.mock("@/common/obsidianCommunityPlugins.ts", () => ({
getObsidianCommunityPluginManager: vi.fn(),
}));
import { cancelTask } from "@/common/utils.ts";
import { cancelTask, scheduleTask } from "@/common/utils.ts";
import { CustomisationSyncContext } from "./customisationSyncContext";
import { createCustomisationSyncTestDependencies } from "./customisationSyncContext.unit.fixture.ts";
describe("CustomisationSyncContext commands", () => {
it("keeps the legacy dialogue methods as delegates to the host-owned UI", () => {
it("opens the host-owned dialogue from a scheduled configuration Notice", async () => {
const control = {
open: vi.fn(),
close: vi.fn(),
isOpen: vi.fn(),
isOpen: vi.fn(() => false),
};
const showConfigurationNotice = vi.fn();
const updatePluginList = vi.fn(async () => undefined);
const configSync = Object.create(CustomisationSyncContext.prototype) as CustomisationSyncContext;
Object.assign(configSync, {
dependencies: createCustomisationSyncTestDependencies({
getUIControl: () => control,
getSettings: () => ({ usePluginSync: true, notifyPluginOrSettingUpdated: true }) as never,
showConfigurationNotice,
}),
updatePluginList,
});
configSync.showPluginSyncModal();
configSync.hidePluginSyncModal();
await configSync.serviceHandlers.processVirtualDocument({
_id: "ix:example",
path: "ix:example",
} as never);
const scheduledNotice = vi.mocked(scheduleTask).mock.calls[0]?.[2] as (() => void) | undefined;
expect(scheduledNotice).toBeTypeOf("function");
scheduledNotice?.();
const openDialogue = showConfigurationNotice.mock.calls[0]?.[0] as (() => void) | undefined;
expect(openDialogue).toBeTypeOf("function");
openDialogue?.();
expect(control.open).toHaveBeenCalledOnce();
expect(control.close).toHaveBeenCalledOnce();
expect(updatePluginList).toHaveBeenCalledWith(false, "ix:example");
});
it("releases every owned processor and reactive subscription", () => {
@@ -85,4 +98,25 @@ describe("CustomisationSyncContext commands", () => {
expect(setEnumerationActive).toHaveBeenCalledWith(false);
expect(publishScanCount).toHaveBeenCalledWith(0);
});
it("characterises the inherited setting-realisation gates pending separate review", async () => {
const isReady = vi.fn(() => false);
const isSuspended = vi.fn(() => false);
const periodicPluginSweepProcessor = { disable: vi.fn(), enable: vi.fn() };
const scanAllConfigFiles = vi.fn(async () => undefined);
const configSync = Object.create(CustomisationSyncContext.prototype) as CustomisationSyncContext;
Object.assign(configSync, {
dependencies: createCustomisationSyncTestDependencies({ isReady, isSuspended }),
periodicPluginSweepProcessor,
scanAllConfigFiles,
});
await expect(configSync.serviceHandlers.onRealiseSetting()).resolves.toBe(true);
expect(periodicPluginSweepProcessor.disable).toHaveBeenCalledOnce();
expect(isReady).not.toHaveBeenCalled();
expect(isSuspended).toHaveBeenCalledOnce();
expect(scanAllConfigFiles).not.toHaveBeenCalled();
expect(periodicPluginSweepProcessor.enable).not.toHaveBeenCalled();
});
});
@@ -1,5 +1,4 @@
import { describe, expect, it, vi } from "vitest";
import type { FilePathWithPrefix } from "@vrtmrz/livesync-commonlib/compat/common/types";
vi.mock("@/deps.ts", () => ({
diff_match_patch: class DiffMatchPatch {},
@@ -65,7 +64,7 @@ describe("compatibility: Customisation Sync paths", () => {
[".obsidian/workspace", ""],
["notes/example.json", "CONFIG"],
])("classifies %s as %s", (path, expected) => {
expect(createConfigSync().getFileCategory(path)).toBe(expected);
expect(createConfigSync().testing.getFileCategory(path)).toBe(expected);
});
it("keeps other plug-in files outside V1 and disabled plug-in-extra synchronisation", () => {
@@ -73,17 +72,17 @@ describe("compatibility: Customisation Sync paths", () => {
const withoutPluginEtc = createConfigSync({ usePluginEtc: false });
const path = ".obsidian/plugins/example/other.json";
expect(v1.getFileCategory(path)).toBe("");
expect(withoutPluginEtc.getFileCategory(path)).toBe("");
expect(v1.testing.getFileCategory(path)).toBe("");
expect(withoutPluginEtc.testing.getFileCategory(path)).toBe("");
});
it("recognises only classified files below the Obsidian configuration directory", () => {
const configSync = createConfigSync();
expect(configSync.isTargetPath(".obsidian/app.json")).toBe(true);
expect(configSync.isTargetPath(".obsidian/plugins/example/main.js")).toBe(true);
expect(configSync.isTargetPath(".obsidian/workspace")).toBe(false);
expect(configSync.isTargetPath("notes/example.json")).toBe(false);
expect(configSync.testing.isTargetPath(".obsidian/app.json")).toBe(true);
expect(configSync.testing.isTargetPath(".obsidian/plugins/example/main.js")).toBe(true);
expect(configSync.testing.isTargetPath(".obsidian/workspace")).toBe(false);
expect(configSync.testing.isTargetPath("notes/example.json")).toBe(false);
});
it.each([
@@ -92,7 +91,7 @@ describe("compatibility: Customisation Sync paths", () => {
[".obsidian/plugins/example/main.js", "ix:device-a/PLUGIN_MAIN/example.md"],
[".obsidian/plugins/example/data.json", "ix:device-a/PLUGIN_DATA/example.md"],
])("creates the V1 document path for %s", (path, expected) => {
expect(createConfigSync().filenameToUnifiedKey(path)).toBe(expected);
expect(createConfigSync().testing.filenameToUnifiedKey(path)).toBe(expected);
});
it.each([
@@ -101,30 +100,18 @@ describe("compatibility: Customisation Sync paths", () => {
[".obsidian/plugins/example/main.js", "ix:device-a/PLUGIN_MAIN/example%main.js"],
[".obsidian/plugins/example/data.json", "ix:device-a/PLUGIN_DATA/example%data.json"],
])("creates the V2 document path for %s", (path, expected) => {
expect(createConfigSync().filenameWithUnifiedKey(path)).toBe(expected);
expect(createConfigSync().testing.filenameWithUnifiedKey(path)).toBe(expected);
});
it("uses an explicit device name when supplied", () => {
const configSync = createConfigSync();
expect(configSync.filenameToUnifiedKey(".obsidian/app.json", "device-b")).toBe(
expect(configSync.testing.filenameToUnifiedKey(".obsidian/app.json", "device-b")).toBe(
"ix:device-b/CONFIG/app.json.md"
);
expect(configSync.filenameWithUnifiedKey(".obsidian/app.json", "device-b")).toBe(
expect(configSync.testing.filenameWithUnifiedKey(".obsidian/app.json", "device-b")).toBe(
"ix:device-b/CONFIG/app.json%app.json"
);
expect(configSync.unifiedKeyPrefixOfTerminal("device-b")).toBe("ix:device-b/");
});
it("parses a V2 document path and derives its V1 compatibility path", () => {
expect(
createConfigSync().parseUnifiedPath("ix:device-a/PLUGIN_MAIN/example%main.js" as FilePathWithPrefix)
).toEqual({
device: "device-a",
category: "PLUGIN_MAIN",
key: "example",
filename: "main.js",
pathV1: "ix:device-a/PLUGIN_MAIN/example.md",
});
expect(configSync.testing.unifiedKeyPrefixOfTerminal("device-b")).toBe("ix:device-b/");
});
});
@@ -31,6 +31,7 @@ vi.mock("@/common/obsidianCommunityPlugins.ts", () => ({
import { scheduleTask } from "@/common/utils.ts";
import { CustomisationSyncContext } from "./customisationSyncContext.ts";
import { createCustomisationSyncTestDependencies } from "./customisationSyncContext.unit.fixture.ts";
import { CustomisationSyncRecentEventDeduplicator } from "./customisationSyncRecentEventDeduplicator.ts";
const PATH = ".obsidian/plugins/example/data.json" as FilePath;
@@ -43,9 +44,7 @@ function createConfigSync(options: { ready?: boolean; suspended?: boolean; enabl
};
const ownsLocalFile = vi.fn(() => options.owned ?? true);
const statHidden = vi.fn(async () => ({ type: "file", mtime: 1 }));
const recentProcessedInternalFiles = Object.assign([] as string[], {
contains: vi.fn(() => false),
});
const recentProcessedInternalFiles = new CustomisationSyncRecentEventDeduplicator();
const configSync = Object.create(CustomisationSyncContext.prototype) as CustomisationSyncContext;
Object.assign(configSync, {
dependencies: createCustomisationSyncTestDependencies({
@@ -71,7 +70,7 @@ describe("Customisation Sync raw-event admission", () => {
it("schedules a recognised path granted by the composition owner", async () => {
const { configSync, ownsLocalFile } = createConfigSync();
await expect(configSync._anyProcessOptionalFileEvent(PATH)).resolves.toBe(true);
await expect(configSync.serviceHandlers.processOptionalFileEvent(PATH)).resolves.toBe(true);
expect(ownsLocalFile).toHaveBeenCalledWith(PATH);
expect(scheduleTask).toHaveBeenCalledOnce();
});
@@ -79,7 +78,7 @@ describe("Customisation Sync raw-event admission", () => {
it("rejects an event while the host is not ready", async () => {
const { configSync, statHidden } = createConfigSync({ ready: false });
await expect(configSync._anyProcessOptionalFileEvent(PATH)).resolves.toBe(false);
await expect(configSync.serviceHandlers.processOptionalFileEvent(PATH)).resolves.toBe(false);
expect(statHidden).not.toHaveBeenCalled();
expect(scheduleTask).not.toHaveBeenCalled();
});
@@ -87,7 +86,9 @@ describe("Customisation Sync raw-event admission", () => {
it("rejects a path outside the recognised Customisation Sync categories", async () => {
const { configSync, ownsLocalFile } = createConfigSync();
await expect(configSync._anyProcessOptionalFileEvent(".obsidian/workspace" as FilePath)).resolves.toBe(false);
await expect(configSync.serviceHandlers.processOptionalFileEvent(".obsidian/workspace" as FilePath)).resolves.toBe(
false
);
expect(ownsLocalFile).not.toHaveBeenCalled();
expect(scheduleTask).not.toHaveBeenCalled();
});
@@ -95,7 +96,7 @@ describe("Customisation Sync raw-event admission", () => {
it("rejects a recognised path assigned to another owner", async () => {
const { configSync, statHidden } = createConfigSync({ owned: false });
await expect(configSync._anyProcessOptionalFileEvent(PATH)).resolves.toBe(false);
await expect(configSync.serviceHandlers.processOptionalFileEvent(PATH)).resolves.toBe(false);
expect(statHidden).not.toHaveBeenCalled();
expect(scheduleTask).not.toHaveBeenCalled();
});
@@ -106,7 +107,7 @@ describe("Customisation Sync raw-event admission", () => {
] as const)("rejects an event while %s", async (_label, options) => {
const { configSync } = createConfigSync(options);
await expect(configSync._anyProcessOptionalFileEvent(PATH)).resolves.toBe(false);
await expect(configSync.serviceHandlers.processOptionalFileEvent(PATH)).resolves.toBe(false);
expect(scheduleTask).not.toHaveBeenCalled();
});
});
@@ -40,4 +40,46 @@ describe("CustomisationSyncContext state ownership", () => {
expect(first.manifests).not.toBe(second.manifests);
expect(get(first.manifests)).not.toBe(get(second.manifests));
});
it("exposes frozen semantic service and testing views without writable state", () => {
const context = new CustomisationSyncContext(createCustomisationSyncTestDependencies());
expect(Object.isFrozen(context.serviceHandlers)).toBe(true);
expect(Object.keys(context.serviceHandlers).sort()).toEqual(
[
"enableOptionalFeature",
"onBeforeReplicate",
"onDatabaseInitialised",
"onRealiseSetting",
"onResuming",
"processOptionalFileEvent",
"processVirtualDocument",
"suspendExtraSync",
].sort()
);
expect(Object.isFrozen(context.testing)).toBe(true);
expect(Object.keys(context.testing).sort()).toEqual(
[
"applyDataV2",
"configDir",
"createPluginDataExFileV2",
"createPluginDataFromV2",
"deleteConfigOnDatabase",
"filenameToUnifiedKey",
"filenameWithUnifiedKey",
"getFileCategory",
"isTargetPath",
"scanAllConfigFiles",
"scanInternalFiles",
"storeCustomizationFiles",
"unifiedKeyPrefixOfTerminal",
].sort()
);
expect("catalogue" in context.testing).toBe(false);
expect("enumerationActive" in context.testing).toBe(false);
expect("manifests" in context.testing).toBe(false);
context.dispose();
});
});
@@ -1,6 +1,6 @@
import { writable } from "svelte/store";
import type PouchDB from "pouchdb-core";
import { type PluginManifest, parseYaml, normalizePath, type ListedFiles, diff_match_patch } from "@/deps.ts";
import { type PluginManifest, parseYaml, normalizePath, diff_match_patch } from "@/deps.ts";
import type {
EntryDoc,
@@ -25,25 +25,18 @@ import {
import { ICXHeader, PERIODIC_PLUGIN_SWEEP } from "@/common/types.ts";
import {
createBlob,
createSavingEntryFromLoadedEntry,
createTextBlob,
delay,
fireAndForget,
getDocData,
getDocDataAsArray,
isDocContentSame,
isLoadedEntry,
isObjectDifferent,
} from "@vrtmrz/livesync-commonlib/compat/common/utils";
import { digestHash } from "@vrtmrz/livesync-commonlib/compat/string_and_binary/hash";
import {
arrayBufferToBase64,
decodeBinary,
readString,
} from "@vrtmrz/livesync-commonlib/compat/string_and_binary/convert";
import { arrayBufferToBase64, decodeBinary } from "@vrtmrz/livesync-commonlib/compat/string_and_binary/convert";
import { serialized, shareRunningResult } from "octagonal-wheels/concurrency/lock";
import { stripAllPrefixes } from "@vrtmrz/livesync-commonlib/compat/string_and_binary/path";
import { cancelTask, EVEN, isCustomisationSyncMetadata, isPluginMetadata, scheduleTask } from "@/common/utils.ts";
import { cancelTask, EVEN, scheduleTask } from "@/common/utils.ts";
import { QueueProcessor } from "octagonal-wheels/concurrency/processor";
import { reactiveSource, type ReactiveSource } from "octagonal-wheels/dataobject/reactive";
import { base64ToArrayBuffer, base64ToString } from "octagonal-wheels/binary/base64";
@@ -59,13 +52,15 @@ import {
isCustomisationSyncTargetPath,
parseCustomisationSyncV2DocumentPath,
} from "./customisationSyncPaths.ts";
import { createCustomisationSyncCodec, type PluginDataEx, type PluginDataExFile } from "./customisationSyncCodec.ts";
import { createCustomisationSyncCodec, type PluginDataEx } from "./customisationSyncCodec.ts";
import type {
CustomisationSyncDialogView,
CustomisationSyncUIControl,
CustomisationSyncFileCategory,
CustomisationSyncServiceHandlers,
CustomisationSyncTestingView,
IPluginDataExDisplay,
LoadedEntryPluginDataExFile,
PluginDataExDisplay,
} from "./customisationSyncView.ts";
import {
REPLICATION_PROGRESS_PRESENTATIONS,
@@ -75,9 +70,29 @@ import type { LiveSyncLocalDB } from "@vrtmrz/livesync-commonlib/compat/pouchdb/
import type { StorageAccess } from "@vrtmrz/livesync-commonlib/compat/interfaces/StorageAccess";
import type { IPathService, IReplicationService } from "@vrtmrz/livesync-commonlib/compat/services/base/IService";
import type { LogFunction } from "@vrtmrz/livesync-commonlib/compat/services/lib/logUtils";
import {
collectOptionalFileSyncFiles,
type OptionalFileSyncFileTreeDependencies,
} from "@/features/optionalFileSyncFileTree.ts";
import { PluginDataExDisplayV2 } from "./customisationSyncModel.ts";
import {
decodeCustomisationSyncV2File,
loadCustomisationDisplayData,
loadCustomisationV2Entry,
readCustomisationFile,
} from "./customisationSyncReadOperations.ts";
import { CustomisationSyncCatalogueState } from "./customisationSyncCatalogueState.ts";
import { CustomisationSyncRecentEventDeduplicator } from "./customisationSyncRecentEventDeduplicator.ts";
export type { PluginDataEx, PluginDataExFile } from "./customisationSyncCodec.ts";
export type { IPluginDataExDisplay, PluginDataExDisplay } from "./customisationSyncView.ts";
export type {
CustomisationSyncFileCategory,
CustomisationSyncServiceHandlers,
CustomisationSyncTestingView,
IPluginDataExDisplay,
PluginDataExDisplay,
} from "./customisationSyncView.ts";
export { PluginDataExDisplayV2 } from "./customisationSyncModel.ts";
const UPDATED_CONFIGURATION_NOTICE_KEY = "config-sync:updated-configuration";
@@ -87,87 +102,8 @@ const {
dummyHead: DUMMY_HEAD,
dummyEnd: DUMMY_END,
} = createCustomisationSyncCodec({ digestHash, parseYaml });
const CUSTOMISATION_SYNC_READ_CODEC = { deserialize, serialize };
function categoryToFolder(category: string, configDir: string = ""): string {
switch (category) {
case "CONFIG":
return `${configDir}/`;
case "THEME":
return `${configDir}/themes/`;
case "SNIPPET":
return `${configDir}/snippets/`;
case "PLUGIN_MAIN":
return `${configDir}/plugins/`;
case "PLUGIN_DATA":
return `${configDir}/plugins/`;
case "PLUGIN_ETC":
return `${configDir}/plugins/`;
default:
return "";
}
}
export class PluginDataExDisplayV2 {
documentPath: FilePathWithPrefix;
category: string;
term: string;
files = [] as LoadedEntryPluginDataExFile[];
name: string;
confKey: string;
constructor(
data: IPluginDataExDisplay,
private readonly manifestLookup: ReadonlyMap<string, PluginManifest>
) {
this.documentPath = `${data.documentPath}` as FilePathWithPrefix;
this.category = `${data.category}`;
this.name = `${data.name}`;
this.term = `${data.term}`;
this.files = [...(data.files as LoadedEntryPluginDataExFile[])];
this.confKey = `${categoryToFolder(this.category, this.term)}${this.name}`;
this.applyLoadedManifest();
}
async setFile(file: LoadedEntryPluginDataExFile) {
const old = this.files.find((e) => e.filename == file.filename);
if (old) {
if (old.mtime == file.mtime && (await isDocContentSame(old.data, file.data))) return;
this.files = this.files.filter((e) => e.filename != file.filename);
}
this.files.push(file);
if (file.filename == "manifest.json") {
this.applyLoadedManifest();
}
}
deleteFile(filename: string) {
this.files = this.files.filter((e) => e.filename != filename);
}
_displayName: string | undefined;
_version: string | undefined;
applyLoadedManifest() {
const manifest = this.manifestLookup.get(this.confKey);
if (manifest) {
this._displayName = manifest.name;
if (this.category == "PLUGIN_MAIN" || this.category == "THEME") {
this._version = manifest?.version;
}
}
}
get displayName(): string {
// if (this._displayNameBuffer !== symbolUnInitialised) return this._displayNameBuffer;
// return this._bufferManifest().displayName;
return this._displayName || this.name;
}
get version(): string | undefined {
return this._version;
}
get mtime(): number {
return ~~this.files.reduce((a, b) => a + b.mtime, 0) / this.files.length;
}
}
type CustomisationSyncSettings = Pick<
ObsidianLiveSyncSettings,
| "usePluginSync"
@@ -195,7 +131,7 @@ export type CustomisationSyncPeriodicProcessor = {
disable(): void;
};
export type CustomisationSyncContextDependencies = {
export type CustomisationSyncContextDependencies = OptionalFileSyncFileTreeDependencies & {
getSettings(): CustomisationSyncSettings;
getLocalDatabase(): CustomisationSyncDatabase;
storageAccess: CustomisationSyncStorage;
@@ -212,7 +148,6 @@ export type CustomisationSyncContextDependencies = {
isSuspended(): boolean;
askRestart(): void;
createPeriodicProcessor(process: () => Promise<unknown>): CustomisationSyncPeriodicProcessor;
listFiles(path: string): Promise<ListedFiles>;
resolveJsonConflict(
path: FilePath,
files: [LoadedEntryPluginDataExFile, LoadedEntryPluginDataExFile],
@@ -232,19 +167,22 @@ export type CustomisationSyncContextDependencies = {
export class CustomisationSyncContext implements CustomisationSyncDialogView {
private readonly dependencies: CustomisationSyncContextDependencies;
private readonly catalogueState = new CustomisationSyncCatalogueState();
private readonly recentProcessedInternalFiles = new CustomisationSyncRecentEventDeduplicator();
private serviceHandlersView: CustomisationSyncServiceHandlers | undefined;
private testingView: CustomisationSyncTestingView | undefined;
private readonly scanProgress = reactiveSource(0);
private readonly pluginScanningChanged: Parameters<ReactiveSource<number>["onChanged"]>[0] = (event) => {
this.enumerationActive.set(event.value != 0);
this.dependencies.publishScanCount(event.value);
};
readonly catalogue = writable<IPluginDataExDisplay[]>([]);
readonly enumerationActive = writable(false);
readonly migrationProgress = writable(0);
private readonly pluginManifests = new Map<string, PluginManifest>();
readonly manifests = writable(this.pluginManifests);
readonly catalogue = this.catalogueState.catalogue;
readonly migrationProgress = this.catalogueState.migrationProgress;
readonly manifests = this.catalogueState.manifests;
readonly periodicPluginSweepProcessor: CustomisationSyncPeriodicProcessor;
private readonly periodicPluginSweepProcessor: CustomisationSyncPeriodicProcessor;
constructor(dependencies: CustomisationSyncContextDependencies) {
this.dependencies = dependencies;
@@ -253,7 +191,62 @@ export class CustomisationSyncContext implements CustomisationSyncDialogView {
);
this.scanProgress.onChanged(this.pluginScanningChanged);
}
get configDir() {
/**
* Semantic callbacks for registration by the optional-file composition
* feature. The returned object is immutable, and each callback retains its
* context without requiring callers to bind a concrete implementation.
*/
get serviceHandlers(): CustomisationSyncServiceHandlers {
if (!this.serviceHandlersView) {
this.serviceHandlersView = Object.freeze({
processOptionalFileEvent: (path: FilePath) => this.processOptionalFileEvent(path),
processVirtualDocument: (docs: PouchDB.Core.ExistingDocument<EntryDoc>) =>
this.processVirtualDocument(docs),
onRealiseSetting: () => this.realiseSettingSyncMode(),
onResuming: () => this.onResumeProcess(),
onBeforeReplicate: (showMessage: boolean) => this.beforeReplicate(showMessage),
onDatabaseInitialised: (showNotice: boolean) => this.onDatabaseInitialised(showNotice),
suspendExtraSync: () => this.suspendExtraSync(),
enableOptionalFeature: (mode: OptionalSyncFeatureMode) => this.enableOptionalFeature(mode),
});
}
return this.serviceHandlersView;
}
/**
* Narrow internal surface used by maintained real-Obsidian contract tests.
* It intentionally omits the context, queues, and writable stores.
*/
get testing(): CustomisationSyncTestingView {
if (!this.testingView) {
this.testingView = Object.freeze({
configDir: this.configDir,
scanInternalFiles: async () => await this.scanInternalFiles(),
scanAllConfigFiles: async (showMessage: boolean) => await this.scanAllConfigFiles(showMessage),
getFileCategory: (filePath: string) => this.getFileCategory(filePath),
isTargetPath: (filePath: string) => this.isTargetPath(filePath),
filenameToUnifiedKey: (path: string, termOverride?: string) =>
this.filenameToUnifiedKey(path, termOverride),
filenameWithUnifiedKey: (path: string, termOverride?: string) =>
this.filenameWithUnifiedKey(path, termOverride),
unifiedKeyPrefixOfTerminal: (termOverride?: string) =>
this.unifiedKeyPrefixOfTerminal(termOverride),
storeCustomizationFiles: async (path: FilePath, termOverride?: string) =>
await this.storeCustomizationFiles(path, termOverride),
deleteConfigOnDatabase: async (path: FilePathWithPrefix, forceWrite?: boolean) =>
await this.deleteConfigOnDatabase(path, forceWrite),
createPluginDataFromV2: (path: FilePathWithPrefix) => this.createPluginDataFromV2(path),
createPluginDataExFileV2: async (path: FilePathWithPrefix, loaded?: LoadedEntry) =>
await this.createPluginDataExFileV2(path, loaded),
applyDataV2: async (data: PluginDataExDisplayV2, content?: string) =>
await this.applyDataV2(data, content),
});
}
return this.testingView;
}
private get configDir() {
return this.dependencies.getConfigDir();
}
@@ -277,11 +270,11 @@ export class CustomisationSyncContext implements CustomisationSyncDialogView {
return this.dependencies.path.getPath(entry);
}
_isMainReady() {
private _isMainReady() {
return this.dependencies.isReady();
}
_isMainSuspended() {
private _isMainSuspended() {
return this.dependencies.isSuspended();
}
@@ -289,13 +282,13 @@ export class CustomisationSyncContext implements CustomisationSyncDialogView {
this.dependencies.log(message, level, key);
}
get useV2() {
private get useV2() {
return this.settings.usePluginSyncV2;
}
get useSyncPluginEtc() {
private get useSyncPluginEtc() {
return this.settings.usePluginEtc;
}
isThisModuleEnabled() {
private isThisModuleEnabled() {
return this.settings.usePluginSync;
}
@@ -355,11 +348,11 @@ export class CustomisationSyncContext implements CustomisationSyncDialogView {
): Promise<boolean> {
const dataACopy =
dataA instanceof PluginDataExDisplayV2
? new PluginDataExDisplayV2(dataA, this.pluginManifests)
? new PluginDataExDisplayV2(dataA, this.catalogueState.manifestLookup)
: { ...dataA };
const dataBCopy =
dataB instanceof PluginDataExDisplayV2
? new PluginDataExDisplayV2(dataB, this.pluginManifests)
? new PluginDataExDisplayV2(dataB, this.catalogueState.manifestLookup)
: { ...dataB };
dataACopy.files = dataACopy.files.filter((file) => file.filename == filename);
dataBCopy.files = dataBCopy.files.filter((file) => file.filename == filename);
@@ -372,14 +365,6 @@ export class CustomisationSyncContext implements CustomisationSyncDialogView {
await this.updatePluginList(false, this.filenameToUnifiedKey(path, deviceName));
}
pluginList: IPluginDataExDisplay[] = [];
showPluginSyncModal() {
this.dependencies.getUIControl()?.open();
}
hidePluginSyncModal() {
this.dependencies.getUIControl()?.close();
}
dispose() {
cancelTask(UPDATED_CONFIGURATION_NOTICE_KEY);
this.periodicPluginSweepProcessor?.disable();
@@ -391,30 +376,21 @@ export class CustomisationSyncContext implements CustomisationSyncDialogView {
this.dependencies.hideConfigurationNotice();
}
private setManifest(key: string, manifest: PluginManifest) {
const old = this.pluginManifests.get(key);
if (old && !isObjectDifferent(manifest, old)) return;
this.pluginManifests.set(key, manifest);
this.manifests.set(this.pluginManifests);
}
getFileCategory(
filePath: string
): "CONFIG" | "THEME" | "SNIPPET" | "PLUGIN_MAIN" | "PLUGIN_ETC" | "PLUGIN_DATA" | "" {
private getFileCategory(filePath: string): CustomisationSyncFileCategory {
return getCustomisationSyncFileCategory(filePath, {
configDir: this.configDir,
useV2: this.useV2,
usePluginEtc: this.useSyncPluginEtc,
});
}
isTargetPath(filePath: string): boolean {
private isTargetPath(filePath: string): boolean {
return isCustomisationSyncTargetPath(filePath, {
configDir: this.configDir,
useV2: this.useV2,
usePluginEtc: this.useSyncPluginEtc,
});
}
async _everyOnDatabaseInitialized(showNotice: boolean) {
private async onDatabaseInitialised(showNotice: boolean) {
if (!this.isThisModuleEnabled()) return true;
try {
this._log("Scanning customizations...");
@@ -426,7 +402,7 @@ export class CustomisationSyncContext implements CustomisationSyncDialogView {
}
return true;
}
async _everyBeforeReplicate(showNotice: boolean) {
private async beforeReplicate(showNotice: boolean) {
if (!this.isThisModuleEnabled()) return true;
if (this.settings.autoSweepPlugins) {
await this.scanAllConfigFiles(showNotice);
@@ -434,7 +410,7 @@ export class CustomisationSyncContext implements CustomisationSyncDialogView {
}
return true;
}
async _everyOnResumeProcess(): Promise<boolean> {
private async onResumeProcess(): Promise<boolean> {
if (!this.isThisModuleEnabled()) return true;
if (this._isMainSuspended()) {
return true;
@@ -450,44 +426,10 @@ export class CustomisationSyncContext implements CustomisationSyncDialogView {
return true;
}
async reloadPluginList(showMessage: boolean) {
this.pluginList = [];
this.loadedManifest_mTime.clear();
this.catalogue.set(this.pluginList);
this.catalogueState.clearForReload();
await this.updatePluginList(showMessage);
}
async loadPluginData(path: FilePathWithPrefix): Promise<PluginDataExDisplay | false> {
const wx = await this.localDatabase.getDBEntry(path, undefined, false, false);
if (wx) {
const data = deserialize(getDocDataAsArray(wx.data), {}) as PluginDataEx;
const xFiles = [] as PluginDataExFile[];
let missingHash = false;
for (const file of data.files) {
const work = { ...file, data: [] as string[] };
if (!file.hash) {
// debugger;
const tempStr = getDocDataAsArray(work.data);
const hash = digestHash(tempStr);
file.hash = hash;
missingHash = true;
}
work.data = [file.hash];
xFiles.push(work);
}
if (missingHash) {
this._log(`Digest created for ${path} to improve checking`, LOG_LEVEL_VERBOSE);
wx.data = serialize(data);
fireAndForget(() => this.localDatabase.putDBEntry(createSavingEntryFromLoadedEntry(wx)));
}
return {
...data,
documentPath: this.getPath(wx),
files: xFiles,
} satisfies PluginDataExDisplay;
}
return false;
}
pluginScanProcessor = new QueueProcessor(
private pluginScanProcessor = new QueueProcessor(
async (v: AnyEntry[]) => {
const plugin = v[0];
if (this.useV2) {
@@ -495,16 +437,16 @@ export class CustomisationSyncContext implements CustomisationSyncDialogView {
return [];
}
const path = plugin.path || this.getPath(plugin);
const oldEntry = this.pluginList.find((e) => e.documentPath == path);
const oldEntry = this.catalogueState.findPlugin(path);
if (oldEntry && oldEntry.mtime == plugin.mtime) return [];
try {
const pluginData = await this.loadPluginData(path);
const pluginData = await loadCustomisationDisplayData(
this.dependencies,
path,
CUSTOMISATION_SYNC_READ_CODEC
);
if (pluginData) {
let newList = [...this.pluginList];
newList = newList.filter((x) => x.documentPath != pluginData.documentPath);
newList.push(pluginData);
this.pluginList = newList;
this.catalogue.set(newList);
this.catalogueState.replacePlugin(pluginData);
}
// Failed to load
return [];
@@ -525,20 +467,23 @@ export class CustomisationSyncContext implements CustomisationSyncDialogView {
}
).startPipeline();
pluginScanProcessorV2 = new QueueProcessor(
// Compatibility question: no production path currently enqueues work into
// this second processor. Preserve its construction and disposal until the
// intended V2 scan path, or its safe removal, has focused coverage.
private pluginScanProcessorV2 = new QueueProcessor(
async (v: AnyEntry[]) => {
const plugin = v[0];
const path = plugin.path || this.getPath(plugin);
const oldEntry = this.pluginList.find((e) => e.documentPath == path);
const oldEntry = this.catalogueState.findPlugin(path);
if (oldEntry && oldEntry.mtime == plugin.mtime) return [];
try {
const pluginData = await this.loadPluginData(path);
const pluginData = await loadCustomisationDisplayData(
this.dependencies,
path,
CUSTOMISATION_SYNC_READ_CODEC
);
if (pluginData) {
let newList = [...this.pluginList];
newList = newList.filter((x) => x.documentPath != pluginData.documentPath);
newList.push(pluginData);
this.pluginList = newList;
this.catalogue.set(newList);
this.catalogueState.replacePlugin(pluginData);
}
// Failed to load
return [];
@@ -559,7 +504,7 @@ export class CustomisationSyncContext implements CustomisationSyncDialogView {
}
).startPipeline();
filenameToUnifiedKey(path: string, termOverRide?: string) {
private filenameToUnifiedKey(path: string, termOverRide?: string): FilePathWithPrefix {
const term = termOverRide || this.dependencies.getDeviceAndVaultName();
return createCustomisationSyncV1DocumentPath(path, term, {
configDir: this.configDir,
@@ -568,7 +513,7 @@ export class CustomisationSyncContext implements CustomisationSyncDialogView {
});
}
filenameWithUnifiedKey(path: string, termOverRide?: string) {
private filenameWithUnifiedKey(path: string, termOverRide?: string): FilePathWithPrefix {
const term = termOverRide || this.dependencies.getDeviceAndVaultName();
return createCustomisationSyncV2DocumentPath(path, term, {
configDir: this.configDir,
@@ -577,88 +522,38 @@ export class CustomisationSyncContext implements CustomisationSyncDialogView {
});
}
unifiedKeyPrefixOfTerminal(termOverRide?: string) {
private unifiedKeyPrefixOfTerminal(termOverRide?: string): string {
const term = termOverRide || this.dependencies.getDeviceAndVaultName();
return createCustomisationSyncDevicePrefix(term);
}
parseUnifiedPath(unifiedPath: FilePathWithPrefix): {
category: string;
device: string;
key: string;
filename: string;
pathV1: FilePathWithPrefix;
} {
return parseCustomisationSyncV2DocumentPath(unifiedPath);
}
loadedManifest_mTime = new Map<string, number>();
async createPluginDataExFileV2(
private async createPluginDataExFileV2(
unifiedPathV2: FilePathWithPrefix,
loaded?: LoadedEntry
): Promise<false | LoadedEntryPluginDataExFile> {
const { category, key, filename, device } = this.parseUnifiedPath(unifiedPathV2);
if (!loaded) {
const d = await this.localDatabase.getDBEntry(unifiedPathV2);
if (!d) {
this._log(`The file ${unifiedPathV2} is not found`, LOG_LEVEL_VERBOSE);
return false;
}
if (!isLoadedEntry(d)) {
this._log(`The file ${unifiedPathV2} is not a note`, LOG_LEVEL_VERBOSE);
return false;
}
loaded = d;
}
const confKey = `${categoryToFolder(category, device)}${key}`;
const relativeFilename =
`${categoryToFolder(category, "")}${category == "CONFIG" || category == "SNIPPET" ? "" : key + "/"}${filename}`.substring(
1
);
const dataSrc = getDocData(loaded.data);
const dataStart = dataSrc.indexOf(DUMMY_END);
const data = dataSrc.substring(dataStart + DUMMY_END.length);
const file: LoadedEntryPluginDataExFile = {
...loaded,
hash: "",
data: [base64ToString(data)],
filename: relativeFilename,
displayName: filename,
};
if (filename == "manifest.json") {
// Same as previously loaded
if (
this.loadedManifest_mTime.get(confKey) != file.mtime &&
this.pluginManifests.get(confKey) == undefined
) {
try {
const parsedManifest = JSON.parse(base64ToString(data)) as PluginManifest;
this.setManifest(confKey, parsedManifest);
this.pluginList
.filter((e) => e instanceof PluginDataExDisplayV2 && e.confKey == confKey)
.forEach((e) => (e as PluginDataExDisplayV2).applyLoadedManifest());
this.catalogue.set(this.pluginList);
} catch (ex) {
// Compatibility: a caller-supplied entry bypasses the database lookup
// and the isLoadedEntry check performed by loadCustomisationV2Entry.
const loadedEntry = loaded ?? (await loadCustomisationV2Entry(this.dependencies, unifiedPathV2));
if (!loadedEntry) return false;
const { confKey, file, isManifest } = decodeCustomisationSyncV2File(unifiedPathV2, loadedEntry, DUMMY_END);
if (isManifest) {
this.catalogueState.processManifest(
confKey,
file.mtime,
() => JSON.parse(file.data[0]) as PluginManifest,
(error) => {
this._log(
`The file ${loaded.path} seems to manifest, but could not be decoded as JSON`,
`The file ${loadedEntry.path} seems to manifest, but could not be decoded as JSON`,
LOG_LEVEL_VERBOSE
);
this._log(ex, LOG_LEVEL_VERBOSE);
this._log(error, LOG_LEVEL_VERBOSE);
}
this.loadedManifest_mTime.set(confKey, file.mtime);
} else {
this.pluginList
.filter((e) => e instanceof PluginDataExDisplayV2 && e.confKey == confKey)
.forEach((e) => (e as PluginDataExDisplayV2).applyLoadedManifest());
this.catalogue.set(this.pluginList);
}
// }
);
}
return file;
}
createPluginDataFromV2(unifiedPathV2: FilePathWithPrefix) {
const { category, device, key, pathV1 } = this.parseUnifiedPath(unifiedPathV2);
private createPluginDataFromV2(unifiedPathV2: FilePathWithPrefix) {
const { category, device, key, pathV1 } = parseCustomisationSyncV2DocumentPath(unifiedPathV2);
if (category == "") return;
const ret: PluginDataExDisplayV2 = new PluginDataExDisplayV2(
@@ -670,21 +565,18 @@ export class CustomisationSyncContext implements CustomisationSyncDialogView {
files: [],
mtime: 0,
},
this.pluginManifests
this.catalogueState.manifestLookup
);
return ret;
}
updatingV2Count = 0;
async updatePluginListV2(showMessage: boolean, unifiedFilenameWithKey: FilePathWithPrefix): Promise<void> {
private async updatePluginListV2(showMessage: boolean, unifiedFilenameWithKey: FilePathWithPrefix): Promise<void> {
try {
this.updatingV2Count++;
this.migrationProgress.set(this.updatingV2Count);
this.catalogueState.beginUpdate();
// const unifiedFilenameWithKey = this.filenameWithUnifiedKey(updatedDocumentPath);
const { pathV1 } = this.parseUnifiedPath(unifiedFilenameWithKey);
const { pathV1 } = parseCustomisationSyncV2DocumentPath(unifiedFilenameWithKey);
const oldEntry = this.pluginList.find((e) => e.documentPath == pathV1);
const oldEntry = this.catalogueState.findPlugin(pathV1);
let entry: PluginDataExDisplayV2 | undefined = undefined;
if (!oldEntry || !(oldEntry instanceof PluginDataExDisplayV2)) {
@@ -697,28 +589,19 @@ export class CustomisationSyncContext implements CustomisationSyncDialogView {
}
if (!entry) return;
const file = await this.createPluginDataExFileV2(unifiedFilenameWithKey);
if (file) {
await entry.setFile(file);
} else {
entry.deleteFile(unifiedFilenameWithKey);
if (entry.files.length == 0) {
this.pluginList = this.pluginList.filter((e) => e.documentPath != pathV1);
}
}
const newList = this.pluginList.filter((e) => e.documentPath != entry.documentPath);
newList.push(entry);
this.pluginList = newList;
// Compatibility: the inherited update always re-adds an empty V2
// row after deleting its final file.
await this.catalogueState.updateV2Plugin(entry, file, unifiedFilenameWithKey);
scheduleTask("updatePluginListV2", 100, () => {
this.catalogue.set(this.pluginList);
this.catalogueState.publishCatalogue();
});
} finally {
this.updatingV2Count--;
this.migrationProgress.set(this.updatingV2Count);
this.catalogueState.endUpdate();
}
}
async migrateV1ToV2(showMessage: boolean, entry: AnyEntry): Promise<void> {
private async migrateV1ToV2(showMessage: boolean, entry: AnyEntry): Promise<void> {
const v1Path = entry.path;
this._log(`Migrating ${entry.path} to V2`, showMessage ? LOG_LEVEL_NOTICE : LOG_LEVEL_INFO);
if (entry.deleted) {
@@ -789,13 +672,11 @@ export class CustomisationSyncContext implements CustomisationSyncDialogView {
async updatePluginList(showMessage: boolean, updatedDocumentPath?: FilePathWithPrefix): Promise<void> {
if (!this.isThisModuleEnabled()) {
this.pluginScanProcessor.clearQueue();
this.pluginList = [];
this.catalogue.set(this.pluginList);
this.catalogueState.clearForDisabledRefresh();
return;
}
try {
this.updatingV2Count++;
this.migrationProgress.set(this.updatingV2Count);
this.catalogueState.beginUpdate();
const updatedDocumentId = updatedDocumentPath ? await this.path2id(updatedDocumentPath) : "";
const plugins = updatedDocumentPath
? this.localDatabase.findEntries(updatedDocumentId, updatedDocumentId + "\u{10ffff}", {
@@ -817,8 +698,7 @@ export class CustomisationSyncContext implements CustomisationSyncDialogView {
}
} finally {
this.enumerationActive.set(false);
this.updatingV2Count--;
this.migrationProgress.set(this.updatingV2Count);
this.catalogueState.endUpdate();
}
this.enumerationActive.set(false);
// return entries;
@@ -892,7 +772,7 @@ export class CustomisationSyncContext implements CustomisationSyncDialogView {
return false;
}
}
async applyDataV2(data: PluginDataExDisplayV2, content?: string): Promise<boolean> {
private async applyDataV2(data: PluginDataExDisplayV2, content?: string): Promise<boolean> {
const baseDir = this.configDir;
try {
if (content) {
@@ -1010,10 +890,10 @@ export class CustomisationSyncContext implements CustomisationSyncDialogView {
if (data.documentPath) {
const delList = [];
if (this.useV2) {
const deleteList = this.pluginList
.filter((e) => e.documentPath == data.documentPath)
.filter((e) => e instanceof PluginDataExDisplayV2)
.map((e) => e.files)
const deleteList = this.catalogueState
.findPlugins(data.documentPath)
.filter((entry) => entry instanceof PluginDataExDisplayV2)
.map((entry) => entry.files)
.flat();
for (const e of deleteList) {
delList.push(e.path);
@@ -1039,7 +919,7 @@ export class CustomisationSyncContext implements CustomisationSyncDialogView {
return false;
}
}
async _anyModuleParsedReplicationResultItem(docs: PouchDB.Core.ExistingDocument<EntryDoc>) {
private async processVirtualDocument(docs: PouchDB.Core.ExistingDocument<EntryDoc>) {
if (!docs._id.startsWith(ICXHeader)) return false;
if (this.isThisModuleEnabled()) {
await this.updatePluginList(
@@ -1050,14 +930,18 @@ export class CustomisationSyncContext implements CustomisationSyncDialogView {
if (this.isThisModuleEnabled() && this.settings.notifyPluginOrSettingUpdated) {
if (!this.dependencies.getUIControl()?.isOpen()) {
scheduleTask(UPDATED_CONFIGURATION_NOTICE_KEY, 1000, () => {
this.dependencies.showConfigurationNotice(() => this.showPluginSyncModal());
this.dependencies.showConfigurationNotice(() => this.dependencies.getUIControl()?.open());
});
}
}
return true;
}
async _everyRealizeSettingSyncMode(): Promise<boolean> {
private async realiseSettingSyncMode(): Promise<boolean> {
this.periodicPluginSweepProcessor?.disable();
// Compatibility question: this inherited callback checks the method
// reference rather than invoking it, then proceeds only while the host is
// suspended. Preserve both gates until their intended lifecycle semantics
// are verified and corrected under a separate behavioural test.
if (!this._isMainReady) return true;
if (!this._isMainSuspended()) return true;
if (!this.isThisModuleEnabled()) return true;
@@ -1072,55 +956,7 @@ export class CustomisationSyncContext implements CustomisationSyncDialogView {
return true;
}
recentProcessedInternalFiles = [] as string[];
async makeEntryFromFile(path: FilePath): Promise<false | PluginDataExFile> {
const stat = await this.storageAccess.statHidden(path);
let version: string | undefined;
let displayName: string | undefined;
if (!stat) {
return false;
}
const contentBin = await this.storageAccess.readHiddenFileBinary(path);
let content: string[];
try {
content = await arrayBufferToBase64(contentBin);
if (path.toLowerCase().endsWith("/manifest.json")) {
const v = readString(new Uint8Array(contentBin));
try {
const json: unknown = JSON.parse(v);
if (typeof json === "object" && json !== null) {
if ("version" in json) {
version = String(json.version);
}
if ("name" in json) {
displayName = String(json.name);
}
}
} catch (ex) {
this._log(
`Configuration sync data: ${path} looks like manifest, but could not read the version`,
LOG_LEVEL_INFO
);
this._log(ex, LOG_LEVEL_VERBOSE);
}
}
} catch (ex) {
this._log(`The file ${path} could not be encoded`);
this._log(ex, LOG_LEVEL_VERBOSE);
return false;
}
const mtime = stat.mtime;
return {
filename: path.substring(this.configDir.length + 1),
data: content,
mtime,
size: stat.size,
version,
displayName: displayName,
};
}
async storeCustomisationFileV2(path: FilePath, term: string, force = false) {
private async storeCustomisationFileV2(path: FilePath, term: string, force = false) {
const vf = this.filenameWithUnifiedKey(path, term);
return await serialized(`plugin-${vf}`, async () => {
const prefixedFileName = vf;
@@ -1199,7 +1035,7 @@ export class CustomisationSyncContext implements CustomisationSyncDialogView {
}
});
}
async storeCustomizationFiles(path: FilePath, termOverRide?: string) {
private async storeCustomizationFiles(path: FilePath, termOverRide?: string) {
const term = termOverRide || this.dependencies.getDeviceAndVaultName();
if (term == "") {
this._log($msg("We have to configure the device name"), LOG_LEVEL_NOTICE);
@@ -1247,7 +1083,7 @@ export class CustomisationSyncContext implements CustomisationSyncDialogView {
fileTargets = ["manifest.json", "theme.css"].map((e) => `${parentPath}/${e}` as FilePath);
}
for (const target of fileTargets) {
const data = await this.makeEntryFromFile(target);
const data = await readCustomisationFile(this.dependencies, target, this.configDir);
if (data == false) {
this._log(`Config: skipped (Possibly is not exist): ${target} `, LOG_LEVEL_VERBOSE);
continue;
@@ -1343,11 +1179,11 @@ export class CustomisationSyncContext implements CustomisationSyncDialogView {
}
});
}
async _anyProcessOptionalFileEvent(path: FilePath): Promise<boolean> {
private async processOptionalFileEvent(path: FilePath): Promise<boolean> {
return await this.watchVaultRawEventsAsync(path);
}
async watchVaultRawEventsAsync(path: FilePath) {
private async watchVaultRawEventsAsync(path: FilePath) {
if (!this._isMainReady()) return false;
if (this._isMainSuspended()) return false;
if (!this.isThisModuleEnabled()) return false;
@@ -1360,12 +1196,11 @@ export class CustomisationSyncContext implements CustomisationSyncDialogView {
// this._log(`Customization file detected: ${path}`, LOG_LEVEL_VERBOSE);
const storageMTime = ~~(((stat && stat.mtime) || 0) / 1000);
const key = `${path}-${storageMTime}`;
if (this.recentProcessedInternalFiles.contains(key)) {
if (!this.recentProcessedInternalFiles.admit(key)) {
// If recently processed, it may caused by self.
// return true to prevent pass the event to the next.
return true;
}
this.recentProcessedInternalFiles = [key, ...this.recentProcessedInternalFiles].slice(0, 100);
// To prevent saving half-collected file sets.
const keySchedule = this.filenameToUnifiedKey(path);
scheduleTask(keySchedule, 100, async () => {
@@ -1480,7 +1315,7 @@ export class CustomisationSyncContext implements CustomisationSyncDialogView {
});
}
async deleteConfigOnDatabase(prefixedFileName: FilePathWithPrefix, forceWrite = false) {
private async deleteConfigOnDatabase(prefixedFileName: FilePathWithPrefix, forceWrite = false): Promise<boolean> {
// const id = await this.path2id(prefixedFileName);
const mtime = new Date().getTime();
return await serialized("file-x-" + prefixedFileName, async () => {
@@ -1518,24 +1353,22 @@ export class CustomisationSyncContext implements CustomisationSyncDialogView {
});
}
async scanInternalFiles(): Promise<FilePath[]> {
const filenames = (await this.getFiles(this.configDir, 2))
private async scanInternalFiles(): Promise<FilePath[]> {
const filenames = (
await collectOptionalFileSyncFiles(this.dependencies, this.configDir, {
maxDepth: 2,
onError: (path, error) => {
this._log(`Could not traverse(CustomisationSync):${path}`, LOG_LEVEL_INFO);
this._log(error, LOG_LEVEL_VERBOSE);
},
})
)
.filter((e) => e.startsWith("."))
.filter((e) => !e.startsWith(".trash"));
return filenames as FilePath[];
}
_anyGetOptionalConflictCheckMethod(path: FilePathWithPrefix): Promise<boolean | "newer"> {
if (isPluginMetadata(path)) {
return Promise.resolve("newer");
}
if (isCustomisationSyncMetadata(path)) {
return Promise.resolve("newer");
}
return Promise.resolve(false);
}
_allSuspendExtraSync(): Promise<boolean> {
private suspendExtraSync(): Promise<boolean> {
if (this.settings.usePluginSync || this.settings.autoSweepPlugins) {
this._log(
"Customisation sync have been temporarily disabled. Please enable them after the fetching, if you need them.",
@@ -1547,11 +1380,11 @@ export class CustomisationSyncContext implements CustomisationSyncDialogView {
return Promise.resolve(true);
}
async _allConfigureOptionalSyncFeature(mode: OptionalSyncFeatureMode) {
await this.configureHiddenFileSync(mode);
private async enableOptionalFeature(mode: OptionalSyncFeatureMode): Promise<boolean> {
await this.configureCustomisationSync(mode);
return true;
}
async configureHiddenFileSync(mode: OptionalSyncFeatureMode) {
private async configureCustomisationSync(mode: OptionalSyncFeatureMode) {
if (mode == "DISABLE") {
await this.dependencies.applySettings(
{
@@ -1584,21 +1417,4 @@ export class CustomisationSyncContext implements CustomisationSyncDialogView {
await this.scanAllConfigFiles(true);
}
}
async getFiles(path: string, lastDepth: number) {
if (lastDepth == -1) return [];
let w: ListedFiles;
try {
w = await this.dependencies.listFiles(path);
} catch (ex) {
this._log(`Could not traverse(CustomisationSync):${path}`, LOG_LEVEL_INFO);
this._log(ex, LOG_LEVEL_VERBOSE);
return [];
}
let files = [...w.files];
for (const v of w.folders) {
files = files.concat(await this.getFiles(v, lastDepth - 1));
}
return files;
}
}
@@ -0,0 +1,129 @@
import { describe, expect, it, vi } from "vitest";
vi.mock("@/deps.ts", () => ({
diff_match_patch: class DiffMatchPatch {},
normalizePath: vi.fn((path: string) => path),
parseYaml: vi.fn(),
Platform: {},
}));
vi.mock("@/common/types.ts", () => ({
ICXHeader: "ix:",
PERIODIC_PLUGIN_SWEEP: 60,
}));
vi.mock("@/common/utils.ts", () => ({
cancelTask: vi.fn(),
EVEN: Symbol("even"),
isCustomisationSyncMetadata: vi.fn(),
isPluginMetadata: vi.fn(),
scheduleTask: vi.fn(),
}));
vi.mock("@/common/PeriodicProcessor.ts", () => ({
PeriodicProcessor: class PeriodicProcessor {},
}));
vi.mock("@/common/translation", () => ({
$msg: vi.fn((message: string) => message),
}));
vi.mock("@/common/obsidianCommunityPlugins.ts", () => ({
getObsidianCommunityPluginManager: vi.fn(),
}));
import {
LOG_LEVEL_VERBOSE,
type FilePathWithPrefix,
type LoadedEntry,
} from "@vrtmrz/livesync-commonlib/compat/common/types";
import { digestHash } from "@vrtmrz/livesync-commonlib/compat/string_and_binary/hash";
import { createCustomisationSyncCodec } from "./customisationSyncCodec.ts";
import { CustomisationSyncCatalogueState } from "./customisationSyncCatalogueState.ts";
import { CustomisationSyncContext } from "./customisationSyncContext.ts";
import { createCustomisationSyncTestDependencies } from "./customisationSyncContext.unit.fixture.ts";
const path = "ix:device-a/PLUGIN_MAIN/example%manifest.json" as FilePathWithPrefix;
const confKey = "device-a/plugins/example";
const codec = createCustomisationSyncCodec({
digestHash,
parseYaml: () => undefined,
});
function loadedManifest(manifestSource: string, mtime: number): LoadedEntry {
const data = `${codec.dummyHead}${codec.dummyEnd}${btoa(manifestSource)}`;
return {
_id: "entry-id",
_rev: "1-a",
path,
type: "plain",
datatype: "plain",
data,
ctime: 10,
mtime,
size: data.length,
children: [],
eden: {},
} as unknown as LoadedEntry;
}
function createContext() {
const log = vi.fn();
const catalogueState = new CustomisationSyncCatalogueState();
const pluginManifests = catalogueState.manifestLookup;
const loadedManifest_mTime = catalogueState.loadedManifestMTime;
const setManifests = vi.spyOn(catalogueState.manifests, "set");
const setCatalogue = vi.spyOn(catalogueState.catalogue, "set");
const context = Object.create(CustomisationSyncContext.prototype) as CustomisationSyncContext;
Object.assign(context, {
dependencies: createCustomisationSyncTestDependencies({
log,
getLocalDatabase: () => ({ getDBEntry: async () => false }) as never,
}),
catalogueState,
});
return {
context,
loadedManifest_mTime,
log,
pluginManifests,
setCatalogue,
setManifests,
};
}
describe("compatibility: Customisation Sync V2 manifest state", () => {
it("keeps the first parsed manifest when a later file has a different mtime", async () => {
const { context, loadedManifest_mTime, pluginManifests, setManifests } = createContext();
await context.testing.createPluginDataExFileV2(
path,
loadedManifest(JSON.stringify({ id: "example", name: "First", version: "1.0.0" }), 20)
);
await context.testing.createPluginDataExFileV2(
path,
loadedManifest(JSON.stringify({ id: "example", name: "Second", version: "2.0.0" }), 30)
);
expect(pluginManifests.get(confKey)).toMatchObject({ name: "First", version: "1.0.0" });
expect(loadedManifest_mTime.get(confKey)).toBe(20);
expect(setManifests).toHaveBeenCalledOnce();
});
it("records a failed manifest mtime and does not retry the same revision", async () => {
const { context, loadedManifest_mTime, log, pluginManifests, setCatalogue } = createContext();
const invalid = loadedManifest("{invalid", 20);
await expect(context.testing.createPluginDataExFileV2(path, invalid)).resolves.toMatchObject({
filename: "plugins/example/manifest.json",
});
await context.testing.createPluginDataExFileV2(path, invalid);
expect(pluginManifests.has(confKey)).toBe(false);
expect(loadedManifest_mTime.get(confKey)).toBe(20);
expect(log).toHaveBeenCalledTimes(2);
expect(log).toHaveBeenNthCalledWith(
1,
`The file ${path} seems to manifest, but could not be decoded as JSON`,
LOG_LEVEL_VERBOSE,
undefined
);
expect(log).toHaveBeenNthCalledWith(2, expect.any(SyntaxError), LOG_LEVEL_VERBOSE, undefined);
expect(setCatalogue).toHaveBeenCalledOnce();
});
});
@@ -0,0 +1,67 @@
import type { PluginManifest } from "@/deps.ts";
import type { FilePathWithPrefix } from "@vrtmrz/livesync-commonlib/compat/common/types";
import { isDocContentSame } from "@vrtmrz/livesync-commonlib/compat/common/utils";
import { getCustomisationSyncCategoryFolder } from "./customisationSyncPaths.ts";
import type { IPluginDataExDisplay, LoadedEntryPluginDataExFile } from "./customisationSyncView.ts";
export class PluginDataExDisplayV2 {
documentPath: FilePathWithPrefix;
category: string;
term: string;
files: LoadedEntryPluginDataExFile[];
name: string;
confKey: string;
_displayName: string | undefined;
_version: string | undefined;
constructor(
data: IPluginDataExDisplay,
private readonly manifestLookup: ReadonlyMap<string, PluginManifest>
) {
this.documentPath = `${data.documentPath}` as FilePathWithPrefix;
this.category = `${data.category}`;
this.name = `${data.name}`;
this.term = `${data.term}`;
this.files = [...(data.files as LoadedEntryPluginDataExFile[])];
this.confKey = `${getCustomisationSyncCategoryFolder(this.category, this.term)}${this.name}`;
this.applyLoadedManifest();
}
async setFile(file: LoadedEntryPluginDataExFile): Promise<void> {
const old = this.files.find((entry) => entry.filename == file.filename);
if (old) {
if (old.mtime == file.mtime && (await isDocContentSame(old.data, file.data))) return;
this.files = this.files.filter((entry) => entry.filename != file.filename);
}
this.files.push(file);
if (file.filename == "manifest.json") {
this.applyLoadedManifest();
}
}
deleteFile(filename: string): void {
this.files = this.files.filter((entry) => entry.filename != filename);
}
applyLoadedManifest(): void {
const manifest = this.manifestLookup.get(this.confKey);
if (manifest) {
this._displayName = manifest.name;
if (this.category == "PLUGIN_MAIN" || this.category == "THEME") {
this._version = manifest.version;
}
}
}
get displayName(): string {
return this._displayName || this.name;
}
get version(): string | undefined {
return this._version;
}
get mtime(): number {
return ~~this.files.reduce((sum, file) => sum + file.mtime, 0) / this.files.length;
}
}
@@ -0,0 +1,58 @@
import { describe, expect, it } from "vitest";
import type { PluginManifest } from "@/deps.ts";
import type { FilePathWithPrefix } from "@vrtmrz/livesync-commonlib/compat/common/types";
import { PluginDataExDisplayV2 } from "./customisationSyncModel.ts";
import type { IPluginDataExDisplay, LoadedEntryPluginDataExFile } from "./customisationSyncView.ts";
function file(filename: string, mtime: number, data: string[]): LoadedEntryPluginDataExFile {
return { filename, mtime, data, size: data.join("").length } as LoadedEntryPluginDataExFile;
}
function display(files: LoadedEntryPluginDataExFile[] = []): IPluginDataExDisplay {
return {
documentPath: "ix:device-a/PLUGIN_MAIN/example%main.js" as FilePathWithPrefix,
category: "PLUGIN_MAIN",
name: "example",
term: "device-a",
files,
mtime: 0,
};
}
describe("PluginDataExDisplayV2", () => {
it("projects manifest identity and file modification time", () => {
const manifests = new Map([
["device-a/plugins/example", { name: "Example plug-in", version: "1.2.3" } as PluginManifest],
]);
const model = new PluginDataExDisplayV2(
display([file("main.js", 10, ["main"]), file("data.json", 20, ["data"])]),
manifests
);
expect(model.confKey).toBe("device-a/plugins/example");
expect(model.displayName).toBe("Example plug-in");
expect(model.version).toBe("1.2.3");
expect(model.mtime).toBe(15);
});
it("retains an unchanged file and replaces changed content", async () => {
const original = file("main.js", 10, ["same"]);
const model = new PluginDataExDisplayV2(display([original]), new Map());
await model.setFile(file("main.js", 10, ["same"]));
expect(model.files[0]).toBe(original);
const changed = file("main.js", 10, ["changed"]);
await model.setFile(changed);
expect(model.files).toEqual([changed]);
});
it("deletes only the named file", () => {
const retained = file("styles.css", 20, ["css"]);
const model = new PluginDataExDisplayV2(display([file("main.js", 10, ["main"]), retained]), new Map());
model.deleteFile("main.js");
expect(model.files).toEqual([retained]);
});
});
@@ -17,6 +17,23 @@ export type CustomisationSyncPathOptions = {
usePluginEtc: boolean;
};
export function getCustomisationSyncCategoryFolder(category: string, configDir: string = ""): string {
switch (category) {
case "CONFIG":
return `${configDir}/`;
case "THEME":
return `${configDir}/themes/`;
case "SNIPPET":
return `${configDir}/snippets/`;
case "PLUGIN_MAIN":
case "PLUGIN_DATA":
case "PLUGIN_ETC":
return `${configDir}/plugins/`;
default:
return "";
}
}
export function getCustomisationSyncFileCategory(
filePath: string,
options: CustomisationSyncPathOptions
@@ -5,6 +5,7 @@ import {
createCustomisationSyncDevicePrefix,
createCustomisationSyncV1DocumentPath,
createCustomisationSyncV2DocumentPath,
getCustomisationSyncCategoryFolder,
getCustomisationSyncFileCategory,
isCustomisationSyncTargetPath,
getCustomisationSyncSettingKey,
@@ -20,6 +21,18 @@ const currentOptions: CustomisationSyncPathOptions = {
};
describe("compatibility: Customisation Sync path operations", () => {
it.each([
["CONFIG", ".obsidian/"],
["THEME", ".obsidian/themes/"],
["SNIPPET", ".obsidian/snippets/"],
["PLUGIN_MAIN", ".obsidian/plugins/"],
["PLUGIN_DATA", ".obsidian/plugins/"],
["PLUGIN_ETC", ".obsidian/plugins/"],
["UNKNOWN", ""],
])("maps category %s to folder %s", (category, expected) => {
expect(getCustomisationSyncCategoryFolder(category, ".obsidian")).toBe(expected);
});
it.each([
[".obsidian/app.json", "CONFIG"],
[".obsidian/themes/Minimal/theme.css", "THEME"],
@@ -0,0 +1,208 @@
import type {
FilePath,
FilePathWithPrefix,
LoadedEntry,
LOG_LEVEL,
} from "@vrtmrz/livesync-commonlib/compat/common/types";
import { LOG_LEVEL_INFO, LOG_LEVEL_VERBOSE } from "@vrtmrz/livesync-commonlib/compat/common/types";
import type { StorageAccess } from "@vrtmrz/livesync-commonlib/compat/interfaces/StorageAccess";
import type { LiveSyncLocalDB } from "@vrtmrz/livesync-commonlib/compat/pouchdb/LiveSyncLocalDB";
import type { IPathService } from "@vrtmrz/livesync-commonlib/compat/services/base/IService";
import type { LogFunction } from "@vrtmrz/livesync-commonlib/compat/services/lib/logUtils";
import {
createSavingEntryFromLoadedEntry,
fireAndForget,
getDocData,
getDocDataAsArray,
isLoadedEntry,
} from "@vrtmrz/livesync-commonlib/compat/common/utils";
import { digestHash } from "@vrtmrz/livesync-commonlib/compat/string_and_binary/hash";
import { arrayBufferToBase64, readString } from "@vrtmrz/livesync-commonlib/compat/string_and_binary/convert";
import { base64ToString } from "octagonal-wheels/binary/base64";
import type { PluginDataEx, PluginDataExFile } from "./customisationSyncCodec.ts";
import { getCustomisationSyncCategoryFolder, parseCustomisationSyncV2DocumentPath } from "./customisationSyncPaths.ts";
import type { LoadedEntryPluginDataExFile, PluginDataExDisplay } from "./customisationSyncView.ts";
type CustomisationSyncLogDependency = {
log: LogFunction;
};
type CustomisationSyncStorageMethods<Method extends keyof StorageAccess> = {
storageAccess: Pick<StorageAccess, Method>;
};
type CustomisationSyncDatabaseMethods<Method extends keyof LiveSyncLocalDB> = {
getLocalDatabase(): Pick<LiveSyncLocalDB, Method>;
};
type CustomisationSyncPathMethods<Method extends keyof IPathService> = {
path: Pick<IPathService, Method>;
};
export type CustomisationSyncFileReaderDependencies = CustomisationSyncStorageMethods<
"readHiddenFileBinary" | "statHidden"
> &
CustomisationSyncLogDependency;
export type CustomisationSyncDisplayLoaderDependencies = CustomisationSyncDatabaseMethods<"getDBEntry" | "putDBEntry"> &
CustomisationSyncPathMethods<"getPath"> &
CustomisationSyncLogDependency;
export type CustomisationSyncV2EntryLoaderDependencies = CustomisationSyncDatabaseMethods<"getDBEntry"> &
CustomisationSyncLogDependency;
export type CustomisationSyncReadCodec = {
deserialize<T>(source: string[], defaultValue: T): T;
serialize(data: PluginDataEx): string;
};
export type DecodedCustomisationSyncV2File = {
confKey: string;
file: LoadedEntryPluginDataExFile;
isManifest: boolean;
};
function log(dependencies: CustomisationSyncLogDependency, message: unknown, level?: LOG_LEVEL, key?: string): void {
dependencies.log(message, level, key);
}
export async function readCustomisationFile(
dependencies: CustomisationSyncFileReaderDependencies,
path: FilePath,
configDir: string
): Promise<false | PluginDataExFile> {
const stat = await dependencies.storageAccess.statHidden(path);
let version: string | undefined;
let displayName: string | undefined;
if (!stat) {
return false;
}
const contentBin = await dependencies.storageAccess.readHiddenFileBinary(path);
let content: string[];
try {
content = await arrayBufferToBase64(contentBin);
if (path.toLowerCase().endsWith("/manifest.json")) {
const manifestSource = readString(new Uint8Array(contentBin));
try {
const manifest: unknown = JSON.parse(manifestSource);
if (typeof manifest === "object" && manifest !== null) {
if ("version" in manifest) {
version = String(manifest.version);
}
if ("name" in manifest) {
displayName = String(manifest.name);
}
}
} catch (error) {
log(
dependencies,
`Configuration sync data: ${path} looks like manifest, but could not read the version`,
LOG_LEVEL_INFO
);
log(dependencies, error, LOG_LEVEL_VERBOSE);
}
}
} catch (error) {
log(dependencies, `The file ${path} could not be encoded`);
log(dependencies, error, LOG_LEVEL_VERBOSE);
return false;
}
return {
// Compatibility: target validation belongs to the caller. The legacy
// reader derives this name positionally without checking the prefix.
filename: path.substring(configDir.length + 1),
data: content,
mtime: stat.mtime,
size: stat.size,
version,
displayName,
};
}
export async function loadCustomisationDisplayData(
dependencies: CustomisationSyncDisplayLoaderDependencies,
path: FilePathWithPrefix,
codec: CustomisationSyncReadCodec
): Promise<PluginDataExDisplay | false> {
const loaded = await dependencies.getLocalDatabase().getDBEntry(path, undefined, false, false);
if (!loaded) {
return false;
}
const data = codec.deserialize(getDocDataAsArray(loaded.data), {}) as PluginDataEx;
const displayFiles: PluginDataExFile[] = [];
let missingHash = false;
for (const file of data.files) {
const displayFile = { ...file, data: [] as string[] };
if (!file.hash) {
// Compatibility question: the inherited implementation clears the
// display copy before calculating this temporary hash, so callers
// see digestHash([]) until the asynchronously repaired document is
// loaded again. The serialiser still writes the real content hash.
const temporaryHashSource = getDocDataAsArray(displayFile.data);
file.hash = digestHash(temporaryHashSource);
missingHash = true;
}
displayFile.data = [file.hash];
displayFiles.push(displayFile);
}
if (missingHash) {
log(dependencies, `Digest created for ${path} to improve checking`, LOG_LEVEL_VERBOSE);
loaded.data = codec.serialize(data);
// Compatibility: catalogue loading does not wait for the repair write.
fireAndForget(() => dependencies.getLocalDatabase().putDBEntry(createSavingEntryFromLoadedEntry(loaded)));
}
return {
...data,
documentPath: dependencies.path.getPath(loaded),
files: displayFiles,
} satisfies PluginDataExDisplay;
}
export async function loadCustomisationV2Entry(
dependencies: CustomisationSyncV2EntryLoaderDependencies,
path: FilePathWithPrefix
): Promise<LoadedEntry | false> {
const loaded = await dependencies.getLocalDatabase().getDBEntry(path);
if (!loaded) {
log(dependencies, `The file ${path} is not found`, LOG_LEVEL_VERBOSE);
return false;
}
if (!isLoadedEntry(loaded)) {
log(dependencies, `The file ${path} is not a note`, LOG_LEVEL_VERBOSE);
return false;
}
return loaded;
}
export function decodeCustomisationSyncV2File(
path: FilePathWithPrefix,
loaded: LoadedEntry,
dummyEnd: string
): DecodedCustomisationSyncV2File {
const { category, key, filename, device } = parseCustomisationSyncV2DocumentPath(path);
const categoryFolder = getCustomisationSyncCategoryFolder(category, device);
const confKey = `${categoryFolder}${key}`;
const relativeFilename =
`${getCustomisationSyncCategoryFolder(category, "")}${category == "CONFIG" || category == "SNIPPET" ? "" : key + "/"}${filename}`.substring(
1
);
const source = getDocData(loaded.data);
const dataStart = source.indexOf(dummyEnd);
// Compatibility question: a missing marker is not rejected. substring()
// starts at dummyEnd.length - 1, preserving the old best-effort decode.
const encodedData = source.substring(dataStart + dummyEnd.length);
const file: LoadedEntryPluginDataExFile = {
...loaded,
hash: "",
data: [base64ToString(encodedData)],
filename: relativeFilename,
displayName: filename,
};
return {
confKey,
file,
isManifest: filename == "manifest.json",
};
}
@@ -0,0 +1,222 @@
import { describe, expect, it, vi } from "vitest";
import {
LOG_LEVEL_INFO,
LOG_LEVEL_VERBOSE,
type FilePath,
type FilePathWithPrefix,
type LoadedEntry,
type UXStat,
} from "@vrtmrz/livesync-commonlib/compat/common/types";
import { digestHash } from "@vrtmrz/livesync-commonlib/compat/string_and_binary/hash";
import { createCustomisationSyncCodec, type PluginDataEx } from "./customisationSyncCodec.ts";
import {
decodeCustomisationSyncV2File,
loadCustomisationDisplayData,
loadCustomisationV2Entry,
readCustomisationFile,
} from "./customisationSyncReadOperations.ts";
const configDir = ".obsidian";
const filePath = ".obsidian/plugins/example/manifest.json" as FilePath;
const documentPath = "ix:device-a/PLUGIN_MAIN/example.md" as FilePathWithPrefix;
const stat = { ctime: 10, mtime: 20, size: 42, type: "file" } as UXStat;
const codec = createCustomisationSyncCodec({
digestHash,
parseYaml: () => undefined,
});
function createDependencies() {
const localDatabase = {
getDBEntry: vi.fn(),
putDBEntry: vi.fn(async (_entry: unknown) => ({ ok: true, id: "id", rev: "1-a" })),
};
const storageAccess = {
statHidden: vi.fn(async () => stat as UXStat | null),
readHiddenFileBinary: vi.fn(),
};
const log = vi.fn();
const dependencies = {
getLocalDatabase: () => localDatabase as never,
storageAccess: storageAccess as never,
path: {
getPath: vi.fn((entry: { path: FilePathWithPrefix }) => entry.path),
} as never,
log,
};
return { dependencies, localDatabase, log, storageAccess };
}
function loadedEntry(path: FilePathWithPrefix, data: string): LoadedEntry {
return {
_id: "entry-id",
_rev: "1-a",
path,
type: "plain",
datatype: "plain",
data,
ctime: 10,
mtime: 20,
size: data.length,
children: [],
eden: {},
} as unknown as LoadedEntry;
}
function pluginData(hash?: string): PluginDataEx {
return {
category: "PLUGIN_MAIN",
name: "example",
term: "device-a",
mtime: 20,
files: [
{
filename: "plugins/example/main.js",
data: ["payload"],
mtime: 20,
size: 7,
hash,
},
],
};
}
describe("Customisation Sync read operations", () => {
it("does not read content when a local file is missing", async () => {
const { dependencies, storageAccess } = createDependencies();
storageAccess.statHidden.mockResolvedValue(null);
await expect(readCustomisationFile(dependencies, filePath, configDir)).resolves.toBe(false);
expect(storageAccess.readHiddenFileBinary).not.toHaveBeenCalled();
});
it("propagates a storage read failure without converting it to an encoding failure", async () => {
const { dependencies, log, storageAccess } = createDependencies();
const error = new Error("read failed");
storageAccess.readHiddenFileBinary.mockRejectedValue(error);
await expect(readCustomisationFile(dependencies, filePath, configDir)).rejects.toBe(error);
expect(log).not.toHaveBeenCalled();
});
it("encodes a manifest and extracts its display metadata", async () => {
const { dependencies, storageAccess } = createDependencies();
const source = JSON.stringify({ name: "Example plug-in", version: "1.2.3" });
storageAccess.readHiddenFileBinary.mockResolvedValue(new TextEncoder().encode(source).buffer);
await expect(readCustomisationFile(dependencies, filePath, configDir)).resolves.toEqual({
filename: "plugins/example/manifest.json",
data: [btoa(source)],
mtime: 20,
size: 42,
version: "1.2.3",
displayName: "Example plug-in",
});
});
it("keeps an unreadable manifest as file data and reports only the metadata failure", async () => {
const { dependencies, log, storageAccess } = createDependencies();
const errorSource = "{invalid";
storageAccess.readHiddenFileBinary.mockResolvedValue(new TextEncoder().encode(errorSource).buffer);
await expect(readCustomisationFile(dependencies, filePath, configDir)).resolves.toMatchObject({
filename: "plugins/example/manifest.json",
data: [btoa(errorSource)],
version: undefined,
displayName: undefined,
});
expect(log).toHaveBeenNthCalledWith(
1,
`Configuration sync data: ${filePath} looks like manifest, but could not read the version`,
LOG_LEVEL_INFO,
undefined
);
expect(log).toHaveBeenNthCalledWith(2, expect.any(SyntaxError), LOG_LEVEL_VERBOSE, undefined);
});
it("loads V1 display data without retaining file content", async () => {
const { dependencies, localDatabase } = createDependencies();
const data = pluginData("known-hash");
localDatabase.getDBEntry.mockResolvedValue(loadedEntry(documentPath, JSON.stringify(data)));
await expect(loadCustomisationDisplayData(dependencies, documentPath, codec)).resolves.toEqual({
...data,
documentPath,
files: [{ ...data.files[0], data: ["known-hash"] }],
});
expect(localDatabase.getDBEntry).toHaveBeenCalledWith(documentPath, undefined, false, false);
expect(localDatabase.putDBEntry).not.toHaveBeenCalled();
});
it("preserves the inherited transient empty-data hash while repairing a V1 document", async () => {
const { dependencies, localDatabase, log } = createDependencies();
const data = pluginData();
localDatabase.getDBEntry.mockResolvedValue(loadedEntry(documentPath, JSON.stringify(data)));
const result = await loadCustomisationDisplayData(dependencies, documentPath, codec);
expect(result).toMatchObject({ files: [{ data: [digestHash([])] }] });
expect(localDatabase.putDBEntry).toHaveBeenCalledOnce();
const saving = localDatabase.putDBEntry.mock.calls[0][0] as { data: Blob };
expect(await saving.data.text()).toContain(digestHash(["payload"]));
expect(log).toHaveBeenCalledWith(
`Digest created for ${documentPath} to improve checking`,
LOG_LEVEL_VERBOSE,
undefined
);
});
it("returns false when a V1 document is absent", async () => {
const { dependencies, localDatabase } = createDependencies();
localDatabase.getDBEntry.mockResolvedValue(false);
await expect(loadCustomisationDisplayData(dependencies, documentPath, codec)).resolves.toBe(false);
});
it("distinguishes an absent V2 entry from a non-note database entry", async () => {
const { dependencies, localDatabase, log } = createDependencies();
const path = "ix:device-a/PLUGIN_MAIN/example%main.js" as FilePathWithPrefix;
localDatabase.getDBEntry.mockResolvedValueOnce(false);
await expect(loadCustomisationV2Entry(dependencies, path)).resolves.toBe(false);
expect(log).toHaveBeenLastCalledWith(`The file ${path} is not found`, LOG_LEVEL_VERBOSE, undefined);
localDatabase.getDBEntry.mockResolvedValueOnce({ path, type: "leaf" });
await expect(loadCustomisationV2Entry(dependencies, path)).resolves.toBe(false);
expect(log).toHaveBeenLastCalledWith(`The file ${path} is not a note`, LOG_LEVEL_VERBOSE, undefined);
});
it("returns a loaded V2 entry after the exact single-argument database lookup", async () => {
const { dependencies, localDatabase } = createDependencies();
const path = "ix:device-a/PLUGIN_MAIN/example%main.js" as FilePathWithPrefix;
const loaded = loadedEntry(path, "data");
localDatabase.getDBEntry.mockResolvedValue(loaded);
await expect(loadCustomisationV2Entry(dependencies, path)).resolves.toBe(loaded);
expect(localDatabase.getDBEntry).toHaveBeenCalledWith(path);
});
it("decodes a V2 payload into its relative Customisation Sync filename", () => {
const path = "ix:device-a/PLUGIN_MAIN/example%main.js" as FilePathWithPrefix;
const loaded = loadedEntry(path, `${codec.dummyHead}${codec.dummyEnd}${btoa("console.log('example');")}`);
expect(decodeCustomisationSyncV2File(path, loaded, codec.dummyEnd)).toEqual({
confKey: "device-a/plugins/example",
isManifest: false,
file: {
...loaded,
filename: "plugins/example/main.js",
displayName: "main.js",
hash: "",
data: ["console.log('example');"],
},
});
});
it("preserves the inherited best-effort offset when a V2 marker is missing", () => {
const path = "ix:device-a/CONFIG/app.json%app.json" as FilePathWithPrefix;
const loaded = loadedEntry(path, `00${btoa("hello")}`);
expect(decodeCustomisationSyncV2File(path, loaded, "END").file.data).toEqual(["hello"]);
});
});
@@ -0,0 +1,17 @@
const MAX_RECENT_CUSTOMISATION_EVENTS = 100;
/** Keeps the bounded newest-first raw-event keys used by Customisation Sync. */
export class CustomisationSyncRecentEventDeduplicator {
private keys: string[] = [];
/**
* Records a key when it is new and returns whether the caller should act.
* Native Array#includes is intentional: the old `.contains` extension is
* not available in every runtime where the feature is exercised.
*/
admit(key: string): boolean {
if (this.keys.includes(key)) return false;
this.keys = [key, ...this.keys].slice(0, MAX_RECENT_CUSTOMISATION_EVENTS);
return true;
}
}
@@ -0,0 +1,24 @@
import { describe, expect, it } from "vitest";
import { CustomisationSyncRecentEventDeduplicator } from "./customisationSyncRecentEventDeduplicator.ts";
describe("Customisation Sync recent raw-event keys", () => {
it("admits a key once and keeps newer keys first", () => {
const history = new CustomisationSyncRecentEventDeduplicator();
expect(history.admit("old")).toBe(true);
expect(history.admit("new")).toBe(true);
expect(history.admit("old")).toBe(false);
});
it("evicts the oldest key when the newest-first history exceeds 100 entries", () => {
const history = new CustomisationSyncRecentEventDeduplicator();
for (let index = 0; index < 101; index++) {
expect(history.admit(`key-${index}`)).toBe(true);
}
expect(history.admit("key-0")).toBe(true);
expect(history.admit("key-100")).toBe(false);
});
});
@@ -1,15 +1,22 @@
import type { PluginManifest } from "@/deps.ts";
import type {
EntryDoc,
FilePathWithPrefix,
FilePath,
LoadedEntry,
PluginSyncSettingEntry,
SYNC_MODE,
} from "@vrtmrz/livesync-commonlib/compat/common/types";
import type PouchDB from "pouchdb-core";
import type { Readable } from "svelte/store";
import type { PluginDataExFile } from "./customisationSyncCodec.ts";
import type { CustomisationSyncFileCategory } from "./customisationSyncPaths.ts";
import type { OptionalSyncFeatureMode } from "@/features/optionalSyncFeatures.ts";
import type { PluginDataExDisplayV2 } from "./customisationSyncModel.ts";
export type LoadedEntryPluginDataExFile = LoadedEntry & PluginDataExFile;
export type { CustomisationSyncFileCategory } from "./customisationSyncPaths.ts";
export interface IPluginDataExDisplay {
documentPath: FilePathWithPrefix;
@@ -33,6 +40,48 @@ export type PluginDataExDisplay = {
mtime: number;
};
/**
* Semantic callbacks registered by the optional-file composition feature.
*
* The context owns the implementations, while the optional-file composition
* adapts these operations to Commonlib's aggregation contracts. Consumers
* receive only callable operations, not the context or its private state.
*/
export interface CustomisationSyncServiceHandlers {
readonly processOptionalFileEvent: (path: FilePath) => Promise<boolean>;
readonly processVirtualDocument: (docs: PouchDB.Core.ExistingDocument<EntryDoc>) => Promise<boolean>;
readonly onRealiseSetting: () => Promise<boolean>;
readonly onResuming: () => Promise<boolean>;
readonly onBeforeReplicate: (showMessage: boolean) => Promise<boolean>;
readonly onDatabaseInitialised: (showNotice: boolean) => Promise<boolean>;
readonly suspendExtraSync: () => Promise<boolean>;
readonly enableOptionalFeature: (mode: OptionalSyncFeatureMode) => Promise<boolean>;
}
/**
* Explicit internal operations used by maintained real-Obsidian contract
* tests. This is deliberately narrower than the concrete context and does
* not expose reactive stores, queues, or host dependencies.
*/
export interface CustomisationSyncTestingView {
readonly configDir: string;
scanInternalFiles(): Promise<FilePath[]>;
scanAllConfigFiles(showMessage: boolean): Promise<void>;
getFileCategory(filePath: string): CustomisationSyncFileCategory;
isTargetPath(filePath: string): boolean;
filenameToUnifiedKey(path: string, termOverride?: string): FilePathWithPrefix;
filenameWithUnifiedKey(path: string, termOverride?: string): FilePathWithPrefix;
unifiedKeyPrefixOfTerminal(termOverride?: string): string;
storeCustomizationFiles(path: FilePath, termOverride?: string): Promise<unknown>;
deleteConfigOnDatabase(prefixedFileName: FilePathWithPrefix, forceWrite?: boolean): Promise<boolean>;
createPluginDataFromV2(unifiedPathV2: FilePathWithPrefix): PluginDataExDisplayV2 | undefined;
createPluginDataExFileV2(
unifiedPathV2: FilePathWithPrefix,
loaded?: LoadedEntry
): Promise<false | LoadedEntryPluginDataExFile>;
applyDataV2(data: PluginDataExDisplayV2, content?: string): Promise<boolean>;
}
/** Stable catalogue and operation surface consumed by the Obsidian dialogue. */
export interface CustomisationSyncDialogView {
readonly catalogue: Readable<IPluginDataExDisplay[]>;
@@ -0,0 +1,260 @@
import {
LOG_LEVEL_DEBUG,
LOG_LEVEL_VERBOSE,
type FilePath,
type FilePathWithPrefix,
type LoadedEntry,
type LOG_LEVEL,
type MetaEntry,
type UXFileInfo,
} from "@vrtmrz/livesync-commonlib/compat/common/types";
import type { StorageAccess } from "@vrtmrz/livesync-commonlib/compat/interfaces/StorageAccess";
import type { LogFunction } from "@vrtmrz/livesync-commonlib/compat/services/lib/logUtils";
import { addPrefix } from "@vrtmrz/livesync-commonlib/compat/string_and_binary/path";
import { serialized } from "octagonal-wheels/concurrency/lock";
import { Semaphore } from "octagonal-wheels/concurrency/semaphore";
import { ICHeader } from "@/common/types.ts";
import type { HiddenFileSyncConflictResolution } from "./hiddenFileSyncConflictResolution.ts";
import type { HiddenFileSyncDatabaseExtractionOperations } from "./hiddenFileSyncDatabaseExtractionOperations.ts";
import type { HiddenFileSyncDatabaseWriteOperations } from "./hiddenFileSyncDatabaseWriteOperations.ts";
import type { HiddenFileSyncProcessedState } from "./hiddenFileSyncProcessedState.ts";
import { getHiddenFileSyncComparisonMTime } from "./hiddenFileSyncState.ts";
import { compareMTime, TARGET_IS_NEW } from "@/common/utils.ts";
type HiddenFileSyncStorageChangeAccess = Pick<StorageAccess, "statHidden">;
export type HiddenFileSyncChangeProcessorDependencies = {
storageAccess: HiddenFileSyncStorageChangeAccess;
readFileWithInfo(path: FilePath): Promise<UXFileInfo>;
loadDatabaseMetadata(path: FilePathWithPrefix): Promise<MetaEntry | LoadedEntry | false>;
databaseWriteOperations: Pick<HiddenFileSyncDatabaseWriteOperations, "store" | "delete">;
databaseExtractionOperations: Pick<HiddenFileSyncDatabaseExtractionOperations, "extract">;
processedState: Pick<
HiddenFileSyncProcessedState,
| "fileToStatKey"
| "getLastProcessedFileKey"
| "getLastProcessedFileMTime"
| "updateLastProcessedFile"
| "updateLastProcessed"
>;
conflictResolution: Pick<HiddenFileSyncConflictResolution, "queue">;
log: LogFunction;
publishActivity(eventCount: number, processingCount: number): void;
};
export type HiddenFileSyncDatabaseChangeOptions = Readonly<{
preventDoubleProcess?: boolean;
onlyNew?: boolean;
metaEntry?: MetaEntry | false;
includeDeletion?: boolean;
}>;
export type HiddenFileSyncChangeProcessor = {
processStorageChange(
path: FilePath,
onlyNew?: boolean,
forceWrite?: boolean,
includeDeleted?: boolean
): Promise<boolean | undefined>;
processDatabaseChange(
path: FilePath,
headerLine: string,
options?: HiddenFileSyncDatabaseChangeOptions
): Promise<boolean>;
dispose(): void;
};
class HiddenFileSyncChangeProcessorOwner implements HiddenFileSyncChangeProcessor {
private readonly semaphore = Semaphore(10);
private eventCount = 0;
private processingCount = 0;
private disposed = false;
constructor(private readonly dependencies: HiddenFileSyncChangeProcessorDependencies) {}
async processStorageChange(
path: FilePath,
onlyNew = false,
forceWrite = false,
includeDeleted = true
): Promise<boolean | undefined> {
try {
return await this.serialiseForEvent(path, async () => {
let stat = await this.dependencies.storageAccess.statHidden(path);
// Sometimes a folder is delivered as a file event.
if (stat != null && stat.type != "file") {
return false;
}
const key = await this.dependencies.processedState.fileToStatKey(path, stat);
// A raw event can occur while the file is being read. Scans
// still enumerate every path, but event admission skips this
// exact already-settled key.
const lastKey = this.dependencies.processedState.getLastProcessedFileKey(path);
if (lastKey == key) {
this.log(`${path} Already processed.`, LOG_LEVEL_DEBUG);
return true;
}
// Read the stat and content as one operation. The stat is
// deliberately compared again below: a file can change while
// the first stat is in flight.
const fileInfo = await this.dependencies.readFileWithInfo(path);
const cacheMTime = getHiddenFileSyncComparisonMTime(fileInfo.stat);
const statMtime = getHiddenFileSyncComparisonMTime(stat);
if (cacheMTime != statMtime) {
this.log(`Hidden file:${path} is changed.`, LOG_LEVEL_VERBOSE);
stat = fileInfo.stat;
}
// Compatibility: the storage marker advances before the
// database operation. A later write failure can therefore
// leave this event marked as processed until a scan or state
// change causes it to be reconsidered.
this.dependencies.processedState.updateLastProcessedFile(path, stat!);
const lastIsNotFound = !lastKey || lastKey.endsWith("-0-0");
const nowIsNotFound = fileInfo.deleted;
const type = lastIsNotFound && nowIsNotFound ? "invalid" : nowIsNotFound ? "delete" : "modified";
if (type == "invalid") {
// Maybe the folder was deleted.
return false;
}
const storageMTimeActual = getHiddenFileSyncComparisonMTime(stat);
const storageMTime =
storageMTimeActual == 0
? this.dependencies.processedState.getLastProcessedFileMTime(path)
: storageMTimeActual;
if (onlyNew) {
const prefixedFileName = addPrefix(path, ICHeader);
const fileOnDatabase = await this.dependencies.loadDatabaseMetadata(prefixedFileName);
const databaseMTime = getHiddenFileSyncComparisonMTime(fileOnDatabase, includeDeleted);
const difference = compareMTime(storageMTime, databaseMTime);
if (difference != TARGET_IS_NEW) {
this.log(`Hidden file:${path} is not new.`, LOG_LEVEL_VERBOSE);
// OnlyNew does not handle a deletion. Preserve the
// inherited partial settlement when both values exist.
if (fileOnDatabase && stat) {
this.dependencies.processedState.updateLastProcessed(path, fileOnDatabase, stat);
}
return true;
}
}
if (type == "delete") {
this.log(`Deletion detected: ${path}`);
return await this.dependencies.databaseWriteOperations.delete(path, forceWrite);
}
if (type == "modified") {
this.log(`Modification detected:${path}`, LOG_LEVEL_VERBOSE);
const result = await this.dependencies.databaseWriteOperations.store(fileInfo, forceWrite);
const resultText = result === undefined ? "Nothing changed" : result ? "Updated" : "Failed";
this.log(`${resultText}: ${path} ${resultText}`, LOG_LEVEL_VERBOSE);
return result;
}
return false;
});
} catch (error) {
this.log(`Failed to process hidden file:${path}`);
this.log(error, LOG_LEVEL_VERBOSE);
}
// Could not be processed, but it was this operation's event. Return
// true to prevent a later handler from claiming it.
return true;
}
async processDatabaseChange(
path: FilePath,
headerLine: string,
options: HiddenFileSyncDatabaseChangeOptions = {}
): Promise<boolean> {
const {
preventDoubleProcess = false,
onlyNew = false,
metaEntry = false,
includeDeletion = true,
} = options;
return await this.serialiseForEvent(path, async () => {
try {
const prefixedPath = addPrefix(path, ICHeader);
const docMeta = metaEntry
? metaEntry
: await this.dependencies.loadDatabaseMetadata(prefixedPath);
if (docMeta === false) {
this.log(`${headerLine}: Failed to read detail of ${path}`);
throw new Error(`Failed to read detail ${path}`);
}
if (docMeta._conflicts && docMeta._conflicts.length > 0) {
this.dependencies.conflictResolution.queue(path);
this.log(`${headerLine} Hidden file conflicted, enqueued to resolve`);
return true;
}
const extracted = await this.dependencies.databaseExtractionOperations.extract(path, {
metaEntry: docMeta,
preventDoubleProcess,
onlyNew,
includeDeletion,
});
if (extracted) {
this.log(`${headerLine} Hidden file processed`);
}
} catch (error) {
this.log(`${headerLine} Failed to process hidden file`);
this.log(error, LOG_LEVEL_VERBOSE);
}
// Compatibility: recognition consumes the database event even when
// extraction returned false or threw. A later scan or state change,
// rather than handler fall-through, is responsible for retrying it.
return true;
});
}
dispose(): void {
if (this.disposed) return;
this.disposed = true;
this.eventCount = 0;
this.processingCount = 0;
this.publishActivity();
}
private async serialiseForEvent<Result>(file: FilePath, operation: () => Promise<Result>): Promise<Result> {
this.eventCount++;
this.publishActivity();
const release = await this.semaphore.acquire();
try {
return await serialized(`hidden-file-event:${file}`, async () => {
this.processingCount++;
this.publishActivity();
try {
return await operation();
} finally {
this.processingCount = Math.max(0, this.processingCount - 1);
this.publishActivity();
}
});
} finally {
release();
this.eventCount = Math.max(0, this.eventCount - 1);
this.publishActivity();
}
}
private publishActivity(): void {
this.dependencies.publishActivity(
this.disposed ? 0 : this.eventCount,
this.disposed ? 0 : this.processingCount
);
}
private log(message: unknown, level?: LOG_LEVEL, key?: string): void {
this.dependencies.log(message, level, key);
}
}
export function createHiddenFileSyncChangeProcessor(
dependencies: HiddenFileSyncChangeProcessorDependencies
): HiddenFileSyncChangeProcessor {
return new HiddenFileSyncChangeProcessorOwner(dependencies);
}
@@ -0,0 +1,159 @@
import { describe, expect, it, vi } from "vitest";
import type { FilePath, MetaEntry, UXFileInfo, UXStat } from "@vrtmrz/livesync-commonlib/compat/common/types";
vi.mock("@/deps.ts", () => ({}));
import {
createHiddenFileSyncChangeProcessor,
type HiddenFileSyncChangeProcessorDependencies,
} from "./hiddenFileSyncChangeProcessor.ts";
const path = ".obsidian/app.json" as FilePath;
const stat = { ctime: 1, mtime: 2, size: 3, type: "file" } as UXStat;
function fileInfo(): UXFileInfo {
return {
path,
name: "app.json",
isInternal: true,
deleted: false,
body: new Blob(["{}"]),
stat,
} as UXFileInfo;
}
function metadata(): MetaEntry {
return {
_id: "i:app",
_rev: "2-current",
path: `i:${path}`,
type: "plain",
datatype: "plain",
ctime: 1,
mtime: 2,
size: 3,
children: [],
eden: {},
deleted: false,
} as unknown as MetaEntry;
}
function createDependencies(
overrides: Partial<HiddenFileSyncChangeProcessorDependencies> = {}
): HiddenFileSyncChangeProcessorDependencies {
const state = {
fileToStatKey: vi.fn(async () => "2-3"),
getLastProcessedFileKey: vi.fn(() => undefined),
getLastProcessedFileMTime: vi.fn(() => 0),
databaseStateKey: vi.fn(() => "2-3-2-current--1"),
getLastProcessedDatabaseKey: vi.fn(() => undefined),
updateLastProcessedFile: vi.fn(),
updateLastProcessedDatabase: vi.fn(),
updateLastProcessed: vi.fn(),
};
return {
storageAccess: {
statHidden: vi.fn(async () => stat),
},
readFileWithInfo: vi.fn(async () => fileInfo()),
loadDatabaseMetadata: vi.fn(async () => metadata()),
databaseWriteOperations: {
store: vi.fn(async () => true),
delete: vi.fn(async () => true),
},
databaseExtractionOperations: {
extract: vi.fn(async () => true),
},
processedState: state,
conflictResolution: { queue: vi.fn() },
log: vi.fn(),
publishActivity: vi.fn(),
...overrides,
} as HiddenFileSyncChangeProcessorDependencies;
}
describe("HiddenFileSyncChangeProcessor activity and serialisation", () => {
it("publishes admission, processing, and release transitions", async () => {
const dependencies = createDependencies();
const processor = createHiddenFileSyncChangeProcessor(dependencies);
await expect(processor.processStorageChange(path)).resolves.toBe(true);
const publishActivity = vi.mocked(dependencies.publishActivity);
expect(publishActivity.mock.calls).toEqual([
[1, 0],
[1, 1],
[1, 0],
[0, 0],
]);
processor.dispose();
});
it("serialises same-path storage changes while allowing each event to settle", async () => {
let active = 0;
let maximumActive = 0;
let releaseFirst!: () => void;
const firstStarted = new Promise<void>((resolve) => {
const write = resolve;
releaseFirst = write;
});
const dependencies = createDependencies({
databaseWriteOperations: {
store: vi.fn(async () => {
active++;
maximumActive = Math.max(maximumActive, active);
if (active == 1) await firstStarted;
active--;
return true;
}),
delete: vi.fn(async () => true),
},
});
const processor = createHiddenFileSyncChangeProcessor(dependencies);
const first = processor.processStorageChange(path);
await vi.waitFor(() => expect(dependencies.databaseWriteOperations.store).toHaveBeenCalledOnce());
const second = processor.processStorageChange(path);
await new Promise<void>((resolve) => setTimeout(resolve, 0));
expect(dependencies.databaseWriteOperations.store).toHaveBeenCalledOnce();
releaseFirst();
await expect(first).resolves.toBe(true);
await expect(second).resolves.toBe(true);
expect(maximumActive).toBe(1);
expect(dependencies.databaseWriteOperations.store).toHaveBeenCalledTimes(2);
processor.dispose();
});
});
describe("HiddenFileSyncChangeProcessor compatibility settlement", () => {
it("consumes database events when metadata loading fails", async () => {
const error = new Error("metadata unavailable");
const dependencies = createDependencies({
loadDatabaseMetadata: vi.fn(async () => {
throw error;
}),
});
const processor = createHiddenFileSyncChangeProcessor(dependencies);
await expect(processor.processDatabaseChange(path, "[Replication]")).resolves.toBe(true);
expect(dependencies.log).toHaveBeenCalledWith("[Replication] Failed to process hidden file", undefined, undefined);
expect(dependencies.log).toHaveBeenCalledWith(error, expect.any(Number), undefined);
processor.dispose();
});
it("advances the storage marker before a failed database write", async () => {
const dependencies = createDependencies({
databaseWriteOperations: {
store: vi.fn(async () => false),
delete: vi.fn(async () => true),
},
});
const processor = createHiddenFileSyncChangeProcessor(dependencies);
await expect(processor.processStorageChange(path)).resolves.toBe(false);
expect(dependencies.processedState.updateLastProcessedFile).toHaveBeenCalledWith(path, stat);
expect(dependencies.databaseWriteOperations.store).toHaveBeenCalledOnce();
processor.dispose();
});
});
@@ -0,0 +1,426 @@
import {
LOG_LEVEL_INFO,
LOG_LEVEL_VERBOSE,
type DocumentID,
type FilePath,
type FilePathWithPrefix,
type LoadedEntry,
type LOG_LEVEL,
type MetaEntry,
type UXStat,
} from "@vrtmrz/livesync-commonlib/compat/common/types";
import type { LogFunction } from "@vrtmrz/livesync-commonlib/compat/services/lib/logUtils";
import { isInternalMetadata } from "@vrtmrz/livesync-commonlib/compat/common/typeUtils";
import { stripAllPrefixes } from "@vrtmrz/livesync-commonlib/compat/string_and_binary/path";
import { QueueProcessor } from "octagonal-wheels/concurrency/processor";
import type { InternalFileInfo } from "@/common/types.ts";
import { getHiddenFileSyncComparisonMTime } from "./hiddenFileSyncState.ts";
export type HiddenFileSyncConflictPath = FilePath | FilePathWithPrefix;
export type HiddenFileSyncRevisionInfo = {
rev: string;
status: string;
};
export type HiddenFileSyncRevisionHistory = MetaEntry & {
_revs_info?: HiddenFileSyncRevisionInfo[];
};
export type HiddenFileSyncJsonResolution = {
keepRevision?: string;
mergedText?: string;
};
export type HiddenFileSyncConflictDatabase = {
scanConflictedEntries(): AsyncIterable<MetaEntry>;
getDocumentId(path: HiddenFileSyncConflictPath): Promise<DocumentID>;
loadCurrentMetadata(id: DocumentID): Promise<MetaEntry>;
loadConflictingMetadata(id: DocumentID, revision: string): Promise<MetaEntry>;
loadRevisionHistory(id: DocumentID): Promise<HiddenFileSyncRevisionHistory>;
loadRevisionEntry(path: HiddenFileSyncConflictPath, revision: string): Promise<LoadedEntry | false>;
mergeJson(
path: FilePathWithPrefix,
baseRevision: string,
currentRevision: string,
conflictedRevision: string
): Promise<string | false>;
removeRevision(id: DocumentID, revision: string): Promise<unknown>;
deleteRevision(entry: LoadedEntry): Promise<boolean>;
};
export type HiddenFileSyncConflictStorage = {
ensureDirectory(path: FilePath): Promise<void>;
writeFile(path: FilePath, data: string): Promise<UXStat | null>;
triggerEvent(path: FilePath): Promise<void>;
};
export type HiddenFileSyncConflictReconciliation = {
storeFile(file: InternalFileInfo, forceWrite?: boolean): Promise<boolean | undefined>;
extractFile(path: FilePath): Promise<boolean | undefined>;
};
export type HiddenFileSyncConflictInteraction = {
resolveJsonConflict(
path: FilePath,
docs: [LoadedEntry, LoadedEntry],
apply: (resolution: HiddenFileSyncJsonResolution) => Promise<boolean>
): Promise<boolean>;
};
/** Read-only queue counters retained for the real-Obsidian contract tests. */
export type HiddenFileSyncConflictProcessorTestingView = {
readonly remaining: number;
readonly totalRemaining: number;
readonly nowProcessing: number;
};
/** Focused conflict operations exposed through the Hidden File Sync test view. */
export interface HiddenFileSyncConflictTestingView {
resolveAll(): Promise<void>;
resolveJson(docA: LoadedEntry, docB: LoadedEntry): Promise<boolean>;
readonly pendingPaths: readonly HiddenFileSyncConflictPath[];
readonly processor: HiddenFileSyncConflictProcessorTestingView;
}
export type HiddenFileSyncConflictResolutionDependencies = {
database: HiddenFileSyncConflictDatabase;
storage: HiddenFileSyncConflictStorage;
reconciliation: HiddenFileSyncConflictReconciliation;
interaction: HiddenFileSyncConflictInteraction;
shouldOverwrite(path: FilePath): boolean;
log: LogFunction;
};
export interface HiddenFileSyncConflictResolution {
queue(path: HiddenFileSyncConflictPath): void;
resolveAll(): Promise<void>;
resolveJson(docA: LoadedEntry, docB: LoadedEntry): Promise<boolean>;
dispose(): void;
readonly testing: HiddenFileSyncConflictTestingView;
}
type PendingJsonConflict = {
id: DocumentID;
doc: MetaEntry;
path: HiddenFileSyncConflictPath;
revA: string;
revB: string;
};
export function selectHiddenFileSyncRevisionToDelete(
currentDoc: MetaEntry,
currentRevision: string,
conflictedDoc: MetaEntry,
conflictedRevision: string
): string {
const currentMTime = getHiddenFileSyncComparisonMTime(currentDoc, true);
const conflictedMTime = getHiddenFileSyncComparisonMTime(conflictedDoc, true);
// Compatibility: an equal mtime keeps the current leaf and deletes the
// conflicted leaf. A different tie-breaker would alter existing winners.
return currentMTime < conflictedMTime ? currentRevision : conflictedRevision;
}
export function findHiddenFileSyncMergeBase(
revisions: readonly HiddenFileSyncRevisionInfo[] | undefined,
conflictedRevision: string
): string {
const conflictedGeneration = Number(conflictedRevision.split("-")[0]);
// Compatibility question: this is the first available lower generation
// from the current branch, not a proven nearest shared ancestor. Changing
// it requires a separate conflict-history decision.
return (
revisions?.find(({ rev, status }) => status == "available" && Number(rev.split("-")[0]) < conflictedGeneration)
?.rev ?? ""
);
}
class HiddenFileSyncConflictResolutionOwner implements HiddenFileSyncConflictResolution {
private readonly pendingPaths = new Set<HiddenFileSyncConflictPath>();
private readonly processor: QueueProcessor<HiddenFileSyncConflictPath, PendingJsonConflict>;
private disposed = false;
readonly testing: HiddenFileSyncConflictTestingView;
constructor(private readonly dependencies: HiddenFileSyncConflictResolutionDependencies) {
const interactionProcessor = new QueueProcessor<PendingJsonConflict, void>(
async (results) => {
const { id, doc, path, revA, revB } = results[0];
// Compatibility question: these reads intentionally remain
// outside the catch below. A rejected read can leave the path
// pending until another lifecycle event reconstructs the owner.
const docAMerge = await this.dependencies.database.loadRevisionEntry(path, revA);
const docBMerge = await this.dependencies.database.loadRevisionEntry(path, revB);
try {
if (docAMerge != false && docBMerge != false) {
if (await this.resolveJson(docAMerge, docBMerge)) {
this.requeue(path);
} else {
this.finish(path);
}
return;
}
await this.resolveByNewerEntry(id, path, doc, revA, revB);
} catch (error) {
this.finish(path);
throw error;
}
},
{
suspended: false,
batchSize: 1,
concurrentLimit: 1,
delay: 10,
keepResultUntilDownstreamConnected: false,
yieldThreshold: 10,
}
);
this.processor = new QueueProcessor<HiddenFileSyncConflictPath, PendingJsonConflict>(
async (paths) => await this.processPath(paths[0]),
{
suspended: false,
batchSize: 1,
concurrentLimit: 5,
delay: 10,
keepResultUntilDownstreamConnected: true,
yieldThreshold: 10,
pipeTo: interactionProcessor,
}
);
const pendingPaths = () => [...this.pendingPaths];
const processor = this.processor;
const processorView = Object.freeze({
get remaining() {
return processor.remaining;
},
get totalRemaining() {
return processor.totalRemaining;
},
get nowProcessing() {
return processor.nowProcessing;
},
});
this.testing = Object.freeze({
resolveAll: async () => await this.resolveAll(),
resolveJson: async (docA: LoadedEntry, docB: LoadedEntry) => await this.resolveJson(docA, docB),
get pendingPaths() {
return pendingPaths();
},
processor: processorView,
});
}
queue(path: HiddenFileSyncConflictPath): void {
if (this.disposed) return;
// Compatibility: this deliberately deduplicates exact strings only.
// Prefixed and unprefixed forms of one path can therefore coexist.
if (this.pendingPaths.has(path)) return;
this.pendingPaths.add(path);
// Compatibility question: if QueueProcessor throws during this
// synchronous admission, the pending marker is retained. No current
// caller expects enqueue to throw.
this.processor.enqueue(path);
}
async resolveAll(): Promise<void> {
// Creating the iterator and awaiting the completed pipeline remain
// outside the catch. Only iteration failures are logged and swallowed
// by this operation.
const conflicted = this.dependencies.database.scanConflictedEntries();
// Do not suspend ordinary conflict admission during the scan.
// QueueProcessor v2 can lose its resume event when scan completion
// races with the suspended pump, leaving every admitted path pending.
try {
for await (const doc of conflicted) {
if (!("_conflicts" in doc)) continue;
if (isInternalMetadata(doc._id)) {
this.queue(doc.path);
}
}
} catch (error) {
this.log("something went wrong on resolving all conflicted internal files");
this.log(error, LOG_LEVEL_VERBOSE);
}
await this.processor.waitForAllProcessed();
}
async resolveJson(docA: LoadedEntry, docB: LoadedEntry): Promise<boolean> {
this.log("Opening data-merging dialog", LOG_LEVEL_VERBOSE);
const docs: [LoadedEntry, LoadedEntry] = [docA, docB];
const storageFilePath = stripAllPrefixes(docA.path);
const displayFilename = `${storageFilePath}`;
return await this.dependencies.interaction.resolveJsonConflict(
storageFilePath,
docs,
async ({ keepRevision: keep, mergedText: result }) => {
try {
let needFlush = false;
if (!result && !keep) {
this.log(`Skipped merging: ${displayFilename}`);
return false;
}
// Compatibility question: the selected revision is not
// validated against these two documents. An unknown value
// consequently deletes both revisions without writing a
// merged result. The sequential effects are also not
// transactional, so an earlier deletion survives a later
// failure.
for (const doc of docs) {
if (doc._rev != keep) {
if (await this.dependencies.database.deleteRevision(doc)) {
this.log(`Conflicted revision has been deleted: ${displayFilename}`);
needFlush = true;
}
}
}
if (!keep && result) {
await this.dependencies.storage.ensureDirectory(storageFilePath);
const stat = await this.dependencies.storage.writeFile(storageFilePath, result);
if (!stat) {
throw new Error("Stat failed");
}
const mtime = getHiddenFileSyncComparisonMTime(stat);
// Compatibility: interactive merged text forces the
// database write, whereas automatic merge below uses
// the writer's default admission policy.
await this.dependencies.reconciliation.storeFile(
{
path: storageFilePath,
mtime,
ctime: stat.ctime ?? mtime,
size: stat.size ?? 0,
},
true
);
await this.dependencies.storage.triggerEvent(storageFilePath);
this.log(`STORAGE <-- DB:${displayFilename}: written (hidden,merged)`);
}
if (needFlush) {
if (await this.dependencies.reconciliation.extractFile(storageFilePath)) {
this.log(`STORAGE --> DB:${displayFilename}: extracted (hidden,merged)`);
} else {
this.log(`STORAGE --> DB:${displayFilename}: extracted (hidden,merged) Failed`);
}
}
return true;
} catch (error) {
this.log("Could not merge conflicted json");
this.log(error, LOG_LEVEL_VERBOSE);
return false;
}
}
);
}
dispose(): void {
if (this.disposed) return;
this.disposed = true;
// QueueProcessor termination cascades downstream, but cannot cancel an
// already-running database operation or dialogue callback.
this.processor.terminate();
this.pendingPaths.clear();
}
private async processPath(path: HiddenFileSyncConflictPath): Promise<PendingJsonConflict[]> {
try {
const id = await this.dependencies.database.getDocumentId(path);
const doc = await this.dependencies.database.loadCurrentMetadata(id);
if (doc._conflicts === undefined || doc._conflicts.length == 0) {
this.finish(path);
return [];
}
this.log(`Hidden file conflicted:${path}`);
// Compatibility: sorting mutates the loaded Metadata object before
// it is forwarded to the manual-resolution stage.
const conflicts = doc._conflicts.sort((a, b) => Number(a.split("-")[0]) - Number(b.split("-")[0]));
const revA = doc._rev!;
const revB = conflicts[0];
if (path.endsWith(".json")) {
const revisionHistory = await this.dependencies.database.loadRevisionHistory(id);
const commonBase = findHiddenFileSyncMergeBase(revisionHistory._revs_info, revB);
const result = await this.dependencies.database.mergeJson(doc.path, commonBase, revA, revB);
if (result) {
this.log(`Object merge:${path}`, LOG_LEVEL_INFO);
const filename = stripAllPrefixes(path);
await this.dependencies.storage.ensureDirectory(filename);
const stat = await this.dependencies.storage.writeFile(filename, result);
if (!stat) {
throw new Error(`HiddenFileSyncConflictResolution: Failed to stat file ${filename}`);
}
await this.dependencies.reconciliation.storeFile({ path: filename, ...stat });
// Compatibility question: extraction is attempted before
// the conflicted branch is removed, so its conflict guard
// normally refuses it. Requeueing eventually reflects the
// winner; changing the order needs a separate decision.
await this.dependencies.reconciliation.extractFile(filename);
await this.dependencies.database.removeRevision(id, revB);
this.requeue(path);
return [];
}
this.log(`Object merge is not applicable.`, LOG_LEVEL_VERBOSE);
if (this.dependencies.shouldOverwrite(stripAllPrefixes(path))) {
this.log(`Overwrite rule applied for conflicted hidden file: ${path}`, LOG_LEVEL_INFO);
await this.resolveByNewerEntry(id, path, doc, revA, revB);
return [];
}
return [{ path, revA, revB, id, doc }];
}
await this.resolveByNewerEntry(id, path, doc, revA, revB);
return [];
} catch (error) {
this.finish(path);
this.log(`Failed to resolve conflict (Hidden): ${path}`);
this.log(error, LOG_LEVEL_VERBOSE);
return [];
}
}
private async resolveByNewerEntry(
id: DocumentID,
path: HiddenFileSyncConflictPath,
currentDoc: MetaEntry,
currentRevision: string,
conflictedRevision: string
): Promise<void> {
const conflictedDoc = await this.dependencies.database.loadConflictingMetadata(id, conflictedRevision);
const revisionToDelete = selectHiddenFileSyncRevisionToDelete(
currentDoc,
currentRevision,
conflictedDoc,
conflictedRevision
);
// Compatibility: the database result is ignored. The following conflict
// read, rather than the deletion response, decides settlement.
await this.dependencies.database.removeRevision(id, revisionToDelete);
this.log(`Older one has been deleted:${path}`);
const current = await this.dependencies.database.loadCurrentMetadata(id);
if (current._conflicts?.length === 0) {
await this.dependencies.reconciliation.extractFile(stripAllPrefixes(path));
this.finish(path);
} else {
// Compatibility: an absent _conflicts field is not considered
// settled here, although the main path treats it as conflict-free.
this.requeue(path);
}
}
private finish(path: HiddenFileSyncConflictPath): void {
this.pendingPaths.delete(path);
}
private requeue(path: HiddenFileSyncConflictPath): void {
this.finish(path);
this.queue(path);
}
private log(message: unknown, level?: LOG_LEVEL, key?: string): void {
this.dependencies.log(message, level, key);
}
}
export function createHiddenFileSyncConflictResolution(
dependencies: HiddenFileSyncConflictResolutionDependencies
): HiddenFileSyncConflictResolution {
return new HiddenFileSyncConflictResolutionOwner(dependencies);
}
@@ -0,0 +1,422 @@
import { describe, expect, it, vi } from "vitest";
import {
LOG_LEVEL_VERBOSE,
type DocumentID,
type FilePath,
type FilePathWithPrefix,
type LoadedEntry,
type MetaEntry,
type UXStat,
} from "@vrtmrz/livesync-commonlib/compat/common/types";
import {
createHiddenFileSyncConflictResolution,
findHiddenFileSyncMergeBase,
selectHiddenFileSyncRevisionToDelete,
type HiddenFileSyncConflictDatabase,
type HiddenFileSyncConflictInteraction,
type HiddenFileSyncConflictReconciliation,
type HiddenFileSyncConflictResolutionDependencies,
type HiddenFileSyncConflictStorage,
type HiddenFileSyncJsonResolution,
type HiddenFileSyncRevisionHistory,
} from "./hiddenFileSyncConflictResolution.ts";
const path = ".obsidian/plugins/example/data.json" as FilePath;
const prefixedPath = `i:${path}` as FilePathWithPrefix;
const id = "i:hidden-entry-id" as DocumentID;
function metadata(
revision: string,
mtime: number,
overrides: Partial<HiddenFileSyncRevisionHistory> = {}
): HiddenFileSyncRevisionHistory {
return {
_id: id,
_rev: revision,
path: prefixedPath,
type: "plain",
datatype: "plain",
ctime: 10,
mtime,
size: 20,
children: [],
eden: {},
deleted: false,
...overrides,
} as unknown as MetaEntry;
}
function loadedEntry(revision: string, content: string): LoadedEntry {
return {
...metadata(revision, 20),
data: content,
} as LoadedEntry;
}
function entries(...values: MetaEntry[]): AsyncIterable<MetaEntry> {
return {
async *[Symbol.asyncIterator]() {
yield* values;
},
};
}
type DependencyOverrides = {
database?: Partial<HiddenFileSyncConflictDatabase>;
storage?: Partial<HiddenFileSyncConflictStorage>;
reconciliation?: Partial<HiddenFileSyncConflictReconciliation>;
interaction?: Partial<HiddenFileSyncConflictInteraction>;
shouldOverwrite?: HiddenFileSyncConflictResolutionDependencies["shouldOverwrite"];
log?: HiddenFileSyncConflictResolutionDependencies["log"];
};
function createDependencies(overrides: DependencyOverrides = {}): HiddenFileSyncConflictResolutionDependencies {
const database: HiddenFileSyncConflictDatabase = {
scanConflictedEntries: () => entries(),
getDocumentId: vi.fn(async () => id),
loadCurrentMetadata: vi.fn(async () => metadata("1-current", 10)),
loadConflictingMetadata: vi.fn(async () => metadata("1-conflict", 10)),
loadRevisionHistory: vi.fn(async () => metadata("1-current", 10, { _revs_info: [] })),
loadRevisionEntry: vi.fn(async (): Promise<LoadedEntry | false> => false),
mergeJson: vi.fn(async (): Promise<string | false> => false),
removeRevision: vi.fn(async () => true),
deleteRevision: vi.fn(async () => true),
...overrides.database,
};
const storage: HiddenFileSyncConflictStorage = {
ensureDirectory: vi.fn(async () => undefined),
writeFile: vi.fn(async () => null),
triggerEvent: vi.fn(async () => undefined),
...overrides.storage,
};
const reconciliation: HiddenFileSyncConflictReconciliation = {
storeFile: vi.fn(async () => true),
extractFile: vi.fn(async () => true),
...overrides.reconciliation,
};
const interaction: HiddenFileSyncConflictInteraction = {
resolveJsonConflict: vi.fn(async () => false),
...overrides.interaction,
};
return {
database,
storage,
reconciliation,
interaction,
shouldOverwrite: overrides.shouldOverwrite ?? (() => false),
log: overrides.log ?? vi.fn(),
};
}
describe("Hidden File Sync conflict policy", () => {
it("keeps the current revision when both mtimes are equal", () => {
const current = metadata("3-current", 20);
const conflicted = metadata("2-conflict", 20);
expect(selectHiddenFileSyncRevisionToDelete(current, current._rev!, conflicted, conflicted._rev!)).toBe(
conflicted._rev
);
});
it("selects the first available lower-generation revision as the merge base", () => {
expect(
findHiddenFileSyncMergeBase(
[
{ rev: "4-current", status: "available" },
{ rev: "3-missing", status: "missing" },
{ rev: "2-base", status: "available" },
{ rev: "1-older", status: "available" },
],
"3-conflict"
)
).toBe("2-base");
expect(findHiddenFileSyncMergeBase(undefined, "3-conflict")).toBe("");
});
});
describe("Hidden File Sync conflict queue", () => {
it("continues processing queued conflict notifications while a full scan is in progress", async () => {
let releaseScan!: () => void;
let markScanStarted!: () => void;
const scanGate = new Promise<void>((resolve) => {
releaseScan = resolve;
});
const scanStarted = new Promise<void>((resolve) => {
markScanStarted = resolve;
});
const loadCurrentMetadata = vi.fn(async () => metadata("1-current", 10));
const dependencies = createDependencies({
database: {
scanConflictedEntries: () => ({
async *[Symbol.asyncIterator]() {
markScanStarted();
await scanGate;
},
}),
loadCurrentMetadata,
},
});
const resolution = createHiddenFileSyncConflictResolution(dependencies);
const resolvingAll = resolution.resolveAll();
await scanStarted;
// Cross the macrotask boundary which allowed the legacy suspended
// processor to stop before a database notification arrived.
await new Promise<void>((resolve) => setTimeout(resolve, 0));
resolution.queue(prefixedPath);
try {
await vi.waitFor(() => expect(loadCurrentMetadata).toHaveBeenCalledOnce(), {
interval: 10,
timeout: 250,
});
} finally {
releaseScan();
await resolvingAll;
resolution.dispose();
}
});
it("deduplicates exact paths and does not accept work after disposal", async () => {
const loadCurrentMetadata = vi.fn(async () => metadata("1-current", 10));
const dependencies = createDependencies({ database: { loadCurrentMetadata } });
const resolution = createHiddenFileSyncConflictResolution(dependencies);
resolution.queue(prefixedPath);
resolution.queue(prefixedPath);
await resolution.resolveAll();
expect(loadCurrentMetadata).toHaveBeenCalledOnce();
resolution.dispose();
resolution.queue(`i:${path}.other` as FilePathWithPrefix);
expect(loadCurrentMetadata).toHaveBeenCalledOnce();
});
it("retains prefixed and unprefixed path forms as separate compatibility keys", async () => {
const loadCurrentMetadata = vi.fn(async () => metadata("1-current", 10));
const dependencies = createDependencies({ database: { loadCurrentMetadata } });
const resolution = createHiddenFileSyncConflictResolution(dependencies);
resolution.queue(path);
resolution.queue(prefixedPath);
await resolution.resolveAll();
expect(loadCurrentMetadata).toHaveBeenCalledTimes(2);
resolution.dispose();
});
it("deletes the conflicted revision on an mtime tie, then extracts", async () => {
const events: string[] = [];
const current = metadata("3-current", 20, { _conflicts: ["2-conflict"] });
const settled = metadata("3-current", 20, { _conflicts: [] });
const dependencies = createDependencies({
database: {
scanConflictedEntries: () => entries(current),
loadCurrentMetadata: vi.fn().mockResolvedValueOnce(current).mockResolvedValueOnce(settled),
loadConflictingMetadata: vi.fn(async () => metadata("2-conflict", 20)),
removeRevision: vi.fn(async (_id, revision) => {
events.push(`remove:${revision}`);
return true;
}),
},
reconciliation: {
extractFile: vi.fn(async () => {
events.push("extract");
return true;
}),
},
});
const resolution = createHiddenFileSyncConflictResolution(dependencies);
await resolution.resolveAll();
expect(events).toEqual(["remove:2-conflict", "extract"]);
resolution.dispose();
});
it("stores and extracts an automatic merge before removing the conflicted revision", async () => {
const events: string[] = [];
const current = metadata("3-current", 30, { _conflicts: ["2-conflict"] });
const settled = metadata("4-merged", 40, { _conflicts: [] });
const stat = { ctime: 10, mtime: 20, size: 30, type: "file" } as UXStat;
const mergeJson = vi.fn(async () => '{"merged":true}');
const dependencies = createDependencies({
database: {
scanConflictedEntries: () => entries(current),
loadCurrentMetadata: vi.fn().mockResolvedValueOnce(current).mockResolvedValueOnce(settled),
loadRevisionHistory: vi.fn(async () =>
metadata("3-current", 30, {
_revs_info: [
{ rev: "3-current", status: "available" },
{ rev: "1-base", status: "available" },
],
})
),
mergeJson,
removeRevision: vi.fn(async () => {
events.push("remove");
return true;
}),
},
storage: {
ensureDirectory: vi.fn(async () => {
events.push("ensure");
}),
writeFile: vi.fn(async () => {
events.push("write");
return stat;
}),
},
reconciliation: {
storeFile: vi.fn(async () => {
events.push("store");
return true;
}),
extractFile: vi.fn(async () => {
events.push("extract");
return false;
}),
},
});
const resolution = createHiddenFileSyncConflictResolution(dependencies);
await resolution.resolveAll();
expect(mergeJson).toHaveBeenCalledWith(prefixedPath, "1-base", "3-current", "2-conflict");
expect(events).toEqual(["ensure", "write", "store", "extract", "remove"]);
resolution.dispose();
});
});
describe("Hidden File Sync JSON conflict application", () => {
function createJsonResolutionFixture(
jsonResolution: HiddenFileSyncJsonResolution,
deletionResult: boolean | Error = true
) {
const events: string[] = [];
const docA = loadedEntry("3-current", '{"current":true}');
const docB = loadedEntry("2-conflict", '{"conflict":true}');
const deleteRevision = vi.fn(async (entry: LoadedEntry) => {
events.push(`delete:${entry._rev}`);
if (deletionResult instanceof Error) {
if (entry._rev === docB._rev) throw deletionResult;
return true;
}
return deletionResult;
});
const extractFile = vi.fn(async () => {
events.push("extract");
return false;
});
const storeFile = vi.fn(async () => {
events.push("store");
return true;
});
const stat = { ctime: 11, mtime: 21, size: 22, type: "file" } as UXStat;
const log = vi.fn();
const dependencies = createDependencies({
database: { deleteRevision },
storage: {
ensureDirectory: vi.fn(async () => {
events.push("ensure");
}),
writeFile: vi.fn(async () => {
events.push("write");
return stat;
}),
triggerEvent: vi.fn(async () => {
events.push("trigger");
}),
},
reconciliation: { extractFile, storeFile },
interaction: {
resolveJsonConflict: vi.fn(async (_path, _docs, apply) => await apply(jsonResolution)),
},
log,
});
return {
deleteRevision,
docA,
docB,
events,
extractFile,
log,
resolution: createHiddenFileSyncConflictResolution(dependencies),
stat,
storeFile,
};
}
it("returns false without changing data when no resolution is selected", async () => {
const fixture = createJsonResolutionFixture({});
await expect(fixture.resolution.resolveJson(fixture.docA, fixture.docB)).resolves.toBe(false);
expect(fixture.events).toEqual([]);
fixture.resolution.dispose();
});
it("keeps the selected revision but reports success when follow-up extraction fails", async () => {
const fixture = createJsonResolutionFixture({ keepRevision: "3-current" });
await expect(fixture.resolution.resolveJson(fixture.docA, fixture.docB)).resolves.toBe(true);
expect(fixture.deleteRevision).toHaveBeenCalledTimes(1);
expect(fixture.deleteRevision).toHaveBeenCalledWith(fixture.docB);
expect(fixture.events).toEqual([`delete:${fixture.docB._rev}`, "extract"]);
expect(fixture.log).toHaveBeenCalledWith(
`STORAGE --> DB:${path}: extracted (hidden,merged) Failed`,
undefined,
undefined
);
fixture.resolution.dispose();
});
it("deletes both supplied revisions when the selected revision is unknown", async () => {
const fixture = createJsonResolutionFixture({ keepRevision: "9-unknown" });
await expect(fixture.resolution.resolveJson(fixture.docA, fixture.docB)).resolves.toBe(true);
expect(fixture.events).toEqual([`delete:${fixture.docA._rev}`, `delete:${fixture.docB._rev}`, "extract"]);
expect(fixture.storeFile).not.toHaveBeenCalled();
fixture.resolution.dispose();
});
it("deletes both revisions before writing and storing a merged result", async () => {
const fixture = createJsonResolutionFixture({ mergedText: '{"merged":true}' });
await expect(fixture.resolution.resolveJson(fixture.docA, fixture.docB)).resolves.toBe(true);
expect(fixture.storeFile).toHaveBeenCalledWith(
{
path,
ctime: fixture.stat.ctime,
mtime: fixture.stat.mtime,
size: fixture.stat.size,
},
true
);
expect(fixture.events).toEqual([
`delete:${fixture.docA._rev}`,
`delete:${fixture.docB._rev}`,
"ensure",
"write",
"store",
"trigger",
"extract",
]);
fixture.resolution.dispose();
});
it("keeps an earlier successful deletion when a later deletion throws", async () => {
const error = new Error("second deletion failed");
const fixture = createJsonResolutionFixture({ mergedText: '{"merged":true}' }, error);
await expect(fixture.resolution.resolveJson(fixture.docA, fixture.docB)).resolves.toBe(false);
expect(fixture.events).toEqual([`delete:${fixture.docA._rev}`, `delete:${fixture.docB._rev}`]);
expect(fixture.storeFile).not.toHaveBeenCalled();
expect(fixture.log).toHaveBeenCalledWith(error, LOG_LEVEL_VERBOSE, undefined);
fixture.resolution.dispose();
});
});
@@ -0,0 +1,129 @@
import { describe, expect, it, vi } from "vitest";
import {
type DocumentID,
type FilePath,
type FilePathWithPrefix,
type LoadedEntry,
type MetaEntry,
type UXStat,
} from "@vrtmrz/livesync-commonlib/compat/common/types";
import { ICHeader, ICHeaderEnd } from "@/common/types.ts";
vi.mock("@/deps.ts", () => ({}));
vi.mock("./configureHiddenFileSyncMode.ts", () => ({
configureHiddenFileSyncMode: vi.fn(),
}));
import { HiddenFileSyncContext } from "./hiddenFileSyncContext.ts";
describe("HiddenFileSyncContext operation composition", () => {
it("composes the conflict owner from the current database and path capabilities", async () => {
const path = ".obsidian/app.json" as FilePath;
const prefixedPath = `i:${path}` as FilePathWithPrefix;
const metadata = {
_id: "i:hidden-entry-id" as DocumentID,
_rev: "2-current",
path: prefixedPath,
type: "plain",
datatype: "plain",
ctime: 10,
mtime: 20,
size: 20,
children: [],
eden: {},
deleted: false,
_conflicts: [],
} as unknown as MetaEntry;
const findEntries = vi.fn(() => ({
async *[Symbol.asyncIterator]() {
yield metadata;
},
}));
const getRaw = vi.fn(async () => metadata);
const path2id = vi.fn(async () => metadata._id);
const periodicProcessor = { enable: vi.fn(), disable: vi.fn() };
const context = new HiddenFileSyncContext({
createPeriodicProcessor: vi.fn(() => periodicProcessor),
getLocalDatabase: () => ({ findEntries, getRaw }),
path: { path2id },
log: vi.fn(),
publishActivity: vi.fn(),
closeJsonConflictDialogs: vi.fn(),
hideConfigurationChangeNotice: vi.fn(),
} as never);
await context.testing.conflictResolution.resolveAll();
expect(findEntries).toHaveBeenCalledWith(ICHeader, ICHeaderEnd, { conflicts: true });
expect(path2id).toHaveBeenCalledWith(prefixedPath, ICHeader);
expect(getRaw).toHaveBeenCalledWith(metadata._id, { conflicts: true });
context.dispose();
});
it("applies a selected live revision through the narrow repair view", async () => {
const path = ".obsidian/plugins/example/data.json" as FilePath;
const prefixedPath = `i:${path}` as FilePathWithPrefix;
const revision = "2-selected";
const metadata = {
_id: "hidden-entry-id" as DocumentID,
_rev: revision,
path: prefixedPath,
type: "plain",
datatype: "plain",
ctime: 10,
mtime: 20,
size: 20,
children: [],
eden: {},
deleted: false,
} as unknown as MetaEntry;
const loaded = {
...metadata,
data: '{"value":"database"}',
} as LoadedEntry;
const stat = { ctime: 10, mtime: 20, size: 20, type: "file" } as UXStat;
const statHidden = vi.fn<() => Promise<UXStat | null>>().mockResolvedValueOnce(null).mockResolvedValue(stat);
const writeHiddenFileAuto = vi.fn(async () => true);
const getDBEntryFromMeta = vi.fn(async () => loaded);
const fetchEntryMeta = vi.fn(async () => metadata);
const getConflictedRevs = vi.fn(async () => [] as string[]);
const markChangesAreSame = vi.fn();
const periodicProcessor = { enable: vi.fn(), disable: vi.fn() };
const context = new HiddenFileSyncContext({
createPeriodicProcessor: vi.fn(() => periodicProcessor),
isIgnoredByIgnoreFile: vi.fn(async () => false),
databaseFileAccess: {
fetchEntryMeta,
getConflictedRevs,
},
getLocalDatabase: () => ({ getDBEntryFromMeta }),
storageAccess: {
statHidden,
isExistsIncludeHidden: vi.fn(async () => false),
ensureDir: vi.fn(async () => true),
writeHiddenFileAuto,
},
path: {
markChangesAreSame,
unmarkChanges: vi.fn(),
},
getSettings: () => ({ suppressNotifyHiddenFilesChange: true }),
log: vi.fn(),
publishActivity: vi.fn(),
closeJsonConflictDialogs: vi.fn(),
hideConfigurationChangeNotice: vi.fn(),
} as never);
await expect(context.repair.extractInternalFileRevisionFromDatabase(path, revision, true)).resolves.toBe(true);
expect(fetchEntryMeta).toHaveBeenCalledWith(prefixedPath, revision, true);
expect(getConflictedRevs).toHaveBeenCalledWith(prefixedPath);
expect(getDBEntryFromMeta).toHaveBeenCalledWith(metadata, false, true);
expect(writeHiddenFileAuto).toHaveBeenCalledWith(path, '{"value":"database"}', {
ctime: metadata.ctime,
mtime: metadata.mtime,
});
expect(markChangesAreSame).toHaveBeenCalledWith(path, metadata.mtime, stat.mtime);
context.dispose();
});
});
@@ -10,58 +10,58 @@ import { HiddenFileSyncContext } from "./hiddenFileSyncContext.ts";
const PATH = ".obsidian/plugins/example/data.json" as FilePath;
function isTargetFile(context: HiddenFileSyncContext, path: FilePath): Promise<boolean> {
return (context as unknown as { isTargetFile(path: FilePath): Promise<boolean> }).isTargetFile(path);
}
function isTargetFileEligible(context: HiddenFileSyncContext, path: FilePath): Promise<boolean> {
return (context as unknown as { isTargetFileEligible(path: FilePath): Promise<boolean> }).isTargetFileEligible(path);
}
function createHiddenFileSync(
options: { owned?: boolean; ignoredByIgnoreFile?: boolean; patternMatch?: boolean } = {}
) {
const ownsLocalFile = vi.fn(() => options.owned ?? true);
const isIgnoredByIgnoreFile = vi.fn(async () => options.ignoredByIgnoreFile ?? false);
const isTargetFileInPatterns = vi.fn(() => options.patternMatch ?? true);
const patternTest = vi.fn(() => options.patternMatch ?? true);
const parseRegExpSettings = vi.fn(() => ({
ignoreFilter: [],
targetFilter: options.patternMatch === undefined ? [] : [{ test: patternTest }],
}));
const hiddenFileSync = Object.create(HiddenFileSyncContext.prototype) as HiddenFileSyncContext;
Object.assign(hiddenFileSync, {
dependencies: { ownsLocalFile, isIgnoredByIgnoreFile },
isTargetFileInPatterns,
parseRegExpSettings,
});
return { hiddenFileSync, isIgnoredByIgnoreFile, isTargetFileInPatterns, ownsLocalFile };
return { hiddenFileSync, isIgnoredByIgnoreFile, parseRegExpSettings, ownsLocalFile };
}
describe("Hidden File Sync local-path admission", () => {
it("checks composition ownership before Hidden File Sync filters", async () => {
const { hiddenFileSync, isTargetFileInPatterns, ownsLocalFile } = createHiddenFileSync({ owned: false });
const { hiddenFileSync, parseRegExpSettings, ownsLocalFile } = createHiddenFileSync({ owned: false });
await expect(hiddenFileSync.isTargetFile(PATH)).resolves.toBe(false);
await expect(isTargetFile(hiddenFileSync, PATH)).resolves.toBe(false);
expect(ownsLocalFile).toHaveBeenCalledWith(PATH);
expect(isTargetFileInPatterns).not.toHaveBeenCalled();
expect(parseRegExpSettings).not.toHaveBeenCalled();
});
it("keeps target patterns and ignore-file results as Hidden File Sync eligibility", async () => {
const patternExcluded = createHiddenFileSync({ patternMatch: false });
await expect(patternExcluded.hiddenFileSync.isTargetFile(PATH)).resolves.toBe(false);
await expect(isTargetFile(patternExcluded.hiddenFileSync, PATH)).resolves.toBe(false);
expect(patternExcluded.isIgnoredByIgnoreFile).not.toHaveBeenCalled();
const ignoreFileExcluded = createHiddenFileSync({ ignoredByIgnoreFile: true });
await expect(ignoreFileExcluded.hiddenFileSync.isTargetFile(PATH)).resolves.toBe(false);
await expect(isTargetFile(ignoreFileExcluded.hiddenFileSync, PATH)).resolves.toBe(false);
expect(ignoreFileExcluded.isIgnoredByIgnoreFile).toHaveBeenCalledWith(PATH);
const admitted = createHiddenFileSync();
await expect(admitted.hiddenFileSync.isTargetFile(PATH)).resolves.toBe(true);
await expect(isTargetFile(admitted.hiddenFileSync, PATH)).resolves.toBe(true);
});
it("exposes eligibility without consulting the composition owner", async () => {
it("evaluates eligibility without consulting the composition owner", async () => {
const { hiddenFileSync, ownsLocalFile } = createHiddenFileSync({ owned: false });
await expect(hiddenFileSync.isTargetFileEligible(PATH)).resolves.toBe(true);
await expect(isTargetFileEligible(hiddenFileSync, PATH)).resolves.toBe(true);
expect(ownsLocalFile).not.toHaveBeenCalled();
});
});
describe("compatibility: Hidden File Sync path shape", () => {
const hiddenFileSync = Object.create(HiddenFileSyncContext.prototype) as HiddenFileSyncContext;
it.each([
[".obsidian/app.json", true],
[".trash/app.json", false],
["notes/app.json", false],
])("recognises %s as a Hidden File Sync path=%s", (path, expected) => {
expect(hiddenFileSync.isHiddenFileSyncHandlingPath(path as FilePath)).toBe(expected);
});
});
@@ -1,5 +1,4 @@
import { describe, expect, it, vi } from "vitest";
import type { FilePath } from "@vrtmrz/livesync-commonlib/compat/common/types";
vi.mock("@/deps.ts", () => ({}));
vi.mock("./configureHiddenFileSyncMode.ts", () => ({
@@ -8,12 +7,29 @@ vi.mock("./configureHiddenFileSyncMode.ts", () => ({
import { HiddenFileSyncContext } from "./hiddenFileSyncContext.ts";
function createContext() {
function getProcessedState(context: HiddenFileSyncContext): unknown {
return (context as unknown as { readonly processedState: unknown }).processedState;
}
function getPrivate<T>(context: HiddenFileSyncContext, key: string): T {
return (context as unknown as Record<string, T>)[key];
}
function createContext(processedFiles = new Map()) {
const periodicProcessor = { enable: vi.fn(), disable: vi.fn() };
const publishActivity = vi.fn();
const hideConfigurationChangeNotice = vi.fn();
const keyValueDatabase = {
get: vi.fn(async (key: IDBValidKey) => {
if (key == "hidden-file-lastProcessed") return processedFiles;
return new Map();
}),
};
const context = new HiddenFileSyncContext({
createPeriodicProcessor: vi.fn(() => periodicProcessor),
getKeyValueDatabase: () => keyValueDatabase,
getSettings: () => ({ syncInternalFiles: true }),
log: vi.fn(),
publishActivity,
closeJsonConflictDialogs: vi.fn(),
hideConfigurationChangeNotice,
@@ -22,71 +38,41 @@ function createContext() {
}
describe("HiddenFileSyncContext state ownership", () => {
it("owns queues, caches, concurrency controls, and processors per context instance", () => {
it("owns caches, concurrency controls, processors, and conflict owners per context instance", () => {
const first = createContext();
const second = createContext();
first.context.pendingConflictChecks.add("i:.obsidian/first.json" as never);
first.context.queuedNotificationFiles.add(".obsidian/plugins/first");
first.context.cacheFileRegExps.set("first", []);
getPrivate<Set<string>>(first.context, "queuedNotificationFiles").add(".obsidian/plugins/first");
getPrivate<Map<string, unknown[]>>(first.context, "cacheFileRegExps").set("first", []);
expect(second.context.pendingConflictChecks).toEqual(new Set());
expect(second.context.queuedNotificationFiles).toEqual(new Set());
expect(second.context.cacheFileRegExps).toEqual(new Map());
expect(first.context.conflictResolutionProcessor).not.toBe(second.context.conflictResolutionProcessor);
expect(first.context.semaphore).not.toBe(second.context.semaphore);
expect(first.context.periodicInternalFileScanProcessor).toBe(first.periodicProcessor);
expect(second.context.periodicInternalFileScanProcessor).toBe(second.periodicProcessor);
expect(getPrivate<Set<string>>(second.context, "queuedNotificationFiles").size).toBe(0);
expect(getPrivate<Map<string, unknown[]>>(second.context, "cacheFileRegExps")).toEqual(new Map());
expect(first.context.testing.conflictResolution).not.toBe(second.context.testing.conflictResolution);
expect(getProcessedState(first.context)).not.toBe(getProcessedState(second.context));
expect(getPrivate<unknown>(first.context, "changeProcessor")).not.toBe(
getPrivate<unknown>(second.context, "changeProcessor")
);
expect(getPrivate<unknown>(first.context, "periodicInternalFileScanProcessor")).toBe(first.periodicProcessor);
expect(getPrivate<unknown>(second.context, "periodicInternalFileScanProcessor")).toBe(second.periodicProcessor);
first.context.dispose();
second.context.dispose();
});
it("publishes instance-owned event and processing counts at each transition", async () => {
const { context, publishActivity } = createContext();
const path = ".obsidian/app.json" as FilePath;
await context.serializedForEvent(path, async () => {
expect(publishActivity).toHaveBeenLastCalledWith(1, 1);
});
expect(publishActivity.mock.calls).toEqual([
[1, 0],
[1, 1],
[1, 0],
[0, 0],
]);
context.dispose();
});
it.each([
[new Map(), true],
[new Map([[".obsidian/app.json", "1-2-3"]]), false],
])(
"preserves start-up scan notice selection for the processed-file cache",
async (processedFiles, forcedNotice) => {
const { context } = createContext();
const performStartupScan = vi.fn(async () => undefined);
const keyValueDatabase = {
get: vi.fn(async (key: IDBValidKey) => {
if (key == "hidden-file-lastProcessed") return processedFiles;
return new Map();
}),
};
Object.assign(context, {
dependencies: {
createPeriodicProcessor: vi.fn(),
getKeyValueDatabase: () => keyValueDatabase,
getSettings: () => ({ syncInternalFiles: true }),
log: vi.fn(),
},
performStartupScan,
});
const { context } = createContext(processedFiles);
const applyOfflineChanges = vi.fn(async () => undefined);
context.applyOfflineChanges = applyOfflineChanges;
await context._everyOnDatabaseInitialized(false);
await context.serviceHandlers.onDatabaseInitialised(false);
expect(performStartupScan).toHaveBeenCalledWith(forcedNotice);
context.conflictResolutionProcessor.terminate();
expect(applyOfflineChanges).toHaveBeenCalledWith(forcedNotice);
context.dispose();
}
);
});
File diff suppressed because it is too large Load Diff
@@ -1,12 +1,5 @@
import { describe, expect, it, vi } from "vitest";
import {
type DocumentID,
LOG_LEVEL_NOTICE,
type FilePath,
type FilePathWithPrefix,
type MetaEntry,
type UXFileInfo,
} from "@vrtmrz/livesync-commonlib/compat/common/types";
import { LOG_LEVEL_NOTICE } from "@vrtmrz/livesync-commonlib/compat/common/types";
vi.mock("@/deps.ts", () => ({}));
vi.mock("./configureHiddenFileSyncMode.ts", () => ({
@@ -16,81 +9,27 @@ vi.mock("./configureHiddenFileSyncMode.ts", () => ({
import { HiddenFileSyncContext } from "./hiddenFileSyncContext.ts";
import { configureHiddenFileSyncMode } from "./configureHiddenFileSyncMode.ts";
function createHiddenRevisionOperation() {
const path = ".obsidian/plugins/example/data.json" as FilePath;
const file = {
path,
name: "data.json",
isInternal: true,
body: new Blob(['{"value":"vault"}']),
stat: {
ctime: 1,
mtime: 2,
size: 17,
type: "file",
},
} as UXFileInfo;
const selected = {
_id: "i:example" as DocumentID,
_rev: "2-selected",
path: `i:${path}` as FilePathWithPrefix,
ctime: 1,
mtime: 2,
size: 17,
type: "plain",
datatype: "plain",
children: [],
eden: {},
deleted: false,
} as MetaEntry;
const winner = {
...selected,
_rev: "3-winner",
} as MetaEntry;
const databaseFileAccess = {
fetchEntryMeta: vi.fn(async (_path: unknown, revision?: string) =>
revision === selected._rev ? selected : winner
),
getConflictedRevs: vi.fn(async () => [selected._rev]),
fetchEntryFromMeta: vi.fn(async () => ({ ...selected, data: '{"value":"database"}' })),
storeWithBaseRevision: vi.fn(async () => "3-vault-child"),
};
const hiddenFileSync = Object.create(HiddenFileSyncContext.prototype) as HiddenFileSyncContext;
Object.assign(hiddenFileSync, {
dependencies: {
databaseFileAccess,
isIgnoredByIgnoreFile: vi.fn(async () => false),
},
loadFileWithInfo: vi.fn(async () => file),
updateLastProcessed: vi.fn(),
_log: vi.fn(),
});
return {
hiddenFileSync,
path,
file,
selected,
winner,
databaseFileAccess,
};
function callPrivate<T extends (...args: never[]) => unknown>(context: HiddenFileSyncContext, key: string): T {
const operation = (context as unknown as Record<string, T>)[key];
return operation.bind(context) as T;
}
describe("HiddenFileSyncContext configuration-change notices", () => {
it("releases processors, transient queues, the pattern cache, activity, and the host Notice effect", () => {
const periodicInternalFileScanProcessor = { disable: vi.fn() };
const conflictResolutionProcessor = { terminate: vi.fn() };
const pendingConflictChecks = new Set(["i:.obsidian/example.json"]);
const conflictResolution = { dispose: vi.fn() };
const queuedNotificationFiles = new Set([".obsidian/plugins/example"]);
const cacheFileRegExps = new Map([["patterns", []]]);
const publishActivity = vi.fn();
const changeProcessor = { dispose: vi.fn() };
const hideConfigurationChangeNotice = vi.fn();
const closeJsonConflictDialogs = vi.fn();
const hiddenFileSync = Object.create(HiddenFileSyncContext.prototype) as HiddenFileSyncContext;
Object.assign(hiddenFileSync, {
dependencies: { publishActivity, hideConfigurationChangeNotice, closeJsonConflictDialogs },
periodicInternalFileScanProcessor,
conflictResolutionProcessor,
pendingConflictChecks,
conflictResolution,
changeProcessor,
queuedNotificationFiles,
cacheFileRegExps,
eventCount: 4,
@@ -101,11 +40,10 @@ describe("HiddenFileSyncContext configuration-change notices", () => {
hiddenFileSync.dispose();
expect(periodicInternalFileScanProcessor.disable).toHaveBeenCalledOnce();
expect(conflictResolutionProcessor.terminate).toHaveBeenCalledOnce();
expect(pendingConflictChecks.size).toBe(0);
expect(conflictResolution.dispose).toHaveBeenCalledOnce();
expect(queuedNotificationFiles.size).toBe(0);
expect(cacheFileRegExps.size).toBe(0);
expect(publishActivity).toHaveBeenCalledWith(0, 0);
expect(changeProcessor.dispose).toHaveBeenCalledOnce();
expect(closeJsonConflictDialogs).toHaveBeenCalledOnce();
expect(hideConfigurationChangeNotice).toHaveBeenCalledOnce();
});
@@ -122,7 +60,7 @@ describe("HiddenFileSyncContext configuration-change notices", () => {
_isMainSuspended: vi.fn(() => false),
});
expect(hiddenFileSync.isReady()).toBe(false);
expect(callPrivate<() => boolean>(hiddenFileSync, "isReady")()).toBe(false);
});
it("settles one batch of changed folders through the host notification effect", () => {
@@ -133,14 +71,16 @@ describe("HiddenFileSyncContext configuration-change notices", () => {
queuedNotificationFiles: new Set([".obsidian/plugins/alpha", ".obsidian/plugins/beta", ".obsidian"]),
});
hiddenFileSync.notifyConfigChange();
callPrivate<() => void>(hiddenFileSync, "notifyConfigChange")();
expect(showConfigurationChangeNotice).toHaveBeenCalledWith([
".obsidian/plugins/alpha",
".obsidian/plugins/beta",
".obsidian",
]);
expect(hiddenFileSync.queuedNotificationFiles.size).toBe(0);
expect((hiddenFileSync as unknown as { queuedNotificationFiles: Set<string> }).queuedNotificationFiles.size).toBe(
0
);
});
it("keeps subordinate initialisation phases below Notice level so one progress Notice owns the scan", async () => {
@@ -200,7 +140,7 @@ describe("HiddenFileSyncContext configuration-change notices", () => {
_log: log,
});
await hiddenFileSync.configureHiddenFileSync("MERGE");
await callPrivate<(mode: "MERGE") => Promise<void>>(hiddenFileSync, "configureHiddenFileSync")("MERGE");
expect(createProgress).toHaveBeenCalledWith("[⚙ Initialise]\n", LOG_LEVEL_NOTICE);
expect(events[0]).toBe("progress:Preparing Hidden File Sync...");
@@ -232,119 +172,10 @@ describe("HiddenFileSyncContext configuration-change notices", () => {
_log: vi.fn(),
});
await expect(hiddenFileSync.configureHiddenFileSync("MERGE")).rejects.toBe(error);
await expect(
callPrivate<(mode: "MERGE") => Promise<void>>(hiddenFileSync, "configureHiddenFileSync")("MERGE")
).rejects.toBe(error);
expect(progress.done).toHaveBeenCalledWith("Failed");
});
});
describe("HiddenFileSyncContext reconciliation settlement", () => {
it("compatibility: consumes a selected database file even when reading its Metadata fails", async () => {
const error = new Error("metadata unavailable");
const getDBEntryMeta = vi.fn(async () => {
throw error;
});
const log = vi.fn();
const hiddenFileSync = Object.create(HiddenFileSyncContext.prototype) as HiddenFileSyncContext;
Object.assign(hiddenFileSync, {
dependencies: {
getLocalDatabase: () => ({ getDBEntryMeta }),
log,
},
serializedForEvent: vi.fn(async (_path: FilePath, operation: () => Promise<boolean>) => await operation()),
});
await expect(
hiddenFileSync.trackDatabaseFileModification(".obsidian/app.json" as FilePath, "[Replication]")
).resolves.toBe(true);
expect(log).toHaveBeenCalledWith("[Replication] Failed to process hidden file", undefined, undefined);
expect(log).toHaveBeenCalledWith(error, expect.any(Number), undefined);
});
});
describe("HiddenFileSyncContext exact revision repair operations", () => {
it("stores the current hidden Vault file as a child of the selected live revision", async () => {
const { hiddenFileSync, file, selected, databaseFileAccess } = createHiddenRevisionOperation();
await expect(hiddenFileSync.storeInternalFileToDatabaseWithBaseRevision(file, selected._rev!)).resolves.toBe(
true
);
expect(databaseFileAccess.storeWithBaseRevision).toHaveBeenCalledWith(
expect.objectContaining({
path: file.path,
body: file.body,
isInternal: true,
}),
selected._rev,
true
);
expect(hiddenFileSync.updateLastProcessed).toHaveBeenCalledWith(
file.path,
expect.objectContaining({ _rev: "3-vault-child" }),
file.stat
);
});
it("refuses to extend a hidden-file revision which is no longer live", async () => {
const { hiddenFileSync, file, selected, databaseFileAccess } = createHiddenRevisionOperation();
databaseFileAccess.getConflictedRevs.mockResolvedValue([]);
await expect(hiddenFileSync.storeInternalFileToDatabaseWithBaseRevision(file, selected._rev!)).resolves.toBe(
false
);
expect(databaseFileAccess.storeWithBaseRevision).not.toHaveBeenCalled();
expect(hiddenFileSync.updateLastProcessed).not.toHaveBeenCalled();
});
it("does not create a hidden-file child when asked only to mark a revision which differs from the Vault", async () => {
const { hiddenFileSync, file, selected, databaseFileAccess } = createHiddenRevisionOperation();
await expect(
hiddenFileSync.storeInternalFileToDatabaseWithBaseRevision(file, selected._rev!, false)
).resolves.toBe(false);
expect(databaseFileAccess.storeWithBaseRevision).not.toHaveBeenCalled();
expect(hiddenFileSync.updateLastProcessed).not.toHaveBeenCalled();
});
it("marks a matching hidden-file revision without creating a child", async () => {
const { hiddenFileSync, file, selected, databaseFileAccess } = createHiddenRevisionOperation();
databaseFileAccess.fetchEntryFromMeta.mockResolvedValue({
...selected,
data: '{"value":"vault"}',
});
await expect(
hiddenFileSync.storeInternalFileToDatabaseWithBaseRevision(file, selected._rev!, false)
).resolves.toBe(true);
expect(databaseFileAccess.storeWithBaseRevision).not.toHaveBeenCalled();
expect(hiddenFileSync.updateLastProcessed).toHaveBeenCalledWith(file.path, selected, file.stat);
});
it("applies the selected live hidden-file revision through the existing extraction path", async () => {
const { hiddenFileSync, path, selected } = createHiddenRevisionOperation();
const extract = vi.fn(async () => true);
hiddenFileSync.extractInternalFileFromDatabase = extract;
await expect(hiddenFileSync.extractInternalFileRevisionFromDatabase(path, selected._rev!, true)).resolves.toBe(
true
);
expect(extract).toHaveBeenCalledWith(path, true, undefined, true, false, true, selected._rev);
});
it("does not apply a hidden-file revision which ceased to be live", async () => {
const { hiddenFileSync, path, selected, databaseFileAccess } = createHiddenRevisionOperation();
databaseFileAccess.getConflictedRevs.mockResolvedValue([]);
await expect(hiddenFileSync.extractInternalFileRevisionFromDatabase(path, selected._rev!, true)).resolves.toBe(
false
);
expect(databaseFileAccess.fetchEntryFromMeta).not.toHaveBeenCalled();
});
});
@@ -0,0 +1,223 @@
import {
LOG_LEVEL_INFO,
LOG_LEVEL_VERBOSE,
type FilePath,
type FilePathWithPrefix,
type LoadedEntry,
type LOG_LEVEL,
type MetaEntry,
type UXStat,
} from "@vrtmrz/livesync-commonlib/compat/common/types";
import { compareMTime, TARGET_IS_NEW } from "@/common/utils.ts";
import type { LogFunction } from "@vrtmrz/livesync-commonlib/compat/services/lib/logUtils";
import { addPrefix } from "@vrtmrz/livesync-commonlib/compat/string_and_binary/path";
import { ICHeader } from "@/common/types.ts";
import { getHiddenFileSyncComparisonMTime } from "./hiddenFileSyncState.ts";
import type { HiddenFileSyncRemovalResult } from "./hiddenFileSyncStorage.ts";
import {
serialiseHiddenFileOperation,
type HiddenFileSyncFileSerialisationDependencies,
} from "./hiddenFileSyncFileOperations.ts";
export type HiddenFileSyncDatabaseExtractionOptions = Readonly<{
force?: boolean;
metaEntry?: MetaEntry | LoadedEntry;
preventDoubleProcess?: boolean;
onlyNew?: boolean;
includeDeletion?: boolean;
requiredLiveRevision?: string;
}>;
type HiddenFileSyncDatabaseExtractionStorageDependencies = {
statStorageFile(path: FilePath): Promise<UXStat | null>;
writeStorageFile(path: FilePath, entry: LoadedEntry, force: boolean): Promise<false | UXStat>;
deleteStorageFile(path: FilePath): Promise<HiddenFileSyncRemovalResult>;
};
type HiddenFileSyncDatabaseExtractionReadDependencies = {
loadDatabaseMetadata(path: FilePathWithPrefix): Promise<MetaEntry | LoadedEntry | false>;
loadLiveRevision(path: FilePathWithPrefix, revision: string): Promise<MetaEntry | false>;
loadDatabaseEntry(entry: MetaEntry | LoadedEntry): Promise<LoadedEntry | false>;
};
export type HiddenFileSyncDatabaseExtractionProcessedState = {
databaseStateKey(entry: MetaEntry | LoadedEntry): string;
getLastProcessedDatabaseKey(path: FilePath): string | undefined;
getLastProcessedFileMTime(path: FilePath): number;
updateLastProcessedDatabase(path: FilePath, entry: string | MetaEntry | LoadedEntry): void;
updateLastProcessedFile(path: FilePath, storageFile: string | UXStat): void;
updateLastProcessed(path: FilePath, databaseEntry: MetaEntry | LoadedEntry, storageFile: UXStat): void;
updateLastProcessedDeletion(path: FilePath, databaseEntry: MetaEntry | LoadedEntry | false): void;
};
export type HiddenFileSyncDatabaseExtractionDependencies = HiddenFileSyncFileSerialisationDependencies &
HiddenFileSyncDatabaseExtractionStorageDependencies &
HiddenFileSyncDatabaseExtractionReadDependencies & {
isIgnoredByIgnoreFile(path: string): Promise<boolean>;
queueNotification(path: FilePath): void;
processedState: HiddenFileSyncDatabaseExtractionProcessedState;
log: LogFunction;
};
export type HiddenFileSyncDatabaseExtractionOperations = {
extract(path: FilePath, options?: HiddenFileSyncDatabaseExtractionOptions): Promise<boolean | undefined>;
extractRevision(path: FilePath, revision: string, force?: boolean): Promise<boolean>;
};
function log(
dependencies: Pick<HiddenFileSyncDatabaseExtractionDependencies, "log">,
message: unknown,
level?: LOG_LEVEL,
key?: string
): void {
dependencies.log(message, level, key);
}
export async function extractHiddenFileFromDatabase(
dependencies: HiddenFileSyncDatabaseExtractionDependencies,
storageFilePath: FilePath,
options: HiddenFileSyncDatabaseExtractionOptions = {}
): Promise<boolean | undefined> {
const {
force = false,
metaEntry,
preventDoubleProcess = true,
onlyNew = false,
includeDeletion = true,
requiredLiveRevision,
} = options;
const prefixedFileName = addPrefix(storageFilePath, ICHeader);
// Compatibility: admission happens outside the per-file lock, so ignore
// policy errors propagate instead of becoming a false extraction result.
if (await dependencies.isIgnoredByIgnoreFile(storageFilePath)) {
return undefined;
}
return await serialiseHiddenFileOperation(dependencies, prefixedFileName, async () => {
try {
// A caller-supplied Metadata entry is trusted as-is. It can change
// before this lock is acquired; only exact-revision repair performs
// an in-lock liveness check.
const metaOnDatabase = requiredLiveRevision
? await dependencies.loadLiveRevision(prefixedFileName, requiredLiveRevision)
: metaEntry
? metaEntry
: await dependencies.loadDatabaseMetadata(prefixedFileName);
// Compatibility: the exact-revision loader validates revision-tree
// membership but normally returns a leaf without `_conflicts`.
// Repair can therefore apply one live branch while ordinary
// reflection remains blocked by conflicted winning Metadata.
if (metaOnDatabase === false) {
throw new Error(`File not found on database.:${storageFilePath}`);
}
if (metaOnDatabase._conflicts?.length) {
log(
dependencies,
`Hidden file ${storageFilePath} has conflicted revisions, to keep in safe, writing to storage has been prevented`,
LOG_LEVEL_INFO
);
return false;
}
if (preventDoubleProcess) {
const key = dependencies.processedState.databaseStateKey(metaOnDatabase);
if (dependencies.processedState.getLastProcessedDatabaseKey(storageFilePath) == key && !force) {
// Compatibility question: the force suffix is unreachable
// because this branch is entered only when force is false.
log(
dependencies,
`STORAGE <-- DB: ${storageFilePath}: skipped (hidden, overwrite${force ? ", force" : ""}) (Previously processed)`
);
return undefined;
}
}
if (onlyNew) {
const databaseMTime = getHiddenFileSyncComparisonMTime(metaOnDatabase, includeDeletion);
const storageStat = await dependencies.statStorageFile(storageFilePath);
const storageMTimeActual = storageStat?.mtime ?? 0;
const storageMTime =
storageMTimeActual == 0
? dependencies.processedState.getLastProcessedFileMTime(storageFilePath)
: storageMTimeActual;
const difference = compareMTime(storageMTime, databaseMTime);
if (difference != TARGET_IS_NEW) {
log(
dependencies,
`STORAGE <-- DB: ${storageFilePath}: skipped (hidden, overwrite${force ? ", force" : ""}) (Not new)`
);
// Compatibility: a declined candidate is settled as
// processed, including a deletion excluded by
// includeDeletion. This prevents later scans retrying it.
dependencies.processedState.updateLastProcessedDatabase(storageFilePath, metaOnDatabase);
if (storageStat) dependencies.processedState.updateLastProcessedFile(storageFilePath, storageStat);
return undefined;
}
}
const deleted = metaOnDatabase.deleted || metaOnDatabase._deleted || false;
if (deleted) {
const result = await dependencies.deleteStorageFile(storageFilePath);
if (result == "OK") {
dependencies.processedState.updateLastProcessedDeletion(storageFilePath, metaOnDatabase);
return true;
}
if (result == "ALREADY") {
// Compatibility question: an already absent file updates
// only the database key. It does not record the missing
// storage key or call unmarkChanges through deletion state.
dependencies.processedState.updateLastProcessedDatabase(storageFilePath, metaOnDatabase);
return true;
}
return false;
}
const fileOnDatabase = await dependencies.loadDatabaseEntry(metaOnDatabase);
if (fileOnDatabase === false) {
throw new Error(`Failed to read file from database:${storageFilePath}`);
}
const resultStat = await dependencies.writeStorageFile(storageFilePath, fileOnDatabase, force);
if (resultStat) {
// Compatibility question: the storage writer also returns the
// existing stat when content is unchanged. That no-op still
// settles state and queues a configuration notification here.
dependencies.processedState.updateLastProcessed(storageFilePath, metaOnDatabase, resultStat);
dependencies.queueNotification(storageFilePath);
log(
dependencies,
`STORAGE <-- DB: ${storageFilePath}: written (hidden, overwrite${force ? ", force" : ""}) Done`
);
return true;
}
return false;
} catch (error) {
log(
dependencies,
`STORAGE <-- DB: ${storageFilePath}: written (hidden, overwrite${force ? ", force" : ""}) Failed`
);
log(dependencies, error, LOG_LEVEL_VERBOSE);
return false;
}
});
}
export async function extractHiddenFileRevisionFromDatabase(
dependencies: HiddenFileSyncDatabaseExtractionDependencies,
storageFilePath: FilePath,
revision: string,
force = false
): Promise<boolean> {
return Boolean(
await extractHiddenFileFromDatabase(dependencies, storageFilePath, {
force,
requiredLiveRevision: revision,
})
);
}
export function createHiddenFileSyncDatabaseExtractionOperations(
dependencies: HiddenFileSyncDatabaseExtractionDependencies
): HiddenFileSyncDatabaseExtractionOperations {
return Object.freeze({
extract: async (path, options) => await extractHiddenFileFromDatabase(dependencies, path, options),
extractRevision: async (path, revision, force) =>
await extractHiddenFileRevisionFromDatabase(dependencies, path, revision, force),
});
}
@@ -0,0 +1,366 @@
import { describe, expect, it, vi } from "vitest";
import {
LOG_LEVEL_INFO,
LOG_LEVEL_VERBOSE,
type DocumentID,
type FilePath,
type FilePathWithPrefix,
type LoadedEntry,
type MetaEntry,
type UXStat,
} from "@vrtmrz/livesync-commonlib/compat/common/types";
import type { HiddenFileSyncRemovalResult } from "./hiddenFileSyncStorage.ts";
vi.mock("@/deps.ts", () => ({}));
import {
createHiddenFileSyncDatabaseExtractionOperations,
extractHiddenFileFromDatabase,
extractHiddenFileRevisionFromDatabase,
type HiddenFileSyncDatabaseExtractionDependencies,
} from "./hiddenFileSyncDatabaseExtractionOperations.ts";
import { toHiddenFileSyncDatabaseStateKey } from "./hiddenFileSyncState.ts";
const path = ".obsidian/plugins/example/data.json" as FilePath;
const prefixedPath = `i:${path}` as FilePathWithPrefix;
const id = "hidden-entry-id" as DocumentID;
function metadata(overrides: Partial<MetaEntry> = {}): MetaEntry {
return {
_id: id,
_rev: "2-current",
path: prefixedPath,
type: "plain",
datatype: "plain",
ctime: 10,
mtime: 20,
size: 20,
children: [],
eden: {},
deleted: false,
...overrides,
} as unknown as MetaEntry;
}
function loadedEntry(entry = metadata()): LoadedEntry {
return {
...entry,
data: '{"value":"database"}',
} as LoadedEntry;
}
function storageStat(mtime = 20): UXStat {
return {
ctime: 11,
mtime,
size: 20,
type: "file",
};
}
function createDependencies(entry = metadata()) {
const events: string[] = [];
const serialiseFileOperation = vi.fn(async (_key: string, operation: () => Promise<unknown>) => {
events.push("lock:start");
try {
return await operation();
} finally {
events.push("lock:end");
}
});
const log = vi.fn();
const isIgnoredByIgnoreFile = vi.fn(async () => false);
const loadDatabaseMetadata = vi.fn(async () => entry as MetaEntry | false);
const loadLiveRevision = vi.fn(async (_path: FilePathWithPrefix, revision: string) =>
revision === entry._rev ? (entry as MetaEntry) : false
);
const loadDatabaseEntry = vi.fn(async () => loadedEntry(entry) as LoadedEntry | false);
const statStorageFile = vi.fn(async () => storageStat() as UXStat | null);
const writeStorageFile = vi.fn(async () => {
events.push("storage:write");
return storageStat(21) as UXStat | false;
});
const deleteStorageFile = vi.fn(async (): Promise<HiddenFileSyncRemovalResult> => {
events.push("storage:delete");
return "OK" as const;
});
const getLastProcessedDatabaseKey = vi.fn(() => undefined as string | undefined);
const getLastProcessedFileMTime = vi.fn(() => 0);
const updateLastProcessed = vi.fn(() => events.push("state:file"));
const updateLastProcessedDatabase = vi.fn(() => events.push("state:database"));
const updateLastProcessedFile = vi.fn(() => events.push("state:storage"));
const updateLastProcessedDeletion = vi.fn(() => events.push("state:deletion"));
const processedState = {
databaseStateKey: toHiddenFileSyncDatabaseStateKey,
getLastProcessedDatabaseKey,
getLastProcessedFileMTime,
updateLastProcessed,
updateLastProcessedDatabase,
updateLastProcessedFile,
updateLastProcessedDeletion,
};
const queueNotification = vi.fn(() => events.push("notification"));
const dependencies = {
serialiseFileOperation,
isIgnoredByIgnoreFile,
loadDatabaseMetadata,
loadLiveRevision,
loadDatabaseEntry,
statStorageFile,
writeStorageFile,
deleteStorageFile,
processedState,
queueNotification,
log,
} as HiddenFileSyncDatabaseExtractionDependencies;
return {
deleteStorageFile,
dependencies,
entry,
events,
getLastProcessedDatabaseKey,
getLastProcessedFileMTime,
isIgnoredByIgnoreFile,
loadDatabaseEntry,
loadDatabaseMetadata,
loadLiveRevision,
log,
queueNotification,
serialiseFileOperation,
statStorageFile,
updateLastProcessed,
updateLastProcessedDatabase,
updateLastProcessedDeletion,
updateLastProcessedFile,
writeStorageFile,
};
}
describe("hidden-file database-to-storage admission", () => {
it("returns undefined for an ignored path without taking the file lock", async () => {
const fixture = createDependencies();
fixture.isIgnoredByIgnoreFile.mockResolvedValue(true);
await expect(extractHiddenFileFromDatabase(fixture.dependencies, path)).resolves.toBeUndefined();
expect(fixture.serialiseFileOperation).not.toHaveBeenCalled();
expect(fixture.loadDatabaseMetadata).not.toHaveBeenCalled();
});
it("propagates ignore-policy errors before taking the guarded path", async () => {
const fixture = createDependencies();
const error = new Error("ignore policy unavailable");
fixture.isIgnoredByIgnoreFile.mockRejectedValue(error);
await expect(extractHiddenFileFromDatabase(fixture.dependencies, path)).rejects.toBe(error);
expect(fixture.serialiseFileOperation).not.toHaveBeenCalled();
expect(fixture.log).not.toHaveBeenCalled();
});
it("prevents a conflicted entry from reaching storage", async () => {
const entry = metadata({ _conflicts: ["2-conflict"] });
const fixture = createDependencies(entry);
await expect(extractHiddenFileFromDatabase(fixture.dependencies, path, { force: true })).resolves.toBe(false);
expect(fixture.loadDatabaseEntry).not.toHaveBeenCalled();
expect(fixture.writeStorageFile).not.toHaveBeenCalled();
expect(fixture.log).toHaveBeenCalledWith(
`Hidden file ${path} has conflicted revisions, to keep in safe, writing to storage has been prevented`,
LOG_LEVEL_INFO,
undefined
);
});
});
describe("hidden-file database-to-storage processed-state policy", () => {
it("skips a previously processed revision without settling state again", async () => {
const fixture = createDependencies();
fixture.getLastProcessedDatabaseKey.mockReturnValue(toHiddenFileSyncDatabaseStateKey(fixture.entry));
await expect(extractHiddenFileFromDatabase(fixture.dependencies, path)).resolves.toBeUndefined();
expect(fixture.loadDatabaseEntry).not.toHaveBeenCalled();
expect(fixture.updateLastProcessed).not.toHaveBeenCalled();
expect(fixture.log).toHaveBeenCalledWith(
`STORAGE <-- DB: ${path}: skipped (hidden, overwrite) (Previously processed)`,
undefined,
undefined
);
});
it("allows force to bypass the previously processed revision", async () => {
const fixture = createDependencies();
fixture.getLastProcessedDatabaseKey.mockReturnValue(toHiddenFileSyncDatabaseStateKey(fixture.entry));
await expect(extractHiddenFileFromDatabase(fixture.dependencies, path, { force: true })).resolves.toBe(true);
expect(fixture.writeStorageFile).toHaveBeenCalledWith(path, loadedEntry(fixture.entry), true);
});
it("settles both sides when onlyNew declines an equally old database entry", async () => {
const fixture = createDependencies();
await expect(
extractHiddenFileFromDatabase(fixture.dependencies, path, {
metaEntry: fixture.entry,
preventDoubleProcess: false,
onlyNew: true,
})
).resolves.toBeUndefined();
expect(fixture.loadDatabaseMetadata).not.toHaveBeenCalled();
expect(fixture.loadDatabaseEntry).not.toHaveBeenCalled();
expect(fixture.updateLastProcessedDatabase).toHaveBeenCalledWith(path, fixture.entry);
expect(fixture.updateLastProcessedFile).toHaveBeenCalledWith(path, storageStat());
expect(fixture.events).toEqual(["lock:start", "state:database", "state:storage", "lock:end"]);
});
it("uses the last known mtime when onlyNew sees a zero storage mtime", async () => {
const fixture = createDependencies(metadata({ mtime: 30 }));
fixture.statStorageFile.mockResolvedValue(storageStat(0));
fixture.getLastProcessedFileMTime.mockReturnValue(40);
await expect(
extractHiddenFileFromDatabase(fixture.dependencies, path, { onlyNew: true })
).resolves.toBeUndefined();
expect(fixture.getLastProcessedFileMTime).toHaveBeenCalledWith(path);
expect(fixture.writeStorageFile).not.toHaveBeenCalled();
});
});
describe("hidden-file database-to-storage application", () => {
it("settles state and queues a notification inside the file lock after a successful write", async () => {
const fixture = createDependencies();
await expect(extractHiddenFileFromDatabase(fixture.dependencies, path)).resolves.toBe(true);
expect(fixture.updateLastProcessed).toHaveBeenCalledWith(path, fixture.entry, storageStat(21));
expect(fixture.queueNotification).toHaveBeenCalledWith(path);
expect(fixture.serialiseFileOperation).toHaveBeenCalledWith(`file-${prefixedPath}`, expect.any(Function));
expect(fixture.events).toEqual(["lock:start", "storage:write", "state:file", "notification", "lock:end"]);
});
it("settles and notifies when the storage writer reports unchanged content with its existing stat", async () => {
const fixture = createDependencies();
fixture.writeStorageFile.mockResolvedValue(storageStat());
await expect(extractHiddenFileFromDatabase(fixture.dependencies, path)).resolves.toBe(true);
expect(fixture.updateLastProcessed).toHaveBeenCalledWith(path, fixture.entry, storageStat());
expect(fixture.queueNotification).toHaveBeenCalledWith(path);
});
it("returns false without settlement when the storage writer fails", async () => {
const fixture = createDependencies();
fixture.writeStorageFile.mockResolvedValue(false);
await expect(extractHiddenFileFromDatabase(fixture.dependencies, path)).resolves.toBe(false);
expect(fixture.updateLastProcessed).not.toHaveBeenCalled();
expect(fixture.queueNotification).not.toHaveBeenCalled();
});
it("records a successful storage deletion through deletion settlement", async () => {
const entry = metadata({ deleted: true });
const fixture = createDependencies(entry);
await expect(extractHiddenFileFromDatabase(fixture.dependencies, path)).resolves.toBe(true);
expect(fixture.updateLastProcessedDeletion).toHaveBeenCalledWith(path, entry);
expect(fixture.updateLastProcessedDatabase).not.toHaveBeenCalled();
expect(fixture.events).toEqual(["lock:start", "storage:delete", "state:deletion", "lock:end"]);
});
it("marks an already absent deleted file as database-processed only", async () => {
const entry = metadata({ deleted: true });
const fixture = createDependencies(entry);
fixture.deleteStorageFile.mockImplementation(async () => {
fixture.events.push("storage:delete");
return "ALREADY";
});
await expect(extractHiddenFileFromDatabase(fixture.dependencies, path)).resolves.toBe(true);
expect(fixture.updateLastProcessedDatabase).toHaveBeenCalledWith(path, entry);
expect(fixture.updateLastProcessedDeletion).not.toHaveBeenCalled();
expect(fixture.events).toEqual(["lock:start", "storage:delete", "state:database", "lock:end"]);
});
it("returns false without settling state when a storage deletion fails", async () => {
const fixture = createDependencies(metadata({ _deleted: true }));
fixture.deleteStorageFile.mockResolvedValue(false);
await expect(extractHiddenFileFromDatabase(fixture.dependencies, path)).resolves.toBe(false);
expect(fixture.updateLastProcessedDeletion).not.toHaveBeenCalled();
expect(fixture.updateLastProcessedDatabase).not.toHaveBeenCalled();
});
it("turns a database content-read failure into false and reports the inherited write diagnostic", async () => {
const fixture = createDependencies();
const error = new Error("content unavailable");
fixture.loadDatabaseEntry.mockRejectedValue(error);
await expect(extractHiddenFileFromDatabase(fixture.dependencies, path, { force: true })).resolves.toBe(false);
expect(fixture.log).toHaveBeenNthCalledWith(
1,
`STORAGE <-- DB: ${path}: written (hidden, overwrite, force) Failed`,
undefined,
undefined
);
expect(fixture.log).toHaveBeenNthCalledWith(2, error, LOG_LEVEL_VERBOSE, undefined);
});
});
describe("selected-revision hidden-file database-to-storage application", () => {
it("applies a selected live revision without reading ordinary Metadata", async () => {
const fixture = createDependencies();
await expect(
extractHiddenFileRevisionFromDatabase(fixture.dependencies, path, fixture.entry._rev!, true)
).resolves.toBe(true);
expect(fixture.loadLiveRevision).toHaveBeenCalledWith(prefixedPath, fixture.entry._rev);
expect(fixture.loadDatabaseMetadata).not.toHaveBeenCalled();
expect(fixture.writeStorageFile).toHaveBeenCalledWith(path, loadedEntry(fixture.entry), true);
});
it("can apply a selected live branch while ordinary Metadata reports a conflict", async () => {
const selected = metadata({ _rev: "2-selected", _conflicts: undefined });
const fixture = createDependencies(metadata({ _rev: "3-winner", _conflicts: [selected._rev!] }));
fixture.loadLiveRevision.mockResolvedValue(selected);
fixture.loadDatabaseEntry.mockResolvedValue(loadedEntry(selected));
await expect(
extractHiddenFileRevisionFromDatabase(fixture.dependencies, path, selected._rev!, true)
).resolves.toBe(true);
expect(fixture.loadDatabaseMetadata).not.toHaveBeenCalled();
expect(fixture.writeStorageFile).toHaveBeenCalledWith(path, loadedEntry(selected), true);
});
it("returns false when the selected revision ceased to be live", async () => {
const fixture = createDependencies();
fixture.loadLiveRevision.mockResolvedValue(false);
await expect(extractHiddenFileRevisionFromDatabase(fixture.dependencies, path, "2-stale", true)).resolves.toBe(
false
);
expect(fixture.loadDatabaseEntry).not.toHaveBeenCalled();
expect(fixture.writeStorageFile).not.toHaveBeenCalled();
});
it("exposes frozen operations with the exact-revision Boolean contract", async () => {
const fixture = createDependencies();
fixture.isIgnoredByIgnoreFile.mockResolvedValue(true);
const operations = createHiddenFileSyncDatabaseExtractionOperations(fixture.dependencies);
expect(Object.isFrozen(operations)).toBe(true);
await expect(operations.extractRevision(path, fixture.entry._rev!)).resolves.toBe(false);
});
});
@@ -0,0 +1,111 @@
import {
LOG_LEVEL_NOTICE,
LOG_LEVEL_VERBOSE,
type FilePath,
type FilePathWithPrefix,
type LoadedEntry,
type LOG_LEVEL,
type MetaEntry,
} from "@vrtmrz/livesync-commonlib/compat/common/types";
import type { DatabaseFileAccess } from "@vrtmrz/livesync-commonlib/compat/interfaces/DatabaseFileAccess";
import type { LiveSyncLocalDB } from "@vrtmrz/livesync-commonlib/compat/pouchdb/LiveSyncLocalDB";
import type { IPathService } from "@vrtmrz/livesync-commonlib/compat/services/base/IService";
import type { LogFunction } from "@vrtmrz/livesync-commonlib/compat/services/lib/logUtils";
import { addPrefix, stripAllPrefixes } from "@vrtmrz/livesync-commonlib/compat/string_and_binary/path";
import { ICHeader } from "@/common/types.ts";
type HiddenFileSyncLogDependency = {
log: LogFunction;
};
type HiddenFileSyncDatabaseMethods<Method extends keyof LiveSyncLocalDB> = {
getLocalDatabase(): Pick<LiveSyncLocalDB, Method>;
};
type HiddenFileSyncDatabaseFileAccessMethods<Method extends keyof DatabaseFileAccess> = {
databaseFileAccess: Pick<DatabaseFileAccess, Method>;
};
type HiddenFileSyncPathMethods<Method extends keyof IPathService> = {
path: Pick<IPathService, Method>;
};
export type HiddenFileSyncBaseEntryLoaderDependencies = HiddenFileSyncDatabaseMethods<"getDBEntry" | "getDBEntryMeta"> &
HiddenFileSyncPathMethods<"path2id"> &
HiddenFileSyncLogDependency;
export type HiddenFileSyncLiveRevisionLoaderDependencies = HiddenFileSyncDatabaseFileAccessMethods<
"fetchEntryMeta" | "getConflictedRevs"
> &
HiddenFileSyncLogDependency;
function log(dependencies: HiddenFileSyncLogDependency, message: unknown, level?: LOG_LEVEL, key?: string): void {
dependencies.log(message, level, key);
}
export async function loadHiddenFileSyncBaseEntry(
dependencies: HiddenFileSyncBaseEntryLoaderDependencies,
file: FilePath,
includeContent = true
): Promise<LoadedEntry | false> {
const prefixedFileName = addPrefix(file, ICHeader);
// Compatibility question: path-to-ID conversion is performed even when an
// entry already exists, and it sits outside the guarded database lookup.
// Preserve this ordering and error propagation until reviewed separately.
const id = await dependencies.path.path2id(prefixedFileName, ICHeader);
try {
const old = includeContent
? await dependencies.getLocalDatabase().getDBEntry(prefixedFileName, undefined, false, true)
: await dependencies.getLocalDatabase().getDBEntryMeta(prefixedFileName, { conflicts: true }, true);
if (old !== false) {
return old;
}
// Compatibility question: getDBEntry() also returns false when content
// or Chunks cannot be read. The inherited behaviour treats that exactly
// like absence and synthesises a fresh base entry.
return {
_id: id,
data: [],
path: prefixedFileName,
mtime: 0,
ctime: 0,
datatype: "newnote",
children: [],
size: 0,
deleted: false,
type: "newnote",
eden: {},
};
} catch (error) {
log(dependencies, "Getting base save data failed");
log(dependencies, error, LOG_LEVEL_VERBOSE);
return false;
}
}
export async function loadLiveHiddenFileSyncRevision(
dependencies: HiddenFileSyncLiveRevisionLoaderDependencies,
prefixedFileName: FilePathWithPrefix,
revision: string
): Promise<MetaEntry | false> {
const [selected, current, conflicts] = await Promise.all([
dependencies.databaseFileAccess.fetchEntryMeta(prefixedFileName, revision, true),
dependencies.databaseFileAccess.fetchEntryMeta(prefixedFileName, undefined, true),
dependencies.databaseFileAccess.getConflictedRevs(prefixedFileName),
]);
const liveRevisions = new Set([...(current && current._rev ? [current._rev] : []), ...conflicts]);
if (!selected || selected._rev !== revision || !liveRevisions.has(revision)) {
// Compatibility: missing, mismatched, and stale selections share the
// same user-facing diagnostic and false result.
log(
dependencies,
`Could not use hidden-file revision ${revision} of ${stripAllPrefixes(prefixedFileName)}; the selected revision is no longer live`,
LOG_LEVEL_NOTICE
);
return false;
}
// Compatibility: liveness is revision-tree membership only. A deleted
// Metadata leaf remains selectable while its revision is still live.
return selected;
}
@@ -0,0 +1,202 @@
import { describe, expect, it, vi } from "vitest";
import {
LOG_LEVEL_NOTICE,
LOG_LEVEL_VERBOSE,
type DocumentID,
type FilePath,
type FilePathWithPrefix,
type LoadedEntry,
type MetaEntry,
} from "@vrtmrz/livesync-commonlib/compat/common/types";
import { loadHiddenFileSyncBaseEntry, loadLiveHiddenFileSyncRevision } from "./hiddenFileSyncDatabaseLoaders.ts";
const path = ".obsidian/app.json" as FilePath;
const prefixedPath = `i:${path}` as FilePathWithPrefix;
const id = "hidden-entry-id" as DocumentID;
function loadedEntry(): LoadedEntry {
return {
_id: id,
_rev: "1-a",
path: prefixedPath,
type: "plain",
datatype: "plain",
data: "content",
ctime: 10,
mtime: 20,
size: 7,
children: [],
eden: {},
deleted: false,
} as LoadedEntry;
}
function createBaseEntryDependencies() {
const getDBEntry = vi.fn();
const getDBEntryMeta = vi.fn();
const path2id = vi.fn(async () => id);
const log = vi.fn();
const dependencies = {
getLocalDatabase: () => ({ getDBEntry, getDBEntryMeta }) as never,
path: { path2id } as never,
log,
};
return { dependencies, getDBEntry, getDBEntryMeta, log, path2id };
}
function metaEntry(revision: string): MetaEntry {
return {
...loadedEntry(),
_rev: revision,
data: undefined,
} as unknown as MetaEntry;
}
function createLiveRevisionDependencies() {
const selected = metaEntry("2-selected");
const current = metaEntry("3-current");
const fetchEntryMeta = vi.fn(async (_path: unknown, revision?: string) => {
if (revision === undefined || revision === current._rev) return current;
if (revision === selected._rev) return selected;
return false;
});
const getConflictedRevs = vi.fn(async () => [selected._rev!]);
const log = vi.fn();
const dependencies = {
databaseFileAccess: { fetchEntryMeta, getConflictedRevs } as never,
log,
};
return { current, dependencies, fetchEntryMeta, getConflictedRevs, log, selected };
}
describe("Hidden File Sync base-entry loader", () => {
it("synthesises a new empty base whenever the content lookup reports false", async () => {
const { dependencies, getDBEntry, path2id } = createBaseEntryDependencies();
getDBEntry.mockResolvedValue(false);
await expect(loadHiddenFileSyncBaseEntry(dependencies, path)).resolves.toEqual({
_id: id,
data: [],
path: prefixedPath,
mtime: 0,
ctime: 0,
datatype: "newnote",
children: [],
size: 0,
deleted: false,
type: "newnote",
eden: {},
});
expect(path2id).toHaveBeenCalledWith(prefixedPath, "i:");
expect(getDBEntry).toHaveBeenCalledWith(prefixedPath, undefined, false, true);
});
it("returns an existing content entry unchanged", async () => {
const { dependencies, getDBEntry, path2id } = createBaseEntryDependencies();
const existing = loadedEntry();
getDBEntry.mockResolvedValue(existing);
await expect(loadHiddenFileSyncBaseEntry(dependencies, path, true)).resolves.toBe(existing);
expect(path2id).toHaveBeenCalledWith(prefixedPath, "i:");
});
it("uses the conflict-aware metadata lookup when content is not requested", async () => {
const { dependencies, getDBEntry, getDBEntryMeta } = createBaseEntryDependencies();
const existing = loadedEntry();
getDBEntryMeta.mockResolvedValue(existing);
await expect(loadHiddenFileSyncBaseEntry(dependencies, path, false)).resolves.toBe(existing);
expect(getDBEntry).not.toHaveBeenCalled();
expect(getDBEntryMeta).toHaveBeenCalledWith(prefixedPath, { conflicts: true }, true);
});
it("turns a database lookup failure into a logged false result", async () => {
const { dependencies, getDBEntry, log } = createBaseEntryDependencies();
const error = new Error("database unavailable");
getDBEntry.mockRejectedValue(error);
await expect(loadHiddenFileSyncBaseEntry(dependencies, path)).resolves.toBe(false);
expect(log).toHaveBeenNthCalledWith(1, "Getting base save data failed", undefined, undefined);
expect(log).toHaveBeenNthCalledWith(2, error, LOG_LEVEL_VERBOSE, undefined);
});
it("propagates path-to-ID failures which occur before the guarded lookup", async () => {
const { dependencies, getDBEntry, log, path2id } = createBaseEntryDependencies();
const error = new Error("ID conversion failed");
path2id.mockRejectedValue(error);
await expect(loadHiddenFileSyncBaseEntry(dependencies, path)).rejects.toBe(error);
expect(getDBEntry).not.toHaveBeenCalled();
expect(log).not.toHaveBeenCalled();
});
});
describe("Hidden File Sync live-revision loader", () => {
it("accepts the current winning revision", async () => {
const { current, dependencies, fetchEntryMeta, getConflictedRevs } = createLiveRevisionDependencies();
await expect(loadLiveHiddenFileSyncRevision(dependencies, prefixedPath, current._rev!)).resolves.toBe(current);
expect(fetchEntryMeta).toHaveBeenNthCalledWith(1, prefixedPath, current._rev, true);
expect(fetchEntryMeta).toHaveBeenNthCalledWith(2, prefixedPath, undefined, true);
expect(getConflictedRevs).toHaveBeenCalledWith(prefixedPath);
});
it("accepts a selected conflict leaf while it remains live", async () => {
const { dependencies, selected } = createLiveRevisionDependencies();
await expect(loadLiveHiddenFileSyncRevision(dependencies, prefixedPath, selected._rev!)).resolves.toBe(
selected
);
});
it("accepts a live conflict even when there is no current winner", async () => {
const { dependencies, fetchEntryMeta, selected } = createLiveRevisionDependencies();
fetchEntryMeta.mockImplementation(async (_path: unknown, revision?: string) =>
revision === selected._rev ? selected : false
);
await expect(loadLiveHiddenFileSyncRevision(dependencies, prefixedPath, selected._rev!)).resolves.toBe(
selected
);
});
it("accepts deleted metadata while its revision remains live", async () => {
const { dependencies, fetchEntryMeta, selected } = createLiveRevisionDependencies();
const deleted = { ...selected, deleted: true } as MetaEntry;
fetchEntryMeta.mockImplementation(async (_path: unknown, revision?: string) =>
revision === undefined ? metaEntry("3-current") : deleted
);
await expect(loadLiveHiddenFileSyncRevision(dependencies, prefixedPath, selected._rev!)).resolves.toBe(deleted);
});
it("rejects a selected revision which is no longer current or conflicted", async () => {
const { dependencies, getConflictedRevs, log, selected } = createLiveRevisionDependencies();
getConflictedRevs.mockResolvedValue([]);
await expect(loadLiveHiddenFileSyncRevision(dependencies, prefixedPath, selected._rev!)).resolves.toBe(false);
expect(log).toHaveBeenCalledWith(
`Could not use hidden-file revision ${selected._rev} of ${path}; the selected revision is no longer live`,
LOG_LEVEL_NOTICE,
undefined
);
});
it("rejects a lookup result whose revision does not match the selection", async () => {
const { dependencies, fetchEntryMeta, log, selected } = createLiveRevisionDependencies();
fetchEntryMeta.mockResolvedValue(metaEntry("2-other"));
await expect(loadLiveHiddenFileSyncRevision(dependencies, prefixedPath, selected._rev!)).resolves.toBe(false);
expect(log).toHaveBeenCalledOnce();
});
it("propagates database-file-access failures", async () => {
const { dependencies, fetchEntryMeta, log, selected } = createLiveRevisionDependencies();
const error = new Error("revision lookup failed");
fetchEntryMeta.mockRejectedValue(error);
await expect(loadLiveHiddenFileSyncRevision(dependencies, prefixedPath, selected._rev!)).rejects.toBe(error);
expect(log).not.toHaveBeenCalled();
});
});
@@ -0,0 +1,337 @@
import {
LOG_LEVEL_NOTICE,
LOG_LEVEL_VERBOSE,
type FilePath,
type FilePathWithPrefix,
type LoadedEntry,
type LOG_LEVEL,
type MetaEntry,
type SavingEntry,
type UXFileInfo,
} from "@vrtmrz/livesync-commonlib/compat/common/types";
import { isDocContentSame, readAsBlob } from "@vrtmrz/livesync-commonlib/compat/common/utils";
import type { LogFunction } from "@vrtmrz/livesync-commonlib/compat/services/lib/logUtils";
import { addPrefix, stripAllPrefixes } from "@vrtmrz/livesync-commonlib/compat/string_and_binary/path";
import type { InternalFileInfo } from "@/common/types.ts";
import { ICHeader } from "@/common/types.ts";
import {
serialiseHiddenFileOperation,
type HiddenFileSyncFileSerialisationDependencies,
type HiddenFileSyncFileSerialiser,
} from "./hiddenFileSyncFileOperations.ts";
export type { HiddenFileSyncFileSerialiser } from "./hiddenFileSyncFileOperations.ts";
type HiddenFileSyncDatabaseWriteResponse = {
readonly ok: boolean;
readonly rev: string;
};
export type HiddenFileSyncDatabaseWriteProcessedState = {
updateLastProcessed(path: FilePath, databaseEntry: MetaEntry | LoadedEntry, storageFile: UXFileInfo["stat"]): void;
updateLastProcessedDeletion(path: FilePath, databaseEntry: MetaEntry | LoadedEntry | false): void;
};
export type HiddenFileSyncDatabaseWriteDependencies = {
serialiseFileOperation: HiddenFileSyncFileSerialiser;
isIgnoredByIgnoreFile(path: string): Promise<boolean>;
readFileWithInfo(path: FilePath): Promise<UXFileInfo>;
loadBaseEntry(path: FilePath): Promise<LoadedEntry | false>;
loadBaseMetadata(path: FilePath): Promise<LoadedEntry | false>;
loadLiveRevision(path: FilePathWithPrefix, revision: string): Promise<MetaEntry | false>;
fetchEntryFromMeta(meta: MetaEntry, waitForReady: boolean, skipCheck: boolean): Promise<LoadedEntry | false>;
storeWithBaseRevision(file: UXFileInfo, baseRevision: string, skipCheck: boolean): Promise<string | false>;
putDatabaseEntry(entry: SavingEntry): Promise<false | HiddenFileSyncDatabaseWriteResponse>;
putRaw(entry: LoadedEntry): Promise<HiddenFileSyncDatabaseWriteResponse>;
removeRevision(id: LoadedEntry["_id"], revision: string): Promise<boolean>;
processedState: HiddenFileSyncDatabaseWriteProcessedState;
now(): number;
log: LogFunction;
};
type HiddenFileSyncDatabaseWriteCommonDependencies = Pick<
HiddenFileSyncDatabaseWriteDependencies,
"isIgnoredByIgnoreFile" | "log" | "readFileWithInfo" | "serialiseFileOperation"
>;
export type StoreHiddenFileInDatabaseDependencies = HiddenFileSyncDatabaseWriteCommonDependencies &
Pick<HiddenFileSyncDatabaseWriteDependencies, "loadBaseEntry" | "putDatabaseEntry" | "processedState">;
export type StoreHiddenFileWithBaseRevisionDependencies = HiddenFileSyncDatabaseWriteCommonDependencies &
Pick<
HiddenFileSyncDatabaseWriteDependencies,
"fetchEntryFromMeta" | "loadLiveRevision" | "storeWithBaseRevision" | "processedState"
>;
export type DeleteHiddenFileFromDatabaseDependencies = Pick<
HiddenFileSyncDatabaseWriteDependencies,
| "isIgnoredByIgnoreFile"
| "loadBaseMetadata"
| "log"
| "now"
| "putRaw"
| "removeRevision"
| "serialiseFileOperation"
| "processedState"
>;
export type HiddenFileSyncDatabaseWriteOperations = {
store(file: InternalFileInfo | UXFileInfo, forceWrite?: boolean): Promise<boolean | undefined>;
storeWithBaseRevision(
file: InternalFileInfo | UXFileInfo,
baseRevision: string,
createIfDifferent?: boolean
): Promise<boolean>;
delete(path: FilePath, forceWrite?: boolean): Promise<boolean | undefined>;
};
type HiddenFileSyncDatabaseWriteGuardDependencies = HiddenFileSyncFileSerialisationDependencies &
Pick<HiddenFileSyncDatabaseWriteDependencies, "log">;
function log(
dependencies: Pick<HiddenFileSyncDatabaseWriteDependencies, "log">,
message: unknown,
level?: LOG_LEVEL,
key?: string
) {
dependencies.log(message, level, key);
}
async function runGuardedDatabaseWrite<Result>(
dependencies: HiddenFileSyncDatabaseWriteGuardDependencies,
prefixedFileName: FilePathWithPrefix,
failureMessage: string,
operation: () => Promise<Result>
): Promise<Result | false> {
return await serialiseHiddenFileOperation(dependencies, prefixedFileName, async () => {
try {
return await operation();
} catch (error) {
log(dependencies, failureMessage);
log(dependencies, error, LOG_LEVEL_VERBOSE);
return false;
}
});
}
async function resolvePresentFileInfo(
dependencies: Pick<HiddenFileSyncDatabaseWriteDependencies, "readFileWithInfo">,
file: InternalFileInfo | UXFileInfo,
storeFilePath: FilePath
): Promise<UXFileInfo> {
const fileInfo = "stat" in file && "body" in file ? file : await dependencies.readFileWithInfo(storeFilePath);
if (fileInfo.deleted) {
throw new Error(`Hidden file:${storeFilePath} is deleted. This should not be occurred.`);
}
return fileInfo;
}
export async function storeHiddenFileInDatabase(
dependencies: StoreHiddenFileInDatabaseDependencies,
file: InternalFileInfo | UXFileInfo,
forceWrite = false
): Promise<boolean | undefined> {
const storeFilePath = stripAllPrefixes(file.path);
const storageFilePath = file.path;
// Compatibility: all three admission checks sit outside the guarded lock,
// so policy errors propagate. The ordinary path reports an ignored file as
// undefined, while the selected-revision path reports false.
if (await dependencies.isIgnoredByIgnoreFile(storageFilePath)) {
return undefined;
}
const prefixedFileName = addPrefix(storeFilePath, ICHeader);
return await runGuardedDatabaseWrite(
dependencies,
prefixedFileName,
`STORAGE --> DB:${storageFilePath}: (hidden) Failed`,
async () => {
const fileInfo = await resolvePresentFileInfo(dependencies, file, storeFilePath);
const baseData = await dependencies.loadBaseEntry(storeFilePath);
if (baseData === false) throw new Error("Failed to load base data");
if (baseData._rev && !forceWrite) {
const isSame = await isDocContentSame(readAsBlob(baseData), fileInfo.body);
if (isSame) {
dependencies.processedState.updateLastProcessed(storeFilePath, baseData, fileInfo.stat);
return undefined;
}
}
const saveData: SavingEntry = {
...baseData,
data: fileInfo.body,
mtime: fileInfo.stat.mtime,
size: fileInfo.stat.size,
children: [],
deleted: false,
type: baseData.datatype,
};
// Compatibility question: ctime comes from the old database base,
// not from the storage stat. A newly synthesised base therefore
// stores ctime 0. Preserve this until cross-device effects are known.
const ret = await dependencies.putDatabaseEntry(saveData);
if (ret && ret.ok) {
saveData._rev = ret.rev;
dependencies.processedState.updateLastProcessed(storeFilePath, saveData, fileInfo.stat);
}
const success = ret && ret.ok;
log(dependencies, `STORAGE --> DB:${storageFilePath}: (hidden) ${success ? "Done" : "Failed"}`);
return success;
}
);
}
export async function storeHiddenFileWithBaseRevision(
dependencies: StoreHiddenFileWithBaseRevisionDependencies,
file: InternalFileInfo | UXFileInfo,
baseRevision: string,
createIfDifferent = true
): Promise<boolean> {
const storeFilePath = stripAllPrefixes(file.path);
const storageFilePath = file.path;
if (await dependencies.isIgnoredByIgnoreFile(storageFilePath)) {
return false;
}
const prefixedFileName = addPrefix(storeFilePath, ICHeader);
return await runGuardedDatabaseWrite(
dependencies,
prefixedFileName,
`STORAGE --> DB:${storageFilePath}: (hidden, selected branch) Failed`,
async () => {
// The live check intentionally precedes the storage read. It avoids
// work for a stale selection, but does not make the later write atomic.
const baseData = await dependencies.loadLiveRevision(prefixedFileName, baseRevision);
if (baseData === false) {
return false;
}
const fileInfo = await resolvePresentFileInfo(dependencies, file, storeFilePath);
if (!baseData.deleted && !baseData._deleted) {
const loadedBase = await dependencies.fetchEntryFromMeta(baseData, true, true);
if (loadedBase && (await isDocContentSame(readAsBlob(loadedBase), fileInfo.body))) {
dependencies.processedState.updateLastProcessed(storeFilePath, baseData, fileInfo.stat);
return true;
}
}
if (!createIfDifferent) {
log(
dependencies,
`Could not mark hidden file ${storeFilePath} as revision ${baseRevision}; the storage content differs`,
LOG_LEVEL_NOTICE
);
return false;
}
const storedRevision = await dependencies.storeWithBaseRevision(
{
...fileInfo,
path: storeFilePath,
name: fileInfo.name || storeFilePath.split("/").pop() || "",
isInternal: true,
},
baseRevision,
true
);
if (storedRevision === false) {
return false;
}
// Compatibility question: spreading a selected PouchDB tombstone
// retains `_deleted: true` in the processed-state entry even though
// `deleted` is reset. The stored child itself is created separately.
dependencies.processedState.updateLastProcessed(
storeFilePath,
{
...baseData,
_rev: storedRevision,
path: prefixedFileName,
ctime: fileInfo.stat.ctime,
mtime: fileInfo.stat.mtime,
size: fileInfo.stat.size,
deleted: false,
},
fileInfo.stat
);
log(dependencies, `STORAGE --> DB:${storageFilePath}: (hidden, selected branch) Done`);
return true;
}
);
}
export async function deleteHiddenFileFromDatabase(
dependencies: DeleteHiddenFileFromDatabaseDependencies,
filenameSrc: FilePath,
forceWrite = false
): Promise<boolean | undefined> {
const storeFilePath = filenameSrc;
const storageFilePath = filenameSrc;
const displayFileName = filenameSrc;
const prefixedFileName = addPrefix(storeFilePath, ICHeader);
// Compatibility question: the timestamp is captured before the ignore
// check and before waiting for the per-file lock.
const mtime = dependencies.now();
// Compatibility question: forceWrite is part of the inherited call
// contract, but it has never changed deletion behaviour.
void forceWrite;
if (await dependencies.isIgnoredByIgnoreFile(storageFilePath)) {
return undefined;
}
return await runGuardedDatabaseWrite(
dependencies,
prefixedFileName,
`STORAGE -x> DB: ${displayFileName}: (hidden) Failed`,
async () => {
const baseData = await dependencies.loadBaseMetadata(storeFilePath);
if (baseData === false) throw new Error("Failed to load base data during deleting");
if (baseData._conflicts !== undefined) {
// Compatibility question: these removals are sequential but not
// atomic. Earlier branches remain removed if a later call fails.
for (const conflictRev of baseData._conflicts) {
await dependencies.removeRevision(baseData._id, conflictRev);
log(
dependencies,
`STORAGE -x> DB: ${displayFileName}: (hidden) conflict removed ${baseData._rev} => ${conflictRev}`,
LOG_LEVEL_VERBOSE
);
}
}
// Compatibility question: only the domain `deleted` marker is
// checked here. `_deleted: true` alone causes another tombstone write.
if (baseData.deleted) {
log(dependencies, `STORAGE -x> DB: ${displayFileName}: (hidden) already deleted`, LOG_LEVEL_VERBOSE);
dependencies.processedState.updateLastProcessedDeletion(storeFilePath, baseData);
return true;
}
const saveData: LoadedEntry = {
...baseData,
mtime,
size: 0,
children: [],
deleted: true,
type: baseData.datatype,
};
// A synthesised base has no revision; the inherited behaviour still
// writes it as a tombstone when the requested path was absent.
const ret = await dependencies.putRaw(saveData);
if (ret && ret.ok) {
log(dependencies, `STORAGE -x> DB: ${displayFileName}: (hidden) Done`);
saveData._rev = ret.rev;
dependencies.processedState.updateLastProcessedDeletion(storeFilePath, saveData);
return true;
} else {
log(dependencies, `STORAGE -x> DB: ${displayFileName}: (hidden) Failed`);
return false;
}
}
);
}
export function createHiddenFileSyncDatabaseWriteOperations(
dependencies: HiddenFileSyncDatabaseWriteDependencies
): HiddenFileSyncDatabaseWriteOperations {
return Object.freeze({
store: async (file, forceWrite) => await storeHiddenFileInDatabase(dependencies, file, forceWrite),
storeWithBaseRevision: async (file, baseRevision, createIfDifferent) =>
await storeHiddenFileWithBaseRevision(dependencies, file, baseRevision, createIfDifferent),
delete: async (path, forceWrite) => await deleteHiddenFileFromDatabase(dependencies, path, forceWrite),
});
}
@@ -0,0 +1,372 @@
import { describe, expect, it, vi } from "vitest";
import {
LOG_LEVEL_NOTICE,
LOG_LEVEL_VERBOSE,
type DocumentID,
type FilePath,
type FilePathWithPrefix,
type LoadedEntry,
type MetaEntry,
type UXFileInfo,
} from "@vrtmrz/livesync-commonlib/compat/common/types";
import {
deleteHiddenFileFromDatabase,
storeHiddenFileInDatabase,
storeHiddenFileWithBaseRevision,
type HiddenFileSyncDatabaseWriteDependencies,
} from "./hiddenFileSyncDatabaseWriteOperations.ts";
const path = ".obsidian/plugins/example/data.json" as FilePath;
const prefixedPath = `i:${path}` as FilePathWithPrefix;
const id = "hidden-entry-id" as DocumentID;
function fileInfo(content = '{"value":"vault"}'): UXFileInfo {
return {
path,
name: "data.json",
isInternal: true,
body: new Blob([content]),
stat: {
ctime: 41,
mtime: 42,
size: content.length,
type: "file",
},
deleted: false,
} as UXFileInfo;
}
function loadedEntry(overrides: Partial<LoadedEntry> = {}): LoadedEntry {
return {
_id: id,
_rev: "2-current",
path: prefixedPath,
type: "plain",
datatype: "plain",
data: '{"value":"database"}',
ctime: 10,
mtime: 20,
size: 20,
children: [],
eden: {},
deleted: false,
...overrides,
} as LoadedEntry;
}
function createDependencies(base = loadedEntry()) {
const events: string[] = [];
const serialiseFileOperation = vi.fn(async (_key: string, operation: () => Promise<unknown>) => {
events.push("lock:start");
try {
return await operation();
} finally {
events.push("lock:end");
}
});
const isIgnoredByIgnoreFile = vi.fn(async () => false);
const readFileWithInfo = vi.fn(async () => fileInfo());
const loadBaseEntry = vi.fn(async () => base as LoadedEntry | false);
const loadBaseMetadata = vi.fn(async () => base as LoadedEntry | false);
const loadLiveRevision = vi.fn(async (_path: FilePathWithPrefix, revision: string) =>
revision === base._rev ? (base as MetaEntry) : false
);
const fetchEntryFromMeta = vi.fn(async () => base as LoadedEntry | false);
const storeWithBaseRevision = vi.fn(async () => "3-selected-child" as string | false);
const putDatabaseEntry = vi.fn(async (_entry: unknown) => ({ ok: true, id, rev: "3-written" }));
const putRaw = vi.fn(async (_entry: LoadedEntry) => ({ ok: true, id, rev: "3-deleted" }));
const removeRevision = vi.fn(async () => true);
const updateLastProcessed = vi.fn(() => events.push("state:file"));
const updateLastProcessedDeletion = vi.fn(() => events.push("state:deletion"));
const processedState = {
updateLastProcessed,
updateLastProcessedDeletion,
};
const now = vi.fn(() => 1_000);
const log = vi.fn();
const dependencies = {
serialiseFileOperation,
isIgnoredByIgnoreFile,
readFileWithInfo,
loadBaseEntry,
loadBaseMetadata,
loadLiveRevision,
fetchEntryFromMeta,
storeWithBaseRevision,
putDatabaseEntry,
putRaw,
removeRevision,
processedState,
now,
log,
} as unknown as HiddenFileSyncDatabaseWriteDependencies;
return {
base,
dependencies,
events,
fetchEntryFromMeta,
isIgnoredByIgnoreFile,
loadBaseEntry,
loadBaseMetadata,
loadLiveRevision,
log,
now,
putDatabaseEntry,
putRaw,
readFileWithInfo,
removeRevision,
serialiseFileOperation,
storeWithBaseRevision,
updateLastProcessed,
updateLastProcessedDeletion,
};
}
describe("ordinary hidden-file database writes", () => {
it("keeps the synthetic base ctime and settles the new revision inside the file lock", async () => {
const base = loadedEntry({
_rev: undefined,
ctime: 0,
data: [],
datatype: "newnote",
type: "newnote",
});
const fixture = createDependencies(base);
const file = fileInfo();
await expect(storeHiddenFileInDatabase(fixture.dependencies, file)).resolves.toBe(true);
expect(fixture.putDatabaseEntry).toHaveBeenCalledWith(
expect.objectContaining({ ctime: 0, mtime: file.stat.mtime, data: file.body })
);
expect(fixture.updateLastProcessed).toHaveBeenCalledWith(
path,
expect.objectContaining({ _rev: "3-written", ctime: 0 }),
file.stat
);
expect(fixture.serialiseFileOperation).toHaveBeenCalledWith(`file-${prefixedPath}`, expect.any(Function));
expect(fixture.events).toEqual(["lock:start", "state:file", "lock:end"]);
});
it("settles matching content without writing or emitting a transfer log", async () => {
const base = loadedEntry({ data: '{"value":"vault"}' });
const fixture = createDependencies(base);
const file = fileInfo();
await expect(storeHiddenFileInDatabase(fixture.dependencies, file)).resolves.toBeUndefined();
expect(fixture.putDatabaseEntry).not.toHaveBeenCalled();
expect(fixture.updateLastProcessed).toHaveBeenCalledWith(path, base, file.stat);
expect(fixture.log).not.toHaveBeenCalled();
});
it("writes matching content when forceWrite is enabled", async () => {
const fixture = createDependencies(loadedEntry({ data: '{"value":"vault"}' }));
await expect(storeHiddenFileInDatabase(fixture.dependencies, fileInfo(), true)).resolves.toBe(true);
expect(fixture.putDatabaseEntry).toHaveBeenCalledOnce();
});
it("returns false and reports both messages when a guarded write fails", async () => {
const fixture = createDependencies();
const error = new Error("storage read failed");
fixture.readFileWithInfo.mockRejectedValue(error);
await expect(
storeHiddenFileInDatabase(fixture.dependencies, {
path,
ctime: 1,
mtime: 2,
size: 3,
})
).resolves.toBe(false);
expect(fixture.log).toHaveBeenNthCalledWith(1, `STORAGE --> DB:${path}: (hidden) Failed`, undefined, undefined);
expect(fixture.log).toHaveBeenNthCalledWith(2, error, LOG_LEVEL_VERBOSE, undefined);
});
});
describe("selected-revision hidden-file database writes", () => {
it("stores the Vault content as a child of a selected live revision", async () => {
const fixture = createDependencies();
const file = fileInfo();
await expect(storeHiddenFileWithBaseRevision(fixture.dependencies, file, fixture.base._rev!)).resolves.toBe(
true
);
expect(fixture.storeWithBaseRevision).toHaveBeenCalledWith(
expect.objectContaining({ path, body: file.body, isInternal: true }),
fixture.base._rev,
true
);
expect(fixture.updateLastProcessed).toHaveBeenCalledWith(
path,
expect.objectContaining({ _rev: "3-selected-child" }),
file.stat
);
});
it("validates liveness before reading storage and refuses a stale revision", async () => {
const fixture = createDependencies();
fixture.loadLiveRevision.mockResolvedValue(false);
await expect(
storeHiddenFileWithBaseRevision(fixture.dependencies, { path, ctime: 1, mtime: 2, size: 3 }, "2-stale")
).resolves.toBe(false);
expect(fixture.readFileWithInfo).not.toHaveBeenCalled();
expect(fixture.storeWithBaseRevision).not.toHaveBeenCalled();
expect(fixture.updateLastProcessed).not.toHaveBeenCalled();
});
it("marks matching content without creating a child", async () => {
const base = loadedEntry({ data: '{"value":"vault"}' });
const fixture = createDependencies(base);
const file = fileInfo();
await expect(storeHiddenFileWithBaseRevision(fixture.dependencies, file, base._rev!, false)).resolves.toBe(
true
);
expect(fixture.storeWithBaseRevision).not.toHaveBeenCalled();
expect(fixture.updateLastProcessed).toHaveBeenCalledWith(path, base, file.stat);
});
it("reports differing content without creating a child when requested", async () => {
const fixture = createDependencies();
await expect(
storeHiddenFileWithBaseRevision(fixture.dependencies, fileInfo(), fixture.base._rev!, false)
).resolves.toBe(false);
expect(fixture.storeWithBaseRevision).not.toHaveBeenCalled();
expect(fixture.updateLastProcessed).not.toHaveBeenCalled();
expect(fixture.log).toHaveBeenCalledWith(
`Could not mark hidden file ${path} as revision ${fixture.base._rev}; the storage content differs`,
LOG_LEVEL_NOTICE,
undefined
);
});
it("keeps a selected branch's _deleted marker in the processed-state entry", async () => {
const base = loadedEntry({ deleted: true, _deleted: true });
const fixture = createDependencies(base);
const file = fileInfo();
await expect(storeHiddenFileWithBaseRevision(fixture.dependencies, file, base._rev!)).resolves.toBe(true);
expect(fixture.fetchEntryFromMeta).not.toHaveBeenCalled();
expect(fixture.updateLastProcessed).toHaveBeenCalledWith(
path,
expect.objectContaining({ _rev: "3-selected-child", deleted: false, _deleted: true }),
file.stat
);
});
});
describe("hidden-file database deletions", () => {
it("removes conflicts before accepting an already deleted entry", async () => {
const base = loadedEntry({ deleted: true, _conflicts: ["2-conflict"] });
const fixture = createDependencies(base);
await expect(deleteHiddenFileFromDatabase(fixture.dependencies, path, true)).resolves.toBe(true);
expect(fixture.removeRevision).toHaveBeenCalledWith(id, "2-conflict");
expect(fixture.putRaw).not.toHaveBeenCalled();
expect(fixture.updateLastProcessedDeletion).toHaveBeenCalledWith(path, base);
});
it("writes a deletion when only the PouchDB _deleted marker is present", async () => {
const base = loadedEntry({ deleted: false, _deleted: true });
const fixture = createDependencies(base);
await expect(deleteHiddenFileFromDatabase(fixture.dependencies, path)).resolves.toBe(true);
expect(fixture.putRaw).toHaveBeenCalledWith(expect.objectContaining({ deleted: true, _deleted: true }));
expect(fixture.updateLastProcessedDeletion).toHaveBeenCalledWith(
path,
expect.objectContaining({ _rev: "3-deleted" })
);
});
it("writes a tombstone for a synthetic base without a revision", async () => {
const base = loadedEntry({ _rev: undefined, data: [], datatype: "newnote", type: "newnote" });
const fixture = createDependencies(base);
let submitted: LoadedEntry | undefined;
fixture.putRaw.mockImplementation(async (entry: LoadedEntry) => {
submitted = { ...entry };
return { ok: true, id, rev: "3-deleted" };
});
await expect(deleteHiddenFileFromDatabase(fixture.dependencies, path)).resolves.toBe(true);
expect(submitted).toEqual(expect.objectContaining({ _rev: undefined, deleted: true, type: "newnote" }));
});
it("keeps earlier conflict removals when a later removal fails", async () => {
const base = loadedEntry({ _conflicts: ["2-first", "2-second"] });
const fixture = createDependencies(base);
const error = new Error("second removal failed");
fixture.removeRevision.mockResolvedValueOnce(true).mockRejectedValueOnce(error);
await expect(deleteHiddenFileFromDatabase(fixture.dependencies, path)).resolves.toBe(false);
expect(fixture.removeRevision.mock.calls).toEqual([
[id, "2-first"],
[id, "2-second"],
]);
expect(fixture.putRaw).not.toHaveBeenCalled();
expect(fixture.updateLastProcessedDeletion).not.toHaveBeenCalled();
expect(fixture.log).toHaveBeenLastCalledWith(error, LOG_LEVEL_VERBOSE, undefined);
});
it("captures the deletion time before evaluating ignore policy", async () => {
const fixture = createDependencies();
const events: string[] = [];
fixture.now.mockImplementation(() => {
events.push("now");
return 1_000;
});
fixture.isIgnoredByIgnoreFile.mockImplementation(async () => {
events.push("ignore");
return true;
});
await expect(deleteHiddenFileFromDatabase(fixture.dependencies, path)).resolves.toBeUndefined();
expect(events).toEqual(["now", "ignore"]);
expect(fixture.serialiseFileOperation).not.toHaveBeenCalled();
});
});
describe("hidden-file database write admission", () => {
it("preserves the different ignored results of the three write paths", async () => {
const fixture = createDependencies();
fixture.isIgnoredByIgnoreFile.mockResolvedValue(true);
const file = fileInfo();
await expect(storeHiddenFileInDatabase(fixture.dependencies, file)).resolves.toBeUndefined();
await expect(storeHiddenFileWithBaseRevision(fixture.dependencies, file, fixture.base._rev!)).resolves.toBe(
false
);
await expect(deleteHiddenFileFromDatabase(fixture.dependencies, path)).resolves.toBeUndefined();
expect(fixture.loadBaseEntry).not.toHaveBeenCalled();
expect(fixture.loadBaseMetadata).not.toHaveBeenCalled();
expect(fixture.loadLiveRevision).not.toHaveBeenCalled();
});
it("propagates ignore-policy failures before entering the guarded lock", async () => {
const fixture = createDependencies();
const error = new Error("ignore policy unavailable");
fixture.isIgnoredByIgnoreFile.mockRejectedValue(error);
await expect(storeHiddenFileInDatabase(fixture.dependencies, fileInfo())).rejects.toBe(error);
expect(fixture.serialiseFileOperation).not.toHaveBeenCalled();
expect(fixture.log).not.toHaveBeenCalled();
});
});
@@ -0,0 +1,18 @@
import type { FilePathWithPrefix } from "@vrtmrz/livesync-commonlib/compat/common/types";
export type HiddenFileSyncFileSerialiser = <Result>(key: string, operation: () => Promise<Result>) => Promise<Result>;
export type HiddenFileSyncFileSerialisationDependencies = {
serialiseFileOperation: HiddenFileSyncFileSerialiser;
};
export async function serialiseHiddenFileOperation<Result>(
dependencies: HiddenFileSyncFileSerialisationDependencies,
prefixedFileName: FilePathWithPrefix,
operation: () => Promise<Result>
): Promise<Result> {
// Compatibility question: this inherited lock uses `file-`, whereas the
// Commonlib database writer uses `file:`. The two writers therefore do not
// mutually exclude one another; changing the key needs concurrency tests.
return await dependencies.serialiseFileOperation(`file-${prefixedFileName}`, operation);
}
@@ -0,0 +1,22 @@
import type { CustomRegExp } from "@vrtmrz/livesync-commonlib/compat/common/utils";
export type HiddenFileSyncPathFilters = {
ignoreFilter: readonly CustomRegExp[];
targetFilter: readonly CustomRegExp[];
};
export function isHiddenFileSyncPath(path: string): boolean {
// Compatibility: this prefix check also excludes names such as `.trashcan`.
// Keep the broader exclusion until a separate path-policy decision changes it.
return path.startsWith(".") && !path.startsWith(".trash");
}
export function matchesHiddenFileSyncPatterns(path: string, filters: HiddenFileSyncPathFilters): boolean {
if (filters.ignoreFilter.some((pattern) => pattern.test(path))) {
return false;
}
if (filters.targetFilter.length > 0) {
return filters.targetFilter.some((pattern) => pattern.test(path));
}
return true;
}
@@ -0,0 +1,41 @@
import { describe, expect, it, vi } from "vitest";
import type { CustomRegExp } from "@vrtmrz/livesync-commonlib/compat/common/utils";
import { isHiddenFileSyncPath, matchesHiddenFileSyncPatterns } from "./hiddenFileSyncPathPolicy.ts";
const pattern = (matches: (path: string) => boolean) => ({ test: vi.fn(matches) }) as unknown as CustomRegExp;
describe("isHiddenFileSyncPath", () => {
it.each([
[".obsidian/app.json", true],
[".git/config", true],
[".trash/app.json", false],
[".trashcan/app.json", false],
["notes/.hidden", false],
])("classifies %s as a Hidden File Sync path=%s", (path, expected) => {
expect(isHiddenFileSyncPath(path)).toBe(expected);
});
});
describe("matchesHiddenFileSyncPatterns", () => {
it("allows every path when no filters are configured", () => {
expect(matchesHiddenFileSyncPatterns(".obsidian/app.json", { ignoreFilter: [], targetFilter: [] })).toBe(true);
});
it("uses target patterns as an allow-list", () => {
const targetFilter = [pattern((path) => path.endsWith(".json"))];
expect(matchesHiddenFileSyncPatterns(".obsidian/app.json", { ignoreFilter: [], targetFilter })).toBe(true);
expect(matchesHiddenFileSyncPatterns(".obsidian/theme.css", { ignoreFilter: [], targetFilter })).toBe(false);
});
it("gives ignore patterns precedence over target patterns", () => {
const matchesEverything = pattern(() => true);
expect(
matchesHiddenFileSyncPatterns(".obsidian/app.json", {
ignoreFilter: [matchesEverything],
targetFilter: [matchesEverything],
})
).toBe(false);
});
});
@@ -0,0 +1,202 @@
import {
LOG_LEVEL_VERBOSE,
type FilePath,
type LoadedEntry,
type LOG_LEVEL,
type MetaEntry,
type UXStat,
} from "@vrtmrz/livesync-commonlib/compat/common/types";
import type { KeyValueDatabase } from "@vrtmrz/livesync-commonlib/compat/interfaces/KeyValueDatabase";
import type { LiveSyncLocalDB } from "@vrtmrz/livesync-commonlib/compat/pouchdb/LiveSyncLocalDB";
import type { IPathService } from "@vrtmrz/livesync-commonlib/compat/services/base/IService";
import type { LogFunction } from "@vrtmrz/livesync-commonlib/compat/services/lib/logUtils";
import { addPrefix } from "@vrtmrz/livesync-commonlib/compat/string_and_binary/path";
import { ICHeader } from "@/common/types.ts";
import { autosaveCache, type MapLike } from "@/common/utils.ts";
import {
getHiddenFileSyncComparisonMTime,
toHiddenFileSyncDatabaseStateKey,
toHiddenFileSyncStorageStateKey,
} from "./hiddenFileSyncState.ts";
type HiddenFileSyncProcessedStateDatabase = Pick<LiveSyncLocalDB, "getDBEntryMeta">;
type HiddenFileSyncProcessedStateStorage = {
statHidden(path: FilePath): Promise<UXStat | null>;
};
export type HiddenFileSyncProcessedStateDependencies = {
getKeyValueDatabase(): KeyValueDatabase;
getLocalDatabase(): HiddenFileSyncProcessedStateDatabase;
storageAccess: HiddenFileSyncProcessedStateStorage;
path: Pick<IPathService, "markChangesAreSame" | "unmarkChanges">;
log: LogFunction;
};
export class HiddenFileSyncProcessedState {
private fileInfoLastProcessed: MapLike<string, string> = new Map();
private fileInfoLastKnown: MapLike<string, number> = new Map();
private databaseInfoLastProcessed: MapLike<string, string> = new Map();
constructor(private readonly dependencies: HiddenFileSyncProcessedStateDependencies) {}
private log(message: unknown, level?: LOG_LEVEL, key?: string): void {
this.dependencies.log(message, level, key);
}
async initialise(): Promise<void> {
// Compatibility question: these reads intentionally remain
// sequential and in this order.
// Compatibility question: autosaveCache has no flush or disposal
// boundary, so a delayed write from an earlier database lifecycle can
// outlive this state owner. Preserve that timing until database
// replacement and unload behaviour have focused coverage.
this.fileInfoLastProcessed = await autosaveCache(
this.dependencies.getKeyValueDatabase(),
"hidden-file-lastProcessed"
);
this.databaseInfoLastProcessed = await autosaveCache(
this.dependencies.getKeyValueDatabase(),
"hidden-file-lastProcessed-database"
);
this.fileInfoLastKnown = await autosaveCache(this.dependencies.getKeyValueDatabase(), "hidden-file-lastKnown");
}
getLastProcessedFileCount(): number {
return this.fileInfoLastProcessed.size;
}
getLastProcessedFileKeys(): IterableIterator<string> {
return this.fileInfoLastProcessed.keys();
}
hasLastProcessedFile(file: FilePath): boolean {
return this.fileInfoLastProcessed.has(file);
}
hasLastProcessedDatabase(file: FilePath): boolean {
return this.databaseInfoLastProcessed.has(file);
}
async fileToStatKey(file: FilePath, stat: UXStat | null = null): Promise<string> {
// Compatibility question: `null` means 'stat not supplied' here
// rather than 'file missing', so a failed earlier stat causes another
// read. Keep that retry until its event-processing effect is
// characterised.
if (!stat) stat = await this.dependencies.storageAccess.statHidden(file);
return this.storageStateKey(stat);
}
storageStateKey(stat: UXStat | null): string {
return toHiddenFileSyncStorageStateKey(stat);
}
databaseStateKey(doc: MetaEntry | LoadedEntry): string {
return toHiddenFileSyncDatabaseStateKey(doc);
}
updateLastProcessedFile(file: FilePath, keySrc: string | UXStat): void {
const key = typeof keySrc == "string" ? keySrc : this.storageStateKey(keySrc);
const splitted = key.split("-");
if (splitted[0] != "0") {
// Compatibility: a zero storage marker does not replace the last
// known non-zero mtime. Deletion therefore retains that fallback.
this.fileInfoLastKnown.set(file, Number(splitted[0]));
}
this.fileInfoLastProcessed.set(file, key);
}
async updateLastProcessedAsActualFile(file: FilePath, stat?: UXStat | null): Promise<void> {
if (!stat) stat = await this.dependencies.storageAccess.statHidden(file);
// Compatibility: adoption updates only the processed marker. It does
// not update the last-known non-zero mtime cache.
this.fileInfoLastProcessed.set(file, this.storageStateKey(stat));
}
resetLastProcessedFile(targetFiles: FilePath[] | false): void {
if (targetFiles) {
for (const key of targetFiles) {
this.fileInfoLastProcessed.delete(key);
}
} else {
this.log(`Delete all processed mark.`, LOG_LEVEL_VERBOSE);
// THINKING: Should we...
// - delete all `Known file` processed mark? (This is current implementation)
// - delete all `Existing file` processed mark?
// - delete all files inside the config folder of current device mark?
this.fileInfoLastProcessed.clear();
}
}
getLastProcessedFileMTime(file: FilePath): number {
const key = this.fileInfoLastKnown.get(file);
if (!key) return 0;
return key;
}
getLastProcessedFileKey(file: FilePath): string | undefined {
return this.fileInfoLastProcessed.get(file);
}
getLastProcessedDatabaseKey(file: FilePath): string | undefined {
return this.databaseInfoLastProcessed.get(file);
}
updateLastProcessedDatabase(file: FilePath, keySrc: string | MetaEntry | LoadedEntry): void {
const key = typeof keySrc == "string" ? keySrc : this.databaseStateKey(keySrc);
this.databaseInfoLastProcessed.set(file, key);
}
updateLastProcessed(path: FilePath, db: MetaEntry | LoadedEntry, stat: UXStat): void {
this.updateLastProcessedDatabase(path, db);
this.updateLastProcessedFile(path, this.storageStateKey(stat));
const dbMTime = getHiddenFileSyncComparisonMTime(db);
const storageMTime = getHiddenFileSyncComparisonMTime(stat);
if (dbMTime == 0 || storageMTime == 0) {
this.dependencies.path.unmarkChanges(path);
} else {
this.dependencies.path.markChangesAreSame(
path,
getHiddenFileSyncComparisonMTime(db),
getHiddenFileSyncComparisonMTime(stat)
);
}
}
updateLastProcessedDeletion(path: FilePath, db: MetaEntry | LoadedEntry | false): void {
this.dependencies.path.unmarkChanges(path);
if (db) this.updateLastProcessedDatabase(path, db);
this.updateLastProcessedFile(path, this.storageStateKey(null));
}
async updateLastProcessedAsActualDatabase(
file: FilePath,
doc?: MetaEntry | LoadedEntry | null | false
): Promise<void> {
const dbPath = addPrefix(file, ICHeader);
if (!doc) doc = await this.dependencies.getLocalDatabase().getDBEntryMeta(dbPath);
if (!doc) return;
this.databaseInfoLastProcessed.set(file, this.databaseStateKey(doc));
}
resetLastProcessedDatabase(targetFiles: FilePath[] | false): void {
if (targetFiles) {
for (const key of targetFiles) {
this.databaseInfoLastProcessed.delete(key);
}
} else {
this.log(`Delete all processed mark.`, LOG_LEVEL_VERBOSE);
// THINKING: Should we...
// - delete all `Known file` processed mark? (This is current implementation)
// - delete all `Existing file` processed mark?
// - delete all files inside the config folder of current device mark?
this.databaseInfoLastProcessed.clear();
}
}
}
export function createHiddenFileSyncProcessedState(
dependencies: HiddenFileSyncProcessedStateDependencies
): HiddenFileSyncProcessedState {
return new HiddenFileSyncProcessedState(dependencies);
}
@@ -0,0 +1,182 @@
import { describe, expect, it, vi } from "vitest";
import {
LOG_LEVEL_VERBOSE,
type FilePath,
type LoadedEntry,
type MetaEntry,
type UXStat,
} from "@vrtmrz/livesync-commonlib/compat/common/types";
import type { KeyValueDatabase } from "@vrtmrz/livesync-commonlib/compat/interfaces/KeyValueDatabase";
vi.mock("@/deps.ts", () => ({}));
import {
createHiddenFileSyncProcessedState,
type HiddenFileSyncProcessedStateDependencies,
} from "./hiddenFileSyncProcessedState.ts";
import { toHiddenFileSyncDatabaseStateKey } from "./hiddenFileSyncState.ts";
const path = ".obsidian/plugins/example/data.json" as FilePath;
function stat(mtime: number, size = 20): UXStat {
return { ctime: mtime, mtime, size, type: "file" };
}
function metadata(overrides: Partial<MetaEntry> = {}): MetaEntry {
return {
_id: "hidden-entry-id",
_rev: "2-current",
path: `i:${path}`,
type: "plain",
datatype: "plain",
ctime: 10,
mtime: 20,
size: 20,
children: [],
eden: {},
deleted: false,
...overrides,
} as unknown as MetaEntry;
}
function createState() {
const events: string[] = [];
const caches = new Map<string, Map<unknown, unknown>>([
["hidden-file-lastProcessed", new Map([[path, "40-20"]])],
["hidden-file-lastProcessed-database", new Map([[path, "20-20-2-current--1"]])],
["hidden-file-lastKnown", new Map([[path, 40]])],
]);
let activeReads = 0;
let maximumActiveReads = 0;
const keyValueDatabase = {
get: vi.fn(async (key: IDBValidKey) => {
events.push(`get:${String(key)}`);
activeReads++;
maximumActiveReads = Math.max(maximumActiveReads, activeReads);
await Promise.resolve();
activeReads--;
return caches.get(String(key));
}),
set: vi.fn(async () => "ok"),
} as unknown as KeyValueDatabase;
const getDBEntryMeta = vi.fn(async () => false as false | LoadedEntry);
const statHidden = vi.fn(async () => stat(41));
const markChangesAreSame = vi.fn(() => undefined);
const unmarkChanges = vi.fn();
const log = vi.fn();
const dependencies: HiddenFileSyncProcessedStateDependencies = {
getKeyValueDatabase: () => keyValueDatabase,
getLocalDatabase: () => ({ getDBEntryMeta }),
storageAccess: { statHidden },
path: { markChangesAreSame, unmarkChanges },
log,
};
const state = createHiddenFileSyncProcessedState(dependencies);
return {
dependencies,
events,
getDBEntryMeta,
keyValueDatabase,
log,
markChangesAreSame,
maximumActiveReads: () => maximumActiveReads,
state,
statHidden,
unmarkChanges,
};
}
describe("Hidden File Sync processed state", () => {
it("loads the three autosave caches sequentially under their existing keys", async () => {
const fixture = createState();
await fixture.state.initialise();
expect(fixture.events).toEqual([
"get:hidden-file-lastProcessed",
"get:hidden-file-lastProcessed-database",
"get:hidden-file-lastKnown",
]);
expect(fixture.maximumActiveReads()).toBe(1);
expect(fixture.state.getLastProcessedFileCount()).toBe(1);
expect(fixture.state.getLastProcessedFileKey(path)).toBe("40-20");
expect(fixture.state.getLastProcessedDatabaseKey(path)).toBe("20-20-2-current--1");
expect(fixture.state.getLastProcessedFileMTime(path)).toBe(40);
});
it("re-reads a null storage stat and retains the last known mtime on deletion", async () => {
const fixture = createState();
await fixture.state.initialise();
fixture.statHidden.mockResolvedValueOnce(stat(45, 3));
await expect(fixture.state.fileToStatKey(path, null)).resolves.toBe("45-3");
expect(fixture.statHidden).toHaveBeenCalledWith(path);
fixture.state.updateLastProcessedFile(path, stat(45, 3));
fixture.state.updateLastProcessedDeletion(path, metadata({ mtime: 50, size: 0 }));
expect(fixture.state.getLastProcessedFileKey(path)).toBe("0-0");
expect(fixture.state.getLastProcessedFileMTime(path)).toBe(45);
expect(fixture.state.getLastProcessedDatabaseKey(path)).toBe(
toHiddenFileSyncDatabaseStateKey(metadata({ mtime: 50, size: 0 }))
);
expect(fixture.unmarkChanges).toHaveBeenCalledWith(path);
});
it("settles combined state before applying the matching-path marker", async () => {
const fixture = createState();
await fixture.state.initialise();
const document = metadata({ mtime: 60, size: 9 });
const storageStat = stat(61, 9);
fixture.state.updateLastProcessed(path, document, storageStat);
expect(fixture.markChangesAreSame).toHaveBeenCalledWith(path, 60, 61);
expect(fixture.unmarkChanges).not.toHaveBeenCalled();
expect(fixture.state.getLastProcessedDatabaseKey(path)).toBe(toHiddenFileSyncDatabaseStateKey(document));
expect(fixture.state.getLastProcessedFileKey(path)).toBe("61-9");
});
it("resets each processed side without clearing last-known storage mtimes", async () => {
const fixture = createState();
await fixture.state.initialise();
fixture.state.updateLastProcessedFile(path, stat(70, 4));
fixture.state.updateLastProcessedDatabase(path, "database-key");
fixture.state.resetLastProcessedFile([path]);
expect(fixture.state.getLastProcessedFileKey(path)).toBeUndefined();
expect(fixture.state.getLastProcessedDatabaseKey(path)).toBe("database-key");
expect(fixture.state.getLastProcessedFileMTime(path)).toBe(70);
fixture.state.resetLastProcessedDatabase([path]);
expect(fixture.state.getLastProcessedDatabaseKey(path)).toBeUndefined();
});
it("does not settle a database marker for a false or missing database document", async () => {
const fixture = createState();
await fixture.state.initialise();
await fixture.state.updateLastProcessedAsActualDatabase(path, false);
expect(fixture.getDBEntryMeta).toHaveBeenCalledWith(`i:${path}`);
expect(fixture.state.getLastProcessedDatabaseKey(path)).toBe("20-20-2-current--1");
fixture.getDBEntryMeta.mockResolvedValueOnce(false);
await fixture.state.updateLastProcessedAsActualDatabase(path);
expect(fixture.state.getLastProcessedDatabaseKey(path)).toBe("20-20-2-current--1");
});
it("logs and clears both sides when a full reset is requested", async () => {
const fixture = createState();
await fixture.state.initialise();
fixture.state.resetLastProcessedFile(false);
fixture.state.resetLastProcessedDatabase(false);
expect(fixture.log).toHaveBeenNthCalledWith(1, "Delete all processed mark.", LOG_LEVEL_VERBOSE, undefined);
expect(fixture.log).toHaveBeenNthCalledWith(2, "Delete all processed mark.", LOG_LEVEL_VERBOSE, undefined);
expect(fixture.state.getLastProcessedFileCount()).toBe(0);
expect(fixture.state.getLastProcessedFileKey(path)).toBeUndefined();
expect(fixture.state.getLastProcessedDatabaseKey(path)).toBeUndefined();
expect(fixture.state.getLastProcessedFileMTime(path)).toBe(40);
});
});
@@ -0,0 +1,48 @@
import type {
FilePathWithPrefix,
LoadedEntry,
MetaEntry,
UXFileInfo,
UXStat,
} from "@vrtmrz/livesync-commonlib/compat/common/types";
import { displayRev } from "@vrtmrz/livesync-commonlib/compat/common/utils";
import { stripAllPrefixes } from "@vrtmrz/livesync-commonlib/compat/string_and_binary/path";
export function toHiddenFileSyncStorageStateKey(stat: UXStat | null): string {
return `${stat?.mtime ?? 0}-${stat?.size ?? 0}`;
}
export function toHiddenFileSyncDatabaseStateKey(doc: LoadedEntry | MetaEntry): string {
// Compatibility: the deletion marker includes its own hyphen, producing `--0`
// or `--1` after the revision. Existing device-local state uses this format.
return `${doc.mtime}-${doc.size}-${doc._rev}-${doc._deleted || doc.deleted || false ? "-0" : "-1"}`;
}
export function getHiddenFileSyncComparisonMTime(
source: MetaEntry | LoadedEntry | UXFileInfo | UXStat | false | null | undefined,
includeDeleted = false
): number {
if (source === null || source === false || source === undefined) return 0;
if (!includeDeleted) {
if ("deleted" in source && source.deleted) return 0;
if ("_deleted" in source && source._deleted) return 0;
}
if ("stat" in source) return source.stat?.mtime ?? 0;
return source.mtime ?? 0;
}
export function describeHiddenFileSyncDocument(doc: LoadedEntry, prefixedPath: FilePathWithPrefix) {
const id = doc._id;
const path = stripAllPrefixes(prefixedPath);
const rev = doc._rev;
return {
id,
rev,
revDisplay: rev ? displayRev(rev) : "0-NOREVS",
prefixedPath,
path,
isDeleted: doc._deleted || doc.deleted || false,
shortenedId: id.substring(0, 10),
shortenedPath: path.substring(0, 10),
};
}
@@ -0,0 +1,70 @@
import { describe, expect, it } from "vitest";
import type { FilePathWithPrefix, LoadedEntry, UXStat } from "@vrtmrz/livesync-commonlib/compat/common/types";
import {
describeHiddenFileSyncDocument,
getHiddenFileSyncComparisonMTime,
toHiddenFileSyncDatabaseStateKey,
toHiddenFileSyncStorageStateKey,
} from "./hiddenFileSyncState.ts";
describe("Hidden File Sync state keys", () => {
it("represents a missing storage file with zero values", () => {
expect(toHiddenFileSyncStorageStateKey(null)).toBe("0-0");
});
it("uses storage modification time and size", () => {
expect(toHiddenFileSyncStorageStateKey({ mtime: 123, size: 456 } as UXStat)).toBe("123-456");
});
it.each([
[false, "123-456-3-example--1"],
[true, "123-456-3-example--0"],
])("includes database revision and deletion state=%s", (deleted, expected) => {
const doc = { mtime: 123, size: 456, _rev: "3-example", deleted } as LoadedEntry;
expect(toHiddenFileSyncDatabaseStateKey(doc)).toBe(expected);
});
});
describe("getHiddenFileSyncComparisonMTime", () => {
const absentSources = [null, false, undefined] as const;
it.each(absentSources)("returns zero for an absent source=%s", (source) => {
expect(getHiddenFileSyncComparisonMTime(source)).toBe(0);
});
it("reads a direct stat or a file-info stat", () => {
expect(getHiddenFileSyncComparisonMTime({ mtime: 123 } as UXStat)).toBe(123);
expect(getHiddenFileSyncComparisonMTime({ stat: { mtime: 456 } } as never)).toBe(456);
});
it("treats deleted entries as zero unless deletion time is requested", () => {
const deleted = { mtime: 123, deleted: true } as LoadedEntry;
expect(getHiddenFileSyncComparisonMTime(deleted)).toBe(0);
expect(getHiddenFileSyncComparisonMTime(deleted, true)).toBe(123);
});
});
describe("describeHiddenFileSyncDocument", () => {
it("derives the unprefixed path and diagnostic revision fields", () => {
const doc = {
_id: "0123456789abcdef",
_rev: "3-example",
mtime: 123,
size: 456,
deleted: true,
} as LoadedEntry;
expect(describeHiddenFileSyncDocument(doc, "i:.obsidian/app.json" as FilePathWithPrefix)).toEqual({
id: "0123456789abcdef",
rev: "3-example",
revDisplay: "3-exampl",
prefixedPath: "i:.obsidian/app.json",
path: ".obsidian/app.json",
isDeleted: true,
shortenedId: "0123456789",
shortenedPath: ".obsidian/",
});
});
});
@@ -0,0 +1,210 @@
import {
LOG_LEVEL_DEBUG,
LOG_LEVEL_VERBOSE,
type FilePath,
type LoadedEntry,
type LOG_LEVEL,
type UXDataWriteOptions,
type UXFileInfo,
type UXStat,
} from "@vrtmrz/livesync-commonlib/compat/common/types";
import type { StorageAccess } from "@vrtmrz/livesync-commonlib/compat/interfaces/StorageAccess";
import type { LogFunction } from "@vrtmrz/livesync-commonlib/compat/services/lib/logUtils";
import {
createBlob,
isDocContentSame,
readContent,
} from "@vrtmrz/livesync-commonlib/compat/common/utils";
export type HiddenFileSyncStorageAccess = Pick<
StorageAccess,
| "ensureDir"
| "isExistsIncludeHidden"
| "readHiddenFileAuto"
| "removeHidden"
| "statHidden"
| "triggerHiddenFile"
| "writeHiddenFileAuto"
>;
type HiddenFileSyncStorageMethods<Method extends keyof HiddenFileSyncStorageAccess> = {
storageAccess: Pick<HiddenFileSyncStorageAccess, Method>;
};
type HiddenFileSyncLogDependency = {
log: LogFunction;
};
export type HiddenFileSyncStorageDependencies = HiddenFileSyncStorageMethods<keyof HiddenFileSyncStorageAccess> &
HiddenFileSyncLogDependency;
export type HiddenFileSyncRemovalResult = "OK" | "ALREADY" | false;
function log(
dependencies: HiddenFileSyncLogDependency,
message: unknown,
level?: LOG_LEVEL,
key?: string
): void {
dependencies.log(message, level, key);
}
export async function readHiddenFileWithInfo(
dependencies: HiddenFileSyncStorageMethods<"readHiddenFileAuto" | "statHidden">,
path: FilePath
): Promise<UXFileInfo> {
const stat = await dependencies.storageAccess.statHidden(path);
if (!stat) {
return {
name: path.split("/").pop() ?? "",
path,
stat: {
size: 0,
mtime: 0,
ctime: 0,
type: "file",
},
isInternal: true,
deleted: true,
body: createBlob(new Uint8Array(0)),
};
}
const content = await dependencies.storageAccess.readHiddenFileAuto(path);
return {
name: path.split("/").pop() ?? "",
path,
stat,
isInternal: true,
deleted: false,
body: createBlob(content),
};
}
export async function ensureHiddenFileDirectory(
dependencies: HiddenFileSyncStorageMethods<"ensureDir" | "isExistsIncludeHidden">,
path: FilePath
): Promise<void> {
if (!(await dependencies.storageAccess.isExistsIncludeHidden(path))) {
// StorageAccess expects the complete target path and ensures its parent.
await dependencies.storageAccess.ensureDir(path);
}
}
export async function writeHiddenFile(
dependencies: HiddenFileSyncStorageMethods<"statHidden" | "writeHiddenFileAuto">,
path: FilePath,
data: string | ArrayBuffer,
options?: UXDataWriteOptions
): Promise<UXStat | null> {
// Compatibility: the writer's Boolean result is ignored. The post-write stat
// has historically decided whether the operation produced a usable file.
await dependencies.storageAccess.writeHiddenFileAuto(path, data, options);
return await dependencies.storageAccess.statHidden(path);
}
export async function removeHiddenFile(
dependencies: HiddenFileSyncStorageMethods<"isExistsIncludeHidden" | "removeHidden"> &
HiddenFileSyncLogDependency,
path: FilePath
): Promise<HiddenFileSyncRemovalResult> {
try {
if (!(await dependencies.storageAccess.isExistsIncludeHidden(path))) {
return "ALREADY";
}
if (await dependencies.storageAccess.removeHidden(path)) {
return "OK";
}
} catch (error) {
log(dependencies, `Failed to remove file:${path}`);
log(dependencies, error, LOG_LEVEL_VERBOSE);
}
return false;
}
export async function triggerHiddenFileEvent(
dependencies: HiddenFileSyncStorageMethods<"triggerHiddenFile"> & HiddenFileSyncLogDependency,
path: FilePath
): Promise<void> {
try {
await dependencies.storageAccess.triggerHiddenFile(path);
} catch (error) {
log(dependencies, "Failed to call internal API(reconcileInternalFile)", LOG_LEVEL_VERBOSE);
log(dependencies, error, LOG_LEVEL_VERBOSE);
}
}
export async function isHiddenFileWriteRequired(
dependencies: HiddenFileSyncStorageMethods<"readHiddenFileAuto"> & HiddenFileSyncLogDependency,
path: FilePath,
content: string | ArrayBuffer
): Promise<boolean> {
try {
const storageContent = await dependencies.storageAccess.readHiddenFileAuto(path);
return !(await isDocContentSame(storageContent, content));
} catch (error) {
log(dependencies, `Cannot check the content of ${path}`);
log(dependencies, error, LOG_LEVEL_VERBOSE);
// Compatibility: an unreadable current file is treated as requiring a
// write. Changing this policy needs a separate recovery decision.
return true;
}
}
export async function writeHiddenFileFromDatabase(
dependencies: HiddenFileSyncStorageMethods<
"ensureDir" | "isExistsIncludeHidden" | "readHiddenFileAuto" | "statHidden" | "writeHiddenFileAuto"
> &
HiddenFileSyncLogDependency,
path: FilePath,
fileOnDatabase: LoadedEntry,
force: boolean
): Promise<false | UXStat> {
try {
const statBefore = await dependencies.storageAccess.statHidden(path);
const isExisting = statBefore != null;
const content = readContent(fileOnDatabase);
await ensureHiddenFileDirectory(dependencies, path);
const writeRequired =
force || !isExisting || (isExisting && (await isHiddenFileWriteRequired(dependencies, path, content)));
if (!writeRequired) {
log(dependencies, `STORAGE <-- DB: ${path}: skipped (hidden) Not changed`, LOG_LEVEL_DEBUG);
return statBefore;
}
const statAfter = await writeHiddenFile(dependencies, path, content, {
mtime: fileOnDatabase.mtime,
ctime: fileOnDatabase.ctime,
});
if (statAfter == null) {
log(dependencies, `STORAGE <-- DB: ${path}: written (hidden,new${force ? ", force" : ""}) Failed (writeResult)`);
return false;
}
log(dependencies, `STORAGE <-- DB: ${path}: written (hidden, overwrite${force ? ", force" : ""})`);
// Compatibility question: ordinary database reflection does not trigger
// a raw storage event here; deletion and manual JSON merging do. Preserve
// this until the event-loop consequences of changing it are characterised.
return statAfter;
} catch (error) {
log(dependencies, `STORAGE <-- DB: ${path}: written (hidden, overwrite${force ? ", force" : ""}) Failed`);
log(dependencies, error, LOG_LEVEL_VERBOSE);
return false;
}
}
export async function deleteHiddenFileFromStorage(
dependencies: HiddenFileSyncStorageMethods<"isExistsIncludeHidden" | "removeHidden" | "triggerHiddenFile"> &
HiddenFileSyncLogDependency,
path: FilePath
): Promise<HiddenFileSyncRemovalResult> {
const result = await removeHiddenFile(dependencies, path);
if (result === false) {
log(dependencies, `STORAGE <x- DB: ${path}: deleting (hidden) Failed`);
return false;
}
if (result === "OK") {
await triggerHiddenFileEvent(dependencies, path);
}
log(dependencies, `STORAGE <x- DB: ${path}: deleting (hidden) ${result == "OK" ? "Done" : "Already not found"}`);
return result;
}
@@ -0,0 +1,230 @@
import { describe, expect, it, vi } from "vitest";
import {
LOG_LEVEL_VERBOSE,
type FilePath,
type LoadedEntry,
type UXStat,
} from "@vrtmrz/livesync-commonlib/compat/common/types";
import {
deleteHiddenFileFromStorage,
ensureHiddenFileDirectory,
isHiddenFileWriteRequired,
readHiddenFileWithInfo,
removeHiddenFile,
triggerHiddenFileEvent,
writeHiddenFile,
writeHiddenFileFromDatabase,
} from "./hiddenFileSyncStorage.ts";
const path = ".obsidian/plugins/example/data.json" as FilePath;
const stat = { ctime: 10, mtime: 20, size: 4, type: "file" } as UXStat;
function createStorageDependencies() {
const storageAccess = {
ensureDir: vi.fn(async () => true),
isExistsIncludeHidden: vi.fn(async () => true),
readHiddenFileAuto: vi.fn(async () => "data" as string | ArrayBuffer),
removeHidden: vi.fn(async () => true),
statHidden: vi.fn(async () => stat as UXStat | null),
triggerHiddenFile: vi.fn(async () => undefined),
writeHiddenFileAuto: vi.fn(async () => true),
};
const log = vi.fn();
return { dependencies: { storageAccess, log }, log, storageAccess };
}
function databaseEntry(content: string, mtime = 30, ctime = 15): LoadedEntry {
return {
path,
type: "plain",
datatype: "plain",
data: content,
mtime,
ctime,
} as LoadedEntry;
}
describe("Hidden File Sync storage operations", () => {
it("represents a missing hidden file as a deleted empty file", async () => {
const { dependencies, storageAccess } = createStorageDependencies();
storageAccess.statHidden.mockResolvedValue(null);
const result = await readHiddenFileWithInfo(dependencies, path);
expect(result).toMatchObject({
name: "data.json",
path,
isInternal: true,
deleted: true,
stat: { ctime: 0, mtime: 0, size: 0, type: "file" },
});
expect(await result.body.text()).toBe("");
expect(storageAccess.readHiddenFileAuto).not.toHaveBeenCalled();
});
it("loads an existing hidden file with its storage stat", async () => {
const { dependencies, storageAccess } = createStorageDependencies();
storageAccess.readHiddenFileAuto.mockResolvedValue("data");
const result = await readHiddenFileWithInfo(dependencies, path);
expect(result).toMatchObject({ name: "data.json", path, isInternal: true, deleted: false, stat });
expect(await result.body.text()).toBe("data");
});
it("ensures a directory only when the target does not exist", async () => {
const { dependencies, storageAccess } = createStorageDependencies();
await ensureHiddenFileDirectory(dependencies, path);
expect(storageAccess.ensureDir).not.toHaveBeenCalled();
storageAccess.isExistsIncludeHidden.mockResolvedValue(false);
await ensureHiddenFileDirectory(dependencies, path);
expect(storageAccess.ensureDir).toHaveBeenCalledOnce();
expect(storageAccess.ensureDir).toHaveBeenCalledWith(path);
});
it("writes a hidden file and returns the resulting stat", async () => {
const { dependencies, storageAccess } = createStorageDependencies();
await expect(writeHiddenFile(dependencies, path, "data", { mtime: 30, ctime: 15 })).resolves.toBe(stat);
expect(storageAccess.writeHiddenFileAuto).toHaveBeenCalledWith(path, "data", { mtime: 30, ctime: 15 });
expect(storageAccess.statHidden).toHaveBeenCalledWith(path);
});
it("uses the post-write stat even when the storage writer reports false", async () => {
const { dependencies, storageAccess } = createStorageDependencies();
storageAccess.writeHiddenFileAuto.mockResolvedValue(false);
await expect(writeHiddenFile(dependencies, path, "data")).resolves.toBe(stat);
expect(storageAccess.statHidden).toHaveBeenCalledAfter(storageAccess.writeHiddenFileAuto);
});
it("distinguishes an absent, removed, and unremovable file", async () => {
const { dependencies, storageAccess } = createStorageDependencies();
storageAccess.isExistsIncludeHidden.mockResolvedValueOnce(false);
await expect(removeHiddenFile(dependencies, path)).resolves.toBe("ALREADY");
expect(storageAccess.removeHidden).not.toHaveBeenCalled();
storageAccess.isExistsIncludeHidden.mockResolvedValueOnce(true);
storageAccess.removeHidden.mockResolvedValueOnce(true);
await expect(removeHiddenFile(dependencies, path)).resolves.toBe("OK");
storageAccess.isExistsIncludeHidden.mockResolvedValueOnce(true);
storageAccess.removeHidden.mockResolvedValueOnce(false);
await expect(removeHiddenFile(dependencies, path)).resolves.toBe(false);
});
it("turns a removal error into a logged false result", async () => {
const { dependencies, log, storageAccess } = createStorageDependencies();
const error = new Error("remove failed");
storageAccess.isExistsIncludeHidden.mockRejectedValue(error);
await expect(removeHiddenFile(dependencies, path)).resolves.toBe(false);
expect(log).toHaveBeenNthCalledWith(1, `Failed to remove file:${path}`, undefined, undefined);
expect(log).toHaveBeenNthCalledWith(2, error, LOG_LEVEL_VERBOSE, undefined);
});
it("treats a content-read failure as requiring a write", async () => {
const { dependencies, log, storageAccess } = createStorageDependencies();
const error = new Error("read failed");
storageAccess.readHiddenFileAuto.mockRejectedValue(error);
await expect(isHiddenFileWriteRequired(dependencies, path, "data")).resolves.toBe(true);
expect(log).toHaveBeenNthCalledWith(1, `Cannot check the content of ${path}`, undefined, undefined);
expect(log).toHaveBeenNthCalledWith(2, error, LOG_LEVEL_VERBOSE, undefined);
});
it("compares binary content without involving the context", async () => {
const { dependencies, storageAccess } = createStorageDependencies();
storageAccess.readHiddenFileAuto.mockResolvedValue(new Uint8Array([1, 2, 3]).buffer);
await expect(
isHiddenFileWriteRequired(dependencies, path, new Uint8Array([1, 2, 3]).buffer)
).resolves.toBe(false);
await expect(
isHiddenFileWriteRequired(dependencies, path, new Uint8Array([1, 2, 4]).buffer)
).resolves.toBe(true);
});
it("skips an unchanged database file and preserves its current stat", async () => {
const { dependencies, storageAccess } = createStorageDependencies();
storageAccess.readHiddenFileAuto.mockResolvedValue("data");
await expect(writeHiddenFileFromDatabase(dependencies, path, databaseEntry("data"), false)).resolves.toBe(
stat
);
expect(storageAccess.ensureDir).not.toHaveBeenCalled();
expect(storageAccess.writeHiddenFileAuto).not.toHaveBeenCalled();
});
it("writes changed database content with its original timestamps", async () => {
const { dependencies, storageAccess } = createStorageDependencies();
storageAccess.readHiddenFileAuto.mockResolvedValue("old");
await expect(writeHiddenFileFromDatabase(dependencies, path, databaseEntry("new"), false)).resolves.toBe(
stat
);
expect(storageAccess.writeHiddenFileAuto).toHaveBeenCalledWith(path, "new", { mtime: 30, ctime: 15 });
expect(storageAccess.triggerHiddenFile).not.toHaveBeenCalled();
});
it("forces a write without reading existing content", async () => {
const { dependencies, storageAccess } = createStorageDependencies();
await expect(writeHiddenFileFromDatabase(dependencies, path, databaseEntry("data"), true)).resolves.toBe(stat);
expect(storageAccess.readHiddenFileAuto).not.toHaveBeenCalled();
expect(storageAccess.writeHiddenFileAuto).toHaveBeenCalledOnce();
});
it("returns false when a completed write has no resulting stat", async () => {
const { dependencies, storageAccess } = createStorageDependencies();
storageAccess.statHidden.mockResolvedValue(null);
await expect(writeHiddenFileFromDatabase(dependencies, path, databaseEntry("data"), false)).resolves.toBe(
false
);
expect(storageAccess.writeHiddenFileAuto).toHaveBeenCalledOnce();
});
it("triggers a storage event only after an actual deletion", async () => {
const { dependencies, log, storageAccess } = createStorageDependencies();
storageAccess.isExistsIncludeHidden.mockResolvedValueOnce(false);
await expect(deleteHiddenFileFromStorage(dependencies, path)).resolves.toBe("ALREADY");
expect(storageAccess.triggerHiddenFile).not.toHaveBeenCalled();
log.mockClear();
storageAccess.isExistsIncludeHidden.mockResolvedValueOnce(true);
storageAccess.removeHidden.mockResolvedValueOnce(true);
await expect(deleteHiddenFileFromStorage(dependencies, path)).resolves.toBe("OK");
expect(storageAccess.triggerHiddenFile).toHaveBeenCalledOnce();
expect(storageAccess.triggerHiddenFile).toHaveBeenCalledWith(path);
expect(storageAccess.triggerHiddenFile).toHaveBeenCalledAfter(storageAccess.removeHidden);
expect(log).toHaveBeenCalledAfter(storageAccess.triggerHiddenFile);
});
it("does not trigger a storage event after a failed deletion", async () => {
const { dependencies, storageAccess } = createStorageDependencies();
storageAccess.removeHidden.mockResolvedValue(false);
await expect(deleteHiddenFileFromStorage(dependencies, path)).resolves.toBe(false);
expect(storageAccess.triggerHiddenFile).not.toHaveBeenCalled();
});
it("swallows and logs storage-event failures", async () => {
const { dependencies, log, storageAccess } = createStorageDependencies();
const error = new Error("event failed");
storageAccess.triggerHiddenFile.mockRejectedValue(error);
await expect(triggerHiddenFileEvent(dependencies, path)).resolves.toBeUndefined();
expect(log).toHaveBeenNthCalledWith(
1,
"Failed to call internal API(reconcileInternalFile)",
LOG_LEVEL_VERBOSE,
undefined
);
expect(log).toHaveBeenNthCalledWith(2, error, LOG_LEVEL_VERBOSE, undefined);
});
});
@@ -1,5 +1,12 @@
import type { InternalFileInfo } from "@/common/types.ts";
import type { FilePath } from "@vrtmrz/livesync-commonlib/compat/common/types";
import type {
FilePath,
FilePathWithPrefix,
LoadedEntry,
UXFileInfo,
} from "@vrtmrz/livesync-commonlib/compat/common/types";
import type { OptionalSyncFeatureMode } from "@/features/optionalSyncFeatures.ts";
import type { HiddenFileSyncConflictTestingView } from "./hiddenFileSyncConflictResolution.ts";
export type HiddenFileSyncInitialisationDirection = "push" | "pull" | "safe" | "pullForce" | "pushForce";
@@ -12,6 +19,91 @@ export interface HiddenFileSyncInitialisationView {
): Promise<void>;
}
/**
* Semantic callbacks consumed by the Commonlib service registries.
*
* The context owns the implementations, while the optional-file composition
* adapts this view to Commonlib's aggregation contracts. Callers do not bind
* the context or depend on registry-oriented method names.
*/
export interface HiddenFileSyncServiceHandlerView {
readonly processOptionalFileEvent: (path: FilePath) => Promise<boolean>;
readonly processOptionalSyncFiles: (doc: LoadedEntry) => Promise<boolean>;
readonly onSettingLoaded: () => Promise<boolean>;
readonly realiseSettingSyncMode: () => Promise<boolean>;
readonly onResuming: () => Promise<boolean>;
readonly beforeReplicate: (showNotice: boolean) => Promise<boolean>;
readonly onDatabaseInitialised: (showNotice: boolean) => Promise<boolean>;
readonly suspendExtraSync: () => Promise<boolean>;
readonly configureOptionalSyncFeature: (mode: OptionalSyncFeatureMode) => Promise<boolean>;
readonly isTargetFileEligible: (path: FilePath) => Promise<boolean>;
readonly queueConflict: (path: FilePathWithPrefix) => Promise<boolean>;
}
export function createHiddenFileSyncServiceHandlerView(
operations: HiddenFileSyncServiceHandlerView
): HiddenFileSyncServiceHandlerView {
const view: HiddenFileSyncServiceHandlerView = {
processOptionalFileEvent: async (path) => await operations.processOptionalFileEvent(path),
processOptionalSyncFiles: async (doc) => await operations.processOptionalSyncFiles(doc),
onSettingLoaded: async () => await operations.onSettingLoaded(),
realiseSettingSyncMode: async () => await operations.realiseSettingSyncMode(),
onResuming: async () => await operations.onResuming(),
beforeReplicate: async (showNotice) => await operations.beforeReplicate(showNotice),
onDatabaseInitialised: async (showNotice) => await operations.onDatabaseInitialised(showNotice),
suspendExtraSync: async () => await operations.suspendExtraSync(),
configureOptionalSyncFeature: async (mode) => await operations.configureOptionalSyncFeature(mode),
isTargetFileEligible: async (path) => await operations.isTargetFileEligible(path),
queueConflict: async (path) => await operations.queueConflict(path),
};
return Object.freeze(view);
}
export type HiddenFileSyncTestingRebuild = (
showNotice: boolean,
targetFiles?: FilePath[] | false
) => Promise<FilePath[]>;
export type HiddenFileSyncTestingRebuildInterceptor = (
runRebuild: HiddenFileSyncTestingRebuild,
showNotice: boolean,
targetFiles?: FilePath[] | false
) => Promise<FilePath[]>;
/** Operations exposed to the real-Obsidian contract tests. */
export interface HiddenFileSyncTestingView extends HiddenFileSyncCommandView {
readonly conflictResolution: HiddenFileSyncConflictTestingView;
readFileWithInfo(path: FilePath): Promise<UXFileInfo>;
showConfigurationChangeNotice(updatedFolders: readonly string[]): void;
interceptRebuildMerging(interceptor: HiddenFileSyncTestingRebuildInterceptor): () => void;
}
export type HiddenFileSyncTestingViewOperations = HiddenFileSyncTestingView;
/**
* Build the frozen testing seam. Tests can observe behaviour and install a
* scoped timing interceptor, but cannot access mutable context state.
*/
export function createHiddenFileSyncTestingView(
operations: HiddenFileSyncTestingViewOperations
): HiddenFileSyncTestingView {
const view: HiddenFileSyncTestingView = {
isManualCommandAvailable: () => operations.isManualCommandAvailable(),
scanAllStorageChanges: async (showNotice) => await operations.scanAllStorageChanges(showNotice),
scanAllDatabaseChanges: async (showNotice) => await operations.scanAllDatabaseChanges(showNotice),
applyOfflineChanges: async (showNotice) => await operations.applyOfflineChanges(showNotice),
updateSettingCache: () => operations.updateSettingCache(),
initialiseInternalFileSync: async (direction, showMessage, targetFiles) =>
await operations.initialiseInternalFileSync(direction, showMessage, targetFiles),
conflictResolution: operations.conflictResolution,
readFileWithInfo: async (path) => await operations.readFileWithInfo(path),
showConfigurationChangeNotice: (updatedFolders) =>
operations.showConfigurationChangeNotice(updatedFolders),
interceptRebuildMerging: (interceptor) => operations.interceptRebuildMerging(interceptor),
};
return Object.freeze(view);
}
/** Exact-revision operations needed by the Hatch repair pane. */
export interface HiddenFileSyncRepairView {
scanInternalFiles(): Promise<InternalFileInfo[]>;
@@ -28,6 +120,19 @@ export interface HiddenFileSyncRepairView {
): Promise<boolean>;
}
export function createHiddenFileSyncRepairView(operations: HiddenFileSyncRepairView): HiddenFileSyncRepairView {
const view: HiddenFileSyncRepairView = {
scanInternalFiles: async () => await operations.scanInternalFiles(),
storeInternalFileToDatabase: async (file, forceWrite) =>
await operations.storeInternalFileToDatabase(file, forceWrite),
storeInternalFileToDatabaseWithBaseRevision: async (file, baseRevision, createIfDifferent) =>
await operations.storeInternalFileToDatabaseWithBaseRevision(file, baseRevision, createIfDifferent),
extractInternalFileRevisionFromDatabase: async (storageFilePath, revision, force) =>
await operations.extractInternalFileRevisionFromDatabase(storageFilePath, revision, force),
};
return Object.freeze(view);
}
/** Operations consumed by the host-owned Hidden File Sync commands. */
export interface HiddenFileSyncCommandView extends HiddenFileSyncInitialisationView {
isManualCommandAvailable(): boolean;
@@ -0,0 +1,132 @@
import { describe, expect, it, vi } from "vitest";
import type { FilePath } from "@vrtmrz/livesync-commonlib/compat/common/types";
import {
createHiddenFileSyncRepairView,
createHiddenFileSyncServiceHandlerView,
createHiddenFileSyncTestingView,
type HiddenFileSyncRepairView,
type HiddenFileSyncServiceHandlerView,
type HiddenFileSyncTestingViewOperations,
} from "./hiddenFileSyncViews.ts";
describe("Hidden File Sync repair view", () => {
it("exposes only frozen repair operations and preserves their receiver", async () => {
const path = ".obsidian/app.json" as FilePath;
const file = { path, ctime: 1, mtime: 2, size: 3 };
const source = {
marker: "source",
scanInternalFiles: vi.fn(async function (this: { marker: string }) {
expect(this.marker).toBe("source");
return [file];
}),
storeInternalFileToDatabase: vi.fn(async function (this: { marker: string }) {
expect(this.marker).toBe("source");
return true;
}),
storeInternalFileToDatabaseWithBaseRevision: vi.fn(async function (this: { marker: string }) {
expect(this.marker).toBe("source");
return true;
}),
extractInternalFileRevisionFromDatabase: vi.fn(async function (this: { marker: string }) {
expect(this.marker).toBe("source");
return true;
}),
} as unknown as HiddenFileSyncRepairView;
const view = createHiddenFileSyncRepairView(source);
expect(view).not.toBe(source);
expect(Object.isFrozen(view)).toBe(true);
expect(Object.keys(view).sort()).toEqual(
[
"extractInternalFileRevisionFromDatabase",
"scanInternalFiles",
"storeInternalFileToDatabase",
"storeInternalFileToDatabaseWithBaseRevision",
].sort()
);
await expect(view.scanInternalFiles()).resolves.toEqual([file]);
await expect(view.storeInternalFileToDatabase(file)).resolves.toBe(true);
await expect(view.storeInternalFileToDatabaseWithBaseRevision(file, "2-selected", false)).resolves.toBe(true);
await expect(view.extractInternalFileRevisionFromDatabase(path, "2-selected", true)).resolves.toBe(true);
expect(source.storeInternalFileToDatabaseWithBaseRevision).toHaveBeenCalledWith(file, "2-selected", false);
});
});
describe("Hidden File Sync service-handler view", () => {
it("forwards semantic callbacks through a frozen view", async () => {
const operations = {
processOptionalFileEvent: vi.fn(async () => true),
processOptionalSyncFiles: vi.fn(async () => true),
onSettingLoaded: vi.fn(async () => true),
realiseSettingSyncMode: vi.fn(async () => true),
onResuming: vi.fn(async () => true),
beforeReplicate: vi.fn(async () => true),
onDatabaseInitialised: vi.fn(async () => true),
suspendExtraSync: vi.fn(async () => true),
configureOptionalSyncFeature: vi.fn(async () => true),
isTargetFileEligible: vi.fn(async () => true),
queueConflict: vi.fn(async () => true),
} satisfies HiddenFileSyncServiceHandlerView;
const view = createHiddenFileSyncServiceHandlerView(operations);
expect(Object.isFrozen(view)).toBe(true);
await view.processOptionalFileEvent(".obsidian/app.json" as FilePath);
await view.processOptionalSyncFiles({} as never);
await view.onSettingLoaded();
await view.realiseSettingSyncMode();
await view.onResuming();
await view.beforeReplicate(true);
await view.onDatabaseInitialised(false);
await view.suspendExtraSync();
await view.configureOptionalSyncFeature("MERGE");
await view.isTargetFileEligible(".obsidian/app.json" as FilePath);
await view.queueConflict("i:.obsidian/app.json" as never);
expect(operations.processOptionalFileEvent).toHaveBeenCalledWith(".obsidian/app.json");
expect(operations.beforeReplicate).toHaveBeenCalledWith(true);
expect(operations.onDatabaseInitialised).toHaveBeenCalledWith(false);
expect(operations.configureOptionalSyncFeature).toHaveBeenCalledWith("MERGE");
expect(operations.queueConflict).toHaveBeenCalledWith("i:.obsidian/app.json");
});
});
describe("Hidden File Sync testing view", () => {
it("keeps test operations focused while retaining a frozen E2E surface", async () => {
const conflictResolution = {
resolveAll: vi.fn(async () => undefined),
resolveJson: vi.fn(async () => true),
pendingPaths: [],
processor: { remaining: 0, totalRemaining: 0, nowProcessing: 0 },
};
const restoreRebuild = vi.fn();
const operations = {
isManualCommandAvailable: vi.fn(() => true),
scanAllStorageChanges: vi.fn(async () => undefined),
scanAllDatabaseChanges: vi.fn(async () => undefined),
applyOfflineChanges: vi.fn(async () => undefined),
updateSettingCache: vi.fn(),
initialiseInternalFileSync: vi.fn(async () => undefined),
conflictResolution,
readFileWithInfo: vi.fn(async () => ({}) as never),
showConfigurationChangeNotice: vi.fn(),
interceptRebuildMerging: vi.fn(() => restoreRebuild),
} satisfies HiddenFileSyncTestingViewOperations;
const view = createHiddenFileSyncTestingView(operations);
expect(Object.isFrozen(view)).toBe(true);
await view.scanAllStorageChanges(true);
await view.readFileWithInfo(".obsidian/app.json" as FilePath);
view.showConfigurationChangeNotice([".obsidian"]);
expect(operations.scanAllStorageChanges).toHaveBeenCalledWith(true);
expect(operations.readFileWithInfo).toHaveBeenCalledWith(".obsidian/app.json");
expect(operations.showConfigurationChangeNotice).toHaveBeenCalledWith([".obsidian"]);
const interceptor = vi.fn(async () => [] as FilePath[]);
expect(view.interceptRebuildMerging(interceptor)).toBe(restoreRebuild);
expect(operations.interceptRebuildMerging).toHaveBeenCalledWith(interceptor);
});
});
+51
View File
@@ -0,0 +1,51 @@
export type OptionalFileSyncDirectoryListing = {
files: readonly string[];
folders: readonly string[];
};
export type OptionalFileSyncFileTreeDependencies = {
listFiles(path: string): Promise<OptionalFileSyncDirectoryListing>;
};
export type OptionalFileSyncFileTreeOptions = {
maxDepth?: number;
shouldInclude?(path: string): boolean | Promise<boolean>;
onError?(path: string, error: unknown): void;
};
export async function collectOptionalFileSyncFiles(
dependencies: OptionalFileSyncFileTreeDependencies,
path: string,
options: OptionalFileSyncFileTreeOptions = {}
): Promise<string[]> {
if (options.maxDepth !== undefined && options.maxDepth < 0) {
return [];
}
let listing: OptionalFileSyncDirectoryListing;
try {
listing = await dependencies.listFiles(path);
} catch (error) {
options.onError?.(path, error);
return [];
}
const files: string[] = [];
for (const file of listing.files) {
if ((await options.shouldInclude?.(file)) ?? true) {
files.push(file);
}
}
for (const folder of listing.folders) {
if (!((await options.shouldInclude?.(folder)) ?? true)) {
continue;
}
files.push(
...(await collectOptionalFileSyncFiles(dependencies, folder, {
...options,
maxDepth: options.maxDepth === undefined ? undefined : options.maxDepth - 1,
}))
);
}
return files;
}
@@ -0,0 +1,48 @@
import { describe, expect, it, vi } from "vitest";
import { collectOptionalFileSyncFiles } from "./optionalFileSyncFileTree.ts";
describe("collectOptionalFileSyncFiles", () => {
it("collects files to the requested directory depth", async () => {
const listings = new Map([
["root", { files: ["root/file"], folders: ["root/one"] }],
["root/one", { files: ["root/one/file"], folders: ["root/one/two"] }],
["root/one/two", { files: ["root/one/two/file"], folders: ["root/one/two/three"] }],
["root/one/two/three", { files: ["root/one/two/three/file"], folders: [] }],
]);
const listFiles = vi.fn(async (path: string) => listings.get(path) ?? { files: [], folders: [] });
await expect(collectOptionalFileSyncFiles({ listFiles }, "root", { maxDepth: 2 })).resolves.toEqual([
"root/file",
"root/one/file",
"root/one/two/file",
]);
expect(listFiles).not.toHaveBeenCalledWith("root/one/two/three");
});
it("uses the same asynchronous filter for files and directory traversal", async () => {
const listFiles = vi.fn(async (path: string) =>
path == "root"
? { files: ["root/include", "root/skip"], folders: ["root/allowed", "root/blocked"] }
: { files: [`${path}/include`], folders: [] }
);
const shouldInclude = vi.fn(async (path: string) => !path.includes("skip") && !path.includes("blocked"));
await expect(collectOptionalFileSyncFiles({ listFiles }, "root", { shouldInclude })).resolves.toEqual([
"root/include",
"root/allowed/include",
]);
expect(listFiles).not.toHaveBeenCalledWith("root/blocked");
});
it("reports an unreadable directory and keeps the successful part of the traversal", async () => {
const failure = new Error("unreadable");
const listFiles = vi.fn(async (path: string) => {
if (path == "root/failing") throw failure;
return { files: ["root/file"], folders: ["root/failing"] };
});
const onError = vi.fn();
await expect(collectOptionalFileSyncFiles({ listFiles }, "root", { onError })).resolves.toEqual(["root/file"]);
expect(onError).toHaveBeenCalledWith("root/failing", failure);
});
});
@@ -11,9 +11,9 @@ import { getObsidianCommunityPluginManager } from "@/common/obsidianCommunityPlu
import { JsonResolveModal } from "@/features/HiddenFileCommon/JsonResolveModal.ts";
import type {
HiddenFileSyncContextDependencies,
HiddenFileSyncJsonResolution,
HiddenFileSyncProgress,
} from "@/features/HiddenFileSync/hiddenFileSyncContext.ts";
import type { HiddenFileSyncJsonResolution } from "@/features/HiddenFileSync/hiddenFileSyncConflictResolution.ts";
import { MARK_DONE } from "@/modules/features/ModuleLog.ts";
import type { LiveSyncCore } from "@/main.ts";
@@ -18,8 +18,14 @@ describe("Optional File Sync ownership boundary", () => {
expect(mainSource).not.toContain("optionalFileSync.testing");
expect(featureSource).toContain("new CustomisationSyncContext");
expect(featureSource).toContain("new HiddenFileSyncContext");
expect(featureSource).toContain("customisationSync.serviceHandlers");
expect(featureSource).toContain("hiddenFileSync.serviceHandlers");
expect(featureSource).toContain("customisationSync.testing");
expect(featureSource).toContain("hiddenFileSync.testing");
expect(customisationSource).not.toContain("onBindFunction(");
expect(hiddenSource).not.toContain("onBindFunction(");
expect(customisationSource).not.toMatch(/\b_(?:any|every|all)[A-Z]/);
expect(hiddenSource).not.toMatch(/\b_(?:any|every|all)[A-Z]/);
expect(customisationSource).not.toContain("LiveSyncCommands");
expect(hiddenSource).not.toContain("LiveSyncCommands");
expect(customisationSource).not.toContain("extends LiveSyncContext");
@@ -15,6 +15,7 @@ import {
type CustomisationSyncFileCategory,
type CustomisationSyncPathOptions,
} from "@/features/ConfigSync/customisationSyncPaths.ts";
import { isHiddenFileSyncPath } from "@/features/HiddenFileSync/hiddenFileSyncPathPolicy.ts";
export type OptionalFileSyncOwner = "customisation" | "hidden-file" | "none";
@@ -58,10 +59,6 @@ export type CustomisationSyncDocumentOwnershipInput = {
pluginSyncExtendedSetting: Readonly<Record<string, PluginSyncSettingEntry>>;
};
function isHiddenFileSyncPath(path: string): boolean {
return path.startsWith(".") && !path.startsWith(".trash");
}
/** Select the sole local writer from persisted feature settings. */
export function selectOptionalFileSyncOwner(
input: OptionalFileSyncOwnerSelectionInput
+38 -70
View File
@@ -10,12 +10,16 @@ import {
CustomisationSyncContext,
type CustomisationSyncContextDependencies,
} from "@/features/ConfigSync/customisationSyncContext.ts";
import type { CustomisationSyncDialogView } from "@/features/ConfigSync/customisationSyncView.ts";
import type {
CustomisationSyncDialogView,
CustomisationSyncTestingView,
} from "@/features/ConfigSync/customisationSyncView.ts";
import { HiddenFileSyncContext } from "@/features/HiddenFileSync/hiddenFileSyncContext.ts";
import type {
HiddenFileSyncCommandView,
HiddenFileSyncInitialisationView,
HiddenFileSyncRepairView,
HiddenFileSyncTestingView,
} from "@/features/HiddenFileSync/hiddenFileSyncViews.ts";
import type { LiveSyncCore } from "@/main.ts";
import {
@@ -38,10 +42,10 @@ export interface OptionalFileSyncFeature {
readonly hiddenFileSyncCommands: HiddenFileSyncCommandView;
readonly hiddenFileSyncInitialisation: HiddenFileSyncInitialisationView;
readonly hiddenFileSyncRepair: HiddenFileSyncRepairView;
/** @internal Direct runtime access for the repository's real-Obsidian contract tests. */
/** @internal Focused operations for the repository's real-Obsidian contract tests. */
readonly testing: {
readonly customisationSync: CustomisationSyncContext;
readonly hiddenFileSync: HiddenFileSyncContext;
readonly customisationSync: CustomisationSyncTestingView;
readonly hiddenFileSync: HiddenFileSyncTestingView;
};
}
@@ -96,6 +100,9 @@ export function useOptionalFileSync(
const hiddenFileSync = createHiddenFileSync({
ownsLocalFile: ownsLocalFile("hidden-file"),
});
const customisationHandlers = customisationSync.serviceHandlers;
const hiddenFileHandlers = hiddenFileSync.serviceHandlers;
const hiddenFileSyncRepair = hiddenFileSync.repair;
const { services } = host;
const disposers: (() => void)[] = [];
@@ -108,7 +115,7 @@ export function useOptionalFileSync(
const routeLocalPath = async (path: FilePath) => {
const selected = selectOptionalFileSyncOwner(ownerSelectionInput(path));
const hiddenFileEligible =
selected.owner == "hidden-file" ? await hiddenFileSync.isTargetFileEligible(path) : false;
selected.owner == "hidden-file" ? await hiddenFileHandlers.isTargetFileEligible(path) : false;
const ready = services.appLifecycle.isReady() && !services.appLifecycle.isSuspended();
return routeOptionalFileSyncPath({
...ownerSelectionInput(path),
@@ -123,86 +130,44 @@ export function useOptionalFileSync(
const localPath = normaliseLocalPath(path);
const decision = await routeLocalPath(localPath);
if (decision.owner == "customisation") {
return await customisationSync._anyProcessOptionalFileEvent(localPath);
return await customisationHandlers.processOptionalFileEvent(localPath);
}
if (decision.owner == "hidden-file") {
return await hiddenFileSync._anyProcessOptionalFileEvent(localPath);
return await hiddenFileHandlers.processOptionalFileEvent(localPath);
}
return false;
})
);
register(
services.conflict.getOptionalConflictCheckMethod.addHandler(async (path: FilePathWithPrefix) => {
services.conflict.getOptionalConflictCheckMethod.addHandler((path: FilePathWithPrefix) => {
if (isPluginMetadata(path) || isCustomisationSyncMetadata(path)) {
return await customisationSync._anyGetOptionalConflictCheckMethod(path);
return Promise.resolve("newer");
}
if (isInternalMetadata(path)) {
return await hiddenFileSync._anyGetOptionalConflictCheckMethod(path);
return hiddenFileHandlers.queueConflict(path);
}
return false;
return Promise.resolve(false);
})
);
register(services.replication.processVirtualDocument.addHandler(customisationHandlers.processVirtualDocument));
register(
services.replication.processVirtualDocument.addHandler(
customisationSync._anyModuleParsedReplicationResultItem.bind(customisationSync)
)
);
register(
services.replication.processOptionalSynchroniseResult.addHandler(
hiddenFileSync._anyProcessOptionalSyncFiles.bind(hiddenFileSync)
)
);
register(
services.appLifecycle.onSettingLoaded.addHandler(
hiddenFileSync._everyOnloadAfterLoadSettings.bind(hiddenFileSync)
)
services.replication.processOptionalSynchroniseResult.addHandler(hiddenFileHandlers.processOptionalSyncFiles)
);
register(services.appLifecycle.onSettingLoaded.addHandler(hiddenFileHandlers.onSettingLoaded));
register(
services.setting.onRealiseSetting.addHandler(
customisationSync._everyRealizeSettingSyncMode.bind(customisationSync)
)
);
register(
services.setting.onRealiseSetting.addHandler(hiddenFileSync._everyRealizeSettingSyncMode.bind(hiddenFileSync))
);
register(
services.appLifecycle.onResuming.addHandler(customisationSync._everyOnResumeProcess.bind(customisationSync))
);
register(services.appLifecycle.onResuming.addHandler(hiddenFileSync._everyOnResumeProcess.bind(hiddenFileSync)));
register(
services.replication.onBeforeReplicate.addHandler(
customisationSync._everyBeforeReplicate.bind(customisationSync)
)
);
register(
services.replication.onBeforeReplicate.addHandler(hiddenFileSync._everyBeforeReplicate.bind(hiddenFileSync))
);
register(
services.databaseEvents.onDatabaseInitialised.addHandler(
customisationSync._everyOnDatabaseInitialized.bind(customisationSync)
)
);
register(
services.databaseEvents.onDatabaseInitialised.addHandler(
hiddenFileSync._everyOnDatabaseInitialized.bind(hiddenFileSync)
)
);
register(
services.setting.suspendExtraSync.addHandler(customisationSync._allSuspendExtraSync.bind(customisationSync))
);
register(services.setting.suspendExtraSync.addHandler(hiddenFileSync._allSuspendExtraSync.bind(hiddenFileSync)));
register(
services.setting.enableOptionalFeature.addHandler(
customisationSync._allConfigureOptionalSyncFeature.bind(customisationSync)
)
);
register(
services.setting.enableOptionalFeature.addHandler(
hiddenFileSync._allConfigureOptionalSyncFeature.bind(hiddenFileSync)
)
);
register(services.setting.onRealiseSetting.addHandler(customisationHandlers.onRealiseSetting));
register(services.setting.onRealiseSetting.addHandler(hiddenFileHandlers.realiseSettingSyncMode));
register(services.appLifecycle.onResuming.addHandler(customisationHandlers.onResuming));
register(services.appLifecycle.onResuming.addHandler(hiddenFileHandlers.onResuming));
register(services.replication.onBeforeReplicate.addHandler(customisationHandlers.onBeforeReplicate));
register(services.replication.onBeforeReplicate.addHandler(hiddenFileHandlers.beforeReplicate));
register(services.databaseEvents.onDatabaseInitialised.addHandler(customisationHandlers.onDatabaseInitialised));
register(services.databaseEvents.onDatabaseInitialised.addHandler(hiddenFileHandlers.onDatabaseInitialised));
register(services.setting.suspendExtraSync.addHandler(customisationHandlers.suspendExtraSync));
register(services.setting.suspendExtraSync.addHandler(hiddenFileHandlers.suspendExtraSync));
register(services.setting.enableOptionalFeature.addHandler(customisationHandlers.enableOptionalFeature));
register(services.setting.enableOptionalFeature.addHandler(hiddenFileHandlers.configureOptionalSyncFeature));
register(
services.vault.isTargetFileInExtra.addHandler(
async (file: string | UXFileInfoStub) => (await routeLocalPath(normaliseLocalPath(file))).owner != "none"
@@ -234,7 +199,10 @@ export function useOptionalFileSync(
customisationSync,
hiddenFileSyncCommands: hiddenFileSync,
hiddenFileSyncInitialisation: hiddenFileSync,
hiddenFileSyncRepair: hiddenFileSync,
testing: Object.freeze({ customisationSync, hiddenFileSync }),
hiddenFileSyncRepair,
testing: Object.freeze({
customisationSync: customisationSync.testing,
hiddenFileSync: hiddenFileSync.testing,
}),
});
}
@@ -38,6 +38,15 @@ function handlerRegistry() {
};
}
function hiddenFileSyncRepairFixture() {
return Object.freeze({
scanInternalFiles: vi.fn(async () => []),
storeInternalFileToDatabase: vi.fn(async () => true),
storeInternalFileToDatabaseWithBaseRevision: vi.fn(async () => true),
extractInternalFileRevisionFromDatabase: vi.fn(async () => true),
});
}
function createFixture() {
const processOptionalFileEvent = handlerRegistry();
const getOptionalConflictCheckMethod = handlerRegistry();
@@ -54,31 +63,42 @@ function createFixture() {
const onUnload = handlerRegistry();
const calls: string[] = [];
const customisationHandlers = {
processOptionalFileEvent: vi.fn(async () => false),
processVirtualDocument: vi.fn(async () => false),
onRealiseSetting: vi.fn(async () => true),
onResuming: vi.fn(async () => true),
onBeforeReplicate: vi.fn(async () => true),
onDatabaseInitialised: vi.fn(async () => true),
suspendExtraSync: vi.fn(async () => true),
enableOptionalFeature: vi.fn(async () => true),
};
const customisationTesting = Object.freeze({ kind: "customisation-testing" });
const customisationSync = {
dispose: vi.fn(() => calls.push("customisation:unload")),
_anyProcessOptionalFileEvent: vi.fn(async () => false),
_anyGetOptionalConflictCheckMethod: vi.fn(async () => false),
_anyModuleParsedReplicationResultItem: vi.fn(async () => false),
_everyRealizeSettingSyncMode: vi.fn(async () => true),
_everyOnResumeProcess: vi.fn(async () => true),
_everyBeforeReplicate: vi.fn(async () => true),
_everyOnDatabaseInitialized: vi.fn(async () => true),
_allSuspendExtraSync: vi.fn(async () => true),
_allConfigureOptionalSyncFeature: vi.fn(async () => true),
serviceHandlers: Object.freeze(customisationHandlers),
testing: customisationTesting,
};
const hiddenFileHandlers = {
onSettingLoaded: vi.fn(async () => true),
processOptionalFileEvent: vi.fn(async () => false),
queueConflict: vi.fn(async () => true),
processOptionalSyncFiles: vi.fn(async () => false),
realiseSettingSyncMode: vi.fn(async () => true),
onResuming: vi.fn(async () => true),
beforeReplicate: vi.fn(async () => true),
onDatabaseInitialised: vi.fn(async () => true),
suspendExtraSync: vi.fn(async () => true),
configureOptionalSyncFeature: vi.fn(async () => true),
isTargetFileEligible: vi.fn(async () => true),
};
const hiddenFileSyncRepair = hiddenFileSyncRepairFixture();
const hiddenFileTesting = Object.freeze({ kind: "hidden-file-testing" });
const hiddenFileSync = {
dispose: vi.fn(() => calls.push("hidden:unload")),
_everyOnloadAfterLoadSettings: vi.fn(async () => true),
_anyProcessOptionalFileEvent: vi.fn(async () => false),
_anyGetOptionalConflictCheckMethod: vi.fn(async () => false),
_anyProcessOptionalSyncFiles: vi.fn(async () => false),
_everyRealizeSettingSyncMode: vi.fn(async () => true),
_everyOnResumeProcess: vi.fn(async () => true),
_everyBeforeReplicate: vi.fn(async () => true),
_everyOnDatabaseInitialized: vi.fn(async () => true),
_allSuspendExtraSync: vi.fn(async () => true),
_allConfigureOptionalSyncFeature: vi.fn(async () => true),
isTargetFileEligible: vi.fn(async () => true),
serviceHandlers: Object.freeze(hiddenFileHandlers),
testing: hiddenFileTesting,
repair: hiddenFileSyncRepair,
};
const settings = {
usePluginSync: true,
@@ -121,9 +141,13 @@ function createFixture() {
return {
calls,
customisationHandlers,
customisationSync,
customisationTesting,
feature,
hiddenFileHandlers,
hiddenFileSync,
hiddenFileTesting,
settings,
contextDependencies: {
customisation: () => customisationDependencies,
@@ -148,31 +172,39 @@ function createFixture() {
}
function createAggregateFixture() {
const customisationHandlers = {
processOptionalFileEvent: vi.fn(async () => false),
processVirtualDocument: vi.fn(async () => false),
onRealiseSetting: vi.fn(async () => true),
onResuming: vi.fn(async () => true),
onBeforeReplicate: vi.fn(async () => true),
onDatabaseInitialised: vi.fn(async () => true),
suspendExtraSync: vi.fn(async () => true),
enableOptionalFeature: vi.fn(async () => true),
};
const customisationSync = {
dispose: vi.fn(),
_anyProcessOptionalFileEvent: vi.fn(async () => false),
_anyGetOptionalConflictCheckMethod: vi.fn(async (): Promise<boolean | "newer"> => false),
_anyModuleParsedReplicationResultItem: vi.fn(async () => false),
_everyRealizeSettingSyncMode: vi.fn(async () => true),
_everyOnResumeProcess: vi.fn(async () => true),
_everyBeforeReplicate: vi.fn(async () => true),
_everyOnDatabaseInitialized: vi.fn(async () => true),
_allSuspendExtraSync: vi.fn(async () => true),
_allConfigureOptionalSyncFeature: vi.fn(async () => true),
serviceHandlers: Object.freeze(customisationHandlers),
testing: Object.freeze({ kind: "customisation-testing" }),
};
const hiddenFileHandlers = {
onSettingLoaded: vi.fn(async () => true),
processOptionalFileEvent: vi.fn(async () => false),
queueConflict: vi.fn(async () => true),
processOptionalSyncFiles: vi.fn(async () => false),
realiseSettingSyncMode: vi.fn(async () => true),
onResuming: vi.fn(async () => true),
beforeReplicate: vi.fn(async () => true),
onDatabaseInitialised: vi.fn(async () => true),
suspendExtraSync: vi.fn(async () => true),
configureOptionalSyncFeature: vi.fn(async () => true),
isTargetFileEligible: vi.fn(async () => true),
};
const hiddenFileSync = {
dispose: vi.fn(),
_everyOnloadAfterLoadSettings: vi.fn(async () => true),
_anyProcessOptionalFileEvent: vi.fn(async () => false),
_anyGetOptionalConflictCheckMethod: vi.fn(async (): Promise<boolean | "newer"> => false),
_anyProcessOptionalSyncFiles: vi.fn(async () => false),
_everyRealizeSettingSyncMode: vi.fn(async () => true),
_everyOnResumeProcess: vi.fn(async () => true),
_everyBeforeReplicate: vi.fn(async () => true),
_everyOnDatabaseInitialized: vi.fn(async () => true),
_allSuspendExtraSync: vi.fn(async () => true),
_allConfigureOptionalSyncFeature: vi.fn(async () => true),
isTargetFileEligible: vi.fn(async () => true),
serviceHandlers: Object.freeze(hiddenFileHandlers),
testing: Object.freeze({ kind: "hidden-file-testing" }),
repair: hiddenFileSyncRepairFixture(),
};
const settings = {
usePluginSync: true,
@@ -217,7 +249,7 @@ function createAggregateFixture() {
createHiddenFileSync: () => hiddenFileSync as never,
});
return { customisationSync, hiddenFileSync, services, settings };
return { customisationHandlers, customisationSync, hiddenFileHandlers, hiddenFileSync, services, settings };
}
describe("useOptionalFileSync", () => {
@@ -239,10 +271,10 @@ describe("useOptionalFileSync", () => {
});
it("routes Selective and Automatic paths to exactly one local owner", async () => {
const { customisationSync, hiddenFileSync, registries, settings } = createFixture();
customisationSync._anyProcessOptionalFileEvent.mockResolvedValue(true);
hiddenFileSync._anyProcessOptionalFileEvent.mockResolvedValue(true);
hiddenFileSync.isTargetFileEligible.mockResolvedValue(false);
const { customisationHandlers, hiddenFileHandlers, registries, settings } = createFixture();
customisationHandlers.processOptionalFileEvent.mockResolvedValue(true);
hiddenFileHandlers.processOptionalFileEvent.mockResolvedValue(true);
hiddenFileHandlers.isTargetFileEligible.mockResolvedValue(false);
await expect(registries.isTargetFileInExtra.handlers[0]!(".obsidian/plugins/example/data.json")).resolves.toBe(
true
@@ -250,9 +282,9 @@ describe("useOptionalFileSync", () => {
await expect(
registries.processOptionalFileEvent.handlers[0]!(".obsidian/plugins/example/data.json")
).resolves.toBe(true);
expect(customisationSync._anyProcessOptionalFileEvent).toHaveBeenCalledOnce();
expect(hiddenFileSync._anyProcessOptionalFileEvent).not.toHaveBeenCalled();
expect(hiddenFileSync.isTargetFileEligible).not.toHaveBeenCalled();
expect(customisationHandlers.processOptionalFileEvent).toHaveBeenCalledOnce();
expect(hiddenFileHandlers.processOptionalFileEvent).not.toHaveBeenCalled();
expect(hiddenFileHandlers.isTargetFileEligible).not.toHaveBeenCalled();
settings.pluginSyncExtendedSetting = {
"PLUGIN_DATA/example": {
@@ -261,53 +293,53 @@ describe("useOptionalFileSync", () => {
files: [],
},
};
hiddenFileSync.isTargetFileEligible.mockResolvedValue(true);
customisationSync._anyProcessOptionalFileEvent.mockClear();
hiddenFileHandlers.isTargetFileEligible.mockResolvedValue(true);
customisationHandlers.processOptionalFileEvent.mockClear();
await expect(
registries.processOptionalFileEvent.handlers[0]!(".obsidian/plugins/example/data.json")
).resolves.toBe(true);
expect(hiddenFileSync._anyProcessOptionalFileEvent).toHaveBeenCalledOnce();
expect(customisationSync._anyProcessOptionalFileEvent).not.toHaveBeenCalled();
expect(hiddenFileHandlers.processOptionalFileEvent).toHaveBeenCalledOnce();
expect(customisationHandlers.processOptionalFileEvent).not.toHaveBeenCalled();
});
it("does not fall back to the other context when the selected owner skips or fails", async () => {
const { customisationSync, hiddenFileSync, services } = createAggregateFixture();
customisationSync._anyProcessOptionalFileEvent.mockResolvedValueOnce(false);
const { customisationHandlers, hiddenFileHandlers, services } = createAggregateFixture();
customisationHandlers.processOptionalFileEvent.mockResolvedValueOnce(false);
await expect(services.fileProcessing.processOptionalFileEvent(".obsidian/app.json")).resolves.toBe(false);
expect(hiddenFileSync._anyProcessOptionalFileEvent).not.toHaveBeenCalled();
expect(hiddenFileHandlers.processOptionalFileEvent).not.toHaveBeenCalled();
customisationSync._anyProcessOptionalFileEvent.mockRejectedValueOnce(new Error("customisation failed"));
customisationHandlers.processOptionalFileEvent.mockRejectedValueOnce(new Error("customisation failed"));
await expect(services.fileProcessing.processOptionalFileEvent(".obsidian/app.json")).resolves.toBe(false);
expect(hiddenFileSync._anyProcessOptionalFileEvent).not.toHaveBeenCalled();
expect(hiddenFileHandlers.processOptionalFileEvent).not.toHaveBeenCalled();
});
it("dispatches conflict documents by their persisted namespace", async () => {
const { customisationSync, hiddenFileSync, services } = createAggregateFixture();
customisationSync._anyGetOptionalConflictCheckMethod.mockResolvedValue("newer");
hiddenFileSync._anyGetOptionalConflictCheckMethod.mockResolvedValue(true);
const { hiddenFileHandlers, services } = createAggregateFixture();
await expect(services.conflict.getOptionalConflictCheckMethod("ix:device/app.json")).resolves.toBe("newer");
expect(hiddenFileSync._anyGetOptionalConflictCheckMethod).not.toHaveBeenCalled();
expect(hiddenFileHandlers.queueConflict).not.toHaveBeenCalled();
await expect(services.conflict.getOptionalConflictCheckMethod("i:.obsidian/example.json")).resolves.toBe(true);
expect(customisationSync._anyGetOptionalConflictCheckMethod).toHaveBeenCalledOnce();
expect(hiddenFileHandlers.queueConflict).toHaveBeenCalledOnce();
expect(hiddenFileHandlers.queueConflict).toHaveBeenCalledWith("i:.obsidian/example.json");
await expect(services.conflict.getOptionalConflictCheckMethod("notes/example.md")).resolves.toBe(false);
});
it("keeps persisted document acceptance separate from current local ownership", async () => {
const { customisationSync, hiddenFileSync, registries, settings } = createFixture();
const { customisationHandlers, hiddenFileHandlers, registries, settings } = createFixture();
settings.usePluginSync = false;
settings.syncInternalFiles = false;
await registries.processVirtualDocument.handlers[0]!({ _id: "ix:device-a/CONFIG/app.json.md" });
await registries.processOptionalSynchroniseResult.handlers[0]!({ _id: "i:.obsidian/app.json" });
expect(customisationSync._anyModuleParsedReplicationResultItem).toHaveBeenCalledOnce();
expect(hiddenFileSync._anyProcessOptionalSyncFiles).toHaveBeenCalledOnce();
expect(customisationHandlers.processVirtualDocument).toHaveBeenCalledOnce();
expect(hiddenFileHandlers.processOptionalSyncFiles).toHaveBeenCalledOnce();
});
it("routes Ignore mode to neither local context", async () => {
const { customisationSync, hiddenFileSync, registries, settings } = createFixture();
const { customisationHandlers, hiddenFileHandlers, registries, settings } = createFixture();
settings.pluginSyncExtendedSetting = {
"PLUGIN_DATA/example": {
key: "PLUGIN_DATA/example",
@@ -319,8 +351,8 @@ describe("useOptionalFileSync", () => {
await expect(
registries.processOptionalFileEvent.handlers[0]!(".obsidian/plugins/example/data.json")
).resolves.toBe(false);
expect(customisationSync._anyProcessOptionalFileEvent).not.toHaveBeenCalled();
expect(hiddenFileSync._anyProcessOptionalFileEvent).not.toHaveBeenCalled();
expect(customisationHandlers.processOptionalFileEvent).not.toHaveBeenCalled();
expect(hiddenFileHandlers.processOptionalFileEvent).not.toHaveBeenCalled();
});
it("injects the same static ownership policy into both scan contexts", () => {
@@ -351,11 +383,12 @@ describe("useOptionalFileSync", () => {
});
it("preserves bail-first-failure and settled-unload behaviour", async () => {
const { customisationSync, hiddenFileSync, services } = createAggregateFixture();
customisationSync._everyRealizeSettingSyncMode.mockResolvedValueOnce(false);
const { customisationHandlers, customisationSync, hiddenFileHandlers, hiddenFileSync, services } =
createAggregateFixture();
customisationHandlers.onRealiseSetting.mockResolvedValueOnce(false);
await expect(services.setting.onRealiseSetting()).resolves.toBe(false);
expect(hiddenFileSync._everyRealizeSettingSyncMode).not.toHaveBeenCalled();
expect(hiddenFileHandlers.realiseSettingSyncMode).not.toHaveBeenCalled();
customisationSync.dispose.mockImplementationOnce(() => {
throw new Error("customisation disposal failed");
@@ -380,21 +413,54 @@ describe("useOptionalFileSync", () => {
});
it("strips a database prefix before evaluating a Hidden File Sync target", async () => {
const { hiddenFileSync, registries } = createFixture();
const { hiddenFileHandlers, registries } = createFixture();
await registries.isTargetFileInExtra.handlers[0]!({ path: "i:.obsidian/workspace" });
expect(hiddenFileSync.isTargetFileEligible).toHaveBeenCalledWith(".obsidian/workspace");
expect(hiddenFileHandlers.isTargetFileEligible).toHaveBeenCalledWith(".obsidian/workspace");
});
it("returns focused views without registering either context as an add-on", () => {
const { customisationSync, feature, hiddenFileSync } = createFixture();
it("returns a concrete repair adapter instead of exposing the broad context to UI", async () => {
const {
customisationSync,
customisationTesting,
feature,
hiddenFileSync,
hiddenFileTesting,
} = createFixture();
expect(feature.customisationSync).toBe(customisationSync);
expect(feature.hiddenFileSyncCommands).toBe(hiddenFileSync);
expect(feature.hiddenFileSyncInitialisation).toBe(hiddenFileSync);
expect(feature.hiddenFileSyncRepair).toBe(hiddenFileSync);
expect(feature.testing).toEqual({ customisationSync, hiddenFileSync });
expect(feature.hiddenFileSyncRepair).toBe(hiddenFileSync.repair);
expect(feature.hiddenFileSyncRepair).not.toBe(hiddenFileSync);
expect(Object.isFrozen(feature.hiddenFileSyncRepair)).toBe(true);
expect(Object.keys(feature.hiddenFileSyncRepair).sort()).toEqual(
[
"extractInternalFileRevisionFromDatabase",
"scanInternalFiles",
"storeInternalFileToDatabase",
"storeInternalFileToDatabaseWithBaseRevision",
].sort()
);
const file = {
path: ".obsidian/app.json",
ctime: 1,
mtime: 2,
size: 3,
} as never;
await feature.hiddenFileSyncRepair.storeInternalFileToDatabaseWithBaseRevision(file, "2-selected", false);
expect(hiddenFileSync.repair.storeInternalFileToDatabaseWithBaseRevision).toHaveBeenCalledWith(
file,
"2-selected",
false
);
expect(feature.testing).toEqual({
customisationSync: customisationTesting,
hiddenFileSync: hiddenFileTesting,
});
expect(Object.isFrozen(feature.testing)).toBe(true);
expect(feature.testing.customisationSync).not.toBe(customisationSync);
expect(feature.testing.hiddenFileSync).not.toBe(hiddenFileSync);
});
});
@@ -180,7 +180,7 @@ async function resolveHiddenConflicts(cliBinary: string, env: NodeJS.ProcessEnv)
"(async()=>{",
"const core=app.plugins.plugins['obsidian-livesync'].core;",
"const syncContext=app.plugins.plugins['obsidian-livesync'].optionalFileSync.testing.hiddenFileSync;",
"await syncContext.resolveConflictOnInternalFiles();",
"await syncContext.conflictResolution.resolveAll();",
"await syncContext.scanAllDatabaseChanges(true);",
"return JSON.stringify({ok:true});",
"})()",
@@ -192,40 +192,46 @@ async function resolveHiddenConflicts(cliBinary: string, env: NodeJS.ProcessEnv)
async function autoMergeHiddenJsonConflict(cliBinary: string, env: NodeJS.ProcessEnv, path: string): Promise<void> {
await evalObsidianJson<unknown>(
cliBinary,
[
"(async()=>{",
`const path=${JSON.stringify(path)};`,
"const syncContext=app.plugins.plugins['obsidian-livesync'].optionalFileSync.testing.hiddenFileSync;",
"await syncContext.conflictResolution.resolveAll();",
"await syncContext.scanAllDatabaseChanges(true);",
"return JSON.stringify({ok:true,path});",
"})()",
].join(""),
env
);
}
async function readHiddenConflictDiagnostics(
cliBinary: string,
env: NodeJS.ProcessEnv,
path: string
): Promise<unknown> {
return await evalObsidianJson<unknown>(
cliBinary,
[
"(async()=>{",
`const path=${JSON.stringify(path)};`,
"const prefixedPath=`i:${path}`;",
"const core=app.plugins.plugins['obsidian-livesync'].core;",
"const syncContext=app.plugins.plugins['obsidian-livesync'].optionalFileSync.testing.hiddenFileSync;",
"let doc=false;",
"const owner=app.plugins.plugins['obsidian-livesync'].optionalFileSync.testing.hiddenFileSync.conflictResolution;",
"const entries=[];",
"for await (const entry of core.localDatabase.findEntries('i:','i;',{conflicts:true})){",
" if(entry.path===prefixedPath){ doc=entry; break; }",
" if(entry.path===prefixedPath){",
" entries.push({id:entry._id,path:entry.path,rev:entry._rev,conflicts:entry._conflicts});",
" }",
"}",
"if(!doc) throw new Error(`Could not find hidden conflict candidate: ${path}`);",
"if(!doc._conflicts?.length) throw new Error(`Hidden file has no conflicts: ${path}`);",
"const conflicts=doc._conflicts.sort((a,b)=>Number(a.split('-')[0])-Number(b.split('-')[0]));",
"const conflictedRev=conflicts[0];",
"const conflictedRevNo=Number(conflictedRev.split('-')[0]);",
"const revFrom=await core.localDatabase.getRaw(doc._id,{revs_info:true});",
"const commonBase=(revFrom._revs_info||[])",
" .filter((rev)=>rev.status==='available'&&Number(rev.rev.split('-')[0])<conflictedRevNo)",
" .map((rev)=>rev.rev)[0]||'';",
"const result=await core.localDatabase.managers.conflictManager.mergeObject(",
" doc.path, commonBase, doc._rev, conflictedRev",
");",
"if(!result){",
" throw new Error(`Hidden JSON conflict was not auto-mergeable: ${path}; base=${commonBase}; current=${doc._rev}; conflict=${conflictedRev}`);",
"}",
"await syncContext.ensureDir(path);",
"const stat=await syncContext.writeFile(path,result);",
"if(!stat) throw new Error(`Could not write merged hidden file: ${path}`);",
"await syncContext.storeInternalFileToDatabase({path,mtime:stat.mtime,ctime:stat.ctime,size:stat.size},true);",
"await core.localDatabase.removeRevision(doc._id,conflictedRev);",
"await syncContext.extractInternalFileFromDatabase(path);",
"await syncContext.scanAllDatabaseChanges(true);",
"return JSON.stringify({ok:true,merged:JSON.parse(result)});",
"const modals=Array.from(document.querySelectorAll('.modal-container')).map((element)=>element.textContent?.trim());",
"return JSON.stringify({",
" entries,",
" pendingPaths:Array.from(owner.pendingPaths??[]),",
" processor:{remaining:owner.processor?.remaining,totalRemaining:owner.processor?.totalRemaining,nowProcessing:owner.processor?.nowProcessing},",
" modals,",
"});",
"})()",
].join(""),
env
@@ -250,7 +256,7 @@ async function openHiddenJsonResolveModal(cliBinary: string, env: NodeJS.Process
"const docA=await core.localDatabase.getDBEntry(prefixedPath,{rev:doc._rev});",
"const docB=await core.localDatabase.getDBEntry(prefixedPath,{rev:conflicts[0]});",
"if(docA===false||docB===false) throw new Error(`Could not load conflicted hidden JSON entries: ${path}`);",
"void syncContext.showJSONMergeDialogAndMerge(docA,docB);",
"void syncContext.conflictResolution.resolveJson(docA,docB);",
"return JSON.stringify({ok:true});",
"})()",
].join(""),
@@ -270,11 +276,12 @@ async function storeHiddenFileAsConflict(
"(async()=>{",
`const path=${JSON.stringify(path)};`,
`const baseRev=${JSON.stringify(baseRev)};`,
"const prefixedPath=`i:${path}`;",
"const core=app.plugins.plugins['obsidian-livesync'].core;",
"const syncContext=app.plugins.plugins['obsidian-livesync'].optionalFileSync.testing.hiddenFileSync;",
"const fileInfo=await syncContext.loadFileWithInfo(path);",
"const fileInfo=await syncContext.readFileWithInfo(path);",
"if(fileInfo.deleted) throw new Error(`Hidden file was unexpectedly deleted: ${path}`);",
"const baseData=await syncContext.__loadBaseSaveData(path,true);",
"const baseData=await core.localDatabase.getDBEntry(prefixedPath,undefined,false,true);",
"if(baseData===false) throw new Error(`Could not load base save data: ${path}`);",
"const saveData={",
" ...baseData,",
@@ -430,9 +437,21 @@ async function runJsonConflictRoundTrip(
await createHiddenJsonConflict(context, session, vaultB, mergeJsonPath, base, left, right);
await autoMergeHiddenJsonConflict(context.cliBinary, session.cliEnv, mergeJsonPath);
await pushLocalChanges(context.cliBinary, session.cliEnv);
const mergedOnB = await waitForPathContent(vaultB.path, mergeJsonPath, (content) =>
hasJsonValues(content, { fromA: true, fromB: true })
);
let mergedOnB: string;
try {
mergedOnB = await waitForPathContent(vaultB.path, mergeJsonPath, (content) =>
hasJsonValues(content, { fromA: true, fromB: true })
);
} catch (error) {
const diagnostics = await readHiddenConflictDiagnostics(context.cliBinary, session.cliEnv, mergeJsonPath).catch(
(diagnosticError: unknown) => ({
diagnosticError: diagnosticError instanceof Error ? diagnosticError.message : String(diagnosticError),
})
);
throw new Error(
`${error instanceof Error ? error.message : String(error)}\nConflict diagnostics: ${JSON.stringify(diagnostics)}`
);
}
await session.app.stop();
session = await startConfiguredSession(context, vaultA);
@@ -620,14 +639,11 @@ async function setHiddenFileNoticeFixtures(port: number, itemIds: string[], incl
};
obsidianApp.plugins.enabledPlugins.add(pluginId);
}
syncContext.queuedNotificationFiles.clear();
for (const id of nextItemIds) {
syncContext.queuedNotificationFiles.add(`.obsidian/plugins/livesync-e2e-${id}`);
}
const updatedFolders = nextItemIds.map((id) => `.obsidian/plugins/livesync-e2e-${id}`);
if (nextIncludeRestart) {
syncContext.queuedNotificationFiles.add(core.services.API.getSystemConfigDir());
updatedFolders.push(core.services.API.getSystemConfigDir());
}
syncContext.notifyConfigChange();
syncContext.showConfigurationChangeNotice(updatedFolders);
},
{ nextItemIds: itemIds, nextIncludeRestart: includeRestart }
);
@@ -683,7 +699,6 @@ async function runInitialisationNoticeGrouping(context: RunnerContext, vault: Te
const syncContext = plugin.optionalFileSync.testing.hiddenFileSync;
const setting = core.services.setting;
const originalApplyPartial = setting.applyPartial;
const originalRebuildMerging = syncContext.rebuildMerging;
const state = {
done: false,
reachedPreparation: false,
@@ -731,13 +746,15 @@ async function runInitialisationNoticeGrouping(context: RunnerContext, vault: Te
return await originalApplyPartial.apply(setting, args);
};
syncContext.rebuildMerging = async (...args: unknown[]) => {
state.reachedInitialisation = true;
await new Promise<void>((resolve) => {
state.releaseInitialisation = resolve;
});
return await originalRebuildMerging.apply(syncContext, args);
};
const restoreRebuildMerging = syncContext.interceptRebuildMerging(
async (runRebuild: (...args: unknown[]) => Promise<unknown>, ...args: unknown[]) => {
state.reachedInitialisation = true;
await new Promise<void>((resolve) => {
state.releaseInitialisation = resolve;
});
return await runRebuild(...args);
}
);
void core.services.setting
.enableOptionalFeature("MERGE")
@@ -752,7 +769,7 @@ async function runInitialisationNoticeGrouping(context: RunnerContext, vault: Te
)
.finally(() => {
setting.applyPartial = originalApplyPartial;
syncContext.rebuildMerging = originalRebuildMerging;
restoreRebuildMerging();
const notices = Array.from(document.querySelectorAll<HTMLElement>(".notice"));
const progressNotices = notices.filter((notice) => notice.textContent?.includes("[⚙"));
state.sawStandaloneGatheringNotice ||= notices.some((notice) =>