Refactor optional file synchronisation ownership

This commit is contained in:
vorotamoroz
2026-09-04 05:19:47 +00:00
parent 826413bf84
commit 8449f9d3f5
53 changed files with 6574 additions and 2156 deletions
@@ -20,12 +20,13 @@ one writer for raw events, scheduled scans, and automatic database-to-local
reflection. The constructor-name add-on lookup and the `ConfigSync` and reflection. The constructor-name add-on lookup and the `ConfigSync` and
`HiddenFileSync` add-on identities have been retired. `HiddenFileSync` add-on identities have been retired.
The Customisation Sync private context owns its catalogue, progress, manifest, The Customisation Sync private context coordinates its snapshot operations,
queue, and periodic state. The Hidden File Sync private context owns its scan queues, periodic work, and focused transient-state owners. The Hidden File
processed-state caches, reconciliation and conflict queues, activity counts, Sync private context coordinates reconciliation, periodic work, and the
and periodic state. Each receives live settings and database projections, lifetimes of focused processed-state, change-processing, and
focused storage, path, and exact-revision capabilities, and explicit host conflict-resolution owners. Each receives live settings and database
effects rather than `LiveSyncCore`. projections, focused storage, path, and exact-revision capabilities, and
explicit host effects rather than `LiveSyncCore`.
The corresponding implemented topology is documented in The corresponding implemented topology is documented in
[Optional-file synchronisation architecture](../design_docs/optional_file_sync_architecture.md). [Optional-file synchronisation architecture](../design_docs/optional_file_sync_architecture.md).
@@ -105,7 +106,7 @@ The Customisation Sync runtime will own:
operations; and operations; and
- the state required to serialise and report those operations. - the state required to serialise and report those operations.
The Hidden File Sync runtime will own: The Hidden File Sync runtime and its composed focused owners will own:
- the device-local processed-state records; - the device-local processed-state records;
- one-file storage and database transfer, including exact-revision repair; - one-file storage and database transfer, including exact-revision repair;
@@ -194,15 +195,44 @@ caches, processed-state records, recent-event records, locks, semaphores,
queues, periodic processors, Notices, and event subscriptions must be created queues, periodic processors, Notices, and event subscriptions must be created
and disposed by their feature context or a focused resource owner. and disposed by their feature context or a focused resource owner.
The contexts are orchestration roots, rather than containers for every mutable
detail. `HiddenFileSyncProcessedState` owns the three device-local maps, their
autosave initialisation, exact key formats, retained known mtime, reset rules,
and settlement effects on `IPathService`. Database write and extraction
operations receive this capability through one narrow `processedState` port;
they do not receive bundles of individual state callbacks or implement state
keys themselves.
`CustomisationSyncCatalogueState` separately owns the transient catalogue
rows, manifest lookup and mtime cache, their reactive stores, and catalogue
update progress. A small recent-event deduplicator owns raw-event admission
history. These Customisation Sync owners are deliberately not implementations
of a shared Hidden File Sync state abstraction: the former state is a derived,
in-memory projection, while Hidden File Sync markers are persisted operational
reconciliation state with different identity, invalidation, and deletion
rules.
`HiddenFileSyncChangeProcessor` owns storage and database change processing,
the bounded semaphore, same-path event serialisation, activity counts, and the
inherited order in which processed-state markers and transfer results settle.
This boundary keeps event concurrency and settlement directly testable without
giving the processor full scan, initialisation, notification, or host
responsibilities.
The joint composition may return several views backed by those contexts: The joint composition may return several views backed by those contexts:
- a Customisation Sync catalogue and operation view for its dialogue; - a Customisation Sync catalogue and operation view for its dialogue;
- a Hidden File Sync initialisation view for settings workflows; and - a Hidden File Sync initialisation view for settings workflows;
- a Hidden File Sync repair view for the Hatch pane. - a Hidden File Sync repair view for the Hatch pane;
- immutable semantic handler views for registration by the joint composition;
and
- explicitly internal testing views for maintained real-Obsidian workflows.
Several views over one context do not create several owners. Views expose Several views over one context do not create several owners. Views expose
stable application data and operations rather than PouchDB entries, queue stable application data and named operations rather than PouchDB entries,
objects, mutable settings records, or the complete core. queue objects, mutable settings records, dependency objects, or the complete
core. The testing views also avoid exposing context instances or writable
internal state; time-sensitive E2E work uses a scoped operation interceptor.
Obsidian commands, ribbon actions, dialogues, Notices, plug-in reloads, and Obsidian commands, ribbon actions, dialogues, Notices, plug-in reloads, and
restart scheduling will remain in host-owned composition. The UI will receive restart scheduling will remain in host-owned composition. The UI will receive
@@ -231,10 +261,10 @@ serviceFeature. `LiveSyncBaseCore.getAddOn()` and its constructor-name lookup
have been removed. have been removed.
Production code consumes focused views and does not use the broad context Production code consumes focused views and does not use the broad context
surface. Maintained real-Obsidian E2E workflows use an explicitly internal surface. Maintained real-Obsidian E2E workflows use explicitly internal,
test view exposed by the composed feature. This is a transitional test seam, immutable test views exposed by the composed feature. These are transitional
not a production service locator, and it should be narrowed as those workflows test seams, not production service locators, and they should be narrowed as
move to public operations or commands. those workflows move to public operations or commands.
The retirement was gated on: The retirement was gated on:
@@ -328,11 +358,13 @@ to the non-owner.
- Replace the complete-core dependency with narrow dependencies. - Replace the complete-core dependency with narrow dependencies.
The private context, path module, codec module, focused presentation view, and The private context, path module, codec module, focused presentation view, and
resource teardown are implemented. Catalogue, enumeration, migration, scan, resource teardown are implemented. A focused catalogue-state owner holds the
and manifest state is owned per context instance. The context accepts only rows, manifests, manifest mtime cache, reactive stores, and update progress,
narrow, live projections and explicit effects; an Obsidian adapter at the while a bounded deduplicator owns recent raw-event keys. Enumeration and scan
composition edge owns dialogues, Notices, plug-in reload, restart, lifecycle, queues remain with the orchestrating context. The context accepts only narrow,
Vault access, and compatibility scan telemetry. live projections and explicit effects; an Obsidian adapter at the composition
edge owns dialogues, Notices, plug-in reload, restart, lifecycle, Vault access,
and compatibility scan telemetry.
### Stage 5: extract the Hidden File Sync runtime — implemented ### Stage 5: extract the Hidden File Sync runtime — implemented
@@ -343,16 +375,26 @@ Vault access, and compatibility scan telemetry.
The private context, focused initialisation, repair, and command views, The private context, focused initialisation, repair, and command views,
host-owned command registration, and processor, cache, subscription, and host-owned command registration, and processor, cache, subscription, and
Notice teardown are implemented. The context owns transfer, reconciliation, Notice teardown are implemented. A focused processed-state owner holds all
conflict, and processed-state behaviour through narrow live dependencies. three persisted maps, their key and mtime rules, reset operations, and
An Obsidian adapter owns JSON conflict dialogues, progress presentation, cross-side settlement. Database write and extraction operations consume one
grouped Notices, plug-in reload, restart scheduling, Vault enumeration, and narrow state port. A focused change processor owns storage and database event
compatibility activity publication. processing, bounded concurrency, per-path serialisation, activity publication,
and compatibility settlement order. The context retains scan, initialisation,
notification, and reconciliation orchestration. A focused conflict-resolution
owner owns pending-path admission,
the parallel classification and serial interaction queues, automatic merge,
newer-revision selection, interactive JSON application, settlement, and queue
disposal. An Obsidian adapter owns JSON conflict dialogue instances, progress
presentation, grouped Notices, plug-in reload, restart scheduling, Vault
enumeration, and compatibility activity publication.
### Stage 6: move synchronisation composition — implemented ### Stage 6: move synchronisation composition — implemented
- Register the overlapping Service handlers once through the joint - Register the overlapping Service handlers once through the joint
serviceFeature. serviceFeature.
- Consume immutable semantic handler views rather than exposing registry-style
methods on either context.
- Preserve the characterised lifecycle callback order and Commonlib - Preserve the characterised lifecycle callback order and Commonlib
aggregation semantics through focused tests. aggregation semantics through focused tests.
@@ -438,11 +480,13 @@ migration without first establishing narrow dependencies.
- The composition root gains several focused views but does not gain another - The composition root gains several focused views but does not gain another
runtime service locator. runtime service locator.
- The legacy add-on identity and constructor-name lookup are removed. - The legacy add-on identity and constructor-name lookup are removed.
- The two contexts remain sizeable because each owns one cohesive persisted - The two contexts coordinate separate synchronisation workflows, but their
synchronisation model, but their dependency surfaces are explicit and do dependency surfaces are explicit and do not include the complete core.
not include the complete core. Further extraction should follow a concrete Customisation Sync delegates its derived catalogue and recent-event state,
behavioural boundary rather than create additional serviceFeatures for while Hidden File Sync delegates processed-state, change-processing, and
private operations. conflict lifecycles, to focused owners. Further extraction should follow a
concrete behavioural boundary rather than create additional serviceFeatures
for private operations.
## References ## References
@@ -31,6 +31,9 @@ Obsidian composition (`main.ts`)
| +--> pure local-path and document routing policy | +--> pure local-path and document routing policy
| | | |
| +--> `CustomisationSyncContext` | +--> `CustomisationSyncContext`
| | +-- `CustomisationSyncCatalogueState`
| | +-- recent-event deduplicator
| | +-- immutable service-handler and testing views
| | ^ | | ^
| | +-- narrow dependencies from | | +-- narrow dependencies from
| | `customisationSyncObsidianAdapter` | | `customisationSyncObsidianAdapter`
@@ -39,6 +42,20 @@ Obsidian composition (`main.ts`)
| ^ | ^
| +-- narrow dependencies from | +-- narrow dependencies from
| `hiddenFileSyncObsidianAdapter` | `hiddenFileSyncObsidianAdapter`
| |
| +--> `HiddenFileSyncProcessedState`
| | +-- three autosaved reconciliation maps
| | +-- key, mtime, reset, and settlement rules
| |
| +--> `HiddenFileSyncChangeProcessor`
| | +-- storage and database change processing
| | +-- per-path serialisation and activity state
| |
| +--> `HiddenFileSyncConflictResolution`
| | +-- pending paths and two-stage conflict queue
| | +-- automatic and interactive JSON resolution
| |
| +-- immutable service-handler, command, repair, and testing views
| |
+--> `useCustomisationSyncUI` +--> `useCustomisationSyncUI`
| +-- catalogue and operation view | +-- catalogue and operation view
@@ -58,22 +75,37 @@ application core.
## Ownership ## Ownership
| Owner | Owns | Does not own | | Owner | Owns | Does not own |
| --- | --- | --- | | ------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | --------------------------------------------------------------------------------------------------------------------- |
| `useOptionalFileSync` | Construction of both contexts, handler registration and removal, local-owner selection, namespace dispatch, compatibility callback order, and context disposal order. | Persisted feature state, synchronisation algorithms, commands, dialogues, or Notices. | | `useOptionalFileSync` | Construction of both contexts, handler registration and removal, local-owner selection, namespace dispatch, compatibility callback order, and context disposal order. | Persisted feature state, synchronisation algorithms, commands, dialogues, or Notices. |
| `CustomisationSyncContext` | The `ix:` codec and path rules, catalogue, manifest cache, scan queues, migration progress, snapshot storage and application, and periodic scan state. | Obsidian dialogues, plug-in lifecycle APIs, ribbon actions, or handler registration. | | `CustomisationSyncContext` | The `ix:` codec and path rules, scan queues, snapshot storage and application, periodic scan state, and the lifetimes of its transient-state owners. | Catalogue mutations, recent-event history, Obsidian dialogues, ribbon actions, or handler registration. |
| `HiddenFileSyncContext` | The `i:` transfer rules, device-local processed-state caches, reconciliation, exact-revision repair, conflict queues, notification batching, activity counts, and periodic scan state. | Obsidian conflict dialogues, grouped Notices, plug-in lifecycle APIs, or handler registration. | | `CustomisationSyncCatalogueState` | Transient catalogue rows, manifest lookup and mtime cache, reactive catalogue and manifest stores, and catalogue update progress. | Database or storage I/O, scan scheduling, routing, or persistence. |
| Customisation Sync Obsidian adapter | Obsidian conflict selection, Notice presentation, plug-in reload, restart requests, Vault enumeration, progress telemetry, and platform-derived fallback device names. | Catalogue state, routing, or persisted document operations. | | Customisation Sync event deduplicator | The bounded, newest-first keys used to admit raw configuration events once. | Scheduling, path ownership, catalogue state, or persisted data. |
| Hidden File Sync Obsidian adapter | JSON conflict dialogue lifetime, progress presentation, grouped change Notices, plug-in reload actions, restart scheduling, Vault enumeration, and compatibility activity publication. | Transfer, reconciliation, processed-state, or conflict decisions. | | `HiddenFileSyncContext` | The `i:` scan and reconciliation workflow, exact-revision repair composition, notification batching, periodic scan state, and focused-owner lifetimes. | Change-event serialisation, processed-state representation, conflict queue state, Obsidian dialogues, or service handler registration. |
| `useCustomisationSyncUI` | Command, ribbon, dialogue, open-request subscription, and their unload teardown. | Synchronisation state or Hidden File Sync initialisation behaviour. | | `HiddenFileSyncProcessedState` | Three device-local autosaved maps, exact storage and database keys, retained known mtime, reset behaviour, and storage/database settlement effects. | File transfer, scan scheduling, conflict handling, or presentation. |
| `useHiddenFileSyncCommands` | Hidden File Sync command registration, setting-change subscription, and their unload teardown. | Synchronisation state or command implementation. | | `HiddenFileSyncChangeProcessor` | Storage and database change processing, same-path event serialisation, bounded concurrency, activity counts, and the inherited event-consumption and settlement order. | Full scans, initialisation policy, notification presentation, or conflict interaction. |
| `HiddenFileSyncConflictResolution` | Conflict admission and deduplication, pending paths, the parallel classification and serial interaction queues, automatic merge, newer-revision policy, interactive JSON resolution, and conflict settlement. | Obsidian dialogue instances, general Hidden File Sync scans, processed-state caches, or Service handler registration. |
| Customisation Sync Obsidian adapter | Obsidian conflict selection, Notice presentation, plug-in reload, restart requests, Vault enumeration, progress telemetry, and platform-derived fallback device names. | Catalogue state, routing, or persisted document operations. |
| Hidden File Sync Obsidian adapter | JSON conflict dialogue lifetime, progress presentation, grouped change Notices, plug-in reload actions, restart scheduling, Vault enumeration, and compatibility activity publication. | Transfer, reconciliation, processed-state, or conflict decisions. |
| `useCustomisationSyncUI` | Command, ribbon, dialogue, open-request subscription, and their unload teardown. | Synchronisation state or Hidden File Sync initialisation behaviour. |
| `useHiddenFileSyncCommands` | Hidden File Sync command registration, setting-change subscription, and their unload teardown. | Synchronisation state or command implementation. |
The two domain contexts remain sizeable because they each own one cohesive The two domain contexts coordinate one cohesive synchronisation workflow each.
persisted synchronisation model. Their private operations are not additional Their private operations and focused owners are not additional serviceFeatures:
serviceFeatures: they do not independently register host integration or have they do not independently register host integration or have separate
separate application lifetimes. Extract a further ordinary module or focused application lifetimes. `HiddenFileSyncChangeProcessor` is a focused resource
state owner when a concrete invariant, replacement lifecycle, or independently owner because its semaphore, per-path serialisation, activity counters, and
testable operation justifies that boundary. event settlement form one independently testable lifecycle.
`HiddenFileSyncConflictResolution` is another focused resource owner because
conflict admission, pending-path identity, two serialisation stages, and
disposal form a separate lifecycle.
The two state owners intentionally do not implement a common generic state
contract. Customisation Sync projects transient catalogue and presentation
state from `ix:` documents. Hidden File Sync persists operational markers used
for reconciliation, with distinct path identity, deletion, reset, and retained
mtime rules. Their common boundary is lifecycle ownership by a context, rather
than interchangeable state semantics.
## Routing and handler contracts ## Routing and handler contracts
@@ -85,21 +117,24 @@ static policy selects Hidden File Sync.
The maintained local ownership is: The maintained local ownership is:
| Path mode | Owner | | Path mode | Owner |
| --- | --- | | ------------------------------------------------------------------- | ------------------ |
| Customisation path in Selective or Flagged Selective mode | Customisation Sync | | Customisation path in Selective or Flagged Selective mode | Customisation Sync |
| Customisation path in Automatic mode, with Hidden File Sync enabled | Hidden File Sync | | Customisation path in Automatic mode, with Hidden File Sync enabled | Hidden File Sync |
| Customisation path in Ignore mode | Neither context | | Customisation path in Ignore mode | Neither context |
| Other eligible hidden path | Hidden File Sync | | Other eligible hidden path | Hidden File Sync |
| Disabled, excluded, ignored, or ordinary Vault path | Neither context | | Disabled, excluded, ignored, or ordinary Vault path | Neither context |
Local ownership is distinct from persisted-document recognition. The Local ownership is distinct from persisted-document recognition. The
composition dispatches `ps:` and `ix:` conflict documents to Customisation composition dispatches `ps:` and `ix:` conflict documents to Customisation
Sync and `i:` conflict documents to Hidden File Sync. Existing documents remain Sync and sends `i:` conflict documents through the Hidden File Sync semantic
recognisable after a local mode changes. handler view. Existing documents remain recognisable after a local mode
changes.
The composition adapts the contexts to the existing Commonlib handler Each context exposes an immutable semantic handler view. The composition
contracts: registers these operations and adapts them to the existing Commonlib handler
contracts; registry aggregation names and binding concerns do not leak back
into either context:
- raw optional-file events are offered to exactly one selected owner; - raw optional-file events are offered to exactly one selected owner;
- a selected handler which skips or fails does not fall through to the other - a selected handler which skips or fails does not fall through to the other
@@ -148,23 +183,35 @@ owners:
operations to the Hatch pane. operations to the Hatch pane.
Production consumers receive these views directly. `OptionalFileSyncFeature` Production consumers receive these views directly. `OptionalFileSyncFeature`
also exposes an explicitly internal `testing` view for maintained real-Obsidian also exposes immutable, explicitly internal testing views for maintained
contract tests. That test seam is not a production service locator and should real-Obsidian contract tests. They provide named operations, including a scoped
not be used by application features. rebuild interceptor, without exposing context instances, dependency objects,
queues, or writable stores. These test seams are not production service
locators and should not be used by application features.
## Lifecycle and disposal ## Lifecycle and disposal
The composition is created after the Service Hub and required ServiceModules The composition is created after the Service Hub and required ServiceModules
exist, and before lifecycle-driven feature work begins. Each context creates exist, and before lifecycle-driven feature work begins. Each context creates
its own queues, caches, semaphores, activity state, and periodic processor. its own periodic processor and focused resource owners.
`CustomisationSyncContext` creates one catalogue-state owner and one
recent-event deduplicator. `HiddenFileSyncContext` creates one processed-state
owner before composing database write and extraction operations around its
narrow port, then creates one change processor and one conflict-resolution
owner. The change processor owns its semaphore and activity state.
Full conflict scans admit discovered paths into the same queue without
suspending it, so ordinary database conflict notifications continue during a
slow scan. After enumeration, the operation waits for both classification and
interaction stages to drain.
On application unload, `useOptionalFileSync` first removes every Service On application unload, `useOptionalFileSync` first removes every Service
handler registration. It then disposes Customisation Sync followed by Hidden handler registration. It then disposes Customisation Sync followed by Hidden
File Sync, preserving the former compatibility order. Disposal disables File Sync, preserving the former compatibility order. Disposal disables
periodic admission, terminates queues, clears transient caches and pending periodic admission, disposes the conflict-resolution owner, terminates queues,
sets, cancels scheduled notification work, resets compatibility telemetry, and clears transient caches and pending sets, cancels scheduled notification work,
hides owned Notices. The two presentation serviceFeatures independently remove resets compatibility telemetry, and hides owned Notices. The two presentation
their commands, event subscriptions, ribbon state, and dialogue instances. serviceFeatures independently remove their commands, event subscriptions,
ribbon state, and dialogue instances.
## Persisted compatibility ## Persisted compatibility
@@ -184,9 +231,11 @@ appropriate, a migration decision.
## Verification boundaries ## Verification boundaries
Focused unit tests cover routing, handler aggregation, context state Focused unit tests cover routing, semantic handler views, context state
isolation, teardown, initial cache selection, exact-revision repair, conflict isolation, teardown, initial cache selection, exact-revision repair, change
dialogue adaptation, grouped Notices, and compatibility activity publication. event serialisation and settlement, conflict queue admission, revision
selection, automatic and interactive merge effect ordering, conflict dialogue
adaptation, grouped Notices, and compatibility activity publication.
The boundary test prevents either domain context from regaining core or The boundary test prevents either domain context from regaining core or
Obsidian dependencies. Obsidian dependencies.
@@ -0,0 +1,152 @@
import type { PluginManifest } from "@/deps.ts";
import type { FilePathWithPrefix } from "@vrtmrz/livesync-commonlib/compat/common/types";
import { isObjectDifferent } from "@vrtmrz/livesync-commonlib/compat/common/utils";
import { writable } from "svelte/store";
import { PluginDataExDisplayV2 } from "./customisationSyncModel.ts";
import type { IPluginDataExDisplay } from "./customisationSyncView.ts";
/**
* Owns the transient catalogue projection used by Customisation Sync.
*
* The database and storage operations remain in the context. This owner only
* coordinates the in-memory rows, their reactive publications, manifest
* lookup, and the update counter which is derived from those operations.
*/
export class CustomisationSyncCatalogueState {
private catalogueRows: IPluginDataExDisplay[] = [];
private readonly manifestByKey = new Map<string, PluginManifest>();
private readonly loadedManifestMTimeByKey = new Map<string, number>();
private activeUpdateCount = 0;
readonly catalogue = writable<IPluginDataExDisplay[]>([]);
readonly migrationProgress = writable(0);
readonly manifests = writable(this.manifestByKey);
/** The current manifest lookup passed to V2 display rows. */
get manifestLookup(): ReadonlyMap<string, PluginManifest> {
return this.manifestByKey;
}
/** The current loaded-manifest cache, exposed read-only for diagnostics. */
get loadedManifestMTime(): ReadonlyMap<string, number> {
return this.loadedManifestMTimeByKey;
}
/** Returns the authoritative row for a V1 document path, when present. */
findPlugin(documentPath: FilePathWithPrefix | string): IPluginDataExDisplay | undefined {
return this.catalogueRows.find((entry) => entry.documentPath == documentPath);
}
/** Returns every row matching a document path, preserving legacy duplicates. */
findPlugins(documentPath: FilePathWithPrefix | string): readonly IPluginDataExDisplay[] {
return this.catalogueRows.filter((entry) => entry.documentPath == documentPath);
}
/** Replaces a V1 row and publishes it immediately, preserving legacy order. */
replacePlugin(plugin: IPluginDataExDisplay): void {
const newList = this.catalogueRows.filter((entry) => entry.documentPath != plugin.documentPath);
newList.push(plugin);
this.catalogueRows = newList;
this.catalogue.set(newList);
}
/**
* Replaces a V2 row without publishing it. V2 callers publish through the
* existing delayed task after a cohesive row update has completed.
*/
private replaceV2Plugin(plugin: PluginDataExDisplayV2): void {
const newList = this.catalogueRows.filter((entry) => entry.documentPath != plugin.documentPath);
newList.push(plugin);
this.catalogueRows = newList;
}
/** Applies one loaded or removed V2 file and replaces its catalogue row. */
async updateV2Plugin(
plugin: PluginDataExDisplayV2,
file: Parameters<PluginDataExDisplayV2["setFile"]>[0] | false,
missingFilePath: string
): Promise<void> {
if (file) {
await plugin.setFile(file);
} else {
plugin.deleteFile(missingFilePath);
}
this.replaceV2Plugin(plugin);
}
/** Publishes the current V2 row set when the legacy delayed task fires. */
publishCatalogue(): void {
this.catalogue.set(this.catalogueRows);
}
/**
* Clears rows and loaded manifest mtimes for an explicit reload. The
* manifest map intentionally survives this narrower refresh.
*/
clearForReload(): void {
this.catalogueRows = [];
this.loadedManifestMTimeByKey.clear();
this.catalogue.set(this.catalogueRows);
}
/** Clears only the catalogue rows for a disabled refresh. */
clearForDisabledRefresh(): void {
this.catalogueRows = [];
this.catalogue.set(this.catalogueRows);
}
/** Begins one catalogue update and publishes its progress count. */
beginUpdate(): void {
this.activeUpdateCount++;
this.migrationProgress.set(this.activeUpdateCount);
}
/** Ends one catalogue update and publishes its progress count. */
endUpdate(): void {
this.activeUpdateCount--;
this.migrationProgress.set(this.activeUpdateCount);
}
/**
* Applies a manifest according to the inherited cache rules. A manifest is
* parsed only when no manifest has previously been accepted for the key;
* failed parses still record their mtime, while a later mtime never
* replaces a successfully parsed first manifest.
*/
processManifest(
confKey: string,
mtime: number,
parseManifest: () => PluginManifest,
onParseError: (error: unknown) => void = () => undefined
): void {
let publishCatalogue = false;
if (this.loadedManifestMTimeByKey.get(confKey) != mtime && this.manifestByKey.get(confKey) == undefined) {
try {
this.setManifest(confKey, parseManifest());
this.applyLoadedManifest(confKey);
publishCatalogue = true;
} catch (error) {
onParseError(error);
}
this.loadedManifestMTimeByKey.set(confKey, mtime);
} else {
this.applyLoadedManifest(confKey);
publishCatalogue = true;
}
if (publishCatalogue) this.catalogue.set(this.catalogueRows);
}
private setManifest(key: string, manifest: PluginManifest): void {
const old = this.manifestByKey.get(key);
if (old && !isObjectDifferent(manifest, old)) return;
this.manifestByKey.set(key, manifest);
this.manifests.set(this.manifestByKey);
}
private applyLoadedManifest(confKey: string): void {
this.catalogueRows
.filter((entry) => entry instanceof PluginDataExDisplayV2 && entry.confKey == confKey)
.forEach((entry) => (entry as PluginDataExDisplayV2).applyLoadedManifest());
}
}
@@ -0,0 +1,127 @@
import { get } from "svelte/store";
import { describe, expect, it, vi } from "vitest";
import type { PluginManifest } from "@/deps.ts";
import type { FilePathWithPrefix } from "@vrtmrz/livesync-commonlib/compat/common/types";
import { CustomisationSyncCatalogueState } from "./customisationSyncCatalogueState.ts";
import { PluginDataExDisplayV2 } from "./customisationSyncModel.ts";
import type { IPluginDataExDisplay, LoadedEntryPluginDataExFile } from "./customisationSyncView.ts";
function display(documentPath = "ix:device-a/PLUGIN_MAIN/example.md"): IPluginDataExDisplay {
return {
documentPath: documentPath as FilePathWithPrefix,
category: "PLUGIN_MAIN",
name: "example",
term: "device-a",
files: [],
mtime: 1,
};
}
function file(filename: string, mtime: number): LoadedEntryPluginDataExFile {
return {
path: `ix:device-a/PLUGIN_MAIN/example%${filename}` as FilePathWithPrefix,
filename,
mtime,
data: [filename],
size: filename.length,
} as LoadedEntryPluginDataExFile;
}
describe("Customisation Sync catalogue state", () => {
it("publishes V1 replacement and keeps V2 replacement delayed", async () => {
const state = new CustomisationSyncCatalogueState();
const setCatalogue = vi.spyOn(state.catalogue, "set");
const row = display();
state.replacePlugin(row);
expect(get(state.catalogue)).toEqual([row]);
expect(setCatalogue).toHaveBeenCalledOnce();
const v2 = new PluginDataExDisplayV2(
{
...display(),
files: [file("main.js", 1)],
},
state.manifestLookup
);
await state.updateV2Plugin(v2, file("main.js", 2), "main.js");
expect(get(state.catalogue)).toEqual([row]);
expect(state.findPlugin(row.documentPath)).toBe(v2);
state.publishCatalogue();
expect(get(state.catalogue)).toEqual([v2]);
});
it("retains the first parsed manifest and records failed mtimes", () => {
const state = new CustomisationSyncCatalogueState();
const first = { name: "First", version: "1.0.0" } as PluginManifest;
const parseManifest = vi.fn(() => first);
state.processManifest("device-a/plugins/example", 20, parseManifest);
state.processManifest(
"device-a/plugins/example",
30,
() => ({ name: "Second", version: "2.0.0" }) as PluginManifest
);
expect(state.manifestLookup.get("device-a/plugins/example")).toBe(first);
expect(state.loadedManifestMTime.get("device-a/plugins/example")).toBe(20);
expect(parseManifest).toHaveBeenCalledOnce();
const failedState = new CustomisationSyncCatalogueState();
const onParseError = vi.fn();
const failure = new SyntaxError("invalid");
failedState.processManifest(
"device-a/plugins/failure",
40,
() => {
throw failure;
},
onParseError
);
failedState.processManifest("device-a/plugins/failure", 40, () => first, onParseError);
expect(onParseError).toHaveBeenCalledWith(failure);
expect(failedState.loadedManifestMTime.get("device-a/plugins/failure")).toBe(40);
expect(failedState.manifestLookup.has("device-a/plugins/failure")).toBe(false);
});
it("clears rows and loaded mtimes on reload while retaining manifest lookup", () => {
const state = new CustomisationSyncCatalogueState();
const key = "device-a/plugins/example";
state.processManifest(key, 20, () => ({ name: "Example" }) as PluginManifest);
state.replacePlugin(display());
state.clearForReload();
expect(get(state.catalogue)).toEqual([]);
expect(state.loadedManifestMTime.size).toBe(0);
expect(state.manifestLookup.get(key)).toEqual({ name: "Example" });
expect(get(state.catalogue)).toEqual([]);
});
it("keeps manifest caches through the narrower disabled refresh", () => {
const state = new CustomisationSyncCatalogueState();
const key = "device-a/plugins/example";
state.processManifest(key, 20, () => ({ name: "Example" }) as PluginManifest);
state.replacePlugin(display());
state.clearForDisabledRefresh();
expect(get(state.catalogue)).toEqual([]);
expect(state.loadedManifestMTime.get(key)).toBe(20);
expect(state.manifestLookup.has(key)).toBe(true);
});
it("tracks V2 updates through migration progress", () => {
const state = new CustomisationSyncCatalogueState();
state.beginUpdate();
state.beginUpdate();
expect(get(state.migrationProgress)).toBe(2);
state.endUpdate();
state.endUpdate();
expect(get(state.migrationProgress)).toBe(0);
});
});
@@ -26,29 +26,42 @@ vi.mock("@/common/translation", () => ({
vi.mock("@/common/obsidianCommunityPlugins.ts", () => ({ vi.mock("@/common/obsidianCommunityPlugins.ts", () => ({
getObsidianCommunityPluginManager: vi.fn(), getObsidianCommunityPluginManager: vi.fn(),
})); }));
import { cancelTask } from "@/common/utils.ts"; import { cancelTask, scheduleTask } from "@/common/utils.ts";
import { CustomisationSyncContext } from "./customisationSyncContext"; import { CustomisationSyncContext } from "./customisationSyncContext";
import { createCustomisationSyncTestDependencies } from "./customisationSyncContext.unit.fixture.ts"; import { createCustomisationSyncTestDependencies } from "./customisationSyncContext.unit.fixture.ts";
describe("CustomisationSyncContext commands", () => { describe("CustomisationSyncContext commands", () => {
it("keeps the legacy dialogue methods as delegates to the host-owned UI", () => { it("opens the host-owned dialogue from a scheduled configuration Notice", async () => {
const control = { const control = {
open: vi.fn(), open: vi.fn(),
close: vi.fn(), close: vi.fn(),
isOpen: vi.fn(), isOpen: vi.fn(() => false),
}; };
const showConfigurationNotice = vi.fn();
const updatePluginList = vi.fn(async () => undefined);
const configSync = Object.create(CustomisationSyncContext.prototype) as CustomisationSyncContext; const configSync = Object.create(CustomisationSyncContext.prototype) as CustomisationSyncContext;
Object.assign(configSync, { Object.assign(configSync, {
dependencies: createCustomisationSyncTestDependencies({ dependencies: createCustomisationSyncTestDependencies({
getUIControl: () => control, getUIControl: () => control,
getSettings: () => ({ usePluginSync: true, notifyPluginOrSettingUpdated: true }) as never,
showConfigurationNotice,
}), }),
updatePluginList,
}); });
configSync.showPluginSyncModal(); await configSync.serviceHandlers.processVirtualDocument({
configSync.hidePluginSyncModal(); _id: "ix:example",
path: "ix:example",
} as never);
const scheduledNotice = vi.mocked(scheduleTask).mock.calls[0]?.[2] as (() => void) | undefined;
expect(scheduledNotice).toBeTypeOf("function");
scheduledNotice?.();
const openDialogue = showConfigurationNotice.mock.calls[0]?.[0] as (() => void) | undefined;
expect(openDialogue).toBeTypeOf("function");
openDialogue?.();
expect(control.open).toHaveBeenCalledOnce(); expect(control.open).toHaveBeenCalledOnce();
expect(control.close).toHaveBeenCalledOnce(); expect(updatePluginList).toHaveBeenCalledWith(false, "ix:example");
}); });
it("releases every owned processor and reactive subscription", () => { it("releases every owned processor and reactive subscription", () => {
@@ -85,4 +98,25 @@ describe("CustomisationSyncContext commands", () => {
expect(setEnumerationActive).toHaveBeenCalledWith(false); expect(setEnumerationActive).toHaveBeenCalledWith(false);
expect(publishScanCount).toHaveBeenCalledWith(0); expect(publishScanCount).toHaveBeenCalledWith(0);
}); });
it("characterises the inherited setting-realisation gates pending separate review", async () => {
const isReady = vi.fn(() => false);
const isSuspended = vi.fn(() => false);
const periodicPluginSweepProcessor = { disable: vi.fn(), enable: vi.fn() };
const scanAllConfigFiles = vi.fn(async () => undefined);
const configSync = Object.create(CustomisationSyncContext.prototype) as CustomisationSyncContext;
Object.assign(configSync, {
dependencies: createCustomisationSyncTestDependencies({ isReady, isSuspended }),
periodicPluginSweepProcessor,
scanAllConfigFiles,
});
await expect(configSync.serviceHandlers.onRealiseSetting()).resolves.toBe(true);
expect(periodicPluginSweepProcessor.disable).toHaveBeenCalledOnce();
expect(isReady).not.toHaveBeenCalled();
expect(isSuspended).toHaveBeenCalledOnce();
expect(scanAllConfigFiles).not.toHaveBeenCalled();
expect(periodicPluginSweepProcessor.enable).not.toHaveBeenCalled();
});
}); });
@@ -1,5 +1,4 @@
import { describe, expect, it, vi } from "vitest"; import { describe, expect, it, vi } from "vitest";
import type { FilePathWithPrefix } from "@vrtmrz/livesync-commonlib/compat/common/types";
vi.mock("@/deps.ts", () => ({ vi.mock("@/deps.ts", () => ({
diff_match_patch: class DiffMatchPatch {}, diff_match_patch: class DiffMatchPatch {},
@@ -65,7 +64,7 @@ describe("compatibility: Customisation Sync paths", () => {
[".obsidian/workspace", ""], [".obsidian/workspace", ""],
["notes/example.json", "CONFIG"], ["notes/example.json", "CONFIG"],
])("classifies %s as %s", (path, expected) => { ])("classifies %s as %s", (path, expected) => {
expect(createConfigSync().getFileCategory(path)).toBe(expected); expect(createConfigSync().testing.getFileCategory(path)).toBe(expected);
}); });
it("keeps other plug-in files outside V1 and disabled plug-in-extra synchronisation", () => { it("keeps other plug-in files outside V1 and disabled plug-in-extra synchronisation", () => {
@@ -73,17 +72,17 @@ describe("compatibility: Customisation Sync paths", () => {
const withoutPluginEtc = createConfigSync({ usePluginEtc: false }); const withoutPluginEtc = createConfigSync({ usePluginEtc: false });
const path = ".obsidian/plugins/example/other.json"; const path = ".obsidian/plugins/example/other.json";
expect(v1.getFileCategory(path)).toBe(""); expect(v1.testing.getFileCategory(path)).toBe("");
expect(withoutPluginEtc.getFileCategory(path)).toBe(""); expect(withoutPluginEtc.testing.getFileCategory(path)).toBe("");
}); });
it("recognises only classified files below the Obsidian configuration directory", () => { it("recognises only classified files below the Obsidian configuration directory", () => {
const configSync = createConfigSync(); const configSync = createConfigSync();
expect(configSync.isTargetPath(".obsidian/app.json")).toBe(true); expect(configSync.testing.isTargetPath(".obsidian/app.json")).toBe(true);
expect(configSync.isTargetPath(".obsidian/plugins/example/main.js")).toBe(true); expect(configSync.testing.isTargetPath(".obsidian/plugins/example/main.js")).toBe(true);
expect(configSync.isTargetPath(".obsidian/workspace")).toBe(false); expect(configSync.testing.isTargetPath(".obsidian/workspace")).toBe(false);
expect(configSync.isTargetPath("notes/example.json")).toBe(false); expect(configSync.testing.isTargetPath("notes/example.json")).toBe(false);
}); });
it.each([ it.each([
@@ -92,7 +91,7 @@ describe("compatibility: Customisation Sync paths", () => {
[".obsidian/plugins/example/main.js", "ix:device-a/PLUGIN_MAIN/example.md"], [".obsidian/plugins/example/main.js", "ix:device-a/PLUGIN_MAIN/example.md"],
[".obsidian/plugins/example/data.json", "ix:device-a/PLUGIN_DATA/example.md"], [".obsidian/plugins/example/data.json", "ix:device-a/PLUGIN_DATA/example.md"],
])("creates the V1 document path for %s", (path, expected) => { ])("creates the V1 document path for %s", (path, expected) => {
expect(createConfigSync().filenameToUnifiedKey(path)).toBe(expected); expect(createConfigSync().testing.filenameToUnifiedKey(path)).toBe(expected);
}); });
it.each([ it.each([
@@ -101,30 +100,18 @@ describe("compatibility: Customisation Sync paths", () => {
[".obsidian/plugins/example/main.js", "ix:device-a/PLUGIN_MAIN/example%main.js"], [".obsidian/plugins/example/main.js", "ix:device-a/PLUGIN_MAIN/example%main.js"],
[".obsidian/plugins/example/data.json", "ix:device-a/PLUGIN_DATA/example%data.json"], [".obsidian/plugins/example/data.json", "ix:device-a/PLUGIN_DATA/example%data.json"],
])("creates the V2 document path for %s", (path, expected) => { ])("creates the V2 document path for %s", (path, expected) => {
expect(createConfigSync().filenameWithUnifiedKey(path)).toBe(expected); expect(createConfigSync().testing.filenameWithUnifiedKey(path)).toBe(expected);
}); });
it("uses an explicit device name when supplied", () => { it("uses an explicit device name when supplied", () => {
const configSync = createConfigSync(); const configSync = createConfigSync();
expect(configSync.filenameToUnifiedKey(".obsidian/app.json", "device-b")).toBe( expect(configSync.testing.filenameToUnifiedKey(".obsidian/app.json", "device-b")).toBe(
"ix:device-b/CONFIG/app.json.md" "ix:device-b/CONFIG/app.json.md"
); );
expect(configSync.filenameWithUnifiedKey(".obsidian/app.json", "device-b")).toBe( expect(configSync.testing.filenameWithUnifiedKey(".obsidian/app.json", "device-b")).toBe(
"ix:device-b/CONFIG/app.json%app.json" "ix:device-b/CONFIG/app.json%app.json"
); );
expect(configSync.unifiedKeyPrefixOfTerminal("device-b")).toBe("ix:device-b/"); expect(configSync.testing.unifiedKeyPrefixOfTerminal("device-b")).toBe("ix:device-b/");
});
it("parses a V2 document path and derives its V1 compatibility path", () => {
expect(
createConfigSync().parseUnifiedPath("ix:device-a/PLUGIN_MAIN/example%main.js" as FilePathWithPrefix)
).toEqual({
device: "device-a",
category: "PLUGIN_MAIN",
key: "example",
filename: "main.js",
pathV1: "ix:device-a/PLUGIN_MAIN/example.md",
});
}); });
}); });
@@ -31,6 +31,7 @@ vi.mock("@/common/obsidianCommunityPlugins.ts", () => ({
import { scheduleTask } from "@/common/utils.ts"; import { scheduleTask } from "@/common/utils.ts";
import { CustomisationSyncContext } from "./customisationSyncContext.ts"; import { CustomisationSyncContext } from "./customisationSyncContext.ts";
import { createCustomisationSyncTestDependencies } from "./customisationSyncContext.unit.fixture.ts"; import { createCustomisationSyncTestDependencies } from "./customisationSyncContext.unit.fixture.ts";
import { CustomisationSyncRecentEventDeduplicator } from "./customisationSyncRecentEventDeduplicator.ts";
const PATH = ".obsidian/plugins/example/data.json" as FilePath; const PATH = ".obsidian/plugins/example/data.json" as FilePath;
@@ -43,9 +44,7 @@ function createConfigSync(options: { ready?: boolean; suspended?: boolean; enabl
}; };
const ownsLocalFile = vi.fn(() => options.owned ?? true); const ownsLocalFile = vi.fn(() => options.owned ?? true);
const statHidden = vi.fn(async () => ({ type: "file", mtime: 1 })); const statHidden = vi.fn(async () => ({ type: "file", mtime: 1 }));
const recentProcessedInternalFiles = Object.assign([] as string[], { const recentProcessedInternalFiles = new CustomisationSyncRecentEventDeduplicator();
contains: vi.fn(() => false),
});
const configSync = Object.create(CustomisationSyncContext.prototype) as CustomisationSyncContext; const configSync = Object.create(CustomisationSyncContext.prototype) as CustomisationSyncContext;
Object.assign(configSync, { Object.assign(configSync, {
dependencies: createCustomisationSyncTestDependencies({ dependencies: createCustomisationSyncTestDependencies({
@@ -71,7 +70,7 @@ describe("Customisation Sync raw-event admission", () => {
it("schedules a recognised path granted by the composition owner", async () => { it("schedules a recognised path granted by the composition owner", async () => {
const { configSync, ownsLocalFile } = createConfigSync(); const { configSync, ownsLocalFile } = createConfigSync();
await expect(configSync._anyProcessOptionalFileEvent(PATH)).resolves.toBe(true); await expect(configSync.serviceHandlers.processOptionalFileEvent(PATH)).resolves.toBe(true);
expect(ownsLocalFile).toHaveBeenCalledWith(PATH); expect(ownsLocalFile).toHaveBeenCalledWith(PATH);
expect(scheduleTask).toHaveBeenCalledOnce(); expect(scheduleTask).toHaveBeenCalledOnce();
}); });
@@ -79,7 +78,7 @@ describe("Customisation Sync raw-event admission", () => {
it("rejects an event while the host is not ready", async () => { it("rejects an event while the host is not ready", async () => {
const { configSync, statHidden } = createConfigSync({ ready: false }); const { configSync, statHidden } = createConfigSync({ ready: false });
await expect(configSync._anyProcessOptionalFileEvent(PATH)).resolves.toBe(false); await expect(configSync.serviceHandlers.processOptionalFileEvent(PATH)).resolves.toBe(false);
expect(statHidden).not.toHaveBeenCalled(); expect(statHidden).not.toHaveBeenCalled();
expect(scheduleTask).not.toHaveBeenCalled(); expect(scheduleTask).not.toHaveBeenCalled();
}); });
@@ -87,7 +86,9 @@ describe("Customisation Sync raw-event admission", () => {
it("rejects a path outside the recognised Customisation Sync categories", async () => { it("rejects a path outside the recognised Customisation Sync categories", async () => {
const { configSync, ownsLocalFile } = createConfigSync(); const { configSync, ownsLocalFile } = createConfigSync();
await expect(configSync._anyProcessOptionalFileEvent(".obsidian/workspace" as FilePath)).resolves.toBe(false); await expect(configSync.serviceHandlers.processOptionalFileEvent(".obsidian/workspace" as FilePath)).resolves.toBe(
false
);
expect(ownsLocalFile).not.toHaveBeenCalled(); expect(ownsLocalFile).not.toHaveBeenCalled();
expect(scheduleTask).not.toHaveBeenCalled(); expect(scheduleTask).not.toHaveBeenCalled();
}); });
@@ -95,7 +96,7 @@ describe("Customisation Sync raw-event admission", () => {
it("rejects a recognised path assigned to another owner", async () => { it("rejects a recognised path assigned to another owner", async () => {
const { configSync, statHidden } = createConfigSync({ owned: false }); const { configSync, statHidden } = createConfigSync({ owned: false });
await expect(configSync._anyProcessOptionalFileEvent(PATH)).resolves.toBe(false); await expect(configSync.serviceHandlers.processOptionalFileEvent(PATH)).resolves.toBe(false);
expect(statHidden).not.toHaveBeenCalled(); expect(statHidden).not.toHaveBeenCalled();
expect(scheduleTask).not.toHaveBeenCalled(); expect(scheduleTask).not.toHaveBeenCalled();
}); });
@@ -106,7 +107,7 @@ describe("Customisation Sync raw-event admission", () => {
] as const)("rejects an event while %s", async (_label, options) => { ] as const)("rejects an event while %s", async (_label, options) => {
const { configSync } = createConfigSync(options); const { configSync } = createConfigSync(options);
await expect(configSync._anyProcessOptionalFileEvent(PATH)).resolves.toBe(false); await expect(configSync.serviceHandlers.processOptionalFileEvent(PATH)).resolves.toBe(false);
expect(scheduleTask).not.toHaveBeenCalled(); expect(scheduleTask).not.toHaveBeenCalled();
}); });
}); });
@@ -40,4 +40,46 @@ describe("CustomisationSyncContext state ownership", () => {
expect(first.manifests).not.toBe(second.manifests); expect(first.manifests).not.toBe(second.manifests);
expect(get(first.manifests)).not.toBe(get(second.manifests)); expect(get(first.manifests)).not.toBe(get(second.manifests));
}); });
it("exposes frozen semantic service and testing views without writable state", () => {
const context = new CustomisationSyncContext(createCustomisationSyncTestDependencies());
expect(Object.isFrozen(context.serviceHandlers)).toBe(true);
expect(Object.keys(context.serviceHandlers).sort()).toEqual(
[
"enableOptionalFeature",
"onBeforeReplicate",
"onDatabaseInitialised",
"onRealiseSetting",
"onResuming",
"processOptionalFileEvent",
"processVirtualDocument",
"suspendExtraSync",
].sort()
);
expect(Object.isFrozen(context.testing)).toBe(true);
expect(Object.keys(context.testing).sort()).toEqual(
[
"applyDataV2",
"configDir",
"createPluginDataExFileV2",
"createPluginDataFromV2",
"deleteConfigOnDatabase",
"filenameToUnifiedKey",
"filenameWithUnifiedKey",
"getFileCategory",
"isTargetPath",
"scanAllConfigFiles",
"scanInternalFiles",
"storeCustomizationFiles",
"unifiedKeyPrefixOfTerminal",
].sort()
);
expect("catalogue" in context.testing).toBe(false);
expect("enumerationActive" in context.testing).toBe(false);
expect("manifests" in context.testing).toBe(false);
context.dispose();
});
}); });
@@ -1,6 +1,6 @@
import { writable } from "svelte/store"; import { writable } from "svelte/store";
import type PouchDB from "pouchdb-core"; import type PouchDB from "pouchdb-core";
import { type PluginManifest, parseYaml, normalizePath, type ListedFiles, diff_match_patch } from "@/deps.ts"; import { type PluginManifest, parseYaml, normalizePath, diff_match_patch } from "@/deps.ts";
import type { import type {
EntryDoc, EntryDoc,
@@ -25,25 +25,18 @@ import {
import { ICXHeader, PERIODIC_PLUGIN_SWEEP } from "@/common/types.ts"; import { ICXHeader, PERIODIC_PLUGIN_SWEEP } from "@/common/types.ts";
import { import {
createBlob, createBlob,
createSavingEntryFromLoadedEntry,
createTextBlob, createTextBlob,
delay, delay,
fireAndForget, fireAndForget,
getDocData, getDocData,
getDocDataAsArray, getDocDataAsArray,
isDocContentSame, isDocContentSame,
isLoadedEntry,
isObjectDifferent,
} from "@vrtmrz/livesync-commonlib/compat/common/utils"; } from "@vrtmrz/livesync-commonlib/compat/common/utils";
import { digestHash } from "@vrtmrz/livesync-commonlib/compat/string_and_binary/hash"; import { digestHash } from "@vrtmrz/livesync-commonlib/compat/string_and_binary/hash";
import { import { arrayBufferToBase64, decodeBinary } from "@vrtmrz/livesync-commonlib/compat/string_and_binary/convert";
arrayBufferToBase64,
decodeBinary,
readString,
} from "@vrtmrz/livesync-commonlib/compat/string_and_binary/convert";
import { serialized, shareRunningResult } from "octagonal-wheels/concurrency/lock"; import { serialized, shareRunningResult } from "octagonal-wheels/concurrency/lock";
import { stripAllPrefixes } from "@vrtmrz/livesync-commonlib/compat/string_and_binary/path"; import { stripAllPrefixes } from "@vrtmrz/livesync-commonlib/compat/string_and_binary/path";
import { cancelTask, EVEN, isCustomisationSyncMetadata, isPluginMetadata, scheduleTask } from "@/common/utils.ts"; import { cancelTask, EVEN, scheduleTask } from "@/common/utils.ts";
import { QueueProcessor } from "octagonal-wheels/concurrency/processor"; import { QueueProcessor } from "octagonal-wheels/concurrency/processor";
import { reactiveSource, type ReactiveSource } from "octagonal-wheels/dataobject/reactive"; import { reactiveSource, type ReactiveSource } from "octagonal-wheels/dataobject/reactive";
import { base64ToArrayBuffer, base64ToString } from "octagonal-wheels/binary/base64"; import { base64ToArrayBuffer, base64ToString } from "octagonal-wheels/binary/base64";
@@ -59,13 +52,15 @@ import {
isCustomisationSyncTargetPath, isCustomisationSyncTargetPath,
parseCustomisationSyncV2DocumentPath, parseCustomisationSyncV2DocumentPath,
} from "./customisationSyncPaths.ts"; } from "./customisationSyncPaths.ts";
import { createCustomisationSyncCodec, type PluginDataEx, type PluginDataExFile } from "./customisationSyncCodec.ts"; import { createCustomisationSyncCodec, type PluginDataEx } from "./customisationSyncCodec.ts";
import type { import type {
CustomisationSyncDialogView, CustomisationSyncDialogView,
CustomisationSyncUIControl, CustomisationSyncUIControl,
CustomisationSyncFileCategory,
CustomisationSyncServiceHandlers,
CustomisationSyncTestingView,
IPluginDataExDisplay, IPluginDataExDisplay,
LoadedEntryPluginDataExFile, LoadedEntryPluginDataExFile,
PluginDataExDisplay,
} from "./customisationSyncView.ts"; } from "./customisationSyncView.ts";
import { import {
REPLICATION_PROGRESS_PRESENTATIONS, REPLICATION_PROGRESS_PRESENTATIONS,
@@ -75,9 +70,29 @@ import type { LiveSyncLocalDB } from "@vrtmrz/livesync-commonlib/compat/pouchdb/
import type { StorageAccess } from "@vrtmrz/livesync-commonlib/compat/interfaces/StorageAccess"; import type { StorageAccess } from "@vrtmrz/livesync-commonlib/compat/interfaces/StorageAccess";
import type { IPathService, IReplicationService } from "@vrtmrz/livesync-commonlib/compat/services/base/IService"; import type { IPathService, IReplicationService } from "@vrtmrz/livesync-commonlib/compat/services/base/IService";
import type { LogFunction } from "@vrtmrz/livesync-commonlib/compat/services/lib/logUtils"; import type { LogFunction } from "@vrtmrz/livesync-commonlib/compat/services/lib/logUtils";
import {
collectOptionalFileSyncFiles,
type OptionalFileSyncFileTreeDependencies,
} from "@/features/optionalFileSyncFileTree.ts";
import { PluginDataExDisplayV2 } from "./customisationSyncModel.ts";
import {
decodeCustomisationSyncV2File,
loadCustomisationDisplayData,
loadCustomisationV2Entry,
readCustomisationFile,
} from "./customisationSyncReadOperations.ts";
import { CustomisationSyncCatalogueState } from "./customisationSyncCatalogueState.ts";
import { CustomisationSyncRecentEventDeduplicator } from "./customisationSyncRecentEventDeduplicator.ts";
export type { PluginDataEx, PluginDataExFile } from "./customisationSyncCodec.ts"; export type { PluginDataEx, PluginDataExFile } from "./customisationSyncCodec.ts";
export type { IPluginDataExDisplay, PluginDataExDisplay } from "./customisationSyncView.ts"; export type {
CustomisationSyncFileCategory,
CustomisationSyncServiceHandlers,
CustomisationSyncTestingView,
IPluginDataExDisplay,
PluginDataExDisplay,
} from "./customisationSyncView.ts";
export { PluginDataExDisplayV2 } from "./customisationSyncModel.ts";
const UPDATED_CONFIGURATION_NOTICE_KEY = "config-sync:updated-configuration"; const UPDATED_CONFIGURATION_NOTICE_KEY = "config-sync:updated-configuration";
@@ -87,87 +102,8 @@ const {
dummyHead: DUMMY_HEAD, dummyHead: DUMMY_HEAD,
dummyEnd: DUMMY_END, dummyEnd: DUMMY_END,
} = createCustomisationSyncCodec({ digestHash, parseYaml }); } = createCustomisationSyncCodec({ digestHash, parseYaml });
const CUSTOMISATION_SYNC_READ_CODEC = { deserialize, serialize };
function categoryToFolder(category: string, configDir: string = ""): string {
switch (category) {
case "CONFIG":
return `${configDir}/`;
case "THEME":
return `${configDir}/themes/`;
case "SNIPPET":
return `${configDir}/snippets/`;
case "PLUGIN_MAIN":
return `${configDir}/plugins/`;
case "PLUGIN_DATA":
return `${configDir}/plugins/`;
case "PLUGIN_ETC":
return `${configDir}/plugins/`;
default:
return "";
}
}
export class PluginDataExDisplayV2 {
documentPath: FilePathWithPrefix;
category: string;
term: string;
files = [] as LoadedEntryPluginDataExFile[];
name: string;
confKey: string;
constructor(
data: IPluginDataExDisplay,
private readonly manifestLookup: ReadonlyMap<string, PluginManifest>
) {
this.documentPath = `${data.documentPath}` as FilePathWithPrefix;
this.category = `${data.category}`;
this.name = `${data.name}`;
this.term = `${data.term}`;
this.files = [...(data.files as LoadedEntryPluginDataExFile[])];
this.confKey = `${categoryToFolder(this.category, this.term)}${this.name}`;
this.applyLoadedManifest();
}
async setFile(file: LoadedEntryPluginDataExFile) {
const old = this.files.find((e) => e.filename == file.filename);
if (old) {
if (old.mtime == file.mtime && (await isDocContentSame(old.data, file.data))) return;
this.files = this.files.filter((e) => e.filename != file.filename);
}
this.files.push(file);
if (file.filename == "manifest.json") {
this.applyLoadedManifest();
}
}
deleteFile(filename: string) {
this.files = this.files.filter((e) => e.filename != filename);
}
_displayName: string | undefined;
_version: string | undefined;
applyLoadedManifest() {
const manifest = this.manifestLookup.get(this.confKey);
if (manifest) {
this._displayName = manifest.name;
if (this.category == "PLUGIN_MAIN" || this.category == "THEME") {
this._version = manifest?.version;
}
}
}
get displayName(): string {
// if (this._displayNameBuffer !== symbolUnInitialised) return this._displayNameBuffer;
// return this._bufferManifest().displayName;
return this._displayName || this.name;
}
get version(): string | undefined {
return this._version;
}
get mtime(): number {
return ~~this.files.reduce((a, b) => a + b.mtime, 0) / this.files.length;
}
}
type CustomisationSyncSettings = Pick< type CustomisationSyncSettings = Pick<
ObsidianLiveSyncSettings, ObsidianLiveSyncSettings,
| "usePluginSync" | "usePluginSync"
@@ -195,7 +131,7 @@ export type CustomisationSyncPeriodicProcessor = {
disable(): void; disable(): void;
}; };
export type CustomisationSyncContextDependencies = { export type CustomisationSyncContextDependencies = OptionalFileSyncFileTreeDependencies & {
getSettings(): CustomisationSyncSettings; getSettings(): CustomisationSyncSettings;
getLocalDatabase(): CustomisationSyncDatabase; getLocalDatabase(): CustomisationSyncDatabase;
storageAccess: CustomisationSyncStorage; storageAccess: CustomisationSyncStorage;
@@ -212,7 +148,6 @@ export type CustomisationSyncContextDependencies = {
isSuspended(): boolean; isSuspended(): boolean;
askRestart(): void; askRestart(): void;
createPeriodicProcessor(process: () => Promise<unknown>): CustomisationSyncPeriodicProcessor; createPeriodicProcessor(process: () => Promise<unknown>): CustomisationSyncPeriodicProcessor;
listFiles(path: string): Promise<ListedFiles>;
resolveJsonConflict( resolveJsonConflict(
path: FilePath, path: FilePath,
files: [LoadedEntryPluginDataExFile, LoadedEntryPluginDataExFile], files: [LoadedEntryPluginDataExFile, LoadedEntryPluginDataExFile],
@@ -232,19 +167,22 @@ export type CustomisationSyncContextDependencies = {
export class CustomisationSyncContext implements CustomisationSyncDialogView { export class CustomisationSyncContext implements CustomisationSyncDialogView {
private readonly dependencies: CustomisationSyncContextDependencies; private readonly dependencies: CustomisationSyncContextDependencies;
private readonly catalogueState = new CustomisationSyncCatalogueState();
private readonly recentProcessedInternalFiles = new CustomisationSyncRecentEventDeduplicator();
private serviceHandlersView: CustomisationSyncServiceHandlers | undefined;
private testingView: CustomisationSyncTestingView | undefined;
private readonly scanProgress = reactiveSource(0); private readonly scanProgress = reactiveSource(0);
private readonly pluginScanningChanged: Parameters<ReactiveSource<number>["onChanged"]>[0] = (event) => { private readonly pluginScanningChanged: Parameters<ReactiveSource<number>["onChanged"]>[0] = (event) => {
this.enumerationActive.set(event.value != 0); this.enumerationActive.set(event.value != 0);
this.dependencies.publishScanCount(event.value); this.dependencies.publishScanCount(event.value);
}; };
readonly catalogue = writable<IPluginDataExDisplay[]>([]);
readonly enumerationActive = writable(false); readonly enumerationActive = writable(false);
readonly migrationProgress = writable(0); readonly catalogue = this.catalogueState.catalogue;
private readonly pluginManifests = new Map<string, PluginManifest>(); readonly migrationProgress = this.catalogueState.migrationProgress;
readonly manifests = writable(this.pluginManifests); readonly manifests = this.catalogueState.manifests;
readonly periodicPluginSweepProcessor: CustomisationSyncPeriodicProcessor; private readonly periodicPluginSweepProcessor: CustomisationSyncPeriodicProcessor;
constructor(dependencies: CustomisationSyncContextDependencies) { constructor(dependencies: CustomisationSyncContextDependencies) {
this.dependencies = dependencies; this.dependencies = dependencies;
@@ -253,7 +191,62 @@ export class CustomisationSyncContext implements CustomisationSyncDialogView {
); );
this.scanProgress.onChanged(this.pluginScanningChanged); this.scanProgress.onChanged(this.pluginScanningChanged);
} }
get configDir() {
/**
* Semantic callbacks for registration by the optional-file composition
* feature. The returned object is immutable, and each callback retains its
* context without requiring callers to bind a concrete implementation.
*/
get serviceHandlers(): CustomisationSyncServiceHandlers {
if (!this.serviceHandlersView) {
this.serviceHandlersView = Object.freeze({
processOptionalFileEvent: (path: FilePath) => this.processOptionalFileEvent(path),
processVirtualDocument: (docs: PouchDB.Core.ExistingDocument<EntryDoc>) =>
this.processVirtualDocument(docs),
onRealiseSetting: () => this.realiseSettingSyncMode(),
onResuming: () => this.onResumeProcess(),
onBeforeReplicate: (showMessage: boolean) => this.beforeReplicate(showMessage),
onDatabaseInitialised: (showNotice: boolean) => this.onDatabaseInitialised(showNotice),
suspendExtraSync: () => this.suspendExtraSync(),
enableOptionalFeature: (mode: OptionalSyncFeatureMode) => this.enableOptionalFeature(mode),
});
}
return this.serviceHandlersView;
}
/**
* Narrow internal surface used by maintained real-Obsidian contract tests.
* It intentionally omits the context, queues, and writable stores.
*/
get testing(): CustomisationSyncTestingView {
if (!this.testingView) {
this.testingView = Object.freeze({
configDir: this.configDir,
scanInternalFiles: async () => await this.scanInternalFiles(),
scanAllConfigFiles: async (showMessage: boolean) => await this.scanAllConfigFiles(showMessage),
getFileCategory: (filePath: string) => this.getFileCategory(filePath),
isTargetPath: (filePath: string) => this.isTargetPath(filePath),
filenameToUnifiedKey: (path: string, termOverride?: string) =>
this.filenameToUnifiedKey(path, termOverride),
filenameWithUnifiedKey: (path: string, termOverride?: string) =>
this.filenameWithUnifiedKey(path, termOverride),
unifiedKeyPrefixOfTerminal: (termOverride?: string) =>
this.unifiedKeyPrefixOfTerminal(termOverride),
storeCustomizationFiles: async (path: FilePath, termOverride?: string) =>
await this.storeCustomizationFiles(path, termOverride),
deleteConfigOnDatabase: async (path: FilePathWithPrefix, forceWrite?: boolean) =>
await this.deleteConfigOnDatabase(path, forceWrite),
createPluginDataFromV2: (path: FilePathWithPrefix) => this.createPluginDataFromV2(path),
createPluginDataExFileV2: async (path: FilePathWithPrefix, loaded?: LoadedEntry) =>
await this.createPluginDataExFileV2(path, loaded),
applyDataV2: async (data: PluginDataExDisplayV2, content?: string) =>
await this.applyDataV2(data, content),
});
}
return this.testingView;
}
private get configDir() {
return this.dependencies.getConfigDir(); return this.dependencies.getConfigDir();
} }
@@ -277,11 +270,11 @@ export class CustomisationSyncContext implements CustomisationSyncDialogView {
return this.dependencies.path.getPath(entry); return this.dependencies.path.getPath(entry);
} }
_isMainReady() { private _isMainReady() {
return this.dependencies.isReady(); return this.dependencies.isReady();
} }
_isMainSuspended() { private _isMainSuspended() {
return this.dependencies.isSuspended(); return this.dependencies.isSuspended();
} }
@@ -289,13 +282,13 @@ export class CustomisationSyncContext implements CustomisationSyncDialogView {
this.dependencies.log(message, level, key); this.dependencies.log(message, level, key);
} }
get useV2() { private get useV2() {
return this.settings.usePluginSyncV2; return this.settings.usePluginSyncV2;
} }
get useSyncPluginEtc() { private get useSyncPluginEtc() {
return this.settings.usePluginEtc; return this.settings.usePluginEtc;
} }
isThisModuleEnabled() { private isThisModuleEnabled() {
return this.settings.usePluginSync; return this.settings.usePluginSync;
} }
@@ -355,11 +348,11 @@ export class CustomisationSyncContext implements CustomisationSyncDialogView {
): Promise<boolean> { ): Promise<boolean> {
const dataACopy = const dataACopy =
dataA instanceof PluginDataExDisplayV2 dataA instanceof PluginDataExDisplayV2
? new PluginDataExDisplayV2(dataA, this.pluginManifests) ? new PluginDataExDisplayV2(dataA, this.catalogueState.manifestLookup)
: { ...dataA }; : { ...dataA };
const dataBCopy = const dataBCopy =
dataB instanceof PluginDataExDisplayV2 dataB instanceof PluginDataExDisplayV2
? new PluginDataExDisplayV2(dataB, this.pluginManifests) ? new PluginDataExDisplayV2(dataB, this.catalogueState.manifestLookup)
: { ...dataB }; : { ...dataB };
dataACopy.files = dataACopy.files.filter((file) => file.filename == filename); dataACopy.files = dataACopy.files.filter((file) => file.filename == filename);
dataBCopy.files = dataBCopy.files.filter((file) => file.filename == filename); dataBCopy.files = dataBCopy.files.filter((file) => file.filename == filename);
@@ -372,14 +365,6 @@ export class CustomisationSyncContext implements CustomisationSyncDialogView {
await this.updatePluginList(false, this.filenameToUnifiedKey(path, deviceName)); await this.updatePluginList(false, this.filenameToUnifiedKey(path, deviceName));
} }
pluginList: IPluginDataExDisplay[] = [];
showPluginSyncModal() {
this.dependencies.getUIControl()?.open();
}
hidePluginSyncModal() {
this.dependencies.getUIControl()?.close();
}
dispose() { dispose() {
cancelTask(UPDATED_CONFIGURATION_NOTICE_KEY); cancelTask(UPDATED_CONFIGURATION_NOTICE_KEY);
this.periodicPluginSweepProcessor?.disable(); this.periodicPluginSweepProcessor?.disable();
@@ -391,30 +376,21 @@ export class CustomisationSyncContext implements CustomisationSyncDialogView {
this.dependencies.hideConfigurationNotice(); this.dependencies.hideConfigurationNotice();
} }
private setManifest(key: string, manifest: PluginManifest) { private getFileCategory(filePath: string): CustomisationSyncFileCategory {
const old = this.pluginManifests.get(key);
if (old && !isObjectDifferent(manifest, old)) return;
this.pluginManifests.set(key, manifest);
this.manifests.set(this.pluginManifests);
}
getFileCategory(
filePath: string
): "CONFIG" | "THEME" | "SNIPPET" | "PLUGIN_MAIN" | "PLUGIN_ETC" | "PLUGIN_DATA" | "" {
return getCustomisationSyncFileCategory(filePath, { return getCustomisationSyncFileCategory(filePath, {
configDir: this.configDir, configDir: this.configDir,
useV2: this.useV2, useV2: this.useV2,
usePluginEtc: this.useSyncPluginEtc, usePluginEtc: this.useSyncPluginEtc,
}); });
} }
isTargetPath(filePath: string): boolean { private isTargetPath(filePath: string): boolean {
return isCustomisationSyncTargetPath(filePath, { return isCustomisationSyncTargetPath(filePath, {
configDir: this.configDir, configDir: this.configDir,
useV2: this.useV2, useV2: this.useV2,
usePluginEtc: this.useSyncPluginEtc, usePluginEtc: this.useSyncPluginEtc,
}); });
} }
async _everyOnDatabaseInitialized(showNotice: boolean) { private async onDatabaseInitialised(showNotice: boolean) {
if (!this.isThisModuleEnabled()) return true; if (!this.isThisModuleEnabled()) return true;
try { try {
this._log("Scanning customizations..."); this._log("Scanning customizations...");
@@ -426,7 +402,7 @@ export class CustomisationSyncContext implements CustomisationSyncDialogView {
} }
return true; return true;
} }
async _everyBeforeReplicate(showNotice: boolean) { private async beforeReplicate(showNotice: boolean) {
if (!this.isThisModuleEnabled()) return true; if (!this.isThisModuleEnabled()) return true;
if (this.settings.autoSweepPlugins) { if (this.settings.autoSweepPlugins) {
await this.scanAllConfigFiles(showNotice); await this.scanAllConfigFiles(showNotice);
@@ -434,7 +410,7 @@ export class CustomisationSyncContext implements CustomisationSyncDialogView {
} }
return true; return true;
} }
async _everyOnResumeProcess(): Promise<boolean> { private async onResumeProcess(): Promise<boolean> {
if (!this.isThisModuleEnabled()) return true; if (!this.isThisModuleEnabled()) return true;
if (this._isMainSuspended()) { if (this._isMainSuspended()) {
return true; return true;
@@ -450,44 +426,10 @@ export class CustomisationSyncContext implements CustomisationSyncDialogView {
return true; return true;
} }
async reloadPluginList(showMessage: boolean) { async reloadPluginList(showMessage: boolean) {
this.pluginList = []; this.catalogueState.clearForReload();
this.loadedManifest_mTime.clear();
this.catalogue.set(this.pluginList);
await this.updatePluginList(showMessage); await this.updatePluginList(showMessage);
} }
async loadPluginData(path: FilePathWithPrefix): Promise<PluginDataExDisplay | false> { private pluginScanProcessor = new QueueProcessor(
const wx = await this.localDatabase.getDBEntry(path, undefined, false, false);
if (wx) {
const data = deserialize(getDocDataAsArray(wx.data), {}) as PluginDataEx;
const xFiles = [] as PluginDataExFile[];
let missingHash = false;
for (const file of data.files) {
const work = { ...file, data: [] as string[] };
if (!file.hash) {
// debugger;
const tempStr = getDocDataAsArray(work.data);
const hash = digestHash(tempStr);
file.hash = hash;
missingHash = true;
}
work.data = [file.hash];
xFiles.push(work);
}
if (missingHash) {
this._log(`Digest created for ${path} to improve checking`, LOG_LEVEL_VERBOSE);
wx.data = serialize(data);
fireAndForget(() => this.localDatabase.putDBEntry(createSavingEntryFromLoadedEntry(wx)));
}
return {
...data,
documentPath: this.getPath(wx),
files: xFiles,
} satisfies PluginDataExDisplay;
}
return false;
}
pluginScanProcessor = new QueueProcessor(
async (v: AnyEntry[]) => { async (v: AnyEntry[]) => {
const plugin = v[0]; const plugin = v[0];
if (this.useV2) { if (this.useV2) {
@@ -495,16 +437,16 @@ export class CustomisationSyncContext implements CustomisationSyncDialogView {
return []; return [];
} }
const path = plugin.path || this.getPath(plugin); const path = plugin.path || this.getPath(plugin);
const oldEntry = this.pluginList.find((e) => e.documentPath == path); const oldEntry = this.catalogueState.findPlugin(path);
if (oldEntry && oldEntry.mtime == plugin.mtime) return []; if (oldEntry && oldEntry.mtime == plugin.mtime) return [];
try { try {
const pluginData = await this.loadPluginData(path); const pluginData = await loadCustomisationDisplayData(
this.dependencies,
path,
CUSTOMISATION_SYNC_READ_CODEC
);
if (pluginData) { if (pluginData) {
let newList = [...this.pluginList]; this.catalogueState.replacePlugin(pluginData);
newList = newList.filter((x) => x.documentPath != pluginData.documentPath);
newList.push(pluginData);
this.pluginList = newList;
this.catalogue.set(newList);
} }
// Failed to load // Failed to load
return []; return [];
@@ -525,20 +467,23 @@ export class CustomisationSyncContext implements CustomisationSyncDialogView {
} }
).startPipeline(); ).startPipeline();
pluginScanProcessorV2 = new QueueProcessor( // Compatibility question: no production path currently enqueues work into
// this second processor. Preserve its construction and disposal until the
// intended V2 scan path, or its safe removal, has focused coverage.
private pluginScanProcessorV2 = new QueueProcessor(
async (v: AnyEntry[]) => { async (v: AnyEntry[]) => {
const plugin = v[0]; const plugin = v[0];
const path = plugin.path || this.getPath(plugin); const path = plugin.path || this.getPath(plugin);
const oldEntry = this.pluginList.find((e) => e.documentPath == path); const oldEntry = this.catalogueState.findPlugin(path);
if (oldEntry && oldEntry.mtime == plugin.mtime) return []; if (oldEntry && oldEntry.mtime == plugin.mtime) return [];
try { try {
const pluginData = await this.loadPluginData(path); const pluginData = await loadCustomisationDisplayData(
this.dependencies,
path,
CUSTOMISATION_SYNC_READ_CODEC
);
if (pluginData) { if (pluginData) {
let newList = [...this.pluginList]; this.catalogueState.replacePlugin(pluginData);
newList = newList.filter((x) => x.documentPath != pluginData.documentPath);
newList.push(pluginData);
this.pluginList = newList;
this.catalogue.set(newList);
} }
// Failed to load // Failed to load
return []; return [];
@@ -559,7 +504,7 @@ export class CustomisationSyncContext implements CustomisationSyncDialogView {
} }
).startPipeline(); ).startPipeline();
filenameToUnifiedKey(path: string, termOverRide?: string) { private filenameToUnifiedKey(path: string, termOverRide?: string): FilePathWithPrefix {
const term = termOverRide || this.dependencies.getDeviceAndVaultName(); const term = termOverRide || this.dependencies.getDeviceAndVaultName();
return createCustomisationSyncV1DocumentPath(path, term, { return createCustomisationSyncV1DocumentPath(path, term, {
configDir: this.configDir, configDir: this.configDir,
@@ -568,7 +513,7 @@ export class CustomisationSyncContext implements CustomisationSyncDialogView {
}); });
} }
filenameWithUnifiedKey(path: string, termOverRide?: string) { private filenameWithUnifiedKey(path: string, termOverRide?: string): FilePathWithPrefix {
const term = termOverRide || this.dependencies.getDeviceAndVaultName(); const term = termOverRide || this.dependencies.getDeviceAndVaultName();
return createCustomisationSyncV2DocumentPath(path, term, { return createCustomisationSyncV2DocumentPath(path, term, {
configDir: this.configDir, configDir: this.configDir,
@@ -577,88 +522,38 @@ export class CustomisationSyncContext implements CustomisationSyncDialogView {
}); });
} }
unifiedKeyPrefixOfTerminal(termOverRide?: string) { private unifiedKeyPrefixOfTerminal(termOverRide?: string): string {
const term = termOverRide || this.dependencies.getDeviceAndVaultName(); const term = termOverRide || this.dependencies.getDeviceAndVaultName();
return createCustomisationSyncDevicePrefix(term); return createCustomisationSyncDevicePrefix(term);
} }
parseUnifiedPath(unifiedPath: FilePathWithPrefix): { private async createPluginDataExFileV2(
category: string;
device: string;
key: string;
filename: string;
pathV1: FilePathWithPrefix;
} {
return parseCustomisationSyncV2DocumentPath(unifiedPath);
}
loadedManifest_mTime = new Map<string, number>();
async createPluginDataExFileV2(
unifiedPathV2: FilePathWithPrefix, unifiedPathV2: FilePathWithPrefix,
loaded?: LoadedEntry loaded?: LoadedEntry
): Promise<false | LoadedEntryPluginDataExFile> { ): Promise<false | LoadedEntryPluginDataExFile> {
const { category, key, filename, device } = this.parseUnifiedPath(unifiedPathV2); // Compatibility: a caller-supplied entry bypasses the database lookup
if (!loaded) { // and the isLoadedEntry check performed by loadCustomisationV2Entry.
const d = await this.localDatabase.getDBEntry(unifiedPathV2); const loadedEntry = loaded ?? (await loadCustomisationV2Entry(this.dependencies, unifiedPathV2));
if (!d) { if (!loadedEntry) return false;
this._log(`The file ${unifiedPathV2} is not found`, LOG_LEVEL_VERBOSE); const { confKey, file, isManifest } = decodeCustomisationSyncV2File(unifiedPathV2, loadedEntry, DUMMY_END);
return false; if (isManifest) {
} this.catalogueState.processManifest(
if (!isLoadedEntry(d)) { confKey,
this._log(`The file ${unifiedPathV2} is not a note`, LOG_LEVEL_VERBOSE); file.mtime,
return false; () => JSON.parse(file.data[0]) as PluginManifest,
} (error) => {
loaded = d;
}
const confKey = `${categoryToFolder(category, device)}${key}`;
const relativeFilename =
`${categoryToFolder(category, "")}${category == "CONFIG" || category == "SNIPPET" ? "" : key + "/"}${filename}`.substring(
1
);
const dataSrc = getDocData(loaded.data);
const dataStart = dataSrc.indexOf(DUMMY_END);
const data = dataSrc.substring(dataStart + DUMMY_END.length);
const file: LoadedEntryPluginDataExFile = {
...loaded,
hash: "",
data: [base64ToString(data)],
filename: relativeFilename,
displayName: filename,
};
if (filename == "manifest.json") {
// Same as previously loaded
if (
this.loadedManifest_mTime.get(confKey) != file.mtime &&
this.pluginManifests.get(confKey) == undefined
) {
try {
const parsedManifest = JSON.parse(base64ToString(data)) as PluginManifest;
this.setManifest(confKey, parsedManifest);
this.pluginList
.filter((e) => e instanceof PluginDataExDisplayV2 && e.confKey == confKey)
.forEach((e) => (e as PluginDataExDisplayV2).applyLoadedManifest());
this.catalogue.set(this.pluginList);
} catch (ex) {
this._log( this._log(
`The file ${loaded.path} seems to manifest, but could not be decoded as JSON`, `The file ${loadedEntry.path} seems to manifest, but could not be decoded as JSON`,
LOG_LEVEL_VERBOSE LOG_LEVEL_VERBOSE
); );
this._log(ex, LOG_LEVEL_VERBOSE); this._log(error, LOG_LEVEL_VERBOSE);
} }
this.loadedManifest_mTime.set(confKey, file.mtime); );
} else {
this.pluginList
.filter((e) => e instanceof PluginDataExDisplayV2 && e.confKey == confKey)
.forEach((e) => (e as PluginDataExDisplayV2).applyLoadedManifest());
this.catalogue.set(this.pluginList);
}
// }
} }
return file; return file;
} }
createPluginDataFromV2(unifiedPathV2: FilePathWithPrefix) { private createPluginDataFromV2(unifiedPathV2: FilePathWithPrefix) {
const { category, device, key, pathV1 } = this.parseUnifiedPath(unifiedPathV2); const { category, device, key, pathV1 } = parseCustomisationSyncV2DocumentPath(unifiedPathV2);
if (category == "") return; if (category == "") return;
const ret: PluginDataExDisplayV2 = new PluginDataExDisplayV2( const ret: PluginDataExDisplayV2 = new PluginDataExDisplayV2(
@@ -670,21 +565,18 @@ export class CustomisationSyncContext implements CustomisationSyncDialogView {
files: [], files: [],
mtime: 0, mtime: 0,
}, },
this.pluginManifests this.catalogueState.manifestLookup
); );
return ret; return ret;
} }
updatingV2Count = 0; private async updatePluginListV2(showMessage: boolean, unifiedFilenameWithKey: FilePathWithPrefix): Promise<void> {
async updatePluginListV2(showMessage: boolean, unifiedFilenameWithKey: FilePathWithPrefix): Promise<void> {
try { try {
this.updatingV2Count++; this.catalogueState.beginUpdate();
this.migrationProgress.set(this.updatingV2Count);
// const unifiedFilenameWithKey = this.filenameWithUnifiedKey(updatedDocumentPath); // const unifiedFilenameWithKey = this.filenameWithUnifiedKey(updatedDocumentPath);
const { pathV1 } = this.parseUnifiedPath(unifiedFilenameWithKey); const { pathV1 } = parseCustomisationSyncV2DocumentPath(unifiedFilenameWithKey);
const oldEntry = this.pluginList.find((e) => e.documentPath == pathV1); const oldEntry = this.catalogueState.findPlugin(pathV1);
let entry: PluginDataExDisplayV2 | undefined = undefined; let entry: PluginDataExDisplayV2 | undefined = undefined;
if (!oldEntry || !(oldEntry instanceof PluginDataExDisplayV2)) { if (!oldEntry || !(oldEntry instanceof PluginDataExDisplayV2)) {
@@ -697,28 +589,19 @@ export class CustomisationSyncContext implements CustomisationSyncDialogView {
} }
if (!entry) return; if (!entry) return;
const file = await this.createPluginDataExFileV2(unifiedFilenameWithKey); const file = await this.createPluginDataExFileV2(unifiedFilenameWithKey);
if (file) { // Compatibility: the inherited update always re-adds an empty V2
await entry.setFile(file); // row after deleting its final file.
} else { await this.catalogueState.updateV2Plugin(entry, file, unifiedFilenameWithKey);
entry.deleteFile(unifiedFilenameWithKey);
if (entry.files.length == 0) {
this.pluginList = this.pluginList.filter((e) => e.documentPath != pathV1);
}
}
const newList = this.pluginList.filter((e) => e.documentPath != entry.documentPath);
newList.push(entry);
this.pluginList = newList;
scheduleTask("updatePluginListV2", 100, () => { scheduleTask("updatePluginListV2", 100, () => {
this.catalogue.set(this.pluginList); this.catalogueState.publishCatalogue();
}); });
} finally { } finally {
this.updatingV2Count--; this.catalogueState.endUpdate();
this.migrationProgress.set(this.updatingV2Count);
} }
} }
async migrateV1ToV2(showMessage: boolean, entry: AnyEntry): Promise<void> { private async migrateV1ToV2(showMessage: boolean, entry: AnyEntry): Promise<void> {
const v1Path = entry.path; const v1Path = entry.path;
this._log(`Migrating ${entry.path} to V2`, showMessage ? LOG_LEVEL_NOTICE : LOG_LEVEL_INFO); this._log(`Migrating ${entry.path} to V2`, showMessage ? LOG_LEVEL_NOTICE : LOG_LEVEL_INFO);
if (entry.deleted) { if (entry.deleted) {
@@ -789,13 +672,11 @@ export class CustomisationSyncContext implements CustomisationSyncDialogView {
async updatePluginList(showMessage: boolean, updatedDocumentPath?: FilePathWithPrefix): Promise<void> { async updatePluginList(showMessage: boolean, updatedDocumentPath?: FilePathWithPrefix): Promise<void> {
if (!this.isThisModuleEnabled()) { if (!this.isThisModuleEnabled()) {
this.pluginScanProcessor.clearQueue(); this.pluginScanProcessor.clearQueue();
this.pluginList = []; this.catalogueState.clearForDisabledRefresh();
this.catalogue.set(this.pluginList);
return; return;
} }
try { try {
this.updatingV2Count++; this.catalogueState.beginUpdate();
this.migrationProgress.set(this.updatingV2Count);
const updatedDocumentId = updatedDocumentPath ? await this.path2id(updatedDocumentPath) : ""; const updatedDocumentId = updatedDocumentPath ? await this.path2id(updatedDocumentPath) : "";
const plugins = updatedDocumentPath const plugins = updatedDocumentPath
? this.localDatabase.findEntries(updatedDocumentId, updatedDocumentId + "\u{10ffff}", { ? this.localDatabase.findEntries(updatedDocumentId, updatedDocumentId + "\u{10ffff}", {
@@ -817,8 +698,7 @@ export class CustomisationSyncContext implements CustomisationSyncDialogView {
} }
} finally { } finally {
this.enumerationActive.set(false); this.enumerationActive.set(false);
this.updatingV2Count--; this.catalogueState.endUpdate();
this.migrationProgress.set(this.updatingV2Count);
} }
this.enumerationActive.set(false); this.enumerationActive.set(false);
// return entries; // return entries;
@@ -892,7 +772,7 @@ export class CustomisationSyncContext implements CustomisationSyncDialogView {
return false; return false;
} }
} }
async applyDataV2(data: PluginDataExDisplayV2, content?: string): Promise<boolean> { private async applyDataV2(data: PluginDataExDisplayV2, content?: string): Promise<boolean> {
const baseDir = this.configDir; const baseDir = this.configDir;
try { try {
if (content) { if (content) {
@@ -1010,10 +890,10 @@ export class CustomisationSyncContext implements CustomisationSyncDialogView {
if (data.documentPath) { if (data.documentPath) {
const delList = []; const delList = [];
if (this.useV2) { if (this.useV2) {
const deleteList = this.pluginList const deleteList = this.catalogueState
.filter((e) => e.documentPath == data.documentPath) .findPlugins(data.documentPath)
.filter((e) => e instanceof PluginDataExDisplayV2) .filter((entry) => entry instanceof PluginDataExDisplayV2)
.map((e) => e.files) .map((entry) => entry.files)
.flat(); .flat();
for (const e of deleteList) { for (const e of deleteList) {
delList.push(e.path); delList.push(e.path);
@@ -1039,7 +919,7 @@ export class CustomisationSyncContext implements CustomisationSyncDialogView {
return false; return false;
} }
} }
async _anyModuleParsedReplicationResultItem(docs: PouchDB.Core.ExistingDocument<EntryDoc>) { private async processVirtualDocument(docs: PouchDB.Core.ExistingDocument<EntryDoc>) {
if (!docs._id.startsWith(ICXHeader)) return false; if (!docs._id.startsWith(ICXHeader)) return false;
if (this.isThisModuleEnabled()) { if (this.isThisModuleEnabled()) {
await this.updatePluginList( await this.updatePluginList(
@@ -1050,14 +930,18 @@ export class CustomisationSyncContext implements CustomisationSyncDialogView {
if (this.isThisModuleEnabled() && this.settings.notifyPluginOrSettingUpdated) { if (this.isThisModuleEnabled() && this.settings.notifyPluginOrSettingUpdated) {
if (!this.dependencies.getUIControl()?.isOpen()) { if (!this.dependencies.getUIControl()?.isOpen()) {
scheduleTask(UPDATED_CONFIGURATION_NOTICE_KEY, 1000, () => { scheduleTask(UPDATED_CONFIGURATION_NOTICE_KEY, 1000, () => {
this.dependencies.showConfigurationNotice(() => this.showPluginSyncModal()); this.dependencies.showConfigurationNotice(() => this.dependencies.getUIControl()?.open());
}); });
} }
} }
return true; return true;
} }
async _everyRealizeSettingSyncMode(): Promise<boolean> { private async realiseSettingSyncMode(): Promise<boolean> {
this.periodicPluginSweepProcessor?.disable(); this.periodicPluginSweepProcessor?.disable();
// Compatibility question: this inherited callback checks the method
// reference rather than invoking it, then proceeds only while the host is
// suspended. Preserve both gates until their intended lifecycle semantics
// are verified and corrected under a separate behavioural test.
if (!this._isMainReady) return true; if (!this._isMainReady) return true;
if (!this._isMainSuspended()) return true; if (!this._isMainSuspended()) return true;
if (!this.isThisModuleEnabled()) return true; if (!this.isThisModuleEnabled()) return true;
@@ -1072,55 +956,7 @@ export class CustomisationSyncContext implements CustomisationSyncDialogView {
return true; return true;
} }
recentProcessedInternalFiles = [] as string[]; private async storeCustomisationFileV2(path: FilePath, term: string, force = false) {
async makeEntryFromFile(path: FilePath): Promise<false | PluginDataExFile> {
const stat = await this.storageAccess.statHidden(path);
let version: string | undefined;
let displayName: string | undefined;
if (!stat) {
return false;
}
const contentBin = await this.storageAccess.readHiddenFileBinary(path);
let content: string[];
try {
content = await arrayBufferToBase64(contentBin);
if (path.toLowerCase().endsWith("/manifest.json")) {
const v = readString(new Uint8Array(contentBin));
try {
const json: unknown = JSON.parse(v);
if (typeof json === "object" && json !== null) {
if ("version" in json) {
version = String(json.version);
}
if ("name" in json) {
displayName = String(json.name);
}
}
} catch (ex) {
this._log(
`Configuration sync data: ${path} looks like manifest, but could not read the version`,
LOG_LEVEL_INFO
);
this._log(ex, LOG_LEVEL_VERBOSE);
}
}
} catch (ex) {
this._log(`The file ${path} could not be encoded`);
this._log(ex, LOG_LEVEL_VERBOSE);
return false;
}
const mtime = stat.mtime;
return {
filename: path.substring(this.configDir.length + 1),
data: content,
mtime,
size: stat.size,
version,
displayName: displayName,
};
}
async storeCustomisationFileV2(path: FilePath, term: string, force = false) {
const vf = this.filenameWithUnifiedKey(path, term); const vf = this.filenameWithUnifiedKey(path, term);
return await serialized(`plugin-${vf}`, async () => { return await serialized(`plugin-${vf}`, async () => {
const prefixedFileName = vf; const prefixedFileName = vf;
@@ -1199,7 +1035,7 @@ export class CustomisationSyncContext implements CustomisationSyncDialogView {
} }
}); });
} }
async storeCustomizationFiles(path: FilePath, termOverRide?: string) { private async storeCustomizationFiles(path: FilePath, termOverRide?: string) {
const term = termOverRide || this.dependencies.getDeviceAndVaultName(); const term = termOverRide || this.dependencies.getDeviceAndVaultName();
if (term == "") { if (term == "") {
this._log($msg("We have to configure the device name"), LOG_LEVEL_NOTICE); this._log($msg("We have to configure the device name"), LOG_LEVEL_NOTICE);
@@ -1247,7 +1083,7 @@ export class CustomisationSyncContext implements CustomisationSyncDialogView {
fileTargets = ["manifest.json", "theme.css"].map((e) => `${parentPath}/${e}` as FilePath); fileTargets = ["manifest.json", "theme.css"].map((e) => `${parentPath}/${e}` as FilePath);
} }
for (const target of fileTargets) { for (const target of fileTargets) {
const data = await this.makeEntryFromFile(target); const data = await readCustomisationFile(this.dependencies, target, this.configDir);
if (data == false) { if (data == false) {
this._log(`Config: skipped (Possibly is not exist): ${target} `, LOG_LEVEL_VERBOSE); this._log(`Config: skipped (Possibly is not exist): ${target} `, LOG_LEVEL_VERBOSE);
continue; continue;
@@ -1343,11 +1179,11 @@ export class CustomisationSyncContext implements CustomisationSyncDialogView {
} }
}); });
} }
async _anyProcessOptionalFileEvent(path: FilePath): Promise<boolean> { private async processOptionalFileEvent(path: FilePath): Promise<boolean> {
return await this.watchVaultRawEventsAsync(path); return await this.watchVaultRawEventsAsync(path);
} }
async watchVaultRawEventsAsync(path: FilePath) { private async watchVaultRawEventsAsync(path: FilePath) {
if (!this._isMainReady()) return false; if (!this._isMainReady()) return false;
if (this._isMainSuspended()) return false; if (this._isMainSuspended()) return false;
if (!this.isThisModuleEnabled()) return false; if (!this.isThisModuleEnabled()) return false;
@@ -1360,12 +1196,11 @@ export class CustomisationSyncContext implements CustomisationSyncDialogView {
// this._log(`Customization file detected: ${path}`, LOG_LEVEL_VERBOSE); // this._log(`Customization file detected: ${path}`, LOG_LEVEL_VERBOSE);
const storageMTime = ~~(((stat && stat.mtime) || 0) / 1000); const storageMTime = ~~(((stat && stat.mtime) || 0) / 1000);
const key = `${path}-${storageMTime}`; const key = `${path}-${storageMTime}`;
if (this.recentProcessedInternalFiles.contains(key)) { if (!this.recentProcessedInternalFiles.admit(key)) {
// If recently processed, it may caused by self. // If recently processed, it may caused by self.
// return true to prevent pass the event to the next. // return true to prevent pass the event to the next.
return true; return true;
} }
this.recentProcessedInternalFiles = [key, ...this.recentProcessedInternalFiles].slice(0, 100);
// To prevent saving half-collected file sets. // To prevent saving half-collected file sets.
const keySchedule = this.filenameToUnifiedKey(path); const keySchedule = this.filenameToUnifiedKey(path);
scheduleTask(keySchedule, 100, async () => { scheduleTask(keySchedule, 100, async () => {
@@ -1480,7 +1315,7 @@ export class CustomisationSyncContext implements CustomisationSyncDialogView {
}); });
} }
async deleteConfigOnDatabase(prefixedFileName: FilePathWithPrefix, forceWrite = false) { private async deleteConfigOnDatabase(prefixedFileName: FilePathWithPrefix, forceWrite = false): Promise<boolean> {
// const id = await this.path2id(prefixedFileName); // const id = await this.path2id(prefixedFileName);
const mtime = new Date().getTime(); const mtime = new Date().getTime();
return await serialized("file-x-" + prefixedFileName, async () => { return await serialized("file-x-" + prefixedFileName, async () => {
@@ -1518,24 +1353,22 @@ export class CustomisationSyncContext implements CustomisationSyncDialogView {
}); });
} }
async scanInternalFiles(): Promise<FilePath[]> { private async scanInternalFiles(): Promise<FilePath[]> {
const filenames = (await this.getFiles(this.configDir, 2)) const filenames = (
await collectOptionalFileSyncFiles(this.dependencies, this.configDir, {
maxDepth: 2,
onError: (path, error) => {
this._log(`Could not traverse(CustomisationSync):${path}`, LOG_LEVEL_INFO);
this._log(error, LOG_LEVEL_VERBOSE);
},
})
)
.filter((e) => e.startsWith(".")) .filter((e) => e.startsWith("."))
.filter((e) => !e.startsWith(".trash")); .filter((e) => !e.startsWith(".trash"));
return filenames as FilePath[]; return filenames as FilePath[];
} }
_anyGetOptionalConflictCheckMethod(path: FilePathWithPrefix): Promise<boolean | "newer"> { private suspendExtraSync(): Promise<boolean> {
if (isPluginMetadata(path)) {
return Promise.resolve("newer");
}
if (isCustomisationSyncMetadata(path)) {
return Promise.resolve("newer");
}
return Promise.resolve(false);
}
_allSuspendExtraSync(): Promise<boolean> {
if (this.settings.usePluginSync || this.settings.autoSweepPlugins) { if (this.settings.usePluginSync || this.settings.autoSweepPlugins) {
this._log( this._log(
"Customisation sync have been temporarily disabled. Please enable them after the fetching, if you need them.", "Customisation sync have been temporarily disabled. Please enable them after the fetching, if you need them.",
@@ -1547,11 +1380,11 @@ export class CustomisationSyncContext implements CustomisationSyncDialogView {
return Promise.resolve(true); return Promise.resolve(true);
} }
async _allConfigureOptionalSyncFeature(mode: OptionalSyncFeatureMode) { private async enableOptionalFeature(mode: OptionalSyncFeatureMode): Promise<boolean> {
await this.configureHiddenFileSync(mode); await this.configureCustomisationSync(mode);
return true; return true;
} }
async configureHiddenFileSync(mode: OptionalSyncFeatureMode) { private async configureCustomisationSync(mode: OptionalSyncFeatureMode) {
if (mode == "DISABLE") { if (mode == "DISABLE") {
await this.dependencies.applySettings( await this.dependencies.applySettings(
{ {
@@ -1584,21 +1417,4 @@ export class CustomisationSyncContext implements CustomisationSyncDialogView {
await this.scanAllConfigFiles(true); await this.scanAllConfigFiles(true);
} }
} }
async getFiles(path: string, lastDepth: number) {
if (lastDepth == -1) return [];
let w: ListedFiles;
try {
w = await this.dependencies.listFiles(path);
} catch (ex) {
this._log(`Could not traverse(CustomisationSync):${path}`, LOG_LEVEL_INFO);
this._log(ex, LOG_LEVEL_VERBOSE);
return [];
}
let files = [...w.files];
for (const v of w.folders) {
files = files.concat(await this.getFiles(v, lastDepth - 1));
}
return files;
}
} }
@@ -0,0 +1,129 @@
import { describe, expect, it, vi } from "vitest";
vi.mock("@/deps.ts", () => ({
diff_match_patch: class DiffMatchPatch {},
normalizePath: vi.fn((path: string) => path),
parseYaml: vi.fn(),
Platform: {},
}));
vi.mock("@/common/types.ts", () => ({
ICXHeader: "ix:",
PERIODIC_PLUGIN_SWEEP: 60,
}));
vi.mock("@/common/utils.ts", () => ({
cancelTask: vi.fn(),
EVEN: Symbol("even"),
isCustomisationSyncMetadata: vi.fn(),
isPluginMetadata: vi.fn(),
scheduleTask: vi.fn(),
}));
vi.mock("@/common/PeriodicProcessor.ts", () => ({
PeriodicProcessor: class PeriodicProcessor {},
}));
vi.mock("@/common/translation", () => ({
$msg: vi.fn((message: string) => message),
}));
vi.mock("@/common/obsidianCommunityPlugins.ts", () => ({
getObsidianCommunityPluginManager: vi.fn(),
}));
import {
LOG_LEVEL_VERBOSE,
type FilePathWithPrefix,
type LoadedEntry,
} from "@vrtmrz/livesync-commonlib/compat/common/types";
import { digestHash } from "@vrtmrz/livesync-commonlib/compat/string_and_binary/hash";
import { createCustomisationSyncCodec } from "./customisationSyncCodec.ts";
import { CustomisationSyncCatalogueState } from "./customisationSyncCatalogueState.ts";
import { CustomisationSyncContext } from "./customisationSyncContext.ts";
import { createCustomisationSyncTestDependencies } from "./customisationSyncContext.unit.fixture.ts";
const path = "ix:device-a/PLUGIN_MAIN/example%manifest.json" as FilePathWithPrefix;
const confKey = "device-a/plugins/example";
const codec = createCustomisationSyncCodec({
digestHash,
parseYaml: () => undefined,
});
function loadedManifest(manifestSource: string, mtime: number): LoadedEntry {
const data = `${codec.dummyHead}${codec.dummyEnd}${btoa(manifestSource)}`;
return {
_id: "entry-id",
_rev: "1-a",
path,
type: "plain",
datatype: "plain",
data,
ctime: 10,
mtime,
size: data.length,
children: [],
eden: {},
} as unknown as LoadedEntry;
}
function createContext() {
const log = vi.fn();
const catalogueState = new CustomisationSyncCatalogueState();
const pluginManifests = catalogueState.manifestLookup;
const loadedManifest_mTime = catalogueState.loadedManifestMTime;
const setManifests = vi.spyOn(catalogueState.manifests, "set");
const setCatalogue = vi.spyOn(catalogueState.catalogue, "set");
const context = Object.create(CustomisationSyncContext.prototype) as CustomisationSyncContext;
Object.assign(context, {
dependencies: createCustomisationSyncTestDependencies({
log,
getLocalDatabase: () => ({ getDBEntry: async () => false }) as never,
}),
catalogueState,
});
return {
context,
loadedManifest_mTime,
log,
pluginManifests,
setCatalogue,
setManifests,
};
}
describe("compatibility: Customisation Sync V2 manifest state", () => {
it("keeps the first parsed manifest when a later file has a different mtime", async () => {
const { context, loadedManifest_mTime, pluginManifests, setManifests } = createContext();
await context.testing.createPluginDataExFileV2(
path,
loadedManifest(JSON.stringify({ id: "example", name: "First", version: "1.0.0" }), 20)
);
await context.testing.createPluginDataExFileV2(
path,
loadedManifest(JSON.stringify({ id: "example", name: "Second", version: "2.0.0" }), 30)
);
expect(pluginManifests.get(confKey)).toMatchObject({ name: "First", version: "1.0.0" });
expect(loadedManifest_mTime.get(confKey)).toBe(20);
expect(setManifests).toHaveBeenCalledOnce();
});
it("records a failed manifest mtime and does not retry the same revision", async () => {
const { context, loadedManifest_mTime, log, pluginManifests, setCatalogue } = createContext();
const invalid = loadedManifest("{invalid", 20);
await expect(context.testing.createPluginDataExFileV2(path, invalid)).resolves.toMatchObject({
filename: "plugins/example/manifest.json",
});
await context.testing.createPluginDataExFileV2(path, invalid);
expect(pluginManifests.has(confKey)).toBe(false);
expect(loadedManifest_mTime.get(confKey)).toBe(20);
expect(log).toHaveBeenCalledTimes(2);
expect(log).toHaveBeenNthCalledWith(
1,
`The file ${path} seems to manifest, but could not be decoded as JSON`,
LOG_LEVEL_VERBOSE,
undefined
);
expect(log).toHaveBeenNthCalledWith(2, expect.any(SyntaxError), LOG_LEVEL_VERBOSE, undefined);
expect(setCatalogue).toHaveBeenCalledOnce();
});
});
@@ -0,0 +1,67 @@
import type { PluginManifest } from "@/deps.ts";
import type { FilePathWithPrefix } from "@vrtmrz/livesync-commonlib/compat/common/types";
import { isDocContentSame } from "@vrtmrz/livesync-commonlib/compat/common/utils";
import { getCustomisationSyncCategoryFolder } from "./customisationSyncPaths.ts";
import type { IPluginDataExDisplay, LoadedEntryPluginDataExFile } from "./customisationSyncView.ts";
export class PluginDataExDisplayV2 {
documentPath: FilePathWithPrefix;
category: string;
term: string;
files: LoadedEntryPluginDataExFile[];
name: string;
confKey: string;
_displayName: string | undefined;
_version: string | undefined;
constructor(
data: IPluginDataExDisplay,
private readonly manifestLookup: ReadonlyMap<string, PluginManifest>
) {
this.documentPath = `${data.documentPath}` as FilePathWithPrefix;
this.category = `${data.category}`;
this.name = `${data.name}`;
this.term = `${data.term}`;
this.files = [...(data.files as LoadedEntryPluginDataExFile[])];
this.confKey = `${getCustomisationSyncCategoryFolder(this.category, this.term)}${this.name}`;
this.applyLoadedManifest();
}
async setFile(file: LoadedEntryPluginDataExFile): Promise<void> {
const old = this.files.find((entry) => entry.filename == file.filename);
if (old) {
if (old.mtime == file.mtime && (await isDocContentSame(old.data, file.data))) return;
this.files = this.files.filter((entry) => entry.filename != file.filename);
}
this.files.push(file);
if (file.filename == "manifest.json") {
this.applyLoadedManifest();
}
}
deleteFile(filename: string): void {
this.files = this.files.filter((entry) => entry.filename != filename);
}
applyLoadedManifest(): void {
const manifest = this.manifestLookup.get(this.confKey);
if (manifest) {
this._displayName = manifest.name;
if (this.category == "PLUGIN_MAIN" || this.category == "THEME") {
this._version = manifest.version;
}
}
}
get displayName(): string {
return this._displayName || this.name;
}
get version(): string | undefined {
return this._version;
}
get mtime(): number {
return ~~this.files.reduce((sum, file) => sum + file.mtime, 0) / this.files.length;
}
}
@@ -0,0 +1,58 @@
import { describe, expect, it } from "vitest";
import type { PluginManifest } from "@/deps.ts";
import type { FilePathWithPrefix } from "@vrtmrz/livesync-commonlib/compat/common/types";
import { PluginDataExDisplayV2 } from "./customisationSyncModel.ts";
import type { IPluginDataExDisplay, LoadedEntryPluginDataExFile } from "./customisationSyncView.ts";
function file(filename: string, mtime: number, data: string[]): LoadedEntryPluginDataExFile {
return { filename, mtime, data, size: data.join("").length } as LoadedEntryPluginDataExFile;
}
function display(files: LoadedEntryPluginDataExFile[] = []): IPluginDataExDisplay {
return {
documentPath: "ix:device-a/PLUGIN_MAIN/example%main.js" as FilePathWithPrefix,
category: "PLUGIN_MAIN",
name: "example",
term: "device-a",
files,
mtime: 0,
};
}
describe("PluginDataExDisplayV2", () => {
it("projects manifest identity and file modification time", () => {
const manifests = new Map([
["device-a/plugins/example", { name: "Example plug-in", version: "1.2.3" } as PluginManifest],
]);
const model = new PluginDataExDisplayV2(
display([file("main.js", 10, ["main"]), file("data.json", 20, ["data"])]),
manifests
);
expect(model.confKey).toBe("device-a/plugins/example");
expect(model.displayName).toBe("Example plug-in");
expect(model.version).toBe("1.2.3");
expect(model.mtime).toBe(15);
});
it("retains an unchanged file and replaces changed content", async () => {
const original = file("main.js", 10, ["same"]);
const model = new PluginDataExDisplayV2(display([original]), new Map());
await model.setFile(file("main.js", 10, ["same"]));
expect(model.files[0]).toBe(original);
const changed = file("main.js", 10, ["changed"]);
await model.setFile(changed);
expect(model.files).toEqual([changed]);
});
it("deletes only the named file", () => {
const retained = file("styles.css", 20, ["css"]);
const model = new PluginDataExDisplayV2(display([file("main.js", 10, ["main"]), retained]), new Map());
model.deleteFile("main.js");
expect(model.files).toEqual([retained]);
});
});
@@ -17,6 +17,23 @@ export type CustomisationSyncPathOptions = {
usePluginEtc: boolean; usePluginEtc: boolean;
}; };
export function getCustomisationSyncCategoryFolder(category: string, configDir: string = ""): string {
switch (category) {
case "CONFIG":
return `${configDir}/`;
case "THEME":
return `${configDir}/themes/`;
case "SNIPPET":
return `${configDir}/snippets/`;
case "PLUGIN_MAIN":
case "PLUGIN_DATA":
case "PLUGIN_ETC":
return `${configDir}/plugins/`;
default:
return "";
}
}
export function getCustomisationSyncFileCategory( export function getCustomisationSyncFileCategory(
filePath: string, filePath: string,
options: CustomisationSyncPathOptions options: CustomisationSyncPathOptions
@@ -5,6 +5,7 @@ import {
createCustomisationSyncDevicePrefix, createCustomisationSyncDevicePrefix,
createCustomisationSyncV1DocumentPath, createCustomisationSyncV1DocumentPath,
createCustomisationSyncV2DocumentPath, createCustomisationSyncV2DocumentPath,
getCustomisationSyncCategoryFolder,
getCustomisationSyncFileCategory, getCustomisationSyncFileCategory,
isCustomisationSyncTargetPath, isCustomisationSyncTargetPath,
getCustomisationSyncSettingKey, getCustomisationSyncSettingKey,
@@ -20,6 +21,18 @@ const currentOptions: CustomisationSyncPathOptions = {
}; };
describe("compatibility: Customisation Sync path operations", () => { describe("compatibility: Customisation Sync path operations", () => {
it.each([
["CONFIG", ".obsidian/"],
["THEME", ".obsidian/themes/"],
["SNIPPET", ".obsidian/snippets/"],
["PLUGIN_MAIN", ".obsidian/plugins/"],
["PLUGIN_DATA", ".obsidian/plugins/"],
["PLUGIN_ETC", ".obsidian/plugins/"],
["UNKNOWN", ""],
])("maps category %s to folder %s", (category, expected) => {
expect(getCustomisationSyncCategoryFolder(category, ".obsidian")).toBe(expected);
});
it.each([ it.each([
[".obsidian/app.json", "CONFIG"], [".obsidian/app.json", "CONFIG"],
[".obsidian/themes/Minimal/theme.css", "THEME"], [".obsidian/themes/Minimal/theme.css", "THEME"],
@@ -0,0 +1,208 @@
import type {
FilePath,
FilePathWithPrefix,
LoadedEntry,
LOG_LEVEL,
} from "@vrtmrz/livesync-commonlib/compat/common/types";
import { LOG_LEVEL_INFO, LOG_LEVEL_VERBOSE } from "@vrtmrz/livesync-commonlib/compat/common/types";
import type { StorageAccess } from "@vrtmrz/livesync-commonlib/compat/interfaces/StorageAccess";
import type { LiveSyncLocalDB } from "@vrtmrz/livesync-commonlib/compat/pouchdb/LiveSyncLocalDB";
import type { IPathService } from "@vrtmrz/livesync-commonlib/compat/services/base/IService";
import type { LogFunction } from "@vrtmrz/livesync-commonlib/compat/services/lib/logUtils";
import {
createSavingEntryFromLoadedEntry,
fireAndForget,
getDocData,
getDocDataAsArray,
isLoadedEntry,
} from "@vrtmrz/livesync-commonlib/compat/common/utils";
import { digestHash } from "@vrtmrz/livesync-commonlib/compat/string_and_binary/hash";
import { arrayBufferToBase64, readString } from "@vrtmrz/livesync-commonlib/compat/string_and_binary/convert";
import { base64ToString } from "octagonal-wheels/binary/base64";
import type { PluginDataEx, PluginDataExFile } from "./customisationSyncCodec.ts";
import { getCustomisationSyncCategoryFolder, parseCustomisationSyncV2DocumentPath } from "./customisationSyncPaths.ts";
import type { LoadedEntryPluginDataExFile, PluginDataExDisplay } from "./customisationSyncView.ts";
type CustomisationSyncLogDependency = {
log: LogFunction;
};
type CustomisationSyncStorageMethods<Method extends keyof StorageAccess> = {
storageAccess: Pick<StorageAccess, Method>;
};
type CustomisationSyncDatabaseMethods<Method extends keyof LiveSyncLocalDB> = {
getLocalDatabase(): Pick<LiveSyncLocalDB, Method>;
};
type CustomisationSyncPathMethods<Method extends keyof IPathService> = {
path: Pick<IPathService, Method>;
};
export type CustomisationSyncFileReaderDependencies = CustomisationSyncStorageMethods<
"readHiddenFileBinary" | "statHidden"
> &
CustomisationSyncLogDependency;
export type CustomisationSyncDisplayLoaderDependencies = CustomisationSyncDatabaseMethods<"getDBEntry" | "putDBEntry"> &
CustomisationSyncPathMethods<"getPath"> &
CustomisationSyncLogDependency;
export type CustomisationSyncV2EntryLoaderDependencies = CustomisationSyncDatabaseMethods<"getDBEntry"> &
CustomisationSyncLogDependency;
export type CustomisationSyncReadCodec = {
deserialize<T>(source: string[], defaultValue: T): T;
serialize(data: PluginDataEx): string;
};
export type DecodedCustomisationSyncV2File = {
confKey: string;
file: LoadedEntryPluginDataExFile;
isManifest: boolean;
};
function log(dependencies: CustomisationSyncLogDependency, message: unknown, level?: LOG_LEVEL, key?: string): void {
dependencies.log(message, level, key);
}
export async function readCustomisationFile(
dependencies: CustomisationSyncFileReaderDependencies,
path: FilePath,
configDir: string
): Promise<false | PluginDataExFile> {
const stat = await dependencies.storageAccess.statHidden(path);
let version: string | undefined;
let displayName: string | undefined;
if (!stat) {
return false;
}
const contentBin = await dependencies.storageAccess.readHiddenFileBinary(path);
let content: string[];
try {
content = await arrayBufferToBase64(contentBin);
if (path.toLowerCase().endsWith("/manifest.json")) {
const manifestSource = readString(new Uint8Array(contentBin));
try {
const manifest: unknown = JSON.parse(manifestSource);
if (typeof manifest === "object" && manifest !== null) {
if ("version" in manifest) {
version = String(manifest.version);
}
if ("name" in manifest) {
displayName = String(manifest.name);
}
}
} catch (error) {
log(
dependencies,
`Configuration sync data: ${path} looks like manifest, but could not read the version`,
LOG_LEVEL_INFO
);
log(dependencies, error, LOG_LEVEL_VERBOSE);
}
}
} catch (error) {
log(dependencies, `The file ${path} could not be encoded`);
log(dependencies, error, LOG_LEVEL_VERBOSE);
return false;
}
return {
// Compatibility: target validation belongs to the caller. The legacy
// reader derives this name positionally without checking the prefix.
filename: path.substring(configDir.length + 1),
data: content,
mtime: stat.mtime,
size: stat.size,
version,
displayName,
};
}
export async function loadCustomisationDisplayData(
dependencies: CustomisationSyncDisplayLoaderDependencies,
path: FilePathWithPrefix,
codec: CustomisationSyncReadCodec
): Promise<PluginDataExDisplay | false> {
const loaded = await dependencies.getLocalDatabase().getDBEntry(path, undefined, false, false);
if (!loaded) {
return false;
}
const data = codec.deserialize(getDocDataAsArray(loaded.data), {}) as PluginDataEx;
const displayFiles: PluginDataExFile[] = [];
let missingHash = false;
for (const file of data.files) {
const displayFile = { ...file, data: [] as string[] };
if (!file.hash) {
// Compatibility question: the inherited implementation clears the
// display copy before calculating this temporary hash, so callers
// see digestHash([]) until the asynchronously repaired document is
// loaded again. The serialiser still writes the real content hash.
const temporaryHashSource = getDocDataAsArray(displayFile.data);
file.hash = digestHash(temporaryHashSource);
missingHash = true;
}
displayFile.data = [file.hash];
displayFiles.push(displayFile);
}
if (missingHash) {
log(dependencies, `Digest created for ${path} to improve checking`, LOG_LEVEL_VERBOSE);
loaded.data = codec.serialize(data);
// Compatibility: catalogue loading does not wait for the repair write.
fireAndForget(() => dependencies.getLocalDatabase().putDBEntry(createSavingEntryFromLoadedEntry(loaded)));
}
return {
...data,
documentPath: dependencies.path.getPath(loaded),
files: displayFiles,
} satisfies PluginDataExDisplay;
}
export async function loadCustomisationV2Entry(
dependencies: CustomisationSyncV2EntryLoaderDependencies,
path: FilePathWithPrefix
): Promise<LoadedEntry | false> {
const loaded = await dependencies.getLocalDatabase().getDBEntry(path);
if (!loaded) {
log(dependencies, `The file ${path} is not found`, LOG_LEVEL_VERBOSE);
return false;
}
if (!isLoadedEntry(loaded)) {
log(dependencies, `The file ${path} is not a note`, LOG_LEVEL_VERBOSE);
return false;
}
return loaded;
}
export function decodeCustomisationSyncV2File(
path: FilePathWithPrefix,
loaded: LoadedEntry,
dummyEnd: string
): DecodedCustomisationSyncV2File {
const { category, key, filename, device } = parseCustomisationSyncV2DocumentPath(path);
const categoryFolder = getCustomisationSyncCategoryFolder(category, device);
const confKey = `${categoryFolder}${key}`;
const relativeFilename =
`${getCustomisationSyncCategoryFolder(category, "")}${category == "CONFIG" || category == "SNIPPET" ? "" : key + "/"}${filename}`.substring(
1
);
const source = getDocData(loaded.data);
const dataStart = source.indexOf(dummyEnd);
// Compatibility question: a missing marker is not rejected. substring()
// starts at dummyEnd.length - 1, preserving the old best-effort decode.
const encodedData = source.substring(dataStart + dummyEnd.length);
const file: LoadedEntryPluginDataExFile = {
...loaded,
hash: "",
data: [base64ToString(encodedData)],
filename: relativeFilename,
displayName: filename,
};
return {
confKey,
file,
isManifest: filename == "manifest.json",
};
}
@@ -0,0 +1,222 @@
import { describe, expect, it, vi } from "vitest";
import {
LOG_LEVEL_INFO,
LOG_LEVEL_VERBOSE,
type FilePath,
type FilePathWithPrefix,
type LoadedEntry,
type UXStat,
} from "@vrtmrz/livesync-commonlib/compat/common/types";
import { digestHash } from "@vrtmrz/livesync-commonlib/compat/string_and_binary/hash";
import { createCustomisationSyncCodec, type PluginDataEx } from "./customisationSyncCodec.ts";
import {
decodeCustomisationSyncV2File,
loadCustomisationDisplayData,
loadCustomisationV2Entry,
readCustomisationFile,
} from "./customisationSyncReadOperations.ts";
const configDir = ".obsidian";
const filePath = ".obsidian/plugins/example/manifest.json" as FilePath;
const documentPath = "ix:device-a/PLUGIN_MAIN/example.md" as FilePathWithPrefix;
const stat = { ctime: 10, mtime: 20, size: 42, type: "file" } as UXStat;
const codec = createCustomisationSyncCodec({
digestHash,
parseYaml: () => undefined,
});
function createDependencies() {
const localDatabase = {
getDBEntry: vi.fn(),
putDBEntry: vi.fn(async (_entry: unknown) => ({ ok: true, id: "id", rev: "1-a" })),
};
const storageAccess = {
statHidden: vi.fn(async () => stat as UXStat | null),
readHiddenFileBinary: vi.fn(),
};
const log = vi.fn();
const dependencies = {
getLocalDatabase: () => localDatabase as never,
storageAccess: storageAccess as never,
path: {
getPath: vi.fn((entry: { path: FilePathWithPrefix }) => entry.path),
} as never,
log,
};
return { dependencies, localDatabase, log, storageAccess };
}
function loadedEntry(path: FilePathWithPrefix, data: string): LoadedEntry {
return {
_id: "entry-id",
_rev: "1-a",
path,
type: "plain",
datatype: "plain",
data,
ctime: 10,
mtime: 20,
size: data.length,
children: [],
eden: {},
} as unknown as LoadedEntry;
}
function pluginData(hash?: string): PluginDataEx {
return {
category: "PLUGIN_MAIN",
name: "example",
term: "device-a",
mtime: 20,
files: [
{
filename: "plugins/example/main.js",
data: ["payload"],
mtime: 20,
size: 7,
hash,
},
],
};
}
describe("Customisation Sync read operations", () => {
it("does not read content when a local file is missing", async () => {
const { dependencies, storageAccess } = createDependencies();
storageAccess.statHidden.mockResolvedValue(null);
await expect(readCustomisationFile(dependencies, filePath, configDir)).resolves.toBe(false);
expect(storageAccess.readHiddenFileBinary).not.toHaveBeenCalled();
});
it("propagates a storage read failure without converting it to an encoding failure", async () => {
const { dependencies, log, storageAccess } = createDependencies();
const error = new Error("read failed");
storageAccess.readHiddenFileBinary.mockRejectedValue(error);
await expect(readCustomisationFile(dependencies, filePath, configDir)).rejects.toBe(error);
expect(log).not.toHaveBeenCalled();
});
it("encodes a manifest and extracts its display metadata", async () => {
const { dependencies, storageAccess } = createDependencies();
const source = JSON.stringify({ name: "Example plug-in", version: "1.2.3" });
storageAccess.readHiddenFileBinary.mockResolvedValue(new TextEncoder().encode(source).buffer);
await expect(readCustomisationFile(dependencies, filePath, configDir)).resolves.toEqual({
filename: "plugins/example/manifest.json",
data: [btoa(source)],
mtime: 20,
size: 42,
version: "1.2.3",
displayName: "Example plug-in",
});
});
it("keeps an unreadable manifest as file data and reports only the metadata failure", async () => {
const { dependencies, log, storageAccess } = createDependencies();
const errorSource = "{invalid";
storageAccess.readHiddenFileBinary.mockResolvedValue(new TextEncoder().encode(errorSource).buffer);
await expect(readCustomisationFile(dependencies, filePath, configDir)).resolves.toMatchObject({
filename: "plugins/example/manifest.json",
data: [btoa(errorSource)],
version: undefined,
displayName: undefined,
});
expect(log).toHaveBeenNthCalledWith(
1,
`Configuration sync data: ${filePath} looks like manifest, but could not read the version`,
LOG_LEVEL_INFO,
undefined
);
expect(log).toHaveBeenNthCalledWith(2, expect.any(SyntaxError), LOG_LEVEL_VERBOSE, undefined);
});
it("loads V1 display data without retaining file content", async () => {
const { dependencies, localDatabase } = createDependencies();
const data = pluginData("known-hash");
localDatabase.getDBEntry.mockResolvedValue(loadedEntry(documentPath, JSON.stringify(data)));
await expect(loadCustomisationDisplayData(dependencies, documentPath, codec)).resolves.toEqual({
...data,
documentPath,
files: [{ ...data.files[0], data: ["known-hash"] }],
});
expect(localDatabase.getDBEntry).toHaveBeenCalledWith(documentPath, undefined, false, false);
expect(localDatabase.putDBEntry).not.toHaveBeenCalled();
});
it("preserves the inherited transient empty-data hash while repairing a V1 document", async () => {
const { dependencies, localDatabase, log } = createDependencies();
const data = pluginData();
localDatabase.getDBEntry.mockResolvedValue(loadedEntry(documentPath, JSON.stringify(data)));
const result = await loadCustomisationDisplayData(dependencies, documentPath, codec);
expect(result).toMatchObject({ files: [{ data: [digestHash([])] }] });
expect(localDatabase.putDBEntry).toHaveBeenCalledOnce();
const saving = localDatabase.putDBEntry.mock.calls[0][0] as { data: Blob };
expect(await saving.data.text()).toContain(digestHash(["payload"]));
expect(log).toHaveBeenCalledWith(
`Digest created for ${documentPath} to improve checking`,
LOG_LEVEL_VERBOSE,
undefined
);
});
it("returns false when a V1 document is absent", async () => {
const { dependencies, localDatabase } = createDependencies();
localDatabase.getDBEntry.mockResolvedValue(false);
await expect(loadCustomisationDisplayData(dependencies, documentPath, codec)).resolves.toBe(false);
});
it("distinguishes an absent V2 entry from a non-note database entry", async () => {
const { dependencies, localDatabase, log } = createDependencies();
const path = "ix:device-a/PLUGIN_MAIN/example%main.js" as FilePathWithPrefix;
localDatabase.getDBEntry.mockResolvedValueOnce(false);
await expect(loadCustomisationV2Entry(dependencies, path)).resolves.toBe(false);
expect(log).toHaveBeenLastCalledWith(`The file ${path} is not found`, LOG_LEVEL_VERBOSE, undefined);
localDatabase.getDBEntry.mockResolvedValueOnce({ path, type: "leaf" });
await expect(loadCustomisationV2Entry(dependencies, path)).resolves.toBe(false);
expect(log).toHaveBeenLastCalledWith(`The file ${path} is not a note`, LOG_LEVEL_VERBOSE, undefined);
});
it("returns a loaded V2 entry after the exact single-argument database lookup", async () => {
const { dependencies, localDatabase } = createDependencies();
const path = "ix:device-a/PLUGIN_MAIN/example%main.js" as FilePathWithPrefix;
const loaded = loadedEntry(path, "data");
localDatabase.getDBEntry.mockResolvedValue(loaded);
await expect(loadCustomisationV2Entry(dependencies, path)).resolves.toBe(loaded);
expect(localDatabase.getDBEntry).toHaveBeenCalledWith(path);
});
it("decodes a V2 payload into its relative Customisation Sync filename", () => {
const path = "ix:device-a/PLUGIN_MAIN/example%main.js" as FilePathWithPrefix;
const loaded = loadedEntry(path, `${codec.dummyHead}${codec.dummyEnd}${btoa("console.log('example');")}`);
expect(decodeCustomisationSyncV2File(path, loaded, codec.dummyEnd)).toEqual({
confKey: "device-a/plugins/example",
isManifest: false,
file: {
...loaded,
filename: "plugins/example/main.js",
displayName: "main.js",
hash: "",
data: ["console.log('example');"],
},
});
});
it("preserves the inherited best-effort offset when a V2 marker is missing", () => {
const path = "ix:device-a/CONFIG/app.json%app.json" as FilePathWithPrefix;
const loaded = loadedEntry(path, `00${btoa("hello")}`);
expect(decodeCustomisationSyncV2File(path, loaded, "END").file.data).toEqual(["hello"]);
});
});
@@ -0,0 +1,17 @@
const MAX_RECENT_CUSTOMISATION_EVENTS = 100;
/** Keeps the bounded newest-first raw-event keys used by Customisation Sync. */
export class CustomisationSyncRecentEventDeduplicator {
private keys: string[] = [];
/**
* Records a key when it is new and returns whether the caller should act.
* Native Array#includes is intentional: the old `.contains` extension is
* not available in every runtime where the feature is exercised.
*/
admit(key: string): boolean {
if (this.keys.includes(key)) return false;
this.keys = [key, ...this.keys].slice(0, MAX_RECENT_CUSTOMISATION_EVENTS);
return true;
}
}
@@ -0,0 +1,24 @@
import { describe, expect, it } from "vitest";
import { CustomisationSyncRecentEventDeduplicator } from "./customisationSyncRecentEventDeduplicator.ts";
describe("Customisation Sync recent raw-event keys", () => {
it("admits a key once and keeps newer keys first", () => {
const history = new CustomisationSyncRecentEventDeduplicator();
expect(history.admit("old")).toBe(true);
expect(history.admit("new")).toBe(true);
expect(history.admit("old")).toBe(false);
});
it("evicts the oldest key when the newest-first history exceeds 100 entries", () => {
const history = new CustomisationSyncRecentEventDeduplicator();
for (let index = 0; index < 101; index++) {
expect(history.admit(`key-${index}`)).toBe(true);
}
expect(history.admit("key-0")).toBe(true);
expect(history.admit("key-100")).toBe(false);
});
});
@@ -1,15 +1,22 @@
import type { PluginManifest } from "@/deps.ts"; import type { PluginManifest } from "@/deps.ts";
import type { import type {
EntryDoc,
FilePathWithPrefix, FilePathWithPrefix,
FilePath,
LoadedEntry, LoadedEntry,
PluginSyncSettingEntry, PluginSyncSettingEntry,
SYNC_MODE, SYNC_MODE,
} from "@vrtmrz/livesync-commonlib/compat/common/types"; } from "@vrtmrz/livesync-commonlib/compat/common/types";
import type PouchDB from "pouchdb-core";
import type { Readable } from "svelte/store"; import type { Readable } from "svelte/store";
import type { PluginDataExFile } from "./customisationSyncCodec.ts"; import type { PluginDataExFile } from "./customisationSyncCodec.ts";
import type { CustomisationSyncFileCategory } from "./customisationSyncPaths.ts";
import type { OptionalSyncFeatureMode } from "@/features/optionalSyncFeatures.ts";
import type { PluginDataExDisplayV2 } from "./customisationSyncModel.ts";
export type LoadedEntryPluginDataExFile = LoadedEntry & PluginDataExFile; export type LoadedEntryPluginDataExFile = LoadedEntry & PluginDataExFile;
export type { CustomisationSyncFileCategory } from "./customisationSyncPaths.ts";
export interface IPluginDataExDisplay { export interface IPluginDataExDisplay {
documentPath: FilePathWithPrefix; documentPath: FilePathWithPrefix;
@@ -33,6 +40,48 @@ export type PluginDataExDisplay = {
mtime: number; mtime: number;
}; };
/**
* Semantic callbacks registered by the optional-file composition feature.
*
* The context owns the implementations, while the optional-file composition
* adapts these operations to Commonlib's aggregation contracts. Consumers
* receive only callable operations, not the context or its private state.
*/
export interface CustomisationSyncServiceHandlers {
readonly processOptionalFileEvent: (path: FilePath) => Promise<boolean>;
readonly processVirtualDocument: (docs: PouchDB.Core.ExistingDocument<EntryDoc>) => Promise<boolean>;
readonly onRealiseSetting: () => Promise<boolean>;
readonly onResuming: () => Promise<boolean>;
readonly onBeforeReplicate: (showMessage: boolean) => Promise<boolean>;
readonly onDatabaseInitialised: (showNotice: boolean) => Promise<boolean>;
readonly suspendExtraSync: () => Promise<boolean>;
readonly enableOptionalFeature: (mode: OptionalSyncFeatureMode) => Promise<boolean>;
}
/**
* Explicit internal operations used by maintained real-Obsidian contract
* tests. This is deliberately narrower than the concrete context and does
* not expose reactive stores, queues, or host dependencies.
*/
export interface CustomisationSyncTestingView {
readonly configDir: string;
scanInternalFiles(): Promise<FilePath[]>;
scanAllConfigFiles(showMessage: boolean): Promise<void>;
getFileCategory(filePath: string): CustomisationSyncFileCategory;
isTargetPath(filePath: string): boolean;
filenameToUnifiedKey(path: string, termOverride?: string): FilePathWithPrefix;
filenameWithUnifiedKey(path: string, termOverride?: string): FilePathWithPrefix;
unifiedKeyPrefixOfTerminal(termOverride?: string): string;
storeCustomizationFiles(path: FilePath, termOverride?: string): Promise<unknown>;
deleteConfigOnDatabase(prefixedFileName: FilePathWithPrefix, forceWrite?: boolean): Promise<boolean>;
createPluginDataFromV2(unifiedPathV2: FilePathWithPrefix): PluginDataExDisplayV2 | undefined;
createPluginDataExFileV2(
unifiedPathV2: FilePathWithPrefix,
loaded?: LoadedEntry
): Promise<false | LoadedEntryPluginDataExFile>;
applyDataV2(data: PluginDataExDisplayV2, content?: string): Promise<boolean>;
}
/** Stable catalogue and operation surface consumed by the Obsidian dialogue. */ /** Stable catalogue and operation surface consumed by the Obsidian dialogue. */
export interface CustomisationSyncDialogView { export interface CustomisationSyncDialogView {
readonly catalogue: Readable<IPluginDataExDisplay[]>; readonly catalogue: Readable<IPluginDataExDisplay[]>;
@@ -0,0 +1,260 @@
import {
LOG_LEVEL_DEBUG,
LOG_LEVEL_VERBOSE,
type FilePath,
type FilePathWithPrefix,
type LoadedEntry,
type LOG_LEVEL,
type MetaEntry,
type UXFileInfo,
} from "@vrtmrz/livesync-commonlib/compat/common/types";
import type { StorageAccess } from "@vrtmrz/livesync-commonlib/compat/interfaces/StorageAccess";
import type { LogFunction } from "@vrtmrz/livesync-commonlib/compat/services/lib/logUtils";
import { addPrefix } from "@vrtmrz/livesync-commonlib/compat/string_and_binary/path";
import { serialized } from "octagonal-wheels/concurrency/lock";
import { Semaphore } from "octagonal-wheels/concurrency/semaphore";
import { ICHeader } from "@/common/types.ts";
import type { HiddenFileSyncConflictResolution } from "./hiddenFileSyncConflictResolution.ts";
import type { HiddenFileSyncDatabaseExtractionOperations } from "./hiddenFileSyncDatabaseExtractionOperations.ts";
import type { HiddenFileSyncDatabaseWriteOperations } from "./hiddenFileSyncDatabaseWriteOperations.ts";
import type { HiddenFileSyncProcessedState } from "./hiddenFileSyncProcessedState.ts";
import { getHiddenFileSyncComparisonMTime } from "./hiddenFileSyncState.ts";
import { compareMTime, TARGET_IS_NEW } from "@/common/utils.ts";
type HiddenFileSyncStorageChangeAccess = Pick<StorageAccess, "statHidden">;
export type HiddenFileSyncChangeProcessorDependencies = {
storageAccess: HiddenFileSyncStorageChangeAccess;
readFileWithInfo(path: FilePath): Promise<UXFileInfo>;
loadDatabaseMetadata(path: FilePathWithPrefix): Promise<MetaEntry | LoadedEntry | false>;
databaseWriteOperations: Pick<HiddenFileSyncDatabaseWriteOperations, "store" | "delete">;
databaseExtractionOperations: Pick<HiddenFileSyncDatabaseExtractionOperations, "extract">;
processedState: Pick<
HiddenFileSyncProcessedState,
| "fileToStatKey"
| "getLastProcessedFileKey"
| "getLastProcessedFileMTime"
| "updateLastProcessedFile"
| "updateLastProcessed"
>;
conflictResolution: Pick<HiddenFileSyncConflictResolution, "queue">;
log: LogFunction;
publishActivity(eventCount: number, processingCount: number): void;
};
export type HiddenFileSyncDatabaseChangeOptions = Readonly<{
preventDoubleProcess?: boolean;
onlyNew?: boolean;
metaEntry?: MetaEntry | false;
includeDeletion?: boolean;
}>;
export type HiddenFileSyncChangeProcessor = {
processStorageChange(
path: FilePath,
onlyNew?: boolean,
forceWrite?: boolean,
includeDeleted?: boolean
): Promise<boolean | undefined>;
processDatabaseChange(
path: FilePath,
headerLine: string,
options?: HiddenFileSyncDatabaseChangeOptions
): Promise<boolean>;
dispose(): void;
};
class HiddenFileSyncChangeProcessorOwner implements HiddenFileSyncChangeProcessor {
private readonly semaphore = Semaphore(10);
private eventCount = 0;
private processingCount = 0;
private disposed = false;
constructor(private readonly dependencies: HiddenFileSyncChangeProcessorDependencies) {}
async processStorageChange(
path: FilePath,
onlyNew = false,
forceWrite = false,
includeDeleted = true
): Promise<boolean | undefined> {
try {
return await this.serialiseForEvent(path, async () => {
let stat = await this.dependencies.storageAccess.statHidden(path);
// Sometimes a folder is delivered as a file event.
if (stat != null && stat.type != "file") {
return false;
}
const key = await this.dependencies.processedState.fileToStatKey(path, stat);
// A raw event can occur while the file is being read. Scans
// still enumerate every path, but event admission skips this
// exact already-settled key.
const lastKey = this.dependencies.processedState.getLastProcessedFileKey(path);
if (lastKey == key) {
this.log(`${path} Already processed.`, LOG_LEVEL_DEBUG);
return true;
}
// Read the stat and content as one operation. The stat is
// deliberately compared again below: a file can change while
// the first stat is in flight.
const fileInfo = await this.dependencies.readFileWithInfo(path);
const cacheMTime = getHiddenFileSyncComparisonMTime(fileInfo.stat);
const statMtime = getHiddenFileSyncComparisonMTime(stat);
if (cacheMTime != statMtime) {
this.log(`Hidden file:${path} is changed.`, LOG_LEVEL_VERBOSE);
stat = fileInfo.stat;
}
// Compatibility: the storage marker advances before the
// database operation. A later write failure can therefore
// leave this event marked as processed until a scan or state
// change causes it to be reconsidered.
this.dependencies.processedState.updateLastProcessedFile(path, stat!);
const lastIsNotFound = !lastKey || lastKey.endsWith("-0-0");
const nowIsNotFound = fileInfo.deleted;
const type = lastIsNotFound && nowIsNotFound ? "invalid" : nowIsNotFound ? "delete" : "modified";
if (type == "invalid") {
// Maybe the folder was deleted.
return false;
}
const storageMTimeActual = getHiddenFileSyncComparisonMTime(stat);
const storageMTime =
storageMTimeActual == 0
? this.dependencies.processedState.getLastProcessedFileMTime(path)
: storageMTimeActual;
if (onlyNew) {
const prefixedFileName = addPrefix(path, ICHeader);
const fileOnDatabase = await this.dependencies.loadDatabaseMetadata(prefixedFileName);
const databaseMTime = getHiddenFileSyncComparisonMTime(fileOnDatabase, includeDeleted);
const difference = compareMTime(storageMTime, databaseMTime);
if (difference != TARGET_IS_NEW) {
this.log(`Hidden file:${path} is not new.`, LOG_LEVEL_VERBOSE);
// OnlyNew does not handle a deletion. Preserve the
// inherited partial settlement when both values exist.
if (fileOnDatabase && stat) {
this.dependencies.processedState.updateLastProcessed(path, fileOnDatabase, stat);
}
return true;
}
}
if (type == "delete") {
this.log(`Deletion detected: ${path}`);
return await this.dependencies.databaseWriteOperations.delete(path, forceWrite);
}
if (type == "modified") {
this.log(`Modification detected:${path}`, LOG_LEVEL_VERBOSE);
const result = await this.dependencies.databaseWriteOperations.store(fileInfo, forceWrite);
const resultText = result === undefined ? "Nothing changed" : result ? "Updated" : "Failed";
this.log(`${resultText}: ${path} ${resultText}`, LOG_LEVEL_VERBOSE);
return result;
}
return false;
});
} catch (error) {
this.log(`Failed to process hidden file:${path}`);
this.log(error, LOG_LEVEL_VERBOSE);
}
// Could not be processed, but it was this operation's event. Return
// true to prevent a later handler from claiming it.
return true;
}
async processDatabaseChange(
path: FilePath,
headerLine: string,
options: HiddenFileSyncDatabaseChangeOptions = {}
): Promise<boolean> {
const {
preventDoubleProcess = false,
onlyNew = false,
metaEntry = false,
includeDeletion = true,
} = options;
return await this.serialiseForEvent(path, async () => {
try {
const prefixedPath = addPrefix(path, ICHeader);
const docMeta = metaEntry
? metaEntry
: await this.dependencies.loadDatabaseMetadata(prefixedPath);
if (docMeta === false) {
this.log(`${headerLine}: Failed to read detail of ${path}`);
throw new Error(`Failed to read detail ${path}`);
}
if (docMeta._conflicts && docMeta._conflicts.length > 0) {
this.dependencies.conflictResolution.queue(path);
this.log(`${headerLine} Hidden file conflicted, enqueued to resolve`);
return true;
}
const extracted = await this.dependencies.databaseExtractionOperations.extract(path, {
metaEntry: docMeta,
preventDoubleProcess,
onlyNew,
includeDeletion,
});
if (extracted) {
this.log(`${headerLine} Hidden file processed`);
}
} catch (error) {
this.log(`${headerLine} Failed to process hidden file`);
this.log(error, LOG_LEVEL_VERBOSE);
}
// Compatibility: recognition consumes the database event even when
// extraction returned false or threw. A later scan or state change,
// rather than handler fall-through, is responsible for retrying it.
return true;
});
}
dispose(): void {
if (this.disposed) return;
this.disposed = true;
this.eventCount = 0;
this.processingCount = 0;
this.publishActivity();
}
private async serialiseForEvent<Result>(file: FilePath, operation: () => Promise<Result>): Promise<Result> {
this.eventCount++;
this.publishActivity();
const release = await this.semaphore.acquire();
try {
return await serialized(`hidden-file-event:${file}`, async () => {
this.processingCount++;
this.publishActivity();
try {
return await operation();
} finally {
this.processingCount = Math.max(0, this.processingCount - 1);
this.publishActivity();
}
});
} finally {
release();
this.eventCount = Math.max(0, this.eventCount - 1);
this.publishActivity();
}
}
private publishActivity(): void {
this.dependencies.publishActivity(
this.disposed ? 0 : this.eventCount,
this.disposed ? 0 : this.processingCount
);
}
private log(message: unknown, level?: LOG_LEVEL, key?: string): void {
this.dependencies.log(message, level, key);
}
}
export function createHiddenFileSyncChangeProcessor(
dependencies: HiddenFileSyncChangeProcessorDependencies
): HiddenFileSyncChangeProcessor {
return new HiddenFileSyncChangeProcessorOwner(dependencies);
}
@@ -0,0 +1,159 @@
import { describe, expect, it, vi } from "vitest";
import type { FilePath, MetaEntry, UXFileInfo, UXStat } from "@vrtmrz/livesync-commonlib/compat/common/types";
vi.mock("@/deps.ts", () => ({}));
import {
createHiddenFileSyncChangeProcessor,
type HiddenFileSyncChangeProcessorDependencies,
} from "./hiddenFileSyncChangeProcessor.ts";
const path = ".obsidian/app.json" as FilePath;
const stat = { ctime: 1, mtime: 2, size: 3, type: "file" } as UXStat;
function fileInfo(): UXFileInfo {
return {
path,
name: "app.json",
isInternal: true,
deleted: false,
body: new Blob(["{}"]),
stat,
} as UXFileInfo;
}
function metadata(): MetaEntry {
return {
_id: "i:app",
_rev: "2-current",
path: `i:${path}`,
type: "plain",
datatype: "plain",
ctime: 1,
mtime: 2,
size: 3,
children: [],
eden: {},
deleted: false,
} as unknown as MetaEntry;
}
function createDependencies(
overrides: Partial<HiddenFileSyncChangeProcessorDependencies> = {}
): HiddenFileSyncChangeProcessorDependencies {
const state = {
fileToStatKey: vi.fn(async () => "2-3"),
getLastProcessedFileKey: vi.fn(() => undefined),
getLastProcessedFileMTime: vi.fn(() => 0),
databaseStateKey: vi.fn(() => "2-3-2-current--1"),
getLastProcessedDatabaseKey: vi.fn(() => undefined),
updateLastProcessedFile: vi.fn(),
updateLastProcessedDatabase: vi.fn(),
updateLastProcessed: vi.fn(),
};
return {
storageAccess: {
statHidden: vi.fn(async () => stat),
},
readFileWithInfo: vi.fn(async () => fileInfo()),
loadDatabaseMetadata: vi.fn(async () => metadata()),
databaseWriteOperations: {
store: vi.fn(async () => true),
delete: vi.fn(async () => true),
},
databaseExtractionOperations: {
extract: vi.fn(async () => true),
},
processedState: state,
conflictResolution: { queue: vi.fn() },
log: vi.fn(),
publishActivity: vi.fn(),
...overrides,
} as HiddenFileSyncChangeProcessorDependencies;
}
describe("HiddenFileSyncChangeProcessor activity and serialisation", () => {
it("publishes admission, processing, and release transitions", async () => {
const dependencies = createDependencies();
const processor = createHiddenFileSyncChangeProcessor(dependencies);
await expect(processor.processStorageChange(path)).resolves.toBe(true);
const publishActivity = vi.mocked(dependencies.publishActivity);
expect(publishActivity.mock.calls).toEqual([
[1, 0],
[1, 1],
[1, 0],
[0, 0],
]);
processor.dispose();
});
it("serialises same-path storage changes while allowing each event to settle", async () => {
let active = 0;
let maximumActive = 0;
let releaseFirst!: () => void;
const firstStarted = new Promise<void>((resolve) => {
const write = resolve;
releaseFirst = write;
});
const dependencies = createDependencies({
databaseWriteOperations: {
store: vi.fn(async () => {
active++;
maximumActive = Math.max(maximumActive, active);
if (active == 1) await firstStarted;
active--;
return true;
}),
delete: vi.fn(async () => true),
},
});
const processor = createHiddenFileSyncChangeProcessor(dependencies);
const first = processor.processStorageChange(path);
await vi.waitFor(() => expect(dependencies.databaseWriteOperations.store).toHaveBeenCalledOnce());
const second = processor.processStorageChange(path);
await new Promise<void>((resolve) => setTimeout(resolve, 0));
expect(dependencies.databaseWriteOperations.store).toHaveBeenCalledOnce();
releaseFirst();
await expect(first).resolves.toBe(true);
await expect(second).resolves.toBe(true);
expect(maximumActive).toBe(1);
expect(dependencies.databaseWriteOperations.store).toHaveBeenCalledTimes(2);
processor.dispose();
});
});
describe("HiddenFileSyncChangeProcessor compatibility settlement", () => {
it("consumes database events when metadata loading fails", async () => {
const error = new Error("metadata unavailable");
const dependencies = createDependencies({
loadDatabaseMetadata: vi.fn(async () => {
throw error;
}),
});
const processor = createHiddenFileSyncChangeProcessor(dependencies);
await expect(processor.processDatabaseChange(path, "[Replication]")).resolves.toBe(true);
expect(dependencies.log).toHaveBeenCalledWith("[Replication] Failed to process hidden file", undefined, undefined);
expect(dependencies.log).toHaveBeenCalledWith(error, expect.any(Number), undefined);
processor.dispose();
});
it("advances the storage marker before a failed database write", async () => {
const dependencies = createDependencies({
databaseWriteOperations: {
store: vi.fn(async () => false),
delete: vi.fn(async () => true),
},
});
const processor = createHiddenFileSyncChangeProcessor(dependencies);
await expect(processor.processStorageChange(path)).resolves.toBe(false);
expect(dependencies.processedState.updateLastProcessedFile).toHaveBeenCalledWith(path, stat);
expect(dependencies.databaseWriteOperations.store).toHaveBeenCalledOnce();
processor.dispose();
});
});
@@ -0,0 +1,426 @@
import {
LOG_LEVEL_INFO,
LOG_LEVEL_VERBOSE,
type DocumentID,
type FilePath,
type FilePathWithPrefix,
type LoadedEntry,
type LOG_LEVEL,
type MetaEntry,
type UXStat,
} from "@vrtmrz/livesync-commonlib/compat/common/types";
import type { LogFunction } from "@vrtmrz/livesync-commonlib/compat/services/lib/logUtils";
import { isInternalMetadata } from "@vrtmrz/livesync-commonlib/compat/common/typeUtils";
import { stripAllPrefixes } from "@vrtmrz/livesync-commonlib/compat/string_and_binary/path";
import { QueueProcessor } from "octagonal-wheels/concurrency/processor";
import type { InternalFileInfo } from "@/common/types.ts";
import { getHiddenFileSyncComparisonMTime } from "./hiddenFileSyncState.ts";
export type HiddenFileSyncConflictPath = FilePath | FilePathWithPrefix;
export type HiddenFileSyncRevisionInfo = {
rev: string;
status: string;
};
export type HiddenFileSyncRevisionHistory = MetaEntry & {
_revs_info?: HiddenFileSyncRevisionInfo[];
};
export type HiddenFileSyncJsonResolution = {
keepRevision?: string;
mergedText?: string;
};
export type HiddenFileSyncConflictDatabase = {
scanConflictedEntries(): AsyncIterable<MetaEntry>;
getDocumentId(path: HiddenFileSyncConflictPath): Promise<DocumentID>;
loadCurrentMetadata(id: DocumentID): Promise<MetaEntry>;
loadConflictingMetadata(id: DocumentID, revision: string): Promise<MetaEntry>;
loadRevisionHistory(id: DocumentID): Promise<HiddenFileSyncRevisionHistory>;
loadRevisionEntry(path: HiddenFileSyncConflictPath, revision: string): Promise<LoadedEntry | false>;
mergeJson(
path: FilePathWithPrefix,
baseRevision: string,
currentRevision: string,
conflictedRevision: string
): Promise<string | false>;
removeRevision(id: DocumentID, revision: string): Promise<unknown>;
deleteRevision(entry: LoadedEntry): Promise<boolean>;
};
export type HiddenFileSyncConflictStorage = {
ensureDirectory(path: FilePath): Promise<void>;
writeFile(path: FilePath, data: string): Promise<UXStat | null>;
triggerEvent(path: FilePath): Promise<void>;
};
export type HiddenFileSyncConflictReconciliation = {
storeFile(file: InternalFileInfo, forceWrite?: boolean): Promise<boolean | undefined>;
extractFile(path: FilePath): Promise<boolean | undefined>;
};
export type HiddenFileSyncConflictInteraction = {
resolveJsonConflict(
path: FilePath,
docs: [LoadedEntry, LoadedEntry],
apply: (resolution: HiddenFileSyncJsonResolution) => Promise<boolean>
): Promise<boolean>;
};
/** Read-only queue counters retained for the real-Obsidian contract tests. */
export type HiddenFileSyncConflictProcessorTestingView = {
readonly remaining: number;
readonly totalRemaining: number;
readonly nowProcessing: number;
};
/** Focused conflict operations exposed through the Hidden File Sync test view. */
export interface HiddenFileSyncConflictTestingView {
resolveAll(): Promise<void>;
resolveJson(docA: LoadedEntry, docB: LoadedEntry): Promise<boolean>;
readonly pendingPaths: readonly HiddenFileSyncConflictPath[];
readonly processor: HiddenFileSyncConflictProcessorTestingView;
}
export type HiddenFileSyncConflictResolutionDependencies = {
database: HiddenFileSyncConflictDatabase;
storage: HiddenFileSyncConflictStorage;
reconciliation: HiddenFileSyncConflictReconciliation;
interaction: HiddenFileSyncConflictInteraction;
shouldOverwrite(path: FilePath): boolean;
log: LogFunction;
};
export interface HiddenFileSyncConflictResolution {
queue(path: HiddenFileSyncConflictPath): void;
resolveAll(): Promise<void>;
resolveJson(docA: LoadedEntry, docB: LoadedEntry): Promise<boolean>;
dispose(): void;
readonly testing: HiddenFileSyncConflictTestingView;
}
type PendingJsonConflict = {
id: DocumentID;
doc: MetaEntry;
path: HiddenFileSyncConflictPath;
revA: string;
revB: string;
};
export function selectHiddenFileSyncRevisionToDelete(
currentDoc: MetaEntry,
currentRevision: string,
conflictedDoc: MetaEntry,
conflictedRevision: string
): string {
const currentMTime = getHiddenFileSyncComparisonMTime(currentDoc, true);
const conflictedMTime = getHiddenFileSyncComparisonMTime(conflictedDoc, true);
// Compatibility: an equal mtime keeps the current leaf and deletes the
// conflicted leaf. A different tie-breaker would alter existing winners.
return currentMTime < conflictedMTime ? currentRevision : conflictedRevision;
}
export function findHiddenFileSyncMergeBase(
revisions: readonly HiddenFileSyncRevisionInfo[] | undefined,
conflictedRevision: string
): string {
const conflictedGeneration = Number(conflictedRevision.split("-")[0]);
// Compatibility question: this is the first available lower generation
// from the current branch, not a proven nearest shared ancestor. Changing
// it requires a separate conflict-history decision.
return (
revisions?.find(({ rev, status }) => status == "available" && Number(rev.split("-")[0]) < conflictedGeneration)
?.rev ?? ""
);
}
class HiddenFileSyncConflictResolutionOwner implements HiddenFileSyncConflictResolution {
private readonly pendingPaths = new Set<HiddenFileSyncConflictPath>();
private readonly processor: QueueProcessor<HiddenFileSyncConflictPath, PendingJsonConflict>;
private disposed = false;
readonly testing: HiddenFileSyncConflictTestingView;
constructor(private readonly dependencies: HiddenFileSyncConflictResolutionDependencies) {
const interactionProcessor = new QueueProcessor<PendingJsonConflict, void>(
async (results) => {
const { id, doc, path, revA, revB } = results[0];
// Compatibility question: these reads intentionally remain
// outside the catch below. A rejected read can leave the path
// pending until another lifecycle event reconstructs the owner.
const docAMerge = await this.dependencies.database.loadRevisionEntry(path, revA);
const docBMerge = await this.dependencies.database.loadRevisionEntry(path, revB);
try {
if (docAMerge != false && docBMerge != false) {
if (await this.resolveJson(docAMerge, docBMerge)) {
this.requeue(path);
} else {
this.finish(path);
}
return;
}
await this.resolveByNewerEntry(id, path, doc, revA, revB);
} catch (error) {
this.finish(path);
throw error;
}
},
{
suspended: false,
batchSize: 1,
concurrentLimit: 1,
delay: 10,
keepResultUntilDownstreamConnected: false,
yieldThreshold: 10,
}
);
this.processor = new QueueProcessor<HiddenFileSyncConflictPath, PendingJsonConflict>(
async (paths) => await this.processPath(paths[0]),
{
suspended: false,
batchSize: 1,
concurrentLimit: 5,
delay: 10,
keepResultUntilDownstreamConnected: true,
yieldThreshold: 10,
pipeTo: interactionProcessor,
}
);
const pendingPaths = () => [...this.pendingPaths];
const processor = this.processor;
const processorView = Object.freeze({
get remaining() {
return processor.remaining;
},
get totalRemaining() {
return processor.totalRemaining;
},
get nowProcessing() {
return processor.nowProcessing;
},
});
this.testing = Object.freeze({
resolveAll: async () => await this.resolveAll(),
resolveJson: async (docA: LoadedEntry, docB: LoadedEntry) => await this.resolveJson(docA, docB),
get pendingPaths() {
return pendingPaths();
},
processor: processorView,
});
}
queue(path: HiddenFileSyncConflictPath): void {
if (this.disposed) return;
// Compatibility: this deliberately deduplicates exact strings only.
// Prefixed and unprefixed forms of one path can therefore coexist.
if (this.pendingPaths.has(path)) return;
this.pendingPaths.add(path);
// Compatibility question: if QueueProcessor throws during this
// synchronous admission, the pending marker is retained. No current
// caller expects enqueue to throw.
this.processor.enqueue(path);
}
async resolveAll(): Promise<void> {
// Creating the iterator and awaiting the completed pipeline remain
// outside the catch. Only iteration failures are logged and swallowed
// by this operation.
const conflicted = this.dependencies.database.scanConflictedEntries();
// Do not suspend ordinary conflict admission during the scan.
// QueueProcessor v2 can lose its resume event when scan completion
// races with the suspended pump, leaving every admitted path pending.
try {
for await (const doc of conflicted) {
if (!("_conflicts" in doc)) continue;
if (isInternalMetadata(doc._id)) {
this.queue(doc.path);
}
}
} catch (error) {
this.log("something went wrong on resolving all conflicted internal files");
this.log(error, LOG_LEVEL_VERBOSE);
}
await this.processor.waitForAllProcessed();
}
async resolveJson(docA: LoadedEntry, docB: LoadedEntry): Promise<boolean> {
this.log("Opening data-merging dialog", LOG_LEVEL_VERBOSE);
const docs: [LoadedEntry, LoadedEntry] = [docA, docB];
const storageFilePath = stripAllPrefixes(docA.path);
const displayFilename = `${storageFilePath}`;
return await this.dependencies.interaction.resolveJsonConflict(
storageFilePath,
docs,
async ({ keepRevision: keep, mergedText: result }) => {
try {
let needFlush = false;
if (!result && !keep) {
this.log(`Skipped merging: ${displayFilename}`);
return false;
}
// Compatibility question: the selected revision is not
// validated against these two documents. An unknown value
// consequently deletes both revisions without writing a
// merged result. The sequential effects are also not
// transactional, so an earlier deletion survives a later
// failure.
for (const doc of docs) {
if (doc._rev != keep) {
if (await this.dependencies.database.deleteRevision(doc)) {
this.log(`Conflicted revision has been deleted: ${displayFilename}`);
needFlush = true;
}
}
}
if (!keep && result) {
await this.dependencies.storage.ensureDirectory(storageFilePath);
const stat = await this.dependencies.storage.writeFile(storageFilePath, result);
if (!stat) {
throw new Error("Stat failed");
}
const mtime = getHiddenFileSyncComparisonMTime(stat);
// Compatibility: interactive merged text forces the
// database write, whereas automatic merge below uses
// the writer's default admission policy.
await this.dependencies.reconciliation.storeFile(
{
path: storageFilePath,
mtime,
ctime: stat.ctime ?? mtime,
size: stat.size ?? 0,
},
true
);
await this.dependencies.storage.triggerEvent(storageFilePath);
this.log(`STORAGE <-- DB:${displayFilename}: written (hidden,merged)`);
}
if (needFlush) {
if (await this.dependencies.reconciliation.extractFile(storageFilePath)) {
this.log(`STORAGE --> DB:${displayFilename}: extracted (hidden,merged)`);
} else {
this.log(`STORAGE --> DB:${displayFilename}: extracted (hidden,merged) Failed`);
}
}
return true;
} catch (error) {
this.log("Could not merge conflicted json");
this.log(error, LOG_LEVEL_VERBOSE);
return false;
}
}
);
}
dispose(): void {
if (this.disposed) return;
this.disposed = true;
// QueueProcessor termination cascades downstream, but cannot cancel an
// already-running database operation or dialogue callback.
this.processor.terminate();
this.pendingPaths.clear();
}
private async processPath(path: HiddenFileSyncConflictPath): Promise<PendingJsonConflict[]> {
try {
const id = await this.dependencies.database.getDocumentId(path);
const doc = await this.dependencies.database.loadCurrentMetadata(id);
if (doc._conflicts === undefined || doc._conflicts.length == 0) {
this.finish(path);
return [];
}
this.log(`Hidden file conflicted:${path}`);
// Compatibility: sorting mutates the loaded Metadata object before
// it is forwarded to the manual-resolution stage.
const conflicts = doc._conflicts.sort((a, b) => Number(a.split("-")[0]) - Number(b.split("-")[0]));
const revA = doc._rev!;
const revB = conflicts[0];
if (path.endsWith(".json")) {
const revisionHistory = await this.dependencies.database.loadRevisionHistory(id);
const commonBase = findHiddenFileSyncMergeBase(revisionHistory._revs_info, revB);
const result = await this.dependencies.database.mergeJson(doc.path, commonBase, revA, revB);
if (result) {
this.log(`Object merge:${path}`, LOG_LEVEL_INFO);
const filename = stripAllPrefixes(path);
await this.dependencies.storage.ensureDirectory(filename);
const stat = await this.dependencies.storage.writeFile(filename, result);
if (!stat) {
throw new Error(`HiddenFileSyncConflictResolution: Failed to stat file ${filename}`);
}
await this.dependencies.reconciliation.storeFile({ path: filename, ...stat });
// Compatibility question: extraction is attempted before
// the conflicted branch is removed, so its conflict guard
// normally refuses it. Requeueing eventually reflects the
// winner; changing the order needs a separate decision.
await this.dependencies.reconciliation.extractFile(filename);
await this.dependencies.database.removeRevision(id, revB);
this.requeue(path);
return [];
}
this.log(`Object merge is not applicable.`, LOG_LEVEL_VERBOSE);
if (this.dependencies.shouldOverwrite(stripAllPrefixes(path))) {
this.log(`Overwrite rule applied for conflicted hidden file: ${path}`, LOG_LEVEL_INFO);
await this.resolveByNewerEntry(id, path, doc, revA, revB);
return [];
}
return [{ path, revA, revB, id, doc }];
}
await this.resolveByNewerEntry(id, path, doc, revA, revB);
return [];
} catch (error) {
this.finish(path);
this.log(`Failed to resolve conflict (Hidden): ${path}`);
this.log(error, LOG_LEVEL_VERBOSE);
return [];
}
}
private async resolveByNewerEntry(
id: DocumentID,
path: HiddenFileSyncConflictPath,
currentDoc: MetaEntry,
currentRevision: string,
conflictedRevision: string
): Promise<void> {
const conflictedDoc = await this.dependencies.database.loadConflictingMetadata(id, conflictedRevision);
const revisionToDelete = selectHiddenFileSyncRevisionToDelete(
currentDoc,
currentRevision,
conflictedDoc,
conflictedRevision
);
// Compatibility: the database result is ignored. The following conflict
// read, rather than the deletion response, decides settlement.
await this.dependencies.database.removeRevision(id, revisionToDelete);
this.log(`Older one has been deleted:${path}`);
const current = await this.dependencies.database.loadCurrentMetadata(id);
if (current._conflicts?.length === 0) {
await this.dependencies.reconciliation.extractFile(stripAllPrefixes(path));
this.finish(path);
} else {
// Compatibility: an absent _conflicts field is not considered
// settled here, although the main path treats it as conflict-free.
this.requeue(path);
}
}
private finish(path: HiddenFileSyncConflictPath): void {
this.pendingPaths.delete(path);
}
private requeue(path: HiddenFileSyncConflictPath): void {
this.finish(path);
this.queue(path);
}
private log(message: unknown, level?: LOG_LEVEL, key?: string): void {
this.dependencies.log(message, level, key);
}
}
export function createHiddenFileSyncConflictResolution(
dependencies: HiddenFileSyncConflictResolutionDependencies
): HiddenFileSyncConflictResolution {
return new HiddenFileSyncConflictResolutionOwner(dependencies);
}
@@ -0,0 +1,422 @@
import { describe, expect, it, vi } from "vitest";
import {
LOG_LEVEL_VERBOSE,
type DocumentID,
type FilePath,
type FilePathWithPrefix,
type LoadedEntry,
type MetaEntry,
type UXStat,
} from "@vrtmrz/livesync-commonlib/compat/common/types";
import {
createHiddenFileSyncConflictResolution,
findHiddenFileSyncMergeBase,
selectHiddenFileSyncRevisionToDelete,
type HiddenFileSyncConflictDatabase,
type HiddenFileSyncConflictInteraction,
type HiddenFileSyncConflictReconciliation,
type HiddenFileSyncConflictResolutionDependencies,
type HiddenFileSyncConflictStorage,
type HiddenFileSyncJsonResolution,
type HiddenFileSyncRevisionHistory,
} from "./hiddenFileSyncConflictResolution.ts";
const path = ".obsidian/plugins/example/data.json" as FilePath;
const prefixedPath = `i:${path}` as FilePathWithPrefix;
const id = "i:hidden-entry-id" as DocumentID;
function metadata(
revision: string,
mtime: number,
overrides: Partial<HiddenFileSyncRevisionHistory> = {}
): HiddenFileSyncRevisionHistory {
return {
_id: id,
_rev: revision,
path: prefixedPath,
type: "plain",
datatype: "plain",
ctime: 10,
mtime,
size: 20,
children: [],
eden: {},
deleted: false,
...overrides,
} as unknown as MetaEntry;
}
function loadedEntry(revision: string, content: string): LoadedEntry {
return {
...metadata(revision, 20),
data: content,
} as LoadedEntry;
}
function entries(...values: MetaEntry[]): AsyncIterable<MetaEntry> {
return {
async *[Symbol.asyncIterator]() {
yield* values;
},
};
}
type DependencyOverrides = {
database?: Partial<HiddenFileSyncConflictDatabase>;
storage?: Partial<HiddenFileSyncConflictStorage>;
reconciliation?: Partial<HiddenFileSyncConflictReconciliation>;
interaction?: Partial<HiddenFileSyncConflictInteraction>;
shouldOverwrite?: HiddenFileSyncConflictResolutionDependencies["shouldOverwrite"];
log?: HiddenFileSyncConflictResolutionDependencies["log"];
};
function createDependencies(overrides: DependencyOverrides = {}): HiddenFileSyncConflictResolutionDependencies {
const database: HiddenFileSyncConflictDatabase = {
scanConflictedEntries: () => entries(),
getDocumentId: vi.fn(async () => id),
loadCurrentMetadata: vi.fn(async () => metadata("1-current", 10)),
loadConflictingMetadata: vi.fn(async () => metadata("1-conflict", 10)),
loadRevisionHistory: vi.fn(async () => metadata("1-current", 10, { _revs_info: [] })),
loadRevisionEntry: vi.fn(async (): Promise<LoadedEntry | false> => false),
mergeJson: vi.fn(async (): Promise<string | false> => false),
removeRevision: vi.fn(async () => true),
deleteRevision: vi.fn(async () => true),
...overrides.database,
};
const storage: HiddenFileSyncConflictStorage = {
ensureDirectory: vi.fn(async () => undefined),
writeFile: vi.fn(async () => null),
triggerEvent: vi.fn(async () => undefined),
...overrides.storage,
};
const reconciliation: HiddenFileSyncConflictReconciliation = {
storeFile: vi.fn(async () => true),
extractFile: vi.fn(async () => true),
...overrides.reconciliation,
};
const interaction: HiddenFileSyncConflictInteraction = {
resolveJsonConflict: vi.fn(async () => false),
...overrides.interaction,
};
return {
database,
storage,
reconciliation,
interaction,
shouldOverwrite: overrides.shouldOverwrite ?? (() => false),
log: overrides.log ?? vi.fn(),
};
}
describe("Hidden File Sync conflict policy", () => {
it("keeps the current revision when both mtimes are equal", () => {
const current = metadata("3-current", 20);
const conflicted = metadata("2-conflict", 20);
expect(selectHiddenFileSyncRevisionToDelete(current, current._rev!, conflicted, conflicted._rev!)).toBe(
conflicted._rev
);
});
it("selects the first available lower-generation revision as the merge base", () => {
expect(
findHiddenFileSyncMergeBase(
[
{ rev: "4-current", status: "available" },
{ rev: "3-missing", status: "missing" },
{ rev: "2-base", status: "available" },
{ rev: "1-older", status: "available" },
],
"3-conflict"
)
).toBe("2-base");
expect(findHiddenFileSyncMergeBase(undefined, "3-conflict")).toBe("");
});
});
describe("Hidden File Sync conflict queue", () => {
it("continues processing queued conflict notifications while a full scan is in progress", async () => {
let releaseScan!: () => void;
let markScanStarted!: () => void;
const scanGate = new Promise<void>((resolve) => {
releaseScan = resolve;
});
const scanStarted = new Promise<void>((resolve) => {
markScanStarted = resolve;
});
const loadCurrentMetadata = vi.fn(async () => metadata("1-current", 10));
const dependencies = createDependencies({
database: {
scanConflictedEntries: () => ({
async *[Symbol.asyncIterator]() {
markScanStarted();
await scanGate;
},
}),
loadCurrentMetadata,
},
});
const resolution = createHiddenFileSyncConflictResolution(dependencies);
const resolvingAll = resolution.resolveAll();
await scanStarted;
// Cross the macrotask boundary which allowed the legacy suspended
// processor to stop before a database notification arrived.
await new Promise<void>((resolve) => setTimeout(resolve, 0));
resolution.queue(prefixedPath);
try {
await vi.waitFor(() => expect(loadCurrentMetadata).toHaveBeenCalledOnce(), {
interval: 10,
timeout: 250,
});
} finally {
releaseScan();
await resolvingAll;
resolution.dispose();
}
});
it("deduplicates exact paths and does not accept work after disposal", async () => {
const loadCurrentMetadata = vi.fn(async () => metadata("1-current", 10));
const dependencies = createDependencies({ database: { loadCurrentMetadata } });
const resolution = createHiddenFileSyncConflictResolution(dependencies);
resolution.queue(prefixedPath);
resolution.queue(prefixedPath);
await resolution.resolveAll();
expect(loadCurrentMetadata).toHaveBeenCalledOnce();
resolution.dispose();
resolution.queue(`i:${path}.other` as FilePathWithPrefix);
expect(loadCurrentMetadata).toHaveBeenCalledOnce();
});
it("retains prefixed and unprefixed path forms as separate compatibility keys", async () => {
const loadCurrentMetadata = vi.fn(async () => metadata("1-current", 10));
const dependencies = createDependencies({ database: { loadCurrentMetadata } });
const resolution = createHiddenFileSyncConflictResolution(dependencies);
resolution.queue(path);
resolution.queue(prefixedPath);
await resolution.resolveAll();
expect(loadCurrentMetadata).toHaveBeenCalledTimes(2);
resolution.dispose();
});
it("deletes the conflicted revision on an mtime tie, then extracts", async () => {
const events: string[] = [];
const current = metadata("3-current", 20, { _conflicts: ["2-conflict"] });
const settled = metadata("3-current", 20, { _conflicts: [] });
const dependencies = createDependencies({
database: {
scanConflictedEntries: () => entries(current),
loadCurrentMetadata: vi.fn().mockResolvedValueOnce(current).mockResolvedValueOnce(settled),
loadConflictingMetadata: vi.fn(async () => metadata("2-conflict", 20)),
removeRevision: vi.fn(async (_id, revision) => {
events.push(`remove:${revision}`);
return true;
}),
},
reconciliation: {
extractFile: vi.fn(async () => {
events.push("extract");
return true;
}),
},
});
const resolution = createHiddenFileSyncConflictResolution(dependencies);
await resolution.resolveAll();
expect(events).toEqual(["remove:2-conflict", "extract"]);
resolution.dispose();
});
it("stores and extracts an automatic merge before removing the conflicted revision", async () => {
const events: string[] = [];
const current = metadata("3-current", 30, { _conflicts: ["2-conflict"] });
const settled = metadata("4-merged", 40, { _conflicts: [] });
const stat = { ctime: 10, mtime: 20, size: 30, type: "file" } as UXStat;
const mergeJson = vi.fn(async () => '{"merged":true}');
const dependencies = createDependencies({
database: {
scanConflictedEntries: () => entries(current),
loadCurrentMetadata: vi.fn().mockResolvedValueOnce(current).mockResolvedValueOnce(settled),
loadRevisionHistory: vi.fn(async () =>
metadata("3-current", 30, {
_revs_info: [
{ rev: "3-current", status: "available" },
{ rev: "1-base", status: "available" },
],
})
),
mergeJson,
removeRevision: vi.fn(async () => {
events.push("remove");
return true;
}),
},
storage: {
ensureDirectory: vi.fn(async () => {
events.push("ensure");
}),
writeFile: vi.fn(async () => {
events.push("write");
return stat;
}),
},
reconciliation: {
storeFile: vi.fn(async () => {
events.push("store");
return true;
}),
extractFile: vi.fn(async () => {
events.push("extract");
return false;
}),
},
});
const resolution = createHiddenFileSyncConflictResolution(dependencies);
await resolution.resolveAll();
expect(mergeJson).toHaveBeenCalledWith(prefixedPath, "1-base", "3-current", "2-conflict");
expect(events).toEqual(["ensure", "write", "store", "extract", "remove"]);
resolution.dispose();
});
});
describe("Hidden File Sync JSON conflict application", () => {
function createJsonResolutionFixture(
jsonResolution: HiddenFileSyncJsonResolution,
deletionResult: boolean | Error = true
) {
const events: string[] = [];
const docA = loadedEntry("3-current", '{"current":true}');
const docB = loadedEntry("2-conflict", '{"conflict":true}');
const deleteRevision = vi.fn(async (entry: LoadedEntry) => {
events.push(`delete:${entry._rev}`);
if (deletionResult instanceof Error) {
if (entry._rev === docB._rev) throw deletionResult;
return true;
}
return deletionResult;
});
const extractFile = vi.fn(async () => {
events.push("extract");
return false;
});
const storeFile = vi.fn(async () => {
events.push("store");
return true;
});
const stat = { ctime: 11, mtime: 21, size: 22, type: "file" } as UXStat;
const log = vi.fn();
const dependencies = createDependencies({
database: { deleteRevision },
storage: {
ensureDirectory: vi.fn(async () => {
events.push("ensure");
}),
writeFile: vi.fn(async () => {
events.push("write");
return stat;
}),
triggerEvent: vi.fn(async () => {
events.push("trigger");
}),
},
reconciliation: { extractFile, storeFile },
interaction: {
resolveJsonConflict: vi.fn(async (_path, _docs, apply) => await apply(jsonResolution)),
},
log,
});
return {
deleteRevision,
docA,
docB,
events,
extractFile,
log,
resolution: createHiddenFileSyncConflictResolution(dependencies),
stat,
storeFile,
};
}
it("returns false without changing data when no resolution is selected", async () => {
const fixture = createJsonResolutionFixture({});
await expect(fixture.resolution.resolveJson(fixture.docA, fixture.docB)).resolves.toBe(false);
expect(fixture.events).toEqual([]);
fixture.resolution.dispose();
});
it("keeps the selected revision but reports success when follow-up extraction fails", async () => {
const fixture = createJsonResolutionFixture({ keepRevision: "3-current" });
await expect(fixture.resolution.resolveJson(fixture.docA, fixture.docB)).resolves.toBe(true);
expect(fixture.deleteRevision).toHaveBeenCalledTimes(1);
expect(fixture.deleteRevision).toHaveBeenCalledWith(fixture.docB);
expect(fixture.events).toEqual([`delete:${fixture.docB._rev}`, "extract"]);
expect(fixture.log).toHaveBeenCalledWith(
`STORAGE --> DB:${path}: extracted (hidden,merged) Failed`,
undefined,
undefined
);
fixture.resolution.dispose();
});
it("deletes both supplied revisions when the selected revision is unknown", async () => {
const fixture = createJsonResolutionFixture({ keepRevision: "9-unknown" });
await expect(fixture.resolution.resolveJson(fixture.docA, fixture.docB)).resolves.toBe(true);
expect(fixture.events).toEqual([`delete:${fixture.docA._rev}`, `delete:${fixture.docB._rev}`, "extract"]);
expect(fixture.storeFile).not.toHaveBeenCalled();
fixture.resolution.dispose();
});
it("deletes both revisions before writing and storing a merged result", async () => {
const fixture = createJsonResolutionFixture({ mergedText: '{"merged":true}' });
await expect(fixture.resolution.resolveJson(fixture.docA, fixture.docB)).resolves.toBe(true);
expect(fixture.storeFile).toHaveBeenCalledWith(
{
path,
ctime: fixture.stat.ctime,
mtime: fixture.stat.mtime,
size: fixture.stat.size,
},
true
);
expect(fixture.events).toEqual([
`delete:${fixture.docA._rev}`,
`delete:${fixture.docB._rev}`,
"ensure",
"write",
"store",
"trigger",
"extract",
]);
fixture.resolution.dispose();
});
it("keeps an earlier successful deletion when a later deletion throws", async () => {
const error = new Error("second deletion failed");
const fixture = createJsonResolutionFixture({ mergedText: '{"merged":true}' }, error);
await expect(fixture.resolution.resolveJson(fixture.docA, fixture.docB)).resolves.toBe(false);
expect(fixture.events).toEqual([`delete:${fixture.docA._rev}`, `delete:${fixture.docB._rev}`]);
expect(fixture.storeFile).not.toHaveBeenCalled();
expect(fixture.log).toHaveBeenCalledWith(error, LOG_LEVEL_VERBOSE, undefined);
fixture.resolution.dispose();
});
});
@@ -0,0 +1,129 @@
import { describe, expect, it, vi } from "vitest";
import {
type DocumentID,
type FilePath,
type FilePathWithPrefix,
type LoadedEntry,
type MetaEntry,
type UXStat,
} from "@vrtmrz/livesync-commonlib/compat/common/types";
import { ICHeader, ICHeaderEnd } from "@/common/types.ts";
vi.mock("@/deps.ts", () => ({}));
vi.mock("./configureHiddenFileSyncMode.ts", () => ({
configureHiddenFileSyncMode: vi.fn(),
}));
import { HiddenFileSyncContext } from "./hiddenFileSyncContext.ts";
describe("HiddenFileSyncContext operation composition", () => {
it("composes the conflict owner from the current database and path capabilities", async () => {
const path = ".obsidian/app.json" as FilePath;
const prefixedPath = `i:${path}` as FilePathWithPrefix;
const metadata = {
_id: "i:hidden-entry-id" as DocumentID,
_rev: "2-current",
path: prefixedPath,
type: "plain",
datatype: "plain",
ctime: 10,
mtime: 20,
size: 20,
children: [],
eden: {},
deleted: false,
_conflicts: [],
} as unknown as MetaEntry;
const findEntries = vi.fn(() => ({
async *[Symbol.asyncIterator]() {
yield metadata;
},
}));
const getRaw = vi.fn(async () => metadata);
const path2id = vi.fn(async () => metadata._id);
const periodicProcessor = { enable: vi.fn(), disable: vi.fn() };
const context = new HiddenFileSyncContext({
createPeriodicProcessor: vi.fn(() => periodicProcessor),
getLocalDatabase: () => ({ findEntries, getRaw }),
path: { path2id },
log: vi.fn(),
publishActivity: vi.fn(),
closeJsonConflictDialogs: vi.fn(),
hideConfigurationChangeNotice: vi.fn(),
} as never);
await context.testing.conflictResolution.resolveAll();
expect(findEntries).toHaveBeenCalledWith(ICHeader, ICHeaderEnd, { conflicts: true });
expect(path2id).toHaveBeenCalledWith(prefixedPath, ICHeader);
expect(getRaw).toHaveBeenCalledWith(metadata._id, { conflicts: true });
context.dispose();
});
it("applies a selected live revision through the narrow repair view", async () => {
const path = ".obsidian/plugins/example/data.json" as FilePath;
const prefixedPath = `i:${path}` as FilePathWithPrefix;
const revision = "2-selected";
const metadata = {
_id: "hidden-entry-id" as DocumentID,
_rev: revision,
path: prefixedPath,
type: "plain",
datatype: "plain",
ctime: 10,
mtime: 20,
size: 20,
children: [],
eden: {},
deleted: false,
} as unknown as MetaEntry;
const loaded = {
...metadata,
data: '{"value":"database"}',
} as LoadedEntry;
const stat = { ctime: 10, mtime: 20, size: 20, type: "file" } as UXStat;
const statHidden = vi.fn<() => Promise<UXStat | null>>().mockResolvedValueOnce(null).mockResolvedValue(stat);
const writeHiddenFileAuto = vi.fn(async () => true);
const getDBEntryFromMeta = vi.fn(async () => loaded);
const fetchEntryMeta = vi.fn(async () => metadata);
const getConflictedRevs = vi.fn(async () => [] as string[]);
const markChangesAreSame = vi.fn();
const periodicProcessor = { enable: vi.fn(), disable: vi.fn() };
const context = new HiddenFileSyncContext({
createPeriodicProcessor: vi.fn(() => periodicProcessor),
isIgnoredByIgnoreFile: vi.fn(async () => false),
databaseFileAccess: {
fetchEntryMeta,
getConflictedRevs,
},
getLocalDatabase: () => ({ getDBEntryFromMeta }),
storageAccess: {
statHidden,
isExistsIncludeHidden: vi.fn(async () => false),
ensureDir: vi.fn(async () => true),
writeHiddenFileAuto,
},
path: {
markChangesAreSame,
unmarkChanges: vi.fn(),
},
getSettings: () => ({ suppressNotifyHiddenFilesChange: true }),
log: vi.fn(),
publishActivity: vi.fn(),
closeJsonConflictDialogs: vi.fn(),
hideConfigurationChangeNotice: vi.fn(),
} as never);
await expect(context.repair.extractInternalFileRevisionFromDatabase(path, revision, true)).resolves.toBe(true);
expect(fetchEntryMeta).toHaveBeenCalledWith(prefixedPath, revision, true);
expect(getConflictedRevs).toHaveBeenCalledWith(prefixedPath);
expect(getDBEntryFromMeta).toHaveBeenCalledWith(metadata, false, true);
expect(writeHiddenFileAuto).toHaveBeenCalledWith(path, '{"value":"database"}', {
ctime: metadata.ctime,
mtime: metadata.mtime,
});
expect(markChangesAreSame).toHaveBeenCalledWith(path, metadata.mtime, stat.mtime);
context.dispose();
});
});
@@ -10,58 +10,58 @@ import { HiddenFileSyncContext } from "./hiddenFileSyncContext.ts";
const PATH = ".obsidian/plugins/example/data.json" as FilePath; const PATH = ".obsidian/plugins/example/data.json" as FilePath;
function isTargetFile(context: HiddenFileSyncContext, path: FilePath): Promise<boolean> {
return (context as unknown as { isTargetFile(path: FilePath): Promise<boolean> }).isTargetFile(path);
}
function isTargetFileEligible(context: HiddenFileSyncContext, path: FilePath): Promise<boolean> {
return (context as unknown as { isTargetFileEligible(path: FilePath): Promise<boolean> }).isTargetFileEligible(path);
}
function createHiddenFileSync( function createHiddenFileSync(
options: { owned?: boolean; ignoredByIgnoreFile?: boolean; patternMatch?: boolean } = {} options: { owned?: boolean; ignoredByIgnoreFile?: boolean; patternMatch?: boolean } = {}
) { ) {
const ownsLocalFile = vi.fn(() => options.owned ?? true); const ownsLocalFile = vi.fn(() => options.owned ?? true);
const isIgnoredByIgnoreFile = vi.fn(async () => options.ignoredByIgnoreFile ?? false); const isIgnoredByIgnoreFile = vi.fn(async () => options.ignoredByIgnoreFile ?? false);
const isTargetFileInPatterns = vi.fn(() => options.patternMatch ?? true); const patternTest = vi.fn(() => options.patternMatch ?? true);
const parseRegExpSettings = vi.fn(() => ({
ignoreFilter: [],
targetFilter: options.patternMatch === undefined ? [] : [{ test: patternTest }],
}));
const hiddenFileSync = Object.create(HiddenFileSyncContext.prototype) as HiddenFileSyncContext; const hiddenFileSync = Object.create(HiddenFileSyncContext.prototype) as HiddenFileSyncContext;
Object.assign(hiddenFileSync, { Object.assign(hiddenFileSync, {
dependencies: { ownsLocalFile, isIgnoredByIgnoreFile }, dependencies: { ownsLocalFile, isIgnoredByIgnoreFile },
isTargetFileInPatterns, parseRegExpSettings,
}); });
return { hiddenFileSync, isIgnoredByIgnoreFile, isTargetFileInPatterns, ownsLocalFile }; return { hiddenFileSync, isIgnoredByIgnoreFile, parseRegExpSettings, ownsLocalFile };
} }
describe("Hidden File Sync local-path admission", () => { describe("Hidden File Sync local-path admission", () => {
it("checks composition ownership before Hidden File Sync filters", async () => { it("checks composition ownership before Hidden File Sync filters", async () => {
const { hiddenFileSync, isTargetFileInPatterns, ownsLocalFile } = createHiddenFileSync({ owned: false }); const { hiddenFileSync, parseRegExpSettings, ownsLocalFile } = createHiddenFileSync({ owned: false });
await expect(hiddenFileSync.isTargetFile(PATH)).resolves.toBe(false); await expect(isTargetFile(hiddenFileSync, PATH)).resolves.toBe(false);
expect(ownsLocalFile).toHaveBeenCalledWith(PATH); expect(ownsLocalFile).toHaveBeenCalledWith(PATH);
expect(isTargetFileInPatterns).not.toHaveBeenCalled(); expect(parseRegExpSettings).not.toHaveBeenCalled();
}); });
it("keeps target patterns and ignore-file results as Hidden File Sync eligibility", async () => { it("keeps target patterns and ignore-file results as Hidden File Sync eligibility", async () => {
const patternExcluded = createHiddenFileSync({ patternMatch: false }); const patternExcluded = createHiddenFileSync({ patternMatch: false });
await expect(patternExcluded.hiddenFileSync.isTargetFile(PATH)).resolves.toBe(false); await expect(isTargetFile(patternExcluded.hiddenFileSync, PATH)).resolves.toBe(false);
expect(patternExcluded.isIgnoredByIgnoreFile).not.toHaveBeenCalled(); expect(patternExcluded.isIgnoredByIgnoreFile).not.toHaveBeenCalled();
const ignoreFileExcluded = createHiddenFileSync({ ignoredByIgnoreFile: true }); const ignoreFileExcluded = createHiddenFileSync({ ignoredByIgnoreFile: true });
await expect(ignoreFileExcluded.hiddenFileSync.isTargetFile(PATH)).resolves.toBe(false); await expect(isTargetFile(ignoreFileExcluded.hiddenFileSync, PATH)).resolves.toBe(false);
expect(ignoreFileExcluded.isIgnoredByIgnoreFile).toHaveBeenCalledWith(PATH); expect(ignoreFileExcluded.isIgnoredByIgnoreFile).toHaveBeenCalledWith(PATH);
const admitted = createHiddenFileSync(); const admitted = createHiddenFileSync();
await expect(admitted.hiddenFileSync.isTargetFile(PATH)).resolves.toBe(true); await expect(isTargetFile(admitted.hiddenFileSync, PATH)).resolves.toBe(true);
}); });
it("exposes eligibility without consulting the composition owner", async () => { it("evaluates eligibility without consulting the composition owner", async () => {
const { hiddenFileSync, ownsLocalFile } = createHiddenFileSync({ owned: false }); const { hiddenFileSync, ownsLocalFile } = createHiddenFileSync({ owned: false });
await expect(hiddenFileSync.isTargetFileEligible(PATH)).resolves.toBe(true); await expect(isTargetFileEligible(hiddenFileSync, PATH)).resolves.toBe(true);
expect(ownsLocalFile).not.toHaveBeenCalled(); expect(ownsLocalFile).not.toHaveBeenCalled();
}); });
}); });
describe("compatibility: Hidden File Sync path shape", () => {
const hiddenFileSync = Object.create(HiddenFileSyncContext.prototype) as HiddenFileSyncContext;
it.each([
[".obsidian/app.json", true],
[".trash/app.json", false],
["notes/app.json", false],
])("recognises %s as a Hidden File Sync path=%s", (path, expected) => {
expect(hiddenFileSync.isHiddenFileSyncHandlingPath(path as FilePath)).toBe(expected);
});
});
@@ -1,5 +1,4 @@
import { describe, expect, it, vi } from "vitest"; import { describe, expect, it, vi } from "vitest";
import type { FilePath } from "@vrtmrz/livesync-commonlib/compat/common/types";
vi.mock("@/deps.ts", () => ({})); vi.mock("@/deps.ts", () => ({}));
vi.mock("./configureHiddenFileSyncMode.ts", () => ({ vi.mock("./configureHiddenFileSyncMode.ts", () => ({
@@ -8,12 +7,29 @@ vi.mock("./configureHiddenFileSyncMode.ts", () => ({
import { HiddenFileSyncContext } from "./hiddenFileSyncContext.ts"; import { HiddenFileSyncContext } from "./hiddenFileSyncContext.ts";
function createContext() { function getProcessedState(context: HiddenFileSyncContext): unknown {
return (context as unknown as { readonly processedState: unknown }).processedState;
}
function getPrivate<T>(context: HiddenFileSyncContext, key: string): T {
return (context as unknown as Record<string, T>)[key];
}
function createContext(processedFiles = new Map()) {
const periodicProcessor = { enable: vi.fn(), disable: vi.fn() }; const periodicProcessor = { enable: vi.fn(), disable: vi.fn() };
const publishActivity = vi.fn(); const publishActivity = vi.fn();
const hideConfigurationChangeNotice = vi.fn(); const hideConfigurationChangeNotice = vi.fn();
const keyValueDatabase = {
get: vi.fn(async (key: IDBValidKey) => {
if (key == "hidden-file-lastProcessed") return processedFiles;
return new Map();
}),
};
const context = new HiddenFileSyncContext({ const context = new HiddenFileSyncContext({
createPeriodicProcessor: vi.fn(() => periodicProcessor), createPeriodicProcessor: vi.fn(() => periodicProcessor),
getKeyValueDatabase: () => keyValueDatabase,
getSettings: () => ({ syncInternalFiles: true }),
log: vi.fn(),
publishActivity, publishActivity,
closeJsonConflictDialogs: vi.fn(), closeJsonConflictDialogs: vi.fn(),
hideConfigurationChangeNotice, hideConfigurationChangeNotice,
@@ -22,71 +38,41 @@ function createContext() {
} }
describe("HiddenFileSyncContext state ownership", () => { describe("HiddenFileSyncContext state ownership", () => {
it("owns queues, caches, concurrency controls, and processors per context instance", () => { it("owns caches, concurrency controls, processors, and conflict owners per context instance", () => {
const first = createContext(); const first = createContext();
const second = createContext(); const second = createContext();
first.context.pendingConflictChecks.add("i:.obsidian/first.json" as never); getPrivate<Set<string>>(first.context, "queuedNotificationFiles").add(".obsidian/plugins/first");
first.context.queuedNotificationFiles.add(".obsidian/plugins/first"); getPrivate<Map<string, unknown[]>>(first.context, "cacheFileRegExps").set("first", []);
first.context.cacheFileRegExps.set("first", []);
expect(second.context.pendingConflictChecks).toEqual(new Set()); expect(getPrivate<Set<string>>(second.context, "queuedNotificationFiles").size).toBe(0);
expect(second.context.queuedNotificationFiles).toEqual(new Set()); expect(getPrivate<Map<string, unknown[]>>(second.context, "cacheFileRegExps")).toEqual(new Map());
expect(second.context.cacheFileRegExps).toEqual(new Map()); expect(first.context.testing.conflictResolution).not.toBe(second.context.testing.conflictResolution);
expect(first.context.conflictResolutionProcessor).not.toBe(second.context.conflictResolutionProcessor); expect(getProcessedState(first.context)).not.toBe(getProcessedState(second.context));
expect(first.context.semaphore).not.toBe(second.context.semaphore); expect(getPrivate<unknown>(first.context, "changeProcessor")).not.toBe(
expect(first.context.periodicInternalFileScanProcessor).toBe(first.periodicProcessor); getPrivate<unknown>(second.context, "changeProcessor")
expect(second.context.periodicInternalFileScanProcessor).toBe(second.periodicProcessor); );
expect(getPrivate<unknown>(first.context, "periodicInternalFileScanProcessor")).toBe(first.periodicProcessor);
expect(getPrivate<unknown>(second.context, "periodicInternalFileScanProcessor")).toBe(second.periodicProcessor);
first.context.dispose(); first.context.dispose();
second.context.dispose(); second.context.dispose();
}); });
it("publishes instance-owned event and processing counts at each transition", async () => {
const { context, publishActivity } = createContext();
const path = ".obsidian/app.json" as FilePath;
await context.serializedForEvent(path, async () => {
expect(publishActivity).toHaveBeenLastCalledWith(1, 1);
});
expect(publishActivity.mock.calls).toEqual([
[1, 0],
[1, 1],
[1, 0],
[0, 0],
]);
context.dispose();
});
it.each([ it.each([
[new Map(), true], [new Map(), true],
[new Map([[".obsidian/app.json", "1-2-3"]]), false], [new Map([[".obsidian/app.json", "1-2-3"]]), false],
])( ])(
"preserves start-up scan notice selection for the processed-file cache", "preserves start-up scan notice selection for the processed-file cache",
async (processedFiles, forcedNotice) => { async (processedFiles, forcedNotice) => {
const { context } = createContext(); const { context } = createContext(processedFiles);
const performStartupScan = vi.fn(async () => undefined); const applyOfflineChanges = vi.fn(async () => undefined);
const keyValueDatabase = { context.applyOfflineChanges = applyOfflineChanges;
get: vi.fn(async (key: IDBValidKey) => {
if (key == "hidden-file-lastProcessed") return processedFiles;
return new Map();
}),
};
Object.assign(context, {
dependencies: {
createPeriodicProcessor: vi.fn(),
getKeyValueDatabase: () => keyValueDatabase,
getSettings: () => ({ syncInternalFiles: true }),
log: vi.fn(),
},
performStartupScan,
});
await context._everyOnDatabaseInitialized(false); await context.serviceHandlers.onDatabaseInitialised(false);
expect(performStartupScan).toHaveBeenCalledWith(forcedNotice); expect(applyOfflineChanges).toHaveBeenCalledWith(forcedNotice);
context.conflictResolutionProcessor.terminate(); context.dispose();
} }
); );
}); });
File diff suppressed because it is too large Load Diff
@@ -1,12 +1,5 @@
import { describe, expect, it, vi } from "vitest"; import { describe, expect, it, vi } from "vitest";
import { import { LOG_LEVEL_NOTICE } from "@vrtmrz/livesync-commonlib/compat/common/types";
type DocumentID,
LOG_LEVEL_NOTICE,
type FilePath,
type FilePathWithPrefix,
type MetaEntry,
type UXFileInfo,
} from "@vrtmrz/livesync-commonlib/compat/common/types";
vi.mock("@/deps.ts", () => ({})); vi.mock("@/deps.ts", () => ({}));
vi.mock("./configureHiddenFileSyncMode.ts", () => ({ vi.mock("./configureHiddenFileSyncMode.ts", () => ({
@@ -16,81 +9,27 @@ vi.mock("./configureHiddenFileSyncMode.ts", () => ({
import { HiddenFileSyncContext } from "./hiddenFileSyncContext.ts"; import { HiddenFileSyncContext } from "./hiddenFileSyncContext.ts";
import { configureHiddenFileSyncMode } from "./configureHiddenFileSyncMode.ts"; import { configureHiddenFileSyncMode } from "./configureHiddenFileSyncMode.ts";
function createHiddenRevisionOperation() { function callPrivate<T extends (...args: never[]) => unknown>(context: HiddenFileSyncContext, key: string): T {
const path = ".obsidian/plugins/example/data.json" as FilePath; const operation = (context as unknown as Record<string, T>)[key];
const file = { return operation.bind(context) as T;
path,
name: "data.json",
isInternal: true,
body: new Blob(['{"value":"vault"}']),
stat: {
ctime: 1,
mtime: 2,
size: 17,
type: "file",
},
} as UXFileInfo;
const selected = {
_id: "i:example" as DocumentID,
_rev: "2-selected",
path: `i:${path}` as FilePathWithPrefix,
ctime: 1,
mtime: 2,
size: 17,
type: "plain",
datatype: "plain",
children: [],
eden: {},
deleted: false,
} as MetaEntry;
const winner = {
...selected,
_rev: "3-winner",
} as MetaEntry;
const databaseFileAccess = {
fetchEntryMeta: vi.fn(async (_path: unknown, revision?: string) =>
revision === selected._rev ? selected : winner
),
getConflictedRevs: vi.fn(async () => [selected._rev]),
fetchEntryFromMeta: vi.fn(async () => ({ ...selected, data: '{"value":"database"}' })),
storeWithBaseRevision: vi.fn(async () => "3-vault-child"),
};
const hiddenFileSync = Object.create(HiddenFileSyncContext.prototype) as HiddenFileSyncContext;
Object.assign(hiddenFileSync, {
dependencies: {
databaseFileAccess,
isIgnoredByIgnoreFile: vi.fn(async () => false),
},
loadFileWithInfo: vi.fn(async () => file),
updateLastProcessed: vi.fn(),
_log: vi.fn(),
});
return {
hiddenFileSync,
path,
file,
selected,
winner,
databaseFileAccess,
};
} }
describe("HiddenFileSyncContext configuration-change notices", () => { describe("HiddenFileSyncContext configuration-change notices", () => {
it("releases processors, transient queues, the pattern cache, activity, and the host Notice effect", () => { it("releases processors, transient queues, the pattern cache, activity, and the host Notice effect", () => {
const periodicInternalFileScanProcessor = { disable: vi.fn() }; const periodicInternalFileScanProcessor = { disable: vi.fn() };
const conflictResolutionProcessor = { terminate: vi.fn() }; const conflictResolution = { dispose: vi.fn() };
const pendingConflictChecks = new Set(["i:.obsidian/example.json"]);
const queuedNotificationFiles = new Set([".obsidian/plugins/example"]); const queuedNotificationFiles = new Set([".obsidian/plugins/example"]);
const cacheFileRegExps = new Map([["patterns", []]]); const cacheFileRegExps = new Map([["patterns", []]]);
const publishActivity = vi.fn(); const publishActivity = vi.fn();
const changeProcessor = { dispose: vi.fn() };
const hideConfigurationChangeNotice = vi.fn(); const hideConfigurationChangeNotice = vi.fn();
const closeJsonConflictDialogs = vi.fn(); const closeJsonConflictDialogs = vi.fn();
const hiddenFileSync = Object.create(HiddenFileSyncContext.prototype) as HiddenFileSyncContext; const hiddenFileSync = Object.create(HiddenFileSyncContext.prototype) as HiddenFileSyncContext;
Object.assign(hiddenFileSync, { Object.assign(hiddenFileSync, {
dependencies: { publishActivity, hideConfigurationChangeNotice, closeJsonConflictDialogs }, dependencies: { publishActivity, hideConfigurationChangeNotice, closeJsonConflictDialogs },
periodicInternalFileScanProcessor, periodicInternalFileScanProcessor,
conflictResolutionProcessor, conflictResolution,
pendingConflictChecks, changeProcessor,
queuedNotificationFiles, queuedNotificationFiles,
cacheFileRegExps, cacheFileRegExps,
eventCount: 4, eventCount: 4,
@@ -101,11 +40,10 @@ describe("HiddenFileSyncContext configuration-change notices", () => {
hiddenFileSync.dispose(); hiddenFileSync.dispose();
expect(periodicInternalFileScanProcessor.disable).toHaveBeenCalledOnce(); expect(periodicInternalFileScanProcessor.disable).toHaveBeenCalledOnce();
expect(conflictResolutionProcessor.terminate).toHaveBeenCalledOnce(); expect(conflictResolution.dispose).toHaveBeenCalledOnce();
expect(pendingConflictChecks.size).toBe(0);
expect(queuedNotificationFiles.size).toBe(0); expect(queuedNotificationFiles.size).toBe(0);
expect(cacheFileRegExps.size).toBe(0); expect(cacheFileRegExps.size).toBe(0);
expect(publishActivity).toHaveBeenCalledWith(0, 0); expect(changeProcessor.dispose).toHaveBeenCalledOnce();
expect(closeJsonConflictDialogs).toHaveBeenCalledOnce(); expect(closeJsonConflictDialogs).toHaveBeenCalledOnce();
expect(hideConfigurationChangeNotice).toHaveBeenCalledOnce(); expect(hideConfigurationChangeNotice).toHaveBeenCalledOnce();
}); });
@@ -122,7 +60,7 @@ describe("HiddenFileSyncContext configuration-change notices", () => {
_isMainSuspended: vi.fn(() => false), _isMainSuspended: vi.fn(() => false),
}); });
expect(hiddenFileSync.isReady()).toBe(false); expect(callPrivate<() => boolean>(hiddenFileSync, "isReady")()).toBe(false);
}); });
it("settles one batch of changed folders through the host notification effect", () => { it("settles one batch of changed folders through the host notification effect", () => {
@@ -133,14 +71,16 @@ describe("HiddenFileSyncContext configuration-change notices", () => {
queuedNotificationFiles: new Set([".obsidian/plugins/alpha", ".obsidian/plugins/beta", ".obsidian"]), queuedNotificationFiles: new Set([".obsidian/plugins/alpha", ".obsidian/plugins/beta", ".obsidian"]),
}); });
hiddenFileSync.notifyConfigChange(); callPrivate<() => void>(hiddenFileSync, "notifyConfigChange")();
expect(showConfigurationChangeNotice).toHaveBeenCalledWith([ expect(showConfigurationChangeNotice).toHaveBeenCalledWith([
".obsidian/plugins/alpha", ".obsidian/plugins/alpha",
".obsidian/plugins/beta", ".obsidian/plugins/beta",
".obsidian", ".obsidian",
]); ]);
expect(hiddenFileSync.queuedNotificationFiles.size).toBe(0); expect((hiddenFileSync as unknown as { queuedNotificationFiles: Set<string> }).queuedNotificationFiles.size).toBe(
0
);
}); });
it("keeps subordinate initialisation phases below Notice level so one progress Notice owns the scan", async () => { it("keeps subordinate initialisation phases below Notice level so one progress Notice owns the scan", async () => {
@@ -200,7 +140,7 @@ describe("HiddenFileSyncContext configuration-change notices", () => {
_log: log, _log: log,
}); });
await hiddenFileSync.configureHiddenFileSync("MERGE"); await callPrivate<(mode: "MERGE") => Promise<void>>(hiddenFileSync, "configureHiddenFileSync")("MERGE");
expect(createProgress).toHaveBeenCalledWith("[⚙ Initialise]\n", LOG_LEVEL_NOTICE); expect(createProgress).toHaveBeenCalledWith("[⚙ Initialise]\n", LOG_LEVEL_NOTICE);
expect(events[0]).toBe("progress:Preparing Hidden File Sync..."); expect(events[0]).toBe("progress:Preparing Hidden File Sync...");
@@ -232,119 +172,10 @@ describe("HiddenFileSyncContext configuration-change notices", () => {
_log: vi.fn(), _log: vi.fn(),
}); });
await expect(hiddenFileSync.configureHiddenFileSync("MERGE")).rejects.toBe(error); await expect(
callPrivate<(mode: "MERGE") => Promise<void>>(hiddenFileSync, "configureHiddenFileSync")("MERGE")
).rejects.toBe(error);
expect(progress.done).toHaveBeenCalledWith("Failed"); expect(progress.done).toHaveBeenCalledWith("Failed");
}); });
}); });
describe("HiddenFileSyncContext reconciliation settlement", () => {
it("compatibility: consumes a selected database file even when reading its Metadata fails", async () => {
const error = new Error("metadata unavailable");
const getDBEntryMeta = vi.fn(async () => {
throw error;
});
const log = vi.fn();
const hiddenFileSync = Object.create(HiddenFileSyncContext.prototype) as HiddenFileSyncContext;
Object.assign(hiddenFileSync, {
dependencies: {
getLocalDatabase: () => ({ getDBEntryMeta }),
log,
},
serializedForEvent: vi.fn(async (_path: FilePath, operation: () => Promise<boolean>) => await operation()),
});
await expect(
hiddenFileSync.trackDatabaseFileModification(".obsidian/app.json" as FilePath, "[Replication]")
).resolves.toBe(true);
expect(log).toHaveBeenCalledWith("[Replication] Failed to process hidden file", undefined, undefined);
expect(log).toHaveBeenCalledWith(error, expect.any(Number), undefined);
});
});
describe("HiddenFileSyncContext exact revision repair operations", () => {
it("stores the current hidden Vault file as a child of the selected live revision", async () => {
const { hiddenFileSync, file, selected, databaseFileAccess } = createHiddenRevisionOperation();
await expect(hiddenFileSync.storeInternalFileToDatabaseWithBaseRevision(file, selected._rev!)).resolves.toBe(
true
);
expect(databaseFileAccess.storeWithBaseRevision).toHaveBeenCalledWith(
expect.objectContaining({
path: file.path,
body: file.body,
isInternal: true,
}),
selected._rev,
true
);
expect(hiddenFileSync.updateLastProcessed).toHaveBeenCalledWith(
file.path,
expect.objectContaining({ _rev: "3-vault-child" }),
file.stat
);
});
it("refuses to extend a hidden-file revision which is no longer live", async () => {
const { hiddenFileSync, file, selected, databaseFileAccess } = createHiddenRevisionOperation();
databaseFileAccess.getConflictedRevs.mockResolvedValue([]);
await expect(hiddenFileSync.storeInternalFileToDatabaseWithBaseRevision(file, selected._rev!)).resolves.toBe(
false
);
expect(databaseFileAccess.storeWithBaseRevision).not.toHaveBeenCalled();
expect(hiddenFileSync.updateLastProcessed).not.toHaveBeenCalled();
});
it("does not create a hidden-file child when asked only to mark a revision which differs from the Vault", async () => {
const { hiddenFileSync, file, selected, databaseFileAccess } = createHiddenRevisionOperation();
await expect(
hiddenFileSync.storeInternalFileToDatabaseWithBaseRevision(file, selected._rev!, false)
).resolves.toBe(false);
expect(databaseFileAccess.storeWithBaseRevision).not.toHaveBeenCalled();
expect(hiddenFileSync.updateLastProcessed).not.toHaveBeenCalled();
});
it("marks a matching hidden-file revision without creating a child", async () => {
const { hiddenFileSync, file, selected, databaseFileAccess } = createHiddenRevisionOperation();
databaseFileAccess.fetchEntryFromMeta.mockResolvedValue({
...selected,
data: '{"value":"vault"}',
});
await expect(
hiddenFileSync.storeInternalFileToDatabaseWithBaseRevision(file, selected._rev!, false)
).resolves.toBe(true);
expect(databaseFileAccess.storeWithBaseRevision).not.toHaveBeenCalled();
expect(hiddenFileSync.updateLastProcessed).toHaveBeenCalledWith(file.path, selected, file.stat);
});
it("applies the selected live hidden-file revision through the existing extraction path", async () => {
const { hiddenFileSync, path, selected } = createHiddenRevisionOperation();
const extract = vi.fn(async () => true);
hiddenFileSync.extractInternalFileFromDatabase = extract;
await expect(hiddenFileSync.extractInternalFileRevisionFromDatabase(path, selected._rev!, true)).resolves.toBe(
true
);
expect(extract).toHaveBeenCalledWith(path, true, undefined, true, false, true, selected._rev);
});
it("does not apply a hidden-file revision which ceased to be live", async () => {
const { hiddenFileSync, path, selected, databaseFileAccess } = createHiddenRevisionOperation();
databaseFileAccess.getConflictedRevs.mockResolvedValue([]);
await expect(hiddenFileSync.extractInternalFileRevisionFromDatabase(path, selected._rev!, true)).resolves.toBe(
false
);
expect(databaseFileAccess.fetchEntryFromMeta).not.toHaveBeenCalled();
});
});
@@ -0,0 +1,223 @@
import {
LOG_LEVEL_INFO,
LOG_LEVEL_VERBOSE,
type FilePath,
type FilePathWithPrefix,
type LoadedEntry,
type LOG_LEVEL,
type MetaEntry,
type UXStat,
} from "@vrtmrz/livesync-commonlib/compat/common/types";
import { compareMTime, TARGET_IS_NEW } from "@/common/utils.ts";
import type { LogFunction } from "@vrtmrz/livesync-commonlib/compat/services/lib/logUtils";
import { addPrefix } from "@vrtmrz/livesync-commonlib/compat/string_and_binary/path";
import { ICHeader } from "@/common/types.ts";
import { getHiddenFileSyncComparisonMTime } from "./hiddenFileSyncState.ts";
import type { HiddenFileSyncRemovalResult } from "./hiddenFileSyncStorage.ts";
import {
serialiseHiddenFileOperation,
type HiddenFileSyncFileSerialisationDependencies,
} from "./hiddenFileSyncFileOperations.ts";
export type HiddenFileSyncDatabaseExtractionOptions = Readonly<{
force?: boolean;
metaEntry?: MetaEntry | LoadedEntry;
preventDoubleProcess?: boolean;
onlyNew?: boolean;
includeDeletion?: boolean;
requiredLiveRevision?: string;
}>;
type HiddenFileSyncDatabaseExtractionStorageDependencies = {
statStorageFile(path: FilePath): Promise<UXStat | null>;
writeStorageFile(path: FilePath, entry: LoadedEntry, force: boolean): Promise<false | UXStat>;
deleteStorageFile(path: FilePath): Promise<HiddenFileSyncRemovalResult>;
};
type HiddenFileSyncDatabaseExtractionReadDependencies = {
loadDatabaseMetadata(path: FilePathWithPrefix): Promise<MetaEntry | LoadedEntry | false>;
loadLiveRevision(path: FilePathWithPrefix, revision: string): Promise<MetaEntry | false>;
loadDatabaseEntry(entry: MetaEntry | LoadedEntry): Promise<LoadedEntry | false>;
};
export type HiddenFileSyncDatabaseExtractionProcessedState = {
databaseStateKey(entry: MetaEntry | LoadedEntry): string;
getLastProcessedDatabaseKey(path: FilePath): string | undefined;
getLastProcessedFileMTime(path: FilePath): number;
updateLastProcessedDatabase(path: FilePath, entry: string | MetaEntry | LoadedEntry): void;
updateLastProcessedFile(path: FilePath, storageFile: string | UXStat): void;
updateLastProcessed(path: FilePath, databaseEntry: MetaEntry | LoadedEntry, storageFile: UXStat): void;
updateLastProcessedDeletion(path: FilePath, databaseEntry: MetaEntry | LoadedEntry | false): void;
};
export type HiddenFileSyncDatabaseExtractionDependencies = HiddenFileSyncFileSerialisationDependencies &
HiddenFileSyncDatabaseExtractionStorageDependencies &
HiddenFileSyncDatabaseExtractionReadDependencies & {
isIgnoredByIgnoreFile(path: string): Promise<boolean>;
queueNotification(path: FilePath): void;
processedState: HiddenFileSyncDatabaseExtractionProcessedState;
log: LogFunction;
};
export type HiddenFileSyncDatabaseExtractionOperations = {
extract(path: FilePath, options?: HiddenFileSyncDatabaseExtractionOptions): Promise<boolean | undefined>;
extractRevision(path: FilePath, revision: string, force?: boolean): Promise<boolean>;
};
function log(
dependencies: Pick<HiddenFileSyncDatabaseExtractionDependencies, "log">,
message: unknown,
level?: LOG_LEVEL,
key?: string
): void {
dependencies.log(message, level, key);
}
export async function extractHiddenFileFromDatabase(
dependencies: HiddenFileSyncDatabaseExtractionDependencies,
storageFilePath: FilePath,
options: HiddenFileSyncDatabaseExtractionOptions = {}
): Promise<boolean | undefined> {
const {
force = false,
metaEntry,
preventDoubleProcess = true,
onlyNew = false,
includeDeletion = true,
requiredLiveRevision,
} = options;
const prefixedFileName = addPrefix(storageFilePath, ICHeader);
// Compatibility: admission happens outside the per-file lock, so ignore
// policy errors propagate instead of becoming a false extraction result.
if (await dependencies.isIgnoredByIgnoreFile(storageFilePath)) {
return undefined;
}
return await serialiseHiddenFileOperation(dependencies, prefixedFileName, async () => {
try {
// A caller-supplied Metadata entry is trusted as-is. It can change
// before this lock is acquired; only exact-revision repair performs
// an in-lock liveness check.
const metaOnDatabase = requiredLiveRevision
? await dependencies.loadLiveRevision(prefixedFileName, requiredLiveRevision)
: metaEntry
? metaEntry
: await dependencies.loadDatabaseMetadata(prefixedFileName);
// Compatibility: the exact-revision loader validates revision-tree
// membership but normally returns a leaf without `_conflicts`.
// Repair can therefore apply one live branch while ordinary
// reflection remains blocked by conflicted winning Metadata.
if (metaOnDatabase === false) {
throw new Error(`File not found on database.:${storageFilePath}`);
}
if (metaOnDatabase._conflicts?.length) {
log(
dependencies,
`Hidden file ${storageFilePath} has conflicted revisions, to keep in safe, writing to storage has been prevented`,
LOG_LEVEL_INFO
);
return false;
}
if (preventDoubleProcess) {
const key = dependencies.processedState.databaseStateKey(metaOnDatabase);
if (dependencies.processedState.getLastProcessedDatabaseKey(storageFilePath) == key && !force) {
// Compatibility question: the force suffix is unreachable
// because this branch is entered only when force is false.
log(
dependencies,
`STORAGE <-- DB: ${storageFilePath}: skipped (hidden, overwrite${force ? ", force" : ""}) (Previously processed)`
);
return undefined;
}
}
if (onlyNew) {
const databaseMTime = getHiddenFileSyncComparisonMTime(metaOnDatabase, includeDeletion);
const storageStat = await dependencies.statStorageFile(storageFilePath);
const storageMTimeActual = storageStat?.mtime ?? 0;
const storageMTime =
storageMTimeActual == 0
? dependencies.processedState.getLastProcessedFileMTime(storageFilePath)
: storageMTimeActual;
const difference = compareMTime(storageMTime, databaseMTime);
if (difference != TARGET_IS_NEW) {
log(
dependencies,
`STORAGE <-- DB: ${storageFilePath}: skipped (hidden, overwrite${force ? ", force" : ""}) (Not new)`
);
// Compatibility: a declined candidate is settled as
// processed, including a deletion excluded by
// includeDeletion. This prevents later scans retrying it.
dependencies.processedState.updateLastProcessedDatabase(storageFilePath, metaOnDatabase);
if (storageStat) dependencies.processedState.updateLastProcessedFile(storageFilePath, storageStat);
return undefined;
}
}
const deleted = metaOnDatabase.deleted || metaOnDatabase._deleted || false;
if (deleted) {
const result = await dependencies.deleteStorageFile(storageFilePath);
if (result == "OK") {
dependencies.processedState.updateLastProcessedDeletion(storageFilePath, metaOnDatabase);
return true;
}
if (result == "ALREADY") {
// Compatibility question: an already absent file updates
// only the database key. It does not record the missing
// storage key or call unmarkChanges through deletion state.
dependencies.processedState.updateLastProcessedDatabase(storageFilePath, metaOnDatabase);
return true;
}
return false;
}
const fileOnDatabase = await dependencies.loadDatabaseEntry(metaOnDatabase);
if (fileOnDatabase === false) {
throw new Error(`Failed to read file from database:${storageFilePath}`);
}
const resultStat = await dependencies.writeStorageFile(storageFilePath, fileOnDatabase, force);
if (resultStat) {
// Compatibility question: the storage writer also returns the
// existing stat when content is unchanged. That no-op still
// settles state and queues a configuration notification here.
dependencies.processedState.updateLastProcessed(storageFilePath, metaOnDatabase, resultStat);
dependencies.queueNotification(storageFilePath);
log(
dependencies,
`STORAGE <-- DB: ${storageFilePath}: written (hidden, overwrite${force ? ", force" : ""}) Done`
);
return true;
}
return false;
} catch (error) {
log(
dependencies,
`STORAGE <-- DB: ${storageFilePath}: written (hidden, overwrite${force ? ", force" : ""}) Failed`
);
log(dependencies, error, LOG_LEVEL_VERBOSE);
return false;
}
});
}
export async function extractHiddenFileRevisionFromDatabase(
dependencies: HiddenFileSyncDatabaseExtractionDependencies,
storageFilePath: FilePath,
revision: string,
force = false
): Promise<boolean> {
return Boolean(
await extractHiddenFileFromDatabase(dependencies, storageFilePath, {
force,
requiredLiveRevision: revision,
})
);
}
export function createHiddenFileSyncDatabaseExtractionOperations(
dependencies: HiddenFileSyncDatabaseExtractionDependencies
): HiddenFileSyncDatabaseExtractionOperations {
return Object.freeze({
extract: async (path, options) => await extractHiddenFileFromDatabase(dependencies, path, options),
extractRevision: async (path, revision, force) =>
await extractHiddenFileRevisionFromDatabase(dependencies, path, revision, force),
});
}
@@ -0,0 +1,366 @@
import { describe, expect, it, vi } from "vitest";
import {
LOG_LEVEL_INFO,
LOG_LEVEL_VERBOSE,
type DocumentID,
type FilePath,
type FilePathWithPrefix,
type LoadedEntry,
type MetaEntry,
type UXStat,
} from "@vrtmrz/livesync-commonlib/compat/common/types";
import type { HiddenFileSyncRemovalResult } from "./hiddenFileSyncStorage.ts";
vi.mock("@/deps.ts", () => ({}));
import {
createHiddenFileSyncDatabaseExtractionOperations,
extractHiddenFileFromDatabase,
extractHiddenFileRevisionFromDatabase,
type HiddenFileSyncDatabaseExtractionDependencies,
} from "./hiddenFileSyncDatabaseExtractionOperations.ts";
import { toHiddenFileSyncDatabaseStateKey } from "./hiddenFileSyncState.ts";
const path = ".obsidian/plugins/example/data.json" as FilePath;
const prefixedPath = `i:${path}` as FilePathWithPrefix;
const id = "hidden-entry-id" as DocumentID;
function metadata(overrides: Partial<MetaEntry> = {}): MetaEntry {
return {
_id: id,
_rev: "2-current",
path: prefixedPath,
type: "plain",
datatype: "plain",
ctime: 10,
mtime: 20,
size: 20,
children: [],
eden: {},
deleted: false,
...overrides,
} as unknown as MetaEntry;
}
function loadedEntry(entry = metadata()): LoadedEntry {
return {
...entry,
data: '{"value":"database"}',
} as LoadedEntry;
}
function storageStat(mtime = 20): UXStat {
return {
ctime: 11,
mtime,
size: 20,
type: "file",
};
}
function createDependencies(entry = metadata()) {
const events: string[] = [];
const serialiseFileOperation = vi.fn(async (_key: string, operation: () => Promise<unknown>) => {
events.push("lock:start");
try {
return await operation();
} finally {
events.push("lock:end");
}
});
const log = vi.fn();
const isIgnoredByIgnoreFile = vi.fn(async () => false);
const loadDatabaseMetadata = vi.fn(async () => entry as MetaEntry | false);
const loadLiveRevision = vi.fn(async (_path: FilePathWithPrefix, revision: string) =>
revision === entry._rev ? (entry as MetaEntry) : false
);
const loadDatabaseEntry = vi.fn(async () => loadedEntry(entry) as LoadedEntry | false);
const statStorageFile = vi.fn(async () => storageStat() as UXStat | null);
const writeStorageFile = vi.fn(async () => {
events.push("storage:write");
return storageStat(21) as UXStat | false;
});
const deleteStorageFile = vi.fn(async (): Promise<HiddenFileSyncRemovalResult> => {
events.push("storage:delete");
return "OK" as const;
});
const getLastProcessedDatabaseKey = vi.fn(() => undefined as string | undefined);
const getLastProcessedFileMTime = vi.fn(() => 0);
const updateLastProcessed = vi.fn(() => events.push("state:file"));
const updateLastProcessedDatabase = vi.fn(() => events.push("state:database"));
const updateLastProcessedFile = vi.fn(() => events.push("state:storage"));
const updateLastProcessedDeletion = vi.fn(() => events.push("state:deletion"));
const processedState = {
databaseStateKey: toHiddenFileSyncDatabaseStateKey,
getLastProcessedDatabaseKey,
getLastProcessedFileMTime,
updateLastProcessed,
updateLastProcessedDatabase,
updateLastProcessedFile,
updateLastProcessedDeletion,
};
const queueNotification = vi.fn(() => events.push("notification"));
const dependencies = {
serialiseFileOperation,
isIgnoredByIgnoreFile,
loadDatabaseMetadata,
loadLiveRevision,
loadDatabaseEntry,
statStorageFile,
writeStorageFile,
deleteStorageFile,
processedState,
queueNotification,
log,
} as HiddenFileSyncDatabaseExtractionDependencies;
return {
deleteStorageFile,
dependencies,
entry,
events,
getLastProcessedDatabaseKey,
getLastProcessedFileMTime,
isIgnoredByIgnoreFile,
loadDatabaseEntry,
loadDatabaseMetadata,
loadLiveRevision,
log,
queueNotification,
serialiseFileOperation,
statStorageFile,
updateLastProcessed,
updateLastProcessedDatabase,
updateLastProcessedDeletion,
updateLastProcessedFile,
writeStorageFile,
};
}
describe("hidden-file database-to-storage admission", () => {
it("returns undefined for an ignored path without taking the file lock", async () => {
const fixture = createDependencies();
fixture.isIgnoredByIgnoreFile.mockResolvedValue(true);
await expect(extractHiddenFileFromDatabase(fixture.dependencies, path)).resolves.toBeUndefined();
expect(fixture.serialiseFileOperation).not.toHaveBeenCalled();
expect(fixture.loadDatabaseMetadata).not.toHaveBeenCalled();
});
it("propagates ignore-policy errors before taking the guarded path", async () => {
const fixture = createDependencies();
const error = new Error("ignore policy unavailable");
fixture.isIgnoredByIgnoreFile.mockRejectedValue(error);
await expect(extractHiddenFileFromDatabase(fixture.dependencies, path)).rejects.toBe(error);
expect(fixture.serialiseFileOperation).not.toHaveBeenCalled();
expect(fixture.log).not.toHaveBeenCalled();
});
it("prevents a conflicted entry from reaching storage", async () => {
const entry = metadata({ _conflicts: ["2-conflict"] });
const fixture = createDependencies(entry);
await expect(extractHiddenFileFromDatabase(fixture.dependencies, path, { force: true })).resolves.toBe(false);
expect(fixture.loadDatabaseEntry).not.toHaveBeenCalled();
expect(fixture.writeStorageFile).not.toHaveBeenCalled();
expect(fixture.log).toHaveBeenCalledWith(
`Hidden file ${path} has conflicted revisions, to keep in safe, writing to storage has been prevented`,
LOG_LEVEL_INFO,
undefined
);
});
});
describe("hidden-file database-to-storage processed-state policy", () => {
it("skips a previously processed revision without settling state again", async () => {
const fixture = createDependencies();
fixture.getLastProcessedDatabaseKey.mockReturnValue(toHiddenFileSyncDatabaseStateKey(fixture.entry));
await expect(extractHiddenFileFromDatabase(fixture.dependencies, path)).resolves.toBeUndefined();
expect(fixture.loadDatabaseEntry).not.toHaveBeenCalled();
expect(fixture.updateLastProcessed).not.toHaveBeenCalled();
expect(fixture.log).toHaveBeenCalledWith(
`STORAGE <-- DB: ${path}: skipped (hidden, overwrite) (Previously processed)`,
undefined,
undefined
);
});
it("allows force to bypass the previously processed revision", async () => {
const fixture = createDependencies();
fixture.getLastProcessedDatabaseKey.mockReturnValue(toHiddenFileSyncDatabaseStateKey(fixture.entry));
await expect(extractHiddenFileFromDatabase(fixture.dependencies, path, { force: true })).resolves.toBe(true);
expect(fixture.writeStorageFile).toHaveBeenCalledWith(path, loadedEntry(fixture.entry), true);
});
it("settles both sides when onlyNew declines an equally old database entry", async () => {
const fixture = createDependencies();
await expect(
extractHiddenFileFromDatabase(fixture.dependencies, path, {
metaEntry: fixture.entry,
preventDoubleProcess: false,
onlyNew: true,
})
).resolves.toBeUndefined();
expect(fixture.loadDatabaseMetadata).not.toHaveBeenCalled();
expect(fixture.loadDatabaseEntry).not.toHaveBeenCalled();
expect(fixture.updateLastProcessedDatabase).toHaveBeenCalledWith(path, fixture.entry);
expect(fixture.updateLastProcessedFile).toHaveBeenCalledWith(path, storageStat());
expect(fixture.events).toEqual(["lock:start", "state:database", "state:storage", "lock:end"]);
});
it("uses the last known mtime when onlyNew sees a zero storage mtime", async () => {
const fixture = createDependencies(metadata({ mtime: 30 }));
fixture.statStorageFile.mockResolvedValue(storageStat(0));
fixture.getLastProcessedFileMTime.mockReturnValue(40);
await expect(
extractHiddenFileFromDatabase(fixture.dependencies, path, { onlyNew: true })
).resolves.toBeUndefined();
expect(fixture.getLastProcessedFileMTime).toHaveBeenCalledWith(path);
expect(fixture.writeStorageFile).not.toHaveBeenCalled();
});
});
describe("hidden-file database-to-storage application", () => {
it("settles state and queues a notification inside the file lock after a successful write", async () => {
const fixture = createDependencies();
await expect(extractHiddenFileFromDatabase(fixture.dependencies, path)).resolves.toBe(true);
expect(fixture.updateLastProcessed).toHaveBeenCalledWith(path, fixture.entry, storageStat(21));
expect(fixture.queueNotification).toHaveBeenCalledWith(path);
expect(fixture.serialiseFileOperation).toHaveBeenCalledWith(`file-${prefixedPath}`, expect.any(Function));
expect(fixture.events).toEqual(["lock:start", "storage:write", "state:file", "notification", "lock:end"]);
});
it("settles and notifies when the storage writer reports unchanged content with its existing stat", async () => {
const fixture = createDependencies();
fixture.writeStorageFile.mockResolvedValue(storageStat());
await expect(extractHiddenFileFromDatabase(fixture.dependencies, path)).resolves.toBe(true);
expect(fixture.updateLastProcessed).toHaveBeenCalledWith(path, fixture.entry, storageStat());
expect(fixture.queueNotification).toHaveBeenCalledWith(path);
});
it("returns false without settlement when the storage writer fails", async () => {
const fixture = createDependencies();
fixture.writeStorageFile.mockResolvedValue(false);
await expect(extractHiddenFileFromDatabase(fixture.dependencies, path)).resolves.toBe(false);
expect(fixture.updateLastProcessed).not.toHaveBeenCalled();
expect(fixture.queueNotification).not.toHaveBeenCalled();
});
it("records a successful storage deletion through deletion settlement", async () => {
const entry = metadata({ deleted: true });
const fixture = createDependencies(entry);
await expect(extractHiddenFileFromDatabase(fixture.dependencies, path)).resolves.toBe(true);
expect(fixture.updateLastProcessedDeletion).toHaveBeenCalledWith(path, entry);
expect(fixture.updateLastProcessedDatabase).not.toHaveBeenCalled();
expect(fixture.events).toEqual(["lock:start", "storage:delete", "state:deletion", "lock:end"]);
});
it("marks an already absent deleted file as database-processed only", async () => {
const entry = metadata({ deleted: true });
const fixture = createDependencies(entry);
fixture.deleteStorageFile.mockImplementation(async () => {
fixture.events.push("storage:delete");
return "ALREADY";
});
await expect(extractHiddenFileFromDatabase(fixture.dependencies, path)).resolves.toBe(true);
expect(fixture.updateLastProcessedDatabase).toHaveBeenCalledWith(path, entry);
expect(fixture.updateLastProcessedDeletion).not.toHaveBeenCalled();
expect(fixture.events).toEqual(["lock:start", "storage:delete", "state:database", "lock:end"]);
});
it("returns false without settling state when a storage deletion fails", async () => {
const fixture = createDependencies(metadata({ _deleted: true }));
fixture.deleteStorageFile.mockResolvedValue(false);
await expect(extractHiddenFileFromDatabase(fixture.dependencies, path)).resolves.toBe(false);
expect(fixture.updateLastProcessedDeletion).not.toHaveBeenCalled();
expect(fixture.updateLastProcessedDatabase).not.toHaveBeenCalled();
});
it("turns a database content-read failure into false and reports the inherited write diagnostic", async () => {
const fixture = createDependencies();
const error = new Error("content unavailable");
fixture.loadDatabaseEntry.mockRejectedValue(error);
await expect(extractHiddenFileFromDatabase(fixture.dependencies, path, { force: true })).resolves.toBe(false);
expect(fixture.log).toHaveBeenNthCalledWith(
1,
`STORAGE <-- DB: ${path}: written (hidden, overwrite, force) Failed`,
undefined,
undefined
);
expect(fixture.log).toHaveBeenNthCalledWith(2, error, LOG_LEVEL_VERBOSE, undefined);
});
});
describe("selected-revision hidden-file database-to-storage application", () => {
it("applies a selected live revision without reading ordinary Metadata", async () => {
const fixture = createDependencies();
await expect(
extractHiddenFileRevisionFromDatabase(fixture.dependencies, path, fixture.entry._rev!, true)
).resolves.toBe(true);
expect(fixture.loadLiveRevision).toHaveBeenCalledWith(prefixedPath, fixture.entry._rev);
expect(fixture.loadDatabaseMetadata).not.toHaveBeenCalled();
expect(fixture.writeStorageFile).toHaveBeenCalledWith(path, loadedEntry(fixture.entry), true);
});
it("can apply a selected live branch while ordinary Metadata reports a conflict", async () => {
const selected = metadata({ _rev: "2-selected", _conflicts: undefined });
const fixture = createDependencies(metadata({ _rev: "3-winner", _conflicts: [selected._rev!] }));
fixture.loadLiveRevision.mockResolvedValue(selected);
fixture.loadDatabaseEntry.mockResolvedValue(loadedEntry(selected));
await expect(
extractHiddenFileRevisionFromDatabase(fixture.dependencies, path, selected._rev!, true)
).resolves.toBe(true);
expect(fixture.loadDatabaseMetadata).not.toHaveBeenCalled();
expect(fixture.writeStorageFile).toHaveBeenCalledWith(path, loadedEntry(selected), true);
});
it("returns false when the selected revision ceased to be live", async () => {
const fixture = createDependencies();
fixture.loadLiveRevision.mockResolvedValue(false);
await expect(extractHiddenFileRevisionFromDatabase(fixture.dependencies, path, "2-stale", true)).resolves.toBe(
false
);
expect(fixture.loadDatabaseEntry).not.toHaveBeenCalled();
expect(fixture.writeStorageFile).not.toHaveBeenCalled();
});
it("exposes frozen operations with the exact-revision Boolean contract", async () => {
const fixture = createDependencies();
fixture.isIgnoredByIgnoreFile.mockResolvedValue(true);
const operations = createHiddenFileSyncDatabaseExtractionOperations(fixture.dependencies);
expect(Object.isFrozen(operations)).toBe(true);
await expect(operations.extractRevision(path, fixture.entry._rev!)).resolves.toBe(false);
});
});
@@ -0,0 +1,111 @@
import {
LOG_LEVEL_NOTICE,
LOG_LEVEL_VERBOSE,
type FilePath,
type FilePathWithPrefix,
type LoadedEntry,
type LOG_LEVEL,
type MetaEntry,
} from "@vrtmrz/livesync-commonlib/compat/common/types";
import type { DatabaseFileAccess } from "@vrtmrz/livesync-commonlib/compat/interfaces/DatabaseFileAccess";
import type { LiveSyncLocalDB } from "@vrtmrz/livesync-commonlib/compat/pouchdb/LiveSyncLocalDB";
import type { IPathService } from "@vrtmrz/livesync-commonlib/compat/services/base/IService";
import type { LogFunction } from "@vrtmrz/livesync-commonlib/compat/services/lib/logUtils";
import { addPrefix, stripAllPrefixes } from "@vrtmrz/livesync-commonlib/compat/string_and_binary/path";
import { ICHeader } from "@/common/types.ts";
type HiddenFileSyncLogDependency = {
log: LogFunction;
};
type HiddenFileSyncDatabaseMethods<Method extends keyof LiveSyncLocalDB> = {
getLocalDatabase(): Pick<LiveSyncLocalDB, Method>;
};
type HiddenFileSyncDatabaseFileAccessMethods<Method extends keyof DatabaseFileAccess> = {
databaseFileAccess: Pick<DatabaseFileAccess, Method>;
};
type HiddenFileSyncPathMethods<Method extends keyof IPathService> = {
path: Pick<IPathService, Method>;
};
export type HiddenFileSyncBaseEntryLoaderDependencies = HiddenFileSyncDatabaseMethods<"getDBEntry" | "getDBEntryMeta"> &
HiddenFileSyncPathMethods<"path2id"> &
HiddenFileSyncLogDependency;
export type HiddenFileSyncLiveRevisionLoaderDependencies = HiddenFileSyncDatabaseFileAccessMethods<
"fetchEntryMeta" | "getConflictedRevs"
> &
HiddenFileSyncLogDependency;
function log(dependencies: HiddenFileSyncLogDependency, message: unknown, level?: LOG_LEVEL, key?: string): void {
dependencies.log(message, level, key);
}
export async function loadHiddenFileSyncBaseEntry(
dependencies: HiddenFileSyncBaseEntryLoaderDependencies,
file: FilePath,
includeContent = true
): Promise<LoadedEntry | false> {
const prefixedFileName = addPrefix(file, ICHeader);
// Compatibility question: path-to-ID conversion is performed even when an
// entry already exists, and it sits outside the guarded database lookup.
// Preserve this ordering and error propagation until reviewed separately.
const id = await dependencies.path.path2id(prefixedFileName, ICHeader);
try {
const old = includeContent
? await dependencies.getLocalDatabase().getDBEntry(prefixedFileName, undefined, false, true)
: await dependencies.getLocalDatabase().getDBEntryMeta(prefixedFileName, { conflicts: true }, true);
if (old !== false) {
return old;
}
// Compatibility question: getDBEntry() also returns false when content
// or Chunks cannot be read. The inherited behaviour treats that exactly
// like absence and synthesises a fresh base entry.
return {
_id: id,
data: [],
path: prefixedFileName,
mtime: 0,
ctime: 0,
datatype: "newnote",
children: [],
size: 0,
deleted: false,
type: "newnote",
eden: {},
};
} catch (error) {
log(dependencies, "Getting base save data failed");
log(dependencies, error, LOG_LEVEL_VERBOSE);
return false;
}
}
export async function loadLiveHiddenFileSyncRevision(
dependencies: HiddenFileSyncLiveRevisionLoaderDependencies,
prefixedFileName: FilePathWithPrefix,
revision: string
): Promise<MetaEntry | false> {
const [selected, current, conflicts] = await Promise.all([
dependencies.databaseFileAccess.fetchEntryMeta(prefixedFileName, revision, true),
dependencies.databaseFileAccess.fetchEntryMeta(prefixedFileName, undefined, true),
dependencies.databaseFileAccess.getConflictedRevs(prefixedFileName),
]);
const liveRevisions = new Set([...(current && current._rev ? [current._rev] : []), ...conflicts]);
if (!selected || selected._rev !== revision || !liveRevisions.has(revision)) {
// Compatibility: missing, mismatched, and stale selections share the
// same user-facing diagnostic and false result.
log(
dependencies,
`Could not use hidden-file revision ${revision} of ${stripAllPrefixes(prefixedFileName)}; the selected revision is no longer live`,
LOG_LEVEL_NOTICE
);
return false;
}
// Compatibility: liveness is revision-tree membership only. A deleted
// Metadata leaf remains selectable while its revision is still live.
return selected;
}
@@ -0,0 +1,202 @@
import { describe, expect, it, vi } from "vitest";
import {
LOG_LEVEL_NOTICE,
LOG_LEVEL_VERBOSE,
type DocumentID,
type FilePath,
type FilePathWithPrefix,
type LoadedEntry,
type MetaEntry,
} from "@vrtmrz/livesync-commonlib/compat/common/types";
import { loadHiddenFileSyncBaseEntry, loadLiveHiddenFileSyncRevision } from "./hiddenFileSyncDatabaseLoaders.ts";
const path = ".obsidian/app.json" as FilePath;
const prefixedPath = `i:${path}` as FilePathWithPrefix;
const id = "hidden-entry-id" as DocumentID;
function loadedEntry(): LoadedEntry {
return {
_id: id,
_rev: "1-a",
path: prefixedPath,
type: "plain",
datatype: "plain",
data: "content",
ctime: 10,
mtime: 20,
size: 7,
children: [],
eden: {},
deleted: false,
} as LoadedEntry;
}
function createBaseEntryDependencies() {
const getDBEntry = vi.fn();
const getDBEntryMeta = vi.fn();
const path2id = vi.fn(async () => id);
const log = vi.fn();
const dependencies = {
getLocalDatabase: () => ({ getDBEntry, getDBEntryMeta }) as never,
path: { path2id } as never,
log,
};
return { dependencies, getDBEntry, getDBEntryMeta, log, path2id };
}
function metaEntry(revision: string): MetaEntry {
return {
...loadedEntry(),
_rev: revision,
data: undefined,
} as unknown as MetaEntry;
}
function createLiveRevisionDependencies() {
const selected = metaEntry("2-selected");
const current = metaEntry("3-current");
const fetchEntryMeta = vi.fn(async (_path: unknown, revision?: string) => {
if (revision === undefined || revision === current._rev) return current;
if (revision === selected._rev) return selected;
return false;
});
const getConflictedRevs = vi.fn(async () => [selected._rev!]);
const log = vi.fn();
const dependencies = {
databaseFileAccess: { fetchEntryMeta, getConflictedRevs } as never,
log,
};
return { current, dependencies, fetchEntryMeta, getConflictedRevs, log, selected };
}
describe("Hidden File Sync base-entry loader", () => {
it("synthesises a new empty base whenever the content lookup reports false", async () => {
const { dependencies, getDBEntry, path2id } = createBaseEntryDependencies();
getDBEntry.mockResolvedValue(false);
await expect(loadHiddenFileSyncBaseEntry(dependencies, path)).resolves.toEqual({
_id: id,
data: [],
path: prefixedPath,
mtime: 0,
ctime: 0,
datatype: "newnote",
children: [],
size: 0,
deleted: false,
type: "newnote",
eden: {},
});
expect(path2id).toHaveBeenCalledWith(prefixedPath, "i:");
expect(getDBEntry).toHaveBeenCalledWith(prefixedPath, undefined, false, true);
});
it("returns an existing content entry unchanged", async () => {
const { dependencies, getDBEntry, path2id } = createBaseEntryDependencies();
const existing = loadedEntry();
getDBEntry.mockResolvedValue(existing);
await expect(loadHiddenFileSyncBaseEntry(dependencies, path, true)).resolves.toBe(existing);
expect(path2id).toHaveBeenCalledWith(prefixedPath, "i:");
});
it("uses the conflict-aware metadata lookup when content is not requested", async () => {
const { dependencies, getDBEntry, getDBEntryMeta } = createBaseEntryDependencies();
const existing = loadedEntry();
getDBEntryMeta.mockResolvedValue(existing);
await expect(loadHiddenFileSyncBaseEntry(dependencies, path, false)).resolves.toBe(existing);
expect(getDBEntry).not.toHaveBeenCalled();
expect(getDBEntryMeta).toHaveBeenCalledWith(prefixedPath, { conflicts: true }, true);
});
it("turns a database lookup failure into a logged false result", async () => {
const { dependencies, getDBEntry, log } = createBaseEntryDependencies();
const error = new Error("database unavailable");
getDBEntry.mockRejectedValue(error);
await expect(loadHiddenFileSyncBaseEntry(dependencies, path)).resolves.toBe(false);
expect(log).toHaveBeenNthCalledWith(1, "Getting base save data failed", undefined, undefined);
expect(log).toHaveBeenNthCalledWith(2, error, LOG_LEVEL_VERBOSE, undefined);
});
it("propagates path-to-ID failures which occur before the guarded lookup", async () => {
const { dependencies, getDBEntry, log, path2id } = createBaseEntryDependencies();
const error = new Error("ID conversion failed");
path2id.mockRejectedValue(error);
await expect(loadHiddenFileSyncBaseEntry(dependencies, path)).rejects.toBe(error);
expect(getDBEntry).not.toHaveBeenCalled();
expect(log).not.toHaveBeenCalled();
});
});
describe("Hidden File Sync live-revision loader", () => {
it("accepts the current winning revision", async () => {
const { current, dependencies, fetchEntryMeta, getConflictedRevs } = createLiveRevisionDependencies();
await expect(loadLiveHiddenFileSyncRevision(dependencies, prefixedPath, current._rev!)).resolves.toBe(current);
expect(fetchEntryMeta).toHaveBeenNthCalledWith(1, prefixedPath, current._rev, true);
expect(fetchEntryMeta).toHaveBeenNthCalledWith(2, prefixedPath, undefined, true);
expect(getConflictedRevs).toHaveBeenCalledWith(prefixedPath);
});
it("accepts a selected conflict leaf while it remains live", async () => {
const { dependencies, selected } = createLiveRevisionDependencies();
await expect(loadLiveHiddenFileSyncRevision(dependencies, prefixedPath, selected._rev!)).resolves.toBe(
selected
);
});
it("accepts a live conflict even when there is no current winner", async () => {
const { dependencies, fetchEntryMeta, selected } = createLiveRevisionDependencies();
fetchEntryMeta.mockImplementation(async (_path: unknown, revision?: string) =>
revision === selected._rev ? selected : false
);
await expect(loadLiveHiddenFileSyncRevision(dependencies, prefixedPath, selected._rev!)).resolves.toBe(
selected
);
});
it("accepts deleted metadata while its revision remains live", async () => {
const { dependencies, fetchEntryMeta, selected } = createLiveRevisionDependencies();
const deleted = { ...selected, deleted: true } as MetaEntry;
fetchEntryMeta.mockImplementation(async (_path: unknown, revision?: string) =>
revision === undefined ? metaEntry("3-current") : deleted
);
await expect(loadLiveHiddenFileSyncRevision(dependencies, prefixedPath, selected._rev!)).resolves.toBe(deleted);
});
it("rejects a selected revision which is no longer current or conflicted", async () => {
const { dependencies, getConflictedRevs, log, selected } = createLiveRevisionDependencies();
getConflictedRevs.mockResolvedValue([]);
await expect(loadLiveHiddenFileSyncRevision(dependencies, prefixedPath, selected._rev!)).resolves.toBe(false);
expect(log).toHaveBeenCalledWith(
`Could not use hidden-file revision ${selected._rev} of ${path}; the selected revision is no longer live`,
LOG_LEVEL_NOTICE,
undefined
);
});
it("rejects a lookup result whose revision does not match the selection", async () => {
const { dependencies, fetchEntryMeta, log, selected } = createLiveRevisionDependencies();
fetchEntryMeta.mockResolvedValue(metaEntry("2-other"));
await expect(loadLiveHiddenFileSyncRevision(dependencies, prefixedPath, selected._rev!)).resolves.toBe(false);
expect(log).toHaveBeenCalledOnce();
});
it("propagates database-file-access failures", async () => {
const { dependencies, fetchEntryMeta, log, selected } = createLiveRevisionDependencies();
const error = new Error("revision lookup failed");
fetchEntryMeta.mockRejectedValue(error);
await expect(loadLiveHiddenFileSyncRevision(dependencies, prefixedPath, selected._rev!)).rejects.toBe(error);
expect(log).not.toHaveBeenCalled();
});
});
@@ -0,0 +1,337 @@
import {
LOG_LEVEL_NOTICE,
LOG_LEVEL_VERBOSE,
type FilePath,
type FilePathWithPrefix,
type LoadedEntry,
type LOG_LEVEL,
type MetaEntry,
type SavingEntry,
type UXFileInfo,
} from "@vrtmrz/livesync-commonlib/compat/common/types";
import { isDocContentSame, readAsBlob } from "@vrtmrz/livesync-commonlib/compat/common/utils";
import type { LogFunction } from "@vrtmrz/livesync-commonlib/compat/services/lib/logUtils";
import { addPrefix, stripAllPrefixes } from "@vrtmrz/livesync-commonlib/compat/string_and_binary/path";
import type { InternalFileInfo } from "@/common/types.ts";
import { ICHeader } from "@/common/types.ts";
import {
serialiseHiddenFileOperation,
type HiddenFileSyncFileSerialisationDependencies,
type HiddenFileSyncFileSerialiser,
} from "./hiddenFileSyncFileOperations.ts";
export type { HiddenFileSyncFileSerialiser } from "./hiddenFileSyncFileOperations.ts";
type HiddenFileSyncDatabaseWriteResponse = {
readonly ok: boolean;
readonly rev: string;
};
export type HiddenFileSyncDatabaseWriteProcessedState = {
updateLastProcessed(path: FilePath, databaseEntry: MetaEntry | LoadedEntry, storageFile: UXFileInfo["stat"]): void;
updateLastProcessedDeletion(path: FilePath, databaseEntry: MetaEntry | LoadedEntry | false): void;
};
export type HiddenFileSyncDatabaseWriteDependencies = {
serialiseFileOperation: HiddenFileSyncFileSerialiser;
isIgnoredByIgnoreFile(path: string): Promise<boolean>;
readFileWithInfo(path: FilePath): Promise<UXFileInfo>;
loadBaseEntry(path: FilePath): Promise<LoadedEntry | false>;
loadBaseMetadata(path: FilePath): Promise<LoadedEntry | false>;
loadLiveRevision(path: FilePathWithPrefix, revision: string): Promise<MetaEntry | false>;
fetchEntryFromMeta(meta: MetaEntry, waitForReady: boolean, skipCheck: boolean): Promise<LoadedEntry | false>;
storeWithBaseRevision(file: UXFileInfo, baseRevision: string, skipCheck: boolean): Promise<string | false>;
putDatabaseEntry(entry: SavingEntry): Promise<false | HiddenFileSyncDatabaseWriteResponse>;
putRaw(entry: LoadedEntry): Promise<HiddenFileSyncDatabaseWriteResponse>;
removeRevision(id: LoadedEntry["_id"], revision: string): Promise<boolean>;
processedState: HiddenFileSyncDatabaseWriteProcessedState;
now(): number;
log: LogFunction;
};
type HiddenFileSyncDatabaseWriteCommonDependencies = Pick<
HiddenFileSyncDatabaseWriteDependencies,
"isIgnoredByIgnoreFile" | "log" | "readFileWithInfo" | "serialiseFileOperation"
>;
export type StoreHiddenFileInDatabaseDependencies = HiddenFileSyncDatabaseWriteCommonDependencies &
Pick<HiddenFileSyncDatabaseWriteDependencies, "loadBaseEntry" | "putDatabaseEntry" | "processedState">;
export type StoreHiddenFileWithBaseRevisionDependencies = HiddenFileSyncDatabaseWriteCommonDependencies &
Pick<
HiddenFileSyncDatabaseWriteDependencies,
"fetchEntryFromMeta" | "loadLiveRevision" | "storeWithBaseRevision" | "processedState"
>;
export type DeleteHiddenFileFromDatabaseDependencies = Pick<
HiddenFileSyncDatabaseWriteDependencies,
| "isIgnoredByIgnoreFile"
| "loadBaseMetadata"
| "log"
| "now"
| "putRaw"
| "removeRevision"
| "serialiseFileOperation"
| "processedState"
>;
export type HiddenFileSyncDatabaseWriteOperations = {
store(file: InternalFileInfo | UXFileInfo, forceWrite?: boolean): Promise<boolean | undefined>;
storeWithBaseRevision(
file: InternalFileInfo | UXFileInfo,
baseRevision: string,
createIfDifferent?: boolean
): Promise<boolean>;
delete(path: FilePath, forceWrite?: boolean): Promise<boolean | undefined>;
};
type HiddenFileSyncDatabaseWriteGuardDependencies = HiddenFileSyncFileSerialisationDependencies &
Pick<HiddenFileSyncDatabaseWriteDependencies, "log">;
function log(
dependencies: Pick<HiddenFileSyncDatabaseWriteDependencies, "log">,
message: unknown,
level?: LOG_LEVEL,
key?: string
) {
dependencies.log(message, level, key);
}
async function runGuardedDatabaseWrite<Result>(
dependencies: HiddenFileSyncDatabaseWriteGuardDependencies,
prefixedFileName: FilePathWithPrefix,
failureMessage: string,
operation: () => Promise<Result>
): Promise<Result | false> {
return await serialiseHiddenFileOperation(dependencies, prefixedFileName, async () => {
try {
return await operation();
} catch (error) {
log(dependencies, failureMessage);
log(dependencies, error, LOG_LEVEL_VERBOSE);
return false;
}
});
}
async function resolvePresentFileInfo(
dependencies: Pick<HiddenFileSyncDatabaseWriteDependencies, "readFileWithInfo">,
file: InternalFileInfo | UXFileInfo,
storeFilePath: FilePath
): Promise<UXFileInfo> {
const fileInfo = "stat" in file && "body" in file ? file : await dependencies.readFileWithInfo(storeFilePath);
if (fileInfo.deleted) {
throw new Error(`Hidden file:${storeFilePath} is deleted. This should not be occurred.`);
}
return fileInfo;
}
export async function storeHiddenFileInDatabase(
dependencies: StoreHiddenFileInDatabaseDependencies,
file: InternalFileInfo | UXFileInfo,
forceWrite = false
): Promise<boolean | undefined> {
const storeFilePath = stripAllPrefixes(file.path);
const storageFilePath = file.path;
// Compatibility: all three admission checks sit outside the guarded lock,
// so policy errors propagate. The ordinary path reports an ignored file as
// undefined, while the selected-revision path reports false.
if (await dependencies.isIgnoredByIgnoreFile(storageFilePath)) {
return undefined;
}
const prefixedFileName = addPrefix(storeFilePath, ICHeader);
return await runGuardedDatabaseWrite(
dependencies,
prefixedFileName,
`STORAGE --> DB:${storageFilePath}: (hidden) Failed`,
async () => {
const fileInfo = await resolvePresentFileInfo(dependencies, file, storeFilePath);
const baseData = await dependencies.loadBaseEntry(storeFilePath);
if (baseData === false) throw new Error("Failed to load base data");
if (baseData._rev && !forceWrite) {
const isSame = await isDocContentSame(readAsBlob(baseData), fileInfo.body);
if (isSame) {
dependencies.processedState.updateLastProcessed(storeFilePath, baseData, fileInfo.stat);
return undefined;
}
}
const saveData: SavingEntry = {
...baseData,
data: fileInfo.body,
mtime: fileInfo.stat.mtime,
size: fileInfo.stat.size,
children: [],
deleted: false,
type: baseData.datatype,
};
// Compatibility question: ctime comes from the old database base,
// not from the storage stat. A newly synthesised base therefore
// stores ctime 0. Preserve this until cross-device effects are known.
const ret = await dependencies.putDatabaseEntry(saveData);
if (ret && ret.ok) {
saveData._rev = ret.rev;
dependencies.processedState.updateLastProcessed(storeFilePath, saveData, fileInfo.stat);
}
const success = ret && ret.ok;
log(dependencies, `STORAGE --> DB:${storageFilePath}: (hidden) ${success ? "Done" : "Failed"}`);
return success;
}
);
}
export async function storeHiddenFileWithBaseRevision(
dependencies: StoreHiddenFileWithBaseRevisionDependencies,
file: InternalFileInfo | UXFileInfo,
baseRevision: string,
createIfDifferent = true
): Promise<boolean> {
const storeFilePath = stripAllPrefixes(file.path);
const storageFilePath = file.path;
if (await dependencies.isIgnoredByIgnoreFile(storageFilePath)) {
return false;
}
const prefixedFileName = addPrefix(storeFilePath, ICHeader);
return await runGuardedDatabaseWrite(
dependencies,
prefixedFileName,
`STORAGE --> DB:${storageFilePath}: (hidden, selected branch) Failed`,
async () => {
// The live check intentionally precedes the storage read. It avoids
// work for a stale selection, but does not make the later write atomic.
const baseData = await dependencies.loadLiveRevision(prefixedFileName, baseRevision);
if (baseData === false) {
return false;
}
const fileInfo = await resolvePresentFileInfo(dependencies, file, storeFilePath);
if (!baseData.deleted && !baseData._deleted) {
const loadedBase = await dependencies.fetchEntryFromMeta(baseData, true, true);
if (loadedBase && (await isDocContentSame(readAsBlob(loadedBase), fileInfo.body))) {
dependencies.processedState.updateLastProcessed(storeFilePath, baseData, fileInfo.stat);
return true;
}
}
if (!createIfDifferent) {
log(
dependencies,
`Could not mark hidden file ${storeFilePath} as revision ${baseRevision}; the storage content differs`,
LOG_LEVEL_NOTICE
);
return false;
}
const storedRevision = await dependencies.storeWithBaseRevision(
{
...fileInfo,
path: storeFilePath,
name: fileInfo.name || storeFilePath.split("/").pop() || "",
isInternal: true,
},
baseRevision,
true
);
if (storedRevision === false) {
return false;
}
// Compatibility question: spreading a selected PouchDB tombstone
// retains `_deleted: true` in the processed-state entry even though
// `deleted` is reset. The stored child itself is created separately.
dependencies.processedState.updateLastProcessed(
storeFilePath,
{
...baseData,
_rev: storedRevision,
path: prefixedFileName,
ctime: fileInfo.stat.ctime,
mtime: fileInfo.stat.mtime,
size: fileInfo.stat.size,
deleted: false,
},
fileInfo.stat
);
log(dependencies, `STORAGE --> DB:${storageFilePath}: (hidden, selected branch) Done`);
return true;
}
);
}
export async function deleteHiddenFileFromDatabase(
dependencies: DeleteHiddenFileFromDatabaseDependencies,
filenameSrc: FilePath,
forceWrite = false
): Promise<boolean | undefined> {
const storeFilePath = filenameSrc;
const storageFilePath = filenameSrc;
const displayFileName = filenameSrc;
const prefixedFileName = addPrefix(storeFilePath, ICHeader);
// Compatibility question: the timestamp is captured before the ignore
// check and before waiting for the per-file lock.
const mtime = dependencies.now();
// Compatibility question: forceWrite is part of the inherited call
// contract, but it has never changed deletion behaviour.
void forceWrite;
if (await dependencies.isIgnoredByIgnoreFile(storageFilePath)) {
return undefined;
}
return await runGuardedDatabaseWrite(
dependencies,
prefixedFileName,
`STORAGE -x> DB: ${displayFileName}: (hidden) Failed`,
async () => {
const baseData = await dependencies.loadBaseMetadata(storeFilePath);
if (baseData === false) throw new Error("Failed to load base data during deleting");
if (baseData._conflicts !== undefined) {
// Compatibility question: these removals are sequential but not
// atomic. Earlier branches remain removed if a later call fails.
for (const conflictRev of baseData._conflicts) {
await dependencies.removeRevision(baseData._id, conflictRev);
log(
dependencies,
`STORAGE -x> DB: ${displayFileName}: (hidden) conflict removed ${baseData._rev} => ${conflictRev}`,
LOG_LEVEL_VERBOSE
);
}
}
// Compatibility question: only the domain `deleted` marker is
// checked here. `_deleted: true` alone causes another tombstone write.
if (baseData.deleted) {
log(dependencies, `STORAGE -x> DB: ${displayFileName}: (hidden) already deleted`, LOG_LEVEL_VERBOSE);
dependencies.processedState.updateLastProcessedDeletion(storeFilePath, baseData);
return true;
}
const saveData: LoadedEntry = {
...baseData,
mtime,
size: 0,
children: [],
deleted: true,
type: baseData.datatype,
};
// A synthesised base has no revision; the inherited behaviour still
// writes it as a tombstone when the requested path was absent.
const ret = await dependencies.putRaw(saveData);
if (ret && ret.ok) {
log(dependencies, `STORAGE -x> DB: ${displayFileName}: (hidden) Done`);
saveData._rev = ret.rev;
dependencies.processedState.updateLastProcessedDeletion(storeFilePath, saveData);
return true;
} else {
log(dependencies, `STORAGE -x> DB: ${displayFileName}: (hidden) Failed`);
return false;
}
}
);
}
export function createHiddenFileSyncDatabaseWriteOperations(
dependencies: HiddenFileSyncDatabaseWriteDependencies
): HiddenFileSyncDatabaseWriteOperations {
return Object.freeze({
store: async (file, forceWrite) => await storeHiddenFileInDatabase(dependencies, file, forceWrite),
storeWithBaseRevision: async (file, baseRevision, createIfDifferent) =>
await storeHiddenFileWithBaseRevision(dependencies, file, baseRevision, createIfDifferent),
delete: async (path, forceWrite) => await deleteHiddenFileFromDatabase(dependencies, path, forceWrite),
});
}
@@ -0,0 +1,372 @@
import { describe, expect, it, vi } from "vitest";
import {
LOG_LEVEL_NOTICE,
LOG_LEVEL_VERBOSE,
type DocumentID,
type FilePath,
type FilePathWithPrefix,
type LoadedEntry,
type MetaEntry,
type UXFileInfo,
} from "@vrtmrz/livesync-commonlib/compat/common/types";
import {
deleteHiddenFileFromDatabase,
storeHiddenFileInDatabase,
storeHiddenFileWithBaseRevision,
type HiddenFileSyncDatabaseWriteDependencies,
} from "./hiddenFileSyncDatabaseWriteOperations.ts";
const path = ".obsidian/plugins/example/data.json" as FilePath;
const prefixedPath = `i:${path}` as FilePathWithPrefix;
const id = "hidden-entry-id" as DocumentID;
function fileInfo(content = '{"value":"vault"}'): UXFileInfo {
return {
path,
name: "data.json",
isInternal: true,
body: new Blob([content]),
stat: {
ctime: 41,
mtime: 42,
size: content.length,
type: "file",
},
deleted: false,
} as UXFileInfo;
}
function loadedEntry(overrides: Partial<LoadedEntry> = {}): LoadedEntry {
return {
_id: id,
_rev: "2-current",
path: prefixedPath,
type: "plain",
datatype: "plain",
data: '{"value":"database"}',
ctime: 10,
mtime: 20,
size: 20,
children: [],
eden: {},
deleted: false,
...overrides,
} as LoadedEntry;
}
function createDependencies(base = loadedEntry()) {
const events: string[] = [];
const serialiseFileOperation = vi.fn(async (_key: string, operation: () => Promise<unknown>) => {
events.push("lock:start");
try {
return await operation();
} finally {
events.push("lock:end");
}
});
const isIgnoredByIgnoreFile = vi.fn(async () => false);
const readFileWithInfo = vi.fn(async () => fileInfo());
const loadBaseEntry = vi.fn(async () => base as LoadedEntry | false);
const loadBaseMetadata = vi.fn(async () => base as LoadedEntry | false);
const loadLiveRevision = vi.fn(async (_path: FilePathWithPrefix, revision: string) =>
revision === base._rev ? (base as MetaEntry) : false
);
const fetchEntryFromMeta = vi.fn(async () => base as LoadedEntry | false);
const storeWithBaseRevision = vi.fn(async () => "3-selected-child" as string | false);
const putDatabaseEntry = vi.fn(async (_entry: unknown) => ({ ok: true, id, rev: "3-written" }));
const putRaw = vi.fn(async (_entry: LoadedEntry) => ({ ok: true, id, rev: "3-deleted" }));
const removeRevision = vi.fn(async () => true);
const updateLastProcessed = vi.fn(() => events.push("state:file"));
const updateLastProcessedDeletion = vi.fn(() => events.push("state:deletion"));
const processedState = {
updateLastProcessed,
updateLastProcessedDeletion,
};
const now = vi.fn(() => 1_000);
const log = vi.fn();
const dependencies = {
serialiseFileOperation,
isIgnoredByIgnoreFile,
readFileWithInfo,
loadBaseEntry,
loadBaseMetadata,
loadLiveRevision,
fetchEntryFromMeta,
storeWithBaseRevision,
putDatabaseEntry,
putRaw,
removeRevision,
processedState,
now,
log,
} as unknown as HiddenFileSyncDatabaseWriteDependencies;
return {
base,
dependencies,
events,
fetchEntryFromMeta,
isIgnoredByIgnoreFile,
loadBaseEntry,
loadBaseMetadata,
loadLiveRevision,
log,
now,
putDatabaseEntry,
putRaw,
readFileWithInfo,
removeRevision,
serialiseFileOperation,
storeWithBaseRevision,
updateLastProcessed,
updateLastProcessedDeletion,
};
}
describe("ordinary hidden-file database writes", () => {
it("keeps the synthetic base ctime and settles the new revision inside the file lock", async () => {
const base = loadedEntry({
_rev: undefined,
ctime: 0,
data: [],
datatype: "newnote",
type: "newnote",
});
const fixture = createDependencies(base);
const file = fileInfo();
await expect(storeHiddenFileInDatabase(fixture.dependencies, file)).resolves.toBe(true);
expect(fixture.putDatabaseEntry).toHaveBeenCalledWith(
expect.objectContaining({ ctime: 0, mtime: file.stat.mtime, data: file.body })
);
expect(fixture.updateLastProcessed).toHaveBeenCalledWith(
path,
expect.objectContaining({ _rev: "3-written", ctime: 0 }),
file.stat
);
expect(fixture.serialiseFileOperation).toHaveBeenCalledWith(`file-${prefixedPath}`, expect.any(Function));
expect(fixture.events).toEqual(["lock:start", "state:file", "lock:end"]);
});
it("settles matching content without writing or emitting a transfer log", async () => {
const base = loadedEntry({ data: '{"value":"vault"}' });
const fixture = createDependencies(base);
const file = fileInfo();
await expect(storeHiddenFileInDatabase(fixture.dependencies, file)).resolves.toBeUndefined();
expect(fixture.putDatabaseEntry).not.toHaveBeenCalled();
expect(fixture.updateLastProcessed).toHaveBeenCalledWith(path, base, file.stat);
expect(fixture.log).not.toHaveBeenCalled();
});
it("writes matching content when forceWrite is enabled", async () => {
const fixture = createDependencies(loadedEntry({ data: '{"value":"vault"}' }));
await expect(storeHiddenFileInDatabase(fixture.dependencies, fileInfo(), true)).resolves.toBe(true);
expect(fixture.putDatabaseEntry).toHaveBeenCalledOnce();
});
it("returns false and reports both messages when a guarded write fails", async () => {
const fixture = createDependencies();
const error = new Error("storage read failed");
fixture.readFileWithInfo.mockRejectedValue(error);
await expect(
storeHiddenFileInDatabase(fixture.dependencies, {
path,
ctime: 1,
mtime: 2,
size: 3,
})
).resolves.toBe(false);
expect(fixture.log).toHaveBeenNthCalledWith(1, `STORAGE --> DB:${path}: (hidden) Failed`, undefined, undefined);
expect(fixture.log).toHaveBeenNthCalledWith(2, error, LOG_LEVEL_VERBOSE, undefined);
});
});
describe("selected-revision hidden-file database writes", () => {
it("stores the Vault content as a child of a selected live revision", async () => {
const fixture = createDependencies();
const file = fileInfo();
await expect(storeHiddenFileWithBaseRevision(fixture.dependencies, file, fixture.base._rev!)).resolves.toBe(
true
);
expect(fixture.storeWithBaseRevision).toHaveBeenCalledWith(
expect.objectContaining({ path, body: file.body, isInternal: true }),
fixture.base._rev,
true
);
expect(fixture.updateLastProcessed).toHaveBeenCalledWith(
path,
expect.objectContaining({ _rev: "3-selected-child" }),
file.stat
);
});
it("validates liveness before reading storage and refuses a stale revision", async () => {
const fixture = createDependencies();
fixture.loadLiveRevision.mockResolvedValue(false);
await expect(
storeHiddenFileWithBaseRevision(fixture.dependencies, { path, ctime: 1, mtime: 2, size: 3 }, "2-stale")
).resolves.toBe(false);
expect(fixture.readFileWithInfo).not.toHaveBeenCalled();
expect(fixture.storeWithBaseRevision).not.toHaveBeenCalled();
expect(fixture.updateLastProcessed).not.toHaveBeenCalled();
});
it("marks matching content without creating a child", async () => {
const base = loadedEntry({ data: '{"value":"vault"}' });
const fixture = createDependencies(base);
const file = fileInfo();
await expect(storeHiddenFileWithBaseRevision(fixture.dependencies, file, base._rev!, false)).resolves.toBe(
true
);
expect(fixture.storeWithBaseRevision).not.toHaveBeenCalled();
expect(fixture.updateLastProcessed).toHaveBeenCalledWith(path, base, file.stat);
});
it("reports differing content without creating a child when requested", async () => {
const fixture = createDependencies();
await expect(
storeHiddenFileWithBaseRevision(fixture.dependencies, fileInfo(), fixture.base._rev!, false)
).resolves.toBe(false);
expect(fixture.storeWithBaseRevision).not.toHaveBeenCalled();
expect(fixture.updateLastProcessed).not.toHaveBeenCalled();
expect(fixture.log).toHaveBeenCalledWith(
`Could not mark hidden file ${path} as revision ${fixture.base._rev}; the storage content differs`,
LOG_LEVEL_NOTICE,
undefined
);
});
it("keeps a selected branch's _deleted marker in the processed-state entry", async () => {
const base = loadedEntry({ deleted: true, _deleted: true });
const fixture = createDependencies(base);
const file = fileInfo();
await expect(storeHiddenFileWithBaseRevision(fixture.dependencies, file, base._rev!)).resolves.toBe(true);
expect(fixture.fetchEntryFromMeta).not.toHaveBeenCalled();
expect(fixture.updateLastProcessed).toHaveBeenCalledWith(
path,
expect.objectContaining({ _rev: "3-selected-child", deleted: false, _deleted: true }),
file.stat
);
});
});
describe("hidden-file database deletions", () => {
it("removes conflicts before accepting an already deleted entry", async () => {
const base = loadedEntry({ deleted: true, _conflicts: ["2-conflict"] });
const fixture = createDependencies(base);
await expect(deleteHiddenFileFromDatabase(fixture.dependencies, path, true)).resolves.toBe(true);
expect(fixture.removeRevision).toHaveBeenCalledWith(id, "2-conflict");
expect(fixture.putRaw).not.toHaveBeenCalled();
expect(fixture.updateLastProcessedDeletion).toHaveBeenCalledWith(path, base);
});
it("writes a deletion when only the PouchDB _deleted marker is present", async () => {
const base = loadedEntry({ deleted: false, _deleted: true });
const fixture = createDependencies(base);
await expect(deleteHiddenFileFromDatabase(fixture.dependencies, path)).resolves.toBe(true);
expect(fixture.putRaw).toHaveBeenCalledWith(expect.objectContaining({ deleted: true, _deleted: true }));
expect(fixture.updateLastProcessedDeletion).toHaveBeenCalledWith(
path,
expect.objectContaining({ _rev: "3-deleted" })
);
});
it("writes a tombstone for a synthetic base without a revision", async () => {
const base = loadedEntry({ _rev: undefined, data: [], datatype: "newnote", type: "newnote" });
const fixture = createDependencies(base);
let submitted: LoadedEntry | undefined;
fixture.putRaw.mockImplementation(async (entry: LoadedEntry) => {
submitted = { ...entry };
return { ok: true, id, rev: "3-deleted" };
});
await expect(deleteHiddenFileFromDatabase(fixture.dependencies, path)).resolves.toBe(true);
expect(submitted).toEqual(expect.objectContaining({ _rev: undefined, deleted: true, type: "newnote" }));
});
it("keeps earlier conflict removals when a later removal fails", async () => {
const base = loadedEntry({ _conflicts: ["2-first", "2-second"] });
const fixture = createDependencies(base);
const error = new Error("second removal failed");
fixture.removeRevision.mockResolvedValueOnce(true).mockRejectedValueOnce(error);
await expect(deleteHiddenFileFromDatabase(fixture.dependencies, path)).resolves.toBe(false);
expect(fixture.removeRevision.mock.calls).toEqual([
[id, "2-first"],
[id, "2-second"],
]);
expect(fixture.putRaw).not.toHaveBeenCalled();
expect(fixture.updateLastProcessedDeletion).not.toHaveBeenCalled();
expect(fixture.log).toHaveBeenLastCalledWith(error, LOG_LEVEL_VERBOSE, undefined);
});
it("captures the deletion time before evaluating ignore policy", async () => {
const fixture = createDependencies();
const events: string[] = [];
fixture.now.mockImplementation(() => {
events.push("now");
return 1_000;
});
fixture.isIgnoredByIgnoreFile.mockImplementation(async () => {
events.push("ignore");
return true;
});
await expect(deleteHiddenFileFromDatabase(fixture.dependencies, path)).resolves.toBeUndefined();
expect(events).toEqual(["now", "ignore"]);
expect(fixture.serialiseFileOperation).not.toHaveBeenCalled();
});
});
describe("hidden-file database write admission", () => {
it("preserves the different ignored results of the three write paths", async () => {
const fixture = createDependencies();
fixture.isIgnoredByIgnoreFile.mockResolvedValue(true);
const file = fileInfo();
await expect(storeHiddenFileInDatabase(fixture.dependencies, file)).resolves.toBeUndefined();
await expect(storeHiddenFileWithBaseRevision(fixture.dependencies, file, fixture.base._rev!)).resolves.toBe(
false
);
await expect(deleteHiddenFileFromDatabase(fixture.dependencies, path)).resolves.toBeUndefined();
expect(fixture.loadBaseEntry).not.toHaveBeenCalled();
expect(fixture.loadBaseMetadata).not.toHaveBeenCalled();
expect(fixture.loadLiveRevision).not.toHaveBeenCalled();
});
it("propagates ignore-policy failures before entering the guarded lock", async () => {
const fixture = createDependencies();
const error = new Error("ignore policy unavailable");
fixture.isIgnoredByIgnoreFile.mockRejectedValue(error);
await expect(storeHiddenFileInDatabase(fixture.dependencies, fileInfo())).rejects.toBe(error);
expect(fixture.serialiseFileOperation).not.toHaveBeenCalled();
expect(fixture.log).not.toHaveBeenCalled();
});
});
@@ -0,0 +1,18 @@
import type { FilePathWithPrefix } from "@vrtmrz/livesync-commonlib/compat/common/types";
export type HiddenFileSyncFileSerialiser = <Result>(key: string, operation: () => Promise<Result>) => Promise<Result>;
export type HiddenFileSyncFileSerialisationDependencies = {
serialiseFileOperation: HiddenFileSyncFileSerialiser;
};
export async function serialiseHiddenFileOperation<Result>(
dependencies: HiddenFileSyncFileSerialisationDependencies,
prefixedFileName: FilePathWithPrefix,
operation: () => Promise<Result>
): Promise<Result> {
// Compatibility question: this inherited lock uses `file-`, whereas the
// Commonlib database writer uses `file:`. The two writers therefore do not
// mutually exclude one another; changing the key needs concurrency tests.
return await dependencies.serialiseFileOperation(`file-${prefixedFileName}`, operation);
}
@@ -0,0 +1,22 @@
import type { CustomRegExp } from "@vrtmrz/livesync-commonlib/compat/common/utils";
export type HiddenFileSyncPathFilters = {
ignoreFilter: readonly CustomRegExp[];
targetFilter: readonly CustomRegExp[];
};
export function isHiddenFileSyncPath(path: string): boolean {
// Compatibility: this prefix check also excludes names such as `.trashcan`.
// Keep the broader exclusion until a separate path-policy decision changes it.
return path.startsWith(".") && !path.startsWith(".trash");
}
export function matchesHiddenFileSyncPatterns(path: string, filters: HiddenFileSyncPathFilters): boolean {
if (filters.ignoreFilter.some((pattern) => pattern.test(path))) {
return false;
}
if (filters.targetFilter.length > 0) {
return filters.targetFilter.some((pattern) => pattern.test(path));
}
return true;
}
@@ -0,0 +1,41 @@
import { describe, expect, it, vi } from "vitest";
import type { CustomRegExp } from "@vrtmrz/livesync-commonlib/compat/common/utils";
import { isHiddenFileSyncPath, matchesHiddenFileSyncPatterns } from "./hiddenFileSyncPathPolicy.ts";
const pattern = (matches: (path: string) => boolean) => ({ test: vi.fn(matches) }) as unknown as CustomRegExp;
describe("isHiddenFileSyncPath", () => {
it.each([
[".obsidian/app.json", true],
[".git/config", true],
[".trash/app.json", false],
[".trashcan/app.json", false],
["notes/.hidden", false],
])("classifies %s as a Hidden File Sync path=%s", (path, expected) => {
expect(isHiddenFileSyncPath(path)).toBe(expected);
});
});
describe("matchesHiddenFileSyncPatterns", () => {
it("allows every path when no filters are configured", () => {
expect(matchesHiddenFileSyncPatterns(".obsidian/app.json", { ignoreFilter: [], targetFilter: [] })).toBe(true);
});
it("uses target patterns as an allow-list", () => {
const targetFilter = [pattern((path) => path.endsWith(".json"))];
expect(matchesHiddenFileSyncPatterns(".obsidian/app.json", { ignoreFilter: [], targetFilter })).toBe(true);
expect(matchesHiddenFileSyncPatterns(".obsidian/theme.css", { ignoreFilter: [], targetFilter })).toBe(false);
});
it("gives ignore patterns precedence over target patterns", () => {
const matchesEverything = pattern(() => true);
expect(
matchesHiddenFileSyncPatterns(".obsidian/app.json", {
ignoreFilter: [matchesEverything],
targetFilter: [matchesEverything],
})
).toBe(false);
});
});
@@ -0,0 +1,202 @@
import {
LOG_LEVEL_VERBOSE,
type FilePath,
type LoadedEntry,
type LOG_LEVEL,
type MetaEntry,
type UXStat,
} from "@vrtmrz/livesync-commonlib/compat/common/types";
import type { KeyValueDatabase } from "@vrtmrz/livesync-commonlib/compat/interfaces/KeyValueDatabase";
import type { LiveSyncLocalDB } from "@vrtmrz/livesync-commonlib/compat/pouchdb/LiveSyncLocalDB";
import type { IPathService } from "@vrtmrz/livesync-commonlib/compat/services/base/IService";
import type { LogFunction } from "@vrtmrz/livesync-commonlib/compat/services/lib/logUtils";
import { addPrefix } from "@vrtmrz/livesync-commonlib/compat/string_and_binary/path";
import { ICHeader } from "@/common/types.ts";
import { autosaveCache, type MapLike } from "@/common/utils.ts";
import {
getHiddenFileSyncComparisonMTime,
toHiddenFileSyncDatabaseStateKey,
toHiddenFileSyncStorageStateKey,
} from "./hiddenFileSyncState.ts";
type HiddenFileSyncProcessedStateDatabase = Pick<LiveSyncLocalDB, "getDBEntryMeta">;
type HiddenFileSyncProcessedStateStorage = {
statHidden(path: FilePath): Promise<UXStat | null>;
};
export type HiddenFileSyncProcessedStateDependencies = {
getKeyValueDatabase(): KeyValueDatabase;
getLocalDatabase(): HiddenFileSyncProcessedStateDatabase;
storageAccess: HiddenFileSyncProcessedStateStorage;
path: Pick<IPathService, "markChangesAreSame" | "unmarkChanges">;
log: LogFunction;
};
export class HiddenFileSyncProcessedState {
private fileInfoLastProcessed: MapLike<string, string> = new Map();
private fileInfoLastKnown: MapLike<string, number> = new Map();
private databaseInfoLastProcessed: MapLike<string, string> = new Map();
constructor(private readonly dependencies: HiddenFileSyncProcessedStateDependencies) {}
private log(message: unknown, level?: LOG_LEVEL, key?: string): void {
this.dependencies.log(message, level, key);
}
async initialise(): Promise<void> {
// Compatibility question: these reads intentionally remain
// sequential and in this order.
// Compatibility question: autosaveCache has no flush or disposal
// boundary, so a delayed write from an earlier database lifecycle can
// outlive this state owner. Preserve that timing until database
// replacement and unload behaviour have focused coverage.
this.fileInfoLastProcessed = await autosaveCache(
this.dependencies.getKeyValueDatabase(),
"hidden-file-lastProcessed"
);
this.databaseInfoLastProcessed = await autosaveCache(
this.dependencies.getKeyValueDatabase(),
"hidden-file-lastProcessed-database"
);
this.fileInfoLastKnown = await autosaveCache(this.dependencies.getKeyValueDatabase(), "hidden-file-lastKnown");
}
getLastProcessedFileCount(): number {
return this.fileInfoLastProcessed.size;
}
getLastProcessedFileKeys(): IterableIterator<string> {
return this.fileInfoLastProcessed.keys();
}
hasLastProcessedFile(file: FilePath): boolean {
return this.fileInfoLastProcessed.has(file);
}
hasLastProcessedDatabase(file: FilePath): boolean {
return this.databaseInfoLastProcessed.has(file);
}
async fileToStatKey(file: FilePath, stat: UXStat | null = null): Promise<string> {
// Compatibility question: `null` means 'stat not supplied' here
// rather than 'file missing', so a failed earlier stat causes another
// read. Keep that retry until its event-processing effect is
// characterised.
if (!stat) stat = await this.dependencies.storageAccess.statHidden(file);
return this.storageStateKey(stat);
}
storageStateKey(stat: UXStat | null): string {
return toHiddenFileSyncStorageStateKey(stat);
}
databaseStateKey(doc: MetaEntry | LoadedEntry): string {
return toHiddenFileSyncDatabaseStateKey(doc);
}
updateLastProcessedFile(file: FilePath, keySrc: string | UXStat): void {
const key = typeof keySrc == "string" ? keySrc : this.storageStateKey(keySrc);
const splitted = key.split("-");
if (splitted[0] != "0") {
// Compatibility: a zero storage marker does not replace the last
// known non-zero mtime. Deletion therefore retains that fallback.
this.fileInfoLastKnown.set(file, Number(splitted[0]));
}
this.fileInfoLastProcessed.set(file, key);
}
async updateLastProcessedAsActualFile(file: FilePath, stat?: UXStat | null): Promise<void> {
if (!stat) stat = await this.dependencies.storageAccess.statHidden(file);
// Compatibility: adoption updates only the processed marker. It does
// not update the last-known non-zero mtime cache.
this.fileInfoLastProcessed.set(file, this.storageStateKey(stat));
}
resetLastProcessedFile(targetFiles: FilePath[] | false): void {
if (targetFiles) {
for (const key of targetFiles) {
this.fileInfoLastProcessed.delete(key);
}
} else {
this.log(`Delete all processed mark.`, LOG_LEVEL_VERBOSE);
// THINKING: Should we...
// - delete all `Known file` processed mark? (This is current implementation)
// - delete all `Existing file` processed mark?
// - delete all files inside the config folder of current device mark?
this.fileInfoLastProcessed.clear();
}
}
getLastProcessedFileMTime(file: FilePath): number {
const key = this.fileInfoLastKnown.get(file);
if (!key) return 0;
return key;
}
getLastProcessedFileKey(file: FilePath): string | undefined {
return this.fileInfoLastProcessed.get(file);
}
getLastProcessedDatabaseKey(file: FilePath): string | undefined {
return this.databaseInfoLastProcessed.get(file);
}
updateLastProcessedDatabase(file: FilePath, keySrc: string | MetaEntry | LoadedEntry): void {
const key = typeof keySrc == "string" ? keySrc : this.databaseStateKey(keySrc);
this.databaseInfoLastProcessed.set(file, key);
}
updateLastProcessed(path: FilePath, db: MetaEntry | LoadedEntry, stat: UXStat): void {
this.updateLastProcessedDatabase(path, db);
this.updateLastProcessedFile(path, this.storageStateKey(stat));
const dbMTime = getHiddenFileSyncComparisonMTime(db);
const storageMTime = getHiddenFileSyncComparisonMTime(stat);
if (dbMTime == 0 || storageMTime == 0) {
this.dependencies.path.unmarkChanges(path);
} else {
this.dependencies.path.markChangesAreSame(
path,
getHiddenFileSyncComparisonMTime(db),
getHiddenFileSyncComparisonMTime(stat)
);
}
}
updateLastProcessedDeletion(path: FilePath, db: MetaEntry | LoadedEntry | false): void {
this.dependencies.path.unmarkChanges(path);
if (db) this.updateLastProcessedDatabase(path, db);
this.updateLastProcessedFile(path, this.storageStateKey(null));
}
async updateLastProcessedAsActualDatabase(
file: FilePath,
doc?: MetaEntry | LoadedEntry | null | false
): Promise<void> {
const dbPath = addPrefix(file, ICHeader);
if (!doc) doc = await this.dependencies.getLocalDatabase().getDBEntryMeta(dbPath);
if (!doc) return;
this.databaseInfoLastProcessed.set(file, this.databaseStateKey(doc));
}
resetLastProcessedDatabase(targetFiles: FilePath[] | false): void {
if (targetFiles) {
for (const key of targetFiles) {
this.databaseInfoLastProcessed.delete(key);
}
} else {
this.log(`Delete all processed mark.`, LOG_LEVEL_VERBOSE);
// THINKING: Should we...
// - delete all `Known file` processed mark? (This is current implementation)
// - delete all `Existing file` processed mark?
// - delete all files inside the config folder of current device mark?
this.databaseInfoLastProcessed.clear();
}
}
}
export function createHiddenFileSyncProcessedState(
dependencies: HiddenFileSyncProcessedStateDependencies
): HiddenFileSyncProcessedState {
return new HiddenFileSyncProcessedState(dependencies);
}
@@ -0,0 +1,182 @@
import { describe, expect, it, vi } from "vitest";
import {
LOG_LEVEL_VERBOSE,
type FilePath,
type LoadedEntry,
type MetaEntry,
type UXStat,
} from "@vrtmrz/livesync-commonlib/compat/common/types";
import type { KeyValueDatabase } from "@vrtmrz/livesync-commonlib/compat/interfaces/KeyValueDatabase";
vi.mock("@/deps.ts", () => ({}));
import {
createHiddenFileSyncProcessedState,
type HiddenFileSyncProcessedStateDependencies,
} from "./hiddenFileSyncProcessedState.ts";
import { toHiddenFileSyncDatabaseStateKey } from "./hiddenFileSyncState.ts";
const path = ".obsidian/plugins/example/data.json" as FilePath;
function stat(mtime: number, size = 20): UXStat {
return { ctime: mtime, mtime, size, type: "file" };
}
function metadata(overrides: Partial<MetaEntry> = {}): MetaEntry {
return {
_id: "hidden-entry-id",
_rev: "2-current",
path: `i:${path}`,
type: "plain",
datatype: "plain",
ctime: 10,
mtime: 20,
size: 20,
children: [],
eden: {},
deleted: false,
...overrides,
} as unknown as MetaEntry;
}
function createState() {
const events: string[] = [];
const caches = new Map<string, Map<unknown, unknown>>([
["hidden-file-lastProcessed", new Map([[path, "40-20"]])],
["hidden-file-lastProcessed-database", new Map([[path, "20-20-2-current--1"]])],
["hidden-file-lastKnown", new Map([[path, 40]])],
]);
let activeReads = 0;
let maximumActiveReads = 0;
const keyValueDatabase = {
get: vi.fn(async (key: IDBValidKey) => {
events.push(`get:${String(key)}`);
activeReads++;
maximumActiveReads = Math.max(maximumActiveReads, activeReads);
await Promise.resolve();
activeReads--;
return caches.get(String(key));
}),
set: vi.fn(async () => "ok"),
} as unknown as KeyValueDatabase;
const getDBEntryMeta = vi.fn(async () => false as false | LoadedEntry);
const statHidden = vi.fn(async () => stat(41));
const markChangesAreSame = vi.fn(() => undefined);
const unmarkChanges = vi.fn();
const log = vi.fn();
const dependencies: HiddenFileSyncProcessedStateDependencies = {
getKeyValueDatabase: () => keyValueDatabase,
getLocalDatabase: () => ({ getDBEntryMeta }),
storageAccess: { statHidden },
path: { markChangesAreSame, unmarkChanges },
log,
};
const state = createHiddenFileSyncProcessedState(dependencies);
return {
dependencies,
events,
getDBEntryMeta,
keyValueDatabase,
log,
markChangesAreSame,
maximumActiveReads: () => maximumActiveReads,
state,
statHidden,
unmarkChanges,
};
}
describe("Hidden File Sync processed state", () => {
it("loads the three autosave caches sequentially under their existing keys", async () => {
const fixture = createState();
await fixture.state.initialise();
expect(fixture.events).toEqual([
"get:hidden-file-lastProcessed",
"get:hidden-file-lastProcessed-database",
"get:hidden-file-lastKnown",
]);
expect(fixture.maximumActiveReads()).toBe(1);
expect(fixture.state.getLastProcessedFileCount()).toBe(1);
expect(fixture.state.getLastProcessedFileKey(path)).toBe("40-20");
expect(fixture.state.getLastProcessedDatabaseKey(path)).toBe("20-20-2-current--1");
expect(fixture.state.getLastProcessedFileMTime(path)).toBe(40);
});
it("re-reads a null storage stat and retains the last known mtime on deletion", async () => {
const fixture = createState();
await fixture.state.initialise();
fixture.statHidden.mockResolvedValueOnce(stat(45, 3));
await expect(fixture.state.fileToStatKey(path, null)).resolves.toBe("45-3");
expect(fixture.statHidden).toHaveBeenCalledWith(path);
fixture.state.updateLastProcessedFile(path, stat(45, 3));
fixture.state.updateLastProcessedDeletion(path, metadata({ mtime: 50, size: 0 }));
expect(fixture.state.getLastProcessedFileKey(path)).toBe("0-0");
expect(fixture.state.getLastProcessedFileMTime(path)).toBe(45);
expect(fixture.state.getLastProcessedDatabaseKey(path)).toBe(
toHiddenFileSyncDatabaseStateKey(metadata({ mtime: 50, size: 0 }))
);
expect(fixture.unmarkChanges).toHaveBeenCalledWith(path);
});
it("settles combined state before applying the matching-path marker", async () => {
const fixture = createState();
await fixture.state.initialise();
const document = metadata({ mtime: 60, size: 9 });
const storageStat = stat(61, 9);
fixture.state.updateLastProcessed(path, document, storageStat);
expect(fixture.markChangesAreSame).toHaveBeenCalledWith(path, 60, 61);
expect(fixture.unmarkChanges).not.toHaveBeenCalled();
expect(fixture.state.getLastProcessedDatabaseKey(path)).toBe(toHiddenFileSyncDatabaseStateKey(document));
expect(fixture.state.getLastProcessedFileKey(path)).toBe("61-9");
});
it("resets each processed side without clearing last-known storage mtimes", async () => {
const fixture = createState();
await fixture.state.initialise();
fixture.state.updateLastProcessedFile(path, stat(70, 4));
fixture.state.updateLastProcessedDatabase(path, "database-key");
fixture.state.resetLastProcessedFile([path]);
expect(fixture.state.getLastProcessedFileKey(path)).toBeUndefined();
expect(fixture.state.getLastProcessedDatabaseKey(path)).toBe("database-key");
expect(fixture.state.getLastProcessedFileMTime(path)).toBe(70);
fixture.state.resetLastProcessedDatabase([path]);
expect(fixture.state.getLastProcessedDatabaseKey(path)).toBeUndefined();
});
it("does not settle a database marker for a false or missing database document", async () => {
const fixture = createState();
await fixture.state.initialise();
await fixture.state.updateLastProcessedAsActualDatabase(path, false);
expect(fixture.getDBEntryMeta).toHaveBeenCalledWith(`i:${path}`);
expect(fixture.state.getLastProcessedDatabaseKey(path)).toBe("20-20-2-current--1");
fixture.getDBEntryMeta.mockResolvedValueOnce(false);
await fixture.state.updateLastProcessedAsActualDatabase(path);
expect(fixture.state.getLastProcessedDatabaseKey(path)).toBe("20-20-2-current--1");
});
it("logs and clears both sides when a full reset is requested", async () => {
const fixture = createState();
await fixture.state.initialise();
fixture.state.resetLastProcessedFile(false);
fixture.state.resetLastProcessedDatabase(false);
expect(fixture.log).toHaveBeenNthCalledWith(1, "Delete all processed mark.", LOG_LEVEL_VERBOSE, undefined);
expect(fixture.log).toHaveBeenNthCalledWith(2, "Delete all processed mark.", LOG_LEVEL_VERBOSE, undefined);
expect(fixture.state.getLastProcessedFileCount()).toBe(0);
expect(fixture.state.getLastProcessedFileKey(path)).toBeUndefined();
expect(fixture.state.getLastProcessedDatabaseKey(path)).toBeUndefined();
expect(fixture.state.getLastProcessedFileMTime(path)).toBe(40);
});
});
@@ -0,0 +1,48 @@
import type {
FilePathWithPrefix,
LoadedEntry,
MetaEntry,
UXFileInfo,
UXStat,
} from "@vrtmrz/livesync-commonlib/compat/common/types";
import { displayRev } from "@vrtmrz/livesync-commonlib/compat/common/utils";
import { stripAllPrefixes } from "@vrtmrz/livesync-commonlib/compat/string_and_binary/path";
export function toHiddenFileSyncStorageStateKey(stat: UXStat | null): string {
return `${stat?.mtime ?? 0}-${stat?.size ?? 0}`;
}
export function toHiddenFileSyncDatabaseStateKey(doc: LoadedEntry | MetaEntry): string {
// Compatibility: the deletion marker includes its own hyphen, producing `--0`
// or `--1` after the revision. Existing device-local state uses this format.
return `${doc.mtime}-${doc.size}-${doc._rev}-${doc._deleted || doc.deleted || false ? "-0" : "-1"}`;
}
export function getHiddenFileSyncComparisonMTime(
source: MetaEntry | LoadedEntry | UXFileInfo | UXStat | false | null | undefined,
includeDeleted = false
): number {
if (source === null || source === false || source === undefined) return 0;
if (!includeDeleted) {
if ("deleted" in source && source.deleted) return 0;
if ("_deleted" in source && source._deleted) return 0;
}
if ("stat" in source) return source.stat?.mtime ?? 0;
return source.mtime ?? 0;
}
export function describeHiddenFileSyncDocument(doc: LoadedEntry, prefixedPath: FilePathWithPrefix) {
const id = doc._id;
const path = stripAllPrefixes(prefixedPath);
const rev = doc._rev;
return {
id,
rev,
revDisplay: rev ? displayRev(rev) : "0-NOREVS",
prefixedPath,
path,
isDeleted: doc._deleted || doc.deleted || false,
shortenedId: id.substring(0, 10),
shortenedPath: path.substring(0, 10),
};
}
@@ -0,0 +1,70 @@
import { describe, expect, it } from "vitest";
import type { FilePathWithPrefix, LoadedEntry, UXStat } from "@vrtmrz/livesync-commonlib/compat/common/types";
import {
describeHiddenFileSyncDocument,
getHiddenFileSyncComparisonMTime,
toHiddenFileSyncDatabaseStateKey,
toHiddenFileSyncStorageStateKey,
} from "./hiddenFileSyncState.ts";
describe("Hidden File Sync state keys", () => {
it("represents a missing storage file with zero values", () => {
expect(toHiddenFileSyncStorageStateKey(null)).toBe("0-0");
});
it("uses storage modification time and size", () => {
expect(toHiddenFileSyncStorageStateKey({ mtime: 123, size: 456 } as UXStat)).toBe("123-456");
});
it.each([
[false, "123-456-3-example--1"],
[true, "123-456-3-example--0"],
])("includes database revision and deletion state=%s", (deleted, expected) => {
const doc = { mtime: 123, size: 456, _rev: "3-example", deleted } as LoadedEntry;
expect(toHiddenFileSyncDatabaseStateKey(doc)).toBe(expected);
});
});
describe("getHiddenFileSyncComparisonMTime", () => {
const absentSources = [null, false, undefined] as const;
it.each(absentSources)("returns zero for an absent source=%s", (source) => {
expect(getHiddenFileSyncComparisonMTime(source)).toBe(0);
});
it("reads a direct stat or a file-info stat", () => {
expect(getHiddenFileSyncComparisonMTime({ mtime: 123 } as UXStat)).toBe(123);
expect(getHiddenFileSyncComparisonMTime({ stat: { mtime: 456 } } as never)).toBe(456);
});
it("treats deleted entries as zero unless deletion time is requested", () => {
const deleted = { mtime: 123, deleted: true } as LoadedEntry;
expect(getHiddenFileSyncComparisonMTime(deleted)).toBe(0);
expect(getHiddenFileSyncComparisonMTime(deleted, true)).toBe(123);
});
});
describe("describeHiddenFileSyncDocument", () => {
it("derives the unprefixed path and diagnostic revision fields", () => {
const doc = {
_id: "0123456789abcdef",
_rev: "3-example",
mtime: 123,
size: 456,
deleted: true,
} as LoadedEntry;
expect(describeHiddenFileSyncDocument(doc, "i:.obsidian/app.json" as FilePathWithPrefix)).toEqual({
id: "0123456789abcdef",
rev: "3-example",
revDisplay: "3-exampl",
prefixedPath: "i:.obsidian/app.json",
path: ".obsidian/app.json",
isDeleted: true,
shortenedId: "0123456789",
shortenedPath: ".obsidian/",
});
});
});
@@ -0,0 +1,210 @@
import {
LOG_LEVEL_DEBUG,
LOG_LEVEL_VERBOSE,
type FilePath,
type LoadedEntry,
type LOG_LEVEL,
type UXDataWriteOptions,
type UXFileInfo,
type UXStat,
} from "@vrtmrz/livesync-commonlib/compat/common/types";
import type { StorageAccess } from "@vrtmrz/livesync-commonlib/compat/interfaces/StorageAccess";
import type { LogFunction } from "@vrtmrz/livesync-commonlib/compat/services/lib/logUtils";
import {
createBlob,
isDocContentSame,
readContent,
} from "@vrtmrz/livesync-commonlib/compat/common/utils";
export type HiddenFileSyncStorageAccess = Pick<
StorageAccess,
| "ensureDir"
| "isExistsIncludeHidden"
| "readHiddenFileAuto"
| "removeHidden"
| "statHidden"
| "triggerHiddenFile"
| "writeHiddenFileAuto"
>;
type HiddenFileSyncStorageMethods<Method extends keyof HiddenFileSyncStorageAccess> = {
storageAccess: Pick<HiddenFileSyncStorageAccess, Method>;
};
type HiddenFileSyncLogDependency = {
log: LogFunction;
};
export type HiddenFileSyncStorageDependencies = HiddenFileSyncStorageMethods<keyof HiddenFileSyncStorageAccess> &
HiddenFileSyncLogDependency;
export type HiddenFileSyncRemovalResult = "OK" | "ALREADY" | false;
function log(
dependencies: HiddenFileSyncLogDependency,
message: unknown,
level?: LOG_LEVEL,
key?: string
): void {
dependencies.log(message, level, key);
}
export async function readHiddenFileWithInfo(
dependencies: HiddenFileSyncStorageMethods<"readHiddenFileAuto" | "statHidden">,
path: FilePath
): Promise<UXFileInfo> {
const stat = await dependencies.storageAccess.statHidden(path);
if (!stat) {
return {
name: path.split("/").pop() ?? "",
path,
stat: {
size: 0,
mtime: 0,
ctime: 0,
type: "file",
},
isInternal: true,
deleted: true,
body: createBlob(new Uint8Array(0)),
};
}
const content = await dependencies.storageAccess.readHiddenFileAuto(path);
return {
name: path.split("/").pop() ?? "",
path,
stat,
isInternal: true,
deleted: false,
body: createBlob(content),
};
}
export async function ensureHiddenFileDirectory(
dependencies: HiddenFileSyncStorageMethods<"ensureDir" | "isExistsIncludeHidden">,
path: FilePath
): Promise<void> {
if (!(await dependencies.storageAccess.isExistsIncludeHidden(path))) {
// StorageAccess expects the complete target path and ensures its parent.
await dependencies.storageAccess.ensureDir(path);
}
}
export async function writeHiddenFile(
dependencies: HiddenFileSyncStorageMethods<"statHidden" | "writeHiddenFileAuto">,
path: FilePath,
data: string | ArrayBuffer,
options?: UXDataWriteOptions
): Promise<UXStat | null> {
// Compatibility: the writer's Boolean result is ignored. The post-write stat
// has historically decided whether the operation produced a usable file.
await dependencies.storageAccess.writeHiddenFileAuto(path, data, options);
return await dependencies.storageAccess.statHidden(path);
}
export async function removeHiddenFile(
dependencies: HiddenFileSyncStorageMethods<"isExistsIncludeHidden" | "removeHidden"> &
HiddenFileSyncLogDependency,
path: FilePath
): Promise<HiddenFileSyncRemovalResult> {
try {
if (!(await dependencies.storageAccess.isExistsIncludeHidden(path))) {
return "ALREADY";
}
if (await dependencies.storageAccess.removeHidden(path)) {
return "OK";
}
} catch (error) {
log(dependencies, `Failed to remove file:${path}`);
log(dependencies, error, LOG_LEVEL_VERBOSE);
}
return false;
}
export async function triggerHiddenFileEvent(
dependencies: HiddenFileSyncStorageMethods<"triggerHiddenFile"> & HiddenFileSyncLogDependency,
path: FilePath
): Promise<void> {
try {
await dependencies.storageAccess.triggerHiddenFile(path);
} catch (error) {
log(dependencies, "Failed to call internal API(reconcileInternalFile)", LOG_LEVEL_VERBOSE);
log(dependencies, error, LOG_LEVEL_VERBOSE);
}
}
export async function isHiddenFileWriteRequired(
dependencies: HiddenFileSyncStorageMethods<"readHiddenFileAuto"> & HiddenFileSyncLogDependency,
path: FilePath,
content: string | ArrayBuffer
): Promise<boolean> {
try {
const storageContent = await dependencies.storageAccess.readHiddenFileAuto(path);
return !(await isDocContentSame(storageContent, content));
} catch (error) {
log(dependencies, `Cannot check the content of ${path}`);
log(dependencies, error, LOG_LEVEL_VERBOSE);
// Compatibility: an unreadable current file is treated as requiring a
// write. Changing this policy needs a separate recovery decision.
return true;
}
}
export async function writeHiddenFileFromDatabase(
dependencies: HiddenFileSyncStorageMethods<
"ensureDir" | "isExistsIncludeHidden" | "readHiddenFileAuto" | "statHidden" | "writeHiddenFileAuto"
> &
HiddenFileSyncLogDependency,
path: FilePath,
fileOnDatabase: LoadedEntry,
force: boolean
): Promise<false | UXStat> {
try {
const statBefore = await dependencies.storageAccess.statHidden(path);
const isExisting = statBefore != null;
const content = readContent(fileOnDatabase);
await ensureHiddenFileDirectory(dependencies, path);
const writeRequired =
force || !isExisting || (isExisting && (await isHiddenFileWriteRequired(dependencies, path, content)));
if (!writeRequired) {
log(dependencies, `STORAGE <-- DB: ${path}: skipped (hidden) Not changed`, LOG_LEVEL_DEBUG);
return statBefore;
}
const statAfter = await writeHiddenFile(dependencies, path, content, {
mtime: fileOnDatabase.mtime,
ctime: fileOnDatabase.ctime,
});
if (statAfter == null) {
log(dependencies, `STORAGE <-- DB: ${path}: written (hidden,new${force ? ", force" : ""}) Failed (writeResult)`);
return false;
}
log(dependencies, `STORAGE <-- DB: ${path}: written (hidden, overwrite${force ? ", force" : ""})`);
// Compatibility question: ordinary database reflection does not trigger
// a raw storage event here; deletion and manual JSON merging do. Preserve
// this until the event-loop consequences of changing it are characterised.
return statAfter;
} catch (error) {
log(dependencies, `STORAGE <-- DB: ${path}: written (hidden, overwrite${force ? ", force" : ""}) Failed`);
log(dependencies, error, LOG_LEVEL_VERBOSE);
return false;
}
}
export async function deleteHiddenFileFromStorage(
dependencies: HiddenFileSyncStorageMethods<"isExistsIncludeHidden" | "removeHidden" | "triggerHiddenFile"> &
HiddenFileSyncLogDependency,
path: FilePath
): Promise<HiddenFileSyncRemovalResult> {
const result = await removeHiddenFile(dependencies, path);
if (result === false) {
log(dependencies, `STORAGE <x- DB: ${path}: deleting (hidden) Failed`);
return false;
}
if (result === "OK") {
await triggerHiddenFileEvent(dependencies, path);
}
log(dependencies, `STORAGE <x- DB: ${path}: deleting (hidden) ${result == "OK" ? "Done" : "Already not found"}`);
return result;
}
@@ -0,0 +1,230 @@
import { describe, expect, it, vi } from "vitest";
import {
LOG_LEVEL_VERBOSE,
type FilePath,
type LoadedEntry,
type UXStat,
} from "@vrtmrz/livesync-commonlib/compat/common/types";
import {
deleteHiddenFileFromStorage,
ensureHiddenFileDirectory,
isHiddenFileWriteRequired,
readHiddenFileWithInfo,
removeHiddenFile,
triggerHiddenFileEvent,
writeHiddenFile,
writeHiddenFileFromDatabase,
} from "./hiddenFileSyncStorage.ts";
const path = ".obsidian/plugins/example/data.json" as FilePath;
const stat = { ctime: 10, mtime: 20, size: 4, type: "file" } as UXStat;
function createStorageDependencies() {
const storageAccess = {
ensureDir: vi.fn(async () => true),
isExistsIncludeHidden: vi.fn(async () => true),
readHiddenFileAuto: vi.fn(async () => "data" as string | ArrayBuffer),
removeHidden: vi.fn(async () => true),
statHidden: vi.fn(async () => stat as UXStat | null),
triggerHiddenFile: vi.fn(async () => undefined),
writeHiddenFileAuto: vi.fn(async () => true),
};
const log = vi.fn();
return { dependencies: { storageAccess, log }, log, storageAccess };
}
function databaseEntry(content: string, mtime = 30, ctime = 15): LoadedEntry {
return {
path,
type: "plain",
datatype: "plain",
data: content,
mtime,
ctime,
} as LoadedEntry;
}
describe("Hidden File Sync storage operations", () => {
it("represents a missing hidden file as a deleted empty file", async () => {
const { dependencies, storageAccess } = createStorageDependencies();
storageAccess.statHidden.mockResolvedValue(null);
const result = await readHiddenFileWithInfo(dependencies, path);
expect(result).toMatchObject({
name: "data.json",
path,
isInternal: true,
deleted: true,
stat: { ctime: 0, mtime: 0, size: 0, type: "file" },
});
expect(await result.body.text()).toBe("");
expect(storageAccess.readHiddenFileAuto).not.toHaveBeenCalled();
});
it("loads an existing hidden file with its storage stat", async () => {
const { dependencies, storageAccess } = createStorageDependencies();
storageAccess.readHiddenFileAuto.mockResolvedValue("data");
const result = await readHiddenFileWithInfo(dependencies, path);
expect(result).toMatchObject({ name: "data.json", path, isInternal: true, deleted: false, stat });
expect(await result.body.text()).toBe("data");
});
it("ensures a directory only when the target does not exist", async () => {
const { dependencies, storageAccess } = createStorageDependencies();
await ensureHiddenFileDirectory(dependencies, path);
expect(storageAccess.ensureDir).not.toHaveBeenCalled();
storageAccess.isExistsIncludeHidden.mockResolvedValue(false);
await ensureHiddenFileDirectory(dependencies, path);
expect(storageAccess.ensureDir).toHaveBeenCalledOnce();
expect(storageAccess.ensureDir).toHaveBeenCalledWith(path);
});
it("writes a hidden file and returns the resulting stat", async () => {
const { dependencies, storageAccess } = createStorageDependencies();
await expect(writeHiddenFile(dependencies, path, "data", { mtime: 30, ctime: 15 })).resolves.toBe(stat);
expect(storageAccess.writeHiddenFileAuto).toHaveBeenCalledWith(path, "data", { mtime: 30, ctime: 15 });
expect(storageAccess.statHidden).toHaveBeenCalledWith(path);
});
it("uses the post-write stat even when the storage writer reports false", async () => {
const { dependencies, storageAccess } = createStorageDependencies();
storageAccess.writeHiddenFileAuto.mockResolvedValue(false);
await expect(writeHiddenFile(dependencies, path, "data")).resolves.toBe(stat);
expect(storageAccess.statHidden).toHaveBeenCalledAfter(storageAccess.writeHiddenFileAuto);
});
it("distinguishes an absent, removed, and unremovable file", async () => {
const { dependencies, storageAccess } = createStorageDependencies();
storageAccess.isExistsIncludeHidden.mockResolvedValueOnce(false);
await expect(removeHiddenFile(dependencies, path)).resolves.toBe("ALREADY");
expect(storageAccess.removeHidden).not.toHaveBeenCalled();
storageAccess.isExistsIncludeHidden.mockResolvedValueOnce(true);
storageAccess.removeHidden.mockResolvedValueOnce(true);
await expect(removeHiddenFile(dependencies, path)).resolves.toBe("OK");
storageAccess.isExistsIncludeHidden.mockResolvedValueOnce(true);
storageAccess.removeHidden.mockResolvedValueOnce(false);
await expect(removeHiddenFile(dependencies, path)).resolves.toBe(false);
});
it("turns a removal error into a logged false result", async () => {
const { dependencies, log, storageAccess } = createStorageDependencies();
const error = new Error("remove failed");
storageAccess.isExistsIncludeHidden.mockRejectedValue(error);
await expect(removeHiddenFile(dependencies, path)).resolves.toBe(false);
expect(log).toHaveBeenNthCalledWith(1, `Failed to remove file:${path}`, undefined, undefined);
expect(log).toHaveBeenNthCalledWith(2, error, LOG_LEVEL_VERBOSE, undefined);
});
it("treats a content-read failure as requiring a write", async () => {
const { dependencies, log, storageAccess } = createStorageDependencies();
const error = new Error("read failed");
storageAccess.readHiddenFileAuto.mockRejectedValue(error);
await expect(isHiddenFileWriteRequired(dependencies, path, "data")).resolves.toBe(true);
expect(log).toHaveBeenNthCalledWith(1, `Cannot check the content of ${path}`, undefined, undefined);
expect(log).toHaveBeenNthCalledWith(2, error, LOG_LEVEL_VERBOSE, undefined);
});
it("compares binary content without involving the context", async () => {
const { dependencies, storageAccess } = createStorageDependencies();
storageAccess.readHiddenFileAuto.mockResolvedValue(new Uint8Array([1, 2, 3]).buffer);
await expect(
isHiddenFileWriteRequired(dependencies, path, new Uint8Array([1, 2, 3]).buffer)
).resolves.toBe(false);
await expect(
isHiddenFileWriteRequired(dependencies, path, new Uint8Array([1, 2, 4]).buffer)
).resolves.toBe(true);
});
it("skips an unchanged database file and preserves its current stat", async () => {
const { dependencies, storageAccess } = createStorageDependencies();
storageAccess.readHiddenFileAuto.mockResolvedValue("data");
await expect(writeHiddenFileFromDatabase(dependencies, path, databaseEntry("data"), false)).resolves.toBe(
stat
);
expect(storageAccess.ensureDir).not.toHaveBeenCalled();
expect(storageAccess.writeHiddenFileAuto).not.toHaveBeenCalled();
});
it("writes changed database content with its original timestamps", async () => {
const { dependencies, storageAccess } = createStorageDependencies();
storageAccess.readHiddenFileAuto.mockResolvedValue("old");
await expect(writeHiddenFileFromDatabase(dependencies, path, databaseEntry("new"), false)).resolves.toBe(
stat
);
expect(storageAccess.writeHiddenFileAuto).toHaveBeenCalledWith(path, "new", { mtime: 30, ctime: 15 });
expect(storageAccess.triggerHiddenFile).not.toHaveBeenCalled();
});
it("forces a write without reading existing content", async () => {
const { dependencies, storageAccess } = createStorageDependencies();
await expect(writeHiddenFileFromDatabase(dependencies, path, databaseEntry("data"), true)).resolves.toBe(stat);
expect(storageAccess.readHiddenFileAuto).not.toHaveBeenCalled();
expect(storageAccess.writeHiddenFileAuto).toHaveBeenCalledOnce();
});
it("returns false when a completed write has no resulting stat", async () => {
const { dependencies, storageAccess } = createStorageDependencies();
storageAccess.statHidden.mockResolvedValue(null);
await expect(writeHiddenFileFromDatabase(dependencies, path, databaseEntry("data"), false)).resolves.toBe(
false
);
expect(storageAccess.writeHiddenFileAuto).toHaveBeenCalledOnce();
});
it("triggers a storage event only after an actual deletion", async () => {
const { dependencies, log, storageAccess } = createStorageDependencies();
storageAccess.isExistsIncludeHidden.mockResolvedValueOnce(false);
await expect(deleteHiddenFileFromStorage(dependencies, path)).resolves.toBe("ALREADY");
expect(storageAccess.triggerHiddenFile).not.toHaveBeenCalled();
log.mockClear();
storageAccess.isExistsIncludeHidden.mockResolvedValueOnce(true);
storageAccess.removeHidden.mockResolvedValueOnce(true);
await expect(deleteHiddenFileFromStorage(dependencies, path)).resolves.toBe("OK");
expect(storageAccess.triggerHiddenFile).toHaveBeenCalledOnce();
expect(storageAccess.triggerHiddenFile).toHaveBeenCalledWith(path);
expect(storageAccess.triggerHiddenFile).toHaveBeenCalledAfter(storageAccess.removeHidden);
expect(log).toHaveBeenCalledAfter(storageAccess.triggerHiddenFile);
});
it("does not trigger a storage event after a failed deletion", async () => {
const { dependencies, storageAccess } = createStorageDependencies();
storageAccess.removeHidden.mockResolvedValue(false);
await expect(deleteHiddenFileFromStorage(dependencies, path)).resolves.toBe(false);
expect(storageAccess.triggerHiddenFile).not.toHaveBeenCalled();
});
it("swallows and logs storage-event failures", async () => {
const { dependencies, log, storageAccess } = createStorageDependencies();
const error = new Error("event failed");
storageAccess.triggerHiddenFile.mockRejectedValue(error);
await expect(triggerHiddenFileEvent(dependencies, path)).resolves.toBeUndefined();
expect(log).toHaveBeenNthCalledWith(
1,
"Failed to call internal API(reconcileInternalFile)",
LOG_LEVEL_VERBOSE,
undefined
);
expect(log).toHaveBeenNthCalledWith(2, error, LOG_LEVEL_VERBOSE, undefined);
});
});
@@ -1,5 +1,12 @@
import type { InternalFileInfo } from "@/common/types.ts"; import type { InternalFileInfo } from "@/common/types.ts";
import type { FilePath } from "@vrtmrz/livesync-commonlib/compat/common/types"; import type {
FilePath,
FilePathWithPrefix,
LoadedEntry,
UXFileInfo,
} from "@vrtmrz/livesync-commonlib/compat/common/types";
import type { OptionalSyncFeatureMode } from "@/features/optionalSyncFeatures.ts";
import type { HiddenFileSyncConflictTestingView } from "./hiddenFileSyncConflictResolution.ts";
export type HiddenFileSyncInitialisationDirection = "push" | "pull" | "safe" | "pullForce" | "pushForce"; export type HiddenFileSyncInitialisationDirection = "push" | "pull" | "safe" | "pullForce" | "pushForce";
@@ -12,6 +19,91 @@ export interface HiddenFileSyncInitialisationView {
): Promise<void>; ): Promise<void>;
} }
/**
* Semantic callbacks consumed by the Commonlib service registries.
*
* The context owns the implementations, while the optional-file composition
* adapts this view to Commonlib's aggregation contracts. Callers do not bind
* the context or depend on registry-oriented method names.
*/
export interface HiddenFileSyncServiceHandlerView {
readonly processOptionalFileEvent: (path: FilePath) => Promise<boolean>;
readonly processOptionalSyncFiles: (doc: LoadedEntry) => Promise<boolean>;
readonly onSettingLoaded: () => Promise<boolean>;
readonly realiseSettingSyncMode: () => Promise<boolean>;
readonly onResuming: () => Promise<boolean>;
readonly beforeReplicate: (showNotice: boolean) => Promise<boolean>;
readonly onDatabaseInitialised: (showNotice: boolean) => Promise<boolean>;
readonly suspendExtraSync: () => Promise<boolean>;
readonly configureOptionalSyncFeature: (mode: OptionalSyncFeatureMode) => Promise<boolean>;
readonly isTargetFileEligible: (path: FilePath) => Promise<boolean>;
readonly queueConflict: (path: FilePathWithPrefix) => Promise<boolean>;
}
export function createHiddenFileSyncServiceHandlerView(
operations: HiddenFileSyncServiceHandlerView
): HiddenFileSyncServiceHandlerView {
const view: HiddenFileSyncServiceHandlerView = {
processOptionalFileEvent: async (path) => await operations.processOptionalFileEvent(path),
processOptionalSyncFiles: async (doc) => await operations.processOptionalSyncFiles(doc),
onSettingLoaded: async () => await operations.onSettingLoaded(),
realiseSettingSyncMode: async () => await operations.realiseSettingSyncMode(),
onResuming: async () => await operations.onResuming(),
beforeReplicate: async (showNotice) => await operations.beforeReplicate(showNotice),
onDatabaseInitialised: async (showNotice) => await operations.onDatabaseInitialised(showNotice),
suspendExtraSync: async () => await operations.suspendExtraSync(),
configureOptionalSyncFeature: async (mode) => await operations.configureOptionalSyncFeature(mode),
isTargetFileEligible: async (path) => await operations.isTargetFileEligible(path),
queueConflict: async (path) => await operations.queueConflict(path),
};
return Object.freeze(view);
}
export type HiddenFileSyncTestingRebuild = (
showNotice: boolean,
targetFiles?: FilePath[] | false
) => Promise<FilePath[]>;
export type HiddenFileSyncTestingRebuildInterceptor = (
runRebuild: HiddenFileSyncTestingRebuild,
showNotice: boolean,
targetFiles?: FilePath[] | false
) => Promise<FilePath[]>;
/** Operations exposed to the real-Obsidian contract tests. */
export interface HiddenFileSyncTestingView extends HiddenFileSyncCommandView {
readonly conflictResolution: HiddenFileSyncConflictTestingView;
readFileWithInfo(path: FilePath): Promise<UXFileInfo>;
showConfigurationChangeNotice(updatedFolders: readonly string[]): void;
interceptRebuildMerging(interceptor: HiddenFileSyncTestingRebuildInterceptor): () => void;
}
export type HiddenFileSyncTestingViewOperations = HiddenFileSyncTestingView;
/**
* Build the frozen testing seam. Tests can observe behaviour and install a
* scoped timing interceptor, but cannot access mutable context state.
*/
export function createHiddenFileSyncTestingView(
operations: HiddenFileSyncTestingViewOperations
): HiddenFileSyncTestingView {
const view: HiddenFileSyncTestingView = {
isManualCommandAvailable: () => operations.isManualCommandAvailable(),
scanAllStorageChanges: async (showNotice) => await operations.scanAllStorageChanges(showNotice),
scanAllDatabaseChanges: async (showNotice) => await operations.scanAllDatabaseChanges(showNotice),
applyOfflineChanges: async (showNotice) => await operations.applyOfflineChanges(showNotice),
updateSettingCache: () => operations.updateSettingCache(),
initialiseInternalFileSync: async (direction, showMessage, targetFiles) =>
await operations.initialiseInternalFileSync(direction, showMessage, targetFiles),
conflictResolution: operations.conflictResolution,
readFileWithInfo: async (path) => await operations.readFileWithInfo(path),
showConfigurationChangeNotice: (updatedFolders) =>
operations.showConfigurationChangeNotice(updatedFolders),
interceptRebuildMerging: (interceptor) => operations.interceptRebuildMerging(interceptor),
};
return Object.freeze(view);
}
/** Exact-revision operations needed by the Hatch repair pane. */ /** Exact-revision operations needed by the Hatch repair pane. */
export interface HiddenFileSyncRepairView { export interface HiddenFileSyncRepairView {
scanInternalFiles(): Promise<InternalFileInfo[]>; scanInternalFiles(): Promise<InternalFileInfo[]>;
@@ -28,6 +120,19 @@ export interface HiddenFileSyncRepairView {
): Promise<boolean>; ): Promise<boolean>;
} }
export function createHiddenFileSyncRepairView(operations: HiddenFileSyncRepairView): HiddenFileSyncRepairView {
const view: HiddenFileSyncRepairView = {
scanInternalFiles: async () => await operations.scanInternalFiles(),
storeInternalFileToDatabase: async (file, forceWrite) =>
await operations.storeInternalFileToDatabase(file, forceWrite),
storeInternalFileToDatabaseWithBaseRevision: async (file, baseRevision, createIfDifferent) =>
await operations.storeInternalFileToDatabaseWithBaseRevision(file, baseRevision, createIfDifferent),
extractInternalFileRevisionFromDatabase: async (storageFilePath, revision, force) =>
await operations.extractInternalFileRevisionFromDatabase(storageFilePath, revision, force),
};
return Object.freeze(view);
}
/** Operations consumed by the host-owned Hidden File Sync commands. */ /** Operations consumed by the host-owned Hidden File Sync commands. */
export interface HiddenFileSyncCommandView extends HiddenFileSyncInitialisationView { export interface HiddenFileSyncCommandView extends HiddenFileSyncInitialisationView {
isManualCommandAvailable(): boolean; isManualCommandAvailable(): boolean;
@@ -0,0 +1,132 @@
import { describe, expect, it, vi } from "vitest";
import type { FilePath } from "@vrtmrz/livesync-commonlib/compat/common/types";
import {
createHiddenFileSyncRepairView,
createHiddenFileSyncServiceHandlerView,
createHiddenFileSyncTestingView,
type HiddenFileSyncRepairView,
type HiddenFileSyncServiceHandlerView,
type HiddenFileSyncTestingViewOperations,
} from "./hiddenFileSyncViews.ts";
describe("Hidden File Sync repair view", () => {
it("exposes only frozen repair operations and preserves their receiver", async () => {
const path = ".obsidian/app.json" as FilePath;
const file = { path, ctime: 1, mtime: 2, size: 3 };
const source = {
marker: "source",
scanInternalFiles: vi.fn(async function (this: { marker: string }) {
expect(this.marker).toBe("source");
return [file];
}),
storeInternalFileToDatabase: vi.fn(async function (this: { marker: string }) {
expect(this.marker).toBe("source");
return true;
}),
storeInternalFileToDatabaseWithBaseRevision: vi.fn(async function (this: { marker: string }) {
expect(this.marker).toBe("source");
return true;
}),
extractInternalFileRevisionFromDatabase: vi.fn(async function (this: { marker: string }) {
expect(this.marker).toBe("source");
return true;
}),
} as unknown as HiddenFileSyncRepairView;
const view = createHiddenFileSyncRepairView(source);
expect(view).not.toBe(source);
expect(Object.isFrozen(view)).toBe(true);
expect(Object.keys(view).sort()).toEqual(
[
"extractInternalFileRevisionFromDatabase",
"scanInternalFiles",
"storeInternalFileToDatabase",
"storeInternalFileToDatabaseWithBaseRevision",
].sort()
);
await expect(view.scanInternalFiles()).resolves.toEqual([file]);
await expect(view.storeInternalFileToDatabase(file)).resolves.toBe(true);
await expect(view.storeInternalFileToDatabaseWithBaseRevision(file, "2-selected", false)).resolves.toBe(true);
await expect(view.extractInternalFileRevisionFromDatabase(path, "2-selected", true)).resolves.toBe(true);
expect(source.storeInternalFileToDatabaseWithBaseRevision).toHaveBeenCalledWith(file, "2-selected", false);
});
});
describe("Hidden File Sync service-handler view", () => {
it("forwards semantic callbacks through a frozen view", async () => {
const operations = {
processOptionalFileEvent: vi.fn(async () => true),
processOptionalSyncFiles: vi.fn(async () => true),
onSettingLoaded: vi.fn(async () => true),
realiseSettingSyncMode: vi.fn(async () => true),
onResuming: vi.fn(async () => true),
beforeReplicate: vi.fn(async () => true),
onDatabaseInitialised: vi.fn(async () => true),
suspendExtraSync: vi.fn(async () => true),
configureOptionalSyncFeature: vi.fn(async () => true),
isTargetFileEligible: vi.fn(async () => true),
queueConflict: vi.fn(async () => true),
} satisfies HiddenFileSyncServiceHandlerView;
const view = createHiddenFileSyncServiceHandlerView(operations);
expect(Object.isFrozen(view)).toBe(true);
await view.processOptionalFileEvent(".obsidian/app.json" as FilePath);
await view.processOptionalSyncFiles({} as never);
await view.onSettingLoaded();
await view.realiseSettingSyncMode();
await view.onResuming();
await view.beforeReplicate(true);
await view.onDatabaseInitialised(false);
await view.suspendExtraSync();
await view.configureOptionalSyncFeature("MERGE");
await view.isTargetFileEligible(".obsidian/app.json" as FilePath);
await view.queueConflict("i:.obsidian/app.json" as never);
expect(operations.processOptionalFileEvent).toHaveBeenCalledWith(".obsidian/app.json");
expect(operations.beforeReplicate).toHaveBeenCalledWith(true);
expect(operations.onDatabaseInitialised).toHaveBeenCalledWith(false);
expect(operations.configureOptionalSyncFeature).toHaveBeenCalledWith("MERGE");
expect(operations.queueConflict).toHaveBeenCalledWith("i:.obsidian/app.json");
});
});
describe("Hidden File Sync testing view", () => {
it("keeps test operations focused while retaining a frozen E2E surface", async () => {
const conflictResolution = {
resolveAll: vi.fn(async () => undefined),
resolveJson: vi.fn(async () => true),
pendingPaths: [],
processor: { remaining: 0, totalRemaining: 0, nowProcessing: 0 },
};
const restoreRebuild = vi.fn();
const operations = {
isManualCommandAvailable: vi.fn(() => true),
scanAllStorageChanges: vi.fn(async () => undefined),
scanAllDatabaseChanges: vi.fn(async () => undefined),
applyOfflineChanges: vi.fn(async () => undefined),
updateSettingCache: vi.fn(),
initialiseInternalFileSync: vi.fn(async () => undefined),
conflictResolution,
readFileWithInfo: vi.fn(async () => ({}) as never),
showConfigurationChangeNotice: vi.fn(),
interceptRebuildMerging: vi.fn(() => restoreRebuild),
} satisfies HiddenFileSyncTestingViewOperations;
const view = createHiddenFileSyncTestingView(operations);
expect(Object.isFrozen(view)).toBe(true);
await view.scanAllStorageChanges(true);
await view.readFileWithInfo(".obsidian/app.json" as FilePath);
view.showConfigurationChangeNotice([".obsidian"]);
expect(operations.scanAllStorageChanges).toHaveBeenCalledWith(true);
expect(operations.readFileWithInfo).toHaveBeenCalledWith(".obsidian/app.json");
expect(operations.showConfigurationChangeNotice).toHaveBeenCalledWith([".obsidian"]);
const interceptor = vi.fn(async () => [] as FilePath[]);
expect(view.interceptRebuildMerging(interceptor)).toBe(restoreRebuild);
expect(operations.interceptRebuildMerging).toHaveBeenCalledWith(interceptor);
});
});
+51
View File
@@ -0,0 +1,51 @@
export type OptionalFileSyncDirectoryListing = {
files: readonly string[];
folders: readonly string[];
};
export type OptionalFileSyncFileTreeDependencies = {
listFiles(path: string): Promise<OptionalFileSyncDirectoryListing>;
};
export type OptionalFileSyncFileTreeOptions = {
maxDepth?: number;
shouldInclude?(path: string): boolean | Promise<boolean>;
onError?(path: string, error: unknown): void;
};
export async function collectOptionalFileSyncFiles(
dependencies: OptionalFileSyncFileTreeDependencies,
path: string,
options: OptionalFileSyncFileTreeOptions = {}
): Promise<string[]> {
if (options.maxDepth !== undefined && options.maxDepth < 0) {
return [];
}
let listing: OptionalFileSyncDirectoryListing;
try {
listing = await dependencies.listFiles(path);
} catch (error) {
options.onError?.(path, error);
return [];
}
const files: string[] = [];
for (const file of listing.files) {
if ((await options.shouldInclude?.(file)) ?? true) {
files.push(file);
}
}
for (const folder of listing.folders) {
if (!((await options.shouldInclude?.(folder)) ?? true)) {
continue;
}
files.push(
...(await collectOptionalFileSyncFiles(dependencies, folder, {
...options,
maxDepth: options.maxDepth === undefined ? undefined : options.maxDepth - 1,
}))
);
}
return files;
}
@@ -0,0 +1,48 @@
import { describe, expect, it, vi } from "vitest";
import { collectOptionalFileSyncFiles } from "./optionalFileSyncFileTree.ts";
describe("collectOptionalFileSyncFiles", () => {
it("collects files to the requested directory depth", async () => {
const listings = new Map([
["root", { files: ["root/file"], folders: ["root/one"] }],
["root/one", { files: ["root/one/file"], folders: ["root/one/two"] }],
["root/one/two", { files: ["root/one/two/file"], folders: ["root/one/two/three"] }],
["root/one/two/three", { files: ["root/one/two/three/file"], folders: [] }],
]);
const listFiles = vi.fn(async (path: string) => listings.get(path) ?? { files: [], folders: [] });
await expect(collectOptionalFileSyncFiles({ listFiles }, "root", { maxDepth: 2 })).resolves.toEqual([
"root/file",
"root/one/file",
"root/one/two/file",
]);
expect(listFiles).not.toHaveBeenCalledWith("root/one/two/three");
});
it("uses the same asynchronous filter for files and directory traversal", async () => {
const listFiles = vi.fn(async (path: string) =>
path == "root"
? { files: ["root/include", "root/skip"], folders: ["root/allowed", "root/blocked"] }
: { files: [`${path}/include`], folders: [] }
);
const shouldInclude = vi.fn(async (path: string) => !path.includes("skip") && !path.includes("blocked"));
await expect(collectOptionalFileSyncFiles({ listFiles }, "root", { shouldInclude })).resolves.toEqual([
"root/include",
"root/allowed/include",
]);
expect(listFiles).not.toHaveBeenCalledWith("root/blocked");
});
it("reports an unreadable directory and keeps the successful part of the traversal", async () => {
const failure = new Error("unreadable");
const listFiles = vi.fn(async (path: string) => {
if (path == "root/failing") throw failure;
return { files: ["root/file"], folders: ["root/failing"] };
});
const onError = vi.fn();
await expect(collectOptionalFileSyncFiles({ listFiles }, "root", { onError })).resolves.toEqual(["root/file"]);
expect(onError).toHaveBeenCalledWith("root/failing", failure);
});
});
@@ -11,9 +11,9 @@ import { getObsidianCommunityPluginManager } from "@/common/obsidianCommunityPlu
import { JsonResolveModal } from "@/features/HiddenFileCommon/JsonResolveModal.ts"; import { JsonResolveModal } from "@/features/HiddenFileCommon/JsonResolveModal.ts";
import type { import type {
HiddenFileSyncContextDependencies, HiddenFileSyncContextDependencies,
HiddenFileSyncJsonResolution,
HiddenFileSyncProgress, HiddenFileSyncProgress,
} from "@/features/HiddenFileSync/hiddenFileSyncContext.ts"; } from "@/features/HiddenFileSync/hiddenFileSyncContext.ts";
import type { HiddenFileSyncJsonResolution } from "@/features/HiddenFileSync/hiddenFileSyncConflictResolution.ts";
import { MARK_DONE } from "@/modules/features/ModuleLog.ts"; import { MARK_DONE } from "@/modules/features/ModuleLog.ts";
import type { LiveSyncCore } from "@/main.ts"; import type { LiveSyncCore } from "@/main.ts";
@@ -18,8 +18,14 @@ describe("Optional File Sync ownership boundary", () => {
expect(mainSource).not.toContain("optionalFileSync.testing"); expect(mainSource).not.toContain("optionalFileSync.testing");
expect(featureSource).toContain("new CustomisationSyncContext"); expect(featureSource).toContain("new CustomisationSyncContext");
expect(featureSource).toContain("new HiddenFileSyncContext"); expect(featureSource).toContain("new HiddenFileSyncContext");
expect(featureSource).toContain("customisationSync.serviceHandlers");
expect(featureSource).toContain("hiddenFileSync.serviceHandlers");
expect(featureSource).toContain("customisationSync.testing");
expect(featureSource).toContain("hiddenFileSync.testing");
expect(customisationSource).not.toContain("onBindFunction("); expect(customisationSource).not.toContain("onBindFunction(");
expect(hiddenSource).not.toContain("onBindFunction("); expect(hiddenSource).not.toContain("onBindFunction(");
expect(customisationSource).not.toMatch(/\b_(?:any|every|all)[A-Z]/);
expect(hiddenSource).not.toMatch(/\b_(?:any|every|all)[A-Z]/);
expect(customisationSource).not.toContain("LiveSyncCommands"); expect(customisationSource).not.toContain("LiveSyncCommands");
expect(hiddenSource).not.toContain("LiveSyncCommands"); expect(hiddenSource).not.toContain("LiveSyncCommands");
expect(customisationSource).not.toContain("extends LiveSyncContext"); expect(customisationSource).not.toContain("extends LiveSyncContext");
@@ -15,6 +15,7 @@ import {
type CustomisationSyncFileCategory, type CustomisationSyncFileCategory,
type CustomisationSyncPathOptions, type CustomisationSyncPathOptions,
} from "@/features/ConfigSync/customisationSyncPaths.ts"; } from "@/features/ConfigSync/customisationSyncPaths.ts";
import { isHiddenFileSyncPath } from "@/features/HiddenFileSync/hiddenFileSyncPathPolicy.ts";
export type OptionalFileSyncOwner = "customisation" | "hidden-file" | "none"; export type OptionalFileSyncOwner = "customisation" | "hidden-file" | "none";
@@ -58,10 +59,6 @@ export type CustomisationSyncDocumentOwnershipInput = {
pluginSyncExtendedSetting: Readonly<Record<string, PluginSyncSettingEntry>>; pluginSyncExtendedSetting: Readonly<Record<string, PluginSyncSettingEntry>>;
}; };
function isHiddenFileSyncPath(path: string): boolean {
return path.startsWith(".") && !path.startsWith(".trash");
}
/** Select the sole local writer from persisted feature settings. */ /** Select the sole local writer from persisted feature settings. */
export function selectOptionalFileSyncOwner( export function selectOptionalFileSyncOwner(
input: OptionalFileSyncOwnerSelectionInput input: OptionalFileSyncOwnerSelectionInput
+38 -70
View File
@@ -10,12 +10,16 @@ import {
CustomisationSyncContext, CustomisationSyncContext,
type CustomisationSyncContextDependencies, type CustomisationSyncContextDependencies,
} from "@/features/ConfigSync/customisationSyncContext.ts"; } from "@/features/ConfigSync/customisationSyncContext.ts";
import type { CustomisationSyncDialogView } from "@/features/ConfigSync/customisationSyncView.ts"; import type {
CustomisationSyncDialogView,
CustomisationSyncTestingView,
} from "@/features/ConfigSync/customisationSyncView.ts";
import { HiddenFileSyncContext } from "@/features/HiddenFileSync/hiddenFileSyncContext.ts"; import { HiddenFileSyncContext } from "@/features/HiddenFileSync/hiddenFileSyncContext.ts";
import type { import type {
HiddenFileSyncCommandView, HiddenFileSyncCommandView,
HiddenFileSyncInitialisationView, HiddenFileSyncInitialisationView,
HiddenFileSyncRepairView, HiddenFileSyncRepairView,
HiddenFileSyncTestingView,
} from "@/features/HiddenFileSync/hiddenFileSyncViews.ts"; } from "@/features/HiddenFileSync/hiddenFileSyncViews.ts";
import type { LiveSyncCore } from "@/main.ts"; import type { LiveSyncCore } from "@/main.ts";
import { import {
@@ -38,10 +42,10 @@ export interface OptionalFileSyncFeature {
readonly hiddenFileSyncCommands: HiddenFileSyncCommandView; readonly hiddenFileSyncCommands: HiddenFileSyncCommandView;
readonly hiddenFileSyncInitialisation: HiddenFileSyncInitialisationView; readonly hiddenFileSyncInitialisation: HiddenFileSyncInitialisationView;
readonly hiddenFileSyncRepair: HiddenFileSyncRepairView; readonly hiddenFileSyncRepair: HiddenFileSyncRepairView;
/** @internal Direct runtime access for the repository's real-Obsidian contract tests. */ /** @internal Focused operations for the repository's real-Obsidian contract tests. */
readonly testing: { readonly testing: {
readonly customisationSync: CustomisationSyncContext; readonly customisationSync: CustomisationSyncTestingView;
readonly hiddenFileSync: HiddenFileSyncContext; readonly hiddenFileSync: HiddenFileSyncTestingView;
}; };
} }
@@ -96,6 +100,9 @@ export function useOptionalFileSync(
const hiddenFileSync = createHiddenFileSync({ const hiddenFileSync = createHiddenFileSync({
ownsLocalFile: ownsLocalFile("hidden-file"), ownsLocalFile: ownsLocalFile("hidden-file"),
}); });
const customisationHandlers = customisationSync.serviceHandlers;
const hiddenFileHandlers = hiddenFileSync.serviceHandlers;
const hiddenFileSyncRepair = hiddenFileSync.repair;
const { services } = host; const { services } = host;
const disposers: (() => void)[] = []; const disposers: (() => void)[] = [];
@@ -108,7 +115,7 @@ export function useOptionalFileSync(
const routeLocalPath = async (path: FilePath) => { const routeLocalPath = async (path: FilePath) => {
const selected = selectOptionalFileSyncOwner(ownerSelectionInput(path)); const selected = selectOptionalFileSyncOwner(ownerSelectionInput(path));
const hiddenFileEligible = const hiddenFileEligible =
selected.owner == "hidden-file" ? await hiddenFileSync.isTargetFileEligible(path) : false; selected.owner == "hidden-file" ? await hiddenFileHandlers.isTargetFileEligible(path) : false;
const ready = services.appLifecycle.isReady() && !services.appLifecycle.isSuspended(); const ready = services.appLifecycle.isReady() && !services.appLifecycle.isSuspended();
return routeOptionalFileSyncPath({ return routeOptionalFileSyncPath({
...ownerSelectionInput(path), ...ownerSelectionInput(path),
@@ -123,86 +130,44 @@ export function useOptionalFileSync(
const localPath = normaliseLocalPath(path); const localPath = normaliseLocalPath(path);
const decision = await routeLocalPath(localPath); const decision = await routeLocalPath(localPath);
if (decision.owner == "customisation") { if (decision.owner == "customisation") {
return await customisationSync._anyProcessOptionalFileEvent(localPath); return await customisationHandlers.processOptionalFileEvent(localPath);
} }
if (decision.owner == "hidden-file") { if (decision.owner == "hidden-file") {
return await hiddenFileSync._anyProcessOptionalFileEvent(localPath); return await hiddenFileHandlers.processOptionalFileEvent(localPath);
} }
return false; return false;
}) })
); );
register( register(
services.conflict.getOptionalConflictCheckMethod.addHandler(async (path: FilePathWithPrefix) => { services.conflict.getOptionalConflictCheckMethod.addHandler((path: FilePathWithPrefix) => {
if (isPluginMetadata(path) || isCustomisationSyncMetadata(path)) { if (isPluginMetadata(path) || isCustomisationSyncMetadata(path)) {
return await customisationSync._anyGetOptionalConflictCheckMethod(path); return Promise.resolve("newer");
} }
if (isInternalMetadata(path)) { if (isInternalMetadata(path)) {
return await hiddenFileSync._anyGetOptionalConflictCheckMethod(path); return hiddenFileHandlers.queueConflict(path);
} }
return false; return Promise.resolve(false);
}) })
); );
register(services.replication.processVirtualDocument.addHandler(customisationHandlers.processVirtualDocument));
register( register(
services.replication.processVirtualDocument.addHandler( services.replication.processOptionalSynchroniseResult.addHandler(hiddenFileHandlers.processOptionalSyncFiles)
customisationSync._anyModuleParsedReplicationResultItem.bind(customisationSync)
)
);
register(
services.replication.processOptionalSynchroniseResult.addHandler(
hiddenFileSync._anyProcessOptionalSyncFiles.bind(hiddenFileSync)
)
);
register(
services.appLifecycle.onSettingLoaded.addHandler(
hiddenFileSync._everyOnloadAfterLoadSettings.bind(hiddenFileSync)
)
); );
register(services.appLifecycle.onSettingLoaded.addHandler(hiddenFileHandlers.onSettingLoaded));
register( register(services.setting.onRealiseSetting.addHandler(customisationHandlers.onRealiseSetting));
services.setting.onRealiseSetting.addHandler( register(services.setting.onRealiseSetting.addHandler(hiddenFileHandlers.realiseSettingSyncMode));
customisationSync._everyRealizeSettingSyncMode.bind(customisationSync) register(services.appLifecycle.onResuming.addHandler(customisationHandlers.onResuming));
) register(services.appLifecycle.onResuming.addHandler(hiddenFileHandlers.onResuming));
); register(services.replication.onBeforeReplicate.addHandler(customisationHandlers.onBeforeReplicate));
register( register(services.replication.onBeforeReplicate.addHandler(hiddenFileHandlers.beforeReplicate));
services.setting.onRealiseSetting.addHandler(hiddenFileSync._everyRealizeSettingSyncMode.bind(hiddenFileSync)) register(services.databaseEvents.onDatabaseInitialised.addHandler(customisationHandlers.onDatabaseInitialised));
); register(services.databaseEvents.onDatabaseInitialised.addHandler(hiddenFileHandlers.onDatabaseInitialised));
register( register(services.setting.suspendExtraSync.addHandler(customisationHandlers.suspendExtraSync));
services.appLifecycle.onResuming.addHandler(customisationSync._everyOnResumeProcess.bind(customisationSync)) register(services.setting.suspendExtraSync.addHandler(hiddenFileHandlers.suspendExtraSync));
); register(services.setting.enableOptionalFeature.addHandler(customisationHandlers.enableOptionalFeature));
register(services.appLifecycle.onResuming.addHandler(hiddenFileSync._everyOnResumeProcess.bind(hiddenFileSync))); register(services.setting.enableOptionalFeature.addHandler(hiddenFileHandlers.configureOptionalSyncFeature));
register(
services.replication.onBeforeReplicate.addHandler(
customisationSync._everyBeforeReplicate.bind(customisationSync)
)
);
register(
services.replication.onBeforeReplicate.addHandler(hiddenFileSync._everyBeforeReplicate.bind(hiddenFileSync))
);
register(
services.databaseEvents.onDatabaseInitialised.addHandler(
customisationSync._everyOnDatabaseInitialized.bind(customisationSync)
)
);
register(
services.databaseEvents.onDatabaseInitialised.addHandler(
hiddenFileSync._everyOnDatabaseInitialized.bind(hiddenFileSync)
)
);
register(
services.setting.suspendExtraSync.addHandler(customisationSync._allSuspendExtraSync.bind(customisationSync))
);
register(services.setting.suspendExtraSync.addHandler(hiddenFileSync._allSuspendExtraSync.bind(hiddenFileSync)));
register(
services.setting.enableOptionalFeature.addHandler(
customisationSync._allConfigureOptionalSyncFeature.bind(customisationSync)
)
);
register(
services.setting.enableOptionalFeature.addHandler(
hiddenFileSync._allConfigureOptionalSyncFeature.bind(hiddenFileSync)
)
);
register( register(
services.vault.isTargetFileInExtra.addHandler( services.vault.isTargetFileInExtra.addHandler(
async (file: string | UXFileInfoStub) => (await routeLocalPath(normaliseLocalPath(file))).owner != "none" async (file: string | UXFileInfoStub) => (await routeLocalPath(normaliseLocalPath(file))).owner != "none"
@@ -234,7 +199,10 @@ export function useOptionalFileSync(
customisationSync, customisationSync,
hiddenFileSyncCommands: hiddenFileSync, hiddenFileSyncCommands: hiddenFileSync,
hiddenFileSyncInitialisation: hiddenFileSync, hiddenFileSyncInitialisation: hiddenFileSync,
hiddenFileSyncRepair: hiddenFileSync, hiddenFileSyncRepair,
testing: Object.freeze({ customisationSync, hiddenFileSync }), testing: Object.freeze({
customisationSync: customisationSync.testing,
hiddenFileSync: hiddenFileSync.testing,
}),
}); });
} }
@@ -38,6 +38,15 @@ function handlerRegistry() {
}; };
} }
function hiddenFileSyncRepairFixture() {
return Object.freeze({
scanInternalFiles: vi.fn(async () => []),
storeInternalFileToDatabase: vi.fn(async () => true),
storeInternalFileToDatabaseWithBaseRevision: vi.fn(async () => true),
extractInternalFileRevisionFromDatabase: vi.fn(async () => true),
});
}
function createFixture() { function createFixture() {
const processOptionalFileEvent = handlerRegistry(); const processOptionalFileEvent = handlerRegistry();
const getOptionalConflictCheckMethod = handlerRegistry(); const getOptionalConflictCheckMethod = handlerRegistry();
@@ -54,31 +63,42 @@ function createFixture() {
const onUnload = handlerRegistry(); const onUnload = handlerRegistry();
const calls: string[] = []; const calls: string[] = [];
const customisationHandlers = {
processOptionalFileEvent: vi.fn(async () => false),
processVirtualDocument: vi.fn(async () => false),
onRealiseSetting: vi.fn(async () => true),
onResuming: vi.fn(async () => true),
onBeforeReplicate: vi.fn(async () => true),
onDatabaseInitialised: vi.fn(async () => true),
suspendExtraSync: vi.fn(async () => true),
enableOptionalFeature: vi.fn(async () => true),
};
const customisationTesting = Object.freeze({ kind: "customisation-testing" });
const customisationSync = { const customisationSync = {
dispose: vi.fn(() => calls.push("customisation:unload")), dispose: vi.fn(() => calls.push("customisation:unload")),
_anyProcessOptionalFileEvent: vi.fn(async () => false), serviceHandlers: Object.freeze(customisationHandlers),
_anyGetOptionalConflictCheckMethod: vi.fn(async () => false), testing: customisationTesting,
_anyModuleParsedReplicationResultItem: vi.fn(async () => false),
_everyRealizeSettingSyncMode: vi.fn(async () => true),
_everyOnResumeProcess: vi.fn(async () => true),
_everyBeforeReplicate: vi.fn(async () => true),
_everyOnDatabaseInitialized: vi.fn(async () => true),
_allSuspendExtraSync: vi.fn(async () => true),
_allConfigureOptionalSyncFeature: vi.fn(async () => true),
}; };
const hiddenFileHandlers = {
onSettingLoaded: vi.fn(async () => true),
processOptionalFileEvent: vi.fn(async () => false),
queueConflict: vi.fn(async () => true),
processOptionalSyncFiles: vi.fn(async () => false),
realiseSettingSyncMode: vi.fn(async () => true),
onResuming: vi.fn(async () => true),
beforeReplicate: vi.fn(async () => true),
onDatabaseInitialised: vi.fn(async () => true),
suspendExtraSync: vi.fn(async () => true),
configureOptionalSyncFeature: vi.fn(async () => true),
isTargetFileEligible: vi.fn(async () => true),
};
const hiddenFileSyncRepair = hiddenFileSyncRepairFixture();
const hiddenFileTesting = Object.freeze({ kind: "hidden-file-testing" });
const hiddenFileSync = { const hiddenFileSync = {
dispose: vi.fn(() => calls.push("hidden:unload")), dispose: vi.fn(() => calls.push("hidden:unload")),
_everyOnloadAfterLoadSettings: vi.fn(async () => true), serviceHandlers: Object.freeze(hiddenFileHandlers),
_anyProcessOptionalFileEvent: vi.fn(async () => false), testing: hiddenFileTesting,
_anyGetOptionalConflictCheckMethod: vi.fn(async () => false), repair: hiddenFileSyncRepair,
_anyProcessOptionalSyncFiles: vi.fn(async () => false),
_everyRealizeSettingSyncMode: vi.fn(async () => true),
_everyOnResumeProcess: vi.fn(async () => true),
_everyBeforeReplicate: vi.fn(async () => true),
_everyOnDatabaseInitialized: vi.fn(async () => true),
_allSuspendExtraSync: vi.fn(async () => true),
_allConfigureOptionalSyncFeature: vi.fn(async () => true),
isTargetFileEligible: vi.fn(async () => true),
}; };
const settings = { const settings = {
usePluginSync: true, usePluginSync: true,
@@ -121,9 +141,13 @@ function createFixture() {
return { return {
calls, calls,
customisationHandlers,
customisationSync, customisationSync,
customisationTesting,
feature, feature,
hiddenFileHandlers,
hiddenFileSync, hiddenFileSync,
hiddenFileTesting,
settings, settings,
contextDependencies: { contextDependencies: {
customisation: () => customisationDependencies, customisation: () => customisationDependencies,
@@ -148,31 +172,39 @@ function createFixture() {
} }
function createAggregateFixture() { function createAggregateFixture() {
const customisationHandlers = {
processOptionalFileEvent: vi.fn(async () => false),
processVirtualDocument: vi.fn(async () => false),
onRealiseSetting: vi.fn(async () => true),
onResuming: vi.fn(async () => true),
onBeforeReplicate: vi.fn(async () => true),
onDatabaseInitialised: vi.fn(async () => true),
suspendExtraSync: vi.fn(async () => true),
enableOptionalFeature: vi.fn(async () => true),
};
const customisationSync = { const customisationSync = {
dispose: vi.fn(), dispose: vi.fn(),
_anyProcessOptionalFileEvent: vi.fn(async () => false), serviceHandlers: Object.freeze(customisationHandlers),
_anyGetOptionalConflictCheckMethod: vi.fn(async (): Promise<boolean | "newer"> => false), testing: Object.freeze({ kind: "customisation-testing" }),
_anyModuleParsedReplicationResultItem: vi.fn(async () => false), };
_everyRealizeSettingSyncMode: vi.fn(async () => true), const hiddenFileHandlers = {
_everyOnResumeProcess: vi.fn(async () => true), onSettingLoaded: vi.fn(async () => true),
_everyBeforeReplicate: vi.fn(async () => true), processOptionalFileEvent: vi.fn(async () => false),
_everyOnDatabaseInitialized: vi.fn(async () => true), queueConflict: vi.fn(async () => true),
_allSuspendExtraSync: vi.fn(async () => true), processOptionalSyncFiles: vi.fn(async () => false),
_allConfigureOptionalSyncFeature: vi.fn(async () => true), realiseSettingSyncMode: vi.fn(async () => true),
onResuming: vi.fn(async () => true),
beforeReplicate: vi.fn(async () => true),
onDatabaseInitialised: vi.fn(async () => true),
suspendExtraSync: vi.fn(async () => true),
configureOptionalSyncFeature: vi.fn(async () => true),
isTargetFileEligible: vi.fn(async () => true),
}; };
const hiddenFileSync = { const hiddenFileSync = {
dispose: vi.fn(), dispose: vi.fn(),
_everyOnloadAfterLoadSettings: vi.fn(async () => true), serviceHandlers: Object.freeze(hiddenFileHandlers),
_anyProcessOptionalFileEvent: vi.fn(async () => false), testing: Object.freeze({ kind: "hidden-file-testing" }),
_anyGetOptionalConflictCheckMethod: vi.fn(async (): Promise<boolean | "newer"> => false), repair: hiddenFileSyncRepairFixture(),
_anyProcessOptionalSyncFiles: vi.fn(async () => false),
_everyRealizeSettingSyncMode: vi.fn(async () => true),
_everyOnResumeProcess: vi.fn(async () => true),
_everyBeforeReplicate: vi.fn(async () => true),
_everyOnDatabaseInitialized: vi.fn(async () => true),
_allSuspendExtraSync: vi.fn(async () => true),
_allConfigureOptionalSyncFeature: vi.fn(async () => true),
isTargetFileEligible: vi.fn(async () => true),
}; };
const settings = { const settings = {
usePluginSync: true, usePluginSync: true,
@@ -217,7 +249,7 @@ function createAggregateFixture() {
createHiddenFileSync: () => hiddenFileSync as never, createHiddenFileSync: () => hiddenFileSync as never,
}); });
return { customisationSync, hiddenFileSync, services, settings }; return { customisationHandlers, customisationSync, hiddenFileHandlers, hiddenFileSync, services, settings };
} }
describe("useOptionalFileSync", () => { describe("useOptionalFileSync", () => {
@@ -239,10 +271,10 @@ describe("useOptionalFileSync", () => {
}); });
it("routes Selective and Automatic paths to exactly one local owner", async () => { it("routes Selective and Automatic paths to exactly one local owner", async () => {
const { customisationSync, hiddenFileSync, registries, settings } = createFixture(); const { customisationHandlers, hiddenFileHandlers, registries, settings } = createFixture();
customisationSync._anyProcessOptionalFileEvent.mockResolvedValue(true); customisationHandlers.processOptionalFileEvent.mockResolvedValue(true);
hiddenFileSync._anyProcessOptionalFileEvent.mockResolvedValue(true); hiddenFileHandlers.processOptionalFileEvent.mockResolvedValue(true);
hiddenFileSync.isTargetFileEligible.mockResolvedValue(false); hiddenFileHandlers.isTargetFileEligible.mockResolvedValue(false);
await expect(registries.isTargetFileInExtra.handlers[0]!(".obsidian/plugins/example/data.json")).resolves.toBe( await expect(registries.isTargetFileInExtra.handlers[0]!(".obsidian/plugins/example/data.json")).resolves.toBe(
true true
@@ -250,9 +282,9 @@ describe("useOptionalFileSync", () => {
await expect( await expect(
registries.processOptionalFileEvent.handlers[0]!(".obsidian/plugins/example/data.json") registries.processOptionalFileEvent.handlers[0]!(".obsidian/plugins/example/data.json")
).resolves.toBe(true); ).resolves.toBe(true);
expect(customisationSync._anyProcessOptionalFileEvent).toHaveBeenCalledOnce(); expect(customisationHandlers.processOptionalFileEvent).toHaveBeenCalledOnce();
expect(hiddenFileSync._anyProcessOptionalFileEvent).not.toHaveBeenCalled(); expect(hiddenFileHandlers.processOptionalFileEvent).not.toHaveBeenCalled();
expect(hiddenFileSync.isTargetFileEligible).not.toHaveBeenCalled(); expect(hiddenFileHandlers.isTargetFileEligible).not.toHaveBeenCalled();
settings.pluginSyncExtendedSetting = { settings.pluginSyncExtendedSetting = {
"PLUGIN_DATA/example": { "PLUGIN_DATA/example": {
@@ -261,53 +293,53 @@ describe("useOptionalFileSync", () => {
files: [], files: [],
}, },
}; };
hiddenFileSync.isTargetFileEligible.mockResolvedValue(true); hiddenFileHandlers.isTargetFileEligible.mockResolvedValue(true);
customisationSync._anyProcessOptionalFileEvent.mockClear(); customisationHandlers.processOptionalFileEvent.mockClear();
await expect( await expect(
registries.processOptionalFileEvent.handlers[0]!(".obsidian/plugins/example/data.json") registries.processOptionalFileEvent.handlers[0]!(".obsidian/plugins/example/data.json")
).resolves.toBe(true); ).resolves.toBe(true);
expect(hiddenFileSync._anyProcessOptionalFileEvent).toHaveBeenCalledOnce(); expect(hiddenFileHandlers.processOptionalFileEvent).toHaveBeenCalledOnce();
expect(customisationSync._anyProcessOptionalFileEvent).not.toHaveBeenCalled(); expect(customisationHandlers.processOptionalFileEvent).not.toHaveBeenCalled();
}); });
it("does not fall back to the other context when the selected owner skips or fails", async () => { it("does not fall back to the other context when the selected owner skips or fails", async () => {
const { customisationSync, hiddenFileSync, services } = createAggregateFixture(); const { customisationHandlers, hiddenFileHandlers, services } = createAggregateFixture();
customisationSync._anyProcessOptionalFileEvent.mockResolvedValueOnce(false); customisationHandlers.processOptionalFileEvent.mockResolvedValueOnce(false);
await expect(services.fileProcessing.processOptionalFileEvent(".obsidian/app.json")).resolves.toBe(false); await expect(services.fileProcessing.processOptionalFileEvent(".obsidian/app.json")).resolves.toBe(false);
expect(hiddenFileSync._anyProcessOptionalFileEvent).not.toHaveBeenCalled(); expect(hiddenFileHandlers.processOptionalFileEvent).not.toHaveBeenCalled();
customisationSync._anyProcessOptionalFileEvent.mockRejectedValueOnce(new Error("customisation failed")); customisationHandlers.processOptionalFileEvent.mockRejectedValueOnce(new Error("customisation failed"));
await expect(services.fileProcessing.processOptionalFileEvent(".obsidian/app.json")).resolves.toBe(false); await expect(services.fileProcessing.processOptionalFileEvent(".obsidian/app.json")).resolves.toBe(false);
expect(hiddenFileSync._anyProcessOptionalFileEvent).not.toHaveBeenCalled(); expect(hiddenFileHandlers.processOptionalFileEvent).not.toHaveBeenCalled();
}); });
it("dispatches conflict documents by their persisted namespace", async () => { it("dispatches conflict documents by their persisted namespace", async () => {
const { customisationSync, hiddenFileSync, services } = createAggregateFixture(); const { hiddenFileHandlers, services } = createAggregateFixture();
customisationSync._anyGetOptionalConflictCheckMethod.mockResolvedValue("newer");
hiddenFileSync._anyGetOptionalConflictCheckMethod.mockResolvedValue(true);
await expect(services.conflict.getOptionalConflictCheckMethod("ix:device/app.json")).resolves.toBe("newer"); await expect(services.conflict.getOptionalConflictCheckMethod("ix:device/app.json")).resolves.toBe("newer");
expect(hiddenFileSync._anyGetOptionalConflictCheckMethod).not.toHaveBeenCalled(); expect(hiddenFileHandlers.queueConflict).not.toHaveBeenCalled();
await expect(services.conflict.getOptionalConflictCheckMethod("i:.obsidian/example.json")).resolves.toBe(true); await expect(services.conflict.getOptionalConflictCheckMethod("i:.obsidian/example.json")).resolves.toBe(true);
expect(customisationSync._anyGetOptionalConflictCheckMethod).toHaveBeenCalledOnce(); expect(hiddenFileHandlers.queueConflict).toHaveBeenCalledOnce();
expect(hiddenFileHandlers.queueConflict).toHaveBeenCalledWith("i:.obsidian/example.json");
await expect(services.conflict.getOptionalConflictCheckMethod("notes/example.md")).resolves.toBe(false);
}); });
it("keeps persisted document acceptance separate from current local ownership", async () => { it("keeps persisted document acceptance separate from current local ownership", async () => {
const { customisationSync, hiddenFileSync, registries, settings } = createFixture(); const { customisationHandlers, hiddenFileHandlers, registries, settings } = createFixture();
settings.usePluginSync = false; settings.usePluginSync = false;
settings.syncInternalFiles = false; settings.syncInternalFiles = false;
await registries.processVirtualDocument.handlers[0]!({ _id: "ix:device-a/CONFIG/app.json.md" }); await registries.processVirtualDocument.handlers[0]!({ _id: "ix:device-a/CONFIG/app.json.md" });
await registries.processOptionalSynchroniseResult.handlers[0]!({ _id: "i:.obsidian/app.json" }); await registries.processOptionalSynchroniseResult.handlers[0]!({ _id: "i:.obsidian/app.json" });
expect(customisationSync._anyModuleParsedReplicationResultItem).toHaveBeenCalledOnce(); expect(customisationHandlers.processVirtualDocument).toHaveBeenCalledOnce();
expect(hiddenFileSync._anyProcessOptionalSyncFiles).toHaveBeenCalledOnce(); expect(hiddenFileHandlers.processOptionalSyncFiles).toHaveBeenCalledOnce();
}); });
it("routes Ignore mode to neither local context", async () => { it("routes Ignore mode to neither local context", async () => {
const { customisationSync, hiddenFileSync, registries, settings } = createFixture(); const { customisationHandlers, hiddenFileHandlers, registries, settings } = createFixture();
settings.pluginSyncExtendedSetting = { settings.pluginSyncExtendedSetting = {
"PLUGIN_DATA/example": { "PLUGIN_DATA/example": {
key: "PLUGIN_DATA/example", key: "PLUGIN_DATA/example",
@@ -319,8 +351,8 @@ describe("useOptionalFileSync", () => {
await expect( await expect(
registries.processOptionalFileEvent.handlers[0]!(".obsidian/plugins/example/data.json") registries.processOptionalFileEvent.handlers[0]!(".obsidian/plugins/example/data.json")
).resolves.toBe(false); ).resolves.toBe(false);
expect(customisationSync._anyProcessOptionalFileEvent).not.toHaveBeenCalled(); expect(customisationHandlers.processOptionalFileEvent).not.toHaveBeenCalled();
expect(hiddenFileSync._anyProcessOptionalFileEvent).not.toHaveBeenCalled(); expect(hiddenFileHandlers.processOptionalFileEvent).not.toHaveBeenCalled();
}); });
it("injects the same static ownership policy into both scan contexts", () => { it("injects the same static ownership policy into both scan contexts", () => {
@@ -351,11 +383,12 @@ describe("useOptionalFileSync", () => {
}); });
it("preserves bail-first-failure and settled-unload behaviour", async () => { it("preserves bail-first-failure and settled-unload behaviour", async () => {
const { customisationSync, hiddenFileSync, services } = createAggregateFixture(); const { customisationHandlers, customisationSync, hiddenFileHandlers, hiddenFileSync, services } =
customisationSync._everyRealizeSettingSyncMode.mockResolvedValueOnce(false); createAggregateFixture();
customisationHandlers.onRealiseSetting.mockResolvedValueOnce(false);
await expect(services.setting.onRealiseSetting()).resolves.toBe(false); await expect(services.setting.onRealiseSetting()).resolves.toBe(false);
expect(hiddenFileSync._everyRealizeSettingSyncMode).not.toHaveBeenCalled(); expect(hiddenFileHandlers.realiseSettingSyncMode).not.toHaveBeenCalled();
customisationSync.dispose.mockImplementationOnce(() => { customisationSync.dispose.mockImplementationOnce(() => {
throw new Error("customisation disposal failed"); throw new Error("customisation disposal failed");
@@ -380,21 +413,54 @@ describe("useOptionalFileSync", () => {
}); });
it("strips a database prefix before evaluating a Hidden File Sync target", async () => { it("strips a database prefix before evaluating a Hidden File Sync target", async () => {
const { hiddenFileSync, registries } = createFixture(); const { hiddenFileHandlers, registries } = createFixture();
await registries.isTargetFileInExtra.handlers[0]!({ path: "i:.obsidian/workspace" }); await registries.isTargetFileInExtra.handlers[0]!({ path: "i:.obsidian/workspace" });
expect(hiddenFileSync.isTargetFileEligible).toHaveBeenCalledWith(".obsidian/workspace"); expect(hiddenFileHandlers.isTargetFileEligible).toHaveBeenCalledWith(".obsidian/workspace");
}); });
it("returns focused views without registering either context as an add-on", () => { it("returns a concrete repair adapter instead of exposing the broad context to UI", async () => {
const { customisationSync, feature, hiddenFileSync } = createFixture(); const {
customisationSync,
customisationTesting,
feature,
hiddenFileSync,
hiddenFileTesting,
} = createFixture();
expect(feature.customisationSync).toBe(customisationSync); expect(feature.customisationSync).toBe(customisationSync);
expect(feature.hiddenFileSyncCommands).toBe(hiddenFileSync); expect(feature.hiddenFileSyncCommands).toBe(hiddenFileSync);
expect(feature.hiddenFileSyncInitialisation).toBe(hiddenFileSync); expect(feature.hiddenFileSyncInitialisation).toBe(hiddenFileSync);
expect(feature.hiddenFileSyncRepair).toBe(hiddenFileSync); expect(feature.hiddenFileSyncRepair).toBe(hiddenFileSync.repair);
expect(feature.testing).toEqual({ customisationSync, hiddenFileSync }); expect(feature.hiddenFileSyncRepair).not.toBe(hiddenFileSync);
expect(Object.isFrozen(feature.hiddenFileSyncRepair)).toBe(true);
expect(Object.keys(feature.hiddenFileSyncRepair).sort()).toEqual(
[
"extractInternalFileRevisionFromDatabase",
"scanInternalFiles",
"storeInternalFileToDatabase",
"storeInternalFileToDatabaseWithBaseRevision",
].sort()
);
const file = {
path: ".obsidian/app.json",
ctime: 1,
mtime: 2,
size: 3,
} as never;
await feature.hiddenFileSyncRepair.storeInternalFileToDatabaseWithBaseRevision(file, "2-selected", false);
expect(hiddenFileSync.repair.storeInternalFileToDatabaseWithBaseRevision).toHaveBeenCalledWith(
file,
"2-selected",
false
);
expect(feature.testing).toEqual({
customisationSync: customisationTesting,
hiddenFileSync: hiddenFileTesting,
});
expect(Object.isFrozen(feature.testing)).toBe(true); expect(Object.isFrozen(feature.testing)).toBe(true);
expect(feature.testing.customisationSync).not.toBe(customisationSync);
expect(feature.testing.hiddenFileSync).not.toBe(hiddenFileSync);
}); });
}); });
@@ -180,7 +180,7 @@ async function resolveHiddenConflicts(cliBinary: string, env: NodeJS.ProcessEnv)
"(async()=>{", "(async()=>{",
"const core=app.plugins.plugins['obsidian-livesync'].core;", "const core=app.plugins.plugins['obsidian-livesync'].core;",
"const syncContext=app.plugins.plugins['obsidian-livesync'].optionalFileSync.testing.hiddenFileSync;", "const syncContext=app.plugins.plugins['obsidian-livesync'].optionalFileSync.testing.hiddenFileSync;",
"await syncContext.resolveConflictOnInternalFiles();", "await syncContext.conflictResolution.resolveAll();",
"await syncContext.scanAllDatabaseChanges(true);", "await syncContext.scanAllDatabaseChanges(true);",
"return JSON.stringify({ok:true});", "return JSON.stringify({ok:true});",
"})()", "})()",
@@ -192,40 +192,46 @@ async function resolveHiddenConflicts(cliBinary: string, env: NodeJS.ProcessEnv)
async function autoMergeHiddenJsonConflict(cliBinary: string, env: NodeJS.ProcessEnv, path: string): Promise<void> { async function autoMergeHiddenJsonConflict(cliBinary: string, env: NodeJS.ProcessEnv, path: string): Promise<void> {
await evalObsidianJson<unknown>( await evalObsidianJson<unknown>(
cliBinary,
[
"(async()=>{",
`const path=${JSON.stringify(path)};`,
"const syncContext=app.plugins.plugins['obsidian-livesync'].optionalFileSync.testing.hiddenFileSync;",
"await syncContext.conflictResolution.resolveAll();",
"await syncContext.scanAllDatabaseChanges(true);",
"return JSON.stringify({ok:true,path});",
"})()",
].join(""),
env
);
}
async function readHiddenConflictDiagnostics(
cliBinary: string,
env: NodeJS.ProcessEnv,
path: string
): Promise<unknown> {
return await evalObsidianJson<unknown>(
cliBinary, cliBinary,
[ [
"(async()=>{", "(async()=>{",
`const path=${JSON.stringify(path)};`, `const path=${JSON.stringify(path)};`,
"const prefixedPath=`i:${path}`;", "const prefixedPath=`i:${path}`;",
"const core=app.plugins.plugins['obsidian-livesync'].core;", "const core=app.plugins.plugins['obsidian-livesync'].core;",
"const syncContext=app.plugins.plugins['obsidian-livesync'].optionalFileSync.testing.hiddenFileSync;", "const owner=app.plugins.plugins['obsidian-livesync'].optionalFileSync.testing.hiddenFileSync.conflictResolution;",
"let doc=false;", "const entries=[];",
"for await (const entry of core.localDatabase.findEntries('i:','i;',{conflicts:true})){", "for await (const entry of core.localDatabase.findEntries('i:','i;',{conflicts:true})){",
" if(entry.path===prefixedPath){ doc=entry; break; }", " if(entry.path===prefixedPath){",
" entries.push({id:entry._id,path:entry.path,rev:entry._rev,conflicts:entry._conflicts});",
" }",
"}", "}",
"if(!doc) throw new Error(`Could not find hidden conflict candidate: ${path}`);", "const modals=Array.from(document.querySelectorAll('.modal-container')).map((element)=>element.textContent?.trim());",
"if(!doc._conflicts?.length) throw new Error(`Hidden file has no conflicts: ${path}`);", "return JSON.stringify({",
"const conflicts=doc._conflicts.sort((a,b)=>Number(a.split('-')[0])-Number(b.split('-')[0]));", " entries,",
"const conflictedRev=conflicts[0];", " pendingPaths:Array.from(owner.pendingPaths??[]),",
"const conflictedRevNo=Number(conflictedRev.split('-')[0]);", " processor:{remaining:owner.processor?.remaining,totalRemaining:owner.processor?.totalRemaining,nowProcessing:owner.processor?.nowProcessing},",
"const revFrom=await core.localDatabase.getRaw(doc._id,{revs_info:true});", " modals,",
"const commonBase=(revFrom._revs_info||[])", "});",
" .filter((rev)=>rev.status==='available'&&Number(rev.rev.split('-')[0])<conflictedRevNo)",
" .map((rev)=>rev.rev)[0]||'';",
"const result=await core.localDatabase.managers.conflictManager.mergeObject(",
" doc.path, commonBase, doc._rev, conflictedRev",
");",
"if(!result){",
" throw new Error(`Hidden JSON conflict was not auto-mergeable: ${path}; base=${commonBase}; current=${doc._rev}; conflict=${conflictedRev}`);",
"}",
"await syncContext.ensureDir(path);",
"const stat=await syncContext.writeFile(path,result);",
"if(!stat) throw new Error(`Could not write merged hidden file: ${path}`);",
"await syncContext.storeInternalFileToDatabase({path,mtime:stat.mtime,ctime:stat.ctime,size:stat.size},true);",
"await core.localDatabase.removeRevision(doc._id,conflictedRev);",
"await syncContext.extractInternalFileFromDatabase(path);",
"await syncContext.scanAllDatabaseChanges(true);",
"return JSON.stringify({ok:true,merged:JSON.parse(result)});",
"})()", "})()",
].join(""), ].join(""),
env env
@@ -250,7 +256,7 @@ async function openHiddenJsonResolveModal(cliBinary: string, env: NodeJS.Process
"const docA=await core.localDatabase.getDBEntry(prefixedPath,{rev:doc._rev});", "const docA=await core.localDatabase.getDBEntry(prefixedPath,{rev:doc._rev});",
"const docB=await core.localDatabase.getDBEntry(prefixedPath,{rev:conflicts[0]});", "const docB=await core.localDatabase.getDBEntry(prefixedPath,{rev:conflicts[0]});",
"if(docA===false||docB===false) throw new Error(`Could not load conflicted hidden JSON entries: ${path}`);", "if(docA===false||docB===false) throw new Error(`Could not load conflicted hidden JSON entries: ${path}`);",
"void syncContext.showJSONMergeDialogAndMerge(docA,docB);", "void syncContext.conflictResolution.resolveJson(docA,docB);",
"return JSON.stringify({ok:true});", "return JSON.stringify({ok:true});",
"})()", "})()",
].join(""), ].join(""),
@@ -270,11 +276,12 @@ async function storeHiddenFileAsConflict(
"(async()=>{", "(async()=>{",
`const path=${JSON.stringify(path)};`, `const path=${JSON.stringify(path)};`,
`const baseRev=${JSON.stringify(baseRev)};`, `const baseRev=${JSON.stringify(baseRev)};`,
"const prefixedPath=`i:${path}`;",
"const core=app.plugins.plugins['obsidian-livesync'].core;", "const core=app.plugins.plugins['obsidian-livesync'].core;",
"const syncContext=app.plugins.plugins['obsidian-livesync'].optionalFileSync.testing.hiddenFileSync;", "const syncContext=app.plugins.plugins['obsidian-livesync'].optionalFileSync.testing.hiddenFileSync;",
"const fileInfo=await syncContext.loadFileWithInfo(path);", "const fileInfo=await syncContext.readFileWithInfo(path);",
"if(fileInfo.deleted) throw new Error(`Hidden file was unexpectedly deleted: ${path}`);", "if(fileInfo.deleted) throw new Error(`Hidden file was unexpectedly deleted: ${path}`);",
"const baseData=await syncContext.__loadBaseSaveData(path,true);", "const baseData=await core.localDatabase.getDBEntry(prefixedPath,undefined,false,true);",
"if(baseData===false) throw new Error(`Could not load base save data: ${path}`);", "if(baseData===false) throw new Error(`Could not load base save data: ${path}`);",
"const saveData={", "const saveData={",
" ...baseData,", " ...baseData,",
@@ -430,9 +437,21 @@ async function runJsonConflictRoundTrip(
await createHiddenJsonConflict(context, session, vaultB, mergeJsonPath, base, left, right); await createHiddenJsonConflict(context, session, vaultB, mergeJsonPath, base, left, right);
await autoMergeHiddenJsonConflict(context.cliBinary, session.cliEnv, mergeJsonPath); await autoMergeHiddenJsonConflict(context.cliBinary, session.cliEnv, mergeJsonPath);
await pushLocalChanges(context.cliBinary, session.cliEnv); await pushLocalChanges(context.cliBinary, session.cliEnv);
const mergedOnB = await waitForPathContent(vaultB.path, mergeJsonPath, (content) => let mergedOnB: string;
hasJsonValues(content, { fromA: true, fromB: true }) try {
); mergedOnB = await waitForPathContent(vaultB.path, mergeJsonPath, (content) =>
hasJsonValues(content, { fromA: true, fromB: true })
);
} catch (error) {
const diagnostics = await readHiddenConflictDiagnostics(context.cliBinary, session.cliEnv, mergeJsonPath).catch(
(diagnosticError: unknown) => ({
diagnosticError: diagnosticError instanceof Error ? diagnosticError.message : String(diagnosticError),
})
);
throw new Error(
`${error instanceof Error ? error.message : String(error)}\nConflict diagnostics: ${JSON.stringify(diagnostics)}`
);
}
await session.app.stop(); await session.app.stop();
session = await startConfiguredSession(context, vaultA); session = await startConfiguredSession(context, vaultA);
@@ -620,14 +639,11 @@ async function setHiddenFileNoticeFixtures(port: number, itemIds: string[], incl
}; };
obsidianApp.plugins.enabledPlugins.add(pluginId); obsidianApp.plugins.enabledPlugins.add(pluginId);
} }
syncContext.queuedNotificationFiles.clear(); const updatedFolders = nextItemIds.map((id) => `.obsidian/plugins/livesync-e2e-${id}`);
for (const id of nextItemIds) {
syncContext.queuedNotificationFiles.add(`.obsidian/plugins/livesync-e2e-${id}`);
}
if (nextIncludeRestart) { if (nextIncludeRestart) {
syncContext.queuedNotificationFiles.add(core.services.API.getSystemConfigDir()); updatedFolders.push(core.services.API.getSystemConfigDir());
} }
syncContext.notifyConfigChange(); syncContext.showConfigurationChangeNotice(updatedFolders);
}, },
{ nextItemIds: itemIds, nextIncludeRestart: includeRestart } { nextItemIds: itemIds, nextIncludeRestart: includeRestart }
); );
@@ -683,7 +699,6 @@ async function runInitialisationNoticeGrouping(context: RunnerContext, vault: Te
const syncContext = plugin.optionalFileSync.testing.hiddenFileSync; const syncContext = plugin.optionalFileSync.testing.hiddenFileSync;
const setting = core.services.setting; const setting = core.services.setting;
const originalApplyPartial = setting.applyPartial; const originalApplyPartial = setting.applyPartial;
const originalRebuildMerging = syncContext.rebuildMerging;
const state = { const state = {
done: false, done: false,
reachedPreparation: false, reachedPreparation: false,
@@ -731,13 +746,15 @@ async function runInitialisationNoticeGrouping(context: RunnerContext, vault: Te
return await originalApplyPartial.apply(setting, args); return await originalApplyPartial.apply(setting, args);
}; };
syncContext.rebuildMerging = async (...args: unknown[]) => { const restoreRebuildMerging = syncContext.interceptRebuildMerging(
state.reachedInitialisation = true; async (runRebuild: (...args: unknown[]) => Promise<unknown>, ...args: unknown[]) => {
await new Promise<void>((resolve) => { state.reachedInitialisation = true;
state.releaseInitialisation = resolve; await new Promise<void>((resolve) => {
}); state.releaseInitialisation = resolve;
return await originalRebuildMerging.apply(syncContext, args); });
}; return await runRebuild(...args);
}
);
void core.services.setting void core.services.setting
.enableOptionalFeature("MERGE") .enableOptionalFeature("MERGE")
@@ -752,7 +769,7 @@ async function runInitialisationNoticeGrouping(context: RunnerContext, vault: Te
) )
.finally(() => { .finally(() => {
setting.applyPartial = originalApplyPartial; setting.applyPartial = originalApplyPartial;
syncContext.rebuildMerging = originalRebuildMerging; restoreRebuildMerging();
const notices = Array.from(document.querySelectorAll<HTMLElement>(".notice")); const notices = Array.from(document.querySelectorAll<HTMLElement>(".notice"));
const progressNotices = notices.filter((notice) => notice.textContent?.includes("[⚙")); const progressNotices = notices.filter((notice) => notice.textContent?.includes("[⚙"));
state.sawStandaloneGatheringNotice ||= notices.some((notice) => state.sawStandaloneGatheringNotice ||= notices.some((notice) =>