Compare commits

..
137 changed files with 16309 additions and 5275 deletions
+2 -15
View File
@@ -158,6 +158,8 @@ Use interaction-based, London School unit tests for the composition boundary. Ve
See [Service feature and legacy Module boundaries](docs/design_docs/service_feature_and_legacy_module_boundaries.md) for the selection criteria, current examples, reasons to avoid new `AbstractModule` subclasses, incremental migration guidance, and test shapes. Commonlib's [service feature composition guide](https://github.com/vrtmrz/livesync-commonlib/blob/main/docs/service-feature-composition.md) defines the shared host-neutral boundary.
The implemented joint composition, local-owner routing, private contexts, host adapters, and focused views for Customisation Sync and Hidden File Sync are documented in [Optional-file synchronisation architecture](docs/design_docs/optional_file_sync_architecture.md).
Legacy Modules remain grouped by directory:
- `core/` contains platform-independent core behaviour;
@@ -238,21 +240,6 @@ Commonlib owns the typed English fallback for messages requested by its services
- Dev mode creates `ls-debug/` folder in `.obsidian/` for debug outputs (e.g., missing translations)
- This causes pretty significant performance overhead.
#### Diagnostic and notice ownership
- A Commonlib or service operation should normally record detailed diagnostics at `LOG_LEVEL_VERBOSE` and return a typed result which lets its caller distinguish complete, partial, and failed outcomes. Do not make callers infer an outcome by parsing log text.
- Detailed diagnostics may be long and remain in English when they are intended for tracing and the generated report. Include enough context to identify the operation, affected target, and remaining state or retry behaviour.
- The application boundary which owns the workflow should decide whether to raise `LOG_LEVEL_NOTICE`. It has the interaction context to describe the user-visible consequence and the next useful action; an internal stage description alone is not a useful notice.
- When several files fail, issue one concise summary notice after the operation returns. Keep the per-file paths and technical causes at verbose level so that the notice remains readable and the generated report remains traceable.
- Commonlib should raise a notice only when its contract explicitly owns user presentation and no higher-level caller can add the required workflow context.
The ordinary start-up scan provides a concrete comparison:
- Good verbose diagnostic: `Offline scan failed to synchronise ${path} between storage and the local database; this path remains eligible for a later scan.` It identifies the operation, the two states being reconciled, the exact target, and what can happen next. Its length is appropriate for a report.
- Notice which needs more context: `Local database initialisation did not complete. See the log for details.` It describes an internal stage, but does not tell the user whether synchronisation can continue, what may be affected, or how to obtain the detailed log.
- Good application notice for a partial result: `Not all files could be synchronised. Check the affected files. Generate a report to review the detailed log.` It states the observable consequence, gives a proportionate action, and leaves the per-file evidence in the report.
- Good application notice for a failed result: `Self-hosted LiveSync cannot synchronise. Generate a report to review the detailed log.` It states the operational consequence without exposing the internal initialisation stage.
## Common Patterns
### Service feature implementation
@@ -0,0 +1,553 @@
---
date: 2026-09-03
commonlib-version: "0.1.21"
self-hosted-livesync-version: "1.0.24"
status: accepted
---
# Architectural Decision Record: Customisation Sync and Hidden File Sync ownership
## Status
Accepted and implemented. The completed structural stages
extract the Customisation Sync path and codec operations, characterise the
current shared routing behaviour, remove the production UI dependency cycle,
move host presentation and Hidden File Sync commands into serviceFeatures,
give one optional-file serviceFeature ownership of both runtime contexts and
all Service handler registration, and replace both runtimes' complete-core
dependencies with narrow host adapters. A pure local-path policy now selects
one writer for raw events, scheduled scans, and automatic database-to-local
reflection. The constructor-name add-on lookup and the `ConfigSync` and
`HiddenFileSync` add-on identities have been retired.
The Customisation Sync private context coordinates lifecycle, raw-event
admission, configuration and periodic policy, view composition, and the
lifetimes of focused path, snapshot, application, scan, and catalogue owners.
The Hidden File Sync private context coordinates lifecycle and handler
admission around focused path-admission, notification, processed-state,
change-processing, conflict-resolution, and reconciliation owners. Each
receives live settings and database projections, focused storage, path, and
exact-revision capabilities, and explicit host effects rather than
`LiveSyncCore`.
The corresponding implemented topology is documented in
[Optional-file synchronisation architecture](../design_docs/optional_file_sync_architecture.md).
## Context
Customisation Sync and Hidden File Sync are supported, advanced, opt-in
features with different data and interaction contracts:
- Customisation Sync stores device-scoped snapshots in the `ix:` namespace.
It supports grouped V1 documents and per-file V2 documents, presents a
catalogue, and applies selected remote state only after an explicit action.
- Hidden File Sync stores one mirrored hidden file in each `i:` Metadata
document. It scans and reconciles local and database state, applies eligible
remote changes automatically, and maintains device-local processed-state
records for offline and conflict handling.
In the baseline implementation, the `ConfigSync` and `HiddenFileSync` add-ons
each combined domain operations, mutable state, Service handler registration,
lifecycle work, settings policy, commands, and Obsidian presentation. Both
received the complete core through `LiveSyncCommands`, and both registered
handlers for optional file events, conflicts, settings, application lifecycle,
database initialisation, and replication.
Some of those Commonlib handlers short-circuit at the first successful or
non-empty result. The construction order of `ConfigSync` before
`HiddenFileSync` is therefore observable behaviour. The Obsidian storage-event
manager also asks the Vault Service whether a configuration-directory file is
an extra target, while the corresponding predicate is currently registered by
`HiddenFileSync`. Local file ownership is consequently distributed across the
storage-event manager, both add-ons, settings entries, target and ignore
patterns, and handler registration order.
In the baseline implementation, user-interface, maintenance, and
real-Obsidian E2E consumers located the concrete add-ons through `getAddOn()`.
Removing either add-on before migrating those consumers would therefore have
combined an ownership change, a lifecycle change, and a consumer migration in
one step.
The baseline Customisation Sync presentation also formed a runtime import
cycle: `ConfigSync` imported its Obsidian dialogue, the dialogue imported its
Svelte pane, and the pane imported `ConfigSync` and `HiddenFileSync` to locate
both add-ons. The child row component imported `ConfigSync` again. The settings
tab also imported `main.ts` as a runtime binding even though it used the
plug-in class only as a type. Bundling survived because those bindings were
read after module evaluation, but that timing was not an architectural
contract. Presentation therefore had to move off the concrete add-ons before
the domain runtimes could be extracted.
## Decision drivers
The redesign must:
1. preserve `ix:` and `i:` data, including V1 and V2 Customisation Sync data;
2. give each local path at most one synchronisation owner;
3. remove handler registration order as the source of path ownership;
4. preserve explicit Customisation Sync application and automatic Hidden File
Sync reflection as separate behaviours;
5. make mutable state and resource disposal ownership explicit;
6. keep Obsidian presentation outside host-neutral operations;
7. migrate existing add-on consumers to focused views before retiring add-on
identity; and
8. permit each migration stage to be verified independently.
## Decision
### Retain two domain runtimes
Customisation Sync and Hidden File Sync will remain separate domain runtimes.
They will not be implemented as modes of one generic file-sync engine.
The Customisation Sync runtime will own:
- V1 and V2 codecs and document-path compatibility;
- the device catalogue and snapshot repository;
- scanning, migration, logical deletion, comparison, and explicit apply
operations; and
- the state required to serialise and report those operations.
The Hidden File Sync runtime and its composed focused owners will own:
- the device-local processed-state records;
- one-file storage and database transfer, including exact-revision repair;
- start-up, offline, periodic, and pre-replication reconciliation;
- conflict queues and automatic or interactive JSON resolution; and
- the state required to serialise and report those operations.
Neither runtime will accept `LiveSyncBaseCore` as its domain dependency.
Operations will receive narrow Services, ServiceModules, settings projections,
and host effects.
### Compose one owner for overlapping handlers
One joint serviceFeature constructs both private runtime contexts and owns
registration into the overlapping optional-file, conflict, target, settings,
database, replication, and application-lifecycle handlers. A single
optional-file callback selects its local owner before invoking either context,
and a single conflict callback dispatches the disjoint `ps:`, `ix:`, and `i:`
namespaces. Callback order and add-on construction order no longer select the
local file owner. Config-before-Hidden order remains explicit only for shared
lifecycle handlers whose side effects still require compatibility.
The composition feature will remain small. Codecs, path functions,
repositories, state transitions, and transfer operations are ordinary modules
or private contexts rather than separate serviceFeatures. No new
ServiceModule is justified while those capabilities have one composition
owner and no independent long-lived consumers.
Internally, operations will use a typed settlement which distinguishes at
least handled, skipped, and failed work. The composition boundary will adapt
that settlement to the existing Commonlib Boolean or first-result handler
contracts. Existing short-circuit and failure behaviour must be characterised
before an adapter changes it.
### Make local path ownership explicit
A pure routing policy classifies a local path as owned by Customisation
Sync, owned by Hidden File Sync, or ignored with a reason. Its inputs will
include:
- the Obsidian configuration directory;
- whether each feature is enabled and ready;
- the Customisation Sync category and extended mode;
- Hidden File Sync target and ignore patterns; and
- the ignore-file result where that asynchronous policy applies.
The maintained ownership intent is:
| Path and mode | Local owner |
| ------------------------------------------------------------ | ------------------ |
| Recognised Customisation Sync path in Selective mode | Customisation Sync |
| Recognised Customisation Sync path in Flagged Selective mode | Customisation Sync |
| Recognised Customisation Sync path in Automatic mode | Hidden File Sync |
| Recognised Customisation Sync path in Ignore mode | Neither feature |
| Other eligible hidden path | Hidden File Sync |
| Excluded, ignored, disabled, or ordinary Vault path | Neither feature |
This table is now used for raw-event dispatch and is injected into both
contexts for scheduled or database-driven local work. Default and persisted
Selective entries therefore have the same Customisation Sync owner. Hidden
File Sync target patterns and ignore-file results are evaluated only after the
policy selects Hidden File Sync, so they cannot prevent a Selective
Customisation Sync event. Automatic mode has no local owner while Hidden File
Sync is disabled, and Ignore mode has no local owner in either context.
Commonlib's ordinary event queue rejects dot paths while Hidden File Sync is
disabled. The Obsidian raw-event boundary therefore dispatches an event which
the policy assigns to Customisation Sync directly after retaining the queue's
configuration, suspension, and modification-time gates. Events handled while
Hidden File Sync is enabled continue through the ordinary queue.
This is a local-write routing table, not a database-document acceptance table.
Existing `ix:` documents must remain recognisable after a local mode change,
and they must not fall through to ordinary Vault reflection. Existing `i:`
documents retain their own namespace and selection rules. The implementation
therefore keeps namespace-specific database handlers separate from the
local-path policy rather than exposing one general `isTargetPath()` predicate.
A narrower document decision controls whether a local Customisation Sync scan
may mutate an existing `ix:` document; it does not control whether that
document is recognised or consumed.
### Use private contexts and focused views
Each mutable object will have one named owner. Catalogue stores, manifest
caches, processed-state records, recent-event records, locks, semaphores,
queues, periodic processors, Notices, and event subscriptions must be created
and disposed by their feature context or a focused resource owner.
The contexts are orchestration roots, rather than containers for every mutable
detail. `HiddenFileSyncProcessedState` owns the three device-local maps, their
autosave initialisation, exact key formats, retained known mtime, reset rules,
and settlement effects on `IPathService`. Database write and extraction
operations receive this capability through one narrow `processedState` port;
they do not receive bundles of individual state callbacks or implement state
keys themselves.
`CatalogueState` separately owns the transient catalogue rows, manifest lookup
and mtime cache, their reactive stores, and catalogue update progress. A small
recent-event deduplicator owns raw-event admission history. These Customisation
Sync owners are deliberately not implementations of a shared Hidden File Sync
state abstraction: the former state is a derived, in-memory projection, while
Hidden File Sync markers are persisted operational reconciliation state with
different identity, invalidation, and deletion rules.
`CatalogueOperations` owns catalogue enumeration, one queue and its progress
subscription, and composes `CatalogueV1`, `CatalogueV2`, and
`CatalogueMigration`. V1 and V2 are mutually exclusive as the selected write
format, but both persisted formats can coexist during migration. Queue work
therefore reads the live setting when it starts, and the shared state continues
to recognise V2 rows independently of that setting. `CatalogueMigration`
remains the distinct bridge from grouped V1 binders to per-file V2 documents.
`SnapshotPersistence` owns the host-neutral V1 grouped and V2 per-file writes
and logical deletion. It returns explicit mutation and refresh outcomes, so it
neither owns catalogue state nor calls back through the context.
`SnapshotOperations` applies those outcomes with the inherited awaited V1 or
fire-and-forget V2 timing. `ApplicationOperations` owns compare, apply,
duplicate, and delete workflows, while `ScanOperations` owns configuration-file
enumeration and V1/V2 reconciliation. Both depend on narrow snapshot and
catalogue ports instead of the context.
The newly extracted catalogue, snapshot, application, scan, and reconciliation
modules omit a Customisation Sync or Hidden File Sync prefix because their
feature directories already supply that scope. Public contexts and views retain
their domain names for compatibility.
`CustomisationSyncPathOperations` binds live configuration-directory, mode,
and device-name projections to the pure category and V1/V2 key functions. It
has no host registration or stateful application lifetime. The context uses
this capability internally; its path helpers are not re-exported through the
real-Obsidian testing view.
`HiddenFileSyncPathAdmission` owns the ownership-first eligibility sequence
and its parsed-pattern cache. `HiddenFileSyncChangeNotifier` owns the pending
folder set, delayed delivery, suppression checks, scheduled-task cancellation,
and Notice show/hide effect calls. The Obsidian adapter still owns the actual
Notice instance. These owners make cache and notification behaviour directly
testable without making either concern a serviceFeature.
`HiddenFileSyncChangeProcessor` owns storage and database change processing,
the bounded semaphore, same-path event serialisation, activity counts, and the
inherited order in which processed-state markers and transfer results settle.
This boundary keeps event concurrency and settlement directly testable without
giving the processor full scan, initialisation, notification, or host
responsibilities.
`Reconciliation` owns storage and database enumeration, full scans, offline
comparison, rebuild direction and ordering, processed-state adoption,
initialisation sequencing, and the scoped rebuild interceptor used by maintained
real-Obsidian tests. Storage and database scans remain together because every
offline and initialisation path coordinates both sides.
The joint composition may return several views backed by those contexts:
- a Customisation Sync catalogue and operation view for its dialogue;
- a Hidden File Sync initialisation view for settings workflows;
- a Hidden File Sync repair view for the Hatch pane;
- immutable semantic handler views for registration by the joint composition;
and
- explicitly internal testing views for maintained real-Obsidian workflows.
Several views over one context do not create several owners. Views expose
stable application data and named operations rather than PouchDB entries,
queue objects, mutable settings records, dependency objects, or the complete
core. The testing views also avoid exposing context instances or writable
internal state; time-sensitive E2E work uses a scoped operation interceptor.
Obsidian commands, ribbon actions, dialogues, Notices, plug-in reloads, and
restart scheduling will remain in host-owned composition. The UI will receive
focused views instead of locating `ConfigSync` or `HiddenFileSync` and calling
one add-on from the other.
The Customisation Sync dialogue has its own host-owned presentation
serviceFeature. It owns command, ribbon, event subscription, dialogue reuse,
and dialogue disposal, and consumes the catalogue and Hidden File Sync
initialisation views. Neither domain runtime imports its Obsidian dialogue or
Svelte components. This presentation feature is separate from the joint
synchronisation owner because it registers host UI rather than overlapping
file, conflict, or replication handlers.
Hidden File Sync commands and their setting-change event subscription are
owned by a second host serviceFeature. It consumes only the command view and
releases its lifecycle and event registrations during application unload.
### Retire compatibility façades after consumer migration
The concrete `ConfigSync` and `HiddenFileSync` add-on identities were retained
only until their consumers and handler ownership had migrated. They have now
been replaced by private `CustomisationSyncContext` and
`HiddenFileSyncContext` instances constructed solely by the joint
serviceFeature. `LiveSyncBaseCore.getAddOn()` and its constructor-name lookup
have been removed.
Production code consumes focused views and does not use the broad context
surface. Maintained real-Obsidian E2E workflows use explicitly internal,
immutable test views exposed by the composed feature. These are transitional
test seams, not production service locators, and they should be narrowed as
those workflows move to public operations or commands.
The retirement was gated on:
- production UI and maintenance consumers no longer use `getAddOn()` for these
features;
- maintained E2E helpers use commands or an explicit test view;
- no handler depending on add-on construction order; and
- unload and replacement tests prove that every owned processor,
subscription, queue, dialogue, and Notice is released.
## Persisted compatibility
This decision does not authorise a data migration.
- `ix:` remains the Customisation Sync Metadata namespace.
- `i:` remains the Hidden File Sync Metadata namespace.
- Metadata continues to reference Chunks rather than embedding raw file
content as an ordinary Metadata field.
- V1 grouped Customisation Sync documents remain readable.
- V2 per-file Customisation Sync paths and the existing V1-to-V2 migration
remain readable and idempotent.
- Device and Vault terms, logical deletions, revision identifiers, and current
path derivation remain compatibility inputs.
- Hidden File Sync processed-state keys remain device-local state unless a
separately tested migration is introduced.
Pure extraction must preserve exact case, depth, prefix, delimiter, and
fallback behaviour even where a later correction appears desirable.
## Lifecycle and failure boundaries
Feature composition will occur after required Services and ServiceModules
exist and before lifecycle-driven work begins. Command and Obsidian UI
registration will occur at the readiness point required by the host.
The owner will:
- start periodic work only while its feature is enabled, ready, and resumed;
- coalesce or serialise work within the feature instance rather than through
process-global string keys where practical;
- stop admission before disposing queues or processors;
- unsubscribe every registered local event listener; and
- close owned dialogues and Notices during unload.
A failed operation must not be converted to handled success merely to stop a
later handler, unless a characterisation test proves that the existing
contract deliberately consumes that failure. Such compatibility adaptations
must be visible at the composition boundary.
## Migration and verification sequence
### Stage 1: characterise pure and routing contracts — implemented
- Cover current Customisation Sync category and V1/V2 document-path rules.
- Cover codec round trips, legacy JSON and YAML fallbacks, and migration
sentinels before extracting the codec.
- Record the effective Selective, Automatic, Ignore, and Flagged Selective
routing matrix, including feature-disabled and pattern-excluded cases.
- Record handler registration, short-circuiting, and teardown behaviour.
### Stage 2: extract pure operations and the presentation boundary — implemented
- Move Customisation Sync path and document-key functions behind the existing
façade methods.
- Move the V1/V2 codec with its hash and YAML dependencies made explicit.
- Inject catalogue, initialisation, and repair views into production UI.
- Move Customisation Sync command, ribbon, event subscription, and dialogue
lifetime into a host-owned presentation serviceFeature.
- Prohibit presentation imports of either concrete add-on or the application
core, and prohibit the runtime from importing its dialogue.
### Stage 3: make routing ownership explicit — implemented
- Introduce the pure routing policy, initially adapting both runtime contexts
to it at the joint composition boundary.
- Preserve the characterised legacy routing until each discrepancy has its own
behavioural decision and regression test.
One optional-file handler now dispatches exactly one selected context, and one
namespace router handles optional conflicts. Both contexts receive the same
static ownership projections for raw events and scans. The final raw-event
decision additionally includes lifecycle readiness, Hidden File Sync patterns,
and the asynchronous ignore-file result. Handler failure does not fall through
to the non-owner.
### Stage 4: extract the Customisation Sync runtime — implemented
- Move catalogue, snapshot repository, scan, apply, compare, delete, and
migration operations into a private context.
- Replace module-global mutable state with context-owned state.
- Replace the complete-core dependency with narrow dependencies.
The private context, path module, codec module, focused presentation view, and
resource teardown are implemented. A focused path capability binds live
settings and device identity to the pure path functions. A focused catalogue
owner holds one queue, its progress subscription, and shared state, while
separate V1, V2, and migration modules hold format-specific behaviour. A
bounded deduplicator owns recent raw-event keys. Host-neutral snapshot
persistence owns V1 grouped and V2 per-file writes, unchanged-content checks,
and logical deletion. A snapshot coordinator applies its explicit refresh
outcomes without a catalogue-to-context callback cycle. Focused application
and scan owners contain selected-snapshot workflows and full local/database
reconciliation, respectively. The context retains raw-event admission and
scheduling, configuration and periodic policy, owner lifetime, and view
composition. It accepts only narrow, live projections and explicit effects; an
Obsidian adapter at the composition edge owns dialogues, Notices, plug-in
reload, restart, lifecycle, Vault access, and compatibility scan telemetry.
### Stage 5: extract the Hidden File Sync runtime — implemented
- Move processed-state, transfer, reconciliation, conflict, and notification
operations into explicit owners.
- Preserve exact-revision repair and current initialisation directions.
- Replace the complete-core dependency with narrow dependencies.
The private context, focused initialisation, repair, and command views,
host-owned command registration, and processor, cache, subscription, and
Notice teardown are implemented. A focused processed-state owner holds all
three persisted maps, their key and mtime rules, reset operations, and
cross-side settlement. Database write and extraction operations consume one
narrow state port. A focused path-admission owner holds the pattern cache and
the ownership, static-path, pattern, and ignore-file sequence. A focused change
notifier owns folder batching, delayed delivery, and teardown of its scheduled
work and Notice effect. A focused change processor owns storage and database
event processing, bounded concurrency, per-path serialisation, activity
publication, and compatibility settlement order. A focused reconciliation
owner owns storage and database scans, offline comparison, rebuilds,
processed-state adoption, initialisation direction and ordering, and its scoped
testing interceptor. A focused conflict-resolution owner owns pending-path admission,
the parallel classification and serial interaction queues, automatic merge,
newer-revision selection, interactive JSON application, settlement, and queue
disposal. An Obsidian adapter owns JSON conflict dialogue instances, progress
presentation, grouped Notices, plug-in reload, restart scheduling, Vault
enumeration, and compatibility activity publication.
### Stage 6: move synchronisation composition — implemented
- Register the overlapping Service handlers once through the joint
serviceFeature.
- Consume immutable semantic handler views rather than exposing registry-style
methods on either context.
- Preserve the characterised lifecycle callback order and Commonlib
aggregation semantics through focused tests.
### Stage 7: retire the façades — implemented
- Remove add-on identity and constructor-order dependencies.
- Migrate maintained E2E workflows to the explicit feature test boundary.
- Remove constructor-name service lookup from the core.
The implemented topology is documented separately from this migration record
in [Optional-file synchronisation architecture](../design_docs/optional_file_sync_architecture.md).
Each stage will run focused unit tests. Changes to Customisation Sync and
Hidden File Sync will run their respective real-Obsidian E2E workflows. The
composition switch will additionally require a mixed-ownership workflow which
proves that one local path is never written by both features.
## Non-goals
This migration does not:
- merge the two persisted namespaces or sync models;
- move implementation into Commonlib before another maintained host needs the
capability;
- change feature maturity, default enablement, setup, or initialisation;
- replace current conflict policy with a generic conflict engine;
- rename existing setting keys, command identifiers, or user-interface labels;
or
- correct unrelated suspicious behaviour while extracting code.
## Characterisation gates
Stage 3 resolved the routing-specific gates with focused regressions:
- an unrecognised eligible hidden path is owned by Hidden File Sync;
- default and persisted Selective entries are owned by Customisation Sync;
- raw Customisation Sync remains available while Hidden File Sync is disabled;
- Hidden File Sync patterns and ignore-file results gate only its selected
paths;
- Automatic mode without an enabled and ready Hidden File Sync owner is
ignored rather than falling back to Customisation Sync;
- a selected handler which skips or fails does not fall through to the other
context; and
- Customisation Sync raw admission now invokes the readiness predicate and
rejects unrecognised or non-owned paths.
The extraction preserves the existing V1 and V2 plug-in application paths,
exact-revision repair, initial cache conditions, and the selected Hidden File
Sync database-processing settlement. These remain compatibility gates for any
future behavioural change. Any defect correction requires its own failing
regression.
## Alternatives rejected
### Convert each extracted file into a serviceFeature
This would reproduce distributed registration and lifetime ownership under
more function names. Pure operations and one private context are the narrower
boundary.
### Merge both features into one generic hidden-file engine
This would obscure the explicit-apply snapshot contract, the automatic mirror
contract, and their incompatible persistence and conflict semantics.
### Remove both add-ons before characterisation and consumer migration
This would change identity, ordering, lifecycle, UI, maintenance, and E2E
boundaries simultaneously. Retaining identity through the earlier migration
stages gave each structural change a smaller failure surface.
### Move the runtimes to Commonlib first
There is no second maintained consumer for the complete feature behaviour at
present. Moving the monoliths across the package boundary would enlarge the
migration without first establishing narrow dependencies.
## Consequences
- Local path ownership can become explicit at one composition boundary.
- Persisted formats and supported user workflows remain stable during the
migration.
- The composition root gains several focused views but does not gain another
runtime service locator.
- The legacy add-on identity and constructor-name lookup are removed.
- The two contexts coordinate separate synchronisation workflows, but their
dependency surfaces are explicit and do not include the complete core.
Customisation Sync delegates its path binding, snapshot persistence and
refresh sequencing, selected-snapshot application, scan reconciliation,
derived catalogue, catalogue queue, and recent-event state, while Hidden File
Sync delegates path admission, notification, processed-state,
change-processing, reconciliation, and conflict lifecycles to focused owners.
Further extraction should follow a concrete behavioural boundary rather than
create additional serviceFeatures for private operations.
## References
- [Feature maturity for 1.0](2026_07_feature_maturity_for_1_0.md)
- [Service feature and legacy Module boundaries](../design_docs/service_feature_and_legacy_module_boundaries.md)
- [Optional-file synchronisation architecture](../design_docs/optional_file_sync_architecture.md)
- [Hidden File Sync guide](../tips/hidden-file-sync.md)
- [Settings reference](../settings.md#6-customisation-sync-advanced)
- [Development guide](../../devs.md#service-composition-and-legacy-modules)
@@ -0,0 +1,318 @@
---
date: 2026-09-03
commonlib-version: "0.1.21"
self-hosted-livesync-version: "1.0.24"
status: accepted
---
# Optional-file synchronisation architecture
## Purpose and scope
This document describes the implemented ownership and composition of
Customisation Sync and Hidden File Sync. These features share host event
boundaries, but retain separate persisted data, state, and application
behaviour. The corresponding decision history and migration constraints are
recorded in the
[Customisation Sync and Hidden File Sync ownership ADR](../adr/2026_09_customisation_and_hidden_file_sync_ownership.md).
The implementation is private application architecture. It does not define a
third-party extension API, a runtime feature registry, or a generic hidden-file
engine.
## Topology
```text
Obsidian composition (`main.ts`)
|
+--> `useOptionalFileSync`
| |
| +--> one Service-handler registration owner
| +--> pure local-path and document routing policy
| |
| +--> `CustomisationSyncContext`
| | +-- `CustomisationSyncPathOperations`
| | +-- `SnapshotPersistence`
| | +-- `SnapshotOperations`
| | +-- `ApplicationOperations`
| | +-- `ScanOperations`
| | +-- recent-event deduplicator
| | +-- immutable service-handler and testing views
| | |
| | +--> `CatalogueOperations`
| | +-- `CatalogueState`
| | +-- one catalogue queue and progress lifecycle
| | +-- `CatalogueV1`
| | +-- `CatalogueV2`
| | +-- `CatalogueMigration`
| | ^
| | +-- narrow dependencies from
| | `customisationSyncObsidianAdapter`
| |
| +--> `HiddenFileSyncContext`
| ^
| +-- narrow dependencies from
| `hiddenFileSyncObsidianAdapter`
| |
| +--> `HiddenFileSyncProcessedState`
| | +-- three autosaved reconciliation maps
| | +-- key, mtime, reset, and settlement rules
| |
| +--> `HiddenFileSyncChangeProcessor`
| | +-- storage and database change processing
| | +-- per-path serialisation and activity state
| |
| +--> `HiddenFileSyncConflictResolution`
| | +-- pending paths and two-stage conflict queue
| | +-- automatic and interactive JSON resolution
| |
| +--> `HiddenFileSyncPathAdmission`
| | +-- ownership, path, pattern, and ignore-file admission
| | +-- per-context parsed-pattern cache
| |
| +--> `HiddenFileSyncChangeNotifier`
| | +-- changed-folder batching and scheduled delivery
| | +-- suppression and Notice-effect teardown
| |
| +--> `Reconciliation`
| | +-- storage and database scans
| | +-- offline reconciliation, rebuilds, and initialisation
| |
| +-- immutable service-handler, command, repair, and testing views
|
+--> `useCustomisationSyncUI`
| +-- catalogue and operation view
| +-- Hidden File Sync initialisation view
|
+--> `useHiddenFileSyncCommands`
+-- Hidden File Sync command view
Hatch settings consumer
+-- Hidden File Sync exact-revision repair view
```
`useOptionalFileSync` is the only owner of the overlapping synchronisation
registrations. The presentation serviceFeatures receive focused views from
that composition; they do not locate either concrete context through the
application core.
## Ownership
| Owner | Owns | Does not own |
| ------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `useOptionalFileSync` | Construction of both contexts, handler registration and removal, local-owner selection, namespace dispatch, compatibility callback order, and context disposal order. | Persisted feature state, synchronisation algorithms, commands, dialogues, or Notices. |
| `CustomisationSyncContext` | Lifecycle and view composition, raw-event admission and scheduling, configuration transitions, periodic scan policy, and focused-owner lifetimes. | Scan reconciliation, snapshot writes or refresh sequencing, catalogue internals, application and comparison algorithms, Obsidian dialogues, ribbon actions, or handler registration. |
| `CustomisationSyncPathOperations` | Binding live configuration-directory, mode, and device-name projections to the pure category, target-path, V1 key, V2 key, and device-prefix functions. | I/O, mutable state, local-owner selection, or persistence. |
| `SnapshotPersistence` | V1 grouped and V2 per-file local-to-database writes, unchanged-content checks, logical deletion, and explicit catalogue-refresh outcomes. | Catalogue state or refresh execution, scans, lifecycle policy, application, dialogues, or plug-in reload. |
| `SnapshotOperations` | Applying persistence outcomes to the catalogue with the inherited awaited V1 and fire-and-forget V2 refresh timing. | Snapshot encoding, catalogue state, scans, lifecycle policy, or host effects. |
| `ApplicationOperations` | Comparing, applying, duplicating, and deleting selected Customisation Sync snapshots through narrow storage, snapshot, and catalogue ports. | Catalogue enumeration, raw-event admission, periodic scheduling, or view composition. |
| `ScanOperations` | Configuration-file enumeration and V1/V2 reconciliation with local and database state. | Periodic scheduling, raw-event admission, snapshot persistence details, or catalogue state. |
| `CatalogueOperations` | Catalogue enumeration and publication, one format-dispatching queue and its progress lifecycle, and composition of the state, V1, V2, and migration modules. | Local-file scanning, snapshot application, raw-event routing, dialogues, or handler registration. |
| `CatalogueState` | Transient catalogue rows, manifest lookup and mtime cache, reactive catalogue and manifest stores, and catalogue update progress. | Database or storage I/O, scan scheduling, routing, or persistence. |
| `CatalogueV1` | Loading and publishing grouped V1 catalogue rows. | V2 decoding, migration, queue lifetime, or local-file scanning. |
| `CatalogueV2` | Building and updating per-file V2 rows and manifests. | V1 loading, migration, queue lifetime, or local-file scanning. |
| `CatalogueMigration` | Translating a grouped V1 binder into V2 per-file documents, deleting the migrated binder, and applying its required V1 refresh. | General catalogue enumeration, queue ownership, or local-file scanning. |
| Customisation Sync event deduplicator | The bounded, newest-first keys used to admit raw configuration events once. | Scheduling, path ownership, catalogue state, or persisted data. |
| `HiddenFileSyncContext` | Lifecycle and view composition, handler admission, configuration transitions, periodic scan state, exact-revision repair composition, and focused-owner lifetimes. | Scan and rebuild algorithms, path admission state, change-event serialisation, processed-state representation, notification batching, conflict queue state, Obsidian dialogues, or service handler registration. |
| `HiddenFileSyncProcessedState` | Three device-local autosaved maps, exact storage and database keys, retained known mtime, reset behaviour, and storage/database settlement effects. | File transfer, scan scheduling, conflict handling, or presentation. |
| `HiddenFileSyncChangeProcessor` | Storage and database change processing, same-path event serialisation, bounded concurrency, activity counts, and the inherited event-consumption and settlement order. | Full scans, initialisation policy, notification presentation, or conflict interaction. |
| `Reconciliation` | Storage and database enumeration, full scans, offline reconciliation, rebuild direction and ordering, processed-state adoption, initialisation sequencing, and the scoped rebuild interceptor. | Individual transfer implementation, conflict interaction, path-pattern state, periodic scheduling, or host lifecycle. |
| `HiddenFileSyncConflictResolution` | Conflict admission and deduplication, pending paths, the parallel classification and serial interaction queues, automatic merge, newer-revision policy, interactive JSON resolution, and conflict settlement. | Obsidian dialogue instances, general Hidden File Sync scans, processed-state caches, or Service handler registration. |
| `HiddenFileSyncPathAdmission` | The ownership-first eligibility sequence, hidden-path and pattern policy, asynchronous ignore-file check, and the per-context parsed-pattern cache. | Composition-level owner selection, scans, transfer, or persistence. |
| `HiddenFileSyncChangeNotifier` | Changed-folder deduplication, delayed delivery, live suppression and configuration-directory checks, scheduled-task cancellation, and the host Notice show/hide effects. | The Obsidian Notice instance, file extraction, or scan policy. |
| Customisation Sync Obsidian adapter | Obsidian conflict selection, Notice presentation, plug-in reload, restart requests, Vault enumeration, progress telemetry, and platform-derived fallback device names. | Catalogue state, routing, or persisted document operations. |
| Hidden File Sync Obsidian adapter | JSON conflict dialogue lifetime, progress presentation, grouped change Notices, plug-in reload actions, restart scheduling, Vault enumeration, and compatibility activity publication. | Transfer, reconciliation, processed-state, or conflict decisions. |
| `useCustomisationSyncUI` | Command, ribbon, dialogue, open-request subscription, and their unload teardown. | Synchronisation state or Hidden File Sync initialisation behaviour. |
| `useHiddenFileSyncCommands` | Hidden File Sync command registration, setting-change subscription, and their unload teardown. | Synchronisation state or command implementation. |
The two domain contexts coordinate one cohesive synchronisation workflow each.
Their private operations and focused owners are not additional serviceFeatures:
they do not independently register host integration or have separate
application lifetimes. The newly extracted catalogue, snapshot, application,
scan, and reconciliation modules omit the domain prefix because their feature
directories already supply that scope; public context and view names retain it
for compatibility. `CustomisationSyncPathOperations` is a stateless capability
which binds live inputs to pure path functions.
`SnapshotPersistence` is a host-neutral operation boundary: it returns
structured mutation and refresh outcomes without reaching into the catalogue
or presentation. `SnapshotOperations` consumes those outcomes and preserves
their refresh timing. `ApplicationOperations` and `ScanOperations` depend on
those narrow ports instead of calling back through the context.
`CatalogueOperations` owns one catalogue queue, its progress subscription, and
the shared state projected by the format-specific modules. The live V2 setting
selects V1 loading or V1-to-V2 migration when each queued item starts. Persisted
V1 and V2 documents can coexist during migration, so V2 documents remain
recognisable regardless of the currently selected write format. `CatalogueV1`
and `CatalogueV2` contain only their format-specific catalogue behaviour, while
`CatalogueMigration` remains the explicit bridge between them. The Hidden File
Sync path-admission owner holds the parsed-pattern cache, and its change
notifier holds the pending folder set and scheduled-task lifetime.
`HiddenFileSyncChangeProcessor` is a focused resource owner because its
semaphore, per-path serialisation, activity counters, and event settlement form
one independently testable lifecycle.
`HiddenFileSyncConflictResolution` is another focused resource owner because
conflict admission, pending-path identity, two serialisation stages, and
disposal form a separate lifecycle.
`Reconciliation` keeps storage and database enumeration together because
offline comparison, rebuilds, and each initialisation direction depend on both
sides and their ordered processed-state adoption.
The two state owners intentionally do not implement a common generic state
contract. Customisation Sync projects transient catalogue and presentation
state from `ix:` documents. Hidden File Sync persists operational markers used
for reconciliation, with distinct path identity, deletion, reset, and retained
mtime rules. Their common boundary is lifecycle ownership by a context, rather
than interchangeable state semantics.
## Routing and handler contracts
Local file ownership is selected before either context processes an event.
`optionalFileSyncRouting.ts` combines the configuration directory, feature
enablement, Customisation Sync mode, and current path category. Hidden File
Sync path admission then checks ownership again before reading its current
target patterns, ignore patterns, or ignore-file result. This keeps the same
guard available to raw events, scheduled scans, and database reflection
without duplicating the policy or its cache in the context.
The maintained local ownership is:
| Path mode | Owner |
| ------------------------------------------------------------------- | ------------------ |
| Customisation path in Selective or Flagged Selective mode | Customisation Sync |
| Customisation path in Automatic mode, with Hidden File Sync enabled | Hidden File Sync |
| Customisation path in Ignore mode | Neither context |
| Other eligible hidden path | Hidden File Sync |
| Disabled, excluded, ignored, or ordinary Vault path | Neither context |
Local ownership is distinct from persisted-document recognition. The
composition dispatches `ps:` and `ix:` conflict documents to Customisation
Sync and sends `i:` conflict documents through the Hidden File Sync semantic
handler view. Existing documents remain recognisable after a local mode
changes.
Each context exposes an immutable semantic handler view. The composition
registers these operations and adapts them to the existing Commonlib handler
contracts; registry aggregation names and binding concerns do not leak back
into either context:
- raw optional-file events are offered to exactly one selected owner;
- a selected handler which skips or fails does not fall through to the other
context;
- Customisation Sync receives virtual Customisation documents;
- Hidden File Sync receives optional synchronisation results for `i:`
documents;
- shared lifecycle handlers retain Customisation-before-Hidden order; and
- the Vault extra-target handler returns the final routed ownership decision.
## Domain dependency boundaries
Neither context imports `main.ts`, accepts `LiveSyncCore`, extends
`LiveSyncContext`, or reaches through `this.core`, `this.services`, or
`this.app`.
Each context receives live getter projections for settings and the local
database. This is important because both can be replaced during settings or
database lifecycle work. Stable ServiceModules, such as storage access and
database file access, are supplied as focused method projections. Host effects
are named individually in the dependency contract.
The Hidden File Sync exact-revision boundary uses only
`fetchEntryMeta`, `getConflictedRevs`, `fetchEntryFromMeta`, and
`storeWithBaseRevision`. A selected revision is checked against the current
winner and conflict leaves before it can be applied or extended. The repair
view therefore does not expose the complete database file-access module.
Obsidian-specific adapters can import the application core as a type and read
the required Services and ServiceModules at composition. The resulting
dependency objects are the only bridge from either domain context to Obsidian
presentation and host lifecycle APIs.
## Views and consumers
One context can back several focused views without creating several state
owners:
- `CustomisationSyncDialogView` supplies catalogue and explicit application
operations to the Customisation Sync dialogue;
- `HiddenFileSyncInitialisationView` supplies the initialisation directions
needed by that dialogue;
- `HiddenFileSyncCommandView` supplies availability and scan operations to
host-owned commands; and
- `HiddenFileSyncRepairView` supplies local inspection and exact-revision
operations to the Hatch pane.
Production consumers receive these views directly. `OptionalFileSyncFeature`
also exposes immutable, explicitly internal testing views for maintained
real-Obsidian contract tests. They provide named operations, including a scoped
rebuild interceptor, without exposing context instances, dependency objects,
queues, or writable stores. These test seams are not production service
locators and should not be used by application features. Path categorisation
and key derivation are tested directly through their focused capability rather
than being re-exported through a broad context testing view.
## Lifecycle and disposal
The composition is created after the Service Hub and required ServiceModules
exist, and before lifecycle-driven feature work begins. Each context creates
its own periodic processor and focused resource owners.
`CustomisationSyncContext` creates one path capability, one snapshot-persistence
boundary, one snapshot coordinator, one application owner, one scan owner, one
catalogue owner, and one recent-event deduplicator. The catalogue owner creates
its shared state, one queue and progress subscription, and the V1, V2, and
migration modules. `HiddenFileSyncContext` creates one
path-admission owner, one change notifier, and one processed-state owner before
composing database write and extraction operations around their narrow ports.
It then creates one change processor, one conflict-resolution owner, and one
reconciliation owner. The change processor owns its semaphore and activity
state; the reconciliation owner owns its scoped testing interceptor.
Full conflict scans admit discovered paths into the same queue without
suspending it, so ordinary database conflict notifications continue during a
slow scan. After enumeration, the operation waits for both classification and
interaction stages to drain.
On application unload, `useOptionalFileSync` first removes every Service
handler registration. It then disposes Customisation Sync followed by Hidden
File Sync, preserving the former compatibility order. Disposal disables
periodic admission, disposes the conflict-resolution owner, terminates queues,
clears transient caches and pending sets, cancels scheduled notification work,
resets compatibility telemetry, and hides owned Notices. The two presentation
serviceFeatures independently remove their commands, event subscriptions,
ribbon state, and dialogue instances.
## Persisted compatibility
This architecture does not migrate persisted data:
- Customisation Sync remains in the `ix:` namespace and continues to read V1
grouped and V2 per-file data;
- Hidden File Sync remains in the `i:` namespace;
- file content remains in Chunks referenced by Metadata rather than being
embedded as ordinary Metadata content;
- exact PouchDB revision identifiers remain part of repair and conflict
operations; and
- Hidden File Sync processed-state keys remain device-local key-value data.
Any change to these contracts requires separate compatibility tests and, when
appropriate, a migration decision.
## Verification boundaries
Focused unit tests cover routing, path-option binding, Hidden File Sync
admission ordering and cache invalidation, semantic handler views, context
owner isolation, teardown, initial cache selection, exact-revision repair,
Customisation Sync scan reconciliation and context delegation, single-queue
catalogue disposal and publication, live V1/V2 dispatch, V1 and V2 snapshot
persistence outcomes, logical-deletion idempotence, refresh ordering,
application operations, Hidden File Sync change-event serialisation and
settlement, reconciliation direction and scan ordering, notification batching,
conflict queue admission, revision selection, automatic and interactive merge
effect ordering, conflict dialogue adaptation, grouped Notices, and
compatibility activity publication.
The boundary test prevents either domain context from regaining core or
Obsidian dependencies.
Changes to local routing or transfer require the maintained Customisation Sync
and Hidden File Sync real-Obsidian workflows. A composition change also
requires the mixed-ownership case which proves that one local path is not
written by both contexts.
@@ -1,70 +0,0 @@
---
date: 2026-09-04
commonlib-version: "0.1.21"
self-hosted-livesync-version: "1.0.24"
status: unreleased
---
# Path component length compatibility
## Purpose
File systems place limits on each file or folder name, rather than applying one
common limit to an entire Vault-relative path. Those limits are also expressed
in different units. Self-hosted LiveSync therefore treats 255 UTF-8 bytes as a
focused Android and Linux compatibility warning, not as a universal definition
of a valid path.
## Basis for the 255-byte warning
- The Linux kernel documentation gives ext4 a maximum file-name length of
[255 bytes](https://www.kernel.org/doc/html/latest/filesystems/ext4/directory.html).
- The F2FS on-disk header defines
[`F2FS_NAME_LEN` as 255](https://android.googlesource.com/kernel/common/+/88d92fb1c034922572bab93482ac9cc61d4ba43c/include/linux/f2fs_fs.h)
and stores names in byte arrays.
- Android's MediaProvider uses a
[`MAX_FILENAME_BYTES` value of 255](https://android.googlesource.com/platform/packages/providers/MediaProvider/+/bae279463/src/com/android/providers/media/util/FileUtils.java)
when building file names. Its source notes that emulated storage can write to
ext4 through FUSE, where names are encoded as UTF-8.
- Android 11 and later use
[FUSE for emulated storage](https://source.android.com/docs/core/storage/fuse-passthrough),
with requests passing through to the underlying file system.
Together, these provide a conservative compatibility boundary for file names
which may reach Android or Linux storage. They do not show that every Android
device, storage provider, or Linux file system has the same limit.
## Why the rule is not universal
Other platforms describe component limits differently. Microsoft's file-system
comparison documents limits in
[Unicode characters](https://learn.microsoft.com/en-us/windows/win32/fileio/filesystem-functionality-comparison),
not UTF-8 bytes. Apple's HFS Plus format stores a name as up to
[255 16-bit `UniChar` values](https://developer.apple.com/library/archive/technotes/tn/tn1150.html).
Apple's APFS guidance discusses valid UTF-8 names, normalisation, and case
sensitivity, but does not establish a universal
[255-byte component rule](https://developer.apple.com/library/archive/documentation/FileManagement/Conceptual/APFS_Guide/FAQ/FAQ.html).
A name can consequently exceed 255 UTF-8 bytes and still work on one platform,
or fail for another platform-specific reason while remaining below this
boundary.
## Product policy
Self-hosted LiveSync applies the warning as follows:
1. split the Vault-relative path on `/` and inspect each non-empty component;
2. measure each component after UTF-8 encoding;
3. accept 255 bytes without this warning and warn at 256 bytes or more;
4. identify every over-limit file or folder name in the active-file status;
5. do not reject, truncate, or rename the path; and
6. treat the result of the real storage operation as authoritative.
If a scan cannot process an individual file, its path is recorded in the
verbose log and remains eligible for a later retry. Ordinary start-up may still
become ready so that unaffected files can synchronise. Explicit Fetch and
Rebuild operations retain strict scan completion because they establish an
authoritative local or remote state.
This policy does not replace the existing checks for reserved characters,
case collisions, ignore rules, or configured file-size limits.
+1 -1
View File
@@ -721,7 +721,7 @@ When a saved setting changes whether a normal file can be reflected, LiveSync re
## 6. Customisation sync (Advanced)
Customisation Sync is a supported, advanced opt-in feature. Its current per-file implementation is covered by a two-Vault real-Obsidian workflow for snippets, configuration files, and plug-in files. Hidden File Sync is a separate feature with different setup, selection, and conflict behaviour; do not use both features to manage the same files.
Customisation Sync is a supported, advanced opt-in feature. Its current per-file implementation is covered by a two-Vault real-Obsidian workflow for configuration files, themes, snippets, and plug-in main, data, and supplementary files. Hidden File Sync is a separate feature with different setup, selection, and conflict behaviour; do not use both features to manage the same files.
### 1. Customisation Sync
-3
View File
@@ -22,9 +22,6 @@ Note: The figure is drawn as single-directional, between two devices for demonst
defines the current revision-tree and file-provenance rules.
- [Chunk Retrieval and Waiting](design_docs/chunk_retrieval_and_waiting.md)
defines missing-Chunk arrival and quiescence handling.
- [Path component length compatibility](design_docs/path_component_length_compatibility.md)
explains why 255 UTF-8 bytes is an Android and Linux compatibility warning,
rather than a universal rule for deciding whether a path is valid.
- [Data Compression](specs_data_compression.md) and [Garbage Collection
V3](specs_garbage_collection.md) describe their respective storage and
maintenance contracts.
+4 -4
View File
@@ -23,7 +23,7 @@
"@smithy/types": "^4.14.3",
"@smithy/util-retry": "^4.4.5",
"@vrtmrz/browser-ui-kit": "0.1.0",
"@vrtmrz/livesync-commonlib": "0.1.22",
"@vrtmrz/livesync-commonlib": "0.1.21",
"@vrtmrz/obsidian-plugin-kit": "0.1.4",
"@vrtmrz/ui-interactions": "0.1.2",
"diff-match-patch": "^1.0.5",
@@ -4620,9 +4620,9 @@
}
},
"node_modules/@vrtmrz/livesync-commonlib": {
"version": "0.1.22",
"resolved": "https://registry.npmjs.org/@vrtmrz/livesync-commonlib/-/livesync-commonlib-0.1.22.tgz",
"integrity": "sha512-8TsFo6xgEO/uZzkQ4TE3yydUyK8pCbuMm0C4DC/8KhG8z06N6hQQmwR7bV+a3Zgt9A5tXPImTFJWTMUIxJYV2g==",
"version": "0.1.21",
"resolved": "https://registry.npmjs.org/@vrtmrz/livesync-commonlib/-/livesync-commonlib-0.1.21.tgz",
"integrity": "sha512-AGuZ3eqBP37HJXEkTSpJ5M5bvTx2lYNq+6Q5NuCPZeGdbv7g6cujGvccVR5ozGfKdHGSyFZND5x1oFS9crRhUg==",
"license": "MIT",
"dependencies": {
"@aws-sdk/client-s3": "^3.808.0",
+1 -1
View File
@@ -177,7 +177,7 @@
"@smithy/types": "^4.14.3",
"@smithy/util-retry": "^4.4.5",
"@vrtmrz/browser-ui-kit": "0.1.0",
"@vrtmrz/livesync-commonlib": "0.1.22",
"@vrtmrz/livesync-commonlib": "0.1.21",
"@vrtmrz/obsidian-plugin-kit": "0.1.4",
"@vrtmrz/ui-interactions": "0.1.2",
"diff-match-patch": "^1.0.5",
-13
View File
@@ -58,18 +58,6 @@ export class LiveSyncBaseCore<
this.services.appLifecycle.onUnload.addHandler(() => Promise.resolve(addOn.onunload()).then(() => true));
}
/**
* Get an add-on by its class name. Returns undefined if not found.
* @param cls
* @returns
*/
getAddOn<T extends TCommands>(cls: string) {
for (const addon of this.addOns) {
if (addon.constructor.name == cls) return addon as T;
}
return undefined;
}
constructor(
serviceHub: InjectableServiceHub<T>,
serviceModuleInitialiser: (
@@ -299,5 +287,4 @@ export class LiveSyncBaseCore<
export interface IMinimumLiveSyncCommands {
onunload(): void;
onload(): void | Promise<void>;
constructor: { name: string };
}
+3 -5
View File
@@ -82,11 +82,9 @@ RUN apt-get update \
WORKDIR /deps
# Remove build-only dependencies before resolving the standalone runtime tree.
# npm --omit=dev omits them from disk, but still resolves their peer graph.
COPY src/apps/cli/package.json ./package.json
RUN npm pkg delete devDependencies \
&& npm install --omit=dev
# package.json lists only the packages that the CLI requires
COPY src/apps/cli/package.json ./package.json
RUN npm install --omit=dev
# ─────────────────────────────────────────────────────────────────────────────
# Stage 3 — runtime
+2 -5
View File
@@ -15,10 +15,7 @@ import { stripAllPrefixes } from "@vrtmrz/livesync-commonlib/compat/string_and_b
import type { CLICommandContext, CLIOptions } from "./types";
import { toArrayBuffer, toDatabaseRelativePath } from "./utils";
import { collectPeers, openP2PHost, parseTimeoutSeconds, syncWithPeer } from "./p2p";
import {
performFullScan,
VaultScanResults,
} from "@vrtmrz/livesync-commonlib/compat/serviceFeatures/offlineScanner";
import { performFullScan } from "@vrtmrz/livesync-commonlib/compat/serviceFeatures/offlineScanner";
import { UnresolvedErrorManager } from "@vrtmrz/livesync-commonlib/compat/services/base/UnresolvedErrorManager";
import { compatGlobal } from "@vrtmrz/livesync-commonlib/compat/common/coreEnvFunctions";
import { fsPromises as fs, path } from "@vrtmrz/livesync-commonlib/node";
@@ -532,7 +529,7 @@ export async function runCommand(options: CLIOptions, context: CLICommandContext
writeStderrLine(standardIo, "[Command] mirror");
const log = (msg: unknown) => writeStderrLine(standardIo, `[Mirror] ${String(msg)}`);
const errorManager = new UnresolvedErrorManager(core.services.appLifecycle, core.services.context.events);
return (await performFullScan(core, log, errorManager, false, true)) === VaultScanResults.COMPLETED;
return await performFullScan(core, log, errorManager, false, true);
}
if (options.command === "remote-add") {
@@ -363,18 +363,6 @@ export const allMessages: Readonly<Record<string, Readonly<Record<string, string
zh: "新增连接",
"zh-tw": "新增連線",
},
"AddOn Module (ConfigSync) has not been loaded. This is very unexpected situation. Please report this issue.": {
def: "AddOn Module (ConfigSync) has not been loaded. This is very unexpected situation. Please report this issue.",
es: "El módulo complementario (ConfigSync) no se ha cargado. Esta situación es muy inesperada. Informa de este problema.",
ko: "애드온 모듈(ConfigSync)이 로드되지 않았습니다. 매우 예기치 못한 상황입니다. 이 문제를 신고해 주세요.",
"zh-tw": "附加模組(ConfigSync)尚未載入。這是非常異常的情況,請回報此問題。",
},
"AddOn Module (HiddenFileSync) has not been loaded. This is very unexpected situation. Please report this issue.": {
def: "AddOn Module (HiddenFileSync) has not been loaded. This is very unexpected situation. Please report this issue.",
es: "El módulo complementario (HiddenFileSync) no se ha cargado. Esta situación es muy inesperada. Informa de este problema.",
ko: "애드온 모듈(HiddenFileSync)이 로드되지 않았습니다. 매우 예기치 못한 상황입니다. 이 문제를 신고해 주세요.",
"zh-tw": "附加模組(HiddenFileSync)尚未載入。這是非常異常的情況,請回報此問題。",
},
Advanced: {
def: "Advanced",
es: "Avanzado",
@@ -4212,9 +4200,6 @@ export const allMessages: Readonly<Record<string, Readonly<Record<string, string
zh: "等待就绪...",
"zh-tw": "正在等待就緒⋯",
},
"moduleLog.pathComponentTooLong": {
def: "This path contains a file or folder name longer than ${maxBytes} UTF-8 bytes. It may not work on some Android and Linux file systems.",
},
"moduleLog.showLog": {
def: "Show Log",
es: "Mostrar registro",
@@ -10417,9 +10402,6 @@ export const allMessages: Readonly<Record<string, Readonly<Record<string, string
zh: "Use Remote Configuration",
"zh-tw": "使用遠端設定",
},
"Ui.Common.LocalDatabaseInitialisationFailed": {
def: "Self-hosted LiveSync cannot synchronise. Generate a report to review the detailed log.",
},
"Ui.Common.Signal.Caution": {
def: "CAUTION",
es: "PRECAUCIÓN",
@@ -10448,9 +10430,6 @@ export const allMessages: Readonly<Record<string, Readonly<Record<string, string
zh: "警告",
"zh-tw": "警告",
},
"Ui.Common.SomeFilesCouldNotBeSynchronised": {
def: "Not all files could be synchronised. Check the affected files. Generate a report to review the detailed log.",
},
"Ui.Settings.Advanced.LocalDatabaseTweak": {
def: "Local Database Tweak",
es: "Ajuste fino de la base de datos local",
-5
View File
@@ -46,8 +46,6 @@
"Active Remote Configuration": "Active Remote Configuration",
"Add default patterns": "Add default patterns",
"Add new connection": "Add new connection",
"AddOn Module (ConfigSync) has not been loaded. This is very unexpected situation. Please report this issue.": "AddOn Module (ConfigSync) has not been loaded. This is very unexpected situation. Please report this issue.",
"AddOn Module (HiddenFileSync) has not been loaded. This is very unexpected situation. Please report this issue.": "AddOn Module (HiddenFileSync) has not been loaded. This is very unexpected situation. Please report this issue.",
"Advanced": "Advanced",
"Advanced Settings": "Advanced Settings",
"After restarting, the data on this device will be uploaded to the server as the 'master copy'. Please be aware that any unintended data currently on the server will be completely overwritten.": "After restarting, the data on this device will be uploaded to the server as the 'master copy'. Please be aware that any unintended data currently on the server will be completely overwritten.",
@@ -483,7 +481,6 @@
"moduleLiveSyncMain.optionResumeAndRestart": "Resume and restart Obsidian",
"moduleLiveSyncMain.titleScramEnabled": "Scram Enabled",
"moduleLocalDatabase.logWaitingForReady": "Waiting for ready...",
"moduleLog.pathComponentTooLong": "This path contains a file or folder name longer than ${maxBytes} UTF-8 bytes. It may not work on some Android and Linux file systems.",
"moduleLog.showLog": "Show Log",
"moduleMigration.fix0256.buttons.checkItLater": "Check it later",
"moduleMigration.fix0256.buttons.DismissForever": "I have fixed it, and do not ask again",
@@ -1143,12 +1140,10 @@
"TweakMismatchResolve.Title.AutoAcceptCompatible": "Auto-Accept Available",
"TweakMismatchResolve.Title.TweakResolving": "Configuration Mismatch Detected",
"TweakMismatchResolve.Title.UseRemoteConfig": "Use Remote Configuration",
"Ui.Common.LocalDatabaseInitialisationFailed": "Self-hosted LiveSync cannot synchronise. Generate a report to review the detailed log.",
"Ui.Common.Signal.Caution": "CAUTION",
"Ui.Common.Signal.Danger": "DANGER",
"Ui.Common.Signal.Notice": "NOTICE",
"Ui.Common.Signal.Warning": "WARNING",
"Ui.Common.SomeFilesCouldNotBeSynchronised": "Not all files could be synchronised. Check the affected files. Generate a report to review the detailed log.",
"Ui.Settings.Advanced.LocalDatabaseTweak": "Local Database Tweak",
"Ui.Settings.Advanced.MemoryCache": "Memory Cache",
"Ui.Settings.Advanced.TransferTweak": "Transfer Tweak",
-2
View File
@@ -46,8 +46,6 @@
"Active Remote Configuration": "Configuración remota activa",
"Add default patterns": "Añadir patrones predeterminados",
"Add new connection": "Añadir conexión",
"AddOn Module (ConfigSync) has not been loaded. This is very unexpected situation. Please report this issue.": "El módulo complementario (ConfigSync) no se ha cargado. Esta situación es muy inesperada. Informa de este problema.",
"AddOn Module (HiddenFileSync) has not been loaded. This is very unexpected situation. Please report this issue.": "El módulo complementario (HiddenFileSync) no se ha cargado. Esta situación es muy inesperada. Informa de este problema.",
"Advanced": "Avanzado",
"Advanced Settings": "Ajustes avanzados",
"After restarting, the data on this device will be uploaded to the server as the 'master copy'. Please be aware that any unintended data currently on the server will be completely overwritten.": "Tras reiniciar, los datos de este dispositivo se subirán al servidor como «copia maestra». Ten en cuenta que cualquier dato no deseado que haya ahora en el servidor se sobrescribirá por completo.",
-2
View File
@@ -46,8 +46,6 @@
"Active Remote Configuration": "활성 원격 구성",
"Add default patterns": "기본 패턴 추가",
"Add new connection": "연결 추가",
"AddOn Module (ConfigSync) has not been loaded. This is very unexpected situation. Please report this issue.": "애드온 모듈(ConfigSync)이 로드되지 않았습니다. 매우 예기치 못한 상황입니다. 이 문제를 신고해 주세요.",
"AddOn Module (HiddenFileSync) has not been loaded. This is very unexpected situation. Please report this issue.": "애드온 모듈(HiddenFileSync)이 로드되지 않았습니다. 매우 예기치 못한 상황입니다. 이 문제를 신고해 주세요.",
"Advanced": "고급",
"Advanced Settings": "고급 설정",
"After restarting, the data on this device will be uploaded to the server as the 'master copy'. Please be aware that any unintended data currently on the server will be completely overwritten.": "재시작하면 이 기기의 데이터가 '원본'으로서 서버에 업로드됩니다. 현재 서버에 있는 의도치 않은 데이터는 모두 완전히 덮어써진다는 점에 유의해 주세요.",
-2
View File
@@ -46,8 +46,6 @@
"Active Remote Configuration": "目前啟用的遠端設定",
"Add default patterns": "新增預設模式",
"Add new connection": "新增連線",
"AddOn Module (ConfigSync) has not been loaded. This is very unexpected situation. Please report this issue.": "附加模組(ConfigSync)尚未載入。這是非常異常的情況,請回報此問題。",
"AddOn Module (HiddenFileSync) has not been loaded. This is very unexpected situation. Please report this issue.": "附加模組(HiddenFileSync)尚未載入。這是非常異常的情況,請回報此問題。",
"Advanced": "進階",
"Advanced Settings": "進階設定",
"After restarting, the data on this device will be uploaded to the server as the 'master copy'. Please be aware that any unintended data currently on the server will be completely overwritten.": "重新啟動後,此裝置上的資料將以「主要複本」的形式上傳到伺服器。請注意,伺服器上任何非預期的現有資料都會被完全覆寫。",
-11
View File
@@ -58,12 +58,6 @@ Activate: Activate
Active Remote Configuration: Active Remote Configuration
Add default patterns: Add default patterns
Add new connection: Add new connection
AddOn Module (ConfigSync) has not been loaded. This is very unexpected situation. Please report this issue.:
AddOn Module (ConfigSync) has not been loaded. This is very unexpected
situation. Please report this issue.
AddOn Module (HiddenFileSync) has not been loaded. This is very unexpected situation. Please report this issue.:
AddOn Module (HiddenFileSync) has not been loaded. This is very unexpected
situation. Please report this issue.
Advanced: Advanced
Advanced Settings: Advanced Settings
After restarting, the data on this device will be uploaded to the server as the 'master copy'. Please be aware that any unintended data currently on the server will be completely overwritten.:
@@ -732,9 +726,6 @@ moduleLiveSyncMain:
moduleLocalDatabase:
logWaitingForReady: Waiting for ready...
moduleLog:
pathComponentTooLong: >-
This path contains a file or folder name longer than ${maxBytes} UTF-8
bytes. It may not work on some Android and Linux file systems.
showLog: Show Log
moduleMigration:
fix0256:
@@ -2129,8 +2120,6 @@ xxhash64 (Fastest): xxhash64 (Fastest)
"This feature enables direct synchronisation between devices. No server is required, but both devices must be online at the same time for synchronisation to occur, and some features may be limited. Internet connection is only required to signalling (detecting peers) and not for data transfer.": "This feature enables direct synchronisation between devices. No server is required, but both devices must be online at the same time for synchronisation to occur, and some features may be limited. Internet connection is only required to signalling (detecting peers) and not for data transfer."
Ui:
Common:
LocalDatabaseInitialisationFailed: Self-hosted LiveSync cannot synchronise. Generate a report to review the detailed log.
SomeFilesCouldNotBeSynchronised: Not all files could be synchronised. Check the affected files. Generate a report to review the detailed log.
Signal:
Caution: CAUTION
Danger: DANGER
-6
View File
@@ -57,12 +57,6 @@ Activate: Activar
Active Remote Configuration: Configuración remota activa
Add default patterns: Añadir patrones predeterminados
Add new connection: Añadir conexión
AddOn Module (ConfigSync) has not been loaded. This is very unexpected situation. Please report this issue.:
El módulo complementario (ConfigSync) no se ha cargado. Esta situación es muy
inesperada. Informa de este problema.
AddOn Module (HiddenFileSync) has not been loaded. This is very unexpected situation. Please report this issue.:
El módulo complementario (HiddenFileSync) no se ha cargado. Esta situación es
muy inesperada. Informa de este problema.
Advanced: Avanzado
Advanced Settings: Ajustes avanzados
After restarting, the data on this device will be uploaded to the server as the 'master copy'. Please be aware that any unintended data currently on the server will be completely overwritten.:
-2
View File
@@ -47,8 +47,6 @@ Activate: 활성화
Active Remote Configuration: 활성 원격 구성
Add default patterns: 기본 패턴 추가
Add new connection: 연결 추가
AddOn Module (ConfigSync) has not been loaded. This is very unexpected situation. Please report this issue.: 애드온 모듈(ConfigSync)이 로드되지 않았습니다. 매우 예기치 못한 상황입니다. 이 문제를 신고해 주세요.
AddOn Module (HiddenFileSync) has not been loaded. This is very unexpected situation. Please report this issue.: 애드온 모듈(HiddenFileSync)이 로드되지 않았습니다. 매우 예기치 못한 상황입니다. 이 문제를 신고해 주세요.
Advanced: 고급
Advanced Settings: 고급 설정
After restarting, the data on this device will be uploaded to the server as the 'master copy'. Please be aware that any unintended data currently on the server will be completely overwritten.: 재시작하면 이 기기의 데이터가 '원본'으로서 서버에 업로드됩니다. 현재 서버에 있는 의도치 않은 데이터는 모두 완전히 덮어써진다는 점에 유의해 주세요.
-2
View File
@@ -139,8 +139,6 @@ username: 使用者名稱
"↑: Overwrite Remote": ↑:覆寫遠端
"↓: Overwrite Local": ↓:覆寫本機
"⇅: Use newer": ⇅:使用較新版本
AddOn Module (ConfigSync) has not been loaded. This is very unexpected situation. Please report this issue.: 附加模組(ConfigSync)尚未載入。這是非常異常的情況,請回報此問題。
AddOn Module (HiddenFileSync) has not been loaded. This is very unexpected situation. Please report this issue.: 附加模組(HiddenFileSync)尚未載入。這是非常異常的情況,請回報此問題。
All the same or non-existent: 全部相同或不存在
Apply All Selected: 套用所有已選取項目
Automatic: 自動
-26
View File
@@ -1,26 +0,0 @@
export const ANDROID_LINUX_PATH_COMPONENT_UTF8_WARNING_BOUNDARY = 255;
export interface OversizedPathComponent {
component: string;
utf8Bytes: number;
}
const utf8Encoder = new TextEncoder();
/**
* Return path components which exceed the conservative Android/Linux
* compatibility boundary.
*
* Obsidian paths use forward slashes. The limit applies to each file or
* folder name, not to the combined Vault-relative path.
*/
export function findPathComponentsExceedingUtf8Limit(
path: string,
maxBytes: number = ANDROID_LINUX_PATH_COMPONENT_UTF8_WARNING_BOUNDARY
): OversizedPathComponent[] {
return path
.split("/")
.filter((component) => component.length > 0)
.map((component) => ({ component, utf8Bytes: utf8Encoder.encode(component).byteLength }))
.filter(({ utf8Bytes }) => utf8Bytes > maxBytes);
}
-46
View File
@@ -1,46 +0,0 @@
import { describe, expect, it } from "vitest";
import {
ANDROID_LINUX_PATH_COMPONENT_UTF8_WARNING_BOUNDARY,
findPathComponentsExceedingUtf8Limit,
} from "./pathCompatibility.ts";
describe("findPathComponentsExceedingUtf8Limit", () => {
it("accepts 255 UTF-8 bytes and reports 256 UTF-8 bytes", () => {
expect(findPathComponentsExceedingUtf8Limit("a".repeat(255))).toEqual([]);
expect(findPathComponentsExceedingUtf8Limit("a".repeat(256))).toEqual([
{
component: "a".repeat(256),
utf8Bytes: 256,
},
]);
});
it("counts UTF-8 bytes rather than JavaScript characters", () => {
expect(findPathComponentsExceedingUtf8Limit("界".repeat(85))).toEqual([]);
expect(findPathComponentsExceedingUtf8Limit(`${"界".repeat(85)}a`)).toEqual([
{
component: `${"界".repeat(85)}a`,
utf8Bytes: 256,
},
]);
});
it("does not apply the component limit to the whole path", () => {
const path = `${"a".repeat(200)}/${"b".repeat(200)}`;
expect(new TextEncoder().encode(path).byteLength).toBeGreaterThan(
ANDROID_LINUX_PATH_COMPONENT_UTF8_WARNING_BOUNDARY
);
expect(findPathComponentsExceedingUtf8Limit(path)).toEqual([]);
});
it("reports an oversized folder component as well as an oversized file name", () => {
const folder = "界".repeat(86);
const file = `${"b".repeat(256)}.md`;
expect(findPathComponentsExceedingUtf8Limit(`parent/${folder}/${file}`)).toEqual([
{ component: folder, utf8Bytes: 258 },
{ component: file, utf8Bytes: 259 },
]);
});
});
-17
View File
@@ -21,23 +21,6 @@ describe("LiveSync-owned translation catalogue", () => {
expect($msg("moduleCheckRemoteSize.optionIncreaseLimit", { newMax: "800" }, "def")).toBe("increase to 800MB");
});
it("keeps the active-file path compatibility warning concise", () => {
const oversizedComponent = `${"界".repeat(86)} (258 bytes)`;
expect(
$msg(
"moduleLog.pathComponentTooLong",
{
maxBytes: "255",
components: oversizedComponent,
},
"def"
)
).toBe(
"This path contains a file or folder name longer than 255 UTF-8 bytes. It may not work on some Android and Linux file systems."
);
});
it("uses Commonlib's canonical English when the application catalogue has no translation", () => {
setLang("es");
@@ -1,113 +0,0 @@
import { describe, expect, it, vi } from "vitest";
vi.mock("@/deps.ts", () => ({
addIcon: vi.fn(),
diff_match_patch: class DiffMatchPatch {},
normalizePath: vi.fn((path: string) => path),
parseYaml: vi.fn(),
Platform: {},
}));
vi.mock("./PluginDialogModal.ts", () => ({
PluginDialogModal: class PluginDialogModal {},
}));
vi.mock("@/features/HiddenFileCommon/JsonResolveModal.ts", () => ({
JsonResolveModal: class JsonResolveModal {},
}));
vi.mock("@/modules/features/InteractiveConflictResolving/ConflictResolveModal.ts", () => ({
ConflictResolveModal: class ConflictResolveModal {},
}));
vi.mock("@/features/LiveSyncCommands.ts", () => ({
LiveSyncCommands: class LiveSyncCommands {
core!: { services: unknown };
get services() {
return this.core.services;
}
},
}));
vi.mock("@/common/types.ts", () => ({
ICXHeader: "ix:",
PERIODIC_PLUGIN_SWEEP: 60,
}));
vi.mock("@/common/utils.ts", () => ({
cancelTask: vi.fn(),
EVEN: Symbol("even"),
isCustomisationSyncMetadata: vi.fn(),
isPluginMetadata: vi.fn(),
scheduleTask: vi.fn(),
}));
vi.mock("@/common/PeriodicProcessor.ts", () => ({
PeriodicProcessor: class PeriodicProcessor {},
}));
vi.mock("@/common/events.ts", () => ({
EVENT_REQUEST_OPEN_PLUGIN_SYNC_DIALOG: "open-plugin-sync",
eventHub: {
onEvent: vi.fn(),
},
}));
vi.mock("@/common/translation", () => ({
$msg: vi.fn((message: string) => message),
}));
vi.mock("@/common/obsidianCommunityPlugins.ts", () => ({
getObsidianCommunityPluginManager: vi.fn(),
}));
import { cancelTask } from "@/common/utils.ts";
import { ConfigSync } from "./CmdConfigSync";
describe("ConfigSync commands", () => {
it("shows the Customisation Sync command only whilst the feature is enabled", () => {
const commands: Array<{
id: string;
checkCallback?: (checking: boolean) => boolean | void;
}> = [];
const settings = {
usePluginSync: false,
};
const showPluginSyncModal = vi.fn();
const configSync = Object.create(ConfigSync.prototype) as ConfigSync;
Object.assign(configSync, {
core: {
settings,
services: {
API: {
addCommand: vi.fn((command) => commands.push(command)),
},
},
},
addRibbonIcon: vi.fn(() => ({
addClass: vi.fn(),
})),
showPluginSyncModal,
});
configSync.onload();
const command = commands.find(({ id }) => id === "livesync-plugin-dialog-ex");
expect(command?.checkCallback?.(true)).toBe(false);
settings.usePluginSync = true;
expect(command?.checkCallback?.(true)).toBe(true);
expect(command?.checkCallback?.(false)).toBe(true);
expect(showPluginSyncModal).toHaveBeenCalledOnce();
});
it("cancels the pending configuration Notice before releasing its owned UI", () => {
const notices = { hide: vi.fn() };
const periodicPluginSweepProcessor = { disable: vi.fn() };
const configSync = Object.create(ConfigSync.prototype) as ConfigSync;
Object.assign(configSync, {
core: {
services: {
context: { notices },
},
},
periodicPluginSweepProcessor,
});
configSync.onunload();
expect(cancelTask).toHaveBeenCalledWith("config-sync:updated-configuration");
expect(notices.hide).toHaveBeenCalledWith("config-sync:updated-configuration");
expect(periodicPluginSweepProcessor.disable).toHaveBeenCalledOnce();
});
});
File diff suppressed because it is too large Load Diff
+11 -31
View File
@@ -1,15 +1,9 @@
<script lang="ts">
import {
ConfigSync,
PluginDataExDisplayV2,
type IPluginDataExDisplay,
type PluginDataExFile,
} from "./CmdConfigSync.ts";
import type { CustomisationSyncDialogView, IPluginDataExDisplay } from "./customisationSyncView.ts";
import type { PluginDataExFile } from "./customisationSyncCodec.ts";
import { Logger } from "@vrtmrz/livesync-commonlib/compat/common/logger";
import { type FilePath, LOG_LEVEL_INFO, LOG_LEVEL_NOTICE, LOG_LEVEL_VERBOSE } from "@vrtmrz/livesync-commonlib/compat/common/types";
import { LOG_LEVEL_INFO, LOG_LEVEL_NOTICE, LOG_LEVEL_VERBOSE } from "@vrtmrz/livesync-commonlib/compat/common/types";
import { getDocData, timeDeltaToHumanReadable, unique } from "@vrtmrz/livesync-commonlib/compat/common/utils";
import type ObsidianLiveSyncPlugin from "@/main";
// import { askString } from "../../common/utils";
import { Menu } from "@/deps.ts";
import { $msg as translateMessage } from "@/common/translation";
@@ -28,15 +22,9 @@
) => Promise<boolean>;
export let deleteData: (data: IPluginDataExDisplay) => Promise<boolean>;
export let hidden: boolean;
export let plugin: ObsidianLiveSyncPlugin;
export let customisationSync: CustomisationSyncDialogView;
export let isMaintenanceMode: boolean = false;
export let isFlagged: boolean = false;
$: core = plugin.core;
const addOn = plugin.core.getAddOn<ConfigSync>(ConfigSync.name)!;
if (!addOn) {
Logger(`Could not load the add-on ${ConfigSync.name}`, LOG_LEVEL_INFO);
throw new Error(`Could not load the add-on ${ConfigSync.name}`);
}
export let selected = "";
let freshness = "";
@@ -263,7 +251,7 @@
const local = list.find((e) => e.term == thisTerm);
const selectedItem = list.find((e) => e.term == selected);
if (selectedItem && (await applyData(selectedItem))) {
addOn.updatePluginList(true, local?.documentPath);
void customisationSync.updatePluginList(true, local?.documentPath);
}
}
async function compareSelected() {
@@ -279,18 +267,12 @@
if (local && remote) {
if (!filename) {
if (await compareData(local, remote)) {
addOn.updatePluginList(true, local.documentPath);
void customisationSync.updatePluginList(true, local.documentPath);
}
return;
} else {
const localCopy =
local instanceof PluginDataExDisplayV2 ? new PluginDataExDisplayV2(local) : { ...local };
const remoteCopy =
remote instanceof PluginDataExDisplayV2 ? new PluginDataExDisplayV2(remote) : { ...remote };
localCopy.files = localCopy.files.filter((e) => e.filename == filename);
remoteCopy.files = remoteCopy.files.filter((e) => e.filename == filename);
if (await compareData(localCopy, remoteCopy, true)) {
addOn.updatePluginList(true, local.documentPath);
if (await customisationSync.compareFileUsingDisplayData(local, remote, filename)) {
void customisationSync.updatePluginList(true, local.documentPath);
}
}
return;
@@ -333,7 +315,7 @@
const selectedItem = list.find((e) => e.term == selected);
// const deletedPath = selectedItem.documentPath;
if (selectedItem && (await deleteData(selectedItem))) {
addOn.reloadPluginList(true);
void customisationSync.reloadPluginList(true);
}
}
async function duplicateItem() {
@@ -342,7 +324,7 @@
Logger(`Could not find local item`, LOG_LEVEL_VERBOSE);
return;
}
const duplicateTermName = await core.confirm.askString(
const duplicateTermName = await customisationSync.askString(
translateMessage("Duplicate"),
translateMessage("device name"),
""
@@ -352,9 +334,7 @@
Logger(translateMessage('We can not use "/" to the device name'), LOG_LEVEL_NOTICE);
return;
}
const key = `${plugin.core.services.API.getSystemConfigDir()}/${local.files[0].filename}`;
await addOn.storeCustomizationFiles(key as FilePath, duplicateTermName);
await addOn.updatePluginList(false, addOn.filenameToUnifiedKey(key, duplicateTermName));
await customisationSync.duplicateData(local, duplicateTermName);
}
}
</script>
+16 -6
View File
@@ -1,17 +1,24 @@
import { mount, unmount } from "svelte";
import { App, Modal } from "@/deps.ts";
import ObsidianLiveSyncPlugin from "@/main.ts";
import { type App, Modal } from "@/deps.ts";
import type { HiddenFileSyncInitialisationView } from "@/features/HiddenFileSync/hiddenFileSyncViews.ts";
import type { CustomisationSyncDialogView } from "./customisationSyncView.ts";
import PluginPane from "./PluginPane.svelte";
export class PluginDialogModal extends Modal {
plugin: ObsidianLiveSyncPlugin;
customisationSync: CustomisationSyncDialogView;
hiddenFileSync: HiddenFileSyncInitialisationView;
component: ReturnType<typeof mount> | undefined;
isOpened() {
return this.component != undefined;
}
constructor(app: App, plugin: ObsidianLiveSyncPlugin) {
constructor(
app: App,
customisationSync: CustomisationSyncDialogView,
hiddenFileSync: HiddenFileSyncInitialisationView
) {
super(app);
this.plugin = plugin;
this.customisationSync = customisationSync;
this.hiddenFileSync = hiddenFileSync;
}
override onOpen() {
@@ -25,7 +32,10 @@ export class PluginDialogModal extends Modal {
if (!this.component) {
this.component = mount(PluginPane, {
target: contentEl,
props: { plugin: this.plugin, core: this.plugin.core },
props: {
customisationSync: this.customisationSync,
hiddenFileSync: this.hiddenFileSync,
},
});
}
}
@@ -0,0 +1,46 @@
import { beforeEach, describe, expect, it, vi } from "vitest";
const svelteMocks = vi.hoisted(() => ({
mount: vi.fn(),
unmount: vi.fn(),
}));
vi.mock("svelte", () => svelteMocks);
vi.mock("@/deps.ts", () => ({
Modal: class Modal {
contentEl = { setCssStyles: vi.fn() };
titleEl = { setText: vi.fn() };
},
}));
vi.mock("./PluginPane.svelte", () => ({ default: "PluginPane" }));
import { PluginDialogModal } from "./PluginDialogModal.ts";
describe("PluginDialogModal", () => {
beforeEach(() => {
vi.clearAllMocks();
});
it("mounts the focused views once and releases the component on close", () => {
const component = { component: "customisation-sync" };
const customisationSync = { catalogue: {} };
const hiddenFileSync = { initialiseInternalFileSync: vi.fn() };
svelteMocks.mount.mockReturnValue(component);
const modal = new PluginDialogModal({} as never, customisationSync as never, hiddenFileSync);
modal.onOpen();
modal.onOpen();
expect(svelteMocks.mount).toHaveBeenCalledOnce();
expect(svelteMocks.mount).toHaveBeenCalledWith("PluginPane", {
target: modal.contentEl,
props: { customisationSync, hiddenFileSync },
});
expect(modal.isOpened()).toBe(true);
modal.onClose();
expect(svelteMocks.unmount).toHaveBeenCalledWith(component);
expect(modal.isOpened()).toBe(false);
});
});
+36 -80
View File
@@ -1,14 +1,6 @@
<script lang="ts">
import { onMount } from "svelte";
import ObsidianLiveSyncPlugin from "@/main";
import {
ConfigSync,
type IPluginDataExDisplay,
pluginIsEnumerating,
pluginList,
pluginManifestStore,
pluginV2Progress,
} from "./CmdConfigSync.ts";
import type { CustomisationSyncDialogView, IPluginDataExDisplay } from "./customisationSyncView.ts";
import PluginCombo from "./PluginCombo.svelte";
import { Menu, type PluginManifest } from "@/deps.ts";
import { unique } from "@vrtmrz/livesync-commonlib/compat/common/utils";
@@ -19,38 +11,19 @@
type SYNC_MODE,
MODE_SHINY,
} from "@vrtmrz/livesync-commonlib/compat/common/types";
import { normalizePath } from "@/deps";
import { HiddenFileSync } from "@/features/HiddenFileSync/CmdHiddenFileSync.ts";
import { LOG_LEVEL_NOTICE, Logger } from "octagonal-wheels/common/logger";
import type { LiveSyncBaseCore } from "@/LiveSyncBaseCore.ts";
import type { HiddenFileSyncInitialisationView } from "@/features/HiddenFileSync/hiddenFileSyncViews.ts";
import { $msg as translateMessage } from "@/common/translation";
import {
REPLICATION_PROGRESS_PRESENTATIONS,
USER_INITIATED_REPLICATION_AUTHORITY,
} from "@vrtmrz/livesync-commonlib/replication";
export let plugin: ObsidianLiveSyncPlugin;
export let core :LiveSyncBaseCore;
// $: core = plugin.core;
export let customisationSync: CustomisationSyncDialogView;
export let hiddenFileSync: HiddenFileSyncInitialisationView;
$: hideNotApplicable = false;
$: thisTerm = core.services.setting.getDeviceAndVaultName();
$: thisTerm = customisationSync.getDeviceAndVaultName();
const addOn = core.getAddOn<ConfigSync>(ConfigSync.name)!;
if (!addOn) {
const msg = translateMessage(
"AddOn Module (ConfigSync) has not been loaded. This is very unexpected situation. Please report this issue."
);
Logger(msg, LOG_LEVEL_NOTICE);
throw new Error(msg);
}
const addOnHiddenFileSync = core.getAddOn<HiddenFileSync>(HiddenFileSync.name) as HiddenFileSync;
if (!addOnHiddenFileSync) {
const msg = translateMessage(
"AddOn Module (HiddenFileSync) has not been loaded. This is very unexpected situation. Please report this issue."
);
Logger(msg, LOG_LEVEL_NOTICE);
throw new Error(msg);
}
const catalogue = customisationSync.catalogue;
const enumerationActive = customisationSync.enumerationActive;
const migrationProgress = customisationSync.migrationProgress;
const manifests = customisationSync.manifests;
let list: IPluginDataExDisplay[] = [];
@@ -61,21 +34,17 @@
let applyAllPluse = 0;
let isMaintenanceMode = false;
async function requestUpdate() {
await addOn.updatePluginList(true);
await customisationSync.updatePluginList(true);
}
async function requestReload() {
await addOn.reloadPluginList(true);
await customisationSync.reloadPluginList(true);
}
let allTerms = [] as string[];
pluginList.subscribe((e) => {
list = e;
allTerms = unique(list.map((e) => e.term));
});
pluginIsEnumerating.subscribe((e) => {
loading = e;
});
onMount(async () => {
requestUpdate();
let allTerms: string[] = [];
$: list = $catalogue;
$: allTerms = unique(list.map((entry) => entry.term));
$: loading = $enumerationActive;
onMount(() => {
void requestUpdate();
});
function filterList(list: IPluginDataExDisplay[], categories: string[]) {
@@ -104,15 +73,11 @@
SNIPPET: translateMessage("Snippets"),
};
async function scanAgain() {
await addOn.scanAllConfigFiles(true);
await customisationSync.scanAllConfigFiles(true);
await requestUpdate();
}
async function replicate() {
await core.services.replication.replicateUserInitiated({
trigger: "manual",
progressPresentation: REPLICATION_PROGRESS_PRESENTATIONS.NOTICE,
interaction: USER_INITIATED_REPLICATION_AUTHORITY,
});
await customisationSync.synchronise();
}
function selectAllNewest(selectMode: boolean) {
selectNewestPulse++;
@@ -126,17 +91,17 @@
applyAllPluse++;
}
async function applyData(data: IPluginDataExDisplay): Promise<boolean> {
return await addOn.applyData(data);
return await customisationSync.applyData(data);
}
async function compareData(
docA: IPluginDataExDisplay,
docB: IPluginDataExDisplay,
compareEach = false
): Promise<boolean> {
return await addOn.compareUsingDisplayData(docA, docB, compareEach);
return await customisationSync.compareUsingDisplayData(docA, docB, compareEach);
}
async function deleteData(data: IPluginDataExDisplay): Promise<boolean> {
return await addOn.deleteData(data);
return await customisationSync.deleteData(data);
}
function askMode(evt: MouseEvent, title: string, key: string) {
const menu = new Menu();
@@ -161,9 +126,8 @@
}
function applyAutomaticSync(key: string, direction: "pushForce" | "pullForce" | "safe") {
setMode(key, MODE_AUTOMATIC);
const configDir = normalizePath(plugin.core.services.API.getSystemConfigDir());
const files = (plugin.core.settings.pluginSyncExtendedSetting[key]?.files ?? []).map((e) => `${configDir}/${e}`);
addOnHiddenFileSync.initialiseInternalFileSync(direction, true, files);
const files = customisationSync.getConfiguredTargetFiles(key);
void hiddenFileSync.initialiseInternalFileSync(direction, true, files);
}
function askOverwriteModeForAutomatic(evt: MouseEvent, key: string) {
const menu = new Menu();
@@ -196,7 +160,7 @@
applyData,
compareData,
deleteData,
plugin,
customisationSync,
isMaintenanceMode,
};
@@ -236,22 +200,12 @@
);
if (mode == MODE_SELECTIVE) {
automaticList.delete(key);
delete plugin.core.settings.pluginSyncExtendedSetting[key];
automaticListDisp = automaticList;
} else {
automaticList.set(key, mode);
automaticListDisp = automaticList;
if (!(key in plugin.core.settings.pluginSyncExtendedSetting)) {
plugin.core.settings.pluginSyncExtendedSetting[key] = {
key,
mode,
files: [],
};
}
plugin.core.settings.pluginSyncExtendedSetting[key].files = files;
plugin.core.settings.pluginSyncExtendedSetting[key].mode = mode;
}
core.services.setting.saveSettingData();
customisationSync.updateConfiguredMode(key, mode, files);
}
function getIcon(mode: SYNC_MODE) {
if (mode in ICONS) {
@@ -264,7 +218,7 @@
let automaticListDisp = new Map<string, SYNC_MODE>();
// apply current configuration to the dialogue
for (const { key, mode } of Object.values(plugin.core.settings.pluginSyncExtendedSetting)) {
for (const { key, mode } of customisationSync.getConfiguredModes()) {
automaticList.set(key, mode);
}
@@ -273,7 +227,7 @@
let displayKeys: Record<string, string[]> = {};
function computeDisplayKeys(list: IPluginDataExDisplay[]) {
const extraKeys = Object.keys(plugin.core.settings.pluginSyncExtendedSetting);
const extraKeys = customisationSync.getConfiguredModes().map(({ key }) => key);
return [
...list,
...extraKeys
@@ -303,7 +257,7 @@
for (const item of deleteItems) {
await deleteData(item);
}
addOn.reloadPluginList(true);
void customisationSync.reloadPluginList(true);
}
let nameMap = new Map<string, string>();
@@ -324,7 +278,7 @@
}
nameMap = newMap;
}
$: updateNameMap($pluginManifestStore);
$: updateNameMap($manifests);
let displayEntries = [] as [string, string][];
$: {
@@ -335,7 +289,7 @@
$: {
pluginEntries = groupBy(filterList(list, ["PLUGIN_MAIN", "PLUGIN_DATA", "PLUGIN_ETC"]), "name");
}
let useSyncPluginEtc = plugin.core.settings.usePluginEtc;
let useSyncPluginEtc = customisationSync.isPluginEtcEnabled();
</script>
<div class="buttonsWrap">
@@ -357,8 +311,10 @@
</div>
</div>
<div class="loading">
{#if loading || $pluginV2Progress !== 0}
<span>{translateMessage("Updating list...")}{$pluginV2Progress == 0 ? "" : ` (${$pluginV2Progress})`}</span>
{#if loading || $migrationProgress !== 0}
<span
>{translateMessage("Updating list...")}{$migrationProgress == 0 ? "" : ` (${$migrationProgress})`}</span
>
{/if}
</div>
<div class="list">
@@ -0,0 +1,346 @@
import { diff_match_patch, parseYaml } from "@/deps.ts";
import type {
diff_result,
FilePath,
FilePathWithPrefix,
LOG_LEVEL,
} from "@vrtmrz/livesync-commonlib/compat/common/types";
import { LOG_LEVEL_NOTICE, LOG_LEVEL_VERBOSE } from "@vrtmrz/livesync-commonlib/compat/common/types";
import { delay, getDocData, getDocDataAsArray, isDocContentSame } from "@vrtmrz/livesync-commonlib/compat/common/utils";
import { decodeBinary } from "@vrtmrz/livesync-commonlib/compat/string_and_binary/convert";
import { digestHash } from "@vrtmrz/livesync-commonlib/compat/string_and_binary/hash";
import { serialized } from "octagonal-wheels/concurrency/lock";
import { base64ToArrayBuffer, base64ToString } from "octagonal-wheels/binary/base64";
import { LiveSyncError } from "@vrtmrz/livesync-commonlib/compat/common/LSError";
import type { StorageAccess } from "@vrtmrz/livesync-commonlib/compat/interfaces/StorageAccess";
import type { LiveSyncLocalDB } from "@vrtmrz/livesync-commonlib/compat/pouchdb/LiveSyncLocalDB";
import type { LogFunction } from "@vrtmrz/livesync-commonlib/compat/services/lib/logUtils";
import { createCustomisationSyncCodec, type PluginDataEx } from "./customisationSyncCodec.ts";
import type { CatalogueOperations } from "./catalogueOperations.ts";
import type { CustomisationSyncPathOperations } from "./customisationSyncPathOperations.ts";
import type { SnapshotOperations } from "./snapshotOperations.ts";
import { PluginDataExDisplayV2 } from "./customisationSyncModel.ts";
import type { IPluginDataExDisplay, LoadedEntryPluginDataExFile } from "./customisationSyncView.ts";
const { deserialize } = createCustomisationSyncCodec({ digestHash, parseYaml });
type ApplicationDatabase = Pick<LiveSyncLocalDB, "getDBEntry">;
type ApplicationStorage = Pick<
StorageAccess,
"ensureDir" | "readHiddenFileBinary" | "readHiddenFileText" | "writeHiddenFileAuto"
>;
type ApplicationPath = Pick<CustomisationSyncPathOperations, "filenameToUnifiedKey">;
type ApplicationSnapshotOperations = Pick<
SnapshotOperations,
"isV2Enabled" | "storeCustomisationFileV2" | "storeCustomizationFiles" | "deleteConfigOnDatabase"
>;
type ApplicationCatalogue = Pick<
CatalogueOperations,
"findPlugins" | "manifestLookup" | "updatePluginList" | "updatePluginListV2"
>;
export type ApplicationOperationsDependencies = {
getLocalDatabase(): ApplicationDatabase;
storageAccess: ApplicationStorage;
path: ApplicationPath;
log: LogFunction;
getConfigDir(): string;
getDeviceAndVaultName(): string;
resolveJsonConflict(
path: FilePath,
files: [LoadedEntryPluginDataExFile, LoadedEntryPluginDataExFile],
remoteName: string,
apply: (content: string) => Promise<boolean>
): Promise<boolean>;
selectTextFile(path: FilePath, diffResult: diff_result, remoteName: string): Promise<"A" | "B" | false>;
reloadPlugin(configDir: string, pluginName: string): Promise<void>;
askRestart(): void;
snapshotOperations: ApplicationSnapshotOperations;
catalogueOperations: ApplicationCatalogue;
};
/**
* Owns the Customisation Sync dialogue's compare, apply, duplicate, and
* delete workflows. It deliberately consumes the shared snapshot capability
* and catalogue owner, leaving lifecycle, event admission, and scanning in
* the context.
*/
export class ApplicationOperations {
constructor(private readonly dependencies: ApplicationOperationsDependencies) {}
private get configDir() {
return this.dependencies.getConfigDir();
}
private get localDatabase() {
return this.dependencies.getLocalDatabase();
}
private get storageAccess() {
return this.dependencies.storageAccess;
}
private _log(message: unknown, level?: LOG_LEVEL, key?: string) {
this.dependencies.log(message, level, key);
}
async compareFileUsingDisplayData(
dataA: IPluginDataExDisplay,
dataB: IPluginDataExDisplay,
filename: string
): Promise<boolean> {
const dataACopy =
dataA instanceof PluginDataExDisplayV2
? new PluginDataExDisplayV2(dataA, this.dependencies.catalogueOperations.manifestLookup)
: { ...dataA };
const dataBCopy =
dataB instanceof PluginDataExDisplayV2
? new PluginDataExDisplayV2(dataB, this.dependencies.catalogueOperations.manifestLookup)
: { ...dataB };
dataACopy.files = dataACopy.files.filter((file) => file.filename == filename);
dataBCopy.files = dataBCopy.files.filter((file) => file.filename == filename);
return await this.compareUsingDisplayData(dataACopy, dataBCopy, true);
}
async compareUsingDisplayData(dataA: IPluginDataExDisplay, dataB: IPluginDataExDisplay, compareEach = false) {
const loadFile = async (data: IPluginDataExDisplay) => {
if (data instanceof PluginDataExDisplayV2 || compareEach) {
return data.files[0] as LoadedEntryPluginDataExFile;
}
const loadDoc = await this.localDatabase.getDBEntry(data.documentPath);
if (!loadDoc) return false;
const pluginData = deserialize(getDocDataAsArray(loadDoc.data), {}) as PluginDataEx;
pluginData.documentPath = data.documentPath;
const file = pluginData.files[0];
const doc = { ...loadDoc, ...file, datatype: "newnote" } as LoadedEntryPluginDataExFile;
return doc;
};
const fileA = await loadFile(dataA);
const fileB = await loadFile(dataB);
this._log(`Comparing: ${dataA.documentPath} <-> ${dataB.documentPath}`, LOG_LEVEL_VERBOSE);
if (!fileA || !fileB) {
this._log(
`Could not load ${dataA.name} for comparison: ${!fileA ? dataA.term : ""}${!fileB ? dataB.term : ""}`,
LOG_LEVEL_NOTICE
);
return false;
}
const path = fileA.filename.split("/").pop() as FilePath;
if (path.endsWith(".json")) {
return serialized("config:merge-data", async () => {
this._log("Opening data-merging dialog", LOG_LEVEL_VERBOSE);
return await this.dependencies.resolveJsonConflict(path, [fileA, fileB], dataB.term, async (result) => {
try {
return await this.applyData(dataA, result);
} catch (ex) {
this._log("Could not apply merged file");
this._log(ex, LOG_LEVEL_VERBOSE);
return false;
}
});
});
} else {
const dmp = new diff_match_patch();
let docAData = getDocData(fileA.data);
let docBData = getDocData(fileB.data);
if (fileA?.datatype != "plain") {
docAData = base64ToString(docAData);
}
if (fileB?.datatype != "plain") {
docBData = base64ToString(docBData);
}
const diffMap = dmp.diff_linesToChars_(docAData, docBData);
const diff = dmp.diff_main(diffMap.chars1, diffMap.chars2, false);
dmp.diff_charsToLines_(diff, diffMap.lineArray);
dmp.diff_cleanupSemantic(diff);
const diffResult: diff_result = {
left: { rev: "A", ...fileA, data: docAData },
right: { rev: "B", ...fileB, data: docBData },
diff: diff,
};
const ret = await this.dependencies.selectTextFile(path, diffResult, dataB.term);
if (ret === false) return false;
const resultContent = ret == "A" ? docAData : ret == "B" ? docBData : undefined;
if (resultContent) {
return await this.applyData(dataA, resultContent);
}
return false;
}
}
async duplicateData(data: IPluginDataExDisplay, deviceName: string): Promise<void> {
const path = `${this.configDir}/${data.files[0].filename}` as FilePath;
await this.dependencies.snapshotOperations.storeCustomizationFiles(path, deviceName);
await this.dependencies.catalogueOperations.updatePluginList(
false,
this.dependencies.path.filenameToUnifiedKey(path, deviceName)
);
}
async applyDataV2(data: PluginDataExDisplayV2, content?: string): Promise<boolean> {
const baseDir = this.configDir;
try {
if (content) {
// Preserve the inherited truthiness check: an explicitly empty
// replacement is treated as the no-content path.
const filename = data.files[0].filename;
this._log(`Applying ${filename} of ${data.displayName || data.name}..`);
const path = `${baseDir}/${filename}` as FilePath;
await this.storageAccess.ensureDir(path);
// If the content has applied, modified time will be updated to the current time.
await this.storageAccess.writeHiddenFileAuto(path, content);
await this.dependencies.snapshotOperations.storeCustomisationFileV2(
path,
this.dependencies.getDeviceAndVaultName()
);
} else {
const files = data.files;
for (const f of files) {
// If files have applied, modified time will be updated to the current time.
const stat = { mtime: f.mtime, ctime: f.ctime };
const path = `${baseDir}/${f.filename}` as FilePath;
this._log(`Applying ${f.filename} of ${data.displayName || data.name}..`);
// const contentEach = createBlob(f.data);
await this.storageAccess.ensureDir(path);
if (f.datatype == "newnote") {
let oldData;
try {
oldData = await this.storageAccess.readHiddenFileBinary(path);
} catch (ex) {
this._log(`Could not read the file ${f.filename}`, LOG_LEVEL_VERBOSE);
this._log(ex, LOG_LEVEL_VERBOSE);
oldData = new ArrayBuffer(0);
}
const content = base64ToArrayBuffer(f.data);
if (await isDocContentSame(oldData, content)) {
this._log(`The file ${f.filename} is already up-to-date`, LOG_LEVEL_VERBOSE);
continue;
}
await this.storageAccess.writeHiddenFileAuto(path, content, stat);
} else {
let oldData;
try {
oldData = await this.storageAccess.readHiddenFileText(path);
} catch (ex) {
this._log(`Could not read the file ${f.filename}`, LOG_LEVEL_VERBOSE);
this._log(ex, LOG_LEVEL_VERBOSE);
oldData = "";
}
const content = getDocData(f.data);
if (await isDocContentSame(oldData, content)) {
this._log(`The file ${f.filename} is already up-to-date`, LOG_LEVEL_VERBOSE);
continue;
}
await this.storageAccess.writeHiddenFileAuto(path, content, stat);
}
this._log(`Applied ${f.filename} of ${data.displayName || data.name}..`);
await this.dependencies.snapshotOperations.storeCustomisationFileV2(
path,
this.dependencies.getDeviceAndVaultName()
);
}
}
} catch (ex) {
this._log(`Applying ${data.displayName || data.name}.. Failed`, LOG_LEVEL_NOTICE);
this._log(ex, LOG_LEVEL_VERBOSE);
return false;
}
return true;
}
async applyData(data: IPluginDataExDisplay, content?: string): Promise<boolean> {
this._log(`Applying ${data.displayName || data.name}..`);
if (data instanceof PluginDataExDisplayV2) {
return this.applyDataV2(data, content);
}
return this.applyDataV1(data, content);
}
private async applyDataV1(data: IPluginDataExDisplay, content?: string): Promise<boolean> {
const baseDir = this.configDir;
try {
if (!data.documentPath) throw new LiveSyncError("InternalError: Document path not exist");
const dx = await this.localDatabase.getDBEntry(data.documentPath);
if (dx == false) {
throw new LiveSyncError("Not found on database");
}
const loadedData = deserialize(getDocDataAsArray(dx.data), {}) as PluginDataEx;
for (const f of loadedData.files) {
this._log(`Applying ${f.filename} of ${data.displayName || data.name}..`);
try {
// console.dir(f);
const path = `${baseDir}/${f.filename}`;
await this.storageAccess.ensureDir(path);
if (!content) {
const dt = decodeBinary(f.data);
await this.storageAccess.writeHiddenFileAuto(path, dt);
} else {
await this.storageAccess.writeHiddenFileAuto(path, content);
}
this._log(`Applying ${f.filename} of ${data.displayName || data.name}.. Done`);
} catch (ex) {
this._log(`Applying ${f.filename} of ${data.displayName || data.name}.. Failed`);
this._log(ex, LOG_LEVEL_VERBOSE);
}
}
const uPath = `${baseDir}/${loadedData.files[0].filename}` as FilePath;
await this.dependencies.snapshotOperations.storeCustomizationFiles(uPath);
// The inherited workflow refreshes once through persistence, then
// explicitly refreshes again with the dialogue's notice flag.
await this.dependencies.catalogueOperations.updatePluginList(true, uPath);
await delay(100);
this._log(`Config ${data.displayName || data.name} has been applied`, LOG_LEVEL_NOTICE);
if (data.category == "PLUGIN_DATA" || data.category == "PLUGIN_MAIN") {
await this.dependencies.reloadPlugin(baseDir, data.name);
} else if (data.category == "CONFIG") {
this.dependencies.askRestart();
}
return true;
} catch (ex) {
this._log(`Applying ${data.displayName || data.name}.. Failed`);
this._log(ex, LOG_LEVEL_VERBOSE);
return false;
}
}
async deleteData(data: PluginDataEx): Promise<boolean> {
try {
if (data.documentPath) {
const delList: FilePathWithPrefix[] = [];
if (this.dependencies.snapshotOperations.isV2Enabled()) {
const deleteList = this.dependencies.catalogueOperations
.findPlugins(data.documentPath)
.filter((entry) => entry instanceof PluginDataExDisplayV2)
.map((entry) => entry.files)
.flat();
for (const e of deleteList) {
delList.push(e.path);
}
}
delList.push(data.documentPath);
const p = delList.map(async (e) => {
await this.dependencies.snapshotOperations.deleteConfigOnDatabase(e);
// Preserve the inherited unconditional refresh after the
// persistence wrapper, including when it emitted no refresh.
await this.dependencies.catalogueOperations.updatePluginList(false, e);
});
await Promise.allSettled(p);
// Preserve the inherited success result even when individual
// deletion/refresh promises settle unsuccessfully.
this._log(
`Deleted: ${data.category}/${data.name} of ${data.category} (${delList.length} items)`,
LOG_LEVEL_NOTICE
);
}
return true;
} catch (ex) {
this._log(`Failed to delete: ${data.documentPath}`, LOG_LEVEL_NOTICE);
this._log(ex, LOG_LEVEL_VERBOSE);
return false;
}
}
}
@@ -0,0 +1,306 @@
import { describe, expect, it, vi } from "vitest";
const asyncHarness = vi.hoisted(() => ({
delay: vi.fn(async () => undefined),
fireAndForget: vi.fn((operation: () => unknown) => {
void operation();
}),
}));
vi.mock("@/deps.ts", () => ({
diff_match_patch: class DiffMatchPatch {},
parseYaml: vi.fn(),
}));
vi.mock("@/common/translation", () => ({
$msg: vi.fn((message: string) => message),
}));
vi.mock("@vrtmrz/livesync-commonlib/compat/common/utils", async (importOriginal) => {
const actual = await importOriginal<typeof import("@vrtmrz/livesync-commonlib/compat/common/utils")>();
return {
...actual,
delay: asyncHarness.delay,
fireAndForget: asyncHarness.fireAndForget,
};
});
import type { FilePath, FilePathWithPrefix, LoadedEntry } from "@vrtmrz/livesync-commonlib/compat/common/types";
import type { PluginManifest } from "@/deps.ts";
import { digestHash } from "@vrtmrz/livesync-commonlib/compat/string_and_binary/hash";
import { createCustomisationSyncCodec, type PluginDataEx } from "./customisationSyncCodec.ts";
import { ApplicationOperations, type ApplicationOperationsDependencies } from "./applicationOperations.ts";
import { PluginDataExDisplayV2 } from "./customisationSyncModel.ts";
import type { SnapshotPersistenceResult } from "./snapshotPersistence.ts";
import { SnapshotOperations } from "./snapshotOperations.ts";
import type { IPluginDataExDisplay } from "./customisationSyncView.ts";
const codec = createCustomisationSyncCodec({ digestHash, parseYaml: () => undefined });
function createOperations() {
const events: string[] = [];
let usePluginSyncV2 = false;
type PersistenceResult = SnapshotPersistenceResult<true>;
const getDBEntry = vi.fn(async (_path: FilePathWithPrefix) => false as false | LoadedEntry);
const ensureDir = vi.fn(async (_path: string) => {
events.push("ensure-dir");
return true;
});
const readHiddenFileBinary = vi.fn(async (_path: string) => new ArrayBuffer(0));
const readHiddenFileText = vi.fn(async (_path: string) => "");
const writeHiddenFileAuto = vi.fn(async (_path: string, _data: string | ArrayBuffer) => {
events.push("write-file");
return true;
});
const storeCustomisationFileV2 = vi.fn(
async (): Promise<PersistenceResult> => ({
value: true,
status: "saved" as const,
refreshes: [] as const,
})
);
const storeCustomizationFiles = vi.fn(
async (): Promise<PersistenceResult> => ({
value: true,
status: "saved" as const,
refreshes: [] as const,
})
);
const deleteConfigOnDatabase = vi.fn(
async (): Promise<PersistenceResult> => ({
value: true,
status: "deleted" as const,
refreshes: [] as const,
})
);
const updatePluginList = vi.fn(async (showMessage: boolean, _path?: FilePathWithPrefix | FilePath) => {
events.push(`refresh-v1:${showMessage}`);
});
const updatePluginListV2 = vi.fn(async (_showMessage: boolean, _path: FilePathWithPrefix) => {
events.push("refresh-v2");
});
const findPlugins = vi.fn(() => [] as readonly IPluginDataExDisplay[]);
const reloadPlugin = vi.fn(async (_configDir: string, _pluginName: string) => {
events.push("reload-plugin");
});
const askRestart = vi.fn(() => {
events.push("ask-restart");
});
const catalogueOperations = {
findPlugins,
manifestLookup: new Map<string, PluginManifest>(),
updatePluginList,
updatePluginListV2,
};
const snapshotOperations = new SnapshotOperations({
getSettings: () => ({ usePluginSyncV2 }),
getDeviceAndVaultName: () => "device-a",
log: vi.fn(),
snapshotPersistence: {
storeCustomisationFileV2,
storeCustomizationFiles,
deleteConfigOnDatabase,
},
catalogueOperations,
});
const dependencies: ApplicationOperationsDependencies = {
getLocalDatabase: () => ({ getDBEntry }),
storageAccess: {
ensureDir,
readHiddenFileBinary,
readHiddenFileText,
writeHiddenFileAuto,
},
path: {
filenameToUnifiedKey: (path, term) => `ix:${term}/CONFIG/${path.split("/").pop()}.md` as FilePathWithPrefix,
},
log: vi.fn(),
getConfigDir: () => ".obsidian",
getDeviceAndVaultName: () => "device-a",
resolveJsonConflict: vi.fn(async () => false),
selectTextFile: vi.fn(async (): Promise<"A" | "B" | false> => false),
reloadPlugin,
askRestart,
snapshotOperations,
catalogueOperations,
};
return {
application: new ApplicationOperations(dependencies),
dependencies,
events,
setUseV2: (value: boolean) => {
usePluginSyncV2 = value;
},
getDBEntry,
persistence: { deleteConfigOnDatabase, storeCustomisationFileV2, storeCustomizationFiles },
catalogue: { findPlugins, updatePluginList, updatePluginListV2 },
storage: { ensureDir, readHiddenFileBinary, readHiddenFileText, writeHiddenFileAuto },
reloadPlugin,
askRestart,
};
}
const display = {
documentPath: "ix:device-a/PLUGIN_DATA/example.md" as FilePathWithPrefix,
category: "PLUGIN_DATA",
name: "example",
term: "device-a",
files: [
{ filename: "plugins/example/data.json", data: ["a"], mtime: 1, size: 1 },
{ filename: "plugins/example/other.json", data: ["b"], mtime: 2, size: 1 },
],
mtime: 2,
} satisfies IPluginDataExDisplay;
describe("Customisation Sync application operations", () => {
it("keeps file-level comparison clones and duplication behaviour inside the owner", async () => {
const fixture = createOperations();
const compareUsingDisplayData = vi
.spyOn(fixture.application, "compareUsingDisplayData")
.mockResolvedValue(true);
await expect(
fixture.application.compareFileUsingDisplayData(display, display, "plugins/example/data.json")
).resolves.toBe(true);
const [left, right, compareEach] = compareUsingDisplayData.mock.calls[0];
expect(left.files.map((file) => file.filename)).toEqual(["plugins/example/data.json"]);
expect(right.files.map((file) => file.filename)).toEqual(["plugins/example/data.json"]);
expect(compareEach).toBe(true);
expect(display.files).toHaveLength(2);
await fixture.application.duplicateData(display, "device-b");
expect(fixture.persistence.storeCustomizationFiles).toHaveBeenCalledWith(
".obsidian/plugins/example/data.json",
"device-b"
);
expect(fixture.catalogue.updatePluginList).toHaveBeenCalledWith(false, "ix:device-b/CONFIG/data.json.md");
});
it("uses the compared filename for legacy file comparisons", async () => {
const fixture = createOperations();
await expect(
fixture.application.compareFileUsingDisplayData(display, display, "plugins/example/data.json")
).resolves.toBe(false);
expect(fixture.dependencies.resolveJsonConflict).toHaveBeenCalledWith(
"data.json",
expect.any(Array),
"device-a",
expect.any(Function)
);
});
it("awaits V1 refreshes before the explicit effect and preserves reload ordering", async () => {
const fixture = createOperations();
fixture.setUseV2(false);
fixture.persistence.storeCustomizationFiles.mockImplementation(async () => {
fixture.events.push("persist");
return {
value: true,
status: "saved" as const,
refreshes: [
{
mode: "v1" as const,
timing: "await" as const,
path: "ix:device-a/PLUGIN_MAIN/example.md" as FilePathWithPrefix,
},
],
};
});
fixture.getDBEntry.mockResolvedValue({
data: codec.serialize({
category: "PLUGIN_MAIN",
name: "example",
term: "device-a",
files: [{ filename: "plugins/example/main.js", data: ["source"], mtime: 1, size: 6 }],
mtime: 1,
} satisfies PluginDataEx),
} as LoadedEntry);
const data = { ...display, category: "PLUGIN_MAIN", name: "example" } satisfies IPluginDataExDisplay;
await expect(fixture.application.applyData(data, "replacement")).resolves.toBe(true);
expect(fixture.events).toEqual([
"ensure-dir",
"write-file",
"persist",
"refresh-v1:false",
"refresh-v1:true",
"reload-plugin",
]);
expect(fixture.reloadPlugin).toHaveBeenCalledWith(".obsidian", "example");
expect(asyncHarness.delay).toHaveBeenCalledWith(100);
});
it("starts V2 catalogue refreshes without awaiting them", async () => {
const fixture = createOperations();
let releaseRefresh!: () => void;
const refresh = new Promise<void>((resolve) => {
releaseRefresh = resolve;
});
fixture.setUseV2(true);
fixture.persistence.storeCustomisationFileV2.mockResolvedValue({
value: true,
status: "saved",
refreshes: [
{
mode: "v2",
timing: "fire-and-forget",
path: "ix:device-a/CONFIG/app.json%app.json" as FilePathWithPrefix,
},
],
});
fixture.catalogue.updatePluginListV2.mockImplementation(async () => await refresh);
const data = new PluginDataExDisplayV2(
{
...display,
files: [{ filename: "app.json", data: ["source"], mtime: 1, size: 6 }],
},
new Map()
);
await expect(fixture.application.applyData(data, "replacement")).resolves.toBe(true);
expect(fixture.catalogue.updatePluginListV2).toHaveBeenCalledWith(
false,
"ix:device-a/CONFIG/app.json%app.json"
);
releaseRefresh();
await refresh;
});
it("deletes the V2 files and binder through the direct owners", async () => {
const fixture = createOperations();
fixture.setUseV2(true);
const v2Path = "ix:device-a/PLUGIN_DATA/example%data.json" as FilePathWithPrefix;
const binderPath = "ix:device-a/PLUGIN_DATA/example.md" as FilePathWithPrefix;
const v2Entry = new PluginDataExDisplayV2(
{
...display,
documentPath: binderPath,
files: [
{
filename: "data.json",
path: v2Path,
data: ["source"],
mtime: 1,
ctime: 1,
size: 6,
datatype: "plain",
} as never,
],
},
new Map()
);
fixture.catalogue.findPlugins.mockReturnValue([v2Entry]);
await expect(
fixture.application.deleteData({
...display,
documentPath: binderPath,
})
).resolves.toBe(true);
expect(fixture.persistence.deleteConfigOnDatabase).toHaveBeenNthCalledWith(1, v2Path, false);
expect(fixture.persistence.deleteConfigOnDatabase).toHaveBeenNthCalledWith(2, binderPath, false);
expect(fixture.catalogue.updatePluginList).toHaveBeenNthCalledWith(1, false, v2Path);
expect(fixture.catalogue.updatePluginList).toHaveBeenNthCalledWith(2, false, binderPath);
});
});
@@ -0,0 +1,126 @@
import { createBlob, getDocDataAsArray } from "@vrtmrz/livesync-commonlib/compat/common/utils";
import type {
AnyEntry,
FilePathWithPrefix,
LOG_LEVEL,
SavingEntry,
} from "@vrtmrz/livesync-commonlib/compat/common/types";
import { LOG_LEVEL_INFO, LOG_LEVEL_NOTICE, LOG_LEVEL_VERBOSE } from "@vrtmrz/livesync-commonlib/compat/common/types";
import type { LiveSyncLocalDB } from "@vrtmrz/livesync-commonlib/compat/pouchdb/LiveSyncLocalDB";
import type { IPathService } from "@vrtmrz/livesync-commonlib/compat/services/base/IService";
import type { LogFunction } from "@vrtmrz/livesync-commonlib/compat/services/lib/logUtils";
import { ICXHeader } from "@/common/types.ts";
import type { SnapshotPersistence } from "./snapshotPersistence.ts";
import type { CustomisationSyncReadCodec } from "./customisationSyncReadOperations.ts";
type CatalogueMigrationDatabase = Pick<LiveSyncLocalDB, "getDBEntry" | "putDBEntry">;
type CatalogueMigrationCodec = Pick<CustomisationSyncReadCodec, "deserialize"> & {
dummyHead: string;
dummyEnd: string;
};
export type CatalogueMigrationDependencies = {
getLocalDatabase(): CatalogueMigrationDatabase;
path: Pick<IPathService, "path2id">;
log: LogFunction;
snapshotPersistence: Pick<SnapshotPersistence, "deleteConfigOnDatabase">;
refreshV1(showMessage: boolean, path: FilePathWithPrefix): Promise<void>;
codec: CatalogueMigrationCodec;
};
/** Bridges persisted V1 binders into the V2 per-file document format. */
export class CatalogueMigration {
constructor(private readonly dependencies: CatalogueMigrationDependencies) {}
private _log(message: unknown, level?: LOG_LEVEL, key?: string): void {
this.dependencies.log(message, level, key);
}
async migrateV1ToV2(showMessage: boolean, entry: AnyEntry): Promise<void> {
const v1Path = entry.path;
this._log(`Migrating ${entry.path} to V2`, showMessage ? LOG_LEVEL_NOTICE : LOG_LEVEL_INFO);
if (entry.deleted) {
this._log(`The entry ${v1Path} is already deleted`, LOG_LEVEL_VERBOSE);
return;
}
// Compatibility question: the inherited conjunction admits any `ix:`
// path or any `.md` path, although the log describes a stricter binder
// check. Preserve it until malformed migration candidates are covered.
if (!v1Path.endsWith(".md") && !v1Path.startsWith(ICXHeader)) {
this._log(`The entry ${v1Path} is not a customisation sync binder`, LOG_LEVEL_VERBOSE);
return;
}
if (v1Path.indexOf("%") !== -1) {
this._log(`The entry ${v1Path} is already migrated`, LOG_LEVEL_VERBOSE);
return;
}
const loadedEntry = await this.dependencies.getLocalDatabase().getDBEntry(v1Path);
if (!loadedEntry) {
this._log(`The entry ${v1Path} is not found`, LOG_LEVEL_VERBOSE);
return;
}
const pluginData = this.dependencies.codec.deserialize(getDocDataAsArray(loadedEntry.data), {}) as {
category: string;
files: Array<{ filename: string; data: string[] }>;
};
const prefixPath = v1Path.slice(0, -".md".length) + "%";
const category = pluginData.category;
for (const f of pluginData.files) {
const stripTable: Record<string, number> = {
CONFIG: 0,
THEME: 2,
SNIPPET: 1,
PLUGIN_MAIN: 2,
PLUGIN_DATA: 2,
PLUGIN_ETC: 2,
};
const deletePrefixCount = stripTable?.[category] ?? 1;
const relativeFilename = f.filename.split("/").slice(deletePrefixCount).join("/");
const v2Path = (prefixPath + relativeFilename) as FilePathWithPrefix;
this._log(`Migrating ${v1Path} / ${relativeFilename} to ${v2Path}`, LOG_LEVEL_VERBOSE);
const newId = await this.dependencies.path.path2id(v2Path);
const data = createBlob([
this.dependencies.codec.dummyHead,
this.dependencies.codec.dummyEnd,
...getDocDataAsArray(f.data),
]);
const saving: SavingEntry = {
...loadedEntry,
_rev: undefined,
_id: newId,
path: v2Path,
data,
datatype: "plain",
type: "plain",
children: [],
eden: {},
};
const result = await this.dependencies.getLocalDatabase().putDBEntry(saving);
if (result && result.ok) {
this._log(`Migrated ${v1Path} / ${f.filename} to ${v2Path}`, LOG_LEVEL_INFO);
const deletion = await this.dependencies.snapshotPersistence.deleteConfigOnDatabase(v1Path);
const deleted = deletion.value;
if (deleted) {
this._log(`Deleted ${v1Path} successfully`, LOG_LEVEL_INFO);
} else {
this._log(`Failed to delete ${v1Path}`, LOG_LEVEL_NOTICE);
}
// Compatibility: the inherited migration called the context
// deletion wrapper, which awaited its V1 catalogue refresh.
// Apply that refresh explicitly now that deletion is a host-
// neutral persistence operation, and only when deletion emitted
// the same mutation outcome.
for (const refresh of deletion.refreshes) {
if (refresh.mode == "v1" && refresh.timing == "await") {
await this.dependencies.refreshV1(false, refresh.path);
}
}
}
}
}
}
@@ -0,0 +1,216 @@
import { parseYaml } from "@/deps.ts";
import { writable } from "svelte/store";
import type {
AnyEntry,
FilePathWithPrefix,
LoadedEntry,
LOG_LEVEL,
ObsidianLiveSyncSettings,
} from "@vrtmrz/livesync-commonlib/compat/common/types";
import { ICXHeader } from "@/common/types.ts";
import { fireAndForget } from "@vrtmrz/livesync-commonlib/compat/common/utils";
import { digestHash } from "@vrtmrz/livesync-commonlib/compat/string_and_binary/hash";
import { QueueProcessor } from "octagonal-wheels/concurrency/processor";
import { reactiveSource, type ReactiveSource } from "octagonal-wheels/dataobject/reactive";
import type { LiveSyncLocalDB } from "@vrtmrz/livesync-commonlib/compat/pouchdb/LiveSyncLocalDB";
import type { IPathService } from "@vrtmrz/livesync-commonlib/compat/services/base/IService";
import type { LogFunction } from "@vrtmrz/livesync-commonlib/compat/services/lib/logUtils";
import { CatalogueMigration } from "./catalogueMigration.ts";
import { CatalogueState } from "./catalogueState.ts";
import { CatalogueV1 } from "./catalogueV1.ts";
import { CatalogueV2 } from "./catalogueV2.ts";
import { createCustomisationSyncCodec } from "./customisationSyncCodec.ts";
import type { SnapshotPersistence } from "./snapshotPersistence.ts";
import type { IPluginDataExDisplay, LoadedEntryPluginDataExFile } from "./customisationSyncView.ts";
const {
serialize,
deserialize,
dummyHead: DUMMY_HEAD,
dummyEnd: DUMMY_END,
} = createCustomisationSyncCodec({ digestHash, parseYaml });
const READ_CODEC = { deserialize, serialize };
const MIGRATION_CODEC = { deserialize, dummyHead: DUMMY_HEAD, dummyEnd: DUMMY_END };
const V2_CODEC = { dummyEnd: DUMMY_END };
type CatalogueSettings = Pick<ObsidianLiveSyncSettings, "usePluginSync" | "usePluginSyncV2">;
type CatalogueDatabase = Pick<LiveSyncLocalDB, "findEntries" | "getDBEntry" | "putDBEntry">;
export type CatalogueOperationsDependencies = {
getSettings(): CatalogueSettings;
getLocalDatabase(): CatalogueDatabase;
path: Pick<IPathService, "getPath" | "path2id">;
log: LogFunction;
snapshotPersistence: Pick<SnapshotPersistence, "deleteConfigOnDatabase">;
publishScanCount(count: number): void;
};
/** Coordinates the shared catalogue state, scan queue, and format modules. */
export class CatalogueOperations {
private readonly dependencies: CatalogueOperationsDependencies;
private readonly catalogueState = new CatalogueState();
private readonly scanProgress = reactiveSource(0);
private readonly pluginScanningChanged: Parameters<ReactiveSource<number>["onChanged"]>[0] = (event) => {
this.enumerationActive.set(event.value != 0);
this.dependencies.publishScanCount(event.value);
};
private readonly pluginScanProcessor: QueueProcessor<AnyEntry, AnyEntry>;
private readonly catalogueV1: CatalogueV1;
private readonly catalogueV2: CatalogueV2;
private readonly catalogueMigration: CatalogueMigration;
readonly enumerationActive = writable(false);
readonly catalogue = this.catalogueState.catalogue;
readonly migrationProgress = this.catalogueState.migrationProgress;
readonly manifests = this.catalogueState.manifests;
constructor(dependencies: CatalogueOperationsDependencies) {
this.dependencies = dependencies;
this.catalogueV1 = new CatalogueV1({
getLocalDatabase: () => this.dependencies.getLocalDatabase(),
path: {
getPath: (entry) => this.getPath(entry),
},
log: (message, level, key) => this._log(message, level, key),
state: this.catalogueState,
});
this.catalogueV2 = new CatalogueV2({
getLocalDatabase: () => this.dependencies.getLocalDatabase(),
log: (message, level, key) => this._log(message, level, key),
state: this.catalogueState,
codec: V2_CODEC,
});
this.catalogueMigration = new CatalogueMigration({
getLocalDatabase: () => this.dependencies.getLocalDatabase(),
path: {
path2id: (path) => this.path2id(path),
},
log: (message, level, key) => this._log(message, level, key),
snapshotPersistence: this.dependencies.snapshotPersistence,
refreshV1: async (showMessage, path) => await this.updatePluginList(showMessage, path),
codec: MIGRATION_CODEC,
});
this.scanProgress.onChanged(this.pluginScanningChanged);
// The single queue deliberately chooses V1 loading or migration when
// each item starts. Settings can change after enqueueing an item.
this.pluginScanProcessor = new QueueProcessor(
async (v: AnyEntry[]) => {
const plugin = v[0];
if (this.dependencies.getSettings().usePluginSyncV2) {
await this.migrateV1ToV2(false, plugin);
return [];
}
await this.catalogueV1.load(plugin, READ_CODEC);
return [];
},
{
suspended: false,
batchSize: 1,
concurrentLimit: 10,
delay: 100,
yieldThreshold: 10,
maintainDelay: false,
totalRemainingReactiveSource: this.scanProgress,
}
).startPipeline();
}
private get settings() {
return this.dependencies.getSettings();
}
private get localDatabase() {
return this.dependencies.getLocalDatabase();
}
private getPath(entry: AnyEntry): FilePathWithPrefix {
return this.dependencies.path.getPath(entry);
}
private async path2id(filename: FilePathWithPrefix) {
return await this.dependencies.path.path2id(filename);
}
private _log(message: unknown, level?: LOG_LEVEL, key?: string): void {
this.dependencies.log(message, level, key);
}
/** The current manifest lookup passed to V2 display rows. */
get manifestLookup() {
return this.catalogueState.manifestLookup;
}
/** Returns every row matching a document path, preserving legacy duplicates. */
findPlugins(documentPath: FilePathWithPrefix | string): readonly IPluginDataExDisplay[] {
return this.catalogueState.findPlugins(documentPath);
}
dispose(): void {
this.pluginScanProcessor.terminate();
this.scanProgress.offChanged(this.pluginScanningChanged);
this.enumerationActive.set(false);
this.dependencies.publishScanCount(0);
}
async reloadPluginList(showMessage: boolean): Promise<void> {
this.catalogueState.clearForReload();
await this.updatePluginList(showMessage);
}
async updatePluginList(showMessage: boolean, updatedDocumentPath?: FilePathWithPrefix): Promise<void> {
if (!this.settings.usePluginSync) {
this.pluginScanProcessor.clearQueue();
this.catalogueState.clearForDisabledRefresh();
return;
}
try {
this.catalogueState.beginUpdate();
const updatedDocumentId = updatedDocumentPath ? await this.path2id(updatedDocumentPath) : "";
const plugins = updatedDocumentPath
? this.localDatabase.findEntries(updatedDocumentId, updatedDocumentId + "\u{10ffff}", {
include_docs: true,
key: updatedDocumentId,
limit: 1,
})
: this.localDatabase.findEntries(ICXHeader + "", `${ICXHeader}\u{10ffff}`, { include_docs: true });
for await (const v of plugins) {
if (v.deleted || v._deleted) continue;
if (v.path.indexOf("%") !== -1) {
fireAndForget(() => this.updatePluginListV2(showMessage, v.path));
continue;
}
const path = v.path || this.getPath(v);
if (updatedDocumentPath && updatedDocumentPath != path) continue;
this.pluginScanProcessor.enqueue(v);
}
} finally {
this.enumerationActive.set(false);
this.catalogueState.endUpdate();
}
this.enumerationActive.set(false);
}
async createPluginDataExFileV2(
unifiedPathV2: FilePathWithPrefix,
loaded?: LoadedEntry
): Promise<false | LoadedEntryPluginDataExFile> {
return await this.catalogueV2.createPluginDataExFileV2(unifiedPathV2, loaded);
}
createPluginDataFromV2(unifiedPathV2: FilePathWithPrefix) {
return this.catalogueV2.createPluginDataFromV2(unifiedPathV2);
}
async updatePluginListV2(showMessage: boolean, unifiedFilenameWithKey: FilePathWithPrefix): Promise<void> {
await this.catalogueV2.updatePluginListV2(showMessage, unifiedFilenameWithKey);
}
private async migrateV1ToV2(showMessage: boolean, entry: AnyEntry): Promise<void> {
await this.catalogueMigration.migrateV1ToV2(showMessage, entry);
}
}
@@ -0,0 +1,222 @@
import { get } from "svelte/store";
import { beforeEach, describe, expect, it, vi } from "vitest";
const testState = vi.hoisted(() => ({
processors: [] as Array<{
clearQueue: ReturnType<typeof vi.fn>;
enqueue: ReturnType<typeof vi.fn>;
terminate: ReturnType<typeof vi.fn>;
startPipeline: ReturnType<typeof vi.fn>;
process: (entries: AnyEntry[]) => Promise<AnyEntry[]>;
}>,
reactiveSources: [] as Array<{
value: number;
onChanged: ReturnType<typeof vi.fn>;
offChanged: ReturnType<typeof vi.fn>;
}>,
}));
vi.mock("@/deps.ts", () => ({
parseYaml: vi.fn(),
}));
vi.mock("@/common/types.ts", () => ({
ICXHeader: "ix:",
}));
vi.mock("@/common/utils.ts", () => ({
fireAndForget: vi.fn(),
scheduleTask: vi.fn(),
}));
vi.mock("octagonal-wheels/concurrency/processor", () => ({
QueueProcessor: class QueueProcessor {
clearQueue = vi.fn();
enqueue = vi.fn();
terminate = vi.fn();
startPipeline = vi.fn(() => this);
process: (entries: AnyEntry[]) => Promise<AnyEntry[]>;
constructor(process: (entries: AnyEntry[]) => Promise<AnyEntry[]>) {
this.process = process;
testState.processors.push(this);
}
},
}));
vi.mock("octagonal-wheels/dataobject/reactive", () => ({
reactiveSource: vi.fn((value: number) => {
const source = {
value,
onChanged: vi.fn(),
offChanged: vi.fn(),
};
testState.reactiveSources.push(source);
return source;
}),
}));
import { scheduleTask } from "@/common/utils.ts";
import type {
AnyEntry,
DocumentID,
FilePathWithPrefix,
LoadedEntry,
} from "@vrtmrz/livesync-commonlib/compat/common/types";
import { digestHash } from "@vrtmrz/livesync-commonlib/compat/string_and_binary/hash";
import { createCustomisationSyncCodec } from "./customisationSyncCodec.ts";
import { CatalogueOperations, type CatalogueOperationsDependencies } from "./catalogueOperations.ts";
import type { SnapshotPersistenceResult } from "./snapshotPersistence.ts";
const codec = createCustomisationSyncCodec({
digestHash,
parseYaml: () => undefined,
});
const v2Path = "ix:device-a/PLUGIN_DATA/example%data.json" as FilePathWithPrefix;
const v1Path = "ix:device-a/CONFIG/app.json.md" as FilePathWithPrefix;
function loadedEntry(): LoadedEntry {
const data = `${codec.dummyHead}${codec.dummyEnd}${btoa("example data")}`;
return {
_id: "entry-id",
_rev: "1-a",
path: v2Path,
type: "plain",
datatype: "plain",
data,
ctime: 10,
mtime: 20,
size: data.length,
children: [],
eden: {},
} as unknown as LoadedEntry;
}
function createOperations() {
const settings = { usePluginSync: true, usePluginSyncV2: true };
const database = {
findEntries: vi.fn(async function* () {
// No refresh entries are needed by the focused V2 test.
}),
getDBEntry: vi.fn(async () => loadedEntry()),
putDBEntry: vi.fn(async () => ({ ok: true, id: "entry-id", rev: "2-b" })),
};
const deleteConfigOnDatabase = vi.fn(
async (): Promise<SnapshotPersistenceResult<boolean>> => ({
value: true,
status: "missing",
refreshes: [],
})
);
const snapshotPersistence = {
deleteConfigOnDatabase,
};
const dependencies: CatalogueOperationsDependencies = {
getSettings: () => settings,
getLocalDatabase: () => database,
path: {
getPath: (entry) => entry.path,
path2id: async (path) => path as unknown as DocumentID,
},
log: vi.fn(),
snapshotPersistence,
publishScanCount: vi.fn(),
};
return {
database,
dependencies,
settings,
operations: new CatalogueOperations(dependencies),
snapshotPersistence,
};
}
describe("Customisation Sync catalogue operations", () => {
beforeEach(() => {
testState.processors.length = 0;
testState.reactiveSources.length = 0;
vi.clearAllMocks();
});
it("owns and releases the active shared scan processor and its progress subscription", () => {
const { dependencies, operations } = createOperations();
operations.enumerationActive.set(true);
operations.dispose();
expect(testState.processors).toHaveLength(1);
expect(testState.processors[0].terminate).toHaveBeenCalledOnce();
expect(testState.reactiveSources[0].offChanged).toHaveBeenCalledOnce();
expect(get(operations.enumerationActive)).toBe(false);
expect(dependencies.publishScanCount).toHaveBeenCalledWith(0);
});
it("chooses loading or migration when queued work starts", async () => {
const { operations, settings } = createOperations();
const migrate = vi
.spyOn(
operations as unknown as { migrateV1ToV2: (showMessage: boolean, entry: AnyEntry) => Promise<void> },
"migrateV1ToV2"
)
.mockResolvedValue(undefined);
const entry = { path: v1Path, deleted: false } as AnyEntry;
settings.usePluginSyncV2 = false;
await testState.processors[0].process([entry]);
expect(migrate).not.toHaveBeenCalled();
settings.usePluginSyncV2 = true;
await testState.processors[0].process([entry]);
expect(migrate).toHaveBeenCalledOnce();
operations.dispose();
});
it("keeps V2 row publication delayed behind the process-global refresh task", async () => {
const { operations } = createOperations();
await operations.updatePluginListV2(false, v2Path);
expect(get(operations.catalogue)).toEqual([]);
expect(scheduleTask).toHaveBeenCalledWith("updatePluginListV2", 100, expect.any(Function));
const publish = vi.mocked(scheduleTask).mock.calls[0]?.[2] as (() => void) | undefined;
publish?.();
expect(get(operations.catalogue)).toHaveLength(1);
expect(get(operations.catalogue)[0]).toMatchObject({
documentPath: "ix:device-a/PLUGIN_DATA/example.md",
files: [{ filename: "plugins/example/data.json" }],
});
operations.dispose();
});
it("uses the persistence deletion outcome and explicitly awaits migration refresh", async () => {
const { database, operations, snapshotPersistence } = createOperations();
const loadedV1 = {
...loadedEntry(),
path: v1Path,
data: codec.serialize({
category: "CONFIG",
name: "app.json",
term: "device-a",
files: [{ filename: "app.json", data: [btoa("config")], mtime: 10, size: 6 }],
mtime: 10,
}),
} as LoadedEntry;
database.getDBEntry.mockResolvedValue(loadedV1);
snapshotPersistence.deleteConfigOnDatabase.mockResolvedValue({
value: true,
status: "deleted",
refreshes: [{ mode: "v1", timing: "await", path: v1Path }],
});
const refresh = vi.spyOn(operations, "updatePluginList").mockResolvedValue(undefined);
await (
operations as unknown as {
migrateV1ToV2(showMessage: boolean, entry: LoadedEntry): Promise<void>;
}
).migrateV1ToV2(false, { path: v1Path, deleted: false } as LoadedEntry);
expect(database.putDBEntry).toHaveBeenCalledOnce();
expect(snapshotPersistence.deleteConfigOnDatabase).toHaveBeenCalledWith(v1Path);
expect(refresh).toHaveBeenCalledWith(false, v1Path);
operations.dispose();
});
});
+153
View File
@@ -0,0 +1,153 @@
import type { PluginManifest } from "@/deps.ts";
import type { FilePathWithPrefix } from "@vrtmrz/livesync-commonlib/compat/common/types";
import { isObjectDifferent } from "@vrtmrz/livesync-commonlib/compat/common/utils";
import { writable } from "svelte/store";
import { PluginDataExDisplayV2 } from "./customisationSyncModel.ts";
import type { IPluginDataExDisplay } from "./customisationSyncView.ts";
/**
* Owns the transient catalogue projection used by Customisation Sync.
*
* The database and storage operations remain in the catalogue modules. This
* owner only coordinates the in-memory rows, their reactive publications,
* manifest lookup, and the update counter which is derived from those
* operations.
*/
export class CatalogueState {
private catalogueRows: IPluginDataExDisplay[] = [];
private readonly manifestByKey = new Map<string, PluginManifest>();
private readonly loadedManifestMTimeByKey = new Map<string, number>();
private activeUpdateCount = 0;
readonly catalogue = writable<IPluginDataExDisplay[]>([]);
readonly migrationProgress = writable(0);
readonly manifests = writable(this.manifestByKey);
/** The current manifest lookup passed to V2 display rows. */
get manifestLookup(): ReadonlyMap<string, PluginManifest> {
return this.manifestByKey;
}
/** The current loaded-manifest cache, exposed read-only for diagnostics. */
get loadedManifestMTime(): ReadonlyMap<string, number> {
return this.loadedManifestMTimeByKey;
}
/** Returns the authoritative row for a document path, when present. */
findPlugin(documentPath: FilePathWithPrefix | string): IPluginDataExDisplay | undefined {
return this.catalogueRows.find((entry) => entry.documentPath == documentPath);
}
/** Returns every row matching a document path, preserving legacy duplicates. */
findPlugins(documentPath: FilePathWithPrefix | string): readonly IPluginDataExDisplay[] {
return this.catalogueRows.filter((entry) => entry.documentPath == documentPath);
}
/** Replaces a V1 row and publishes it immediately, preserving legacy order. */
replacePlugin(plugin: IPluginDataExDisplay): void {
const newList = this.catalogueRows.filter((entry) => entry.documentPath != plugin.documentPath);
newList.push(plugin);
this.catalogueRows = newList;
this.catalogue.set(newList);
}
/**
* Replaces a V2 row without publishing it. V2 callers publish through the
* existing delayed task after a cohesive row update has completed.
*/
private replaceV2Plugin(plugin: PluginDataExDisplayV2): void {
const newList = this.catalogueRows.filter((entry) => entry.documentPath != plugin.documentPath);
newList.push(plugin);
this.catalogueRows = newList;
}
/** Applies one loaded or removed V2 file and replaces its catalogue row. */
async updateV2Plugin(
plugin: PluginDataExDisplayV2,
file: Parameters<PluginDataExDisplayV2["setFile"]>[0] | false,
missingFilePath: string
): Promise<void> {
if (file) {
await plugin.setFile(file);
} else {
plugin.deleteFile(missingFilePath);
}
this.replaceV2Plugin(plugin);
}
/** Publishes the current V2 row set when the legacy delayed task fires. */
publishCatalogue(): void {
this.catalogue.set(this.catalogueRows);
}
/**
* Clears rows and loaded manifest mtimes for an explicit reload. The
* manifest map intentionally survives this narrower refresh.
*/
clearForReload(): void {
this.catalogueRows = [];
this.loadedManifestMTimeByKey.clear();
this.catalogue.set(this.catalogueRows);
}
/** Clears only the catalogue rows for a disabled refresh. */
clearForDisabledRefresh(): void {
this.catalogueRows = [];
this.catalogue.set(this.catalogueRows);
}
/** Begins one catalogue update and publishes its progress count. */
beginUpdate(): void {
this.activeUpdateCount++;
this.migrationProgress.set(this.activeUpdateCount);
}
/** Ends one catalogue update and publishes its progress count. */
endUpdate(): void {
this.activeUpdateCount--;
this.migrationProgress.set(this.activeUpdateCount);
}
/**
* Applies a manifest according to the inherited cache rules. A manifest is
* parsed only when no manifest has previously been accepted for the key;
* failed parses still record their mtime, while a later mtime never
* replaces a successfully parsed first manifest.
*/
processManifest(
confKey: string,
mtime: number,
parseManifest: () => PluginManifest,
onParseError: (error: unknown) => void = () => undefined
): void {
let publishCatalogue = false;
if (this.loadedManifestMTimeByKey.get(confKey) != mtime && this.manifestByKey.get(confKey) == undefined) {
try {
this.setManifest(confKey, parseManifest());
this.applyLoadedManifest(confKey);
publishCatalogue = true;
} catch (error) {
onParseError(error);
}
this.loadedManifestMTimeByKey.set(confKey, mtime);
} else {
this.applyLoadedManifest(confKey);
publishCatalogue = true;
}
if (publishCatalogue) this.catalogue.set(this.catalogueRows);
}
private setManifest(key: string, manifest: PluginManifest): void {
const old = this.manifestByKey.get(key);
if (old && !isObjectDifferent(manifest, old)) return;
this.manifestByKey.set(key, manifest);
this.manifests.set(this.manifestByKey);
}
private applyLoadedManifest(confKey: string): void {
this.catalogueRows
.filter((entry) => entry instanceof PluginDataExDisplayV2 && entry.confKey == confKey)
.forEach((entry) => (entry as PluginDataExDisplayV2).applyLoadedManifest());
}
}
@@ -0,0 +1,127 @@
import { get } from "svelte/store";
import { describe, expect, it, vi } from "vitest";
import type { PluginManifest } from "@/deps.ts";
import type { FilePathWithPrefix } from "@vrtmrz/livesync-commonlib/compat/common/types";
import { CatalogueState } from "./catalogueState.ts";
import { PluginDataExDisplayV2 } from "./customisationSyncModel.ts";
import type { IPluginDataExDisplay, LoadedEntryPluginDataExFile } from "./customisationSyncView.ts";
function display(documentPath = "ix:device-a/PLUGIN_MAIN/example.md"): IPluginDataExDisplay {
return {
documentPath: documentPath as FilePathWithPrefix,
category: "PLUGIN_MAIN",
name: "example",
term: "device-a",
files: [],
mtime: 1,
};
}
function file(filename: string, mtime: number): LoadedEntryPluginDataExFile {
return {
path: `ix:device-a/PLUGIN_MAIN/example%${filename}` as FilePathWithPrefix,
filename,
mtime,
data: [filename],
size: filename.length,
} as LoadedEntryPluginDataExFile;
}
describe("Customisation Sync catalogue state", () => {
it("publishes V1 replacement and keeps V2 replacement delayed", async () => {
const state = new CatalogueState();
const setCatalogue = vi.spyOn(state.catalogue, "set");
const row = display();
state.replacePlugin(row);
expect(get(state.catalogue)).toEqual([row]);
expect(setCatalogue).toHaveBeenCalledOnce();
const v2 = new PluginDataExDisplayV2(
{
...display(),
files: [file("main.js", 1)],
},
state.manifestLookup
);
await state.updateV2Plugin(v2, file("main.js", 2), "main.js");
expect(get(state.catalogue)).toEqual([row]);
expect(state.findPlugin(row.documentPath)).toBe(v2);
state.publishCatalogue();
expect(get(state.catalogue)).toEqual([v2]);
});
it("retains the first parsed manifest and records failed mtimes", () => {
const state = new CatalogueState();
const first = { name: "First", version: "1.0.0" } as PluginManifest;
const parseManifest = vi.fn(() => first);
state.processManifest("device-a/plugins/example", 20, parseManifest);
state.processManifest(
"device-a/plugins/example",
30,
() => ({ name: "Second", version: "2.0.0" }) as PluginManifest
);
expect(state.manifestLookup.get("device-a/plugins/example")).toBe(first);
expect(state.loadedManifestMTime.get("device-a/plugins/example")).toBe(20);
expect(parseManifest).toHaveBeenCalledOnce();
const failedState = new CatalogueState();
const onParseError = vi.fn();
const failure = new SyntaxError("invalid");
failedState.processManifest(
"device-a/plugins/failure",
40,
() => {
throw failure;
},
onParseError
);
failedState.processManifest("device-a/plugins/failure", 40, () => first, onParseError);
expect(onParseError).toHaveBeenCalledWith(failure);
expect(failedState.loadedManifestMTime.get("device-a/plugins/failure")).toBe(40);
expect(failedState.manifestLookup.has("device-a/plugins/failure")).toBe(false);
});
it("clears rows and loaded mtimes on reload while retaining manifest lookup", () => {
const state = new CatalogueState();
const key = "device-a/plugins/example";
state.processManifest(key, 20, () => ({ name: "Example" }) as PluginManifest);
state.replacePlugin(display());
state.clearForReload();
expect(get(state.catalogue)).toEqual([]);
expect(state.loadedManifestMTime.size).toBe(0);
expect(state.manifestLookup.get(key)).toEqual({ name: "Example" });
expect(get(state.catalogue)).toEqual([]);
});
it("keeps manifest caches through the narrower disabled refresh", () => {
const state = new CatalogueState();
const key = "device-a/plugins/example";
state.processManifest(key, 20, () => ({ name: "Example" }) as PluginManifest);
state.replacePlugin(display());
state.clearForDisabledRefresh();
expect(get(state.catalogue)).toEqual([]);
expect(state.loadedManifestMTime.get(key)).toBe(20);
expect(state.manifestLookup.has(key)).toBe(true);
});
it("tracks V2 updates through migration progress", () => {
const state = new CatalogueState();
state.beginUpdate();
state.beginUpdate();
expect(get(state.migrationProgress)).toBe(2);
state.endUpdate();
state.endUpdate();
expect(get(state.migrationProgress)).toBe(0);
});
});
+50
View File
@@ -0,0 +1,50 @@
import type { AnyEntry, LOG_LEVEL } from "@vrtmrz/livesync-commonlib/compat/common/types";
import { LOG_LEVEL_NOTICE, LOG_LEVEL_VERBOSE } from "@vrtmrz/livesync-commonlib/compat/common/types";
import type { LiveSyncLocalDB } from "@vrtmrz/livesync-commonlib/compat/pouchdb/LiveSyncLocalDB";
import type { IPathService } from "@vrtmrz/livesync-commonlib/compat/services/base/IService";
import type { LogFunction } from "@vrtmrz/livesync-commonlib/compat/services/lib/logUtils";
import { CatalogueState } from "./catalogueState.ts";
import { loadCustomisationDisplayData, type CustomisationSyncReadCodec } from "./customisationSyncReadOperations.ts";
type CatalogueV1Database = Pick<LiveSyncLocalDB, "getDBEntry" | "putDBEntry">;
export type CatalogueV1Dependencies = {
getLocalDatabase(): CatalogueV1Database;
path: Pick<IPathService, "getPath">;
log: LogFunction;
state: CatalogueState;
};
/** Loads and publishes legacy V1 catalogue rows. */
export class CatalogueV1 {
constructor(private readonly dependencies: CatalogueV1Dependencies) {}
private _log(message: unknown, level?: LOG_LEVEL, key?: string): void {
this.dependencies.log(message, level, key);
}
async load(entry: AnyEntry, codec: CustomisationSyncReadCodec): Promise<void> {
const path = entry.path || this.dependencies.path.getPath(entry);
const oldEntry = this.dependencies.state.findPlugin(path);
if (oldEntry && oldEntry.mtime == entry.mtime) return;
try {
const pluginData = await loadCustomisationDisplayData(
{
getLocalDatabase: () => this.dependencies.getLocalDatabase(),
path: this.dependencies.path,
log: this.dependencies.log,
},
path,
codec
);
if (pluginData) {
this.dependencies.state.replacePlugin(pluginData);
}
// Failed to load
} catch (ex) {
this._log(`Something happened at enumerating customization :${path}`, LOG_LEVEL_NOTICE);
this._log(ex, LOG_LEVEL_VERBOSE);
}
}
}
+123
View File
@@ -0,0 +1,123 @@
import type { PluginManifest } from "@/deps.ts";
import type { FilePathWithPrefix, LoadedEntry, LOG_LEVEL } from "@vrtmrz/livesync-commonlib/compat/common/types";
import { LOG_LEVEL_VERBOSE } from "@vrtmrz/livesync-commonlib/compat/common/types";
import type { LiveSyncLocalDB } from "@vrtmrz/livesync-commonlib/compat/pouchdb/LiveSyncLocalDB";
import type { LogFunction } from "@vrtmrz/livesync-commonlib/compat/services/lib/logUtils";
import { scheduleTask } from "@/common/utils.ts";
import { CatalogueState } from "./catalogueState.ts";
import { PluginDataExDisplayV2 } from "./customisationSyncModel.ts";
import { parseCustomisationSyncV2DocumentPath } from "./customisationSyncPaths.ts";
import { decodeCustomisationSyncV2File, loadCustomisationV2Entry } from "./customisationSyncReadOperations.ts";
import type { LoadedEntryPluginDataExFile } from "./customisationSyncView.ts";
type CatalogueV2Database = Pick<LiveSyncLocalDB, "getDBEntry">;
export type CatalogueV2Dependencies = {
getLocalDatabase(): CatalogueV2Database;
log: LogFunction;
state: CatalogueState;
codec: { dummyEnd: string };
};
/** Builds, updates, and publishes V2 catalogue rows and manifests. */
export class CatalogueV2 {
constructor(private readonly dependencies: CatalogueV2Dependencies) {}
private _log(message: unknown, level?: LOG_LEVEL, key?: string): void {
this.dependencies.log(message, level, key);
}
get manifestLookup() {
return this.dependencies.state.manifestLookup;
}
async createPluginDataExFileV2(
unifiedPathV2: FilePathWithPrefix,
loaded?: LoadedEntry
): Promise<false | LoadedEntryPluginDataExFile> {
// Compatibility: a caller-supplied entry bypasses the database lookup
// and the isLoadedEntry check performed by loadCustomisationV2Entry.
const loadedEntry =
loaded ??
(await loadCustomisationV2Entry(
{
getLocalDatabase: () => this.dependencies.getLocalDatabase(),
log: this.dependencies.log,
},
unifiedPathV2
));
if (!loadedEntry) return false;
const { confKey, file, isManifest } = decodeCustomisationSyncV2File(
unifiedPathV2,
loadedEntry,
this.dependencies.codec.dummyEnd
);
if (isManifest) {
this.dependencies.state.processManifest(
confKey,
file.mtime,
() => JSON.parse(file.data[0]) as PluginManifest,
(error) => {
this._log(
`The file ${loadedEntry.path} seems to manifest, but could not be decoded as JSON`,
LOG_LEVEL_VERBOSE
);
this._log(error, LOG_LEVEL_VERBOSE);
}
);
}
return file;
}
createPluginDataFromV2(unifiedPathV2: FilePathWithPrefix): PluginDataExDisplayV2 | undefined {
const { category, device, key, pathV1 } = parseCustomisationSyncV2DocumentPath(unifiedPathV2);
if (category == "") return;
return new PluginDataExDisplayV2(
{
documentPath: pathV1,
category,
name: key,
term: `${device}`,
files: [],
mtime: 0,
},
this.dependencies.state.manifestLookup
);
}
async updatePluginListV2(showMessage: boolean, unifiedFilenameWithKey: FilePathWithPrefix): Promise<void> {
// The public parameter is retained for the established catalogue
// signature; V2 publication has never used it.
void showMessage;
try {
this.dependencies.state.beginUpdate();
const { pathV1 } = parseCustomisationSyncV2DocumentPath(unifiedFilenameWithKey);
const oldEntry = this.dependencies.state.findPlugin(pathV1);
let entry: PluginDataExDisplayV2 | undefined;
// Compatibility question: when a V1 row is found first for this
// logical path, the inherited implementation constructs a fresh
// V2 row rather than looking for another existing V2 row. Preserve
// that selection until mixed-format catalogue races are covered.
if (!oldEntry || !(oldEntry instanceof PluginDataExDisplayV2)) {
entry = this.createPluginDataFromV2(unifiedFilenameWithKey);
} else {
entry = oldEntry;
}
if (!entry) return;
const file = await this.createPluginDataExFileV2(unifiedFilenameWithKey);
// Compatibility: the inherited update always re-adds an empty V2
// row after deleting its final file.
await this.dependencies.state.updateV2Plugin(entry, file, unifiedFilenameWithKey);
scheduleTask("updatePluginListV2", 100, () => {
this.dependencies.state.publishCatalogue();
});
} finally {
this.dependencies.state.endUpdate();
}
}
}
@@ -0,0 +1,209 @@
import type { FilePathWithPrefix } from "@vrtmrz/livesync-commonlib/compat/common/types";
const FIELD_DELIMITER = "\u200b";
const LINE_DELIMITER = "\n";
export type PluginDataExFile = {
filename: string;
data: string[];
mtime: number;
size: number;
version?: string;
hash?: string;
displayName?: string;
};
export type PluginDataEx = {
documentPath?: FilePathWithPrefix;
category: string;
name: string;
displayName?: string;
term: string;
files: PluginDataExFile[];
version?: string;
mtime: number;
};
export type CustomisationSyncCodecDependencies = {
digestHash(data: string[]): string;
parseYaml(source: string): unknown;
};
function splitWithDelimiters(sources: string[]): string[] {
const result: string[] = [];
for (const str of sources) {
let startIndex = 0;
const maxLen = str.length;
let i = -1;
let fieldIndex;
let lineIndex;
do {
fieldIndex = str.indexOf(FIELD_DELIMITER, startIndex);
lineIndex = str.indexOf(LINE_DELIMITER, startIndex);
if (fieldIndex == -1 && lineIndex == -1) {
break;
}
if (fieldIndex == -1) {
i = lineIndex;
} else if (lineIndex == -1) {
i = fieldIndex;
} else {
i = fieldIndex < lineIndex ? fieldIndex : lineIndex;
}
result.push(str.slice(startIndex, i + 1));
startIndex = i + 1;
} while (i < maxLen);
if (startIndex < maxLen) {
result.push(str.slice(startIndex));
}
}
// Preserve the legacy trailing-empty-chunk behaviour.
if (sources[sources.length - 1] == "") {
result.push("");
}
return result;
}
function getTokenizer(source: string[]) {
const sources = splitWithDelimiters(source);
sources[0] = sources[0].substring(1);
let pos = 0;
let lineRunOut = false;
return {
next(): string {
if (lineRunOut) {
return "";
}
if (pos >= sources.length) {
return "";
}
const item = sources[pos];
if (!item.endsWith(LINE_DELIMITER)) {
pos++;
} else {
lineRunOut = true;
}
if (item.endsWith(FIELD_DELIMITER) || item.endsWith(LINE_DELIMITER)) {
return item.substring(0, item.length - 1);
}
return item + this.next();
},
nextLine() {
if (lineRunOut) {
pos++;
} else {
while (!sources[pos].endsWith(LINE_DELIMITER)) {
pos++;
if (pos >= sources.length) break;
}
pos++;
}
lineRunOut = false;
},
};
}
function deserializeCustomFormat(source: string[]): PluginDataEx {
const tokens = getTokenizer(source);
const category = tokens.next();
const name = tokens.next();
const term = tokens.next();
tokens.nextLine();
const version = tokens.next();
tokens.nextLine();
const mtime = Number(tokens.next());
tokens.nextLine();
const result: PluginDataEx = {
category,
name,
term,
version,
mtime,
files: [],
};
let filename = "";
do {
filename = tokens.next();
if (!filename) break;
const displayName = tokens.next();
const fileVersion = tokens.next();
tokens.nextLine();
const fileMtime = Number(tokens.next());
const size = Number(tokens.next());
const hash = tokens.next();
tokens.nextLine();
const data: string[] = [];
let piece = "";
do {
piece = tokens.next();
if (piece == "") break;
data.push(piece);
} while (piece != "");
result.files.push({
filename,
displayName,
version: fileVersion,
mtime: fileMtime,
size,
data,
hash,
});
tokens.nextLine();
} while (filename);
return result;
}
export function createCustomisationSyncCodec(dependencies: CustomisationSyncCodecDependencies) {
function serialize(data: PluginDataEx): string {
// Group fields with similar entropy around newlines to retain the existing chunking characteristics.
let result = ":";
result += data.category + FIELD_DELIMITER + data.name + FIELD_DELIMITER + data.term + LINE_DELIMITER;
result += (data.version ?? "") + LINE_DELIMITER;
result += data.mtime + LINE_DELIMITER;
for (const file of data.files) {
result +=
file.filename +
FIELD_DELIMITER +
(file.displayName ?? "") +
FIELD_DELIMITER +
(file.version ?? "") +
LINE_DELIMITER;
const hash = dependencies.digestHash(file.data ?? []);
result += file.mtime + FIELD_DELIMITER + file.size + FIELD_DELIMITER + hash + LINE_DELIMITER;
for (const piece of file.data ?? []) {
result += piece + FIELD_DELIMITER;
}
result += LINE_DELIMITER;
}
return result;
}
function deserialize<T>(source: string[], defaultValue: T): T {
try {
if (source[0][0] == ":") {
return deserializeCustomFormat(source) as T;
}
return JSON.parse(source.join("")) as T;
} catch {
try {
return dependencies.parseYaml(source.join("")) as T;
} catch {
return defaultValue;
}
}
}
const dummyHead = serialize({
category: "CONFIG",
name: "migrated",
files: [],
mtime: 0,
term: "-",
displayName: "MIRAGED",
});
const dummyEnd = FIELD_DELIMITER + LINE_DELIMITER + "\u200c";
return { serialize, deserialize, dummyHead, dummyEnd };
}
@@ -0,0 +1,94 @@
import { describe, expect, it, vi } from "vitest";
import { createCustomisationSyncCodec, type PluginDataEx } from "./customisationSyncCodec.ts";
function createCodec() {
const digestHash = vi.fn((data: string[]) => `digest:${data.join("|")}`);
const parseYaml = vi.fn((_source: string): unknown => {
throw new Error("Invalid YAML");
});
return {
codec: createCustomisationSyncCodec({ digestHash, parseYaml }),
digestHash,
parseYaml,
};
}
const data: PluginDataEx = {
category: "PLUGIN_DATA",
name: "example",
term: "device-a",
version: "1.2.3",
mtime: 123,
files: [
{
filename: ".obsidian/plugins/example/data.json",
displayName: "data.json",
version: "2.0.0",
mtime: 120,
size: 6,
data: ["YWJj", "ZGVm"],
},
],
};
describe("compatibility: Customisation Sync codec", () => {
it("preserves the existing custom wire format", () => {
const { codec, digestHash } = createCodec();
expect(codec.serialize(data)).toBe(
":PLUGIN_DATA\u200bexample\u200bdevice-a\n" +
"1.2.3\n" +
"123\n" +
".obsidian/plugins/example/data.json\u200bdata.json\u200b2.0.0\n" +
"120\u200b6\u200bdigest:YWJj|ZGVm\n" +
"YWJj\u200bZGVm\u200b\n"
);
expect(digestHash).toHaveBeenCalledWith(["YWJj", "ZGVm"]);
});
it("round-trips the custom format across arbitrary source chunks", () => {
const { codec } = createCodec();
const serialised = codec.serialize(data);
const source = [serialised.slice(0, 13), serialised.slice(13, 47), serialised.slice(47), ""];
expect(codec.deserialize<PluginDataEx>(source, {} as PluginDataEx)).toEqual({
...data,
files: [
{
...data.files[0],
hash: "digest:YWJj|ZGVm",
},
],
});
});
it("retains JSON as the first legacy fallback", () => {
const { codec, parseYaml } = createCodec();
expect(codec.deserialize(['{"value":1}'], { value: 0 })).toEqual({ value: 1 });
expect(parseYaml).not.toHaveBeenCalled();
});
it("uses the injected YAML parser after JSON parsing fails", () => {
const parseYaml = vi.fn(() => ({ value: 2 }));
const codec = createCustomisationSyncCodec({ digestHash: vi.fn(() => "hash"), parseYaml });
expect(codec.deserialize(["value: 2"], { value: 0 })).toEqual({ value: 2 });
expect(parseYaml).toHaveBeenCalledWith("value: 2");
});
it("returns the supplied default when every decoder rejects the input", () => {
const { codec } = createCodec();
const defaultValue = { retained: true };
expect(codec.deserialize([], defaultValue)).toBe(defaultValue);
});
it("preserves the V2 migration sentinels", () => {
const { codec } = createCodec();
expect(codec.dummyHead).toBe(":CONFIG\u200bmigrated\u200b-\n\n0\n");
expect(codec.dummyEnd).toBe("\u200b\n\u200c");
});
});
@@ -0,0 +1,108 @@
import { describe, expect, it, vi } from "vitest";
vi.mock("@/deps.ts", () => ({
diff_match_patch: class DiffMatchPatch {},
normalizePath: vi.fn((path: string) => path),
parseYaml: vi.fn(),
Platform: {},
}));
vi.mock("@/common/types.ts", () => ({
ICXHeader: "ix:",
PERIODIC_PLUGIN_SWEEP: 60,
}));
vi.mock("@/common/utils.ts", () => ({
cancelTask: vi.fn(),
EVEN: Symbol("even"),
isCustomisationSyncMetadata: vi.fn(),
isPluginMetadata: vi.fn(),
scheduleTask: vi.fn(),
}));
vi.mock("@/common/PeriodicProcessor.ts", () => ({
PeriodicProcessor: class PeriodicProcessor {},
}));
vi.mock("@/common/translation", () => ({
$msg: vi.fn((message: string) => message),
}));
vi.mock("@/common/obsidianCommunityPlugins.ts", () => ({
getObsidianCommunityPluginManager: vi.fn(),
}));
import { cancelTask, scheduleTask } from "@/common/utils.ts";
import { CustomisationSyncContext } from "./customisationSyncContext";
import { createCustomisationSyncTestDependencies } from "./customisationSyncContext.unit.fixture.ts";
describe("CustomisationSyncContext commands", () => {
it("opens the host-owned dialogue from a scheduled configuration Notice", async () => {
const control = {
open: vi.fn(),
close: vi.fn(),
isOpen: vi.fn(() => false),
};
const showConfigurationNotice = vi.fn();
const updatePluginList = vi.fn(async () => undefined);
const configSync = Object.create(CustomisationSyncContext.prototype) as CustomisationSyncContext;
Object.assign(configSync, {
dependencies: createCustomisationSyncTestDependencies({
getUIControl: () => control,
getSettings: () => ({ usePluginSync: true, notifyPluginOrSettingUpdated: true }) as never,
showConfigurationNotice,
}),
updatePluginList,
});
await configSync.serviceHandlers.processVirtualDocument({
_id: "ix:example",
path: "ix:example",
} as never);
const scheduledNotice = vi.mocked(scheduleTask).mock.calls[0]?.[2] as (() => void) | undefined;
expect(scheduledNotice).toBeTypeOf("function");
scheduledNotice?.();
const openDialogue = showConfigurationNotice.mock.calls[0]?.[0] as (() => void) | undefined;
expect(openDialogue).toBeTypeOf("function");
openDialogue?.();
expect(control.open).toHaveBeenCalledOnce();
expect(updatePluginList).toHaveBeenCalledWith(false, "ix:example");
});
it("delegates catalogue resource release during disposal", () => {
const hideConfigurationNotice = vi.fn();
const periodicPluginSweepProcessor = { disable: vi.fn() };
const catalogueOperations = { dispose: vi.fn() };
const configSync = Object.create(CustomisationSyncContext.prototype) as CustomisationSyncContext;
Object.assign(configSync, {
dependencies: createCustomisationSyncTestDependencies({
hideConfigurationNotice,
}),
periodicPluginSweepProcessor,
catalogueOperations,
});
configSync.dispose();
expect(cancelTask).toHaveBeenCalledWith("config-sync:updated-configuration");
expect(hideConfigurationNotice).toHaveBeenCalledOnce();
expect(periodicPluginSweepProcessor.disable).toHaveBeenCalledOnce();
expect(catalogueOperations.dispose).toHaveBeenCalledOnce();
});
it("characterises the inherited setting-realisation gates pending separate review", async () => {
const isReady = vi.fn(() => false);
const isSuspended = vi.fn(() => false);
const periodicPluginSweepProcessor = { disable: vi.fn(), enable: vi.fn() };
const scanAllConfigFiles = vi.fn(async () => undefined);
const configSync = Object.create(CustomisationSyncContext.prototype) as CustomisationSyncContext;
Object.assign(configSync, {
dependencies: createCustomisationSyncTestDependencies({ isReady, isSuspended }),
periodicPluginSweepProcessor,
scanAllConfigFiles,
});
await expect(configSync.serviceHandlers.onRealiseSetting()).resolves.toBe(true);
expect(periodicPluginSweepProcessor.disable).toHaveBeenCalledOnce();
expect(isReady).not.toHaveBeenCalled();
expect(isSuspended).toHaveBeenCalledOnce();
expect(scanAllConfigFiles).not.toHaveBeenCalled();
expect(periodicPluginSweepProcessor.enable).not.toHaveBeenCalled();
});
});
@@ -0,0 +1,80 @@
import { get } from "svelte/store";
import { describe, expect, it, vi } from "vitest";
vi.mock("@/deps.ts", () => ({
diff_match_patch: class DiffMatchPatch {},
normalizePath: vi.fn((path: string) => path),
parseYaml: vi.fn(),
Platform: {},
}));
vi.mock("@/common/PeriodicProcessor.ts", () => ({
PeriodicProcessor: class PeriodicProcessor {
disable = vi.fn();
enable = vi.fn();
},
}));
vi.mock("octagonal-wheels/concurrency/processor", () => ({
QueueProcessor: class QueueProcessor {
clearQueue = vi.fn();
enqueue = vi.fn();
terminate = vi.fn();
startPipeline() {
return this;
}
},
}));
vi.mock("@/common/obsidianCommunityPlugins.ts", () => ({
getObsidianCommunityPluginManager: vi.fn(),
}));
import { CustomisationSyncContext } from "./customisationSyncContext.ts";
import { createCustomisationSyncTestDependencies } from "./customisationSyncContext.unit.fixture.ts";
describe("CustomisationSyncContext composition", () => {
it("does not share catalogue or presentation state between context instances", () => {
const first = new CustomisationSyncContext(createCustomisationSyncTestDependencies());
const second = new CustomisationSyncContext(createCustomisationSyncTestDependencies());
expect(first.catalogue).not.toBe(second.catalogue);
expect(first.enumerationActive).not.toBe(second.enumerationActive);
expect(first.migrationProgress).not.toBe(second.migrationProgress);
expect(first.manifests).not.toBe(second.manifests);
expect(get(first.manifests)).not.toBe(get(second.manifests));
});
it("exposes frozen semantic service and testing views without writable state", () => {
const context = new CustomisationSyncContext(createCustomisationSyncTestDependencies());
expect(Object.isFrozen(context.serviceHandlers)).toBe(true);
expect(Object.keys(context.serviceHandlers).sort()).toEqual(
[
"enableOptionalFeature",
"onBeforeReplicate",
"onDatabaseInitialised",
"onRealiseSetting",
"onResuming",
"processOptionalFileEvent",
"processVirtualDocument",
"suspendExtraSync",
].sort()
);
expect(Object.isFrozen(context.testing)).toBe(true);
expect(Object.keys(context.testing).sort()).toEqual(
[
"applyDataV2",
"configDir",
"createPluginDataExFileV2",
"createPluginDataFromV2",
"deleteConfigOnDatabase",
"scanAllConfigFiles",
"scanInternalFiles",
"storeCustomizationFiles",
].sort()
);
expect("catalogue" in context.testing).toBe(false);
expect("enumerationActive" in context.testing).toBe(false);
expect("manifests" in context.testing).toBe(false);
context.dispose();
});
});
@@ -0,0 +1,117 @@
import { beforeEach, describe, expect, it, vi } from "vitest";
import type { FilePath } from "@vrtmrz/livesync-commonlib/compat/common/types";
vi.mock("@/deps.ts", () => ({
diff_match_patch: class DiffMatchPatch {},
normalizePath: vi.fn((path: string) => path),
parseYaml: vi.fn(),
Platform: {},
}));
vi.mock("@/common/types.ts", () => ({
ICXHeader: "ix:",
PERIODIC_PLUGIN_SWEEP: 60,
}));
vi.mock("@/common/utils.ts", () => ({
cancelTask: vi.fn(),
EVEN: Symbol("even"),
isCustomisationSyncMetadata: vi.fn(),
isPluginMetadata: vi.fn(),
scheduleTask: vi.fn(),
}));
vi.mock("@/common/PeriodicProcessor.ts", () => ({
PeriodicProcessor: class PeriodicProcessor {},
}));
vi.mock("@/common/translation", () => ({
$msg: vi.fn((message: string) => message),
}));
vi.mock("@/common/obsidianCommunityPlugins.ts", () => ({
getObsidianCommunityPluginManager: vi.fn(),
}));
import { scheduleTask } from "@/common/utils.ts";
import { CustomisationSyncContext } from "./customisationSyncContext.ts";
import { createCustomisationSyncTestDependencies } from "./customisationSyncContext.unit.fixture.ts";
import { CustomisationSyncRecentEventDeduplicator } from "./customisationSyncRecentEventDeduplicator.ts";
const PATH = ".obsidian/plugins/example/data.json" as FilePath;
function createConfigSync(options: { ready?: boolean; suspended?: boolean; enabled?: boolean; owned?: boolean } = {}) {
const settings = {
usePluginSync: true,
usePluginSyncV2: true,
usePluginEtc: true,
pluginSyncExtendedSetting: {},
};
const ownsLocalFile = vi.fn(() => options.owned ?? true);
const statHidden = vi.fn(async () => ({ type: "file", mtime: 1 }));
const recentProcessedInternalFiles = new CustomisationSyncRecentEventDeduplicator();
const configSync = Object.create(CustomisationSyncContext.prototype) as CustomisationSyncContext;
Object.assign(configSync, {
dependencies: createCustomisationSyncTestDependencies({
getConfigDir: () => ".obsidian",
getSettings: () => settings as never,
storageAccess: { statHidden } as never,
ownsLocalFile,
}),
_isMainReady: vi.fn(() => options.ready ?? true),
_isMainSuspended: vi.fn(() => options.suspended ?? false),
isThisModuleEnabled: vi.fn(() => options.enabled ?? true),
pathOperations: {
isTargetPath: vi.fn((path: FilePath) => path == PATH),
filenameToUnifiedKey: vi.fn(() => "ix:device-a/PLUGIN_DATA/example.md"),
},
recentProcessedInternalFiles,
_log: vi.fn(),
});
return { configSync, ownsLocalFile, statHidden };
}
describe("Customisation Sync raw-event admission", () => {
beforeEach(() => {
vi.clearAllMocks();
});
it("schedules a recognised path granted by the composition owner", async () => {
const { configSync, ownsLocalFile } = createConfigSync();
await expect(configSync.serviceHandlers.processOptionalFileEvent(PATH)).resolves.toBe(true);
expect(ownsLocalFile).toHaveBeenCalledWith(PATH);
expect(scheduleTask).toHaveBeenCalledOnce();
});
it("rejects an event while the host is not ready", async () => {
const { configSync, statHidden } = createConfigSync({ ready: false });
await expect(configSync.serviceHandlers.processOptionalFileEvent(PATH)).resolves.toBe(false);
expect(statHidden).not.toHaveBeenCalled();
expect(scheduleTask).not.toHaveBeenCalled();
});
it("rejects a path outside the recognised Customisation Sync categories", async () => {
const { configSync, ownsLocalFile } = createConfigSync();
await expect(
configSync.serviceHandlers.processOptionalFileEvent(".obsidian/workspace" as FilePath)
).resolves.toBe(false);
expect(ownsLocalFile).not.toHaveBeenCalled();
expect(scheduleTask).not.toHaveBeenCalled();
});
it("rejects a recognised path assigned to another owner", async () => {
const { configSync, statHidden } = createConfigSync({ owned: false });
await expect(configSync.serviceHandlers.processOptionalFileEvent(PATH)).resolves.toBe(false);
expect(statHidden).not.toHaveBeenCalled();
expect(scheduleTask).not.toHaveBeenCalled();
});
it.each([
["suspended", { suspended: true }],
["disabled", { enabled: false }],
] as const)("rejects an event while %s", async (_label, options) => {
const { configSync } = createConfigSync(options);
await expect(configSync.serviceHandlers.processOptionalFileEvent(PATH)).resolves.toBe(false);
expect(scheduleTask).not.toHaveBeenCalled();
});
});
@@ -0,0 +1,37 @@
import { describe, expect, it, vi } from "vitest";
vi.mock("@/deps.ts", () => ({
diff_match_patch: class DiffMatchPatch {},
normalizePath: vi.fn((path: string) => path),
parseYaml: vi.fn(),
Platform: {},
}));
vi.mock("@/common/types.ts", () => ({
ICXHeader: "ix:",
PERIODIC_PLUGIN_SWEEP: 60,
}));
vi.mock("@/common/utils.ts", () => ({
cancelTask: vi.fn(),
EVEN: Symbol("even"),
isCustomisationSyncMetadata: vi.fn(),
isPluginMetadata: vi.fn(),
scheduleTask: vi.fn(),
}));
vi.mock("@/common/translation", () => ({
$msg: vi.fn((message: string) => message),
}));
import { CustomisationSyncContext } from "./customisationSyncContext.ts";
describe("Customisation Sync scan delegation", () => {
it("preserves the public scan argument and result through the focused owner", async () => {
const scanAllConfigFiles = vi.fn(async (_showMessage: boolean) => undefined);
const context = Object.create(CustomisationSyncContext.prototype) as CustomisationSyncContext;
Object.assign(context, { scanOperations: { scanAllConfigFiles } });
await expect(context.scanAllConfigFiles(true)).resolves.toBeUndefined();
expect(scanAllConfigFiles).toHaveBeenCalledOnce();
expect(scanAllConfigFiles).toHaveBeenCalledWith(true);
});
});
@@ -0,0 +1,566 @@
import type PouchDB from "pouchdb-core";
import { normalizePath } from "@/deps.ts";
import type {
EntryDoc,
LoadedEntry,
FilePathWithPrefix,
FilePath,
AnyEntry,
diff_result,
SYNC_MODE,
ObsidianLiveSyncSettings,
LOG_LEVEL,
} from "@vrtmrz/livesync-commonlib/compat/common/types";
import { LOG_LEVEL_NOTICE, LOG_LEVEL_VERBOSE, MODE_SELECTIVE } from "@vrtmrz/livesync-commonlib/compat/common/types";
import { ICXHeader, PERIODIC_PLUGIN_SWEEP } from "@/common/types.ts";
import { cancelTask, scheduleTask } from "@/common/utils.ts";
import { $msg } from "@/common/translation";
import type { OptionalSyncFeatureMode } from "@/features/optionalSyncFeatures.ts";
import type {
CustomisationSyncDialogView,
CustomisationSyncUIControl,
CustomisationSyncServiceHandlers,
CustomisationSyncTestingView,
IPluginDataExDisplay,
LoadedEntryPluginDataExFile,
} from "./customisationSyncView.ts";
import {
REPLICATION_PROGRESS_PRESENTATIONS,
USER_INITIATED_REPLICATION_AUTHORITY,
} from "@vrtmrz/livesync-commonlib/replication";
import type { LiveSyncLocalDB } from "@vrtmrz/livesync-commonlib/compat/pouchdb/LiveSyncLocalDB";
import type { StorageAccess } from "@vrtmrz/livesync-commonlib/compat/interfaces/StorageAccess";
import type { IPathService, IReplicationService } from "@vrtmrz/livesync-commonlib/compat/services/base/IService";
import type { LogFunction } from "@vrtmrz/livesync-commonlib/compat/services/lib/logUtils";
import type { OptionalFileSyncFileTreeDependencies } from "@/features/optionalFileSyncFileTree.ts";
import { PluginDataExDisplayV2 } from "./customisationSyncModel.ts";
import { ApplicationOperations, type ApplicationOperationsDependencies } from "./applicationOperations.ts";
import { CustomisationSyncRecentEventDeduplicator } from "./customisationSyncRecentEventDeduplicator.ts";
import { CatalogueOperations, type CatalogueOperationsDependencies } from "./catalogueOperations.ts";
import { SnapshotPersistence, type SnapshotPersistenceDependencies } from "./snapshotPersistence.ts";
import { SnapshotOperations } from "./snapshotOperations.ts";
import { ScanOperations, type ScanOperationsDependencies } from "./scanOperations.ts";
import {
createCustomisationSyncPathOperations,
type CustomisationSyncPathOperations,
} from "./customisationSyncPathOperations.ts";
export type { PluginDataEx, PluginDataExFile } from "./customisationSyncCodec.ts";
export type {
CustomisationSyncFileCategory,
CustomisationSyncServiceHandlers,
CustomisationSyncTestingView,
IPluginDataExDisplay,
PluginDataExDisplay,
} from "./customisationSyncView.ts";
export { PluginDataExDisplayV2 } from "./customisationSyncModel.ts";
const UPDATED_CONFIGURATION_NOTICE_KEY = "config-sync:updated-configuration";
type CustomisationSyncSettings = Pick<
ObsidianLiveSyncSettings,
| "usePluginSync"
| "usePluginSyncV2"
| "usePluginEtc"
| "pluginSyncExtendedSetting"
| "autoSweepPlugins"
| "autoSweepPluginsPeriodic"
| "watchInternalFileChanges"
| "notifyPluginOrSettingUpdated"
>;
type CustomisationSyncDatabase = Pick<
LiveSyncLocalDB,
"allDocsRaw" | "findEntries" | "getDBEntry" | "getDBEntryFromMeta" | "getDBEntryMeta" | "putDBEntry" | "putRaw"
>;
type CustomisationSyncStorage = Pick<
StorageAccess,
"ensureDir" | "readHiddenFileBinary" | "readHiddenFileText" | "statHidden" | "writeHiddenFileAuto"
>;
export type CustomisationSyncPeriodicProcessor = {
enable(interval: number): void;
disable(): void;
};
export type CustomisationSyncContextDependencies = OptionalFileSyncFileTreeDependencies & {
getSettings(): CustomisationSyncSettings;
getLocalDatabase(): CustomisationSyncDatabase;
storageAccess: CustomisationSyncStorage;
path: Pick<IPathService, "getPath" | "isMarkedAsSameChanges" | "markChangesAreSame" | "path2id">;
log: LogFunction;
getConfigDir(): string;
getDeviceAndVaultName(): string;
setDeviceAndVaultName(name: string): void;
saveSettingData(): Promise<void>;
applySettings(partial: Partial<ObsidianLiveSyncSettings>, saveImmediately?: boolean): Promise<void>;
replicateUserInitiated: IReplicationService["replicateUserInitiated"];
askString(title: string, key: string, placeholder: string): Promise<string | false>;
isReady(): boolean;
isSuspended(): boolean;
askRestart(): void;
createPeriodicProcessor(process: () => Promise<unknown>): CustomisationSyncPeriodicProcessor;
resolveJsonConflict(
path: FilePath,
files: [LoadedEntryPluginDataExFile, LoadedEntryPluginDataExFile],
remoteName: string,
apply: (content: string) => Promise<boolean>
): Promise<boolean>;
selectTextFile(path: FilePath, diffResult: diff_result, remoteName: string): Promise<"A" | "B" | false>;
reloadPlugin(configDir: string, pluginName: string): Promise<void>;
getFallbackDeviceName(): string;
showConfigurationNotice(openDialog: () => void): void;
hideConfigurationNotice(): void;
getUIControl(): CustomisationSyncUIControl | undefined;
ownsLocalFile(path: FilePath): boolean;
ownsLocalDocument(path: FilePathWithPrefix): boolean;
publishScanCount(count: number): void;
};
export class CustomisationSyncContext implements CustomisationSyncDialogView {
private readonly dependencies: CustomisationSyncContextDependencies;
private readonly pathOperations: CustomisationSyncPathOperations;
private readonly snapshotPersistence: SnapshotPersistence;
private readonly snapshotOperations: SnapshotOperations;
private readonly catalogueOperations: CatalogueOperations;
private readonly applicationOperations: ApplicationOperations;
private readonly scanOperations: ScanOperations;
private readonly recentProcessedInternalFiles = new CustomisationSyncRecentEventDeduplicator();
private serviceHandlersView: CustomisationSyncServiceHandlers | undefined;
private testingView: CustomisationSyncTestingView | undefined;
private readonly periodicPluginSweepProcessor: CustomisationSyncPeriodicProcessor;
constructor(dependencies: CustomisationSyncContextDependencies) {
this.dependencies = dependencies;
this.pathOperations = createCustomisationSyncPathOperations({
getConfigDir: () => dependencies.getConfigDir(),
getUseV2: () => dependencies.getSettings().usePluginSyncV2,
getUsePluginEtc: () => dependencies.getSettings().usePluginEtc,
getDeviceAndVaultName: () => dependencies.getDeviceAndVaultName(),
});
const snapshotPersistenceDependencies: SnapshotPersistenceDependencies = {
getLocalDatabase: () => dependencies.getLocalDatabase(),
storageAccess: dependencies.storageAccess,
path: {
...this.pathOperations,
path2id: (filename, prefix) => dependencies.path.path2id(filename, prefix),
isMarkedAsSameChanges: (file, mtimes) => dependencies.path.isMarkedAsSameChanges(file, mtimes),
markChangesAreSame: (file, newMtime, oldMtime) =>
dependencies.path.markChangesAreSame(file, newMtime, oldMtime),
},
log: (message, level, key) => dependencies.log(message, level, key),
getConfigDir: () => dependencies.getConfigDir(),
};
this.snapshotPersistence = new SnapshotPersistence(snapshotPersistenceDependencies);
this.catalogueOperations = new CatalogueOperations({
getSettings: () => {
const settings = dependencies.getSettings();
return {
usePluginSync: settings.usePluginSync,
usePluginSyncV2: settings.usePluginSyncV2,
};
},
getLocalDatabase: () => dependencies.getLocalDatabase(),
path: {
getPath: (entry) => dependencies.path.getPath(entry),
path2id: (filename, prefix) => dependencies.path.path2id(filename, prefix),
},
log: (message, level, key) => dependencies.log(message, level, key),
snapshotPersistence: this.snapshotPersistence,
publishScanCount: (count) => dependencies.publishScanCount(count),
} satisfies CatalogueOperationsDependencies);
this.snapshotOperations = new SnapshotOperations({
getSettings: () => ({ usePluginSyncV2: dependencies.getSettings().usePluginSyncV2 }),
getDeviceAndVaultName: () => dependencies.getDeviceAndVaultName(),
log: (message, level, key) => dependencies.log(message, level, key),
snapshotPersistence: this.snapshotPersistence,
catalogueOperations: this.catalogueOperations,
});
const applicationOperationsDependencies: ApplicationOperationsDependencies = {
getLocalDatabase: () => ({ getDBEntry: (path) => dependencies.getLocalDatabase().getDBEntry(path) }),
storageAccess: dependencies.storageAccess,
path: {
filenameToUnifiedKey: (path, termOverride) =>
this.pathOperations.filenameToUnifiedKey(path, termOverride),
},
log: (message, level, key) => dependencies.log(message, level, key),
getConfigDir: () => dependencies.getConfigDir(),
getDeviceAndVaultName: () => dependencies.getDeviceAndVaultName(),
resolveJsonConflict: (path, files, remoteName, apply) =>
dependencies.resolveJsonConflict(path, files, remoteName, apply),
selectTextFile: (path, diffResult, remoteName) => dependencies.selectTextFile(path, diffResult, remoteName),
reloadPlugin: (configDir, pluginName) => dependencies.reloadPlugin(configDir, pluginName),
askRestart: () => dependencies.askRestart(),
snapshotOperations: this.snapshotOperations,
catalogueOperations: this.catalogueOperations,
};
this.applicationOperations = new ApplicationOperations(applicationOperationsDependencies);
this.scanOperations = new ScanOperations({
listFiles: async (path) => await dependencies.listFiles(path),
getSettings: () => ({ usePluginSyncV2: dependencies.getSettings().usePluginSyncV2 }),
getLocalDatabase: () => dependencies.getLocalDatabase(),
path: {
getPath: (entry) => dependencies.path.getPath(entry),
isTargetPath: (path) => this.pathOperations.isTargetPath(path),
filenameToUnifiedKey: (path, termOverride) =>
this.pathOperations.filenameToUnifiedKey(path, termOverride),
filenameWithUnifiedKey: (path, termOverride) =>
this.pathOperations.filenameWithUnifiedKey(path, termOverride),
unifiedKeyPrefixOfTerminal: (termOverride) =>
this.pathOperations.unifiedKeyPrefixOfTerminal(termOverride),
},
log: (message, level, key) => dependencies.log(message, level, key),
getConfigDir: () => dependencies.getConfigDir(),
getDeviceAndVaultName: () => dependencies.getDeviceAndVaultName(),
ownsLocalFile: (path) => dependencies.ownsLocalFile(path),
ownsLocalDocument: (path) => dependencies.ownsLocalDocument(path),
snapshotOperations: this.snapshotOperations,
catalogueOperations: this.catalogueOperations,
} satisfies ScanOperationsDependencies);
this.periodicPluginSweepProcessor = dependencies.createPeriodicProcessor(
async () => await this.scanAllConfigFiles(false)
);
}
get catalogue() {
return this.catalogueOperations.catalogue;
}
get enumerationActive() {
return this.catalogueOperations.enumerationActive;
}
get migrationProgress() {
return this.catalogueOperations.migrationProgress;
}
get manifests() {
return this.catalogueOperations.manifests;
}
/**
* Semantic callbacks for registration by the optional-file composition
* feature. The returned object is immutable, and each callback retains its
* context without requiring callers to bind a concrete implementation.
*/
get serviceHandlers(): CustomisationSyncServiceHandlers {
if (!this.serviceHandlersView) {
this.serviceHandlersView = Object.freeze({
processOptionalFileEvent: (path: FilePath) => this.processOptionalFileEvent(path),
processVirtualDocument: (docs: PouchDB.Core.ExistingDocument<EntryDoc>) =>
this.processVirtualDocument(docs),
onRealiseSetting: () => this.realiseSettingSyncMode(),
onResuming: () => this.onResumeProcess(),
onBeforeReplicate: (showMessage: boolean) => this.beforeReplicate(showMessage),
onDatabaseInitialised: (showNotice: boolean) => this.onDatabaseInitialised(showNotice),
suspendExtraSync: () => this.suspendExtraSync(),
enableOptionalFeature: (mode: OptionalSyncFeatureMode) => this.enableOptionalFeature(mode),
});
}
return this.serviceHandlersView;
}
/**
* Narrow internal surface used by maintained real-Obsidian contract tests.
* It intentionally omits the context, queues, and writable stores.
*/
get testing(): CustomisationSyncTestingView {
if (!this.testingView) {
this.testingView = Object.freeze({
configDir: this.configDir,
scanInternalFiles: async () => await this.scanOperations.scanInternalFiles(),
scanAllConfigFiles: async (showMessage: boolean) => await this.scanAllConfigFiles(showMessage),
storeCustomizationFiles: async (path: FilePath, termOverride?: string) =>
await this.snapshotOperations.storeCustomizationFiles(path, termOverride),
deleteConfigOnDatabase: async (path: FilePathWithPrefix, forceWrite?: boolean) =>
await this.snapshotOperations.deleteConfigOnDatabase(path, forceWrite),
createPluginDataFromV2: (path: FilePathWithPrefix) =>
this.catalogueOperations.createPluginDataFromV2(path),
createPluginDataExFileV2: async (path: FilePathWithPrefix, loaded?: LoadedEntry) =>
await this.catalogueOperations.createPluginDataExFileV2(path, loaded),
applyDataV2: async (data: PluginDataExDisplayV2, content?: string) =>
await this.applicationOperations.applyDataV2(data, content),
});
}
return this.testingView;
}
private get configDir() {
return this.dependencies.getConfigDir();
}
private get settings() {
return this.dependencies.getSettings();
}
private get storageAccess() {
return this.dependencies.storageAccess;
}
private getPath(entry: AnyEntry): FilePathWithPrefix {
return this.dependencies.path.getPath(entry);
}
private _isMainReady() {
return this.dependencies.isReady();
}
private _isMainSuspended() {
return this.dependencies.isSuspended();
}
private _log(message: unknown, level?: LOG_LEVEL, key?: string) {
this.dependencies.log(message, level, key);
}
private get useSyncPluginEtc() {
return this.settings.usePluginEtc;
}
private isThisModuleEnabled() {
return this.settings.usePluginSync;
}
isEnabled(): boolean {
return this.isThisModuleEnabled();
}
getDeviceAndVaultName(): string {
return this.dependencies.getDeviceAndVaultName();
}
getConfiguredModes() {
return Object.values(this.settings.pluginSyncExtendedSetting).map((entry) => ({
...entry,
files: [...entry.files],
}));
}
isPluginEtcEnabled(): boolean {
return this.useSyncPluginEtc;
}
updateConfiguredMode(key: string, mode: SYNC_MODE, files: string[]): void {
if (mode == MODE_SELECTIVE) {
delete this.settings.pluginSyncExtendedSetting[key];
} else {
this.settings.pluginSyncExtendedSetting[key] = {
key,
mode,
files: [...files],
};
}
void this.dependencies.saveSettingData();
}
getConfiguredTargetFiles(key: string): string[] {
const configDir = normalizePath(this.configDir);
return (this.settings.pluginSyncExtendedSetting[key]?.files ?? []).map((path) => `${configDir}/${path}`);
}
async synchronise(): Promise<void> {
await this.dependencies.replicateUserInitiated({
trigger: "manual",
progressPresentation: REPLICATION_PROGRESS_PRESENTATIONS.NOTICE,
interaction: USER_INITIATED_REPLICATION_AUTHORITY,
});
}
askString(title: string, key: string, placeholder: string): Promise<string | false> {
return this.dependencies.askString(title, key, placeholder);
}
async compareFileUsingDisplayData(
dataA: IPluginDataExDisplay,
dataB: IPluginDataExDisplay,
filename: string
): Promise<boolean> {
return await this.applicationOperations.compareFileUsingDisplayData(dataA, dataB, filename);
}
async duplicateData(data: IPluginDataExDisplay, deviceName: string): Promise<void> {
await this.applicationOperations.duplicateData(data, deviceName);
}
dispose() {
cancelTask(UPDATED_CONFIGURATION_NOTICE_KEY);
this.periodicPluginSweepProcessor?.disable();
this.catalogueOperations.dispose();
this.dependencies.hideConfigurationNotice();
}
private async onDatabaseInitialised(showNotice: boolean) {
if (!this.isThisModuleEnabled()) return true;
try {
this._log("Scanning customizations...");
await this.scanAllConfigFiles(showNotice);
this._log("Scanning customizations : done");
} catch (ex) {
this._log("Scanning customizations : failed");
this._log(ex, LOG_LEVEL_VERBOSE);
}
return true;
}
private async beforeReplicate(showNotice: boolean) {
if (!this.isThisModuleEnabled()) return true;
if (this.settings.autoSweepPlugins) {
await this.scanAllConfigFiles(showNotice);
return true;
}
return true;
}
private async onResumeProcess(): Promise<boolean> {
if (!this.isThisModuleEnabled()) return true;
if (this._isMainSuspended()) {
return true;
}
if (this.settings.autoSweepPlugins) {
await this.scanAllConfigFiles(false);
}
this.periodicPluginSweepProcessor.enable(
this.settings.autoSweepPluginsPeriodic && !this.settings.watchInternalFileChanges
? PERIODIC_PLUGIN_SWEEP * 1000
: 0
);
return true;
}
async reloadPluginList(showMessage: boolean) {
await this.catalogueOperations.reloadPluginList(showMessage);
}
async updatePluginList(showMessage: boolean, updatedDocumentPath?: FilePathWithPrefix): Promise<void> {
await this.catalogueOperations.updatePluginList(showMessage, updatedDocumentPath);
}
async compareUsingDisplayData(dataA: IPluginDataExDisplay, dataB: IPluginDataExDisplay, compareEach = false) {
return await this.applicationOperations.compareUsingDisplayData(dataA, dataB, compareEach);
}
async applyData(data: IPluginDataExDisplay, content?: string): Promise<boolean> {
return await this.applicationOperations.applyData(data, content);
}
async deleteData(data: IPluginDataExDisplay): Promise<boolean> {
return await this.applicationOperations.deleteData(data);
}
private async processVirtualDocument(docs: PouchDB.Core.ExistingDocument<EntryDoc>) {
if (!docs._id.startsWith(ICXHeader)) return false;
if (this.isThisModuleEnabled()) {
await this.updatePluginList(
false,
(docs as AnyEntry).path ? (docs as AnyEntry).path : this.getPath(docs as AnyEntry)
);
}
if (this.isThisModuleEnabled() && this.settings.notifyPluginOrSettingUpdated) {
if (!this.dependencies.getUIControl()?.isOpen()) {
scheduleTask(UPDATED_CONFIGURATION_NOTICE_KEY, 1000, () => {
this.dependencies.showConfigurationNotice(() => this.dependencies.getUIControl()?.open());
});
}
}
return true;
}
private async realiseSettingSyncMode(): Promise<boolean> {
this.periodicPluginSweepProcessor?.disable();
// Compatibility question: this inherited callback checks the method
// reference rather than invoking it, then proceeds only while the host is
// suspended. Preserve both gates until their intended lifecycle semantics
// are verified and corrected under a separate behavioural test.
if (!this._isMainReady) return true;
if (!this._isMainSuspended()) return true;
if (!this.isThisModuleEnabled()) return true;
if (this.settings.autoSweepPlugins) {
await this.scanAllConfigFiles(false);
}
this.periodicPluginSweepProcessor.enable(
this.settings.autoSweepPluginsPeriodic && !this.settings.watchInternalFileChanges
? PERIODIC_PLUGIN_SWEEP * 1000
: 0
);
return true;
}
private async processOptionalFileEvent(path: FilePath): Promise<boolean> {
return await this.watchVaultRawEventsAsync(path);
}
private async watchVaultRawEventsAsync(path: FilePath) {
if (!this._isMainReady()) return false;
if (this._isMainSuspended()) return false;
if (!this.isThisModuleEnabled()) return false;
if (!this.pathOperations.isTargetPath(path)) return false;
if (!this.dependencies.ownsLocalFile(path)) return false;
const stat = await this.storageAccess.statHidden(path);
// Make sure that target is a file.
if (stat && stat.type != "file") return false;
// this._log(`Customization file detected: ${path}`, LOG_LEVEL_VERBOSE);
const storageMTime = ~~(((stat && stat.mtime) || 0) / 1000);
const key = `${path}-${storageMTime}`;
if (!this.recentProcessedInternalFiles.admit(key)) {
// If recently processed, it may caused by self.
// return true to prevent pass the event to the next.
return true;
}
// To prevent saving half-collected file sets.
const keySchedule = this.pathOperations.filenameToUnifiedKey(path);
scheduleTask(keySchedule, 100, async () => {
await this.snapshotOperations.storeCustomizationFiles(path);
});
// Okay, it may handled after 100ms.
// This was my own job.
return true;
}
async scanAllConfigFiles(showMessage: boolean): Promise<void> {
await this.scanOperations.scanAllConfigFiles(showMessage);
}
private suspendExtraSync(): Promise<boolean> {
if (this.settings.usePluginSync || this.settings.autoSweepPlugins) {
this._log(
"Customisation sync have been temporarily disabled. Please enable them after the fetching, if you need them.",
LOG_LEVEL_NOTICE
);
this.settings.usePluginSync = false;
this.settings.autoSweepPlugins = false;
}
return Promise.resolve(true);
}
private async enableOptionalFeature(mode: OptionalSyncFeatureMode): Promise<boolean> {
await this.configureCustomisationSync(mode);
return true;
}
private async configureCustomisationSync(mode: OptionalSyncFeatureMode) {
if (mode == "DISABLE") {
await this.dependencies.applySettings(
{
usePluginSync: false,
},
true
);
return;
}
if (mode == "CUSTOMIZE") {
if (!this.dependencies.getDeviceAndVaultName()) {
let name = await this.dependencies.askString(
$msg("Device name"),
$msg("Please set this device name"),
`desktop`
);
if (!name) {
name = this.dependencies.getFallbackDeviceName();
}
this.dependencies.setDeviceAndVaultName(name);
}
await this.dependencies.applySettings(
{
usePluginSync: true,
useAdvancedMode: true,
},
true
);
await this.scanAllConfigFiles(true);
}
}
}
@@ -0,0 +1,49 @@
import type { CustomisationSyncContextDependencies } from "./customisationSyncContext.ts";
/** Minimal inert dependency set for focused Customisation Sync unit tests. */
export function createCustomisationSyncTestDependencies(
overrides: Partial<CustomisationSyncContextDependencies> = {}
): CustomisationSyncContextDependencies {
const defaults = {
getSettings: () => ({
usePluginSync: true,
usePluginSyncV2: true,
usePluginEtc: true,
pluginSyncExtendedSetting: {},
autoSweepPlugins: false,
autoSweepPluginsPeriodic: false,
watchInternalFileChanges: false,
notifyPluginOrSettingUpdated: false,
}),
getLocalDatabase: () => ({}),
storageAccess: {},
path: {},
log: () => undefined,
getConfigDir: () => ".config-dir",
getDeviceAndVaultName: () => "device-a",
setDeviceAndVaultName: () => undefined,
saveSettingData: () => Promise.resolve(),
applySettings: () => Promise.resolve(),
replicateUserInitiated: () => Promise.resolve(),
askString: () => Promise.resolve(false),
isReady: () => true,
isSuspended: () => false,
askRestart: () => undefined,
createPeriodicProcessor: () => ({
enable: () => undefined,
disable: () => undefined,
}),
listFiles: () => Promise.resolve({ files: [], folders: [] }),
resolveJsonConflict: () => Promise.resolve(false),
selectTextFile: () => Promise.resolve(false),
reloadPlugin: () => Promise.resolve(),
getFallbackDeviceName: () => "desktop-test",
showConfigurationNotice: () => undefined,
hideConfigurationNotice: () => undefined,
getUIControl: () => undefined,
ownsLocalFile: () => true,
ownsLocalDocument: () => true,
publishScanCount: () => undefined,
} as unknown as CustomisationSyncContextDependencies;
return { ...defaults, ...overrides };
}
@@ -0,0 +1,148 @@
import { describe, expect, it, vi } from "vitest";
vi.mock("@/deps.ts", () => ({
diff_match_patch: class DiffMatchPatch {},
normalizePath: vi.fn((path: string) => path),
parseYaml: vi.fn(),
Platform: {},
}));
vi.mock("@/common/types.ts", () => ({
ICXHeader: "ix:",
PERIODIC_PLUGIN_SWEEP: 60,
}));
vi.mock("@/common/utils.ts", () => ({
cancelTask: vi.fn(),
EVEN: Symbol("even"),
isCustomisationSyncMetadata: vi.fn(),
isPluginMetadata: vi.fn(),
scheduleTask: vi.fn(),
}));
vi.mock("@/common/PeriodicProcessor.ts", () => ({
PeriodicProcessor: class PeriodicProcessor {},
}));
vi.mock("@/common/translation", () => ({
$msg: vi.fn((message: string) => message),
}));
vi.mock("@/common/obsidianCommunityPlugins.ts", () => ({
getObsidianCommunityPluginManager: vi.fn(),
}));
vi.mock("octagonal-wheels/concurrency/processor", () => ({
QueueProcessor: class QueueProcessor {
clearQueue = vi.fn();
enqueue = vi.fn();
terminate = vi.fn();
startPipeline() {
return this;
}
},
}));
import {
LOG_LEVEL_VERBOSE,
type FilePathWithPrefix,
type LoadedEntry,
} from "@vrtmrz/livesync-commonlib/compat/common/types";
import { digestHash } from "@vrtmrz/livesync-commonlib/compat/string_and_binary/hash";
import { createCustomisationSyncCodec } from "./customisationSyncCodec.ts";
import { CatalogueOperations } from "./catalogueOperations.ts";
import { CustomisationSyncContext } from "./customisationSyncContext.ts";
import { createCustomisationSyncTestDependencies } from "./customisationSyncContext.unit.fixture.ts";
const path = "ix:device-a/PLUGIN_MAIN/example%manifest.json" as FilePathWithPrefix;
const confKey = "device-a/plugins/example";
const codec = createCustomisationSyncCodec({
digestHash,
parseYaml: () => undefined,
});
function loadedManifest(manifestSource: string, mtime: number): LoadedEntry {
const data = `${codec.dummyHead}${codec.dummyEnd}${btoa(manifestSource)}`;
return {
_id: "entry-id",
_rev: "1-a",
path,
type: "plain",
datatype: "plain",
data,
ctime: 10,
mtime,
size: data.length,
children: [],
eden: {},
} as unknown as LoadedEntry;
}
function createContext() {
const log = vi.fn();
const snapshotPersistence = {
deleteConfigOnDatabase: vi.fn(async () => ({ value: true, status: "missing" as const, refreshes: [] })),
};
const catalogueOperations = new CatalogueOperations({
...createCustomisationSyncTestDependencies({
log,
getLocalDatabase: () => ({ getDBEntry: async () => false }) as never,
}),
snapshotPersistence,
publishScanCount: vi.fn(),
});
const pluginManifests = catalogueOperations.manifestLookup;
const setManifests = vi.spyOn(catalogueOperations.manifests, "set");
const setCatalogue = vi.spyOn(catalogueOperations.catalogue, "set");
const context = Object.create(CustomisationSyncContext.prototype) as CustomisationSyncContext;
Object.assign(context, {
dependencies: createCustomisationSyncTestDependencies({
log,
getLocalDatabase: () => ({ getDBEntry: async () => false }) as never,
}),
catalogueOperations,
});
return {
catalogueOperations,
context,
log,
pluginManifests,
setCatalogue,
setManifests,
};
}
describe("compatibility: Customisation Sync V2 manifest state", () => {
it("keeps the first parsed manifest when a later file has a different mtime", async () => {
const { catalogueOperations, context, pluginManifests, setManifests } = createContext();
await context.testing.createPluginDataExFileV2(
path,
loadedManifest(JSON.stringify({ id: "example", name: "First", version: "1.0.0" }), 20)
);
await context.testing.createPluginDataExFileV2(
path,
loadedManifest(JSON.stringify({ id: "example", name: "Second", version: "2.0.0" }), 30)
);
expect(pluginManifests.get(confKey)).toMatchObject({ name: "First", version: "1.0.0" });
expect(setManifests).toHaveBeenCalledOnce();
catalogueOperations.dispose();
});
it("records a failed manifest mtime and does not retry the same revision", async () => {
const { catalogueOperations, context, log, pluginManifests, setCatalogue } = createContext();
const invalid = loadedManifest("{invalid", 20);
await expect(context.testing.createPluginDataExFileV2(path, invalid)).resolves.toMatchObject({
filename: "plugins/example/manifest.json",
});
await context.testing.createPluginDataExFileV2(path, invalid);
expect(pluginManifests.has(confKey)).toBe(false);
expect(log).toHaveBeenCalledTimes(2);
expect(log).toHaveBeenNthCalledWith(
1,
`The file ${path} seems to manifest, but could not be decoded as JSON`,
LOG_LEVEL_VERBOSE,
undefined
);
expect(log).toHaveBeenNthCalledWith(2, expect.any(SyntaxError), LOG_LEVEL_VERBOSE, undefined);
expect(setCatalogue).toHaveBeenCalledOnce();
catalogueOperations.dispose();
});
});
@@ -0,0 +1,138 @@
import { describe, expect, it, vi } from "vitest";
import {
type FilePathWithPrefix,
MODE_AUTOMATIC,
MODE_SELECTIVE,
type PluginSyncSettingEntry,
} from "@vrtmrz/livesync-commonlib/compat/common/types";
import {
REPLICATION_PROGRESS_PRESENTATIONS,
USER_INITIATED_REPLICATION_AUTHORITY,
} from "@vrtmrz/livesync-commonlib/replication";
vi.mock("@/deps.ts", () => ({
diff_match_patch: class DiffMatchPatch {},
normalizePath: vi.fn((path: string) => path),
parseYaml: vi.fn(),
Platform: {},
}));
vi.mock("@/common/types.ts", () => ({
ICXHeader: "ix:",
PERIODIC_PLUGIN_SWEEP: 60,
}));
vi.mock("@/common/utils.ts", () => ({
cancelTask: vi.fn(),
EVEN: Symbol("even"),
isCustomisationSyncMetadata: vi.fn(),
isPluginMetadata: vi.fn(),
scheduleTask: vi.fn(),
}));
vi.mock("@/common/PeriodicProcessor.ts", () => ({
PeriodicProcessor: class PeriodicProcessor {},
}));
vi.mock("@/common/translation", () => ({
$msg: vi.fn((message: string) => message),
}));
vi.mock("@/common/obsidianCommunityPlugins.ts", () => ({
getObsidianCommunityPluginManager: vi.fn(),
}));
import { CustomisationSyncContext } from "./customisationSyncContext.ts";
import { createCustomisationSyncTestDependencies } from "./customisationSyncContext.unit.fixture.ts";
import type { IPluginDataExDisplay } from "./customisationSyncView.ts";
function createConfigSync() {
const saveSettingData = vi.fn(async () => undefined);
const replicateUserInitiated = vi.fn(async () => undefined);
const askString = vi.fn(async () => "device-b" as string | false);
const pluginSyncExtendedSetting: Record<string, PluginSyncSettingEntry> = {
"PLUGIN_DATA/example": {
key: "PLUGIN_DATA/example",
mode: MODE_AUTOMATIC,
files: ["plugins/example/data.json"],
},
};
const settings = {
usePluginSync: true,
usePluginSyncV2: true,
usePluginEtc: true,
pluginSyncExtendedSetting,
};
const configSync = Object.create(CustomisationSyncContext.prototype) as CustomisationSyncContext;
Object.assign(configSync, {
dependencies: createCustomisationSyncTestDependencies({
getConfigDir: () => ".obsidian",
getSettings: () => settings as never,
saveSettingData,
replicateUserInitiated: replicateUserInitiated as never,
askString,
}),
});
return { askString, configSync, replicateUserInitiated, saveSettingData, settings };
}
const display = {
documentPath: "ix:device-a/PLUGIN_DATA/example%data.json" as FilePathWithPrefix,
category: "PLUGIN_DATA",
name: "example",
term: "device-a",
files: [
{ filename: "data.json", data: ["a"], mtime: 1, size: 1 },
{ filename: "other.json", data: ["b"], mtime: 2, size: 1 },
],
mtime: 2,
} satisfies IPluginDataExDisplay;
describe("CustomisationSyncContext dialogue view", () => {
it("projects and updates Customisation Sync modes without exposing mutable settings", () => {
const { configSync, saveSettingData, settings } = createConfigSync();
const projected = configSync.getConfiguredModes();
projected[0].files.push("changed-in-view");
expect(settings.pluginSyncExtendedSetting["PLUGIN_DATA/example"].files).toEqual(["plugins/example/data.json"]);
expect(configSync.getConfiguredTargetFiles("PLUGIN_DATA/example")).toEqual([
".obsidian/plugins/example/data.json",
]);
configSync.updateConfiguredMode("PLUGIN_MAIN/example", MODE_AUTOMATIC, ["plugins/example/main.js"]);
expect(settings.pluginSyncExtendedSetting["PLUGIN_MAIN/example"]).toEqual({
key: "PLUGIN_MAIN/example",
mode: MODE_AUTOMATIC,
files: ["plugins/example/main.js"],
});
configSync.updateConfiguredMode("PLUGIN_MAIN/example", MODE_SELECTIVE, []);
expect(settings.pluginSyncExtendedSetting).not.toHaveProperty("PLUGIN_MAIN/example");
expect(saveSettingData).toHaveBeenCalledTimes(2);
});
it("routes host operations through the focused view", async () => {
const { askString, configSync, replicateUserInitiated } = createConfigSync();
await configSync.synchronise();
await expect(configSync.askString("Duplicate", "device name", "")).resolves.toBe("device-b");
expect(replicateUserInitiated).toHaveBeenCalledWith({
trigger: "manual",
progressPresentation: REPLICATION_PROGRESS_PRESENTATIONS.NOTICE,
interaction: USER_INITIATED_REPLICATION_AUTHORITY,
});
expect(askString).toHaveBeenCalledWith("Duplicate", "device name", "");
});
it("delegates file-level comparison and duplication to the application owner", async () => {
const { configSync } = createConfigSync();
const compareFileUsingDisplayData = vi.fn(async () => true);
const duplicateData = vi.fn(async () => undefined);
Object.assign(configSync, {
applicationOperations: { compareFileUsingDisplayData, duplicateData },
});
await expect(configSync.compareFileUsingDisplayData(display, display, "data.json")).resolves.toBe(true);
expect(compareFileUsingDisplayData).toHaveBeenCalledWith(display, display, "data.json");
await configSync.duplicateData(display, "device-b");
expect(duplicateData).toHaveBeenCalledWith(display, "device-b");
});
});
@@ -0,0 +1,67 @@
import type { PluginManifest } from "@/deps.ts";
import type { FilePathWithPrefix } from "@vrtmrz/livesync-commonlib/compat/common/types";
import { isDocContentSame } from "@vrtmrz/livesync-commonlib/compat/common/utils";
import { getCustomisationSyncCategoryFolder } from "./customisationSyncPaths.ts";
import type { IPluginDataExDisplay, LoadedEntryPluginDataExFile } from "./customisationSyncView.ts";
export class PluginDataExDisplayV2 {
documentPath: FilePathWithPrefix;
category: string;
term: string;
files: LoadedEntryPluginDataExFile[];
name: string;
confKey: string;
_displayName: string | undefined;
_version: string | undefined;
constructor(
data: IPluginDataExDisplay,
private readonly manifestLookup: ReadonlyMap<string, PluginManifest>
) {
this.documentPath = `${data.documentPath}` as FilePathWithPrefix;
this.category = `${data.category}`;
this.name = `${data.name}`;
this.term = `${data.term}`;
this.files = [...(data.files as LoadedEntryPluginDataExFile[])];
this.confKey = `${getCustomisationSyncCategoryFolder(this.category, this.term)}${this.name}`;
this.applyLoadedManifest();
}
async setFile(file: LoadedEntryPluginDataExFile): Promise<void> {
const old = this.files.find((entry) => entry.filename == file.filename);
if (old) {
if (old.mtime == file.mtime && (await isDocContentSame(old.data, file.data))) return;
this.files = this.files.filter((entry) => entry.filename != file.filename);
}
this.files.push(file);
if (file.filename == "manifest.json") {
this.applyLoadedManifest();
}
}
deleteFile(filename: string): void {
this.files = this.files.filter((entry) => entry.filename != filename);
}
applyLoadedManifest(): void {
const manifest = this.manifestLookup.get(this.confKey);
if (manifest) {
this._displayName = manifest.name;
if (this.category == "PLUGIN_MAIN" || this.category == "THEME") {
this._version = manifest.version;
}
}
}
get displayName(): string {
return this._displayName || this.name;
}
get version(): string | undefined {
return this._version;
}
get mtime(): number {
return ~~this.files.reduce((sum, file) => sum + file.mtime, 0) / this.files.length;
}
}
@@ -0,0 +1,58 @@
import { describe, expect, it } from "vitest";
import type { PluginManifest } from "@/deps.ts";
import type { FilePathWithPrefix } from "@vrtmrz/livesync-commonlib/compat/common/types";
import { PluginDataExDisplayV2 } from "./customisationSyncModel.ts";
import type { IPluginDataExDisplay, LoadedEntryPluginDataExFile } from "./customisationSyncView.ts";
function file(filename: string, mtime: number, data: string[]): LoadedEntryPluginDataExFile {
return { filename, mtime, data, size: data.join("").length } as LoadedEntryPluginDataExFile;
}
function display(files: LoadedEntryPluginDataExFile[] = []): IPluginDataExDisplay {
return {
documentPath: "ix:device-a/PLUGIN_MAIN/example%main.js" as FilePathWithPrefix,
category: "PLUGIN_MAIN",
name: "example",
term: "device-a",
files,
mtime: 0,
};
}
describe("PluginDataExDisplayV2", () => {
it("projects manifest identity and file modification time", () => {
const manifests = new Map([
["device-a/plugins/example", { name: "Example plug-in", version: "1.2.3" } as PluginManifest],
]);
const model = new PluginDataExDisplayV2(
display([file("main.js", 10, ["main"]), file("data.json", 20, ["data"])]),
manifests
);
expect(model.confKey).toBe("device-a/plugins/example");
expect(model.displayName).toBe("Example plug-in");
expect(model.version).toBe("1.2.3");
expect(model.mtime).toBe(15);
});
it("retains an unchanged file and replaces changed content", async () => {
const original = file("main.js", 10, ["same"]);
const model = new PluginDataExDisplayV2(display([original]), new Map());
await model.setFile(file("main.js", 10, ["same"]));
expect(model.files[0]).toBe(original);
const changed = file("main.js", 10, ["changed"]);
await model.setFile(changed);
expect(model.files).toEqual([changed]);
});
it("deletes only the named file", () => {
const retained = file("styles.css", 20, ["css"]);
const model = new PluginDataExDisplayV2(display([file("main.js", 10, ["main"]), retained]), new Map());
model.deleteFile("main.js");
expect(model.files).toEqual([retained]);
});
});
@@ -0,0 +1,59 @@
import type { FilePathWithPrefix } from "@vrtmrz/livesync-commonlib/compat/common/types";
import {
createCustomisationSyncDevicePrefix,
createCustomisationSyncV1DocumentPath,
createCustomisationSyncV2DocumentPath,
getCustomisationSyncFileCategory,
isCustomisationSyncTargetPath,
type CustomisationSyncFileCategory,
type CustomisationSyncPathOptions,
} from "./customisationSyncPaths.ts";
/** Live projections needed to derive Customisation Sync paths. */
export type CustomisationSyncPathOperationsDependencies = Readonly<{
getConfigDir: () => string;
getUseV2: () => boolean;
getUsePluginEtc: () => boolean;
getDeviceAndVaultName: () => string;
}>;
/** Path operations exposed to the Customisation Sync context. */
export type CustomisationSyncPathOperations = Readonly<{
getFileCategory(filePath: string): CustomisationSyncFileCategory;
isTargetPath(filePath: string): boolean;
filenameToUnifiedKey(path: string, termOverride?: string): FilePathWithPrefix;
filenameWithUnifiedKey(path: string, termOverride?: string): FilePathWithPrefix;
unifiedKeyPrefixOfTerminal(termOverride?: string): FilePathWithPrefix;
}>;
function getPathOptions(dependencies: CustomisationSyncPathOperationsDependencies): CustomisationSyncPathOptions {
return {
configDir: dependencies.getConfigDir(),
useV2: dependencies.getUseV2(),
usePluginEtc: dependencies.getUsePluginEtc(),
};
}
export function createCustomisationSyncPathOperations(
dependencies: CustomisationSyncPathOperationsDependencies
): CustomisationSyncPathOperations {
return Object.freeze({
getFileCategory: (filePath: string) => getCustomisationSyncFileCategory(filePath, getPathOptions(dependencies)),
isTargetPath: (filePath: string) => isCustomisationSyncTargetPath(filePath, getPathOptions(dependencies)),
filenameToUnifiedKey: (path: string, termOverride?: string) =>
createCustomisationSyncV1DocumentPath(
path,
termOverride || dependencies.getDeviceAndVaultName(),
getPathOptions(dependencies)
),
filenameWithUnifiedKey: (path: string, termOverride?: string) =>
createCustomisationSyncV2DocumentPath(
path,
termOverride || dependencies.getDeviceAndVaultName(),
getPathOptions(dependencies)
),
unifiedKeyPrefixOfTerminal: (termOverride?: string) =>
createCustomisationSyncDevicePrefix(termOverride || dependencies.getDeviceAndVaultName()),
});
}
@@ -0,0 +1,90 @@
import { describe, expect, it } from "vitest";
import {
createCustomisationSyncPathOperations,
type CustomisationSyncPathOperationsDependencies,
} from "./customisationSyncPathOperations.ts";
type PathState = {
configDir: string;
useV2: boolean;
usePluginEtc: boolean;
deviceAndVaultName: string;
};
function createOperations(state: PathState) {
const dependencies: CustomisationSyncPathOperationsDependencies = {
getConfigDir: () => state.configDir,
getUseV2: () => state.useV2,
getUsePluginEtc: () => state.usePluginEtc,
getDeviceAndVaultName: () => state.deviceAndVaultName,
};
return createCustomisationSyncPathOperations(dependencies);
}
describe("Customisation Sync path operations", () => {
it("reads category and target settings through live getters", () => {
const state: PathState = {
configDir: ".obsidian",
useV2: true,
usePluginEtc: true,
deviceAndVaultName: "device-a",
};
const operations = createOperations(state);
const extraPluginFile = ".obsidian/plugins/example/settings.json";
expect(operations.getFileCategory(extraPluginFile)).toBe("PLUGIN_ETC");
expect(operations.isTargetPath(extraPluginFile)).toBe(true);
state.useV2 = false;
expect(operations.getFileCategory(extraPluginFile)).toBe("");
expect(operations.isTargetPath(extraPluginFile)).toBe(false);
state.useV2 = true;
state.usePluginEtc = false;
expect(operations.getFileCategory(extraPluginFile)).toBe("");
state.configDir = ".config";
expect(operations.isTargetPath(extraPluginFile)).toBe(false);
expect(operations.isTargetPath(".config/plugins/example/settings.json")).toBe(false);
});
it("derives V1, V2, and device-prefix paths from the current term and settings", () => {
const state: PathState = {
configDir: ".obsidian",
useV2: true,
usePluginEtc: true,
deviceAndVaultName: "device-a",
};
const operations = createOperations(state);
const path = ".obsidian/plugins/example/main.js";
expect(operations.filenameToUnifiedKey(path)).toBe("ix:device-a/PLUGIN_MAIN/example.md");
expect(operations.filenameWithUnifiedKey(path)).toBe("ix:device-a/PLUGIN_MAIN/example%main.js");
expect(operations.unifiedKeyPrefixOfTerminal()).toBe("ix:device-a/");
state.deviceAndVaultName = "device-b";
expect(operations.filenameToUnifiedKey(path)).toBe("ix:device-b/PLUGIN_MAIN/example.md");
expect(operations.filenameWithUnifiedKey(path)).toBe("ix:device-b/PLUGIN_MAIN/example%main.js");
expect(operations.unifiedKeyPrefixOfTerminal()).toBe("ix:device-b/");
});
it("keeps the existing override fallback semantics", () => {
const state: PathState = {
configDir: ".obsidian",
useV2: true,
usePluginEtc: true,
deviceAndVaultName: "device-a",
};
const operations = createOperations(state);
const path = ".obsidian/app.json";
expect(operations.filenameToUnifiedKey(path, "device-b")).toBe("ix:device-b/CONFIG/app.json.md");
expect(operations.filenameWithUnifiedKey(path, "device-b")).toBe("ix:device-b/CONFIG/app.json%app.json");
expect(operations.unifiedKeyPrefixOfTerminal("device-b")).toBe("ix:device-b/");
expect(operations.filenameToUnifiedKey(path, "")).toBe("ix:device-a/CONFIG/app.json.md");
expect(operations.filenameWithUnifiedKey(path, "")).toBe("ix:device-a/CONFIG/app.json%app.json");
expect(operations.unifiedKeyPrefixOfTerminal("")).toBe("ix:device-a/");
});
});
@@ -0,0 +1,130 @@
import { ICXHeader } from "@/common/types.ts";
import type { FilePathWithPrefix } from "@vrtmrz/livesync-commonlib/compat/common/types";
import { stripAllPrefixes } from "@vrtmrz/livesync-commonlib/compat/string_and_binary/path";
export type CustomisationSyncFileCategory =
| "CONFIG"
| "THEME"
| "SNIPPET"
| "PLUGIN_MAIN"
| "PLUGIN_ETC"
| "PLUGIN_DATA"
| "";
export type CustomisationSyncPathOptions = {
configDir: string;
useV2: boolean;
usePluginEtc: boolean;
};
export function getCustomisationSyncCategoryFolder(category: string, configDir: string = ""): string {
switch (category) {
case "CONFIG":
return `${configDir}/`;
case "THEME":
return `${configDir}/themes/`;
case "SNIPPET":
return `${configDir}/snippets/`;
case "PLUGIN_MAIN":
case "PLUGIN_DATA":
case "PLUGIN_ETC":
return `${configDir}/plugins/`;
default:
return "";
}
}
export function getCustomisationSyncFileCategory(
filePath: string,
options: CustomisationSyncPathOptions
): CustomisationSyncFileCategory {
if (filePath.split("/").length == 2 && filePath.endsWith(".json")) return "CONFIG";
if (filePath.split("/").length == 4 && filePath.startsWith(`${options.configDir}/themes/`)) return "THEME";
if (filePath.startsWith(`${options.configDir}/snippets/`) && filePath.endsWith(".css")) return "SNIPPET";
if (filePath.startsWith(`${options.configDir}/plugins/`)) {
if (filePath.endsWith("/styles.css") || filePath.endsWith("/manifest.json") || filePath.endsWith("/main.js")) {
return "PLUGIN_MAIN";
}
if (filePath.endsWith("/data.json")) {
return "PLUGIN_DATA";
}
return options.useV2 && options.usePluginEtc ? "PLUGIN_ETC" : "";
}
return "";
}
export function isCustomisationSyncTargetPath(filePath: string, options: CustomisationSyncPathOptions): boolean {
if (!filePath.startsWith(options.configDir)) return false;
return getCustomisationSyncFileCategory(filePath, options) != "";
}
export function getCustomisationSyncSettingKey(
filePath: string,
options: CustomisationSyncPathOptions
): string | undefined {
if (!isCustomisationSyncTargetPath(filePath, options)) return undefined;
const category = getCustomisationSyncFileCategory(filePath, options);
const name =
category == "CONFIG" || category == "SNIPPET"
? filePath.split("/").slice(-1)[0]
: filePath.split("/").slice(-2)[0];
return name ? `${category}/${name}` : undefined;
}
export function getCustomisationSyncSettingKeyFromDocumentPath(documentPath: FilePathWithPrefix): string | undefined {
const [, category, ...rest] = stripAllPrefixes(documentPath).split("/");
if (!category || rest.length == 0) return undefined;
if (!["CONFIG", "THEME", "SNIPPET", "PLUGIN_MAIN", "PLUGIN_ETC", "PLUGIN_DATA"].includes(category)) {
return undefined;
}
const encodedName = category == "CONFIG" || category == "SNIPPET" ? rest.join("/") : rest[0];
const name = encodedName.split("%")[0].replace(/\.md$/, "");
return name ? `${category}/${name}` : undefined;
}
export function createCustomisationSyncV1DocumentPath(
filePath: string,
device: string,
options: CustomisationSyncPathOptions
): FilePathWithPrefix {
const category = getCustomisationSyncFileCategory(filePath, options);
const name =
category == "CONFIG" || category == "SNIPPET"
? filePath.split("/").slice(-1)[0]
: category == "PLUGIN_ETC"
? filePath.split("/").slice(-2).join("/")
: filePath.split("/").slice(-2)[0];
return `${ICXHeader}${device}/${category}/${name}.md` as FilePathWithPrefix;
}
export function createCustomisationSyncV2DocumentPath(
filePath: string,
device: string,
options: CustomisationSyncPathOptions
): FilePathWithPrefix {
const category = getCustomisationSyncFileCategory(filePath, options);
const name =
category == "CONFIG" || category == "SNIPPET"
? filePath.split("/").slice(-1)[0]
: filePath.split("/").slice(-2)[0];
const baseName = category == "CONFIG" || category == "SNIPPET" ? name : filePath.split("/").slice(3).join("/");
return `${ICXHeader}${device}/${category}/${name}%${baseName}` as FilePathWithPrefix;
}
export function createCustomisationSyncDevicePrefix(device: string): FilePathWithPrefix {
return `${ICXHeader}${device}/` as FilePathWithPrefix;
}
export function parseCustomisationSyncV2DocumentPath(unifiedPath: FilePathWithPrefix): {
category: string;
device: string;
key: string;
filename: string;
pathV1: FilePathWithPrefix;
} {
const [device, category, ...rest] = stripAllPrefixes(unifiedPath).split("/");
const relativePath = rest.join("/");
const [key, filename] = relativePath.split("%");
const pathV1 = (unifiedPath.split("%")[0] + ".md") as FilePathWithPrefix;
return { device, category, key, filename, pathV1 };
}
@@ -0,0 +1,117 @@
import { describe, expect, it } from "vitest";
import type { FilePathWithPrefix } from "@vrtmrz/livesync-commonlib/compat/common/types";
import {
createCustomisationSyncDevicePrefix,
createCustomisationSyncV1DocumentPath,
createCustomisationSyncV2DocumentPath,
getCustomisationSyncCategoryFolder,
getCustomisationSyncFileCategory,
isCustomisationSyncTargetPath,
getCustomisationSyncSettingKey,
getCustomisationSyncSettingKeyFromDocumentPath,
parseCustomisationSyncV2DocumentPath,
type CustomisationSyncPathOptions,
} from "./customisationSyncPaths.ts";
const currentOptions: CustomisationSyncPathOptions = {
configDir: ".obsidian",
useV2: true,
usePluginEtc: true,
};
describe("compatibility: Customisation Sync path operations", () => {
it.each([
["CONFIG", ".obsidian/"],
["THEME", ".obsidian/themes/"],
["SNIPPET", ".obsidian/snippets/"],
["PLUGIN_MAIN", ".obsidian/plugins/"],
["PLUGIN_DATA", ".obsidian/plugins/"],
["PLUGIN_ETC", ".obsidian/plugins/"],
["UNKNOWN", ""],
])("maps category %s to folder %s", (category, expected) => {
expect(getCustomisationSyncCategoryFolder(category, ".obsidian")).toBe(expected);
});
it.each([
[".obsidian/app.json", "CONFIG"],
[".obsidian/themes/Minimal/theme.css", "THEME"],
[".obsidian/snippets/example.css", "SNIPPET"],
[".obsidian/plugins/example/manifest.json", "PLUGIN_MAIN"],
[".obsidian/plugins/example/data.json", "PLUGIN_DATA"],
[".obsidian/plugins/example/extra.json", "PLUGIN_ETC"],
] as const)("classifies the maintained path %s as %s", (path, expected) => {
expect(getCustomisationSyncFileCategory(path, currentOptions)).toBe(expected);
});
it("preserves the exact depth and case-sensitive category rules", () => {
expect(getCustomisationSyncFileCategory(".obsidian/themes/Minimal/assets/theme.css", currentOptions)).toBe("");
expect(getCustomisationSyncFileCategory(".obsidian/snippets/example.CSS", currentOptions)).toBe("");
expect(getCustomisationSyncFileCategory(".Obsidian/plugins/example/main.js", currentOptions)).toBe("");
});
it("requires both V2 and plug-in-extra support for other plug-in files", () => {
const path = ".obsidian/plugins/example/extra.json";
expect(getCustomisationSyncFileCategory(path, { ...currentOptions, useV2: false })).toBe("");
expect(getCustomisationSyncFileCategory(path, { ...currentOptions, usePluginEtc: false })).toBe("");
});
it("keeps category classification separate from configuration-directory targeting", () => {
expect(getCustomisationSyncFileCategory("notes/example.json", currentOptions)).toBe("CONFIG");
expect(isCustomisationSyncTargetPath("notes/example.json", currentOptions)).toBe(false);
expect(isCustomisationSyncTargetPath(".obsidian/app.json", currentOptions)).toBe(true);
});
it.each([
[".obsidian/app.json", "CONFIG/app.json"],
[".obsidian/themes/Minimal/theme.css", "THEME/Minimal"],
[".obsidian/snippets/example.css", "SNIPPET/example.css"],
[".obsidian/plugins/example/main.js", "PLUGIN_MAIN/example"],
[".obsidian/plugins/example/data.json", "PLUGIN_DATA/example"],
[".obsidian/plugins/example/extra.json", "PLUGIN_ETC/example"],
] as const)("maps the local path %s to setting key %s", (path, expected) => {
expect(getCustomisationSyncSettingKey(path, currentOptions)).toBe(expected);
});
it.each([
["ix:device-a/CONFIG/app.json.md", "CONFIG/app.json"],
["ix:device-a/CONFIG/app.json%app.json", "CONFIG/app.json"],
["ix:device-a/THEME/Minimal.md", "THEME/Minimal"],
["ix:device-a/PLUGIN_DATA/example%data.json", "PLUGIN_DATA/example"],
["ix:device-a/PLUGIN_ETC/example/extra.json.md", "PLUGIN_ETC/example"],
] as const)("maps the persisted path %s to setting key %s", (path, expected) => {
expect(getCustomisationSyncSettingKeyFromDocumentPath(path as FilePathWithPrefix)).toBe(expected);
});
it.each([
[".obsidian/app.json", "ix:device-a/CONFIG/app.json.md"],
[".obsidian/themes/Minimal/theme.css", "ix:device-a/THEME/Minimal.md"],
[".obsidian/plugins/example/main.js", "ix:device-a/PLUGIN_MAIN/example.md"],
[".obsidian/plugins/example/extra.json", "ix:device-a/PLUGIN_ETC/example/extra.json.md"],
] as const)("creates the persisted V1 path for %s", (path, expected) => {
expect(createCustomisationSyncV1DocumentPath(path, "device-a", currentOptions)).toBe(expected);
});
it.each([
[".obsidian/app.json", "ix:device-a/CONFIG/app.json%app.json"],
[".obsidian/themes/Minimal/theme.css", "ix:device-a/THEME/Minimal%theme.css"],
[".obsidian/plugins/example/main.js", "ix:device-a/PLUGIN_MAIN/example%main.js"],
[".obsidian/plugins/example/extra.json", "ix:device-a/PLUGIN_ETC/example%extra.json"],
] as const)("creates the persisted V2 path for %s", (path, expected) => {
expect(createCustomisationSyncV2DocumentPath(path, "device-a", currentOptions)).toBe(expected);
});
it("creates and parses device-scoped V2 paths", () => {
expect(createCustomisationSyncDevicePrefix("device-a")).toBe("ix:device-a/");
expect(
parseCustomisationSyncV2DocumentPath("ix:device-a/PLUGIN_MAIN/example%main.js" as FilePathWithPrefix)
).toEqual({
device: "device-a",
category: "PLUGIN_MAIN",
key: "example",
filename: "main.js",
pathV1: "ix:device-a/PLUGIN_MAIN/example.md",
});
});
});
@@ -0,0 +1,208 @@
import type {
FilePath,
FilePathWithPrefix,
LoadedEntry,
LOG_LEVEL,
} from "@vrtmrz/livesync-commonlib/compat/common/types";
import { LOG_LEVEL_INFO, LOG_LEVEL_VERBOSE } from "@vrtmrz/livesync-commonlib/compat/common/types";
import type { StorageAccess } from "@vrtmrz/livesync-commonlib/compat/interfaces/StorageAccess";
import type { LiveSyncLocalDB } from "@vrtmrz/livesync-commonlib/compat/pouchdb/LiveSyncLocalDB";
import type { IPathService } from "@vrtmrz/livesync-commonlib/compat/services/base/IService";
import type { LogFunction } from "@vrtmrz/livesync-commonlib/compat/services/lib/logUtils";
import {
createSavingEntryFromLoadedEntry,
fireAndForget,
getDocData,
getDocDataAsArray,
isLoadedEntry,
} from "@vrtmrz/livesync-commonlib/compat/common/utils";
import { digestHash } from "@vrtmrz/livesync-commonlib/compat/string_and_binary/hash";
import { arrayBufferToBase64, readString } from "@vrtmrz/livesync-commonlib/compat/string_and_binary/convert";
import { base64ToString } from "octagonal-wheels/binary/base64";
import type { PluginDataEx, PluginDataExFile } from "./customisationSyncCodec.ts";
import { getCustomisationSyncCategoryFolder, parseCustomisationSyncV2DocumentPath } from "./customisationSyncPaths.ts";
import type { LoadedEntryPluginDataExFile, PluginDataExDisplay } from "./customisationSyncView.ts";
type CustomisationSyncLogDependency = {
log: LogFunction;
};
type CustomisationSyncStorageMethods<Method extends keyof StorageAccess> = {
storageAccess: Pick<StorageAccess, Method>;
};
type CustomisationSyncDatabaseMethods<Method extends keyof LiveSyncLocalDB> = {
getLocalDatabase(): Pick<LiveSyncLocalDB, Method>;
};
type CustomisationSyncPathMethods<Method extends keyof IPathService> = {
path: Pick<IPathService, Method>;
};
export type CustomisationSyncFileReaderDependencies = CustomisationSyncStorageMethods<
"readHiddenFileBinary" | "statHidden"
> &
CustomisationSyncLogDependency;
export type CustomisationSyncDisplayLoaderDependencies = CustomisationSyncDatabaseMethods<"getDBEntry" | "putDBEntry"> &
CustomisationSyncPathMethods<"getPath"> &
CustomisationSyncLogDependency;
export type CustomisationSyncV2EntryLoaderDependencies = CustomisationSyncDatabaseMethods<"getDBEntry"> &
CustomisationSyncLogDependency;
export type CustomisationSyncReadCodec = {
deserialize<T>(source: string[], defaultValue: T): T;
serialize(data: PluginDataEx): string;
};
export type DecodedCustomisationSyncV2File = {
confKey: string;
file: LoadedEntryPluginDataExFile;
isManifest: boolean;
};
function log(dependencies: CustomisationSyncLogDependency, message: unknown, level?: LOG_LEVEL, key?: string): void {
dependencies.log(message, level, key);
}
export async function readCustomisationFile(
dependencies: CustomisationSyncFileReaderDependencies,
path: FilePath,
configDir: string
): Promise<false | PluginDataExFile> {
const stat = await dependencies.storageAccess.statHidden(path);
let version: string | undefined;
let displayName: string | undefined;
if (!stat) {
return false;
}
const contentBin = await dependencies.storageAccess.readHiddenFileBinary(path);
let content: string[];
try {
content = await arrayBufferToBase64(contentBin);
if (path.toLowerCase().endsWith("/manifest.json")) {
const manifestSource = readString(new Uint8Array(contentBin));
try {
const manifest: unknown = JSON.parse(manifestSource);
if (typeof manifest === "object" && manifest !== null) {
if ("version" in manifest) {
version = String(manifest.version);
}
if ("name" in manifest) {
displayName = String(manifest.name);
}
}
} catch (error) {
log(
dependencies,
`Configuration sync data: ${path} looks like manifest, but could not read the version`,
LOG_LEVEL_INFO
);
log(dependencies, error, LOG_LEVEL_VERBOSE);
}
}
} catch (error) {
log(dependencies, `The file ${path} could not be encoded`);
log(dependencies, error, LOG_LEVEL_VERBOSE);
return false;
}
return {
// Compatibility: target validation belongs to the caller. The legacy
// reader derives this name positionally without checking the prefix.
filename: path.substring(configDir.length + 1),
data: content,
mtime: stat.mtime,
size: stat.size,
version,
displayName,
};
}
export async function loadCustomisationDisplayData(
dependencies: CustomisationSyncDisplayLoaderDependencies,
path: FilePathWithPrefix,
codec: CustomisationSyncReadCodec
): Promise<PluginDataExDisplay | false> {
const loaded = await dependencies.getLocalDatabase().getDBEntry(path, undefined, false, false);
if (!loaded) {
return false;
}
const data = codec.deserialize(getDocDataAsArray(loaded.data), {}) as PluginDataEx;
const displayFiles: PluginDataExFile[] = [];
let missingHash = false;
for (const file of data.files) {
const displayFile = { ...file, data: [] as string[] };
if (!file.hash) {
// Compatibility question: the inherited implementation clears the
// display copy before calculating this temporary hash, so callers
// see digestHash([]) until the asynchronously repaired document is
// loaded again. The serialiser still writes the real content hash.
const temporaryHashSource = getDocDataAsArray(displayFile.data);
file.hash = digestHash(temporaryHashSource);
missingHash = true;
}
displayFile.data = [file.hash];
displayFiles.push(displayFile);
}
if (missingHash) {
log(dependencies, `Digest created for ${path} to improve checking`, LOG_LEVEL_VERBOSE);
loaded.data = codec.serialize(data);
// Compatibility: catalogue loading does not wait for the repair write.
fireAndForget(() => dependencies.getLocalDatabase().putDBEntry(createSavingEntryFromLoadedEntry(loaded)));
}
return {
...data,
documentPath: dependencies.path.getPath(loaded),
files: displayFiles,
} satisfies PluginDataExDisplay;
}
export async function loadCustomisationV2Entry(
dependencies: CustomisationSyncV2EntryLoaderDependencies,
path: FilePathWithPrefix
): Promise<LoadedEntry | false> {
const loaded = await dependencies.getLocalDatabase().getDBEntry(path);
if (!loaded) {
log(dependencies, `The file ${path} is not found`, LOG_LEVEL_VERBOSE);
return false;
}
if (!isLoadedEntry(loaded)) {
log(dependencies, `The file ${path} is not a note`, LOG_LEVEL_VERBOSE);
return false;
}
return loaded;
}
export function decodeCustomisationSyncV2File(
path: FilePathWithPrefix,
loaded: LoadedEntry,
dummyEnd: string
): DecodedCustomisationSyncV2File {
const { category, key, filename, device } = parseCustomisationSyncV2DocumentPath(path);
const categoryFolder = getCustomisationSyncCategoryFolder(category, device);
const confKey = `${categoryFolder}${key}`;
const relativeFilename =
`${getCustomisationSyncCategoryFolder(category, "")}${category == "CONFIG" || category == "SNIPPET" ? "" : key + "/"}${filename}`.substring(
1
);
const source = getDocData(loaded.data);
const dataStart = source.indexOf(dummyEnd);
// Compatibility question: a missing marker is not rejected. substring()
// starts at dummyEnd.length - 1, preserving the old best-effort decode.
const encodedData = source.substring(dataStart + dummyEnd.length);
const file: LoadedEntryPluginDataExFile = {
...loaded,
hash: "",
data: [base64ToString(encodedData)],
filename: relativeFilename,
displayName: filename,
};
return {
confKey,
file,
isManifest: filename == "manifest.json",
};
}
@@ -0,0 +1,222 @@
import { describe, expect, it, vi } from "vitest";
import {
LOG_LEVEL_INFO,
LOG_LEVEL_VERBOSE,
type FilePath,
type FilePathWithPrefix,
type LoadedEntry,
type UXStat,
} from "@vrtmrz/livesync-commonlib/compat/common/types";
import { digestHash } from "@vrtmrz/livesync-commonlib/compat/string_and_binary/hash";
import { createCustomisationSyncCodec, type PluginDataEx } from "./customisationSyncCodec.ts";
import {
decodeCustomisationSyncV2File,
loadCustomisationDisplayData,
loadCustomisationV2Entry,
readCustomisationFile,
} from "./customisationSyncReadOperations.ts";
const configDir = ".obsidian";
const filePath = ".obsidian/plugins/example/manifest.json" as FilePath;
const documentPath = "ix:device-a/PLUGIN_MAIN/example.md" as FilePathWithPrefix;
const stat = { ctime: 10, mtime: 20, size: 42, type: "file" } as UXStat;
const codec = createCustomisationSyncCodec({
digestHash,
parseYaml: () => undefined,
});
function createDependencies() {
const localDatabase = {
getDBEntry: vi.fn(),
putDBEntry: vi.fn(async (_entry: unknown) => ({ ok: true, id: "id", rev: "1-a" })),
};
const storageAccess = {
statHidden: vi.fn(async () => stat as UXStat | null),
readHiddenFileBinary: vi.fn(),
};
const log = vi.fn();
const dependencies = {
getLocalDatabase: () => localDatabase as never,
storageAccess: storageAccess as never,
path: {
getPath: vi.fn((entry: { path: FilePathWithPrefix }) => entry.path),
} as never,
log,
};
return { dependencies, localDatabase, log, storageAccess };
}
function loadedEntry(path: FilePathWithPrefix, data: string): LoadedEntry {
return {
_id: "entry-id",
_rev: "1-a",
path,
type: "plain",
datatype: "plain",
data,
ctime: 10,
mtime: 20,
size: data.length,
children: [],
eden: {},
} as unknown as LoadedEntry;
}
function pluginData(hash?: string): PluginDataEx {
return {
category: "PLUGIN_MAIN",
name: "example",
term: "device-a",
mtime: 20,
files: [
{
filename: "plugins/example/main.js",
data: ["payload"],
mtime: 20,
size: 7,
hash,
},
],
};
}
describe("Customisation Sync read operations", () => {
it("does not read content when a local file is missing", async () => {
const { dependencies, storageAccess } = createDependencies();
storageAccess.statHidden.mockResolvedValue(null);
await expect(readCustomisationFile(dependencies, filePath, configDir)).resolves.toBe(false);
expect(storageAccess.readHiddenFileBinary).not.toHaveBeenCalled();
});
it("propagates a storage read failure without converting it to an encoding failure", async () => {
const { dependencies, log, storageAccess } = createDependencies();
const error = new Error("read failed");
storageAccess.readHiddenFileBinary.mockRejectedValue(error);
await expect(readCustomisationFile(dependencies, filePath, configDir)).rejects.toBe(error);
expect(log).not.toHaveBeenCalled();
});
it("encodes a manifest and extracts its display metadata", async () => {
const { dependencies, storageAccess } = createDependencies();
const source = JSON.stringify({ name: "Example plug-in", version: "1.2.3" });
storageAccess.readHiddenFileBinary.mockResolvedValue(new TextEncoder().encode(source).buffer);
await expect(readCustomisationFile(dependencies, filePath, configDir)).resolves.toEqual({
filename: "plugins/example/manifest.json",
data: [btoa(source)],
mtime: 20,
size: 42,
version: "1.2.3",
displayName: "Example plug-in",
});
});
it("keeps an unreadable manifest as file data and reports only the metadata failure", async () => {
const { dependencies, log, storageAccess } = createDependencies();
const errorSource = "{invalid";
storageAccess.readHiddenFileBinary.mockResolvedValue(new TextEncoder().encode(errorSource).buffer);
await expect(readCustomisationFile(dependencies, filePath, configDir)).resolves.toMatchObject({
filename: "plugins/example/manifest.json",
data: [btoa(errorSource)],
version: undefined,
displayName: undefined,
});
expect(log).toHaveBeenNthCalledWith(
1,
`Configuration sync data: ${filePath} looks like manifest, but could not read the version`,
LOG_LEVEL_INFO,
undefined
);
expect(log).toHaveBeenNthCalledWith(2, expect.any(SyntaxError), LOG_LEVEL_VERBOSE, undefined);
});
it("loads V1 display data without retaining file content", async () => {
const { dependencies, localDatabase } = createDependencies();
const data = pluginData("known-hash");
localDatabase.getDBEntry.mockResolvedValue(loadedEntry(documentPath, JSON.stringify(data)));
await expect(loadCustomisationDisplayData(dependencies, documentPath, codec)).resolves.toEqual({
...data,
documentPath,
files: [{ ...data.files[0], data: ["known-hash"] }],
});
expect(localDatabase.getDBEntry).toHaveBeenCalledWith(documentPath, undefined, false, false);
expect(localDatabase.putDBEntry).not.toHaveBeenCalled();
});
it("preserves the inherited transient empty-data hash while repairing a V1 document", async () => {
const { dependencies, localDatabase, log } = createDependencies();
const data = pluginData();
localDatabase.getDBEntry.mockResolvedValue(loadedEntry(documentPath, JSON.stringify(data)));
const result = await loadCustomisationDisplayData(dependencies, documentPath, codec);
expect(result).toMatchObject({ files: [{ data: [digestHash([])] }] });
expect(localDatabase.putDBEntry).toHaveBeenCalledOnce();
const saving = localDatabase.putDBEntry.mock.calls[0][0] as { data: Blob };
expect(await saving.data.text()).toContain(digestHash(["payload"]));
expect(log).toHaveBeenCalledWith(
`Digest created for ${documentPath} to improve checking`,
LOG_LEVEL_VERBOSE,
undefined
);
});
it("returns false when a V1 document is absent", async () => {
const { dependencies, localDatabase } = createDependencies();
localDatabase.getDBEntry.mockResolvedValue(false);
await expect(loadCustomisationDisplayData(dependencies, documentPath, codec)).resolves.toBe(false);
});
it("distinguishes an absent V2 entry from a non-note database entry", async () => {
const { dependencies, localDatabase, log } = createDependencies();
const path = "ix:device-a/PLUGIN_MAIN/example%main.js" as FilePathWithPrefix;
localDatabase.getDBEntry.mockResolvedValueOnce(false);
await expect(loadCustomisationV2Entry(dependencies, path)).resolves.toBe(false);
expect(log).toHaveBeenLastCalledWith(`The file ${path} is not found`, LOG_LEVEL_VERBOSE, undefined);
localDatabase.getDBEntry.mockResolvedValueOnce({ path, type: "leaf" });
await expect(loadCustomisationV2Entry(dependencies, path)).resolves.toBe(false);
expect(log).toHaveBeenLastCalledWith(`The file ${path} is not a note`, LOG_LEVEL_VERBOSE, undefined);
});
it("returns a loaded V2 entry after the exact single-argument database lookup", async () => {
const { dependencies, localDatabase } = createDependencies();
const path = "ix:device-a/PLUGIN_MAIN/example%main.js" as FilePathWithPrefix;
const loaded = loadedEntry(path, "data");
localDatabase.getDBEntry.mockResolvedValue(loaded);
await expect(loadCustomisationV2Entry(dependencies, path)).resolves.toBe(loaded);
expect(localDatabase.getDBEntry).toHaveBeenCalledWith(path);
});
it("decodes a V2 payload into its relative Customisation Sync filename", () => {
const path = "ix:device-a/PLUGIN_MAIN/example%main.js" as FilePathWithPrefix;
const loaded = loadedEntry(path, `${codec.dummyHead}${codec.dummyEnd}${btoa("console.log('example');")}`);
expect(decodeCustomisationSyncV2File(path, loaded, codec.dummyEnd)).toEqual({
confKey: "device-a/plugins/example",
isManifest: false,
file: {
...loaded,
filename: "plugins/example/main.js",
displayName: "main.js",
hash: "",
data: ["console.log('example');"],
},
});
});
it("preserves the inherited best-effort offset when a V2 marker is missing", () => {
const path = "ix:device-a/CONFIG/app.json%app.json" as FilePathWithPrefix;
const loaded = loadedEntry(path, `00${btoa("hello")}`);
expect(decodeCustomisationSyncV2File(path, loaded, "END").file.data).toEqual(["hello"]);
});
});
@@ -0,0 +1,17 @@
const MAX_RECENT_CUSTOMISATION_EVENTS = 100;
/** Keeps the bounded newest-first raw-event keys used by Customisation Sync. */
export class CustomisationSyncRecentEventDeduplicator {
private keys: string[] = [];
/**
* Records a key when it is new and returns whether the caller should act.
* Native Array#includes is intentional: the old `.contains` extension is
* not available in every runtime where the feature is exercised.
*/
admit(key: string): boolean {
if (this.keys.includes(key)) return false;
this.keys = [key, ...this.keys].slice(0, MAX_RECENT_CUSTOMISATION_EVENTS);
return true;
}
}
@@ -0,0 +1,24 @@
import { describe, expect, it } from "vitest";
import { CustomisationSyncRecentEventDeduplicator } from "./customisationSyncRecentEventDeduplicator.ts";
describe("Customisation Sync recent raw-event keys", () => {
it("admits a key once and keeps newer keys first", () => {
const history = new CustomisationSyncRecentEventDeduplicator();
expect(history.admit("old")).toBe(true);
expect(history.admit("new")).toBe(true);
expect(history.admit("old")).toBe(false);
});
it("evicts the oldest key when the newest-first history exceeds 100 entries", () => {
const history = new CustomisationSyncRecentEventDeduplicator();
for (let index = 0; index < 101; index++) {
expect(history.admit(`key-${index}`)).toBe(true);
}
expect(history.admit("key-0")).toBe(true);
expect(history.admit("key-100")).toBe(false);
});
});
@@ -0,0 +1,47 @@
import { readFileSync } from "node:fs";
import { describe, expect, it } from "vitest";
const uiSources = [
["PluginDialogModal", readFileSync(new URL("./PluginDialogModal.ts", import.meta.url), "utf8")],
["PluginPane", readFileSync(new URL("./PluginPane.svelte", import.meta.url), "utf8")],
["PluginCombo", readFileSync(new URL("./PluginCombo.svelte", import.meta.url), "utf8")],
[
"PaneCustomisationSync",
readFileSync(
new URL("../../modules/features/SettingDialogue/PaneCustomisationSync.ts", import.meta.url),
"utf8"
),
],
[
"PaneHatch",
readFileSync(new URL("../../modules/features/SettingDialogue/PaneHatch.ts", import.meta.url), "utf8"),
],
] as const;
const customisationSyncSource = readFileSync(new URL("./customisationSyncContext.ts", import.meta.url), "utf8");
const settingTabSource = readFileSync(
new URL("../../modules/features/SettingDialogue/ObsidianLiveSyncSettingTab.ts", import.meta.url),
"utf8"
);
const settingModuleSource = readFileSync(
new URL("../../modules/features/ModuleObsidianSettingTab.ts", import.meta.url),
"utf8"
);
describe("optional-file synchronisation UI dependency boundary", () => {
it.each(uiSources)("keeps %s independent from concrete add-ons and the application core", (_name, source) => {
expect(source).not.toMatch(/from ["'][^"']*Cmd(?:Config|HiddenFile)Sync(?:\.ts)?["']/);
expect(source).not.toMatch(/from ["'][^"']*main(?:\.ts)?["']/);
expect(source).not.toContain("getAddOn(");
expect(source).not.toContain("getAddOn<");
});
it("keeps the Customisation Sync runtime independent from its Obsidian dialogue", () => {
expect(customisationSyncSource).not.toMatch(/from ["'][^"']*PluginDialogModal(?:\.ts)?["']/);
});
it("keeps the settings presentation out of the runtime cycle and off the compatibility lookup", () => {
expect(settingTabSource).not.toMatch(/^import(?!\s+type\b)[^;]*from ["'][^"']*main(?:\.ts)?["'];/mu);
expect(settingModuleSource).not.toContain("getAddOn(");
expect(settingModuleSource).not.toContain("getAddOn<");
});
});
@@ -0,0 +1,118 @@
import type { PluginManifest } from "@/deps.ts";
import type {
EntryDoc,
FilePathWithPrefix,
FilePath,
LoadedEntry,
PluginSyncSettingEntry,
SYNC_MODE,
} from "@vrtmrz/livesync-commonlib/compat/common/types";
import type PouchDB from "pouchdb-core";
import type { Readable } from "svelte/store";
import type { PluginDataExFile } from "./customisationSyncCodec.ts";
import type { OptionalSyncFeatureMode } from "@/features/optionalSyncFeatures.ts";
import type { PluginDataExDisplayV2 } from "./customisationSyncModel.ts";
export type LoadedEntryPluginDataExFile = LoadedEntry & PluginDataExFile;
export type { CustomisationSyncFileCategory } from "./customisationSyncPaths.ts";
export interface IPluginDataExDisplay {
documentPath: FilePathWithPrefix;
category: string;
name: string;
term: string;
displayName?: string;
files: (LoadedEntryPluginDataExFile | PluginDataExFile)[];
version?: string;
mtime: number;
}
export type PluginDataExDisplay = {
documentPath: FilePathWithPrefix;
category: string;
name: string;
term: string;
displayName?: string;
files: PluginDataExFile[];
version?: string;
mtime: number;
};
/**
* Semantic callbacks registered by the optional-file composition feature.
*
* The context owns the implementations, while the optional-file composition
* adapts these operations to Commonlib's aggregation contracts. Consumers
* receive only callable operations, not the context or its private state.
*/
export interface CustomisationSyncServiceHandlers {
readonly processOptionalFileEvent: (path: FilePath) => Promise<boolean>;
readonly processVirtualDocument: (docs: PouchDB.Core.ExistingDocument<EntryDoc>) => Promise<boolean>;
readonly onRealiseSetting: () => Promise<boolean>;
readonly onResuming: () => Promise<boolean>;
readonly onBeforeReplicate: (showMessage: boolean) => Promise<boolean>;
readonly onDatabaseInitialised: (showNotice: boolean) => Promise<boolean>;
readonly suspendExtraSync: () => Promise<boolean>;
readonly enableOptionalFeature: (mode: OptionalSyncFeatureMode) => Promise<boolean>;
}
/**
* Explicit internal operations used by maintained real-Obsidian contract
* tests. This is deliberately narrower than the concrete context and does
* not expose reactive stores, queues, or host dependencies.
*/
export interface CustomisationSyncTestingView {
readonly configDir: string;
scanInternalFiles(): Promise<FilePath[]>;
scanAllConfigFiles(showMessage: boolean): Promise<void>;
storeCustomizationFiles(path: FilePath, termOverride?: string): Promise<unknown>;
deleteConfigOnDatabase(prefixedFileName: FilePathWithPrefix, forceWrite?: boolean): Promise<boolean>;
createPluginDataFromV2(unifiedPathV2: FilePathWithPrefix): PluginDataExDisplayV2 | undefined;
createPluginDataExFileV2(
unifiedPathV2: FilePathWithPrefix,
loaded?: LoadedEntry
): Promise<false | LoadedEntryPluginDataExFile>;
applyDataV2(data: PluginDataExDisplayV2, content?: string): Promise<boolean>;
}
/** Stable catalogue and operation surface consumed by the Obsidian dialogue. */
export interface CustomisationSyncDialogView {
readonly catalogue: Readable<IPluginDataExDisplay[]>;
readonly enumerationActive: Readable<boolean>;
readonly migrationProgress: Readable<number>;
readonly manifests: Readable<Map<string, PluginManifest>>;
isEnabled(): boolean;
getDeviceAndVaultName(): string;
getConfiguredModes(): PluginSyncSettingEntry[];
isPluginEtcEnabled(): boolean;
updateConfiguredMode(key: string, mode: SYNC_MODE, files: string[]): void;
getConfiguredTargetFiles(key: string): string[];
updatePluginList(showMessage: boolean, updatedDocumentPath?: FilePathWithPrefix): Promise<void>;
reloadPluginList(showMessage: boolean): Promise<void>;
scanAllConfigFiles(showMessage: boolean): Promise<void>;
synchronise(): Promise<void>;
applyData(data: IPluginDataExDisplay): Promise<boolean>;
compareUsingDisplayData(
dataA: IPluginDataExDisplay,
dataB: IPluginDataExDisplay,
compareEach?: boolean
): Promise<boolean>;
compareFileUsingDisplayData(
dataA: IPluginDataExDisplay,
dataB: IPluginDataExDisplay,
filename: string
): Promise<boolean>;
deleteData(data: IPluginDataExDisplay): Promise<boolean>;
duplicateData(data: IPluginDataExDisplay, deviceName: string): Promise<void>;
askString(title: string, key: string, placeholder: string): Promise<string | false>;
}
/** Narrow control returned by the host-owned dialogue composition feature. */
export interface CustomisationSyncUIControl {
open(): void;
close(): void;
isOpen(): boolean;
}
+197
View File
@@ -0,0 +1,197 @@
import type {
AnyEntry,
FilePath,
FilePathWithPrefix,
InternalFileEntry,
LOG_LEVEL,
ObsidianLiveSyncSettings,
} from "@vrtmrz/livesync-commonlib/compat/common/types";
import { LOG_LEVEL_INFO, LOG_LEVEL_NOTICE, LOG_LEVEL_VERBOSE } from "@vrtmrz/livesync-commonlib/compat/common/types";
import { fireAndForget } from "@vrtmrz/livesync-commonlib/compat/common/utils";
import type { LiveSyncLocalDB } from "@vrtmrz/livesync-commonlib/compat/pouchdb/LiveSyncLocalDB";
import type { LogFunction } from "@vrtmrz/livesync-commonlib/compat/services/lib/logUtils";
import { shareRunningResult } from "octagonal-wheels/concurrency/lock";
import { Semaphore } from "octagonal-wheels/concurrency/semaphore";
import { $msg } from "@/common/translation";
import { ICXHeader } from "@/common/types.ts";
import {
collectOptionalFileSyncFiles,
type OptionalFileSyncFileTreeDependencies,
} from "@/features/optionalFileSyncFileTree.ts";
import type { CatalogueOperations } from "./catalogueOperations.ts";
import type { CustomisationSyncPathOperations } from "./customisationSyncPathOperations.ts";
import type { SnapshotOperations } from "./snapshotOperations.ts";
type ScanSettings = Pick<ObsidianLiveSyncSettings, "usePluginSyncV2">;
type ScanDatabase = Pick<LiveSyncLocalDB, "allDocsRaw" | "findEntries">;
type ScanPathOperations = Pick<
CustomisationSyncPathOperations,
"isTargetPath" | "filenameToUnifiedKey" | "filenameWithUnifiedKey" | "unifiedKeyPrefixOfTerminal"
> & {
getPath(entry: AnyEntry): FilePathWithPrefix;
};
type ScanSnapshotOperations = Pick<
SnapshotOperations,
"storeCustomisationFileV2" | "storeCustomizationFiles" | "deleteConfigOnDatabase"
>;
type ScanCatalogueOperations = Pick<CatalogueOperations, "updatePluginList">;
export type ScanOperationsDependencies = OptionalFileSyncFileTreeDependencies & {
getSettings(): ScanSettings;
getLocalDatabase(): ScanDatabase;
path: ScanPathOperations;
log: LogFunction;
getConfigDir(): string;
getDeviceAndVaultName(): string;
ownsLocalFile(path: FilePath): boolean;
ownsLocalDocument(path: FilePathWithPrefix): boolean;
snapshotOperations: ScanSnapshotOperations;
catalogueOperations: ScanCatalogueOperations;
};
/**
* Owns Customisation Sync file enumeration and reconciliation with the local
* database. Snapshot writes and catalogue publication remain explicit ports so
* scans do not depend on the context or its lifecycle.
*/
export class ScanOperations {
constructor(private readonly dependencies: ScanOperationsDependencies) {}
private get localDatabase() {
return this.dependencies.getLocalDatabase();
}
private getPath(entry: AnyEntry): FilePathWithPrefix {
return this.dependencies.path.getPath(entry);
}
private _log(message: unknown, level?: LOG_LEVEL, key?: string) {
this.dependencies.log(message, level, key);
}
async scanInternalFiles(): Promise<FilePath[]> {
const filenames = (
await collectOptionalFileSyncFiles(this.dependencies, this.dependencies.getConfigDir(), {
maxDepth: 2,
onError: (path, error) => {
this._log(`Could not traverse(CustomisationSync):${path}`, LOG_LEVEL_INFO);
this._log(error, LOG_LEVEL_VERBOSE);
},
})
)
.filter((e) => e.startsWith("."))
.filter((e) => !e.startsWith(".trash"));
return filenames as FilePath[];
}
async scanAllConfigFiles(showMessage: boolean): Promise<void> {
await shareRunningResult("scanAllConfigFiles", async () => {
const logLevel = showMessage ? LOG_LEVEL_NOTICE : LOG_LEVEL_INFO;
this._log("Scanning customizing files.", logLevel, "scan-all-config");
const term = this.dependencies.getDeviceAndVaultName();
if (term == "") {
this._log($msg("We have to configure the device name"), LOG_LEVEL_NOTICE);
return;
}
const filesAll = await this.scanInternalFiles();
if (this.dependencies.getSettings().usePluginSyncV2) {
await this.scanV2ConfigFiles(filesAll, term);
} else {
await this.scanV1ConfigFiles(filesAll, term);
}
});
}
private async scanV2ConfigFiles(filesAll: readonly FilePath[], term: string): Promise<void> {
const filesAllUnified = filesAll
.filter((e) => this.dependencies.path.isTargetPath(e))
.map((e) => [this.dependencies.path.filenameWithUnifiedKey(e, term), e] as [FilePathWithPrefix, FilePath]);
const localFileMap = new Map(filesAllUnified.map((e) => [e[0], e[1]]));
const prefix = this.dependencies.path.unifiedKeyPrefixOfTerminal(term);
const entries = this.localDatabase.findEntries(prefix + "", `${prefix}\u{10ffff}`, {
include_docs: true,
});
const tasks = [] as (() => Promise<void>)[];
const concurrency = 10;
const semaphore = Semaphore(concurrency);
for await (const item of entries) {
if (item.path.indexOf("%") !== -1) {
continue;
}
tasks.push(async () => {
const releaser = await semaphore.acquire();
try {
const unifiedFilenameWithKey = `${item._id}` as FilePathWithPrefix;
const localPath = localFileMap.get(unifiedFilenameWithKey);
if (localPath) {
if (this.dependencies.ownsLocalFile(localPath)) {
await this.dependencies.snapshotOperations.storeCustomisationFileV2(localPath, term);
}
localFileMap.delete(unifiedFilenameWithKey);
} else if (this.dependencies.ownsLocalDocument(this.getPath(item))) {
await this.dependencies.snapshotOperations.deleteConfigOnDatabase(unifiedFilenameWithKey);
}
} catch (ex) {
this._log(`scanAllConfigFiles - Error: ${item._id}`, LOG_LEVEL_VERBOSE);
this._log(ex, LOG_LEVEL_VERBOSE);
} finally {
releaser();
}
});
}
await Promise.all(tasks.map((e) => e()));
// Extra files
const taskExtra = [] as (() => Promise<void>)[];
for (const [, filePath] of localFileMap) {
if (!this.dependencies.ownsLocalFile(filePath)) continue;
taskExtra.push(async () => {
const releaser = await semaphore.acquire();
try {
await this.dependencies.snapshotOperations.storeCustomisationFileV2(filePath, term);
} catch (ex) {
this._log(`scanAllConfigFiles - Error: ${filePath}`, LOG_LEVEL_VERBOSE);
this._log(ex, LOG_LEVEL_VERBOSE);
} finally {
releaser();
}
});
}
await Promise.all(taskExtra.map((e) => e()));
fireAndForget(() => this.dependencies.catalogueOperations.updatePluginList(false));
}
private async scanV1ConfigFiles(filesAll: readonly FilePath[], term: string): Promise<void> {
const files = filesAll
.filter((e) => this.dependencies.path.isTargetPath(e))
.map((e) => ({ key: this.dependencies.path.filenameToUnifiedKey(e), file: e }));
const virtualPathsOfLocalFiles = [...new Set(files.map((e) => e.key))];
const filesOnDB = (
(
await this.localDatabase.allDocsRaw({
startkey: ICXHeader + "",
endkey: `${ICXHeader}\u{10ffff}`,
include_docs: true,
})
).rows.map((e) => e.doc) as InternalFileEntry[]
).filter((e) => !e.deleted);
let deleteCandidate = filesOnDB.map((e) => this.getPath(e)).filter((e) => e.startsWith(`${ICXHeader}${term}/`));
for (const vp of virtualPathsOfLocalFiles) {
const p = files.find((e) => e.key == vp)?.file;
if (!p) {
this._log(`scanAllConfigFiles - File not found: ${vp}`, LOG_LEVEL_VERBOSE);
continue;
}
if (this.dependencies.ownsLocalFile(p)) {
await this.dependencies.snapshotOperations.storeCustomizationFiles(p);
}
deleteCandidate = deleteCandidate.filter((e) => e != vp);
}
for (const vp of deleteCandidate) {
if (this.dependencies.ownsLocalDocument(vp)) {
await this.dependencies.snapshotOperations.deleteConfigOnDatabase(vp);
}
}
fireAndForget(() => this.dependencies.catalogueOperations.updatePluginList(false));
}
}
@@ -0,0 +1,327 @@
import { beforeEach, describe, expect, it, vi } from "vitest";
import {
LOG_LEVEL_INFO,
LOG_LEVEL_NOTICE,
LOG_LEVEL_VERBOSE,
type FilePath,
type FilePathWithPrefix,
} from "@vrtmrz/livesync-commonlib/compat/common/types";
const asyncHarness = vi.hoisted(() => ({
fireAndForget: vi.fn((operation: () => unknown) => {
void operation();
}),
}));
vi.mock("@/common/translation", () => ({
$msg: vi.fn((message: string) => message),
}));
vi.mock("@vrtmrz/livesync-commonlib/compat/common/utils", async (importOriginal) => {
const actual = await importOriginal<typeof import("@vrtmrz/livesync-commonlib/compat/common/utils")>();
return {
...actual,
fireAndForget: asyncHarness.fireAndForget,
};
});
import { ScanOperations, type ScanOperationsDependencies } from "./scanOperations.ts";
type ScanEntry = {
_id: FilePathWithPrefix;
path: FilePathWithPrefix;
deleted?: boolean;
};
type FixtureOptions = {
useV2?: boolean;
usePluginSync?: boolean;
term?: string;
files?: FilePath[];
targetFiles?: FilePath[];
databaseEntries?: ScanEntry[];
v1Paths?: Record<string, FilePathWithPrefix>;
v2Paths?: Record<string, FilePathWithPrefix>;
ownsLocalFile?: (path: FilePath) => boolean;
ownsLocalDocument?: (path: FilePathWithPrefix) => boolean;
listFiles?: (path: string) => Promise<{ files: readonly string[]; folders: readonly string[] }>;
};
function asyncEntries<T>(entries: readonly T[]) {
return {
async *[Symbol.asyncIterator]() {
yield* entries;
},
};
}
function createFixture(options: FixtureOptions = {}) {
const files = options.files ?? [];
const term = options.term ?? "device-a";
const databaseEntries = options.databaseEntries ?? [];
const v1Paths = options.v1Paths ?? {};
const v2Paths = options.v2Paths ?? {};
const log = vi.fn();
const allDocsRaw = vi.fn(async () => ({
rows: databaseEntries.map((doc) => ({ id: doc._id, doc })),
}));
const findEntries = vi.fn(() => asyncEntries(databaseEntries));
const storeCustomisationFileV2 = vi.fn(async (_path: FilePath, _term: string) => true);
const storeCustomizationFiles = vi.fn(async (_path: FilePath) => true);
const deleteConfigOnDatabase = vi.fn(async (_path: FilePathWithPrefix) => true);
const updatePluginList = vi.fn(async (_showMessage: boolean) => undefined);
const listFiles = vi.fn(
options.listFiles ??
(async () => ({ files, folders: [] }) as { files: readonly string[]; folders: readonly string[] })
);
const dependencies = {
listFiles,
getSettings: () => ({ usePluginSyncV2: options.useV2 ?? false, usePluginSync: options.usePluginSync ?? true }),
getLocalDatabase: () => ({ allDocsRaw, findEntries }),
path: {
isTargetPath: (path: string) => options.targetFiles?.includes(path as FilePath) ?? true,
filenameToUnifiedKey: (path: string) =>
v1Paths[path] ?? (`ix:${term}/CONFIG/${path.split("/").pop()}.md` as FilePathWithPrefix),
filenameWithUnifiedKey: (path: string) =>
v2Paths[path] ??
(`ix:${term}/CONFIG/${path.split("/").pop()}%${path.split("/").pop()}` as FilePathWithPrefix),
unifiedKeyPrefixOfTerminal: (termOverride?: string) => `ix:${termOverride ?? term}/` as FilePathWithPrefix,
getPath: (entry: ScanEntry) => entry.path,
},
log,
getConfigDir: () => ".obsidian",
getDeviceAndVaultName: () => term,
ownsLocalFile: options.ownsLocalFile ?? (() => true),
ownsLocalDocument: options.ownsLocalDocument ?? (() => true),
snapshotOperations: {
storeCustomisationFileV2,
storeCustomizationFiles,
deleteConfigOnDatabase,
},
catalogueOperations: { updatePluginList },
} as unknown as ScanOperationsDependencies;
return {
operations: new ScanOperations(dependencies),
dependencies,
listFiles,
log,
allDocsRaw,
findEntries,
snapshot: { storeCustomisationFileV2, storeCustomizationFiles, deleteConfigOnDatabase },
catalogue: { updatePluginList },
};
}
describe("ScanOperations", () => {
beforeEach(() => {
asyncHarness.fireAndForget.mockClear();
});
it("filters the bounded file tree and logs traversal errors", async () => {
const traversalError = new Error("cannot read folder");
const listFiles = vi.fn(async (path: string) => {
switch (path) {
case ".obsidian":
return {
files: [".obsidian/app.json", "settings.json", ".trash/root.json"],
folders: [".obsidian/plugins", ".trash", ".obsidian/unreadable"],
};
case ".obsidian/plugins":
return {
files: [".obsidian/plugins/example/data.json"],
folders: [".obsidian/plugins/example"],
};
case ".obsidian/plugins/example":
return {
files: [".obsidian/plugins/example/manifest.json"],
folders: [".obsidian/plugins/example/deeper"],
};
case ".trash":
return { files: [".trash/ignored.json"], folders: [] };
case ".obsidian/unreadable":
throw traversalError;
default:
throw new Error(`unexpected traversal: ${path}`);
}
});
const fixture = createFixture({ listFiles });
await expect(fixture.operations.scanInternalFiles()).resolves.toEqual([
".obsidian/app.json",
".obsidian/plugins/example/data.json",
".obsidian/plugins/example/manifest.json",
]);
expect(listFiles).not.toHaveBeenCalledWith(".obsidian/plugins/example/deeper");
expect(fixture.log).toHaveBeenCalledWith(
"Could not traverse(CustomisationSync):.obsidian/unreadable",
LOG_LEVEL_INFO,
undefined
);
expect(fixture.log).toHaveBeenCalledWith(traversalError, LOG_LEVEL_VERBOSE, undefined);
});
it("stops before traversal when the device term is empty", async () => {
const fixture = createFixture({ term: "", usePluginSync: false, files: [".obsidian/app.json"] as FilePath[] });
await fixture.operations.scanAllConfigFiles(true);
expect(fixture.listFiles).not.toHaveBeenCalled();
expect(fixture.allDocsRaw).not.toHaveBeenCalled();
expect(fixture.findEntries).not.toHaveBeenCalled();
expect(fixture.snapshot.storeCustomizationFiles).not.toHaveBeenCalled();
expect(fixture.snapshot.storeCustomisationFileV2).not.toHaveBeenCalled();
expect(fixture.snapshot.deleteConfigOnDatabase).not.toHaveBeenCalled();
expect(fixture.catalogue.updatePluginList).not.toHaveBeenCalled();
expect(fixture.log).toHaveBeenCalledWith("Scanning customizing files.", LOG_LEVEL_NOTICE, "scan-all-config");
expect(fixture.log).toHaveBeenCalledWith("We have to configure the device name", LOG_LEVEL_NOTICE, undefined);
});
it("dispatches according to the current V1/V2 setting on each scan", async () => {
const path = ".obsidian/app.json" as FilePath;
const fixture = createFixture({ files: [path], useV2: false });
let useV2 = false;
fixture.dependencies.getSettings = () => ({ usePluginSyncV2: useV2 });
await fixture.operations.scanAllConfigFiles(false);
useV2 = true;
await fixture.operations.scanAllConfigFiles(false);
expect(fixture.snapshot.storeCustomizationFiles).toHaveBeenCalledWith(path);
expect(fixture.snapshot.storeCustomizationFiles).toHaveBeenCalledTimes(1);
expect(fixture.snapshot.storeCustomisationFileV2).toHaveBeenCalledWith(path, "device-a");
expect(fixture.snapshot.storeCustomisationFileV2).toHaveBeenCalledTimes(1);
});
it("routes V1 ownership and deletes only stale owned documents", async () => {
const ownedPath = ".obsidian/app.json" as FilePath;
const unownedPath = ".obsidian/appearance.json" as FilePath;
const ownedDocument = "ix:device-a/CONFIG/app.json.md" as FilePathWithPrefix;
const unownedDocument = "ix:device-a/CONFIG/appearance.json.md" as FilePathWithPrefix;
const staleDocument = "ix:device-a/CONFIG/stale.json.md" as FilePathWithPrefix;
const fixture = createFixture({
useV2: false,
usePluginSync: false,
files: [ownedPath, unownedPath],
v1Paths: {
[ownedPath]: ownedDocument,
[unownedPath]: unownedDocument,
},
databaseEntries: [
{ _id: ownedDocument, path: ownedDocument },
{ _id: unownedDocument, path: unownedDocument },
{ _id: staleDocument, path: staleDocument },
],
ownsLocalFile: (path) => path == ownedPath,
});
await fixture.operations.scanAllConfigFiles(false);
expect(fixture.snapshot.storeCustomizationFiles).toHaveBeenCalledWith(ownedPath);
expect(fixture.snapshot.storeCustomizationFiles).toHaveBeenCalledTimes(1);
expect(fixture.snapshot.deleteConfigOnDatabase).toHaveBeenCalledWith(staleDocument);
expect(fixture.snapshot.deleteConfigOnDatabase).toHaveBeenCalledTimes(1);
expect(fixture.catalogue.updatePluginList).toHaveBeenCalledWith(false);
expect(fixture.catalogue.updatePluginList).toHaveBeenCalledTimes(1);
expect(fixture.allDocsRaw).toHaveBeenCalledWith({
startkey: "ix:",
endkey: "ix:\u{10ffff}",
include_docs: true,
});
});
it("propagates V1 snapshot failures without publishing a final refresh", async () => {
const path = ".obsidian/app.json" as FilePath;
const failure = new Error("V1 write failed");
const fixture = createFixture({ files: [path] });
fixture.snapshot.storeCustomizationFiles.mockRejectedValueOnce(failure);
await expect(fixture.operations.scanAllConfigFiles(false)).rejects.toBe(failure);
expect(fixture.catalogue.updatePluginList).not.toHaveBeenCalled();
});
it("routes V2 ownership, removes matched keys, and deletes stale documents", async () => {
const ownedPath = ".obsidian/app.json" as FilePath;
const unownedPath = ".obsidian/appearance.json" as FilePath;
const extraPath = ".obsidian/plugins/example/main.js" as FilePath;
const ownedDocument = "ix:device-a/CONFIG/app.json%app.json" as FilePathWithPrefix;
const unownedDocument = "ix:device-a/CONFIG/appearance.json%appearance.json" as FilePathWithPrefix;
const staleDocument = "ix:device-a/CONFIG/stale.json" as FilePathWithPrefix;
const skippedDocument = "ix:device-a/CONFIG/skipped%app.json" as FilePathWithPrefix;
const owned = new Set([ownedPath, extraPath]);
const fixture = createFixture({
useV2: true,
files: [ownedPath, unownedPath, extraPath],
v2Paths: {
[ownedPath]: ownedDocument,
[unownedPath]: unownedDocument,
[extraPath]: "ix:device-a/PLUGIN_MAIN/example%main.js" as FilePathWithPrefix,
},
databaseEntries: [
{ _id: ownedDocument, path: "ix:device-a/CONFIG/app.json.md" as FilePathWithPrefix },
{ _id: unownedDocument, path: "ix:device-a/CONFIG/appearance.json.md" as FilePathWithPrefix },
{ _id: staleDocument, path: staleDocument },
{ _id: skippedDocument, path: skippedDocument },
],
ownsLocalFile: (path) => owned.has(path),
});
await fixture.operations.scanAllConfigFiles(false);
expect(fixture.snapshot.storeCustomisationFileV2).toHaveBeenCalledWith(ownedPath, "device-a");
expect(fixture.snapshot.storeCustomisationFileV2).toHaveBeenCalledWith(extraPath, "device-a");
expect(fixture.snapshot.storeCustomisationFileV2).toHaveBeenCalledTimes(2);
expect(fixture.snapshot.storeCustomisationFileV2).not.toHaveBeenCalledWith(unownedPath, "device-a");
expect(fixture.snapshot.deleteConfigOnDatabase).toHaveBeenCalledWith(staleDocument);
expect(fixture.snapshot.deleteConfigOnDatabase).toHaveBeenCalledTimes(1);
expect(fixture.catalogue.updatePluginList).toHaveBeenCalledWith(false);
expect(fixture.findEntries).toHaveBeenCalledWith("ix:device-a/", "ix:device-a/\u{10ffff}", {
include_docs: true,
});
});
it("catches and logs each V2 entry failure before publishing the refresh", async () => {
const path = ".obsidian/app.json" as FilePath;
const document = "ix:device-a/CONFIG/app.json%app.json" as FilePathWithPrefix;
const failure = new Error("V2 write failed");
const fixture = createFixture({
useV2: true,
files: [path],
v2Paths: { [path]: document },
databaseEntries: [{ _id: document, path: "ix:device-a/CONFIG/app.json.md" as FilePathWithPrefix }],
});
fixture.snapshot.storeCustomisationFileV2.mockRejectedValueOnce(failure);
await fixture.operations.scanAllConfigFiles(false);
expect(fixture.log).toHaveBeenCalledWith(
`scanAllConfigFiles - Error: ${document}`,
LOG_LEVEL_VERBOSE,
undefined
);
expect(fixture.log).toHaveBeenCalledWith(failure, LOG_LEVEL_VERBOSE, undefined);
expect(fixture.catalogue.updatePluginList).toHaveBeenCalledWith(false);
});
it("starts the final catalogue refresh without awaiting it", async () => {
const path = ".obsidian/app.json" as FilePath;
const fixture = createFixture({ files: [path] });
let releaseRefresh!: () => void;
const refresh = new Promise<void>((resolve) => {
releaseRefresh = resolve;
});
const refreshStarted = vi.fn();
fixture.catalogue.updatePluginList.mockImplementation(async () => {
refreshStarted();
await refresh;
});
await fixture.operations.scanAllConfigFiles(false);
expect(asyncHarness.fireAndForget).toHaveBeenCalledOnce();
expect(refreshStarted).toHaveBeenCalledOnce();
expect(fixture.catalogue.updatePluginList).toHaveBeenCalledWith(false);
releaseRefresh();
await refresh;
});
});
@@ -0,0 +1,82 @@
import type {
FilePath,
FilePathWithPrefix,
LOG_LEVEL,
ObsidianLiveSyncSettings,
} from "@vrtmrz/livesync-commonlib/compat/common/types";
import { LOG_LEVEL_NOTICE } from "@vrtmrz/livesync-commonlib/compat/common/types";
import { fireAndForget } from "@vrtmrz/livesync-commonlib/compat/common/utils";
import { $msg } from "@/common/translation";
import type { CatalogueOperations } from "./catalogueOperations.ts";
import type { SnapshotPersistence, SnapshotRefresh } from "./snapshotPersistence.ts";
type SnapshotSettings = Pick<ObsidianLiveSyncSettings, "usePluginSyncV2">;
type SnapshotPersistencePort = Pick<
SnapshotPersistence,
"storeCustomisationFileV2" | "storeCustomizationFiles" | "deleteConfigOnDatabase"
>;
type SnapshotCatalogue = Pick<CatalogueOperations, "updatePluginList" | "updatePluginListV2">;
export type SnapshotOperationsDependencies = {
getSettings(): SnapshotSettings;
getDeviceAndVaultName(): string;
log(message: unknown, level?: LOG_LEVEL, key?: string): void;
snapshotPersistence: SnapshotPersistencePort;
catalogueOperations: SnapshotCatalogue;
};
/**
* Adapts host-neutral Customisation Sync snapshot mutations to catalogue
* refreshes. Current-term selection and the inherited refresh timing live in
* this owner so scan, dialogue, and testing callers share one policy.
*/
export class SnapshotOperations {
constructor(private readonly dependencies: SnapshotOperationsDependencies) {}
private _log(message: unknown, level?: LOG_LEVEL, key?: string) {
this.dependencies.log(message, level, key);
}
isV2Enabled(): boolean {
return this.dependencies.getSettings().usePluginSyncV2;
}
private async applyPersistenceRefreshes(refreshes: readonly SnapshotRefresh[]) {
for (const refresh of refreshes) {
if (refresh.mode == "v2" && refresh.timing == "fire-and-forget") {
fireAndForget(() => this.dependencies.catalogueOperations.updatePluginListV2(false, refresh.path));
} else if (refresh.mode == "v1" && refresh.timing == "await") {
await this.dependencies.catalogueOperations.updatePluginList(false, refresh.path);
}
}
}
async storeCustomisationFileV2(path: FilePath, term: string, force = false) {
const persistence = await this.dependencies.snapshotPersistence.storeCustomisationFileV2(path, term, force);
await this.applyPersistenceRefreshes(persistence.refreshes);
return persistence.value;
}
async storeCustomizationFiles(path: FilePath, termOverride?: string) {
const term = termOverride || this.dependencies.getDeviceAndVaultName();
if (term == "") {
this._log($msg("We have to configure the device name"), LOG_LEVEL_NOTICE);
return;
}
const persistence = this.isV2Enabled()
? await this.dependencies.snapshotPersistence.storeCustomisationFileV2(path, term)
: await this.dependencies.snapshotPersistence.storeCustomizationFiles(path, term);
await this.applyPersistenceRefreshes(persistence.refreshes);
return persistence.value;
}
async deleteConfigOnDatabase(prefixedFileName: FilePathWithPrefix, forceWrite = false): Promise<boolean> {
const persistence = await this.dependencies.snapshotPersistence.deleteConfigOnDatabase(
prefixedFileName,
forceWrite
);
await this.applyPersistenceRefreshes(persistence.refreshes);
return persistence.value;
}
}
@@ -0,0 +1,137 @@
import { describe, expect, it, vi } from "vitest";
const asyncHarness = vi.hoisted(() => ({
fireAndForget: vi.fn((operation: () => unknown) => {
void operation();
}),
}));
vi.mock("@/common/translation", () => ({
$msg: vi.fn((message: string) => message),
}));
vi.mock("@vrtmrz/livesync-commonlib/compat/common/utils", async (importOriginal) => {
const actual = await importOriginal<typeof import("@vrtmrz/livesync-commonlib/compat/common/utils")>();
return {
...actual,
fireAndForget: asyncHarness.fireAndForget,
};
});
import type { FilePath, FilePathWithPrefix } from "@vrtmrz/livesync-commonlib/compat/common/types";
import type { SnapshotPersistenceResult } from "./snapshotPersistence.ts";
import { SnapshotOperations, type SnapshotOperationsDependencies } from "./snapshotOperations.ts";
const CONFIG_PATH = ".obsidian/app.json" as FilePath;
const V1_PATH = "ix:device-b/CONFIG/app.json.md" as FilePathWithPrefix;
const V2_PATH = "ix:device-a/CONFIG/app.json%app.json" as FilePathWithPrefix;
function createOperations(usePluginSyncV2: boolean) {
const events: string[] = [];
type PersistenceResult = SnapshotPersistenceResult<true>;
const storeCustomisationFileV2 = vi.fn(
async (_path: FilePath, _term: string, _force?: boolean): Promise<PersistenceResult> => ({
value: true,
status: "saved",
refreshes: [],
})
);
const storeCustomizationFiles = vi.fn(
async (_path: FilePath, _term: string): Promise<PersistenceResult> => ({
value: true,
status: "saved",
refreshes: [],
})
);
const deleteConfigOnDatabase = vi.fn(
async (_path: FilePathWithPrefix, _force?: boolean): Promise<PersistenceResult> => ({
value: true,
status: "deleted",
refreshes: [],
})
);
const updatePluginList = vi.fn(async () => {
events.push("refresh-v1");
});
const updatePluginListV2 = vi.fn(async () => {
events.push("refresh-v2");
});
const dependencies: SnapshotOperationsDependencies = {
getSettings: () => ({ usePluginSyncV2 }),
getDeviceAndVaultName: () => "device-a",
log: vi.fn(),
snapshotPersistence: {
storeCustomisationFileV2,
storeCustomizationFiles,
deleteConfigOnDatabase,
},
catalogueOperations: { updatePluginList, updatePluginListV2 },
};
return {
operations: new SnapshotOperations(dependencies),
events,
persistence: { storeCustomisationFileV2, storeCustomizationFiles, deleteConfigOnDatabase },
catalogue: { updatePluginList, updatePluginListV2 },
};
}
describe("Snapshot Operations", () => {
it("selects V1 persistence with an override term and awaits its refresh", async () => {
const fixture = createOperations(false);
fixture.persistence.storeCustomizationFiles.mockImplementation(async (_path: FilePath, term: string) => {
fixture.events.push(`persist:${term}`);
return {
value: true,
status: "saved",
refreshes: [{ mode: "v1", timing: "await", path: V1_PATH }],
};
});
await expect(fixture.operations.storeCustomizationFiles(CONFIG_PATH, "device-b")).resolves.toBe(true);
expect(fixture.persistence.storeCustomizationFiles).toHaveBeenCalledWith(CONFIG_PATH, "device-b");
expect(fixture.events).toEqual(["persist:device-b", "refresh-v1"]);
expect(fixture.catalogue.updatePluginList).toHaveBeenCalledWith(false, V1_PATH);
});
it("selects V2 persistence with the current term and does not await its refresh", async () => {
const fixture = createOperations(true);
let releaseRefresh!: () => void;
const refresh = new Promise<void>((resolve) => {
releaseRefresh = resolve;
});
fixture.persistence.storeCustomisationFileV2.mockImplementation(async (_path: FilePath, term: string) => {
fixture.events.push(`persist:${term}`);
return {
value: true,
status: "saved",
refreshes: [{ mode: "v2", timing: "fire-and-forget", path: V2_PATH }],
};
});
fixture.catalogue.updatePluginListV2.mockImplementation(async () => {
fixture.events.push("refresh-v2-start");
await refresh;
fixture.events.push("refresh-v2-end");
});
await expect(fixture.operations.storeCustomizationFiles(CONFIG_PATH)).resolves.toBe(true);
expect(fixture.events).toEqual(["persist:device-a", "refresh-v2-start"]);
releaseRefresh();
await refresh;
expect(fixture.events).toEqual(["persist:device-a", "refresh-v2-start", "refresh-v2-end"]);
});
it("returns the persistence result after applying deletion refreshes", async () => {
const fixture = createOperations(false);
fixture.persistence.deleteConfigOnDatabase.mockImplementation(async () => ({
value: true,
status: "deleted",
refreshes: [{ mode: "v1", timing: "await", path: V1_PATH }],
}));
await expect(fixture.operations.deleteConfigOnDatabase(V1_PATH)).resolves.toBe(true);
expect(fixture.persistence.deleteConfigOnDatabase).toHaveBeenCalledWith(V1_PATH, false);
expect(fixture.catalogue.updatePluginList).toHaveBeenCalledWith(false, V1_PATH);
});
});
@@ -0,0 +1,402 @@
import { parseYaml } from "@/deps.ts";
import type {
FilePath,
FilePathWithPrefix,
InternalFileEntry,
LOG_LEVEL,
SavingEntry,
} from "@vrtmrz/livesync-commonlib/compat/common/types";
import { LOG_LEVEL_DEBUG, LOG_LEVEL_VERBOSE } from "@vrtmrz/livesync-commonlib/compat/common/types";
import {
createBlob,
createTextBlob,
getDocData,
getDocDataAsArray,
isDocContentSame,
} from "@vrtmrz/livesync-commonlib/compat/common/utils";
import { EVEN } from "@vrtmrz/livesync-commonlib/compat/common/models/shared.const.symbols";
import { digestHash } from "@vrtmrz/livesync-commonlib/compat/string_and_binary/hash";
import { arrayBufferToBase64 } from "@vrtmrz/livesync-commonlib/compat/string_and_binary/convert";
import type { LiveSyncLocalDB } from "@vrtmrz/livesync-commonlib/compat/pouchdb/LiveSyncLocalDB";
import type { StorageAccess } from "@vrtmrz/livesync-commonlib/compat/interfaces/StorageAccess";
import type { IPathService } from "@vrtmrz/livesync-commonlib/compat/services/base/IService";
import type { LogFunction } from "@vrtmrz/livesync-commonlib/compat/services/lib/logUtils";
import { base64ToArrayBuffer } from "octagonal-wheels/binary/base64";
import { serialized } from "octagonal-wheels/concurrency/lock";
import { LiveSyncError } from "@vrtmrz/livesync-commonlib/compat/common/LSError";
import { createCustomisationSyncCodec, type PluginDataEx } from "./customisationSyncCodec.ts";
import type { CustomisationSyncPathOperations } from "./customisationSyncPathOperations.ts";
import { readCustomisationFile } from "./customisationSyncReadOperations.ts";
const {
serialize,
deserialize,
dummyHead: DUMMY_HEAD,
dummyEnd: DUMMY_END,
} = createCustomisationSyncCodec({ digestHash, parseYaml });
type SnapshotPersistenceDatabase = Pick<
LiveSyncLocalDB,
"getDBEntryFromMeta" | "getDBEntryMeta" | "putDBEntry" | "putRaw"
>;
type SnapshotPersistenceStorage = Pick<StorageAccess, "readHiddenFileBinary" | "statHidden">;
type SnapshotPersistencePath = Pick<
CustomisationSyncPathOperations,
"getFileCategory" | "filenameToUnifiedKey" | "filenameWithUnifiedKey"
> &
Pick<IPathService, "isMarkedAsSameChanges" | "markChangesAreSame" | "path2id">;
export type SnapshotPersistenceDependencies = {
getLocalDatabase(): SnapshotPersistenceDatabase;
storageAccess: SnapshotPersistenceStorage;
path: SnapshotPersistencePath;
log: LogFunction;
getConfigDir(): string;
};
export type SnapshotRefresh = {
mode: "v1" | "v2";
timing: "await" | "fire-and-forget";
path: FilePathWithPrefix;
};
export type SnapshotPersistenceStatus = "saved" | "skipped" | "missing" | "deleted" | "already-deleted" | "failed";
export type SnapshotPersistenceResult<Value> = {
value: Value;
status: SnapshotPersistenceStatus;
refreshes: readonly SnapshotRefresh[];
};
type DatabaseSaveResult = Awaited<ReturnType<SnapshotPersistenceDatabase["putDBEntry"]>>;
type StoreResultValue = DatabaseSaveResult | true | undefined;
function result<Value>(
value: Value,
status: SnapshotPersistenceStatus,
refreshes: readonly SnapshotRefresh[] = []
): SnapshotPersistenceResult<Value> {
return { value, status, refreshes };
}
/**
* Persists local Customisation Sync snapshots without owning catalogue state,
* lifecycle, replication, or user-interface behaviour.
*/
export class SnapshotPersistence {
private readonly dependencies: SnapshotPersistenceDependencies;
constructor(dependencies: SnapshotPersistenceDependencies) {
this.dependencies = dependencies;
}
private _log(message: unknown, level?: LOG_LEVEL, key?: string) {
this.dependencies.log(message, level, key);
}
private async readFile(path: FilePath) {
return await readCustomisationFile(
{
storageAccess: this.dependencies.storageAccess,
log: this.dependencies.log,
},
path,
this.dependencies.getConfigDir()
);
}
// Compatibility question: the inherited force parameter is not read.
// Preserve it until its intended write-bypass semantics are decided.
async storeCustomisationFileV2(
path: FilePath,
term: string,
force = false
): Promise<SnapshotPersistenceResult<StoreResultValue>> {
void force;
const vf = this.dependencies.path.filenameWithUnifiedKey(path, term);
return await serialized(`plugin-${vf}`, async () => {
const prefixedFileName = vf;
const id = await this.dependencies.path.path2id(prefixedFileName);
const stat = await this.dependencies.storageAccess.statHidden(path);
if (!stat) {
return result(false, "missing");
}
const mtime = stat.mtime;
const content = await this.dependencies.storageAccess.readHiddenFileBinary(path);
const contentBlob = createBlob([DUMMY_HEAD, DUMMY_END, ...(await arrayBufferToBase64(content))]);
// const contentBlob = createBlob(content);
try {
const old = await this.dependencies
.getLocalDatabase()
.getDBEntryMeta(prefixedFileName, undefined, false);
let saveData: SavingEntry;
if (old === false) {
saveData = {
_id: id,
path: prefixedFileName,
data: contentBlob,
mtime,
ctime: mtime,
datatype: "plain",
size: contentBlob.size,
children: [],
deleted: false,
type: "plain",
eden: {},
};
} else {
// Compatibility question: this inherited marker check
// precedes loading the old document and can suppress a
// content comparison. Preserve that event-suppression
// ordering until its scan contract is reviewed.
if (
this.dependencies.path.isMarkedAsSameChanges(prefixedFileName, [old.mtime, mtime + 1]) == EVEN
) {
this._log(
`STORAGE --> DB:${prefixedFileName}: (config) Skipped (Already checked the same)`,
LOG_LEVEL_DEBUG
);
return result(undefined, "skipped");
}
const docXDoc = await this.dependencies.getLocalDatabase().getDBEntryFromMeta(old, false, false);
if (docXDoc == false) {
throw new LiveSyncError("Could not load the document");
}
const dataSrc = getDocData(docXDoc.data);
const dataStart = dataSrc.indexOf(DUMMY_END);
const oldContent = dataSrc.substring(dataStart + DUMMY_END.length);
const oldContentArray = base64ToArrayBuffer(oldContent);
if (await isDocContentSame(oldContentArray, content)) {
this._log(
`STORAGE --> DB:${prefixedFileName}: (config) Skipped (the same content)`,
LOG_LEVEL_VERBOSE
);
this.dependencies.path.markChangesAreSame(prefixedFileName, old.mtime, mtime + 1);
return result(true, "skipped");
}
saveData = {
...old,
data: contentBlob,
mtime,
size: contentBlob.size,
datatype: "plain",
children: [],
deleted: false,
type: "plain",
};
}
const ret = await this.dependencies.getLocalDatabase().putDBEntry(saveData);
this._log(`STORAGE --> DB:${prefixedFileName}: (config) Done`);
// Compatibility question: the inherited refresh path omits the
// explicit term override and therefore uses the current term.
// Preserve that path until its cross-device semantics are reviewed.
return result(ret, "saved", [
{
mode: "v2",
timing: "fire-and-forget",
path: this.dependencies.path.filenameWithUnifiedKey(path),
},
]);
} catch (ex) {
this._log(`STORAGE --> DB:${prefixedFileName}: (config) Failed`);
this._log(ex, LOG_LEVEL_VERBOSE);
return result(false, "failed");
}
});
}
async storeCustomizationFiles(path: FilePath, term: string): Promise<SnapshotPersistenceResult<StoreResultValue>> {
const vf = this.dependencies.path.filenameToUnifiedKey(path, term);
// console.warn(`Storing ${path} to ${bareVF} :--> ${keyedVF}`);
return await serialized(`plugin-${vf}`, async () => {
const category = this.dependencies.path.getFileCategory(path);
let mtime = 0;
let fileTargets = [] as FilePath[];
// let savePath = "";
const name =
category == "CONFIG" || category == "SNIPPET"
? path.split("/").reverse()[0]
: path.split("/").reverse()[1];
const parentPath = path.split("/").slice(0, -1).join("/");
const prefixedFileName = this.dependencies.path.filenameToUnifiedKey(path, term);
const id = await this.dependencies.path.path2id(prefixedFileName);
const dt: PluginDataEx = {
category: category,
files: [],
name: name,
mtime: 0,
term: term,
};
// let scheduleKey = "";
if (
category == "CONFIG" ||
category == "SNIPPET" ||
category == "PLUGIN_ETC" ||
category == "PLUGIN_DATA"
) {
fileTargets = [path];
if (category == "PLUGIN_ETC") {
dt.displayName = path.split("/").slice(-1).join("/");
}
} else if (category == "PLUGIN_MAIN") {
fileTargets = ["manifest.json", "main.js", "styles.css"].map((e) => `${parentPath}/${e}` as FilePath);
} else if (category == "THEME") {
fileTargets = ["manifest.json", "theme.css"].map((e) => `${parentPath}/${e}` as FilePath);
}
for (const target of fileTargets) {
const data = await this.readFile(target);
if (data == false) {
this._log(`Config: skipped (Possibly is not exist): ${target} `, LOG_LEVEL_VERBOSE);
continue;
}
if (data.version) {
dt.version = data.version;
}
if (data.displayName) {
dt.displayName = data.displayName;
}
// Compatibility question: the inherited aggregation uses an
// average rather than the newest member mtime. Preserve that
// scan behaviour until its timestamp policy is reviewed.
mtime = mtime == 0 ? data.mtime : (data.mtime + mtime) / 2;
dt.files.push(data);
}
dt.mtime = mtime;
// Compatibility question: the inherited empty-file path performs a
// deletion refresh and then an unconditional explicit refresh. Keep
// both outcomes, including the extra refresh when deletion succeeds.
if (dt.files.length == 0) {
this._log(`Nothing left: deleting.. ${path}`);
const deletion = await this.deleteConfigOnDatabase(prefixedFileName);
return result(undefined, deletion.status, [
...deletion.refreshes,
{ mode: "v1", timing: "await", path: prefixedFileName },
]);
}
const content = createTextBlob(serialize(dt));
try {
const old = await this.dependencies
.getLocalDatabase()
.getDBEntryMeta(prefixedFileName, undefined, false);
let saveData: SavingEntry;
if (old === false) {
saveData = {
_id: id,
path: prefixedFileName,
data: content,
mtime,
ctime: mtime,
datatype: "newnote",
size: content.size,
children: [],
deleted: false,
type: "newnote",
eden: {},
};
} else {
if (old.mtime == mtime) {
// this._log(`STORAGE --> DB:${prefixedFileName}: (config) Skipped (Same time)`, LOG_LEVEL_VERBOSE);
return result(true, "skipped");
}
const oldC = await this.dependencies.getLocalDatabase().getDBEntryFromMeta(old, false, false);
if (oldC) {
const d = deserialize(getDocDataAsArray(oldC.data), {}) as PluginDataEx;
if (d.files.length == dt.files.length) {
// Compatibility question: the inherited comparison
// looks up each current file by the previous filename
// and compares a missing lookup as empty content.
// Preserve this rename/empty-file behaviour for now.
const diffs = d.files
.map((previous) => ({
prev: previous,
curr: dt.files.find((e) => e.filename == previous.filename),
}))
.map(async (e) => {
try {
return await isDocContentSame(e.curr?.data ?? [], e.prev.data);
} catch {
return false;
}
});
const isSame = (await Promise.all(diffs)).every((e) => e == true);
if (isSame) {
this._log(
`STORAGE --> DB:${prefixedFileName}: (config) Skipped (Same content)`,
LOG_LEVEL_VERBOSE
);
return result(true, "skipped");
}
}
}
saveData = {
...old,
data: content,
mtime,
size: content.size,
datatype: "newnote",
children: [],
deleted: false,
type: "newnote",
};
}
const ret = await this.dependencies.getLocalDatabase().putDBEntry(saveData);
this._log(`STORAGE --> DB:${prefixedFileName}: (config) Done`);
return result(ret, "saved", [{ mode: "v1", timing: "await", path: saveData.path }]);
} catch (ex) {
this._log(`STORAGE --> DB:${prefixedFileName}: (config) Failed`);
this._log(ex, LOG_LEVEL_VERBOSE);
return result(false, "failed");
}
});
}
// Compatibility question: the inherited forceWrite parameter is not read.
// Preserve it until callers define whether deletion should bypass a marker.
async deleteConfigOnDatabase(
prefixedFileName: FilePathWithPrefix,
forceWrite = false
): Promise<SnapshotPersistenceResult<boolean>> {
void forceWrite;
// const id = await this.path2id(prefixedFileName);
const mtime = new Date().getTime();
return await serialized("file-x-" + prefixedFileName, async () => {
try {
const old = (await this.dependencies
.getLocalDatabase()
.getDBEntryMeta(prefixedFileName, undefined, false)) as InternalFileEntry | false;
let saveData: InternalFileEntry;
if (old === false) {
this._log(`STORAGE -x> DB:${prefixedFileName}: (config) already deleted (Not found on database)`);
return result(true, "missing");
} else {
if (old.deleted) {
this._log(`STORAGE -x> DB:${prefixedFileName}: (config) already deleted`);
return result(true, "already-deleted");
}
saveData = {
...old,
mtime,
size: 0,
children: [],
deleted: true,
type: "newnote",
};
}
await this.dependencies.getLocalDatabase().putRaw(saveData);
this._log(`STORAGE -x> DB:${prefixedFileName}: (config) Done`);
return result(true, "deleted", [{ mode: "v1", timing: "await", path: prefixedFileName }]);
} catch (ex) {
this._log(`STORAGE -x> DB:${prefixedFileName}: (config) Failed`);
this._log(ex, LOG_LEVEL_VERBOSE);
return result(false, "failed");
}
});
}
}
@@ -0,0 +1,219 @@
import { describe, expect, it, vi } from "vitest";
vi.mock("@/deps.ts", () => ({
diff_match_patch: class DiffMatchPatch {},
normalizePath: vi.fn((path: string) => path),
parseYaml: vi.fn(),
}));
vi.mock("@/common/utils.ts", () => ({
EVEN: Symbol("even"),
cancelTask: vi.fn(),
fireAndForget: vi.fn(),
scheduleTask: vi.fn(),
}));
vi.mock("@/common/types.ts", () => ({
ICXHeader: "ix:",
PERIODIC_PLUGIN_SWEEP: 60,
}));
vi.mock("@/common/translation", () => ({
$msg: vi.fn((message: string) => message),
}));
vi.mock("@/common/obsidianCommunityPlugins.ts", () => ({
getObsidianCommunityPluginManager: vi.fn(),
}));
vi.mock("@/features/optionalFileSyncFileTree.ts", () => ({
collectOptionalFileSyncFiles: vi.fn(),
}));
import type { FilePath, FilePathWithPrefix, LoadedEntry, UXStat } from "@vrtmrz/livesync-commonlib/compat/common/types";
import { EVEN } from "@vrtmrz/livesync-commonlib/compat/common/models/shared.const.symbols";
import { createCustomisationSyncCodec } from "./customisationSyncCodec.ts";
import { SnapshotPersistence, type SnapshotPersistenceDependencies } from "./snapshotPersistence.ts";
const CONFIG_PATH = ".obsidian/app.json" as FilePath;
const V1_PATH = "ix:device-a/CONFIG/app.json.md" as FilePathWithPrefix;
const V2_PATH = "ix:device-a/CONFIG/app.json%app.json" as FilePathWithPrefix;
const codec = createCustomisationSyncCodec({
digestHash: (source) => source.join(""),
parseYaml: () => undefined,
});
function loadedV2Entry(source: string, mtime = 10): LoadedEntry {
const data = `${codec.dummyHead}${codec.dummyEnd}${btoa(source)}`;
return {
_id: "entry-id",
_rev: "1-a",
path: V2_PATH,
type: "plain",
datatype: "plain",
data,
ctime: mtime,
mtime,
size: data.length,
children: [],
eden: {},
} as unknown as LoadedEntry;
}
function createPersistence(
options: {
category?: "CONFIG" | "PLUGIN_MAIN";
old?: false | LoadedEntry;
stat?: UXStat | null;
content?: string;
currentTerm?: string;
} = {}
) {
const currentTerm = options.currentTerm ?? "device-a";
const statHidden = vi.fn(
async (_path: string): Promise<UXStat | null> =>
options.stat === undefined ? { type: "file", ctime: 10, mtime: 10, size: 5 } : options.stat
);
const readHiddenFileBinary = vi.fn(
async (_path: string) => new TextEncoder().encode(options.content ?? "hello").buffer
);
const getDBEntryMeta = vi.fn(async () => options.old ?? false);
const getDBEntryFromMeta = vi.fn(async (entry: LoadedEntry) => entry);
const putDBEntry = vi.fn(async () => ({ ok: true, id: "entry-id", rev: "2-b" }));
const putRaw = vi.fn(async () => ({ ok: true, id: "entry-id", rev: "2-b" }));
const filenameToUnifiedKey = vi.fn(
(_path: string, term?: string) =>
`ix:${term ?? currentTerm}/${options.category ?? "CONFIG"}/app.json.md` as FilePathWithPrefix
);
const filenameWithUnifiedKey = vi.fn(
(_path: string, term?: string) =>
`ix:${term ?? currentTerm}/${options.category ?? "CONFIG"}/app.json%app.json` as FilePathWithPrefix
);
const dependencies: SnapshotPersistenceDependencies = {
getLocalDatabase: () => ({ getDBEntryMeta, getDBEntryFromMeta, putDBEntry, putRaw }),
storageAccess: { statHidden, readHiddenFileBinary },
path: {
getFileCategory: () => options.category ?? "CONFIG",
filenameToUnifiedKey,
filenameWithUnifiedKey,
path2id: vi.fn(async (path) => path),
isMarkedAsSameChanges: vi.fn(),
markChangesAreSame: vi.fn(),
},
log: vi.fn(),
getConfigDir: () => ".obsidian",
};
return {
database: { getDBEntryMeta, getDBEntryFromMeta, putDBEntry, putRaw },
dependencies,
filenameToUnifiedKey,
filenameWithUnifiedKey,
persistence: new SnapshotPersistence(dependencies),
readHiddenFileBinary,
statHidden,
};
}
describe("Customisation Sync snapshot persistence", () => {
it("persists a V2 file and returns a fire-and-forget catalogue refresh", async () => {
const fixture = createPersistence();
const mutation = await fixture.persistence.storeCustomisationFileV2(CONFIG_PATH, "device-a");
expect(mutation).toMatchObject({
value: { ok: true, id: "entry-id", rev: "2-b" },
status: "saved",
refreshes: [{ mode: "v2", timing: "fire-and-forget", path: V2_PATH }],
});
expect(fixture.database.putDBEntry).toHaveBeenCalledOnce();
expect(fixture.filenameWithUnifiedKey).toHaveBeenNthCalledWith(1, CONFIG_PATH, "device-a");
expect(fixture.filenameWithUnifiedKey).toHaveBeenNthCalledWith(2, CONFIG_PATH);
});
it("aggregates the V1 plug-in file set and returns an awaited refresh", async () => {
const fixture = createPersistence({ category: "PLUGIN_MAIN" });
const mutation = await fixture.persistence.storeCustomizationFiles(
".obsidian/plugins/example/main.js" as FilePath,
"device-a"
);
expect(mutation).toMatchObject({
value: { ok: true },
status: "saved",
refreshes: [{ mode: "v1", timing: "await", path: "ix:device-a/PLUGIN_MAIN/app.json.md" }],
});
expect(fixture.readHiddenFileBinary).toHaveBeenCalledTimes(3);
expect(fixture.database.putDBEntry).toHaveBeenCalledOnce();
});
it("keeps the inherited duplicate V1 refresh on the empty-file deletion path", async () => {
const old = {
...loadedV2Entry("old"),
path: V1_PATH,
datatype: "newnote",
type: "newnote",
deleted: false,
} as LoadedEntry;
const fixture = createPersistence({ old, stat: null });
const mutation = await fixture.persistence.storeCustomizationFiles(CONFIG_PATH, "device-a");
expect(mutation.value).toBeUndefined();
expect(mutation.status).toBe("deleted");
expect(mutation.refreshes).toEqual([
{ mode: "v1", timing: "await", path: V1_PATH },
{ mode: "v1", timing: "await", path: V1_PATH },
]);
expect(fixture.database.putRaw).toHaveBeenCalledOnce();
});
it.each([
["missing", false, "missing"],
["already deleted", { ...loadedV2Entry("old"), deleted: true } as LoadedEntry, "already-deleted"],
] as const)("treats an absent or %s document as a successful no-op", async (_label, old, status) => {
const fixture = createPersistence({ old, stat: null });
const mutation = await fixture.persistence.deleteConfigOnDatabase(V1_PATH);
expect(mutation).toMatchObject({ value: true, status, refreshes: [] });
expect(fixture.database.putRaw).not.toHaveBeenCalled();
});
it("returns an awaited refresh only when deletion writes a live document", async () => {
const old = {
...loadedV2Entry("old"),
path: V1_PATH,
deleted: false,
} as LoadedEntry;
const fixture = createPersistence({ old });
const mutation = await fixture.persistence.deleteConfigOnDatabase(V1_PATH);
expect(mutation).toMatchObject({
value: true,
status: "deleted",
refreshes: [{ mode: "v1", timing: "await", path: V1_PATH }],
});
expect(fixture.database.putRaw).toHaveBeenCalledOnce();
});
it("preserves the V2 marker and same-content skips", async () => {
const markerFixture = createPersistence({ old: loadedV2Entry("old") });
const marker = markerFixture.dependencies.path.isMarkedAsSameChanges as ReturnType<typeof vi.fn>;
marker.mockReturnValue(EVEN);
await expect(
markerFixture.persistence.storeCustomisationFileV2(CONFIG_PATH, "device-a")
).resolves.toMatchObject({
value: undefined,
status: "skipped",
refreshes: [],
});
expect(markerFixture.database.putDBEntry).not.toHaveBeenCalled();
const sameContentFixture = createPersistence({ old: loadedV2Entry("hello") });
await expect(
sameContentFixture.persistence.storeCustomisationFileV2(CONFIG_PATH, "device-a")
).resolves.toMatchObject({
value: true,
status: "skipped",
refreshes: [],
});
expect(sameContentFixture.database.putDBEntry).not.toHaveBeenCalled();
});
});
File diff suppressed because it is too large Load Diff
@@ -1,470 +0,0 @@
import { describe, expect, it, vi } from "vitest";
import {
type DocumentID,
LOG_LEVEL_NOTICE,
type FilePath,
type FilePathWithPrefix,
type MetaEntry,
type UXFileInfo,
} from "@vrtmrz/livesync-commonlib/compat/common/types";
vi.mock("@/deps.ts", () => ({}));
vi.mock("@/features/HiddenFileCommon/JsonResolveModal.ts", () => ({
JsonResolveModal: class JsonResolveModal {},
}));
vi.mock("@/features/LiveSyncCommands.ts", () => ({
LiveSyncCommands: class LiveSyncCommands {
plugin!: { app: unknown };
core!: { services: unknown; settings: unknown };
get app() {
return this.plugin.app;
}
get services() {
return this.core.services;
}
get settings() {
return this.core.settings;
}
},
}));
vi.mock("./configureHiddenFileSyncMode.ts", () => ({
configureHiddenFileSyncMode: vi.fn(),
}));
import { HiddenFileSync } from "./CmdHiddenFileSync.ts";
import { configureHiddenFileSyncMode } from "./configureHiddenFileSyncMode.ts";
function createHiddenRevisionOperation() {
const path = ".obsidian/plugins/example/data.json" as FilePath;
const file = {
path,
name: "data.json",
isInternal: true,
body: new Blob(["{\"value\":\"vault\"}"]),
stat: {
ctime: 1,
mtime: 2,
size: 17,
type: "file",
},
} as UXFileInfo;
const selected = {
_id: "i:example" as DocumentID,
_rev: "2-selected",
path: `i:${path}` as FilePathWithPrefix,
ctime: 1,
mtime: 2,
size: 17,
type: "plain",
datatype: "plain",
children: [],
eden: {},
deleted: false,
} as MetaEntry;
const winner = {
...selected,
_rev: "3-winner",
} as MetaEntry;
const databaseFileAccess = {
fetchEntryMeta: vi.fn(
async (_path: unknown, revision?: string) =>
revision === selected._rev ? selected : winner
),
getConflictedRevs: vi.fn(async () => [selected._rev]),
fetchEntryFromMeta: vi.fn(async () => ({ ...selected, data: "{\"value\":\"database\"}" })),
storeWithBaseRevision: vi.fn(async () => "3-vault-child"),
};
const hiddenFileSync = Object.create(HiddenFileSync.prototype) as HiddenFileSync;
Object.assign(hiddenFileSync, {
core: {
services: {
vault: {
isIgnoredByIgnoreFile: vi.fn(async () => false),
},
},
databaseFileAccess,
},
loadFileWithInfo: vi.fn(async () => file),
updateLastProcessed: vi.fn(),
_log: vi.fn(),
});
return {
hiddenFileSync,
path,
file,
selected,
winner,
databaseFileAccess,
};
}
describe("HiddenFileSync configuration-change notices", () => {
it("shows manual Hidden File Sync commands only when the feature, Advanced mode, and runtime are ready", () => {
const commands: Array<{
id: string;
checkCallback?: (checking: boolean) => boolean | void;
}> = [];
const settings = {
syncInternalFiles: false,
useAdvancedMode: false,
};
const hiddenFileSync = Object.create(HiddenFileSync.prototype) as HiddenFileSync;
Object.assign(hiddenFileSync, {
core: {
settings,
services: {
API: {
addCommand: vi.fn((command) => commands.push(command)),
},
},
},
_isMainReady: vi.fn(() => true),
_isMainSuspended: vi.fn(() => false),
_isDatabaseReady: vi.fn(() => true),
});
hiddenFileSync.onload();
const commandIds = [
"livesync-sync-internal",
"livesync-scaninternal-storage",
"livesync-scaninternal-database",
"livesync-internal-scan-offline-changes",
];
for (const commandId of commandIds) {
const command = commands.find(({ id }) => id === commandId);
expect(command?.checkCallback?.(true)).toBe(false);
}
settings.syncInternalFiles = true;
settings.useAdvancedMode = true;
for (const commandId of commandIds) {
const command = commands.find(({ id }) => id === commandId);
expect(command?.checkCallback?.(true)).toBe(true);
}
});
it("does not report Hidden File Sync as ready before the main runtime is ready", () => {
const hiddenFileSync = Object.create(HiddenFileSync.prototype) as HiddenFileSync;
Object.assign(hiddenFileSync, {
core: {
settings: {
syncInternalFiles: true,
},
},
_isMainReady: vi.fn(() => false),
_isMainSuspended: vi.fn(() => false),
});
expect(hiddenFileSync.isReady()).toBe(false);
});
it("groups plug-in reloads and an Obsidian restart into one finished Notice", async () => {
const noticeGroups = {
setItem: vi.fn(),
finish: vi.fn(() => true),
removeItem: vi.fn(() => true),
};
const plugin = {
app: {
plugins: {
manifests: {
alpha: {
id: "alpha",
name: "Alpha",
dir: ".obsidian/plugins/alpha",
},
beta: {
id: "beta",
name: "Beta",
dir: ".obsidian/plugins/beta",
},
},
enabledPlugins: new Set(["alpha", "beta"]),
unloadPlugin: vi.fn(async () => undefined),
loadPlugin: vi.fn(async () => undefined),
},
},
};
const core = {
confirm: { askInPopup: vi.fn() },
services: {
context: { noticeGroups },
API: { getSystemConfigDir: vi.fn(() => ".obsidian") },
appLifecycle: {
isReloadingScheduled: vi.fn(() => false),
scheduleRestart: vi.fn(),
},
},
};
const hiddenFileSync = Object.create(HiddenFileSync.prototype) as HiddenFileSync;
Object.assign(hiddenFileSync, {
plugin,
core,
queuedNotificationFiles: new Set([".obsidian/plugins/alpha", ".obsidian/plugins/beta", ".obsidian"]),
_log: vi.fn(),
});
hiddenFileSync.notifyConfigChange();
expect(noticeGroups.setItem).toHaveBeenNthCalledWith(1, "hidden-file-changes", "plugin:alpha", {
message: "Files in Alpha were updated.",
action: expect.objectContaining({ label: "Reload Alpha" }),
});
expect(noticeGroups.setItem).toHaveBeenNthCalledWith(2, "hidden-file-changes", "plugin:beta", {
message: "Files in Beta were updated.",
action: expect.objectContaining({ label: "Reload Beta" }),
});
expect(noticeGroups.setItem).toHaveBeenNthCalledWith(3, "hidden-file-changes", "restart", {
message: "Other Obsidian settings files were updated.",
action: expect.objectContaining({ label: "Schedule an Obsidian restart" }),
});
expect(noticeGroups.setItem.mock.calls.every(([groupKey]) => groupKey === "hidden-file-changes")).toBe(true);
expect(noticeGroups.finish).toHaveBeenCalledWith("hidden-file-changes", { durationMs: 20_000 });
expect(core.confirm.askInPopup).not.toHaveBeenCalled();
const reloadAction = (noticeGroups.setItem.mock.calls[0]?.[2] as { action: { onSelect: () => void } }).action
.onSelect;
reloadAction();
await vi.waitFor(() => {
expect(plugin.app.plugins.unloadPlugin).toHaveBeenCalledWith("alpha");
expect(plugin.app.plugins.loadPlugin).toHaveBeenCalledWith("alpha");
expect(noticeGroups.removeItem).toHaveBeenCalledWith("hidden-file-changes", "plugin:alpha");
});
const restartAction = (noticeGroups.setItem.mock.calls[2]?.[2] as { action: { onSelect: () => void } }).action
.onSelect;
restartAction();
expect(core.services.appLifecycle.scheduleRestart).toHaveBeenCalledOnce();
expect(noticeGroups.removeItem).toHaveBeenCalledWith("hidden-file-changes", "restart");
});
it("keeps subordinate initialisation phases below Notice level so one progress Notice owns the scan", async () => {
const progress = {
log: vi.fn(),
once: vi.fn(),
done: vi.fn(),
};
const rebuildMerging = vi.fn(async () => []);
const adoptCurrentStorageFilesAsProcessed = vi.fn(async () => undefined);
const adoptCurrentDatabaseFilesAsProcessed = vi.fn(async () => undefined);
const scanAllStorageChanges = vi.fn(async () => undefined);
const scanAllDatabaseChanges = vi.fn(async () => undefined);
const hiddenFileSync = Object.create(HiddenFileSync.prototype) as HiddenFileSync;
Object.assign(hiddenFileSync, {
_progress: vi.fn(() => progress),
rebuildMerging,
adoptCurrentStorageFilesAsProcessed,
adoptCurrentDatabaseFilesAsProcessed,
scanAllStorageChanges,
scanAllDatabaseChanges,
});
await hiddenFileSync.initialiseInternalFileSync("safe", true);
expect(rebuildMerging).toHaveBeenCalledWith(false, false);
expect(scanAllStorageChanges).toHaveBeenCalledWith(false, true, false);
expect(scanAllDatabaseChanges).toHaveBeenCalledWith(false, true, false);
expect(progress.done).toHaveBeenCalledOnce();
});
it("retirement guard: does not restore separate gathering and restart Notices", async () => {
vi.mocked(configureHiddenFileSyncMode).mockImplementation(async (_mode, handlers) => {
await handlers.enable();
await handlers.initialise("safe");
return "enabled";
});
const events: string[] = [];
const progress = {
log: vi.fn((message: string) => {
events.push(`progress:${message}`);
}),
once: vi.fn(),
done: vi.fn(),
};
const createProgress = vi.fn(() => progress);
const applyPartial = vi.fn(async () => {
events.push("apply-settings");
});
const initialiseInternalFileSync = vi.fn(async () => undefined);
const log = vi.fn();
const hiddenFileSync = Object.create(HiddenFileSync.prototype) as HiddenFileSync;
Object.assign(hiddenFileSync, {
core: {
services: {
setting: { applyPartial },
},
},
initialiseInternalFileSync,
_progress: createProgress,
_log: log,
});
await hiddenFileSync.configureHiddenFileSync("MERGE");
expect(createProgress).toHaveBeenCalledWith("[⚙ Initialise]\n", LOG_LEVEL_NOTICE);
expect(events[0]).toBe("progress:Preparing Hidden File Sync...");
expect(initialiseInternalFileSync).toHaveBeenCalledWith("safe", true, false, progress);
expect(log).not.toHaveBeenCalledWith("Gathering files for enabling Hidden File Sync", LOG_LEVEL_NOTICE);
expect(log).not.toHaveBeenCalledWith("Done! Restarting the app is strongly recommended!", LOG_LEVEL_NOTICE);
expect(log).toHaveBeenCalledWith("Hidden File Sync initialisation completed.", expect.any(Number));
});
it("closes the preparation Notice when enabling Hidden File Sync fails", async () => {
vi.mocked(configureHiddenFileSyncMode).mockImplementation(async (_mode, handlers) => {
await handlers.enable();
return "enabled";
});
const error = new Error("setting persistence failed");
const progress = {
log: vi.fn(),
once: vi.fn(),
done: vi.fn(),
};
const hiddenFileSync = Object.create(HiddenFileSync.prototype) as HiddenFileSync;
Object.assign(hiddenFileSync, {
core: {
services: {
setting: {
applyPartial: vi.fn(async () => {
throw error;
}),
},
},
},
_progress: vi.fn(() => progress),
_log: vi.fn(),
});
await expect(hiddenFileSync.configureHiddenFileSync("MERGE")).rejects.toBe(error);
expect(progress.done).toHaveBeenCalledWith("Failed");
});
});
describe("HiddenFileSync exact revision repair operations", () => {
it("stores the current hidden Vault file as a child of the selected live revision", async () => {
const {
hiddenFileSync,
file,
selected,
databaseFileAccess,
} = createHiddenRevisionOperation();
await expect(
hiddenFileSync.storeInternalFileToDatabaseWithBaseRevision(file, selected._rev!)
).resolves.toBe(true);
expect(databaseFileAccess.storeWithBaseRevision).toHaveBeenCalledWith(
expect.objectContaining({
path: file.path,
body: file.body,
isInternal: true,
}),
selected._rev,
true
);
expect(hiddenFileSync.updateLastProcessed).toHaveBeenCalledWith(
file.path,
expect.objectContaining({ _rev: "3-vault-child" }),
file.stat
);
});
it("refuses to extend a hidden-file revision which is no longer live", async () => {
const {
hiddenFileSync,
file,
selected,
databaseFileAccess,
} = createHiddenRevisionOperation();
databaseFileAccess.getConflictedRevs.mockResolvedValue([]);
await expect(
hiddenFileSync.storeInternalFileToDatabaseWithBaseRevision(file, selected._rev!)
).resolves.toBe(false);
expect(databaseFileAccess.storeWithBaseRevision).not.toHaveBeenCalled();
expect(hiddenFileSync.updateLastProcessed).not.toHaveBeenCalled();
});
it("does not create a hidden-file child when asked only to mark a revision which differs from the Vault", async () => {
const {
hiddenFileSync,
file,
selected,
databaseFileAccess,
} = createHiddenRevisionOperation();
await expect(
hiddenFileSync.storeInternalFileToDatabaseWithBaseRevision(
file,
selected._rev!,
false
)
).resolves.toBe(false);
expect(databaseFileAccess.storeWithBaseRevision).not.toHaveBeenCalled();
expect(hiddenFileSync.updateLastProcessed).not.toHaveBeenCalled();
});
it("marks a matching hidden-file revision without creating a child", async () => {
const {
hiddenFileSync,
file,
selected,
databaseFileAccess,
} = createHiddenRevisionOperation();
databaseFileAccess.fetchEntryFromMeta.mockResolvedValue({
...selected,
data: "{\"value\":\"vault\"}",
});
await expect(
hiddenFileSync.storeInternalFileToDatabaseWithBaseRevision(
file,
selected._rev!,
false
)
).resolves.toBe(true);
expect(databaseFileAccess.storeWithBaseRevision).not.toHaveBeenCalled();
expect(hiddenFileSync.updateLastProcessed).toHaveBeenCalledWith(
file.path,
selected,
file.stat
);
});
it("applies the selected live hidden-file revision through the existing extraction path", async () => {
const {
hiddenFileSync,
path,
selected,
} = createHiddenRevisionOperation();
const extract = vi.fn(async () => true);
hiddenFileSync.extractInternalFileFromDatabase = extract;
await expect(
hiddenFileSync.extractInternalFileRevisionFromDatabase(path, selected._rev!, true)
).resolves.toBe(true);
expect(extract).toHaveBeenCalledWith(path, true, undefined, true, false, true, selected._rev);
});
it("does not apply a hidden-file revision which ceased to be live", async () => {
const {
hiddenFileSync,
path,
selected,
databaseFileAccess,
} = createHiddenRevisionOperation();
databaseFileAccess.getConflictedRevs.mockResolvedValue([]);
await expect(
hiddenFileSync.extractInternalFileRevisionFromDatabase(path, selected._rev!, true)
).resolves.toBe(false);
expect(databaseFileAccess.fetchEntryFromMeta).not.toHaveBeenCalled();
});
});
@@ -0,0 +1,78 @@
import type { FilePath, ObsidianLiveSyncSettings } from "@vrtmrz/livesync-commonlib/compat/common/types";
const HIDDEN_FILE_NOTIFICATION_TASK = "notify-config-change";
const HIDDEN_FILE_NOTIFICATION_DELAY_MS = 1000;
export type HiddenFileSyncChangeNotifierSettings = Pick<ObsidianLiveSyncSettings, "suppressNotifyHiddenFilesChange">;
export type HiddenFileSyncChangeNotifierTaskScheduler = (
key: string,
timeout: number,
operation: () => Promise<unknown> | void
) => void;
export type HiddenFileSyncChangeNotifierDependencies = {
getSettings(): HiddenFileSyncChangeNotifierSettings;
getConfigDir(): string;
scheduleTask: HiddenFileSyncChangeNotifierTaskScheduler;
cancelTask(key: string): void;
showConfigurationChangeNotice(updatedFolders: readonly string[]): void;
hideConfigurationChangeNotice(): void;
};
export type HiddenFileSyncChangeNotifier = {
queueNotification(path: FilePath): void;
/** Compatibility seam used by the real-Obsidian Hidden File Sync fixture. */
showConfigurationChangeNotice(updatedFolders: readonly string[]): void;
dispose(): void;
};
class HiddenFileSyncChangeNotifierOwner implements HiddenFileSyncChangeNotifier {
private readonly queuedNotificationFiles = new Set<string>();
private disposed = false;
constructor(private readonly dependencies: HiddenFileSyncChangeNotifierDependencies) {}
queueNotification(path: FilePath): void {
if (this.disposed) return;
if (this.dependencies.getSettings().suppressNotifyHiddenFilesChange) return;
const configDir = this.dependencies.getConfigDir();
if (!path.startsWith(configDir)) return;
const folder = path.split("/").slice(0, -1).join("/");
this.queuedNotificationFiles.add(folder);
this.dependencies.scheduleTask(HIDDEN_FILE_NOTIFICATION_TASK, HIDDEN_FILE_NOTIFICATION_DELAY_MS, () => {
this.flush();
});
}
showConfigurationChangeNotice(updatedFolders: readonly string[]): void {
this.queuedNotificationFiles.clear();
for (const folder of updatedFolders) {
this.queuedNotificationFiles.add(folder);
}
this.flush();
}
dispose(): void {
if (this.disposed) return;
this.disposed = true;
this.queuedNotificationFiles.clear();
this.dependencies.cancelTask(HIDDEN_FILE_NOTIFICATION_TASK);
this.dependencies.hideConfigurationChangeNotice();
}
private flush(): void {
const updatedFolders = [...this.queuedNotificationFiles];
this.queuedNotificationFiles.clear();
if (this.disposed) return;
this.dependencies.showConfigurationChangeNotice(updatedFolders);
}
}
export function createHiddenFileSyncChangeNotifier(
dependencies: HiddenFileSyncChangeNotifierDependencies
): HiddenFileSyncChangeNotifier {
return new HiddenFileSyncChangeNotifierOwner(dependencies);
}
@@ -0,0 +1,136 @@
import { describe, expect, it, vi } from "vitest";
import type { FilePath } from "@vrtmrz/livesync-commonlib/compat/common/types";
import {
createHiddenFileSyncChangeNotifier,
type HiddenFileSyncChangeNotifierDependencies,
} from "./hiddenFileSyncChangeNotifier.ts";
type ScheduledOperation = {
key: string;
timeout: number;
operation: () => Promise<unknown> | void;
};
function createFixture(overrides: Partial<HiddenFileSyncChangeNotifierDependencies> = {}): {
notifier: ReturnType<typeof createHiddenFileSyncChangeNotifier>;
scheduled: ScheduledOperation[];
settings: { suppressNotifyHiddenFilesChange: boolean };
configDir: { value: string };
showConfigurationChangeNotice: ReturnType<typeof vi.fn>;
hideConfigurationChangeNotice: ReturnType<typeof vi.fn>;
scheduleTask: ReturnType<typeof vi.fn>;
cancelTask: ReturnType<typeof vi.fn>;
} {
const scheduled: ScheduledOperation[] = [];
const settings = { suppressNotifyHiddenFilesChange: false };
const configDir = { value: ".obsidian" };
const showConfigurationChangeNotice = vi.fn();
const hideConfigurationChangeNotice = vi.fn();
const scheduleTask = vi.fn<HiddenFileSyncChangeNotifierDependencies["scheduleTask"]>((key, timeout, operation) => {
scheduled.push({ key, timeout, operation });
});
const cancelTask = vi.fn<HiddenFileSyncChangeNotifierDependencies["cancelTask"]>();
const dependencies: HiddenFileSyncChangeNotifierDependencies = {
getSettings: () => settings,
getConfigDir: () => configDir.value,
scheduleTask,
cancelTask,
showConfigurationChangeNotice,
hideConfigurationChangeNotice,
...overrides,
};
return {
notifier: createHiddenFileSyncChangeNotifier(dependencies),
scheduled,
settings,
configDir,
showConfigurationChangeNotice,
hideConfigurationChangeNotice,
scheduleTask,
cancelTask,
};
}
describe("Hidden File Sync change notifier", () => {
it("queues distinct parent folders and flushes them in insertion order", () => {
const fixture = createFixture();
fixture.notifier.queueNotification(".obsidian/plugins/alpha/data.json" as FilePath);
fixture.notifier.queueNotification(".obsidian/plugins/beta/data.json" as FilePath);
fixture.notifier.queueNotification(".obsidian/plugins/alpha/main.js" as FilePath);
expect(fixture.scheduleTask).toHaveBeenCalledTimes(3);
expect(fixture.scheduleTask).toHaveBeenLastCalledWith("notify-config-change", 1000, expect.any(Function));
fixture.scheduled[fixture.scheduled.length - 1]?.operation();
expect(fixture.showConfigurationChangeNotice).toHaveBeenCalledWith([
".obsidian/plugins/alpha",
".obsidian/plugins/beta",
]);
});
it("uses live suppression and configuration-directory dependencies", () => {
const fixture = createFixture();
fixture.settings.suppressNotifyHiddenFilesChange = true;
fixture.notifier.queueNotification(".obsidian/plugins/suppressed/data.json" as FilePath);
fixture.settings.suppressNotifyHiddenFilesChange = false;
fixture.notifier.queueNotification("other/plugins/outside/data.json" as FilePath);
fixture.configDir.value = "other";
fixture.notifier.queueNotification("other/plugins/inside/data.json" as FilePath);
expect(fixture.scheduled).toHaveLength(1);
fixture.scheduled[0]?.operation();
expect(fixture.showConfigurationChangeNotice).toHaveBeenCalledWith(["other/plugins/inside"]);
});
it("clears the batch before displaying it", () => {
const fixture = createFixture();
fixture.showConfigurationChangeNotice.mockImplementation(() => {
fixture.notifier.queueNotification(".obsidian/plugins/new/data.json" as FilePath);
});
fixture.notifier.queueNotification(".obsidian/plugins/old/data.json" as FilePath);
fixture.scheduled[0]?.operation();
expect(fixture.showConfigurationChangeNotice).toHaveBeenNthCalledWith(1, [".obsidian/plugins/old"]);
fixture.scheduled[1]?.operation();
expect(fixture.showConfigurationChangeNotice).toHaveBeenNthCalledWith(2, [".obsidian/plugins/new"]);
});
it("supports the immediate fixture seam without scheduling another task", () => {
const fixture = createFixture();
fixture.notifier.showConfigurationChangeNotice([
".obsidian/plugins/alpha",
".obsidian/plugins/beta",
".obsidian/plugins/alpha",
]);
expect(fixture.showConfigurationChangeNotice).toHaveBeenCalledWith([
".obsidian/plugins/alpha",
".obsidian/plugins/beta",
]);
expect(fixture.scheduleTask).not.toHaveBeenCalled();
});
it("cancels pending work, hides the Notice, and ignores later work on disposal", () => {
const fixture = createFixture();
fixture.notifier.queueNotification(".obsidian/plugins/example/data.json" as FilePath);
fixture.notifier.dispose();
fixture.notifier.dispose();
fixture.scheduled[0]?.operation();
fixture.notifier.queueNotification(".obsidian/plugins/later/data.json" as FilePath);
expect(fixture.cancelTask).toHaveBeenCalledOnce();
expect(fixture.cancelTask).toHaveBeenCalledWith("notify-config-change");
expect(fixture.hideConfigurationChangeNotice).toHaveBeenCalledOnce();
expect(fixture.showConfigurationChangeNotice).not.toHaveBeenCalled();
expect(fixture.scheduleTask).toHaveBeenCalledOnce();
});
});
@@ -0,0 +1,260 @@
import {
LOG_LEVEL_DEBUG,
LOG_LEVEL_VERBOSE,
type FilePath,
type FilePathWithPrefix,
type LoadedEntry,
type LOG_LEVEL,
type MetaEntry,
type UXFileInfo,
} from "@vrtmrz/livesync-commonlib/compat/common/types";
import type { StorageAccess } from "@vrtmrz/livesync-commonlib/compat/interfaces/StorageAccess";
import type { LogFunction } from "@vrtmrz/livesync-commonlib/compat/services/lib/logUtils";
import { addPrefix } from "@vrtmrz/livesync-commonlib/compat/string_and_binary/path";
import { serialized } from "octagonal-wheels/concurrency/lock";
import { Semaphore } from "octagonal-wheels/concurrency/semaphore";
import { ICHeader } from "@/common/types.ts";
import type { HiddenFileSyncConflictResolution } from "./hiddenFileSyncConflictResolution.ts";
import type { HiddenFileSyncDatabaseExtractionOperations } from "./hiddenFileSyncDatabaseExtractionOperations.ts";
import type { HiddenFileSyncDatabaseWriteOperations } from "./hiddenFileSyncDatabaseWriteOperations.ts";
import type { HiddenFileSyncProcessedState } from "./hiddenFileSyncProcessedState.ts";
import { getHiddenFileSyncComparisonMTime } from "./hiddenFileSyncState.ts";
import { compareMTime, TARGET_IS_NEW } from "@/common/utils.ts";
type HiddenFileSyncStorageChangeAccess = Pick<StorageAccess, "statHidden">;
export type HiddenFileSyncChangeProcessorDependencies = {
storageAccess: HiddenFileSyncStorageChangeAccess;
readFileWithInfo(path: FilePath): Promise<UXFileInfo>;
loadDatabaseMetadata(path: FilePathWithPrefix): Promise<MetaEntry | LoadedEntry | false>;
databaseWriteOperations: Pick<HiddenFileSyncDatabaseWriteOperations, "store" | "delete">;
databaseExtractionOperations: Pick<HiddenFileSyncDatabaseExtractionOperations, "extract">;
processedState: Pick<
HiddenFileSyncProcessedState,
| "fileToStatKey"
| "getLastProcessedFileKey"
| "getLastProcessedFileMTime"
| "updateLastProcessedFile"
| "updateLastProcessed"
>;
conflictResolution: Pick<HiddenFileSyncConflictResolution, "queue">;
log: LogFunction;
publishActivity(eventCount: number, processingCount: number): void;
};
export type HiddenFileSyncDatabaseChangeOptions = Readonly<{
preventDoubleProcess?: boolean;
onlyNew?: boolean;
metaEntry?: MetaEntry | false;
includeDeletion?: boolean;
}>;
export type HiddenFileSyncChangeProcessor = {
processStorageChange(
path: FilePath,
onlyNew?: boolean,
forceWrite?: boolean,
includeDeleted?: boolean
): Promise<boolean | undefined>;
processDatabaseChange(
path: FilePath,
headerLine: string,
options?: HiddenFileSyncDatabaseChangeOptions
): Promise<boolean>;
dispose(): void;
};
class HiddenFileSyncChangeProcessorOwner implements HiddenFileSyncChangeProcessor {
private readonly semaphore = Semaphore(10);
private eventCount = 0;
private processingCount = 0;
private disposed = false;
constructor(private readonly dependencies: HiddenFileSyncChangeProcessorDependencies) {}
async processStorageChange(
path: FilePath,
onlyNew = false,
forceWrite = false,
includeDeleted = true
): Promise<boolean | undefined> {
try {
return await this.serialiseForEvent(path, async () => {
let stat = await this.dependencies.storageAccess.statHidden(path);
// Sometimes a folder is delivered as a file event.
if (stat != null && stat.type != "file") {
return false;
}
const key = await this.dependencies.processedState.fileToStatKey(path, stat);
// A raw event can occur while the file is being read. Scans
// still enumerate every path, but event admission skips this
// exact already-settled key.
const lastKey = this.dependencies.processedState.getLastProcessedFileKey(path);
if (lastKey == key) {
this.log(`${path} Already processed.`, LOG_LEVEL_DEBUG);
return true;
}
// Read the stat and content as one operation. The stat is
// deliberately compared again below: a file can change while
// the first stat is in flight.
const fileInfo = await this.dependencies.readFileWithInfo(path);
const cacheMTime = getHiddenFileSyncComparisonMTime(fileInfo.stat);
const statMtime = getHiddenFileSyncComparisonMTime(stat);
if (cacheMTime != statMtime) {
this.log(`Hidden file:${path} is changed.`, LOG_LEVEL_VERBOSE);
stat = fileInfo.stat;
}
// Compatibility: the storage marker advances before the
// database operation. A later write failure can therefore
// leave this event marked as processed until a scan or state
// change causes it to be reconsidered.
this.dependencies.processedState.updateLastProcessedFile(path, stat!);
const lastIsNotFound = !lastKey || lastKey.endsWith("-0-0");
const nowIsNotFound = fileInfo.deleted;
const type = lastIsNotFound && nowIsNotFound ? "invalid" : nowIsNotFound ? "delete" : "modified";
if (type == "invalid") {
// Maybe the folder was deleted.
return false;
}
const storageMTimeActual = getHiddenFileSyncComparisonMTime(stat);
const storageMTime =
storageMTimeActual == 0
? this.dependencies.processedState.getLastProcessedFileMTime(path)
: storageMTimeActual;
if (onlyNew) {
const prefixedFileName = addPrefix(path, ICHeader);
const fileOnDatabase = await this.dependencies.loadDatabaseMetadata(prefixedFileName);
const databaseMTime = getHiddenFileSyncComparisonMTime(fileOnDatabase, includeDeleted);
const difference = compareMTime(storageMTime, databaseMTime);
if (difference != TARGET_IS_NEW) {
this.log(`Hidden file:${path} is not new.`, LOG_LEVEL_VERBOSE);
// OnlyNew does not handle a deletion. Preserve the
// inherited partial settlement when both values exist.
if (fileOnDatabase && stat) {
this.dependencies.processedState.updateLastProcessed(path, fileOnDatabase, stat);
}
return true;
}
}
if (type == "delete") {
this.log(`Deletion detected: ${path}`);
return await this.dependencies.databaseWriteOperations.delete(path, forceWrite);
}
if (type == "modified") {
this.log(`Modification detected:${path}`, LOG_LEVEL_VERBOSE);
const result = await this.dependencies.databaseWriteOperations.store(fileInfo, forceWrite);
const resultText = result === undefined ? "Nothing changed" : result ? "Updated" : "Failed";
this.log(`${resultText}: ${path} ${resultText}`, LOG_LEVEL_VERBOSE);
return result;
}
return false;
});
} catch (error) {
this.log(`Failed to process hidden file:${path}`);
this.log(error, LOG_LEVEL_VERBOSE);
}
// Could not be processed, but it was this operation's event. Return
// true to prevent a later handler from claiming it.
return true;
}
async processDatabaseChange(
path: FilePath,
headerLine: string,
options: HiddenFileSyncDatabaseChangeOptions = {}
): Promise<boolean> {
const {
preventDoubleProcess = false,
onlyNew = false,
metaEntry = false,
includeDeletion = true,
} = options;
return await this.serialiseForEvent(path, async () => {
try {
const prefixedPath = addPrefix(path, ICHeader);
const docMeta = metaEntry
? metaEntry
: await this.dependencies.loadDatabaseMetadata(prefixedPath);
if (docMeta === false) {
this.log(`${headerLine}: Failed to read detail of ${path}`);
throw new Error(`Failed to read detail ${path}`);
}
if (docMeta._conflicts && docMeta._conflicts.length > 0) {
this.dependencies.conflictResolution.queue(path);
this.log(`${headerLine} Hidden file conflicted, enqueued to resolve`);
return true;
}
const extracted = await this.dependencies.databaseExtractionOperations.extract(path, {
metaEntry: docMeta,
preventDoubleProcess,
onlyNew,
includeDeletion,
});
if (extracted) {
this.log(`${headerLine} Hidden file processed`);
}
} catch (error) {
this.log(`${headerLine} Failed to process hidden file`);
this.log(error, LOG_LEVEL_VERBOSE);
}
// Compatibility: recognition consumes the database event even when
// extraction returned false or threw. A later scan or state change,
// rather than handler fall-through, is responsible for retrying it.
return true;
});
}
dispose(): void {
if (this.disposed) return;
this.disposed = true;
this.eventCount = 0;
this.processingCount = 0;
this.publishActivity();
}
private async serialiseForEvent<Result>(file: FilePath, operation: () => Promise<Result>): Promise<Result> {
this.eventCount++;
this.publishActivity();
const release = await this.semaphore.acquire();
try {
return await serialized(`hidden-file-event:${file}`, async () => {
this.processingCount++;
this.publishActivity();
try {
return await operation();
} finally {
this.processingCount = Math.max(0, this.processingCount - 1);
this.publishActivity();
}
});
} finally {
release();
this.eventCount = Math.max(0, this.eventCount - 1);
this.publishActivity();
}
}
private publishActivity(): void {
this.dependencies.publishActivity(
this.disposed ? 0 : this.eventCount,
this.disposed ? 0 : this.processingCount
);
}
private log(message: unknown, level?: LOG_LEVEL, key?: string): void {
this.dependencies.log(message, level, key);
}
}
export function createHiddenFileSyncChangeProcessor(
dependencies: HiddenFileSyncChangeProcessorDependencies
): HiddenFileSyncChangeProcessor {
return new HiddenFileSyncChangeProcessorOwner(dependencies);
}
@@ -0,0 +1,159 @@
import { describe, expect, it, vi } from "vitest";
import type { FilePath, MetaEntry, UXFileInfo, UXStat } from "@vrtmrz/livesync-commonlib/compat/common/types";
vi.mock("@/deps.ts", () => ({}));
import {
createHiddenFileSyncChangeProcessor,
type HiddenFileSyncChangeProcessorDependencies,
} from "./hiddenFileSyncChangeProcessor.ts";
const path = ".obsidian/app.json" as FilePath;
const stat = { ctime: 1, mtime: 2, size: 3, type: "file" } as UXStat;
function fileInfo(): UXFileInfo {
return {
path,
name: "app.json",
isInternal: true,
deleted: false,
body: new Blob(["{}"]),
stat,
} as UXFileInfo;
}
function metadata(): MetaEntry {
return {
_id: "i:app",
_rev: "2-current",
path: `i:${path}`,
type: "plain",
datatype: "plain",
ctime: 1,
mtime: 2,
size: 3,
children: [],
eden: {},
deleted: false,
} as unknown as MetaEntry;
}
function createDependencies(
overrides: Partial<HiddenFileSyncChangeProcessorDependencies> = {}
): HiddenFileSyncChangeProcessorDependencies {
const state = {
fileToStatKey: vi.fn(async () => "2-3"),
getLastProcessedFileKey: vi.fn(() => undefined),
getLastProcessedFileMTime: vi.fn(() => 0),
databaseStateKey: vi.fn(() => "2-3-2-current--1"),
getLastProcessedDatabaseKey: vi.fn(() => undefined),
updateLastProcessedFile: vi.fn(),
updateLastProcessedDatabase: vi.fn(),
updateLastProcessed: vi.fn(),
};
return {
storageAccess: {
statHidden: vi.fn(async () => stat),
},
readFileWithInfo: vi.fn(async () => fileInfo()),
loadDatabaseMetadata: vi.fn(async () => metadata()),
databaseWriteOperations: {
store: vi.fn(async () => true),
delete: vi.fn(async () => true),
},
databaseExtractionOperations: {
extract: vi.fn(async () => true),
},
processedState: state,
conflictResolution: { queue: vi.fn() },
log: vi.fn(),
publishActivity: vi.fn(),
...overrides,
} as HiddenFileSyncChangeProcessorDependencies;
}
describe("HiddenFileSyncChangeProcessor activity and serialisation", () => {
it("publishes admission, processing, and release transitions", async () => {
const dependencies = createDependencies();
const processor = createHiddenFileSyncChangeProcessor(dependencies);
await expect(processor.processStorageChange(path)).resolves.toBe(true);
const publishActivity = vi.mocked(dependencies.publishActivity);
expect(publishActivity.mock.calls).toEqual([
[1, 0],
[1, 1],
[1, 0],
[0, 0],
]);
processor.dispose();
});
it("serialises same-path storage changes while allowing each event to settle", async () => {
let active = 0;
let maximumActive = 0;
let releaseFirst!: () => void;
const firstStarted = new Promise<void>((resolve) => {
const write = resolve;
releaseFirst = write;
});
const dependencies = createDependencies({
databaseWriteOperations: {
store: vi.fn(async () => {
active++;
maximumActive = Math.max(maximumActive, active);
if (active == 1) await firstStarted;
active--;
return true;
}),
delete: vi.fn(async () => true),
},
});
const processor = createHiddenFileSyncChangeProcessor(dependencies);
const first = processor.processStorageChange(path);
await vi.waitFor(() => expect(dependencies.databaseWriteOperations.store).toHaveBeenCalledOnce());
const second = processor.processStorageChange(path);
await new Promise<void>((resolve) => setTimeout(resolve, 0));
expect(dependencies.databaseWriteOperations.store).toHaveBeenCalledOnce();
releaseFirst();
await expect(first).resolves.toBe(true);
await expect(second).resolves.toBe(true);
expect(maximumActive).toBe(1);
expect(dependencies.databaseWriteOperations.store).toHaveBeenCalledTimes(2);
processor.dispose();
});
});
describe("HiddenFileSyncChangeProcessor compatibility settlement", () => {
it("consumes database events when metadata loading fails", async () => {
const error = new Error("metadata unavailable");
const dependencies = createDependencies({
loadDatabaseMetadata: vi.fn(async () => {
throw error;
}),
});
const processor = createHiddenFileSyncChangeProcessor(dependencies);
await expect(processor.processDatabaseChange(path, "[Replication]")).resolves.toBe(true);
expect(dependencies.log).toHaveBeenCalledWith("[Replication] Failed to process hidden file", undefined, undefined);
expect(dependencies.log).toHaveBeenCalledWith(error, expect.any(Number), undefined);
processor.dispose();
});
it("advances the storage marker before a failed database write", async () => {
const dependencies = createDependencies({
databaseWriteOperations: {
store: vi.fn(async () => false),
delete: vi.fn(async () => true),
},
});
const processor = createHiddenFileSyncChangeProcessor(dependencies);
await expect(processor.processStorageChange(path)).resolves.toBe(false);
expect(dependencies.processedState.updateLastProcessedFile).toHaveBeenCalledWith(path, stat);
expect(dependencies.databaseWriteOperations.store).toHaveBeenCalledOnce();
processor.dispose();
});
});
@@ -0,0 +1,426 @@
import {
LOG_LEVEL_INFO,
LOG_LEVEL_VERBOSE,
type DocumentID,
type FilePath,
type FilePathWithPrefix,
type LoadedEntry,
type LOG_LEVEL,
type MetaEntry,
type UXStat,
} from "@vrtmrz/livesync-commonlib/compat/common/types";
import type { LogFunction } from "@vrtmrz/livesync-commonlib/compat/services/lib/logUtils";
import { isInternalMetadata } from "@vrtmrz/livesync-commonlib/compat/common/typeUtils";
import { stripAllPrefixes } from "@vrtmrz/livesync-commonlib/compat/string_and_binary/path";
import { QueueProcessor } from "octagonal-wheels/concurrency/processor";
import type { InternalFileInfo } from "@/common/types.ts";
import { getHiddenFileSyncComparisonMTime } from "./hiddenFileSyncState.ts";
export type HiddenFileSyncConflictPath = FilePath | FilePathWithPrefix;
export type HiddenFileSyncRevisionInfo = {
rev: string;
status: string;
};
export type HiddenFileSyncRevisionHistory = MetaEntry & {
_revs_info?: HiddenFileSyncRevisionInfo[];
};
export type HiddenFileSyncJsonResolution = {
keepRevision?: string;
mergedText?: string;
};
export type HiddenFileSyncConflictDatabase = {
scanConflictedEntries(): AsyncIterable<MetaEntry>;
getDocumentId(path: HiddenFileSyncConflictPath): Promise<DocumentID>;
loadCurrentMetadata(id: DocumentID): Promise<MetaEntry>;
loadConflictingMetadata(id: DocumentID, revision: string): Promise<MetaEntry>;
loadRevisionHistory(id: DocumentID): Promise<HiddenFileSyncRevisionHistory>;
loadRevisionEntry(path: HiddenFileSyncConflictPath, revision: string): Promise<LoadedEntry | false>;
mergeJson(
path: FilePathWithPrefix,
baseRevision: string,
currentRevision: string,
conflictedRevision: string
): Promise<string | false>;
removeRevision(id: DocumentID, revision: string): Promise<unknown>;
deleteRevision(entry: LoadedEntry): Promise<boolean>;
};
export type HiddenFileSyncConflictStorage = {
ensureDirectory(path: FilePath): Promise<void>;
writeFile(path: FilePath, data: string): Promise<UXStat | null>;
triggerEvent(path: FilePath): Promise<void>;
};
export type HiddenFileSyncConflictReconciliation = {
storeFile(file: InternalFileInfo, forceWrite?: boolean): Promise<boolean | undefined>;
extractFile(path: FilePath): Promise<boolean | undefined>;
};
export type HiddenFileSyncConflictInteraction = {
resolveJsonConflict(
path: FilePath,
docs: [LoadedEntry, LoadedEntry],
apply: (resolution: HiddenFileSyncJsonResolution) => Promise<boolean>
): Promise<boolean>;
};
/** Read-only queue counters retained for the real-Obsidian contract tests. */
export type HiddenFileSyncConflictProcessorTestingView = {
readonly remaining: number;
readonly totalRemaining: number;
readonly nowProcessing: number;
};
/** Focused conflict operations exposed through the Hidden File Sync test view. */
export interface HiddenFileSyncConflictTestingView {
resolveAll(): Promise<void>;
resolveJson(docA: LoadedEntry, docB: LoadedEntry): Promise<boolean>;
readonly pendingPaths: readonly HiddenFileSyncConflictPath[];
readonly processor: HiddenFileSyncConflictProcessorTestingView;
}
export type HiddenFileSyncConflictResolutionDependencies = {
database: HiddenFileSyncConflictDatabase;
storage: HiddenFileSyncConflictStorage;
reconciliation: HiddenFileSyncConflictReconciliation;
interaction: HiddenFileSyncConflictInteraction;
shouldOverwrite(path: FilePath): boolean;
log: LogFunction;
};
export interface HiddenFileSyncConflictResolution {
queue(path: HiddenFileSyncConflictPath): void;
resolveAll(): Promise<void>;
resolveJson(docA: LoadedEntry, docB: LoadedEntry): Promise<boolean>;
dispose(): void;
readonly testing: HiddenFileSyncConflictTestingView;
}
type PendingJsonConflict = {
id: DocumentID;
doc: MetaEntry;
path: HiddenFileSyncConflictPath;
revA: string;
revB: string;
};
export function selectHiddenFileSyncRevisionToDelete(
currentDoc: MetaEntry,
currentRevision: string,
conflictedDoc: MetaEntry,
conflictedRevision: string
): string {
const currentMTime = getHiddenFileSyncComparisonMTime(currentDoc, true);
const conflictedMTime = getHiddenFileSyncComparisonMTime(conflictedDoc, true);
// Compatibility: an equal mtime keeps the current leaf and deletes the
// conflicted leaf. A different tie-breaker would alter existing winners.
return currentMTime < conflictedMTime ? currentRevision : conflictedRevision;
}
export function findHiddenFileSyncMergeBase(
revisions: readonly HiddenFileSyncRevisionInfo[] | undefined,
conflictedRevision: string
): string {
const conflictedGeneration = Number(conflictedRevision.split("-")[0]);
// Compatibility question: this is the first available lower generation
// from the current branch, not a proven nearest shared ancestor. Changing
// it requires a separate conflict-history decision.
return (
revisions?.find(({ rev, status }) => status == "available" && Number(rev.split("-")[0]) < conflictedGeneration)
?.rev ?? ""
);
}
class HiddenFileSyncConflictResolutionOwner implements HiddenFileSyncConflictResolution {
private readonly pendingPaths = new Set<HiddenFileSyncConflictPath>();
private readonly processor: QueueProcessor<HiddenFileSyncConflictPath, PendingJsonConflict>;
private disposed = false;
readonly testing: HiddenFileSyncConflictTestingView;
constructor(private readonly dependencies: HiddenFileSyncConflictResolutionDependencies) {
const interactionProcessor = new QueueProcessor<PendingJsonConflict, void>(
async (results) => {
const { id, doc, path, revA, revB } = results[0];
// Compatibility question: these reads intentionally remain
// outside the catch below. A rejected read can leave the path
// pending until another lifecycle event reconstructs the owner.
const docAMerge = await this.dependencies.database.loadRevisionEntry(path, revA);
const docBMerge = await this.dependencies.database.loadRevisionEntry(path, revB);
try {
if (docAMerge != false && docBMerge != false) {
if (await this.resolveJson(docAMerge, docBMerge)) {
this.requeue(path);
} else {
this.finish(path);
}
return;
}
await this.resolveByNewerEntry(id, path, doc, revA, revB);
} catch (error) {
this.finish(path);
throw error;
}
},
{
suspended: false,
batchSize: 1,
concurrentLimit: 1,
delay: 10,
keepResultUntilDownstreamConnected: false,
yieldThreshold: 10,
}
);
this.processor = new QueueProcessor<HiddenFileSyncConflictPath, PendingJsonConflict>(
async (paths) => await this.processPath(paths[0]),
{
suspended: false,
batchSize: 1,
concurrentLimit: 5,
delay: 10,
keepResultUntilDownstreamConnected: true,
yieldThreshold: 10,
pipeTo: interactionProcessor,
}
);
const pendingPaths = () => [...this.pendingPaths];
const processor = this.processor;
const processorView = Object.freeze({
get remaining() {
return processor.remaining;
},
get totalRemaining() {
return processor.totalRemaining;
},
get nowProcessing() {
return processor.nowProcessing;
},
});
this.testing = Object.freeze({
resolveAll: async () => await this.resolveAll(),
resolveJson: async (docA: LoadedEntry, docB: LoadedEntry) => await this.resolveJson(docA, docB),
get pendingPaths() {
return pendingPaths();
},
processor: processorView,
});
}
queue(path: HiddenFileSyncConflictPath): void {
if (this.disposed) return;
// Compatibility: this deliberately deduplicates exact strings only.
// Prefixed and unprefixed forms of one path can therefore coexist.
if (this.pendingPaths.has(path)) return;
this.pendingPaths.add(path);
// Compatibility question: if QueueProcessor throws during this
// synchronous admission, the pending marker is retained. No current
// caller expects enqueue to throw.
this.processor.enqueue(path);
}
async resolveAll(): Promise<void> {
// Creating the iterator and awaiting the completed pipeline remain
// outside the catch. Only iteration failures are logged and swallowed
// by this operation.
const conflicted = this.dependencies.database.scanConflictedEntries();
// Do not suspend ordinary conflict admission during the scan.
// QueueProcessor v2 can lose its resume event when scan completion
// races with the suspended pump, leaving every admitted path pending.
try {
for await (const doc of conflicted) {
if (!("_conflicts" in doc)) continue;
if (isInternalMetadata(doc._id)) {
this.queue(doc.path);
}
}
} catch (error) {
this.log("something went wrong on resolving all conflicted internal files");
this.log(error, LOG_LEVEL_VERBOSE);
}
await this.processor.waitForAllProcessed();
}
async resolveJson(docA: LoadedEntry, docB: LoadedEntry): Promise<boolean> {
this.log("Opening data-merging dialog", LOG_LEVEL_VERBOSE);
const docs: [LoadedEntry, LoadedEntry] = [docA, docB];
const storageFilePath = stripAllPrefixes(docA.path);
const displayFilename = `${storageFilePath}`;
return await this.dependencies.interaction.resolveJsonConflict(
storageFilePath,
docs,
async ({ keepRevision: keep, mergedText: result }) => {
try {
let needFlush = false;
if (!result && !keep) {
this.log(`Skipped merging: ${displayFilename}`);
return false;
}
// Compatibility question: the selected revision is not
// validated against these two documents. An unknown value
// consequently deletes both revisions without writing a
// merged result. The sequential effects are also not
// transactional, so an earlier deletion survives a later
// failure.
for (const doc of docs) {
if (doc._rev != keep) {
if (await this.dependencies.database.deleteRevision(doc)) {
this.log(`Conflicted revision has been deleted: ${displayFilename}`);
needFlush = true;
}
}
}
if (!keep && result) {
await this.dependencies.storage.ensureDirectory(storageFilePath);
const stat = await this.dependencies.storage.writeFile(storageFilePath, result);
if (!stat) {
throw new Error("Stat failed");
}
const mtime = getHiddenFileSyncComparisonMTime(stat);
// Compatibility: interactive merged text forces the
// database write, whereas automatic merge below uses
// the writer's default admission policy.
await this.dependencies.reconciliation.storeFile(
{
path: storageFilePath,
mtime,
ctime: stat.ctime ?? mtime,
size: stat.size ?? 0,
},
true
);
await this.dependencies.storage.triggerEvent(storageFilePath);
this.log(`STORAGE <-- DB:${displayFilename}: written (hidden,merged)`);
}
if (needFlush) {
if (await this.dependencies.reconciliation.extractFile(storageFilePath)) {
this.log(`STORAGE --> DB:${displayFilename}: extracted (hidden,merged)`);
} else {
this.log(`STORAGE --> DB:${displayFilename}: extracted (hidden,merged) Failed`);
}
}
return true;
} catch (error) {
this.log("Could not merge conflicted json");
this.log(error, LOG_LEVEL_VERBOSE);
return false;
}
}
);
}
dispose(): void {
if (this.disposed) return;
this.disposed = true;
// QueueProcessor termination cascades downstream, but cannot cancel an
// already-running database operation or dialogue callback.
this.processor.terminate();
this.pendingPaths.clear();
}
private async processPath(path: HiddenFileSyncConflictPath): Promise<PendingJsonConflict[]> {
try {
const id = await this.dependencies.database.getDocumentId(path);
const doc = await this.dependencies.database.loadCurrentMetadata(id);
if (doc._conflicts === undefined || doc._conflicts.length == 0) {
this.finish(path);
return [];
}
this.log(`Hidden file conflicted:${path}`);
// Compatibility: sorting mutates the loaded Metadata object before
// it is forwarded to the manual-resolution stage.
const conflicts = doc._conflicts.sort((a, b) => Number(a.split("-")[0]) - Number(b.split("-")[0]));
const revA = doc._rev!;
const revB = conflicts[0];
if (path.endsWith(".json")) {
const revisionHistory = await this.dependencies.database.loadRevisionHistory(id);
const commonBase = findHiddenFileSyncMergeBase(revisionHistory._revs_info, revB);
const result = await this.dependencies.database.mergeJson(doc.path, commonBase, revA, revB);
if (result) {
this.log(`Object merge:${path}`, LOG_LEVEL_INFO);
const filename = stripAllPrefixes(path);
await this.dependencies.storage.ensureDirectory(filename);
const stat = await this.dependencies.storage.writeFile(filename, result);
if (!stat) {
throw new Error(`HiddenFileSyncConflictResolution: Failed to stat file ${filename}`);
}
await this.dependencies.reconciliation.storeFile({ path: filename, ...stat });
// Compatibility question: extraction is attempted before
// the conflicted branch is removed, so its conflict guard
// normally refuses it. Requeueing eventually reflects the
// winner; changing the order needs a separate decision.
await this.dependencies.reconciliation.extractFile(filename);
await this.dependencies.database.removeRevision(id, revB);
this.requeue(path);
return [];
}
this.log(`Object merge is not applicable.`, LOG_LEVEL_VERBOSE);
if (this.dependencies.shouldOverwrite(stripAllPrefixes(path))) {
this.log(`Overwrite rule applied for conflicted hidden file: ${path}`, LOG_LEVEL_INFO);
await this.resolveByNewerEntry(id, path, doc, revA, revB);
return [];
}
return [{ path, revA, revB, id, doc }];
}
await this.resolveByNewerEntry(id, path, doc, revA, revB);
return [];
} catch (error) {
this.finish(path);
this.log(`Failed to resolve conflict (Hidden): ${path}`);
this.log(error, LOG_LEVEL_VERBOSE);
return [];
}
}
private async resolveByNewerEntry(
id: DocumentID,
path: HiddenFileSyncConflictPath,
currentDoc: MetaEntry,
currentRevision: string,
conflictedRevision: string
): Promise<void> {
const conflictedDoc = await this.dependencies.database.loadConflictingMetadata(id, conflictedRevision);
const revisionToDelete = selectHiddenFileSyncRevisionToDelete(
currentDoc,
currentRevision,
conflictedDoc,
conflictedRevision
);
// Compatibility: the database result is ignored. The following conflict
// read, rather than the deletion response, decides settlement.
await this.dependencies.database.removeRevision(id, revisionToDelete);
this.log(`Older one has been deleted:${path}`);
const current = await this.dependencies.database.loadCurrentMetadata(id);
if (current._conflicts?.length === 0) {
await this.dependencies.reconciliation.extractFile(stripAllPrefixes(path));
this.finish(path);
} else {
// Compatibility: an absent _conflicts field is not considered
// settled here, although the main path treats it as conflict-free.
this.requeue(path);
}
}
private finish(path: HiddenFileSyncConflictPath): void {
this.pendingPaths.delete(path);
}
private requeue(path: HiddenFileSyncConflictPath): void {
this.finish(path);
this.queue(path);
}
private log(message: unknown, level?: LOG_LEVEL, key?: string): void {
this.dependencies.log(message, level, key);
}
}
export function createHiddenFileSyncConflictResolution(
dependencies: HiddenFileSyncConflictResolutionDependencies
): HiddenFileSyncConflictResolution {
return new HiddenFileSyncConflictResolutionOwner(dependencies);
}
@@ -0,0 +1,422 @@
import { describe, expect, it, vi } from "vitest";
import {
LOG_LEVEL_VERBOSE,
type DocumentID,
type FilePath,
type FilePathWithPrefix,
type LoadedEntry,
type MetaEntry,
type UXStat,
} from "@vrtmrz/livesync-commonlib/compat/common/types";
import {
createHiddenFileSyncConflictResolution,
findHiddenFileSyncMergeBase,
selectHiddenFileSyncRevisionToDelete,
type HiddenFileSyncConflictDatabase,
type HiddenFileSyncConflictInteraction,
type HiddenFileSyncConflictReconciliation,
type HiddenFileSyncConflictResolutionDependencies,
type HiddenFileSyncConflictStorage,
type HiddenFileSyncJsonResolution,
type HiddenFileSyncRevisionHistory,
} from "./hiddenFileSyncConflictResolution.ts";
const path = ".obsidian/plugins/example/data.json" as FilePath;
const prefixedPath = `i:${path}` as FilePathWithPrefix;
const id = "i:hidden-entry-id" as DocumentID;
function metadata(
revision: string,
mtime: number,
overrides: Partial<HiddenFileSyncRevisionHistory> = {}
): HiddenFileSyncRevisionHistory {
return {
_id: id,
_rev: revision,
path: prefixedPath,
type: "plain",
datatype: "plain",
ctime: 10,
mtime,
size: 20,
children: [],
eden: {},
deleted: false,
...overrides,
} as unknown as MetaEntry;
}
function loadedEntry(revision: string, content: string): LoadedEntry {
return {
...metadata(revision, 20),
data: content,
} as LoadedEntry;
}
function entries(...values: MetaEntry[]): AsyncIterable<MetaEntry> {
return {
async *[Symbol.asyncIterator]() {
yield* values;
},
};
}
type DependencyOverrides = {
database?: Partial<HiddenFileSyncConflictDatabase>;
storage?: Partial<HiddenFileSyncConflictStorage>;
reconciliation?: Partial<HiddenFileSyncConflictReconciliation>;
interaction?: Partial<HiddenFileSyncConflictInteraction>;
shouldOverwrite?: HiddenFileSyncConflictResolutionDependencies["shouldOverwrite"];
log?: HiddenFileSyncConflictResolutionDependencies["log"];
};
function createDependencies(overrides: DependencyOverrides = {}): HiddenFileSyncConflictResolutionDependencies {
const database: HiddenFileSyncConflictDatabase = {
scanConflictedEntries: () => entries(),
getDocumentId: vi.fn(async () => id),
loadCurrentMetadata: vi.fn(async () => metadata("1-current", 10)),
loadConflictingMetadata: vi.fn(async () => metadata("1-conflict", 10)),
loadRevisionHistory: vi.fn(async () => metadata("1-current", 10, { _revs_info: [] })),
loadRevisionEntry: vi.fn(async (): Promise<LoadedEntry | false> => false),
mergeJson: vi.fn(async (): Promise<string | false> => false),
removeRevision: vi.fn(async () => true),
deleteRevision: vi.fn(async () => true),
...overrides.database,
};
const storage: HiddenFileSyncConflictStorage = {
ensureDirectory: vi.fn(async () => undefined),
writeFile: vi.fn(async () => null),
triggerEvent: vi.fn(async () => undefined),
...overrides.storage,
};
const reconciliation: HiddenFileSyncConflictReconciliation = {
storeFile: vi.fn(async () => true),
extractFile: vi.fn(async () => true),
...overrides.reconciliation,
};
const interaction: HiddenFileSyncConflictInteraction = {
resolveJsonConflict: vi.fn(async () => false),
...overrides.interaction,
};
return {
database,
storage,
reconciliation,
interaction,
shouldOverwrite: overrides.shouldOverwrite ?? (() => false),
log: overrides.log ?? vi.fn(),
};
}
describe("Hidden File Sync conflict policy", () => {
it("keeps the current revision when both mtimes are equal", () => {
const current = metadata("3-current", 20);
const conflicted = metadata("2-conflict", 20);
expect(selectHiddenFileSyncRevisionToDelete(current, current._rev!, conflicted, conflicted._rev!)).toBe(
conflicted._rev
);
});
it("selects the first available lower-generation revision as the merge base", () => {
expect(
findHiddenFileSyncMergeBase(
[
{ rev: "4-current", status: "available" },
{ rev: "3-missing", status: "missing" },
{ rev: "2-base", status: "available" },
{ rev: "1-older", status: "available" },
],
"3-conflict"
)
).toBe("2-base");
expect(findHiddenFileSyncMergeBase(undefined, "3-conflict")).toBe("");
});
});
describe("Hidden File Sync conflict queue", () => {
it("continues processing queued conflict notifications while a full scan is in progress", async () => {
let releaseScan!: () => void;
let markScanStarted!: () => void;
const scanGate = new Promise<void>((resolve) => {
releaseScan = resolve;
});
const scanStarted = new Promise<void>((resolve) => {
markScanStarted = resolve;
});
const loadCurrentMetadata = vi.fn(async () => metadata("1-current", 10));
const dependencies = createDependencies({
database: {
scanConflictedEntries: () => ({
async *[Symbol.asyncIterator]() {
markScanStarted();
await scanGate;
},
}),
loadCurrentMetadata,
},
});
const resolution = createHiddenFileSyncConflictResolution(dependencies);
const resolvingAll = resolution.resolveAll();
await scanStarted;
// Cross the macrotask boundary which allowed the legacy suspended
// processor to stop before a database notification arrived.
await new Promise<void>((resolve) => setTimeout(resolve, 0));
resolution.queue(prefixedPath);
try {
await vi.waitFor(() => expect(loadCurrentMetadata).toHaveBeenCalledOnce(), {
interval: 10,
timeout: 250,
});
} finally {
releaseScan();
await resolvingAll;
resolution.dispose();
}
});
it("deduplicates exact paths and does not accept work after disposal", async () => {
const loadCurrentMetadata = vi.fn(async () => metadata("1-current", 10));
const dependencies = createDependencies({ database: { loadCurrentMetadata } });
const resolution = createHiddenFileSyncConflictResolution(dependencies);
resolution.queue(prefixedPath);
resolution.queue(prefixedPath);
await resolution.resolveAll();
expect(loadCurrentMetadata).toHaveBeenCalledOnce();
resolution.dispose();
resolution.queue(`i:${path}.other` as FilePathWithPrefix);
expect(loadCurrentMetadata).toHaveBeenCalledOnce();
});
it("retains prefixed and unprefixed path forms as separate compatibility keys", async () => {
const loadCurrentMetadata = vi.fn(async () => metadata("1-current", 10));
const dependencies = createDependencies({ database: { loadCurrentMetadata } });
const resolution = createHiddenFileSyncConflictResolution(dependencies);
resolution.queue(path);
resolution.queue(prefixedPath);
await resolution.resolveAll();
expect(loadCurrentMetadata).toHaveBeenCalledTimes(2);
resolution.dispose();
});
it("deletes the conflicted revision on an mtime tie, then extracts", async () => {
const events: string[] = [];
const current = metadata("3-current", 20, { _conflicts: ["2-conflict"] });
const settled = metadata("3-current", 20, { _conflicts: [] });
const dependencies = createDependencies({
database: {
scanConflictedEntries: () => entries(current),
loadCurrentMetadata: vi.fn().mockResolvedValueOnce(current).mockResolvedValueOnce(settled),
loadConflictingMetadata: vi.fn(async () => metadata("2-conflict", 20)),
removeRevision: vi.fn(async (_id, revision) => {
events.push(`remove:${revision}`);
return true;
}),
},
reconciliation: {
extractFile: vi.fn(async () => {
events.push("extract");
return true;
}),
},
});
const resolution = createHiddenFileSyncConflictResolution(dependencies);
await resolution.resolveAll();
expect(events).toEqual(["remove:2-conflict", "extract"]);
resolution.dispose();
});
it("stores and extracts an automatic merge before removing the conflicted revision", async () => {
const events: string[] = [];
const current = metadata("3-current", 30, { _conflicts: ["2-conflict"] });
const settled = metadata("4-merged", 40, { _conflicts: [] });
const stat = { ctime: 10, mtime: 20, size: 30, type: "file" } as UXStat;
const mergeJson = vi.fn(async () => '{"merged":true}');
const dependencies = createDependencies({
database: {
scanConflictedEntries: () => entries(current),
loadCurrentMetadata: vi.fn().mockResolvedValueOnce(current).mockResolvedValueOnce(settled),
loadRevisionHistory: vi.fn(async () =>
metadata("3-current", 30, {
_revs_info: [
{ rev: "3-current", status: "available" },
{ rev: "1-base", status: "available" },
],
})
),
mergeJson,
removeRevision: vi.fn(async () => {
events.push("remove");
return true;
}),
},
storage: {
ensureDirectory: vi.fn(async () => {
events.push("ensure");
}),
writeFile: vi.fn(async () => {
events.push("write");
return stat;
}),
},
reconciliation: {
storeFile: vi.fn(async () => {
events.push("store");
return true;
}),
extractFile: vi.fn(async () => {
events.push("extract");
return false;
}),
},
});
const resolution = createHiddenFileSyncConflictResolution(dependencies);
await resolution.resolveAll();
expect(mergeJson).toHaveBeenCalledWith(prefixedPath, "1-base", "3-current", "2-conflict");
expect(events).toEqual(["ensure", "write", "store", "extract", "remove"]);
resolution.dispose();
});
});
describe("Hidden File Sync JSON conflict application", () => {
function createJsonResolutionFixture(
jsonResolution: HiddenFileSyncJsonResolution,
deletionResult: boolean | Error = true
) {
const events: string[] = [];
const docA = loadedEntry("3-current", '{"current":true}');
const docB = loadedEntry("2-conflict", '{"conflict":true}');
const deleteRevision = vi.fn(async (entry: LoadedEntry) => {
events.push(`delete:${entry._rev}`);
if (deletionResult instanceof Error) {
if (entry._rev === docB._rev) throw deletionResult;
return true;
}
return deletionResult;
});
const extractFile = vi.fn(async () => {
events.push("extract");
return false;
});
const storeFile = vi.fn(async () => {
events.push("store");
return true;
});
const stat = { ctime: 11, mtime: 21, size: 22, type: "file" } as UXStat;
const log = vi.fn();
const dependencies = createDependencies({
database: { deleteRevision },
storage: {
ensureDirectory: vi.fn(async () => {
events.push("ensure");
}),
writeFile: vi.fn(async () => {
events.push("write");
return stat;
}),
triggerEvent: vi.fn(async () => {
events.push("trigger");
}),
},
reconciliation: { extractFile, storeFile },
interaction: {
resolveJsonConflict: vi.fn(async (_path, _docs, apply) => await apply(jsonResolution)),
},
log,
});
return {
deleteRevision,
docA,
docB,
events,
extractFile,
log,
resolution: createHiddenFileSyncConflictResolution(dependencies),
stat,
storeFile,
};
}
it("returns false without changing data when no resolution is selected", async () => {
const fixture = createJsonResolutionFixture({});
await expect(fixture.resolution.resolveJson(fixture.docA, fixture.docB)).resolves.toBe(false);
expect(fixture.events).toEqual([]);
fixture.resolution.dispose();
});
it("keeps the selected revision but reports success when follow-up extraction fails", async () => {
const fixture = createJsonResolutionFixture({ keepRevision: "3-current" });
await expect(fixture.resolution.resolveJson(fixture.docA, fixture.docB)).resolves.toBe(true);
expect(fixture.deleteRevision).toHaveBeenCalledTimes(1);
expect(fixture.deleteRevision).toHaveBeenCalledWith(fixture.docB);
expect(fixture.events).toEqual([`delete:${fixture.docB._rev}`, "extract"]);
expect(fixture.log).toHaveBeenCalledWith(
`STORAGE --> DB:${path}: extracted (hidden,merged) Failed`,
undefined,
undefined
);
fixture.resolution.dispose();
});
it("deletes both supplied revisions when the selected revision is unknown", async () => {
const fixture = createJsonResolutionFixture({ keepRevision: "9-unknown" });
await expect(fixture.resolution.resolveJson(fixture.docA, fixture.docB)).resolves.toBe(true);
expect(fixture.events).toEqual([`delete:${fixture.docA._rev}`, `delete:${fixture.docB._rev}`, "extract"]);
expect(fixture.storeFile).not.toHaveBeenCalled();
fixture.resolution.dispose();
});
it("deletes both revisions before writing and storing a merged result", async () => {
const fixture = createJsonResolutionFixture({ mergedText: '{"merged":true}' });
await expect(fixture.resolution.resolveJson(fixture.docA, fixture.docB)).resolves.toBe(true);
expect(fixture.storeFile).toHaveBeenCalledWith(
{
path,
ctime: fixture.stat.ctime,
mtime: fixture.stat.mtime,
size: fixture.stat.size,
},
true
);
expect(fixture.events).toEqual([
`delete:${fixture.docA._rev}`,
`delete:${fixture.docB._rev}`,
"ensure",
"write",
"store",
"trigger",
"extract",
]);
fixture.resolution.dispose();
});
it("keeps an earlier successful deletion when a later deletion throws", async () => {
const error = new Error("second deletion failed");
const fixture = createJsonResolutionFixture({ mergedText: '{"merged":true}' }, error);
await expect(fixture.resolution.resolveJson(fixture.docA, fixture.docB)).resolves.toBe(false);
expect(fixture.events).toEqual([`delete:${fixture.docA._rev}`, `delete:${fixture.docB._rev}`]);
expect(fixture.storeFile).not.toHaveBeenCalled();
expect(fixture.log).toHaveBeenCalledWith(error, LOG_LEVEL_VERBOSE, undefined);
fixture.resolution.dispose();
});
});
@@ -0,0 +1,129 @@
import { describe, expect, it, vi } from "vitest";
import {
type DocumentID,
type FilePath,
type FilePathWithPrefix,
type LoadedEntry,
type MetaEntry,
type UXStat,
} from "@vrtmrz/livesync-commonlib/compat/common/types";
import { ICHeader, ICHeaderEnd } from "@/common/types.ts";
vi.mock("@/deps.ts", () => ({}));
vi.mock("./configureHiddenFileSyncMode.ts", () => ({
configureHiddenFileSyncMode: vi.fn(),
}));
import { HiddenFileSyncContext } from "./hiddenFileSyncContext.ts";
describe("HiddenFileSyncContext operation composition", () => {
it("composes the conflict owner from the current database and path capabilities", async () => {
const path = ".obsidian/app.json" as FilePath;
const prefixedPath = `i:${path}` as FilePathWithPrefix;
const metadata = {
_id: "i:hidden-entry-id" as DocumentID,
_rev: "2-current",
path: prefixedPath,
type: "plain",
datatype: "plain",
ctime: 10,
mtime: 20,
size: 20,
children: [],
eden: {},
deleted: false,
_conflicts: [],
} as unknown as MetaEntry;
const findEntries = vi.fn(() => ({
async *[Symbol.asyncIterator]() {
yield metadata;
},
}));
const getRaw = vi.fn(async () => metadata);
const path2id = vi.fn(async () => metadata._id);
const periodicProcessor = { enable: vi.fn(), disable: vi.fn() };
const context = new HiddenFileSyncContext({
createPeriodicProcessor: vi.fn(() => periodicProcessor),
getLocalDatabase: () => ({ findEntries, getRaw }),
path: { path2id },
log: vi.fn(),
publishActivity: vi.fn(),
closeJsonConflictDialogs: vi.fn(),
hideConfigurationChangeNotice: vi.fn(),
} as never);
await context.testing.conflictResolution.resolveAll();
expect(findEntries).toHaveBeenCalledWith(ICHeader, ICHeaderEnd, { conflicts: true });
expect(path2id).toHaveBeenCalledWith(prefixedPath, ICHeader);
expect(getRaw).toHaveBeenCalledWith(metadata._id, { conflicts: true });
context.dispose();
});
it("applies a selected live revision through the narrow repair view", async () => {
const path = ".obsidian/plugins/example/data.json" as FilePath;
const prefixedPath = `i:${path}` as FilePathWithPrefix;
const revision = "2-selected";
const metadata = {
_id: "hidden-entry-id" as DocumentID,
_rev: revision,
path: prefixedPath,
type: "plain",
datatype: "plain",
ctime: 10,
mtime: 20,
size: 20,
children: [],
eden: {},
deleted: false,
} as unknown as MetaEntry;
const loaded = {
...metadata,
data: '{"value":"database"}',
} as LoadedEntry;
const stat = { ctime: 10, mtime: 20, size: 20, type: "file" } as UXStat;
const statHidden = vi.fn<() => Promise<UXStat | null>>().mockResolvedValueOnce(null).mockResolvedValue(stat);
const writeHiddenFileAuto = vi.fn(async () => true);
const getDBEntryFromMeta = vi.fn(async () => loaded);
const fetchEntryMeta = vi.fn(async () => metadata);
const getConflictedRevs = vi.fn(async () => [] as string[]);
const markChangesAreSame = vi.fn();
const periodicProcessor = { enable: vi.fn(), disable: vi.fn() };
const context = new HiddenFileSyncContext({
createPeriodicProcessor: vi.fn(() => periodicProcessor),
isIgnoredByIgnoreFile: vi.fn(async () => false),
databaseFileAccess: {
fetchEntryMeta,
getConflictedRevs,
},
getLocalDatabase: () => ({ getDBEntryFromMeta }),
storageAccess: {
statHidden,
isExistsIncludeHidden: vi.fn(async () => false),
ensureDir: vi.fn(async () => true),
writeHiddenFileAuto,
},
path: {
markChangesAreSame,
unmarkChanges: vi.fn(),
},
getSettings: () => ({ suppressNotifyHiddenFilesChange: true }),
log: vi.fn(),
publishActivity: vi.fn(),
closeJsonConflictDialogs: vi.fn(),
hideConfigurationChangeNotice: vi.fn(),
} as never);
await expect(context.repair.extractInternalFileRevisionFromDatabase(path, revision, true)).resolves.toBe(true);
expect(fetchEntryMeta).toHaveBeenCalledWith(prefixedPath, revision, true);
expect(getConflictedRevs).toHaveBeenCalledWith(prefixedPath);
expect(getDBEntryFromMeta).toHaveBeenCalledWith(metadata, false, true);
expect(writeHiddenFileAuto).toHaveBeenCalledWith(path, '{"value":"database"}', {
ctime: metadata.ctime,
mtime: metadata.mtime,
});
expect(markChangesAreSame).toHaveBeenCalledWith(path, metadata.mtime, stat.mtime);
context.dispose();
});
});
@@ -0,0 +1,84 @@
import { describe, expect, it, vi } from "vitest";
vi.mock("@/deps.ts", () => ({}));
vi.mock("./configureHiddenFileSyncMode.ts", () => ({
configureHiddenFileSyncMode: vi.fn(),
}));
import { HiddenFileSyncContext } from "./hiddenFileSyncContext.ts";
function getProcessedState(context: HiddenFileSyncContext): unknown {
return (context as unknown as { readonly processedState: unknown }).processedState;
}
function getPrivate<T>(context: HiddenFileSyncContext, key: string): T {
return (context as unknown as Record<string, T>)[key];
}
function createContext(processedFiles = new Map()) {
const periodicProcessor = { enable: vi.fn(), disable: vi.fn() };
const publishActivity = vi.fn();
const hideConfigurationChangeNotice = vi.fn();
const keyValueDatabase = {
get: vi.fn(async (key: IDBValidKey) => {
if (key == "hidden-file-lastProcessed") return processedFiles;
return new Map();
}),
};
const context = new HiddenFileSyncContext({
createPeriodicProcessor: vi.fn(() => periodicProcessor),
getKeyValueDatabase: () => keyValueDatabase,
getSettings: () => ({ syncInternalFiles: true }),
log: vi.fn(),
publishActivity,
closeJsonConflictDialogs: vi.fn(),
hideConfigurationChangeNotice,
} as never);
return { context, hideConfigurationChangeNotice, periodicProcessor, publishActivity };
}
describe("HiddenFileSyncContext ownership and start-up lifecycle", () => {
it("creates each stateful capability owner per context instance", () => {
const first = createContext();
const second = createContext();
expect(getPrivate<unknown>(first.context, "pathAdmission")).not.toBe(
getPrivate<unknown>(second.context, "pathAdmission")
);
expect(getPrivate<unknown>(first.context, "changeNotifier")).not.toBe(
getPrivate<unknown>(second.context, "changeNotifier")
);
expect(first.context.testing.conflictResolution).not.toBe(second.context.testing.conflictResolution);
expect(getProcessedState(first.context)).not.toBe(getProcessedState(second.context));
expect(getPrivate<unknown>(first.context, "changeProcessor")).not.toBe(
getPrivate<unknown>(second.context, "changeProcessor")
);
expect(getPrivate<unknown>(first.context, "reconciliation")).not.toBe(
getPrivate<unknown>(second.context, "reconciliation")
);
expect(getPrivate<unknown>(first.context, "periodicInternalFileScanProcessor")).toBe(first.periodicProcessor);
expect(getPrivate<unknown>(second.context, "periodicInternalFileScanProcessor")).toBe(second.periodicProcessor);
first.context.dispose();
second.context.dispose();
});
it.each([
[new Map(), true],
[new Map([[".obsidian/app.json", "1-2-3"]]), false],
])(
"preserves start-up scan notice selection for the processed-file cache",
async (processedFiles, forcedNotice) => {
const { context } = createContext(processedFiles);
const reconciliation = getPrivate<{
applyOfflineChanges(showNotice: boolean): Promise<unknown>;
}>(context, "reconciliation");
const applyOfflineChanges = vi.spyOn(reconciliation, "applyOfflineChanges").mockResolvedValue(undefined);
await context.serviceHandlers.onDatabaseInitialised(false);
expect(applyOfflineChanges).toHaveBeenCalledWith(forcedNotice);
context.dispose();
}
);
});
@@ -0,0 +1,600 @@
import {
type AnyEntry,
type LoadedEntry,
type FilePathWithPrefix,
type FilePath,
LOG_LEVEL_INFO,
LOG_LEVEL_NOTICE,
LOG_LEVEL_VERBOSE,
type MetaEntry,
type ObsidianLiveSyncSettings,
type LOG_LEVEL,
} from "@vrtmrz/livesync-commonlib/compat/common/types";
import { ICHeader, ICHeaderEnd } from "@/common/types.ts";
import { type CustomRegExp } from "@vrtmrz/livesync-commonlib/compat/common/utils";
import { isInternalMetadata, cancelTask, scheduleTask } from "@/common/utils.ts";
import { serialized } from "octagonal-wheels/concurrency/lock";
import { addPrefix, stripAllPrefixes } from "@vrtmrz/livesync-commonlib/compat/string_and_binary/path";
import { configureHiddenFileSyncMode, type ConfigureHiddenFileSyncResult } from "./configureHiddenFileSyncMode.ts";
import type { OptionalSyncFeatureMode } from "@/features/optionalSyncFeatures.ts";
import { $msg } from "@/common/translation";
import type { LiveSyncLocalDB } from "@vrtmrz/livesync-commonlib/compat/pouchdb/LiveSyncLocalDB";
import type { DatabaseFileAccess } from "@vrtmrz/livesync-commonlib/compat/interfaces/DatabaseFileAccess";
import type { KeyValueDatabase } from "@vrtmrz/livesync-commonlib/compat/interfaces/KeyValueDatabase";
import type { IPathService } from "@vrtmrz/livesync-commonlib/compat/services/base/IService";
import {
createHiddenFileSyncRepairView,
createHiddenFileSyncServiceHandlerView,
createHiddenFileSyncTestingView,
type HiddenFileSyncCommandView,
type HiddenFileSyncRepairView,
type HiddenFileSyncServiceHandlerView,
type HiddenFileSyncTestingView,
} from "./hiddenFileSyncViews.ts";
import type { OptionalFileSyncFileTreeDependencies } from "@/features/optionalFileSyncFileTree.ts";
import {
deleteHiddenFileFromStorage,
ensureHiddenFileDirectory,
readHiddenFileWithInfo,
triggerHiddenFileEvent,
writeHiddenFile,
writeHiddenFileFromDatabase,
type HiddenFileSyncStorageDependencies,
} from "./hiddenFileSyncStorage.ts";
import { loadHiddenFileSyncBaseEntry, loadLiveHiddenFileSyncRevision } from "./hiddenFileSyncDatabaseLoaders.ts";
import {
createHiddenFileSyncDatabaseWriteOperations,
type HiddenFileSyncDatabaseWriteOperations,
} from "./hiddenFileSyncDatabaseWriteOperations.ts";
import {
createHiddenFileSyncDatabaseExtractionOperations,
type HiddenFileSyncDatabaseExtractionOperations,
} from "./hiddenFileSyncDatabaseExtractionOperations.ts";
import {
createHiddenFileSyncConflictResolution,
type HiddenFileSyncConflictResolution,
type HiddenFileSyncJsonResolution,
} from "./hiddenFileSyncConflictResolution.ts";
import {
createHiddenFileSyncProcessedState,
type HiddenFileSyncProcessedState,
} from "./hiddenFileSyncProcessedState.ts";
import {
createHiddenFileSyncChangeProcessor,
type HiddenFileSyncChangeProcessor,
} from "./hiddenFileSyncChangeProcessor.ts";
import { createHiddenFileSyncPathAdmission, type HiddenFileSyncPathAdmission } from "./hiddenFileSyncPathAdmission.ts";
import {
createHiddenFileSyncChangeNotifier,
type HiddenFileSyncChangeNotifier,
} from "./hiddenFileSyncChangeNotifier.ts";
import {
createReconciliation,
type InitialisationDirection,
type ReconciliationProgress,
type Reconciliation,
} from "./reconciliation.ts";
export type { ReconciliationProgress as HiddenFileSyncProgress } from "./reconciliation.ts";
type SyncDirection = InitialisationDirection;
type HiddenFileSyncSettings = Pick<
ObsidianLiveSyncSettings,
| "syncInternalFiles"
| "syncInternalFilesBeforeReplication"
| "watchInternalFileChanges"
| "useAdvancedMode"
| "syncInternalFilesInterval"
| "syncInternalFileOverwritePatterns"
| "syncInternalFilesTargetPatterns"
| "syncInternalFilesIgnorePatterns"
| "suppressNotifyHiddenFilesChange"
>;
type HiddenFileSyncDatabase = Pick<
LiveSyncLocalDB,
| "allDocsRaw"
| "deleteDBEntry"
| "findEntries"
| "getDBEntry"
| "getDBEntryFromMeta"
| "getDBEntryMeta"
| "getRaw"
| "putDBEntry"
| "putRaw"
| "removeRevision"
> & {
readonly managers: {
readonly conflictManager: Pick<LiveSyncLocalDB["managers"]["conflictManager"], "mergeObject">;
};
};
type HiddenFileSyncDatabaseFileAccess = Pick<
DatabaseFileAccess,
"fetchEntryFromMeta" | "fetchEntryMeta" | "getConflictedRevs" | "storeWithBaseRevision"
>;
export type HiddenFileSyncPeriodicProcessor = {
enable(interval: number): void;
disable(): void;
};
export type HiddenFileSyncContextDependencies = OptionalFileSyncFileTreeDependencies &
HiddenFileSyncStorageDependencies & {
getSettings(): HiddenFileSyncSettings;
getLocalDatabase(): HiddenFileSyncDatabase;
getKeyValueDatabase(): KeyValueDatabase;
databaseFileAccess: HiddenFileSyncDatabaseFileAccess;
path: Pick<IPathService, "getPath" | "markChangesAreSame" | "path2id" | "unmarkChanges">;
createProgress(prefix?: string, level?: LOG_LEVEL): ReconciliationProgress;
createPeriodicProcessor(process: () => Promise<unknown>): HiddenFileSyncPeriodicProcessor;
isReady(): boolean;
isSuspended(): boolean;
isDatabaseReady(): boolean;
isIgnoredByIgnoreFile(path: string): Promise<boolean>;
getConfigDir(): string;
getRootPath(): string;
getFileRegExp(
key:
| "syncInternalFileOverwritePatterns"
| "syncInternalFilesIgnorePatterns"
| "syncInternalFilesTargetPatterns"
): CustomRegExp[];
applySettings(partial: Partial<ObsidianLiveSyncSettings>, saveImmediately?: boolean): Promise<void>;
setSyncInternalFilesEnabled(enabled: boolean): void;
resolveJsonConflict(
path: FilePath,
docs: [LoadedEntry, LoadedEntry],
apply: (resolution: HiddenFileSyncJsonResolution) => Promise<boolean>
): Promise<boolean>;
showConfigurationChangeNotice(updatedFolders: readonly string[]): void;
hideConfigurationChangeNotice(): void;
closeJsonConflictDialogs(): void;
publishActivity(eventCount: number, processingCount: number): void;
ownsLocalFile(path: FilePath): boolean;
};
export class HiddenFileSyncContext implements HiddenFileSyncCommandView {
private readonly dependencies: HiddenFileSyncContextDependencies;
private readonly processedState: HiddenFileSyncProcessedState;
private readonly databaseWriteOperations: HiddenFileSyncDatabaseWriteOperations;
private readonly databaseExtractionOperations: HiddenFileSyncDatabaseExtractionOperations;
private readonly conflictResolution: HiddenFileSyncConflictResolution;
private readonly changeProcessor: HiddenFileSyncChangeProcessor;
private readonly reconciliation: Reconciliation;
private readonly pathAdmission: HiddenFileSyncPathAdmission;
private readonly changeNotifier: HiddenFileSyncChangeNotifier;
readonly serviceHandlers: HiddenFileSyncServiceHandlerView;
readonly testing: HiddenFileSyncTestingView;
readonly repair: HiddenFileSyncRepairView;
private readonly periodicInternalFileScanProcessor: HiddenFileSyncPeriodicProcessor;
private disposed = false;
constructor(dependencies: HiddenFileSyncContextDependencies) {
this.dependencies = dependencies;
this.pathAdmission = createHiddenFileSyncPathAdmission({
getTargetPatternSource: () => dependencies.getSettings().syncInternalFilesTargetPatterns,
getIgnorePatternSource: () => dependencies.getSettings().syncInternalFilesIgnorePatterns,
getFileRegExp: (key) => dependencies.getFileRegExp(key),
isIgnoredByIgnoreFile: async (path) => await dependencies.isIgnoredByIgnoreFile(path),
ownsLocalFile: (path) => dependencies.ownsLocalFile(path),
});
this.changeNotifier = createHiddenFileSyncChangeNotifier({
getSettings: () => dependencies.getSettings(),
getConfigDir: () => dependencies.getConfigDir(),
scheduleTask: (key, timeout, operation) => scheduleTask(key, timeout, operation),
cancelTask: (key) => cancelTask(key),
showConfigurationChangeNotice: (updatedFolders) =>
dependencies.showConfigurationChangeNotice(updatedFolders),
hideConfigurationChangeNotice: () => dependencies.hideConfigurationChangeNotice(),
});
this.processedState = createHiddenFileSyncProcessedState({
getKeyValueDatabase: () => this.dependencies.getKeyValueDatabase(),
getLocalDatabase: () => this.dependencies.getLocalDatabase(),
storageAccess: this.dependencies.storageAccess,
path: this.dependencies.path,
log: (message, level, key) => this.dependencies.log(message, level, key),
});
this.databaseWriteOperations = createHiddenFileSyncDatabaseWriteOperations({
serialiseFileOperation: async (key, operation) => await serialized(key, operation),
isIgnoredByIgnoreFile: async (path) => await dependencies.isIgnoredByIgnoreFile(path),
readFileWithInfo: async (path) => await readHiddenFileWithInfo(dependencies, path),
loadBaseEntry: async (path) => await loadHiddenFileSyncBaseEntry(dependencies, path, true),
loadBaseMetadata: async (path) => await loadHiddenFileSyncBaseEntry(dependencies, path, false),
loadLiveRevision: async (path, revision) =>
await loadLiveHiddenFileSyncRevision(dependencies, path, revision),
fetchEntryFromMeta: async (meta, waitForReady, skipCheck) =>
await dependencies.databaseFileAccess.fetchEntryFromMeta(meta, waitForReady, skipCheck),
storeWithBaseRevision: async (file, baseRevision, skipCheck) =>
await dependencies.databaseFileAccess.storeWithBaseRevision(file, baseRevision, skipCheck),
putDatabaseEntry: async (entry) => await dependencies.getLocalDatabase().putDBEntry(entry),
putRaw: async (entry) => await dependencies.getLocalDatabase().putRaw(entry),
removeRevision: async (id, revision) => await dependencies.getLocalDatabase().removeRevision(id, revision),
processedState: this.processedState,
now: () => new Date().getTime(),
log: (message, level, key) => dependencies.log(message, level, key),
});
this.databaseExtractionOperations = createHiddenFileSyncDatabaseExtractionOperations({
serialiseFileOperation: async (key, operation) => await serialized(key, operation),
isIgnoredByIgnoreFile: async (path) => await dependencies.isIgnoredByIgnoreFile(path),
loadDatabaseMetadata: async (path) =>
await dependencies.getLocalDatabase().getDBEntryMeta(path, { conflicts: true }, true),
loadLiveRevision: async (path, revision) =>
await loadLiveHiddenFileSyncRevision(dependencies, path, revision),
loadDatabaseEntry: async (entry) =>
await dependencies.getLocalDatabase().getDBEntryFromMeta(entry, false, true),
statStorageFile: async (path) => await dependencies.storageAccess.statHidden(path),
writeStorageFile: async (path, entry, force) =>
await writeHiddenFileFromDatabase(dependencies, path, entry, force),
deleteStorageFile: async (path) => await deleteHiddenFileFromStorage(dependencies, path),
processedState: this.processedState,
queueNotification: (path) => this.changeNotifier.queueNotification(path),
log: (message, level, key) => dependencies.log(message, level, key),
});
this.conflictResolution = createHiddenFileSyncConflictResolution({
database: {
scanConflictedEntries: () =>
dependencies.getLocalDatabase().findEntries(ICHeader, ICHeaderEnd, { conflicts: true }),
getDocumentId: async (path) => await dependencies.path.path2id(path, ICHeader),
loadCurrentMetadata: async (id) =>
await dependencies.getLocalDatabase().getRaw<MetaEntry>(id, { conflicts: true }),
loadConflictingMetadata: async (id, revision) =>
await dependencies.getLocalDatabase().getRaw<MetaEntry>(id, { rev: revision }),
loadRevisionHistory: async (id) =>
await dependencies.getLocalDatabase().getRaw(id, { revs_info: true }),
loadRevisionEntry: async (path, revision) =>
await dependencies.getLocalDatabase().getDBEntry(addPrefix(path, ICHeader), { rev: revision }),
mergeJson: async (path, baseRevision, currentRevision, conflictedRevision) =>
await dependencies
.getLocalDatabase()
.managers.conflictManager.mergeObject(path, baseRevision, currentRevision, conflictedRevision),
removeRevision: async (id, revision) =>
await dependencies.getLocalDatabase().removeRevision(id, revision),
deleteRevision: async (entry) =>
await dependencies.getLocalDatabase().deleteDBEntry(dependencies.path.getPath(entry), {
rev: entry._rev,
}),
},
storage: {
ensureDirectory: async (path) => await ensureHiddenFileDirectory(dependencies, path),
writeFile: async (path, data) => await writeHiddenFile(dependencies, path, data),
triggerEvent: async (path) => await triggerHiddenFileEvent(dependencies, path),
},
reconciliation: {
storeFile: async (file, forceWrite) => await this.databaseWriteOperations.store(file, forceWrite),
extractFile: async (path) => await this.databaseExtractionOperations.extract(path),
},
interaction: {
resolveJsonConflict: async (path, docs, apply) =>
await dependencies.resolveJsonConflict(path, docs, apply),
},
shouldOverwrite: (path) =>
dependencies.getFileRegExp("syncInternalFileOverwritePatterns").some((pattern) => pattern.test(path)),
log: (message, level, key) => dependencies.log(message, level, key),
});
this.changeProcessor = createHiddenFileSyncChangeProcessor({
storageAccess: dependencies.storageAccess,
readFileWithInfo: async (path) => await readHiddenFileWithInfo(dependencies, path),
loadDatabaseMetadata: async (path) =>
await dependencies.getLocalDatabase().getDBEntryMeta(path, { conflicts: true }, true),
databaseWriteOperations: this.databaseWriteOperations,
databaseExtractionOperations: this.databaseExtractionOperations,
processedState: this.processedState,
conflictResolution: this.conflictResolution,
log: (message, level, key) => dependencies.log(message, level, key),
publishActivity: (eventCount, processingCount) => dependencies.publishActivity(eventCount, processingCount),
});
this.reconciliation = createReconciliation({
listFiles: async (path) => await dependencies.listFiles(path),
getLocalDatabase: () => dependencies.getLocalDatabase(),
storageAccess: dependencies.storageAccess,
getRootPath: () => dependencies.getRootPath(),
getPath: (entry) => dependencies.path.getPath(entry),
isTargetFile: async (path) => await this.pathAdmission.isTargetFile(path),
isIgnoredByIgnoreFile: async (path) => await dependencies.isIgnoredByIgnoreFile(path),
createProgress: (prefix, level) => dependencies.createProgress(prefix, level),
processedState: this.processedState,
changeProcessor: this.changeProcessor,
log: (message, level, key) => dependencies.log(message, level, key),
});
this.repair = createHiddenFileSyncRepairView({
scanInternalFiles: async () => await this.reconciliation.scanInternalFiles(),
storeInternalFileToDatabase: async (file, forceWrite) =>
await this.databaseWriteOperations.store(file, forceWrite),
storeInternalFileToDatabaseWithBaseRevision: async (file, baseRevision, createIfDifferent) =>
await this.databaseWriteOperations.storeWithBaseRevision(file, baseRevision, createIfDifferent),
extractInternalFileRevisionFromDatabase: async (storageFilePath, revision, force) =>
await this.databaseExtractionOperations.extractRevision(storageFilePath, revision, force),
});
this.serviceHandlers = createHiddenFileSyncServiceHandlerView({
processOptionalFileEvent: async (path) => await this.processOptionalFileEvent(path),
processOptionalSyncFiles: async (doc) => await this.processOptionalSyncFiles(doc),
onSettingLoaded: async () => await this.onSettingLoaded(),
realiseSettingSyncMode: async () => await this.realiseSettingSyncMode(),
onResuming: async () => await this.onResuming(),
beforeReplicate: async (showNotice) => await this.beforeReplicate(showNotice),
onDatabaseInitialised: async (showNotice) => await this.onDatabaseInitialised(showNotice),
suspendExtraSync: async () => await this.suspendExtraSync(),
configureOptionalSyncFeature: async (mode) => await this.configureOptionalSyncFeature(mode),
isTargetFileEligible: async (path) => await this.pathAdmission.isTargetFileEligible(path),
queueConflict: async (path) => await this.queueConflict(path),
});
this.testing = createHiddenFileSyncTestingView({
isManualCommandAvailable: () => this.isManualCommandAvailable(),
scanAllStorageChanges: async (showNotice) => await this.scanAllStorageChanges(showNotice),
scanAllDatabaseChanges: async (showNotice) => await this.scanAllDatabaseChanges(showNotice),
applyOfflineChanges: async (showNotice) => await this.applyOfflineChanges(showNotice),
updateSettingCache: () => this.updateSettingCache(),
initialiseInternalFileSync: async (direction, showMessage, targetFiles) =>
await this.initialiseInternalFileSync(direction, showMessage, targetFiles),
conflictResolution: this.conflictResolution.testing,
readFileWithInfo: async (path) => await readHiddenFileWithInfo(dependencies, path),
showConfigurationChangeNotice: (updatedFolders) =>
this.changeNotifier.showConfigurationChangeNotice(updatedFolders),
interceptRebuildMerging: (interceptor) => this.reconciliation.interceptRebuildMerging(interceptor),
});
this.periodicInternalFileScanProcessor = dependencies.createPeriodicProcessor(
async () =>
this.isThisModuleEnabled() && this._isDatabaseReady() && (await this.scanAllStorageChanges(false))
);
}
private get settings() {
return this.dependencies.getSettings();
}
private getPath(entry: AnyEntry): FilePathWithPrefix {
return this.dependencies.path.getPath(entry);
}
private _isMainReady() {
return this.dependencies.isReady();
}
private _isMainSuspended() {
return this.dependencies.isSuspended();
}
private _isDatabaseReady() {
return this.dependencies.isDatabaseReady();
}
private _log(message: unknown, level?: LOG_LEVEL, key?: string) {
this.dependencies.log(message, level, key);
}
private _progress(prefix: string = "", level: LOG_LEVEL = LOG_LEVEL_NOTICE) {
return this.dependencies.createProgress(prefix, level);
}
private isThisModuleEnabled() {
return this.settings.syncInternalFiles;
}
dispose() {
if (this.disposed) return;
this.disposed = true;
this.periodicInternalFileScanProcessor?.disable();
this.changeProcessor.dispose();
this.reconciliation.dispose();
this.conflictResolution.dispose();
this.pathAdmission.dispose();
this.changeNotifier.dispose();
this.dependencies.closeJsonConflictDialogs();
}
// The key-value database becomes available before this lifecycle callback.
private async onDatabaseInitialised(showNotice: boolean) {
await this.processedState.initialise();
if (this.isThisModuleEnabled()) {
if (this.processedState.getLastProcessedFileCount() == 0) {
this._log(`No cache found. Performing startup scan.`, LOG_LEVEL_VERBOSE);
await this.applyOfflineChanges(true);
} else {
await this.applyOfflineChanges(showNotice);
}
}
return true;
}
private async beforeReplicate(showNotice: boolean) {
if (
this.isThisModuleEnabled() &&
this._isDatabaseReady() &&
this.settings.syncInternalFilesBeforeReplication &&
!this.settings.watchInternalFileChanges
) {
await this.scanAllStorageChanges(showNotice);
}
return true;
}
private onSettingLoaded(): Promise<boolean> {
this.updateSettingCache();
return Promise.resolve(true);
}
updateSettingCache() {
this.pathAdmission.invalidatePatternCache();
}
private isReady() {
if (this.disposed) return false;
if (!this._isMainReady()) return false;
if (this._isMainSuspended()) return false;
if (!this.isThisModuleEnabled()) return false;
return true;
}
isManualCommandAvailable() {
return this.settings.useAdvancedMode && this.isReady() && this._isDatabaseReady();
}
private async onResuming(): Promise<boolean> {
this.periodicInternalFileScanProcessor?.disable();
if (this._isMainSuspended()) return true;
if (this.isThisModuleEnabled()) {
await this.applyOfflineChanges(false);
}
this.periodicInternalFileScanProcessor.enable(
this.isThisModuleEnabled() && this.settings.syncInternalFilesInterval
? this.settings.syncInternalFilesInterval * 1000
: 0
);
return true;
}
private realiseSettingSyncMode(): Promise<boolean> {
this.periodicInternalFileScanProcessor?.disable();
if (this._isMainSuspended()) return Promise.resolve(true);
if (!this._isMainReady()) return Promise.resolve(true);
this.periodicInternalFileScanProcessor.enable(
this.isThisModuleEnabled() && this.settings.syncInternalFilesInterval
? this.settings.syncInternalFilesInterval * 1000
: 0
);
this.pathAdmission.invalidatePatternCache();
return Promise.resolve(true);
}
private async processOptionalFileEvent(path: FilePath): Promise<boolean> {
if (this.isReady()) {
return (await this.reconciliation.processStorageChange(path)) || false;
}
return false;
}
private async processOptionalSyncFiles(doc: LoadedEntry): Promise<boolean> {
if (isInternalMetadata(doc._id)) {
if (this.isThisModuleEnabled()) {
//system file
const filename = this.getPath(doc);
const unprefixedPath = stripAllPrefixes(filename);
if (!(await this.pathAdmission.isTargetFile(stripAllPrefixes(unprefixedPath)))) {
this._log(
`Skipped processing sync file:${unprefixedPath} (Not Hidden File Sync target)`,
LOG_LEVEL_VERBOSE
);
// We should return true, we made sure that document is a internalMetadata.
return true;
}
if (!(await this.reconciliation.processDatabaseDocument(doc))) {
this._log(`Failed to process sync file:${unprefixedPath}`, LOG_LEVEL_NOTICE);
// Do not yield false, this file had been processed.
}
}
return true;
}
return false;
}
// --> Database Event Functions
private queueConflict(path: FilePathWithPrefix): Promise<boolean> {
this.conflictResolution.queue(path);
return Promise.resolve(true);
}
// <-- Database Event Functions
async scanAllStorageChanges(
showNotice: boolean = false,
onlyNew = false,
forceWriteAll = false,
includeDeleted = true
): Promise<unknown> {
return await this.reconciliation.scanAllStorageChanges(showNotice, onlyNew, forceWriteAll, includeDeleted);
}
async scanAllDatabaseChanges(
showNotice: boolean = false,
onlyNew = false,
forceWriteAll = false,
includeDeletion = true
): Promise<unknown> {
return await this.reconciliation.scanAllDatabaseChanges(showNotice, onlyNew, forceWriteAll, includeDeletion);
}
async applyOfflineChanges(showNotice: boolean): Promise<unknown> {
return await this.reconciliation.applyOfflineChanges(showNotice);
}
async initialiseInternalFileSync(
direction: SyncDirection,
showMessage: boolean,
targetFilesSrc: string[] | false = false,
initialisationProgress?: ReconciliationProgress
): Promise<void> {
return await this.reconciliation.initialiseInternalFileSync(
direction,
showMessage,
targetFilesSrc,
initialisationProgress
);
}
// <-- Initialization functions
private suspendExtraSync(): Promise<boolean> {
if (this.settings.syncInternalFiles) {
this._log(
$msg(
"Hidden file synchronization have been temporarily disabled. Please enable them after the fetching, if you need them."
),
LOG_LEVEL_NOTICE
);
this.dependencies.setSyncInternalFilesEnabled(false);
}
return Promise.resolve(true);
}
// --> Configuration handling
private async configureOptionalSyncFeature(mode: OptionalSyncFeatureMode) {
await this.configureHiddenFileSync(mode);
return true;
}
private async configureHiddenFileSync(mode: OptionalSyncFeatureMode) {
let initialisationProgress: ReconciliationProgress | undefined;
let result: ConfigureHiddenFileSyncResult;
try {
result = await configureHiddenFileSyncMode(mode, {
disable: async () => {
await this.dependencies.applySettings(
{
syncInternalFiles: false,
},
true
);
},
enable: async () => {
// Open the one user-visible progress Notice before saving
// the setting. Large Vaults can otherwise appear idle
// before the initial file enumeration begins.
initialisationProgress = this._progress("[⚙ Initialise]\n", LOG_LEVEL_NOTICE);
initialisationProgress.log("Preparing Hidden File Sync...");
await this.dependencies.applySettings(
{
useAdvancedMode: true,
syncInternalFiles: true,
},
true
);
},
initialise: async (direction) => {
await this.initialiseInternalFileSync(direction, true, false, initialisationProgress);
initialisationProgress = undefined;
},
});
} catch (error) {
initialisationProgress?.done("Failed");
throw error;
}
if (result == "ignored" || result == "disabled") {
return;
}
this._log("Hidden File Sync initialisation completed.", LOG_LEVEL_INFO);
}
// <-- Configuration handling
// <-- Local Storage SubFunctions
}
@@ -0,0 +1,132 @@
import { describe, expect, it, vi } from "vitest";
import { LOG_LEVEL_NOTICE } from "@vrtmrz/livesync-commonlib/compat/common/types";
vi.mock("@/deps.ts", () => ({}));
vi.mock("./configureHiddenFileSyncMode.ts", () => ({
configureHiddenFileSyncMode: vi.fn(),
}));
import { HiddenFileSyncContext } from "./hiddenFileSyncContext.ts";
import { configureHiddenFileSyncMode } from "./configureHiddenFileSyncMode.ts";
function callPrivate<T extends (...args: never[]) => unknown>(context: HiddenFileSyncContext, key: string): T {
const operation = (context as unknown as Record<string, T>)[key];
return operation.bind(context) as T;
}
describe("HiddenFileSyncContext lifecycle", () => {
it("releases its processors, capability owners, and host conflict dialogues", () => {
const periodicInternalFileScanProcessor = { disable: vi.fn() };
const conflictResolution = { dispose: vi.fn() };
const changeProcessor = { dispose: vi.fn() };
const reconciliation = { dispose: vi.fn() };
const pathAdmission = { dispose: vi.fn() };
const changeNotifier = { dispose: vi.fn() };
const closeJsonConflictDialogs = vi.fn();
const hiddenFileSync = Object.create(HiddenFileSyncContext.prototype) as HiddenFileSyncContext;
Object.assign(hiddenFileSync, {
dependencies: { closeJsonConflictDialogs },
periodicInternalFileScanProcessor,
conflictResolution,
changeProcessor,
reconciliation,
pathAdmission,
changeNotifier,
eventCount: 4,
processingCount: 2,
});
hiddenFileSync.dispose();
hiddenFileSync.dispose();
expect(periodicInternalFileScanProcessor.disable).toHaveBeenCalledOnce();
expect(conflictResolution.dispose).toHaveBeenCalledOnce();
expect(changeProcessor.dispose).toHaveBeenCalledOnce();
expect(reconciliation.dispose).toHaveBeenCalledOnce();
expect(pathAdmission.dispose).toHaveBeenCalledOnce();
expect(changeNotifier.dispose).toHaveBeenCalledOnce();
expect(closeJsonConflictDialogs).toHaveBeenCalledOnce();
});
it("does not report Hidden File Sync as ready before the main runtime is ready", () => {
const hiddenFileSync = Object.create(HiddenFileSyncContext.prototype) as HiddenFileSyncContext;
Object.assign(hiddenFileSync, {
dependencies: {
getSettings: () => ({
syncInternalFiles: true,
}),
},
_isMainReady: vi.fn(() => false),
_isMainSuspended: vi.fn(() => false),
});
expect(callPrivate<() => boolean>(hiddenFileSync, "isReady")()).toBe(false);
});
it("retirement guard: does not restore separate gathering and restart Notices", async () => {
vi.mocked(configureHiddenFileSyncMode).mockImplementation(async (_mode, handlers) => {
await handlers.enable();
await handlers.initialise("safe");
return "enabled";
});
const events: string[] = [];
const progress = {
log: vi.fn((message: string) => {
events.push(`progress:${message}`);
}),
once: vi.fn(),
done: vi.fn(),
};
const createProgress = vi.fn(() => progress);
const applyPartial = vi.fn(async () => {
events.push("apply-settings");
});
const initialiseInternalFileSync = vi.fn(async () => undefined);
const log = vi.fn();
const hiddenFileSync = Object.create(HiddenFileSyncContext.prototype) as HiddenFileSyncContext;
Object.assign(hiddenFileSync, {
dependencies: { applySettings: applyPartial },
initialiseInternalFileSync,
_progress: createProgress,
_log: log,
});
await callPrivate<(mode: "MERGE") => Promise<void>>(hiddenFileSync, "configureHiddenFileSync")("MERGE");
expect(createProgress).toHaveBeenCalledWith("[⚙ Initialise]\n", LOG_LEVEL_NOTICE);
expect(events[0]).toBe("progress:Preparing Hidden File Sync...");
expect(initialiseInternalFileSync).toHaveBeenCalledWith("safe", true, false, progress);
expect(log).not.toHaveBeenCalledWith("Gathering files for enabling Hidden File Sync", LOG_LEVEL_NOTICE);
expect(log).not.toHaveBeenCalledWith("Done! Restarting the app is strongly recommended!", LOG_LEVEL_NOTICE);
expect(log).toHaveBeenCalledWith("Hidden File Sync initialisation completed.", expect.any(Number));
});
it("closes the preparation Notice when enabling Hidden File Sync fails", async () => {
vi.mocked(configureHiddenFileSyncMode).mockImplementation(async (_mode, handlers) => {
await handlers.enable();
return "enabled";
});
const error = new Error("setting persistence failed");
const progress = {
log: vi.fn(),
once: vi.fn(),
done: vi.fn(),
};
const hiddenFileSync = Object.create(HiddenFileSyncContext.prototype) as HiddenFileSyncContext;
Object.assign(hiddenFileSync, {
dependencies: {
applySettings: vi.fn(async () => {
throw error;
}),
},
_progress: vi.fn(() => progress),
_log: vi.fn(),
});
await expect(
callPrivate<(mode: "MERGE") => Promise<void>>(hiddenFileSync, "configureHiddenFileSync")("MERGE")
).rejects.toBe(error);
expect(progress.done).toHaveBeenCalledWith("Failed");
});
});
@@ -0,0 +1,223 @@
import {
LOG_LEVEL_INFO,
LOG_LEVEL_VERBOSE,
type FilePath,
type FilePathWithPrefix,
type LoadedEntry,
type LOG_LEVEL,
type MetaEntry,
type UXStat,
} from "@vrtmrz/livesync-commonlib/compat/common/types";
import { compareMTime, TARGET_IS_NEW } from "@/common/utils.ts";
import type { LogFunction } from "@vrtmrz/livesync-commonlib/compat/services/lib/logUtils";
import { addPrefix } from "@vrtmrz/livesync-commonlib/compat/string_and_binary/path";
import { ICHeader } from "@/common/types.ts";
import { getHiddenFileSyncComparisonMTime } from "./hiddenFileSyncState.ts";
import type { HiddenFileSyncRemovalResult } from "./hiddenFileSyncStorage.ts";
import {
serialiseHiddenFileOperation,
type HiddenFileSyncFileSerialisationDependencies,
} from "./hiddenFileSyncFileOperations.ts";
export type HiddenFileSyncDatabaseExtractionOptions = Readonly<{
force?: boolean;
metaEntry?: MetaEntry | LoadedEntry;
preventDoubleProcess?: boolean;
onlyNew?: boolean;
includeDeletion?: boolean;
requiredLiveRevision?: string;
}>;
type HiddenFileSyncDatabaseExtractionStorageDependencies = {
statStorageFile(path: FilePath): Promise<UXStat | null>;
writeStorageFile(path: FilePath, entry: LoadedEntry, force: boolean): Promise<false | UXStat>;
deleteStorageFile(path: FilePath): Promise<HiddenFileSyncRemovalResult>;
};
type HiddenFileSyncDatabaseExtractionReadDependencies = {
loadDatabaseMetadata(path: FilePathWithPrefix): Promise<MetaEntry | LoadedEntry | false>;
loadLiveRevision(path: FilePathWithPrefix, revision: string): Promise<MetaEntry | false>;
loadDatabaseEntry(entry: MetaEntry | LoadedEntry): Promise<LoadedEntry | false>;
};
export type HiddenFileSyncDatabaseExtractionProcessedState = {
databaseStateKey(entry: MetaEntry | LoadedEntry): string;
getLastProcessedDatabaseKey(path: FilePath): string | undefined;
getLastProcessedFileMTime(path: FilePath): number;
updateLastProcessedDatabase(path: FilePath, entry: string | MetaEntry | LoadedEntry): void;
updateLastProcessedFile(path: FilePath, storageFile: string | UXStat): void;
updateLastProcessed(path: FilePath, databaseEntry: MetaEntry | LoadedEntry, storageFile: UXStat): void;
updateLastProcessedDeletion(path: FilePath, databaseEntry: MetaEntry | LoadedEntry | false): void;
};
export type HiddenFileSyncDatabaseExtractionDependencies = HiddenFileSyncFileSerialisationDependencies &
HiddenFileSyncDatabaseExtractionStorageDependencies &
HiddenFileSyncDatabaseExtractionReadDependencies & {
isIgnoredByIgnoreFile(path: string): Promise<boolean>;
queueNotification(path: FilePath): void;
processedState: HiddenFileSyncDatabaseExtractionProcessedState;
log: LogFunction;
};
export type HiddenFileSyncDatabaseExtractionOperations = {
extract(path: FilePath, options?: HiddenFileSyncDatabaseExtractionOptions): Promise<boolean | undefined>;
extractRevision(path: FilePath, revision: string, force?: boolean): Promise<boolean>;
};
function log(
dependencies: Pick<HiddenFileSyncDatabaseExtractionDependencies, "log">,
message: unknown,
level?: LOG_LEVEL,
key?: string
): void {
dependencies.log(message, level, key);
}
export async function extractHiddenFileFromDatabase(
dependencies: HiddenFileSyncDatabaseExtractionDependencies,
storageFilePath: FilePath,
options: HiddenFileSyncDatabaseExtractionOptions = {}
): Promise<boolean | undefined> {
const {
force = false,
metaEntry,
preventDoubleProcess = true,
onlyNew = false,
includeDeletion = true,
requiredLiveRevision,
} = options;
const prefixedFileName = addPrefix(storageFilePath, ICHeader);
// Compatibility: admission happens outside the per-file lock, so ignore
// policy errors propagate instead of becoming a false extraction result.
if (await dependencies.isIgnoredByIgnoreFile(storageFilePath)) {
return undefined;
}
return await serialiseHiddenFileOperation(dependencies, prefixedFileName, async () => {
try {
// A caller-supplied Metadata entry is trusted as-is. It can change
// before this lock is acquired; only exact-revision repair performs
// an in-lock liveness check.
const metaOnDatabase = requiredLiveRevision
? await dependencies.loadLiveRevision(prefixedFileName, requiredLiveRevision)
: metaEntry
? metaEntry
: await dependencies.loadDatabaseMetadata(prefixedFileName);
// Compatibility: the exact-revision loader validates revision-tree
// membership but normally returns a leaf without `_conflicts`.
// Repair can therefore apply one live branch while ordinary
// reflection remains blocked by conflicted winning Metadata.
if (metaOnDatabase === false) {
throw new Error(`File not found on database.:${storageFilePath}`);
}
if (metaOnDatabase._conflicts?.length) {
log(
dependencies,
`Hidden file ${storageFilePath} has conflicted revisions, to keep in safe, writing to storage has been prevented`,
LOG_LEVEL_INFO
);
return false;
}
if (preventDoubleProcess) {
const key = dependencies.processedState.databaseStateKey(metaOnDatabase);
if (dependencies.processedState.getLastProcessedDatabaseKey(storageFilePath) == key && !force) {
// Compatibility question: the force suffix is unreachable
// because this branch is entered only when force is false.
log(
dependencies,
`STORAGE <-- DB: ${storageFilePath}: skipped (hidden, overwrite${force ? ", force" : ""}) (Previously processed)`
);
return undefined;
}
}
if (onlyNew) {
const databaseMTime = getHiddenFileSyncComparisonMTime(metaOnDatabase, includeDeletion);
const storageStat = await dependencies.statStorageFile(storageFilePath);
const storageMTimeActual = storageStat?.mtime ?? 0;
const storageMTime =
storageMTimeActual == 0
? dependencies.processedState.getLastProcessedFileMTime(storageFilePath)
: storageMTimeActual;
const difference = compareMTime(storageMTime, databaseMTime);
if (difference != TARGET_IS_NEW) {
log(
dependencies,
`STORAGE <-- DB: ${storageFilePath}: skipped (hidden, overwrite${force ? ", force" : ""}) (Not new)`
);
// Compatibility: a declined candidate is settled as
// processed, including a deletion excluded by
// includeDeletion. This prevents later scans retrying it.
dependencies.processedState.updateLastProcessedDatabase(storageFilePath, metaOnDatabase);
if (storageStat) dependencies.processedState.updateLastProcessedFile(storageFilePath, storageStat);
return undefined;
}
}
const deleted = metaOnDatabase.deleted || metaOnDatabase._deleted || false;
if (deleted) {
const result = await dependencies.deleteStorageFile(storageFilePath);
if (result == "OK") {
dependencies.processedState.updateLastProcessedDeletion(storageFilePath, metaOnDatabase);
return true;
}
if (result == "ALREADY") {
// Compatibility question: an already absent file updates
// only the database key. It does not record the missing
// storage key or call unmarkChanges through deletion state.
dependencies.processedState.updateLastProcessedDatabase(storageFilePath, metaOnDatabase);
return true;
}
return false;
}
const fileOnDatabase = await dependencies.loadDatabaseEntry(metaOnDatabase);
if (fileOnDatabase === false) {
throw new Error(`Failed to read file from database:${storageFilePath}`);
}
const resultStat = await dependencies.writeStorageFile(storageFilePath, fileOnDatabase, force);
if (resultStat) {
// Compatibility question: the storage writer also returns the
// existing stat when content is unchanged. That no-op still
// settles state and queues a configuration notification here.
dependencies.processedState.updateLastProcessed(storageFilePath, metaOnDatabase, resultStat);
dependencies.queueNotification(storageFilePath);
log(
dependencies,
`STORAGE <-- DB: ${storageFilePath}: written (hidden, overwrite${force ? ", force" : ""}) Done`
);
return true;
}
return false;
} catch (error) {
log(
dependencies,
`STORAGE <-- DB: ${storageFilePath}: written (hidden, overwrite${force ? ", force" : ""}) Failed`
);
log(dependencies, error, LOG_LEVEL_VERBOSE);
return false;
}
});
}
export async function extractHiddenFileRevisionFromDatabase(
dependencies: HiddenFileSyncDatabaseExtractionDependencies,
storageFilePath: FilePath,
revision: string,
force = false
): Promise<boolean> {
return Boolean(
await extractHiddenFileFromDatabase(dependencies, storageFilePath, {
force,
requiredLiveRevision: revision,
})
);
}
export function createHiddenFileSyncDatabaseExtractionOperations(
dependencies: HiddenFileSyncDatabaseExtractionDependencies
): HiddenFileSyncDatabaseExtractionOperations {
return Object.freeze({
extract: async (path, options) => await extractHiddenFileFromDatabase(dependencies, path, options),
extractRevision: async (path, revision, force) =>
await extractHiddenFileRevisionFromDatabase(dependencies, path, revision, force),
});
}
@@ -0,0 +1,366 @@
import { describe, expect, it, vi } from "vitest";
import {
LOG_LEVEL_INFO,
LOG_LEVEL_VERBOSE,
type DocumentID,
type FilePath,
type FilePathWithPrefix,
type LoadedEntry,
type MetaEntry,
type UXStat,
} from "@vrtmrz/livesync-commonlib/compat/common/types";
import type { HiddenFileSyncRemovalResult } from "./hiddenFileSyncStorage.ts";
vi.mock("@/deps.ts", () => ({}));
import {
createHiddenFileSyncDatabaseExtractionOperations,
extractHiddenFileFromDatabase,
extractHiddenFileRevisionFromDatabase,
type HiddenFileSyncDatabaseExtractionDependencies,
} from "./hiddenFileSyncDatabaseExtractionOperations.ts";
import { toHiddenFileSyncDatabaseStateKey } from "./hiddenFileSyncState.ts";
const path = ".obsidian/plugins/example/data.json" as FilePath;
const prefixedPath = `i:${path}` as FilePathWithPrefix;
const id = "hidden-entry-id" as DocumentID;
function metadata(overrides: Partial<MetaEntry> = {}): MetaEntry {
return {
_id: id,
_rev: "2-current",
path: prefixedPath,
type: "plain",
datatype: "plain",
ctime: 10,
mtime: 20,
size: 20,
children: [],
eden: {},
deleted: false,
...overrides,
} as unknown as MetaEntry;
}
function loadedEntry(entry = metadata()): LoadedEntry {
return {
...entry,
data: '{"value":"database"}',
} as LoadedEntry;
}
function storageStat(mtime = 20): UXStat {
return {
ctime: 11,
mtime,
size: 20,
type: "file",
};
}
function createDependencies(entry = metadata()) {
const events: string[] = [];
const serialiseFileOperation = vi.fn(async (_key: string, operation: () => Promise<unknown>) => {
events.push("lock:start");
try {
return await operation();
} finally {
events.push("lock:end");
}
});
const log = vi.fn();
const isIgnoredByIgnoreFile = vi.fn(async () => false);
const loadDatabaseMetadata = vi.fn(async () => entry as MetaEntry | false);
const loadLiveRevision = vi.fn(async (_path: FilePathWithPrefix, revision: string) =>
revision === entry._rev ? (entry as MetaEntry) : false
);
const loadDatabaseEntry = vi.fn(async () => loadedEntry(entry) as LoadedEntry | false);
const statStorageFile = vi.fn(async () => storageStat() as UXStat | null);
const writeStorageFile = vi.fn(async () => {
events.push("storage:write");
return storageStat(21) as UXStat | false;
});
const deleteStorageFile = vi.fn(async (): Promise<HiddenFileSyncRemovalResult> => {
events.push("storage:delete");
return "OK" as const;
});
const getLastProcessedDatabaseKey = vi.fn(() => undefined as string | undefined);
const getLastProcessedFileMTime = vi.fn(() => 0);
const updateLastProcessed = vi.fn(() => events.push("state:file"));
const updateLastProcessedDatabase = vi.fn(() => events.push("state:database"));
const updateLastProcessedFile = vi.fn(() => events.push("state:storage"));
const updateLastProcessedDeletion = vi.fn(() => events.push("state:deletion"));
const processedState = {
databaseStateKey: toHiddenFileSyncDatabaseStateKey,
getLastProcessedDatabaseKey,
getLastProcessedFileMTime,
updateLastProcessed,
updateLastProcessedDatabase,
updateLastProcessedFile,
updateLastProcessedDeletion,
};
const queueNotification = vi.fn(() => events.push("notification"));
const dependencies = {
serialiseFileOperation,
isIgnoredByIgnoreFile,
loadDatabaseMetadata,
loadLiveRevision,
loadDatabaseEntry,
statStorageFile,
writeStorageFile,
deleteStorageFile,
processedState,
queueNotification,
log,
} as HiddenFileSyncDatabaseExtractionDependencies;
return {
deleteStorageFile,
dependencies,
entry,
events,
getLastProcessedDatabaseKey,
getLastProcessedFileMTime,
isIgnoredByIgnoreFile,
loadDatabaseEntry,
loadDatabaseMetadata,
loadLiveRevision,
log,
queueNotification,
serialiseFileOperation,
statStorageFile,
updateLastProcessed,
updateLastProcessedDatabase,
updateLastProcessedDeletion,
updateLastProcessedFile,
writeStorageFile,
};
}
describe("hidden-file database-to-storage admission", () => {
it("returns undefined for an ignored path without taking the file lock", async () => {
const fixture = createDependencies();
fixture.isIgnoredByIgnoreFile.mockResolvedValue(true);
await expect(extractHiddenFileFromDatabase(fixture.dependencies, path)).resolves.toBeUndefined();
expect(fixture.serialiseFileOperation).not.toHaveBeenCalled();
expect(fixture.loadDatabaseMetadata).not.toHaveBeenCalled();
});
it("propagates ignore-policy errors before taking the guarded path", async () => {
const fixture = createDependencies();
const error = new Error("ignore policy unavailable");
fixture.isIgnoredByIgnoreFile.mockRejectedValue(error);
await expect(extractHiddenFileFromDatabase(fixture.dependencies, path)).rejects.toBe(error);
expect(fixture.serialiseFileOperation).not.toHaveBeenCalled();
expect(fixture.log).not.toHaveBeenCalled();
});
it("prevents a conflicted entry from reaching storage", async () => {
const entry = metadata({ _conflicts: ["2-conflict"] });
const fixture = createDependencies(entry);
await expect(extractHiddenFileFromDatabase(fixture.dependencies, path, { force: true })).resolves.toBe(false);
expect(fixture.loadDatabaseEntry).not.toHaveBeenCalled();
expect(fixture.writeStorageFile).not.toHaveBeenCalled();
expect(fixture.log).toHaveBeenCalledWith(
`Hidden file ${path} has conflicted revisions, to keep in safe, writing to storage has been prevented`,
LOG_LEVEL_INFO,
undefined
);
});
});
describe("hidden-file database-to-storage processed-state policy", () => {
it("skips a previously processed revision without settling state again", async () => {
const fixture = createDependencies();
fixture.getLastProcessedDatabaseKey.mockReturnValue(toHiddenFileSyncDatabaseStateKey(fixture.entry));
await expect(extractHiddenFileFromDatabase(fixture.dependencies, path)).resolves.toBeUndefined();
expect(fixture.loadDatabaseEntry).not.toHaveBeenCalled();
expect(fixture.updateLastProcessed).not.toHaveBeenCalled();
expect(fixture.log).toHaveBeenCalledWith(
`STORAGE <-- DB: ${path}: skipped (hidden, overwrite) (Previously processed)`,
undefined,
undefined
);
});
it("allows force to bypass the previously processed revision", async () => {
const fixture = createDependencies();
fixture.getLastProcessedDatabaseKey.mockReturnValue(toHiddenFileSyncDatabaseStateKey(fixture.entry));
await expect(extractHiddenFileFromDatabase(fixture.dependencies, path, { force: true })).resolves.toBe(true);
expect(fixture.writeStorageFile).toHaveBeenCalledWith(path, loadedEntry(fixture.entry), true);
});
it("settles both sides when onlyNew declines an equally old database entry", async () => {
const fixture = createDependencies();
await expect(
extractHiddenFileFromDatabase(fixture.dependencies, path, {
metaEntry: fixture.entry,
preventDoubleProcess: false,
onlyNew: true,
})
).resolves.toBeUndefined();
expect(fixture.loadDatabaseMetadata).not.toHaveBeenCalled();
expect(fixture.loadDatabaseEntry).not.toHaveBeenCalled();
expect(fixture.updateLastProcessedDatabase).toHaveBeenCalledWith(path, fixture.entry);
expect(fixture.updateLastProcessedFile).toHaveBeenCalledWith(path, storageStat());
expect(fixture.events).toEqual(["lock:start", "state:database", "state:storage", "lock:end"]);
});
it("uses the last known mtime when onlyNew sees a zero storage mtime", async () => {
const fixture = createDependencies(metadata({ mtime: 30 }));
fixture.statStorageFile.mockResolvedValue(storageStat(0));
fixture.getLastProcessedFileMTime.mockReturnValue(40);
await expect(
extractHiddenFileFromDatabase(fixture.dependencies, path, { onlyNew: true })
).resolves.toBeUndefined();
expect(fixture.getLastProcessedFileMTime).toHaveBeenCalledWith(path);
expect(fixture.writeStorageFile).not.toHaveBeenCalled();
});
});
describe("hidden-file database-to-storage application", () => {
it("settles state and queues a notification inside the file lock after a successful write", async () => {
const fixture = createDependencies();
await expect(extractHiddenFileFromDatabase(fixture.dependencies, path)).resolves.toBe(true);
expect(fixture.updateLastProcessed).toHaveBeenCalledWith(path, fixture.entry, storageStat(21));
expect(fixture.queueNotification).toHaveBeenCalledWith(path);
expect(fixture.serialiseFileOperation).toHaveBeenCalledWith(`file-${prefixedPath}`, expect.any(Function));
expect(fixture.events).toEqual(["lock:start", "storage:write", "state:file", "notification", "lock:end"]);
});
it("settles and notifies when the storage writer reports unchanged content with its existing stat", async () => {
const fixture = createDependencies();
fixture.writeStorageFile.mockResolvedValue(storageStat());
await expect(extractHiddenFileFromDatabase(fixture.dependencies, path)).resolves.toBe(true);
expect(fixture.updateLastProcessed).toHaveBeenCalledWith(path, fixture.entry, storageStat());
expect(fixture.queueNotification).toHaveBeenCalledWith(path);
});
it("returns false without settlement when the storage writer fails", async () => {
const fixture = createDependencies();
fixture.writeStorageFile.mockResolvedValue(false);
await expect(extractHiddenFileFromDatabase(fixture.dependencies, path)).resolves.toBe(false);
expect(fixture.updateLastProcessed).not.toHaveBeenCalled();
expect(fixture.queueNotification).not.toHaveBeenCalled();
});
it("records a successful storage deletion through deletion settlement", async () => {
const entry = metadata({ deleted: true });
const fixture = createDependencies(entry);
await expect(extractHiddenFileFromDatabase(fixture.dependencies, path)).resolves.toBe(true);
expect(fixture.updateLastProcessedDeletion).toHaveBeenCalledWith(path, entry);
expect(fixture.updateLastProcessedDatabase).not.toHaveBeenCalled();
expect(fixture.events).toEqual(["lock:start", "storage:delete", "state:deletion", "lock:end"]);
});
it("marks an already absent deleted file as database-processed only", async () => {
const entry = metadata({ deleted: true });
const fixture = createDependencies(entry);
fixture.deleteStorageFile.mockImplementation(async () => {
fixture.events.push("storage:delete");
return "ALREADY";
});
await expect(extractHiddenFileFromDatabase(fixture.dependencies, path)).resolves.toBe(true);
expect(fixture.updateLastProcessedDatabase).toHaveBeenCalledWith(path, entry);
expect(fixture.updateLastProcessedDeletion).not.toHaveBeenCalled();
expect(fixture.events).toEqual(["lock:start", "storage:delete", "state:database", "lock:end"]);
});
it("returns false without settling state when a storage deletion fails", async () => {
const fixture = createDependencies(metadata({ _deleted: true }));
fixture.deleteStorageFile.mockResolvedValue(false);
await expect(extractHiddenFileFromDatabase(fixture.dependencies, path)).resolves.toBe(false);
expect(fixture.updateLastProcessedDeletion).not.toHaveBeenCalled();
expect(fixture.updateLastProcessedDatabase).not.toHaveBeenCalled();
});
it("turns a database content-read failure into false and reports the inherited write diagnostic", async () => {
const fixture = createDependencies();
const error = new Error("content unavailable");
fixture.loadDatabaseEntry.mockRejectedValue(error);
await expect(extractHiddenFileFromDatabase(fixture.dependencies, path, { force: true })).resolves.toBe(false);
expect(fixture.log).toHaveBeenNthCalledWith(
1,
`STORAGE <-- DB: ${path}: written (hidden, overwrite, force) Failed`,
undefined,
undefined
);
expect(fixture.log).toHaveBeenNthCalledWith(2, error, LOG_LEVEL_VERBOSE, undefined);
});
});
describe("selected-revision hidden-file database-to-storage application", () => {
it("applies a selected live revision without reading ordinary Metadata", async () => {
const fixture = createDependencies();
await expect(
extractHiddenFileRevisionFromDatabase(fixture.dependencies, path, fixture.entry._rev!, true)
).resolves.toBe(true);
expect(fixture.loadLiveRevision).toHaveBeenCalledWith(prefixedPath, fixture.entry._rev);
expect(fixture.loadDatabaseMetadata).not.toHaveBeenCalled();
expect(fixture.writeStorageFile).toHaveBeenCalledWith(path, loadedEntry(fixture.entry), true);
});
it("can apply a selected live branch while ordinary Metadata reports a conflict", async () => {
const selected = metadata({ _rev: "2-selected", _conflicts: undefined });
const fixture = createDependencies(metadata({ _rev: "3-winner", _conflicts: [selected._rev!] }));
fixture.loadLiveRevision.mockResolvedValue(selected);
fixture.loadDatabaseEntry.mockResolvedValue(loadedEntry(selected));
await expect(
extractHiddenFileRevisionFromDatabase(fixture.dependencies, path, selected._rev!, true)
).resolves.toBe(true);
expect(fixture.loadDatabaseMetadata).not.toHaveBeenCalled();
expect(fixture.writeStorageFile).toHaveBeenCalledWith(path, loadedEntry(selected), true);
});
it("returns false when the selected revision ceased to be live", async () => {
const fixture = createDependencies();
fixture.loadLiveRevision.mockResolvedValue(false);
await expect(extractHiddenFileRevisionFromDatabase(fixture.dependencies, path, "2-stale", true)).resolves.toBe(
false
);
expect(fixture.loadDatabaseEntry).not.toHaveBeenCalled();
expect(fixture.writeStorageFile).not.toHaveBeenCalled();
});
it("exposes frozen operations with the exact-revision Boolean contract", async () => {
const fixture = createDependencies();
fixture.isIgnoredByIgnoreFile.mockResolvedValue(true);
const operations = createHiddenFileSyncDatabaseExtractionOperations(fixture.dependencies);
expect(Object.isFrozen(operations)).toBe(true);
await expect(operations.extractRevision(path, fixture.entry._rev!)).resolves.toBe(false);
});
});
@@ -0,0 +1,111 @@
import {
LOG_LEVEL_NOTICE,
LOG_LEVEL_VERBOSE,
type FilePath,
type FilePathWithPrefix,
type LoadedEntry,
type LOG_LEVEL,
type MetaEntry,
} from "@vrtmrz/livesync-commonlib/compat/common/types";
import type { DatabaseFileAccess } from "@vrtmrz/livesync-commonlib/compat/interfaces/DatabaseFileAccess";
import type { LiveSyncLocalDB } from "@vrtmrz/livesync-commonlib/compat/pouchdb/LiveSyncLocalDB";
import type { IPathService } from "@vrtmrz/livesync-commonlib/compat/services/base/IService";
import type { LogFunction } from "@vrtmrz/livesync-commonlib/compat/services/lib/logUtils";
import { addPrefix, stripAllPrefixes } from "@vrtmrz/livesync-commonlib/compat/string_and_binary/path";
import { ICHeader } from "@/common/types.ts";
type HiddenFileSyncLogDependency = {
log: LogFunction;
};
type HiddenFileSyncDatabaseMethods<Method extends keyof LiveSyncLocalDB> = {
getLocalDatabase(): Pick<LiveSyncLocalDB, Method>;
};
type HiddenFileSyncDatabaseFileAccessMethods<Method extends keyof DatabaseFileAccess> = {
databaseFileAccess: Pick<DatabaseFileAccess, Method>;
};
type HiddenFileSyncPathMethods<Method extends keyof IPathService> = {
path: Pick<IPathService, Method>;
};
export type HiddenFileSyncBaseEntryLoaderDependencies = HiddenFileSyncDatabaseMethods<"getDBEntry" | "getDBEntryMeta"> &
HiddenFileSyncPathMethods<"path2id"> &
HiddenFileSyncLogDependency;
export type HiddenFileSyncLiveRevisionLoaderDependencies = HiddenFileSyncDatabaseFileAccessMethods<
"fetchEntryMeta" | "getConflictedRevs"
> &
HiddenFileSyncLogDependency;
function log(dependencies: HiddenFileSyncLogDependency, message: unknown, level?: LOG_LEVEL, key?: string): void {
dependencies.log(message, level, key);
}
export async function loadHiddenFileSyncBaseEntry(
dependencies: HiddenFileSyncBaseEntryLoaderDependencies,
file: FilePath,
includeContent = true
): Promise<LoadedEntry | false> {
const prefixedFileName = addPrefix(file, ICHeader);
// Compatibility question: path-to-ID conversion is performed even when an
// entry already exists, and it sits outside the guarded database lookup.
// Preserve this ordering and error propagation until reviewed separately.
const id = await dependencies.path.path2id(prefixedFileName, ICHeader);
try {
const old = includeContent
? await dependencies.getLocalDatabase().getDBEntry(prefixedFileName, undefined, false, true)
: await dependencies.getLocalDatabase().getDBEntryMeta(prefixedFileName, { conflicts: true }, true);
if (old !== false) {
return old;
}
// Compatibility question: getDBEntry() also returns false when content
// or Chunks cannot be read. The inherited behaviour treats that exactly
// like absence and synthesises a fresh base entry.
return {
_id: id,
data: [],
path: prefixedFileName,
mtime: 0,
ctime: 0,
datatype: "newnote",
children: [],
size: 0,
deleted: false,
type: "newnote",
eden: {},
};
} catch (error) {
log(dependencies, "Getting base save data failed");
log(dependencies, error, LOG_LEVEL_VERBOSE);
return false;
}
}
export async function loadLiveHiddenFileSyncRevision(
dependencies: HiddenFileSyncLiveRevisionLoaderDependencies,
prefixedFileName: FilePathWithPrefix,
revision: string
): Promise<MetaEntry | false> {
const [selected, current, conflicts] = await Promise.all([
dependencies.databaseFileAccess.fetchEntryMeta(prefixedFileName, revision, true),
dependencies.databaseFileAccess.fetchEntryMeta(prefixedFileName, undefined, true),
dependencies.databaseFileAccess.getConflictedRevs(prefixedFileName),
]);
const liveRevisions = new Set([...(current && current._rev ? [current._rev] : []), ...conflicts]);
if (!selected || selected._rev !== revision || !liveRevisions.has(revision)) {
// Compatibility: missing, mismatched, and stale selections share the
// same user-facing diagnostic and false result.
log(
dependencies,
`Could not use hidden-file revision ${revision} of ${stripAllPrefixes(prefixedFileName)}; the selected revision is no longer live`,
LOG_LEVEL_NOTICE
);
return false;
}
// Compatibility: liveness is revision-tree membership only. A deleted
// Metadata leaf remains selectable while its revision is still live.
return selected;
}
@@ -0,0 +1,202 @@
import { describe, expect, it, vi } from "vitest";
import {
LOG_LEVEL_NOTICE,
LOG_LEVEL_VERBOSE,
type DocumentID,
type FilePath,
type FilePathWithPrefix,
type LoadedEntry,
type MetaEntry,
} from "@vrtmrz/livesync-commonlib/compat/common/types";
import { loadHiddenFileSyncBaseEntry, loadLiveHiddenFileSyncRevision } from "./hiddenFileSyncDatabaseLoaders.ts";
const path = ".obsidian/app.json" as FilePath;
const prefixedPath = `i:${path}` as FilePathWithPrefix;
const id = "hidden-entry-id" as DocumentID;
function loadedEntry(): LoadedEntry {
return {
_id: id,
_rev: "1-a",
path: prefixedPath,
type: "plain",
datatype: "plain",
data: "content",
ctime: 10,
mtime: 20,
size: 7,
children: [],
eden: {},
deleted: false,
} as LoadedEntry;
}
function createBaseEntryDependencies() {
const getDBEntry = vi.fn();
const getDBEntryMeta = vi.fn();
const path2id = vi.fn(async () => id);
const log = vi.fn();
const dependencies = {
getLocalDatabase: () => ({ getDBEntry, getDBEntryMeta }) as never,
path: { path2id } as never,
log,
};
return { dependencies, getDBEntry, getDBEntryMeta, log, path2id };
}
function metaEntry(revision: string): MetaEntry {
return {
...loadedEntry(),
_rev: revision,
data: undefined,
} as unknown as MetaEntry;
}
function createLiveRevisionDependencies() {
const selected = metaEntry("2-selected");
const current = metaEntry("3-current");
const fetchEntryMeta = vi.fn(async (_path: unknown, revision?: string) => {
if (revision === undefined || revision === current._rev) return current;
if (revision === selected._rev) return selected;
return false;
});
const getConflictedRevs = vi.fn(async () => [selected._rev!]);
const log = vi.fn();
const dependencies = {
databaseFileAccess: { fetchEntryMeta, getConflictedRevs } as never,
log,
};
return { current, dependencies, fetchEntryMeta, getConflictedRevs, log, selected };
}
describe("Hidden File Sync base-entry loader", () => {
it("synthesises a new empty base whenever the content lookup reports false", async () => {
const { dependencies, getDBEntry, path2id } = createBaseEntryDependencies();
getDBEntry.mockResolvedValue(false);
await expect(loadHiddenFileSyncBaseEntry(dependencies, path)).resolves.toEqual({
_id: id,
data: [],
path: prefixedPath,
mtime: 0,
ctime: 0,
datatype: "newnote",
children: [],
size: 0,
deleted: false,
type: "newnote",
eden: {},
});
expect(path2id).toHaveBeenCalledWith(prefixedPath, "i:");
expect(getDBEntry).toHaveBeenCalledWith(prefixedPath, undefined, false, true);
});
it("returns an existing content entry unchanged", async () => {
const { dependencies, getDBEntry, path2id } = createBaseEntryDependencies();
const existing = loadedEntry();
getDBEntry.mockResolvedValue(existing);
await expect(loadHiddenFileSyncBaseEntry(dependencies, path, true)).resolves.toBe(existing);
expect(path2id).toHaveBeenCalledWith(prefixedPath, "i:");
});
it("uses the conflict-aware metadata lookup when content is not requested", async () => {
const { dependencies, getDBEntry, getDBEntryMeta } = createBaseEntryDependencies();
const existing = loadedEntry();
getDBEntryMeta.mockResolvedValue(existing);
await expect(loadHiddenFileSyncBaseEntry(dependencies, path, false)).resolves.toBe(existing);
expect(getDBEntry).not.toHaveBeenCalled();
expect(getDBEntryMeta).toHaveBeenCalledWith(prefixedPath, { conflicts: true }, true);
});
it("turns a database lookup failure into a logged false result", async () => {
const { dependencies, getDBEntry, log } = createBaseEntryDependencies();
const error = new Error("database unavailable");
getDBEntry.mockRejectedValue(error);
await expect(loadHiddenFileSyncBaseEntry(dependencies, path)).resolves.toBe(false);
expect(log).toHaveBeenNthCalledWith(1, "Getting base save data failed", undefined, undefined);
expect(log).toHaveBeenNthCalledWith(2, error, LOG_LEVEL_VERBOSE, undefined);
});
it("propagates path-to-ID failures which occur before the guarded lookup", async () => {
const { dependencies, getDBEntry, log, path2id } = createBaseEntryDependencies();
const error = new Error("ID conversion failed");
path2id.mockRejectedValue(error);
await expect(loadHiddenFileSyncBaseEntry(dependencies, path)).rejects.toBe(error);
expect(getDBEntry).not.toHaveBeenCalled();
expect(log).not.toHaveBeenCalled();
});
});
describe("Hidden File Sync live-revision loader", () => {
it("accepts the current winning revision", async () => {
const { current, dependencies, fetchEntryMeta, getConflictedRevs } = createLiveRevisionDependencies();
await expect(loadLiveHiddenFileSyncRevision(dependencies, prefixedPath, current._rev!)).resolves.toBe(current);
expect(fetchEntryMeta).toHaveBeenNthCalledWith(1, prefixedPath, current._rev, true);
expect(fetchEntryMeta).toHaveBeenNthCalledWith(2, prefixedPath, undefined, true);
expect(getConflictedRevs).toHaveBeenCalledWith(prefixedPath);
});
it("accepts a selected conflict leaf while it remains live", async () => {
const { dependencies, selected } = createLiveRevisionDependencies();
await expect(loadLiveHiddenFileSyncRevision(dependencies, prefixedPath, selected._rev!)).resolves.toBe(
selected
);
});
it("accepts a live conflict even when there is no current winner", async () => {
const { dependencies, fetchEntryMeta, selected } = createLiveRevisionDependencies();
fetchEntryMeta.mockImplementation(async (_path: unknown, revision?: string) =>
revision === selected._rev ? selected : false
);
await expect(loadLiveHiddenFileSyncRevision(dependencies, prefixedPath, selected._rev!)).resolves.toBe(
selected
);
});
it("accepts deleted metadata while its revision remains live", async () => {
const { dependencies, fetchEntryMeta, selected } = createLiveRevisionDependencies();
const deleted = { ...selected, deleted: true } as MetaEntry;
fetchEntryMeta.mockImplementation(async (_path: unknown, revision?: string) =>
revision === undefined ? metaEntry("3-current") : deleted
);
await expect(loadLiveHiddenFileSyncRevision(dependencies, prefixedPath, selected._rev!)).resolves.toBe(deleted);
});
it("rejects a selected revision which is no longer current or conflicted", async () => {
const { dependencies, getConflictedRevs, log, selected } = createLiveRevisionDependencies();
getConflictedRevs.mockResolvedValue([]);
await expect(loadLiveHiddenFileSyncRevision(dependencies, prefixedPath, selected._rev!)).resolves.toBe(false);
expect(log).toHaveBeenCalledWith(
`Could not use hidden-file revision ${selected._rev} of ${path}; the selected revision is no longer live`,
LOG_LEVEL_NOTICE,
undefined
);
});
it("rejects a lookup result whose revision does not match the selection", async () => {
const { dependencies, fetchEntryMeta, log, selected } = createLiveRevisionDependencies();
fetchEntryMeta.mockResolvedValue(metaEntry("2-other"));
await expect(loadLiveHiddenFileSyncRevision(dependencies, prefixedPath, selected._rev!)).resolves.toBe(false);
expect(log).toHaveBeenCalledOnce();
});
it("propagates database-file-access failures", async () => {
const { dependencies, fetchEntryMeta, log, selected } = createLiveRevisionDependencies();
const error = new Error("revision lookup failed");
fetchEntryMeta.mockRejectedValue(error);
await expect(loadLiveHiddenFileSyncRevision(dependencies, prefixedPath, selected._rev!)).rejects.toBe(error);
expect(log).not.toHaveBeenCalled();
});
});
@@ -0,0 +1,337 @@
import {
LOG_LEVEL_NOTICE,
LOG_LEVEL_VERBOSE,
type FilePath,
type FilePathWithPrefix,
type LoadedEntry,
type LOG_LEVEL,
type MetaEntry,
type SavingEntry,
type UXFileInfo,
} from "@vrtmrz/livesync-commonlib/compat/common/types";
import { isDocContentSame, readAsBlob } from "@vrtmrz/livesync-commonlib/compat/common/utils";
import type { LogFunction } from "@vrtmrz/livesync-commonlib/compat/services/lib/logUtils";
import { addPrefix, stripAllPrefixes } from "@vrtmrz/livesync-commonlib/compat/string_and_binary/path";
import type { InternalFileInfo } from "@/common/types.ts";
import { ICHeader } from "@/common/types.ts";
import {
serialiseHiddenFileOperation,
type HiddenFileSyncFileSerialisationDependencies,
type HiddenFileSyncFileSerialiser,
} from "./hiddenFileSyncFileOperations.ts";
export type { HiddenFileSyncFileSerialiser } from "./hiddenFileSyncFileOperations.ts";
type HiddenFileSyncDatabaseWriteResponse = {
readonly ok: boolean;
readonly rev: string;
};
export type HiddenFileSyncDatabaseWriteProcessedState = {
updateLastProcessed(path: FilePath, databaseEntry: MetaEntry | LoadedEntry, storageFile: UXFileInfo["stat"]): void;
updateLastProcessedDeletion(path: FilePath, databaseEntry: MetaEntry | LoadedEntry | false): void;
};
export type HiddenFileSyncDatabaseWriteDependencies = {
serialiseFileOperation: HiddenFileSyncFileSerialiser;
isIgnoredByIgnoreFile(path: string): Promise<boolean>;
readFileWithInfo(path: FilePath): Promise<UXFileInfo>;
loadBaseEntry(path: FilePath): Promise<LoadedEntry | false>;
loadBaseMetadata(path: FilePath): Promise<LoadedEntry | false>;
loadLiveRevision(path: FilePathWithPrefix, revision: string): Promise<MetaEntry | false>;
fetchEntryFromMeta(meta: MetaEntry, waitForReady: boolean, skipCheck: boolean): Promise<LoadedEntry | false>;
storeWithBaseRevision(file: UXFileInfo, baseRevision: string, skipCheck: boolean): Promise<string | false>;
putDatabaseEntry(entry: SavingEntry): Promise<false | HiddenFileSyncDatabaseWriteResponse>;
putRaw(entry: LoadedEntry): Promise<HiddenFileSyncDatabaseWriteResponse>;
removeRevision(id: LoadedEntry["_id"], revision: string): Promise<boolean>;
processedState: HiddenFileSyncDatabaseWriteProcessedState;
now(): number;
log: LogFunction;
};
type HiddenFileSyncDatabaseWriteCommonDependencies = Pick<
HiddenFileSyncDatabaseWriteDependencies,
"isIgnoredByIgnoreFile" | "log" | "readFileWithInfo" | "serialiseFileOperation"
>;
export type StoreHiddenFileInDatabaseDependencies = HiddenFileSyncDatabaseWriteCommonDependencies &
Pick<HiddenFileSyncDatabaseWriteDependencies, "loadBaseEntry" | "putDatabaseEntry" | "processedState">;
export type StoreHiddenFileWithBaseRevisionDependencies = HiddenFileSyncDatabaseWriteCommonDependencies &
Pick<
HiddenFileSyncDatabaseWriteDependencies,
"fetchEntryFromMeta" | "loadLiveRevision" | "storeWithBaseRevision" | "processedState"
>;
export type DeleteHiddenFileFromDatabaseDependencies = Pick<
HiddenFileSyncDatabaseWriteDependencies,
| "isIgnoredByIgnoreFile"
| "loadBaseMetadata"
| "log"
| "now"
| "putRaw"
| "removeRevision"
| "serialiseFileOperation"
| "processedState"
>;
export type HiddenFileSyncDatabaseWriteOperations = {
store(file: InternalFileInfo | UXFileInfo, forceWrite?: boolean): Promise<boolean | undefined>;
storeWithBaseRevision(
file: InternalFileInfo | UXFileInfo,
baseRevision: string,
createIfDifferent?: boolean
): Promise<boolean>;
delete(path: FilePath, forceWrite?: boolean): Promise<boolean | undefined>;
};
type HiddenFileSyncDatabaseWriteGuardDependencies = HiddenFileSyncFileSerialisationDependencies &
Pick<HiddenFileSyncDatabaseWriteDependencies, "log">;
function log(
dependencies: Pick<HiddenFileSyncDatabaseWriteDependencies, "log">,
message: unknown,
level?: LOG_LEVEL,
key?: string
) {
dependencies.log(message, level, key);
}
async function runGuardedDatabaseWrite<Result>(
dependencies: HiddenFileSyncDatabaseWriteGuardDependencies,
prefixedFileName: FilePathWithPrefix,
failureMessage: string,
operation: () => Promise<Result>
): Promise<Result | false> {
return await serialiseHiddenFileOperation(dependencies, prefixedFileName, async () => {
try {
return await operation();
} catch (error) {
log(dependencies, failureMessage);
log(dependencies, error, LOG_LEVEL_VERBOSE);
return false;
}
});
}
async function resolvePresentFileInfo(
dependencies: Pick<HiddenFileSyncDatabaseWriteDependencies, "readFileWithInfo">,
file: InternalFileInfo | UXFileInfo,
storeFilePath: FilePath
): Promise<UXFileInfo> {
const fileInfo = "stat" in file && "body" in file ? file : await dependencies.readFileWithInfo(storeFilePath);
if (fileInfo.deleted) {
throw new Error(`Hidden file:${storeFilePath} is deleted. This should not be occurred.`);
}
return fileInfo;
}
export async function storeHiddenFileInDatabase(
dependencies: StoreHiddenFileInDatabaseDependencies,
file: InternalFileInfo | UXFileInfo,
forceWrite = false
): Promise<boolean | undefined> {
const storeFilePath = stripAllPrefixes(file.path);
const storageFilePath = file.path;
// Compatibility: all three admission checks sit outside the guarded lock,
// so policy errors propagate. The ordinary path reports an ignored file as
// undefined, while the selected-revision path reports false.
if (await dependencies.isIgnoredByIgnoreFile(storageFilePath)) {
return undefined;
}
const prefixedFileName = addPrefix(storeFilePath, ICHeader);
return await runGuardedDatabaseWrite(
dependencies,
prefixedFileName,
`STORAGE --> DB:${storageFilePath}: (hidden) Failed`,
async () => {
const fileInfo = await resolvePresentFileInfo(dependencies, file, storeFilePath);
const baseData = await dependencies.loadBaseEntry(storeFilePath);
if (baseData === false) throw new Error("Failed to load base data");
if (baseData._rev && !forceWrite) {
const isSame = await isDocContentSame(readAsBlob(baseData), fileInfo.body);
if (isSame) {
dependencies.processedState.updateLastProcessed(storeFilePath, baseData, fileInfo.stat);
return undefined;
}
}
const saveData: SavingEntry = {
...baseData,
data: fileInfo.body,
mtime: fileInfo.stat.mtime,
size: fileInfo.stat.size,
children: [],
deleted: false,
type: baseData.datatype,
};
// Compatibility question: ctime comes from the old database base,
// not from the storage stat. A newly synthesised base therefore
// stores ctime 0. Preserve this until cross-device effects are known.
const ret = await dependencies.putDatabaseEntry(saveData);
if (ret && ret.ok) {
saveData._rev = ret.rev;
dependencies.processedState.updateLastProcessed(storeFilePath, saveData, fileInfo.stat);
}
const success = ret && ret.ok;
log(dependencies, `STORAGE --> DB:${storageFilePath}: (hidden) ${success ? "Done" : "Failed"}`);
return success;
}
);
}
export async function storeHiddenFileWithBaseRevision(
dependencies: StoreHiddenFileWithBaseRevisionDependencies,
file: InternalFileInfo | UXFileInfo,
baseRevision: string,
createIfDifferent = true
): Promise<boolean> {
const storeFilePath = stripAllPrefixes(file.path);
const storageFilePath = file.path;
if (await dependencies.isIgnoredByIgnoreFile(storageFilePath)) {
return false;
}
const prefixedFileName = addPrefix(storeFilePath, ICHeader);
return await runGuardedDatabaseWrite(
dependencies,
prefixedFileName,
`STORAGE --> DB:${storageFilePath}: (hidden, selected branch) Failed`,
async () => {
// The live check intentionally precedes the storage read. It avoids
// work for a stale selection, but does not make the later write atomic.
const baseData = await dependencies.loadLiveRevision(prefixedFileName, baseRevision);
if (baseData === false) {
return false;
}
const fileInfo = await resolvePresentFileInfo(dependencies, file, storeFilePath);
if (!baseData.deleted && !baseData._deleted) {
const loadedBase = await dependencies.fetchEntryFromMeta(baseData, true, true);
if (loadedBase && (await isDocContentSame(readAsBlob(loadedBase), fileInfo.body))) {
dependencies.processedState.updateLastProcessed(storeFilePath, baseData, fileInfo.stat);
return true;
}
}
if (!createIfDifferent) {
log(
dependencies,
`Could not mark hidden file ${storeFilePath} as revision ${baseRevision}; the storage content differs`,
LOG_LEVEL_NOTICE
);
return false;
}
const storedRevision = await dependencies.storeWithBaseRevision(
{
...fileInfo,
path: storeFilePath,
name: fileInfo.name || storeFilePath.split("/").pop() || "",
isInternal: true,
},
baseRevision,
true
);
if (storedRevision === false) {
return false;
}
// Compatibility question: spreading a selected PouchDB tombstone
// retains `_deleted: true` in the processed-state entry even though
// `deleted` is reset. The stored child itself is created separately.
dependencies.processedState.updateLastProcessed(
storeFilePath,
{
...baseData,
_rev: storedRevision,
path: prefixedFileName,
ctime: fileInfo.stat.ctime,
mtime: fileInfo.stat.mtime,
size: fileInfo.stat.size,
deleted: false,
},
fileInfo.stat
);
log(dependencies, `STORAGE --> DB:${storageFilePath}: (hidden, selected branch) Done`);
return true;
}
);
}
export async function deleteHiddenFileFromDatabase(
dependencies: DeleteHiddenFileFromDatabaseDependencies,
filenameSrc: FilePath,
forceWrite = false
): Promise<boolean | undefined> {
const storeFilePath = filenameSrc;
const storageFilePath = filenameSrc;
const displayFileName = filenameSrc;
const prefixedFileName = addPrefix(storeFilePath, ICHeader);
// Compatibility question: the timestamp is captured before the ignore
// check and before waiting for the per-file lock.
const mtime = dependencies.now();
// Compatibility question: forceWrite is part of the inherited call
// contract, but it has never changed deletion behaviour.
void forceWrite;
if (await dependencies.isIgnoredByIgnoreFile(storageFilePath)) {
return undefined;
}
return await runGuardedDatabaseWrite(
dependencies,
prefixedFileName,
`STORAGE -x> DB: ${displayFileName}: (hidden) Failed`,
async () => {
const baseData = await dependencies.loadBaseMetadata(storeFilePath);
if (baseData === false) throw new Error("Failed to load base data during deleting");
if (baseData._conflicts !== undefined) {
// Compatibility question: these removals are sequential but not
// atomic. Earlier branches remain removed if a later call fails.
for (const conflictRev of baseData._conflicts) {
await dependencies.removeRevision(baseData._id, conflictRev);
log(
dependencies,
`STORAGE -x> DB: ${displayFileName}: (hidden) conflict removed ${baseData._rev} => ${conflictRev}`,
LOG_LEVEL_VERBOSE
);
}
}
// Compatibility question: only the domain `deleted` marker is
// checked here. `_deleted: true` alone causes another tombstone write.
if (baseData.deleted) {
log(dependencies, `STORAGE -x> DB: ${displayFileName}: (hidden) already deleted`, LOG_LEVEL_VERBOSE);
dependencies.processedState.updateLastProcessedDeletion(storeFilePath, baseData);
return true;
}
const saveData: LoadedEntry = {
...baseData,
mtime,
size: 0,
children: [],
deleted: true,
type: baseData.datatype,
};
// A synthesised base has no revision; the inherited behaviour still
// writes it as a tombstone when the requested path was absent.
const ret = await dependencies.putRaw(saveData);
if (ret && ret.ok) {
log(dependencies, `STORAGE -x> DB: ${displayFileName}: (hidden) Done`);
saveData._rev = ret.rev;
dependencies.processedState.updateLastProcessedDeletion(storeFilePath, saveData);
return true;
} else {
log(dependencies, `STORAGE -x> DB: ${displayFileName}: (hidden) Failed`);
return false;
}
}
);
}
export function createHiddenFileSyncDatabaseWriteOperations(
dependencies: HiddenFileSyncDatabaseWriteDependencies
): HiddenFileSyncDatabaseWriteOperations {
return Object.freeze({
store: async (file, forceWrite) => await storeHiddenFileInDatabase(dependencies, file, forceWrite),
storeWithBaseRevision: async (file, baseRevision, createIfDifferent) =>
await storeHiddenFileWithBaseRevision(dependencies, file, baseRevision, createIfDifferent),
delete: async (path, forceWrite) => await deleteHiddenFileFromDatabase(dependencies, path, forceWrite),
});
}
@@ -0,0 +1,372 @@
import { describe, expect, it, vi } from "vitest";
import {
LOG_LEVEL_NOTICE,
LOG_LEVEL_VERBOSE,
type DocumentID,
type FilePath,
type FilePathWithPrefix,
type LoadedEntry,
type MetaEntry,
type UXFileInfo,
} from "@vrtmrz/livesync-commonlib/compat/common/types";
import {
deleteHiddenFileFromDatabase,
storeHiddenFileInDatabase,
storeHiddenFileWithBaseRevision,
type HiddenFileSyncDatabaseWriteDependencies,
} from "./hiddenFileSyncDatabaseWriteOperations.ts";
const path = ".obsidian/plugins/example/data.json" as FilePath;
const prefixedPath = `i:${path}` as FilePathWithPrefix;
const id = "hidden-entry-id" as DocumentID;
function fileInfo(content = '{"value":"vault"}'): UXFileInfo {
return {
path,
name: "data.json",
isInternal: true,
body: new Blob([content]),
stat: {
ctime: 41,
mtime: 42,
size: content.length,
type: "file",
},
deleted: false,
} as UXFileInfo;
}
function loadedEntry(overrides: Partial<LoadedEntry> = {}): LoadedEntry {
return {
_id: id,
_rev: "2-current",
path: prefixedPath,
type: "plain",
datatype: "plain",
data: '{"value":"database"}',
ctime: 10,
mtime: 20,
size: 20,
children: [],
eden: {},
deleted: false,
...overrides,
} as LoadedEntry;
}
function createDependencies(base = loadedEntry()) {
const events: string[] = [];
const serialiseFileOperation = vi.fn(async (_key: string, operation: () => Promise<unknown>) => {
events.push("lock:start");
try {
return await operation();
} finally {
events.push("lock:end");
}
});
const isIgnoredByIgnoreFile = vi.fn(async () => false);
const readFileWithInfo = vi.fn(async () => fileInfo());
const loadBaseEntry = vi.fn(async () => base as LoadedEntry | false);
const loadBaseMetadata = vi.fn(async () => base as LoadedEntry | false);
const loadLiveRevision = vi.fn(async (_path: FilePathWithPrefix, revision: string) =>
revision === base._rev ? (base as MetaEntry) : false
);
const fetchEntryFromMeta = vi.fn(async () => base as LoadedEntry | false);
const storeWithBaseRevision = vi.fn(async () => "3-selected-child" as string | false);
const putDatabaseEntry = vi.fn(async (_entry: unknown) => ({ ok: true, id, rev: "3-written" }));
const putRaw = vi.fn(async (_entry: LoadedEntry) => ({ ok: true, id, rev: "3-deleted" }));
const removeRevision = vi.fn(async () => true);
const updateLastProcessed = vi.fn(() => events.push("state:file"));
const updateLastProcessedDeletion = vi.fn(() => events.push("state:deletion"));
const processedState = {
updateLastProcessed,
updateLastProcessedDeletion,
};
const now = vi.fn(() => 1_000);
const log = vi.fn();
const dependencies = {
serialiseFileOperation,
isIgnoredByIgnoreFile,
readFileWithInfo,
loadBaseEntry,
loadBaseMetadata,
loadLiveRevision,
fetchEntryFromMeta,
storeWithBaseRevision,
putDatabaseEntry,
putRaw,
removeRevision,
processedState,
now,
log,
} as unknown as HiddenFileSyncDatabaseWriteDependencies;
return {
base,
dependencies,
events,
fetchEntryFromMeta,
isIgnoredByIgnoreFile,
loadBaseEntry,
loadBaseMetadata,
loadLiveRevision,
log,
now,
putDatabaseEntry,
putRaw,
readFileWithInfo,
removeRevision,
serialiseFileOperation,
storeWithBaseRevision,
updateLastProcessed,
updateLastProcessedDeletion,
};
}
describe("ordinary hidden-file database writes", () => {
it("keeps the synthetic base ctime and settles the new revision inside the file lock", async () => {
const base = loadedEntry({
_rev: undefined,
ctime: 0,
data: [],
datatype: "newnote",
type: "newnote",
});
const fixture = createDependencies(base);
const file = fileInfo();
await expect(storeHiddenFileInDatabase(fixture.dependencies, file)).resolves.toBe(true);
expect(fixture.putDatabaseEntry).toHaveBeenCalledWith(
expect.objectContaining({ ctime: 0, mtime: file.stat.mtime, data: file.body })
);
expect(fixture.updateLastProcessed).toHaveBeenCalledWith(
path,
expect.objectContaining({ _rev: "3-written", ctime: 0 }),
file.stat
);
expect(fixture.serialiseFileOperation).toHaveBeenCalledWith(`file-${prefixedPath}`, expect.any(Function));
expect(fixture.events).toEqual(["lock:start", "state:file", "lock:end"]);
});
it("settles matching content without writing or emitting a transfer log", async () => {
const base = loadedEntry({ data: '{"value":"vault"}' });
const fixture = createDependencies(base);
const file = fileInfo();
await expect(storeHiddenFileInDatabase(fixture.dependencies, file)).resolves.toBeUndefined();
expect(fixture.putDatabaseEntry).not.toHaveBeenCalled();
expect(fixture.updateLastProcessed).toHaveBeenCalledWith(path, base, file.stat);
expect(fixture.log).not.toHaveBeenCalled();
});
it("writes matching content when forceWrite is enabled", async () => {
const fixture = createDependencies(loadedEntry({ data: '{"value":"vault"}' }));
await expect(storeHiddenFileInDatabase(fixture.dependencies, fileInfo(), true)).resolves.toBe(true);
expect(fixture.putDatabaseEntry).toHaveBeenCalledOnce();
});
it("returns false and reports both messages when a guarded write fails", async () => {
const fixture = createDependencies();
const error = new Error("storage read failed");
fixture.readFileWithInfo.mockRejectedValue(error);
await expect(
storeHiddenFileInDatabase(fixture.dependencies, {
path,
ctime: 1,
mtime: 2,
size: 3,
})
).resolves.toBe(false);
expect(fixture.log).toHaveBeenNthCalledWith(1, `STORAGE --> DB:${path}: (hidden) Failed`, undefined, undefined);
expect(fixture.log).toHaveBeenNthCalledWith(2, error, LOG_LEVEL_VERBOSE, undefined);
});
});
describe("selected-revision hidden-file database writes", () => {
it("stores the Vault content as a child of a selected live revision", async () => {
const fixture = createDependencies();
const file = fileInfo();
await expect(storeHiddenFileWithBaseRevision(fixture.dependencies, file, fixture.base._rev!)).resolves.toBe(
true
);
expect(fixture.storeWithBaseRevision).toHaveBeenCalledWith(
expect.objectContaining({ path, body: file.body, isInternal: true }),
fixture.base._rev,
true
);
expect(fixture.updateLastProcessed).toHaveBeenCalledWith(
path,
expect.objectContaining({ _rev: "3-selected-child" }),
file.stat
);
});
it("validates liveness before reading storage and refuses a stale revision", async () => {
const fixture = createDependencies();
fixture.loadLiveRevision.mockResolvedValue(false);
await expect(
storeHiddenFileWithBaseRevision(fixture.dependencies, { path, ctime: 1, mtime: 2, size: 3 }, "2-stale")
).resolves.toBe(false);
expect(fixture.readFileWithInfo).not.toHaveBeenCalled();
expect(fixture.storeWithBaseRevision).not.toHaveBeenCalled();
expect(fixture.updateLastProcessed).not.toHaveBeenCalled();
});
it("marks matching content without creating a child", async () => {
const base = loadedEntry({ data: '{"value":"vault"}' });
const fixture = createDependencies(base);
const file = fileInfo();
await expect(storeHiddenFileWithBaseRevision(fixture.dependencies, file, base._rev!, false)).resolves.toBe(
true
);
expect(fixture.storeWithBaseRevision).not.toHaveBeenCalled();
expect(fixture.updateLastProcessed).toHaveBeenCalledWith(path, base, file.stat);
});
it("reports differing content without creating a child when requested", async () => {
const fixture = createDependencies();
await expect(
storeHiddenFileWithBaseRevision(fixture.dependencies, fileInfo(), fixture.base._rev!, false)
).resolves.toBe(false);
expect(fixture.storeWithBaseRevision).not.toHaveBeenCalled();
expect(fixture.updateLastProcessed).not.toHaveBeenCalled();
expect(fixture.log).toHaveBeenCalledWith(
`Could not mark hidden file ${path} as revision ${fixture.base._rev}; the storage content differs`,
LOG_LEVEL_NOTICE,
undefined
);
});
it("keeps a selected branch's _deleted marker in the processed-state entry", async () => {
const base = loadedEntry({ deleted: true, _deleted: true });
const fixture = createDependencies(base);
const file = fileInfo();
await expect(storeHiddenFileWithBaseRevision(fixture.dependencies, file, base._rev!)).resolves.toBe(true);
expect(fixture.fetchEntryFromMeta).not.toHaveBeenCalled();
expect(fixture.updateLastProcessed).toHaveBeenCalledWith(
path,
expect.objectContaining({ _rev: "3-selected-child", deleted: false, _deleted: true }),
file.stat
);
});
});
describe("hidden-file database deletions", () => {
it("removes conflicts before accepting an already deleted entry", async () => {
const base = loadedEntry({ deleted: true, _conflicts: ["2-conflict"] });
const fixture = createDependencies(base);
await expect(deleteHiddenFileFromDatabase(fixture.dependencies, path, true)).resolves.toBe(true);
expect(fixture.removeRevision).toHaveBeenCalledWith(id, "2-conflict");
expect(fixture.putRaw).not.toHaveBeenCalled();
expect(fixture.updateLastProcessedDeletion).toHaveBeenCalledWith(path, base);
});
it("writes a deletion when only the PouchDB _deleted marker is present", async () => {
const base = loadedEntry({ deleted: false, _deleted: true });
const fixture = createDependencies(base);
await expect(deleteHiddenFileFromDatabase(fixture.dependencies, path)).resolves.toBe(true);
expect(fixture.putRaw).toHaveBeenCalledWith(expect.objectContaining({ deleted: true, _deleted: true }));
expect(fixture.updateLastProcessedDeletion).toHaveBeenCalledWith(
path,
expect.objectContaining({ _rev: "3-deleted" })
);
});
it("writes a tombstone for a synthetic base without a revision", async () => {
const base = loadedEntry({ _rev: undefined, data: [], datatype: "newnote", type: "newnote" });
const fixture = createDependencies(base);
let submitted: LoadedEntry | undefined;
fixture.putRaw.mockImplementation(async (entry: LoadedEntry) => {
submitted = { ...entry };
return { ok: true, id, rev: "3-deleted" };
});
await expect(deleteHiddenFileFromDatabase(fixture.dependencies, path)).resolves.toBe(true);
expect(submitted).toEqual(expect.objectContaining({ _rev: undefined, deleted: true, type: "newnote" }));
});
it("keeps earlier conflict removals when a later removal fails", async () => {
const base = loadedEntry({ _conflicts: ["2-first", "2-second"] });
const fixture = createDependencies(base);
const error = new Error("second removal failed");
fixture.removeRevision.mockResolvedValueOnce(true).mockRejectedValueOnce(error);
await expect(deleteHiddenFileFromDatabase(fixture.dependencies, path)).resolves.toBe(false);
expect(fixture.removeRevision.mock.calls).toEqual([
[id, "2-first"],
[id, "2-second"],
]);
expect(fixture.putRaw).not.toHaveBeenCalled();
expect(fixture.updateLastProcessedDeletion).not.toHaveBeenCalled();
expect(fixture.log).toHaveBeenLastCalledWith(error, LOG_LEVEL_VERBOSE, undefined);
});
it("captures the deletion time before evaluating ignore policy", async () => {
const fixture = createDependencies();
const events: string[] = [];
fixture.now.mockImplementation(() => {
events.push("now");
return 1_000;
});
fixture.isIgnoredByIgnoreFile.mockImplementation(async () => {
events.push("ignore");
return true;
});
await expect(deleteHiddenFileFromDatabase(fixture.dependencies, path)).resolves.toBeUndefined();
expect(events).toEqual(["now", "ignore"]);
expect(fixture.serialiseFileOperation).not.toHaveBeenCalled();
});
});
describe("hidden-file database write admission", () => {
it("preserves the different ignored results of the three write paths", async () => {
const fixture = createDependencies();
fixture.isIgnoredByIgnoreFile.mockResolvedValue(true);
const file = fileInfo();
await expect(storeHiddenFileInDatabase(fixture.dependencies, file)).resolves.toBeUndefined();
await expect(storeHiddenFileWithBaseRevision(fixture.dependencies, file, fixture.base._rev!)).resolves.toBe(
false
);
await expect(deleteHiddenFileFromDatabase(fixture.dependencies, path)).resolves.toBeUndefined();
expect(fixture.loadBaseEntry).not.toHaveBeenCalled();
expect(fixture.loadBaseMetadata).not.toHaveBeenCalled();
expect(fixture.loadLiveRevision).not.toHaveBeenCalled();
});
it("propagates ignore-policy failures before entering the guarded lock", async () => {
const fixture = createDependencies();
const error = new Error("ignore policy unavailable");
fixture.isIgnoredByIgnoreFile.mockRejectedValue(error);
await expect(storeHiddenFileInDatabase(fixture.dependencies, fileInfo())).rejects.toBe(error);
expect(fixture.serialiseFileOperation).not.toHaveBeenCalled();
expect(fixture.log).not.toHaveBeenCalled();
});
});
@@ -0,0 +1,18 @@
import type { FilePathWithPrefix } from "@vrtmrz/livesync-commonlib/compat/common/types";
export type HiddenFileSyncFileSerialiser = <Result>(key: string, operation: () => Promise<Result>) => Promise<Result>;
export type HiddenFileSyncFileSerialisationDependencies = {
serialiseFileOperation: HiddenFileSyncFileSerialiser;
};
export async function serialiseHiddenFileOperation<Result>(
dependencies: HiddenFileSyncFileSerialisationDependencies,
prefixedFileName: FilePathWithPrefix,
operation: () => Promise<Result>
): Promise<Result> {
// Compatibility question: this inherited lock uses `file-`, whereas the
// Commonlib database writer uses `file:`. The two writers therefore do not
// mutually exclude one another; changing the key needs concurrency tests.
return await dependencies.serialiseFileOperation(`file-${prefixedFileName}`, operation);
}
@@ -0,0 +1,73 @@
import type { CustomRegExpSourceList, FilePath } from "@vrtmrz/livesync-commonlib/compat/common/types";
import type { CustomRegExp } from "@vrtmrz/livesync-commonlib/compat/common/utils";
import {
isHiddenFileSyncPath,
matchesHiddenFileSyncPatterns,
type HiddenFileSyncPathFilters,
} from "./hiddenFileSyncPathPolicy.ts";
type HiddenFileSyncPathRegExpKey = "syncInternalFilesIgnorePatterns" | "syncInternalFilesTargetPatterns";
export type HiddenFileSyncPathAdmissionDependencies = {
getTargetPatternSource(): CustomRegExpSourceList<",">;
getIgnorePatternSource(): CustomRegExpSourceList<",">;
getFileRegExp(key: HiddenFileSyncPathRegExpKey): readonly CustomRegExp[];
isIgnoredByIgnoreFile(path: FilePath): Promise<boolean>;
ownsLocalFile(path: FilePath): boolean;
};
export type HiddenFileSyncPathAdmission = {
isTargetFileEligible(path: FilePath): Promise<boolean>;
isTargetFile(path: FilePath): Promise<boolean>;
invalidatePatternCache(): void;
dispose(): void;
};
class HiddenFileSyncPathAdmissionOwner implements HiddenFileSyncPathAdmission {
private readonly cacheFileRegExps = new Map<string, HiddenFileSyncPathFilters>();
constructor(private readonly dependencies: HiddenFileSyncPathAdmissionDependencies) {}
private parseRegExpSettings(): HiddenFileSyncPathFilters {
const targetPatternSource = this.dependencies.getTargetPatternSource();
const ignorePatternSource = this.dependencies.getIgnorePatternSource();
const regExpKey = `${targetPatternSource}||${ignorePatternSource}`;
const cached = this.cacheFileRegExps.get(regExpKey);
if (cached) return cached;
// Keep the inherited parser order: ignore patterns are read before target patterns.
const ignoreFilter = this.dependencies.getFileRegExp("syncInternalFilesIgnorePatterns");
const targetFilter = this.dependencies.getFileRegExp("syncInternalFilesTargetPatterns");
const filters: HiddenFileSyncPathFilters = { ignoreFilter, targetFilter };
this.cacheFileRegExps.clear();
this.cacheFileRegExps.set(regExpKey, filters);
return filters;
}
async isTargetFileEligible(path: FilePath): Promise<boolean> {
const result = matchesHiddenFileSyncPatterns(path, this.parseRegExpSettings()) && isHiddenFileSyncPath(path);
if (!result) return false;
return !(await this.dependencies.isIgnoredByIgnoreFile(path));
}
async isTargetFile(path: FilePath): Promise<boolean> {
// Ownership is checked first so another optional-file owner cannot be filtered by this feature.
if (this.dependencies.ownsLocalFile(path) === false) return false;
return await this.isTargetFileEligible(path);
}
invalidatePatternCache(): void {
this.cacheFileRegExps.clear();
}
dispose(): void {
this.cacheFileRegExps.clear();
}
}
export function createHiddenFileSyncPathAdmission(
dependencies: HiddenFileSyncPathAdmissionDependencies
): HiddenFileSyncPathAdmission {
return new HiddenFileSyncPathAdmissionOwner(dependencies);
}
@@ -0,0 +1,139 @@
import { describe, expect, it, vi } from "vitest";
import type { CustomRegExpSourceList, FilePath } from "@vrtmrz/livesync-commonlib/compat/common/types";
import type { CustomRegExp } from "@vrtmrz/livesync-commonlib/compat/common/utils";
import {
createHiddenFileSyncPathAdmission,
type HiddenFileSyncPathAdmissionDependencies,
} from "./hiddenFileSyncPathAdmission.ts";
const PATH = ".obsidian/plugins/example/data.json" as FilePath;
const pattern = (matches: (path: string) => boolean) => ({ test: vi.fn(matches) }) as unknown as CustomRegExp;
function createDependencies(
overrides: Partial<HiddenFileSyncPathAdmissionDependencies> = {}
): HiddenFileSyncPathAdmissionDependencies & {
getFileRegExp: ReturnType<typeof vi.fn>;
isIgnoredByIgnoreFile: ReturnType<typeof vi.fn>;
ownsLocalFile: ReturnType<typeof vi.fn>;
getTargetPatternSource: ReturnType<typeof vi.fn>;
getIgnorePatternSource: ReturnType<typeof vi.fn>;
} {
const getTargetPatternSource = vi.fn(() => "target" as CustomRegExpSourceList<",">);
const getIgnorePatternSource = vi.fn(() => "ignore" as CustomRegExpSourceList<",">);
const getFileRegExp = vi.fn((key: "syncInternalFilesIgnorePatterns" | "syncInternalFilesTargetPatterns") => {
if (key == "syncInternalFilesIgnorePatterns") return [];
return [];
});
const isIgnoredByIgnoreFile = vi.fn(async () => false);
const ownsLocalFile = vi.fn(() => true);
return {
getTargetPatternSource,
getIgnorePatternSource,
getFileRegExp,
isIgnoredByIgnoreFile,
ownsLocalFile,
...overrides,
} as HiddenFileSyncPathAdmissionDependencies & {
getFileRegExp: ReturnType<typeof vi.fn>;
isIgnoredByIgnoreFile: ReturnType<typeof vi.fn>;
ownsLocalFile: ReturnType<typeof vi.fn>;
getTargetPatternSource: ReturnType<typeof vi.fn>;
getIgnorePatternSource: ReturnType<typeof vi.fn>;
};
}
describe("Hidden File Sync path admission", () => {
it("checks composition ownership before reading pattern settings", async () => {
const dependencies = createDependencies({ ownsLocalFile: vi.fn(() => false) });
const admission = createHiddenFileSyncPathAdmission(dependencies);
await expect(admission.isTargetFile(PATH)).resolves.toBe(false);
expect(dependencies.ownsLocalFile).toHaveBeenCalledWith(PATH);
expect(dependencies.getTargetPatternSource).not.toHaveBeenCalled();
expect(dependencies.getIgnorePatternSource).not.toHaveBeenCalled();
expect(dependencies.getFileRegExp).not.toHaveBeenCalled();
expect(dependencies.isIgnoredByIgnoreFile).not.toHaveBeenCalled();
});
it("checks static path and pattern policy before the asynchronous ignore-file policy", async () => {
const isIgnoredByIgnoreFile = vi.fn(async () => false);
const targetFilter = [pattern(() => false)];
const dependencies = createDependencies({
getFileRegExp: vi.fn((key: "syncInternalFilesIgnorePatterns" | "syncInternalFilesTargetPatterns") =>
key == "syncInternalFilesTargetPatterns" ? targetFilter : []
),
isIgnoredByIgnoreFile,
});
const admission = createHiddenFileSyncPathAdmission(dependencies);
await expect(admission.isTargetFile(PATH)).resolves.toBe(false);
await expect(admission.isTargetFile("notes/example.md" as FilePath)).resolves.toBe(false);
expect(isIgnoredByIgnoreFile).not.toHaveBeenCalled();
});
it("adopts target and ignore patterns while preserving their policy order", async () => {
const calls: string[] = [];
const targetFilter = [pattern(() => true)];
const ignoreFilter = [pattern(() => true)];
const getFileRegExp = vi.fn((key: "syncInternalFilesIgnorePatterns" | "syncInternalFilesTargetPatterns") => {
calls.push(key);
return key == "syncInternalFilesTargetPatterns" ? targetFilter : ignoreFilter;
});
const isIgnoredByIgnoreFile = vi.fn(async () => false);
const dependencies = createDependencies({ getFileRegExp, isIgnoredByIgnoreFile });
const admission = createHiddenFileSyncPathAdmission(dependencies);
await expect(admission.isTargetFile(PATH)).resolves.toBe(false);
expect(calls).toEqual(["syncInternalFilesIgnorePatterns", "syncInternalFilesTargetPatterns"]);
expect(targetFilter[0].test).not.toHaveBeenCalled();
expect(isIgnoredByIgnoreFile).not.toHaveBeenCalled();
});
it("caches parsed filters per owner and refreshes them when sources or settings change", async () => {
const targetFilter = [pattern(() => true)];
const ignoreFilter: CustomRegExp[] = [];
let targetSource = "target" as CustomRegExpSourceList<",">;
const getTargetPatternSource = vi.fn(() => targetSource);
const getFileRegExp = vi.fn((key: "syncInternalFilesIgnorePatterns" | "syncInternalFilesTargetPatterns") =>
key == "syncInternalFilesTargetPatterns" ? targetFilter : ignoreFilter
);
const dependencies = createDependencies({ getTargetPatternSource, getFileRegExp });
const admission = createHiddenFileSyncPathAdmission(dependencies);
await expect(admission.isTargetFile(PATH)).resolves.toBe(true);
await expect(admission.isTargetFile(PATH)).resolves.toBe(true);
expect(getFileRegExp).toHaveBeenCalledTimes(2);
targetSource = "changed" as CustomRegExpSourceList<",">;
await expect(admission.isTargetFile(PATH)).resolves.toBe(true);
expect(getFileRegExp).toHaveBeenCalledTimes(4);
admission.invalidatePatternCache();
await expect(admission.isTargetFile(PATH)).resolves.toBe(true);
expect(getFileRegExp).toHaveBeenCalledTimes(6);
});
it("does not share the pattern cache between owners and clears it on disposal", async () => {
const firstDependencies = createDependencies();
const secondDependencies = createDependencies();
const first = createHiddenFileSyncPathAdmission(firstDependencies);
const second = createHiddenFileSyncPathAdmission(secondDependencies);
await first.isTargetFile(PATH);
await first.isTargetFile(PATH);
await second.isTargetFile(PATH);
expect(firstDependencies.getFileRegExp).toHaveBeenCalledTimes(2);
expect(secondDependencies.getFileRegExp).toHaveBeenCalledTimes(2);
first.dispose();
first.dispose();
await first.isTargetFile(PATH);
expect(firstDependencies.getFileRegExp).toHaveBeenCalledTimes(4);
expect(secondDependencies.getFileRegExp).toHaveBeenCalledTimes(2);
});
});
@@ -0,0 +1,22 @@
import type { CustomRegExp } from "@vrtmrz/livesync-commonlib/compat/common/utils";
export type HiddenFileSyncPathFilters = {
ignoreFilter: readonly CustomRegExp[];
targetFilter: readonly CustomRegExp[];
};
export function isHiddenFileSyncPath(path: string): boolean {
// Compatibility: this prefix check also excludes names such as `.trashcan`.
// Keep the broader exclusion until a separate path-policy decision changes it.
return path.startsWith(".") && !path.startsWith(".trash");
}
export function matchesHiddenFileSyncPatterns(path: string, filters: HiddenFileSyncPathFilters): boolean {
if (filters.ignoreFilter.some((pattern) => pattern.test(path))) {
return false;
}
if (filters.targetFilter.length > 0) {
return filters.targetFilter.some((pattern) => pattern.test(path));
}
return true;
}
@@ -0,0 +1,41 @@
import { describe, expect, it, vi } from "vitest";
import type { CustomRegExp } from "@vrtmrz/livesync-commonlib/compat/common/utils";
import { isHiddenFileSyncPath, matchesHiddenFileSyncPatterns } from "./hiddenFileSyncPathPolicy.ts";
const pattern = (matches: (path: string) => boolean) => ({ test: vi.fn(matches) }) as unknown as CustomRegExp;
describe("isHiddenFileSyncPath", () => {
it.each([
[".obsidian/app.json", true],
[".git/config", true],
[".trash/app.json", false],
[".trashcan/app.json", false],
["notes/.hidden", false],
])("classifies %s as a Hidden File Sync path=%s", (path, expected) => {
expect(isHiddenFileSyncPath(path)).toBe(expected);
});
});
describe("matchesHiddenFileSyncPatterns", () => {
it("allows every path when no filters are configured", () => {
expect(matchesHiddenFileSyncPatterns(".obsidian/app.json", { ignoreFilter: [], targetFilter: [] })).toBe(true);
});
it("uses target patterns as an allow-list", () => {
const targetFilter = [pattern((path) => path.endsWith(".json"))];
expect(matchesHiddenFileSyncPatterns(".obsidian/app.json", { ignoreFilter: [], targetFilter })).toBe(true);
expect(matchesHiddenFileSyncPatterns(".obsidian/theme.css", { ignoreFilter: [], targetFilter })).toBe(false);
});
it("gives ignore patterns precedence over target patterns", () => {
const matchesEverything = pattern(() => true);
expect(
matchesHiddenFileSyncPatterns(".obsidian/app.json", {
ignoreFilter: [matchesEverything],
targetFilter: [matchesEverything],
})
).toBe(false);
});
});
@@ -0,0 +1,202 @@
import {
LOG_LEVEL_VERBOSE,
type FilePath,
type LoadedEntry,
type LOG_LEVEL,
type MetaEntry,
type UXStat,
} from "@vrtmrz/livesync-commonlib/compat/common/types";
import type { KeyValueDatabase } from "@vrtmrz/livesync-commonlib/compat/interfaces/KeyValueDatabase";
import type { LiveSyncLocalDB } from "@vrtmrz/livesync-commonlib/compat/pouchdb/LiveSyncLocalDB";
import type { IPathService } from "@vrtmrz/livesync-commonlib/compat/services/base/IService";
import type { LogFunction } from "@vrtmrz/livesync-commonlib/compat/services/lib/logUtils";
import { addPrefix } from "@vrtmrz/livesync-commonlib/compat/string_and_binary/path";
import { ICHeader } from "@/common/types.ts";
import { autosaveCache, type MapLike } from "@/common/utils.ts";
import {
getHiddenFileSyncComparisonMTime,
toHiddenFileSyncDatabaseStateKey,
toHiddenFileSyncStorageStateKey,
} from "./hiddenFileSyncState.ts";
type HiddenFileSyncProcessedStateDatabase = Pick<LiveSyncLocalDB, "getDBEntryMeta">;
type HiddenFileSyncProcessedStateStorage = {
statHidden(path: FilePath): Promise<UXStat | null>;
};
export type HiddenFileSyncProcessedStateDependencies = {
getKeyValueDatabase(): KeyValueDatabase;
getLocalDatabase(): HiddenFileSyncProcessedStateDatabase;
storageAccess: HiddenFileSyncProcessedStateStorage;
path: Pick<IPathService, "markChangesAreSame" | "unmarkChanges">;
log: LogFunction;
};
export class HiddenFileSyncProcessedState {
private fileInfoLastProcessed: MapLike<string, string> = new Map();
private fileInfoLastKnown: MapLike<string, number> = new Map();
private databaseInfoLastProcessed: MapLike<string, string> = new Map();
constructor(private readonly dependencies: HiddenFileSyncProcessedStateDependencies) {}
private log(message: unknown, level?: LOG_LEVEL, key?: string): void {
this.dependencies.log(message, level, key);
}
async initialise(): Promise<void> {
// Compatibility question: these reads intentionally remain
// sequential and in this order.
// Compatibility question: autosaveCache has no flush or disposal
// boundary, so a delayed write from an earlier database lifecycle can
// outlive this state owner. Preserve that timing until database
// replacement and unload behaviour have focused coverage.
this.fileInfoLastProcessed = await autosaveCache(
this.dependencies.getKeyValueDatabase(),
"hidden-file-lastProcessed"
);
this.databaseInfoLastProcessed = await autosaveCache(
this.dependencies.getKeyValueDatabase(),
"hidden-file-lastProcessed-database"
);
this.fileInfoLastKnown = await autosaveCache(this.dependencies.getKeyValueDatabase(), "hidden-file-lastKnown");
}
getLastProcessedFileCount(): number {
return this.fileInfoLastProcessed.size;
}
getLastProcessedFileKeys(): IterableIterator<string> {
return this.fileInfoLastProcessed.keys();
}
hasLastProcessedFile(file: FilePath): boolean {
return this.fileInfoLastProcessed.has(file);
}
hasLastProcessedDatabase(file: FilePath): boolean {
return this.databaseInfoLastProcessed.has(file);
}
async fileToStatKey(file: FilePath, stat: UXStat | null = null): Promise<string> {
// Compatibility question: `null` means 'stat not supplied' here
// rather than 'file missing', so a failed earlier stat causes another
// read. Keep that retry until its event-processing effect is
// characterised.
if (!stat) stat = await this.dependencies.storageAccess.statHidden(file);
return this.storageStateKey(stat);
}
storageStateKey(stat: UXStat | null): string {
return toHiddenFileSyncStorageStateKey(stat);
}
databaseStateKey(doc: MetaEntry | LoadedEntry): string {
return toHiddenFileSyncDatabaseStateKey(doc);
}
updateLastProcessedFile(file: FilePath, keySrc: string | UXStat): void {
const key = typeof keySrc == "string" ? keySrc : this.storageStateKey(keySrc);
const splitted = key.split("-");
if (splitted[0] != "0") {
// Compatibility: a zero storage marker does not replace the last
// known non-zero mtime. Deletion therefore retains that fallback.
this.fileInfoLastKnown.set(file, Number(splitted[0]));
}
this.fileInfoLastProcessed.set(file, key);
}
async updateLastProcessedAsActualFile(file: FilePath, stat?: UXStat | null): Promise<void> {
if (!stat) stat = await this.dependencies.storageAccess.statHidden(file);
// Compatibility: adoption updates only the processed marker. It does
// not update the last-known non-zero mtime cache.
this.fileInfoLastProcessed.set(file, this.storageStateKey(stat));
}
resetLastProcessedFile(targetFiles: FilePath[] | false): void {
if (targetFiles) {
for (const key of targetFiles) {
this.fileInfoLastProcessed.delete(key);
}
} else {
this.log(`Delete all processed mark.`, LOG_LEVEL_VERBOSE);
// THINKING: Should we...
// - delete all `Known file` processed mark? (This is current implementation)
// - delete all `Existing file` processed mark?
// - delete all files inside the config folder of current device mark?
this.fileInfoLastProcessed.clear();
}
}
getLastProcessedFileMTime(file: FilePath): number {
const key = this.fileInfoLastKnown.get(file);
if (!key) return 0;
return key;
}
getLastProcessedFileKey(file: FilePath): string | undefined {
return this.fileInfoLastProcessed.get(file);
}
getLastProcessedDatabaseKey(file: FilePath): string | undefined {
return this.databaseInfoLastProcessed.get(file);
}
updateLastProcessedDatabase(file: FilePath, keySrc: string | MetaEntry | LoadedEntry): void {
const key = typeof keySrc == "string" ? keySrc : this.databaseStateKey(keySrc);
this.databaseInfoLastProcessed.set(file, key);
}
updateLastProcessed(path: FilePath, db: MetaEntry | LoadedEntry, stat: UXStat): void {
this.updateLastProcessedDatabase(path, db);
this.updateLastProcessedFile(path, this.storageStateKey(stat));
const dbMTime = getHiddenFileSyncComparisonMTime(db);
const storageMTime = getHiddenFileSyncComparisonMTime(stat);
if (dbMTime == 0 || storageMTime == 0) {
this.dependencies.path.unmarkChanges(path);
} else {
this.dependencies.path.markChangesAreSame(
path,
getHiddenFileSyncComparisonMTime(db),
getHiddenFileSyncComparisonMTime(stat)
);
}
}
updateLastProcessedDeletion(path: FilePath, db: MetaEntry | LoadedEntry | false): void {
this.dependencies.path.unmarkChanges(path);
if (db) this.updateLastProcessedDatabase(path, db);
this.updateLastProcessedFile(path, this.storageStateKey(null));
}
async updateLastProcessedAsActualDatabase(
file: FilePath,
doc?: MetaEntry | LoadedEntry | null | false
): Promise<void> {
const dbPath = addPrefix(file, ICHeader);
if (!doc) doc = await this.dependencies.getLocalDatabase().getDBEntryMeta(dbPath);
if (!doc) return;
this.databaseInfoLastProcessed.set(file, this.databaseStateKey(doc));
}
resetLastProcessedDatabase(targetFiles: FilePath[] | false): void {
if (targetFiles) {
for (const key of targetFiles) {
this.databaseInfoLastProcessed.delete(key);
}
} else {
this.log(`Delete all processed mark.`, LOG_LEVEL_VERBOSE);
// THINKING: Should we...
// - delete all `Known file` processed mark? (This is current implementation)
// - delete all `Existing file` processed mark?
// - delete all files inside the config folder of current device mark?
this.databaseInfoLastProcessed.clear();
}
}
}
export function createHiddenFileSyncProcessedState(
dependencies: HiddenFileSyncProcessedStateDependencies
): HiddenFileSyncProcessedState {
return new HiddenFileSyncProcessedState(dependencies);
}
@@ -0,0 +1,182 @@
import { describe, expect, it, vi } from "vitest";
import {
LOG_LEVEL_VERBOSE,
type FilePath,
type LoadedEntry,
type MetaEntry,
type UXStat,
} from "@vrtmrz/livesync-commonlib/compat/common/types";
import type { KeyValueDatabase } from "@vrtmrz/livesync-commonlib/compat/interfaces/KeyValueDatabase";
vi.mock("@/deps.ts", () => ({}));
import {
createHiddenFileSyncProcessedState,
type HiddenFileSyncProcessedStateDependencies,
} from "./hiddenFileSyncProcessedState.ts";
import { toHiddenFileSyncDatabaseStateKey } from "./hiddenFileSyncState.ts";
const path = ".obsidian/plugins/example/data.json" as FilePath;
function stat(mtime: number, size = 20): UXStat {
return { ctime: mtime, mtime, size, type: "file" };
}
function metadata(overrides: Partial<MetaEntry> = {}): MetaEntry {
return {
_id: "hidden-entry-id",
_rev: "2-current",
path: `i:${path}`,
type: "plain",
datatype: "plain",
ctime: 10,
mtime: 20,
size: 20,
children: [],
eden: {},
deleted: false,
...overrides,
} as unknown as MetaEntry;
}
function createState() {
const events: string[] = [];
const caches = new Map<string, Map<unknown, unknown>>([
["hidden-file-lastProcessed", new Map([[path, "40-20"]])],
["hidden-file-lastProcessed-database", new Map([[path, "20-20-2-current--1"]])],
["hidden-file-lastKnown", new Map([[path, 40]])],
]);
let activeReads = 0;
let maximumActiveReads = 0;
const keyValueDatabase = {
get: vi.fn(async (key: IDBValidKey) => {
events.push(`get:${String(key)}`);
activeReads++;
maximumActiveReads = Math.max(maximumActiveReads, activeReads);
await Promise.resolve();
activeReads--;
return caches.get(String(key));
}),
set: vi.fn(async () => "ok"),
} as unknown as KeyValueDatabase;
const getDBEntryMeta = vi.fn(async () => false as false | LoadedEntry);
const statHidden = vi.fn(async () => stat(41));
const markChangesAreSame = vi.fn(() => undefined);
const unmarkChanges = vi.fn();
const log = vi.fn();
const dependencies: HiddenFileSyncProcessedStateDependencies = {
getKeyValueDatabase: () => keyValueDatabase,
getLocalDatabase: () => ({ getDBEntryMeta }),
storageAccess: { statHidden },
path: { markChangesAreSame, unmarkChanges },
log,
};
const state = createHiddenFileSyncProcessedState(dependencies);
return {
dependencies,
events,
getDBEntryMeta,
keyValueDatabase,
log,
markChangesAreSame,
maximumActiveReads: () => maximumActiveReads,
state,
statHidden,
unmarkChanges,
};
}
describe("Hidden File Sync processed state", () => {
it("loads the three autosave caches sequentially under their existing keys", async () => {
const fixture = createState();
await fixture.state.initialise();
expect(fixture.events).toEqual([
"get:hidden-file-lastProcessed",
"get:hidden-file-lastProcessed-database",
"get:hidden-file-lastKnown",
]);
expect(fixture.maximumActiveReads()).toBe(1);
expect(fixture.state.getLastProcessedFileCount()).toBe(1);
expect(fixture.state.getLastProcessedFileKey(path)).toBe("40-20");
expect(fixture.state.getLastProcessedDatabaseKey(path)).toBe("20-20-2-current--1");
expect(fixture.state.getLastProcessedFileMTime(path)).toBe(40);
});
it("re-reads a null storage stat and retains the last known mtime on deletion", async () => {
const fixture = createState();
await fixture.state.initialise();
fixture.statHidden.mockResolvedValueOnce(stat(45, 3));
await expect(fixture.state.fileToStatKey(path, null)).resolves.toBe("45-3");
expect(fixture.statHidden).toHaveBeenCalledWith(path);
fixture.state.updateLastProcessedFile(path, stat(45, 3));
fixture.state.updateLastProcessedDeletion(path, metadata({ mtime: 50, size: 0 }));
expect(fixture.state.getLastProcessedFileKey(path)).toBe("0-0");
expect(fixture.state.getLastProcessedFileMTime(path)).toBe(45);
expect(fixture.state.getLastProcessedDatabaseKey(path)).toBe(
toHiddenFileSyncDatabaseStateKey(metadata({ mtime: 50, size: 0 }))
);
expect(fixture.unmarkChanges).toHaveBeenCalledWith(path);
});
it("settles combined state before applying the matching-path marker", async () => {
const fixture = createState();
await fixture.state.initialise();
const document = metadata({ mtime: 60, size: 9 });
const storageStat = stat(61, 9);
fixture.state.updateLastProcessed(path, document, storageStat);
expect(fixture.markChangesAreSame).toHaveBeenCalledWith(path, 60, 61);
expect(fixture.unmarkChanges).not.toHaveBeenCalled();
expect(fixture.state.getLastProcessedDatabaseKey(path)).toBe(toHiddenFileSyncDatabaseStateKey(document));
expect(fixture.state.getLastProcessedFileKey(path)).toBe("61-9");
});
it("resets each processed side without clearing last-known storage mtimes", async () => {
const fixture = createState();
await fixture.state.initialise();
fixture.state.updateLastProcessedFile(path, stat(70, 4));
fixture.state.updateLastProcessedDatabase(path, "database-key");
fixture.state.resetLastProcessedFile([path]);
expect(fixture.state.getLastProcessedFileKey(path)).toBeUndefined();
expect(fixture.state.getLastProcessedDatabaseKey(path)).toBe("database-key");
expect(fixture.state.getLastProcessedFileMTime(path)).toBe(70);
fixture.state.resetLastProcessedDatabase([path]);
expect(fixture.state.getLastProcessedDatabaseKey(path)).toBeUndefined();
});
it("does not settle a database marker for a false or missing database document", async () => {
const fixture = createState();
await fixture.state.initialise();
await fixture.state.updateLastProcessedAsActualDatabase(path, false);
expect(fixture.getDBEntryMeta).toHaveBeenCalledWith(`i:${path}`);
expect(fixture.state.getLastProcessedDatabaseKey(path)).toBe("20-20-2-current--1");
fixture.getDBEntryMeta.mockResolvedValueOnce(false);
await fixture.state.updateLastProcessedAsActualDatabase(path);
expect(fixture.state.getLastProcessedDatabaseKey(path)).toBe("20-20-2-current--1");
});
it("logs and clears both sides when a full reset is requested", async () => {
const fixture = createState();
await fixture.state.initialise();
fixture.state.resetLastProcessedFile(false);
fixture.state.resetLastProcessedDatabase(false);
expect(fixture.log).toHaveBeenNthCalledWith(1, "Delete all processed mark.", LOG_LEVEL_VERBOSE, undefined);
expect(fixture.log).toHaveBeenNthCalledWith(2, "Delete all processed mark.", LOG_LEVEL_VERBOSE, undefined);
expect(fixture.state.getLastProcessedFileCount()).toBe(0);
expect(fixture.state.getLastProcessedFileKey(path)).toBeUndefined();
expect(fixture.state.getLastProcessedDatabaseKey(path)).toBeUndefined();
expect(fixture.state.getLastProcessedFileMTime(path)).toBe(40);
});
});
@@ -0,0 +1,48 @@
import type {
FilePathWithPrefix,
LoadedEntry,
MetaEntry,
UXFileInfo,
UXStat,
} from "@vrtmrz/livesync-commonlib/compat/common/types";
import { displayRev } from "@vrtmrz/livesync-commonlib/compat/common/utils";
import { stripAllPrefixes } from "@vrtmrz/livesync-commonlib/compat/string_and_binary/path";
export function toHiddenFileSyncStorageStateKey(stat: UXStat | null): string {
return `${stat?.mtime ?? 0}-${stat?.size ?? 0}`;
}
export function toHiddenFileSyncDatabaseStateKey(doc: LoadedEntry | MetaEntry): string {
// Compatibility: the deletion marker includes its own hyphen, producing `--0`
// or `--1` after the revision. Existing device-local state uses this format.
return `${doc.mtime}-${doc.size}-${doc._rev}-${doc._deleted || doc.deleted || false ? "-0" : "-1"}`;
}
export function getHiddenFileSyncComparisonMTime(
source: MetaEntry | LoadedEntry | UXFileInfo | UXStat | false | null | undefined,
includeDeleted = false
): number {
if (source === null || source === false || source === undefined) return 0;
if (!includeDeleted) {
if ("deleted" in source && source.deleted) return 0;
if ("_deleted" in source && source._deleted) return 0;
}
if ("stat" in source) return source.stat?.mtime ?? 0;
return source.mtime ?? 0;
}
export function describeHiddenFileSyncDocument(doc: LoadedEntry, prefixedPath: FilePathWithPrefix) {
const id = doc._id;
const path = stripAllPrefixes(prefixedPath);
const rev = doc._rev;
return {
id,
rev,
revDisplay: rev ? displayRev(rev) : "0-NOREVS",
prefixedPath,
path,
isDeleted: doc._deleted || doc.deleted || false,
shortenedId: id.substring(0, 10),
shortenedPath: path.substring(0, 10),
};
}
@@ -0,0 +1,70 @@
import { describe, expect, it } from "vitest";
import type { FilePathWithPrefix, LoadedEntry, UXStat } from "@vrtmrz/livesync-commonlib/compat/common/types";
import {
describeHiddenFileSyncDocument,
getHiddenFileSyncComparisonMTime,
toHiddenFileSyncDatabaseStateKey,
toHiddenFileSyncStorageStateKey,
} from "./hiddenFileSyncState.ts";
describe("Hidden File Sync state keys", () => {
it("represents a missing storage file with zero values", () => {
expect(toHiddenFileSyncStorageStateKey(null)).toBe("0-0");
});
it("uses storage modification time and size", () => {
expect(toHiddenFileSyncStorageStateKey({ mtime: 123, size: 456 } as UXStat)).toBe("123-456");
});
it.each([
[false, "123-456-3-example--1"],
[true, "123-456-3-example--0"],
])("includes database revision and deletion state=%s", (deleted, expected) => {
const doc = { mtime: 123, size: 456, _rev: "3-example", deleted } as LoadedEntry;
expect(toHiddenFileSyncDatabaseStateKey(doc)).toBe(expected);
});
});
describe("getHiddenFileSyncComparisonMTime", () => {
const absentSources = [null, false, undefined] as const;
it.each(absentSources)("returns zero for an absent source=%s", (source) => {
expect(getHiddenFileSyncComparisonMTime(source)).toBe(0);
});
it("reads a direct stat or a file-info stat", () => {
expect(getHiddenFileSyncComparisonMTime({ mtime: 123 } as UXStat)).toBe(123);
expect(getHiddenFileSyncComparisonMTime({ stat: { mtime: 456 } } as never)).toBe(456);
});
it("treats deleted entries as zero unless deletion time is requested", () => {
const deleted = { mtime: 123, deleted: true } as LoadedEntry;
expect(getHiddenFileSyncComparisonMTime(deleted)).toBe(0);
expect(getHiddenFileSyncComparisonMTime(deleted, true)).toBe(123);
});
});
describe("describeHiddenFileSyncDocument", () => {
it("derives the unprefixed path and diagnostic revision fields", () => {
const doc = {
_id: "0123456789abcdef",
_rev: "3-example",
mtime: 123,
size: 456,
deleted: true,
} as LoadedEntry;
expect(describeHiddenFileSyncDocument(doc, "i:.obsidian/app.json" as FilePathWithPrefix)).toEqual({
id: "0123456789abcdef",
rev: "3-example",
revDisplay: "3-exampl",
prefixedPath: "i:.obsidian/app.json",
path: ".obsidian/app.json",
isDeleted: true,
shortenedId: "0123456789",
shortenedPath: ".obsidian/",
});
});
});
@@ -0,0 +1,210 @@
import {
LOG_LEVEL_DEBUG,
LOG_LEVEL_VERBOSE,
type FilePath,
type LoadedEntry,
type LOG_LEVEL,
type UXDataWriteOptions,
type UXFileInfo,
type UXStat,
} from "@vrtmrz/livesync-commonlib/compat/common/types";
import type { StorageAccess } from "@vrtmrz/livesync-commonlib/compat/interfaces/StorageAccess";
import type { LogFunction } from "@vrtmrz/livesync-commonlib/compat/services/lib/logUtils";
import {
createBlob,
isDocContentSame,
readContent,
} from "@vrtmrz/livesync-commonlib/compat/common/utils";
export type HiddenFileSyncStorageAccess = Pick<
StorageAccess,
| "ensureDir"
| "isExistsIncludeHidden"
| "readHiddenFileAuto"
| "removeHidden"
| "statHidden"
| "triggerHiddenFile"
| "writeHiddenFileAuto"
>;
type HiddenFileSyncStorageMethods<Method extends keyof HiddenFileSyncStorageAccess> = {
storageAccess: Pick<HiddenFileSyncStorageAccess, Method>;
};
type HiddenFileSyncLogDependency = {
log: LogFunction;
};
export type HiddenFileSyncStorageDependencies = HiddenFileSyncStorageMethods<keyof HiddenFileSyncStorageAccess> &
HiddenFileSyncLogDependency;
export type HiddenFileSyncRemovalResult = "OK" | "ALREADY" | false;
function log(
dependencies: HiddenFileSyncLogDependency,
message: unknown,
level?: LOG_LEVEL,
key?: string
): void {
dependencies.log(message, level, key);
}
export async function readHiddenFileWithInfo(
dependencies: HiddenFileSyncStorageMethods<"readHiddenFileAuto" | "statHidden">,
path: FilePath
): Promise<UXFileInfo> {
const stat = await dependencies.storageAccess.statHidden(path);
if (!stat) {
return {
name: path.split("/").pop() ?? "",
path,
stat: {
size: 0,
mtime: 0,
ctime: 0,
type: "file",
},
isInternal: true,
deleted: true,
body: createBlob(new Uint8Array(0)),
};
}
const content = await dependencies.storageAccess.readHiddenFileAuto(path);
return {
name: path.split("/").pop() ?? "",
path,
stat,
isInternal: true,
deleted: false,
body: createBlob(content),
};
}
export async function ensureHiddenFileDirectory(
dependencies: HiddenFileSyncStorageMethods<"ensureDir" | "isExistsIncludeHidden">,
path: FilePath
): Promise<void> {
if (!(await dependencies.storageAccess.isExistsIncludeHidden(path))) {
// StorageAccess expects the complete target path and ensures its parent.
await dependencies.storageAccess.ensureDir(path);
}
}
export async function writeHiddenFile(
dependencies: HiddenFileSyncStorageMethods<"statHidden" | "writeHiddenFileAuto">,
path: FilePath,
data: string | ArrayBuffer,
options?: UXDataWriteOptions
): Promise<UXStat | null> {
// Compatibility: the writer's Boolean result is ignored. The post-write stat
// has historically decided whether the operation produced a usable file.
await dependencies.storageAccess.writeHiddenFileAuto(path, data, options);
return await dependencies.storageAccess.statHidden(path);
}
export async function removeHiddenFile(
dependencies: HiddenFileSyncStorageMethods<"isExistsIncludeHidden" | "removeHidden"> &
HiddenFileSyncLogDependency,
path: FilePath
): Promise<HiddenFileSyncRemovalResult> {
try {
if (!(await dependencies.storageAccess.isExistsIncludeHidden(path))) {
return "ALREADY";
}
if (await dependencies.storageAccess.removeHidden(path)) {
return "OK";
}
} catch (error) {
log(dependencies, `Failed to remove file:${path}`);
log(dependencies, error, LOG_LEVEL_VERBOSE);
}
return false;
}
export async function triggerHiddenFileEvent(
dependencies: HiddenFileSyncStorageMethods<"triggerHiddenFile"> & HiddenFileSyncLogDependency,
path: FilePath
): Promise<void> {
try {
await dependencies.storageAccess.triggerHiddenFile(path);
} catch (error) {
log(dependencies, "Failed to call internal API(reconcileInternalFile)", LOG_LEVEL_VERBOSE);
log(dependencies, error, LOG_LEVEL_VERBOSE);
}
}
export async function isHiddenFileWriteRequired(
dependencies: HiddenFileSyncStorageMethods<"readHiddenFileAuto"> & HiddenFileSyncLogDependency,
path: FilePath,
content: string | ArrayBuffer
): Promise<boolean> {
try {
const storageContent = await dependencies.storageAccess.readHiddenFileAuto(path);
return !(await isDocContentSame(storageContent, content));
} catch (error) {
log(dependencies, `Cannot check the content of ${path}`);
log(dependencies, error, LOG_LEVEL_VERBOSE);
// Compatibility: an unreadable current file is treated as requiring a
// write. Changing this policy needs a separate recovery decision.
return true;
}
}
export async function writeHiddenFileFromDatabase(
dependencies: HiddenFileSyncStorageMethods<
"ensureDir" | "isExistsIncludeHidden" | "readHiddenFileAuto" | "statHidden" | "writeHiddenFileAuto"
> &
HiddenFileSyncLogDependency,
path: FilePath,
fileOnDatabase: LoadedEntry,
force: boolean
): Promise<false | UXStat> {
try {
const statBefore = await dependencies.storageAccess.statHidden(path);
const isExisting = statBefore != null;
const content = readContent(fileOnDatabase);
await ensureHiddenFileDirectory(dependencies, path);
const writeRequired =
force || !isExisting || (isExisting && (await isHiddenFileWriteRequired(dependencies, path, content)));
if (!writeRequired) {
log(dependencies, `STORAGE <-- DB: ${path}: skipped (hidden) Not changed`, LOG_LEVEL_DEBUG);
return statBefore;
}
const statAfter = await writeHiddenFile(dependencies, path, content, {
mtime: fileOnDatabase.mtime,
ctime: fileOnDatabase.ctime,
});
if (statAfter == null) {
log(dependencies, `STORAGE <-- DB: ${path}: written (hidden,new${force ? ", force" : ""}) Failed (writeResult)`);
return false;
}
log(dependencies, `STORAGE <-- DB: ${path}: written (hidden, overwrite${force ? ", force" : ""})`);
// Compatibility question: ordinary database reflection does not trigger
// a raw storage event here; deletion and manual JSON merging do. Preserve
// this until the event-loop consequences of changing it are characterised.
return statAfter;
} catch (error) {
log(dependencies, `STORAGE <-- DB: ${path}: written (hidden, overwrite${force ? ", force" : ""}) Failed`);
log(dependencies, error, LOG_LEVEL_VERBOSE);
return false;
}
}
export async function deleteHiddenFileFromStorage(
dependencies: HiddenFileSyncStorageMethods<"isExistsIncludeHidden" | "removeHidden" | "triggerHiddenFile"> &
HiddenFileSyncLogDependency,
path: FilePath
): Promise<HiddenFileSyncRemovalResult> {
const result = await removeHiddenFile(dependencies, path);
if (result === false) {
log(dependencies, `STORAGE <x- DB: ${path}: deleting (hidden) Failed`);
return false;
}
if (result === "OK") {
await triggerHiddenFileEvent(dependencies, path);
}
log(dependencies, `STORAGE <x- DB: ${path}: deleting (hidden) ${result == "OK" ? "Done" : "Already not found"}`);
return result;
}
@@ -0,0 +1,230 @@
import { describe, expect, it, vi } from "vitest";
import {
LOG_LEVEL_VERBOSE,
type FilePath,
type LoadedEntry,
type UXStat,
} from "@vrtmrz/livesync-commonlib/compat/common/types";
import {
deleteHiddenFileFromStorage,
ensureHiddenFileDirectory,
isHiddenFileWriteRequired,
readHiddenFileWithInfo,
removeHiddenFile,
triggerHiddenFileEvent,
writeHiddenFile,
writeHiddenFileFromDatabase,
} from "./hiddenFileSyncStorage.ts";
const path = ".obsidian/plugins/example/data.json" as FilePath;
const stat = { ctime: 10, mtime: 20, size: 4, type: "file" } as UXStat;
function createStorageDependencies() {
const storageAccess = {
ensureDir: vi.fn(async () => true),
isExistsIncludeHidden: vi.fn(async () => true),
readHiddenFileAuto: vi.fn(async () => "data" as string | ArrayBuffer),
removeHidden: vi.fn(async () => true),
statHidden: vi.fn(async () => stat as UXStat | null),
triggerHiddenFile: vi.fn(async () => undefined),
writeHiddenFileAuto: vi.fn(async () => true),
};
const log = vi.fn();
return { dependencies: { storageAccess, log }, log, storageAccess };
}
function databaseEntry(content: string, mtime = 30, ctime = 15): LoadedEntry {
return {
path,
type: "plain",
datatype: "plain",
data: content,
mtime,
ctime,
} as LoadedEntry;
}
describe("Hidden File Sync storage operations", () => {
it("represents a missing hidden file as a deleted empty file", async () => {
const { dependencies, storageAccess } = createStorageDependencies();
storageAccess.statHidden.mockResolvedValue(null);
const result = await readHiddenFileWithInfo(dependencies, path);
expect(result).toMatchObject({
name: "data.json",
path,
isInternal: true,
deleted: true,
stat: { ctime: 0, mtime: 0, size: 0, type: "file" },
});
expect(await result.body.text()).toBe("");
expect(storageAccess.readHiddenFileAuto).not.toHaveBeenCalled();
});
it("loads an existing hidden file with its storage stat", async () => {
const { dependencies, storageAccess } = createStorageDependencies();
storageAccess.readHiddenFileAuto.mockResolvedValue("data");
const result = await readHiddenFileWithInfo(dependencies, path);
expect(result).toMatchObject({ name: "data.json", path, isInternal: true, deleted: false, stat });
expect(await result.body.text()).toBe("data");
});
it("ensures a directory only when the target does not exist", async () => {
const { dependencies, storageAccess } = createStorageDependencies();
await ensureHiddenFileDirectory(dependencies, path);
expect(storageAccess.ensureDir).not.toHaveBeenCalled();
storageAccess.isExistsIncludeHidden.mockResolvedValue(false);
await ensureHiddenFileDirectory(dependencies, path);
expect(storageAccess.ensureDir).toHaveBeenCalledOnce();
expect(storageAccess.ensureDir).toHaveBeenCalledWith(path);
});
it("writes a hidden file and returns the resulting stat", async () => {
const { dependencies, storageAccess } = createStorageDependencies();
await expect(writeHiddenFile(dependencies, path, "data", { mtime: 30, ctime: 15 })).resolves.toBe(stat);
expect(storageAccess.writeHiddenFileAuto).toHaveBeenCalledWith(path, "data", { mtime: 30, ctime: 15 });
expect(storageAccess.statHidden).toHaveBeenCalledWith(path);
});
it("uses the post-write stat even when the storage writer reports false", async () => {
const { dependencies, storageAccess } = createStorageDependencies();
storageAccess.writeHiddenFileAuto.mockResolvedValue(false);
await expect(writeHiddenFile(dependencies, path, "data")).resolves.toBe(stat);
expect(storageAccess.statHidden).toHaveBeenCalledAfter(storageAccess.writeHiddenFileAuto);
});
it("distinguishes an absent, removed, and unremovable file", async () => {
const { dependencies, storageAccess } = createStorageDependencies();
storageAccess.isExistsIncludeHidden.mockResolvedValueOnce(false);
await expect(removeHiddenFile(dependencies, path)).resolves.toBe("ALREADY");
expect(storageAccess.removeHidden).not.toHaveBeenCalled();
storageAccess.isExistsIncludeHidden.mockResolvedValueOnce(true);
storageAccess.removeHidden.mockResolvedValueOnce(true);
await expect(removeHiddenFile(dependencies, path)).resolves.toBe("OK");
storageAccess.isExistsIncludeHidden.mockResolvedValueOnce(true);
storageAccess.removeHidden.mockResolvedValueOnce(false);
await expect(removeHiddenFile(dependencies, path)).resolves.toBe(false);
});
it("turns a removal error into a logged false result", async () => {
const { dependencies, log, storageAccess } = createStorageDependencies();
const error = new Error("remove failed");
storageAccess.isExistsIncludeHidden.mockRejectedValue(error);
await expect(removeHiddenFile(dependencies, path)).resolves.toBe(false);
expect(log).toHaveBeenNthCalledWith(1, `Failed to remove file:${path}`, undefined, undefined);
expect(log).toHaveBeenNthCalledWith(2, error, LOG_LEVEL_VERBOSE, undefined);
});
it("treats a content-read failure as requiring a write", async () => {
const { dependencies, log, storageAccess } = createStorageDependencies();
const error = new Error("read failed");
storageAccess.readHiddenFileAuto.mockRejectedValue(error);
await expect(isHiddenFileWriteRequired(dependencies, path, "data")).resolves.toBe(true);
expect(log).toHaveBeenNthCalledWith(1, `Cannot check the content of ${path}`, undefined, undefined);
expect(log).toHaveBeenNthCalledWith(2, error, LOG_LEVEL_VERBOSE, undefined);
});
it("compares binary content without involving the context", async () => {
const { dependencies, storageAccess } = createStorageDependencies();
storageAccess.readHiddenFileAuto.mockResolvedValue(new Uint8Array([1, 2, 3]).buffer);
await expect(
isHiddenFileWriteRequired(dependencies, path, new Uint8Array([1, 2, 3]).buffer)
).resolves.toBe(false);
await expect(
isHiddenFileWriteRequired(dependencies, path, new Uint8Array([1, 2, 4]).buffer)
).resolves.toBe(true);
});
it("skips an unchanged database file and preserves its current stat", async () => {
const { dependencies, storageAccess } = createStorageDependencies();
storageAccess.readHiddenFileAuto.mockResolvedValue("data");
await expect(writeHiddenFileFromDatabase(dependencies, path, databaseEntry("data"), false)).resolves.toBe(
stat
);
expect(storageAccess.ensureDir).not.toHaveBeenCalled();
expect(storageAccess.writeHiddenFileAuto).not.toHaveBeenCalled();
});
it("writes changed database content with its original timestamps", async () => {
const { dependencies, storageAccess } = createStorageDependencies();
storageAccess.readHiddenFileAuto.mockResolvedValue("old");
await expect(writeHiddenFileFromDatabase(dependencies, path, databaseEntry("new"), false)).resolves.toBe(
stat
);
expect(storageAccess.writeHiddenFileAuto).toHaveBeenCalledWith(path, "new", { mtime: 30, ctime: 15 });
expect(storageAccess.triggerHiddenFile).not.toHaveBeenCalled();
});
it("forces a write without reading existing content", async () => {
const { dependencies, storageAccess } = createStorageDependencies();
await expect(writeHiddenFileFromDatabase(dependencies, path, databaseEntry("data"), true)).resolves.toBe(stat);
expect(storageAccess.readHiddenFileAuto).not.toHaveBeenCalled();
expect(storageAccess.writeHiddenFileAuto).toHaveBeenCalledOnce();
});
it("returns false when a completed write has no resulting stat", async () => {
const { dependencies, storageAccess } = createStorageDependencies();
storageAccess.statHidden.mockResolvedValue(null);
await expect(writeHiddenFileFromDatabase(dependencies, path, databaseEntry("data"), false)).resolves.toBe(
false
);
expect(storageAccess.writeHiddenFileAuto).toHaveBeenCalledOnce();
});
it("triggers a storage event only after an actual deletion", async () => {
const { dependencies, log, storageAccess } = createStorageDependencies();
storageAccess.isExistsIncludeHidden.mockResolvedValueOnce(false);
await expect(deleteHiddenFileFromStorage(dependencies, path)).resolves.toBe("ALREADY");
expect(storageAccess.triggerHiddenFile).not.toHaveBeenCalled();
log.mockClear();
storageAccess.isExistsIncludeHidden.mockResolvedValueOnce(true);
storageAccess.removeHidden.mockResolvedValueOnce(true);
await expect(deleteHiddenFileFromStorage(dependencies, path)).resolves.toBe("OK");
expect(storageAccess.triggerHiddenFile).toHaveBeenCalledOnce();
expect(storageAccess.triggerHiddenFile).toHaveBeenCalledWith(path);
expect(storageAccess.triggerHiddenFile).toHaveBeenCalledAfter(storageAccess.removeHidden);
expect(log).toHaveBeenCalledAfter(storageAccess.triggerHiddenFile);
});
it("does not trigger a storage event after a failed deletion", async () => {
const { dependencies, storageAccess } = createStorageDependencies();
storageAccess.removeHidden.mockResolvedValue(false);
await expect(deleteHiddenFileFromStorage(dependencies, path)).resolves.toBe(false);
expect(storageAccess.triggerHiddenFile).not.toHaveBeenCalled();
});
it("swallows and logs storage-event failures", async () => {
const { dependencies, log, storageAccess } = createStorageDependencies();
const error = new Error("event failed");
storageAccess.triggerHiddenFile.mockRejectedValue(error);
await expect(triggerHiddenFileEvent(dependencies, path)).resolves.toBeUndefined();
expect(log).toHaveBeenNthCalledWith(
1,
"Failed to call internal API(reconcileInternalFile)",
LOG_LEVEL_VERBOSE,
undefined
);
expect(log).toHaveBeenNthCalledWith(2, error, LOG_LEVEL_VERBOSE, undefined);
});
});
@@ -0,0 +1,143 @@
import type { InternalFileInfo } from "@/common/types.ts";
import type {
FilePath,
FilePathWithPrefix,
LoadedEntry,
UXFileInfo,
} from "@vrtmrz/livesync-commonlib/compat/common/types";
import type { OptionalSyncFeatureMode } from "@/features/optionalSyncFeatures.ts";
import type { HiddenFileSyncConflictTestingView } from "./hiddenFileSyncConflictResolution.ts";
export type HiddenFileSyncInitialisationDirection = "push" | "pull" | "safe" | "pullForce" | "pushForce";
/** Initialisation operation needed by the Customisation Sync dialogue. */
export interface HiddenFileSyncInitialisationView {
initialiseInternalFileSync(
direction: HiddenFileSyncInitialisationDirection,
showMessage: boolean,
targetFiles?: string[] | false
): Promise<void>;
}
/**
* Semantic callbacks consumed by the Commonlib service registries.
*
* The context owns the implementations, while the optional-file composition
* adapts this view to Commonlib's aggregation contracts. Callers do not bind
* the context or depend on registry-oriented method names.
*/
export interface HiddenFileSyncServiceHandlerView {
readonly processOptionalFileEvent: (path: FilePath) => Promise<boolean>;
readonly processOptionalSyncFiles: (doc: LoadedEntry) => Promise<boolean>;
readonly onSettingLoaded: () => Promise<boolean>;
readonly realiseSettingSyncMode: () => Promise<boolean>;
readonly onResuming: () => Promise<boolean>;
readonly beforeReplicate: (showNotice: boolean) => Promise<boolean>;
readonly onDatabaseInitialised: (showNotice: boolean) => Promise<boolean>;
readonly suspendExtraSync: () => Promise<boolean>;
readonly configureOptionalSyncFeature: (mode: OptionalSyncFeatureMode) => Promise<boolean>;
readonly isTargetFileEligible: (path: FilePath) => Promise<boolean>;
readonly queueConflict: (path: FilePathWithPrefix) => Promise<boolean>;
}
export function createHiddenFileSyncServiceHandlerView(
operations: HiddenFileSyncServiceHandlerView
): HiddenFileSyncServiceHandlerView {
const view: HiddenFileSyncServiceHandlerView = {
processOptionalFileEvent: async (path) => await operations.processOptionalFileEvent(path),
processOptionalSyncFiles: async (doc) => await operations.processOptionalSyncFiles(doc),
onSettingLoaded: async () => await operations.onSettingLoaded(),
realiseSettingSyncMode: async () => await operations.realiseSettingSyncMode(),
onResuming: async () => await operations.onResuming(),
beforeReplicate: async (showNotice) => await operations.beforeReplicate(showNotice),
onDatabaseInitialised: async (showNotice) => await operations.onDatabaseInitialised(showNotice),
suspendExtraSync: async () => await operations.suspendExtraSync(),
configureOptionalSyncFeature: async (mode) => await operations.configureOptionalSyncFeature(mode),
isTargetFileEligible: async (path) => await operations.isTargetFileEligible(path),
queueConflict: async (path) => await operations.queueConflict(path),
};
return Object.freeze(view);
}
export type HiddenFileSyncTestingRebuild = (
showNotice: boolean,
targetFiles?: FilePath[] | false
) => Promise<FilePath[]>;
export type HiddenFileSyncTestingRebuildInterceptor = (
runRebuild: HiddenFileSyncTestingRebuild,
showNotice: boolean,
targetFiles?: FilePath[] | false
) => Promise<FilePath[]>;
/** Operations exposed to the real-Obsidian contract tests. */
export interface HiddenFileSyncTestingView extends HiddenFileSyncCommandView {
readonly conflictResolution: HiddenFileSyncConflictTestingView;
readFileWithInfo(path: FilePath): Promise<UXFileInfo>;
showConfigurationChangeNotice(updatedFolders: readonly string[]): void;
interceptRebuildMerging(interceptor: HiddenFileSyncTestingRebuildInterceptor): () => void;
}
export type HiddenFileSyncTestingViewOperations = HiddenFileSyncTestingView;
/**
* Build the frozen testing seam. Tests can observe behaviour and install a
* scoped timing interceptor, but cannot access mutable context state.
*/
export function createHiddenFileSyncTestingView(
operations: HiddenFileSyncTestingViewOperations
): HiddenFileSyncTestingView {
const view: HiddenFileSyncTestingView = {
isManualCommandAvailable: () => operations.isManualCommandAvailable(),
scanAllStorageChanges: async (showNotice) => await operations.scanAllStorageChanges(showNotice),
scanAllDatabaseChanges: async (showNotice) => await operations.scanAllDatabaseChanges(showNotice),
applyOfflineChanges: async (showNotice) => await operations.applyOfflineChanges(showNotice),
updateSettingCache: () => operations.updateSettingCache(),
initialiseInternalFileSync: async (direction, showMessage, targetFiles) =>
await operations.initialiseInternalFileSync(direction, showMessage, targetFiles),
conflictResolution: operations.conflictResolution,
readFileWithInfo: async (path) => await operations.readFileWithInfo(path),
showConfigurationChangeNotice: (updatedFolders) =>
operations.showConfigurationChangeNotice(updatedFolders),
interceptRebuildMerging: (interceptor) => operations.interceptRebuildMerging(interceptor),
};
return Object.freeze(view);
}
/** Exact-revision operations needed by the Hatch repair pane. */
export interface HiddenFileSyncRepairView {
scanInternalFiles(): Promise<InternalFileInfo[]>;
storeInternalFileToDatabase(file: InternalFileInfo, forceWrite?: boolean): Promise<boolean | undefined>;
storeInternalFileToDatabaseWithBaseRevision(
file: InternalFileInfo,
baseRevision: string,
createIfDifferent?: boolean
): Promise<boolean>;
extractInternalFileRevisionFromDatabase(
storageFilePath: FilePath,
revision: string,
force?: boolean
): Promise<boolean>;
}
export function createHiddenFileSyncRepairView(operations: HiddenFileSyncRepairView): HiddenFileSyncRepairView {
const view: HiddenFileSyncRepairView = {
scanInternalFiles: async () => await operations.scanInternalFiles(),
storeInternalFileToDatabase: async (file, forceWrite) =>
await operations.storeInternalFileToDatabase(file, forceWrite),
storeInternalFileToDatabaseWithBaseRevision: async (file, baseRevision, createIfDifferent) =>
await operations.storeInternalFileToDatabaseWithBaseRevision(file, baseRevision, createIfDifferent),
extractInternalFileRevisionFromDatabase: async (storageFilePath, revision, force) =>
await operations.extractInternalFileRevisionFromDatabase(storageFilePath, revision, force),
};
return Object.freeze(view);
}
/** Operations consumed by the host-owned Hidden File Sync commands. */
export interface HiddenFileSyncCommandView extends HiddenFileSyncInitialisationView {
isManualCommandAvailable(): boolean;
scanAllStorageChanges(showNotice: boolean): Promise<unknown>;
scanAllDatabaseChanges(showNotice: boolean): Promise<unknown>;
applyOfflineChanges(showNotice: boolean): Promise<unknown>;
updateSettingCache(): void;
}
@@ -0,0 +1,132 @@
import { describe, expect, it, vi } from "vitest";
import type { FilePath } from "@vrtmrz/livesync-commonlib/compat/common/types";
import {
createHiddenFileSyncRepairView,
createHiddenFileSyncServiceHandlerView,
createHiddenFileSyncTestingView,
type HiddenFileSyncRepairView,
type HiddenFileSyncServiceHandlerView,
type HiddenFileSyncTestingViewOperations,
} from "./hiddenFileSyncViews.ts";
describe("Hidden File Sync repair view", () => {
it("exposes only frozen repair operations and preserves their receiver", async () => {
const path = ".obsidian/app.json" as FilePath;
const file = { path, ctime: 1, mtime: 2, size: 3 };
const source = {
marker: "source",
scanInternalFiles: vi.fn(async function (this: { marker: string }) {
expect(this.marker).toBe("source");
return [file];
}),
storeInternalFileToDatabase: vi.fn(async function (this: { marker: string }) {
expect(this.marker).toBe("source");
return true;
}),
storeInternalFileToDatabaseWithBaseRevision: vi.fn(async function (this: { marker: string }) {
expect(this.marker).toBe("source");
return true;
}),
extractInternalFileRevisionFromDatabase: vi.fn(async function (this: { marker: string }) {
expect(this.marker).toBe("source");
return true;
}),
} as unknown as HiddenFileSyncRepairView;
const view = createHiddenFileSyncRepairView(source);
expect(view).not.toBe(source);
expect(Object.isFrozen(view)).toBe(true);
expect(Object.keys(view).sort()).toEqual(
[
"extractInternalFileRevisionFromDatabase",
"scanInternalFiles",
"storeInternalFileToDatabase",
"storeInternalFileToDatabaseWithBaseRevision",
].sort()
);
await expect(view.scanInternalFiles()).resolves.toEqual([file]);
await expect(view.storeInternalFileToDatabase(file)).resolves.toBe(true);
await expect(view.storeInternalFileToDatabaseWithBaseRevision(file, "2-selected", false)).resolves.toBe(true);
await expect(view.extractInternalFileRevisionFromDatabase(path, "2-selected", true)).resolves.toBe(true);
expect(source.storeInternalFileToDatabaseWithBaseRevision).toHaveBeenCalledWith(file, "2-selected", false);
});
});
describe("Hidden File Sync service-handler view", () => {
it("forwards semantic callbacks through a frozen view", async () => {
const operations = {
processOptionalFileEvent: vi.fn(async () => true),
processOptionalSyncFiles: vi.fn(async () => true),
onSettingLoaded: vi.fn(async () => true),
realiseSettingSyncMode: vi.fn(async () => true),
onResuming: vi.fn(async () => true),
beforeReplicate: vi.fn(async () => true),
onDatabaseInitialised: vi.fn(async () => true),
suspendExtraSync: vi.fn(async () => true),
configureOptionalSyncFeature: vi.fn(async () => true),
isTargetFileEligible: vi.fn(async () => true),
queueConflict: vi.fn(async () => true),
} satisfies HiddenFileSyncServiceHandlerView;
const view = createHiddenFileSyncServiceHandlerView(operations);
expect(Object.isFrozen(view)).toBe(true);
await view.processOptionalFileEvent(".obsidian/app.json" as FilePath);
await view.processOptionalSyncFiles({} as never);
await view.onSettingLoaded();
await view.realiseSettingSyncMode();
await view.onResuming();
await view.beforeReplicate(true);
await view.onDatabaseInitialised(false);
await view.suspendExtraSync();
await view.configureOptionalSyncFeature("MERGE");
await view.isTargetFileEligible(".obsidian/app.json" as FilePath);
await view.queueConflict("i:.obsidian/app.json" as never);
expect(operations.processOptionalFileEvent).toHaveBeenCalledWith(".obsidian/app.json");
expect(operations.beforeReplicate).toHaveBeenCalledWith(true);
expect(operations.onDatabaseInitialised).toHaveBeenCalledWith(false);
expect(operations.configureOptionalSyncFeature).toHaveBeenCalledWith("MERGE");
expect(operations.queueConflict).toHaveBeenCalledWith("i:.obsidian/app.json");
});
});
describe("Hidden File Sync testing view", () => {
it("keeps test operations focused while retaining a frozen E2E surface", async () => {
const conflictResolution = {
resolveAll: vi.fn(async () => undefined),
resolveJson: vi.fn(async () => true),
pendingPaths: [],
processor: { remaining: 0, totalRemaining: 0, nowProcessing: 0 },
};
const restoreRebuild = vi.fn();
const operations = {
isManualCommandAvailable: vi.fn(() => true),
scanAllStorageChanges: vi.fn(async () => undefined),
scanAllDatabaseChanges: vi.fn(async () => undefined),
applyOfflineChanges: vi.fn(async () => undefined),
updateSettingCache: vi.fn(),
initialiseInternalFileSync: vi.fn(async () => undefined),
conflictResolution,
readFileWithInfo: vi.fn(async () => ({}) as never),
showConfigurationChangeNotice: vi.fn(),
interceptRebuildMerging: vi.fn(() => restoreRebuild),
} satisfies HiddenFileSyncTestingViewOperations;
const view = createHiddenFileSyncTestingView(operations);
expect(Object.isFrozen(view)).toBe(true);
await view.scanAllStorageChanges(true);
await view.readFileWithInfo(".obsidian/app.json" as FilePath);
view.showConfigurationChangeNotice([".obsidian"]);
expect(operations.scanAllStorageChanges).toHaveBeenCalledWith(true);
expect(operations.readFileWithInfo).toHaveBeenCalledWith(".obsidian/app.json");
expect(operations.showConfigurationChangeNotice).toHaveBeenCalledWith([".obsidian"]);
const interceptor = vi.fn(async () => [] as FilePath[]);
expect(view.interceptRebuildMerging(interceptor)).toBe(restoreRebuild);
expect(operations.interceptRebuildMerging).toHaveBeenCalledWith(interceptor);
});
});
@@ -0,0 +1,683 @@
import {
type AnyEntry,
type DocumentID,
type FilePath,
type FilePathWithPrefix,
type LoadedEntry,
LOG_LEVEL_INFO,
LOG_LEVEL_NOTICE,
LOG_LEVEL_VERBOSE,
type LOG_LEVEL,
type MetaEntry,
} from "@vrtmrz/livesync-commonlib/compat/common/types";
import type { StorageAccess } from "@vrtmrz/livesync-commonlib/compat/interfaces/StorageAccess";
import type { LiveSyncLocalDB } from "@vrtmrz/livesync-commonlib/compat/pouchdb/LiveSyncLocalDB";
import type { LogFunction } from "@vrtmrz/livesync-commonlib/compat/services/lib/logUtils";
import { tryGetFilePath } from "@vrtmrz/livesync-commonlib/compat/common/utils.doc";
import { stripAllPrefixes } from "@vrtmrz/livesync-commonlib/compat/string_and_binary/path";
import { Semaphore } from "octagonal-wheels/concurrency/semaphore";
import { serialized, skipIfDuplicated } from "octagonal-wheels/concurrency/lock";
import { type InternalFileInfo, ICHeader, ICHeaderEnd } from "@/common/types.ts";
import {
BASE_IS_NEW,
compareMTime,
EVEN,
getLogLevel,
isInternalMetadata,
onlyInNTimes,
TARGET_IS_NEW,
} from "@/common/utils.ts";
import {
collectOptionalFileSyncFiles,
type OptionalFileSyncFileTreeDependencies,
} from "@/features/optionalFileSyncFileTree.ts";
import type { HiddenFileSyncChangeProcessor } from "./hiddenFileSyncChangeProcessor.ts";
import type { HiddenFileSyncProcessedState } from "./hiddenFileSyncProcessedState.ts";
import { describeHiddenFileSyncDocument, getHiddenFileSyncComparisonMTime } from "./hiddenFileSyncState.ts";
import type {
HiddenFileSyncInitialisationDirection as InitialisationDirection,
HiddenFileSyncTestingRebuild,
HiddenFileSyncTestingRebuildInterceptor,
} from "./hiddenFileSyncViews.ts";
export type { HiddenFileSyncInitialisationDirection as InitialisationDirection } from "./hiddenFileSyncViews.ts";
export type ReconciliationProgress = {
log(message: string): void;
once(message: string): void;
done(message?: string): void;
};
type ReconciliationDatabase = Pick<LiveSyncLocalDB, "allDocsRaw">;
type ReconciliationStorage = Pick<StorageAccess, "statHidden">;
type ReconciliationProcessedState = Pick<
HiddenFileSyncProcessedState,
| "databaseStateKey"
| "getLastProcessedDatabaseKey"
| "getLastProcessedFileKey"
| "getLastProcessedFileMTime"
| "hasLastProcessedDatabase"
| "hasLastProcessedFile"
| "getLastProcessedFileKeys"
| "resetLastProcessedDatabase"
| "resetLastProcessedFile"
| "storageStateKey"
| "updateLastProcessed"
| "updateLastProcessedAsActualDatabase"
| "updateLastProcessedAsActualFile"
>;
type ReconciliationChangeProcessor = Pick<
HiddenFileSyncChangeProcessor,
"processStorageChange" | "processDatabaseChange"
>;
export type ReconciliationDependencies = OptionalFileSyncFileTreeDependencies & {
getLocalDatabase(): ReconciliationDatabase;
storageAccess: ReconciliationStorage;
getRootPath(): string;
getPath(entry: AnyEntry): FilePathWithPrefix;
isTargetFile(path: FilePath): Promise<boolean>;
isIgnoredByIgnoreFile(path: string): Promise<boolean>;
createProgress(prefix?: string, level?: LOG_LEVEL): ReconciliationProgress;
processedState: ReconciliationProcessedState;
changeProcessor: ReconciliationChangeProcessor;
log: LogFunction;
};
export type Reconciliation = {
processStorageChange(
path: FilePath,
onlyNew?: boolean,
forceWrite?: boolean,
includeDeleted?: boolean
): Promise<boolean | undefined>;
processDatabaseDocument(doc: LoadedEntry): Promise<boolean>;
scanInternalFiles(): Promise<InternalFileInfo[]>;
scanAllStorageChanges(
showNotice?: boolean,
onlyNew?: boolean,
forceWriteAll?: boolean,
includeDeleted?: boolean
): Promise<unknown>;
scanAllDatabaseChanges(
showNotice?: boolean,
onlyNew?: boolean,
forceWriteAll?: boolean,
includeDeletion?: boolean
): Promise<unknown>;
applyOfflineChanges(showNotice: boolean): Promise<unknown>;
initialiseInternalFileSync(
direction: InitialisationDirection,
showMessage: boolean,
targetFilesSrc?: string[] | false,
initialisationProgress?: ReconciliationProgress
): Promise<void>;
interceptRebuildMerging(interceptor: HiddenFileSyncTestingRebuildInterceptor): () => void;
dispose(): void;
};
class ReconciliationOwner implements Reconciliation {
private rebuildMergingHook: HiddenFileSyncTestingRebuild | undefined;
constructor(private readonly dependencies: ReconciliationDependencies) {}
private get localDatabase() {
return this.dependencies.getLocalDatabase();
}
private get storageAccess() {
return this.dependencies.storageAccess;
}
private getPath(entry: AnyEntry): FilePathWithPrefix {
return this.dependencies.getPath(entry);
}
private _log(message: unknown, level?: LOG_LEVEL, key?: string): void {
this.dependencies.log(message, level, key);
}
private _verbose(message: unknown, key?: string): void {
this._log(message, LOG_LEVEL_VERBOSE, key);
}
private _progress(prefix: string = "", level: LOG_LEVEL = LOG_LEVEL_NOTICE): ReconciliationProgress {
return this.dependencies.createProgress(prefix, level);
}
async processStorageChange(
path: FilePath,
onlyNew = false,
forceWrite = false,
includeDeleted = true
): Promise<boolean | undefined> {
if (!(await this.dependencies.isTargetFile(path))) {
this._log(
`Storage file tracking: Hidden file skipped: ${path} is filtered out by the defined patterns.`,
LOG_LEVEL_VERBOSE
);
return false;
}
return await this.dependencies.changeProcessor.processStorageChange(path, onlyNew, forceWrite, includeDeleted);
}
async processDatabaseDocument(doc: LoadedEntry): Promise<boolean> {
const info = describeHiddenFileSyncDocument(doc, this.getPath(doc));
const path = info.path;
const headerLine = `Tracking DB ${info.path} (${info.revDisplay}) :`;
const ret = await this.trackDatabaseFileModification(path, headerLine);
this._log(`${headerLine} Done: ${info.shortenedId})`, LOG_LEVEL_VERBOSE);
return ret;
}
private async scanInternalFileNames(): Promise<FilePath[]> {
const findRoot = this.dependencies.getRootPath();
const filenames = await collectOptionalFileSyncFiles(this.dependencies, findRoot, {
shouldInclude: (path) => this.dependencies.isTargetFile(path as FilePath),
onError: (path, error) => {
this._log(`Could not traverse(HiddenSync):${path}`, LOG_LEVEL_INFO);
this._log(error, LOG_LEVEL_VERBOSE);
},
});
return filenames as FilePath[];
}
async scanInternalFiles(): Promise<InternalFileInfo[]> {
const fileNames = await this.scanInternalFileNames();
const files = fileNames.map(async (e) => {
return {
path: e,
stat: await this.storageAccess.statHidden(e),
};
});
const result: InternalFileInfo[] = [];
for (const f of files) {
const w = await f;
if (await this.dependencies.isIgnoredByIgnoreFile(w.path)) {
continue;
}
const mtime = w.stat?.mtime ?? 0;
const ctime = w.stat?.ctime ?? mtime;
const size = w.stat?.size ?? 0;
result.push({
...w,
mtime,
ctime,
size,
});
}
return result;
}
private async adoptCurrentStorageFilesAsProcessed(targetFiles: FilePath[] | false): Promise<void> {
const allFiles = await this.scanInternalFileNames();
const files = targetFiles ? allFiles.filter((e) => targetFiles.some((t) => e.indexOf(t) !== -1)) : allFiles;
for (const file of files) {
await this.dependencies.processedState.updateLastProcessedAsActualFile(file);
}
}
private async adoptCurrentDatabaseFilesAsProcessed(targetFiles: FilePath[] | false): Promise<void> {
const allFiles = await this.getAllDatabaseFiles();
const files = targetFiles
? allFiles.filter((e) => targetFiles.some((t) => e.path.indexOf(t) !== -1))
: allFiles;
for (const file of files) {
const path = stripAllPrefixes(this.getPath(file));
await this.dependencies.processedState.updateLastProcessedAsActualDatabase(path, file);
}
}
private async trackScannedStorageChanges(
processFiles: FilePath[],
showNotice: boolean = false,
onlyNew = false,
forceWriteAll = false,
includeDeleted = true
): Promise<void> {
const logLevel = getLogLevel(showNotice);
const p = this._progress(`[⚙ Storage -> DB ]\n`, logLevel);
const notifyProgress = onlyInNTimes(100, (progress) => p.log(`${progress}/${processFiles.length}`));
const processes = processFiles.map(async (file, i) => {
try {
await this.processStorageChange(file, onlyNew, forceWriteAll, includeDeleted);
notifyProgress();
} catch (ex) {
p.once(`Failed to process storage change file:${file}`);
this._log(ex, LOG_LEVEL_VERBOSE);
}
});
await Promise.all(processes);
p.done();
}
async scanAllStorageChanges(
showNotice: boolean = false,
onlyNew = false,
forceWriteAll = false,
includeDeleted = true
): Promise<unknown> {
return await skipIfDuplicated("scanAllStorageChanges", async () => {
const logLevel = getLogLevel(showNotice);
const p = this._progress(`[⚙ Scanning Storage -> DB ]\n`, logLevel);
p.log(`Scanning storage files...`);
const knownNames = [...this.dependencies.processedState.getLastProcessedFileKeys()] as FilePath[];
const existNames = await this.scanInternalFileNames();
const files = new Set([...knownNames, ...existNames]);
this._log(
`Known/Exist ${knownNames.length}/${existNames.length}, Totally ${files.size} files.`,
LOG_LEVEL_VERBOSE
);
const taskNameAndMeta = [...files].map(async (e) => [e, await this.storageAccess.statHidden(e)] as const);
const nameAndMeta = await Promise.all(taskNameAndMeta);
const processFiles = nameAndMeta
.filter(([path, stat]) => {
if (forceWriteAll) return true;
const key = this.dependencies.processedState.getLastProcessedFileKey(path);
const newKey = this.dependencies.processedState.storageStateKey(stat);
return key != newKey;
})
.map(([path, stat]) => path);
const staticsMessage = `[Storage hidden file statics]
Known files: ${knownNames.length}
Actual files: ${existNames.length}
All files: ${files.size}
Offline Changed files: ${processFiles.length}`;
// this._log(staticsMessage, logLevel, "scan-changes");
p.once(staticsMessage);
await this.trackScannedStorageChanges(processFiles, showNotice, onlyNew, forceWriteAll, includeDeleted);
p.done();
});
}
private async trackScannedDatabaseChange(
processFiles: MetaEntry[],
showNotice: boolean = false,
onlyNew = false,
forceWriteAll = false,
includeDeletion = true
): Promise<void> {
const logLevel = getLogLevel(showNotice);
const p = this._progress(`[⚙ DB -> Storage ]\n`, logLevel);
const notifyProgress = onlyInNTimes(100, (progress) => p.log(`${progress}/${processFiles.length}`));
const processes = processFiles.map(async (file) => {
try {
const path = stripAllPrefixes(this.getPath(file));
if (!(await this.dependencies.isTargetFile(path))) {
this._log(
`Database file tracking: Hidden file skipped: ${path} is filtered out by the defined patterns.`,
LOG_LEVEL_VERBOSE
);
} else {
await this.trackDatabaseFileModification(
path,
"[Hidden file scan]",
!forceWriteAll,
onlyNew,
file,
includeDeletion
);
}
notifyProgress();
} catch (ex) {
this._log(`Failed to process storage change file:${tryGetFilePath(file)}`, logLevel);
this._log(ex, LOG_LEVEL_VERBOSE);
}
});
await Promise.all(processes);
p.done();
}
async applyOfflineChanges(showNotice: boolean): Promise<unknown> {
const logLevel = getLogLevel(showNotice);
return await serialized("applyOfflineChanges", async () => {
const p = this._progress("[⚙ Apply untracked changes ]\n", logLevel);
this._log(`Track changes.`, logLevel);
p.log("Enumerating local files...");
const currentStorageFiles = await this.scanInternalFileNames();
p.log("Enumerating database files...");
const currentDatabaseFiles = await this.getAllDatabaseFiles();
const allDatabaseMap = Object.fromEntries(
currentDatabaseFiles.map((e) => [stripAllPrefixes(this.getPath(e)), e])
);
const currentDatabaseFileNames = [...Object.keys(allDatabaseMap)] as FilePath[];
const untrackedLocal = currentStorageFiles.filter(
(e) => !this.dependencies.processedState.hasLastProcessedFile(e)
);
const untrackedDatabase = currentDatabaseFileNames.filter(
(e) => !this.dependencies.processedState.hasLastProcessedDatabase(e)
);
const bothUntracked = untrackedLocal.filter((e) => untrackedDatabase.indexOf(e) !== -1);
p.log("Applying untracked changes...");
const stat = `Tracking statics:
Local files: ${currentStorageFiles.length}
Database files: ${currentDatabaseFileNames.length}
Untracked local files: ${untrackedLocal.length}
Untracked database files: ${untrackedDatabase.length}
Common untracked files: ${bothUntracked.length}`;
p.once(stat);
const semaphores = Semaphore(10);
const notifyProgress = onlyInNTimes(25, (progress) => p.log(`${progress}/${bothUntracked.length}`));
const allProcesses = bothUntracked.map(async (file) => {
notifyProgress();
const rel = await semaphores.acquire();
try {
const fileStat = await this.storageAccess.statHidden(file);
if (fileStat == null) {
// This should not be happened. But, if it happens, we should skip this.
this._log(`Unexpected error: Failed to stat file during applyOfflineChange :${file}`);
return;
}
const dbInfo = allDatabaseMap[file];
if (dbInfo.deleted || dbInfo._deleted) {
// Applying deletion can be harmful if the local file is not tracked.
// So, we should skip this.
return;
}
const fileMTime = getHiddenFileSyncComparisonMTime(fileStat);
const dbMTime = getHiddenFileSyncComparisonMTime(dbInfo);
const diff = compareMTime(fileMTime, dbMTime);
if (diff == BASE_IS_NEW) {
// Local file is newer than the database file.
// So, we should apply the local file to the database.
await this.processStorageChange(file, true);
} else if (diff == TARGET_IS_NEW) {
// Database file is newer than the local file.
// So, we should apply the database file to the local file.
await this.trackDatabaseFileModification(file, "[Apply]", true, true, dbInfo);
} else if (diff == EVEN) {
// Both are same, we may skip this but should update the last processed key.
this.dependencies.processedState.updateLastProcessed(file, dbInfo, fileStat);
}
} finally {
rel();
}
});
await Promise.all(allProcesses);
await this.scanAllStorageChanges(showNotice);
await this.scanAllDatabaseChanges(showNotice);
p.done();
});
}
async scanAllDatabaseChanges(
showNotice: boolean = false,
onlyNew = false,
forceWriteAll = false,
includeDeletion = true
): Promise<unknown> {
return await skipIfDuplicated("scanAllDatabaseChanges", async () => {
const databaseFiles = await this.getAllDatabaseFiles();
const files = databaseFiles.filter((e) => {
const doc = e;
const key = this.dependencies.processedState.databaseStateKey(doc);
const path = stripAllPrefixes(this.getPath(doc));
const lastKey = this.dependencies.processedState.getLastProcessedDatabaseKey(path);
return lastKey != key;
});
const logLevel = getLogLevel(showNotice);
const staticsMessage = `[Database hidden file statics]
All files: ${databaseFiles.length}
Offline Changed files: ${files.length}`;
this._log(staticsMessage, logLevel, "scan-changes");
return await this.trackScannedDatabaseChange(files, showNotice, onlyNew, forceWriteAll, includeDeletion);
});
}
private async useDatabaseFiles(files: MetaEntry[], showNotice = false, onlyNew = false): Promise<boolean> {
const logLevel = getLogLevel(showNotice);
const p = this._progress(`[⚙ Scanning DB -> Storage ]\n`, logLevel);
p.log("Scanning database files...");
const notifyProgress = onlyInNTimes(25, (progress) => p.log(`${progress}/${files.length}`));
const processFiles = files.map(async (file) => {
try {
const path = stripAllPrefixes(this.getPath(file));
await this.trackDatabaseFileModification(path, "[Scanning]", true, onlyNew, file);
notifyProgress();
} catch (ex) {
this._log(`Failed to process database changes:${tryGetFilePath(file)}`);
this._log(ex, LOG_LEVEL_VERBOSE);
}
return;
});
await Promise.all(processFiles);
p.done();
return true;
}
private async trackDatabaseFileModification(
path: FilePath,
headerLine: string,
preventDoubleProcess = false,
onlyNew = false,
meta: MetaEntry | false = false,
includeDeletion = true
): Promise<boolean> {
return await this.dependencies.changeProcessor.processDatabaseChange(path, headerLine, {
preventDoubleProcess,
onlyNew,
metaEntry: meta,
includeDeletion,
});
}
private async rebuildMerging(showNotice: boolean, targetFiles: FilePath[] | false = false): Promise<FilePath[]> {
const logLevel = getLogLevel(showNotice);
const p = this._progress("[⚙ Rebuild by Merge ]\n", logLevel);
this._log(`Rebuilding hidden files from the storage and the local database.`, logLevel);
p.log("Enumerating local files...");
const currentStorageFilesAll = await this.scanInternalFileNames();
const currentStorageFiles = targetFiles
? currentStorageFilesAll.filter((e) => targetFiles.some((f) => f == e))
: currentStorageFilesAll;
p.log("Enumerating database files...");
const allDatabaseFiles = await this.getAllDatabaseFiles();
const allDatabaseMap = new Map(allDatabaseFiles.map((e) => [stripAllPrefixes(this.getPath(e)), e]));
const currentDatabaseFiles = targetFiles
? allDatabaseFiles.filter((e) => targetFiles.some((f) => f == stripAllPrefixes(this.getPath(e))))
: allDatabaseFiles;
const allFileNames = new Set([
...currentStorageFiles,
...currentDatabaseFiles.map((e) => stripAllPrefixes(this.getPath(e))),
]);
const storageToDatabase = [] as FilePath[];
const databaseToStorage = [] as MetaEntry[];
const eachProgress = onlyInNTimes(100, (progress) => p.log(`Checking ${progress}/${allFileNames.size}`));
for (const file of allFileNames) {
eachProgress();
const storageMTime = await this.storageAccess.statHidden(file);
const mtimeStorage = getHiddenFileSyncComparisonMTime(storageMTime);
const dbEntry = allDatabaseMap.get(file)!;
const mtimeDB = getHiddenFileSyncComparisonMTime(dbEntry);
const diff = compareMTime(mtimeStorage, mtimeDB);
if (diff == BASE_IS_NEW) {
storageToDatabase.push(file);
} else if (diff == TARGET_IS_NEW) {
databaseToStorage.push(dbEntry);
} else if (diff == EVEN) {
// For safety, storage to database.
storageToDatabase.push(file);
}
}
p.once(
`Storage to Database: ${storageToDatabase.length} files\n Database to Storage: ${databaseToStorage.length} files`
);
this.dependencies.processedState.resetLastProcessedDatabase(targetFiles);
this.dependencies.processedState.resetLastProcessedFile(targetFiles);
const processes = [
this.trackScannedStorageChanges(storageToDatabase, showNotice, false, true),
this.useDatabaseFiles(databaseToStorage, showNotice, false),
];
p.log("Start processing...");
await Promise.all(processes);
p.done();
return [...allFileNames];
}
private async runRebuildMerging(showNotice: boolean, targetFiles: FilePath[] | false = false): Promise<FilePath[]> {
return this.rebuildMergingHook
? await this.rebuildMergingHook(showNotice, targetFiles)
: await this.rebuildMerging(showNotice, targetFiles);
}
private async rebuildFromStorage(
showNotice: boolean,
targetFiles: FilePath[] | false = false,
onlyNew = false
): Promise<FilePath[]> {
// reset processed file markers
const logLevel = getLogLevel(showNotice);
this._verbose(`Rebuilding hidden files from the storage.`);
this._log(`Rebuilding hidden files from the storage.`, logLevel);
const p = this._progress("[⚙ Rebuild by Storage ]\n", logLevel);
p.log("Enumerating local files...");
const currentFilesAll = await this.scanInternalFileNames();
const currentFiles = targetFiles
? currentFilesAll.filter((e) => targetFiles.some((f) => f == e))
: currentFilesAll;
p.once(`Storage to Database: ${currentFiles.length} files.`);
p.log("Start processing...");
this.dependencies.processedState.resetLastProcessedFile(targetFiles);
await this.trackScannedStorageChanges(currentFiles, showNotice, onlyNew, true);
p.done();
return currentFiles;
}
private async getAllDatabaseFiles(): Promise<MetaEntry[]> {
const allFiles = (
await this.localDatabase.allDocsRaw({ startkey: ICHeader, endkey: ICHeaderEnd, include_docs: true })
).rows
.filter((e) => isInternalMetadata(e.id as DocumentID))
.map((e) => e.doc) as MetaEntry[];
const files = [] as MetaEntry[];
for (const file of allFiles) {
if (await this.dependencies.isTargetFile(stripAllPrefixes(this.getPath(file)))) {
files.push(file);
}
}
return files;
}
private async rebuildFromDatabase(
showNotice: boolean,
targetFiles: FilePath[] | false = false,
onlyNew = false
): Promise<MetaEntry[]> {
const logLevel = getLogLevel(showNotice);
this._verbose(`Rebuilding hidden files from the local database.`);
this._log(`Rebuilding hidden files from the local database.`, logLevel);
const p = this._progress("[⚙ Rebuild by Database ]\n", logLevel);
p.log("Enumerating database files...");
const allFiles = await this.getAllDatabaseFiles();
// THINKING: Should we exclude conflicted or deleted files?
// Current implementation is to include all files, and following processes will handle for them.
// However, in perspective of performance and future-proofing, I feel somewhat justified in doing it here.
const currentFiles = targetFiles
? allFiles.filter((e) => targetFiles.some((f) => f == stripAllPrefixes(this.getPath(e))))
: allFiles;
p.once(`Database to Storage: ${currentFiles.length} files.`);
this.dependencies.processedState.resetLastProcessedDatabase(targetFiles);
p.log("Start processing...");
await this.useDatabaseFiles(currentFiles, showNotice, onlyNew);
p.done();
return currentFiles;
}
async initialiseInternalFileSync(
direction: InitialisationDirection,
showMessage: boolean,
// filesAll: InternalFileInfo[] | false = false,
targetFilesSrc: string[] | false = false,
initialisationProgress?: ReconciliationProgress
): Promise<void> {
const logLevel = showMessage ? LOG_LEVEL_NOTICE : LOG_LEVEL_INFO;
const p = initialisationProgress ?? this._progress("[⚙ Initialise]\n", logLevel);
// Compatibility question: the legacy preflight was already disabled.
// Enabling it would change initialisation timing and could open a
// conflict dialogue while the feature is being configured.
// p.log("Resolving conflicts before starting...");
// await this.conflictResolution.resolveAll();
p.log("Initialising hidden files sync...");
// The initialisation progress owns the user-visible Notice. Its child
// rebuild and scan operations still write ordinary log entries, but
// must not each create another keep-alive Notice.
const showChildNotices = false;
// TODO: Handling ignore files cannot be performed to the hidden files.
const targetFiles = targetFilesSrc
? targetFilesSrc.map((e) => stripAllPrefixes(e as FilePathWithPrefix))
: false;
if (direction == "pushForce" || direction == "push") {
const onlyNew = direction == "push";
p.log(`Started: Storage --> Database ${onlyNew ? "(Only New)" : ""}`);
const updatedFiles = await this.rebuildFromStorage(showChildNotices, targetFiles, onlyNew);
// making doubly sure, No more losing files.
// I did so many times during the development.
await this.adoptCurrentStorageFilesAsProcessed(updatedFiles);
await this.adoptCurrentDatabaseFilesAsProcessed(updatedFiles);
// And, scan other changes on the database (i.e. files which are on only other devices)
p.log("Checking for remaining storage and database changes...");
await this.scanAllStorageChanges(showChildNotices, true, false);
await this.scanAllDatabaseChanges(showChildNotices, true, false);
}
if (direction == "pullForce" || direction == "pull") {
const onlyNew = direction == "pull";
p.log(`Started: Database --> Storage ${onlyNew ? "(Only New)" : ""}`);
const updatedEntries = await this.rebuildFromDatabase(showChildNotices, targetFiles, onlyNew);
const updatedFiles = updatedEntries.map((e) => stripAllPrefixes(this.getPath(e)));
// making doubly sure, No more losing files.
await this.adoptCurrentStorageFilesAsProcessed(updatedFiles);
await this.adoptCurrentDatabaseFilesAsProcessed(updatedFiles);
// And, scan other changes on the database (i.e. files which are on only other devices)
p.log("Checking for remaining database and storage changes...");
await this.scanAllDatabaseChanges(showChildNotices, true, false);
await this.scanAllStorageChanges(showChildNotices, true, false);
}
if (direction == "safe") {
p.log(`Started: Database <--> Storage (by modified date)`);
const updatedFiles = await this.runRebuildMerging(showChildNotices, targetFiles);
await this.adoptCurrentStorageFilesAsProcessed(updatedFiles);
await this.adoptCurrentDatabaseFilesAsProcessed(updatedFiles);
// And, scan other changes on the database (i.e. files which are on only other devices)
p.log("Checking for remaining storage and database changes...");
await this.scanAllStorageChanges(showChildNotices, true, false);
await this.scanAllDatabaseChanges(showChildNotices, true, false);
}
p.done();
}
interceptRebuildMerging(interceptor: HiddenFileSyncTestingRebuildInterceptor): () => void {
const previousHook = this.rebuildMergingHook;
const runRebuild = async (showNotice: boolean, targetFiles?: FilePath[] | false) =>
await this.rebuildMerging(showNotice, targetFiles);
const hook: HiddenFileSyncTestingRebuild = async (showNotice, targetFiles) =>
await interceptor(runRebuild, showNotice, targetFiles);
this.rebuildMergingHook = hook;
return () => {
if (this.rebuildMergingHook === hook) {
this.rebuildMergingHook = previousHook;
}
};
}
dispose(): void {
this.rebuildMergingHook = undefined;
}
}
export function createReconciliation(dependencies: ReconciliationDependencies): Reconciliation {
return new ReconciliationOwner(dependencies);
}
@@ -0,0 +1,168 @@
import { describe, expect, it, vi } from "vitest";
import {
LOG_LEVEL_INFO,
LOG_LEVEL_NOTICE,
type FilePath,
type MetaEntry,
type UXStat,
} from "@vrtmrz/livesync-commonlib/compat/common/types";
vi.mock("@/deps.ts", () => ({}));
import {
createReconciliation,
type ReconciliationProgress,
type ReconciliationDependencies,
} from "./reconciliation.ts";
const targetPath = ".obsidian/app.json" as FilePath;
const filteredPath = ".obsidian/plugins/other/data.json" as FilePath;
function createFixture(options: { files?: FilePath[]; databaseFiles?: MetaEntry[] } = {}) {
const files = options.files ?? [targetPath];
const databaseFiles = options.databaseFiles ?? [];
const processedFiles = new Map<string, string>();
const progress = {
log: vi.fn(),
once: vi.fn(),
done: vi.fn(),
} satisfies ReconciliationProgress;
const createProgress = vi.fn((_prefix = "", _level = LOG_LEVEL_NOTICE) => progress);
const statHidden = vi.fn(
async (path: FilePath): Promise<UXStat | null> => ({
ctime: 10,
mtime: path == targetPath ? 20 : 30,
size: 10,
type: "file",
})
);
const isTargetFile = vi.fn(async (path: FilePath) => path == targetPath);
const allDocsRaw = vi.fn(async () => ({
rows: databaseFiles.map((doc) => ({ id: doc._id, doc })),
}));
const processedState = {
databaseStateKey: vi.fn((entry: MetaEntry) => `${entry._rev}`),
getLastProcessedDatabaseKey: vi.fn(() => undefined as string | undefined),
getLastProcessedFileKey: vi.fn(() => undefined as string | undefined),
getLastProcessedFileMTime: vi.fn(() => 0),
hasLastProcessedDatabase: vi.fn(() => false),
hasLastProcessedFile: vi.fn(() => false),
getLastProcessedFileKeys: vi.fn(() => processedFiles.keys()),
resetLastProcessedDatabase: vi.fn(),
resetLastProcessedFile: vi.fn(),
storageStateKey: vi.fn((stat: UXStat | null) => `${stat?.mtime ?? 0}`),
updateLastProcessed: vi.fn(),
updateLastProcessedAsActualDatabase: vi.fn(async () => undefined),
updateLastProcessedAsActualFile: vi.fn(async () => undefined),
};
const changeProcessor = {
processStorageChange: vi.fn(async () => true),
processDatabaseChange: vi.fn(async () => true),
};
const dependencies = {
listFiles: vi.fn(async () => ({ files, folders: [] })),
getLocalDatabase: () => ({ allDocsRaw }),
storageAccess: { statHidden },
getRootPath: () => "root",
getPath: (entry: MetaEntry) => entry.path,
isTargetFile,
isIgnoredByIgnoreFile: vi.fn(async () => false),
createProgress,
processedState,
changeProcessor,
log: vi.fn(),
} as unknown as ReconciliationDependencies;
return {
dependencies,
progress,
createProgress,
isTargetFile,
statHidden,
processedState,
changeProcessor,
};
}
function metadata(path: FilePath = targetPath): MetaEntry {
return {
_id: `i:${path}`,
_rev: "2-current",
path: `i:${path}`,
type: "plain",
datatype: "plain",
ctime: 10,
mtime: 20,
size: 10,
children: [],
eden: {},
deleted: false,
} as unknown as MetaEntry;
}
describe("Reconciliation", () => {
it("keeps push initialisation direction and follow-up scan order", async () => {
const fixture = createFixture();
const reconciliation = createReconciliation(fixture.dependencies);
const order: string[] = [];
const scanStorageChanges = vi.spyOn(reconciliation, "scanAllStorageChanges").mockImplementation(async () => {
order.push("storage-scan");
});
const scanDatabaseChanges = vi.spyOn(reconciliation, "scanAllDatabaseChanges").mockImplementation(async () => {
order.push("database-scan");
});
await reconciliation.initialiseInternalFileSync("push", true);
expect(fixture.changeProcessor.processStorageChange).toHaveBeenCalledWith(targetPath, true, true, true);
expect(order).toEqual(["storage-scan", "database-scan"]);
expect(scanStorageChanges).toHaveBeenCalledWith(false, true, false);
expect(scanDatabaseChanges).toHaveBeenCalledWith(false, true, false);
expect(fixture.createProgress).toHaveBeenCalledWith("[⚙ Initialise]\n", LOG_LEVEL_NOTICE);
expect(fixture.createProgress).toHaveBeenCalledWith("[⚙ Rebuild by Storage ]\n", LOG_LEVEL_INFO);
expect(fixture.progress.done).toHaveBeenCalledTimes(3);
});
it("restores rebuild interception in stack order without clobbering a newer hook", async () => {
const fixture = createFixture({ files: [] });
const reconciliation = createReconciliation(fixture.dependencies);
const events: string[] = [];
const first = vi.fn(async (run, showNotice, targetFiles) => {
events.push("first:start");
const result = await run(showNotice, targetFiles);
events.push("first:end");
return result;
});
const second = vi.fn(async (run, showNotice, targetFiles) => {
events.push("second:start");
const result = await run(showNotice, targetFiles);
events.push("second:end");
return result;
});
const restoreFirst = reconciliation.interceptRebuildMerging(first);
const restoreSecond = reconciliation.interceptRebuildMerging(second);
restoreFirst();
await reconciliation.initialiseInternalFileSync("safe", false);
expect(second).toHaveBeenCalledOnce();
restoreSecond();
await reconciliation.initialiseInternalFileSync("safe", false);
expect(first).toHaveBeenCalledOnce();
expect(events[0]).toBe("second:start");
expect(events[events.length - 1]).toBe("first:end");
});
it("uses admission and processed-state keys when selecting storage scan work", async () => {
const fixture = createFixture({ files: [targetPath, filteredPath] });
const reconciliation = createReconciliation(fixture.dependencies);
await reconciliation.scanAllStorageChanges(false);
expect(fixture.isTargetFile).toHaveBeenCalledWith(targetPath);
expect(fixture.isTargetFile).toHaveBeenCalledWith(filteredPath);
expect(fixture.processedState.getLastProcessedFileKey).toHaveBeenCalledWith(targetPath);
expect(fixture.changeProcessor.processStorageChange).toHaveBeenCalledWith(targetPath, false, false, true);
expect(fixture.changeProcessor.processStorageChange).not.toHaveBeenCalledWith(filteredPath, false, false, true);
expect(fixture.progress.once).toHaveBeenCalledWith(expect.stringContaining("Offline Changed files: 1"));
});
});
+3 -80
View File
@@ -1,93 +1,16 @@
import { LOG_LEVEL_VERBOSE } from "octagonal-wheels/common/logger";
import {
LOG_LEVEL_INFO,
LOG_LEVEL_NOTICE,
type AnyEntry,
type DocumentID,
type FilePath,
type FilePathWithPrefix,
type LOG_LEVEL,
} from "@vrtmrz/livesync-commonlib/compat/common/types";
import { MARK_DONE } from "@/modules/features/ModuleLog.ts";
import type { LiveSyncCore } from "@/main.ts";
// import { __$checkInstanceBinding } from "@vrtmrz/livesync-commonlib/compat/dev/checks";
import { createInstanceLogFunction } from "@vrtmrz/livesync-commonlib/compat/services/lib/logUtils";
let noticeIndex = 0;
export abstract class LiveSyncCommands {
core: LiveSyncCore;
get app() {
return this.services.context.app;
}
get settings() {
return this.core.settings;
}
get localDatabase() {
return this.core.localDatabase;
}
get services() {
return this.core.services;
}
async path2id(filename: FilePathWithPrefix | FilePath, prefix?: string): Promise<DocumentID> {
return await this.services.path.path2id(filename, prefix);
}
getPath(entry: AnyEntry): FilePathWithPrefix {
return this.services.path.getPath(entry);
}
import { LiveSyncContext } from "./LiveSyncContext.ts";
export abstract class LiveSyncCommands extends LiveSyncContext {
constructor(core: LiveSyncCore) {
this.core = core;
super(core);
this.onBindFunction(this.core, this.core.services);
this._log = createInstanceLogFunction(this.constructor.name, this.services.API);
// __$checkInstanceBinding(this);
}
abstract onunload(): void;
abstract onload(): void | Promise<void>;
_isMainReady() {
return this.services.appLifecycle.isReady();
}
_isMainSuspended() {
return this.services.appLifecycle.isSuspended();
}
_isDatabaseReady() {
return this.services.database.isDatabaseReady();
}
_log: ReturnType<typeof createInstanceLogFunction>;
_verbose = (msg: unknown, key?: string) => {
this._log(msg, LOG_LEVEL_VERBOSE, key);
};
_info = (msg: unknown, key?: string) => {
this._log(msg, LOG_LEVEL_INFO, key);
};
_notice = (msg: unknown, key?: string) => {
this._log(msg, LOG_LEVEL_NOTICE, key);
};
_progress = (prefix: string = "", level: LOG_LEVEL = LOG_LEVEL_NOTICE) => {
const key = `keepalive-progress-${noticeIndex++}`;
return {
log: (msg: string) => {
this._log(prefix + msg, level, key);
},
once: (msg: string) => {
this._log(prefix + msg, level);
},
done: (msg: string = "Done") => {
this._log(prefix + msg + MARK_DONE, level, key);
},
};
};
_debug = (msg: unknown, key?: string) => {
this._log(msg, LOG_LEVEL_VERBOSE, key);
};
onBindFunction(core: LiveSyncCore, services: typeof core.services) {
// Override if needed.
}

Some files were not shown because too many files have changed in this diff Show More