Refactor optional file synchronisation ownership

docs/adr/2026_09_customisation_and_hidden_file_sync_ownership.md

@@ -20,12 +20,13 @@ one writer for raw events, scheduled scans, and automatic database-to-local
 reflection. The constructor-name add-on lookup and the `ConfigSync` and
 `HiddenFileSync` add-on identities have been retired.
 
-The Customisation Sync private context owns its catalogue, progress, manifest,
-queue, and periodic state. The Hidden File Sync private context owns its
-processed-state caches, reconciliation and conflict queues, activity counts,
-and periodic state. Each receives live settings and database projections,
-focused storage, path, and exact-revision capabilities, and explicit host
-effects rather than `LiveSyncCore`.
+The Customisation Sync private context coordinates its snapshot operations,
+scan queues, periodic work, and focused transient-state owners. The Hidden File
+Sync private context coordinates reconciliation, periodic work, and the
+lifetimes of focused processed-state, change-processing, and
+conflict-resolution owners. Each receives live settings and database
+projections, focused storage, path, and exact-revision capabilities, and
+explicit host effects rather than `LiveSyncCore`.
 
 The corresponding implemented topology is documented in
 [Optional-file synchronisation architecture](../design_docs/optional_file_sync_architecture.md).
@@ -105,7 +106,7 @@ The Customisation Sync runtime will own:
   operations; and
 - the state required to serialise and report those operations.
 
-The Hidden File Sync runtime will own:
+The Hidden File Sync runtime and its composed focused owners will own:
 
 - the device-local processed-state records;
 - one-file storage and database transfer, including exact-revision repair;
@@ -194,15 +195,44 @@ caches, processed-state records, recent-event records, locks, semaphores,
 queues, periodic processors, Notices, and event subscriptions must be created
 and disposed by their feature context or a focused resource owner.
 
+The contexts are orchestration roots, rather than containers for every mutable
+detail. `HiddenFileSyncProcessedState` owns the three device-local maps, their
+autosave initialisation, exact key formats, retained known mtime, reset rules,
+and settlement effects on `IPathService`. Database write and extraction
+operations receive this capability through one narrow `processedState` port;
+they do not receive bundles of individual state callbacks or implement state
+keys themselves.
+
+`CustomisationSyncCatalogueState` separately owns the transient catalogue
+rows, manifest lookup and mtime cache, their reactive stores, and catalogue
+update progress. A small recent-event deduplicator owns raw-event admission
+history. These Customisation Sync owners are deliberately not implementations
+of a shared Hidden File Sync state abstraction: the former state is a derived,
+in-memory projection, while Hidden File Sync markers are persisted operational
+reconciliation state with different identity, invalidation, and deletion
+rules.
+
+`HiddenFileSyncChangeProcessor` owns storage and database change processing,
+the bounded semaphore, same-path event serialisation, activity counts, and the
+inherited order in which processed-state markers and transfer results settle.
+This boundary keeps event concurrency and settlement directly testable without
+giving the processor full scan, initialisation, notification, or host
+responsibilities.
+
 The joint composition may return several views backed by those contexts:
 
 - a Customisation Sync catalogue and operation view for its dialogue;
-- a Hidden File Sync initialisation view for settings workflows; and
-- a Hidden File Sync repair view for the Hatch pane.
+- a Hidden File Sync initialisation view for settings workflows;
+- a Hidden File Sync repair view for the Hatch pane;
+- immutable semantic handler views for registration by the joint composition;
+  and
+- explicitly internal testing views for maintained real-Obsidian workflows.
 
 Several views over one context do not create several owners. Views expose
-stable application data and operations rather than PouchDB entries, queue
-objects, mutable settings records, or the complete core.
+stable application data and named operations rather than PouchDB entries,
+queue objects, mutable settings records, dependency objects, or the complete
+core. The testing views also avoid exposing context instances or writable
+internal state; time-sensitive E2E work uses a scoped operation interceptor.
 
 Obsidian commands, ribbon actions, dialogues, Notices, plug-in reloads, and
 restart scheduling will remain in host-owned composition. The UI will receive
@@ -231,10 +261,10 @@ serviceFeature. `LiveSyncBaseCore.getAddOn()` and its constructor-name lookup
 have been removed.
 
 Production code consumes focused views and does not use the broad context
-surface. Maintained real-Obsidian E2E workflows use an explicitly internal
-test view exposed by the composed feature. This is a transitional test seam,
-not a production service locator, and it should be narrowed as those workflows
-move to public operations or commands.
+surface. Maintained real-Obsidian E2E workflows use explicitly internal,
+immutable test views exposed by the composed feature. These are transitional
+test seams, not production service locators, and they should be narrowed as
+those workflows move to public operations or commands.
 
 The retirement was gated on:
 
@@ -328,11 +358,13 @@ to the non-owner.
 - Replace the complete-core dependency with narrow dependencies.
 
 The private context, path module, codec module, focused presentation view, and
-resource teardown are implemented. Catalogue, enumeration, migration, scan,
-and manifest state is owned per context instance. The context accepts only
-narrow, live projections and explicit effects; an Obsidian adapter at the
-composition edge owns dialogues, Notices, plug-in reload, restart, lifecycle,
-Vault access, and compatibility scan telemetry.
+resource teardown are implemented. A focused catalogue-state owner holds the
+rows, manifests, manifest mtime cache, reactive stores, and update progress,
+while a bounded deduplicator owns recent raw-event keys. Enumeration and scan
+queues remain with the orchestrating context. The context accepts only narrow,
+live projections and explicit effects; an Obsidian adapter at the composition
+edge owns dialogues, Notices, plug-in reload, restart, lifecycle, Vault access,
+and compatibility scan telemetry.
 
 ### Stage 5: extract the Hidden File Sync runtime — implemented
 
@@ -343,16 +375,26 @@ Vault access, and compatibility scan telemetry.
 
 The private context, focused initialisation, repair, and command views,
 host-owned command registration, and processor, cache, subscription, and
-Notice teardown are implemented. The context owns transfer, reconciliation,
-conflict, and processed-state behaviour through narrow live dependencies.
-An Obsidian adapter owns JSON conflict dialogues, progress presentation,
-grouped Notices, plug-in reload, restart scheduling, Vault enumeration, and
-compatibility activity publication.
+Notice teardown are implemented. A focused processed-state owner holds all
+three persisted maps, their key and mtime rules, reset operations, and
+cross-side settlement. Database write and extraction operations consume one
+narrow state port. A focused change processor owns storage and database event
+processing, bounded concurrency, per-path serialisation, activity publication,
+and compatibility settlement order. The context retains scan, initialisation,
+notification, and reconciliation orchestration. A focused conflict-resolution
+owner owns pending-path admission,
+the parallel classification and serial interaction queues, automatic merge,
+newer-revision selection, interactive JSON application, settlement, and queue
+disposal. An Obsidian adapter owns JSON conflict dialogue instances, progress
+presentation, grouped Notices, plug-in reload, restart scheduling, Vault
+enumeration, and compatibility activity publication.
 
 ### Stage 6: move synchronisation composition — implemented
 
 - Register the overlapping Service handlers once through the joint
   serviceFeature.
+- Consume immutable semantic handler views rather than exposing registry-style
+  methods on either context.
 - Preserve the characterised lifecycle callback order and Commonlib
   aggregation semantics through focused tests.
 
@@ -438,11 +480,13 @@ migration without first establishing narrow dependencies.
 - The composition root gains several focused views but does not gain another
   runtime service locator.
 - The legacy add-on identity and constructor-name lookup are removed.
-- The two contexts remain sizeable because each owns one cohesive persisted
-  synchronisation model, but their dependency surfaces are explicit and do
-  not include the complete core. Further extraction should follow a concrete
-  behavioural boundary rather than create additional serviceFeatures for
-  private operations.
+- The two contexts coordinate separate synchronisation workflows, but their
+  dependency surfaces are explicit and do not include the complete core.
+  Customisation Sync delegates its derived catalogue and recent-event state,
+  while Hidden File Sync delegates processed-state, change-processing, and
+  conflict lifecycles, to focused owners. Further extraction should follow a
+  concrete behavioural boundary rather than create additional serviceFeatures
+  for private operations.
 
 ## References
 

docs/design_docs/optional_file_sync_architecture.md

@@ -31,6 +31,9 @@ Obsidian composition (`main.ts`)
   |      +--> pure local-path and document routing policy
   |      |
   |      +--> `CustomisationSyncContext`
+  |      |      +-- `CustomisationSyncCatalogueState`
+  |      |      +-- recent-event deduplicator
+  |      |      +-- immutable service-handler and testing views
   |      |      ^
   |      |      +-- narrow dependencies from
   |      |          `customisationSyncObsidianAdapter`
@@ -39,6 +42,20 @@ Obsidian composition (`main.ts`)
   |             ^
   |             +-- narrow dependencies from
   |                 `hiddenFileSyncObsidianAdapter`
+  |             |
+  |             +--> `HiddenFileSyncProcessedState`
+  |             |      +-- three autosaved reconciliation maps
+  |             |      +-- key, mtime, reset, and settlement rules
+  |             |
+  |             +--> `HiddenFileSyncChangeProcessor`
+  |             |      +-- storage and database change processing
+  |             |      +-- per-path serialisation and activity state
+  |             |
+  |             +--> `HiddenFileSyncConflictResolution`
+  |             |      +-- pending paths and two-stage conflict queue
+  |             |      +-- automatic and interactive JSON resolution
+  |             |
+  |             +-- immutable service-handler, command, repair, and testing views
   |
   +--> `useCustomisationSyncUI`
   |      +-- catalogue and operation view
@@ -58,22 +75,37 @@ application core.
 
 ## Ownership
 
-| Owner | Owns | Does not own |
-| --- | --- | --- |
-| `useOptionalFileSync` | Construction of both contexts, handler registration and removal, local-owner selection, namespace dispatch, compatibility callback order, and context disposal order. | Persisted feature state, synchronisation algorithms, commands, dialogues, or Notices. |
-| `CustomisationSyncContext` | The `ix:` codec and path rules, catalogue, manifest cache, scan queues, migration progress, snapshot storage and application, and periodic scan state. | Obsidian dialogues, plug-in lifecycle APIs, ribbon actions, or handler registration. |
-| `HiddenFileSyncContext` | The `i:` transfer rules, device-local processed-state caches, reconciliation, exact-revision repair, conflict queues, notification batching, activity counts, and periodic scan state. | Obsidian conflict dialogues, grouped Notices, plug-in lifecycle APIs, or handler registration. |
-| Customisation Sync Obsidian adapter | Obsidian conflict selection, Notice presentation, plug-in reload, restart requests, Vault enumeration, progress telemetry, and platform-derived fallback device names. | Catalogue state, routing, or persisted document operations. |
-| Hidden File Sync Obsidian adapter | JSON conflict dialogue lifetime, progress presentation, grouped change Notices, plug-in reload actions, restart scheduling, Vault enumeration, and compatibility activity publication. | Transfer, reconciliation, processed-state, or conflict decisions. |
-| `useCustomisationSyncUI` | Command, ribbon, dialogue, open-request subscription, and their unload teardown. | Synchronisation state or Hidden File Sync initialisation behaviour. |
-| `useHiddenFileSyncCommands` | Hidden File Sync command registration, setting-change subscription, and their unload teardown. | Synchronisation state or command implementation. |
+| Owner                                 | Owns                                                                                                                                                                                                          | Does not own                                                                                                          |
+| ------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | --------------------------------------------------------------------------------------------------------------------- |
+| `useOptionalFileSync`                 | Construction of both contexts, handler registration and removal, local-owner selection, namespace dispatch, compatibility callback order, and context disposal order.                                         | Persisted feature state, synchronisation algorithms, commands, dialogues, or Notices.                                 |
+| `CustomisationSyncContext`            | The `ix:` codec and path rules, scan queues, snapshot storage and application, periodic scan state, and the lifetimes of its transient-state owners.                                                          | Catalogue mutations, recent-event history, Obsidian dialogues, ribbon actions, or handler registration.               |
+| `CustomisationSyncCatalogueState`     | Transient catalogue rows, manifest lookup and mtime cache, reactive catalogue and manifest stores, and catalogue update progress.                                                                             | Database or storage I/O, scan scheduling, routing, or persistence.                                                    |
+| Customisation Sync event deduplicator | The bounded, newest-first keys used to admit raw configuration events once.                                                                                                                                   | Scheduling, path ownership, catalogue state, or persisted data.                                                       |
+| `HiddenFileSyncContext`               | The `i:` scan and reconciliation workflow, exact-revision repair composition, notification batching, periodic scan state, and focused-owner lifetimes.                                                        | Change-event serialisation, processed-state representation, conflict queue state, Obsidian dialogues, or service handler registration. |
+| `HiddenFileSyncProcessedState`        | Three device-local autosaved maps, exact storage and database keys, retained known mtime, reset behaviour, and storage/database settlement effects.                                                           | File transfer, scan scheduling, conflict handling, or presentation.                                                   |
+| `HiddenFileSyncChangeProcessor`       | Storage and database change processing, same-path event serialisation, bounded concurrency, activity counts, and the inherited event-consumption and settlement order.                                      | Full scans, initialisation policy, notification presentation, or conflict interaction.                               |
+| `HiddenFileSyncConflictResolution`    | Conflict admission and deduplication, pending paths, the parallel classification and serial interaction queues, automatic merge, newer-revision policy, interactive JSON resolution, and conflict settlement. | Obsidian dialogue instances, general Hidden File Sync scans, processed-state caches, or Service handler registration. |
+| Customisation Sync Obsidian adapter   | Obsidian conflict selection, Notice presentation, plug-in reload, restart requests, Vault enumeration, progress telemetry, and platform-derived fallback device names.                                        | Catalogue state, routing, or persisted document operations.                                                           |
+| Hidden File Sync Obsidian adapter     | JSON conflict dialogue lifetime, progress presentation, grouped change Notices, plug-in reload actions, restart scheduling, Vault enumeration, and compatibility activity publication.                        | Transfer, reconciliation, processed-state, or conflict decisions.                                                     |
+| `useCustomisationSyncUI`              | Command, ribbon, dialogue, open-request subscription, and their unload teardown.                                                                                                                              | Synchronisation state or Hidden File Sync initialisation behaviour.                                                   |
+| `useHiddenFileSyncCommands`           | Hidden File Sync command registration, setting-change subscription, and their unload teardown.                                                                                                                | Synchronisation state or command implementation.                                                                      |
 
-The two domain contexts remain sizeable because they each own one cohesive
-persisted synchronisation model. Their private operations are not additional
-serviceFeatures: they do not independently register host integration or have
-separate application lifetimes. Extract a further ordinary module or focused
-state owner when a concrete invariant, replacement lifecycle, or independently
-testable operation justifies that boundary.
+The two domain contexts coordinate one cohesive synchronisation workflow each.
+Their private operations and focused owners are not additional serviceFeatures:
+they do not independently register host integration or have separate
+application lifetimes. `HiddenFileSyncChangeProcessor` is a focused resource
+owner because its semaphore, per-path serialisation, activity counters, and
+event settlement form one independently testable lifecycle.
+`HiddenFileSyncConflictResolution` is another focused resource owner because
+conflict admission, pending-path identity, two serialisation stages, and
+disposal form a separate lifecycle.
+
+The two state owners intentionally do not implement a common generic state
+contract. Customisation Sync projects transient catalogue and presentation
+state from `ix:` documents. Hidden File Sync persists operational markers used
+for reconciliation, with distinct path identity, deletion, reset, and retained
+mtime rules. Their common boundary is lifecycle ownership by a context, rather
+than interchangeable state semantics.
 
 ## Routing and handler contracts
 
@@ -85,21 +117,24 @@ static policy selects Hidden File Sync.
 
 The maintained local ownership is:
 
-| Path mode | Owner |
-| --- | --- |
-| Customisation path in Selective or Flagged Selective mode | Customisation Sync |
-| Customisation path in Automatic mode, with Hidden File Sync enabled | Hidden File Sync |
-| Customisation path in Ignore mode | Neither context |
-| Other eligible hidden path | Hidden File Sync |
-| Disabled, excluded, ignored, or ordinary Vault path | Neither context |
+| Path mode                                                           | Owner              |
+| ------------------------------------------------------------------- | ------------------ |
+| Customisation path in Selective or Flagged Selective mode           | Customisation Sync |
+| Customisation path in Automatic mode, with Hidden File Sync enabled | Hidden File Sync   |
+| Customisation path in Ignore mode                                   | Neither context    |
+| Other eligible hidden path                                          | Hidden File Sync   |
+| Disabled, excluded, ignored, or ordinary Vault path                 | Neither context    |
 
 Local ownership is distinct from persisted-document recognition. The
 composition dispatches `ps:` and `ix:` conflict documents to Customisation
-Sync and `i:` conflict documents to Hidden File Sync. Existing documents remain
-recognisable after a local mode changes.
+Sync and sends `i:` conflict documents through the Hidden File Sync semantic
+handler view. Existing documents remain recognisable after a local mode
+changes.
 
-The composition adapts the contexts to the existing Commonlib handler
-contracts:
+Each context exposes an immutable semantic handler view. The composition
+registers these operations and adapts them to the existing Commonlib handler
+contracts; registry aggregation names and binding concerns do not leak back
+into either context:
 
 - raw optional-file events are offered to exactly one selected owner;
 - a selected handler which skips or fails does not fall through to the other
@@ -148,23 +183,35 @@ owners:
   operations to the Hatch pane.
 
 Production consumers receive these views directly. `OptionalFileSyncFeature`
-also exposes an explicitly internal `testing` view for maintained real-Obsidian
-contract tests. That test seam is not a production service locator and should
-not be used by application features.
+also exposes immutable, explicitly internal testing views for maintained
+real-Obsidian contract tests. They provide named operations, including a scoped
+rebuild interceptor, without exposing context instances, dependency objects,
+queues, or writable stores. These test seams are not production service
+locators and should not be used by application features.
 
 ## Lifecycle and disposal
 
 The composition is created after the Service Hub and required ServiceModules
 exist, and before lifecycle-driven feature work begins. Each context creates
-its own queues, caches, semaphores, activity state, and periodic processor.
+its own periodic processor and focused resource owners.
+`CustomisationSyncContext` creates one catalogue-state owner and one
+recent-event deduplicator. `HiddenFileSyncContext` creates one processed-state
+owner before composing database write and extraction operations around its
+narrow port, then creates one change processor and one conflict-resolution
+owner. The change processor owns its semaphore and activity state.
+Full conflict scans admit discovered paths into the same queue without
+suspending it, so ordinary database conflict notifications continue during a
+slow scan. After enumeration, the operation waits for both classification and
+interaction stages to drain.
 
 On application unload, `useOptionalFileSync` first removes every Service
 handler registration. It then disposes Customisation Sync followed by Hidden
 File Sync, preserving the former compatibility order. Disposal disables
-periodic admission, terminates queues, clears transient caches and pending
-sets, cancels scheduled notification work, resets compatibility telemetry, and
-hides owned Notices. The two presentation serviceFeatures independently remove
-their commands, event subscriptions, ribbon state, and dialogue instances.
+periodic admission, disposes the conflict-resolution owner, terminates queues,
+clears transient caches and pending sets, cancels scheduled notification work,
+resets compatibility telemetry, and hides owned Notices. The two presentation
+serviceFeatures independently remove their commands, event subscriptions,
+ribbon state, and dialogue instances.
 
 ## Persisted compatibility
 
@@ -184,9 +231,11 @@ appropriate, a migration decision.
 
 ## Verification boundaries
 
-Focused unit tests cover routing, handler aggregation, context state
-isolation, teardown, initial cache selection, exact-revision repair, conflict
-dialogue adaptation, grouped Notices, and compatibility activity publication.
+Focused unit tests cover routing, semantic handler views, context state
+isolation, teardown, initial cache selection, exact-revision repair, change
+event serialisation and settlement, conflict queue admission, revision
+selection, automatic and interactive merge effect ordering, conflict dialogue
+adaptation, grouped Notices, and compatibility activity publication.
 The boundary test prevents either domain context from regaining core or
 Obsidian dependencies.