From 7aa41baf089a9ba0a282f590fb732803b18388da Mon Sep 17 00:00:00 2001 From: vorotamoroz Date: Thu, 27 Aug 2026 08:58:48 +0000 Subject: [PATCH] Document replicator capability and lifecycle contracts --- ...eplicator_capabilities_01_core_contract.md | 516 ++++++++++++++++++ ...r_capabilities_02_p2p_service_lifecycle.md | 301 ++++++++++ ...plicator_capabilities_03_migration_plan.md | 268 +++++++++ 3 files changed, 1085 insertions(+) create mode 100644 docs/adr/2026_08_replicator_capabilities_01_core_contract.md create mode 100644 docs/adr/2026_08_replicator_capabilities_02_p2p_service_lifecycle.md create mode 100644 docs/adr/2026_08_replicator_capabilities_03_migration_plan.md diff --git a/docs/adr/2026_08_replicator_capabilities_01_core_contract.md b/docs/adr/2026_08_replicator_capabilities_01_core_contract.md new file mode 100644 index 00000000..1bf66f34 --- /dev/null +++ b/docs/adr/2026_08_replicator_capabilities_01_core_contract.md @@ -0,0 +1,516 @@ +--- +date: 2026-08-27 +commonlib-version: "0.1.19" +self-hosted-livesync-version: "1.0.21" +status: proposed +series: replicator-capabilities-and-lifecycle +part: 1 of 3 +--- + +# Architectural Decision Record: Replicator Capabilities and Lifecycle Orchestration — Part 1: Core Contract + +Series navigation: this is Part 1 of 3. Continue with [Part 2: P2P service and +session lifecycle](2026_08_replicator_capabilities_02_p2p_service_lifecycle.md), +then [Part 3: migration plan and verification](2026_08_replicator_capabilities_03_migration_plan.md). + +## Status + +Proposed. This record defines the provider, capability, lifecycle, interaction, +ownership, and probe boundaries required by current Self-hosted LiveSync +consumers. It is the generic part of the series; the P2P-specific ownership +rules live in Part 2, and implementation sequencing lives in Part 3. + +The accepted P2P room and transport lifecycle record remains authoritative for +the current P2P implementation until Stage 3 in Part 3 is complete. The +supersession boundary for that record is stated in Part 2 and is not repeated +here. + +## Context + +Self-hosted LiveSync currently represents CouchDB, Object Storage, and P2P with +one `LiveSyncAbstractReplicator` base class. That base class requires finite +and continuous replication, full upload and download, remote creation and +reset, lock administration, preferred-tweak Metadata, on-demand Chunk reads, +remote status, integrity inspection, and connected-device inspection. + +These operations do not share one support boundary: + +- CouchDB supports unattended OneShot Sync, Continuous replication, central + remote administration, and CouchDB-specific inspection and maintenance. +- Object Storage supports finite journal synchronisation, central reset and + lock Metadata, full upload and download, and storage-size inspection. It has + no continuous changes feed, CouchDB Chunk source, CouchDB integrity + inspection, or CouchDB device registry. +- P2P supports peer-targeted finite transfer and an independently owned room, + signalling, watch, and broadcast lifecycle. It has no central database to + create, reset, lock, inspect for size, or upload during first-device setup. + It also has peer-driven AutoSync and AutoWatch paths when the room is open. + Those paths are detailed in Part 2. + +The abstract class makes absent capabilities look like operations. Current +implementations express absence through thrown errors, `false`, empty arrays, +zero counts, silent success, and a dummy all-zero Security Seed. Callers +cannot tell whether an operation was performed, was inapplicable, or could not +be performed. + +A neutral value is correct only when it is the documented identity for every +caller. For example, Object Storage has no remote-Chunk role, whereas a +supported CouchDB Chunk request may legitimately return an empty array. A +zero integrity count cannot safely stand for an inspection which did not run. + +Issue 1140 exposes the lifecycle consequence. In version 1.0.21, +`ModuleReplicatorCouchDB` owns the application resume callback and excludes +Object Storage and P2P by `remoteType`. Object Storage accepts `syncOnStart`, +but no journal synchronisation starts at resume. Periodic, event-driven, and +manual calls later reach the active Replicator successfully. + +The construction boundary is also overloaded. `getNewReplicator()` is an +order-dependent first-result handler which ignores false results and catches +handler errors. It is used for active Replicator acquisition, trial settings, +and temporary command instances. P2P construction can replace and close a +service-owned current transport, so an apparently temporary request can +disturb an active or adjunct transport. + +Fast Setup is a separate boundary. Streaming Fetch is a CouchDB-specific +initial transfer which uses CouchDB HTTP settings and a Security Seed supplier; +it does not need a full Replicator or an owned PouchDB connection. It must not +be made a generic Replicator capability merely because the current code obtains +one as a supplier. + +Finally, current operation results can hide failure. Object Storage can discard +a failed or stopped journal result and report success. A headless P2P path can +complete but return `undefined`, which the caller interprets as failure. Remote +mutations can be caught or ignored before Rebuilder continuation, and an +offline integrity inspection can be represented as zero. The contract must +make those outcomes truthful. + +## Decision drivers + +The design must: + +1. cover every operation used by the plug-in, CLI, WebApp, WebPeer, Setup, + Rebuilder, and maintenance flows; +2. keep lifecycle policy independent of provider class names; +3. make an omitted support decision a compile-time error when a current + provider or capability is added; +4. carry trigger and interaction policy through `ReplicationService`, so an + automatic trigger cannot open a dialogue; +5. distinguish capability absence, unavailable observation, and an observed + empty value where that difference affects safety; +6. retain simple neutral results where they are safe identities for every + caller; +7. give active Replicator, flow-specific probe, and transport resources one + explicit owner and disposal boundary; and +8. report replication, central-remote mutation, and maintenance outcomes + truthfully while preserving source compatibility during migration. + +## Decision + +### Use precise ownership terms + +- A **provider definition** is the host-composed, exhaustive declaration for + one current remote kind. +- The **active Replicator** is the selected main-remote handle owned by + `ReplicatorService`. +- A **probe** is a short-lived, flow-specific validation resource whose caller + owns disposal. +- The **P2P service** is the stable Commonlib implementation which supplies + narrow P2P contract views. Its room-session ownership is defined in Part 2. +- A **room session** and **session epoch** are P2P terms defined in Part 2; an + epoch is an internal fence, not a public capability or logical room name. + +An active Replicator is a handle, not necessarily the owner of every transport +which it uses. In particular, the active P2P Replicator is a non-owning adapter +over the P2P service. Its ownership consequences are specified once in Part 2. + +### Separate policy, provider selection, and the active Replicator + +Application lifecycle policy decides **when** synchronisation is requested. +The selected provider and its active Replicator decide **how**, and whether, +that request can be performed. A provider module must not subscribe to the +application resume lifecycle merely because it can construct a transport. + +Self-hosted LiveSync will have one LiveSync-owned core ServiceModule as the +replication lifecycle coordinator. It uses `AppLifecycleService`, persisted +settings, `ReplicationService`, and the active support declaration. It does +not branch on `remoteType` or use `instanceof` as a capability test. +Commonlib owns the trigger-aware replication contract; the host owns +application lifecycle wiring. + +The existing `onResumed` event remains the eligible-resume boundary after +initial readiness, settings application, and visibility recovery. It is not +redefined as a once-per-process event. The coordinator coalesces duplicate work +within one lifecycle generation and preserves readiness and suspension gates: + +- configured Continuous replication starts only through an active Continuous + role; +- otherwise, configured `syncOnStart` runs through an unattended OneShot role + when that role is supported; +- an unsupported configured policy produces an explicit unsupported or + not-implemented result; and +- automatic start-up, periodic, file-event, and merge triggers never open a + dialogue. A target-requiring operation is available to an explicit user + action or to a flow with a configured target. + +`ReplicationService` remains responsible for readiness checks, bounded finite +activity, failure processing, and replication timing. It exposes distinct +user-initiated and unattended entry points, or a typed request which carries +interaction authority. The coordinator never calls a concrete Replicator's +`openReplication()` directly. + +`P2P_AutoStart` remains a separate P2P room policy. It is not central +Continuous replication and is not `syncOnStart`; its service lifecycle is +specified in Part 2. Reopening after `EVENT_DATABASE_REBUILT` is a +flow-authorised continuation requested by the Rebuilder, not evidence that +AutoStart is enabled. + +The CLI daemon owns its initial finite convergence before its mirror scan. +Restored settings mark that convergence as satisfied for the current lifecycle +generation, so `syncOnStart` does not repeat it. In `--interval` mode, the +daemon poller is the sole recurring remote-poll scheduler. In changes-feed mode, +the coordinator starts one configured Continuous session when supported; +otherwise it may enable the configured generic periodic timer. Continuous has +precedence when both are configured. + +### Use a fixed current-provider definition + +Commonlib defines the canonical current remote kinds, provider contract, +capability catalogue, support-decision type, and typed provider builder. Each +host composition explicitly declares the provider kinds which it includes and +supplies an exhaustive definition table for that set. The current catalogue is +CouchDB, Object Storage, and P2P; it is not a public third-party registration +API. + +CouchDB is part of every current host composition. Object Storage and P2P are +compile-time composition choices and may be included or omitted without +changing the generic lifecycle coordinator. Adding another current provider +requires a Commonlib kind and support declaration, host composition, +Setup/profile schema handling, and provider-specific tests. It does not require +a runtime plug-in registry or behaviour for unknown provider kinds. + +Each provider definition supplies: + +- canonical kind and diagnostic name; +- complete support metadata for the current catalogue; +- active Replicator construction; +- provider configuration identity and an explicit same-kind rebind-or-replace + policy; +- flow-specific probe and initial-transfer dependency factories; and +- provider-specific entry points required by existing current flows. + +The host composes CouchDB and Object Storage definitions directly. A module is +retained only when it owns separate state or behaviour; a factory-registration- +only module instance is not required. The stateful P2P service is composed +independently and may be an adjunct beside a different selected main provider. + +The definition table uses `satisfies` against required record keys. Adding a +current provider or catalogue entry without a support decision is a compile-time +error. A typed builder correlates each `supported` decision with its required +role and rejects a role for an absent capability. + +Support has three stable states: + +```typescript +type CapabilitySupport = + | { readonly kind: "supported" } + | { readonly kind: "not-implemented"; readonly reason: CapabilityReason } + | { readonly kind: "not-applicable"; readonly reason: CapabilityReason }; +``` + +`not-implemented` means that the provider model exists but the current +implementation does not supply it. `not-applicable` means that the model does +not exist for that provider, such as central database locking for P2P. Reasons +are stable codes, not arbitrary user-facing strings. + +Historical empty `remoteType` is resolved explicitly as CouchDB at the +persistence/profile boundary. Capability selection then uses that canonical +kind; it never infers CouchDB by truthiness or by a negative test such as +'neither Object Storage nor P2P'. + +If a provider caches effective connection settings, its declared rebind or +replacement policy keeps the active adapter current after a same-kind profile +change. Reconciliation is serialised with active work and leaves no old +credentials, Security Seed state, journal checkpoint, adapter cache, or +diagnostic connection reachable from the reconciled handle. + +### Separate exhaustive support metadata from narrow runtime roles + +Every provider definition contains a required support record over the complete +current catalogue. An active Replicator exposes only the narrow roles marked as +supported by its definition. This gives compile-time completeness without a +giant runtime facade or a collection of Boolean flags. + +The generic catalogue covers user-initiated and unattended OneShot Sync, +ordinary long-lived Continuous replication, full upload and download, central +reset and lock administration, preferred-tweak Metadata read and write, +on-demand remote Chunk reads, remote storage status, compromised-Chunk +inspection, central Security Seed, and a request to stop active transfer. + +Full upload/download, reset/lock, and Metadata read/write remain separate roles. +Provider initialisation remains flow-specific: CouchDB may create a database, +whereas Object Storage prepares its Security Seed and does not provision a +bucket. Provider-specific CouchDB maintenance, Object Storage journal +checkpoint maintenance, and P2P room operations are narrowed facets, not +generic feature tests. P2P facets are defined in Part 2. + +Local node identity initialisation is a database and Replicator lifecycle +concern, not a remote capability. Replication statistics remain a +`ReplicatorService` telemetry sink. + +### Make unattended work explicit and truthful + +User-initiated and unattended OneShot Sync are separate roles. Interaction +authority is an upper bound on local interaction, not an instruction to display +a dialogue. An operation may apply a stricter veto, but cannot request an +interaction which its caller did not permit. Remote refusal and incoming-peer +consent remain independent decisions. + +```typescript +type UnattendedTrigger = "resume" | "periodic" | "database-event" | "editor-save" | "file-open" | "merge" | "daemon"; + +interface InteractionPermissions { + readonly peerSelection: boolean; + readonly localPeerAdmission: boolean; + readonly configurationExchange: boolean; + readonly failureRecovery: boolean; +} + +type PermittedInteractionPermissions = + | (InteractionPermissions & { readonly peerSelection: true }) + | (InteractionPermissions & { readonly localPeerAdmission: true }) + | (InteractionPermissions & { readonly configurationExchange: true }) + | (InteractionPermissions & { readonly failureRecovery: true }); + +type InteractionAuthority = + | typeof NO_INTERACTION + | { readonly kind: "permitted"; readonly permissions: PermittedInteractionPermissions }; + +const NO_INTERACTION = { kind: "forbidden" } as const; + +interface UserInitiatedOneShotRequest { + readonly trigger: "manual"; + readonly interaction: InteractionAuthority; +} + +interface UserInitiatedOneShot { + run(request: UserInitiatedOneShotRequest): Promise; +} + +interface UnattendedOneShot { + run(request: { + readonly trigger: UnattendedTrigger; + readonly interaction: typeof NO_INTERACTION; + }): Promise; +} +``` + +`NO_INTERACTION` is the only all-false authority. Shared immutable authority +values are reused by hosts, avoiding a permission allocation per operation. +Operation-specific requests, including P2P configuration exchange, carry the +same upper bound and expose only relevant permissions. An unattended path may +use persisted or automatic acceptance policy, but cannot obtain local +interaction authority implicitly. + +Outcomes do not collapse failure into `void` or `boolean`: + +```typescript +const REPLICATION_COMPLETED = { status: "completed" } as const; +const REPLICATION_CANCELLED = { status: "cancelled" } as const; + +type ReplicationOutcome = + | typeof REPLICATION_COMPLETED + | typeof REPLICATION_CANCELLED + | { readonly status: "blocked"; readonly reason: ReplicationBlockReason } + | { readonly status: "partial"; readonly detail: PartialReplicationDetail } + | { readonly status: "failed"; readonly error: unknown }; +``` + +Completed and cancelled values are shared singletons or literals. Blocked, +partial, and failed results may carry diagnostic detail. Central provider +initialisation, reset, lock, unlock, and resolution settle only after their +defined remote write succeeds; a Rebuilder must not continue after an ignored +mutation failure. + +### Distinguish observation from an identity result only when required + +Capability availability and operation results are separate. A supported +operation may fail, while an inapplicable operation must not be called or +reported as a network attempt. + +For an observation where an empty value changes a safety decision, use a tagged +result: + +```typescript +type RemoteObservation = + | { readonly kind: "observed"; readonly value: T } + | { readonly kind: "unavailable"; readonly error?: unknown }; +``` + +Compromised-Chunk inspection is the current example. `observed: 0` proves that +the supported inspection found no matching entries; `unavailable` says that it +did not complete, including while offline. A caller must handle every result +before declaring the remote clean. + +Do not wrap every result. When an empty array is the safe, documented identity +for every current caller, retain it. High-frequency Chunk, document, +changes-feed, and queue paths keep arrays, iterators, and scalars unless a +measurement and a safety distinction justify a tag. No wrapper is added per +Chunk, document, changes-feed row, or queue item. + +### Give active ownership and probes explicit boundaries + +`ReplicatorService` is the sole owner of the active Replicator. Replacement is +an atomic transition under its transition lock: + +```text +active -> retiring -> disposed -> replacement published +``` + +Acquisitions wait for the transition and receive only the replacement. A fenced +old handle cannot start work. Disposal stops Continuous activity, awaits work +which cannot be cancelled, and settles or reports every operation owned by the +active adapter before publication. If bounded retirement cannot settle, +replacement fails visibly and no new handle is published; the old handle is +disposed when late work settles. + +The generic stop role is an idempotent request to stop a provider transfer after +transport work begins. It does not promise cancellation of readiness checks, +Security Seed acquisition, or external storage calls which do not consume a +cancellation signal. P2P declares this role `not-implemented` until its lower +level transfer and RPC work have a real stop path. + +`getNewReplicator()` is not a general temporary-instance API. Setup and settings +flows request narrow probes, such as connection, preferred-tweak, or isolated +P2P signalling validation. A resource-returning factory returns an owned +resource with idempotent asynchronous `dispose()`. Trial settings are passed +to the probe itself and cannot silently read active settings. A probe cannot +replace the active Replicator or the P2P service. + +Streaming Fetch receives an owned CouchDB HTTP configuration and +`RemoteSecuritySeed` supplier directly. The supplier is bound to the selected +configuration identity, invalidates cached seed state on reconciliation, and +is disposed by the initial-transfer flow. It does not construct a temporary +full Replicator. + +### Keep initialisation workflows explicit + +- CouchDB and Object Storage first-device rebuilds reset and lock their central + remote, then perform the established convergence upload. +- A P2P first-device rebuild prepares the local database without pretending to + reset, lock, or upload to a central remote. +- CouchDB and Object Storage Fetch use their central full-download flow. +- A P2P additional-device Fetch selects a peer and performs one full download. +- CouchDB Streaming Fetch remains the separate initial-transfer service above. + +The Rebuilder requests capabilities and does not cast a generic Replicator to a +concrete class. Its `EVENT_DATABASE_REBUILT` continuation is separately +authorised and does not imply `syncOnStart` or P2P AutoStart. + +## Target capability matrix for current providers + +`S` means supported, `NI` means not implemented, and `NA` means not applicable. +Configuration and reachability are request preconditions or outcomes, not +support states. + +| Generic role | CouchDB | Object Storage | P2P | +| --------------------------------------- | ------- | -------------- | --- | +| User-initiated OneShot Sync | S | S | S | +| Unattended OneShot Sync without UI | S | S | S | +| Ordinary long-lived Continuous session | S | NA | NA | +| Full upload to a central remote | S | S | NA | +| Full download | S | S | S | +| Central remote reset | S | S | NA | +| Central remote lock and resolution | S | S | NA | +| Preferred-tweak Metadata read and write | S | S | NA | +| On-demand remote Chunk source | S | NA | NA | +| Remote storage status | S | S | NA | +| Compromised-Chunk inspection | S | NA | NA | +| Central-remote Security Seed | S | S | NA | +| Request to stop active transfer | S | S | NI | + +P2P unattended OneShot means a role exists which uses configured target names +without opening a dialogue. Peer-room, watch, acceptance, and broadcast roles +are provider-specific facets, not the ordinary Continuous role; their trigger +matrix and de-duplication rules are owned by Part 2. + +## Alternatives rejected + +### Move the resume handler and retain a `remoteType` switch + +This would fix issue 1140 narrowly, but would leave future providers and +triggers subject to the same omission. It would not stop automatic P2P calls +from reaching an interactive method, and capability semantics would remain +implicit. + +### Use only optional methods or Boolean support flags + +Optional runtime roles are useful, but they do not force a support decision when +a provider or catalogue entry is added. Exhaustive support metadata plus a +typed builder keeps runtime interfaces small while requiring the decision at +compile time. + +### Use only a provider-discriminated union + +A provider union is appropriate for provider-specific maintenance after +explicit narrowing. It cannot model adjunct P2P beside another selected main +provider and is not a generic feature test. + +### Tag every value and hot-path return + +Uniform wrappers would add allocation and noise to Chunk, document, +changes-feed, and queue paths without improving semantics where an identity is +safe. Tags remain for low-frequency observations and outcomes which affect +control flow. + +### Retain `getNewReplicator()` as the trial and command factory + +The handler is order-dependent, can suppress construction errors, and can +replace a feature-owned P2P transport. Flow-specific probes and the stable P2P +service make ownership explicit. + +### Treat neutral compatibility results as supported operations + +Dummy zero counts, empty Security Seeds, false values, and silent mutations +lose distinctions required for safety and recovery. A neutral value remains +only when every caller proves it to be the operation's identity. + +## Consequences + +- `syncOnStart` becomes an application policy for every unattended finite + provider which supports it, rather than a CouchDB module behaviour. +- Adding a current provider or capability requires an explicit compile-time + support decision. +- Unsupported central operations are no longer represented as successful P2P + no-ops or transport errors. +- Safe identities remain simple, while safety-sensitive observations are + explicit. +- Setup and trial configuration cannot replace an active or adjunct transport. +- Central mutations and integrity checks cannot silently turn failure or + unavailability into success. +- P2P remains first-class without being mislabelled as central Continuous + replication. + +## Non-goals + +- Do not introduce third-party remote-provider registration. +- Do not redesign every replication result or user-facing message in one + change. +- Do not make Streaming Fetch a generic Replicator capability. +- Do not make P2P pretend to own a central remote database. +- Do not infer support from `remoteType`, constructor identity, a falsy result, + or a neutral value. +- Do not redefine `syncOnStart` as a once-per-process setting. +- Do not change established profile persistence or Setup flag-file restart + ordering as part of this capability split. + +## References + +- [Part 2: P2P service and session lifecycle](2026_08_replicator_capabilities_02_p2p_service_lifecycle.md) +- [Part 3: migration plan and verification](2026_08_replicator_capabilities_03_migration_plan.md) +- [Bounded Remote Activity](2026_07_bounded_remote_activity.md) +- [Make Onboarding Profile-Aware](2026_07_multiple_remote_onboarding.md) +- [P2P Room and Transport Lifecycle](2026_07_p2p_transport_lifecycle.md) +- [P2P Transport Compatibility Controls](2026_08_p2p_transport_compatibility.md) +- [CouchDB Remote Connection Ownership](2026_08_couchdb_remote_connection_ownership.md) +- [Package the Common Library Behind Explicit Host Boundaries](2026_07_common_library_package_boundary.md) +- [Self-hosted LiveSync issue 1140](https://github.com/vrtmrz/obsidian-livesync/issues/1140) diff --git a/docs/adr/2026_08_replicator_capabilities_02_p2p_service_lifecycle.md b/docs/adr/2026_08_replicator_capabilities_02_p2p_service_lifecycle.md new file mode 100644 index 00000000..26d900b5 --- /dev/null +++ b/docs/adr/2026_08_replicator_capabilities_02_p2p_service_lifecycle.md @@ -0,0 +1,301 @@ +--- +date: 2026-08-27 +commonlib-version: "0.1.19" +self-hosted-livesync-version: "1.0.21" +status: proposed +series: replicator-capabilities-and-lifecycle +part: 2 of 3 +--- + +# Architectural Decision Record: Replicator Capabilities and Lifecycle Orchestration — Part 2: P2P Service and Session Lifecycle + +Series navigation: this is Part 2 of 3. Read [Part 1: core contract](2026_08_replicator_capabilities_01_core_contract.md) +first, then continue with [Part 3: migration plan and verification](2026_08_replicator_capabilities_03_migration_plan.md). + +## Status + +Proposed. This record defines the P2P service owner, room-session boundary, +narrow contract views, automation demands, replacement fencing, and trigger +semantics. Generic provider and capability rules are owned by Part 1; the +implementation and verification order is owned by Part 3. + +The accepted [P2P Room and Transport Lifecycle](2026_07_p2p_transport_lifecycle.md) +record remains the description of the current implementation until Stage 3 in +Part 3 is complete. Stage 3 then supersedes only that record's replaceable +LiveSync P2P Replicator and current-result ownership. Its decisions about +serialised room operations, `room.leave()`, Trystero-owned physical peers, and +relay reconnection remain in force. + +## Scope and context + +The Commonlib P2P feature currently spans `TrysteroReplicatorP2PServer`, +`TrysteroReplicator`, and the LiveSync-specific `LiveSyncTrysteroReplicator`. +The current result resolves a replaceable Replicator while the room, signalling, +watch, broadcast, diagnostics, platform-event subscriptions, and database +feeds have longer-lived relationships. Installing that replaceable object as +the active Replicator makes ordinary active-handle disposal capable of closing +an adjunct or policy-owned room. + +The same room membership serves several independent behaviours: + +- P2P AutoStart opens the room and signalling service; +- AutoSync reacts to an advertised and accepted peer with one finite transfer; +- AutoWatch follows later changes broadcast by a selected peer; +- AutoBroadcast publishes local database changes; +- explicit commands pull, push, or synchronise against a selected peer; +- incoming `reqSync` requests pull from an accepted peer; and +- diagnostics and platform events observe the transport. + +These behaviours must share one room owner, but they must not share one +implicit policy. A setting change may replace the room session while another +provider remains the selected main remote. A setup probe must not close that +session. A finite operation may need a room even when AutoStart is disabled. + +## Decision + +### One stable P2P service owns each room session + +The host composes one stable P2P service independently of the selected main +provider. It may be present as an adjunct beside CouchDB or Object Storage, or +its non-owning adapter may serve as the selected main P2P Replicator. The host +owns the service lifetime; the service owns all LiveSync-specific room +resources. + +A **P2P room session** is one active room membership and every resource whose +validity depends on that membership: + +- the Trystero room, RPC actions, and `RpcRoom`; +- its internal session epoch and cancellation signal; +- advertisement state and temporary peer decisions; +- peer-bound RPC clients and remote database proxies; +- connection, diagnostic, and platform-event subscriptions; and +- RPC publication, the watch set, database and broadcast change feeds, and + in-flight finite-transfer de-duplication bound to the local database. + +A **session epoch** is an internal fence for one room session. It is not a +public capability, a persisted profile identifier, or a synonym for the +logical room. Every callback, snapshot, and operation token carrying +session-bound state is checked against the current epoch. + +Closing or replacing a session fences its epoch, stops new operations, settles +or fails in-flight work, closes RPC and client resources, and leaves the room +in the transport-owned order. Persisted peer acceptance decisions survive; +temporary decisions, advertisements, clients, listeners, and feeds do not. +The underlying WebRTC peer remains under Trystero's shared-peer ownership as +specified by the accepted lifecycle record. + +### Expose seven narrow contract views + +The service does not expose a room session, raw host, or concrete Replicator to +ordinary consumers. It supplies these seven views over the same owner: + +1. `P2PTransportLifecycle` observes room state and accepts explicit, + user-owned connect or disconnect requests. +2. `P2PPeerDirectory` supplies peer snapshots and peer arrival or departure. +3. `P2PPeerAdmission` evaluates incoming peers and administers temporary or + persisted acceptance decisions. +4. `P2PTargetedTransfer` performs pull, requested push, and bidirectional + finite synchronisation against an explicit peer. +5. `P2PChangeRelay` administers peer watch and local-change broadcast. +6. `P2PConfigurationExchange` performs peer configuration exchange under its + declared interaction authority. +7. `P2PDiagnostics` supplies status and RTC diagnostics without exposing raw + room or peer connections. + +These are stable service-level contract views, not seven wrapper allocations or +independent state owners. One implementation may satisfy several views. +Advertisement and admission state remain under one peer-access owner, while +pull, push, and bidirectional transfer remain under one transfer owner. A +consumer which needs more than one view receives those views explicitly; it +does not receive a general P2P context or service locator. + +The views resolve the current published session at invocation. An operation +which carries an old epoch returns a stale or blocked result rather than +dispatching into a replacement. The epoch remains internal. The active P2P +Replicator is a non-owning adapter over the views, and its disposal cannot +implicitly leave the service-owned room. + +### Make explicit disconnect a veto, not another policy demand + +`P2PTransportLifecycle` distinguishes user intent from automation: + +- explicit connect resumes relay reconnection and establishes a user-owned + room demand; +- explicit disconnect is the sole force-close path, retires the current + session, invalidates all demands under the retirement contract, pauses relay + reconnection, and establishes a service-lifetime veto against AutoStart; and +- only a later explicit connect clears that veto. + +Automated policies and finite operations acquire or release only their own +demands. They cannot close a room held by another demand and cannot override an +explicit disconnect veto. This is a lifecycle veto, not the opposite of +`InteractionAuthority`: local interaction authority is the upper bound used by +operations, as specified in Part 1. An operation may impose a stricter veto, +but an unattended operation cannot gain permission to open a dialogue. + +### Separate automation policy from room ownership + +P2P automation is a composed service feature. It owns `P2P_AutoStart`, +AutoSync, AutoWatch, and AutoBroadcast policy, including delayed work and +automatic-trigger coalescing. It consumes the transport, peer, admission, +transfer, and change-relay views but does not own their mutable state. + +`P2PChangeRelay` owns the actual watch set and database changes feed. +`P2PTargetedTransfer` owns in-flight transfer de-duplication. Incoming-peer +consent is distinct from a local caller's authority to select a peer or open a +dialogue. Persisted or automatic acceptance may authorise an unattended path; +it cannot create local interaction authority implicitly. + +Every finite operation which needs a room obtains its own internal, +session-epoch-bound demand. Acquiring another demand does not open another +session. Releasing it never closes a session still required by AutoStart, +another finite operation, or another host consumer. Demand bookkeeping is +internal to the service and is not a general consumer contract; it cannot turn +a finite transfer into persistent transport policy. + +### Reconcile session settings atomically + +The effective P2P session binding is derived from the selected profile, all +settings which affect transport or session-bound automation, and the current +local database identity. It is not a new persisted profile identifier or a +device-local override. The service reconciles this binding independently of +the selected main provider, so an adjunct room can be replaced while CouchDB +or Object Storage remains active. + +A change to any binding input retires the whole room session and opens a +replacement when policy still requires one. Replacing a policy-only setting +may cause a temporary disconnect, but it preserves one atomic listener and +policy boundary: advertisements and temporary peer decisions are reacquired, +while persisted peer decisions survive. AutoStart reconnects when it remains +enabled and no explicit-disconnect veto is active. No old listener, credential, +client, or policy demand remains reachable after replacement. + +Reconciliation is serialised with room lifecycle operations: + +1. fence new session work; +2. allow already-started finite transfers to settle, because the current lower + level transfer may not yet have a real cancellation contract; +3. close and leave the old room in transport-owned order; +4. open and validate the candidate session; and +5. publish the replacement only after it has opened successfully. + +The service never exposes a partly initialised candidate or silently interrupts +a transfer without a cancellation contract. If bounded settlement or candidate +opening fails, reconciliation reports the failure and publishes no mixed old +and new session. A candidate-open failure leaves one observable disconnected +state with policy demands unsatisfied; it does not revive the fenced session or +start an unbounded retry loop. A later lifecycle trigger or explicit connect may +retry. + +### Order local database replacement across both owners + +The database lifecycle transition owns ordering above `ReplicatorService` and +the P2P service: + +1. fence acquisition of active Replicator and P2P room work; +2. retire the active adapter; +3. settle and retire P2P database-bound feeds and publication; +4. publish the replacement local database identity only after both boundaries + have settled; +5. rebind or replace the active provider; +6. reconcile P2P against the new database identity; and +7. let the Rebuilder request its separately authorised reopen. + +Each service serialises its own resources. The database transition owns the +cross-service ordering rather than introducing one transport-wide lock. + +### De-duplicate by logical lifecycle, not by room epoch + +Baseline AutoSync transfers are de-duplicated by peer and application lifecycle +generation. Trigger provenance and session epoch are not part of the key, so a +reconnect alone cannot repeat an in-flight or completed baseline transfer. + +In-flight de-duplication belongs to the current room session. Completed baseline +history belongs to the stable automation owner and survives transport-only or +policy-only session replacement. Only completed per-peer baselines are settled; +partial requests record completed peers only, while blocked, cancelled, failed, +and incomplete peers remain eligible for a later bounded retry. + +The automation owner clears settled records when the logical peer namespace or +local database identity changes. It retains them across transport-only and +policy-only replacement. Watch may follow a later advertised change, but does +not immediately repeat a completed baseline transfer. + +### Define the P2P trigger matrix + +| Trigger | Required preconditions | Effect | +| ----------------------- | --------------------------------------------------------------------- | -------------------------------------------- | +| `P2P_AutoStart` | P2P enabled, active lifecycle generation, and no user disconnect veto | Open room; do not itself transfer files | +| `P2P_AutoSyncPeers` | Open room, matching advertisement, and accepted peer policy | Run one bidirectional finite synchronisation | +| `P2P_AutoWatchPeers` | Open room, matching accepted advertisement, and remote broadcasting | Pull later announced updates | +| `P2P_AutoBroadcast` | Open room and local broadcasting enabled | Announce later local database changes | +| `P2P_SyncOnReplication` | Configured names and advertisements received within a bounded wait | Run target-aware unattended OneShot Sync | +| Explicit peer command | Supplied peer target and accepted connection | Run user-owned finite synchronisation | +| Incoming `reqSync` | Accepted peer and ordinary readiness | Pull from requesting peer | + +`P2P_AutoStart` is a transport policy, not central Continuous replication and +not `syncOnStart`. AutoSync, AutoWatch, and accepted incoming requests remain +unattended when their persisted policies permit them. A configured-target +request waits for advertisement for a bounded period; it does not inspect a +possibly stale snapshot immediately after opening. Missing, undiscovered, +unaccepted, or partly successful targets are explicit operation results. An +unknown peer never opens an acceptance dialogue on an unattended path. + +Delayed opens belong to the lifecycle generation which scheduled them. +Suspension cancels them or makes them harmless, and the callback rechecks +current settings and suspension state before opening. Ordinary automation uses +the trigger-aware readiness policy rather than bypassing readiness, pending-file +settlement, clean-up, or version gates. Fetch and Rebuild retain their +separately authorised bypasses. + +### Keep probes isolated from the active service + +Setup and settings use a separately owned `P2PConnectionProbe`. It does not +borrow or replace the current room session, and its `dispose()` cannot close the +active service. Because the current implementation may use process-global +relay sockets, a probe must either allocate isolated transport resources or the +host must reject concurrent probing while the active service uses those +sockets. It must not pause or close active relay sockets as a side effect of a +short-lived check. + +### Keep provider composition explicit + +The stable P2P service can be composed even when P2P is not the selected main +provider, while the generic provider table in Part 1 can include or omit the +P2P provider at compile time. No runtime unknown-provider registry is implied. +When P2P is selected as main, its active adapter delegates to this same service +and does not publish a second P2P lifecycle owner. + +## Consequences + +- P2P room, peer, watch, acceptance, transfer, configuration, and diagnostics + have one explicit owner and seven focused contracts. +- Disposing an active adapter cannot close a policy-owned or adjunct room. +- Explicit user disconnect has a clear veto boundary and cannot be undone by + AutoStart or a finite operation. +- Finite operations can request a room without changing persistent room policy. +- Settings and local database replacement cannot publish mixed-session state. +- Reconnects do not repeat completed baseline transfers merely because the room + epoch changed. +- Setup probes can validate P2P without mutating the active transport. + +## Non-goals + +- Do not make the P2P service a general service locator. +- Do not expose room, raw host, peer connection, or concrete Replicator state + to ordinary consumers. +- Do not make P2P own a central remote database. +- Do not reinterpret P2P AutoStart as `syncOnStart` or central Continuous + replication. +- Do not replace the accepted Trystero physical-peer and relay ownership + decisions. +- Do not add unbounded retry loops or pretend that a missing lower-level stop + operation is end-to-end cancellation. + +## References + +- [Part 1: core contract](2026_08_replicator_capabilities_01_core_contract.md) +- [Part 3: migration plan and verification](2026_08_replicator_capabilities_03_migration_plan.md) +- [P2P Room and Transport Lifecycle](2026_07_p2p_transport_lifecycle.md) +- [P2P Transport Compatibility Controls](2026_08_p2p_transport_compatibility.md) +- [Bounded Remote Activity](2026_07_bounded_remote_activity.md) diff --git a/docs/adr/2026_08_replicator_capabilities_03_migration_plan.md b/docs/adr/2026_08_replicator_capabilities_03_migration_plan.md new file mode 100644 index 00000000..8deec4f9 --- /dev/null +++ b/docs/adr/2026_08_replicator_capabilities_03_migration_plan.md @@ -0,0 +1,268 @@ +--- +date: 2026-08-27 +commonlib-version: "0.1.19" +self-hosted-livesync-version: "1.0.21" +status: proposed +series: replicator-capabilities-and-lifecycle +part: 3 of 3 +--- + +# Architectural Decision Record: Replicator Capabilities and Lifecycle Orchestration — Part 3: Migration Plan and Verification + +Series navigation: this is Part 3 of 3. Start with [Part 1: core contract](2026_08_replicator_capabilities_01_core_contract.md), +then [Part 2: P2P service and session lifecycle](2026_08_replicator_capabilities_02_p2p_service_lifecycle.md). +This part owns implementation sequencing and verification; it does not add +another runtime contract. + +## Status + +Proposed. The stages below are an implementation and verification order, not +independently releasable states. Commonlib and Self-hosted LiveSync must not +publish temporary support boundaries described by an incomplete stage. A +release follows only after the target matrix, ownership boundaries, and all +production-consumer migrations in Parts 1 and 2 are complete. + +## Migration rules + +Each Commonlib change first runs its focused unit and type-contract tests, +builds and validates the packed artefact, installs that exact artefact in +Self-hosted LiveSync, and runs focused downstream tests before either +repository advances. Compatibility methods remain until every production +consumer has migrated. + +The smallest vertical contract which fixes issue 1140 takes priority over +unrelated probe, Fast Fetch, maintenance, integrity, and facade work. A stage +must not claim completion when it only changes a type declaration while a +production caller still uses the old ownership or interaction path. + +## Stage 1: reproduce the current boundary failures + +Add the smallest regression which configures an Object Storage active provider, +enables `syncOnStart`, invokes the resume lifecycle after readiness, and +expects one unattended finite synchronisation. Run it against 1.0.21 and +confirm the expected failure: the CouchDB-owned lifecycle handler excludes +Object Storage. Cover both an ordinary Object Storage profile and a migrated +profile which retains `liveSync: true`; unsupported Continuous must not suppress +the supported `syncOnStart` OneShot policy. + +Add passing characterisation tests which inventory automatic P2P periodic, +database-save, editor-save, file-open, and merge triggers, together with P2P +AutoSync, AutoWatch, incoming-request, and nested finite-activity paths. These +tests record current ownership and dialogue behaviour so that later stages do +not accidentally remove a valid automatic path. + +## Stage 2: fix issue 1140 through the minimum vertical contract + +Add the fixed provider definitions and the minimum user-initiated, unattended, +and Continuous roles to Commonlib. Give `ReplicationService` typed entry points +which carry trigger and interaction policy. Immediately before changing +Object Storage handling, add a failing test in which a stopped or failed +journal transfer must not produce a completed outcome; then propagate the +actual journal result. + +Add the LiveSync-owned lifecycle coordinator, remove the resume handler from +`ModuleReplicatorCouchDB`, and route CouchDB Continuous and OneShot Sync plus +Object Storage `syncOnStart` through `ReplicationService`. Migrate every +automatic caller to the unattended entry point in this stage, so periodic and +event calls cannot fall back to an interactive P2P role. Migrate manual commands +to the user-initiated entry point. + +Replace the factory-registration-only responsibilities of +`ModuleReplicatorCouchDB` and `ModuleReplicatorMinIO` with composed provider +definitions. Retain a module only for separately identified stateful +behaviour; do not retain an instance merely to add a construction handler. + +At this boundary, existing P2P AutoSync, AutoWatch, and incoming-request +entry points receive the same non-interactive readiness and accepted-peer gate. +The no-interaction authority reaches counterpart RPC authorisation and +broadcast progress notifications. An unknown peer is blocked rather than +prompting. + +Until Stage 4 supplies target-aware unattended P2P, each host composition +declares generic `P2P_SyncOnReplication` as `not-implemented`. Its automatic +request settles without UI with an explicit blocked result. Existing AutoSync, +AutoWatch, and accepted incoming-request paths continue with the Stage 2 gate. +This is a temporary migration state, not the target matrix in Part 1. + +Apply and test the CLI scheduling precedence defined in Part 1, so the daemon +and lifecycle coordinator cannot schedule duplicate initial or recurring work. + +## Stage 3: make P2P transport ownership truthful + +Introduce the stable P2P service and its narrow contract views around the +existing implementation while preserving transfer semantics and changing the +necessary ownership and lifecycle behaviour. Make the active P2P Replicator a +non-owning adapter. Give the service exclusive ownership of room sessions, +session-epoch fencing, effective session binding, room open/close/replacement, +and settings reconciliation. Consolidate overlapping resume and +settings-event handlers. + +Migrate Obsidian panes and commands to lifecycle, peer, admission, transfer, +change-relay, configuration-exchange, and diagnostic views as required. Migrate +CLI and WebPeer away from concrete-class checks and raw host or room access. +Preserve RTC diagnostics through `P2PDiagnostics`, rather than retaining +`rawHost`. The compatibility facade may delegate during this stage, but new +consumers cannot receive it. + +Separate active-adapter release, room-session leave, and the stop request. Keep +the P2P stop role `not-implemented` until a lower-level operation exists. Add +internal session-demand ownership for finite operations and policy-held +AutoStart without exposing that bookkeeping as a general consumer API. + +Add ownership regressions immediately before implementation: + +- replacing or disposing the active main adapter does not close a policy-owned + or adjunct room; +- a disposed session fences late callbacks and clients; +- a P2P setting change replaces an adjunct room while another provider remains + the active main remote; +- persisted peer decisions survive replacement, while temporary decisions and + advertisements do not; +- local database replacement retires database-bound feeds and publication; +- repeated replacement does not retain platform-event subscriptions; +- policy-only change waits for an already-started finite transfer to settle; +- explicit disconnect suppresses AutoStart and relay reconnection until + explicit connect; and +- database replacement fences both active-provider and P2P work before + publishing the new database identity, while a failed candidate leaves one + observable disconnected state without reviving the fenced session. + +When this stage lands, add a supersession note to the accepted P2P lifecycle +record and update `devs.md` from the replaceable concrete Replicator getter to +the stable contract views. Preserve the accepted Trystero peer and relay +ownership rules rather than rewriting their historical verification. + +## Stage 4: add target-aware unattended P2P orchestration + +Before implementation, add failing regressions for the headless result-loss +path, delayed advertisement, an unaccepted peer, overlapping configured-target +and AutoSync requests, suspension before delayed open, and a finite-operation +demand beside policy-owned AutoStart demand. + +Implement target-aware unattended P2P work without UI. Add bounded +advertisement waiting, peer-acceptance outcomes, finite-operation demands beside +policy demands, lifecycle-generation cancellation for delayed opens, and +de-duplication across AutoSync, AutoWatch, and configured-target requests. +Route P2P automation through trigger-aware readiness without central-remote +Security Seed preflight. Add the missing finite-activity boundary to direct +shared-pane synchronisation. + +Keep the detailed wait, session-demand, de-duplication, and session-epoch state +machine in Part 2 rather than expanding the generic provider contract. If +implementation evidence requires a refinement, amend Part 2 before completing +this stage. After Stage 3 is complete, Part 2 supersedes the +replaceable-Replicator and current-result ownership portions of the accepted +July 2026 record; its Trystero peer and relay decisions remain unchanged. + +## Stage 5: separate active construction and flow-specific probes + +Make active creation a private, exhaustive `ReplicatorService` operation with +an explicit same-kind rebind-or-replace policy. Migrate every non-active caller +before restricting `getNewReplicator()`: CouchDB connection and passphrase +checks, Object Storage connection and preferred-tweak trials, isolated P2P +Setup, CLI commands, and other host compositions. Prove that every probe leaves +the active main Replicator and adjunct P2P transport unchanged. + +Migrate Streaming Fetch to the owned CouchDB initial-transfer dependencies +defined in Part 1. Add replacement-fence, late-settlement, +configuration-identity, cache-invalidation, and probe-disposal tests before +making active construction private. + +## Stage 6: migrate safety-sensitive and provider-specific consumers + +Migrate Setup, Rebuilder, tweak review, remote-size inspection, on-demand Chunk +retrieval, migration inspection, maintenance, and abort commands. Central +mutations settle truthfully before Rebuilder continuation. Compromised-Chunk +inspection uses `observed` or `unavailable`; an offline check is never zero. +Object Storage and P2P no longer supply dummy integrity counts, remote Chunk +results, or Security Seeds through the compatibility facade. + +Move local node identity initialisation out of the provider facade. Add +idempotent asynchronous disposal and settlement tests, same-kind profile +rebind-or-replace tests, and bounded activity around the host-owned garbage- +collection workflow's CouchDB OneShots. Each consumer migration removes the +corresponding generic `remoteType` or `instanceof` feature branch and adds a +focused test. + +Transport-specific settings may continue to display provider kind. Provider +identity is valid for labels and provider-specific configuration; it is not a +generic feature test. + +## Stage 7: retire the giant compatibility facade + +After every current consumer has migrated, stop requiring unsupported methods +on `LiveSyncAbstractReplicator`. Retain or remove compatibility exports at the +next Commonlib compatibility boundary. Do not retain dummy methods only because +the former abstract base declared them. + +## Verification + +### Commonlib unit and type-contract tests + +Cover: + +- exhaustive host-composed definitions for CouchDB, Object Storage, and P2P; +- complete support declarations and matching runtime roles; +- user-initiated, unattended, blocked, partial, cancelled, and failed OneShot + outcomes; +- truthful Object Storage stop or transfer failure and headless P2P outcomes; +- observed and unavailable compromised-Chunk results, including offline; +- active, retiring, disposed, and replacement-published states, including + rejection of new work during retirement and late settlement; +- same-kind rebind or replacement, configuration-identity invalidation, + idempotent asynchronous disposal, and settlement ordering; +- probes which cannot replace the active Replicator or P2P service and dispose + owned resources; +- the deliberately narrow active-transfer stop request, including work it + does not claim to cancel; +- central full-transfer and remote-administration boundaries; and +- provider-specific P2P, CouchDB, and journal facets. + +### Self-hosted LiveSync unit tests + +Cover: + +- Object Storage `syncOnStart` through resume, including a migrated profile + which retains `liveSync: true`; +- existing CouchDB Continuous and OneShot paths; +- periodic, database-save, editor-save, file-open, merge, and daemon triggers + remaining free of dialogues; +- manual P2P and configured peer-targeted flows remaining available; +- P2P AutoStart cancellation across suspension, bounded advertisement waiting, + accepted peers without unattended dialogues, remote-broadcast prerequisites, + session-demand reference counts, and overlapping peer policies; +- the seven P2P service views sharing one room owner without exposing a raw + host, room, or concrete Replicator; +- session replacement fencing callbacks, clients, temporary decisions, + advertisements, and database-bound feeds while retaining persisted decisions; +- counterpart RPC authorisation and broadcast progress preserving no-dialogue + authority; +- Setup and settings validation through probes; +- first-device and additional-device initialisation for each current provider; +- P2P as the main remote and as an adjunct transport; and +- CLI, WebApp, and WebPeer composition against the same contracts, including + daemon scheduling after settings restoration and P2P AutoStart reconciliation + after a settings change. + +### Focused integration and real-Obsidian verification + +Cover an Object Storage change arriving immediately after start-up without +waiting for the periodic interval, ordinary CouchDB start-up, and P2P start-up +without an unexpected selection dialogue or transport replacement. P2P +validation also covers an accepted configured peer advertising after room open, +a watched peer whose remote side broadcasts, and suspension before a delayed +AutoStart callback. + +No temporary stage is a release candidate. Release readiness requires the +target capability matrix, production-consumer migration, focused downstream +checks with the exact packed Commonlib artefact, and the real-runtime checks +appropriate to the changed boundary. + +## References + +- [Part 1: core contract](2026_08_replicator_capabilities_01_core_contract.md) +- [Part 2: P2P service and session lifecycle](2026_08_replicator_capabilities_02_p2p_service_lifecycle.md) +- [P2P Room and Transport Lifecycle](2026_07_p2p_transport_lifecycle.md) +- [P2P Transport Compatibility Controls](2026_08_p2p_transport_compatibility.md) +- [Bounded Remote Activity](2026_07_bounded_remote_activity.md) +- [Self-hosted LiveSync issue 1140](https://github.com/vrtmrz/obsidian-livesync/issues/1140)