obsidian-livesync/docs/adr/2026_07_adaptive_journal_webdav.md

Architectural Decision Record: Gate Adaptive WebDAV with an Endpoint Safety Check

Status

Proposed as an experimental provider after the S3 Adaptive path has completed adapter, CLI, host, and real-host acceptance.

Context

WebDAV exposes an object-shaped interface suitable for immutable Commit Bundles and Packs, but method support and conditional semantics vary across servers, gateways, reverse proxies, and authentication layers. A server can advertise WebDAV while failing binary fidelity, replacing an object despite If-None-Match: *, returning incomplete listings, or ignoring Range.

The common protocol decision is recorded in Adaptive Journal as an explicit protocol. The Pack background, flat object mapping, and detailed safety-check sequence are specified in the Adaptive Journal Sync design. Commonlib owns the implemented commit-bundle-v1 wire contract.

Decision

Adaptive WebDAV uses the same immutable Commit Bundle and external Pack semantics proven by S3, with a WebDAV-specific flat object mapping inside the configured collection. Its Catalogue is derived locally from authenticated routes in received Bundles rather than stored as a separate mutable remote object. Logical writer ordering comes from the host ID, writer epoch, and dense sequence embedded in authenticated records. Correctness does not depend on a server-provided startAfter order.

Before a new Adaptive repository becomes writable, a non-destructive endpoint safety checker exercises random reserved probe keys and reports observed semantic capabilities:

• binary write and exact read-back;
• read-after-write visibility;
• complete collection listing for the probe keys;
• If-None-Match: * preventing replacement;
• delete visibility; and
• exact byte-range behaviour, reported separately as optional.

The checker is an implementation acceptance target for compatible servers, not a claim that every WebDAV server must support Adaptive Journal. It touches no repository objects, attempts to remove every probe, reports incomplete cleanup, and never interprets authentication, permission, timeout, malformed response, or server failure as absence.

Conditional create, binary fidelity, complete listing, read-after-write visibility, and delete visibility are required. Range remains optional. The user selects whole-pack or Range retrieval based on their endpoint and own latency and throughput preference; the checker does not benchmark or recommend a policy.

Successful mutations do not receive unconditional confirmation requests. An ambiguous response is classified as verify-first, and the exact immutable key is read before retrying. The existing opaque WebDAV layout and Adaptive layout are detected separately; a mismatch requires a remote rebuild or another namespace.

Working hypotheses and illustrative estimates

The following values are planning assumptions for the first WebDAV experiment. They are not benchmark results, a provider-pricing statement, or a performance guarantee. The implementation must record the corresponding measurements before this ADR presents either Journal format as generally faster or smaller.

Cost and lifecycle assumptions

The initial deployment model is a self-hosted or quota-priced WebDAV service without a direct per-request tariff. Request count still matters because every request consumes a round trip, server work, connection capacity, and a share of any rate limit. Commercial services, gateways, and managed hosting may use a different charging model, so the adapter must not infer cost policy from the WebDAV label.

Storage capacity is expected to be the more visible constraint. RFC 4331 defines optional DAV:quota-used-bytes and DAV:quota-available-bytes properties for collections. When both are available, a connection diagnostic may report them as server-supplied evidence. Their absence means 'unknown', not 'unlimited', and quota reporting is not an Adaptive safety requirement.

The first experiment has no ordinary remote Garbage Collection or repacking guarantee. Commit Bundles and external Packs remain immutable, unreachable data may remain retained, and an interrupted publication may leave an unreferenced Pack. A remote Rebuild is therefore the only guaranteed way to reclaim all experimental history. This makes retained bytes and object count acceptance measurements rather than later operational details.

Adaptive and Opaque Journal consume the same local PouchDB Chunk split. A smaller average changed Chunk can reduce retained history in either format. Adaptive adds authenticated routes and independently framed records, while Opaque adds its own container, compression, and encryption overhead. No comparative storage saving is assumed before both representations are measured with the same changes.

Ordinary request model

The object-profile model is shared with S3. Let:

• B be the number of Metadata batches published as distinct Commit Bundles;
• X be the number of new external Pack objects needed by those batches;
• J be the number of Opaque Journal objects produced for the same changes;
• W be the number of visible Writer streams;
• K be the number of those Writer descriptors not already cached by the opened process;
• P be the number of additional complete Pack-container reads needed for missing Chunks; and
• M be the number of missing Chunk frames fetched with Range.

After the endpoint safety result and repository binding have been cached for the active configuration, ordinary publication performs approximately J Opaque PUTs or B + X Adaptive PUTs. Every Adaptive batch creates one Bundle; each external Pack adds one preceding PUT. A confirmed conditional create does not add a read-back. An ambiguous PUT adds one exact-key GET before any retry and remains outside the successful-first-attempt estimate.

The generic object-store receive path performs approximately:

Retrieval policy	Requests without WebDAV listing reuse
whole-pack	1 + W + K + B + P
range	1 + W + K + B + M

The 1 + W term represents one Writer listing and one Commit listing per Writer. A flat WebDAV collection normally cannot apply those logical prefixes on the server: each call may require another Depth-one PROPFIND over the same collection. The WebDAV target is therefore one complete listing at the start of a receive phase, filtered locally for every Writer and Commit prefix in that phase. This receive-phase listing must expire before the next receive phase; it must never become an unbounded cross-synchronisation cache. A Commit which becomes visible after the listing response is handled by the next receive phase, which is consistent with immutable eventual replication.

With that explicitly scoped reuse, the request estimates become 1 + K + B + P and 1 + K + B + M. For 100 new inline-Pack Bundles which require no additional Pack read, the difference is illustrative rather than a benchmark:

Visible Writers	Descriptor state	Repeated-prefix listing	One receive-phase listing
1	first receive (K = 1)	103	102
1	warm process (K = 0)	102	101
10	first receive (K = 10)	121	111
10	warm process (K = 0)	111	101

Opaque catch-up is approximately one listing plus one GET per new Journal object, or 1 + J. Holding B = J = 100 would therefore give 101 operations. That comparison does not assume that the two batchers produce equal object counts for a real editing history.

Listing-volume hypothesis

Request count alone understates WebDAV discovery cost. Let N be the number of resources in the configured flat collection and L the average XML response bytes per resource for a minimal Depth-one PROPFIND. Repeating the listing for each logical prefix transfers roughly (1 + W) * N * L; one receive-phase listing transfers roughly N * L.

Assuming L = 1 KiB only to make the scale visible gives:

Collection resources (N)	One listing	Repeated listing, W = 1	Repeated listing, W = 10
1,000	about 1 MiB	about 2 MiB	about 11 MiB
10,000	about 10 MiB	about 20 MiB	about 107 MiB
100,000	about 98 MiB	about 195 MiB	about 1.05 GiB

Actual XML size, selected properties, URL length, compression, server implementation, and collection scope can change these values materially. The adapter should request only the properties it needs and measure both response bytes and wall time. The table explains the reuse target; it is not an estimate of every WebDAV server.

Ignoring cleaned probe resources and abandoned Packs, the current Commit Bundle layout contains approximately 1 + W + B + X remote objects: one manifest, one descriptor per Writer, one object per Bundle, and one per external Pack. Object count therefore grows with synchronisation batches even when the retained payload is small, and it feeds back into subsequent PROPFIND cost.

Retained-byte hypothesis

For a localised edit history, let E be the number of separately published changes and D the average bytes of newly created encoded Chunk frames per change. Before Garbage Collection or Rebuild, retained Chunk bytes grow approximately as E * D, plus Metadata, Commit, route, frame, server-metadata, and abandoned-publication overhead.

For a 10 MiB incompressible binary file whose content-defined split resumes after one changed Chunk:

Published changes (E)	Average changed Chunk	Approximate new Chunk bytes retained	New Bundle objects when each Pack stays inline
100	1 MiB	100 MiB	100
1,000	1 MiB	about 0.98 GiB	1,000
100	256 KiB	25 MiB	100
1,000	256 KiB	250 MiB	1,000

These figures exclude Metadata and protocol overhead and do not predict text splitting. They show why a 10 MiB live file can consume much more than 10 MiB remotely after many committed edits. They also show why reducing Chunk size is a storage-versus-record-count decision shared with Opaque Journal, not an Adaptive-only recommendation.

Pack-target hypothesis

Changing the external Pack target does not normally duplicate Chunk frames: a Pack is their concatenation without capacity padding. Smaller targets chiefly increase object count, route data, PUTs, listing work, and later whole-Pack GETs. Larger targets increase peak buffer size, retry cost, transfer duration, and exposure to mobile watchdog or proxy timeouts.

For about 100 MiB of newly required encoded Chunk frames in one batch, the following serial-transfer model ignores RTT, TLS setup, encryption, server processing, contention, and retries:

Pack target	External Packs (X)	PUTs including the Bundle	One target-sized upload at 5 / 10 Mbit/s
8 MiB	13	14	about 13.4 / 6.7 seconds
16 MiB	7	8	about 26.8 / 13.4 seconds
32 MiB	4	5	about 53.7 / 26.8 seconds
64 MiB	2	3	about 107.4 / 53.7 seconds

The total Chunk payload remains about 100 MiB in each row. The initial WebDAV experiment therefore retains the current 32 MiB preferred target as a compromise, not as a universal optimum or a user-facing recommendation. Measurements on the disposable server must record peak memory, per-PUT duration, retry behaviour, total object count, and watchdog survival. A later decision may lower the internal target for timeout-constrained endpoints without changing the wire format.

Staged acceptance

Adapter and Commonlib integration

Unit tests own HTTP status classification, conditional-create verification, flat-name round trips, complete listing, probe isolation, cleanup reporting, Range validation, and whole-pack fallback. A disposable WebDAV integration runs the safety checker before exercising Adaptive Metadata and Chunk synchronisation. The integration also proves that one receive phase reuses a single collection listing, then invalidates it so the next phase can observe a new Commit.

CLI end-to-end acceptance

The built CLI applies an Adaptive WebDAV Setup URI to a second independent database and synchronises text and binary Chunk-backed files through a real disposable server. One client uses whole-pack retrieval; Range is added to this layer only when the selected test server proves it. The CLI test does not repeat the complete endpoint capability matrix.

Host settings and UI

The WebDAV dialogue exposes Adaptive mode only with clear capability-check results. It persists the expected repository ID and the selected retrieval policy, defaults to whole-pack, and reports that Range is optional. Focused tests own profile and Setup URI preservation without contacting a server.

Real-host end-to-end acceptance

One real-Obsidian workflow uses the same known disposable WebDAV implementation, runs the safety gate, and proves a representative Chunk-backed transfer. Servers outside that fixture are diagnosed by the checker rather than added to a large real-host matrix.

Alternatives rejected

Trust advertised WebDAV methods

Method advertisement does not prove the conditional, listing, visibility, and byte semantics required for immutable publication.

Treat Range as mandatory

Whole-pack retrieval is correct and often competitive for throughput. Requiring Range would exclude otherwise safe servers for an optional optimisation.

Add server-specific compatibility branches

The remote may be any implementation or proxy composition. Semantic checks produce a maintainable contract, while a growing server-name table would remain incomplete and become stale.

Consequences

• WebDAV remains experimental because suitability is endpoint-specific.
• The safety checker gives a concrete reason when a server cannot host Adaptive Journal.
• Common object-pack behaviour is inherited from the earlier S3 boundary, keeping WebDAV-specific tests focused on HTTP semantics and name mapping.
• Optional Range support can improve request efficiency without becoming a data-availability requirement.