public/obsidian-livesync
1
0
Fork 0
mirror of https://github.com/vrtmrz/obsidian-livesync.git synced 2026-08-26 05:17:06 +00:00
Code Issues Packages Projects Releases Wiki Activity

docs: model adaptive WebDAV trade-offs

Browse Source
This commit is contained in:
vorotamoroz
2026-08-01 04:16:37 +00:00
parent 0890a97222
commit 133ef34d7b
1 changed files with 154 additions and 10 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/adr/2026_07_adaptive_journal_webdav.md
+154 -10
View File
@@ -7,20 +7,24 @@ acceptance.
## Context
WebDAV exposes an object-shaped interface suitable for immutable packs, but method support and conditional semantics
vary across servers, gateways, reverse proxies, and authentication layers. A server can advertise WebDAV while failing
binary fidelity, replacing an object despite `If-None-Match: *`, returning incomplete listings, or ignoring Range.
WebDAV exposes an object-shaped interface suitable for immutable Commit Bundles and Packs, but method support and
conditional semantics vary across servers, gateways, reverse proxies, and authentication layers. A server can
advertise WebDAV while failing binary fidelity, replacing an object despite `If-None-Match: *`, returning incomplete
listings, or ignoring Range.
The common protocol decision is recorded in
[Adaptive Journal as an explicit protocol](2026_07_adaptive_journal_protocol.md). The pack format, flat object mapping,
and detailed safety-check sequence are specified in the
[Adaptive Journal Sync design](../design_docs/adaptive_journal_sync.md).
[Adaptive Journal as an explicit protocol](2026_07_adaptive_journal_protocol.md). The Pack background, flat object
mapping, and detailed safety-check sequence are specified in the
[Adaptive Journal Sync design](../design_docs/adaptive_journal_sync.md). Commonlib owns the implemented
`commit-bundle-v1` wire contract.
## Decision
Adaptive WebDAV uses the same immutable pack and catalogue semantics proven by S3, with a WebDAV-specific flat object
mapping inside the configured collection. Logical writer ordering comes from the host ID, writer epoch, and dense
sequence embedded in authenticated records. Correctness does not depend on a server-provided `startAfter` order.
Adaptive WebDAV uses the same immutable Commit Bundle and external Pack semantics proven by S3, with a WebDAV-specific
flat object mapping inside the configured collection. Its Catalogue is derived locally from authenticated routes in
received Bundles rather than stored as a separate mutable remote object. Logical writer ordering comes from the host
ID, writer epoch, and dense sequence embedded in authenticated records. Correctness does not depend on a
server-provided `startAfter` order.
Before a new Adaptive repository becomes writable, a non-destructive endpoint safety checker exercises random reserved
probe keys and reports observed semantic capabilities:
@@ -44,13 +48,153 @@ Successful mutations do not receive unconditional confirmation requests. An ambi
`verify-first`, and the exact immutable key is read before retrying. The existing opaque WebDAV layout and Adaptive
layout are detected separately; a mismatch requires a remote rebuild or another namespace.
## Working hypotheses and illustrative estimates
The following values are planning assumptions for the first WebDAV experiment. They are not benchmark results, a
provider-pricing statement, or a performance guarantee. The implementation must record the corresponding measurements
before this ADR presents either Journal format as generally faster or smaller.
### Cost and lifecycle assumptions
The initial deployment model is a self-hosted or quota-priced WebDAV service without a direct per-request tariff.
Request count still matters because every request consumes a round trip, server work, connection capacity, and a share
of any rate limit. Commercial services, gateways, and managed hosting may use a different charging model, so the
adapter must not infer cost policy from the WebDAV label.
Storage capacity is expected to be the more visible constraint. RFC 4331 defines optional
[`DAV:quota-used-bytes` and `DAV:quota-available-bytes`](https://www.rfc-editor.org/rfc/rfc4331.html) properties for
collections. When both are available, a connection diagnostic may report them as server-supplied evidence. Their
absence means 'unknown', not 'unlimited', and quota reporting is not an Adaptive safety requirement.
The first experiment has no ordinary remote Garbage Collection or repacking guarantee. Commit Bundles and external
Packs remain immutable, unreachable data may remain retained, and an interrupted publication may leave an unreferenced
Pack. A remote Rebuild is therefore the only guaranteed way to reclaim all experimental history. This makes retained
bytes and object count acceptance measurements rather than later operational details.
Adaptive and Opaque Journal consume the same local PouchDB Chunk split. A smaller average changed Chunk can reduce
retained history in either format. Adaptive adds authenticated routes and independently framed records, while Opaque
adds its own container, compression, and encryption overhead. No comparative storage saving is assumed before both
representations are measured with the same changes.
### Ordinary request model
The object-profile model is shared with S3. Let:
- `B` be the number of Metadata batches published as distinct Commit Bundles;
- `X` be the number of new external Pack objects needed by those batches;
- `J` be the number of Opaque Journal objects produced for the same changes;
- `W` be the number of visible Writer streams;
- `K` be the number of those Writer descriptors not already cached by the opened process;
- `P` be the number of additional complete Pack-container reads needed for missing Chunks; and
- `M` be the number of missing Chunk frames fetched with Range.
After the endpoint safety result and repository binding have been cached for the active configuration, ordinary
publication performs approximately `J` Opaque PUTs or `B + X` Adaptive PUTs. Every Adaptive batch creates one Bundle;
each external Pack adds one preceding PUT. A confirmed conditional create does not add a read-back. An ambiguous PUT
adds one exact-key GET before any retry and remains outside the successful-first-attempt estimate.
The generic object-store receive path performs approximately:
| Retrieval policy | Requests without WebDAV listing reuse |
| --- | ---: |
| `whole-pack` | `1 + W + K + B + P` |
| `range` | `1 + W + K + B + M` |
The `1 + W` term represents one Writer listing and one Commit listing per Writer. A flat WebDAV collection normally
cannot apply those logical prefixes on the server: each call may require another Depth-one `PROPFIND` over the same
collection. The WebDAV target is therefore one complete listing at the start of a receive phase, filtered locally for
every Writer and Commit prefix in that phase. This receive-phase listing must expire before the next receive phase; it
must never become an unbounded cross-synchronisation cache. A Commit which becomes visible after the listing response
is handled by the next receive phase, which is consistent with immutable eventual replication.
With that explicitly scoped reuse, the request estimates become `1 + K + B + P` and `1 + K + B + M`. For 100 new
inline-Pack Bundles which require no additional Pack read, the difference is illustrative rather than a benchmark:
| Visible Writers | Descriptor state | Repeated-prefix listing | One receive-phase listing |
| ---: | --- | ---: | ---: |
| 1 | first receive (`K = 1`) | 103 | 102 |
| 1 | warm process (`K = 0`) | 102 | 101 |
| 10 | first receive (`K = 10`) | 121 | 111 |
| 10 | warm process (`K = 0`) | 111 | 101 |
Opaque catch-up is approximately one listing plus one GET per new Journal object, or `1 + J`. Holding `B = J = 100`
would therefore give 101 operations. That comparison does not assume that the two batchers produce equal object counts
for a real editing history.
### Listing-volume hypothesis
Request count alone understates WebDAV discovery cost. Let `N` be the number of resources in the configured flat
collection and `L` the average XML response bytes per resource for a minimal Depth-one `PROPFIND`. Repeating the
listing for each logical prefix transfers roughly `(1 + W) * N * L`; one receive-phase listing transfers roughly
`N * L`.
Assuming `L = 1 KiB` only to make the scale visible gives:
| Collection resources (`N`) | One listing | Repeated listing, `W = 1` | Repeated listing, `W = 10` |
| ---: | ---: | ---: | ---: |
| 1,000 | about 1 MiB | about 2 MiB | about 11 MiB |
| 10,000 | about 10 MiB | about 20 MiB | about 107 MiB |
| 100,000 | about 98 MiB | about 195 MiB | about 1.05 GiB |
Actual XML size, selected properties, URL length, compression, server implementation, and collection scope can change
these values materially. The adapter should request only the properties it needs and measure both response bytes and
wall time. The table explains the reuse target; it is not an estimate of every WebDAV server.
Ignoring cleaned probe resources and abandoned Packs, the current Commit Bundle layout contains approximately
`1 + W + B + X` remote objects: one manifest, one descriptor per Writer, one object per Bundle, and one per external
Pack. Object count therefore grows with synchronisation batches even when the retained payload is small, and it feeds
back into subsequent `PROPFIND` cost.
### Retained-byte hypothesis
For a localised edit history, let `E` be the number of separately published changes and `D` the average bytes of newly
created encoded Chunk frames per change. Before Garbage Collection or Rebuild, retained Chunk bytes grow approximately
as `E * D`, plus Metadata, Commit, route, frame, server-metadata, and abandoned-publication overhead.
For a 10 MiB incompressible binary file whose content-defined split resumes after one changed Chunk:
| Published changes (`E`) | Average changed Chunk | Approximate new Chunk bytes retained | New Bundle objects when each Pack stays inline |
| ---: | ---: | ---: | ---: |
| 100 | 1 MiB | 100 MiB | 100 |
| 1,000 | 1 MiB | about 0.98 GiB | 1,000 |
| 100 | 256 KiB | 25 MiB | 100 |
| 1,000 | 256 KiB | 250 MiB | 1,000 |
These figures exclude Metadata and protocol overhead and do not predict text splitting. They show why a 10 MiB live
file can consume much more than 10 MiB remotely after many committed edits. They also show why reducing Chunk size is a
storage-versus-record-count decision shared with Opaque Journal, not an Adaptive-only recommendation.
### Pack-target hypothesis
Changing the external Pack target does not normally duplicate Chunk frames: a Pack is their concatenation without
capacity padding. Smaller targets chiefly increase object count, route data, PUTs, listing work, and later whole-Pack
GETs. Larger targets increase peak buffer size, retry cost, transfer duration, and exposure to mobile watchdog or proxy
timeouts.
For about 100 MiB of newly required encoded Chunk frames in one batch, the following serial-transfer model ignores RTT,
TLS setup, encryption, server processing, contention, and retries:
| Pack target | External Packs (`X`) | PUTs including the Bundle | One target-sized upload at 5 / 10 Mbit/s |
| ---: | ---: | ---: | ---: |
| 8 MiB | 13 | 14 | about 13.4 / 6.7 seconds |
| 16 MiB | 7 | 8 | about 26.8 / 13.4 seconds |
| 32 MiB | 4 | 5 | about 53.7 / 26.8 seconds |
| 64 MiB | 2 | 3 | about 107.4 / 53.7 seconds |
The total Chunk payload remains about 100 MiB in each row. The initial WebDAV experiment therefore retains the current
32 MiB preferred target as a compromise, not as a universal optimum or a user-facing recommendation. Measurements on
the disposable server must record peak memory, per-PUT duration, retry behaviour, total object count, and watchdog
survival. A later decision may lower the internal target for timeout-constrained endpoints without changing the wire
format.
## Staged acceptance
### Adapter and Commonlib integration
Unit tests own HTTP status classification, conditional-create verification, flat-name round trips, complete listing,
probe isolation, cleanup reporting, Range validation, and whole-pack fallback. A disposable WebDAV integration runs
the safety checker before exercising Adaptive Metadata and Chunk synchronisation.
the safety checker before exercising Adaptive Metadata and Chunk synchronisation. The integration also proves that one
receive phase reuses a single collection listing, then invalidates it so the next phase can observe a new Commit.
### CLI end-to-end acceptance