mirror of
https://github.com/vrtmrz/obsidian-livesync.git
synced 2026-08-30 23:37:08 +00:00
97 lines
5.3 KiB
Markdown
97 lines
5.3 KiB
Markdown
# Architectural Decision Record: Use S3 as the Reference Adaptive Object Store
|
|
|
|
## Status
|
|
|
|
Proposed as an improvement to the maintained S3-compatible Journal path. Adaptive Journal remains explicit and
|
|
opt-in; the existing opaque format remains the compatibility default.
|
|
|
|
## Context
|
|
|
|
S3-compatible Object Storage already supplies the maintained Journal object model and is the smallest provider on
|
|
which to prove Adaptive immutable packs. It has a standard conditional-create request, paginated listing, binary
|
|
objects, and optional byte-range retrieval, but compatible endpoints can still differ in consistency, proxy behaviour,
|
|
and Range handling.
|
|
|
|
The common protocol decision is recorded in
|
|
[Adaptive Journal as an explicit protocol](2026_07_adaptive_journal_protocol.md). The complete object-pack and catalogue
|
|
formats are specified in the [Adaptive Journal Sync design](../design_docs/adaptive_journal_sync.md).
|
|
|
|
## Decision
|
|
|
|
S3-compatible storage is the reference implementation of the Adaptive object-store strategy.
|
|
|
|
- It stores the manifest, writer control records, Metadata batches, commits, Chunk packs, indexes, and catalogue records
|
|
as immutable objects under the configured bucket namespace.
|
|
- Manifest and every immutable publication use conditional create. A successful create response is sufficient; the
|
|
client does not add a confirmation request solely for caution.
|
|
- A lost or ambiguous mutation response returns `verify-first`. The caller reads the exact key before retrying.
|
|
- A conditional-request conflict which does not establish an existing immutable value remains retryable and is not
|
|
reported as a successful create.
|
|
- Listing follows every continuation token and proves complete prefix enumeration. Folder marker objects and stale
|
|
capability-probe objects are not repository data.
|
|
- Format inspection distinguishes empty, `opaque-v1`, `adaptive-v1`, and mixed repositories. Mixed or mismatched data
|
|
fails closed, and reset remains an explicit batched operation.
|
|
|
|
The portable retrieval policy is `whole-pack`. A user may select `range` when the endpoint capability probe has
|
|
confirmed exact byte-range behaviour. Range responses must use `206`, carry a matching `Content-Range`, and return the
|
|
requested number of bytes. Loss of optional Range support falls back to whole-pack retrieval after reporting the
|
|
capability change; it does not make valid packs unreadable.
|
|
|
|
Capability results which prove support or lack of support may be cached for the endpoint identity. A transient probe
|
|
failure is not cached as a permanent result. Probe objects use a reserved random prefix and are removed when possible;
|
|
incomplete cleanup is reported and ignored by format detection.
|
|
|
|
## Staged acceptance
|
|
|
|
### Adapter and Commonlib integration
|
|
|
|
Focused tests cover conditional creation, ambiguous responses, binary fidelity, read-after-write visibility, paginated
|
|
listing, deletion visibility, Range validation, format detection, reset batching, and probe cleanup. A disposable
|
|
MinIO integration proves two-client Adaptive Metadata and Chunk synchronisation and both retrieval paths.
|
|
|
|
### CLI end-to-end acceptance
|
|
|
|
One real-MinIO scenario uses two independent CLI databases. Device A uses whole-pack retrieval. Device B receives an
|
|
Adaptive S3 Setup URI selecting Range retrieval. The scenario verifies that the URI preserves the format and policy,
|
|
that both devices can publish and receive more than one synchronisation round, and that text and binary content are
|
|
reconstructed from Chunks.
|
|
|
|
This test owns the built CLI, settings persistence, Setup URI decoding, and headless composition. It does not repeat
|
|
every S3 error classification already owned by Commonlib.
|
|
|
|
### Host settings and UI
|
|
|
|
The Object Storage dialogue exposes `opaque-v1` and `adaptive-v1`, with whole-pack as the Adaptive default and Range as
|
|
an explicit preference. It preserves the expected repository ID and read policy through saved profiles and Setup URI
|
|
handling. Focused host tests own normalisation, validation, persistence, and warnings; they do not contact S3.
|
|
|
|
### Real-host end-to-end acceptance
|
|
|
|
One real-Obsidian workflow applies an Adaptive S3 configuration and proves a representative Chunk-backed file transfer
|
|
against disposable MinIO. It verifies the Obsidian composition boundary only. The CLI suite remains the broader
|
|
headless synchronisation acceptance test.
|
|
|
|
## Alternatives rejected
|
|
|
|
### Re-upload a mutable pack when one Chunk changes
|
|
|
|
Packs are immutable publication units. A changed logical Chunk is added to a new pack, and a new catalogue delta makes
|
|
that location discoverable. Existing packs are replaced only by a separately protected compaction process.
|
|
|
|
### Require Range for Adaptive S3
|
|
|
|
Whole-pack reads often provide better throughput and work on more endpoints. Range is a deployment preference, not a
|
|
correctness requirement.
|
|
|
|
### Confirm every successful mutation with another request
|
|
|
|
This doubles request count in the ordinary success path. Exact-key verification is reserved for ambiguous outcomes and
|
|
explicit diagnostics.
|
|
|
|
## Consequences
|
|
|
|
- S3 establishes the object-store contract before the more variable WebDAV implementation.
|
|
- The two retrieval policies are exercised without duplicating the complete CLI scenario.
|
|
- Existing opaque S3 repositories remain readable and never become Adaptive implicitly.
|
|
- Endpoint capability evidence, rather than the S3-compatible label alone, controls optional behaviour.
|