mirror of
https://github.com/vrtmrz/obsidian-livesync.git
synced 2026-10-30 21:08:18 +00:00
Merge main and retain Fetch recovery notes
This commit is contained in:
commit
8422c19881
47 files changed
+2695
-1303
No files matched your search
@@ -0,0 +1,94 @@
|
||||
# Documentation index
|
||||
|
||||
Choose a document by the task and its status. User guides describe supported operations; implementation references describe contracts and their limits; ADRs record why a decision was made. Proposals and historical designs are grouped separately below.
|
||||
|
||||
A document's original date and reviewed-version fields describe its baseline. Read its implementation or publication note before treating that baseline as the current release. A published feature does not by itself establish that every proposal or validation step in its design is complete.
|
||||
|
||||
## Setup and everyday use
|
||||
|
||||
| Task | Start here |
|
||||
|---|---|
|
||||
| Get started | [What you need](../README.md#what-you-need), then [Start synchronising](../README.md#start-synchronising) |
|
||||
| Choose a CouchDB setup | [Own server](setup_own_server.md), [Docker stack](../docker/README.md), [Fly.io](setup_flyio.md), or [IBM Cloudant (managed)](setup_cloudant.md) |
|
||||
| Configure CouchDB clients | [Quick setup](quick_setup.md) |
|
||||
| Find hosting services proposed by the community | [Third-party hosting listings](../README.md#hosting-services-proposed-by-the-community) |
|
||||
| Set up S3-compatible storage | [Object Storage Setup](setup_object_storage.md) |
|
||||
| Set up peer-to-peer synchronisation | [P2P communication model](p2p.md), then [Peer-to-Peer Setup](setup_p2p.md) |
|
||||
| Diagnose P2P connectivity | [Browser-assisted check](../src/apps/webpeer/README.md#p2p-connection-check), [network tips](tips/p2p-sync-tips.md), and [self-hosted TURN](../docker/coturn/README.md) |
|
||||
| Understand a setting or command | [Settings reference](settings.md) |
|
||||
| Read or customise synchronisation status | [Status indicators and CSS snippets](tips/status-display.md) |
|
||||
| Investigate or recover from a problem | [Troubleshooting](troubleshooting.md), then the relevant [recovery procedure](recovery.md) |
|
||||
| Prepare an issue report | [Issue reporting](to_issue_reporting.md) |
|
||||
|
||||
The project overview links to Japanese and Chinese documentation.
|
||||
|
||||
## Implementation and technical references
|
||||
|
||||
Use these references for the area being changed, together with its relevant ADR. Check the document's stated scope and validation limits before relying on a guarantee.
|
||||
|
||||
| Area | References |
|
||||
|---|---|
|
||||
| Overall synchronisation and terminology | [Technical overview](tech_info.md), [glossary](glossary.md), and [terminology conventions](terms.md) |
|
||||
| Metadata, Chunks, and database representations | [Data structures](datastructure.md) |
|
||||
| Conflict resolution, provenance, and history restoration | [Conflict and provenance contract](specs_conflict_resolution.md), [Document History restoration](design_docs/document_history_revision_restoration.md), and [local save completion](adr/2026_10_local_file_save_completion_and_batching.md) |
|
||||
| Replicator and host boundaries | [Replicator architecture snapshot](design_docs/replicator_architecture.md) and [service feature and legacy Module boundaries](design_docs/service_feature_and_legacy_module_boundaries.md) |
|
||||
| Chunk retrieval and maintenance | [Chunk retrieval and waiting](design_docs/chunk_retrieval_and_waiting.md), [Garbage Collection V3](specs_garbage_collection.md), and [Data Compression](specs_data_compression.md) |
|
||||
| Document identity and validation | [Unicode path identity](design_docs/unicode_path_identity.md), [configurable ID derivation](design_docs/configurable_id_derivation.md), [Metadata ID validation and repair](design_docs/metadata_document_id_validation_and_repair.md), and [path component compatibility](design_docs/path_component_length_compatibility.md) |
|
||||
| Settings compatibility and profiles | [Tweak compatibility and recovery](design_docs/tweak_compatibility.md) and [remote configuration management](design_docs_of_remote_configurations.md) |
|
||||
| Encryption and sharing | [Metadata encryption](design_docs/internal_metadata_encryption.md) and [time-bound Setup URIs](design_docs/time_bound_setup_uri.md) |
|
||||
| P2P connection preparation | [TURN connection settings](design_docs/renewable_turn_credentials.md) |
|
||||
| Journal storage internals | [Journal Replicator 2nd Edition](design_docs_of_journalsync_2nd.md) |
|
||||
|
||||
The installed `@vrtmrz/livesync-commonlib` package is authoritative for shared runtime contracts. Use the exact dependency selected by [package.json](../package.json) and its lockfile when checking behaviour. The [Commonlib documentation](https://github.com/vrtmrz/livesync-commonlib/tree/main/docs) is maintained in that repository; this index does not replace its package contracts.
|
||||
|
||||
## Architectural decisions
|
||||
|
||||
Read the decisions relevant to the change, rather than treating the whole ADR directory as a prerequisite. A decision's acceptance, its implementation, its release, and its remaining verification work are separate facts.
|
||||
|
||||
| Decision area | Records |
|
||||
|---|---|
|
||||
| Shared package boundary | [Common library package boundary](adr/2026_07_common_library_package_boundary.md) |
|
||||
| Replicator capabilities and migration | [Core contract](adr/2026_08_replicator_capabilities_01_core_contract.md), [P2P service lifecycle](adr/2026_08_replicator_capabilities_02_p2p_service_lifecycle.md), and [migration plan](adr/2026_08_replicator_capabilities_03_migration_plan.md) |
|
||||
| Remote resources and Fast Setup | [CouchDB connection ownership](adr/2026_08_couchdb_remote_connection_ownership.md), [Fast Fetch eligibility](adr/2026_08_fast_fetch_transport_eligibility.md), and [persistence and completion](adr/2026_08_fast_fetch_persistence_and_completion.md) |
|
||||
| P2P lifecycle and setup | [Transport lifecycle](adr/2026_07_p2p_transport_lifecycle.md), [connection check](adr/2026_07_p2p_connection_check.md), [profile-aware onboarding](adr/2026_07_multiple_remote_onboarding.md), and [compatibility controls](adr/2026_08_p2p_transport_compatibility.md) |
|
||||
| Settings interface integration | [Declarative settings adapter](adr/2026_08_declarative_settings_adapter.md) |
|
||||
| Saving and path identity | [Local file save completion and batching](adr/2026_10_local_file_save_completion_and_batching.md) and [Unicode path identity](adr/2026_10_unicode_path_identity.md) |
|
||||
| Activity and Chunk arrival | [Bounded remote activity](adr/2026_07_bounded_remote_activity.md) and [Chunk arrival quiescence](adr/2026_07_chunk_arrival_quiescence.md) |
|
||||
| Compatibility and validation | [Release notes and database compatibility gates](adr/2026_07_release_notes_and_database_compatibility.md) and [real Obsidian E2E](adr/2026_06_real_obsidian_e2e.md) |
|
||||
|
||||
## Proposals and incomplete explanations
|
||||
|
||||
These records are useful for design work. Their proposed representation or behaviour is not a current runtime contract.
|
||||
|
||||
| Document | Recorded state |
|
||||
|---|---|
|
||||
| [Tiered Chunk Storage with Live Compaction](design_docs/tired_chunk_pack.md) | In design |
|
||||
| [Tiered Chunk Storage for Bucket Sync](design_docs/tired_chunk_pack_bucket.md) | In design |
|
||||
| [Design intent for Metadata and Chunks](design_docs/intention_of_chunks.md) | Incomplete explanation; use the data-structure reference for current representations |
|
||||
| [Feature maturity for 1.0](adr/2026_07_feature_maturity_for_1_0.md) | Proposed; publication of 1.0 alone does not establish formal acceptance of this record |
|
||||
|
||||
## Historical and retired designs
|
||||
|
||||
These documents preserve earlier motivations, approaches, or results. Follow their maintained replacement for current operations.
|
||||
|
||||
| Historical material | Current starting point |
|
||||
|---|---|
|
||||
| [Original Tweak-sharing proposal](design_docs_of_sharing_tweak_value.md) | [Tweak compatibility and recovery](design_docs/tweak_compatibility.md) |
|
||||
| [First Journal Sync design](design_docs_of_journalsync.md) | [Journal Replicator 2nd Edition](design_docs_of_journalsync_2nd.md) |
|
||||
| [Eden design and operational results](design_docs_of_keep_newborn_chunks.md#results-from-actual-operation) | The document records its deprecated, frozen status; use [Data structures](datastructure.md) for representations still accepted by current code |
|
||||
| [Withdrawn Chunk Aggregation by Prefix](design_docs/chunk_aggregation_by_prefix.md) | Retained as a withdrawn proposal, not an implemented storage layout |
|
||||
|
||||
Dated measurements, including the [Data Compression benchmark](specs_data_compression.md#reproducible-benchmark), remain evidence for their recorded revision and environment. A current rerun is a separate measurement.
|
||||
|
||||
## Development, validation, and release history
|
||||
|
||||
- [Development guide](../devs.md): build, module ownership, and release workflow.
|
||||
- [Contributing](../CONTRIBUTING.md): contribution and documentation checks.
|
||||
- Independent applications: [CLI](../src/apps/cli/README.md), [WebApp](../src/apps/webapp/README.md), and [WebPeer](../src/apps/webpeer/README.md).
|
||||
- [Setup utilities](../utils/readme.md): provisioning and Setup URI generation.
|
||||
- [Review Harness](review_harness.md): bounded real-device observations and performance measurements.
|
||||
- [Test procedures](../test/README.md): choose the suite which owns the boundary being changed.
|
||||
- [Change notes](../updates.md): current release and unreleased changes.
|
||||
- Published release archives: [1.0](releases/1.0.md), [1.0 previews](releases/1.0-previews.md), [0.25](releases/0.25.md), and [earlier releases](releases/legacy.md).
|
||||
|
||||
When a feature ships or a design is replaced, update its implementation note and this index if its role changes. Preserve the original decision and measurement history, state the maintained replacement, and keep existing links working.
|
||||
@@ -2,7 +2,10 @@
|
||||
|
||||
## Status
|
||||
|
||||
Accepted — the user-facing controls will be introduced in stages. This record defines their boundaries before Commonlib settings and LiveSync interfaces are changed.
|
||||
Accepted. The P2P message-size presets and connection-path choice are present
|
||||
in current Self-hosted LiveSync 1.0.42 with Commonlib 0.2.2. The proposed
|
||||
`Test TURN server` action and concise route diagnostics in the Obsidian
|
||||
interface remain future work.
|
||||
|
||||
## Context
|
||||
|
||||
@@ -11,10 +14,12 @@ WebRTC connectivity depends on both devices, their browsers or embedded WebViews
|
||||
The current P2P transport has several relevant properties:
|
||||
|
||||
- Trystero supplies the Nostr signalling strategy and the browser-owned WebRTC connection.
|
||||
- Commonlib limits one RPC wire payload to 15,360 bytes so it remains below Trystero's own action-chunk boundary.
|
||||
- Trystero supplies ordinary STUN servers and accepts an optional TURN server list with one username and credential.
|
||||
- ICE chooses a direct, server-reflexive, or TURN-relayed path automatically.
|
||||
- Commonlib can collect raw WebRTC statistics, but LiveSync does not yet present the selected candidate route in a concise diagnostic result.
|
||||
- Commonlib defaults to a 15,360-byte RPC wire payload limit, below Trystero's own action-chunk boundary; the presets below can lower it.
|
||||
- Manual TURN settings use one URL list, username, and credential. Managed preparation can supply multiple ICE server entries with their own credentials.
|
||||
- ICE chooses a direct, server-reflexive, or TURN-relayed path automatically unless the profile selects `TURN relay only`.
|
||||
- Commonlib can collect raw WebRTC statistics. The CLI reports the selected
|
||||
candidate route, but the Obsidian interface does not yet present it in a
|
||||
concise diagnostic result.
|
||||
|
||||
Issue reports suggest that reducing the application payload may improve some mobile and constrained-network paths. A VPN such as Tailscale may also turn an unreliable route into a reliable one. These observations are consistent with NAT, path-MTU, fragmentation, or intermediary behaviour, but they do not prove one universal cause. Browser WebRTC implementations retain responsibility for SCTP, DTLS, ICE, packetisation, congestion control, and retransmission.
|
||||
|
||||
@@ -24,7 +29,7 @@ One low-level number cannot represent all of these concerns. Users need a small
|
||||
|
||||
### Message-size presets
|
||||
|
||||
LiveSync will expose a `P2P message size` choice with four presets:
|
||||
LiveSync exposes a `P2P message size` choice with four presets:
|
||||
|
||||
| Label | Maximum RPC wire payload | Intended use |
|
||||
| ----------------------- | -----------------------: | ------------------------------------------------------------------------------- |
|
||||
@@ -37,11 +42,11 @@ This value limits Commonlib RPC wire payloads before Trystero applies its own fr
|
||||
|
||||
The bound applies to outgoing messages. A device which only lowers its own value still receives messages produced under the sender's value. The selected preset therefore belongs to the P2P profile and is included in an encrypted Setup URI for additional devices. A device which was configured earlier must be changed separately; the interface and troubleshooting guidance must state that the same conservative preset should be selected on every participating device. An absent key preserves the current 15,360-byte default.
|
||||
|
||||
Automatic negotiation or fallback between presets is deferred. A failed ordered data channel may require connection replacement before a smaller retry can prove anything, and changing transport parameters during a replication session would broaden the lifecycle contract considerably. The first implementation remains explicit, stable for one room lifetime, and inspectable.
|
||||
Automatic negotiation or fallback between presets is deferred. A failed ordered data channel may require connection replacement before a smaller retry can prove anything, and changing transport parameters during a replication session would broaden the lifecycle contract considerably. The implementation remains explicit, stable for one room lifetime, and inspectable.
|
||||
|
||||
### Connection path
|
||||
|
||||
LiveSync will expose a separate `Connection path` choice:
|
||||
LiveSync exposes a separate `Connection path` choice:
|
||||
|
||||
- `Automatic` retains normal ICE selection and is the default.
|
||||
- `TURN relay only` supplies `iceTransportPolicy: 'relay'` and prevents direct or server-reflexive candidates from being selected.
|
||||
@@ -81,11 +86,15 @@ Where the browser exposes the evidence, the result should report:
|
||||
- UDP, TCP, or TLS transport; and
|
||||
- a bounded failure or inconclusive result.
|
||||
|
||||
Ordinary P2P diagnostics should later summarise the selected candidate pair as direct, server-reflexive, or relayed, with its transport. Raw `getStats()` output remains supporting evidence rather than the primary interface.
|
||||
The proposed Obsidian P2P diagnostics should summarise the selected candidate
|
||||
pair as direct, server-reflexive, or relayed, with its transport. The CLI's
|
||||
candidate-route output does not provide this user-facing summary. Raw
|
||||
`getStats()` output remains supporting evidence rather than the primary
|
||||
interface.
|
||||
|
||||
### Placement and defaults
|
||||
|
||||
These controls belong inside `P2P Configuration` under a `Connection compatibility` section. They do not require the repository-wide Advanced, Power User, or Edge Case modes. P2P itself remains a supported opt-in feature.
|
||||
These controls appear inside `P2P Configuration` under a `Connection compatibility` section. They do not require the repository-wide Advanced, Power User, or Edge Case modes. P2P itself remains a supported opt-in feature.
|
||||
|
||||
Existing profiles retain the following defaults:
|
||||
|
||||
@@ -161,7 +170,8 @@ A layer-4 TLS router could share one public address between distinct CouchDB and
|
||||
|
||||
## Verification
|
||||
|
||||
The first implementation stage must add focused tests before production changes:
|
||||
The following list records the original focused-test plan for the message-size
|
||||
and connection-path controls. It remains useful as regression coverage:
|
||||
|
||||
- settings-schema defaults for absent keys;
|
||||
- P2P connection-string and Setup URI round trips which retain both transport compatibility settings;
|
||||
|
||||
@@ -2,15 +2,17 @@
|
||||
date: 2026-10-24
|
||||
commonlib-version: "0.2.0"
|
||||
self-hosted-livesync-version: "1.0.40"
|
||||
status: accepted-unreleased
|
||||
status: accepted
|
||||
---
|
||||
|
||||
# Architectural Decision Record: Local File Save Completion and Batching
|
||||
|
||||
## Status
|
||||
|
||||
Accepted on 2026-10-25. The package versions above identify the investigated
|
||||
baseline, rather than a released implementation of this decision.
|
||||
Accepted on 2026-10-25. The recovery contracts and LiveSync mode batching in
|
||||
implementation-order steps 1–3 are present in current Self-hosted LiveSync
|
||||
1.0.42 with Commonlib 0.2.2. The version fields above retain the investigated
|
||||
baseline. Further same-path coalescing in step 4 remains a follow-up.
|
||||
|
||||
## Context
|
||||
|
||||
@@ -349,6 +351,9 @@ eliminating genuine concurrent-edit conflicts or as a confirmed fix for #1207.
|
||||
|
||||
## Ownership and implementation order
|
||||
|
||||
The list below records the implementation sequence. Steps 1–3 are published
|
||||
in the current version line; the further coalescing in step 4 remains planned.
|
||||
|
||||
1. **Commonlib recovery:** event ownership, ordered snapshot persistence,
|
||||
successful completion and failure propagation, finite flush boundaries, and
|
||||
same-time content reconciliation. Add focused failing regressions before
|
||||
@@ -373,6 +378,11 @@ automation transaction API require a further design decision.
|
||||
|
||||
## Verification required
|
||||
|
||||
The matrix below records the acceptance criteria for implementation steps 1–3
|
||||
and remains useful for regression work. Publication in the current version line
|
||||
does not extend the historical test results above to untested platforms or
|
||||
automation workflows.
|
||||
|
||||
Use controlled promises at the manager and persistence boundaries for focused
|
||||
regressions, and real PouchDB tests for content and ancestry. The acceptance
|
||||
matrix is:
|
||||
@@ -395,13 +405,13 @@ matrix is:
|
||||
| Old settings, mode changes, explicit opt-in, and settings import | Existing LiveSync mode users retain immediate saving until they choose batching. |
|
||||
| Incoming identical content, an edit, a deletion, or delete/recreate while waiting | Preserve genuine branches without extra conflicts or loss of local work. |
|
||||
|
||||
Keep the existing E2E comparison between batching disabled, an ignored legacy
|
||||
setting, and explicitly enabled batching. Repeat both directions, manual
|
||||
synchronisation, saved-snapshot restart, and controlled in-progress restart with
|
||||
the fix. Record the original failure separately from repaired results; a pass
|
||||
with a different timestamp bucket or a test-only bundle is not verification of
|
||||
the production fix. Run platform-specific lifecycle checks before extending the
|
||||
evidence to Windows or mobile.
|
||||
The original E2E plan compared batching disabled, an ignored legacy setting, and
|
||||
explicitly enabled batching in both directions, with manual synchronisation,
|
||||
saved-snapshot restart, and controlled in-progress restart. Record the original
|
||||
failure separately from repaired results; a pass with a different timestamp
|
||||
bucket or a test-only bundle is not verification of the production fix. Keep
|
||||
platform-specific lifecycle evidence bounded to the tested environments before
|
||||
extending conclusions to Windows or mobile.
|
||||
|
||||
## Alternatives and consequences
|
||||
|
||||
|
||||
@@ -2,15 +2,16 @@
|
||||
date: 2026-10-17
|
||||
commonlib-version: "0.1.42"
|
||||
self-hosted-livesync-version: "1.0.37"
|
||||
status: unreleased
|
||||
status: released
|
||||
---
|
||||
|
||||
# Architectural Decision Record: Unicode Path Identity for CouchDB
|
||||
|
||||
## Status
|
||||
|
||||
Accepted on 2026-10-17. The feature remains unreleased. Local validation
|
||||
evidence and platform limits are recorded in the design document.
|
||||
Accepted on 2026-10-17. Unicode NFC path identity was released in Self-hosted
|
||||
LiveSync 1.0.38 with Commonlib 0.1.42. Local validation evidence and platform
|
||||
limits are recorded in the design document.
|
||||
|
||||
## Context
|
||||
|
||||
|
||||
@@ -2,7 +2,7 @@
|
||||
date: 2026-09-29
|
||||
commonlib-version: "0.1.33"
|
||||
self-hosted-livesync-version: "1.0.32"
|
||||
status: unreleased
|
||||
status: released
|
||||
---
|
||||
|
||||
# Configurable ID derivation
|
||||
@@ -21,12 +21,12 @@ including its passphrase-persistence correction at commit
|
||||
the `next` tag, provides the construction described here. It replaces the
|
||||
independent Chunk algorithm from the `0.1.32` prerelease without a
|
||||
compatibility branch or a new settings version. Legacy ID generation remains
|
||||
unchanged. LiveSync pins the published `0.1.33` package and its registry
|
||||
integrity in the lockfile.
|
||||
unchanged. At that implementation baseline, LiveSync pinned the published
|
||||
`0.1.33` package and its registry integrity in the lockfile.
|
||||
Its [Internal Metadata encryption design](https://github.com/vrtmrz/obsidian-livesync/blob/126d6eadb858a79a08ad7f600061e54fc8d31196/docs/design_docs/internal_metadata_encryption.md)
|
||||
remains the basis for Properties encryption and CouchDB feature admission.
|
||||
This document records an unreleased LiveSync feature. It does not select a
|
||||
plug-in release version.
|
||||
Independent ID derivation shipped in Self-hosted LiveSync 1.0.33. This document
|
||||
retains the design and package baseline recorded above.
|
||||
|
||||
## Feasibility
|
||||
|
||||
|
||||
@@ -10,40 +10,45 @@ This document uses the independent revision properties defined under
|
||||
[Revision](../glossary.md#revision) and the general state model in
|
||||
[Conflict resolution and revision provenance](../specs_conflict_resolution.md).
|
||||
|
||||
Document History can reconstruct an available historical revision from its
|
||||
Chunks and write that content to the Vault through **Back to this revision**.
|
||||
The current action writes directly through the storage adapter. It does not
|
||||
create a new Metadata revision, clear a logical deletion through a successor,
|
||||
or record which database revision produced the restored Vault file.
|
||||
Before the current restoration implementation, Document History could
|
||||
reconstruct an available historical revision from its Chunks and write that
|
||||
content to the Vault through **Back to this revision**. That action wrote
|
||||
directly through the storage adapter. It did not create a new Metadata revision,
|
||||
clear a logical deletion through a successor, or record which database revision
|
||||
produced the restored Vault file.
|
||||
|
||||
This leaves a logically deleted Metadata document at `deleted: true` after the
|
||||
file has returned to the Vault. A later ordinary Vault save may create a
|
||||
non-deleted successor, but restoration must not depend on an unrelated later
|
||||
file event.
|
||||
That left a logically deleted Metadata document at `deleted: true` after the
|
||||
file had returned to the Vault. A later ordinary Vault save could create a
|
||||
non-deleted successor, but restoration needed to succeed without depending on
|
||||
an unrelated later file event.
|
||||
|
||||
The same action does not inspect or preserve revision-tree intent explicitly
|
||||
when conflicts exist. Document History currently displays the ancestry of the
|
||||
PouchDB winner. It does not display the complete revision tree or the ancestry
|
||||
of every conflict leaf.
|
||||
Restoration does not resolve conflicts or provide a branch-aware history view.
|
||||
Document History displays the ancestry of the PouchDB winner; it does not
|
||||
display the complete revision tree or the ancestry of every conflict leaf.
|
||||
|
||||
This design covers restoration of one readable historical revision of a normal
|
||||
Vault file. It creates a new non-deleted successor revision, reflects that exact
|
||||
revision to the Vault, and leaves every other conflict branch available for the
|
||||
existing conflict workflow.
|
||||
|
||||
The current **Back to this revision** implementation writes that successor
|
||||
revision, reflects the exact created revision to the Vault, and records it as
|
||||
device-local provenance. The storage-only behaviour described above is the
|
||||
pre-fix history.
|
||||
|
||||
## Evidence
|
||||
|
||||
A real-Obsidian exercise created a Markdown document, removed it through the
|
||||
Vault API, and waited for LiveSync to store a logical-deletion successor. The
|
||||
Metadata retained all referenced Chunks, and Document History could reconstruct
|
||||
the deleted content.
|
||||
A real-Obsidian exercise before the current restoration implementation created
|
||||
a Markdown document, removed it through the Vault API, and waited for LiveSync
|
||||
to store a logical-deletion successor. The Metadata retained all referenced
|
||||
Chunks, and Document History could reconstruct the deleted content.
|
||||
|
||||
Selecting **Back to this revision** restored the file and its exact bytes to the
|
||||
Vault. The local database nevertheless retained the same deleted current
|
||||
revision after file processing had settled. An additional ordinary Vault save
|
||||
then created a new non-deleted successor. This demonstrates that content
|
||||
reconstruction works and that the missing operation is the database-aware
|
||||
restoration step.
|
||||
then created a new non-deleted successor. This historical reproduction showed
|
||||
that content reconstruction worked while the database-aware restoration step
|
||||
was missing.
|
||||
|
||||
## Revision-tree decision
|
||||
|
||||
|
||||
@@ -2,14 +2,15 @@
|
||||
date: 2026-09-27
|
||||
commonlib-version: "0.1.30"
|
||||
self-hosted-livesync-version: "1.0.32"
|
||||
status: unreleased
|
||||
status: released
|
||||
---
|
||||
|
||||
# Internal Metadata encryption and remote feature changes
|
||||
|
||||
This document defines the LiveSync integration of Commonlib's remote feature
|
||||
contract and encrypted Metadata for Hidden File Sync and Customisation Sync.
|
||||
It describes unreleased behaviour being implemented in this branch.
|
||||
The implementation shipped in Self-hosted LiveSync 1.0.33; this document
|
||||
retains its original design and implementation context.
|
||||
|
||||
Commonlib's companion `docs/remote-feature-compatibility.md` is the
|
||||
source of truth for the wire document, identifiers, validation, and shared
|
||||
@@ -159,11 +160,11 @@ after alignment. With the preference subsequently OFF, verify that Fast Fetch
|
||||
still decodes encrypted Metadata and preserves the remote feature declaration.
|
||||
|
||||
Keep the primary-language settings and troubleshooting guides, the
|
||||
database-compatibility ADR, and Unreleased notes aligned with this behaviour.
|
||||
Keep the detailed shared protocol in Commonlib and link to it after publication;
|
||||
do not maintain another copy of its wire schema here. Translations are a separate
|
||||
change. Update tested-version evidence when the implementation and its
|
||||
validation have been accepted.
|
||||
database-compatibility ADR, and release notes consistent with this behaviour.
|
||||
Commonlib's companion document remains the source of truth for the shared
|
||||
protocol; this document does not repeat its wire schema. Translations remain a
|
||||
separate change. The version fields above retain the original design baseline;
|
||||
add later tested-version evidence only when validation is recorded.
|
||||
|
||||
Related application contracts: [Replicator architecture](replicator_architecture.md),
|
||||
[Tweak compatibility](tweak_compatibility.md), and
|
||||
|
||||
@@ -2,7 +2,7 @@
|
||||
date: 2026-09-04
|
||||
commonlib-version: "0.1.21"
|
||||
self-hosted-livesync-version: "1.0.24"
|
||||
status: unreleased
|
||||
status: released
|
||||
---
|
||||
|
||||
# Path component length compatibility
|
||||
@@ -11,9 +11,9 @@ status: unreleased
|
||||
|
||||
File systems place limits on each file or folder name, rather than applying one
|
||||
common limit to an entire Vault-relative path. Those limits are also expressed
|
||||
in different units. Self-hosted LiveSync therefore treats 255 UTF-8 bytes as a
|
||||
focused Android and Linux compatibility warning, not as a universal definition
|
||||
of a valid path.
|
||||
in different units. The warning shipped in Self-hosted LiveSync 1.0.26. It
|
||||
treats 255 UTF-8 bytes as a focused Android and Linux compatibility warning,
|
||||
not as a universal definition of a valid path.
|
||||
|
||||
## Basis for the 255-byte warning
|
||||
|
||||
|
||||
@@ -2,7 +2,7 @@
|
||||
date: 2026-09-16
|
||||
commonlib-version: "0.1.25"
|
||||
self-hosted-livesync-version: "1.0.28"
|
||||
status: unreleased
|
||||
status: released
|
||||
---
|
||||
|
||||
# TURN credentials in P2P connection settings
|
||||
@@ -14,6 +14,9 @@ by acquiring temporary TURN credentials on the device before opening a P2P room.
|
||||
The [P2P transport compatibility ADR](../adr/2026_08_p2p_transport_compatibility.md)
|
||||
records the connection and persistence policy.
|
||||
|
||||
Managed Cloudflare TURN credentials shipped in Self-hosted LiveSync 1.0.29.
|
||||
This document records the design and its remaining runtime verification limits.
|
||||
|
||||
LiveSync prepares a connection copy of `P2PSyncSetting`. Commonlib owns the room
|
||||
lifecycle and consumes the resulting ICE settings. Service-specific HTTP and
|
||||
validation remain under `src/integrations/`; Commonlib has no provider catalogue
|
||||
|
||||
@@ -3,40 +3,42 @@ date: 2026-09-28
|
||||
commonlib-version: "0.1.33"
|
||||
feasibility-probe-version: "0.1.27"
|
||||
self-hosted-livesync-version: "1.0.32"
|
||||
status: implementing
|
||||
status: released
|
||||
---
|
||||
|
||||
# Time-bound Setup URIs
|
||||
|
||||
## Purpose and status
|
||||
|
||||
Offer two modes when generating a [Setup URI](../glossary.md):
|
||||
The published Obsidian generator offers two choices when creating a [Setup URI](../glossary.md):
|
||||
|
||||
- **Ephemeral**, selected by default, derives an effective passphrase from the
|
||||
entered passphrase and the current fixed time window.
|
||||
- **Persistent** uses the entered passphrase and the existing URI format
|
||||
unchanged. It also serves as the compatibility mode for older readers.
|
||||
- **Time-bound** (the `ephemeral` derivation mode), selected by default,
|
||||
derives an effective passphrase from the entered passphrase and the current
|
||||
fixed time window.
|
||||
- **Compatible (no time limit)** (the `persistent` derivation mode) uses the
|
||||
entered passphrase and the existing URI format unchanged. It also serves as
|
||||
the compatibility mode for older readers.
|
||||
|
||||
The generation mode choice shows the exact time until which an Ephemeral URI
|
||||
can be opened through the ordinary reader. Import still requires only the URI and
|
||||
the entered passphrase. Neither the mode nor a timestamp is stored in the URI.
|
||||
The Time-bound choice shows the exact time until which the URI can be opened
|
||||
through the ordinary reader. Import still requires only the URI and the entered
|
||||
passphrase. Neither the mode nor a timestamp is stored in the URI.
|
||||
|
||||
The design is technically feasible with the existing encryption primitives.
|
||||
An executable protocol probe passes 22 cases against Commonlib 0.1.27 and
|
||||
octagonal-wheels 0.1.54. The implementation was first published as Commonlib 0.1.32
|
||||
on the npm `next` tag. LiveSync now pins Commonlib 0.1.33 from that tag in its npm
|
||||
dependency and independent Deno tool imports. A mobile performance bound remains
|
||||
outstanding.
|
||||
Both URI availability choices shipped in Self-hosted LiveSync 1.0.33. The
|
||||
initial executable protocol probe passed 22 cases against Commonlib 0.1.27 and
|
||||
octagonal-wheels 0.1.54. The implementation was first published as Commonlib
|
||||
0.1.32 on the npm `next` tag. Current Self-hosted LiveSync 1.0.42 uses Commonlib
|
||||
0.2.2. A mobile performance bound remains outstanding.
|
||||
|
||||
This document records the proposed key derivation and integration contract. The
|
||||
companion probe demonstrates them without changing an application entry point.
|
||||
This document records the protocol and the integration plan used for the
|
||||
published feature. The companion probe remains a standalone design check; it
|
||||
does not exercise an application entry point.
|
||||
|
||||
## Scope
|
||||
|
||||
The first version offers Ephemeral and Persistent only. Ephemeral uses one
|
||||
fixed seven-day window shared by every implementation. Arbitrary start dates,
|
||||
custom end dates, selectable window lengths, and rolling seven-day lifetimes
|
||||
are outside this version.
|
||||
The published protocol offers the Ephemeral and Persistent modes only.
|
||||
Ephemeral uses one fixed seven-day window shared by every implementation.
|
||||
Arbitrary start dates, custom end dates, selectable window lengths, and rolling
|
||||
seven-day lifetimes are outside this version.
|
||||
|
||||
Keep the current settings filtering, remote profiles, main and P2P selections,
|
||||
and short/full export variants. Time binding affects opening the exported
|
||||
@@ -255,15 +257,16 @@ separately shared message.
|
||||
## Ownership and compatibility
|
||||
|
||||
Commonlib owns preparation of exported settings, the time-window calculation,
|
||||
passphrase derivation, format parsing, and dual-candidate decoding. Make these
|
||||
changes in the Commonlib repository. LiveSync must consume a validated packed
|
||||
artefact and then an exact reviewed package version, following
|
||||
passphrase derivation, format parsing, and dual-candidate decoding. These
|
||||
changes were made in the Commonlib repository. LiveSync consumes the reviewed
|
||||
Commonlib package, following
|
||||
[the Commonlib dependency workflow](../../devs.md#commonlib-dependency).
|
||||
|
||||
Preserve the existing positional encoder API. Its third and fourth arguments
|
||||
already mean properties to remove and whether to omit default values. Add a
|
||||
separate encoder, provisionally `encodeTimeBoundSetupURI`, with an options
|
||||
object containing the mode and the existing export options. Its result is:
|
||||
The implementation preserves the existing positional encoder API. Its third
|
||||
and fourth arguments retain their meanings: properties to remove and whether
|
||||
to omit default values. It adds a separate `encodeTimeBoundSetupURI` encoder
|
||||
with an options object containing the mode and the existing export options.
|
||||
Its result is:
|
||||
|
||||
```typescript
|
||||
type TimeBoundSetupURIResult = {
|
||||
@@ -272,27 +275,27 @@ type TimeBoundSetupURIResult = {
|
||||
};
|
||||
```
|
||||
|
||||
The new encoder defaults to Ephemeral. Preserve the old encoder's legacy
|
||||
behaviour for callers which have not migrated. Extend
|
||||
`decodeSettingsFromSetupURI` to try both candidates for `%$` while retaining its
|
||||
The new encoder defaults to Ephemeral. The old encoder retains its legacy
|
||||
behaviour for callers which have not migrated. The updated
|
||||
`decodeSettingsFromSetupURI` tries both candidates for `%$` while retaining its
|
||||
settings return contract and historical format handling. Persistent generation
|
||||
through the new API delegates to the old encoder. Time injection belongs in
|
||||
through the new API delegates to the old encoder. Time injection remains in
|
||||
internal test seams, not an import option exposed to users.
|
||||
|
||||
| Surface | Required work |
|
||||
| Surface | Published integration |
|
||||
| ------------------------------------------------------------------------------------ | -------------------------------------------------------------------------------------------------------------------------------------------------------- |
|
||||
| [Obsidian generation](../../src/serviceFeatures/setupObsidian/setupUri.ts) | Reuse the passphrase and copy dialogues, add a mode choice with the exact current window end, and use the new encoder for all three copy variants and the copy event. Preserve each variant's export filters. |
|
||||
| [Obsidian import](../../src/modules/features/SetupWizard/dialogs/UseSetupURI.svelte) | Consume the updated decoder and retain one authentication failure message. The existing prefix check indicates URI shape, not successful authentication. |
|
||||
| [Protocol handler](../../src/serviceFeatures/setupObsidian/setupProtocol.ts) | Keep the existing `settings` reconstruction; test encoded delimiters for both modes. |
|
||||
| [CLI setup](../../src/apps/cli/commands/runCommand.ts) | Consume the updated decoder and leave settings unchanged after any rejection. |
|
||||
| [WebPeer generator](../../src/apps/webpeer/src/P2PCheckSetup.ts) | Choose a mode explicitly and return availability to its result UI; keep Ephemeral as the default. |
|
||||
| [Setup utility](../../utils/setup/generate_setup_uri.ts) | Add an explicit mode input, default to Ephemeral, and return/print the end time. Persistent is an explicit choice for durable provisioning output. |
|
||||
| [Setup utility package facade](../../utils/setup/livesync-commonlib.ts) | Update its independent Commonlib pin and associated lockfiles; updating the root npm dependency alone is insufficient. |
|
||||
| [Obsidian generation](../../src/serviceFeatures/setupObsidian/setupUri.ts) | Reuses the passphrase and copy dialogues, presents the mode choice and exact current window end, and uses the encoder for all copy variants and the copy event while preserving each variant's export filters. |
|
||||
| [Obsidian import](../../src/modules/features/SetupWizard/dialogs/UseSetupURI.svelte) | Consumes the updated decoder and retains one authentication failure message. The existing prefix check indicates URI shape, not successful authentication. |
|
||||
| [Protocol handler](../../src/serviceFeatures/setupObsidian/setupProtocol.ts) | Retains the existing `settings` reconstruction; encoded delimiters remain part of protocol validation for both modes. |
|
||||
| [CLI setup](../../src/apps/cli/commands/runCommand.ts) | Uses the updated decoder and leaves settings unchanged after a rejection. |
|
||||
| [WebPeer generator](../../src/apps/webpeer/src/P2PCheckSetup.ts) | Explicitly selects Time-bound mode and returns availability to its result UI; this generator does not offer a Persistent choice. |
|
||||
| [Setup utility](../../utils/setup/generate_setup_uri.ts) | Accepts an explicit mode, defaults to Time-bound, and returns or prints the end time. Persistent is selected for durable provisioning output. |
|
||||
| [Setup utility package facade](../../utils/setup/livesync-commonlib.ts) | Retains its independent Commonlib pin and associated lockfiles, separate from the root npm dependency. |
|
||||
|
||||
The Fly.io wrapper delegates to the setup utility and should inherit the same
|
||||
contract. All maintained generators must migrate explicitly before this
|
||||
feature is described as the default across applications. Non-interactive
|
||||
automation which needs a durable URI must select Persistent.
|
||||
The Fly.io wrapper delegates to the setup utility and inherits the same
|
||||
contract. The initial plan required each maintained generator to select a mode
|
||||
explicitly. Non-interactive automation which needs a durable URI selects
|
||||
Persistent.
|
||||
|
||||
WebPeer hides an Ephemeral URI and disables its copy and additional-device
|
||||
actions after the window ends. Its connection monitor remains available for a
|
||||
@@ -309,8 +312,9 @@ The assessed, unchanged Commonlib 0.1.27 decoder opens Persistent with the
|
||||
entered passphrase and rejects Ephemeral with that same passphrase. Persistent
|
||||
compatibility means compatibility with readers which already support the
|
||||
ordinary generated `%$` representation; it does not add that representation
|
||||
to still older readers. Historical supported versions and their settings
|
||||
schemas still need compatibility fixtures before rollout.
|
||||
to still older readers. This design does not establish compatibility with every
|
||||
historical reader or settings schema; compatibility fixtures are needed before
|
||||
making broader claims about them.
|
||||
|
||||
Keep the original raw-passphrase behaviour of existing URIs. They are
|
||||
indistinguishable from newly generated Persistent URIs to the updated reader.
|
||||
@@ -358,68 +362,72 @@ All 22 cases passed:
|
||||
and raw-passphrase candidates;
|
||||
- the deliberate limitation that clock rollback reproduces an old key.
|
||||
|
||||
This demonstrates the proposed mechanism and baseline interoperability, not
|
||||
completed application support, formal mode indistinguishability, or a mobile
|
||||
latency bound. The present encryption API may perform two PBKDF2 derivations
|
||||
on a cold import. Do not infer production performance from this small probe,
|
||||
whose operations can reuse the dependency's in-memory key cache.
|
||||
The probe demonstrates feasibility and baseline interoperability. Application
|
||||
integration is recorded above; the probe does not establish formal mode
|
||||
indistinguishability or a mobile latency bound. The encryption API may perform
|
||||
two PBKDF2 derivations on a cold import. Do not infer production performance
|
||||
from this small probe, whose operations can reuse the dependency's in-memory
|
||||
key cache.
|
||||
|
||||
Implementation validation must cover Commonlib unit and packed-package tests,
|
||||
plus shared fixed-input protocol vectors across consumers. Add targeted LiveSync
|
||||
tests for the generation metadata, cancellation, a window change during mode
|
||||
selection, encoded deep links, and rejected imports without settings writes. Extend CLI, WebPeer,
|
||||
and Deno setup-tool round trips, including their Persistent selection.
|
||||
The original implementation validation plan covered Commonlib unit and
|
||||
packed-package tests, plus shared fixed-input protocol vectors across consumers.
|
||||
It called for targeted LiveSync tests of generation metadata, cancellation, a
|
||||
window change during mode selection, encoded deep links, and rejected imports
|
||||
without settings writes, alongside CLI, WebPeer, and Deno setup-tool round trips
|
||||
for both modes.
|
||||
|
||||
Then run the required LiveSync checks, unit suite, and production build, one
|
||||
broad process at a time with `NODE_OPTIONS=--max-old-space-size=3072` and bounded
|
||||
test workers. Use the existing focused real-Obsidian Setup URI workflow for
|
||||
copy/paste, deep-link import, cancellation, and successful setup. Verify
|
||||
selection-boundary handling with an injected test clock and mobile behaviour on a
|
||||
supported runtime; do not change the host's system clock. No remote database
|
||||
service is needed for the protocol tests themselves.
|
||||
The plan also called for the LiveSync checks, unit suite, and production build,
|
||||
one broad process at a time with `NODE_OPTIONS=--max-old-space-size=3072` and
|
||||
bounded test workers. It used the focused real-Obsidian Setup URI workflow for
|
||||
copy/paste, deep-link import, cancellation, and successful setup. Selection-
|
||||
boundary handling was to use an injected test clock; mobile behaviour requires
|
||||
a supported runtime and must not change the host's system clock. Protocol tests
|
||||
do not require a remote database service.
|
||||
|
||||
There is no identified blocker in the Time-bound URI protocol. The Commonlib
|
||||
candidate based on 0.1.31 passed its type, unit, boundary, release-process,
|
||||
and packed-package gates before the 0.1.32 release. A clean `npm ci` against
|
||||
the published 0.1.32 artefact and the frozen Deno tool lock resolve the same
|
||||
registry integrity. LiveSync passes its source checks, production build, and
|
||||
1,034 unit tests on a local stack with [PR #1222](https://github.com/vrtmrz/obsidian-livesync/pull/1222).
|
||||
The two CLI installer tests need a runner which permits child processes; both
|
||||
pass under normal process permissions. The Deno setup-tool suite, CLI setup
|
||||
The original protocol assessment found no blocker in the Time-bound URI
|
||||
protocol. The Commonlib candidate based on 0.1.31 passed its type, unit,
|
||||
boundary, release-process, and packed-package gates before the 0.1.32 release. A
|
||||
clean `npm ci` against the published 0.1.32 artefact and the frozen Deno tool
|
||||
lock resolved the same registry integrity. In the original implementation
|
||||
baseline, LiveSync passed its source checks, production build, and 1,034 unit
|
||||
tests on a local stack with [PR #1222](https://github.com/vrtmrz/obsidian-livesync/pull/1222).
|
||||
The two CLI installer tests needed a runner which permits child processes; both
|
||||
passed under normal process permissions. The Deno setup-tool suite, CLI setup
|
||||
and file-operation contract, and WebPeer browser tests passed before stacking.
|
||||
|
||||
Focused real-Obsidian generation checks pass on desktop and emulated mobile,
|
||||
The recorded real-Obsidian generation checks passed on desktop and emulated mobile,
|
||||
including the displayed Time-bound end, Compatible output, and mobile touch
|
||||
targets. A two-device CouchDB workflow now passes on the local stack: the
|
||||
provisioning tool creates a generation 12 database, the first device generates
|
||||
targets. A two-device CouchDB workflow passed on the local stack: the
|
||||
provisioning tool created a generation 12 database, the first device generated
|
||||
a Time-bound Setup URI after declaring its required generation 13 feature, and
|
||||
the second device imports it. An ordinary note completes a round trip, and a
|
||||
hidden snippet synchronises. The separate real-Obsidian tests for live remote
|
||||
the second device imported it. An ordinary note completed a round trip, and a
|
||||
hidden snippet synchronised. The separate real-Obsidian tests for live remote
|
||||
feature changes, internal Metadata migration, and a percent-prefixed E2EE
|
||||
passphrase also pass with published Commonlib 0.1.32. Unit and protocol tests
|
||||
cover window boundaries and preserve the existing distinction between an empty
|
||||
passphrase and cancellation. The WebPeer browser test confirms that its monitor
|
||||
remains available after the URI window ends for a device which has already
|
||||
imported the URI.
|
||||
passphrase also passed with published Commonlib 0.1.32. Unit and protocol tests
|
||||
covered window boundaries and preserved the existing distinction between an
|
||||
empty passphrase and cancellation. The WebPeer browser test confirmed that its
|
||||
monitor remained available after the URI window ended for a device which had
|
||||
already imported the URI.
|
||||
|
||||
The current Commonlib 0.1.33 integration includes the merged host changes from
|
||||
PR #1222 and PR #1225. npm 10 and normal clean installations preserve the lockfile;
|
||||
source checks, the production build, 1,064 unit tests, and ten Deno setup-tool
|
||||
tests pass. Both URI modes preserve ID recovery and explicit legacy ID selection.
|
||||
The Commonlib 0.1.33 integration included the merged host changes from
|
||||
PR #1222 and PR #1225. At that baseline, npm 10 and normal clean installations
|
||||
preserved the lockfile; source checks, the production build, 1,064 unit tests,
|
||||
and ten Deno setup-tool tests passed. Both URI modes preserved ID recovery and
|
||||
explicit legacy ID selection.
|
||||
|
||||
Real Obsidian tests cover independent ID keys through Time-bound, Compatible,
|
||||
and QR setup, encrypted local persistence, natural restarts of both devices,
|
||||
and bidirectional note transfers with stable document and Chunk IDs. Incorrect
|
||||
passphrases and past-window URIs leave runtime and persisted settings unchanged.
|
||||
The past clock belongs to an isolated fixture worker, not Obsidian or the host.
|
||||
CouchDB tests also cover a custom ID source and recovery code, rejection of
|
||||
incompatible document keys before remote writes, and the Doctor's decline,
|
||||
reminder, dismissal, and later acceptance paths. The detailed commands and the
|
||||
remaining published-artefact and physical-device checks are recorded in the
|
||||
Recorded real-Obsidian tests covered independent ID keys through Time-bound,
|
||||
Compatible, and QR setup, encrypted local persistence, natural restarts of both
|
||||
devices, and bidirectional note transfers with stable document and Chunk IDs.
|
||||
Incorrect passphrases and past-window URIs left runtime and persisted settings
|
||||
unchanged. The past clock belonged to an isolated fixture worker, not Obsidian
|
||||
or the host. CouchDB tests also covered a custom ID source and recovery code,
|
||||
rejection of incompatible document keys before remote writes, and the Doctor's
|
||||
decline, reminder, dismissal, and later acceptance paths. The detailed commands,
|
||||
remaining published-artefact checks, and physical-device checks are recorded in the
|
||||
[real Obsidian test guide](../../test/e2e-obsidian/README.md#combined-setup-and-security-regression-checks).
|
||||
|
||||
Compatible preserves the URI encryption format; each receiving client must
|
||||
still support the shared settings and the remote's declared features. The host
|
||||
compatibility checks from PR #1222 remain necessary. These results do not
|
||||
establish a mobile performance bound. The first implementation changes only
|
||||
primary-language resources; translation changes require separate scope.
|
||||
establish a mobile performance bound. The initial implementation changed only
|
||||
primary-language resources; translation changes remain a separate scope.
|
||||
@@ -2,15 +2,16 @@
|
||||
date: 2026-10-17
|
||||
commonlib-version: "0.1.42"
|
||||
self-hosted-livesync-version: "1.0.37"
|
||||
status: unreleased
|
||||
status: released
|
||||
---
|
||||
|
||||
# Tweak compatibility and recovery
|
||||
|
||||
This document describes the compatibility and recovery contract, including
|
||||
unreleased changes after Self-hosted LiveSync 1.0.37 using Commonlib 0.1.42.
|
||||
Commonlib owns the interpretation of synchronisation settings; LiveSync owns
|
||||
the dialogues and operations which consume that result.
|
||||
This document describes the compatibility and recovery contract introduced
|
||||
after the Self-hosted LiveSync 1.0.37 and Commonlib 0.1.42 baseline recorded
|
||||
above. The current assessment is included in Self-hosted LiveSync 1.0.42 with
|
||||
Commonlib 0.2.2. Commonlib owns the interpretation of synchronisation settings;
|
||||
LiveSync owns the dialogues and operations which consume that result.
|
||||
|
||||
## Shared assessment
|
||||
|
||||
|
||||
@@ -2,7 +2,7 @@
|
||||
date: 2026-10-19
|
||||
commonlib-version: "0.1.43"
|
||||
self-hosted-livesync-version: "1.0.38"
|
||||
status: unreleased
|
||||
status: released
|
||||
---
|
||||
|
||||
# Unicode path identity
|
||||
@@ -17,8 +17,10 @@ Commonlib's [consumer contract](https://github.com/vrtmrz/livesync-commonlib/blo
|
||||
defines the shared package boundary.
|
||||
|
||||
NFC identity shipped in Self-hosted LiveSync `1.0.38` with Commonlib `0.1.42`.
|
||||
The deletion handling in reconstruction below uses Commonlib `0.1.43` and
|
||||
remains unreleased in Self-hosted LiveSync.
|
||||
Conflict-aware deletion handling during reconstruction was added separately
|
||||
and is included in current Commonlib `0.2.2`, used by Self-hosted LiveSync
|
||||
`1.0.42`. This later handling is distinct from the initial NFC identity
|
||||
release.
|
||||
|
||||
## Scope
|
||||
|
||||
|
||||
@@ -1,5 +1,7 @@
|
||||
## The design document of the journal sync
|
||||
|
||||
Historical design: this is the first Journal Sync design. Its original transfer sequence and discussion of possible storage backends are retained as history. For the maintained pipeline and storage-adapter design, read [Journal Replicator 2nd Edition](design_docs_of_journalsync_2nd.md) and the [Replicator architecture](design_docs/replicator_architecture.md). Use [Object Storage Setup](setup_object_storage.md) for supported client setup. A possible backend discussed here is not evidence that an adapter is available in the current release.
|
||||
|
||||
Original title: Synchronise without CouchDB
|
||||
|
||||
### Goal
|
||||
@@ -47,4 +49,4 @@ This implementation replaces the synchronisation performed by CouchDB. Therefore
|
||||
|
||||
### Consideration and Conclusion
|
||||
|
||||
This design offers a novel approach to journal synchronisation without relying on CouchDB. It leverages PouchDB's journaling capabilities and leverages simple server-side storage for efficient data exchange. Hence, the new design could be said to have gotten a broader outlook.
|
||||
This design offers a novel approach to journal synchronisation without relying on CouchDB. It leverages PouchDB's journaling capabilities and leverages simple server-side storage for efficient data exchange. Hence, the new design could be said to have gotten a broader outlook.
|
||||
@@ -1,5 +1,9 @@
|
||||
# The design document of remote configuration management
|
||||
|
||||
## Implementation status
|
||||
|
||||
This document describes the profile catalogue and its runtime projections as used by Self-hosted LiveSync 1.0.42 with Commonlib 0.2.2. Commonlib owns profile migration, activation, and protected settings persistence. The [Replicator architecture](design_docs/replicator_architecture.md) and [P2P service lifecycle decision](adr/2026_08_replicator_capabilities_02_p2p_service_lifecycle.md) describe the separate P2P service boundary.
|
||||
|
||||
## Goal
|
||||
|
||||
- Allow us to manage multiple remote connections in a single vault.
|
||||
@@ -33,7 +37,9 @@ Once multiple remotes of the same type became desirable, the previous model no l
|
||||
The current settings now have two layers for remote configuration.
|
||||
|
||||
1. A stored collection of named remotes.
|
||||
2. One active remote projected into the legacy flat settings fields.
|
||||
2. One active main remote projected into the legacy flat settings fields.
|
||||
|
||||
The P2P service also has its own selected profile, recorded by `P2P_ActiveRemoteConfigurationId`. This permits the P2P service to use a selected P2P profile alongside a CouchDB or Object Storage main remote; it does not introduce a second main remote.
|
||||
|
||||
This means the replication and database layers can continue to read the effective remote from the existing settings fields, while the settings dialogue and migration logic can manage many stored remotes.
|
||||
|
||||
@@ -54,6 +60,7 @@ type RemoteConfiguration = {
|
||||
type RemoteConfigurations = {
|
||||
remoteConfigurations: Record<string, RemoteConfiguration>;
|
||||
activeConfigurationId: string;
|
||||
P2P_ActiveRemoteConfigurationId: string;
|
||||
};
|
||||
```
|
||||
|
||||
@@ -116,7 +123,11 @@ Therefore:
|
||||
- Import trusts the supplied connection string as-is after parsing.
|
||||
- We should regard exported connection strings as sensitive information, much like Setup URI or a credentials-bearing configuration file.
|
||||
|
||||
The `isEncrypted` field is currently reserved for future expansion. At present, the connection string itself is stored plainly inside the settings data, in the same sense that the effective runtime configuration can contain usable remote credentials.
|
||||
The `isEncrypted` field identifies an encrypted profile URI in persisted settings. When settings are saved with configuration encryption enabled, Commonlib encrypts the profile URIs and records `isEncrypted: true`. Loading protected settings decrypts the URIs into the runtime settings and clears that runtime flag before profile activation. The runtime settings and the persisted representation therefore have different purposes.
|
||||
|
||||
Protection depends on the configured settings-passphrase policy and a usable key. A plaintext configuration is not made confidential by the presence of the field alone. Consult [Encrypting sensitive configuration items](settings.md#encrypting-sensitive-configuration-items) for that policy.
|
||||
|
||||
Clipboard export serialises the usable connection string. Treat that exported value as sensitive even when its saved profile URI is encrypted at rest; storage protection does not encrypt the clipboard representation.
|
||||
|
||||
### Migration strategy
|
||||
|
||||
@@ -129,7 +140,7 @@ Rules:
|
||||
- If no remote list exists and the legacy fields contain a Peer-to-Peer configuration, create `legacy-p2p`.
|
||||
- If more than one legacy remote is populated, create all possible entries and select the active one according to `remoteType`.
|
||||
|
||||
This migration is intentionally additive. It does not remove the flat fields because they remain necessary as the active runtime projection.
|
||||
This migration is intentionally additive. It does not remove the flat fields because they remain necessary as the active runtime projection. A migrated legacy P2P profile also initialises the separate P2P selection. Loading an existing profile catalogue does not unconditionally replace an intentionally empty P2P selection.
|
||||
|
||||
### Normalisation and application paths
|
||||
|
||||
@@ -190,7 +201,7 @@ The feature should be tested from four viewpoints.
|
||||
|
||||
## Outlook
|
||||
|
||||
Import/export configuration strings should also be encrypted in the future, but this is a separate feature that can be added on top of the current design.
|
||||
A separately encrypted representation for clipboard import and export remains a possible extension. Encryption of profile URIs in persisted settings is already implemented as described above.
|
||||
|
||||
## Consideration and conclusion
|
||||
|
||||
@@ -203,4 +214,4 @@ Its core idea is modest but effective.
|
||||
- Project it into the legacy settings for runtime use.
|
||||
- Normalise every imported settings object through the same path.
|
||||
|
||||
This keeps the implementation understandable and migration-friendly. It also opens the door for future work, such as encrypted per-remote storage, richer remote metadata, or remote-scoped options, without forcing another large redesign of how remotes are represented.
|
||||
This keeps the implementation understandable and migration-friendly. It also opens the door for future work, such as richer remote metadata or remote-scoped options, without forcing another large redesign of how remotes are represented.
|
||||
@@ -1,6 +1,6 @@
|
||||
# Sharing `Tweak values`
|
||||
|
||||
NOTE: This is the planned feature design document. This is planned, but not be implemented now (v0.23.3). This has not reached the design freeze and will be added to from time to time.
|
||||
Historical design: this document records the original proposal written for v0.23.3. Tweak sharing and compatibility assessment are implemented in published releases. Use [Tweak compatibility and recovery](design_docs/tweak_compatibility.md) for the maintained contract, and [Settings](settings.md) for the available controls. The proposal and test plan below are retained as design history; they do not define the current behaviour.
|
||||
|
||||
## Goal
|
||||
|
||||
@@ -52,4 +52,4 @@ B: On the setting dialogue.
|
||||
- However, this might be an essential feature. Further efforts are desired.
|
||||
|
||||
### Consideration and Conclusion
|
||||
To be described after implemented, tested, and, released.
|
||||
The published implementation has evolved beyond this proposal, including classification of compatible differences and automatic alignment where configured. Its current decisions, recovery requirements, and remaining limitations are recorded in [Tweak compatibility and recovery](design_docs/tweak_compatibility.md). The original proposal does not establish that every item in its test plan was implemented unchanged.
|
||||
@@ -19,10 +19,11 @@ The command and view are registered only when developer tools are enabled. Disab
|
||||
| --- | --- | --- | --- |
|
||||
| Settings lifecycle | Automatic | Read-only | Exposes the seven synchronisation choices as typed observations and, for a genuinely new Vault, compares the selected recommendations with Commonlib's new-Vault contract. Existing-Vault setting preservation remains an automated migration and `settings-ui` E2E responsibility. |
|
||||
| Compatibility review boundary | Guided | Device-local state | Observes the current dedicated compatibility controller and opens its actual review action. The Harness does not restore Change Log acknowledgement, `lastReadUpdates`, or a separate manual Pass or Fail result. |
|
||||
| P2P composition | Automatic | Read-only | Confirms that the live P2P result resolves one replicator bound to the active Obsidian services. Peer discovery, replacement, reconnection, and transfer remain unit, CLI, and Compose E2E responsibilities. |
|
||||
| ID generation performance | Automatic, individual | Read-only | Measures legacy and independent IDs using fixed in-memory inputs, including key derivation and heap samples where available. Keep Obsidian in the foreground. |
|
||||
| File content hash performance | Manual | Read-only | Compares SHA-256 and xxHash64 on fixed in-memory Blob inputs. Input creation is excluded from hash timings; engine initialisation and warm-up are reported separately. |
|
||||
| Vault fixture round trip | Automatic, explicit | Dedicated Vault fixtures | After explicit confirmation, creates, reads, modifies, renames, and removes one owned fixture tree. General Vault reflection remains covered by its separate real-Obsidian E2E workflow. |
|
||||
|
||||
**Automatic** runs only the two read-only local observations. **Full review** also starts the guided compatibility observation and asks before the Vault fixture scenario writes anything. A scenario can also be run individually.
|
||||
**Automatic** runs only Settings lifecycle. **Full review** visits all listed scenarios, including the guided compatibility review and the two performance measurements, and asks before the Vault fixture scenario writes anything. Each scenario can also be run individually; a scenario's mode label does not by itself include it in the Automatic action.
|
||||
|
||||
These observations deliberately do not repeat stronger automated workflows. They exist to record which contracts the current real-device composition exposes while a maintainer reviews an immutable artefact.
|
||||
|
||||
|
||||
@@ -161,6 +161,8 @@ Following panes will be shown when you enable this setting.
|
||||
|
||||
### 1. Appearance
|
||||
|
||||
For icon meanings, progress counters, and CSS snippets, see the [status display guide](tips/status-display.md).
|
||||
|
||||
#### Display Language
|
||||
|
||||
Setting key: displayLanguage
|
||||
|
||||
+50
-23
@@ -3,16 +3,16 @@
|
||||
|
||||
This is how to configure fly.io and CouchDB on it for Self-hosted LiveSync.
|
||||
|
||||
The CouchDB instance is yours to manage. You can configure it yourself or set up another instance and switch to it, following the preservation and recovery guidance below.
|
||||
|
||||
> [!WARNING]
|
||||
> It is **your** instance. In Obsidian, we have files locally. Hence, do not hesitate to destroy the remote database if you feel something have got weird. We can launch and switch to the new CouchDB instance anytime[^1].
|
||||
>
|
||||
[^1]: Actually, I am always building the database for reproduction of the issue like so.
|
||||
> Before replacing or deleting an existing CouchDB instance, back up every available Vault and the remote database, then [choose the authoritative copy](recovery.md#first-choose-the-authoritative-copy). Stop or suspend synchronisation on other devices while reviewing recovery options. Local files alone do not establish that every needed edit or revision has been preserved.
|
||||
|
||||
> [!NOTE]
|
||||
> **What and why is the Fly.io?**
|
||||
> At some point, we started to experience problems related to our IBM Cloudant account. At the same time, Self-hosted LiveSync started to improve its functionality, requiring CouchDB in a more natural state to use all its features.
|
||||
>
|
||||
> Then we found Fly.io. Fly.io is the PaaS Platform, which can be useable for a very reasonable price. It generally falls within the `Free Allowances` range in most cases.
|
||||
> Fly.io charges according to current account and resource usage, including Machines, persistent volumes, and network usage. Check [Fly.io pricing](https://fly.io/pricing/) and the [pricing documentation](https://fly.io/docs/about/pricing/) before deploying. A time-limited trial does not provide ongoing free hosting.
|
||||
|
||||
## Required materials
|
||||
|
||||
@@ -31,10 +31,10 @@ This is how to configure fly.io and CouchDB on it for Self-hosted LiveSync.
|
||||
4. Copy the Setup-URI and Use it in the Obsidian.
|
||||
5. You have been synchronised. Use the Setup-URI in subsequent devices.
|
||||
|
||||
Steps 4 and 5 are detailed in the [Quick Setup](./quick_setup.md#1-using-setup-uris).
|
||||
For steps 4 and 5, see [Set up the first device](./quick_setup.md#set-up-the-first-device), then [Add another device](./quick_setup.md#add-another-device) for subsequent devices.
|
||||
|
||||
> [!NOTE]
|
||||
> Your automatically configured configurations will be shown on the result in the Colab note like below, and **it will not be saved**. Please make a note of it somewhere.
|
||||
> Save your own generated connection details and Setup URI securely. This is a historical Colab output example from 2024; its values are illustrative and will differ for each deployment. Replace all example credentials before deployment.
|
||||
> ```
|
||||
> -- YOUR CONFIGURATION --
|
||||
> URL : https://billowing-dawn-6619.fly.dev
|
||||
@@ -45,16 +45,18 @@ Steps 4 and 5 are detailed in the [Quick Setup](./quick_setup.md#1-using-setup-u
|
||||
|
||||
### B. Scripted Setup
|
||||
|
||||
Please refer to the document of [deploy-server.sh](../utils/readme.md#deploy-serversh).
|
||||
Please refer to the document of [deploy-server.sh](../utils/readme.md#flyio-deployment).
|
||||
|
||||
### C. Manual Setup
|
||||
|
||||
| Used in the text | Meaning and where to use | Memo |
|
||||
| ---------------- | --------------------------- | ------------------------------------------------------------------------ |
|
||||
| campanella | Username | It is less likely to fail if it consists only of letters and numbers. |
|
||||
| dfusiuada9suy | Password | |
|
||||
| `<YOUR UNIQUE COUCHDB USERNAME>` | Username | Choose a unique value using only letters and numbers. |
|
||||
| `<YOUR UNIQUE COUCHDB PASSWORD>` | Password | Generate a strong, unique password for this instance. |
|
||||
| nrt | Region to make the instance | We can use any [region](https://fly.io/docs/reference/regions/) near us. |
|
||||
|
||||
Choose an available, unique Fly.io application name and replace both credential placeholders before deployment. Historical output below illustrates the command responses; use your own application name, hostname, and credentials.
|
||||
|
||||
#### 1. Install flyctl
|
||||
|
||||
- Mac or Linux
|
||||
@@ -97,8 +99,8 @@ For more information, please refer to [Sign up](https://fly.io/docs/hands-on/sig
|
||||
|
||||
|
||||
```bash
|
||||
$ fly launch --name=billowing-dawn-6619 --env="COUCHDB_USER=campanella" --copy-config=true --detach --no-deploy --region nrt --yes
|
||||
$ fly secrets set COUCHDB_PASSWORD=dfusiuada9suy
|
||||
$ fly launch --name='<YOUR UNIQUE APP NAME>' --env="COUCHDB_USER=<YOUR UNIQUE COUCHDB USERNAME>" --copy-config=true --detach --no-deploy --region nrt --yes
|
||||
$ fly secrets set COUCHDB_PASSWORD='<YOUR UNIQUE COUCHDB PASSWORD>'
|
||||
```
|
||||
|
||||
#### 4. Deploy
|
||||
@@ -170,24 +172,24 @@ Visit your newly deployed app at https://billowing-dawn-6619.fly.dev/
|
||||
|
||||
After the initial setup, CouchDB needs some more customisations to be used from Self-hosted LiveSync. It can be configured in browsers or by HTTP-REST APIs.
|
||||
|
||||
This section is set up using the REST API.
|
||||
This section uses the REST API. The macOS and Linux commands require [jq](https://jqlang.org/download/) to encode credentials as JSON. Windows uses PowerShell's built-in `ConvertTo-Json`.
|
||||
|
||||
1. Prepare environment variables.
|
||||
|
||||
- Mac or Linux:
|
||||
|
||||
```bash
|
||||
export couchHost=https://billowing-dawn-6619.fly.dev
|
||||
export couchUser=campanella
|
||||
export couchPwd=dfusiuada9suy
|
||||
export couchHost='https://<YOUR APP HOSTNAME>'
|
||||
export couchUser='<YOUR UNIQUE COUCHDB USERNAME>'
|
||||
export couchPwd='<YOUR UNIQUE COUCHDB PASSWORD>'
|
||||
```
|
||||
|
||||
- Windows
|
||||
|
||||
```powershell
|
||||
set couchHost https://billowing-dawn-6619.fly.dev
|
||||
set couchUser campanella
|
||||
set couchPwd dfusiuada9suy
|
||||
set couchHost 'https://<YOUR APP HOSTNAME>'
|
||||
set couchUser '<YOUR UNIQUE COUCHDB USERNAME>'
|
||||
set couchPwd '<YOUR UNIQUE COUCHDB PASSWORD>'
|
||||
$creds = [System.Convert]::ToBase64String([System.Text.Encoding]::UTF8.GetBytes("${couchUser}:${couchPwd}"))
|
||||
```
|
||||
|
||||
@@ -196,13 +198,36 @@ $creds = [System.Convert]::ToBase64String([System.Text.Encoding]::UTF8.GetBytes(
|
||||
- Mac or Linux
|
||||
|
||||
```bash
|
||||
curl -X POST "${couchHost}/_cluster_setup" -H "Content-Type: application/json" -d "{\"action\":\"enable_single_node\",\"username\":\"${couchUser}\",\"password\":\"${couchPwd}\",\"bind_address\":\"0.0.0.0\",\"port\":5984,\"singlenode\":true}" --user "${couchUser}:${couchPwd}"
|
||||
clusterSetupJSON=$(jq -n \
|
||||
--arg username "$couchUser" \
|
||||
--arg password "$couchPwd" \
|
||||
'{
|
||||
action: "enable_single_node",
|
||||
username: $username,
|
||||
password: $password,
|
||||
bind_address: "0.0.0.0",
|
||||
port: 5984,
|
||||
singlenode: true
|
||||
}') &&
|
||||
curl -X POST "${couchHost}/_cluster_setup" \
|
||||
-H "Content-Type: application/json" \
|
||||
--data-binary "$clusterSetupJSON" \
|
||||
--user "${couchUser}:${couchPwd}"
|
||||
```
|
||||
|
||||
- Windows
|
||||
|
||||
```powershell
|
||||
iwr -UseBasicParsing -Method 'POST' -ContentType 'application/json; charset=utf-8' -Headers @{ 'Authorization' = 'Basic ' + $creds } "${couchHost}/_cluster_setup" -Body "{""action"":""enable_single_node"",""username"":""${couchUser}"",""password"":""${couchPwd}"",""bind_address"":""0.0.0.0"",""port"":5984,""singlenode"":true}"
|
||||
$clusterSetupJSON = @{
|
||||
action = 'enable_single_node'
|
||||
username = $couchUser
|
||||
password = $couchPwd
|
||||
bind_address = '0.0.0.0'
|
||||
port = 5984
|
||||
singlenode = $true
|
||||
} | ConvertTo-Json -Compress
|
||||
|
||||
iwr -UseBasicParsing -Method 'POST' -ContentType 'application/json; charset=utf-8' -Headers @{ 'Authorization' = 'Basic ' + $creds } "${couchHost}/_cluster_setup" -Body $clusterSetupJSON
|
||||
```
|
||||
|
||||
Note: if the response code is not 200. We have to retry the request once again.
|
||||
@@ -242,11 +267,13 @@ Note: Each of these should also be repeated until finished in 200.
|
||||
|
||||
#### 6. Use it from Self-hosted LiveSync
|
||||
|
||||
Now the CouchDB is ready to use from Self-hosted LiveSync. We can use `https://billowing-dawn-6619.fly.dev` in URI, `campanella` in `Username` and `dfusiuada9suy` in `Password` on Self-hosted LiveSync. The `Database name` could be anything you want.
|
||||
Please refer to the [Minimal Setup of the Quick Setup](./quick_setup.md#2-minimal-setup).
|
||||
Now the CouchDB is ready to use from Self-hosted LiveSync. Use your Fly.io app's hostname in `URI`, and the unique credentials configured above in `Username` and `Password`. The `Database name` could be anything you want.
|
||||
Please refer to [Configure CouchDB manually on the first device](./quick_setup.md#configure-couchdb-manually-on-the-first-device).
|
||||
|
||||
## Delete the Instance
|
||||
|
||||
Before deleting the instance, preserve the data you still need and follow [Recovery and flag files](recovery.md#first-choose-the-authoritative-copy).
|
||||
|
||||
If you want to delete the CouchDB instance, you can do that in [fly.io Dashboard](https://fly.io/dashboard/personal)
|
||||
|
||||
If you have done with [B. Scripted Setup](#b-scripted-setup), we can use [delete-server.sh](../utils/readme.md#delete-serversh).
|
||||
If you have done with [B. Scripted Setup](#b-scripted-setup), we can use [delete-server.sh](../utils/readme.md#flyio-deployment).
|
||||
@@ -18,14 +18,14 @@ The public generator applies the Object Storage preset and records the connectio
|
||||
|
||||
```sh
|
||||
export remote_type=s3
|
||||
export endpoint=https://objects.example.com
|
||||
export access_key=<ACCESS KEY>
|
||||
export secret_key=<SECRET KEY>
|
||||
export endpoint='https://objects.example.com'
|
||||
export access_key='<ACCESS KEY>'
|
||||
export secret_key='<SECRET KEY>'
|
||||
export bucket=vault-data
|
||||
export region=auto
|
||||
export bucket_prefix=my-vault
|
||||
export LIVESYNC_PASSPHRASE=<A STRONG VAULT ENCRYPTION PASSPHRASE>
|
||||
export uri_passphrase=<A SEPARATE SETUP URI PASSPHRASE>
|
||||
export LIVESYNC_PASSPHRASE='<A STRONG VAULT ENCRYPTION PASSPHRASE>'
|
||||
export uri_passphrase='<A SEPARATE SETUP URI PASSPHRASE>'
|
||||
deno run --minimum-dependency-age=0 --allow-env https://raw.githubusercontent.com/vrtmrz/obsidian-livesync/main/utils/setup/generate_setup_uri.ts
|
||||
```
|
||||
|
||||
|
||||
+24
-18
@@ -15,7 +15,7 @@
|
||||
- [4. Starting Docker Compose in background](#4-run-the-docker-compose-file-in-the-background)
|
||||
- [C. Install CouchDB directly](#c-install-couchdb-directly)
|
||||
- [2. Run couchdb-init.sh for initialise](#2-run-couchdb-initsh-for-initialise)
|
||||
- [3. Expose CouchDB to the Internet](#3-expose-couchdb-to-the-internet)
|
||||
- [3. Make CouchDB reachable from your devices](#3-make-couchdb-reachable-from-your-devices)
|
||||
- [4. Client Setup](#4-client-setup)
|
||||
- [1. Generate the setup URI on a desktop device or server](#1-generate-the-setup-uri-on-a-desktop-device-or-server)
|
||||
- [2. Setup Self-hosted LiveSync to Obsidian](#2-setup-self-hosted-livesync-to-obsidian)
|
||||
@@ -122,14 +122,14 @@ The `username` and `password` in this step are CouchDB administrator credentials
|
||||
|
||||
```
|
||||
export hostname=http://localhost:5984
|
||||
export username=<INSERT COUCHDB ADMINISTRATOR USERNAME HERE>
|
||||
export password=<INSERT COUCHDB ADMINISTRATOR PASSWORD HERE>
|
||||
export username='<INSERT COUCHDB ADMINISTRATOR USERNAME HERE>'
|
||||
export password='<INSERT COUCHDB ADMINISTRATOR PASSWORD HERE>'
|
||||
export database=obsidiannotes
|
||||
curl -s https://raw.githubusercontent.com/vrtmrz/obsidian-livesync/main/utils/couchdb/couchdb-init.sh | bash
|
||||
```
|
||||
|
||||
> [!TIP]
|
||||
> In fish, `hostname` is read-only and cannot be exported. Pass the values through `env` instead: `curl -s https://raw.githubusercontent.com/vrtmrz/obsidian-livesync/main/utils/couchdb/couchdb-init.sh | env hostname=http://localhost:5984 username=<...> password=<...> database=obsidiannotes bash`.
|
||||
> In fish, `hostname` is read-only and cannot be exported. Pass the values through `env` instead: `curl -s https://raw.githubusercontent.com/vrtmrz/obsidian-livesync/main/utils/couchdb/couchdb-init.sh | env hostname='http://localhost:5984' username='<INSERT COUCHDB ADMINISTRATOR USERNAME HERE>' password='<INSERT COUCHDB ADMINISTRATOR PASSWORD HERE>' database=obsidiannotes bash`.
|
||||
|
||||
If it results like the following:
|
||||
```
|
||||
@@ -140,21 +140,25 @@ The wrapper runs the exact registry-pinned Commonlib consumer. When `database` i
|
||||
|
||||
If you are using Docker Compose and the above command does not work or displays `ERROR: Hostname missing`, you can try running the following command, replacing the placeholders with your own values:
|
||||
```
|
||||
curl -s https://raw.githubusercontent.com/vrtmrz/obsidian-livesync/main/utils/couchdb/couchdb-init.sh | hostname=http://<YOUR SERVER IP>:5984 username=<INSERT COUCHDB ADMINISTRATOR USERNAME HERE> password=<INSERT COUCHDB ADMINISTRATOR PASSWORD HERE> database=obsidiannotes bash
|
||||
curl -s https://raw.githubusercontent.com/vrtmrz/obsidian-livesync/main/utils/couchdb/couchdb-init.sh | hostname='http://<YOUR SERVER IP>:5984' username='<INSERT COUCHDB ADMINISTRATOR USERNAME HERE>' password='<INSERT COUCHDB ADMINISTRATOR PASSWORD HERE>' database=obsidiannotes bash
|
||||
```
|
||||
|
||||
## 3. Expose CouchDB to the Internet
|
||||
<a id="3-expose-couchdb-to-the-internet"></a>
|
||||
|
||||
- You can skip this instruction if you using only in intranet and only with desktop devices.
|
||||
- For mobile devices, Obsidian requires a valid SSL certificate. Usually, it needs exposing the internet.
|
||||
## 3. Make CouchDB reachable from your devices
|
||||
|
||||
Whatever solutions we can use. For simplicity, the following sample uses Cloudflare Zero Trust for testing.
|
||||
- You can skip this step if you use CouchDB only on a trusted local network with desktop devices.
|
||||
- For mobile devices, Obsidian requires HTTPS with a valid certificate. HTTPS does not require making CouchDB publicly accessible.
|
||||
|
||||
Choose how your devices reach your CouchDB server. You can use a [reverse proxy you operate](#reverse-proxies), private-network access, or a tunnel service of your choice. The [Docker profiles](../docker/README.md#profile-details) provide examples.
|
||||
|
||||
Cloudflare is one optional choice. The following example uses a [Quick Tunnel](https://developers.cloudflare.com/tunnel/get-started/quick-tunnels/) for temporary testing. It prints a random hostname which changes each time you create a tunnel, and the URL stops working when the `cloudflared` process stops. Quick Tunnels are intended for testing and development, not production. For ongoing use, configure your chosen connection method; the [Cloudflare Tunnel guide](https://developers.cloudflare.com/tunnel/get-started/) applies if you choose that provider.
|
||||
|
||||
```
|
||||
cloudflared tunnel --url http://localhost:5984
|
||||
```
|
||||
|
||||
You will then get the following output:
|
||||
The following 2024 output is an example of the format. Its hostname is illustrative only; use the URL printed by your own tunnel instead.
|
||||
|
||||
```
|
||||
2024-02-14T10:35:25Z INF Thank you for trying Cloudflare Tunnel. Doing so, without a Cloudflare account, is a quick way to experiment and try it out. However, be aware that these account-less Tunnels have no uptime guarantee. If you intend to use Tunnels in production you should use a pre-created named tunnel by following: https://developers.cloudflare.com/cloudflare-one/connections/connect-apps
|
||||
@@ -167,7 +171,7 @@ You will then get the following output:
|
||||
:
|
||||
:
|
||||
```
|
||||
Now `https://tiles-photograph-routine-groundwater.trycloudflare.com` is our server. Make it into the background once, please.
|
||||
Use the URL printed by your tunnel as the server address. Keep the tunnel process running while you configure and test the client.
|
||||
|
||||
|
||||
## 4. Client Setup
|
||||
@@ -178,13 +182,15 @@ Now `https://tiles-photograph-routine-groundwater.trycloudflare.com` is our serv
|
||||
|
||||
The `LIVESYNC_USER` and `LIVESYNC_PASS` here are the credentials which Self-hosted LiveSync will store for routine access to this database. They may be the administrator credentials from step 2. If you have separately configured a CouchDB account with the required access to this database, use that account instead. Neither the provisioning wrapper nor the Setup URI generator creates that separate account.
|
||||
|
||||
Use the server URL through which your devices reach CouchDB. For the temporary Quick Tunnel example above, use the URL printed by your own running tunnel.
|
||||
|
||||
```bash
|
||||
export LIVESYNC_HOST=https://tiles-photograph-routine-groundwater.trycloudflare.com
|
||||
export LIVESYNC_HOST='<YOUR COUCHDB SERVER URL>'
|
||||
export LIVESYNC_DB=obsidiannotes
|
||||
export LIVESYNC_USER=<INSERT COUCHDB USERNAME FOR LIVESYNC>
|
||||
export LIVESYNC_PASS=<INSERT THE COUCHDB PASSWORD>
|
||||
export LIVESYNC_PASSPHRASE=<INSERT A STRONG VAULT ENCRYPTION PASSPHRASE>
|
||||
export uri_passphrase=<INSERT A SEPARATE SETUP URI PASSPHRASE> # Optional
|
||||
export LIVESYNC_USER='<INSERT COUCHDB USERNAME FOR LIVESYNC>'
|
||||
export LIVESYNC_PASS='<INSERT THE COUCHDB PASSWORD>'
|
||||
export LIVESYNC_PASSPHRASE='<INSERT A STRONG VAULT ENCRYPTION PASSPHRASE>'
|
||||
export uri_passphrase='<INSERT A SEPARATE SETUP URI PASSPHRASE>' # Optional
|
||||
deno run --minimum-dependency-age=0 --allow-env https://raw.githubusercontent.com/vrtmrz/obsidian-livesync/main/utils/setup/generate_setup_uri.ts
|
||||
```
|
||||
|
||||
@@ -199,9 +205,9 @@ The generator consumes the exact registry-pinned Commonlib release used by the p
|
||||
|
||||
By default, the URI is Ephemeral and can be opened only until the exact UTC time printed by the generator. This is the end of a fixed seven-day window, not seven days after creation. Set `uri_mode=persistent` before running the command if the URI needs no time condition or must be readable by an older client that supports the existing encrypted format.
|
||||
|
||||
You will then get the following output:
|
||||
The output has the following form. The timestamp and values below are illustrative:
|
||||
|
||||
```bash
|
||||
```text
|
||||
Generated couchdb Setup URI.
|
||||
Ephemeral: usable until 2026-10-01T00:00:00.000Z (UTC).
|
||||
Your passphrase for the Setup URI is: H7vX...a-random-32-character-value
|
||||
|
||||
+5
-5
@@ -127,11 +127,11 @@ The public Setup URI generator is also available when configuration must be crea
|
||||
|
||||
```sh
|
||||
export remote_type=p2p
|
||||
export p2p_relays=wss://relay.example.com
|
||||
export p2p_room_id=<A PRIVATE ROOM ID> # Optional; generated when omitted
|
||||
export p2p_passphrase=<A PRIVATE P2P PASSPHRASE> # Optional; generated when omitted
|
||||
export LIVESYNC_PASSPHRASE=<A STRONG VAULT ENCRYPTION PASSPHRASE>
|
||||
export uri_passphrase=<A SEPARATE SETUP URI PASSPHRASE>
|
||||
export p2p_relays='wss://relay.example.com'
|
||||
export p2p_room_id='<A PRIVATE ROOM ID>' # Optional; generated when omitted
|
||||
export p2p_passphrase='<A PRIVATE P2P PASSPHRASE>' # Optional; generated when omitted
|
||||
export LIVESYNC_PASSPHRASE='<A STRONG VAULT ENCRYPTION PASSPHRASE>'
|
||||
export uri_passphrase='<A SEPARATE SETUP URI PASSPHRASE>'
|
||||
deno run --minimum-dependency-age=0 --allow-env https://raw.githubusercontent.com/vrtmrz/obsidian-livesync/main/utils/setup/generate_setup_uri.ts
|
||||
```
|
||||
|
||||
|
||||
@@ -33,7 +33,11 @@ The current benchmark measures the Node CLI process. It does not establish Obsid
|
||||
|
||||
## Reproducible benchmark
|
||||
|
||||
The benchmark is implemented under `src/apps/cli/testdeno` and packaged by `test/bench-network`. It uses the real CLI mirror and synchronisation path, Commonlib 0.1.0-rc.4, CouchDB 3.5.0, the V3 Rabin–Karp splitter, optional Data Compression, and E2EE V2. It runs each of these conditions three times in rotating order:
|
||||
The benchmark is implemented under `src/apps/cli/testdeno` and packaged by `test/bench-network`. It exercises the real CLI mirror and synchronisation path. The recorded three-repeat result below used Commonlib 0.1.0-rc.4, CouchDB 3.5.0, the V3 Rabin–Karp splitter, optional Data Compression, and E2EE V2.
|
||||
|
||||
The current Compose runner installs the checked-out repository's package and lockfile with `npm ci`; Self-hosted LiveSync 1.0.42 selects Commonlib 0.2.2. Running the command below measures that checkout, rather than recreating the older package environment automatically. Keep the recorded results associated with their original measurement date and dependency version.
|
||||
|
||||
With the repeat count shown below, it runs each of these conditions three times in rotating order:
|
||||
|
||||
- E2EE off, compression off;
|
||||
- E2EE off, compression on;
|
||||
|
||||
+11
-11
@@ -1,8 +1,8 @@
|
||||
# Fast Setup (Simple Fetch)
|
||||
|
||||
Fast Setup is a streamlined, user-friendly data retrieval and initialisation flow designed to simplify setting up secondary devices or recovering databases.
|
||||
Fast Setup (Simple Fetch) is the preferred flow for initial replication to a new or empty secondary device. It can also be used for recovery, but a retrieval choice may replace local files. Preserve valuable local and remote copies before choosing how to handle a Vault that already contains work. See [Recovery and flag files](../recovery.md#first-choose-the-authoritative-copy) for guidance on identifying the authoritative copy.
|
||||
|
||||
Instead of guiding the user through the detailed multi-step setup wizard dialogues, Fast Setup prompts the user with high-level sync decisions and automates database download and local storage scanning in one continuous process.
|
||||
Instead of guiding the user through the detailed multi-step setup wizard dialogues, Fast Setup prompts the user with high-level synchronisation decisions and automates data retrieval and local storage scanning in one process. The retrieval path depends on the selected remote.
|
||||
|
||||
---
|
||||
|
||||
@@ -16,14 +16,14 @@ On the next startup, the plug-in boots in scheduled fetch mode and opens a simpl
|
||||
|
||||
## Technical Characteristics
|
||||
|
||||
Fast Setup leverages several backend optimisations to make the retrieval fast, safe, and clean:
|
||||
Fast Setup uses several retrieval and reflection behaviours:
|
||||
|
||||
1. **Stream-based Replication for Speed**
|
||||
- It fetches all remote metadata via stream reception, which is significantly faster than traditional chunk-by-chunk retrieval.
|
||||
1. **Stream-based Fast Fetch for CouchDB**
|
||||
- The stream-based Fast Fetch path is CouchDB-specific. It retrieves remote Metadata incrementally; Object Storage and P2P use their own retrieval paths.
|
||||
2. **Delayed File Reflection to Prevent Corrupted Warnings**
|
||||
- By suspending file reflection during the download phase, it prevents the plug-in from raising temporary or false "corrupted data synchronisation" or "size mismatch" warnings that can occur during the chunk download process.
|
||||
3. **Time-Based Comparison is Generally Sufficient**
|
||||
- Since the vault is entering a fresh synchronisation or recovery state, comparing files based on their modification timestamps (newer-wins) is highly reliable and sufficient to reconcile files without needing complex manual conflict resolution.
|
||||
- Delaying local file reflection during retrieval suppresses temporary synchronisation warnings while remote data is being fetched.
|
||||
3. **Timestamp Comparison Selects a Whole File**
|
||||
- **Compare time and take newer** compares modification timestamps and selects one whole-file version. It does not merge file contents. Before choosing this policy for a Vault with existing work, preserve valuable local and remote edits and review [Recovery and flag files](../recovery.md#first-choose-the-authoritative-copy).
|
||||
|
||||
---
|
||||
|
||||
@@ -33,11 +33,11 @@ Fast Setup leverages several backend optimisations to make the retrieval fast, s
|
||||
You will be prompted to choose how the retrieved remote data will interact with your existing local files:
|
||||
|
||||
1. **Compare time and take newer (newer-wins)**
|
||||
- Compares the modified time of files and accepts the newer version.
|
||||
- **Recommended if:** You have been using Self-hosted LiveSync and have made changes on multiple devices that you want to merge.
|
||||
- Compares the modified time of files and selects the newer whole-file version; it does not combine edits from both versions.
|
||||
- **Recommended if:** You have preserved both versions and deliberately want to select one according to its modification time.
|
||||
2. **Overwrite all with remote files (remote-wins)**
|
||||
- Remote data is treated as the source of truth.
|
||||
- **Recommended if:** You are setting up a brand new device with an empty or clean vault.
|
||||
- **Recommended if:** You are setting up a new secondary device with an empty Vault.
|
||||
- *Warning: This will overwrite local files with remote files. Please ensure you have a backup of your local vault before proceeding.*
|
||||
3. **Use the detailed flow (legacy)**
|
||||
- Switches back to the detailed, traditional setup wizard dialogues.
|
||||
|
||||
@@ -0,0 +1,98 @@
|
||||
# Synchronisation status and CSS customisation
|
||||
|
||||
Use [General Settings](../settings.md#2-general-settings) to choose whether synchronisation status appears in the status bar, inside the editor, or both. The following sections explain the indicators and how to customise their appearance.
|
||||
|
||||
## Status indicators
|
||||
|
||||
- Activity Indicator
|
||||
- 📲 A finite remote operation is in progress
|
||||
- 🌐N Approximate remote requests currently in progress
|
||||
- Status
|
||||
- ⏹️ Stopped
|
||||
- 🌀 Starting synchronisation
|
||||
- ⏩ Comparing CouchDB change history, including entries already held locally
|
||||
- 💤 LiveSync enabled. Waiting for changes
|
||||
- ⚡️ Synchronisation in progress
|
||||
- ⚠ An error occurred
|
||||
- Statistical Indicators
|
||||
- ↑ Uploaded chunks and metadata
|
||||
- ↓ Downloaded chunks and metadata
|
||||
- Progress Indicators
|
||||
- 📥 Unprocessed transferred items
|
||||
- 📄 Working database operation
|
||||
- 💾 Working write storage processes
|
||||
- ⏳ Working read storage processes
|
||||
- 🛫 Pending read storage processes
|
||||
- 📬 Batched read storage processes
|
||||
- ⚙️ Working or pending storage processes for hidden files
|
||||
- 🛄 Pending initial on-demand chunk requests
|
||||
- 🔁 Pending chunk retries, including retry delays
|
||||
- 🔌 Working customisation items (configuration, snippets, and plug-ins)
|
||||
|
||||
To prevent file and database corruption, please avoid closing Obsidian until all progress indicators have disappeared as much as possible (although the plug-in will attempt to resume if interrupted). This is especially important if you have deleted or renamed files.
|
||||
|
||||
## CouchDB comparison count
|
||||
|
||||
When CouchDB change history is being compared, directional counts can appear before any new documents arrive, for example `Sync: ⏩ (⏩↑ 1200) (⏩↓ 84) ↑ 0 ↓ 0`. The `⏩↑` count is for the push direction, and `⏩↓` is for the pull direction. Each counts candidate changes-feed entries compared during the current replication attempt, including entries which do not need to be transferred and repeated entries on retries. They are feed-entry counts, not counts of Vault files which need to be transferred. The final `↑` and `↓` counts continue to show uploaded and downloaded Chunks and Metadata.
|
||||
|
||||
Each directional count has its own display timer. A positive count remains visible for three seconds after its last change, then clears; a count which reaches zero clears immediately. A change in one direction does not extend the other direction's timer. The ⏩ status icon follows comparison activity independently of these timers.
|
||||
|
||||
## Customising status items with CSS snippets
|
||||
|
||||
The status bar and editor overlay use individual spans with the common `livesync-stat-` class prefix. CSS snippets can hide or style individual items. For example, this hides both directional comparison counts and the P2P summary:
|
||||
|
||||
```css
|
||||
.livesync-stat-ff-push,
|
||||
.livesync-stat-ff-pull,
|
||||
.livesync-stat-p2p {
|
||||
display: none;
|
||||
}
|
||||
```
|
||||
|
||||
The item classes include `livesync-stat-network`, `livesync-stat-sync`, `livesync-stat-sent`, `livesync-stat-push-remaining`, `livesync-stat-arrived`, `livesync-stat-pull-remaining`, `livesync-stat-ff-push`, `livesync-stat-ff-pull`, and `livesync-stat-p2p`. File-processing items use `livesync-stat-file-processing`, `livesync-stat-file-pending`, and `livesync-stat-file-batched`; queue items use `livesync-stat-queue-` followed by `replication`, `database`, `storage`, `chunk-initial`, `chunk-retrying`, `plugin-scan`, `hidden-files`, or `conflicts`. The skipped-file indicator uses `livesync-stat-file-skip`.
|
||||
|
||||
The former `livesync-stat-ff` class has been replaced by `livesync-stat-ff-push` and `livesync-stat-ff-pull`. Update existing CSS snippets to target one or both of the directional classes.
|
||||
|
||||
All items also have `livesync-stat-item`. Numeric children use `livesync-stat-value`, tabular digits, and right alignment. Each numeric item retains its widest displayed digit count during an active period, reducing movement as counts decrease. The editor overlay uses wider gaps between items, while remaining counts stay beside their transfer totals.
|
||||
|
||||
Inactive items receive `livesync-stat-inactive` instead of the HTML `hidden` attribute. The default CSS hides them immediately. They retain their last text, numeric width, and position amongst the other items so that CSS snippets can customise how they disappear. The numeric width starts afresh when an item becomes active again.
|
||||
|
||||
### Animate editor status items
|
||||
|
||||
This optional CSS snippet shrinks inactive items in the editor status line and expands them when they reappear. It applies to all status items in that line. Because inactive items remain in the DOM, their width can transition as their state changes; the initial creation of an item is not guaranteed to animate. Text changes, including on first appearance, do not necessarily animate an item's width. The `@supports` guard leaves the default, immediate hiding behaviour in place in CSS engines which do not support `calc-size()`.
|
||||
|
||||
```css
|
||||
@supports (width: calc-size(auto, size)) {
|
||||
.livesync-status-statusline > .livesync-stat-item {
|
||||
font-variant-numeric: tabular-nums;
|
||||
transition: width 0.125s;
|
||||
display: inline-block;
|
||||
width: calc-size(auto, size);
|
||||
overflow: hidden;
|
||||
white-space: nowrap;
|
||||
}
|
||||
|
||||
.livesync-status-statusline > .livesync-stat-item.livesync-stat-inactive {
|
||||
transition: width 0.46s;
|
||||
display: inline-block;
|
||||
width: calc-size(auto, size * 0);
|
||||
overflow: hidden;
|
||||
white-space: nowrap;
|
||||
margin-inline-start: 0 !important;
|
||||
}
|
||||
|
||||
@media (prefers-reduced-motion: reduce) {
|
||||
.livesync-status-statusline > .livesync-stat-item {
|
||||
transition: none;
|
||||
}
|
||||
|
||||
.livesync-status-statusline > .livesync-stat-item.livesync-stat-inactive {
|
||||
transition: none;
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
The snippet affects the editor overlay only; the status bar keeps its default appearance. See the [`calc-size()` reference](https://developer.chrome.com/docs/css-ui/animate-to-height-auto/) for the CSS feature used to calculate an item's intrinsic width.
|
||||
|
||||
See the [documentation index](../README.md) for setup and troubleshooting guides.
|
||||
Reference in new issue
Block a user