public/obsidian-livesync
1
0 Fork 0
mirror of https://github.com/vrtmrz/obsidian-livesync.git synced 2026-10-29 20:43:20 +00:00
Code Issues Packages Projects Releases Wiki Activity

Merge pull request #1292 from vrtmrz/docs/agent-guidance-review

Browse Source 
Refresh setup, recovery, and documentation guidance
This commit is contained in:
vorotamoroz authored and GitHub committed 2026-10-28 23:31:43 +09:00
2 parents db95f420b9 d550e7a200
commit a2eecce239
36 files changed
+717 -430
Download Patch File Download Diff File Expand all files Collapse all files

No files matched your search

AGENTS.md
+22
View File
@@ -2,6 +2,12 @@
When working on this repository (writing code, comments, documentation, or commits), you MUST follow these guidelines to maintain consistency.
## Documentation Navigation and Maintenance
- Use the [documentation index](docs/README.md) to select the relevant implementation reference, ADR, proposal, or historical record. The required references below still apply.
- When implementation or publication changes a recorded status, update its publication note and supporting references in the same change. Preserve the original decision date, reviewed-version baseline, and remaining follow-ups.
- Mark superseded or frozen documents at their entry point and link to the maintained reference. Keep historical evidence distinguishable from current operating instructions, and preserve existing links when reorganising documentation.
## Required Reference Files
Before making changes to documentation, user-facing text, or settings:
@@ -39,6 +45,13 @@ Always adhere to the following stylistic and spelling rules:
---
## Release Notes
- Follow the release-note scope and history rules in [devs.md](devs.md#release-notes). Structure substantive entries as the user-visible outcome, the method that produces it, and supporting conditions, defaults, limitations, or required actions. Put the outcome first and use subordinate items when the explanation needs them.
- When a pull request explains only its implementation, consult the relevant issue or design decision to establish the problem it resolves. Verify the claimed outcome against the change and its evidence. Distinguish removing a cause from reducing occurrence, detecting a problem, or improving recovery.
---
## Technical & Architecture Rules
1. **Database Structure**:
@@ -56,6 +69,15 @@ Always adhere to the following stylistic and spelling rules:
---
## Failure Assessment and Recovery
- Establish the intended behaviour from [the conflict policy](devs.md#conflict-merge-policy) and [the revision-provenance contract](docs/specs_conflict_resolution.md) before classifying a failure. A visible conflict which preserves both versions can be the correct outcome; manual selection alone does not establish a serious defect.
- Assess preservation of user edits and required revision ancestry separately from extra work, temporary synchronisation delays, or duplicate internal records. A successful automatic merge or an empty queue is not proof of correct recovery. Check the resulting contents and history, safe convergence, and behaviour after restart where applicable.
- Distinguish data still present on a device or in readable history from irretrievable loss. Verify that a recovery source contains the affected edits before treating the failure as fully recoverable. Follow [the recovery guidance](docs/recovery.md) before recommending a rebuild or reset which could discard unique work.
- Give demonstrated irreversible information loss particular weight. Describe the actual burden of safe manual conflict resolution without treating automation as a correctness requirement. If preservation, recovery, impact, or the appropriate response remains unclear, review the evidence and alternatives with the collaborating user before changing the design or accepting the risk; continue independent investigation.
---
## Development & Verification Commands
Before submitting code, you should run verification scripts locally to ensure correct syntax and function.
README.md
+54 -76
View File
@@ -1,11 +1,12 @@
# Self-hosted LiveSync
[Japanese docs](./README_ja.md) - [Chinese docs](./README_cn.md).
Self-hosted LiveSync is a community-developed synchronisation plug-in available on all Obsidian-compatible platforms. It leverages robust server solutions such as CouchDB or object storage systems (e.g., MinIO, S3, R2, etc.) to ensure reliable data synchronisation.
Additionally, it supports peer-to-peer synchronisation using WebRTC, enabling devices to exchange notes without a central data-storage server. A signalling relay is still required for peer discovery. See [How peer-to-peer synchronisation works](./docs/p2p.md).
Find setup guides, troubleshooting, technical references, design decisions, and historical material in the [documentation index](docs/README.md).
![obsidian_live_sync_demo](https://user-images.githubusercontent.com/45774780/137355323-f57a8b09-abf2-4501-836c-8cb7d2ff24a3.gif)
>[!IMPORTANT]
@@ -18,12 +19,8 @@ Additionally, it supports peer-to-peer synchronisation using WebRTC, enabling de
- Use open-source solutions for the server.
- Compatible solutions are supported.
- Support end-to-end encryption.
- Synchronise settings, snippets, themes, and plug-ins via [Customisation Sync (Beta)](docs/settings.md#6-customisation-sync-advanced) or [Hidden File Sync](docs/tips/hidden-file-sync.md).
- Enable supported, opt-in WebRTC peer-to-peer synchronisation.
- No central data-storage server is required, but a signalling relay is still required for peer discovery.
- At least one device containing the required data must be online while another device synchronises.
- Follow the [Peer-to-Peer Setup](docs/setup_p2p.md) after reviewing the [P2P communication model](docs/p2p.md).
- If you would like to see how P2P setup and connectivity behave on your devices, try the guided connection check in the experimental [WebPeer browser utility](https://vrtmrz.github.io/obsidian-livesync/webpeer/). It uses an empty Vault and does not test note synchronisation. See the [WebPeer documentation](src/apps/webpeer/README.md) for its scope and limitations.
- Synchronise settings, snippets, themes, and plug-ins via [Customisation Sync (Advanced)](docs/settings.md#6-customisation-sync-advanced) or [Hidden File Sync](docs/tips/hidden-file-sync.md).
- Synchronise peer-to-peer using WebRTC.
This plug-in may be particularly useful for researchers, engineers, and developers who need to keep their notes fully self-hosted for security reasons. It is also suitable for anyone seeking the peace of mind that comes with knowing their notes remain entirely private.
@@ -34,95 +31,76 @@ This plug-in may be particularly useful for researchers, engineers, and develope
## How to Use
### 3-minute setup - CouchDB on fly.io
Self-hosting lets us choose a server we trust and manage it ourselves. Please set up and maintain your own server, and keep it secure.
**Recommended for beginners**
<a id="setup-workflows"></a>
[![LiveSync Setup onto Fly.io SpeedRun 2024 using Google Colab](https://img.youtube.com/vi/7sa_I1832Xc/0.jpg)](https://www.youtube.com/watch?v=7sa_I1832Xc)
### What you need
1. [Set up CouchDB on fly.io](docs/setup_flyio.md)
2. Configure plug-in in [Quick Setup](docs/quick_setup.md)
| Method | What you need | Nice to have |
| --- | --- | --- |
| CouchDB | A CouchDB-compatible server. | — |
| Object Storage | An S3-compatible service or server. | — |
| Peer-to-Peer | Nothing. | Your own signalling server. |
### Setup workflows
For a CouchDB-compatible server, choose a setup guide for your environment:
Choose a synchronisation method, prepare its server where required, then follow the corresponding client setup:
- [Your own server](docs/setup_own_server.md): run CouchDB on a Raspberry Pi or another server you manage.
- [Docker](docker/README.md): deploy CouchDB with the supplied Docker stack.
- [Fly.io](docs/setup_flyio.md): manage your own CouchDB instance on hosted infrastructure.
- [IBM Cloudant (managed)](docs/setup_cloudant.md): use a database service managed by IBM.
1. CouchDB
1. Prepare the server:
- [Set up your own CouchDB server](docs/setup_own_server.md).
- [Set up CouchDB on fly.io](docs/setup_flyio.md).
2. Configure the clients by following [CouchDB Quick Setup](docs/quick_setup.md).
2. Object Storage
1. Prepare the server. A maintained MinIO server installation guide is not currently available here, so set up an S3-compatible service or server of your choice.
2. Configure the clients by following [Object Storage Setup](docs/setup_object_storage.md).
3. Peer-to-Peer
1. No central data-storage server is required. The project's public signalling relay requires no server provisioning; controlled deployments can provide another compatible relay.
2. Configure the clients by following [Peer-to-Peer Setup](docs/setup_p2p.md).
The linked guides explain the setup and any provider-specific costs.
Each workflow establishes ordinary note synchronisation on the first device, generates a Setup URI for each additional device from that working device, and verifies synchronisation in both directions.
<a id="3-minute-setup---couchdb-on-flyio"></a>
> [!TIP]
> Fly.io is no longer free. Fortunately, we can still use IBM Cloudant despite some limitations. Refer to [Set up IBM Cloudant](docs/setup_cloudant.md).
> We can also use peer-to-peer synchronisation without a central data-storage server; a signalling relay is still used for peer discovery. Alternatively, cheap object storage like Cloudflare R2 can be used for free.
> However, most importantly, we can use a server that we trust. Therefore, please set up your own server.
> CouchDB can also be run on a Raspberry Pi (please be mindful of your server's security).
For the Fly.io video walkthrough, see [automated setup](docs/setup_flyio.md#a-very-automated-setup).
P2P does not require a central data-storage server, but a signalling relay is still required for peer discovery. You can use the project's public signalling relay, so there is no server to provision. At least one device containing the required data must be online while another device synchronises. Follow the [Peer-to-Peer Setup](docs/setup_p2p.md) after reviewing the [P2P communication model](docs/p2p.md).
### Third-party managed CouchDB hosting
If you would like to see how P2P setup and connectivity behave on your devices, try the guided connection check in the experimental [WebPeer browser utility](https://vrtmrz.github.io/obsidian-livesync/webpeer/). It uses an empty Vault and does not test note synchronisation. See the [WebPeer documentation](src/apps/webpeer/README.md) for its scope and limitations.
### Start synchronising
1. Install and configure Self-hosted LiveSync on your first device using [Quick Setup](docs/quick_setup.md) for CouchDB, [Object Storage Setup](docs/setup_object_storage.md), or [Peer-to-Peer Setup](docs/setup_p2p.md).
2. Generate a Setup URI from that working device and follow the same guide to add your other devices.
3. Create test notes and confirm that they synchronise in both directions.
<a id="third-party-managed-couchdb-hosting"></a>
### Hosting services proposed by the community
> [!NOTE]
> The following is a third-party hosting option proposed by Zenith Hosting. It is not an official Self-hosted LiveSync service, and it is neither endorsed nor recommended by this project.
> The services below are operated by third parties. They are not official Self-hosted LiveSync services, and this project neither endorses nor recommends them. The providers proposed the listings in the table.
If you would rather not set up and maintain a server yourself, Zenith Hosting offers a managed CouchDB server which this plug-in can be configured to connect to: [Zenith Hosting](https://zenith.hosting/host/obsidian-livesync). As with any hosted service, your data will reside on a server operated by a third party, so please consider whether that is acceptable for your vault before using it.
If you would rather not set up and maintain a server yourself, the following providers offer managed CouchDB servers which this plug-in can be configured to connect to:
| Provider | Offering |
| --- | --- |
| [Zenith Hosting](https://zenith.hosting/host/obsidian-livesync) | Managed CouchDB hosting. |
Your synchronisation data will reside on a server operated by the provider. Consider whether that is acceptable for your Vault, and review the provider's current terms, pricing, and data-handling policies before using the service.
## Information in the Status Bar
Synchronisation status is shown in the status bar with the following icons.
Synchronisation status is shown in the status bar and inside the editor by default. The main indicators are:
- Activity Indicator
- 📲 A finite remote operation is in progress
- 🌐N Approximate remote requests currently in progress
- Status
- ⏹️ Stopped
- ⏩ Comparing CouchDB change history, including entries already held locally
- 💤 LiveSync enabled. Waiting for changes
- ⚡️ Synchronisation in progress
- ⚠ An error occurred
- Statistical Indicators
- ↑ Uploaded chunks and metadata
- ↓ Downloaded chunks and metadata
- Progress Indicators
- 📥 Unprocessed transferred items
- 📄 Working database operation
- 💾 Working write storage processes
- ⏳ Working read storage processes
- 🛫 Pending read storage processes
- 📬 Batched read storage processes
- ⚙️ Working or pending storage processes for hidden files
- 🛄 Pending initial on-demand chunk requests
- 🔁 Pending chunk retries, including retry delays
- 🔌 Working customisation items (configuration, snippets, and plug-ins)
| Indicator | Meaning |
| --- | --- |
| ⏹️ | Stopped. |
| 🌀 | Starting synchronisation. |
| 💤 | LiveSync mode is waiting for changes. |
| ⚡️ | Synchronisation is in progress. |
| ⏩ | Comparing CouchDB change history, including entries already held locally. |
| ⚠ | An error occurred. |
| ↑ / ↓ | Uploaded / downloaded Chunks and Metadata. |
| 📲 / 🌐N | A finite remote operation is in progress / approximate remote requests in progress. |
To prevent file and database corruption, please avoid closing Obsidian until all progress indicators have disappeared as much as possible (although the plug-in will attempt to resume if interrupted). This is especially important if you have deleted or renamed files.
<a id="customising-status-items-with-css-snippets"></a>
When CouchDB change history is being compared, `(⏩ 1200) ↑ 0 ↓ 0` can appear before any new documents arrive. The count is the number of candidate changes-feed entries compared during the current replication attempt, including repeated entries on retries. It is separate from downloaded Chunks and Metadata, and is neither a Vault file count nor a count of entries already present locally. It appears before the upload count, disappears after three seconds without a change, and reappears when the count changes. The ⏩ status icon follows comparison activity independently of that timer.
See the [status display guide](docs/tips/status-display.md) for detailed progress counters and [CSS customisation](docs/tips/status-display.md#customising-status-items-with-css-snippets).
### Customising status items with CSS snippets
The status bar and editor overlay use individual spans with the common `livesync-stat-` class prefix. CSS snippets can hide or style the items we need. For example, this hides the comparison count and P2P summary:
```css
.livesync-stat-ff,
.livesync-stat-p2p {
display: none;
}
```
The item classes include `livesync-stat-network`, `livesync-stat-sync`, `livesync-stat-sent`, `livesync-stat-push-remaining`, `livesync-stat-arrived`, `livesync-stat-pull-remaining`, `livesync-stat-ff`, and `livesync-stat-p2p`. File-processing items use `livesync-stat-file-processing`, `livesync-stat-file-pending`, and `livesync-stat-file-batched`; queue items use `livesync-stat-queue-` followed by `replication`, `database`, `storage`, `chunk-initial`, `chunk-retrying`, `plugin-scan`, `hidden-files`, or `conflicts`. The skipped-file indicator uses `livesync-stat-file-skip`.
All items also have `livesync-stat-item`. Numeric children use `livesync-stat-value`, tabular digits, and right alignment. Each numeric item retains its widest displayed digit count during an active period, reducing movement as counts decrease. The editor overlay uses wider gaps between items, while remaining counts stay beside their transfer totals.
Inactive items receive `livesync-stat-inactive` instead of the HTML `hidden` attribute. The default CSS hides them immediately. They retain their last text, numeric width, and position amongst the other items so that CSS snippets can customise how they disappear. The numeric width starts afresh when an item becomes active again.
Allow pending file operations to finish before closing Obsidian, especially after deleting or renaming files.
## Tips and Troubleshooting
- If you want a faster and simpler initial replication when setting up subsequent devices, see the [Fast Setup Guide](docs/tips/fast-setup.md).
devs.md
+3 -1
View File
@@ -1,5 +1,7 @@
# Self-hosted LiveSync Development Guide
Use the [documentation index](docs/README.md) to find implementation references, architectural decisions, proposals, and historical material for the area being changed.
## Project Overview
Self-hosted LiveSync is an Obsidian plugin for synchronising vaults across devices using CouchDB, MinIO/S3, or peer-to-peer WebRTC. The codebase uses a modular architecture with TypeScript, Svelte, and PouchDB.
@@ -78,7 +80,7 @@ Regression tests remain in the suite owned by the implementation under test. Plu
- **CLI E2E** (`src/apps/cli/testdeno/`): Host-independent consumer workflows. The canonical Compose P2P suite covers ordinary two-peer synchronisation, replacement of the current Replicator followed by transfer with the same peer, and explicit relay disconnection followed by paused and resumed reconnection. Its lifecycle entry point is included only in the Docker test build and does not add a public CLI command. Run `npm run test:e2e:cli` for the ordinary suite or `npm run test:e2e:cli:p2p` for P2P validation.
- **Self-hosted setup tools** (`utils/couchdb/`, `utils/setup/`, and `utils/flyio/`): Deno contract tests consume the exact locked Commonlib registry package, verify current CouchDB, Object Storage, and random-room P2P Setup URI defaults and remote profiles, and keep CouchDB administration separate from package-owned LiveSync database-version negotiation. `unit-ci` also provisions a real temporary CouchDB database and verifies its version document against the installed Commonlib package. Run `npm run test:setup-tools` for the local contract gate.
- **Real Obsidian E2E** (`test/e2e-obsidian/`): Local-first scripts that launch real Obsidian with temporary vaults and the built Self-hosted LiveSync plug-in. Use these for boot-up sequence, vault reflection, RedFlag flows, Fast Setup (Simple Fetch), settings dialogues, restart-sensitive workflows, Object Storage regressions, and other behaviour that depends on Obsidian itself. Run focused scripts such as `npm run test:e2e:obsidian:two-vault-sync`, or use `npm run test:e2e:obsidian:local-suite:services` to run the broader local suite with CouchDB and RustFS fixtures managed by the wrapper.
- **Real Obsidian E2E** (`test/e2e-obsidian/`): Local-first scripts that launch real Obsidian with temporary vaults and the built Self-hosted LiveSync plug-in. Use these for boot-up sequence, vault reflection, RedFlag flows, Fast Setup (Simple Fetch), settings dialogues, restart-sensitive workflows, Object Storage regressions, and other behaviour that depends on Obsidian itself. After source changes, run the build-owning wrapper, for example `npm run test:e2e:obsidian:focused -- two-vault-sync`, with the required fixture already running. Direct scenario scripts reuse the existing bundle and are suitable for reruns only while that build remains unchanged. Use `npm run test:e2e:obsidian:local-suite:services` to run the broader local suite with CouchDB and RustFS fixtures managed by the wrapper.
- **Docker Services**: Service-backed tests use CouchDB and RustFS (S3). Canonical P2P validation owns its relay through the CLI Compose runner:
docker/.env.example
+4 -4
View File
@@ -34,9 +34,9 @@ ACME_EMAIL=you@yourdomain.com
# PROFILE: tailscale (--profile tailscale)
# =============================================================================
# Tailscale OAuth key (not a regular auth key — must be OAuth for persistent use)
# Generate at: https://login.tailscale.com/admin/settings/oauth
# Scopes needed: devices:write
# Tailscale authentication key for this container.
# OAuth clients require their own scopes and tag configuration; see:
# https://tailscale.com/docs/features/containers/docker/docker-params#ts_authkey
TS_AUTHKEY=tskey-auth-xxxxxxxxxxxxxxxx-xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
# Hostname this node will have on your tailnet (becomes <hostname>.<tailnet>.ts.net)
@@ -47,6 +47,6 @@ TS_HOSTNAME=livesync
# =============================================================================
# Tunnel token from Cloudflare Zero Trust dashboard
# Create at: https://one.dash.cloudflare.com/ → Networks → Tunnels → Create tunnel
# Current setup guide: https://developers.cloudflare.com/tunnel/get-started/
# Copy the token from the "Install connector" step
CF_TUNNEL_TOKEN=eyJhIjoixxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
docker/README.md
+127 -85
View File
@@ -61,11 +61,11 @@ docker compose --profile cloudflare up -d
```bash
# Should return {"status":"ok"}
curl -u admin:yourpassword http://localhost:5984/_up
curl -u '<COUCHDB_USER>:<COUCHDB_PASSWORD>' http://localhost:5984/_up
# Check CORS headers
curl -v -H "Origin: app://obsidian.md" \
-u admin:yourpassword \
-u '<COUCHDB_USER>:<COUCHDB_PASSWORD>' \
http://localhost:5984/
```
@@ -75,7 +75,7 @@ In the Obsidian plugin settings (**Self-hosted LiveSync**):
| Field | Value |
|---|---|
| URI | `https://your-domain-or-ts-hostname:5984` (or `http://localhost:5984` for LAN-only) |
| URI | `https://your-domain-or-ts-hostname` for an HTTPS profile, or `http://localhost:5984` for local HTTP |
| Username | value of `COUCHDB_USER` |
| Password | value of `COUCHDB_PASSWORD` |
| Database name | value of `COUCHDB_DATABASE` (default: `obsidiannotes`) |
@@ -85,10 +85,13 @@ In the Obsidian plugin settings (**Self-hosted LiveSync**):
## Profile Details
### Default (no profile) — LAN / Localhost only
<a id="default-no-profile--lan--localhost-only"></a>
CouchDB is exposed on `http://localhost:5984` (or LAN IP).
**Desktop Obsidian works over HTTP.** Mobile Obsidian requires HTTPS — use a tunnel profile.
### Default (no profile) — Host HTTP access
The Compose file publishes CouchDB on host port 5984 by default, on all host interfaces. Its reachability depends on the host firewall and network; selecting an HTTPS profile does not close that direct HTTP port. Restrict the host port to the intended network, bind it to loopback, or remove the mapping when all access uses the proxy.
Desktop Obsidian can use HTTP. Use HTTPS for mobile clients and connections outside a trusted local network. The supplied HTTPS profiles use port 443, so their client URL normally has no `:5984` suffix.
### `--profile caddy` — Public Domain + Auto TLS
@@ -104,13 +107,14 @@ ACME_EMAIL=you@example.com
Caddy automatically issues a Let's Encrypt certificate. No manual cert management.
### `--profile tailscale` — No Domain Required ✅ Recommended for privacy
<a id="--profile-tailscale--no-domain-required--recommended-for-privacy"></a>
### `--profile tailscale` — Tailnet or public HTTPS
**Requires**:
- Free [Tailscale account](https://login.tailscale.com/)
- Install the Tailscale app on all your Obsidian devices
- Generate an **OAuth key** at: https://login.tailscale.com/admin/settings/oauth
(Scopes: `devices:write`)
- A [Tailscale account](https://login.tailscale.com/).
- The Tailscale app on devices using private tailnet access.
- An authentication key for `TS_AUTHKEY`. Follow the [Docker authentication parameters](https://tailscale.com/docs/features/containers/docker/docker-params#ts_authkey) if using an OAuth client instead.
**Set in `.env`**:
```
@@ -119,33 +123,29 @@ TS_HOSTNAME=livesync
```
**Two sub-modes**:
- **VPN mode** (default): CouchDB accessible only to devices on your Tailnet at
`https://livesync.<tailnet>.ts.net` — completely private
- **Funnel mode**: public HTTPS at `https://livesync.<tailnet>.ts.net` — no domain purchase
Enable in your [Tailscale ACL](https://login.tailscale.com/admin/acls):
```json
"nodeAttrs": [{"target": ["tag:container"], "attr": ["funnel"]}]
```
> **Note on Windows Docker Desktop**: If `/dev/net/tun` is unavailable, add `TS_USERSPACE=true`
> to the tailscale service environment in `docker-compose.yml`.
- **Private tailnet access (VPN / Serve)**: choose this for access from your own devices on your tailnet, subject to its access rules. Set `AllowFunnel` to `false` in `config/ts-serve.json`, and use the Tailscale app on those devices.
- **Public HTTPS (Funnel)**: choose this when devices need to connect over the Internet without joining your tailnet. Keep `AllowFunnel` set to `true`, and allow Funnel for this server in your tailnet's access controls.
Both modes use a Tailscale HTTPS name, such as `https://livesync.<tailnet>.ts.net`, so you do not need to buy a domain. Use the actual DNS name assigned to your server in Tailscale. See the [Tailscale Docker guide](https://tailscale.com/blog/docker-tailscale-guide) for HTTPS and Funnel prerequisites.
Choose the mode before starting the profile. The bundled `config/ts-serve.json` sets `AllowFunnel` to `true`, requesting public HTTPS when the tailnet permits Funnel; change that entry to `false` to keep the Tailscale endpoint private. After starting or restarting the Tailscale service, check the effective configuration with `docker exec livesync-tailscale tailscale serve status`. The separate host HTTP port described above is unaffected by this choice.
The bundled service expects `/dev/net/tun`. On hosts without that device, follow Tailscale's userspace-networking instructions and adjust the device mount as well as `TS_USERSPACE`; changing the environment variable alone leaves the required bind mount in place.
### `--profile cloudflare` — Cloudflare Tunnel
**Requires**:
- Free [Cloudflare account](https://www.cloudflare.com/)
- A domain managed by Cloudflare DNS (can transfer existing domain for free)
- Cloudflare Zero Trust account (free)
- A [Cloudflare account](https://www.cloudflare.com/).
- A domain whose DNS is managed by Cloudflare.
#### Step 1: Create a Cloudflare Tunnel
1. Log in to [Cloudflare Zero Trust](https://one.dash.cloudflare.com/)
2. Navigate to **Networks → Tunnels**
3. Click **Create a tunnel**
4. Choose **Cloudflared** as tunnel type
5. Name your tunnel (e.g., `obsidian-livesync`)
6. Click **Save tunnel**
7. **Copy the tunnel token** — it looks like `eyJhIjoiZX...` (very long, ~400 characters)
1. Open the [Cloudflare dashboard](https://dash.cloudflare.com/) and go to **Networking → Tunnels**.
2. Select **Create Tunnel**, give it a name such as `obsidian-livesync`, and create it.
3. On the connector setup screen, copy the installation command into a text editor without running it. Extract only the tunnel token, the string beginning with `eyJ`, and put it in `CF_TUNNEL_TOKEN` in the next step. Keep the token private.
For an existing tunnel, select it and use **Add a replica** to display the installation command again. This Compose profile starts the connector in step 4. Until then, the dashboard can show the tunnel as disconnected. See the official [tunnel setup](https://developers.cloudflare.com/tunnel/get-started/) and [token instructions](https://developers.cloudflare.com/tunnel/reference/tunnel-tokens/) for the current screens.
#### Step 2: Configure Environment
@@ -157,27 +157,14 @@ COUCHDB_DOMAIN=sync.yourdomain.com # Must be a domain managed by Cloudflare
#### Step 3: Add Public Hostname Route
🚨 **CRITICAL**: Token-based tunnels ignore the local `cloudflared.yml` config file. All routing is controlled from the dashboard.
Configure the route in the Cloudflare dashboard for this remotely managed tunnel. The local `config/cloudflared.yml` is not the routing source for the token-managed profile.
Back in the Zero Trust dashboard, **in the same tunnel creation flow** (or edit your tunnel later):
1. Open the tunnel under **Networking → Tunnels**.
2. Under **Routes**, add a **Published application**.
3. Choose the public hostname, for example `sync.yourdomain.com`.
4. Set the service URL to `http://couchdb:5984` and save the route.
1. Go to the **Public Hostname** tab
2. Click **Add a public hostname**
3. Configure:
- **Subdomain**: `sync` (or your preferred subdomain)
- **Domain**: Select your Cloudflare domain from dropdown
- **Type**: `HTTP`
- **URL**: `couchdb:5984` ← **Do NOT use `localhost`!**
**Why `couchdb:5984` not `localhost:5984`?**
- The `cloudflared` container runs inside Docker on the same network as `couchdb`
- Docker's internal DNS resolves `couchdb` to the correct container
- Using `localhost` would look inside the `cloudflared` container (nothing there)
4. Under **Additional application settings** (expand):
- **No TLS Verify**: Leave **OFF** (CouchDB uses plain HTTP internally, that's fine)
- Leave other settings at defaults
5. Click **Save hostname**
The `cloudflared` and `couchdb` services share a Docker network. The service name reaches CouchDB; `localhost` would address the `cloudflared` container itself. The local connection is HTTP, while the public client URL is HTTPS. Keep TLS certificate verification enabled for any HTTPS origin you configure later. The [current setup guide](https://developers.cloudflare.com/tunnel/get-started/) describes the dashboard flow.
#### Step 4: Start the Stack
@@ -203,7 +190,7 @@ You should see:
curl -I https://sync.yourdomain.com
# Should return {"couchdb":"Welcome",...}
curl -u admin:yourpassword https://sync.yourdomain.com
curl -u '<COUCHDB_USER>:<COUCHDB_PASSWORD>' https://sync.yourdomain.com
```
If you get **404**, see Troubleshooting below.
@@ -220,9 +207,7 @@ In Obsidian → Settings → **Self-hosted LiveSync**:
| Database name | value of `COUCHDB_DATABASE` from `.env` (default: `obsidiannotes`) |
| End-to-end passphrase | *Choose your own* — never stored server-side |
Under **Remote Database Configuration → Advanced**:
- Enable: ✅ **Use Request API to avoid inevitable CORS problem**
(See "Known Issue" below for why this is critical)
If browser requests encounter a CORS failure, consult [Use Request API](../docs/settings.md#use-request-api-to-avoid-inevitable-cors-problem). That setting selects Obsidian's HTTP request adapter. It does not select the synchronisation mode or remove Cloudflare's proxy limits.
---
@@ -241,13 +226,11 @@ docker logs livesync-cloudflared --tail 20
**Fix**: The public hostname rule is missing or incorrect.
1. Go to Zero Trust → Networks → Tunnels → your tunnel → **Edit**
2. Click **Public Hostname** tab
3. Verify a hostname exists with:
- Service Type: `HTTP`
- URL: `couchdb:5984` (NOT `localhost:5984`)
4. If no hostname exists, add it (see Step 3 above)
5. Wait 30 seconds for changes to propagate, then test again
1. Open the tunnel's published application routes in the Cloudflare dashboard.
2. Verify that the requested hostname maps to `http://couchdb:5984`.
3. Add or correct the route if required, then retry after the configuration has propagated.
A connected tunnel alone does not establish that the hostname route or origin is correct.
**Problem: Connection immediately closes / 502 Bad Gateway**
@@ -258,10 +241,10 @@ docker ps --filter "name=livesync-couchdb"
# Status should be: Up (healthy)
docker inspect livesync-couchdb -f '{{.NetworkSettings.Networks}}'
# Should show: livesync-net
# Both services should list the same Compose network.
docker inspect livesync-cloudflared -f '{{.NetworkSettings.Networks}}'
# Should also show: livesync-net
# Compare with the CouchDB network above.
```
**Fix**: If CouchDB is unhealthy, check logs:
@@ -271,41 +254,100 @@ docker logs livesync-couchdb --tail 50
**Problem: 524 Timeout Errors During Sync**
**Root cause**: Cloudflare's proxy has a **100-second idle timeout**. CouchDB's replication protocol uses long-polling on the `_changes` feed, which can idle for longer during quiet periods.
A `524` can indicate that the origin did not respond or accept request data within Cloudflare's proxy timeout. Inspect the failing request, CouchDB response times, and connector logs before selecting a remedy. See [Cloudflare's current 524 guidance](https://developers.cloudflare.com/support/troubleshooting/http-status-codes/cloudflare-5xx-errors/error-524/) for the applicable limits and diagnostic steps.
**Fix**: Switch to short-polling mode in the Obsidian plugin:
1. Obsidian → Settings → Self-hosted LiveSync
2. **Remote Database Configuration → Advanced**
3. Enable: ✅ **Use Request API to avoid inevitable CORS problem**
4. Save and restart sync
This keeps all requests under 100 seconds.
**Alternative**: Use Tailscale or Caddy profiles instead — neither has aggressive timeouts.
**Use Request API** changes the HTTP adapter; it does not turn LiveSync mode into short polling or guarantee that a request completes within a proxy timeout. Review the actual synchronisation mode and [heartbeat/timeout setting](../docs/settings.md#use-timeouts-instead-of-heartbeats), then verify transfers on the intended devices. If the required traffic does not fit the proxy limits, consider an appropriately secured direct HTTPS route through Caddy or private Tailscale access.
---
## Data & Backup
All vault data lives in the `couchdb-data` Docker named volume.
The server database files live in the `couchdb-data` Docker named volume. Keep Vault backups, encryption passphrases, and ID recovery information separately. Back up the Compose configuration and CouchDB configuration as well; the database volume alone does not contain all of them.
The following example creates an archive while CouchDB is stopped. Verify the volume name before running it, especially if you have overridden the Compose project name:
```bash
# Backup
docker run --rm -v obsidian-livesync_couchdb-data:/data \
-v $(pwd)/backup:/backup alpine \
tar czf /backup/couchdb-backup-$(date +%Y%m%d).tar.gz -C /data .
# Restore
docker run --rm -v obsidian-livesync_couchdb-data:/data \
-v $(pwd)/backup:/backup alpine \
tar xzf /backup/couchdb-backup-20260218.tar.gz -C /data
docker volume inspect obsidian-livesync_couchdb-data
mkdir -p backup
docker compose stop couchdb
docker run --rm --network none -v obsidian-livesync_couchdb-data:/data:ro \
-v "$(pwd)/backup:/backup" alpine \
tar czf "/backup/couchdb-backup-$(date +%Y%m%d-%H%M%S).tar.gz" -C /data .
docker compose start couchdb
```
### Restore a backup
Restore into a new volume so that the current database remains available. The following Bash commands run from the `docker/` directory. Use a trusted archive from the backup above, and the same CouchDB image version used to create it. Keep the saved CouchDB configuration available too. Run each block only after the previous step succeeds.
First, choose the archive and create a new recovery volume. Docker generates an unused volume name:
```bash
backupArchive='<BACKUP ARCHIVE FILENAME>'
recoveryImage='<COUCHDB IMAGE TAG OR DIGEST FROM THE BACKUP>'
recoveryVolume=$(docker volume create) &&
docker run --rm --network none \
--mount "type=volume,src=$recoveryVolume,dst=/data" \
--mount "type=bind,src=$(pwd)/backup,dst=/backup,readonly" \
alpine:3.22 tar xzf "/backup/$backupArchive" -C /data
```
If extraction fails, stop here. The new recovery volume may be empty or incomplete; remove it with `docker volume rm "$recoveryVolume"` before retrying. The original database volume is unaffected.
Start a temporary CouchDB with the restored data. It has no external network connection or published ports, so synchronising clients cannot reach it:
```bash
recoveryUser='<COUCHDB USERNAME>'
recoveryPassword='<COUCHDB PASSWORD>'
recoveryContainer=$(docker run -d --network none \
-e COUCHDB_USER="$recoveryUser" \
-e COUCHDB_PASSWORD="$recoveryPassword" \
--mount "type=volume,src=$recoveryVolume,dst=/opt/couchdb/data" \
"$recoveryImage")
```
Wait for CouchDB to start, then inspect its health, database list, and your restored database. Replace the database name with the one you backed up:
```bash
recoveryDatabase='<DATABASE NAME>'
docker exec "$recoveryContainer" curl -fsS \
-u "$recoveryUser:$recoveryPassword" http://127.0.0.1:5984/_up
docker exec "$recoveryContainer" curl -fsS \
-u "$recoveryUser:$recoveryPassword" http://127.0.0.1:5984/_all_dbs
docker exec "$recoveryContainer" curl -fsS \
-u "$recoveryUser:$recoveryPassword" "http://127.0.0.1:5984/$recoveryDatabase"
docker exec "$recoveryContainer" curl -fsS \
-u "$recoveryUser:$recoveryPassword" \
"http://127.0.0.1:5984/$recoveryDatabase/_all_docs?limit=5"
```
Compare the database names, document counts, and expected document IDs with your backup. These checks establish that CouchDB can read the restored files; they do not establish that every required note is present. If a check fails, inspect `docker logs "$recoveryContainer"` and keep the original volume in place.
Once you have verified the restored database, stop and remove only the temporary container, and note the recovery volume name:
```bash
docker stop "$recoveryContainer" && docker rm "$recoveryContainer"
docker volume inspect "$recoveryVolume" --format '{{.Name}}'
```
To use the restored copy, first [choose the authoritative copy](../docs/recovery.md#first-choose-the-authoritative-copy) and suspend synchronisation on your devices. Stop the stack with `docker compose down`, using the same profile options as usual and without `-v`. Save a copy of `docker-compose.yml`, then replace only its `couchdb-data` volume declaration with the following, using the printed recovery volume name. Keep the other volume declarations:
```yaml
volumes:
couchdb-data:
external: true
name: <RECOVERY VOLUME NAME>
```
Use the same CouchDB image version in the `couchdb` service, restore any saved configuration changes, and start the stack with your usual `docker compose up -d` profile command. The original volume and archive remain available. Verify the required notes from a separate test Vault. When the restored remote is your chosen source of truth, follow [Reset Synchronisation on This Device](../docs/recovery.md#reset-synchronisation-on-this-device) on your existing devices before resuming them, preserving any newer local work first.
See [Backing up CouchDB](https://docs.couchdb.org/en/stable/maintenance/backups.html) for database, configuration, and replication-based backup options.
---
## Security Notes
- CouchDB requires authentication for **all** requests (configured by `livesync.ini`)
- The init service enables authenticated CouchDB access through the REST API. Verify that protected database endpoints reject unauthenticated requests before exposing the service.
- Enable **End-to-End Encryption** passphrase in the Obsidian plugin — vault data is
encrypted before it ever leaves your device
- The init container runs once and exits — it has no persistent access
@@ -344,8 +386,8 @@ open http://localhost:5984/_utils
| Init container keeps restarting | CouchDB not healthy yet — wait 30s, check `docker compose logs couchdb` |
| `curl: (52) Empty reply` | CouchDB not fully started — the healthcheck should gate this |
| Mobile can't connect | Needs HTTPS — use tailscale or caddy profile |
| 524 errors with Cloudflare | Enable "Use Request API" toggle in Obsidian plugin |
| `Permission denied` on volumes | Run `docker compose down -v` and retry — first-run volume ownership issue |
| 524 errors with Cloudflare | Inspect the failing request and origin response time; follow the timeout guidance above. |
| `Permission denied` on volumes | Preserve the volume, inspect the CouchDB logs and mount ownership, and correct the specific permission problem. `docker compose down -v` deletes the database volume and is not a repair step. |
| CORS errors in browser | Confirm CouchDB headers: `curl -v -H "Origin: app://obsidian.md" http://localhost:5984/` |
| CouchDB exits immediately, zero logs (Windows) | **Do not add `:ro`** to the `livesync.ini` volume mount. CouchDB's entrypoint runs `chmod 0644` on all files in `/opt/couchdb/etc` — read-only bind mounts cause a silent EPERM crash on Docker Desktop for Windows (WSL2). The compose file is already correct; do not modify it. |
| Settings in `livesync.ini` seem ignored | Settings requiring restart (e.g. bind_address) load at start. Runtime-only settings (require_valid_user, enable_cors) are set by the init container via REST API and take effect immediately without restart. |
docker/config/cloudflared.yml
+7 -8
View File
@@ -2,17 +2,16 @@
# =============================================================================
#
# Prerequisites:
# 1. Create a tunnel in Cloudflare Zero Trust → Networks → Tunnels
# 1. Create a remotely managed tunnel using the current Cloudflare dashboard guide.
# 2. Copy the tunnel token to CF_TUNNEL_TOKEN in your .env
# 3. Add a public hostname in the tunnel config:
# 3. Add a published application route in the dashboard:
# Hostname : couchdb.yourdomain.com (or whatever you set COUCHDB_DOMAIN to)
# Service : http://couchdb:5984
#
# Known issue: Cloudflare's 100-second proxy timeout can interrupt CouchDB's
# long-polling replication change feed, causing 524 errors.
# MITIGATION: In the Obsidian plugin settings, enable:
# "Use Request API to avoid inevitable CORS problem"
# This switches from long-poll to short-poll mode.
# The token-managed Compose profile uses dashboard routing. The local ingress
# example below describes locally managed routing and is not that profile's
# source of routes. Diagnose 524 errors using current Cloudflare guidance.
# The Obsidian Request API setting changes the HTTP adapter, not the sync mode.
#
# =============================================================================
@@ -23,7 +22,7 @@ ingress:
- hostname: ${COUCHDB_DOMAIN}
service: http://couchdb:5984
originRequest:
# Increase timeouts for CouchDB replication streams
# Origin connection settings for this locally managed example
connectTimeout: 30s
keepAliveTimeout: 90s
keepAliveConnections: 100
docker/config/livesync.ini
+4 -5
View File
@@ -1,10 +1,9 @@
; CouchDB local configuration for Self-hosted LiveSync
; This file is volume-mounted into /opt/couchdb/etc/local.d/livesync.ini
; This file is volume-mounted into /opt/couchdb/etc/local.ini
;
; IMPORTANT: Do NOT set require_valid_user here.
; CouchDB needs to start without auth to complete its first-run cluster setup
; (_users, _replicator databases must be created first).
; The couchdb-init service applies auth lockdown via REST API after first-run.
; The couchdb-init service configures require_valid_user and CORS through the
; REST API during provisioning. Verify that the init service completes before
; exposing the database to clients.
[couchdb]
; Max size per document (50MB). Large enough for binary attachments.
docker/coturn/README.md
+1 -1
View File
@@ -51,7 +51,7 @@ Enter both client paths in the P2P profile's TURN server list:
turn:turn.example.com:3478?transport=udp,turn:turn.example.com:3478?transport=tcp
```
Use `TURN_USERNAME` and `TURN_PASSWORD` as the TURN username and credential. Keep the normal `Automatic` ICE policy unless a future LiveSync release offers `TURN relay only` and the direct path needs to be excluded deliberately.
Use `TURN_USERNAME` and `TURN_PASSWORD` as the TURN username and credential. Keep the normal `Automatic` connection path unless the direct path needs to be excluded deliberately. Select `TURN relay only` under `Connection compatibility` to require the configured TURN route.
Both synchronising devices must be able to reach the server. Prove an explicit two-way `Replicate now` round trip on the intended networks before relying on the configuration.
docker/docker-compose.yml
+4 -4
View File
@@ -2,7 +2,7 @@
# =============================================================================
# PROFILES
# --------
# (default) CouchDB only — LAN/localhost access, no TLS
# (default) CouchDB only — host HTTP port, no TLS
# Suitable for desktop-only use or testing.
#
# --profile caddy CouchDB + Caddy reverse proxy
@@ -14,7 +14,7 @@
#
# --profile cloudflare CouchDB + cloudflared tunnel daemon
# Free public HTTPS via Cloudflare. Needs a CF account + tunnel token.
# NOTE: Enable "Use Request API" in the Obsidian plugin to avoid 524 timeouts.
# Review proxy timeouts separately from the Obsidian HTTP adapter setting.
#
# QUICK START (local test):
# cp .env.example .env && edit .env
@@ -147,8 +147,8 @@ services:
# PROFILE: cloudflare — cloudflared tunnel daemon
# Requirements: Cloudflare account, tunnel token from CF Zero Trust dashboard
# Usage: docker compose --profile cloudflare up -d
# NOTE: Enable "Use Request API" toggle in the Obsidian LiveSync plugin settings
# to avoid Cloudflare's 100-second proxy timeout (524 errors).
# NOTE: The Request API setting changes the HTTP adapter, not the sync mode.
# Diagnose proxy timeouts using the Docker README and current provider limits.
# ---------------------------------------------------------------------------
cloudflared:
image: cloudflare/cloudflared:latest
docs/README.md
+94
View File
@@ -0,0 +1,94 @@
# Documentation index
Choose a document by the task and its status. User guides describe supported operations; implementation references describe contracts and their limits; ADRs record why a decision was made. Proposals and historical designs are grouped separately below.
A document's original date and reviewed-version fields describe its baseline. Read its implementation or publication note before treating that baseline as the current release. A published feature does not by itself establish that every proposal or validation step in its design is complete.
## Setup and everyday use
| Task | Start here |
|---|---|
| Get started | [What you need](../README.md#what-you-need), then [Start synchronising](../README.md#start-synchronising) |
| Choose a CouchDB setup | [Own server](setup_own_server.md), [Docker stack](../docker/README.md), [Fly.io](setup_flyio.md), or [IBM Cloudant (managed)](setup_cloudant.md) |
| Configure CouchDB clients | [Quick setup](quick_setup.md) |
| Find hosting services proposed by the community | [Third-party hosting listings](../README.md#hosting-services-proposed-by-the-community) |
| Set up S3-compatible storage | [Object Storage Setup](setup_object_storage.md) |
| Set up peer-to-peer synchronisation | [P2P communication model](p2p.md), then [Peer-to-Peer Setup](setup_p2p.md) |
| Diagnose P2P connectivity | [Browser-assisted check](../src/apps/webpeer/README.md#p2p-connection-check), [network tips](tips/p2p-sync-tips.md), and [self-hosted TURN](../docker/coturn/README.md) |
| Understand a setting or command | [Settings reference](settings.md) |
| Read or customise synchronisation status | [Status indicators and CSS snippets](tips/status-display.md) |
| Investigate or recover from a problem | [Troubleshooting](troubleshooting.md), then the relevant [recovery procedure](recovery.md) |
| Prepare an issue report | [Issue reporting](to_issue_reporting.md) |
The project overview links to Japanese and Chinese documentation.
## Implementation and technical references
Use these references for the area being changed, together with its relevant ADR. Check the document's stated scope and validation limits before relying on a guarantee.
| Area | References |
|---|---|
| Overall synchronisation and terminology | [Technical overview](tech_info.md), [glossary](glossary.md), and [terminology conventions](terms.md) |
| Metadata, Chunks, and database representations | [Data structures](datastructure.md) |
| Conflict resolution, provenance, and history restoration | [Conflict and provenance contract](specs_conflict_resolution.md), [Document History restoration](design_docs/document_history_revision_restoration.md), and [local save completion](adr/2026_10_local_file_save_completion_and_batching.md) |
| Replicator and host boundaries | [Replicator architecture snapshot](design_docs/replicator_architecture.md) and [service feature and legacy Module boundaries](design_docs/service_feature_and_legacy_module_boundaries.md) |
| Chunk retrieval and maintenance | [Chunk retrieval and waiting](design_docs/chunk_retrieval_and_waiting.md), [Garbage Collection V3](specs_garbage_collection.md), and [Data Compression](specs_data_compression.md) |
| Document identity and validation | [Unicode path identity](design_docs/unicode_path_identity.md), [configurable ID derivation](design_docs/configurable_id_derivation.md), [Metadata ID validation and repair](design_docs/metadata_document_id_validation_and_repair.md), and [path component compatibility](design_docs/path_component_length_compatibility.md) |
| Settings compatibility and profiles | [Tweak compatibility and recovery](design_docs/tweak_compatibility.md) and [remote configuration management](design_docs_of_remote_configurations.md) |
| Encryption and sharing | [Metadata encryption](design_docs/internal_metadata_encryption.md) and [time-bound Setup URIs](design_docs/time_bound_setup_uri.md) |
| P2P connection preparation | [TURN connection settings](design_docs/renewable_turn_credentials.md) |
| Journal storage internals | [Journal Replicator 2nd Edition](design_docs_of_journalsync_2nd.md) |
The installed `@vrtmrz/livesync-commonlib` package is authoritative for shared runtime contracts. Use the exact dependency selected by [package.json](../package.json) and its lockfile when checking behaviour. The [Commonlib documentation](https://github.com/vrtmrz/livesync-commonlib/tree/main/docs) is maintained in that repository; this index does not replace its package contracts.
## Architectural decisions
Read the decisions relevant to the change, rather than treating the whole ADR directory as a prerequisite. A decision's acceptance, its implementation, its release, and its remaining verification work are separate facts.
| Decision area | Records |
|---|---|
| Shared package boundary | [Common library package boundary](adr/2026_07_common_library_package_boundary.md) |
| Replicator capabilities and migration | [Core contract](adr/2026_08_replicator_capabilities_01_core_contract.md), [P2P service lifecycle](adr/2026_08_replicator_capabilities_02_p2p_service_lifecycle.md), and [migration plan](adr/2026_08_replicator_capabilities_03_migration_plan.md) |
| Remote resources and Fast Setup | [CouchDB connection ownership](adr/2026_08_couchdb_remote_connection_ownership.md), [Fast Fetch eligibility](adr/2026_08_fast_fetch_transport_eligibility.md), and [persistence and completion](adr/2026_08_fast_fetch_persistence_and_completion.md) |
| P2P lifecycle and setup | [Transport lifecycle](adr/2026_07_p2p_transport_lifecycle.md), [connection check](adr/2026_07_p2p_connection_check.md), [profile-aware onboarding](adr/2026_07_multiple_remote_onboarding.md), and [compatibility controls](adr/2026_08_p2p_transport_compatibility.md) |
| Settings interface integration | [Declarative settings adapter](adr/2026_08_declarative_settings_adapter.md) |
| Saving and path identity | [Local file save completion and batching](adr/2026_10_local_file_save_completion_and_batching.md) and [Unicode path identity](adr/2026_10_unicode_path_identity.md) |
| Activity and Chunk arrival | [Bounded remote activity](adr/2026_07_bounded_remote_activity.md) and [Chunk arrival quiescence](adr/2026_07_chunk_arrival_quiescence.md) |
| Compatibility and validation | [Release notes and database compatibility gates](adr/2026_07_release_notes_and_database_compatibility.md) and [real Obsidian E2E](adr/2026_06_real_obsidian_e2e.md) |
## Proposals and incomplete explanations
These records are useful for design work. Their proposed representation or behaviour is not a current runtime contract.
| Document | Recorded state |
|---|---|
| [Tiered Chunk Storage with Live Compaction](design_docs/tired_chunk_pack.md) | In design |
| [Tiered Chunk Storage for Bucket Sync](design_docs/tired_chunk_pack_bucket.md) | In design |
| [Design intent for Metadata and Chunks](design_docs/intention_of_chunks.md) | Incomplete explanation; use the data-structure reference for current representations |
| [Feature maturity for 1.0](adr/2026_07_feature_maturity_for_1_0.md) | Proposed; publication of 1.0 alone does not establish formal acceptance of this record |
## Historical and retired designs
These documents preserve earlier motivations, approaches, or results. Follow their maintained replacement for current operations.
| Historical material | Current starting point |
|---|---|
| [Original Tweak-sharing proposal](design_docs_of_sharing_tweak_value.md) | [Tweak compatibility and recovery](design_docs/tweak_compatibility.md) |
| [First Journal Sync design](design_docs_of_journalsync.md) | [Journal Replicator 2nd Edition](design_docs_of_journalsync_2nd.md) |
| [Eden design and operational results](design_docs_of_keep_newborn_chunks.md#results-from-actual-operation) | The document records its deprecated, frozen status; use [Data structures](datastructure.md) for representations still accepted by current code |
| [Withdrawn Chunk Aggregation by Prefix](design_docs/chunk_aggregation_by_prefix.md) | Retained as a withdrawn proposal, not an implemented storage layout |
Dated measurements, including the [Data Compression benchmark](specs_data_compression.md#reproducible-benchmark), remain evidence for their recorded revision and environment. A current rerun is a separate measurement.
## Development, validation, and release history
- [Development guide](../devs.md): build, module ownership, and release workflow.
- [Contributing](../CONTRIBUTING.md): contribution and documentation checks.
- Independent applications: [CLI](../src/apps/cli/README.md), [WebApp](../src/apps/webapp/README.md), and [WebPeer](../src/apps/webpeer/README.md).
- [Setup utilities](../utils/readme.md): provisioning and Setup URI generation.
- [Review Harness](review_harness.md): bounded real-device observations and performance measurements.
- [Test procedures](../test/README.md): choose the suite which owns the boundary being changed.
- [Change notes](../updates.md): current release and unreleased changes.
- Published release archives: [1.0](releases/1.0.md), [1.0 previews](releases/1.0-previews.md), [0.25](releases/0.25.md), and [earlier releases](releases/legacy.md).
When a feature ships or a design is replaced, update its implementation note and this index if its role changes. Preserve the original decision and measurement history, state the maintained replacement, and keep existing links working.
docs/adr/2026_08_p2p_transport_compatibility.md
+21 -11
View File
@@ -2,7 +2,10 @@
## Status
Accepted — the user-facing controls will be introduced in stages. This record defines their boundaries before Commonlib settings and LiveSync interfaces are changed.
Accepted. The P2P message-size presets and connection-path choice are present
in current Self-hosted LiveSync 1.0.42 with Commonlib 0.2.2. The proposed
`Test TURN server` action and concise route diagnostics in the Obsidian
interface remain future work.
## Context
@@ -11,10 +14,12 @@ WebRTC connectivity depends on both devices, their browsers or embedded WebViews
The current P2P transport has several relevant properties:
- Trystero supplies the Nostr signalling strategy and the browser-owned WebRTC connection.
- Commonlib limits one RPC wire payload to 15,360 bytes so it remains below Trystero's own action-chunk boundary.
- Trystero supplies ordinary STUN servers and accepts an optional TURN server list with one username and credential.
- ICE chooses a direct, server-reflexive, or TURN-relayed path automatically.
- Commonlib can collect raw WebRTC statistics, but LiveSync does not yet present the selected candidate route in a concise diagnostic result.
- Commonlib defaults to a 15,360-byte RPC wire payload limit, below Trystero's own action-chunk boundary; the presets below can lower it.
- Manual TURN settings use one URL list, username, and credential. Managed preparation can supply multiple ICE server entries with their own credentials.
- ICE chooses a direct, server-reflexive, or TURN-relayed path automatically unless the profile selects `TURN relay only`.
- Commonlib can collect raw WebRTC statistics. The CLI reports the selected
candidate route, but the Obsidian interface does not yet present it in a
concise diagnostic result.
Issue reports suggest that reducing the application payload may improve some mobile and constrained-network paths. A VPN such as Tailscale may also turn an unreliable route into a reliable one. These observations are consistent with NAT, path-MTU, fragmentation, or intermediary behaviour, but they do not prove one universal cause. Browser WebRTC implementations retain responsibility for SCTP, DTLS, ICE, packetisation, congestion control, and retransmission.
@@ -24,7 +29,7 @@ One low-level number cannot represent all of these concerns. Users need a small
### Message-size presets
LiveSync will expose a `P2P message size` choice with four presets:
LiveSync exposes a `P2P message size` choice with four presets:
| Label | Maximum RPC wire payload | Intended use |
| ----------------------- | -----------------------: | ------------------------------------------------------------------------------- |
@@ -37,11 +42,11 @@ This value limits Commonlib RPC wire payloads before Trystero applies its own fr
The bound applies to outgoing messages. A device which only lowers its own value still receives messages produced under the sender's value. The selected preset therefore belongs to the P2P profile and is included in an encrypted Setup URI for additional devices. A device which was configured earlier must be changed separately; the interface and troubleshooting guidance must state that the same conservative preset should be selected on every participating device. An absent key preserves the current 15,360-byte default.
Automatic negotiation or fallback between presets is deferred. A failed ordered data channel may require connection replacement before a smaller retry can prove anything, and changing transport parameters during a replication session would broaden the lifecycle contract considerably. The first implementation remains explicit, stable for one room lifetime, and inspectable.
Automatic negotiation or fallback between presets is deferred. A failed ordered data channel may require connection replacement before a smaller retry can prove anything, and changing transport parameters during a replication session would broaden the lifecycle contract considerably. The implementation remains explicit, stable for one room lifetime, and inspectable.
### Connection path
LiveSync will expose a separate `Connection path` choice:
LiveSync exposes a separate `Connection path` choice:
- `Automatic` retains normal ICE selection and is the default.
- `TURN relay only` supplies `iceTransportPolicy: 'relay'` and prevents direct or server-reflexive candidates from being selected.
@@ -81,11 +86,15 @@ Where the browser exposes the evidence, the result should report:
- UDP, TCP, or TLS transport; and
- a bounded failure or inconclusive result.
Ordinary P2P diagnostics should later summarise the selected candidate pair as direct, server-reflexive, or relayed, with its transport. Raw `getStats()` output remains supporting evidence rather than the primary interface.
The proposed Obsidian P2P diagnostics should summarise the selected candidate
pair as direct, server-reflexive, or relayed, with its transport. The CLI's
candidate-route output does not provide this user-facing summary. Raw
`getStats()` output remains supporting evidence rather than the primary
interface.
### Placement and defaults
These controls belong inside `P2P Configuration` under a `Connection compatibility` section. They do not require the repository-wide Advanced, Power User, or Edge Case modes. P2P itself remains a supported opt-in feature.
These controls appear inside `P2P Configuration` under a `Connection compatibility` section. They do not require the repository-wide Advanced, Power User, or Edge Case modes. P2P itself remains a supported opt-in feature.
Existing profiles retain the following defaults:
@@ -161,7 +170,8 @@ A layer-4 TLS router could share one public address between distinct CouchDB and
## Verification
The first implementation stage must add focused tests before production changes:
The following list records the original focused-test plan for the message-size
and connection-path controls. It remains useful as regression coverage:
- settings-schema defaults for absent keys;
- P2P connection-string and Setup URI round trips which retain both transport compatibility settings;
docs/adr/2026_10_local_file_save_completion_and_batching.md
+20 -10
View File
@@ -2,15 +2,17 @@
date: 2026-10-24
commonlib-version: "0.2.0"
self-hosted-livesync-version: "1.0.40"
status: accepted-unreleased
status: accepted
---
# Architectural Decision Record: Local File Save Completion and Batching
## Status
Accepted on 2026-10-25. The package versions above identify the investigated
baseline, rather than a released implementation of this decision.
Accepted on 2026-10-25. The recovery contracts and LiveSync mode batching in
implementation-order steps 1–3 are present in current Self-hosted LiveSync
1.0.42 with Commonlib 0.2.2. The version fields above retain the investigated
baseline. Further same-path coalescing in step 4 remains a follow-up.
## Context
@@ -349,6 +351,9 @@ eliminating genuine concurrent-edit conflicts or as a confirmed fix for #1207.
## Ownership and implementation order
The list below records the implementation sequence. Steps 1–3 are published
in the current version line; the further coalescing in step 4 remains planned.
1. **Commonlib recovery:** event ownership, ordered snapshot persistence,
successful completion and failure propagation, finite flush boundaries, and
same-time content reconciliation. Add focused failing regressions before
@@ -373,6 +378,11 @@ automation transaction API require a further design decision.
## Verification required
The matrix below records the acceptance criteria for implementation steps 1–3
and remains useful for regression work. Publication in the current version line
does not extend the historical test results above to untested platforms or
automation workflows.
Use controlled promises at the manager and persistence boundaries for focused
regressions, and real PouchDB tests for content and ancestry. The acceptance
matrix is:
@@ -395,13 +405,13 @@ matrix is:
| Old settings, mode changes, explicit opt-in, and settings import | Existing LiveSync mode users retain immediate saving until they choose batching. |
| Incoming identical content, an edit, a deletion, or delete/recreate while waiting | Preserve genuine branches without extra conflicts or loss of local work. |
Keep the existing E2E comparison between batching disabled, an ignored legacy
setting, and explicitly enabled batching. Repeat both directions, manual
synchronisation, saved-snapshot restart, and controlled in-progress restart with
the fix. Record the original failure separately from repaired results; a pass
with a different timestamp bucket or a test-only bundle is not verification of
the production fix. Run platform-specific lifecycle checks before extending the
evidence to Windows or mobile.
The original E2E plan compared batching disabled, an ignored legacy setting, and
explicitly enabled batching in both directions, with manual synchronisation,
saved-snapshot restart, and controlled in-progress restart. Record the original
failure separately from repaired results; a pass with a different timestamp
bucket or a test-only bundle is not verification of the production fix. Keep
platform-specific lifecycle evidence bounded to the tested environments before
extending conclusions to Windows or mobile.
## Alternatives and consequences
docs/adr/2026_10_unicode_path_identity.md
+4 -3
View File
@@ -2,15 +2,16 @@
date: 2026-10-17
commonlib-version: "0.1.42"
self-hosted-livesync-version: "1.0.37"
status: unreleased
status: released
---
# Architectural Decision Record: Unicode Path Identity for CouchDB
## Status
Accepted on 2026-10-17. The feature remains unreleased. Local validation
evidence and platform limits are recorded in the design document.
Accepted on 2026-10-17. Unicode NFC path identity was released in Self-hosted
LiveSync 1.0.38 with Commonlib 0.1.42. Local validation evidence and platform
limits are recorded in the design document.
## Context
docs/design_docs/configurable_id_derivation.md
+5 -5
View File
@@ -2,7 +2,7 @@
date: 2026-09-29
commonlib-version: "0.1.33"
self-hosted-livesync-version: "1.0.32"
status: unreleased
status: released
---
# Configurable ID derivation
@@ -21,12 +21,12 @@ including its passphrase-persistence correction at commit
the `next` tag, provides the construction described here. It replaces the
independent Chunk algorithm from the `0.1.32` prerelease without a
compatibility branch or a new settings version. Legacy ID generation remains
unchanged. LiveSync pins the published `0.1.33` package and its registry
integrity in the lockfile.
unchanged. At that implementation baseline, LiveSync pinned the published
`0.1.33` package and its registry integrity in the lockfile.
Its [Internal Metadata encryption design](https://github.com/vrtmrz/obsidian-livesync/blob/126d6eadb858a79a08ad7f600061e54fc8d31196/docs/design_docs/internal_metadata_encryption.md)
remains the basis for Properties encryption and CouchDB feature admission.
This document records an unreleased LiveSync feature. It does not select a
plug-in release version.
Independent ID derivation shipped in Self-hosted LiveSync 1.0.33. This document
retains the design and package baseline recorded above.
## Feasibility
docs/design_docs/document_history_revision_restoration.md
+25 -20
View File
@@ -10,40 +10,45 @@ This document uses the independent revision properties defined under
[Revision](../glossary.md#revision) and the general state model in
[Conflict resolution and revision provenance](../specs_conflict_resolution.md).
Document History can reconstruct an available historical revision from its
Chunks and write that content to the Vault through **Back to this revision**.
The current action writes directly through the storage adapter. It does not
create a new Metadata revision, clear a logical deletion through a successor,
or record which database revision produced the restored Vault file.
Before the current restoration implementation, Document History could
reconstruct an available historical revision from its Chunks and write that
content to the Vault through **Back to this revision**. That action wrote
directly through the storage adapter. It did not create a new Metadata revision,
clear a logical deletion through a successor, or record which database revision
produced the restored Vault file.
This leaves a logically deleted Metadata document at `deleted: true` after the
file has returned to the Vault. A later ordinary Vault save may create a
non-deleted successor, but restoration must not depend on an unrelated later
file event.
That left a logically deleted Metadata document at `deleted: true` after the
file had returned to the Vault. A later ordinary Vault save could create a
non-deleted successor, but restoration needed to succeed without depending on
an unrelated later file event.
The same action does not inspect or preserve revision-tree intent explicitly
when conflicts exist. Document History currently displays the ancestry of the
PouchDB winner. It does not display the complete revision tree or the ancestry
of every conflict leaf.
Restoration does not resolve conflicts or provide a branch-aware history view.
Document History displays the ancestry of the PouchDB winner; it does not
display the complete revision tree or the ancestry of every conflict leaf.
This design covers restoration of one readable historical revision of a normal
Vault file. It creates a new non-deleted successor revision, reflects that exact
revision to the Vault, and leaves every other conflict branch available for the
existing conflict workflow.
The current **Back to this revision** implementation writes that successor
revision, reflects the exact created revision to the Vault, and records it as
device-local provenance. The storage-only behaviour described above is the
pre-fix history.
## Evidence
A real-Obsidian exercise created a Markdown document, removed it through the
Vault API, and waited for LiveSync to store a logical-deletion successor. The
Metadata retained all referenced Chunks, and Document History could reconstruct
the deleted content.
A real-Obsidian exercise before the current restoration implementation created
a Markdown document, removed it through the Vault API, and waited for LiveSync
to store a logical-deletion successor. The Metadata retained all referenced
Chunks, and Document History could reconstruct the deleted content.
Selecting **Back to this revision** restored the file and its exact bytes to the
Vault. The local database nevertheless retained the same deleted current
revision after file processing had settled. An additional ordinary Vault save
then created a new non-deleted successor. This demonstrates that content
reconstruction works and that the missing operation is the database-aware
restoration step.
then created a new non-deleted successor. This historical reproduction showed
that content reconstruction worked while the database-aware restoration step
was missing.
## Revision-tree decision
docs/design_docs/internal_metadata_encryption.md
+8 -7
View File
@@ -2,14 +2,15 @@
date: 2026-09-27
commonlib-version: "0.1.30"
self-hosted-livesync-version: "1.0.32"
status: unreleased
status: released
---
# Internal Metadata encryption and remote feature changes
This document defines the LiveSync integration of Commonlib's remote feature
contract and encrypted Metadata for Hidden File Sync and Customisation Sync.
It describes unreleased behaviour being implemented in this branch.
The implementation shipped in Self-hosted LiveSync 1.0.33; this document
retains its original design and implementation context.
Commonlib's companion `docs/remote-feature-compatibility.md` is the
source of truth for the wire document, identifiers, validation, and shared
@@ -159,11 +160,11 @@ after alignment. With the preference subsequently OFF, verify that Fast Fetch
still decodes encrypted Metadata and preserves the remote feature declaration.
Keep the primary-language settings and troubleshooting guides, the
database-compatibility ADR, and Unreleased notes aligned with this behaviour.
Keep the detailed shared protocol in Commonlib and link to it after publication;
do not maintain another copy of its wire schema here. Translations are a separate
change. Update tested-version evidence when the implementation and its
validation have been accepted.
database-compatibility ADR, and release notes consistent with this behaviour.
Commonlib's companion document remains the source of truth for the shared
protocol; this document does not repeat its wire schema. Translations remain a
separate change. The version fields above retain the original design baseline;
add later tested-version evidence only when validation is recorded.
Related application contracts: [Replicator architecture](replicator_architecture.md),
[Tweak compatibility](tweak_compatibility.md), and
docs/design_docs/path_component_length_compatibility.md
+4 -4
View File
@@ -2,7 +2,7 @@
date: 2026-09-04
commonlib-version: "0.1.21"
self-hosted-livesync-version: "1.0.24"
status: unreleased
status: released
---
# Path component length compatibility
@@ -11,9 +11,9 @@ status: unreleased
File systems place limits on each file or folder name, rather than applying one
common limit to an entire Vault-relative path. Those limits are also expressed
in different units. Self-hosted LiveSync therefore treats 255 UTF-8 bytes as a
focused Android and Linux compatibility warning, not as a universal definition
of a valid path.
in different units. The warning shipped in Self-hosted LiveSync 1.0.26. It
treats 255 UTF-8 bytes as a focused Android and Linux compatibility warning,
not as a universal definition of a valid path.
## Basis for the 255-byte warning
docs/design_docs/renewable_turn_credentials.md
+4 -1
View File
@@ -2,7 +2,7 @@
date: 2026-09-16
commonlib-version: "0.1.25"
self-hosted-livesync-version: "1.0.28"
status: unreleased
status: released
---
# TURN credentials in P2P connection settings
@@ -14,6 +14,9 @@ by acquiring temporary TURN credentials on the device before opening a P2P room.
The [P2P transport compatibility ADR](../adr/2026_08_p2p_transport_compatibility.md)
records the connection and persistence policy.
Managed Cloudflare TURN credentials shipped in Self-hosted LiveSync 1.0.29.
This document records the design and its remaining runtime verification limits.
LiveSync prepares a connection copy of `P2PSyncSetting`. Commonlib owns the room
lifecycle and consumes the resulting ICE settings. Service-specific HTTP and
validation remain under `src/integrations/`; Commonlib has no provider catalogue
docs/design_docs/time_bound_setup_uri.md
+104 -96
View File
@@ -3,40 +3,42 @@ date: 2026-09-28
commonlib-version: "0.1.33"
feasibility-probe-version: "0.1.27"
self-hosted-livesync-version: "1.0.32"
status: implementing
status: released
---
# Time-bound Setup URIs
## Purpose and status
Offer two modes when generating a [Setup URI](../glossary.md):
The published Obsidian generator offers two choices when creating a [Setup URI](../glossary.md):
- **Ephemeral**, selected by default, derives an effective passphrase from the
entered passphrase and the current fixed time window.
- **Persistent** uses the entered passphrase and the existing URI format
unchanged. It also serves as the compatibility mode for older readers.
- **Time-bound** (the `ephemeral` derivation mode), selected by default,
derives an effective passphrase from the entered passphrase and the current
fixed time window.
- **Compatible (no time limit)** (the `persistent` derivation mode) uses the
entered passphrase and the existing URI format unchanged. It also serves as
the compatibility mode for older readers.
The generation mode choice shows the exact time until which an Ephemeral URI
can be opened through the ordinary reader. Import still requires only the URI and
the entered passphrase. Neither the mode nor a timestamp is stored in the URI.
The Time-bound choice shows the exact time until which the URI can be opened
through the ordinary reader. Import still requires only the URI and the entered
passphrase. Neither the mode nor a timestamp is stored in the URI.
The design is technically feasible with the existing encryption primitives.
An executable protocol probe passes 22 cases against Commonlib 0.1.27 and
octagonal-wheels 0.1.54. The implementation was first published as Commonlib 0.1.32
on the npm `next` tag. LiveSync now pins Commonlib 0.1.33 from that tag in its npm
dependency and independent Deno tool imports. A mobile performance bound remains
outstanding.
Both URI availability choices shipped in Self-hosted LiveSync 1.0.33. The
initial executable protocol probe passed 22 cases against Commonlib 0.1.27 and
octagonal-wheels 0.1.54. The implementation was first published as Commonlib
0.1.32 on the npm `next` tag. Current Self-hosted LiveSync 1.0.42 uses Commonlib
0.2.2. A mobile performance bound remains outstanding.
This document records the proposed key derivation and integration contract. The
companion probe demonstrates them without changing an application entry point.
This document records the protocol and the integration plan used for the
published feature. The companion probe remains a standalone design check; it
does not exercise an application entry point.
## Scope
The first version offers Ephemeral and Persistent only. Ephemeral uses one
fixed seven-day window shared by every implementation. Arbitrary start dates,
custom end dates, selectable window lengths, and rolling seven-day lifetimes
are outside this version.
The published protocol offers the Ephemeral and Persistent modes only.
Ephemeral uses one fixed seven-day window shared by every implementation.
Arbitrary start dates, custom end dates, selectable window lengths, and rolling
seven-day lifetimes are outside this version.
Keep the current settings filtering, remote profiles, main and P2P selections,
and short/full export variants. Time binding affects opening the exported
@@ -255,15 +257,16 @@ separately shared message.
## Ownership and compatibility
Commonlib owns preparation of exported settings, the time-window calculation,
passphrase derivation, format parsing, and dual-candidate decoding. Make these
changes in the Commonlib repository. LiveSync must consume a validated packed
artefact and then an exact reviewed package version, following
passphrase derivation, format parsing, and dual-candidate decoding. These
changes were made in the Commonlib repository. LiveSync consumes the reviewed
Commonlib package, following
[the Commonlib dependency workflow](../../devs.md#commonlib-dependency).
Preserve the existing positional encoder API. Its third and fourth arguments
already mean properties to remove and whether to omit default values. Add a
separate encoder, provisionally `encodeTimeBoundSetupURI`, with an options
object containing the mode and the existing export options. Its result is:
The implementation preserves the existing positional encoder API. Its third
and fourth arguments retain their meanings: properties to remove and whether
to omit default values. It adds a separate `encodeTimeBoundSetupURI` encoder
with an options object containing the mode and the existing export options.
Its result is:
```typescript
type TimeBoundSetupURIResult = {
@@ -272,27 +275,27 @@ type TimeBoundSetupURIResult = {
};
```
The new encoder defaults to Ephemeral. Preserve the old encoder's legacy
behaviour for callers which have not migrated. Extend
`decodeSettingsFromSetupURI` to try both candidates for `%$` while retaining its
The new encoder defaults to Ephemeral. The old encoder retains its legacy
behaviour for callers which have not migrated. The updated
`decodeSettingsFromSetupURI` tries both candidates for `%$` while retaining its
settings return contract and historical format handling. Persistent generation
through the new API delegates to the old encoder. Time injection belongs in
through the new API delegates to the old encoder. Time injection remains in
internal test seams, not an import option exposed to users.
| Surface | Required work |
| Surface | Published integration |
| ------------------------------------------------------------------------------------ | -------------------------------------------------------------------------------------------------------------------------------------------------------- |
| [Obsidian generation](../../src/serviceFeatures/setupObsidian/setupUri.ts) | Reuse the passphrase and copy dialogues, add a mode choice with the exact current window end, and use the new encoder for all three copy variants and the copy event. Preserve each variant's export filters. |
| [Obsidian import](../../src/modules/features/SetupWizard/dialogs/UseSetupURI.svelte) | Consume the updated decoder and retain one authentication failure message. The existing prefix check indicates URI shape, not successful authentication. |
| [Protocol handler](../../src/serviceFeatures/setupObsidian/setupProtocol.ts) | Keep the existing `settings` reconstruction; test encoded delimiters for both modes. |
| [CLI setup](../../src/apps/cli/commands/runCommand.ts) | Consume the updated decoder and leave settings unchanged after any rejection. |
| [WebPeer generator](../../src/apps/webpeer/src/P2PCheckSetup.ts) | Choose a mode explicitly and return availability to its result UI; keep Ephemeral as the default. |
| [Setup utility](../../utils/setup/generate_setup_uri.ts) | Add an explicit mode input, default to Ephemeral, and return/print the end time. Persistent is an explicit choice for durable provisioning output. |
| [Setup utility package facade](../../utils/setup/livesync-commonlib.ts) | Update its independent Commonlib pin and associated lockfiles; updating the root npm dependency alone is insufficient. |
| [Obsidian generation](../../src/serviceFeatures/setupObsidian/setupUri.ts) | Reuses the passphrase and copy dialogues, presents the mode choice and exact current window end, and uses the encoder for all copy variants and the copy event while preserving each variant's export filters. |
| [Obsidian import](../../src/modules/features/SetupWizard/dialogs/UseSetupURI.svelte) | Consumes the updated decoder and retains one authentication failure message. The existing prefix check indicates URI shape, not successful authentication. |
| [Protocol handler](../../src/serviceFeatures/setupObsidian/setupProtocol.ts) | Retains the existing `settings` reconstruction; encoded delimiters remain part of protocol validation for both modes. |
| [CLI setup](../../src/apps/cli/commands/runCommand.ts) | Uses the updated decoder and leaves settings unchanged after a rejection. |
| [WebPeer generator](../../src/apps/webpeer/src/P2PCheckSetup.ts) | Explicitly selects Time-bound mode and returns availability to its result UI; this generator does not offer a Persistent choice. |
| [Setup utility](../../utils/setup/generate_setup_uri.ts) | Accepts an explicit mode, defaults to Time-bound, and returns or prints the end time. Persistent is selected for durable provisioning output. |
| [Setup utility package facade](../../utils/setup/livesync-commonlib.ts) | Retains its independent Commonlib pin and associated lockfiles, separate from the root npm dependency. |
The Fly.io wrapper delegates to the setup utility and should inherit the same
contract. All maintained generators must migrate explicitly before this
feature is described as the default across applications. Non-interactive
automation which needs a durable URI must select Persistent.
The Fly.io wrapper delegates to the setup utility and inherits the same
contract. The initial plan required each maintained generator to select a mode
explicitly. Non-interactive automation which needs a durable URI selects
Persistent.
WebPeer hides an Ephemeral URI and disables its copy and additional-device
actions after the window ends. Its connection monitor remains available for a
@@ -309,8 +312,9 @@ The assessed, unchanged Commonlib 0.1.27 decoder opens Persistent with the
entered passphrase and rejects Ephemeral with that same passphrase. Persistent
compatibility means compatibility with readers which already support the
ordinary generated `%$` representation; it does not add that representation
to still older readers. Historical supported versions and their settings
schemas still need compatibility fixtures before rollout.
to still older readers. This design does not establish compatibility with every
historical reader or settings schema; compatibility fixtures are needed before
making broader claims about them.
Keep the original raw-passphrase behaviour of existing URIs. They are
indistinguishable from newly generated Persistent URIs to the updated reader.
@@ -358,68 +362,72 @@ All 22 cases passed:
and raw-passphrase candidates;
- the deliberate limitation that clock rollback reproduces an old key.
This demonstrates the proposed mechanism and baseline interoperability, not
completed application support, formal mode indistinguishability, or a mobile
latency bound. The present encryption API may perform two PBKDF2 derivations
on a cold import. Do not infer production performance from this small probe,
whose operations can reuse the dependency's in-memory key cache.
The probe demonstrates feasibility and baseline interoperability. Application
integration is recorded above; the probe does not establish formal mode
indistinguishability or a mobile latency bound. The encryption API may perform
two PBKDF2 derivations on a cold import. Do not infer production performance
from this small probe, whose operations can reuse the dependency's in-memory
key cache.
Implementation validation must cover Commonlib unit and packed-package tests,
plus shared fixed-input protocol vectors across consumers. Add targeted LiveSync
tests for the generation metadata, cancellation, a window change during mode
selection, encoded deep links, and rejected imports without settings writes. Extend CLI, WebPeer,
and Deno setup-tool round trips, including their Persistent selection.
The original implementation validation plan covered Commonlib unit and
packed-package tests, plus shared fixed-input protocol vectors across consumers.
It called for targeted LiveSync tests of generation metadata, cancellation, a
window change during mode selection, encoded deep links, and rejected imports
without settings writes, alongside CLI, WebPeer, and Deno setup-tool round trips
for both modes.
Then run the required LiveSync checks, unit suite, and production build, one
broad process at a time with `NODE_OPTIONS=--max-old-space-size=3072` and bounded
test workers. Use the existing focused real-Obsidian Setup URI workflow for
copy/paste, deep-link import, cancellation, and successful setup. Verify
selection-boundary handling with an injected test clock and mobile behaviour on a
supported runtime; do not change the host's system clock. No remote database
service is needed for the protocol tests themselves.
The plan also called for the LiveSync checks, unit suite, and production build,
one broad process at a time with `NODE_OPTIONS=--max-old-space-size=3072` and
bounded test workers. It used the focused real-Obsidian Setup URI workflow for
copy/paste, deep-link import, cancellation, and successful setup. Selection-
boundary handling was to use an injected test clock; mobile behaviour requires
a supported runtime and must not change the host's system clock. Protocol tests
do not require a remote database service.
There is no identified blocker in the Time-bound URI protocol. The Commonlib
candidate based on 0.1.31 passed its type, unit, boundary, release-process,
and packed-package gates before the 0.1.32 release. A clean `npm ci` against
the published 0.1.32 artefact and the frozen Deno tool lock resolve the same
registry integrity. LiveSync passes its source checks, production build, and
1,034 unit tests on a local stack with [PR #1222](https://github.com/vrtmrz/obsidian-livesync/pull/1222).
The two CLI installer tests need a runner which permits child processes; both
pass under normal process permissions. The Deno setup-tool suite, CLI setup
The original protocol assessment found no blocker in the Time-bound URI
protocol. The Commonlib candidate based on 0.1.31 passed its type, unit,
boundary, release-process, and packed-package gates before the 0.1.32 release. A
clean `npm ci` against the published 0.1.32 artefact and the frozen Deno tool
lock resolved the same registry integrity. In the original implementation
baseline, LiveSync passed its source checks, production build, and 1,034 unit
tests on a local stack with [PR #1222](https://github.com/vrtmrz/obsidian-livesync/pull/1222).
The two CLI installer tests needed a runner which permits child processes; both
passed under normal process permissions. The Deno setup-tool suite, CLI setup
and file-operation contract, and WebPeer browser tests passed before stacking.
Focused real-Obsidian generation checks pass on desktop and emulated mobile,
The recorded real-Obsidian generation checks passed on desktop and emulated mobile,
including the displayed Time-bound end, Compatible output, and mobile touch
targets. A two-device CouchDB workflow now passes on the local stack: the
provisioning tool creates a generation 12 database, the first device generates
targets. A two-device CouchDB workflow passed on the local stack: the
provisioning tool created a generation 12 database, the first device generated
a Time-bound Setup URI after declaring its required generation 13 feature, and
the second device imports it. An ordinary note completes a round trip, and a
hidden snippet synchronises. The separate real-Obsidian tests for live remote
the second device imported it. An ordinary note completed a round trip, and a
hidden snippet synchronised. The separate real-Obsidian tests for live remote
feature changes, internal Metadata migration, and a percent-prefixed E2EE
passphrase also pass with published Commonlib 0.1.32. Unit and protocol tests
cover window boundaries and preserve the existing distinction between an empty
passphrase and cancellation. The WebPeer browser test confirms that its monitor
remains available after the URI window ends for a device which has already
imported the URI.
passphrase also passed with published Commonlib 0.1.32. Unit and protocol tests
covered window boundaries and preserved the existing distinction between an
empty passphrase and cancellation. The WebPeer browser test confirmed that its
monitor remained available after the URI window ended for a device which had
already imported the URI.
The current Commonlib 0.1.33 integration includes the merged host changes from
PR #1222 and PR #1225. npm 10 and normal clean installations preserve the lockfile;
source checks, the production build, 1,064 unit tests, and ten Deno setup-tool
tests pass. Both URI modes preserve ID recovery and explicit legacy ID selection.
The Commonlib 0.1.33 integration included the merged host changes from
PR #1222 and PR #1225. At that baseline, npm 10 and normal clean installations
preserved the lockfile; source checks, the production build, 1,064 unit tests,
and ten Deno setup-tool tests passed. Both URI modes preserved ID recovery and
explicit legacy ID selection.
Real Obsidian tests cover independent ID keys through Time-bound, Compatible,
and QR setup, encrypted local persistence, natural restarts of both devices,
and bidirectional note transfers with stable document and Chunk IDs. Incorrect
passphrases and past-window URIs leave runtime and persisted settings unchanged.
The past clock belongs to an isolated fixture worker, not Obsidian or the host.
CouchDB tests also cover a custom ID source and recovery code, rejection of
incompatible document keys before remote writes, and the Doctor's decline,
reminder, dismissal, and later acceptance paths. The detailed commands and the
remaining published-artefact and physical-device checks are recorded in the
Recorded real-Obsidian tests covered independent ID keys through Time-bound,
Compatible, and QR setup, encrypted local persistence, natural restarts of both
devices, and bidirectional note transfers with stable document and Chunk IDs.
Incorrect passphrases and past-window URIs left runtime and persisted settings
unchanged. The past clock belonged to an isolated fixture worker, not Obsidian
or the host. CouchDB tests also covered a custom ID source and recovery code,
rejection of incompatible document keys before remote writes, and the Doctor's
decline, reminder, dismissal, and later acceptance paths. The detailed commands,
remaining published-artefact checks, and physical-device checks are recorded in the
[real Obsidian test guide](../../test/e2e-obsidian/README.md#combined-setup-and-security-regression-checks).
Compatible preserves the URI encryption format; each receiving client must
still support the shared settings and the remote's declared features. The host
compatibility checks from PR #1222 remain necessary. These results do not
establish a mobile performance bound. The first implementation changes only
primary-language resources; translation changes require separate scope.
establish a mobile performance bound. The initial implementation changed only
primary-language resources; translation changes remain a separate scope.
docs/design_docs/tweak_compatibility.md
+6 -5
View File
@@ -2,15 +2,16 @@
date: 2026-10-17
commonlib-version: "0.1.42"
self-hosted-livesync-version: "1.0.37"
status: unreleased
status: released
---
# Tweak compatibility and recovery
This document describes the compatibility and recovery contract, including
unreleased changes after Self-hosted LiveSync 1.0.37 using Commonlib 0.1.42.
Commonlib owns the interpretation of synchronisation settings; LiveSync owns
the dialogues and operations which consume that result.
This document describes the compatibility and recovery contract introduced
after the Self-hosted LiveSync 1.0.37 and Commonlib 0.1.42 baseline recorded
above. The current assessment is included in Self-hosted LiveSync 1.0.42 with
Commonlib 0.2.2. Commonlib owns the interpretation of synchronisation settings;
LiveSync owns the dialogues and operations which consume that result.
## Shared assessment
docs/design_docs/unicode_path_identity.md
+5 -3
View File
@@ -2,7 +2,7 @@
date: 2026-10-19
commonlib-version: "0.1.43"
self-hosted-livesync-version: "1.0.38"
status: unreleased
status: released
---
# Unicode path identity
@@ -17,8 +17,10 @@ Commonlib's [consumer contract](https://github.com/vrtmrz/livesync-commonlib/blo
defines the shared package boundary.
NFC identity shipped in Self-hosted LiveSync `1.0.38` with Commonlib `0.1.42`.
The deletion handling in reconstruction below uses Commonlib `0.1.43` and
remains unreleased in Self-hosted LiveSync.
Conflict-aware deletion handling during reconstruction was added separately
and is included in current Commonlib `0.2.2`, used by Self-hosted LiveSync
`1.0.42`. This later handling is distinct from the initial NFC identity
release.
## Scope
docs/design_docs_of_journalsync.md
+3 -1
View File
@@ -1,5 +1,7 @@
## The design document of the journal sync
Historical design: this is the first Journal Sync design. Its original transfer sequence and discussion of possible storage backends are retained as history. For the maintained pipeline and storage-adapter design, read [Journal Replicator 2nd Edition](design_docs_of_journalsync_2nd.md) and the [Replicator architecture](design_docs/replicator_architecture.md). Use [Object Storage Setup](setup_object_storage.md) for supported client setup. A possible backend discussed here is not evidence that an adapter is available in the current release.
Original title: Synchronise without CouchDB
### Goal
@@ -47,4 +49,4 @@ This implementation replaces the synchronisation performed by CouchDB. Therefore
### Consideration and Conclusion
This design offers a novel approach to journal synchronisation without relying on CouchDB. It leverages PouchDB's journaling capabilities and leverages simple server-side storage for efficient data exchange. Hence, the new design could be said to have gotten a broader outlook.
This design offers a novel approach to journal synchronisation without relying on CouchDB. It leverages PouchDB's journaling capabilities and leverages simple server-side storage for efficient data exchange. Hence, the new design could be said to have gotten a broader outlook.
docs/design_docs_of_remote_configurations.md
+16 -5
View File
@@ -1,5 +1,9 @@
# The design document of remote configuration management
## Implementation status
This document describes the profile catalogue and its runtime projections as used by Self-hosted LiveSync 1.0.42 with Commonlib 0.2.2. Commonlib owns profile migration, activation, and protected settings persistence. The [Replicator architecture](design_docs/replicator_architecture.md) and [P2P service lifecycle decision](adr/2026_08_replicator_capabilities_02_p2p_service_lifecycle.md) describe the separate P2P service boundary.
## Goal
- Allow us to manage multiple remote connections in a single vault.
@@ -33,7 +37,9 @@ Once multiple remotes of the same type became desirable, the previous model no l
The current settings now have two layers for remote configuration.
1. A stored collection of named remotes.
2. One active remote projected into the legacy flat settings fields.
2. One active main remote projected into the legacy flat settings fields.
The P2P service also has its own selected profile, recorded by `P2P_ActiveRemoteConfigurationId`. This permits the P2P service to use a selected P2P profile alongside a CouchDB or Object Storage main remote; it does not introduce a second main remote.
This means the replication and database layers can continue to read the effective remote from the existing settings fields, while the settings dialogue and migration logic can manage many stored remotes.
@@ -54,6 +60,7 @@ type RemoteConfiguration = {
type RemoteConfigurations = {
remoteConfigurations: Record<string, RemoteConfiguration>;
activeConfigurationId: string;
P2P_ActiveRemoteConfigurationId: string;
};
```
@@ -116,7 +123,11 @@ Therefore:
- Import trusts the supplied connection string as-is after parsing.
- We should regard exported connection strings as sensitive information, much like Setup URI or a credentials-bearing configuration file.
The `isEncrypted` field is currently reserved for future expansion. At present, the connection string itself is stored plainly inside the settings data, in the same sense that the effective runtime configuration can contain usable remote credentials.
The `isEncrypted` field identifies an encrypted profile URI in persisted settings. When settings are saved with configuration encryption enabled, Commonlib encrypts the profile URIs and records `isEncrypted: true`. Loading protected settings decrypts the URIs into the runtime settings and clears that runtime flag before profile activation. The runtime settings and the persisted representation therefore have different purposes.
Protection depends on the configured settings-passphrase policy and a usable key. A plaintext configuration is not made confidential by the presence of the field alone. Consult [Encrypting sensitive configuration items](settings.md#encrypting-sensitive-configuration-items) for that policy.
Clipboard export serialises the usable connection string. Treat that exported value as sensitive even when its saved profile URI is encrypted at rest; storage protection does not encrypt the clipboard representation.
### Migration strategy
@@ -129,7 +140,7 @@ Rules:
- If no remote list exists and the legacy fields contain a Peer-to-Peer configuration, create `legacy-p2p`.
- If more than one legacy remote is populated, create all possible entries and select the active one according to `remoteType`.
This migration is intentionally additive. It does not remove the flat fields because they remain necessary as the active runtime projection.
This migration is intentionally additive. It does not remove the flat fields because they remain necessary as the active runtime projection. A migrated legacy P2P profile also initialises the separate P2P selection. Loading an existing profile catalogue does not unconditionally replace an intentionally empty P2P selection.
### Normalisation and application paths
@@ -190,7 +201,7 @@ The feature should be tested from four viewpoints.
## Outlook
Import/export configuration strings should also be encrypted in the future, but this is a separate feature that can be added on top of the current design.
A separately encrypted representation for clipboard import and export remains a possible extension. Encryption of profile URIs in persisted settings is already implemented as described above.
## Consideration and conclusion
@@ -203,4 +214,4 @@ Its core idea is modest but effective.
- Project it into the legacy settings for runtime use.
- Normalise every imported settings object through the same path.
This keeps the implementation understandable and migration-friendly. It also opens the door for future work, such as encrypted per-remote storage, richer remote metadata, or remote-scoped options, without forcing another large redesign of how remotes are represented.
This keeps the implementation understandable and migration-friendly. It also opens the door for future work, such as richer remote metadata or remote-scoped options, without forcing another large redesign of how remotes are represented.
docs/design_docs_of_sharing_tweak_value.md
+2 -2
View File
@@ -1,6 +1,6 @@
# Sharing `Tweak values`
NOTE: This is the planned feature design document. This is planned, but not be implemented now (v0.23.3). This has not reached the design freeze and will be added to from time to time.
Historical design: this document records the original proposal written for v0.23.3. Tweak sharing and compatibility assessment are implemented in published releases. Use [Tweak compatibility and recovery](design_docs/tweak_compatibility.md) for the maintained contract, and [Settings](settings.md) for the available controls. The proposal and test plan below are retained as design history; they do not define the current behaviour.
## Goal
@@ -52,4 +52,4 @@ B: On the setting dialogue.
- However, this might be an essential feature. Further efforts are desired.
### Consideration and Conclusion
To be described after implemented, tested, and, released.
The published implementation has evolved beyond this proposal, including classification of compatible differences and automatic alignment where configured. Its current decisions, recovery requirements, and remaining limitations are recorded in [Tweak compatibility and recovery](design_docs/tweak_compatibility.md). The original proposal does not establish that every item in its test plan was implemented unchanged.
docs/review_harness.md
+3 -2
View File
@@ -19,10 +19,11 @@ The command and view are registered only when developer tools are enabled. Disab
| --- | --- | --- | --- |
| Settings lifecycle | Automatic | Read-only | Exposes the seven synchronisation choices as typed observations and, for a genuinely new Vault, compares the selected recommendations with Commonlib's new-Vault contract. Existing-Vault setting preservation remains an automated migration and `settings-ui` E2E responsibility. |
| Compatibility review boundary | Guided | Device-local state | Observes the current dedicated compatibility controller and opens its actual review action. The Harness does not restore Change Log acknowledgement, `lastReadUpdates`, or a separate manual Pass or Fail result. |
| P2P composition | Automatic | Read-only | Confirms that the live P2P result resolves one replicator bound to the active Obsidian services. Peer discovery, replacement, reconnection, and transfer remain unit, CLI, and Compose E2E responsibilities. |
| ID generation performance | Automatic, individual | Read-only | Measures legacy and independent IDs using fixed in-memory inputs, including key derivation and heap samples where available. Keep Obsidian in the foreground. |
| File content hash performance | Manual | Read-only | Compares SHA-256 and xxHash64 on fixed in-memory Blob inputs. Input creation is excluded from hash timings; engine initialisation and warm-up are reported separately. |
| Vault fixture round trip | Automatic, explicit | Dedicated Vault fixtures | After explicit confirmation, creates, reads, modifies, renames, and removes one owned fixture tree. General Vault reflection remains covered by its separate real-Obsidian E2E workflow. |
**Automatic** runs only the two read-only local observations. **Full review** also starts the guided compatibility observation and asks before the Vault fixture scenario writes anything. A scenario can also be run individually.
**Automatic** runs only Settings lifecycle. **Full review** visits all listed scenarios, including the guided compatibility review and the two performance measurements, and asks before the Vault fixture scenario writes anything. Each scenario can also be run individually; a scenario's mode label does not by itself include it in the Automatic action.
These observations deliberately do not repeat stronger automated workflows. They exist to record which contracts the current real-device composition exposes while a maintainer reviews an immutable artefact.
docs/settings.md
+2
View File
@@ -161,6 +161,8 @@ Following panes will be shown when you enable this setting.
### 1. Appearance
For icon meanings, progress counters, and CSS snippets, see the [status display guide](tips/status-display.md).
#### Display Language
Setting key: displayLanguage
docs/setup_flyio.md
+50 -23
View File
@@ -3,16 +3,16 @@
This is how to configure fly.io and CouchDB on it for Self-hosted LiveSync.
The CouchDB instance is yours to manage. You can configure it yourself or set up another instance and switch to it, following the preservation and recovery guidance below.
> [!WARNING]
> It is **your** instance. In Obsidian, we have files locally. Hence, do not hesitate to destroy the remote database if you feel something have got weird. We can launch and switch to the new CouchDB instance anytime[^1].
>
[^1]: Actually, I am always building the database for reproduction of the issue like so.
> Before replacing or deleting an existing CouchDB instance, back up every available Vault and the remote database, then [choose the authoritative copy](recovery.md#first-choose-the-authoritative-copy). Stop or suspend synchronisation on other devices while reviewing recovery options. Local files alone do not establish that every needed edit or revision has been preserved.
> [!NOTE]
> **What and why is the Fly.io?**
> At some point, we started to experience problems related to our IBM Cloudant account. At the same time, Self-hosted LiveSync started to improve its functionality, requiring CouchDB in a more natural state to use all its features.
>
> Then we found Fly.io. Fly.io is the PaaS Platform, which can be useable for a very reasonable price. It generally falls within the `Free Allowances` range in most cases.
> Fly.io charges according to current account and resource usage, including Machines, persistent volumes, and network usage. Check [Fly.io pricing](https://fly.io/pricing/) and the [pricing documentation](https://fly.io/docs/about/pricing/) before deploying. A time-limited trial does not provide ongoing free hosting.
## Required materials
@@ -31,10 +31,10 @@ This is how to configure fly.io and CouchDB on it for Self-hosted LiveSync.
4. Copy the Setup-URI and Use it in the Obsidian.
5. You have been synchronised. Use the Setup-URI in subsequent devices.
Steps 4 and 5 are detailed in the [Quick Setup](./quick_setup.md#1-using-setup-uris).
For steps 4 and 5, see [Set up the first device](./quick_setup.md#set-up-the-first-device), then [Add another device](./quick_setup.md#add-another-device) for subsequent devices.
> [!NOTE]
> Your automatically configured configurations will be shown on the result in the Colab note like below, and **it will not be saved**. Please make a note of it somewhere.
> Save your own generated connection details and Setup URI securely. This is a historical Colab output example from 2024; its values are illustrative and will differ for each deployment. Replace all example credentials before deployment.
> ```
> -- YOUR CONFIGURATION --
> URL : https://billowing-dawn-6619.fly.dev
@@ -45,16 +45,18 @@ Steps 4 and 5 are detailed in the [Quick Setup](./quick_setup.md#1-using-setup-u
### B. Scripted Setup
Please refer to the document of [deploy-server.sh](../utils/readme.md#deploy-serversh).
Please refer to the document of [deploy-server.sh](../utils/readme.md#flyio-deployment).
### C. Manual Setup
| Used in the text | Meaning and where to use | Memo |
| ---------------- | --------------------------- | ------------------------------------------------------------------------ |
| campanella | Username | It is less likely to fail if it consists only of letters and numbers. |
| dfusiuada9suy | Password | |
| `<YOUR UNIQUE COUCHDB USERNAME>` | Username | Choose a unique value using only letters and numbers. |
| `<YOUR UNIQUE COUCHDB PASSWORD>` | Password | Generate a strong, unique password for this instance. |
| nrt | Region to make the instance | We can use any [region](https://fly.io/docs/reference/regions/) near us. |
Choose an available, unique Fly.io application name and replace both credential placeholders before deployment. Historical output below illustrates the command responses; use your own application name, hostname, and credentials.
#### 1. Install flyctl
- Mac or Linux
@@ -97,8 +99,8 @@ For more information, please refer to [Sign up](https://fly.io/docs/hands-on/sig
```bash
$ fly launch --name=billowing-dawn-6619 --env="COUCHDB_USER=campanella" --copy-config=true --detach --no-deploy --region nrt --yes
$ fly secrets set COUCHDB_PASSWORD=dfusiuada9suy
$ fly launch --name='<YOUR UNIQUE APP NAME>' --env="COUCHDB_USER=<YOUR UNIQUE COUCHDB USERNAME>" --copy-config=true --detach --no-deploy --region nrt --yes
$ fly secrets set COUCHDB_PASSWORD='<YOUR UNIQUE COUCHDB PASSWORD>'
```
#### 4. Deploy
@@ -170,24 +172,24 @@ Visit your newly deployed app at https://billowing-dawn-6619.fly.dev/
After the initial setup, CouchDB needs some more customisations to be used from Self-hosted LiveSync. It can be configured in browsers or by HTTP-REST APIs.
This section is set up using the REST API.
This section uses the REST API. The macOS and Linux commands require [jq](https://jqlang.org/download/) to encode credentials as JSON. Windows uses PowerShell's built-in `ConvertTo-Json`.
1. Prepare environment variables.
- Mac or Linux:
```bash
export couchHost=https://billowing-dawn-6619.fly.dev
export couchUser=campanella
export couchPwd=dfusiuada9suy
export couchHost='https://<YOUR APP HOSTNAME>'
export couchUser='<YOUR UNIQUE COUCHDB USERNAME>'
export couchPwd='<YOUR UNIQUE COUCHDB PASSWORD>'
```
- Windows
```powershell
set couchHost https://billowing-dawn-6619.fly.dev
set couchUser campanella
set couchPwd dfusiuada9suy
set couchHost 'https://<YOUR APP HOSTNAME>'
set couchUser '<YOUR UNIQUE COUCHDB USERNAME>'
set couchPwd '<YOUR UNIQUE COUCHDB PASSWORD>'
$creds = [System.Convert]::ToBase64String([System.Text.Encoding]::UTF8.GetBytes("${couchUser}:${couchPwd}"))
```
@@ -196,13 +198,36 @@ $creds = [System.Convert]::ToBase64String([System.Text.Encoding]::UTF8.GetBytes(
- Mac or Linux
```bash
curl -X POST "${couchHost}/_cluster_setup" -H "Content-Type: application/json" -d "{\"action\":\"enable_single_node\",\"username\":\"${couchUser}\",\"password\":\"${couchPwd}\",\"bind_address\":\"0.0.0.0\",\"port\":5984,\"singlenode\":true}" --user "${couchUser}:${couchPwd}"
clusterSetupJSON=$(jq -n \
--arg username "$couchUser" \
--arg password "$couchPwd" \
'{
action: "enable_single_node",
username: $username,
password: $password,
bind_address: "0.0.0.0",
port: 5984,
singlenode: true
}') &&
curl -X POST "${couchHost}/_cluster_setup" \
-H "Content-Type: application/json" \
--data-binary "$clusterSetupJSON" \
--user "${couchUser}:${couchPwd}"
```
- Windows
```powershell
iwr -UseBasicParsing -Method 'POST' -ContentType 'application/json; charset=utf-8' -Headers @{ 'Authorization' = 'Basic ' + $creds } "${couchHost}/_cluster_setup" -Body "{""action"":""enable_single_node"",""username"":""${couchUser}"",""password"":""${couchPwd}"",""bind_address"":""0.0.0.0"",""port"":5984,""singlenode"":true}"
$clusterSetupJSON = @{
action = 'enable_single_node'
username = $couchUser
password = $couchPwd
bind_address = '0.0.0.0'
port = 5984
singlenode = $true
} | ConvertTo-Json -Compress
iwr -UseBasicParsing -Method 'POST' -ContentType 'application/json; charset=utf-8' -Headers @{ 'Authorization' = 'Basic ' + $creds } "${couchHost}/_cluster_setup" -Body $clusterSetupJSON
```
Note: if the response code is not 200. We have to retry the request once again.
@@ -242,11 +267,13 @@ Note: Each of these should also be repeated until finished in 200.
#### 6. Use it from Self-hosted LiveSync
Now the CouchDB is ready to use from Self-hosted LiveSync. We can use `https://billowing-dawn-6619.fly.dev` in URI, `campanella` in `Username` and `dfusiuada9suy` in `Password` on Self-hosted LiveSync. The `Database name` could be anything you want.
Please refer to the [Minimal Setup of the Quick Setup](./quick_setup.md#2-minimal-setup).
Now the CouchDB is ready to use from Self-hosted LiveSync. Use your Fly.io app's hostname in `URI`, and the unique credentials configured above in `Username` and `Password`. The `Database name` could be anything you want.
Please refer to [Configure CouchDB manually on the first device](./quick_setup.md#configure-couchdb-manually-on-the-first-device).
## Delete the Instance
Before deleting the instance, preserve the data you still need and follow [Recovery and flag files](recovery.md#first-choose-the-authoritative-copy).
If you want to delete the CouchDB instance, you can do that in [fly.io Dashboard](https://fly.io/dashboard/personal)
If you have done with [B. Scripted Setup](#b-scripted-setup), we can use [delete-server.sh](../utils/readme.md#delete-serversh).
If you have done with [B. Scripted Setup](#b-scripted-setup), we can use [delete-server.sh](../utils/readme.md#flyio-deployment).
docs/setup_object_storage.md
+5 -5
View File
@@ -18,14 +18,14 @@ The public generator applies the Object Storage preset and records the connectio
```sh
export remote_type=s3
export endpoint=https://objects.example.com
export access_key=<ACCESS KEY>
export secret_key=<SECRET KEY>
export endpoint='https://objects.example.com'
export access_key='<ACCESS KEY>'
export secret_key='<SECRET KEY>'
export bucket=vault-data
export region=auto
export bucket_prefix=my-vault
export LIVESYNC_PASSPHRASE=<A STRONG VAULT ENCRYPTION PASSPHRASE>
export uri_passphrase=<A SEPARATE SETUP URI PASSPHRASE>
export LIVESYNC_PASSPHRASE='<A STRONG VAULT ENCRYPTION PASSPHRASE>'
export uri_passphrase='<A SEPARATE SETUP URI PASSPHRASE>'
deno run --minimum-dependency-age=0 --allow-env https://raw.githubusercontent.com/vrtmrz/obsidian-livesync/main/utils/setup/generate_setup_uri.ts
```
docs/setup_own_server.md
+24 -18
View File
@@ -15,7 +15,7 @@
- [4. Starting Docker Compose in background](#4-run-the-docker-compose-file-in-the-background)
- [C. Install CouchDB directly](#c-install-couchdb-directly)
- [2. Run couchdb-init.sh for initialise](#2-run-couchdb-initsh-for-initialise)
- [3. Expose CouchDB to the Internet](#3-expose-couchdb-to-the-internet)
- [3. Make CouchDB reachable from your devices](#3-make-couchdb-reachable-from-your-devices)
- [4. Client Setup](#4-client-setup)
- [1. Generate the setup URI on a desktop device or server](#1-generate-the-setup-uri-on-a-desktop-device-or-server)
- [2. Setup Self-hosted LiveSync to Obsidian](#2-setup-self-hosted-livesync-to-obsidian)
@@ -122,14 +122,14 @@ The `username` and `password` in this step are CouchDB administrator credentials
```
export hostname=http://localhost:5984
export username=<INSERT COUCHDB ADMINISTRATOR USERNAME HERE>
export password=<INSERT COUCHDB ADMINISTRATOR PASSWORD HERE>
export username='<INSERT COUCHDB ADMINISTRATOR USERNAME HERE>'
export password='<INSERT COUCHDB ADMINISTRATOR PASSWORD HERE>'
export database=obsidiannotes
curl -s https://raw.githubusercontent.com/vrtmrz/obsidian-livesync/main/utils/couchdb/couchdb-init.sh | bash
```
> [!TIP]
> In fish, `hostname` is read-only and cannot be exported. Pass the values through `env` instead: `curl -s https://raw.githubusercontent.com/vrtmrz/obsidian-livesync/main/utils/couchdb/couchdb-init.sh | env hostname=http://localhost:5984 username=<...> password=<...> database=obsidiannotes bash`.
> In fish, `hostname` is read-only and cannot be exported. Pass the values through `env` instead: `curl -s https://raw.githubusercontent.com/vrtmrz/obsidian-livesync/main/utils/couchdb/couchdb-init.sh | env hostname='http://localhost:5984' username='<INSERT COUCHDB ADMINISTRATOR USERNAME HERE>' password='<INSERT COUCHDB ADMINISTRATOR PASSWORD HERE>' database=obsidiannotes bash`.
If it results like the following:
```
@@ -140,21 +140,25 @@ The wrapper runs the exact registry-pinned Commonlib consumer. When `database` i
If you are using Docker Compose and the above command does not work or displays `ERROR: Hostname missing`, you can try running the following command, replacing the placeholders with your own values:
```
curl -s https://raw.githubusercontent.com/vrtmrz/obsidian-livesync/main/utils/couchdb/couchdb-init.sh | hostname=http://<YOUR SERVER IP>:5984 username=<INSERT COUCHDB ADMINISTRATOR USERNAME HERE> password=<INSERT COUCHDB ADMINISTRATOR PASSWORD HERE> database=obsidiannotes bash
curl -s https://raw.githubusercontent.com/vrtmrz/obsidian-livesync/main/utils/couchdb/couchdb-init.sh | hostname='http://<YOUR SERVER IP>:5984' username='<INSERT COUCHDB ADMINISTRATOR USERNAME HERE>' password='<INSERT COUCHDB ADMINISTRATOR PASSWORD HERE>' database=obsidiannotes bash
```
## 3. Expose CouchDB to the Internet
<a id="3-expose-couchdb-to-the-internet"></a>
- You can skip this instruction if you using only in intranet and only with desktop devices.
- For mobile devices, Obsidian requires a valid SSL certificate. Usually, it needs exposing the internet.
## 3. Make CouchDB reachable from your devices
Whatever solutions we can use. For simplicity, the following sample uses Cloudflare Zero Trust for testing.
- You can skip this step if you use CouchDB only on a trusted local network with desktop devices.
- For mobile devices, Obsidian requires HTTPS with a valid certificate. HTTPS does not require making CouchDB publicly accessible.
Choose how your devices reach your CouchDB server. You can use a [reverse proxy you operate](#reverse-proxies), private-network access, or a tunnel service of your choice. The [Docker profiles](../docker/README.md#profile-details) provide examples.
Cloudflare is one optional choice. The following example uses a [Quick Tunnel](https://developers.cloudflare.com/tunnel/get-started/quick-tunnels/) for temporary testing. It prints a random hostname which changes each time you create a tunnel, and the URL stops working when the `cloudflared` process stops. Quick Tunnels are intended for testing and development, not production. For ongoing use, configure your chosen connection method; the [Cloudflare Tunnel guide](https://developers.cloudflare.com/tunnel/get-started/) applies if you choose that provider.
```
cloudflared tunnel --url http://localhost:5984
```
You will then get the following output:
The following 2024 output is an example of the format. Its hostname is illustrative only; use the URL printed by your own tunnel instead.
```
2024-02-14T10:35:25Z INF Thank you for trying Cloudflare Tunnel. Doing so, without a Cloudflare account, is a quick way to experiment and try it out. However, be aware that these account-less Tunnels have no uptime guarantee. If you intend to use Tunnels in production you should use a pre-created named tunnel by following: https://developers.cloudflare.com/cloudflare-one/connections/connect-apps
@@ -167,7 +171,7 @@ You will then get the following output:
:
:
```
Now `https://tiles-photograph-routine-groundwater.trycloudflare.com` is our server. Make it into the background once, please.
Use the URL printed by your tunnel as the server address. Keep the tunnel process running while you configure and test the client.
## 4. Client Setup
@@ -178,13 +182,15 @@ Now `https://tiles-photograph-routine-groundwater.trycloudflare.com` is our serv
The `LIVESYNC_USER` and `LIVESYNC_PASS` here are the credentials which Self-hosted LiveSync will store for routine access to this database. They may be the administrator credentials from step 2. If you have separately configured a CouchDB account with the required access to this database, use that account instead. Neither the provisioning wrapper nor the Setup URI generator creates that separate account.
Use the server URL through which your devices reach CouchDB. For the temporary Quick Tunnel example above, use the URL printed by your own running tunnel.
```bash
export LIVESYNC_HOST=https://tiles-photograph-routine-groundwater.trycloudflare.com
export LIVESYNC_HOST='<YOUR COUCHDB SERVER URL>'
export LIVESYNC_DB=obsidiannotes
export LIVESYNC_USER=<INSERT COUCHDB USERNAME FOR LIVESYNC>
export LIVESYNC_PASS=<INSERT THE COUCHDB PASSWORD>
export LIVESYNC_PASSPHRASE=<INSERT A STRONG VAULT ENCRYPTION PASSPHRASE>
export uri_passphrase=<INSERT A SEPARATE SETUP URI PASSPHRASE> # Optional
export LIVESYNC_USER='<INSERT COUCHDB USERNAME FOR LIVESYNC>'
export LIVESYNC_PASS='<INSERT THE COUCHDB PASSWORD>'
export LIVESYNC_PASSPHRASE='<INSERT A STRONG VAULT ENCRYPTION PASSPHRASE>'
export uri_passphrase='<INSERT A SEPARATE SETUP URI PASSPHRASE>' # Optional
deno run --minimum-dependency-age=0 --allow-env https://raw.githubusercontent.com/vrtmrz/obsidian-livesync/main/utils/setup/generate_setup_uri.ts
```
@@ -199,9 +205,9 @@ The generator consumes the exact registry-pinned Commonlib release used by the p
By default, the URI is Ephemeral and can be opened only until the exact UTC time printed by the generator. This is the end of a fixed seven-day window, not seven days after creation. Set `uri_mode=persistent` before running the command if the URI needs no time condition or must be readable by an older client that supports the existing encrypted format.
You will then get the following output:
The output has the following form. The timestamp and values below are illustrative:
```bash
```text
Generated couchdb Setup URI.
Ephemeral: usable until 2026-10-01T00:00:00.000Z (UTC).
Your passphrase for the Setup URI is: H7vX...a-random-32-character-value
docs/setup_p2p.md
+5 -5
View File
@@ -127,11 +127,11 @@ The public Setup URI generator is also available when configuration must be crea
```sh
export remote_type=p2p
export p2p_relays=wss://relay.example.com
export p2p_room_id=<A PRIVATE ROOM ID> # Optional; generated when omitted
export p2p_passphrase=<A PRIVATE P2P PASSPHRASE> # Optional; generated when omitted
export LIVESYNC_PASSPHRASE=<A STRONG VAULT ENCRYPTION PASSPHRASE>
export uri_passphrase=<A SEPARATE SETUP URI PASSPHRASE>
export p2p_relays='wss://relay.example.com'
export p2p_room_id='<A PRIVATE ROOM ID>' # Optional; generated when omitted
export p2p_passphrase='<A PRIVATE P2P PASSPHRASE>' # Optional; generated when omitted
export LIVESYNC_PASSPHRASE='<A STRONG VAULT ENCRYPTION PASSPHRASE>'
export uri_passphrase='<A SEPARATE SETUP URI PASSPHRASE>'
deno run --minimum-dependency-age=0 --allow-env https://raw.githubusercontent.com/vrtmrz/obsidian-livesync/main/utils/setup/generate_setup_uri.ts
```
docs/specs_data_compression.md
+5 -1
View File
@@ -33,7 +33,11 @@ The current benchmark measures the Node CLI process. It does not establish Obsid
## Reproducible benchmark
The benchmark is implemented under `src/apps/cli/testdeno` and packaged by `test/bench-network`. It uses the real CLI mirror and synchronisation path, Commonlib 0.1.0-rc.4, CouchDB 3.5.0, the V3 Rabin–Karp splitter, optional Data Compression, and E2EE V2. It runs each of these conditions three times in rotating order:
The benchmark is implemented under `src/apps/cli/testdeno` and packaged by `test/bench-network`. It exercises the real CLI mirror and synchronisation path. The recorded three-repeat result below used Commonlib 0.1.0-rc.4, CouchDB 3.5.0, the V3 Rabin–Karp splitter, optional Data Compression, and E2EE V2.
The current Compose runner installs the checked-out repository's package and lockfile with `npm ci`; Self-hosted LiveSync 1.0.42 selects Commonlib 0.2.2. Running the command below measures that checkout, rather than recreating the older package environment automatically. Keep the recorded results associated with their original measurement date and dependency version.
With the repeat count shown below, it runs each of these conditions three times in rotating order:
- E2EE off, compression off;
- E2EE off, compression on;
docs/tips/fast-setup.md
+11 -11
View File
@@ -1,8 +1,8 @@
# Fast Setup (Simple Fetch)
Fast Setup is a streamlined, user-friendly data retrieval and initialisation flow designed to simplify setting up secondary devices or recovering databases.
Fast Setup (Simple Fetch) is the preferred flow for initial replication to a new or empty secondary device. It can also be used for recovery, but a retrieval choice may replace local files. Preserve valuable local and remote copies before choosing how to handle a Vault that already contains work. See [Recovery and flag files](../recovery.md#first-choose-the-authoritative-copy) for guidance on identifying the authoritative copy.
Instead of guiding the user through the detailed multi-step setup wizard dialogues, Fast Setup prompts the user with high-level sync decisions and automates database download and local storage scanning in one continuous process.
Instead of guiding the user through the detailed multi-step setup wizard dialogues, Fast Setup prompts the user with high-level synchronisation decisions and automates data retrieval and local storage scanning in one process. The retrieval path depends on the selected remote.
---
@@ -16,14 +16,14 @@ On the next startup, the plug-in boots in scheduled fetch mode and opens a simpl
## Technical Characteristics
Fast Setup leverages several backend optimisations to make the retrieval fast, safe, and clean:
Fast Setup uses several retrieval and reflection behaviours:
1. **Stream-based Replication for Speed**
- It fetches all remote metadata via stream reception, which is significantly faster than traditional chunk-by-chunk retrieval.
1. **Stream-based Fast Fetch for CouchDB**
- The stream-based Fast Fetch path is CouchDB-specific. It retrieves remote Metadata incrementally; Object Storage and P2P use their own retrieval paths.
2. **Delayed File Reflection to Prevent Corrupted Warnings**
- By suspending file reflection during the download phase, it prevents the plug-in from raising temporary or false "corrupted data synchronisation" or "size mismatch" warnings that can occur during the chunk download process.
3. **Time-Based Comparison is Generally Sufficient**
- Since the vault is entering a fresh synchronisation or recovery state, comparing files based on their modification timestamps (newer-wins) is highly reliable and sufficient to reconcile files without needing complex manual conflict resolution.
- Delaying local file reflection during retrieval suppresses temporary synchronisation warnings while remote data is being fetched.
3. **Timestamp Comparison Selects a Whole File**
- **Compare time and take newer** compares modification timestamps and selects one whole-file version. It does not merge file contents. Before choosing this policy for a Vault with existing work, preserve valuable local and remote edits and review [Recovery and flag files](../recovery.md#first-choose-the-authoritative-copy).
---
@@ -33,11 +33,11 @@ Fast Setup leverages several backend optimisations to make the retrieval fast, s
You will be prompted to choose how the retrieved remote data will interact with your existing local files:
1. **Compare time and take newer (newer-wins)**
- Compares the modified time of files and accepts the newer version.
- **Recommended if:** You have been using Self-hosted LiveSync and have made changes on multiple devices that you want to merge.
- Compares the modified time of files and selects the newer whole-file version; it does not combine edits from both versions.
- **Recommended if:** You have preserved both versions and deliberately want to select one according to its modification time.
2. **Overwrite all with remote files (remote-wins)**
- Remote data is treated as the source of truth.
- **Recommended if:** You are setting up a brand new device with an empty or clean vault.
- **Recommended if:** You are setting up a new secondary device with an empty Vault.
- *Warning: This will overwrite local files with remote files. Please ensure you have a backup of your local vault before proceeding.*
3. **Use the detailed flow (legacy)**
- Switches back to the detailed, traditional setup wizard dialogues.
docs/tips/status-display.md
+55
View File
@@ -0,0 +1,55 @@
# Synchronisation status and CSS customisation
Use [General Settings](../settings.md#2-general-settings) to choose whether synchronisation status appears in the status bar, inside the editor, or both. The following sections explain the indicators and how to customise their appearance.
## Status indicators
- Activity Indicator
- 📲 A finite remote operation is in progress
- 🌐N Approximate remote requests currently in progress
- Status
- ⏹️ Stopped
- 🌀 Starting synchronisation
- ⏩ Comparing CouchDB change history, including entries already held locally
- 💤 LiveSync enabled. Waiting for changes
- ⚡️ Synchronisation in progress
- ⚠ An error occurred
- Statistical Indicators
- ↑ Uploaded chunks and metadata
- ↓ Downloaded chunks and metadata
- Progress Indicators
- 📥 Unprocessed transferred items
- 📄 Working database operation
- 💾 Working write storage processes
- ⏳ Working read storage processes
- 🛫 Pending read storage processes
- 📬 Batched read storage processes
- ⚙️ Working or pending storage processes for hidden files
- 🛄 Pending initial on-demand chunk requests
- 🔁 Pending chunk retries, including retry delays
- 🔌 Working customisation items (configuration, snippets, and plug-ins)
To prevent file and database corruption, please avoid closing Obsidian until all progress indicators have disappeared as much as possible (although the plug-in will attempt to resume if interrupted). This is especially important if you have deleted or renamed files.
## CouchDB comparison count
When CouchDB change history is being compared, `(⏩ 1200) ↑ 0 ↓ 0` can appear before any new documents arrive. The count is the number of candidate changes-feed entries compared during the current replication attempt, including repeated entries on retries. It is separate from downloaded Chunks and Metadata, and is neither a Vault file count nor a count of entries already present locally. It appears before the upload count, disappears after three seconds without a change, and reappears when the count changes. The ⏩ status icon follows comparison activity independently of that timer.
## Customising status items with CSS snippets
The status bar and editor overlay use individual spans with the common `livesync-stat-` class prefix. CSS snippets can hide or style individual items. For example, this hides the comparison count and P2P summary:
```css
.livesync-stat-ff,
.livesync-stat-p2p {
display: none;
}
```
The item classes include `livesync-stat-network`, `livesync-stat-sync`, `livesync-stat-sent`, `livesync-stat-push-remaining`, `livesync-stat-arrived`, `livesync-stat-pull-remaining`, `livesync-stat-ff`, and `livesync-stat-p2p`. File-processing items use `livesync-stat-file-processing`, `livesync-stat-file-pending`, and `livesync-stat-file-batched`; queue items use `livesync-stat-queue-` followed by `replication`, `database`, `storage`, `chunk-initial`, `chunk-retrying`, `plugin-scan`, `hidden-files`, or `conflicts`. The skipped-file indicator uses `livesync-stat-file-skip`.
All items also have `livesync-stat-item`. Numeric children use `livesync-stat-value`, tabular digits, and right alignment. Each numeric item retains its widest displayed digit count during an active period, reducing movement as counts decrease. The editor overlay uses wider gaps between items, while remaining counts stay beside their transfer totals.
Inactive items receive `livesync-stat-inactive` instead of the HTML `hidden` attribute. The default CSS hides them immediately. They retain their last text, numeric width, and position amongst the other items so that CSS snippets can customise how they disappear. The numeric width starts afresh when an item becomes active again.
See the [documentation index](../README.md) for setup and troubleshooting guides.
src/apps/cli/testdeno/test_dev_deno.md
+3 -3
View File
@@ -146,7 +146,7 @@ These flows do not require Bash on the host. The S3 matrix tasks, environment va
### `test-mirror.ts`
- Verifies six core mirror scenarios.
- Verifies seven core mirror scenarios, including an unrecognised revision origin.
- Does not require an external remote.
### `test-sync-two-local-databases.ts`
@@ -247,11 +247,12 @@ deno task test:daemon
deno task test:push-pull
deno task test:sync-locked-remote
# CouchDB-based tests
# CouchDB and Object Storage tests
deno task test:sync-two-local
deno task test:decoupled-vault
deno task test:remote-commands
deno task test:e2e-couchdb
deno task test:e2e-matrix
# Explicit host-direct P2P diagnostics for cross-platform investigations.
# These are not part of the default suite or release evidence.
@@ -260,7 +261,6 @@ deno task test:p2p-peers
deno task test:p2p-sync
deno task test:p2p-three-nodes
deno task test:p2p-upload-download
deno task test:e2e-matrix
```
---
src/apps/webpeer/README.md
+2
View File
@@ -49,6 +49,8 @@ After the first device connects, **Try another device without resetting** keeps
Serve the production build over HTTPS or from `localhost`. Downloading `check.html` alone is not supported because the page uses built module assets and origin-scoped browser storage. The generated credentials are temporary and must not replace the settings of a production Vault.
The generated Setup URI is Time-bound, and the page shows when its current fixed UTC window ends. After that time, the page hides the URI and QR code and disables the additional-device action. Start a fresh check to configure another device. A monitor for a device which already imported the settings remains available.
The detailed scope and result semantics are recorded in the [browser-assisted P2P connection-check ADR](../../../docs/adr/2026_07_p2p_connection_check.md).
## Storage and lifecycle
utils/readme.md
+5 -5
View File
@@ -55,13 +55,13 @@ The generator defaults to `uri_mode=ephemeral`. The URI opens only during the cu
```sh
export remote_type=s3
export endpoint=https://objects.example.com
export access_key=<ACCESS KEY>
export secret_key=<SECRET KEY>
export endpoint='https://objects.example.com'
export access_key='<ACCESS KEY>'
export secret_key='<SECRET KEY>'
export bucket=vault-data
export region=auto
export bucket_prefix=team-a # Optional
export LIVESYNC_PASSPHRASE=<A STRONG VAULT ENCRYPTION PASSPHRASE>
export LIVESYNC_PASSPHRASE='<A STRONG VAULT ENCRYPTION PASSPHRASE>'
deno run --minimum-dependency-age=0 --config=./flyio/deno.jsonc --frozen --lock=./flyio/deno.lock --allow-env ./setup/generate_setup_uri.ts
```
@@ -71,7 +71,7 @@ Optional Object Storage variables are `use_custom_request_handler`, `force_path_
```sh
export remote_type=p2p
export LIVESYNC_PASSPHRASE=<A STRONG VAULT ENCRYPTION PASSPHRASE>
export LIVESYNC_PASSPHRASE='<A STRONG VAULT ENCRYPTION PASSPHRASE>'
deno run --minimum-dependency-age=0 --config=./flyio/deno.jsonc --frozen --lock=./flyio/deno.lock --allow-env ./setup/generate_setup_uri.ts
```