Compare commits

...

94 Commits

Author SHA1 Message Date
vorotamoroz a722244a92 fix(cli): replace the headless log handler 2026-08-01 06:20:47 +00:00
vorotamoroz 4f14d7f160 feat(cli): support adaptive WebDAV remotes 2026-08-01 06:09:32 +00:00
vorotamoroz 7955ab54d3 refactor: compose Journal remotes generically 2026-08-01 06:09:02 +00:00
vorotamoroz 133ef34d7b docs: model adaptive WebDAV trade-offs 2026-08-01 04:16:37 +00:00
vorotamoroz 0890a97222 test(obsidian): cover adaptive S3 journal upload 2026-08-01 03:52:57 +00:00
vorotamoroz b7da78ce45 feat: expose adaptive journal options for S3 2026-07-31 19:24:40 +00:00
vorotamoroz 53d4bd527e docs: split adaptive journal decisions by provider 2026-07-31 19:24:40 +00:00
vorotamoroz 64870a32ea docs: specify adaptive journal sync design 2026-07-31 19:24:40 +00:00
vorotamoroz 5ede11d032 docs: plan adaptive journal synchronisation 2026-07-31 19:24:40 +00:00
vorotamoroz bcf123c67a test(cli): exercise adaptive S3 pack reads 2026-07-31 19:24:19 +00:00
vorotamoroz f3b0fbb29b test(cli): cover adaptive journal S3 end to end 2026-07-31 19:24:19 +00:00
vorotamoroz 73e34aef7c fix(cli): verify adaptive journal remotes 2026-07-31 19:24:19 +00:00
vorotamoroz 166e431103 fix(cli): persist adaptive writer state 2026-07-31 19:24:19 +00:00
vorotamoroz 3ba9feaac7 Merge pull request #1063 from vrtmrz/feature/p2p-connection-check
Add a browser-assisted P2P connection check
2026-08-01 03:51:06 +09:00
vorotamoroz 42f37753b8 Address WebPeer CSS review warnings 2026-07-31 17:56:26 +00:00
vorotamoroz ec1fc4a97e Fix WebPeer interop test selector 2026-07-31 16:37:09 +00:00
vorotamoroz 146e25907d Link WebPeer from English README 2026-07-31 15:14:00 +00:00
vorotamoroz c97346e262 Unify WebPeer visual design 2026-07-31 15:05:22 +00:00
vorotamoroz 48c398948a Add browser P2P connection check 2026-07-31 14:13:34 +00:00
vorotamoroz e32e3f545e Merge pull request #1062 from vrtmrz/fix/release-promotion-order
Fix release promotion order and remediation rendering
2026-07-31 19:41:01 +09:00
vorotamoroz 0aed7f8556 Add remediation fix to unreleased notes 2026-07-31 10:35:05 +00:00
vorotamoroz 50a51bf07e Use setting control for remediation status 2026-07-31 10:08:34 +00:00
vorotamoroz 1570136703 Fix remediation status element insertion 2026-07-31 09:13:56 +00:00
vorotamoroz 5b6bbc8118 Remove release instruction text assertions 2026-07-31 09:13:21 +00:00
vorotamoroz 54f0262b3f Fix stable release promotion order 2026-07-31 08:55:26 +00:00
vorotamoroz 7d33613017 Merge pull request #1061 from vrtmrz/1_0_2
Releasing 1.0.2
2026-07-31 17:39:40 +09:00
vorotamoroz b700c631ca Releasing 1.0.2 2026-07-31 07:28:27 +00:00
vorotamoroz 00152523db Merge pull request #1015 from zeedif/refactor/i18n-source-keys
refactor(i18n): migrate UI text to source-based localization keys
2026-07-31 15:45:59 +09:00
vorotamoroz fbfb77440d Merge branch 'main' into pr/zeedif/1015 2026-07-31 07:14:14 +01:00
vorotamoroz 89d4667d95 Merge pull request #1059 from zeedif/i18n/spanish-catalogue-coverage
i18n(es): complete the Spanish catalogue coverage
2026-07-31 14:06:38 +09:00
vorotamoroz 21dd24682b Merge branch 'main' into pr/zeedif/1059 2026-07-31 06:00:59 +01:00
vorotamoroz 5c52d69d66 Merge pull request #1055 from motolies/complete-korean-translations
Complete and correct the Korean translations
2026-07-31 13:44:52 +09:00
vorotamoroz a251648e48 Merge pull request #1009 from SeleiXi/feat/history-rev-nav
feat: Add revision prev/next buttons next to history slider
2026-07-31 13:43:14 +09:00
vorotamoroz 57c9726ae5 Merge pull request #1032 from apple-ouyang/fix/keep-replicated-document-apply-active
Keep replicated document application active
2026-07-31 13:41:07 +09:00
vorotamoroz a6977da9e8 Merge pull request #1058 from vrtmrz/issue-1056-preserve-utf8-bom
Preserve UTF-8 BOMs during Vault ingestion
2026-07-31 13:39:27 +09:00
vorotamoroz 397a9b69c8 Merge pull request #1060 from vrtmrz/fix/issue-1036-octagonal-wheels-0.1.52
Use octagonal-wheels 0.1.52 for Node base64 fallback
2026-07-31 13:39:11 +09:00
vorotamoroz ce573f18a6 Update octagonal-wheels for Node base64 fallback 2026-07-31 03:13:53 +00:00
Ouyang Xingyuan 0df1f57f3b fix: keep replicated document application active
Why:
- Large downloaded batches can outlive the finite remote operation and be suspended before every document reaches the Vault.
- The v1.0 activity contract requires local application to remain distinct from remote-operation reporting.

Changes:
- Add an Obsidian-owned local application activity boundary which reuses Wake Lock policy without incrementing remote activity.
- Retain one boundary through queue processing and the final recovery snapshot, releasing and reacquiring it around suspension.
- Log final snapshot failures, preserve processing results, and cover the lifecycle with focused tests and documentation.
2026-07-31 11:06:18 +08:00
Zeedif 0518fab62a i18n(es): complete the Spanish catalogue coverage
Fills every untranslated key in the Spanish catalogue: coverage goes from
643/1132 to 1131/1132 against `en.yaml`.

- Translate the 488 keys that had no Spanish entry, covering the Config
  Doctor, tweak-mismatch resolution, the P2P notes and panes, the Setup
  Wizard, the maintenance and patch panels, Garbage Collection V3 and the
  migration dialogues.
- Register follows the existing catalogue: informal address, `es-ES`
  vocabulary, British-style source wording preserved where it is a proper
  noun (Setup URI, Vault, chunk).
- Placeholders (`${...}`), keyword references (`%{...}`) and Markdown
  callout syntax are kept byte-identical to the English source.
- Regenerate `messagesJson/es.json` and `combinedMessages.prod.ts` with the
  documented `i18n:bake` pipeline.

`obsidianLiveSyncSettingTab.logServerConfigurationCheck` is deliberately left
untranslated: its English value is the key itself, so it looks like a
placeholder rather than a message.

Regenerating the catalogue also normalises three pre-existing entries each in
`ko.json` and `zh.json`, where the committed JSON kept a trailing space before
a newline that YAML cannot represent.

Verified with tsc-check and test:unit.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-07-30 19:09:17 -06:00
vorotamoroz e296708845 Preserve UTF-8 BOMs during Vault ingestion 2026-07-31 01:08:56 +00:00
Zeedif b6746be73e refactor(i18n): route remaining Obsidian UI text through the message catalogue
Continues the source-key migration inside the application boundary introduced
in 1.0.0, where LiveSync owns its catalogue and consumes Commonlib as a
published package.

- Replace hardcoded user-visible strings in the Obsidian UI (Setup Wizard
  dialogues, P2P panes, Customisation Sync panes, Global History, the JSON
  conflict pane and the remote-configuration menu) with `$msg` calls, keeping
  the English source string as the key.
- Wire up strings whose translations already existed in the catalogue but were
  still rendered as literals, for example the whole Intro dialogue.
- Add the new entries to `src/common/messagesYAML/en.yaml` and `es.yaml`, then
  regenerate `messagesJson/` and `combinedMessages.prod.ts` through the
  documented `i18n:bake` pipeline.
- No Commonlib gitlink or catalogue is involved; every change is
  LiveSync-owned.

Regenerating the catalogue also normalises three pre-existing entries each in
`ko.json` and `zh.json`, where the committed JSON kept a trailing space before
a newline that YAML cannot represent.

Verified with tsc-check, tsc-check:apps, svelte-check and lint.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-07-30 18:59:19 -06:00
vorotamoroz c385bd7ce7 Merge pull request #1054 from vrtmrz/refactor/browser-app-composition
Compose browser application runtimes and restore standard Notice placement
2026-07-30 18:01:07 +09:00
vorotamoroz b21c3114e5 fix: clarify browser application guidance 2026-07-30 05:53:42 +00:00
Namwon Kim 17fe6c1518 Complete and correct the Korean translations
Korean coverage was 796 of 1132 keys (70%), so a large part of the UI fell
back to English. Alongside the missing entries, the existing translations
carried a few defects that were visible to users.

Fixes:

- Three messages had lost the `$` from their placeholders, leaving a literal
  `{origin}`, `{org}` and `{from}->{to}` on screen, because `$msg()` only
  substitutes `${...}`.
- `dialog.yourLanguageAvailable` referenced `%{Display language}`, but the key
  is `Display Language`, so the raw token was shown. Corrected to match ja.
- Sixteen onboarding messages ended with the Japanese full stop `。`.
- Four messages hard-coded a particle after a variable (`${measuredSize}가`),
  which is ungrammatical for values ending in a consonant. They now use the
  paired form `이(가)`, `을(를)`, `(으)로`.

Terminology is now consistent with the Obsidian Korean UI and with the
definitions in docs/terms.md:

- `vault` is 보관함, `device` is 기기, `status bar` is 상태 표시줄.
- `chunk` is 청크 everywhere; the Eden settings had rendered it as 변경 기록
  ("change record"), which is not what a chunk is.
- `Compaction` is 압축 정리, separating it from `Data Compression` (데이터 압축),
  which had shared the same word.
- HTTP `Origin` is 출처; it had been 원점, the geometric origin.
- Six English strings that had mapped to two different Korean strings each are
  now unified.

The remaining 336 keys are translated, including the setup wizard outro, the
insecure-chunk and v0.25.6 recovery dialogues, and the database adapter
migration messages.

This ports vrtmrz/livesync-commonlib#79 to this repository, following the
v1.0.0 boundary change that moved application translations here. On top of
the original work, the 17 keys newly introduced in v1.0.0 (the P2P setup
wizard Invitation, OutroNewP2PUser and RebuildEverythingP2P dialogues, and
the moduleCheckRemoteSize notices) are translated as well.

Verified with _tools/checkI18nCoverage.ts (ko: 1132/1132, 0 missing),
tsc --noEmit, and a dev build. Generated resources are committed as produced
by `npm run i18n:bake`; besides the `ko` entries it also trims a stale
trailing space from five es/zh entries in combinedMessages.prod.ts that had
been left behind by the migration.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EtFsro96LHztmoYmRjtfAy
2026-07-30 09:15:41 +09:00
vorotamoroz d4994fdd05 Releasing 1.0.1 2026-07-29 15:57:33 +00:00
SeleiXi 93bbab4253 fix(history): align revision navigation with v1.0
Use the preferred Obsidian element helper and preseed the real-Obsidian scenario before startup. Register the focused test and strengthen revision boundary coverage.
2026-07-29 23:45:28 +08:00
SeleiXi 8b7d410327 chore: merge upstream main into history revision branch 2026-07-29 23:45:12 +08:00
vorotamoroz 68b6358345 build: use stable browser UI kit 0.1.0 2026-07-29 15:16:19 +00:00
vorotamoroz 29c26c43b7 test: validate packaged Pages application subpaths 2026-07-29 11:13:10 +00:00
vorotamoroz 72197a5df0 fix: use standard onboarding notice placement 2026-07-29 09:12:27 +00:00
vorotamoroz 4723771ee0 refactor: compose browser application runtimes 2026-07-29 09:11:57 +00:00
vorotamoroz b22c1bd777 Merge pull request #1052 from vrtmrz/v1_0_0
Integrate Self-hosted LiveSync 1.0.0 into main
2026-07-27 20:14:33 +09:00
vorotamoroz 8876213127 Merge pull request #1033 from vrtmrz/common-library-package-boundary
Consume Commonlib through its published package boundary
2026-07-27 19:48:56 +09:00
vorotamoroz 61ffdde9b5 Merge pull request #1051 from vrtmrz/1_0_0
Releasing 1.0.0
2026-07-27 19:33:37 +09:00
vorotamoroz f1e382c6ed Check repository tools with community lint 2026-07-27 09:07:20 +00:00
vorotamoroz 2b95766d4f Correct stable release promotion order 2026-07-27 08:28:16 +00:00
vorotamoroz c3f2163204 Release 1.0.0 2026-07-27 08:12:33 +00:00
vorotamoroz d5a40a7e3d release: prepare 1.0.0-rc.1 2026-07-27 03:37:46 +00:00
vorotamoroz 3ce6ccf7a6 Polish 1.0.0-rc.0 release notes 2026-07-27 01:57:25 +00:00
github-actions[bot] 8c6824b7f6 Releasing 1.0.0-rc.0 2026-07-27 01:36:33 +00:00
vorotamoroz e35583a752 Reconcile beta.5 metadata for release candidate 2026-07-27 00:52:15 +00:00
vorotamoroz 866575add5 Correct staged release and merge gates 2026-07-26 23:34:52 +00:00
vorotamoroz 4f01e7f687 Keep startup scan on a configured Vault 2026-07-26 16:16:26 +00:00
vorotamoroz 18e4d16c6f Use the stable Commonlib package 2026-07-26 16:15:46 +00:00
vorotamoroz da4b188b38 Clarify the file inspection action 2026-07-26 15:48:01 +00:00
vorotamoroz 12061b7bf3 Reconcile beta.4 metadata for the next preview 2026-07-26 08:28:44 +00:00
vorotamoroz 51a749099d Verify P2P pane in a fresh mobile session 2026-07-26 08:06:48 +00:00
vorotamoroz 4194a0067c Stabilise configured Real Obsidian workflows 2026-07-26 08:06:08 +00:00
vorotamoroz ad762d4ddf Update Commonlib and Obsidian test-session packages 2026-07-26 08:05:39 +00:00
vorotamoroz 127d460e18 Improve conflict inspection and recovery workflow 2026-07-25 11:18:10 +00:00
vorotamoroz 12fc43a69c Improve recovery diagnostics and actions 2026-07-25 09:01:25 +00:00
vorotamoroz bea0e68091 Keep translation details out of startup path 2026-07-25 04:34:53 +00:00
vorotamoroz 99eb9cd46f Record beta.3 and current validation 2026-07-25 03:49:39 +00:00
vorotamoroz a7bc337fd1 Test Security Seed refresh in Real Obsidian 2026-07-25 03:49:18 +00:00
vorotamoroz 0d68b6530f Test Garbage Collection V3 against CouchDB 2026-07-25 03:48:51 +00:00
vorotamoroz c24fc1dc82 Update Commonlib to 0.1.0-rc.12 2026-07-24 19:06:53 +00:00
vorotamoroz c8162fd0cd Make dialogue prose selectable 2026-07-24 17:53:55 +00:00
vorotamoroz 0a7a3b1635 Harden Garbage Collection V3 validation 2026-07-24 17:53:22 +00:00
vorotamoroz 20fb027055 Protect conflict chunks during garbage collection 2026-07-24 17:09:22 +00:00
vorotamoroz 07cba4ce83 Keep unreadable revisions for explicit repair 2026-07-24 16:15:46 +00:00
vorotamoroz 658ad6dfe4 Describe saved connections consistently 2026-07-24 16:15:02 +00:00
vorotamoroz 0e5475b7e3 Limit commands to applicable contexts 2026-07-24 16:13:49 +00:00
vorotamoroz 6afeb0b409 Record beta.2 history and remove release-note links 2026-07-24 09:54:44 +00:00
vorotamoroz d489452583 Improve troubleshooting and compatible setting handling 2026-07-24 08:51:30 +00:00
vorotamoroz 058da1f34c Polish setup guide wording 2026-07-24 05:55:37 +00:00
vorotamoroz 32b72e4b10 Normalise Chinese setup guide line endings 2026-07-24 05:39:40 +00:00
vorotamoroz bdc44920ac Refresh setup guides and capture fixtures 2026-07-24 05:37:13 +00:00
vorotamoroz 24a4ebb8dd Use obsidian-test-session 0.2.5 2026-07-24 03:04:39 +00:00
vorotamoroz 68d22ade76 Use established terms in setup guidance 2026-07-23 17:38:16 +00:00
vorotamoroz bf0fc0aea8 Retire orphaned WebApp tests and classify regression guards 2026-07-23 17:21:16 +00:00
vorotamoroz 24b941f594 Group Hidden File Sync initialisation progress 2026-07-23 16:47:30 +00:00
vorotamoroz c8050322e4 Clarify remote setup and verify manual CouchDB onboarding 2026-07-23 16:46:50 +00:00
vorotamoroz 1df034f12a Refine P2P and manual setup workflows 2026-07-23 15:23:47 +00:00
SeleiXi c4d6b768da Add revision prev/next buttons next to history slider 2026-07-14 00:26:59 +08:00
314 changed files with 30696 additions and 7013 deletions
+40 -1
View File
@@ -7,10 +7,18 @@ on:
- main
paths:
- 'aggregator.html'
- 'src/**'
- 'test/browser-apps/**'
- 'package.json'
- 'package-lock.json'
- '.github/workflows/deploy-pages.yml'
pull_request:
paths:
- 'aggregator.html'
- 'src/**'
- 'test/browser-apps/**'
- 'package.json'
- 'package-lock.json'
- '.github/workflows/deploy-pages.yml'
permissions:
@@ -30,6 +38,29 @@ jobs:
- name: Checkout
uses: actions/checkout@v4
- name: Setup Node.js
uses: actions/setup-node@v4
with:
node-version: '24.x'
cache: 'npm'
- name: Setup Deno
uses: denoland/setup-deno@v2
with:
deno-version: v2.9.2
- name: Install dependencies
run: npm ci
- name: Install Chromium
run: npx playwright@1.62.0 install --with-deps chromium
- name: Validate browser applications
run: npm run test:browser-apps
- name: Validate browser application interoperability
run: npm run test:e2e:browser-apps:interop
- name: Validate aggregator
run: |
test -s aggregator.html
@@ -41,10 +72,18 @@ jobs:
- name: Prepare Pages site
run: |
mkdir -p _site
mkdir -p _site/webapp _site/webpeer
cp aggregator.html _site/aggregator.html
cp -R src/apps/webapp/dist/. _site/webapp/
cp -R src/apps/webpeer/dist/. _site/webpeer/
test -s _site/webapp/index.html
test -s _site/webapp/webapp.html
test -s _site/webpeer/index.html
touch _site/.nojekyll
- name: Validate packaged Pages site
run: npm run test:browser-apps:pages
- name: Configure GitHub Pages
uses: actions/configure-pages@v5
+22 -3
View File
@@ -62,6 +62,7 @@ jobs:
VERSION: ${{ inputs.version }}
EXPECTED_HEAD_SHA: ${{ inputs.expected_head_sha }}
PRERELEASE: ${{ inputs.prerelease }}
PUBLISH_CLI: ${{ inputs.publish_cli }}
run: |
set -euo pipefail
ACTUAL_HEAD_SHA="$(git rev-parse HEAD)"
@@ -73,6 +74,10 @@ jobs:
echo "Version ${VERSION} is a pre-release version, but prerelease was not enabled." >&2
exit 1
fi
if [[ "${VERSION}" != *-* && "${PRERELEASE}" == "true" && "${PUBLISH_CLI}" == "true" ]]; then
echo "A stable version staged as a pre-release must use publish_cli=false so that the CLI latest and major-minor image tags do not advance before BRAT validation." >&2
exit 1
fi
node utils/release-notes.mjs validate "${VERSION}"
- name: Ensure and push release tags
@@ -96,8 +101,15 @@ jobs:
GH_TOKEN: ${{ github.token }}
VERSION: ${{ inputs.version }}
PRERELEASE: ${{ inputs.prerelease }}
PUBLISH_CLI: ${{ inputs.publish_cli }}
run: |
set -euo pipefail
if [[ "${PUBLISH_CLI}" == "true" ]]; then
gh workflow run cli-docker.yml \
--ref "${VERSION}-cli" \
--field dry_run=false \
--field force=false
fi
gh workflow run release.yml \
--ref "${VERSION}" \
--field tag="${VERSION}" \
@@ -113,15 +125,22 @@ jobs:
{
echo "Ensured the plug-in tag \`${VERSION}\` points to the reviewed release commit."
if [[ "${PUBLISH_CLI}" == "true" ]]; then
echo "The CLI tag \`${VERSION}-cli\` was also created; its tag event starts the container workflow."
echo "The CLI tag \`${VERSION}-cli\` was also created, and finalisation explicitly dispatched the CLI container workflow."
else
echo "CLI publication was omitted."
fi
echo ""
echo "Dispatched the plug-in release workflow for \`${VERSION}\`. After approval for the release environment, it creates a draft GitHub Release."
echo ""
if [[ "${PRERELEASE}" == "true" ]]; then
echo "Publish the draft as a pre-release, keep the release pull request in draft, and merge only after BRAT validation succeeds."
if [[ "${VERSION}" == *-* ]]; then
echo "Publish the draft as a pre-release without replacing the latest stable release."
echo "Keep the release pull request in draft and unmerged after BRAT validation; close it only through a separate maintainer action."
elif [[ "${PRERELEASE}" == "true" ]]; then
echo "Publish the draft initially as a pre-release without replacing the latest stable release."
echo "After BRAT validation, remove the pre-release designation and make this exact release the latest stable release before its manifest reaches the default branch."
echo "Confirm the release is no longer a pre-release, then merge the release pull request into its reviewed base branch and integrate the exact release commit into the default branch."
echo "This order prevents Community Directory review from inspecting a stable manifest version while the matching GitHub Release remains a pre-release."
echo "Create the stable CLI tag and publish its latest and major-minor image tags through a separate maintainer gate."
else
echo "Publish the draft as the latest stable release, keep the release pull request in draft, and merge only after BRAT validation succeeds."
fi
+5
View File
@@ -38,6 +38,11 @@ Before submitting a pull request, you must run verification scripts locally to e
```bash
npm run test:unit
```
- When changing the troubleshooting or recovery guides, inspect their current English UI labels and local references:
```bash
npm run inspect:troubleshooting
```
This read-only Inspector prints JSON containing `ok`, `checkedFiles`, `checkedLocalReferences`, and `errors`, and exits unsuccessfully when a contract is stale.
If you have the capability and a suitable environment (such as Linux and Docker), running the CLI End-to-End (E2E) tests is also highly appreciated. Instructions are detailed in [devs.md](devs.md). If you cannot run E2E tests locally, please explicitly ask to run the tests on the CI by stating 'Please run CI tests' in your pull request description.
+10 -12
View File
@@ -4,7 +4,7 @@
Self-hosted LiveSync is a community-developed synchronisation plug-in available on all Obsidian-compatible platforms. It leverages robust server solutions such as CouchDB or object storage systems (e.g., MinIO, S3, R2, etc.) to ensure reliable data synchronisation.
Additionally, it supports peer-to-peer synchronisation using WebRTC, enabling you to synchronise your notes directly between devices without relying on a server. Documentation is available for [Peer-to-Peer Synchronisation](./docs/p2p_sync_updates_2026.md).
Additionally, it supports peer-to-peer synchronisation using WebRTC, enabling devices to exchange notes without a central data-storage server. A signalling relay is still required for peer discovery. See [How peer-to-peer synchronisation works](./docs/p2p.md).
![obsidian_live_sync_demo](https://user-images.githubusercontent.com/45774780/137355323-f57a8b09-abf2-4501-836c-8cb7d2ff24a3.gif)
@@ -19,14 +19,11 @@ Additionally, it supports peer-to-peer synchronisation using WebRTC, enabling yo
- Compatible solutions are supported.
- Support end-to-end encryption.
- Synchronise settings, snippets, themes, and plug-ins via [Customisation Sync (Beta)](docs/settings.md#6-customisation-sync-advanced) or [Hidden File Sync](docs/tips/hidden-file-sync.md).
- Enable WebRTC peer-to-peer synchronisation without requiring a `host` (Experimental).
- This feature is still in the experimental stage. Please exercise caution when using it.
- WebRTC is a peer-to-peer synchronisation method, so **at least one device must be online to synchronise**.
- Instead of keeping your device online as a stable peer, you can use two pseudo-peers:
- [livesync-serverpeer](https://github.com/vrtmrz/livesync-serverpeer): A pseudo-client running on the server for receiving and sending data between devices.
- [webpeer](https://github.com/vrtmrz/obsidian-livesync/tree/main/src/apps/webpeer): A pseudo-client for receiving and sending data between devices.
- A pre-built instance is available at [fancy-syncing.vrtmrz.net/webpeer](https://fancy-syncing.vrtmrz.net/webpeer/) (hosted on the vrtmrz's blog site). This is also peer-to-peer. Feel free to use it.
- For more information, refer to the [English explanatory article](https://fancy-syncing.vrtmrz.net/blog/0034-p2p-sync-en.html) or the [Japanese explanatory article](https://fancy-syncing.vrtmrz.net/blog/0034-p2p-sync).
- Enable supported, opt-in WebRTC peer-to-peer synchronisation.
- No central data-storage server is required, but a signalling relay is still required for peer discovery.
- At least one device containing the required data must be online while another device synchronises.
- Follow the [Peer-to-Peer Setup](docs/setup_p2p.md) after reviewing the [P2P communication model](docs/p2p.md).
- If you would like to see how P2P setup and connectivity behave on your devices, try the guided connection check in the experimental [WebPeer browser utility](https://vrtmrz.github.io/obsidian-livesync/webpeer/). It uses an empty Vault and does not test note synchronisation. See the [WebPeer documentation](src/apps/webpeer/README.md) for its scope and limitations.
This plug-in may be particularly useful for researchers, engineers, and developers who need to keep their notes fully self-hosted for security reasons. It is also suitable for anyone seeking the peace of mind that comes with knowing their notes remain entirely private.
@@ -59,14 +56,14 @@ Choose a synchronisation method, prepare its server where required, then follow
1. Prepare the server. A maintained MinIO server installation guide is not currently available here, so set up an S3-compatible service or server of your choice.
2. Configure the clients by following [Object Storage Setup](docs/setup_object_storage.md).
3. Peer-to-Peer
1. No server setup is required.
1. No central data-storage server is required. The project's public signalling relay requires no server provisioning; controlled deployments can provide another compatible relay.
2. Configure the clients by following [Peer-to-Peer Setup](docs/setup_p2p.md).
Each workflow establishes ordinary note synchronisation on the first device, generates the additional-device Setup URI from that working device, and verifies synchronisation in both directions.
Each workflow establishes ordinary note synchronisation on the first device, generates a Setup URI for each additional device from that working device, and verifies synchronisation in both directions.
> [!TIP]
> Fly.io is no longer free. Fortunately, we can still use IBM Cloudant despite some limitations. Refer to [Set up IBM Cloudant](docs/setup_cloudant.md).
> We can also use peer-to-peer synchronisation without a server. Alternatively, cheap object storage like Cloudflare R2 can be used for free.
> We can also use peer-to-peer synchronisation without a central data-storage server; a signalling relay is still used for peer discovery. Alternatively, cheap object storage like Cloudflare R2 can be used for free.
> However, most importantly, we can use a server that we trust. Therefore, please set up your own server.
> CouchDB can also be run on a Raspberry Pi (please be mindful of your server's security).
@@ -102,6 +99,7 @@ To prevent file and database corruption, please avoid closing Obsidian until all
## Tips and Troubleshooting
- If you want a faster and simpler initial replication when setting up subsequent devices, see the [Fast Setup Guide](docs/tips/fast-setup.md).
- Configure [Hidden File Sync](docs/tips/hidden-file-sync.md) only after ordinary note synchronisation works.
- If Obsidian or LiveSync cannot start normally, use [Recovery and flag files](docs/recovery.md) before changing or resetting a remote database.
- Self-hosted LiveSync 1.0 requires Obsidian 1.7.2 or later. If you need to use 1.0 on an earlier Obsidian version, please [open an issue](https://github.com/vrtmrz/obsidian-livesync/issues/new?template=issue-report.md) with your version, platform, and reason for remaining on it so that we can assess whether extending support is practical. The standard Community Plugins installer otherwise selects an older compatible plug-in release.
- If you are having problems getting the plug-in working, see [Tips and Troubleshooting](docs/troubleshooting.md).
+4 -3
View File
@@ -1,11 +1,12 @@
import { writeFileSync } from "fs";
import { allMessages } from "../src/common/messages/combinedMessages.dev.ts";
import path from "path";
const { writeFileSync } = process.getBuiltinModule("node:fs");
const path = process.getBuiltinModule("node:path");
const __dirname = import.meta.dirname;
const currentPath = __dirname;
const outDir = path.resolve(currentPath, "../src/common/messages/combinedMessages.prod.ts");
console.log(`Writing to ${outDir}`);
process.stdout.write(`Writing to ${outDir}\n`);
writeFileSync(
outDir,
`export const allMessages: Readonly<Record<string, Readonly<Record<string, string>>>> = ${JSON.stringify(allMessages, null, 4)};`
+13 -8
View File
@@ -1,14 +1,14 @@
import { readFile } from "fs/promises";
import { join, resolve } from "path";
import { glob } from "tinyglobby";
import { parse } from "yaml";
import { objectToDotted } from "./messagelib.ts";
const fsPromises = process.getBuiltinModule("node:fs/promises");
const path = process.getBuiltinModule("node:path");
const __dirname = import.meta.dirname;
const targetDir = resolve(join(__dirname, "../src/common/messagesYAML/"));
const targetDir = path.resolve(path.join(__dirname, "../src/common/messagesYAML/"));
const files = (await glob(`*.yaml`, { expandDirectories: false, absolute: true, cwd: targetDir })).sort();
function flattenMessages(src: Record<string, unknown>) {
function flattenMessages(src: unknown) {
return Object.fromEntries(
Object.entries(objectToDotted(src))
.map(([key, value]) => [key.endsWith("._value") ? key.slice(0, -7) : key, value] as const)
@@ -20,9 +20,14 @@ function flattenMessages(src: Record<string, unknown>) {
const localeData = new Map<string, Record<string, string>>();
for (const file of files) {
const segments = file.split(/[/\\]/);
const locale = segments[segments.length - 1]!.replace(/\.yaml$/, "");
const content = await readFile(file, "utf-8");
localeData.set(locale, flattenMessages(parse(content) ?? {}));
const localeFilename = segments[segments.length - 1];
if (localeFilename === undefined) {
throw new Error(`Could not determine the locale name for ${file}`);
}
const locale = localeFilename.replace(/\.yaml$/, "");
const content = await fsPromises.readFile(file, "utf-8");
const parsed: unknown = parse(content);
localeData.set(locale, flattenMessages(parsed ?? {}));
}
const baseLocale = "en";
@@ -55,4 +60,4 @@ const report = Object.fromEntries(
})
);
console.log(JSON.stringify(report, null, 2));
process.stdout.write(`${JSON.stringify(report, null, 2)}\n`);
+2 -3
View File
@@ -1,9 +1,8 @@
import { writeFileSync } from "fs";
import { SUPPORTED_I18N_LANGS, type I18N_LANGS } from "../src/common/rosetta";
import { allMessages } from "../src/common/messages/combinedMessages.dev.ts";
import path from "path";
const { writeFileSync } = process.getBuiltinModule("node:fs");
const path = process.getBuiltinModule("node:path");
const thisFileDir = __dirname;
const outDir = path.join(thisFileDir, "i18n");
+5 -14
View File
@@ -1,26 +1,17 @@
import { writeFileSync } from "fs";
import { allMessages } from "../src/common/messages/combinedMessages.prod.ts";
const __dirname = import.meta.dirname;
import path from "path";
const { writeFileSync } = process.getBuiltinModule("node:fs");
const path = process.getBuiltinModule("node:path");
const thisFileDir = __dirname;
const outDir = path.resolve(thisFileDir, "../src/common/messagesJson");
const out = {} as Record<string, { [key: string]: string | undefined }>;
for (const [key, value] of Object.entries(allMessages)) {
//@ts-ignore
for (const [lang, langValue] of Object.entries(allMessages[key])) {
for (const [lang, langValue] of Object.entries(value)) {
if (!out[lang]) out[lang] = {};
if (lang in value) {
out[lang][key] = langValue as string;
} else {
if (lang === "def") {
out[lang][key] = key;
} else {
out[lang][key] = undefined;
}
}
out[lang][key] = langValue;
}
}
+137
View File
@@ -0,0 +1,137 @@
const fsPromises = process.getBuiltinModule("node:fs/promises");
const path = process.getBuiltinModule("node:path");
const url = process.getBuiltinModule("node:url");
type InspectionError = {
check: "current-label" | "local-reference" | "retired-label";
file: string;
detail: string;
};
export type TroubleshootingDocsInspection = {
ok: boolean;
checkedFiles: string[];
checkedLocalReferences: number;
errors: InspectionError[];
};
const guidePaths = ["docs/troubleshooting.md", "docs/recovery.md", "docs/tips/p2p-sync-tips.md"] as const;
const messageCataloguePath = "src/common/messagesJson/en.json";
const markdownLinkPattern = /!?\[[^\]]*\]\(([^)\s]+)(?:\s+["'][^)]*["'])?\)/gu;
function repositoryRootFromThisFile(): string {
return path.resolve(path.dirname(url.fileURLToPath(import.meta.url)), "..");
}
function normaliseReferenceTarget(rawTarget: string): string {
const withoutAngles = rawTarget.startsWith("<") && rawTarget.endsWith(">") ? rawTarget.slice(1, -1) : rawTarget;
return decodeURIComponent(withoutAngles);
}
function isExternalReference(target: string): boolean {
return /^(?:https?:|mailto:|obsidian:)/u.test(target);
}
async function inspectLocalReferences(
repositoryRoot: string,
documentPath: string,
document: string,
errors: InspectionError[]
): Promise<number> {
let checked = 0;
for (const match of document.matchAll(markdownLinkPattern)) {
const rawTarget = match[1];
if (!rawTarget) continue;
const target = normaliseReferenceTarget(rawTarget);
if (isExternalReference(target) || target.startsWith("#")) continue;
const [pathPart] = target.split("#", 1);
if (!pathPart) continue;
checked++;
const referencedPath = path.resolve(repositoryRoot, path.dirname(documentPath), pathPart);
try {
await fsPromises.access(referencedPath);
} catch {
errors.push({
check: "local-reference",
file: documentPath,
detail: `Missing local reference: ${path.relative(repositoryRoot, referencedPath)}`,
});
}
}
return checked;
}
export async function inspectTroubleshootingDocs(
repositoryRoot = repositoryRootFromThisFile()
): Promise<TroubleshootingDocsInspection> {
const errors: InspectionError[] = [];
const documents = new Map<string, string>();
for (const guidePath of guidePaths) {
documents.set(guidePath, await fsPromises.readFile(path.resolve(repositoryRoot, guidePath), "utf8"));
}
const troubleshooting = documents.get("docs/troubleshooting.md")!;
const catalogue = JSON.parse(
await fsPromises.readFile(path.resolve(repositoryRoot, messageCataloguePath), "utf8")
) as Record<string, string>;
const requiredMessageKeys = [
"TweakMismatchResolve.Action.UseConfigured",
"TweakMismatchResolve.Action.UseMine",
"TweakMismatchResolve.Action.UseRemote",
"TweakMismatchResolve.Action.Dismiss",
"obsidianLiveSyncSettingTab.titleSyncSettingsViaMarkdown",
] as const;
for (const messageKey of requiredMessageKeys) {
const label = catalogue[messageKey];
if (!label) {
errors.push({
check: "current-label",
file: messageCataloguePath,
detail: `The English message catalogue does not define ${messageKey}.`,
});
continue;
}
if (!troubleshooting.includes(label)) {
errors.push({
check: "current-label",
file: "docs/troubleshooting.md",
detail: `The guide does not include the current UI label '${label}'.`,
});
}
}
for (const retiredLabel of ["`Update with mine`", "`Use configured`", "`Sync settings via Markdown files`"]) {
if (troubleshooting.includes(retiredLabel)) {
errors.push({
check: "retired-label",
file: "docs/troubleshooting.md",
detail: `The guide still includes the retired label ${retiredLabel}.`,
});
}
}
let checkedLocalReferences = 0;
for (const [guidePath, document] of documents) {
checkedLocalReferences += await inspectLocalReferences(repositoryRoot, guidePath, document, errors);
}
return {
ok: errors.length === 0,
checkedFiles: [...guidePaths],
checkedLocalReferences,
errors,
};
}
async function runCli(): Promise<void> {
const result = await inspectTroubleshootingDocs();
process.stdout.write(`${JSON.stringify(result, null, 2)}\n`);
if (!result.ok) process.exitCode = 1;
}
const invokedPath = process.argv[1] ? url.pathToFileURL(path.resolve(process.argv[1])).href : undefined;
if (invokedPath === import.meta.url) {
await runCli();
}
@@ -0,0 +1,17 @@
import { describe, expect, it } from "vitest";
import { inspectTroubleshootingDocs } from "./inspect-troubleshooting-docs";
describe("troubleshooting documentation contract", () => {
it("uses current English UI labels and resolves every local guide reference", async () => {
const result = await inspectTroubleshootingDocs();
expect(result.checkedFiles).toEqual([
"docs/troubleshooting.md",
"docs/recovery.md",
"docs/tips/p2p-sync-tips.md",
]);
expect(result.checkedLocalReferences).toBeGreaterThan(0);
expect(result.errors).toEqual([]);
expect(result.ok).toBe(true);
});
});
+14 -10
View File
@@ -1,27 +1,31 @@
// Convert Application convenient Message Resources (JSON) to Human-Editable format (YAML)
import { readFile, writeFile } from "fs/promises";
import { join, resolve } from "path";
import { stringify } from "yaml";
import { glob } from "tinyglobby";
import { dottedToObject } from "./messagelib";
const fsPromises = process.getBuiltinModule("node:fs/promises");
const path = process.getBuiltinModule("node:path");
const __dirname = import.meta.dirname;
const targetDir = resolve(join(__dirname, "../src/common/messagesJson/"));
console.log(`Target directory: ${targetDir}`);
const targetDir = path.resolve(path.join(__dirname, "../src/common/messagesJson/"));
process.stdout.write(`Target directory: ${targetDir}\n`);
const files = await glob(`*.json`, { expandDirectories: false, absolute: true, cwd: targetDir });
for (const file of files) {
const filePath = resolve(file);
console.log(`Processing file: ${filePath}`);
const content = await readFile(filePath, "utf-8");
const jsonDataSrc = JSON.parse(content);
const filePath = path.resolve(file);
process.stdout.write(`Processing file: ${filePath}\n`);
const content = await fsPromises.readFile(filePath, "utf-8");
const jsonDataSrc: unknown = JSON.parse(content);
if (typeof jsonDataSrc !== "object" || jsonDataSrc === null || Array.isArray(jsonDataSrc)) {
throw new TypeError(`Expected ${filePath} to contain a JSON object`);
}
const jsonDataD2 = Object.fromEntries(
Object.entries(jsonDataSrc).sort(([keyA], [keyB]) => keyA.localeCompare(keyB))
);
const jsonData = dottedToObject(jsonDataD2);
const yamlData = stringify(jsonData, { indent: 2 });
const yamlFilePath = filePath.replace(/\.json$/, ".yaml").replace("Json", "YAML");
await writeFile(yamlFilePath, yamlData, "utf-8");
console.log(`Converted ${filePath} to ${yamlFilePath}`);
await fsPromises.writeFile(yamlFilePath, yamlData, "utf-8");
process.stdout.write(`Converted ${filePath} to ${yamlFilePath}\n`);
}
// console.dir(files, { depth: 0 });
+47 -36
View File
@@ -1,38 +1,49 @@
export function objectToDotted(obj: any, prefix = ""): Record<string, any> {
return Object.entries(obj).reduce(
(acc, [key, value]) => {
const newKey = prefix ? `${prefix}.${key}` : key;
if (typeof value === "object" && value !== null && !Array.isArray(value)) {
Object.assign(acc, objectToDotted(value, newKey));
} else {
acc[newKey] = value;
}
return acc;
},
{} as Record<string, any>
);
function isRecord(value: unknown): value is Record<string, unknown> {
return typeof value === "object" && value !== null && !Array.isArray(value);
}
export function dottedToObject(obj: Record<string, any>): Record<string, any> {
return Object.entries(obj).reduce(
(acc, [key, value]) => {
if (key.includes(" ")) {
// Return as is.
return { ...acc, [key]: value }; // Skip keys with spaces
}
const keys = key.split(".");
keys.reduce((nestedAcc, currKey, index) => {
if (currKey in nestedAcc && typeof nestedAcc[currKey] !== "object") {
nestedAcc[currKey] = { _value: nestedAcc[currKey] }; // Convert to object if not already
}
if (index === keys.length - 1) {
nestedAcc[currKey] = value;
} else {
nestedAcc[currKey] = nestedAcc[currKey] || {};
}
return nestedAcc[currKey];
}, acc);
return acc;
},
{} as Record<string, any>
);
export function objectToDotted(obj: unknown, prefix = ""): Record<string, unknown> {
if (!isRecord(obj)) {
throw new TypeError("Expected a message catalogue object");
}
const flattened: Record<string, unknown> = {};
for (const [key, value] of Object.entries(obj)) {
const newKey = prefix ? `${prefix}.${key}` : key;
if (isRecord(value)) {
Object.assign(flattened, objectToDotted(value, newKey));
} else {
flattened[newKey] = value;
}
}
return flattened;
}
export function dottedToObject(obj: unknown): Record<string, unknown> {
if (!isRecord(obj)) {
throw new TypeError("Expected a dotted message catalogue object");
}
const nestedResult: Record<string, unknown> = {};
for (const [key, value] of Object.entries(obj)) {
if (key.includes(" ")) {
nestedResult[key] = value;
continue;
}
const keys = key.split(".");
let nested = nestedResult;
for (const [index, currentKey] of keys.entries()) {
if (index === keys.length - 1) {
nested[currentKey] = value;
continue;
}
const currentValue = nested[currentKey];
if (isRecord(currentValue)) {
nested = currentValue;
continue;
}
const replacement = currentValue === undefined || currentValue === null ? {} : { _value: currentValue };
nested[currentKey] = replacement;
nested = replacement;
}
}
return nestedResult;
}
+41
View File
@@ -0,0 +1,41 @@
import { describe, expect, it } from "vitest";
import { dottedToObject, objectToDotted } from "./messagelib";
describe("message catalogue conversion", () => {
it("flattens nested objects while preserving leaf values", () => {
expect(
objectToDotted({
dialogue: {
title: "Title",
options: ["first", "second"],
},
"literal key": "Literal",
})
).toEqual({
"dialogue.title": "Title",
"dialogue.options": ["first", "second"],
"literal key": "Literal",
});
});
it("preserves an existing leaf under _value when a dotted child follows it", () => {
expect(
dottedToObject({
section: "Base value",
"section.child": "Child value",
"literal key": "Literal",
})
).toEqual({
section: {
_value: "Base value",
child: "Child value",
},
"literal key": "Literal",
});
});
it("rejects non-object catalogue roots", () => {
expect(() => objectToDotted("not an object")).toThrow("Expected a message catalogue object");
expect(() => dottedToObject(["not", "an", "object"])).toThrow("Expected a dotted message catalogue object");
});
});
+11 -10
View File
@@ -1,29 +1,30 @@
// Convert Human-Editable format (YAML) to Application convenient Message Resources (JSON)
import { readFile, writeFile } from "fs/promises";
import { join, resolve } from "path";
import { parse } from "yaml";
import { glob } from "tinyglobby";
import { objectToDotted } from "./messagelib";
const fsPromises = process.getBuiltinModule("node:fs/promises");
const path = process.getBuiltinModule("node:path");
const __dirname = import.meta.dirname;
const targetDir = resolve(join(__dirname, "../src/common/messagesYAML/"));
console.log(`Target directory: ${targetDir}`);
const targetDir = path.resolve(path.join(__dirname, "../src/common/messagesYAML/"));
process.stdout.write(`Target directory: ${targetDir}\n`);
const files = await glob(`*.yaml`, { expandDirectories: false, absolute: true, cwd: targetDir });
for (const file of files) {
const filePath = resolve(file);
const content = await readFile(filePath, "utf-8");
const jsonDataSrc = parse(content);
const filePath = path.resolve(file);
const content = await fsPromises.readFile(filePath, "utf-8");
const jsonDataSrc: unknown = parse(content);
const jsonDataD2 = objectToDotted(jsonDataSrc);
const jsonData = Object.fromEntries(
Object.entries(jsonDataD2)
.map(([key, value]) => [key.endsWith("._value") ? key.slice(0, -7) : key, value])
.map(([key, value]): [string, unknown] => [key.endsWith("._value") ? key.slice(0, -7) : key, value])
.sort(([keyA], [keyB]) => keyA.localeCompare(keyB))
);
const yamlData = JSON.stringify(jsonData, null, 4) + "\n";
const yamlFilePath = filePath.replace(/\.yaml$/, ".json").replace("YAML", "Json");
await writeFile(yamlFilePath, yamlData, "utf-8");
console.log(`Converted ${filePath} to ${yamlFilePath}`);
await fsPromises.writeFile(yamlFilePath, yamlData, "utf-8");
process.stdout.write(`Converted ${filePath} to ${yamlFilePath}\n`);
}
// console.dir(files, { depth: 0 });
+30 -19
View File
@@ -38,6 +38,8 @@ npm run buildDev # Development build (one-time)
npm run test:integration # Run CouchDB-backed integration tests
npm run test:setup-tools # Check provisioning and Setup URI package contracts
npm run test:e2e:cli:p2p # Run canonical P2P validation in Compose
npm run test:browser-apps # Build and run the WebApp and WebPeer app-owned Chromium tests
npm run test:e2e:browser-apps:interop # Run WebApp → WebPeer → CLI in Compose
npm run test:e2e:obsidian:local-suite # Run the real Obsidian local suite
```
@@ -60,13 +62,14 @@ To facilitate development and testing, the build process can automatically copy
### Testing Infrastructure
- ~~**Deno Tests**: Unit tests for platform-independent code (e.g., `HashManager.test.ts`)~~
- This is now obsolete, migrated to vitest.
- **Vitest**:
- **Unit Tests** (`vitest.config.unit.ts`): Unit tests run in Node.js (excluding harnesses and integration tests). Unit tests should be `*.unit.spec.ts` and placed alongside the implementation file (e.g., `ChunkFetcher.unit.spec.ts`). Executed via `npm run test:unit`.
- **Integration Tests** (`vitest.config.integration.ts`): Tests run in Node.js against a real CouchDB instance. Integration tests should be `*.integration.spec.ts` or `*.integration.test.ts` and placed alongside the implementation file (e.g., `StreamingFetch.integration.spec.ts`). Executed via `npm run test:integration`.
- If you add a feature that interacts with the remote database (e.g., replication changes, custom changes feed parameters, or custom HTTP queries), you strongly expected to write an integration test to verify the behaviour against a real CouchDB server.
- If you add a feature that interacts with the remote database (e.g., replication changes, custom changes feed parameters, or custom HTTP queries), you are strongly expected to write an integration test to verify the behaviour against a real CouchDB server.
- **Commonlib Tests**: Commonlib owns unit and package tests for shared RPC, storage, replication, and platform contracts. LiveSync CI verifies the exact packed dependency as a downstream consumer.
Regression tests remain in the suite owned by the implementation under test. Plug-in tests may be co-located with their source, while independent application tests remain under `test/apps/` or `test/browser-apps/` so that they stay outside the Community Review source boundary. Prefix a case or group with `compatibility:` when it protects a persisted input or state which current releases still accept, and with `retirement guard:` when it prevents a removed setting, control, or notification from returning. Remove or replace a compatibility case only when the corresponding input is no longer accepted or an equivalent maintained case preserves the contract. Remove a retirement guard only when another current contract makes the old behaviour unreachable. Do not preserve a disconnected historical test as an executable specification when no maintained runner invokes it; Git history is the reference for retired test infrastructure.
- **CLI E2E** (`src/apps/cli/testdeno/`): Host-independent consumer workflows. The canonical Compose P2P suite covers ordinary two-peer synchronisation, replacement of the current replicator followed by transfer with the same peer, and explicit relay disconnection followed by paused and resumed reconnection. Its lifecycle entry point is included only in the Docker test build and does not add a public CLI command. Run `npm run test:e2e:cli` for the ordinary suite or `npm run test:e2e:cli:p2p` for P2P validation.
- **Self-hosted setup tools** (`utils/couchdb/`, `utils/setup/`, and `utils/flyio/`): Deno contract tests consume the exact locked Commonlib registry package, verify current CouchDB, Object Storage, and random-room P2P Setup URI defaults and remote profiles, and keep CouchDB administration separate from package-owned LiveSync database-version negotiation. `unit-ci` also provisions a real temporary CouchDB database and verifies its version document against the installed Commonlib package. Run `npm run test:setup-tools` for the local contract gate.
- **Real Obsidian E2E** (`test/e2e-obsidian/`): Local-first scripts that launch real Obsidian with temporary vaults and the built Self-hosted LiveSync plug-in. Use these for boot-up sequence, vault reflection, RedFlag flows, Fast Setup (Simple Fetch), settings dialogues, restart-sensitive workflows, Object Storage regressions, and other behaviour that depends on Obsidian itself. Run focused scripts such as `npm run test:e2e:obsidian:two-vault-sync`, or use `npm run test:e2e:obsidian:local-suite:services` to run the broader local suite with CouchDB and MinIO fixtures managed by the wrapper.
@@ -84,9 +87,11 @@ To facilitate development and testing, the build process can automatically copy
- **Test Structure**:
- `test/e2e-obsidian/` - Real Obsidian E2E scripts for local verification
- `test/apps/webapp/` and `test/apps/webpeer/` - app-owned unit tests outside the Community Review source boundary
- `test/browser-apps/webapp/` and `test/browser-apps/webpeer/` - app-owned Deno and Chromium tests outside the Community Review source boundary
- `test/browser-apps/` - Compose-owned WebApp → WebPeer → CLI P2P interoperability and shared browser-test support
- co-located `*.unit.spec.ts` files - Node-based unit tests
- co-located `*.integration.spec.ts` files - service-backed integration tests
- `src/apps/webapp/obsidianMock.ts` - Webapp-only Obsidian compatibility adapter; it is not an E2E Harness
### Import Path Normalisation
@@ -143,7 +148,7 @@ Hence, the new feature should be implemented as follows:
- **LiveSyncLocalDB** (`@vrtmrz/livesync-commonlib/compat/pouchdb/LiveSyncLocalDB`): Local PouchDB database wrapper
- **Replicators** (`@vrtmrz/livesync-commonlib/compat/replication/*`): CouchDB, Journal, and P2P synchronisation engines
- **Service Hub** (`src/modules/services/`): Central service registry using dependency injection
- **Common Library** (`@vrtmrz/livesync-commonlib`): Platform-independent synchronisation logic, shared with the CLI, Webapp, WebPeer, and external tools
- **Common Library** (`@vrtmrz/livesync-commonlib`): Platform-independent synchronisation logic, shared with the CLI, WebApp, WebPeer, and external tools
Commonlib owns the P2P replicator and Trystero transport lifecycle. Host commands, event handlers, and views must retain the Commonlib service-feature result and resolve its current `replicator` at the point of use. They must not snapshot an instance which can be replaced when settings or the local database change, close Trystero-owned raw peers, or install another Trystero transport generation at the application root.
@@ -243,7 +248,10 @@ export class ModuleExample extends AbstractObsidianModule {
- Use SemVer beta identifiers such as `1.0.0-beta.0` for immutable integration previews. Increment the beta number when a published preview needs a correction. Reserve `1.0.0-rc.0` for the first feature- and contract-frozen release candidate. Historical `-patchedN` releases remain unchanged in the release history.
- Publish a pre-release from an immutable reviewed tag, mark its GitHub Release as a pre-release, and do not replace the latest stable release.
- A plug-in review release may omit the CLI image when the CLI artefact is not part of the required validation. When a pre-release CLI image is published, it receives immutable version and SHA-qualified tags only; it must not advance `latest` or a stable major-minor tag.
- Keep the release pull request in draft until the exact published plug-in has passed BRAT validation. If validation fails, prepare the next pre-release version rather than moving the existing tag.
- Keep a hyphenated pre-release's release pull request in draft and unmerged after BRAT validation. Reconcile the published version's metadata into its base branch through a reviewed metadata-only commit, then close the release pull request only through a separate maintainer action.
- Stage a stable version for BRAT by publishing its exact `x.y.z` tag initially as a GitHub pre-release with `prerelease=true` and `publish_cli=false`. The stable manifest version would otherwise make the CLI workflow advance `latest` and the major-minor image tag before validation.
- After a staged stable version passes BRAT validation, remove the GitHub pre-release designation and make the exact release the latest stable release first. Confirm that promotion, then merge its exact release commit into the reviewed base branch and integrate it through the reviewed branch chain into the repository's default branch. This order prevents Community Directory review from inspecting a stable manifest version while the matching GitHub Release remains a pre-release. Publish stable CLI tags through a separate maintainer gate.
- If validation fails, leave every published tag unchanged and prepare the next pre-release or patch version.
## Release Notes
@@ -261,15 +269,17 @@ The `Finalise Release Tags` and `Release Obsidian Plugin` workflows use the `rel
- Run the `Prepare Release PR` workflow with the target version and selected base branch. It creates the release branch, updates versions, confirms that Commonlib is locked to an immutable package version, moves the `## Unreleased` notes to the target version, commits the release preparation, pushes the branch, and opens a draft release PR. The base branch may already select the target development version; the workflow still runs the version lifecycle so that release-only metadata such as `versions.json` is recorded in the release commit.
- Do not tag the release branch when the PR is first created. Polish the release PR first, especially `updates.md`.
- Once the release PR head is fixed, run the `Finalise Release Tags` workflow with its full head commit SHA. It validates the release branch, ensures that the plug-in tag points to that commit, optionally creates the corresponding CLI tag, and dispatches the plug-in release workflow. A CLI tag starts its own container workflow. The finalisation workflow can be retried when existing tags already point to the reviewed commit, but stops if a selected tag points elsewhere.
- The plug-in publishing workflow is intentionally dispatch-only. Pushing a plug-in tag directly does not publish a GitHub Release; use `Finalise Release Tags`, or dispatch `Release Obsidian Plugin` explicitly for recovery or a pre-release. The CLI Docker workflow retains its documented branch, tag, and manual triggers.
- Approve the `Release Obsidian Plugin` workflow for the `release` environment, then inspect the generated draft GitHub Release. For a selected CLI publication, confirm the image tags appropriate to a stable or pre-release version.
- Publish a stable draft as the latest release, or publish a pre-release draft without replacing the latest stable release. In either case, keep the release PR in draft and leave its base branch unchanged until BRAT validation succeeds. Record that state in the PR.
- Once the release PR head is fixed, run the `Finalise Release Tags` workflow with its full head commit SHA. It validates the release branch, ensures that the plug-in tag points to that commit, optionally creates the corresponding CLI tag, and explicitly dispatches the selected plug-in and CLI release workflows. The finalisation workflow can be retried when existing tags already point to the reviewed commit, but stops if a selected tag points elsewhere.
- The plug-in publishing workflow is intentionally dispatch-only. Pushing a plug-in tag directly does not publish a GitHub Release; use `Finalise Release Tags`, or dispatch `Release Obsidian Plugin` explicitly for recovery or a pre-release. When CLI publication is selected, finalisation dispatches the CLI Docker workflow against the reviewed CLI tag instead of relying on a tag created by `GITHUB_TOKEN` to start another workflow.
- For a hyphenated pre-release, run finalisation with `prerelease=true`; CLI publication remains optional. For a stable version awaiting BRAT validation, use `prerelease=true` and `publish_cli=false`.
- Approve the `Release Obsidian Plugin` workflow for the `release` environment, then inspect the generated draft GitHub Release. When a hyphenated pre-release includes the CLI, confirm that it received only its immutable version and SHA-qualified image tags.
- Publish the draft as a GitHub pre-release without replacing the latest stable release. Keep its release pull request in draft and leave its base branch unchanged throughout BRAT validation. Record that state in the pull request.
- Validate the published release through BRAT. Confirm start-up, ordinary bidirectional synchronisation, and any regression scenario relevant to the release.
- After BRAT validation succeeds, mark the release PR ready and merge it into the selected base branch with a merge commit. This keeps the tagged release commit in that branch's history.
- After a hyphenated pre-release passes, keep its release pull request unmerged. Add a reviewed metadata-only commit to the selected base branch which records the published version in `versions.json` and moves its exact tagged release notes out of `## Unreleased`, then close the release pull request only through a separate maintainer action.
- After a stable version passes, remove the GitHub pre-release designation and make that exact release the latest stable release. Confirm that the release is no longer a pre-release, then mark its release pull request ready, merge the exact release commit into the selected base branch with a merge commit, and integrate that exact commit through the reviewed branch chain into the repository's default branch. Confirm that the default branch contains the matching stable metadata. Create the stable CLI tag and publish its `latest` and major-minor image tags, if selected, through a separate maintainer gate.
- If BRAT validation fails, keep the release PR in draft and do not move published tags. Before preparing the next version, add a reviewed metadata-only commit to the selected base branch which records the published version in `versions.json` and moves its exact tagged release notes out of `## Unreleased`. Keep only changes made after that tag under `## Unreleased`. Compare the historical section with `git show <tag>:updates.md`; do not merge the failed release PR or describe it as validated. The next release PR can then rotate only the correction notes while preserving the immutable release history.
- Prepare and publish the next patch or pre-release version from that reconciled base. Leave the failed release PR draft until it is deliberately closed as superseded under a separate maintainer action.
- For a pre-release, set `prerelease=true` in `Finalise Release Tags`. A hyphenated version is rejected unless that input is enabled.
- A hyphenated version is rejected unless `prerelease=true`. A stable version staged with `prerelease=true` is rejected unless `publish_cli=false`.
### Release Cheat Sheet
@@ -289,15 +299,16 @@ The `Finalise Release Tags` and `Release Obsidian Plugin` workflows use the `rel
- `version`: the same target version.
- `release_branch`: leave blank unless the release branch used a custom name.
- `expected_head_sha`: the full head commit SHA reviewed in the release PR.
- `prerelease`: enable for a version such as `1.0.0-rc.0`.
- `publish_cli`: disable when the reviewed release is plug-in-only.
- `prerelease`: enable for a version such as `1.0.0-rc.0`, and also when staging a stable version for BRAT.
- `publish_cli`: optional for a hyphenated pre-release, but disable it when staging a stable version.
5. Approve the `Release Obsidian Plugin` workflow for the `release` environment, then check the generated draft GitHub Release.
6. If CLI publication was selected, confirm that the CLI tag event published the expected image tags.
7. Publish the draft as a stable release or pre-release as selected, but keep the release PR in draft and leave its base branch unchanged.
8. Update the PR state message to describe the published release and state that merging remains on hold until BRAT validation is complete.
6. If a hyphenated pre-release includes the CLI, confirm that the explicitly dispatched CLI workflow published only immutable version and SHA-qualified image tags.
7. Publish the draft as a GitHub pre-release without replacing the latest stable release, but keep the release PR in draft and leave its base branch unchanged.
8. Update the PR state message to describe the published pre-release and state that merging remains on hold.
9. Validate the published release through BRAT, including start-up, ordinary bidirectional synchronisation, and any release-specific regression scenario.
10. After BRAT validation succeeds, mark the release PR ready and merge it into the selected base branch with a merge commit.
11. If validation fails, leave the PR in draft and do not move the published tags. Reconcile the published version's `updates.md` section and `versions.json` entry into the base branch as metadata only, then prepare the next patch or pre-release version from the remaining `## Unreleased` entries.
10. After a hyphenated pre-release passes, keep its release PR unmerged, reconcile its `versions.json` entry and exact release-note section into the selected base branch as metadata only, then close the PR through a separate maintainer action.
11. After a stable version passes, remove the pre-release designation and make the exact release the latest stable release. Confirm that promotion, then mark the release PR ready, merge the exact release commit into the selected base branch, and integrate it through the reviewed branch chain into the repository's default branch. Confirm that the default branch contains the matching stable metadata, then publish stable CLI tags through a separate maintainer gate if selected.
12. If validation fails, leave the PR in draft and do not move the published tags. Reconcile the published version's `updates.md` section and `versions.json` entry into the base branch as metadata only, then prepare the next patch or pre-release version from the remaining `## Unreleased` entries.
## Contribution Guidelines
+1 -1
View File
@@ -200,7 +200,7 @@ Current implementation status:
Current implementation status:
- The mocked Vitest browser suites, their P2P runner, their root-level relay helpers, and the manual `harness-ci` workflow have been removed after maintained suites covered the critical flows.
- Headless CouchDB and Object Storage combinations, with and without encryption, remain owned by the CLI two-Vault matrix. P2P transport replacement and relay lifecycle remain owned by the CLI Compose E2E suite. Real Obsidian owns the visible CouchDB, Object Storage, and P2P Setup URI workflows, including first-device URI generation, second-device import, and two-way Vault synchronisation.
- Headless CouchDB and Object Storage combinations, with and without encryption, remain owned by the CLI two-Vault matrix. P2P transport replacement and relay lifecycle remain owned by the CLI Compose E2E suite. Real Obsidian owns the visible CouchDB, Object Storage, and P2P Setup URI workflows, including URI generation on the first device, import on the second device, and two-way Vault synchronisation.
- The Obsidian compatibility implementation still needed by the Webapp has moved to `src/apps/webapp/obsidianMock.ts`; it is not a retained browser E2E Harness.
- Remaining high-value scenarios, including RedFlag and Fast Setup (Simple Fetch) variants, should be added according to their owning integration boundary rather than copied line by line from the retired suite.
@@ -0,0 +1,90 @@
# Architectural Decision Record: Use Native Batch CAS for Adaptive PostgREST
## Status
Proposed as the final experimental provider, after the S3 and WebDAV delivery sequences have established the common
protocol, CLI, host, and end-to-end boundaries.
## Context
PostgREST can expose PostgreSQL uniqueness, row-level security, bounded set operations, and transactions. Treating it
as another opaque object store works, but leaves Metadata and Chunks combined and cannot use a multi-key Chunk query.
Using one HTTP request per logical Chunk would make latency dominate synchronisation.
The common protocol decision is recorded in
[Adaptive Journal as an explicit protocol](2026_07_adaptive_journal_protocol.md). The tables, binary RPC framing,
transaction rules, and privacy model are specified in the
[Adaptive Journal Sync design](../design_docs/adaptive_journal_sync.md).
## Decision
Adaptive PostgREST separates Metadata publication from immutable Chunk storage physically as well as logically.
- Vault- and repository-scoped Chunk rows use the Remote Chunk key as an insert-only unique address.
- Bounded binary RPCs implement `hasMany`, `getMany`, and `putMany` while preserving input order and per-entry status.
- Writer descriptors, Metadata batches, and commits remain small append-only records.
- A transactional commit verifies the sorted required-Chunk-key digest and the existence of every referenced Chunk
before making Metadata visible.
- Concurrent insertion of a different encrypted frame for the same logical Chunk reads, validates, and accepts the
winning plaintext only when it represents the same logical value.
- Row-level security scopes every operation to the configured Vault. Server-visible Remote Chunk keys reveal equality
within that repository but do not reveal plaintext Chunk IDs.
PostgREST does not use object packs, catalogue deltas, or Range retrieval for native Chunk rows. The shared binary Chunk
record remains independently verifiable, but PostgreSQL supplies the batch index and uniqueness boundary.
The SQL schema and RPC contract are versioned together with the Adaptive format. A missing or incompatible schema is
detected before publication and requires applying the reviewed schema or rebuilding the remote. The client does not
perform an implicit data-format migration.
## Staged acceptance
### Adapter and Commonlib integration
Unit tests own binary-envelope limits, ordering, status decoding, insert conflicts, rollback, error classification,
Vault isolation, and malformed responses. Disposable PostgreSQL and PostgREST integration tests own the real SQL
schema, row-level security, RPC transactions, and two-client Metadata and Chunk synchronisation.
### CLI end-to-end acceptance
The built CLI applies an Adaptive PostgREST Setup URI to a second independent database and synchronises text and binary
Chunk-backed files through disposable PostgreSQL and PostgREST services. The scenario proves bounded native batching at
the headless product boundary without repeating the SQL failure matrix.
### Host settings and UI
The PostgREST dialogue persists the endpoint, Vault identifier, authentication configuration, Adaptive format, and
expected repository ID. Focused tests own connection-string and Setup URI preservation, validation, and format-mismatch
guidance; they do not execute SQL.
### Real-host end-to-end acceptance
One real-Obsidian workflow applies the reviewed schema and performs a representative Chunk-backed transfer. The
disposable Commonlib integration remains authoritative for RLS, transaction rollback, and the complete RPC matrix.
## Alternatives rejected
### Reuse immutable object packs in PostgreSQL
This would preserve portability at the cost of native multi-key lookup and transaction guarantees. The repository
contract already permits a different physical representation.
### Store raw file content with Metadata
Metadata must remain small and must continue to refer to immutable logical Chunks. Combining content with Metadata
would break the maintained PouchDB model and duplicate unchanged content across revisions.
### Encode Chunk bodies as JSON values
Base64 and large JSON arrays add framing and memory overhead and make limits harder to enforce. Bounded binary `bytea`
RPC envelopes provide deterministic lengths and status ordering.
## Consequences
- PostgREST has the largest provider-specific implementation because it includes reviewed SQL, RLS, RPC framing, and
transaction behaviour.
- Placing it last lets the common protocol, CLI, and host boundaries stabilise before introducing that larger surface.
- Native batching can reduce request count substantially relative to object-per-Chunk storage, but actual speed still
depends on database, proxy, network, and workload measurements.
- PostgREST and object stores share logical records and correctness rules without pretending to share a physical
layout.
@@ -0,0 +1,98 @@
# Architectural Decision Record: Introduce Adaptive Journal as an Explicit Protocol
## Status
Proposed for staged implementation. This decision does not change the current default Journal format or promise a
compatible in-place migration.
## Context
The existing Journal protocol publishes PouchDB Metadata and Chunk documents together in opaque immutable objects.
That representation is portable and remains the compatibility baseline, but it prevents a remote from answering a
bounded multi-key Chunk query or amortising object-store requests independently from Metadata publication.
S3-compatible Object Storage, WebDAV, and PostgREST have materially different physical capabilities. Making each one
implement a separate replication algorithm would duplicate ordering, encryption, recovery, and Chunk-delivery rules.
Making every one use an identical physical layout would discard useful native batching and transactions.
The detailed binary formats, state machines, privacy properties, and recovery rules are specified in the
[Adaptive Journal Sync design](../design_docs/adaptive_journal_sync.md).
## Decision
Adaptive Journal is a second, explicit Journal protocol version owned by Commonlib. It keeps one logical repository
contract while allowing provider-specific physical storage.
- Metadata events and raw Chunk records remain separate throughout publication.
- Logical Chunks are immutable and content-addressable. A changed value has a new logical Chunk ID.
- A repository-scoped Remote Chunk key is derived locally from the exact logical Chunk ID after accepting the
repository manifest.
- Chunks become durable before a commit makes Metadata references visible.
- Ordinary publication creates immutable records. Catalogue snapshots, caches, and indexes are derived and
reconstructible.
- Each writer publishes a dense sequence identified by a stable host ID and a persisted random writer epoch. A reader
tracks one frontier per writer stream; it does not depend on a remote `startAfter` ordering contract.
- Remote operations return typed outcomes which distinguish absence, conflict, permanent rejection, retryable failure,
and an ambiguous mutation which must be verified before retrying.
The adaptive repository exposes batched Chunk availability, publication, and retrieval even when its implementation
uses immutable packs internally. The caller does not issue one remote request per Chunk.
### Repository identity and compatibility
A new repository has one immutable, conditionally created manifest. The first device generates its candidate locally,
and the winning manifest fixes the repository ID, Security Seed, protocol parameters, and required capabilities.
Every client pins the accepted repository ID in local repository state. A Setup URI exported from an accepted binding
should carry that non-secret expected ID.
`opaque-v1` and `adaptive-v1` are separate remote formats. Format selection is explicit in the remote configuration,
and a mismatch fails before publication. The implementation detects incompatible remote data, but it does not migrate
it. Changing format requires an explicit remote rebuild or a new remote namespace.
Deprecated data formats do not enlarge the Adaptive protocol. Compatibility remains at the existing decoding
boundaries, and rebuilding the remote is the recovery path for an unsupported Adaptive layout.
### Provider delivery sequence
Each provider is delivered and reviewed through four boundaries, in order:
1. **Adapter and Commonlib integration.** Implement semantic capabilities, typed failure handling, format detection,
unit tests, and disposable real-service integration tests.
2. **CLI end-to-end acceptance.** Use the built CLI and a real disposable service to apply a Setup URI, synchronise two
independent local databases, and restore text and binary Chunk content. This proves the headless product boundary
without Obsidian or Svelte.
3. **Host settings and UI.** Add only provider-specific controls, validation, profile persistence, Setup URI transport,
and focused host tests.
4. **Real-host end-to-end acceptance.** Exercise one representative setup and synchronisation path in a real Obsidian
instance. This test verifies host composition and does not repeat the adapter capability matrix or the complete CLI
conflict suite.
The initial order is S3-compatible Object Storage, WebDAV, then PostgREST. A provider completes these boundaries before
the next provider is presented for integration review. The sequence keeps each review independently attributable and
keeps the real-host tests small.
## Alternatives rejected
### One cross-provider implementation change
This hides which provider requires a shared-core change, makes failures difficult to attribute, and forces reviewers to
understand SQL RPCs, object packs, WebDAV behaviour, CLI composition, and Obsidian UI in one change.
### Add the Host UI before headless acceptance
This makes an application-level failure ambiguous between the protocol, adapter, CLI-independent host composition, and
presentation. The CLI provides the smaller executable boundary first.
### Use one physical representation for every provider
One-object-per-Chunk storage causes excessive object requests, while forcing packs into PostgreSQL discards bounded
multi-key RPCs and transactions. Common semantics do not require common physical storage.
## Consequences
- Commonlib owns protocol correctness and provider adapters; Self-hosted LiveSync owns CLI composition, Obsidian host
integration, and presentation.
- Provider reviews can stop at the first failed boundary without involving later UI or real-host tests.
- The final end-to-end suite remains an acceptance layer rather than a duplicate protocol test suite.
- Adding another provider requires the same semantic contract and staged evidence, not another replication algorithm.
- Remote rebuild remains an explicit operational requirement while Adaptive Journal is evolving.
+96
View File
@@ -0,0 +1,96 @@
# Architectural Decision Record: Use S3 as the Reference Adaptive Object Store
## Status
Proposed as an improvement to the maintained S3-compatible Journal path. Adaptive Journal remains explicit and
opt-in; the existing opaque format remains the compatibility default.
## Context
S3-compatible Object Storage already supplies the maintained Journal object model and is the smallest provider on
which to prove Adaptive immutable packs. It has a standard conditional-create request, paginated listing, binary
objects, and optional byte-range retrieval, but compatible endpoints can still differ in consistency, proxy behaviour,
and Range handling.
The common protocol decision is recorded in
[Adaptive Journal as an explicit protocol](2026_07_adaptive_journal_protocol.md). The complete object-pack and catalogue
formats are specified in the [Adaptive Journal Sync design](../design_docs/adaptive_journal_sync.md).
## Decision
S3-compatible storage is the reference implementation of the Adaptive object-store strategy.
- It stores the manifest, writer control records, Metadata batches, commits, Chunk packs, indexes, and catalogue records
as immutable objects under the configured bucket namespace.
- Manifest and every immutable publication use conditional create. A successful create response is sufficient; the
client does not add a confirmation request solely for caution.
- A lost or ambiguous mutation response returns `verify-first`. The caller reads the exact key before retrying.
- A conditional-request conflict which does not establish an existing immutable value remains retryable and is not
reported as a successful create.
- Listing follows every continuation token and proves complete prefix enumeration. Folder marker objects and stale
capability-probe objects are not repository data.
- Format inspection distinguishes empty, `opaque-v1`, `adaptive-v1`, and mixed repositories. Mixed or mismatched data
fails closed, and reset remains an explicit batched operation.
The portable retrieval policy is `whole-pack`. A user may select `range` when the endpoint capability probe has
confirmed exact byte-range behaviour. Range responses must use `206`, carry a matching `Content-Range`, and return the
requested number of bytes. Loss of optional Range support falls back to whole-pack retrieval after reporting the
capability change; it does not make valid packs unreadable.
Capability results which prove support or lack of support may be cached for the endpoint identity. A transient probe
failure is not cached as a permanent result. Probe objects use a reserved random prefix and are removed when possible;
incomplete cleanup is reported and ignored by format detection.
## Staged acceptance
### Adapter and Commonlib integration
Focused tests cover conditional creation, ambiguous responses, binary fidelity, read-after-write visibility, paginated
listing, deletion visibility, Range validation, format detection, reset batching, and probe cleanup. A disposable
MinIO integration proves two-client Adaptive Metadata and Chunk synchronisation and both retrieval paths.
### CLI end-to-end acceptance
One real-MinIO scenario uses two independent CLI databases. Device A uses whole-pack retrieval. Device B receives an
Adaptive S3 Setup URI selecting Range retrieval. The scenario verifies that the URI preserves the format and policy,
that both devices can publish and receive more than one synchronisation round, and that text and binary content are
reconstructed from Chunks.
This test owns the built CLI, settings persistence, Setup URI decoding, and headless composition. It does not repeat
every S3 error classification already owned by Commonlib.
### Host settings and UI
The Object Storage dialogue exposes `opaque-v1` and `adaptive-v1`, with whole-pack as the Adaptive default and Range as
an explicit preference. It preserves the expected repository ID and read policy through saved profiles and Setup URI
handling. Focused host tests own normalisation, validation, persistence, and warnings; they do not contact S3.
### Real-host end-to-end acceptance
One real-Obsidian workflow applies an Adaptive S3 configuration and proves a representative Chunk-backed file transfer
against disposable MinIO. It verifies the Obsidian composition boundary only. The CLI suite remains the broader
headless synchronisation acceptance test.
## Alternatives rejected
### Re-upload a mutable pack when one Chunk changes
Packs are immutable publication units. A changed logical Chunk is added to a new pack, and a new catalogue delta makes
that location discoverable. Existing packs are replaced only by a separately protected compaction process.
### Require Range for Adaptive S3
Whole-pack reads often provide better throughput and work on more endpoints. Range is a deployment preference, not a
correctness requirement.
### Confirm every successful mutation with another request
This doubles request count in the ordinary success path. Exact-key verification is reserved for ambiguous outcomes and
explicit diagnostics.
## Consequences
- S3 establishes the object-store contract before the more variable WebDAV implementation.
- The two retrieval policies are exercised without duplicating the complete CLI scenario.
- Existing opaque S3 repositories remain readable and never become Adaptive implicitly.
- Endpoint capability evidence, rather than the S3-compatible label alone, controls optional behaviour.
+240
View File
@@ -0,0 +1,240 @@
# Architectural Decision Record: Gate Adaptive WebDAV with an Endpoint Safety Check
## Status
Proposed as an experimental provider after the S3 Adaptive path has completed adapter, CLI, host, and real-host
acceptance.
## Context
WebDAV exposes an object-shaped interface suitable for immutable Commit Bundles and Packs, but method support and
conditional semantics vary across servers, gateways, reverse proxies, and authentication layers. A server can
advertise WebDAV while failing binary fidelity, replacing an object despite `If-None-Match: *`, returning incomplete
listings, or ignoring Range.
The common protocol decision is recorded in
[Adaptive Journal as an explicit protocol](2026_07_adaptive_journal_protocol.md). The Pack background, flat object
mapping, and detailed safety-check sequence are specified in the
[Adaptive Journal Sync design](../design_docs/adaptive_journal_sync.md). Commonlib owns the implemented
`commit-bundle-v1` wire contract.
## Decision
Adaptive WebDAV uses the same immutable Commit Bundle and external Pack semantics proven by S3, with a WebDAV-specific
flat object mapping inside the configured collection. Its Catalogue is derived locally from authenticated routes in
received Bundles rather than stored as a separate mutable remote object. Logical writer ordering comes from the host
ID, writer epoch, and dense sequence embedded in authenticated records. Correctness does not depend on a
server-provided `startAfter` order.
Before a new Adaptive repository becomes writable, a non-destructive endpoint safety checker exercises random reserved
probe keys and reports observed semantic capabilities:
- binary write and exact read-back;
- read-after-write visibility;
- complete collection listing for the probe keys;
- `If-None-Match: *` preventing replacement;
- delete visibility; and
- exact byte-range behaviour, reported separately as optional.
The checker is an implementation acceptance target for compatible servers, not a claim that every WebDAV server must
support Adaptive Journal. It touches no repository objects, attempts to remove every probe, reports incomplete cleanup,
and never interprets authentication, permission, timeout, malformed response, or server failure as absence.
Conditional create, binary fidelity, complete listing, read-after-write visibility, and delete visibility are required.
Range remains optional. The user selects whole-pack or Range retrieval based on their endpoint and own latency and
throughput preference; the checker does not benchmark or recommend a policy.
Successful mutations do not receive unconditional confirmation requests. An ambiguous response is classified as
`verify-first`, and the exact immutable key is read before retrying. The existing opaque WebDAV layout and Adaptive
layout are detected separately; a mismatch requires a remote rebuild or another namespace.
## Working hypotheses and illustrative estimates
The following values are planning assumptions for the first WebDAV experiment. They are not benchmark results, a
provider-pricing statement, or a performance guarantee. The implementation must record the corresponding measurements
before this ADR presents either Journal format as generally faster or smaller.
### Cost and lifecycle assumptions
The initial deployment model is a self-hosted or quota-priced WebDAV service without a direct per-request tariff.
Request count still matters because every request consumes a round trip, server work, connection capacity, and a share
of any rate limit. Commercial services, gateways, and managed hosting may use a different charging model, so the
adapter must not infer cost policy from the WebDAV label.
Storage capacity is expected to be the more visible constraint. RFC 4331 defines optional
[`DAV:quota-used-bytes` and `DAV:quota-available-bytes`](https://www.rfc-editor.org/rfc/rfc4331.html) properties for
collections. When both are available, a connection diagnostic may report them as server-supplied evidence. Their
absence means 'unknown', not 'unlimited', and quota reporting is not an Adaptive safety requirement.
The first experiment has no ordinary remote Garbage Collection or repacking guarantee. Commit Bundles and external
Packs remain immutable, unreachable data may remain retained, and an interrupted publication may leave an unreferenced
Pack. A remote Rebuild is therefore the only guaranteed way to reclaim all experimental history. This makes retained
bytes and object count acceptance measurements rather than later operational details.
Adaptive and Opaque Journal consume the same local PouchDB Chunk split. A smaller average changed Chunk can reduce
retained history in either format. Adaptive adds authenticated routes and independently framed records, while Opaque
adds its own container, compression, and encryption overhead. No comparative storage saving is assumed before both
representations are measured with the same changes.
### Ordinary request model
The object-profile model is shared with S3. Let:
- `B` be the number of Metadata batches published as distinct Commit Bundles;
- `X` be the number of new external Pack objects needed by those batches;
- `J` be the number of Opaque Journal objects produced for the same changes;
- `W` be the number of visible Writer streams;
- `K` be the number of those Writer descriptors not already cached by the opened process;
- `P` be the number of additional complete Pack-container reads needed for missing Chunks; and
- `M` be the number of missing Chunk frames fetched with Range.
After the endpoint safety result and repository binding have been cached for the active configuration, ordinary
publication performs approximately `J` Opaque PUTs or `B + X` Adaptive PUTs. Every Adaptive batch creates one Bundle;
each external Pack adds one preceding PUT. A confirmed conditional create does not add a read-back. An ambiguous PUT
adds one exact-key GET before any retry and remains outside the successful-first-attempt estimate.
The generic object-store receive path performs approximately:
| Retrieval policy | Requests without WebDAV listing reuse |
| --- | ---: |
| `whole-pack` | `1 + W + K + B + P` |
| `range` | `1 + W + K + B + M` |
The `1 + W` term represents one Writer listing and one Commit listing per Writer. A flat WebDAV collection normally
cannot apply those logical prefixes on the server: each call may require another Depth-one `PROPFIND` over the same
collection. The WebDAV target is therefore one complete listing at the start of a receive phase, filtered locally for
every Writer and Commit prefix in that phase. This receive-phase listing must expire before the next receive phase; it
must never become an unbounded cross-synchronisation cache. A Commit which becomes visible after the listing response
is handled by the next receive phase, which is consistent with immutable eventual replication.
With that explicitly scoped reuse, the request estimates become `1 + K + B + P` and `1 + K + B + M`. For 100 new
inline-Pack Bundles which require no additional Pack read, the difference is illustrative rather than a benchmark:
| Visible Writers | Descriptor state | Repeated-prefix listing | One receive-phase listing |
| ---: | --- | ---: | ---: |
| 1 | first receive (`K = 1`) | 103 | 102 |
| 1 | warm process (`K = 0`) | 102 | 101 |
| 10 | first receive (`K = 10`) | 121 | 111 |
| 10 | warm process (`K = 0`) | 111 | 101 |
Opaque catch-up is approximately one listing plus one GET per new Journal object, or `1 + J`. Holding `B = J = 100`
would therefore give 101 operations. That comparison does not assume that the two batchers produce equal object counts
for a real editing history.
### Listing-volume hypothesis
Request count alone understates WebDAV discovery cost. Let `N` be the number of resources in the configured flat
collection and `L` the average XML response bytes per resource for a minimal Depth-one `PROPFIND`. Repeating the
listing for each logical prefix transfers roughly `(1 + W) * N * L`; one receive-phase listing transfers roughly
`N * L`.
Assuming `L = 1 KiB` only to make the scale visible gives:
| Collection resources (`N`) | One listing | Repeated listing, `W = 1` | Repeated listing, `W = 10` |
| ---: | ---: | ---: | ---: |
| 1,000 | about 1 MiB | about 2 MiB | about 11 MiB |
| 10,000 | about 10 MiB | about 20 MiB | about 107 MiB |
| 100,000 | about 98 MiB | about 195 MiB | about 1.05 GiB |
Actual XML size, selected properties, URL length, compression, server implementation, and collection scope can change
these values materially. The adapter should request only the properties it needs and measure both response bytes and
wall time. The table explains the reuse target; it is not an estimate of every WebDAV server.
Ignoring cleaned probe resources and abandoned Packs, the current Commit Bundle layout contains approximately
`1 + W + B + X` remote objects: one manifest, one descriptor per Writer, one object per Bundle, and one per external
Pack. Object count therefore grows with synchronisation batches even when the retained payload is small, and it feeds
back into subsequent `PROPFIND` cost.
### Retained-byte hypothesis
For a localised edit history, let `E` be the number of separately published changes and `D` the average bytes of newly
created encoded Chunk frames per change. Before Garbage Collection or Rebuild, retained Chunk bytes grow approximately
as `E * D`, plus Metadata, Commit, route, frame, server-metadata, and abandoned-publication overhead.
For a 10 MiB incompressible binary file whose content-defined split resumes after one changed Chunk:
| Published changes (`E`) | Average changed Chunk | Approximate new Chunk bytes retained | New Bundle objects when each Pack stays inline |
| ---: | ---: | ---: | ---: |
| 100 | 1 MiB | 100 MiB | 100 |
| 1,000 | 1 MiB | about 0.98 GiB | 1,000 |
| 100 | 256 KiB | 25 MiB | 100 |
| 1,000 | 256 KiB | 250 MiB | 1,000 |
These figures exclude Metadata and protocol overhead and do not predict text splitting. They show why a 10 MiB live
file can consume much more than 10 MiB remotely after many committed edits. They also show why reducing Chunk size is a
storage-versus-record-count decision shared with Opaque Journal, not an Adaptive-only recommendation.
### Pack-target hypothesis
Changing the external Pack target does not normally duplicate Chunk frames: a Pack is their concatenation without
capacity padding. Smaller targets chiefly increase object count, route data, PUTs, listing work, and later whole-Pack
GETs. Larger targets increase peak buffer size, retry cost, transfer duration, and exposure to mobile watchdog or proxy
timeouts.
For about 100 MiB of newly required encoded Chunk frames in one batch, the following serial-transfer model ignores RTT,
TLS setup, encryption, server processing, contention, and retries:
| Pack target | External Packs (`X`) | PUTs including the Bundle | One target-sized upload at 5 / 10 Mbit/s |
| ---: | ---: | ---: | ---: |
| 8 MiB | 13 | 14 | about 13.4 / 6.7 seconds |
| 16 MiB | 7 | 8 | about 26.8 / 13.4 seconds |
| 32 MiB | 4 | 5 | about 53.7 / 26.8 seconds |
| 64 MiB | 2 | 3 | about 107.4 / 53.7 seconds |
The total Chunk payload remains about 100 MiB in each row. The initial WebDAV experiment therefore retains the current
32 MiB preferred target as a compromise, not as a universal optimum or a user-facing recommendation. Measurements on
the disposable server must record peak memory, per-PUT duration, retry behaviour, total object count, and watchdog
survival. A later decision may lower the internal target for timeout-constrained endpoints without changing the wire
format.
## Staged acceptance
### Adapter and Commonlib integration
Unit tests own HTTP status classification, conditional-create verification, flat-name round trips, complete listing,
probe isolation, cleanup reporting, Range validation, and whole-pack fallback. A disposable WebDAV integration runs
the safety checker before exercising Adaptive Metadata and Chunk synchronisation. The integration also proves that one
receive phase reuses a single collection listing, then invalidates it so the next phase can observe a new Commit.
### CLI end-to-end acceptance
The built CLI applies an Adaptive WebDAV Setup URI to a second independent database and synchronises text and binary
Chunk-backed files through a real disposable server. One client uses whole-pack retrieval; Range is added to this layer
only when the selected test server proves it. The CLI test does not repeat the complete endpoint capability matrix.
### Host settings and UI
The WebDAV dialogue exposes Adaptive mode only with clear capability-check results. It persists the expected repository
ID and the selected retrieval policy, defaults to whole-pack, and reports that Range is optional. Focused tests own
profile and Setup URI preservation without contacting a server.
### Real-host end-to-end acceptance
One real-Obsidian workflow uses the same known disposable WebDAV implementation, runs the safety gate, and proves a
representative Chunk-backed transfer. Servers outside that fixture are diagnosed by the checker rather than added to a
large real-host matrix.
## Alternatives rejected
### Trust advertised WebDAV methods
Method advertisement does not prove the conditional, listing, visibility, and byte semantics required for immutable
publication.
### Treat Range as mandatory
Whole-pack retrieval is correct and often competitive for throughput. Requiring Range would exclude otherwise safe
servers for an optional optimisation.
### Add server-specific compatibility branches
The remote may be any implementation or proxy composition. Semantic checks produce a maintainable contract, while a
growing server-name table would remain incomplete and become stale.
## Consequences
- WebDAV remains experimental because suitability is endpoint-specific.
- The safety checker gives a concrete reason when a server cannot host Adaptive Journal.
- Common object-pack behaviour is inherited from the earlier S3 boundary, keeping WebDAV-specific tests focused on HTTP
semantics and name mapping.
- Optional Range support can improve request efficiency without becoming a data-availability requirement.
+9 -5
View File
@@ -37,9 +37,9 @@ The Obsidian host injects the screen wake-lock manager from the `octagonal-wheel
Finite replication enters both counts only after readiness checks have succeeded and leaves them after `openReplication(..., continuous: false, ...)` settles. A successful completion has reached the latest sequence in that operation's scope and is therefore an authoritative quiescence boundary for chunk retrieval. A failed operation does not prove latest state, but can no longer deliver documents from that attempt. Failure handling runs afterwards so a mismatch or recovery dialogue does not retain the activity. This includes the direct start-up synchronisation path as well as manual, event-driven, and periodic calls through `ReplicationService`. The unbounded continuous channel does not enter either boundary, but its finite initial pull-only catch-up does; the one-shot parameter fallback chain remains inside that boundary.
Delivery into the local database and application to the Obsidian Vault are deliberately separate lifetimes. A mobile device can have only a short opportunity to obtain remote data, while applying a large downloaded batch to the Vault is durable, offline-capable work which can continue or resume later. The replication-result queue and its recovery snapshot therefore remain outside `boundedRemoteActivityCount`. Finite remote activity ends when the transfer operation settles, even when the `📥` queue still contains documents awaiting Vault application.
Delivery into the local database and application to the Obsidian Vault are deliberately separate lifetimes. Finite remote activity ends when the transfer operation settles, even when the `📥` queue still contains documents awaiting Vault application. The Obsidian host tracks that queue through a separate `boundedLocalApplicationActivityCount`, so local Vault writes do not increment either remote-activity count or keep the `📲` indicator visible.
This separation also keeps the activity indicators truthful: `📲` describes a finite remote operation and must not remain active solely because local Vault writes are pending. The existing replication-result count continues to describe that local queue. If a future feature offers screen-awake protection while applying downloaded documents, it must use a separately typed local-application activity or power-policy boundary, preserve the current behaviour by default, and avoid incrementing either remote-activity count.
Applying downloaded document changes enters one local-application boundary from the first queued document until the queue and its in-progress set are both empty. The final empty recovery snapshot is attempted before the boundary settles; snapshot failure is logged and still releases the boundary. Additional batches share the existing boundary. Suspending replication-result processing releases it, and resuming reacquires it while work remains. The same host activity runner supplies best-effort Wake Lock protection, while the visibility lifecycle waits for both remote and local bounded activity to finish.
Manual P2P commands which bypass `ReplicationService` enter the broad boundary. Direct P2P pull and push entry points are therefore both protected as finite remote work, covering the Obsidian panes, CLI, and Webapp. A pull or bidirectional synchronisation also enters the narrower finite-replication boundary because it can place documents in the local database. A push-only request remains broad-only: it cannot satisfy a local missing-chunk read and must not present itself as a delivery source. Automatic synchronisation on peer discovery, a pull requested by a remote peer, and a watched pull following a peer progress notification enter both boundaries because each can deliver local documents. A normal P2P peer-selection dialogue represents one broad finite session: it remains inside the boundary while waiting for a peer and while the person may perform repeated synchronisations, then settles when the dialogue closes and any in-flight synchronisation has finished. Closing without synchronising returns a failed result and releases the boundary. The 'Start Sync & Close' action completes its synchronisation before closing. This deliberately protects peer discovery and selection, because display sleep can interrupt discovery or connection establishment and require the person to start detection again. It may therefore retain a Wake Lock longer than the network transfer alone. A transfer performed inside that session temporarily adds a nested activity; the count remains a logical-operation count rather than a connection total.
@@ -76,6 +76,8 @@ P2P does not yet contribute to the physical-request count because it does not ha
The platform activity runner remains injected. Common library and headless consumers can omit it while retaining the same bounded activity count and operation semantics.
The Obsidian-specific `ObsidianReplicatorService` owns `boundedLocalApplicationActivityCount` and reuses the injected activity runner. It is deliberately absent from the common service contract: CLI and Webapp processing continue directly, while the Obsidian host uses the count for Wake Lock and visibility-lifecycle policy without changing remote-operation reporting.
## Non-Goals
- Do not count continuous replication as a bounded activity.
@@ -85,11 +87,11 @@ The platform activity runner remains injected. Common library and headless consu
- Do not guarantee protection against operating-system suspension, closing a laptop lid, forced termination, network loss, or a user-initiated sleep action.
- Do not add a lifecycle timeout which would abort an unusually slow rebuild. A genuinely stalled operation may postpone LiveSync's visibility suspension until it settles, but the platform may still suspend or terminate background work.
- Do not broaden `keepReplicationActiveInBackground`; it remains an opt-in desktop policy for continuous and periodic operation after finite work has ended.
- Do not include offline scans, unrelated local storage reflection, or the durable replication-result queue in this boundary. They are offline-capable and require a separate decision if activity reporting or power policy is added later.
- Do not include offline scans, unrelated local storage reflection, or the durable replication-result queue in either remote-activity count. The queue uses its separate local-application boundary.
## Verification
Before changing the transfer/application separation, add a deterministic regression scenario which leaves downloaded documents queued after finite transfer settles, verifies that remote activity has ended, persists the queued state, and resumes Vault application after suspension or restart. An optional local-application Wake Lock feature requires its own enabled and disabled cases; existing remote-operation E2E is not evidence for that separate policy.
Before changing the transfer/application separation, keep deterministic coverage which leaves downloaded documents queued after finite transfer settles, verifies that remote activity has ended, and retains the separate local-application boundary until Vault application and the final recovery snapshot settle.
Unit tests cover:
@@ -106,6 +108,8 @@ Unit tests cover:
- remote chunk fetching remaining inside the shared boundary from synchronous queue acceptance through local persistence and terminal notification;
- missing-chunk waiters rechecking local storage when observed per-identifier claims and finite replication have settled;
- the finite-replication count excluding other bounded work;
- replicated document application sharing one local boundary, settling after the final recovery snapshot, and releasing around processing suspension;
- local application activity leaving both remote-activity counts unchanged;
- standard, fast, remote, and combined rebuild activity boundaries;
- Rebuilder-owned confirmation and completion dialogues remaining outside rebuild activity;
- fallback from fast fetch avoiding a nested activity boundary;
@@ -125,5 +129,5 @@ The exact Fancy Kit screen wake-lock behaviour is covered by its package and Har
- One finite activity definition drives Wake Lock, lifecycle protection, and status UI without coupling common library code to Obsidian or browser globals.
- Callers can observe accurate logical activity even in CLI and Webapp hosts which do not inject a Wake Lock implementation.
- Rebuild operations now retain Wake Lock and lifecycle protection across their longest interruption-sensitive phases without retaining them for Rebuilder-owned pre-operation or completion dialogues. Post-reset P2P discovery and selection remain protected as an intentional part of completing the rebuild.
- Downloaded documents may remain in the durable Vault-application queue after remote activity has ended, allowing transfer and offline application to follow different mobile lifetimes without presenting local writes as communication.
- Downloaded documents may remain in the durable Vault-application queue after remote activity has ended, while a separate local-application boundary retains best-effort Wake Lock and lifecycle protection without presenting local writes as communication.
- Users can now distinguish the lifetime of a finite remote operation from approximate request activity within it.
@@ -65,7 +65,7 @@ The special meaning would duplicate `activeConfigurationId`, make a user-visible
### Add profile naming and full list editing to onboarding
That would make the first-run path longer and duplicate the established Remote Databases interface. Automatic descriptive names and later renaming keep this change limited to data integrity and consistent selection.
That would make the first-run path longer and duplicate the established Saved connections interface. Automatic descriptive names and later renaming keep this change limited to data integrity and consistent selection.
### Replace the compatibility fields immediately
@@ -77,11 +77,11 @@ Commonlib unit tests cover preserving existing profiles, opaque-ID insertion, ge
Self-hosted LiveSync unit tests cover preserving modern Setup URI profiles and their active selection, retaining legacy Setup URI and QR migration, adding CouchDB and Object Storage profiles beside an existing profile, independent P2P selection, fresh P2P selection as both main and P2P remote, and cancellation without mutation.
The real-Obsidian onboarding E2E owns the invitation, dialogue presentation, safe-area and touch-target checks, cancellation, and command reopening. It does not contact a remote or submit credentials. Remote connection correctness remains owned by the CouchDB, Object Storage, P2P, and two-Vault suites. The end-to-end Setup URI and provisioning acceptance workflow remains a separate release gate.
The real-Obsidian onboarding E2E owns the invitation, dialogue presentation, safe-area and touch-target checks, cancellation, and reopening from the Setup pane. It does not contact a remote or submit credentials. Remote connection correctness remains owned by the CouchDB, Object Storage, P2P, and two-Vault suites. The end-to-end Setup URI and provisioning acceptance workflow remains a separate release gate.
## Consequences
- Manual onboarding and the Remote Databases pane share one Commonlib profile contract.
- Manual onboarding and the Saved connections list share one Commonlib profile contract.
- Existing profiles survive reconfiguration, and a newly configured connection becomes explicitly selectable.
- Modern imports retain user-assigned profile identity and names.
- Legacy Setup URIs continue to work through an isolated compatibility boundary.
+103
View File
@@ -0,0 +1,103 @@
# Architectural Decision Record: Browser-assisted P2P Connection Check
## Status
Accepted — WebPeer provides a disposable, browser-observed connection check with a clean one-device path and an optional same-room second-device attempt.
## Context
P2P availability depends on the browser, device, signalling relay, NAT, firewall, and current network. A Setup URI can make a trial easier, but a connection observed between a browser and one device does not prove that two other devices can connect to each other, and a WebRTC connection does not prove that LiveSync documents can be synchronised correctly.
Showing this experiment inside the ordinary LiveSync interface would mix a temporary connectivity check with the state of a real Vault. It would also make a browser-owned diagnostic result appear to be a plug-in-owned synchronisation result.
The Commonlib diagnostic counters are cumulative within one JavaScript realm. A negotiation may create more than one `RTCPeerConnection`, a failed attempt may later be followed by a successful retry, and closing a successful connection is not a failure. The counters therefore cannot be interpreted as peer counts or as mutually exclusive outcomes.
## Decision
WebPeer provides a separate page labelled 'P2P connection check'. It acts as a temporary reference peer and creates a fresh, random P2P configuration locally in the browser. The page displays an encrypted Setup URI, a QR code containing that URI, and the separate Setup URI passphrase.
The initial implementation is part of the WebPeer production build and is served over HTTPS or from `localhost`. A downloaded copy of `check.html` alone is not a supported standalone application because the build uses separate module assets and origin-scoped browser storage.
Each check has these boundaries:
| Participant | Responsibility |
| -------------------------- | --------------------------------------------------------------------------------------------------- |
| Browser page | Joins the generated room as the diagnostic reference peer and displays its own diagnostic counters. |
| Desktop or mobile LiveSync | Opens the generated Setup URI in a dedicated empty Vault and attempts the ordinary P2P connection. |
| User | Uses an empty Vault for every device and chooses either a fresh room or the explicitly less isolated same-room second-device attempt. |
The generated device configuration:
- uses a dedicated P2P application identifier so that test rooms do not overlap ordinary LiveSync rooms;
- uses random room, P2P, Vault-encryption, and Setup URI secrets;
- enables P2P auto-start so that a dedicated empty test Vault attempts the connection after setup;
- does not enable automatic broadcasting, peer acceptance, watching, or replication;
- omits device-local names from the Setup URI; and
- enables diagnostic WebRTC wrapping only on the browser reference peer.
The browser joins the room only after the user explicitly starts monitoring. Preparing the Setup URI and QR code does not contact the signalling relay. The page must remain open while monitoring.
The primary path checks one target device. The user selects either desktop or mobile before generating the configuration. Starting a fresh check reloads the page, creates a new room, and resets the page-scoped diagnostic baseline; this remains the clearest way to compare devices.
After the first connection succeeds, the page can scroll back to the existing QR code without generating new credentials. It also offers an optional same-room second-device attempt. This keeps the browser and first device connected, records the current diagnostic totals and active `RTCPeerConnection` identifiers as a local baseline, and reuses the exact Setup URI for another empty Vault. It reports an additional connection only when both the successful-connection total and the number of simultaneous active connections increase, with an active connection identifier absent from the baseline. A reconnect which replaces the first connection can therefore increase negotiation counters without satisfying the additional-connection result.
The same-room result remains a convenience rather than an isolated comparison. Its totals are still cumulative, a connection is not a device identity, it depends on the first device remaining connected, and unusual reconnect timing can be harder to interpret than a fresh room. The interface keeps the fresh-check action available and explains this limitation beside the additional result.
The page displays the following Commonlib diagnostic totals:
- new connection states;
- successful connection states;
- failed connection states; and
- closed connection states.
A successful total greater than zero proves that this browser and the target device established at least one WebRTC connection in the generated room. That successful result remains valid if a later closed or failed total increases. A failed total without a successful total describes a failed attempt, not a final verdict, because the transport may retry. If no success has been observed after the documented observation period, the result is inconclusive for P2P on the current network. The page recommends CouchDB for predictable synchronisation when repeated checks on the intended networks do not connect.
The page does not claim to verify document transfer, conflict handling, sustained connectivity, background execution, or a direct desktop-to-mobile path. An optional final test can use two disposable empty Vaults and a note round trip directly between the user's devices. That direct test is more representative, but its result cannot be observed reliably by the browser page without a separate reporting protocol.
## Security and privacy
All credentials are generated with Web Crypto in the browser. The Setup URI is encrypted, its passphrase is shown separately, and neither value is uploaded merely by preparing the check. Their read-only fields disable browser autocomplete and spell-check; copying either value to the system clipboard remains an explicit user action. Starting monitoring uses the configured signalling relay, which is the existing public relay by default, and exposes the usual P2P signalling metadata to that service. WebRTC and optional relay or TURN behaviour retain their existing Commonlib security and privacy properties.
The production page does not accept a relay override from a deployed origin. A `relay` query parameter is honoured only when the page itself is served from a loopback host, and only for `ws:` or `wss:` URLs. This narrow boundary allows the real-Obsidian E2E scenario to use the local Compose relay without making an arbitrary relay query part of the hosted user workflow.
The generated configuration is disposable. It must be used only in a dedicated empty Vault, must not replace a production Vault's settings, and must not be retained as the credentials for a later production setup. The browser page stores no ordinary Vault files for this workflow, although browser-local database infrastructure is opened to satisfy the existing WebPeer runtime contract.
## Alternatives rejected
### Display the result in the LiveSync plug-in
The observation belongs to the browser reference peer and does not prove LiveSync document synchronisation. Keeping the result beside the QR code makes the evidence and its owner explicit.
### Use browser, desktop, and mobile simultaneously as the primary comparison
The diagnostic totals are cumulative and do not attribute an `RTCPeerConnection` to a named target device. Separate random rooms provide a clearer result. Same-room reuse is retained only as an explicit convenience, with a counter baseline and another simultaneous active connection required for its result.
### Treat any failure increment as a final failure
Negotiation can fail and then retry successfully. A later successful increment takes precedence over earlier failure increments.
### Treat browser success as proof of device-to-device synchronisation
Browser-to-device success is a useful preflight check, but it does not exercise the exact desktop-to-mobile network path or a LiveSync note round trip.
### Put the Setup URI passphrase in the QR code
Keeping the encrypted URI and passphrase separate preserves the existing Setup URI boundary and avoids turning one captured QR code into the complete configuration secret.
### Distribute one standalone downloaded HTML file
WebPeer already depends on a compiled module graph, origin-scoped storage, and secure browser capabilities. A hosted production build reuses those boundaries and can be updated consistently; a single self-contained file would require a separate bundling and trust model.
## Verification
Unit tests decode each generated Setup URI and verify its isolation identifier, random credentials, remote-profile selection, automatic-start policy, absence of a device-local name, and browser-only diagnostic setting. Pure outcome tests cover waiting, failed attempts, later success, and closure after success.
The WebPeer browser smoke test verifies that the production page can prepare a desktop or mobile check, render the QR code, display a separately formatted passphrase, expose zeroed counters without contacting the signalling relay, and return to the unchanged QR code. Pure tests verify that a reconnect which replaces the first connection does not satisfy the same-room additional-connection rule. The focused real-Obsidian E2E test serves the production page from loopback, starts the local Compose Nostr relay, applies the browser-generated Setup URI through visible onboarding in one empty Vault, records the same-room baseline, reuses the unchanged URI in a second empty Vault, and requires the browser to observe another simultaneous active connection before retaining a result screenshot. Existing Commonlib tests remain authoritative for diagnostic WebRTC wrapping, Trystero negotiation, and P2P lifecycle behaviour.
## Consequences
- Users can assess whether P2P is plausible on each intended device and network before relying on it.
- Results remain visibly scoped to connectivity rather than synchronisation correctness.
- A fresh test requires a fresh page session and disposable empty Vault; same-room reuse requires another empty Vault and retains cumulative counters.
- CouchDB remains the recommended predictable option when representative P2P checks repeatedly fail.
- Direct device-to-device note transfer remains a separate, optional verification step.
+2 -2
View File
@@ -46,7 +46,7 @@ Lifecycle operations on one `LiveSyncTrysteroReplicator` are serialised. A close
Relay sockets retain their Trystero-provided close handlers. LiveSync pauses relay reconnection, closes the sockets, and later resumes reconnection through Trystero's public functions. It does not replace `socket.onclose`, because Trystero uses that handler to retire and recreate shared relay clients correctly.
P2P setup follows the transport's actual ownership model. First-device initialisation resets and scans the local database, but does not attempt to lock, reset, or upload to a non-existent central remote database. Its confirmation dialogues therefore describe preparing this device and do not present the central-server overwrite warnings or remote-configuration option. An additional device performs one explicit peer-selection and finite Fetch pass, then resumes database and Vault reflection. The generic second convergence pass remains reserved for central remote types because repeating it for P2P would ask the user to select the same peer twice.
P2P setup follows the transport's actual ownership model. Initialising the first device resets and scans the local database, but does not attempt to lock, reset, or upload to a non-existent central remote database. Its confirmation dialogues therefore describe preparing this device and do not present warnings about overwriting a central server or an option to fetch its configuration. An additional device selects a peer once, performs Fetch once, then resumes database and Vault reflection. The generic second convergence pass remains reserved for central remote types because repeating it for P2P would ask the user to select the same peer twice.
## Ownership
@@ -74,7 +74,7 @@ This interferes with Trystero's shared relay clients. The public pause and resum
## Verification
Commonlib unit tests prove that normal P2P host closure calls `room.leave()` without directly closing Trystero-owned peer connections. Additional package tests cover the action API, replaceable peer-event subscriptions, multiple RPC transport disposers, serialised open and close operations, local-only first-device initialisation, and one-pass additional-device Fetch.
Commonlib unit tests prove that normal P2P host closure calls `room.leave()` without directly closing Trystero-owned peer connections. Additional package tests cover the action API, replaceable peer-event subscriptions, multiple RPC transport disposers, serialised open and close operations, initialisation of the first device without a central remote, and Fetch running once for an additional device.
Self-hosted LiveSync unit tests prove that settings and database replacement leave panes on the current replicator, and that an explicit P2P rebuild bypasses the policy intended for ordinary replication.
@@ -60,7 +60,7 @@ Keep configured-state inference separate from new-Vault initialisation. If an ex
### Onboarding activation and initialisation
- Keep an unconfigured Vault outside database initialisation, offline scanning, and configured-only checks. Offer setup through the long-lived onboarding Notice and the permanent command instead of opening a competing dialogue automatically.
- Keep an unconfigured Vault outside database initialisation, offline scanning, and configured-only checks. Offer setup through the long-lived onboarding Notice, and allow the wizard to be reopened from the Setup pane instead of opening a competing dialogue automatically.
- For new-device onboarding, reserve Rebuild before enabling and saving the accepted settings.
- For an unconfigured existing device, reserve Fetch before enabling and saving imported or manually confirmed settings.
- Suspend the current runtime after the flag has been written, apply the accepted settings through the scheduler's preparation callback, and request restart only after that callback succeeds.
File diff suppressed because it is too large Load Diff
+98
View File
@@ -0,0 +1,98 @@
# How peer-to-peer synchronisation works
Peer-to-peer (P2P) synchronisation transfers Vault data between LiveSync devices through WebRTC. It does not require a central database containing a copy of the Vault. It does require a signalling relay so that devices can discover one another and establish a connection.
For the procedure for the first and additional devices, see [Set up peer-to-peer synchronisation](setup_p2p.md). For connection problems, see [Peer-to-Peer Synchronisation Tips](tips/p2p-sync-tips.md).
## Connection model
```mermaid
flowchart LR
A["Device A"] <-->|"Discovery and connection signalling"| S["Signalling relay"]
S <-->|"Discovery and connection signalling"| B["Device B"]
A <-->|"Encrypted Vault synchronisation"| B
A -.->|"Fallback encrypted WebRTC traffic"| T["TURN server"]
T -.-> B
```
The signalling relay and TURN server have different roles:
- The **signalling relay** is required for peer discovery and connection negotiation. LiveSync uses Nostr-compatible WebSocket relays for this role. The relay does not store or transfer Vault contents.
- A **TURN server** is an optional fallback. WebRTC uses it to relay the encrypted peer connection only when the devices cannot establish a direct path through their networks.
## The project's public signalling relay
The project author operates a public signalling relay as a best-effort convenience. Selecting **Use the project's public signalling relay** means that no signalling server needs to be provisioned for an ordinary setup.
The public relay:
- is not a Vault storage service;
- may observe signalling metadata, such as connection timing and network addresses;
- has no availability or log-retention guarantee; and
- can be replaced with another compatible relay at any time by updating every device in the P2P group.
Use a signalling relay which is acceptable for your privacy and availability requirements. A controlled deployment may use its own Nostr-compatible relay.
## Signalling relay and TURN server
Both settings contain server addresses, but they are not interchangeable.
| Setting | Required | Carries Vault contents | Purpose |
| --- | --- | --- | --- |
| **Signalling relay URLs** | Yes | No | Finds peers and exchanges the information needed to establish WebRTC connections. |
| **TURN server URLs** | Only when direct WebRTC connectivity fails | Encrypted WebRTC traffic | Relays traffic between peers when NAT or firewall rules prevent a direct path. |
A TURN provider cannot read LiveSync's encrypted Vault contents, but it can observe connection metadata and traffic volume. Use a provider you trust. The project does not operate an official TURN service.
## P2P Status
The **P2P Status** pane is the current Obsidian interface for P2P connections.
- After a P2P configuration exists, the command **Self-hosted LiveSync: P2P Sync : Open P2P Status** is available from the command palette.
- The P2P ribbon icon appears only after a P2P configuration exists.
- LiveSync does not open the pane merely because Obsidian has started. If the pane was already part of the saved Obsidian workspace, Obsidian may restore it.
- Workspaces containing the retired P2P pane are migrated to the current status pane. The retired command is no longer exposed.
The active P2P remote is selected independently from the main CouchDB or Object Storage remote. Devices can therefore use P2P alongside their main remote without replacing it.
![P2P Status on desktop](../images/p2p-setup/p2p-status-pane.png)
![P2P Status in a mobile layout](../images/p2p-setup/p2p-status-pane-mobile.png)
**Open connection** joins the signalling room and makes the device available for discovery. **Disconnect** leaves the LiveSync room, stops its P2P replication service, and closes the signalling connections. It does not delete the saved P2P profile.
Every participating device must use the same signalling relay set, Group ID, and P2P passphrase. Each device should have a distinct device name. A peer which joins after another device is already connected is advertised to that device; use **Refresh**, or reconnect the device which should be discovered, if a peer is not yet listed.
## Manual and automatic data movement
**Replicate now** performs an explicit bidirectional synchronisation with the selected peer. This is the clearest option when proving a new configuration.
**Announce changes** and **Follow changes** provide a more continuous experience:
- The source device must enable **Announce changes** before it dispatches change notifications.
- A receiving device must enable **Follow changes** for that peer before it fetches in response to those notifications.
- A notification contains no Vault data. It only asks the following peer to fetch through the encrypted P2P connection.
- Missing a notification does not make an explicit later synchronisation unsafe; **Replicate now** still compares the available data.
The peer's **More actions** menu can save these choices for that device:
- **Synchronise when this device connects** runs one synchronisation when that named peer is discovered.
- **Follow whenever this device connects** restores following for that named peer.
- **Include in the P2P synchronisation command** includes that peer when the command for registered targets is run.
![Persistent actions for a detected peer](../images/p2p-setup/guide-p2p-setup-peer-actions-menu.png)
Configure these only after a manual round trip has succeeded. Device names used by persistent rules should remain unique and stable.
## Approval and privacy
A device must approve a peer before serving its data. Permanent approval is stored; session approval lasts only for the current Obsidian session. Check the displayed device name before approving a request.
The encrypted Setup URI contains the shared P2P configuration but deliberately omits the device-specific name. Store the Setup URI and its passphrase separately, and generate a Setup URI for another device from a first device which has completed setup.
## Operational limits
- At least one device which already has the required data must be online while another device fetches it.
- P2P does not provide the continuously available central copy offered by CouchDB or Object Storage. Keep independent backups.
- Mobile operating systems may pause Obsidian in the background. Keep Obsidian visible and the device awake during initial transfer, rebuild, or a large synchronisation.
- Changing from CouchDB to P2P is not a repair operation for a stopped CouchDB setup. Diagnose the existing transport first.
+5 -60
View File
@@ -1,62 +1,7 @@
# User Guide: Peer-to-Peer Synchronisation (2026 Edition)
# Peer-to-peer synchronisation
Peer-to-Peer (P2P) synchronisation has evolved significantly. This guide covers the essential setup and the new features introduced in the 2026 updates.
This address is retained for links to an earlier P2P guide. The time-specific interface description has been replaced by stable documentation:
## 1. Core Concept: Server-less Freedom
P2P synchronisation allows your devices to talk directly to each other using WebRTC. A central server is not required for data storage, ensuring maximum privacy and "freedom."
## 2. Setting Up via P2P Status Pane
You no longer need to navigate through complex menus. Simply open the **P2P Status** (via the ribbon icon or command palette) and click the **⚙ (Cog)** icon.
This opens the **P2P Setup** dialogue where you can configure the essentials:
- **Room ID:** A unique identifier for your synchronisation group.
- **Passphrase:** Your encryption key. Ensure all your devices use the exact same passphrase.
- **Device Name:** A recognisable name for the current device (e.g., `iphone-16`).
Once you have saved the settings, return to the **P2P Status Pane** and click the **Connect** button to join the network.
*Tip: You can also toggle **Auto Connect** in the setup dialogue to automatically join the network whenever Obsidian starts.*
## 3. Real-time Control
The status pane in the right sidebar provides granular control over your synchronisation:
- **Active P2P Remote (new):** P2P now has its own active remote selection, separate from the normal active remote for database replication. Use the combo box next to the cog icon to choose which P2P remote configuration is active for P2P features.
- **Create P2P Remote (new):** Use the **+** button to open the P2P setup dialogue and create a dedicated P2P remote configuration. This is recommended when no P2P active remote has been selected yet.
- **Selection required (new):** If no P2P active remote is selected, the pane asks for selection before P2P target-related changes are saved.
- **Signalling Status:** Shows if you are connected to the relay (🟢 Online).
- **Live-push (Broadcast):** Toggle "Broadcast changes" to notify other peers whenever you make an edit.
- **Replicate now (🔄):** Start immediate bidirectional replication with a visible peer (Pull, then Push).
- **Watch (🔔/🔕):** Enable "Watch" on specific peers to automatically pull changes when they broadcast. This creates a "LiveSync-like" experience.
- **Sync target (🔗/⛓️‍💥):** Mark specific peers as **sync targets**. Peers marked here will be included when you run the **"P2P: Sync with targets"** command (see section 5). Click the button next to a peer to toggle it on (🔗, highlighted) or off (⛓️‍💥). This setting is persisted in your configuration.
## 4. Replication Dialogue
If you want to synchronise with a specific peer manually, use the **Replication** command or button. This opens the **Replication Dialogue** listing available devices.
Inside the dialogue, the **Server Status** card at the top confirms you are still connected while performing the sync.
The status card now shows a stable **Room ID suffix** above **Peer ID**. The Room ID suffix is better for identifying your P2P group, while Peer ID may change between connections.
Two actions are available per peer:
- **Sync** — Starts a bidirectional synchronisation (Pull then Push) and keeps the dialogue open so you can monitor progress or sync with additional peers.
- **Start Sync & Close** — Runs the same bidirectional synchronisation, waits for it to settle, then closes the dialogue. After a successful synchronisation, it also closes the signalling connection.
On supported mobile and desktop devices, LiveSync keeps the screen awake while this peer-selection dialogue is open and while its synchronisations finish. This is intentional: display sleep can interrupt peer discovery or connection establishment and require detection to start again. Wake Lock support remains best effort, does not keep a hidden application running in the background, and does not override operating-system sleep or suspension.
During a P2P rebuild, peer discovery and selection remain protected after the rebuild has started. Keep Obsidian visible until a peer is selected and the transfer finishes, because mobile platform restrictions can still pause or terminate a hidden application.
## 5. Syncing with Registered Targets via Command Palette
You can now trigger a synchronisation with all your pre-registered target peers in one step, without opening any UI.
1. Open the **Command Palette** (`Ctrl/Cmd + P`).
2. Run **"P2P: Sync with targets"**.
This command synchronises with every peer whose **SYNC** toggle is enabled in the **Detected Peers** list. If no targets are registered, or if the P2P server is not running, the command will notify you accordingly.
*Tip: Pair this command with a hotkey for a quick, keyboard-driven sync workflow.*
## 6. Technical Improvements in 2026
- **Decoupled Architecture:** The UI is now strictly separated from the core logic, making the plug-in more stable across different platforms (Mobile, Desktop, and Web).
- **Svelte 5 UI:** The interface has been rebuilt for better responsiveness and clearer status indicators.
- **Security:** All data remains end-to-end encrypted. Even the signalling relay never sees your actual notes.
- [Set up peer-to-peer synchronisation](setup_p2p.md) for configuring the first device, generating a Setup URI for another device, approving the connection, and verifying synchronisation in both directions.
- [How peer-to-peer synchronisation works](p2p.md) for signalling, TURN, privacy, the P2P Status pane, and automatic behaviour.
- [Peer-to-Peer Synchronisation Tips](tips/p2p-sync-tips.md) for connection troubleshooting.
+41 -6
View File
@@ -50,13 +50,13 @@ Create an ordinary test note and allow it to upload before adding another device
## Create a Setup URI for another device
Generate the additional-device Setup URI from the working first device. This captures the settings which that device is actually using, rather than asking another device to reuse the bootstrap URI produced during server provisioning.
Generate a Setup URI for another device from the working first device. This captures the settings which that device is actually using, rather than asking another device to reuse the Setup URI produced during server provisioning.
1. Open the Obsidian command palette on the first device.
2. Run `Self-hosted LiveSync: Copy settings as a new Setup URI`.
3. Enter a new passphrase which will protect this Setup URI, then select `OK`.
![Masked passphrase for a new additional-device Setup URI](../images/quick-setup/guide-quick-setup-copy-setup-uri-passphrase.png)
![Masked passphrase for a new Setup URI for another device](../images/quick-setup/guide-quick-setup-copy-setup-uri-passphrase.png)
4. Copy the resulting Setup URI, then select `OK`.
@@ -75,7 +75,7 @@ Start with a new or separately backed-up Vault. Do not use a production Vault co
5. Paste the new Setup URI generated by the first device, enter its Setup URI passphrase, and select `Test Settings and Continue`.
6. Review `Setup Complete: Preparing to Fetch Synchronisation Data`, then select `Restart and Fetch Data`.
![Additional-device Fetch confirmation](../images/quick-setup/guide-quick-setup-second-fetch.png)
![Fetch confirmation on the additional device](../images/quick-setup/guide-quick-setup-second-fetch.png)
7. For a new or empty Vault, select `Overwrite all with remote files`. For a Vault with local work, stop and choose the appropriate strategy from the [Fast Setup guide](./tips/fast-setup.md).
@@ -83,7 +83,7 @@ Start with a new or separately backed-up Vault. Do not use a production Vault co
8. When asked how to handle extra local files, the conservative choice is `Keep local files even if not on remote`. Select the delete option only when the local Vault is disposable and an exact remote copy is intended.
![Additional-device local file policy](../images/quick-setup/guide-quick-setup-local-file-policy.png)
![Local file policy on the additional device](../images/quick-setup/guide-quick-setup-local-file-policy.png)
9. Allow retrieval, file reflection, and any requested restart to finish. Keep Obsidian open until the LiveSync progress indicators have cleared.
@@ -100,6 +100,41 @@ Add optional features separately so that their ownership and initialisation dire
Do not enable both features for the same files.
## Manual configuration
## Configure CouchDB manually on the first device
If a Setup URI is unavailable, choose `Enter the server information manually` during onboarding. Manual configuration is an advanced path: verify the connection, encryption, remote profile, and synchronisation preset before initialising either side. After the first device works, use `Copy settings as a new Setup URI` from the command palette to add later devices through the recommended path.
Use this path when CouchDB is ready but a Setup URI is unavailable. It configures one first device through the visible onboarding dialogue; it does not provision or repair the CouchDB server. Add later devices with a Setup URI generated by this working first device instead of entering the credentials again.
1. Install and enable Self-hosted LiveSync in the intended Vault.
2. Select the `Welcome to Self-hosted LiveSync` Notice, choose `I am setting this up for the first time`, then confirm that you want to set up a new synchronisation.
3. On `Connection Method`, select `Configure a remote manually`, then select `Proceed with manual configuration`.
![Manual remote configuration option during onboarding](../images/couchdb-manual/guide-couchdb-manual-connection-method.png)
4. On `End-to-End Encryption`, decide how the synchronised data will be protected.
- For an ordinary new Vault, enable `End-to-End Encryption` and enter a strong Vault encryption passphrase.
- Enable `Obfuscate Properties` if remote document properties should also be concealed.
- Store the Vault encryption passphrase securely. It is separate from the passphrase used to protect a Setup URI.
![CouchDB Vault encryption settings with the passphrase masked](../images/couchdb-manual/guide-couchdb-manual-encryption.png)
5. On `Choose a synchronisation remote`, select `CouchDB`, then select `Continue to CouchDB setup`.
![CouchDB option in the list of synchronisation remotes](../images/couchdb-manual/guide-couchdb-manual-remote-selection.png)
6. Enter the complete CouchDB URL, username, password, and database name.
- Obsidian Mobile requires HTTPS. Plain HTTP is suitable only for a trusted local connection from a desktop device.
- Use credentials which are allowed to connect to the selected database and, when configuring the first device, create it if it does not exist.
![Manual CouchDB connection fields with the password masked](../images/couchdb-manual/guide-couchdb-manual-connection-details.png)
7. `Check server requirements` is optional. It sends the displayed credentials to the configured server through Obsidian's internal request API, and some checks require CouchDB administrator access. The initial check is read-only. If it offers a server change, review and confirm that individual change separately.
![Successful optional CouchDB server requirements check](../images/couchdb-manual/guide-couchdb-manual-server-requirements.png)
8. Select `Create or connect to database and continue`. Onboarding requires this connection test to succeed.
9. Review `Setup Complete: Preparing to Initialise Server`, then select `Restart and Initialise Server`.
10. Read the final overwrite warning. Select `I Understand, Overwrite Server` only when this device is intentionally the source of truth and a current backup exists.
11. A newly created database can show `Fetch Remote Configuration Failed` because it does not yet contain a saved preferred configuration. Select `Skip and proceed` only for this known new database.
12. Acknowledge `All optional features are disabled`, then keep Obsidian open until the initialisation progress has cleared.
Create and synchronise an ordinary test note. Once it has reached CouchDB, follow [Create a Setup URI for another device](#create-a-setup-uri-for-another-device), then [Add another device](#add-another-device). This keeps the second device aligned with the remote profile and encryption settings which the first device actually applied.
+120
View File
@@ -0,0 +1,120 @@
# Recovery and flag files
This guide covers emergency suspension, local database recovery, and deliberate remote reconstruction. These operations are not ordinary synchronisation.
> [!IMPORTANT]
> Back up every available Vault before recovery. If a central remote is involved, back up that database or bucket as well. Stop or suspend other LiveSync devices until you have chosen the authoritative copy.
If Obsidian will not start normally, do not give up. Flag files can be created or removed with the operating system's file manager while Obsidian is closed. They are the only supported way to intervene before the ordinary LiveSync boot-up sequence reaches its database and synchronisation work.
## First choose the authoritative copy
Use the least destructive operation which matches the evidence:
- If the correct data is uncertain, suspend all work with `redflag.md`, preserve every copy, and inspect them before proceeding.
- If the central remote is healthy and should win, use **Reset Synchronisation on This Device** or `flag_fetch.md`.
- If this device's Vault is healthy and should replace a damaged or unwanted central remote, use **Overwrite Server Data with This Device's Files** or `flag_rebuild.md`.
- If both the Vault and local database are healthy and the only concern is unused storage, Garbage Collection may be appropriate. It does not repair a damaged database.
Do not switch transport, enable P2P, or run Garbage Collection as a substitute for diagnosing a stopped CouchDB or Object Storage setup.
## Suspend before diagnosis
Close Obsidian completely, then create an empty file or directory named `redflag.md` at the root of the Vault. On the next start, LiveSync enters its emergency suspension state before ordinary database, file-watching, and synchronisation work continues.
While suspended:
1. Back up the Vault and any available remote data.
2. Check which device or remote contains the intended files.
3. Correct only the identified configuration or storage problem.
4. Remove `redflag.md`.
5. Start Obsidian and review the remaining suspension controls under `Hatch` -> `Scram Switches`.
The flag deliberately enables file logging, which may affect performance. Remove it after the emergency has been understood.
## Recover a conflicted or mismatched file
Use this workflow when one file, or a small number of known files, has conflicts, missing chunks, or a difference between the current Vault file and the local LiveSync database. The inspection is device-local: it does not query a remote database or prove that another device has the same chunks.
The `Hatch` recovery controls are ordered by escalation. Running **Recreate chunks for current Vault files** again with unchanged chunk settings and file contents produces the same chunks, and does not alter the revision tree. **Inspect conflicts and file/database differences** then provides actions for exact revisions. **Resolve All conflicted files by the newer one** is last because it applies a modification-time policy in bulk and logically deletes every other live version.
1. Stop editing the affected file, pause replication on the participating devices, and keep a separate copy of every readable version.
2. If another device or backup has the intended content, preserve that copy before changing any revision.
3. If the current Vault file is readable, select **Recreate current chunks**. This can restore only chunks derived from the current Vault contents; it cannot reconstruct unique bytes from an unavailable historical or conflict revision.
4. Select **Inspect conflicts and file/database differences****Begin inspection**.
5. Review the database winner, every conflict revision, and any unavailable shared ancestor separately. Revision identifiers, `Δsize`, `Δtime`, and chunk availability are diagnostic evidence; they do not decide which content is correct.
6. Use the wrench menu on the exact revision:
- **Compare with Vault** opens a read-only comparison for readable text.
- **Apply this revision to Vault** replaces the Vault file with that readable database revision.
- **Mark this revision as the Vault version** records an exact byte-for-byte match without creating a child revision.
- **Store Vault file as a child of this revision** preserves the current Vault bytes on that selected branch.
- **Retry reading revision** retries configured chunk retrieval without changing the revision tree.
- **Apply logical deletion to Vault**, **Discard this branch**, and **Discard unreadable revision** are destructive decisions. Use them only after preserving every version which may still be needed.
7. Synchronise the healthy source if chunks were restored, scan again, and confirm that the expected conflict or difference has disappeared before resuming ordinary editing.
An absent Vault file and a logical-deletion winner already agree and do not require a repair card unless another live branch remains. If the scan reports many unrelated files, or the local database itself is incomplete or corrupt, stop the per-file workflow and use [Reset synchronisation on this device](#reset-synchronisation-on-this-device) from a trusted remote. If the central remote must instead be reconstructed from an authoritative Vault, use [Overwrite server data with this device's files](#overwrite-server-data-with-this-devices-files).
## Reset synchronisation on this device
Use this when the remote copy is trusted but this device's local LiveSync database is incomplete, corrupt, or no longer aligned with it.
The readable flag is `flag_fetch.md`; the legacy name `redflag3.md` remains accepted.
On the next start, LiveSync:
1. pauses ordinary start-up work;
2. asks which remote to use when more than one remote profile exists;
3. asks how to treat existing Vault files;
4. discards and reconstructs the local LiveSync database from the selected remote; and
5. resumes only after the scheduled operation has completed or been cancelled safely.
For P2P, a source peer must be online, discovered, and selected in `P2P Rebuild`. Merely opening an empty signalling room does not complete Fetch. Closing the rebuild dialogue without selecting a peer reports failure and does not treat the local database as restored.
Review the [Fast Setup guide](tips/fast-setup.md) before using this operation on a Vault which contains unsynchronised local work.
## Overwrite server data with this device's files
Use this only when this device's Vault is the authoritative copy and the central remote should be reconstructed from it.
The readable flag is `flag_rebuild.md`; the legacy name `redflag2.md` remains accepted.
For CouchDB and Object Storage, this is destructive to the selected remote state. Other devices may still contain revisions or files which are not present in the authoritative Vault, so keep them stopped until the new remote has been verified and then reset them from that remote.
For a P2P-only setup, there is no central remote database to overwrite. Preparing the first device instead rebuilds its local LiveSync database from its Vault.
## Garbage Collection is not Rebuild
Garbage Collection removes unreferenced chunks while preserving the current database and its revision model. Use it only when:
- the Vault is healthy;
- the local LiveSync database is healthy;
- all relevant devices have synchronised; and
- the remaining historical and deletion state is understood.
Deleted documents, tombstones, live conflicts, and retained metadata are not free. Live conflict branches keep the chunks needed for review, while an ordinary superseded linear revision does not protect its former chunks. Garbage Collection can therefore make old content unreadable and cannot promise the smallest possible remote. Review the [Garbage Collection V3 specification](specs_garbage_collection.md) before using it.
Rebuild is a different operation. It reconstructs the database from a chosen authoritative state and is the more certain way to remove unwanted history or repair a damaged remote, but it is also more disruptive and can discard changes which exist only elsewhere.
## Flag-file reference
Create only the flag required for the chosen operation.
| File at the Vault root | Effect |
| --- | --- |
| `redflag.md` | Suspend ordinary LiveSync work for diagnosis. It remains until removed manually. |
| `flag_fetch.md` or `redflag3.md` | Schedule **Reset Synchronisation on This Device** from the selected remote. |
| `flag_rebuild.md` or `redflag2.md` | Schedule **Overwrite Server Data with This Device's Files**, or local P2P preparation when no central remote exists. |
Flag files themselves are excluded from synchronisation. Fetch and rebuild flags are removed by the scheduled workflow after completion or cancellation; `redflag.md` is a manual emergency stop.
## When the warning continues
If LiveSync still reports emergency suspension after a recovery dialogue has closed:
1. close Obsidian completely;
2. inspect the Vault root for every name in the table above;
3. remove only flags whose intended operation has finished or been abandoned;
4. restart Obsidian; and
5. check `Hatch` -> `Scram Switches` for remaining suspended file watching or database reflection.
If the intended authoritative copy is still uncertain, leave synchronisation suspended and collect a [full report](troubleshooting.md#collect-a-report) before changing the databases again.
+161
View File
@@ -0,0 +1,161 @@
# 1.0 preview release history
This document records the opt-in beta and release-candidate builds published before 1.0.0. Most users upgrading from 0.25.83 only need the consolidated [1.0.0 release notes](../../updates.md).
The prepared `1.0.0-rc.0` tag was not published as a plug-in release and is therefore omitted.
## 1.0.0-rc.1
27th July, 2026
The work towards 1.0 has become so substantial that I have written [an article about it](https://fancy-syncing.vrtmrz.net/blog/0036-livesync-1_0_0-en.html) (linked again here).
### Important
- This candidate retains the plug-in behaviour prepared for rc.0. The version was advanced because release tags are immutable; rc.0 was stopped during CLI validation before a plug-in release was published.
- This remains an opt-in pre-release for BRAT validation and does not replace the latest stable release. The exact rc.1 plug-in and CLI artefacts will be validated separately after publication.
### CLI and release validation
- CLI release validation now generates Setup URIs through the supported ESM package interface, allowing the Docker test to reach the CLI container instead of stopping during test preparation.
- The CLI Docker image now assigns its entrypoint permissions explicitly, so non-root execution does not depend on permissions inherited from the source checkout.
- Release finalisation now explicitly dispatches the CLI container workflow when CLI publication is selected, rather than relying on a workflow-created tag to start another workflow.
- Focused regression tests guard the ESM execution mode and deterministic container entrypoint permissions, while the existing release-workflow tests now require explicit CLI dispatch with non-dry-run, immutable-tag inputs.
### Testing
- The native CLI setup, put, cat, list, information, deletion, conflict-resolution, and revision-retrieval scenario completed with the packaged Commonlib dependency.
- The same scenario completed through the rebuilt non-root Docker image.
- The focused CLI and release-workflow unit tests passed after first demonstrating all three regressions against the unmodified implementation.
## 1.0.0-beta.5
26th July, 2026
### Improved
- **Inspect conflicts and file/database differences** now compares the current Vault file with the database winner and every live conflict revision. Compact, mobile-friendly indicators show missing chunks, `Δsize`, `Δtime`, whether the Vault matches the winner, and whether conflict branches remain.
- Each reported file and live revision now has a compact wrench menu. Its available actions can compare readable text, apply the selected revision to the Vault, record an exact byte match, store the Vault content as a child of the selected branch, retry retrieving missing chunks without changing the revision tree, or discard only the selected live branch after confirmation.
### Fixed
- A winning logical deletion is no longer reported as a missing Vault file when the file is already absent.
### Testing
- Strengthened Real Obsidian coverage for start-up with an existing configuration, Security Seed readiness, failure diagnostics, and P2P status pane placement in separate desktop and mobile sessions.
## 1.0.0-beta.4
25th July, 2026
### Improved
- **Verify and repair all files** now reports the database winner, every conflict revision, missing chunks, and unavailable shared ancestors separately. It can retry an exact revision without changing the tree, while discarding an unreadable live revision requires explicit confirmation.
- Command-palette actions now use clearer names and appear only when their feature and current context make them usable. Renamed commands keep their identifiers, so hotkeys already assigned to them continue to work. The onboarding wizard can be reopened from **Self-hosted LiveSync settings****Setup**.
- Text in setup and review dialogues can now be selected for copying or translation.
- When LiveSync adopts an available interface translation on first start-up, it now continues initialisation and leaves a persistent Notice from which the translation details can be opened, instead of waiting for an unsolicited dialogue.
### Fixed
- An unreadable conflict revision is no longer deleted automatically merely because its chunks are unavailable on the current device.
- Garbage Collection V3 now protects chunks required by every live conflict branch and the available revision ancestry needed to review and merge conflicts, instead of considering only the database winner. The action is offered only for CouchDB because P2P has no central database to compact and does not provide the device inventory required by the workflow. Collection now stops when device progress cannot be verified, and a compaction timeout is no longer followed by a contradictory success message.
### Testing
- Added regressions for revision repair, command availability, selectable dialogues, conflict-aware chunk reachability, device-progress safeguards, and compaction timeouts.
- Added a real CouchDB integration test for logical chunk deletion, shared and conflict chunk retention, compaction completion, downstream replication, and content-addressed chunk recreation.
- Added a real Obsidian encrypted reconnect scenario which replaces the remote Security Seed while one client retains the previous value, verifies that synchronisation refreshes it without restoring the old value, and proves a bidirectional encrypted round-trip.
## 1.0.0-beta.3
24th July, 2026
### Improved
- Enabling Hidden File Sync now opens one progress Notice before its setting is saved and reuses that Notice throughout the initial file scan, instead of stacking separate phase and restart Notices.
- P2P is now presented only after it has been configured: its status pane no longer opens at start-up, its ribbon icon remains hidden for CouchDB-only Vaults, and the retired P2P pane command has been removed. The current pane distinguishes announcing changes, following a peer, and persistent per-device actions. Setup and guidance now distinguish the required signalling relay from optional TURN, and describe the public signalling relay's privacy and availability limits.
- First-device P2P setup now accepts a successfully opened signalling room without requiring another peer to be online. Additional-device Fetch still requires selecting a source peer and completing `P2P Rebuild`.
- Manual CouchDB setup now distinguishes creating a first database from connecting an additional device to an existing one. Settings mode can save an unverified profile explicitly, while onboarding requires a successful connection, and each proposed server-configuration fix requires separate confirmation.
- Differences limited to the chunk hash algorithm, chunk size, or splitter version are now aligned automatically by default. Existing content remains readable, while an explicit opt-out and any difference which also involves an incompatible setting retain manual review.
### Fixed
- Choosing **Apply settings to this device, and fetch again** for a compatible configuration mismatch now applies the remote settings before Fetch, instead of updating the remote database with this device's settings.
- Accepted settings which control how new chunks are created now take effect before synchronisation is retried, rather than leaving the previous hash or splitter active until restart.
### Testing
- Added regressions for P2P configuration, the distinction between setting up the first device and using Fetch on an additional device, the P2P status pane, CouchDB setup policy, and mobile dialogues.
## 1.0.0-beta.2
23rd July, 2026
### Improved
- Choosing **Not now** on a merge conflict now postpones repeated dialogues for that conflict while the active file retains an unresolved-conflict warning. Three or more live versions show their current count and are reviewed one deterministic pair at a time; completed pairs remain resolved across restart. The existing conflict commands can reopen a postponed conflict explicitly, and a later conflict prompts again after the current one has been resolved.
### Fixed
- Answering or externally closing a merge dialogue immediately no longer leaves conflict processing waiting for a response which has already occurred.
### Testing
- Added revision-tree regressions and focused real-Obsidian scenarios for multiple-version review and restart between resolution stages.
## 1.0.0-beta.1
22nd July, 2026
### Important
- This corrected opt-in integration preview follows `1.0.0-beta.0` and does not replace the latest stable release. Update every participating device before resuming synchronisation, and continue to use a current backup while testing with an existing Vault.
### Fixed
- Conflict resolutions made on another device no longer recreate the same conflict when the receiving Vault still contains the exact content of the deleted losing revision. Automatic text and structured-data merge now uses the nearest revision actually shared by both branches instead of inferring ancestry from revision generation numbers.
- Edits, deletions, and renames made while a file is conflicted now extend the exact revision displayed on that device. If LiveSync cannot prove the displayed branch, it preserves the affected branches for review instead of silently applying the operation to the database winner.
### Testing
- Added revision-tree regressions and focused real-Obsidian scenarios for propagated resolutions and file operations performed while a conflict remains active.
## 1.0.0-beta.0
22nd July, 2026
### Important
- This is an opt-in 1.0 integration preview for BRAT and testing with existing Vaults. It does not replace the latest stable release. Use it with a current backup, and update every participating device before resuming synchronisation.
- An upgraded, copied, or restored Vault may pause replication for an explicit compatibility review. The review preserves the existing automatic synchronisation choices and resumes them only after the decision has been saved successfully.
### Improved
- An unconfigured installation now waits for you to start setup. A long-lived Notice offers the setup action, and **Open onboarding wizard** remains available from the command palette instead of the dialogue opening automatically.
- The setup wizard now creates named remote profiles for CouchDB, Object Storage, and P2P. Current Setup URIs preserve their profile names and selections, and the wizard reserves Rebuild or Fetch before the ordinary start-up scan begins.
- Peer-to-Peer Synchronisation (P2P) and Hidden File Sync are supported opt-in features. JWT authentication, ignore files, automatic newer-file conflict resolution, and Garbage Collection V3 remain previews. Customisation Sync remains a supported advanced workflow.
- Data Compression remains available after measurement showed a modest, workload-dependent reduction in stored and transferred chunk data. Its benefits, costs, and reason for remaining disabled by default in 1.0 are described in the Data Compression specification.
- Compatibility review now runs before Config Doctor without overlapping it. Existing Vaults retain their automatic synchronisation choices and explicit file-name case setting. For installations created by earlier releases, LiveSync preserves whether setup had been completed and saves a missing legacy case setting as case-insensitive.
- P2P connections now restart reliably after settings are reapplied or the local database is reset. Setup on an additional device asks you to select the source device once. Disconnecting leaves the LiveSync room and closes its signalling relay connections so that reconnecting can establish a new room.
- Action buttons are stacked vertically, long setup dialogues keep their controls reachable on mobile screens, and persistent Notices no longer cover close controls. Hidden File Sync reload and restart requests are grouped into one message, including the case reported in issue #555.
- Warnings about estimated remote storage size now appear as long-lived clickable Notices instead of timed dialogues. Initial uploads and Rebuild operations no longer prompt to send every chunk in advance; ordinary replication completes the transfer.
- Removed the obsolete **Use the trash bin** control and the setting for fixed chunk revisions. Remote deletion still follows Obsidian's preference, and chunk revisions remain content-derived. The Change Log remains available but no longer opens automatically or tracks unread versions.
### Fixed
- The optional Custom HTTP Handler used by Object Storage now sends the correct byte range from binary request bodies and reports unsupported body types instead of silently sending an empty request.
- When selectors, ignore files, size limits, modification-time limits, or file-name case settings are broadened, LiveSync now rechecks previously received files without requiring another remote update.
- P2P setup on the first device no longer displays reset or upload steps for a central database, and Config Doctor now offers its chunk size recommendation for CouchDB only when a CouchDB remote profile is selected.
### Security
- Fly.io setup now generates CouchDB and Vault encryption secrets with cryptographically secure randomness. Dependency updates prevent excessive CPU use from specially crafted path patterns and `mailto:` links. The CLI rejects path traversal and symbolic-link components detected before Vault operations.
### Miscellaneous
- Self-hosted LiveSync now owns its translation catalogue. Commonlib provides English messages to other applications, and translation contributions can be made directly to the Self-hosted LiveSync repository.
### Testing
- Expanded automated testing in Obsidian for upgrades, synchronisation between two devices, CouchDB, Object Storage, P2P, Hidden File Sync, mobile dialogues, and clean-up after failures.
+63 -25
View File
@@ -12,7 +12,7 @@ The following status applies to optional and compatibility features in the 1.0 l
| -------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| Supported, opt-in | Peer-to-Peer Synchronisation, Hidden File Sync, and Customisation Sync | Maintained and covered by focused real-runtime tests. Enable them only where their separate setup and operational constraints are acceptable. |
| Maintained, advanced | Data Compression | Available as an explicit storage and bandwidth trade-off. It remains disabled by default because the measured processing and memory costs outweigh the mixed-dataset saving. |
| Beta or experimental | JWT authentication, ignore files, automatic newer-file conflict resolution, and Garbage Collection V3 | Retained for explicit testing and specialised use. They remain disabled by default and are not part of the minimum supported setup. |
| Beta or experimental | JWT authentication, ignore files, automatic newer-file conflict resolution, and Garbage Collection V3 for CouchDB | Retained for explicit testing and specialised use. They remain disabled by default and are not part of the minimum supported setup. |
| Compatibility only | V1 dynamic iteration counts, the old IndexedDB adapter, non-current hash algorithms, Eden chunks, and the stored `doNotUseFixedRevisionForChunks` key | Existing settings and data remain readable. New Vaults use the current defaults, and compatibility controls are shown only where a migration or recovery path still needs them. |
| Icon | Description |
@@ -40,7 +40,7 @@ Internal database or settings compatibility reviews use a separate safety dialog
This pane is used for setting up Self-hosted LiveSync. There are several options to set up Self-hosted LiveSync.
An unconfigured installation does not open the onboarding dialogue automatically or scan the Vault into the local database. A long-lived Notice offers the onboarding action, and **Open onboarding wizard** remains available from the command palette after that Notice closes.
An unconfigured installation does not open the onboarding dialogue automatically or scan the Vault into the local database. A long-lived Notice offers the onboarding action. If the Notice is dismissed, open **Self-hosted LiveSync settings****Setup****Rerun Onboarding Wizard**.
Choose the new-device path when this device owns the files which should initialise synchronisation. Choose the existing-device path when it should receive an established remote state. The wizard reserves Rebuild or Fetch respectively before enabling the settings and requesting a restart, so the selected initialisation runs before the ordinary start-up scan.
@@ -169,9 +169,9 @@ Show verbose log. Please enable when you report the logs
## 3. Remote Configuration
### 1. Remote Server
### 1. Connection settings
Self-hosted LiveSync supports multiple remote connection profiles under **Remote Server** -> **Remote Databases**. This allows you to save and switch between multiple databases or bucket configurations in a single vault.
Self-hosted LiveSync stores multiple remote connection profiles under **Connection settings** **Saved connections**. Each profile represents a CouchDB database, an Object Storage connection, or a P2P configuration, and several profiles can be kept in one Vault.
Each profile has an opaque identifier and a presentation name. The name does not need to be unique and is not used to select the profile. The main remote and the P2P remote are selected independently, so code and settings imports must preserve both selections rather than relying on a special identifier such as `default`.
@@ -185,7 +185,7 @@ Each profile has an opaque identifier and a presentation name. The name does not
Setting key: remoteType
The active remote server type. This is automatically projected to the legacy configuration when you activate a connection profile.
The active connection type. This is automatically projected to the legacy configuration when you activate a connection profile.
### 2. Notification
@@ -301,6 +301,18 @@ Setting key: bucketCustomHeaders
Custom HTTP headers to include in every request sent to the Object Storage bucket. Specify them in the format `Header-Name: Value`, with each header on a new line.
#### Journal data format
Setting key: journalFormat
Existing Object Storage profiles use `opaque-v1` unless Adaptive Journal is selected explicitly. `adaptive-v1` uses an authenticated manifest and immutable Commit Bundles under a separate namespace, with larger Packs stored separately. Changing between formats requires an explicit remote Rebuild; LiveSync does not migrate or read both representations.
#### Pack retrieval
Setting key: packReadPolicy
Adaptive Journal can download a complete immutable Pack (`whole-pack`) or request only the required byte ranges (`range`). Complete Pack retrieval is the portable, throughput-oriented default. Selecting Range requires exact byte-range support from the configured S3-compatible endpoint. The connection test verifies that capability, and synchronisation refuses an unsupported selection before writing.
#### Test Connection
#### Apply Settings
@@ -314,7 +326,7 @@ These settings are configured within the CouchDB Setup dialogue when adding (`
Setting key: couchDB_URI
The URI of the CouchDB server.
Note: Only Secure (HTTPS) connections can be used on Obsidian Mobile. The URI must not end with a trailing slash.
Only secure HTTPS connections can be used on Obsidian Mobile. The setup dialogue accepts a complete HTTP or HTTPS URL and normalises it when the settings are applied.
#### Username
@@ -332,14 +344,13 @@ The password used to authenticate with CouchDB.
Setting key: couchDB_DBNAME
The name of the database.
Note: The database name cannot contain capital letters, spaces, or special characters other than `_$()+/-`, and cannot start with an underscore (`_`).
The name of the database. It must not be empty. CouchDB validates the name when the connection is attempted; the setup dialogue does not apply a narrower client-side naming rule.
#### Use Request API to avoid inevitable CORS problem
Setting key: useRequestAPI
This option is labeled **Use Internal API** in the setup dialogue. If enabled, Obsidian's internal request API will be used to bypass CORS restrictions. This is a workaround that may not be compliant with web standards and is less secure. Note that this might break in future Obsidian versions.
This option is labelled **Use Internal API** in the setup dialogue. If enabled, Obsidian's internal request API is used to bypass CORS restrictions. It sends the configured credentials to the CouchDB server through an Obsidian-owned API, so use it only with a server you trust. Configure CouchDB CORS correctly where possible; this compatibility workaround may change in future Obsidian versions.
#### Custom Headers
@@ -383,13 +394,20 @@ Setting key: jwtSub
The subject (`sub`) claim of the JWT, which should match your CouchDB username.
#### Test Database Connection
#### Connection and save actions
Open database connection. If the remote database is not found and you have permission to create a database, the database will be created.
The action depends on why the dialogue was opened:
#### Validate Database Configuration
- Onboarding for the first device uses **Create or connect to database and continue**. It may create the database when it does not exist and the supplied account has permission.
- Onboarding for an additional device uses **Connect to existing database and continue**. It does not create a missing database.
- Adding or editing a saved remote profile uses **Test connection and save**. It does not create a missing database.
- Settings mode also offers **Save without connecting**. The existing profile is updated, but automatic synchronisation may fail until the connection is corrected.
Checks and fixes any potential issues with the database config.
Onboarding requires a successful connection. It does not expose an unverified continuation action.
#### Check server requirements
This optional check reads the CouchDB server configuration through Obsidian's internal request API and sends the configured credentials to that server. Administrator access may be required. The initial check is read-only. Each offered fix names the exact CouchDB setting and proposed value, and requires separate confirmation before making that change.
#### Apply Settings
@@ -401,11 +419,11 @@ Setting key: P2P_Enabled
Enable direct peer-to-peer synchronisation via WebRTC.
#### Relay URL
#### Signalling relay URLs
Setting key: P2P_relays
The WebSocket relay server URL(s) used for coordinating P2P connections via WebRTC. Multiple URLs can be separated by commas.
The Nostr-compatible WebSocket relay URL or URLs used for peer discovery and WebRTC connection negotiation. Multiple URLs can be separated by commas. A signalling relay does not store or transfer Vault contents. See [How peer-to-peer synchronisation works](p2p.md).
#### Group ID
@@ -435,17 +453,17 @@ This option is labeled **Auto Start P2P Connection** in the setup dialogue. If e
Closing a P2P connection leaves the LiveSync P2P room, stops its replication service, closes the signalling relay sockets, and pauses their automatic reconnection. An idle WebRTC connection may remain temporarily under the transport's ownership so that it can be reused, but it cannot carry traffic for the room which has been left. Connecting again resumes relay reconnection and joins a new LiveSync room.
#### Automatically broadcast changes to connected peers
#### Announce changes automatically after connecting
Setting key: P2P_AutoBroadcast
This option is labeled **Auto Broadcast Changes** in the setup dialogue. If enabled, changes will be automatically broadcasted to connected peers, requesting them to fetch the changes.
When enabled, this device notifies connected peers after a local change. The notification contains no Vault data. A receiving peer fetches the change only when it follows this device.
#### TURN Server URLs (comma-separated)
Setting key: P2P_turnServers
A comma-separated list of TURN/STUN server URLs. Used to relay P2P connections when direct WebRTC connection fails due to strict NAT or firewalls. In most cases, these can be left blank.
A comma-separated list of TURN server URLs. TURN is an optional fallback which relays encrypted WebRTC traffic when strict NAT or firewall rules prevent a direct peer connection. It is distinct from the required signalling relay. In most environments, this field can remain blank.
#### TURN Username
@@ -692,6 +710,12 @@ Open the dialogue
#### Make report to inform the issue
#### Copy database information for a file
Select a file to copy its local database information. The command **Copy database information for the active file** performs the same inspection for the file open in the editor.
The report includes the Vault-relative path, document and chunk identifiers, local database revisions, conflicts, and local chunk availability. It does not query the remote or include file contents. Paths and identifiers can still be private metadata, so review the report before sharing it.
#### Write logs into the file
Setting key: writeLogToTheFile
@@ -715,17 +739,19 @@ Stop reflecting database changes to storage files.
### 3. Recovery and Repair
#### Recreate missing chunks for all files
#### Recreate chunks for current Vault files
This will recreate chunks for all files. If there were missing chunks, this may fix the errors.
Recreate chunks from files currently present in the Vault. This can repair missing chunks for those exact current contents after they have been confirmed as authoritative. It cannot reconstruct unavailable historical or conflict content.
#### Inspect conflicts and file/database differences
Compare each Vault file with every current live revision in the local database. Each winner and conflict revision is shown separately with its exact revision identifier, local chunk availability, and relationship to the current Vault file. Unavailable shared ancestors are reported separately because they prevent conservative three-way merging but are not live revisions which can be discarded.
Select **Begin inspection** to run the inspection. Each reported file and live revision has a wrench menu for read-only comparison, applying an exact database revision to the Vault, recording an exact byte match, preserving the Vault file as a child of a selected branch, retrying chunk retrieval, or explicitly discarding a branch. Destructive actions require confirmation. Follow [Recover a conflicted or mismatched file](recovery.md#recover-a-conflicted-or-mismatched-file) before changing revision history.
#### Resolve All conflicted files by the newer one
Resolve all conflicted files by the newer one. Caution: This will overwrite the older one, and cannot resurrect the overwritten one.
#### Verify and repair all files
Compare the content of files between on local database and storage. If not matched, you will be asked which one you want to keep.
After confirmation, resolve every conflict by modification time. This logically deletes every version except the newest one. It is a destructive policy choice and cannot recover content which is already unavailable.
#### Check and convert non-path-obfuscated files
@@ -953,6 +979,12 @@ If disabled(toggled), chunks will be split on the UI thread (Previous behaviour)
Setting key: processSmallFilesInUIThread
If enabled, the file under 1kb will be processed in the UI thread.
#### Automatically align compatible chunk settings
Setting key: autoAcceptCompatibleTweak
Current releases enable this by default when the differences are limited to compatible chunk settings. The side with the newer recorded modification time is used for the chunk hash algorithm, chunk size, or splitter version; the remote value is used when neither side has a recorded time or the times are equal. No dialogue or database reconstruction is required. Existing content remains readable, but changing these values can reduce chunk reuse. Turn this off to review compatible differences manually. Any difference which also involves an incompatible setting always requires an explicit decision.
### 8. Compatibility (Trouble addressed)
#### Do not check configuration mismatch before replication
@@ -1027,6 +1059,12 @@ Purge all download/upload cache.
Delete all data on the remote server.
### 6. Garbage Collection V3 (CouchDB only)
Garbage Collection V3 identifies chunk documents which are not reachable from any current file or live conflict branch, creates logical deletions for those chunks locally, propagates the deletions to CouchDB, and requests remote compaction.
Use it only when the Vault, local database, and remote are healthy, and every relevant device has synchronised. It can make an ordinary superseded file revision unreadable when no live state still needs its chunks. It does not repair corruption or replace a deliberate rebuild. See the [Garbage Collection V3 specification](specs_garbage_collection.md).
### 7. Reset
#### Delete local database to reset or uninstall Self-hosted LiveSync
+8 -8
View File
@@ -1,6 +1,6 @@
# Set up Object Storage
This guide establishes Object Storage synchronisation on a first device, generates an additional-device Setup URI from that working device, and verifies synchronisation in both directions.
This guide establishes Object Storage synchronisation on a first device, generates a Setup URI for another device from that working device, and verifies synchronisation in both directions.
Object Storage uses the S3-compatible API. Prepare the following before starting:
@@ -12,7 +12,7 @@ Object Storage uses the S3-compatible API. Prepare the following before starting
Back up every Vault involved, and do not use Obsidian Sync, iCloud synchronisation, or another synchronisation service on the same Vault.
## Generate the bootstrap Setup URI
## Generate the initial Setup URI
The public generator applies the Object Storage preset and records the connection as the selected remote profile. Run it from a trusted terminal:
@@ -40,13 +40,13 @@ Use a new bucket prefix, or a prefix whose contents you deliberately intend to r
1. Install and enable Self-hosted LiveSync in the intended Vault.
2. Open onboarding from the `Welcome to Self-hosted LiveSync` Notice.
3. Select `I am setting this up for the first time`, then choose the recommended Setup URI method.
4. Paste the bootstrap Setup URI, enter its passphrase, and select `Test Settings and Continue`.
4. Paste the initial Setup URI, enter its passphrase, and select `Test Settings and Continue`.
![Object Storage Setup URI on the first device](../images/object-storage-setup/guide-object-storage-setup-first-setup-uri.png)
5. Select `Restart and Initialise Server`, then read and accept the final overwrite confirmation only when this Vault is the intended source of truth.
![Object Storage first-device initialisation](../images/object-storage-setup/guide-object-storage-setup-first-initialise.png)
![Object Storage initialisation on the first device](../images/object-storage-setup/guide-object-storage-setup-first-initialise.png)
![Final Object Storage overwrite confirmation](../images/object-storage-setup/guide-object-storage-setup-first-rebuild-confirmation.png)
@@ -64,7 +64,7 @@ Generate a fresh Setup URI from the working first device:
1. Run `Self-hosted LiveSync: Copy settings as a new Setup URI` from the command palette.
2. Enter a new Setup URI passphrase.
![Masked passphrase for the additional-device Object Storage Setup URI](../images/object-storage-setup/guide-object-storage-setup-copy-setup-uri-passphrase.png)
![Masked passphrase for the Object Storage Setup URI for another device](../images/object-storage-setup/guide-object-storage-setup-copy-setup-uri-passphrase.png)
3. Copy the resulting URI.
@@ -80,7 +80,7 @@ Start with a new or separately backed-up Vault.
2. Open onboarding, select `I am adding a device to an existing synchronisation setup`, and choose the recommended Setup URI method.
3. Enter the URI generated by the first device and its passphrase.
![First-device Object Storage Setup URI entered on the second device](../images/object-storage-setup/guide-object-storage-setup-second-setup-uri.png)
![Object Storage Setup URI from the first device entered on the second device](../images/object-storage-setup/guide-object-storage-setup-second-setup-uri.png)
4. Select `Restart and Fetch Data`.
@@ -96,7 +96,7 @@ Start with a new or separately backed-up Vault.
Confirm that the first device's test note appears unchanged. Create a second ordinary note on the new device, wait for its journal synchronisation to finish, and confirm that it reaches the first device. Configure optional features only after this two-way check passes.
![First-device note received by the second Object Storage device](../images/object-storage-setup/guide-object-storage-setup-first-to-second.png)
![Note from the first device received by the second Object Storage device](../images/object-storage-setup/guide-object-storage-setup-first-to-second.png)
![Second-device note received by the first Object Storage device](../images/object-storage-setup/guide-object-storage-setup-second-to-first.png)
@@ -104,5 +104,5 @@ Confirm that the first device's test note appears unchanged. Create a second ord
- Treat the endpoint, bucket, prefix, access key, secret key, Vault passphrase, Setup URI, and Setup URI passphrase as sensitive.
- Use a distinct prefix per synchronisation set unless shared data is explicitly intended.
- Do not select first-device initialisation against an existing prefix unless replacing its contents is deliberate.
- Do not initialise the first device against an existing prefix unless replacing its contents is deliberate.
- Object Storage is not a Vault backup. Keep independent backups and test restoration separately.
+3 -3
View File
@@ -167,7 +167,7 @@ Now `https://tiles-photograph-routine-groundwater.trycloudflare.com` is our serv
## 4. Client Setup
> [!TIP]
> Now manual configuration is not recommended for some reasons. However, if you want to do so, please use `Setup wizard`. The recommended extra configurations will be also set.
> A generated Setup URI is the recommended path because it carries the current defaults for a new Vault and the selected remote profile. If a Setup URI cannot be generated, follow [Configure CouchDB manually on the first device](./quick_setup.md#configure-couchdb-manually-on-the-first-device), then generate a new Setup URI from that working device for every additional device.
### 1. Generate the setup URI on a desktop device or server
```bash
@@ -185,7 +185,7 @@ deno run --minimum-dependency-age=0 --allow-env https://raw.githubusercontent.co
>
> If `uri_passphrase` is omitted, the generator creates a cryptographically random value and prints it once.
The generator consumes the exact registry-pinned Commonlib release used by the provisioning utility. It creates a configured CouchDB remote profile, applies the current new-Vault defaults, and encodes them with Commonlib's Setup URI contract.
The generator consumes the exact registry-pinned Commonlib release used by the provisioning utility. It creates a configured CouchDB remote profile, applies the current defaults for a new Vault, and encodes them with Commonlib's Setup URI contract.
You will then get the following output:
@@ -202,7 +202,7 @@ Store the Setup URI and its passphrase separately.
Follow [Quick setup](./quick_setup.md#set-up-the-first-device) for the first device. It covers the current onboarding Notice, Setup URI import, server initialisation, and the safety prompts shown for a newly provisioned database.
After ordinary note synchronisation works, [generate a new Setup URI on that first device](./quick_setup.md#create-a-setup-uri-for-another-device), then follow [Add another device](./quick_setup.md#add-another-device). Do not make the second device depend on retaining the provisioning-time bootstrap URI. Configure optional features only after the normal path is verified; [Hidden File Sync has its own guide](./tips/hidden-file-sync.md).
After ordinary note synchronisation works, [generate a new Setup URI on that first device](./quick_setup.md#create-a-setup-uri-for-another-device), then follow [Add another device](./quick_setup.md#add-another-device). Do not make the second device depend on retaining the initial Setup URI produced during provisioning. Configure optional features only after the normal path is verified; [Hidden File Sync has its own guide](./tips/hidden-file-sync.md).
---
+152 -152
View File
@@ -1,152 +1,152 @@
# 在你自己的服务器上设置 CouchDB
## 目录
- [配置 CouchDB](#配置-CouchDB)
- [运行 CouchDB](#运行-CouchDB)
- [Docker CLI](#docker-cli)
- [Docker Compose](#docker-compose)
- [创建数据库](#创建数据库)
- [从移动设备访问](#从移动设备访问)
- [移动设备测试](#移动设备测试)
- [设置你的域名](#设置你的域名)
---
> 注:提供了 [docker-compose.yml 和 ini 文件](https://github.com/vrtmrz/self-hosted-livesync-server) 可以同时启动 Caddy 和 CouchDB。推荐直接使用该 docker-compose 配置进行搭建。(若使用,请查阅链接中的文档,而不是这个文档)
## 配置 CouchDB
设置 CouchDB 的最简单方法是使用 [CouchDB docker image]((https://hub.docker.com/_/couchdb)).
需要修改一些 `local.ini` 中的配置,以让它可以用于 Self-hosted LiveSync,如下:
```
[couchdb]
single_node=true
max_document_size = 50000000
[chttpd]
require_valid_user = true
max_http_request_size = 4294967296
[chttpd_auth]
require_valid_user = true
authentication_redirect = /_utils/session.html
[httpd]
WWW-Authenticate = Basic realm="couchdb"
enable_cors = true
[cors]
origins = app://obsidian.md,capacitor://localhost,http://localhost
credentials = true
headers = accept, authorization, content-type, origin, referer
methods = GET, PUT, POST, HEAD, DELETE
max_age = 3600
```
## 运行 CouchDB
### Docker CLI
你可以通过指定 `local.ini` 配置运行 CouchDB:
```
$ docker run --rm -it -e COUCHDB_USER=admin -e COUCHDB_PASSWORD=password -v /path/to/local.ini:/opt/couchdb/etc/local.ini -p 5984:5984 couchdb
```
*记得将上述命令中的 local.ini 挂载路径替换成实际的存放路径*
后台运行:
```
$ docker run -d --restart always -e COUCHDB_USER=admin -e COUCHDB_PASSWORD=password -v /path/to/local.ini:/opt/couchdb/etc/local.ini -p 5984:5984 couchdb
```
*记得将上述命令中的 local.ini 挂载路径替换成实际的存放路径*
### Docker Compose
创建一个文件夹, 将你的 `local.ini` 放在文件夹内, 然后在文件夹内创建 `docker-compose.yml`. 请确保对 `local.ini` 有读写权限并且确保在容器运行后能创建 `data` 文件夹. 文件夹结构大概如下:
```
obsidian-livesync
├── docker-compose.yml
└── local.ini
```
可以参照以下内容编辑 `docker-compose.yml`:
```yaml
services:
couchdb:
image: couchdb
container_name: obsidian-livesync
user: 1000:1000
environment:
- COUCHDB_USER=admin
- COUCHDB_PASSWORD=password
volumes:
- ./data:/opt/couchdb/data
- ./local.ini:/opt/couchdb/etc/local.ini
ports:
- 5984:5984
restart: unless-stopped
```
最后, 创建并启动容器:
```
# -d will launch detached so the container runs in background
docker-compose up -d
```
## 创建数据库
CouchDB 部署成功后, 需要手动创建一个数据库, 方便插件连接并同步.
1. 访问 `http://localhost:5984/_utils`, 输入帐号密码后进入管理页面
2. 点击 Create Database, 然后根据个人喜好创建数据库
## 从移动设备访问
如果你想要从移动设备访问 Self-hosted LiveSync,你需要一个合法的 SSL 证书。
### 移动设备测试
测试时,[localhost.run](http://localhost.run/) 这一类的反向隧道服务很实用。(非必须,只是用于终端设备不方便 ssh 的时候的备选方案)
```
$ ssh -R 80:localhost:5984 nokey@localhost.run
Warning: Permanently added the RSA host key for IP address '35.171.254.69' to the list of known hosts.
===============================================================================
Welcome to localhost.run!
Follow your favourite reverse tunnel at [https://twitter.com/localhost_run].
**You need a SSH key to access this service.**
If you get a permission denied follow Gitlab's most excellent howto:
https://docs.gitlab.com/ee/ssh/
*Only rsa and ed25519 keys are supported*
To set up and manage custom domains go to https://admin.localhost.run/
More details on custom domains (and how to enable subdomains of your custom
domain) at https://localhost.run/docs/custom-domains
To explore using localhost.run visit the documentation site:
https://localhost.run/docs/
===============================================================================
** your connection id is xxxxxxxxxxxxxxxxxxxxxxxxxxxx, please mention it if you send me a message about an issue. **
xxxxxxxx.localhost.run tunneled with tls termination, https://xxxxxxxx.localhost.run
Connection to localhost.run closed by remote host.
Connection to localhost.run closed.
```
https://xxxxxxxx.localhost.run 即为临时服务器地址。
### 设置你的域名
设置一个指向你服务器的 A 记录,并根据需要设置反向代理。
Note: 不推荐将 CouchDB 挂载到根目录
可以使用 Caddy 很方便的给服务器加上 SSL 功能
提供了 [docker-compose.yml 和 ini 文件](https://github.com/vrtmrz/self-hosted-livesync-server) 可以同时启动 Caddy 和 CouchDB。
注意检查服务器日志,当心恶意访问。
# 在你自己的服务器上设置 CouchDB
## 目录
- [配置 CouchDB](#配置-CouchDB)
- [运行 CouchDB](#运行-CouchDB)
- [Docker CLI](#docker-cli)
- [Docker Compose](#docker-compose)
- [创建数据库](#创建数据库)
- [从移动设备访问](#从移动设备访问)
- [移动设备测试](#移动设备测试)
- [设置你的域名](#设置你的域名)
---
> 注:提供了 [docker-compose.yml 和 ini 文件](https://github.com/vrtmrz/self-hosted-livesync-server) 可以同时启动 Caddy 和 CouchDB。推荐直接使用该 docker-compose 配置进行搭建。(若使用,请查阅链接中的文档,而不是这个文档)
## 配置 CouchDB
设置 CouchDB 的最简单方法是使用 [CouchDB docker image]((https://hub.docker.com/_/couchdb)).
需要修改一些 `local.ini` 中的配置,以让它可以用于 Self-hosted LiveSync,如下:
```
[couchdb]
single_node=true
max_document_size = 50000000
[chttpd]
require_valid_user = true
max_http_request_size = 4294967296
[chttpd_auth]
require_valid_user = true
authentication_redirect = /_utils/session.html
[httpd]
WWW-Authenticate = Basic realm="couchdb"
enable_cors = true
[cors]
origins = app://obsidian.md,capacitor://localhost,http://localhost
credentials = true
headers = accept, authorization, content-type, origin, referer
methods = GET, PUT, POST, HEAD, DELETE
max_age = 3600
```
## 运行 CouchDB
### Docker CLI
你可以通过指定 `local.ini` 配置运行 CouchDB:
```
$ docker run --rm -it -e COUCHDB_USER=admin -e COUCHDB_PASSWORD=password -v /path/to/local.ini:/opt/couchdb/etc/local.ini -p 5984:5984 couchdb
```
*记得将上述命令中的 local.ini 挂载路径替换成实际的存放路径*
后台运行:
```
$ docker run -d --restart always -e COUCHDB_USER=admin -e COUCHDB_PASSWORD=password -v /path/to/local.ini:/opt/couchdb/etc/local.ini -p 5984:5984 couchdb
```
*记得将上述命令中的 local.ini 挂载路径替换成实际的存放路径*
### Docker Compose
创建一个文件夹, 将你的 `local.ini` 放在文件夹内, 然后在文件夹内创建 `docker-compose.yml`. 请确保对 `local.ini` 有读写权限并且确保在容器运行后能创建 `data` 文件夹. 文件夹结构大概如下:
```
obsidian-livesync
├── docker-compose.yml
└── local.ini
```
可以参照以下内容编辑 `docker-compose.yml`:
```yaml
services:
couchdb:
image: couchdb
container_name: obsidian-livesync
user: 1000:1000
environment:
- COUCHDB_USER=admin
- COUCHDB_PASSWORD=password
volumes:
- ./data:/opt/couchdb/data
- ./local.ini:/opt/couchdb/etc/local.ini
ports:
- 5984:5984
restart: unless-stopped
```
最后, 创建并启动容器:
```
# -d will launch detached so the container runs in background
docker-compose up -d
```
## 创建数据库
CouchDB 部署成功后, 需要手动创建一个数据库, 方便插件连接并同步.
1. 访问 `http://localhost:5984/_utils`, 输入帐号密码后进入管理页面
2. 点击 Create Database, 然后根据个人喜好创建数据库
## 从移动设备访问
如果你想要从移动设备访问 Self-hosted LiveSync,你需要一个合法的 SSL 证书。
### 移动设备测试
测试时,[localhost.run](http://localhost.run/) 这一类的反向隧道服务很实用。(非必须,只是用于终端设备不方便 ssh 的时候的备选方案)
```
$ ssh -R 80:localhost:5984 nokey@localhost.run
Warning: Permanently added the RSA host key for IP address '35.171.254.69' to the list of known hosts.
===============================================================================
Welcome to localhost.run!
Follow your favourite reverse tunnel at [https://twitter.com/localhost_run].
**You need a SSH key to access this service.**
If you get a permission denied follow Gitlab's most excellent howto:
https://docs.gitlab.com/ee/ssh/
*Only rsa and ed25519 keys are supported*
To set up and manage custom domains go to https://admin.localhost.run/
More details on custom domains (and how to enable subdomains of your custom
domain) at https://localhost.run/docs/custom-domains
To explore using localhost.run visit the documentation site:
https://localhost.run/docs/
===============================================================================
** your connection id is xxxxxxxxxxxxxxxxxxxxxxxxxxxx, please mention it if you send me a message about an issue. **
xxxxxxxx.localhost.run tunneled with tls termination, https://xxxxxxxx.localhost.run
Connection to localhost.run closed by remote host.
Connection to localhost.run closed.
```
https://xxxxxxxx.localhost.run 即为临时服务器地址。
### 设置你的域名
设置一个指向你服务器的 A 记录,并根据需要设置反向代理。
Note: 不推荐将 CouchDB 挂载到根目录
可以使用 Caddy 很方便的给服务器加上 SSL 功能
提供了 [docker-compose.yml 和 ini 文件](https://github.com/vrtmrz/self-hosted-livesync-server) 可以同时启动 Caddy 和 CouchDB。
注意检查服务器日志,当心恶意访问。
+55 -38
View File
@@ -1,54 +1,44 @@
# Set up peer-to-peer synchronisation
This guide establishes a peer-to-peer synchronisation set, generates the second-device Setup URI on the working first device, and verifies synchronisation in both directions with explicit peer approval.
This guide configures a working first device through the ordinary user interface, generates a Setup URI for an additional device, and verifies synchronisation in both directions with explicit peer approval.
Peer-to-peer synchronisation has no central copy of the Vault. At least one device containing the required data must be online when another device fetches it. A Nostr-compatible signalling relay helps devices discover each other, while the Vault data travels through the peer connection.
Peer-to-peer synchronisation has no central data-storage server containing a copy of the Vault. A signalling relay is still required for peer discovery. The project's public signalling relay avoids the need to provision one for an ordinary setup; a controlled setup can use another Nostr-compatible relay. Vault data travels through the encrypted peer connection, not through the signalling relay.
See [How peer-to-peer synchronisation works](p2p.md) for the communication model, the public relay policy, and the distinction between signalling and TURN.
Before starting:
- back up both Vaults;
- prepare a signalling relay reachable by both devices;
- decide whether to use the project's public signalling relay or another relay reachable by both devices;
- ensure the networks permit a WebRTC connection, or review the [P2P troubleshooting guidance](./tips/p2p-sync-tips.md);
- disable every other synchronisation service for these Vaults; and
- keep both devices awake and Obsidian open during the initial transfer.
## Generate the bootstrap Setup URI
Run the public generator from a trusted terminal. Supply your own relay for a controlled self-hosted setup:
```sh
export remote_type=p2p
export p2p_relays=wss://relay.example.com
export p2p_room_id=<A PRIVATE ROOM ID> # Optional; generated when omitted
export p2p_passphrase=<A PRIVATE P2P PASSPHRASE> # Optional; generated when omitted
export passphrase=<A STRONG VAULT ENCRYPTION PASSPHRASE>
export uri_passphrase=<A SEPARATE SETUP URI PASSPHRASE>
deno run --minimum-dependency-age=0 --allow-env https://raw.githubusercontent.com/vrtmrz/obsidian-livesync/main/utils/setup/generate_setup_uri.ts
```
The generated Setup URI contains the encrypted room, relay, and Vault settings. It deliberately omits the device-specific peer name. Store the URI and its passphrase separately.
## Set up the first device
1. Install and enable Self-hosted LiveSync in the intended Vault.
2. Open onboarding from the `Welcome to Self-hosted LiveSync` Notice.
3. Select `I am setting this up for the first time`, then choose the recommended Setup URI method.
4. Paste the bootstrap Setup URI, enter its passphrase, and select `Test Settings and Continue`.
![P2P Setup URI on the first device](../images/p2p-setup/guide-p2p-setup-first-setup-uri.png)
5. Complete the first-device initialisation and final confirmation. This initialises the local LiveSync database; P2P has no central remote database to erase.
3. Select `I am setting this up for the first time`, choose manual configuration, then select `Peer-to-Peer only`.
4. In `P2P Configuration`:
- enable P2P;
- select `Use the project's public signalling relay`, or enter your own signalling relay URLs;
- generate or enter a private Group ID;
- enter a strong P2P passphrase;
- enter a unique name for this device; and
- leave automatic start and automatic announcements disabled until the manual round trip succeeds.
5. Select `Test Settings and Continue`. The test joins the signalling relay; it does not require another peer to be online.
6. Complete the initialisation and final confirmation on the first device. This initialises the local LiveSync database; P2P has no central remote database to erase.
![P2P local initialisation on the first device](../images/p2p-setup/guide-p2p-setup-first-initialise.png)
![P2P local database confirmation on the first device](../images/p2p-setup/guide-p2p-setup-first-rebuild-confirmation.png)
6. Keep optional features disabled until ordinary note synchronisation works.
7. Open `Self-hosted LiveSync: Open P2P Replicator` from the command palette. Select `Open connection` if signalling is disconnected.
7. Keep optional features disabled until ordinary note synchronisation works.
8. After saving the P2P profile, open `Self-hosted LiveSync: P2P Sync : Open P2P Status` from the command palette. The P2P ribbon icon provides the same destination. Select `Open connection` if signalling is disconnected.
![First P2P device connected to the signalling relay](../images/p2p-setup/guide-p2p-setup-first-device-connected.png)
8. Create an ordinary test note and wait for the local LiveSync progress indicators to clear.
9. Create an ordinary test note and wait for the local LiveSync progress indicators to clear.
## Generate the second-device Setup URI
@@ -57,7 +47,7 @@ On the working first device:
1. Run `Self-hosted LiveSync: Copy settings as a new Setup URI` from the command palette.
2. Enter a new Setup URI passphrase.
![Masked passphrase for the additional-device P2P Setup URI](../images/p2p-setup/guide-p2p-setup-copy-setup-uri-passphrase.png)
![Masked passphrase for the P2P Setup URI for another device](../images/p2p-setup/guide-p2p-setup-copy-setup-uri-passphrase.png)
3. Copy the resulting URI.
@@ -71,7 +61,7 @@ Keep the first device online. Store the new URI and its passphrase separately.
2. Open onboarding, select `I am adding a device to an existing synchronisation setup`, and choose the recommended Setup URI method.
3. Enter the Setup URI generated by the first device and its passphrase.
![First-device P2P Setup URI entered on the second device](../images/p2p-setup/guide-p2p-setup-second-setup-uri.png)
![P2P Setup URI from the first device entered on the second device](../images/p2p-setup/guide-p2p-setup-second-setup-uri.png)
4. Select `Restart and Fetch Data`.
@@ -83,7 +73,7 @@ Keep the first device online. Store the new URI and its passphrase separately.
![P2P local-file policy](../images/p2p-setup/guide-p2p-setup-local-file-policy.png)
6. In `P2P Rebuild`, confirm that the expected first-device name is shown, then select `Sync`.
6. In `P2P Rebuild`, confirm that the expected name of the first device is shown, then select `Sync`.
![Selecting the first device for P2P Rebuild](../images/p2p-setup/guide-p2p-setup-select-first-device.png)
@@ -93,16 +83,16 @@ Keep the first device online. Store the new URI and its passphrase separately.
8. Keep both devices open until the test note appears on the second device.
![First-device note received by the second P2P device](../images/p2p-setup/guide-p2p-setup-first-to-second.png)
![Note from the first device received by the second P2P device](../images/p2p-setup/guide-p2p-setup-first-to-second.png)
## Verify the return journey
Create a second ordinary note on the second device. With automatic broadcast disabled, start the next finite synchronisation explicitly:
Create a second ordinary note on the second device. Keep automatic announcements disabled, then run and verify the next synchronisation explicitly:
1. Open the P2P Replicator pane on both devices.
2. If the previous finite peer connection no longer appears, select `Disconnect` and then `Open connection` on the first device, followed by the second device. The device which joins last is advertised to devices already in the room.
1. Open `P2P Status` on both devices.
2. If a peer no longer appears, select `Disconnect` and then `Open connection` on the first device, followed by the second device. The device which joins last is advertised to devices which are already in the room.
3. On the first device, select `Refresh`, verify the second-device name, then select `Replicate now`.
4. On the second device, verify the requesting first-device name and select `Accept` or `Accept Temporarily`.
4. On the second device, verify the name of the requesting first device and select `Accept` or `Accept Temporarily`.
![Explicit return-journey connection request on the second device](../images/p2p-setup/guide-p2p-setup-connection-request-2.png)
@@ -110,7 +100,15 @@ Create a second ordinary note on the second device. With automatic broadcast dis
![Second-device note received by the first P2P device](../images/p2p-setup/guide-p2p-setup-second-to-first.png)
The two devices are now proven to share the same room, encryption settings, and data format in both directions. Configure automatic start, automatic broadcast, or optional features separately after this manual path works.
The two devices are now proven to share the same room, encryption settings, and data format in both directions.
After this manual path works, configure automatic behaviour deliberately:
- `Announce changes` on a source device dispatches change notifications while it is connected.
- `Follow changes` on the receiving device fetches after notifications from that peer.
- The peer's `More actions` menu can synchronise or follow whenever that named device connects, or include it in the P2P synchronisation command.
An announcement contains no Vault data and does not transfer a change by itself. The source must announce, the receiver must follow, and both devices must be connected.
## If a peer does not appear
@@ -118,4 +116,23 @@ The two devices are now proven to share the same room, encryption settings, and
- Select `Refresh` after the other device joins.
- Reconnect the device which should be discovered last.
- Check that the Setup URI came from the working first device and that neither device copied a peer name manually.
- Check relay reachability, WebRTC restrictions, VPNs, and TURN considerations in [Peer-to-Peer Synchronisation Tips](./tips/p2p-sync-tips.md).
- Check signalling relay reachability separately from WebRTC connectivity.
- Review VPN and TURN options in [Peer-to-Peer Synchronisation Tips](./tips/p2p-sync-tips.md).
## Controlled or self-hosted setup
The ordinary route above starts in the plug-in UI and can use the project's public signalling relay. For a controlled deployment, prepare your own Nostr-compatible relay and enter it in `Signalling relay URLs` on every device.
The public Setup URI generator is also available when configuration must be created outside Obsidian. Run it from a trusted terminal:
```sh
export remote_type=p2p
export p2p_relays=wss://relay.example.com
export p2p_room_id=<A PRIVATE ROOM ID> # Optional; generated when omitted
export p2p_passphrase=<A PRIVATE P2P PASSPHRASE> # Optional; generated when omitted
export passphrase=<A STRONG VAULT ENCRYPTION PASSPHRASE>
export uri_passphrase=<A SEPARATE SETUP URI PASSPHRASE>
deno run --minimum-dependency-age=0 --allow-env https://raw.githubusercontent.com/vrtmrz/obsidian-livesync/main/utils/setup/generate_setup_uri.ts
```
The generated Setup URI contains the encrypted room, relay, and Vault settings. It deliberately omits the device-specific name. Store the URI and its passphrase separately. After importing it on the first device, continue from the initialisation step above, then generate a fresh Setup URI for an additional device from that working device.
+33 -2
View File
@@ -46,6 +46,37 @@ The all-branch history check prevents a resolved conflict from being recreated m
The compatibility implementation currently selects the newer modification time for differing binary conflicts even when the general **Always overwrite with a newer file** option is disabled. This is existing behaviour, not a new 1.0 guarantee. Changing it to explicit selection only is a separate compatibility decision.
## Unreadable revisions and repair
A document revision can remain in the PouchDB tree while one or more chunks needed to reconstruct its content are unavailable. Missing content is not evidence that the revision is obsolete. LiveSync therefore leaves an unreadable winner or conflict revision in the tree instead of deleting it during automatic conflict processing.
**Hatch****Inspect conflicts and file/database differences** inspects the current winner, every current conflict revision, and the nearest shared ancestor for each conflict. A logical-deletion winner and an absent Vault file already agree, so that state is not reported unless another live branch still requires attention. When the Vault already matches the winner but conflict branches remain, the card shows the compact status `✅ Vault matches winner · ⚠️ Conflicts: N`; matching the winner does not mean that the conflict has been resolved.
Each reported live revision has a compact wrench menu. The available actions depend on the exact revision and current Vault state:
- **Compare with Vault** opens the existing difference dialogue in read-only mode for differing text files.
- **Apply this revision to Vault** writes the selected readable revision, even when it is not the database winner. Replacing an existing file requires confirmation.
- **Mark this revision as the Vault version** is offered when the bytes already match. It records exact device-local provenance without creating a child revision, and refuses the operation if the file changed after inspection.
- **Store Vault file as a child of this revision** preserves the current Vault bytes on the explicitly selected live branch.
- **Apply logical deletion to Vault** removes an existing Vault file after confirmation. An absent file needs no retained deletion provenance.
- **Retry reading revision** attempts the configured chunk-retrieval path again. It does not change the revision tree.
- **Discard this branch** is available for each exact live revision while at least one other live branch remains. It requires confirmation and creates a logical deletion on only the selected branch without changing the current Vault file.
- **Discard unreadable revision** remains available as a recovery action when an unreadable revision is the only live leaf. It requires confirmation because no other database branch remains.
Every mutating action rechecks that the selected revision is still a current live leaf. If another operation resolved or replaced it, the action fails and the card is refreshed instead of extending an obsolete branch.
The card uses compact, mobile-friendly diagnostic rows with an emoji and a text label. `🧩 Missing chunks: N` identifies an unreadable revision. In the database row, `Δsize` is decoded size minus recorded size; in the Vault row, `Δsize vs DB` is Vault size minus decoded database size. `Δtime` is Vault modification time minus database modification time. The ordinary two-second comparison window still labels which side is newer. These values help diagnose a mismatch; path, size, and modification time do not prove revision identity or decide which content should win.
A shared ancestor is informational. An ancestor which is no longer a live revision cannot be discarded independently through this workflow. If its body is unavailable, conservative three-way merge remains disabled, although readable live revisions can still be selected manually.
Logical deletion does not recreate missing bytes, purge the document history, or prove that the deleted version was unimportant. Another replica or backup may still contain the missing chunks. Recover from that source before discarding a revision whenever possible.
**Recreate chunks for current Vault files** can recreate chunks only from files which are readable in the current Vault. It cannot reconstruct unique bytes from an unavailable historical or conflict revision.
Garbage Collection V3 treats every live conflict revision and its nearest available shared ancestor as reachable. Their locally available chunks are retained until the conflict is resolved. After resolution, chunks used only by the discarded branch or no-longer-needed merge ancestry can become eligible for collection. See the [Garbage Collection V3 specification](specs_garbage_collection.md).
A generation-one revision has no parent. When its body is unavailable, LiveSync cannot preserve a changed Vault file as a sibling branch without inventing ancestry. It leaves the operation unresolved. Recover the missing chunks from another replica or backup, or explicitly discard that live revision. If the current Vault file is the intended replacement, it can be stored after the unreadable revision has been logically deleted.
### Two devices independently create the same path
If two devices create the same full synchronised path before either device has
@@ -220,8 +251,8 @@ Do not:
## Verification
Commonlib's real-PouchDB and injected-boundary unit tests cover unequal branch lengths, exact shared ancestry, deterministic ordering of multiple live leaves, a sensible stage followed by reconstruction of a manual pair, content below a deleted losing leaf, recorded and reconstructed branch identity, ambiguous matches, conflict-time editing, logical deletion, case-only rename, cross-path rename, and safe unproven fallbacks.
Commonlib's real-PouchDB and injected-boundary unit tests cover unequal branch lengths, exact shared ancestry, deterministic ordering of multiple live leaves, a sensible stage followed by reconstruction of a manual pair, content below a deleted losing leaf, recorded and reconstructed branch identity, ambiguous matches, conflict-time editing, missing-body preservation when parent metadata is available, refusal to invent a parent for a generation-one revision, logical deletion, case-only rename, cross-path rename, and safe unproven fallbacks.
LiveSync's optional real-Obsidian two-Vault checks have two scopes. `E2E_OBSIDIAN_INCLUDE_MARKDOWN_CONFLICT=true` resolves and edits a Markdown conflict, propagates it to a Vault which still displays the deleted losing content, and requires one live result to remain. `E2E_OBSIDIAN_INCLUDE_CONFLICT_OPERATIONS=true` edits, deletes, case-renames, and cross-path-renames files while conflicts remain active; it verifies the parent revision of each resulting branch, replicates those exact trees, and confirms that the other live branches remain intact.
The focused `test:e2e:obsidian:conflict-dialog-policy` scenario creates three live versions in one real Obsidian Vault. It verifies the count warning, commits a concatenated child of the displayed winner, confirms that the untouched leaf remains as one conflict, postpones that remaining pair, restarts the isolated Obsidian profile, and confirms that only the live pair is reconstructed. It also verifies that an incoming resolution closes a stale dialogue, completes the waiting conflict operation, and clears the warning.
The focused `test:e2e:obsidian:conflict-dialog-policy` scenario creates three live versions in one real Obsidian Vault. It verifies the count warning, commits a concatenated child of the displayed winner, confirms that the untouched leaf remains as one conflict, postpones that remaining pair, restarts the isolated Obsidian profile, and confirms that only the live pair is reconstructed. It also verifies that an incoming resolution closes a stale dialogue, completes the waiting conflict operation, and clears the warning. The repair scenario removes a referenced local chunk, confirms that the exact unreadable live revision remains in the tree, and exercises explicit retry and discard controls without deleting another live revision.
+65
View File
@@ -0,0 +1,65 @@
# Garbage Collection V3
Garbage Collection V3 is a beta maintenance operation for CouchDB remotes. It removes chunk documents which are no longer required by the current local revision tree, propagates those logical deletions to CouchDB, and then requests remote compaction.
It is not a repair operation. Use it only when the Vault, the local LiveSync database, and the CouchDB remote are healthy, every relevant device has synchronised, and recoverable backups exist.
## Supported scope
Garbage Collection V3 is available in Edge Case mode for CouchDB. It is not offered for Object Storage or P2P:
- Object Storage has a different journal and object lifecycle.
- P2P has no central database to compact and cannot provide the accepted-device progress information required by this workflow.
The operation requires **Fetch chunks on demand** to be off so that its local reachability result is not confused with chunks which exist only on the remote.
## Workflow
After the user starts Garbage Collection V3, LiveSync:
1. completes a one-shot bidirectional CouchDB synchronisation;
2. reads the accepted-device list and current progress recorded on the remote;
3. requires parseable progress information, warns when an accepted device has no current information or device progress differs, then requires explicit confirmation;
4. computes the chunks reachable from the local PouchDB revision tree;
5. creates a logical deletion for each locally present chunk which is not reachable;
6. completes a push-only replication so that those deletions reach CouchDB;
7. requests CouchDB compaction and waits for it for up to two minutes; and
8. clears the local chunk caches.
If the initial synchronisation, device inspection, or confirmation fails, the workflow stops before collection. Missing or invalid device progress is treated as a failed inspection rather than offered as a confirmation override. If push-only replication fails, the local logical deletions have already been created, but remote compaction is not started; synchronise again before retrying or using another device. A compaction failure or completion timeout is reported separately and is not also reported as a successful completion.
## Reachability rules
A chunk remains reachable when it is referenced by any of the following:
- the current database winner for a file;
- any other live conflict revision for that file;
- an available revision on either side of a live conflict which is required to describe the divergence; or
- the nearest available revision shared by both live conflict branches.
Chunk identifiers are content-derived and shared between files. Reachability is therefore collected into one set across the database. A chunk used by two or more current files remains protected even when one file is updated or deleted.
An ordinary superseded linear revision does not protect its former chunks. Once no current file or live conflict branch references a chunk, it can be collected. After a conflict is resolved, chunks unique to the discarded branch and to no-longer-needed merge ancestry can also become eligible.
## Consequences
Garbage Collection deliberately trades historical recoverability for storage. A metadata revision may remain in the revision tree after a chunk which only that superseded revision used has been collected, so that historical body can become unreadable. Remote compaction can then discard old CouchDB revision bodies. Tombstones and retained metadata also consume storage, so the operation does not promise the smallest possible database.
Writing the same bytes again produces the same content-derived chunk identifier. If that chunk was collected previously, the normal chunk-writing path creates a new live revision for it, and ordinary replication can transfer it again. This does not recover an older file revision automatically; it only makes the newly written content available.
Garbage Collection does not reconstruct a chunk which is already missing, determine whether an unreadable revision is important, or repair a damaged local database. Use **Inspect conflicts and file/database differences**, another healthy replica, or a backup for those cases. Use **Overwrite Server Data with This Device's Files** only when a chosen Vault is authoritative and a deliberate remote rebuild is required.
## Verification
Commonlib tests use real in-memory PouchDB revision trees to verify:
- collection eligibility after a normal file update;
- protection of chunks shared by multiple current files;
- protection of all live conflict branches and their nearest available shared ancestor;
- eligibility of losing-branch and ancestor-only chunks after conflict resolution;
- propagation of chunk deletion to another PouchDB database; and
- recreation and propagation when the same content is written again.
Self-hosted LiveSync unit tests verify that Garbage Collection V3 uses Commonlib's revision-aware result, deletes only unreachable chunks, performs the initial bidirectional and final push-only replications in order, and requests remote compaction.
A disposable real-CouchDB integration test verifies logical deletion on the server, retention of shared and conflict chunks, compaction completion, replication after collection, and recreation of a content-addressed chunk. CouchDB's choice and timing of physical byte reclamation remain an external database boundary, so the test does not assert a particular reduction in database size.
+4 -4
View File
@@ -27,7 +27,7 @@ All guidelines and conventions listed below are disclosed and maintained solely
- Boot-up sequence (boot-sequence)
- The initialisation process of the plug-in when Obsidian starts. It starts with the loading of the plug-in, setting up core services, loading saved settings, and opening the local database. Once the layout is ready, the plug-in checks for the presence of flag files, runs configuration diagnostics, connects to the remote database, and begins file watching. The sequence finishes once the plug-in is fully ready and operational.
- Broken files (Size mismatch)
- A state where a file's metadata and the actual content stored in its chunks do not match, causing file retrieval or synchronisation failures. These mismatches can be detected and resolved by running validation tools such as `Verify and repair all files` on the Hatch pane.
- A state where a file's metadata and the actual content stored in its chunks do not match, causing file retrieval or synchronisation failures. These mismatches can be inspected with `Inspect conflicts and file/database differences` on the Hatch pane, then handled one exact revision at a time.
- Chunk / Chunks
- Divided units of data stored in the database or object storage to facilitate efficient synchronisation.
- Compaction
@@ -74,8 +74,8 @@ All guidelines and conventions listed below are disclosed and maintained solely
- A privacy option that encrypts file paths and folder names on the remote server.
- plug-in
- We use the hyphenated form `plug-in` in user-facing messages and general documentation, while `plugin` may appear in codebase files, configuration settings, or technical contexts.
- Relay Server (P2P relays)
- A WebSocket-based coordination server used to establish direct WebRTC peer-to-peer connections. The default relay is provided by the plug-in author.
- Signalling relay (P2P)
- A Nostr-compatible WebSocket relay used for peer discovery and WebRTC connection negotiation. It does not store or transfer Vault contents. The project author operates a public relay as a best-effort convenience, and users can provide another compatible relay.
- Remediation (maxMTimeForReflectEvents)
- A recovery setting that restricts the propagation of changes from the database to local storage, ignoring any file events (such as accidental mass deletions) that occurred after a specified date and time.
- Reset Synchronisation on This Device
@@ -95,7 +95,7 @@ All guidelines and conventions listed below are disclosed and maintained solely
- Sync Mode
- The replication trigger mechanism. Users can select from `On Events` (synchronising on local file changes), `Periodic and Events` (synchronising at fixed intervals as well as on events), or `LiveSync` (continuous, real-time synchronisation).
- TURN Server (WebRTC P2P)
- A server type (Traversal Using Relays around NAT) used as a fallback to relay traffic when direct WebRTC peer-to-peer connection is blocked by strict NAT or firewalls.
- A Traversal Using Relays around NAT server used as an optional fallback to relay encrypted WebRTC traffic when strict NAT or firewall rules block a direct peer connection. It is distinct from the signalling relay.
- Update Thinning (Batch database update)
- An optimisation that groups multiple local file edits together over a short delay before committing them to the local database, reducing the number of database write operations.
- WebRTC P2P (Peer-to-Peer)
+4 -1
View File
@@ -47,7 +47,10 @@ A pattern containing only `snippets` does not admit the `.obsidian` parent, so t
![Hidden File Sync initialisation choices](../../images/hidden-file-sync/guide-hidden-file-enable.png)
2. Under `Enable Hidden File Sync`, select the initialisation direction chosen above.
3. Keep Obsidian open while the initial scan and synchronisation finish.
3. Keep Obsidian open while the initial scan and synchronisation finish. A progress Notice appears when preparation begins and remains visible until the initial scan has finished.
![Hidden File Sync initial scan progress Notice](../../images/hidden-file-sync/guide-hidden-file-initial-scan-progress.png)
4. Restart Obsidian when the completion Notice recommends it.
5. Confirm that the expected hidden files, and only those files, are present in the remote synchronisation state.
+44 -11
View File
@@ -10,22 +10,55 @@ authors:
# Peer-to-Peer Synchronisation Tips
For the complete first-device, Setup URI, second-device, and two-way verification procedure, see [Set up peer-to-peer synchronisation](../setup_p2p.md).
For the first device, Setup URI, additional device, and two-way verification procedure, see [Set up peer-to-peer synchronisation](../setup_p2p.md). For the communication and privacy model, see [How peer-to-peer synchronisation works](../p2p.md).
> [!IMPORTANT]
> Peer-to-peer synchronisation is a supported opt-in feature. WebRTC connectivity still depends on the networks, relays, and optional TURN servers available to every device, so a working connection cannot be guaranteed in every environment.
> P2P is a supported opt-in feature, but WebRTC connectivity still depends on the networks available to every device. A direct connection cannot be guaranteed in every environment.
## Difficulties with Peer-to-Peer Synchronisation
## A peer does not appear
It is often the case that peer-to-peer connections do not function correctly, for instance, when using mobile data services.
In such circumstances, we recommend connecting all devices to a single Virtual Private Network (VPN). It is advisable to select a service, such as Tailscale, which facilitates direct communication between peers wherever possible.
Should one be in an environment where even Tailscale is unable to connect, or where it cannot be lawfully installed, please continue reading.
Check discovery before changing any Vault settings:
## A More Detailed Explanation
1. Confirm that both devices use the same **Signalling relay URLs**, Group ID, and P2P passphrase.
2. Confirm that each device has a distinct device name.
3. Open `P2P Status` on both devices and confirm that each shows `Connected`.
4. Select `Refresh` after the other device joins.
5. If the peer remains absent, select `Disconnect`, then `Open connection` on the device which should be advertised again.
The failure of a Peer-to-Peer connection via WebRTC can be attributed to several factors. These may include an unsuccessful UDP hole-punching attempt, or an intermediary gateway intentionally terminating the connection. Troubleshooting this matter is not a simple undertaking. Furthermore, and rather unfortunately, gateway administrators are typically aware of this type of network behaviour. Whilst a legitimate purpose for such traffic can be cited, such as for web conferencing, this is often insufficient to prevent it from being blocked.
The signalling relay discovers peers; it does not prove that the networks can carry a WebRTC data connection.
This situation, however, is the primary reason that our project does not provide a TURN server. Although it is said that a TURN server within WebRTC does not decrypt communications, the project holds the view that the risk of a malicious party impersonating a TURN server must be avoided. Consequently, configuring a TURN server for relay communication is not currently possible through the user interface. Furthermore, there is no official project TURN server, which is to say, one that could be monitored by a third party.
## A peer appears but synchronisation cannot connect
We request that you provide your own server, using your own Fully Qualified Domain Name (FQDN), and subsequently enter its details into the advanced settings.
For testing purposes, Cloudflare's Real-Time TURN Service is exceedingly convenient and offers a generous amount of free data. However, it must be noted that because it is a well-known destination, such traffic is highly conspicuous. There is also a significant possibility that it may be blocked by default. We advise proceeding with caution.
WebRTC may fail when UDP hole punching is blocked by carrier-grade NAT, a firewall, a VPN policy, or an intermediary gateway.
Try these in order:
1. Put both devices on the same ordinary network and retry.
2. Remove a VPN temporarily if it blocks peer traffic, or use a trusted VPN such as Tailscale when it provides a reachable path between the devices.
3. In `P2P Configuration` -> `Advanced Settings`, configure a trusted TURN service.
TURN is a fallback for encrypted WebRTC traffic. It is different from the required signalling relay. The project does not operate an official TURN service. A TURN provider cannot read encrypted Vault contents, but it can observe connection metadata and traffic volume.
## A connected peer does not receive later edits
An open signalling connection does not automatically move every change.
- Use `Replicate now` to prove an explicit bidirectional round trip.
- Enable `Announce changes` on the source device before it dispatches notifications.
- Enable `Follow changes` for that source on the receiving device before it fetches in response.
- Use the peer's `More actions` menu only after the manual round trip works.
If the device was asleep, Obsidian was in the background, or the peer disconnected, run an explicit synchronisation after both devices are visible and connected.
## Mobile limitations
Keep Obsidian visible and the device awake during initial transfer, rebuild, or a large synchronisation. Wake Lock support is best effort and cannot prevent the operating system from suspending or terminating a background application.
## Collect evidence
If the same room works on one network but not another, include both network types in the report. Run `Generate full report for opening the issue with debug info`, remove credentials and private relay details, and state whether:
- both devices reached `Connected`;
- each device appeared in `Detected Peers`;
- a connection request appeared; and
- a TURN server or VPN was in use.
+142 -424
View File
@@ -1,454 +1,172 @@
# Tips and Troubleshooting
- [Tips and Troubleshooting](#tips-and-troubleshooting)
- [Tips](#tips)
- [CORS avoidance](#cors-avoidance)
- [CORS configuration with reverse proxy](#cors-configuration-with-reverse-proxy)
- [Nginx](#nginx)
- [Nginx and subdirectory](#nginx-and-subdirectory)
- [Caddy](#caddy)
- [Caddy and subdirectory](#caddy-and-subdirectory)
- [Apache](#apache)
- [Show all setting panes](#show-all-setting-panes)
- [How to resolve `Tweaks Mismatched of Changed`](#how-to-resolve-tweaks-mismatched-of-changed)
- [Notable bugs and fixes](#notable-bugs-and-fixes)
- [Binary files get bigger on iOS](#binary-files-get-bigger-on-ios)
- [Some setting name has been changed](#some-setting-name-has-been-changed)
- [Questions and Answers](#questions-and-answers)
- [How should I share the settings between multiple devices?](#how-should-i-share-the-settings-between-multiple-devices)
- [What should I enter for the passphrase of Setup-URI?](#what-should-i-enter-for-the-passphrase-of-setup-uri)
- [Why the settings of Self-hosted LiveSync itself is disabled in default?](#why-the-settings-of-self-hosted-livesync-itself-is-disabled-in-default)
- [The plug-in says `something went wrong`.](#the-plug-in-says-something-went-wrong)
- [A large number of files were deleted, and were synchronised!](#a-large-number-of-files-were-deleted-and-were-synchronised)
- [Why `Use an old adapter for compatibility` is somehow enabled in my vault?](#why-use-an-old-adapter-for-compatibility-is-somehow-enabled-in-my-vault)
- [ZIP (or any extensions) files were not synchronised. Why?](#zip-or-any-extensions-files-were-not-synchronised-why)
- [I hope to report the issue, but you said you needs `Report`. How to make it?](#i-hope-to-report-the-issue-but-you-said-you-needs-report-how-to-make-it)
- [Where can I check the log?](#where-can-i-check-the-log)
- [Why are the logs volatile and ephemeral?](#why-are-the-logs-volatile-and-ephemeral)
- [Some network logs are not written into the file.](#some-network-logs-are-not-written-into-the-file)
- [If a file were deleted or trimmed, the capacity of the database should be reduced, right?](#if-a-file-were-deleted-or-trimmed-the-capacity-of-the-database-should-be-reduced-right)
- [How to launch the DevTools](#how-to-launch-the-devtools)
- [On Desktop Devices](#on-desktop-devices)
- [On Android](#on-android)
- [On iOS, iPadOS devices](#on-ios-ipados-devices)
- [How can I use the DevTools?](#how-can-i-use-the-devtools)
- [Checking the network log](#checking-the-network-log)
- [Troubleshooting](#troubleshooting)
- [While using Cloudflare Tunnels, often Obsidian API fallback and `524` error occurs.](#while-using-cloudflare-tunnels-often-obsidian-api-fallback-and-524-error-occurs)
- [On the mobile device, cannot synchronise on the local network!](#on-the-mobile-device-cannot-synchronise-on-the-local-network)
- [I think that something bad happening on the vault...](#i-think-that-something-bad-happening-on-the-vault)
- [Flag Files](#flag-files)
- [Old tips](#old-tips)
# Troubleshooting
<!-- - -->
## Tips
### CORS avoidance
If we are unable to configure CORS properly for any reason (for example, if we cannot configure non-administered network devices), we may choose to ignore CORS.
To use the Obsidian API (also known as the Non-Native API) to bypass CORS, we can enable the toggle ``Use Request API to avoid `inevitable` CORS problem``.
<!-- Add **Long explanation of CORS** here for integrity -->
### CORS configuration with reverse proxy
- IMPORTANT: CouchDB handles CORS by itself. Do not process CORS on the reverse
proxy.
- Do not process `Option` requests on the reverse proxy!
- Make sure `host` and `X-Forwarded-For` headers are forwarded to the CouchDB.
- If you are using a subdirectory, make sure to handle it properly. More
detailed information is in the
[CouchDB documentation](https://docs.couchdb.org/en/stable/best-practices/reverse-proxies.html).
Minimal configurations are as follows:
#### Nginx
```nginx
location / {
proxy_pass http://localhost:5984;
proxy_redirect off;
proxy_buffering off;
proxy_set_header Host $host;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
}
```
#### Nginx and subdirectory
```nginx
location /couchdb {
rewrite ^ $request_uri;
rewrite ^/couchdb/(.*) /$1 break;
proxy_pass http://localhost:5984$uri;
proxy_redirect off;
proxy_buffering off;
proxy_set_header Host $host;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
}
location /_session {
proxy_pass http://localhost:5984/_session;
proxy_redirect off;
proxy_buffering off;
proxy_set_header Host $host;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
}
```
#### Caddy
```caddyfile
domain.com {
reverse_proxy localhost:5984
}
```
#### Caddy and subdirectory
```caddyfile
domain.com {
reverse_proxy /couchdb/* localhost:5984
reverse_proxy /_session/* localhost:5984/_session
}
```
#### Apache
Sorry, Apache is not recommended for CouchDB. Omit the configuration from here.
Please refer to the
[Official documentation](https://docs.couchdb.org/en/stable/best-practices/reverse-proxies.html#reverse-proxying-with-apache-http-server).
### Show all setting panes
Full pane is not shown by default. To show all panes, please toggle all in
`🧙‍♂️ Wizard` -> `Enable extra and advanced features`.
For your information, the all panes are as follows:
![All Panes](all_toggles.png)
### How to resolve `Tweaks Mismatched of Changed`
(Since v0.23.17)
If you have changed some configurations or tweaks which should be unified
between the devices, you will be asked how to reflect (or not) other devices at
the next synchronisation. It also occurs on the device itself, where changes are
made, to prevent unexpected configuration changes from unwanted propagation.\
(We may thank this behaviour if we have synchronised or backed up and restored
Self-hosted LiveSync. At least, for me so).
Following dialogue will be shown: ![Dialogue](tweak_mismatch_dialogue.png)
- If we want to propagate the setting of the device, we should choose
`Update with mine`.
- On other devices, we should choose `Use configured` to accept and use the
configured configuration.
- `Dismiss` can postpone a decision. However, we cannot synchronise until we
have decided.
Rest assured that in most cases we can choose `Use configured`. (Unless you are
certain that you have not changed the configuration).
If we see it for the first time, it reflects the settings of the device that has
been synchronised with the remote for the first time since the upgrade.
Probably, we can accept that.
<!-- Add here -->
## Notable bugs and fixes
### Binary files get bigger on iOS
- Reported at: v0.20.x
- Fixed at: v0.21.2 (Fixed but not reviewed)
- Required action: larger files will not be fixed automatically, please perform
`Verify and repair all files`. If our local database and storage are not
matched, we will be asked to apply which one.
### Some setting name has been changed
- Fixed at: v0.22.6
| Previous name | New name |
| ---------------------------- | ---------------------------------------- |
| Open setup URI | Use the copied setup URI |
| Copy setup URI | Copy current settings as a new setup URI |
| Setup Wizard | Minimal Setup |
| Check database configuration | Check and Fix database configuration |
## Questions and Answers
### How should I share the settings between multiple devices?
- Device setup:
- Using `Setup URI` is the most straightforward way.
- Setting changes during use:
- Use `Sync settings via Markdown files` on the `🔄️ Sync settings` pane.
### What should I enter for the passphrase of Setup-URI?
- Anything you like is OK. However, the recommendation is as follows:
- Include the vault (group) information.
- Include the date of operation.
- Anything random for your security.
- For example, `MyVault-20240901-r4nd0mStr1ng`.
- Why?
- The Setup-URI is encoded; that means it cannot indicate the actual settings. Hence, if you use the same passphrase for multiple vaults, you may accidentally mix up vaults.
### Why the settings of Self-hosted LiveSync itself is disabled in default?
Basically, if we configure all `additionalSuffixOfDatabaseName` the same, we can synchronise this file between multiple devices.
(`additionalSuffixOfDatabaseName` should be unique in each device, not in the synchronised vaults).
However, if we synchronise the settings of Self-hosted LiveSync itself, we may encounter some unexpected behaviours.
For example, if a setting that 'let Self-hosted LiveSync setting be excluded' is synced, it is very unlikely that things will recover automatically after this, and there is little chance we will even notice this. Even if we change our minds and change the settings back on other devices. It could get even worse if incompatible changes are automatically reflected; everything will break.
### The plug-in says `something went wrong`.
There are many cases where this is really unclear. One possibility is that the chunk fetch did not go well.
1. Restarting Obsidian sometimes helps (fetch-order problem).
2. If actually there are no chunks, please perform `Recreate missing chunks for all files` on the `🧰 Hatch` pane at the other devices. And synchronise again. (also restart Obsidian may effect).
3. If the problem persists, please perform `Verify and repair all files` on the `🧰 Hatch` pane. If our local database and storage are not matched, we will be asked to apply which one.
### A large number of files were deleted, and were synchronised!
1. Backup everything important.
- Your local vault.
- Your CouchDB database (this can be done by replicating to another database).
2. Prepare the empty vault
3. Place `redflag.md` at the top of the vault.
4. Apply the settings **BUT DO NOT PROCEED TO RESTORE YET**.
- You can use `Setup URI`, QR Code, or manually apply the settings.
5. Set `Maximum file modification time for reflected file events` in `Remediation` on the `🩹 Patches` pane.
- If you know when the files were deleted, set the time a bit before that.
- If not, bisecting may help us.
6. Delete `redflag.md`.
7. Perform `Reset Synchronisation on This Device` on the `🎛️ Maintenance` pane.
This mode is very fragile. Please be careful.
### Why `Use an old adapter for compatibility` is somehow enabled in my vault?
Because you are a compassionate and experienced user. Before v0.17.16, we used
an old adapter for the local database. At that time, current default adapter has
not been stable. The new adapter has better performance and has a new feature
like purging. Therefore, we should use new adapters and current default is so.
However, when switching from an old adapter to a new adapter, some converting or
local database rebuilding is required, and it takes some time. It was a long
time ago now, but we once inconvenienced everyone in a hurry when we changed the
format of our database. For these reasons, this toggle is automatically on if we
have upgraded from vault which using an old adapter.
When you overwrite server data with this device's files or reset synchronisation on this device again, you will be asked to
switch this.
Therefore, experienced users (especially those stable enough not to have to
overwrite server data) may have this toggle enabled in their Vault. Please
disable it when you have enough time.
### ZIP (or any extensions) files were not synchronised. Why?
It depends on Obsidian detects. May toggling `Detect all extensions` of
`File and links` (setting of Obsidian) will help us.
### I hope to report the issue, but you said you needs `Report`. How to make it?
We can copy the report to the clipboard, by performing
`Generate full report for opening the issue with debug info` command!
### Where can I check the log?
We can launch the log pane by `Show log` on the command palette. And if you have
troubled something, please enable the `Verbose Log` on the `General Setting`
pane.
`Generate full report for opening the issue with debug info` command also contains
the recent 1000 log lines, which is very helpful for debugging. Full-report is
already set to the verbose level, so it contains all the logs without enabling the
`Verbose Log` toggle.
Let me note that please be sure to remove any sensitive information before sharing the report.
However, the logs would not be kept so long and cleared when restarted. If you
want to check the logs, please enable `Write logs into the file` temporarily.
![ScreenShot](../images/write_logs_into_the_file.png)
Start with the symptom which is visible now. Do not reset a database, change transport, or enable P2P merely to see whether the problem disappears.
> [!IMPORTANT]
>
> - Writing logs into the file will impact the performance.
> - Please make sure that you have erased all your confidential information
> before reporting issue.
> If Obsidian will not start, do not give up. Close it, create `redflag.md` at the Vault root with the operating system's file manager, then follow [Recovery and flag files](recovery.md). This is the supported route for intervening before ordinary LiveSync start-up work.
### Why are the logs volatile and ephemeral?
Before changing settings:
To avoid unexpected exposure to our confidential things.
1. Back up the affected Vaults and, where possible, the remote database or bucket.
2. Stop editing on other devices.
3. Confirm that every participating device uses the intended plug-in version.
4. Identify whether the active main remote is CouchDB, Object Storage, or P2P.
5. Open `Show log` and note the first error, rather than only the final summary.
### Some network logs are not written into the file.
For a report, run `Generate full report for opening the issue with debug info`, remove credentials and private server details, and include the steps which caused the symptom.
Especially the CORS error will be reported as a general error to the plug-in for
security reasons. So we cannot detect and log it. We are only able to
investigate them by [Checking the network log](#checking-the-network-log).
## CouchDB does not connect
### If a file were deleted or trimmed, the capacity of the database should be reduced, right?
Check the connection in this order:
No, even though if files were deleted, chunks were not deleted. Self-hosted
LiveSync splits the files into multiple chunks and transfers only newly created.
This behaviour enables us to less traffic. And, the chunks will be shared
between the files to reduce the total usage of the database.
1. Confirm that the URL is complete and points to the intended server.
2. On mobile, use HTTPS with a certificate trusted by the operating system. Plain HTTP and self-signed certificates are not supported.
3. Confirm the username, password, database name, and any custom headers.
4. Confirm that the server responds outside the plug-in and that the database exists on additional devices.
5. Use the setup dialogue's connection test.
6. If basic access works, run **Check server requirements**. Its initial check is read-only. Each offered server change requires separate confirmation.
And one more thing, we can handle the conflicts on any device even though it has
happened on other devices. This means that conflicts will happen in the past,
after the time we have synchronised. Hence we cannot collect and delete the
unused chunks even though if we are not currently referenced.
Configure CouchDB CORS first. Reverse-proxy examples belong in [Set up your own CouchDB server](setup_own_server.md), alongside the rest of the server configuration.
To shrink the database size, `Overwrite Server Data with This Device's Files` is the only reliable and effective way.
But do not worry, if we have synchronised well. We have the actual and real
files. Only it takes a bit of time and traffic.
`Use Internal API` is a compatibility workaround for a trusted server. It sends the configured credentials through Obsidian's internal request API. Enable it only after checking the destination, and do not treat a fallback through that API as proof that the server or proxy is correctly configured.
### How to launch the DevTools
A Cloudflare `524` response means that Cloudflare timed out while waiting for the origin. The response may also lack the CORS headers which would have been present on an ordinary CouchDB response. Correct the long-running server or proxy request first. The advanced CouchDB option `Use timeouts instead of heartbeats` may help only when the underlying operation is otherwise healthy.
#### On Desktop Devices
For JWT-specific setup and key-format errors, see [JWT Authentication on CouchDB](tips/jwt-on-couchdb.md).
We can launch the DevTools by pressing `ctrl`+`shift`+`i` (`Command`+`shift`+`i` on Mac).
## CouchDB was working but synchronisation stopped
#### On Android
Do not switch to P2P or reset the database as the first response. Check:
Please refer to [Remote debug Android devices](https://developer.chrome.com/docs/devtools/remote-debugging/).
Once the DevTools have been launched, everything operates the same as on a PC.
1. the active remote profile and connection state;
2. the plug-in version on every device;
3. the CouchDB response and server logs;
4. pending LiveSync progress indicators;
5. `Check server requirements`; and
6. the LiveSync log and full report.
#### On iOS, iPadOS devices
If the remote is healthy but one device's local database is not, use [Reset Synchronisation on This Device](recovery.md#reset-synchronisation-on-this-device) only after backing up unsynchronised local files.
If we have a Mac, we can inspect from Safari on the Mac. Please refer to [Inspecting iOS and iPadOS](https://developer.apple.com/documentation/safari-developer-tools/inspecting-ios).
## Files are missing or excluded
### How can I use the DevTools?
Check Obsidian's `Detect all file extensions`, LiveSync selectors, ignore files, file-size limits, modification-time limits, and Hidden File Sync rules. A filtered file is different from a file which reached the database but could not be reconstructed from its chunks.
#### Checking the network log
If the log reports missing chunks or a size mismatch:
1. stop editing the affected file and keep a separate copy of any readable content;
2. restart Obsidian once to rule out an interrupted fetch;
3. synchronise a device or restore a backup which still has the correct content;
4. on that healthy device, run `Recreate chunks for current Vault files`, then synchronise;
5. follow [Recover a conflicted or mismatched file](recovery.md#recover-a-conflicted-or-mismatched-file); run `Inspect conflicts and file/database differences` from `Hatch`, then use each revision's wrench menu to review and act on that exact branch; and
6. use `Discard this branch` only after confirming that the exact live branch is no longer wanted. Use the separate `Discard unreadable revision` recovery action only when an unreadable revision is the sole live leaf.
The repair card uses compact diagnostic rows which remain readable in a narrow mobile settings pane. `🧩 Missing chunks: N` marks an unreadable revision. In the database row, `Δsize` means decoded size minus recorded size; `Δsize vs DB` means Vault size minus decoded database size; and `Δtime` means Vault modification time minus database modification time. These are diagnostic values, not a rule for deciding which revision is correct. `✅ Vault matches winner · ⚠️ Conflicts: N` means that the current Vault bytes agree with the database winner while other live branches still need a decision. Every mutating action rechecks that its selected revision is still live. Applying a logical deletion to an existing Vault file requires confirmation; a logical-deletion winner with no Vault file already agrees and is omitted.
`Retry reading revision` does not change the revision tree. `Discard this branch` creates a logical deletion on one exact live revision while another live branch remains and leaves the current Vault file unchanged. If the discarded revision was recorded as the Vault's exact source, that stale device-local provenance is removed. `Discard unreadable revision` provides the corresponding explicit escape hatch for a sole unreadable live leaf. Neither action purges history or reconstructs missing content. An unavailable non-live ancestor cannot be deleted through this workflow; it disables conservative three-way merge but does not prevent explicit selection between readable live revisions.
`Recreate chunks for current Vault files` uses current Vault content. It cannot recreate unique bytes which exist only in an unreadable historical or conflict revision.
## A configuration mismatch dialogue blocks synchronisation
Some settings must match across devices. LiveSync pauses synchronisation when the local and remote values differ rather than propagating an unexpected change silently.
Current releases automatically align compatible settings which control how new chunks are created, by default and where possible. This applies to the chunk hash algorithm, chunk size, and splitter version. Existing content remains readable across these choices, although using different choices can reduce chunk reuse and increase storage or transfer work. An explicit opt-out retains the manual review. A mismatch involving encryption, path obfuscation, file-name case handling, or any combination which includes one of those settings always remains a manual decision.
The available actions depend on when the mismatch is found:
- While checking a remote profile, `Use configured settings` accepts the shared values already stored in that remote. `Dismiss` leaves this device's settings unchanged.
- For a mismatch found before synchronisation, `Apply settings to this device` accepts the remote values. Choose `Update remote database settings` only when this device's values are intended to become the shared values.
- When the change requires local or remote reconstruction, the action itself states that Fetch or Rebuild will follow. Make sure that the intended authoritative copy is available before choosing it.
- `Dismiss` postpones a mismatch found before synchronisation. Synchronisation remains paused until the mismatch is resolved.
![Configuration mismatch dialogue](tweak_mismatch_dialogue.png)
Historic defect notices and renamed controls are retained in the [0.25 release history](releases/0.25.md) and [legacy release history](releases/legacy.md), rather than in the current troubleshooting path.
## Setup and settings questions
### Share a configuration with another device
Generate an encrypted Setup URI from a working device. This preserves the intended remote profiles and selections while allowing the additional device to keep its own device-specific name. Store the URI and its passphrase separately.
For deliberate setting changes during normal use, use `Sync Settings via Markdown` under `Sync settings`.
### Choose a Setup URI passphrase
Use a strong passphrase which is distinct from the Vault encryption passphrase. Record enough context outside the encrypted URI to identify the intended Vault and date, but do not rely on a reused human-readable pattern alone.
### Why synchronising LiveSync's own settings is disabled by default
An automatically propagated transport, database, or exclusion setting can disable the mechanism needed to reverse it. LiveSync therefore keeps its own settings out of Customisation Sync by default. Enable that advanced behaviour only with an independent recovery path and device-specific database suffixes.
### The plug-in reports that something went wrong
Use the first specific error in `Show log` to choose the relevant section. When it names chunks or a size mismatch, follow [Files are missing or excluded](#files-are-missing-or-excluded). Do not rebuild solely from the generic final message.
### A large deletion propagated
Stop every device, preserve the available copies, and follow [Recovery and flag files](recovery.md). If the deletion time is known, `Maximum file modification time for reflected file events` under `Remediation` can limit which remote events are applied while recovering into a separate, backed-up Vault. Treat that as a forensic recovery constraint, not as an ordinary synchronisation setting.
### An old database adapter is still selected
Very old Vaults may retain the compatibility adapter until a deliberate local database migration or reset. Do not toggle it merely to troubleshoot an unrelated current failure. The history and migration notes are in the [legacy release history](releases/legacy.md).
### ZIP or another extension is not synchronised
Enable Obsidian's `Detect all file extensions`, then check LiveSync selectors, ignore rules, and size limits as described in [Files are missing or excluded](#files-are-missing-or-excluded).
## Collect a report
Run `Generate full report for opening the issue with debug info` to copy the current settings summary and recent verbose log lines. Remove credentials, remote URLs, Vault names, file contents, and other private information before sharing it.
When a problem concerns one file, run **Copy database information for the active file**, or use **Hatch****Copy database information for a file** to select another file. The report describes this device's local database view, including the Vault-relative path, document and chunk identifiers, local database revisions, conflicts, and local chunk availability. It does not query the remote server or include file contents. Treat paths and identifiers as private metadata before sharing.
Use `Show log` for live inspection. Logs are intentionally kept in memory for a limited time to reduce accidental disclosure. Enable `Write logs into the file` only while reproducing a problem, then disable it and remove the file after review because persistent logging affects performance and may contain private data.
![Write logs into the file](../images/write_logs_into_the_file.png)
Browser security errors, particularly CORS failures, may reach the plug-in only as a general network error. Use the network inspector when the ordinary log cannot show the rejected response.
## The database remains large after files are deleted
LiveSync stores file metadata, chunks, revision history, conflicts, deletions, and tombstones. Deleting or shortening a file therefore does not immediately remove every object which once represented it.
Garbage Collection V3 can remove unreferenced chunks from a healthy CouchDB setup, but it is appropriate only when the Vault and local database are healthy and all relevant devices have synchronised. Current files and live conflict branches protect their required chunks; an ordinary superseded revision does not. Tombstones and retained metadata are not free, so Garbage Collection does not guarantee a minimal database. Review the [Garbage Collection V3 specification](specs_garbage_collection.md) before using it.
`Overwrite Server Data with This Device's Files` is a separate rebuild operation and is the more certain way to reconstruct a central remote from a chosen authoritative Vault. It is also destructive and may discard changes which exist only on another device. Review [Recovery and flag files](recovery.md#garbage-collection-is-not-rebuild) before choosing between them.
## Inspect a network failure
### Desktop
Open Developer Tools with `Ctrl`+`Shift`+`I`, or `Command`+`Option`+`I` on macOS.
### Android
Follow Chrome's [Remote debug Android devices](https://developer.chrome.com/docs/devtools/remote-debugging/) guide.
### iOS and iPadOS
Use Safari on a Mac and follow Apple's [Inspecting iOS and iPadOS](https://developer.apple.com/documentation/safari-developer-tools/inspecting-ios) guide.
### Network evidence
1. Open the network pane.
2. Find the requests marked in red.\
2. Reproduce the failure and select the request marked in red.
![Errored](../images/devtools1.png)
3. Capture the `Headers`, `Payload`, and, `Response`. **Please be sure to keep
important information confidential**. If the `Response` contains secrets, you
can omitted that. Note: Headers contains a some credentials. **The path of
the request URL, Remote Address, authority, and authorization must be
concealed.**\
3. Record the status, timing, and a sanitised version of the headers, payload, and response.
4. Remove the request path, remote address, authority, authorisation, cookies, credentials, and response secrets before sharing.
![Concealed sample](../images/devtools2.png)
## Troubleshooting
## P2P does not connect or transfer changes
<!-- Add here -->
Use [Peer-to-Peer Synchronisation Tips](tips/p2p-sync-tips.md). Check signalling discovery separately from the WebRTC data path, and confirm which devices announce and follow changes. P2P is not a repair step for another transport.
### While using Cloudflare Tunnels, often Obsidian API fallback and `524` error occurs.
## Obsidian or LiveSync remains suspended
A `524` error occurs when the request to the server is not completed within a
`specified time`. This is a timeout error from Cloudflare. From the reported
issue, it seems to be 100 seconds. (#627).
Follow [Recovery and flag files](recovery.md). A `redflag.md` emergency stop remains active until it is removed outside Obsidian. Fetch and rebuild flags have different, potentially destructive meanings; do not create them merely to clear a warning.
Therefore, this error returns from Cloudflare, not from the server. Hence, the
result contains no CORS field. It means that this response makes the Obsidian
API fallback.
## Further technical context
However, even if the Obsidian API fallback occurs, the request is still not
completed within the `specified time`, 100 seconds.
To solve this issue, we need to configure the timeout settings.
Please enable the toggle in `💪 Power users` -> `CouchDB Connection Tweak` ->
`Use timeouts instead of heartbeats`.
### On the mobile device, cannot synchronise on the local network!
Obsidian mobile is not able to connect to the non-secure end-point, such as
starting with `http://`. Make sure your URI of CouchDB. Also not able to use a
self-signed certificate.
### I think that something bad happening on the vault...
Place the [flag file](#flag-files) on top of the vault, and restart Obsidian. The most simple
way is to create a new note and rename it to `redflag`. Of course, we can put it
without Obsidian.
For example, if there is `redflag.md`, Self-hosted LiveSync suspends all database and storage
processes.
### Scram State and Flag Files (SCRAM Warning Loop)
The plug-in uses a **Scram state** (emergency suspension of all synchronisation processes) to prevent database corruption when severe errors or conflicts are detected. This state is often triggered or persisted by **flag files** placed at the root of the vault.
If you encounter a warning saying **"Scram detected, all sync operations are suspended per SCRAM"** or get caught in an infinite loop where the warning persists even after clicking "Resume", it is likely due to a flag file in your vault.
#### Flag Files
A flag file is a simple Markdown file located at the root of your vault. Its very existence is significant; it may be left blank or contain any text. These files are used so they can be easily placed or deleted from outside Obsidian (e.g., when Obsidian fails to launch).
| Filename | Human-Friendly Name | Description |
| ------------- | ------------------- | --------------------------------------------------------------------------------------- |
| `redflag.md` | - | Suspends all processes (activates Scram). |
| `redflag2.md` | `flag_rebuild.md` | Suspends all processes, and overwrites server data with this device's files. |
| `redflag3.md` | `flag_fetch.md` | Suspends all processes, discards the local database, and resets synchronisation on this device. |
When resetting synchronisation on this device or overwriting server data, restarting Obsidian is performed once for safety reasons. At that time, Self-hosted LiveSync uses these files to determine whether the process should be carried out. (This mechanism is especially useful on mobile devices to force cancellation if the database rebuilding fails). These files are not subject to synchronisation.
Flag-file recovery is handled before the compatibility-review dialogue. `redflag.md` keeps start-up stopped, while a cancelled recovery or one which schedules a restart also prevents a second dialogue from competing with it in that process. When fetch-all or rebuild-all completes and start-up continues, Self-hosted LiveSync can still ask for compatibility review before ordinary synchronisation resumes. Completing a recovery does not automatically acknowledge a database or settings version; use the explicit resume action after reviewing the explanation.
#### How to Resolve the Scram Loop
If you cannot disable Scram, please follow these steps:
1. Close Obsidian completely.
2. Open your system's file manager and check the root directory of your vault.
3. Locate and delete any flag files (such as `redflag.md`, `redflag2.md`, or `redflag3.md`).
4. Launch Obsidian.
5. Go to the settings dialogue -> **Hatch** -> **Scram Switches**, and manually toggle **Suspend file watching** and **Suspend database reflecting** to `false` (disabled) if they have not been reset automatically.
> [!TIP]
> This is the reason why flag files are standard `.md` files: it allows you to manage them externally. On mobile devices, you can use system file manager applications (such as the native **Files** app on iOS/iPadOS or **Files by Google** on Android) to find and delete these files to resolve a lock, or conversely, create/place a new `redflag.md` file (or directory) at the root of your vault to force-suspend synchronisation and stop Obsidian's boot-up sequence if you need to fix a database issue.
### JWT Authentication Errors
#### DataError when configuring JWT authentication
If you encounter a `DataError:` with no additional information in the logs when configuring JWT authentication, this usually indicates a private key formatting issue.
Self-hosted LiveSync requires the private key (for ES256/ES512 algorithms) to be in the **PKCS#8 PEM** format. Standard SEC1 EC private keys (which begin with `-----BEGIN EC PRIVATE KEY-----`) will trigger this error.
To resolve this, convert your private key to PKCS#8 format using the following `openssl` command:
```bash
openssl pkcs8 -topk8 -inform PEM -nocrypt -in private.key -out pkcs8.key
```
Then paste the contents of `pkcs8.key` (which begins with `-----BEGIN PRIVATE KEY-----`) into the JWT Key field.
### Old tips
- Rarely, a file in the database could be corrupted. The plug-in will not write
to local storage when a file looks corrupted. If a local version of the file
is on your device, the corruption could be fixed by editing the local file and
synchronising it. But if the file does not exist on any of your devices, then
it can not be rescued. In this case, you can delete these items from the
settings dialogue.
- To stop the boot-up sequence (eg. for fixing problems on databases), you can
put a `redflag.md` file (or directory) at the root of your vault. Tip for iOS:
a redflag directory can be created at the root of the vault using the File
application.
- Also, with `redflag2.md` placed, we can automatically overwrite server data with this device's files during the boot-up sequence. With `redflag3.md`, we
can discard only the local database and reset synchronisation on this device.
- Q: The database is growing, how can I shrink it down? A: each of the docs is
saved with their past 100 revisions for detecting and resolving conflicts.
Picturing that one device has been offline for a while, and comes online
again. The device has to compare its notes with the remotely saved ones. If
there exists a historic revision in which the note used to be identical, it
could be updated safely (like git fast-forward). Even if that is not in
revision histories, we only have to check the differences after the revision
that both devices commonly have. This is like git's conflict-resolving method.
So, We have to make the database again like an enlarged git repo if you want
to solve the root of the problem.
- And more technical Information is in the [Technical Information](tech_info.md)
- If you want to synchronise files without obsidian, you can use
[filesystem-livesync](https://github.com/vrtmrz/filesystem-livesync).
- WebClipper is also available on Chrome Web
Store:[obsidian-livesync-webclip](https://chrome.google.com/webstore/detail/obsidian-livesync-webclip/jfpaflmpckblieefkegjncjoceapakdf)
Repo is here:
[obsidian-livesync-webclip](https://github.com/vrtmrz/obsidian-livesync-webclip).
(Docs are a work in progress.)
See [Technical Information](tech_info.md) for database and synchronisation internals. Current behaviour belongs in this guide; instructions for older defects remain in the release histories.
Binary file not shown.

Before

Width:  |  Height:  |  Size: 46 KiB

After

Width:  |  Height:  |  Size: 32 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 57 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 47 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 71 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 50 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 60 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 30 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 20 KiB

After

Width:  |  Height:  |  Size: 20 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 71 KiB

After

Width:  |  Height:  |  Size: 71 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 47 KiB

After

Width:  |  Height:  |  Size: 47 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 47 KiB

After

Width:  |  Height:  |  Size: 45 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 20 KiB

After

Width:  |  Height:  |  Size: 19 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 37 KiB

After

Width:  |  Height:  |  Size: 37 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 34 KiB

After

Width:  |  Height:  |  Size: 34 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 6.6 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 71 KiB

After

Width:  |  Height:  |  Size: 71 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 43 KiB

After

Width:  |  Height:  |  Size: 41 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 34 KiB

After

Width:  |  Height:  |  Size: 34 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 27 KiB

After

Width:  |  Height:  |  Size: 28 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 38 KiB

After

Width:  |  Height:  |  Size: 38 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 33 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 51 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 20 KiB

After

Width:  |  Height:  |  Size: 20 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 71 KiB

After

Width:  |  Height:  |  Size: 71 KiB

+1 -1
View File
@@ -1,7 +1,7 @@
{
"id": "obsidian-livesync",
"name": "Self-hosted LiveSync",
"version": "1.0.0-beta.0",
"version": "1.0.2",
"minAppVersion": "1.7.2",
"description": "Community implementation of self-hosted livesync. Reflect your vault changes to some other devices immediately. Please make sure to disable other synchronize solutions to avoid content corruption or duplication.",
"author": "vorotamoroz",
+45 -335
View File
@@ -1,12 +1,12 @@
{
"name": "obsidian-livesync",
"version": "1.0.0-beta.0",
"version": "1.0.2",
"lockfileVersion": 3,
"requires": true,
"packages": {
"": {
"name": "obsidian-livesync",
"version": "1.0.0-beta.0",
"version": "1.0.2",
"license": "MIT",
"workspaces": [
"src/apps/cli",
@@ -22,15 +22,17 @@
"@smithy/querystring-builder": "^4.2.9",
"@smithy/types": "^4.14.3",
"@smithy/util-retry": "^4.4.5",
"@vrtmrz/livesync-commonlib": "0.1.0-rc.11",
"@vrtmrz/obsidian-plugin-kit": "0.1.2",
"@vrtmrz/browser-ui-kit": "0.1.0",
"@vrtmrz/livesync-commonlib": "0.1.0",
"@vrtmrz/obsidian-plugin-kit": "0.1.3",
"@vrtmrz/ui-interactions": "0.1.2",
"diff-match-patch": "^1.0.5",
"fflate": "^0.8.2",
"idb": "^8.0.3",
"markdown-it": "^14.2.0",
"minimatch": "^10.2.5",
"obsidian": "^1.13.1",
"octagonal-wheels": "^0.1.51",
"octagonal-wheels": "^0.1.52",
"qrcode-generator": "^1.4.4",
"xxhash-wasm-102": "npm:xxhash-wasm@^1.0.2"
},
@@ -39,7 +41,6 @@
"@emnapi/core": "1.11.1",
"@emnapi/runtime": "1.11.1",
"@eslint/js": "^9.39.3",
"@playwright/test": "^1.58.2",
"@sveltejs/vite-plugin-svelte": "^7.1.2",
"@tsconfig/svelte": "^5.0.8",
"@types/deno": "^2.5.0",
@@ -57,7 +58,7 @@
"@types/transform-pouch": "^1.0.6",
"@typescript-eslint/parser": "8.56.1",
"@vitest/coverage-v8": "^4.1.8",
"@vrtmrz/obsidian-test-session": "0.2.4",
"@vrtmrz/obsidian-test-session": "0.2.6",
"dotenv-cli": "^11.0.0",
"esbuild": "0.28.1",
"esbuild-plugin-inline-worker": "^0.1.1",
@@ -2201,123 +2202,6 @@
"url": "https://github.com/chalk/wrap-ansi?sponsor=1"
}
},
"node_modules/@istanbuljs/load-nyc-config": {
"version": "1.1.0",
"resolved": "https://registry.npmjs.org/@istanbuljs/load-nyc-config/-/load-nyc-config-1.1.0.tgz",
"integrity": "sha512-VjeHSlIzpv/NyD3N0YuHfXOPDIixcA1q2ZV98wsMqcYlPmv2n3Yb2lYP9XMElnaFVXg5A7YLTeLu6V84uQDjmQ==",
"dev": true,
"license": "ISC",
"dependencies": {
"camelcase": "^5.3.1",
"find-up": "^4.1.0",
"get-package-type": "^0.1.0",
"js-yaml": "^3.13.1",
"resolve-from": "^5.0.0"
},
"engines": {
"node": ">=8"
}
},
"node_modules/@istanbuljs/load-nyc-config/node_modules/argparse": {
"version": "1.0.10",
"resolved": "https://registry.npmjs.org/argparse/-/argparse-1.0.10.tgz",
"integrity": "sha512-o5Roy6tNG4SL/FOkCAN6RzjiakZS25RLYFrcMttJqbdd8BWrnA+fGz57iN5Pb06pvBGvl5gQ0B48dJlslXvoTg==",
"dev": true,
"license": "MIT",
"dependencies": {
"sprintf-js": "~1.0.2"
}
},
"node_modules/@istanbuljs/load-nyc-config/node_modules/find-up": {
"version": "4.1.0",
"resolved": "https://registry.npmjs.org/find-up/-/find-up-4.1.0.tgz",
"integrity": "sha512-PpOwAdQ/YlXQ2vj8a3h8IipDuYRi3wceVQQGYWxNINccq40Anw7BlsEXCMbt1Zt+OLA6Fq9suIpIWD0OsnISlw==",
"dev": true,
"license": "MIT",
"dependencies": {
"locate-path": "^5.0.0",
"path-exists": "^4.0.0"
},
"engines": {
"node": ">=8"
}
},
"node_modules/@istanbuljs/load-nyc-config/node_modules/js-yaml": {
"version": "3.15.0",
"resolved": "https://registry.npmjs.org/js-yaml/-/js-yaml-3.15.0.tgz",
"integrity": "sha512-ttBQIIQPDeLjpPOohtUdXuXUVoA2uIB6fEH9HyJ7234s5mBJ5wTx20njxplLZQgLaOfpmPQA7X2t5AX6tIPbog==",
"dev": true,
"license": "MIT",
"dependencies": {
"argparse": "^1.0.7",
"esprima": "^4.0.0"
},
"bin": {
"js-yaml": "bin/js-yaml.js"
}
},
"node_modules/@istanbuljs/load-nyc-config/node_modules/locate-path": {
"version": "5.0.0",
"resolved": "https://registry.npmjs.org/locate-path/-/locate-path-5.0.0.tgz",
"integrity": "sha512-t7hw9pI+WvuwNJXwk5zVHpyhIqzg2qTlklJOf0mVxGSbe3Fp2VieZcduNYjaLDoy6p9uGpQEGWG87WpMKlNq8g==",
"dev": true,
"license": "MIT",
"dependencies": {
"p-locate": "^4.1.0"
},
"engines": {
"node": ">=8"
}
},
"node_modules/@istanbuljs/load-nyc-config/node_modules/p-limit": {
"version": "2.3.0",
"resolved": "https://registry.npmjs.org/p-limit/-/p-limit-2.3.0.tgz",
"integrity": "sha512-//88mFWSJx8lxCzwdAABTJL2MyWB12+eIY7MDL2SqLmAkeKU9qxRvWuSyTjm3FUmpBEMuFfckAIqEaVGUDxb6w==",
"dev": true,
"license": "MIT",
"dependencies": {
"p-try": "^2.0.0"
},
"engines": {
"node": ">=6"
},
"funding": {
"url": "https://github.com/sponsors/sindresorhus"
}
},
"node_modules/@istanbuljs/load-nyc-config/node_modules/p-locate": {
"version": "4.1.0",
"resolved": "https://registry.npmjs.org/p-locate/-/p-locate-4.1.0.tgz",
"integrity": "sha512-R79ZZ/0wAxKGu3oYMlz8jy/kbhsNrS7SKZ7PxEHBgJ5+F2mtFW2fK2cOtBh1cHYkQsbzFV7I+EoRKe6Yt0oK7A==",
"dev": true,
"license": "MIT",
"dependencies": {
"p-limit": "^2.2.0"
},
"engines": {
"node": ">=8"
}
},
"node_modules/@istanbuljs/load-nyc-config/node_modules/resolve-from": {
"version": "5.0.0",
"resolved": "https://registry.npmjs.org/resolve-from/-/resolve-from-5.0.0.tgz",
"integrity": "sha512-qYg9KP24dD5qka9J47d0aVky0N+b4fTU89LN9iDnjB5waksiC49rvMB0PrUJQGoTmH50XPiqOvAjDfaijGxYZw==",
"dev": true,
"license": "MIT",
"engines": {
"node": ">=8"
}
},
"node_modules/@istanbuljs/schema": {
"version": "0.1.3",
"resolved": "https://registry.npmjs.org/@istanbuljs/schema/-/schema-0.1.3.tgz",
"integrity": "sha512-ZXRY4jNvVgSVQ8DL3LTcakaAtXwTVUxE81hslsyD2AtoXW/wVob10HkOJ1X/pAlcI7D+2YoZKg5do8G/w6RYgA==",
"dev": true,
"license": "MIT",
"engines": {
"node": ">=8"
}
},
"node_modules/@jridgewell/gen-mapping": {
"version": "0.3.13",
"resolved": "https://registry.npmjs.org/@jridgewell/gen-mapping/-/gen-mapping-0.3.13.tgz",
@@ -2732,22 +2616,6 @@
"url": "https://opencollective.com/unts"
}
},
"node_modules/@playwright/test": {
"version": "1.61.0",
"resolved": "https://registry.npmjs.org/@playwright/test/-/test-1.61.0.tgz",
"integrity": "sha512-cKA5B6lpFEMyMGjxF54QihfYpB4FkEGH+qZhtArDEG+wezQAJY8Pq6C7T1SjWz+FFzt3TbyoXBQYk/0292TdJA==",
"dev": true,
"license": "Apache-2.0",
"dependencies": {
"playwright": "1.61.0"
},
"bin": {
"playwright": "cli.js"
},
"engines": {
"node": ">=18"
}
},
"node_modules/@promptbook/utils": {
"version": "0.69.5",
"resolved": "https://registry.npmjs.org/@promptbook/utils/-/utils-0.69.5.tgz",
@@ -4897,10 +4765,19 @@
"url": "https://opencollective.com/vitest"
}
},
"node_modules/@vrtmrz/browser-ui-kit": {
"version": "0.1.0",
"resolved": "https://registry.npmjs.org/@vrtmrz/browser-ui-kit/-/browser-ui-kit-0.1.0.tgz",
"integrity": "sha512-UlMPRZS3lLFcuEKBaAPcSkIY0we3+kj32GHLcOAfxmooy8XaJWh9/VP5MCC+lo06GrZ6graE3vwmAiFsAHwgng==",
"license": "MIT",
"dependencies": {
"@vrtmrz/ui-interactions": "0.1.2"
}
},
"node_modules/@vrtmrz/livesync-commonlib": {
"version": "0.1.0-rc.11",
"resolved": "https://registry.npmjs.org/@vrtmrz/livesync-commonlib/-/livesync-commonlib-0.1.0-rc.11.tgz",
"integrity": "sha512-o811duZFajxDFI6cy7zwtXPg6lihx5e1MH6Xse1sx4HseYurbaDf9DtZJdAtfkjTxl5wOZRSnVMoMLn+I/xD+g==",
"version": "0.1.0",
"resolved": "https://registry.npmjs.org/@vrtmrz/livesync-commonlib/-/livesync-commonlib-0.1.0.tgz",
"integrity": "sha512-rdzEubzLStioanE67pps2XSGZ2UGyMIyeEoKsdPztQLLW8wM05PWApOPbJujIydHcDr2hFanbDFe89Lk7Mn4XQ==",
"license": "MIT",
"dependencies": {
"@aws-sdk/client-s3": "^3.808.0",
@@ -4961,21 +4838,21 @@
}
},
"node_modules/@vrtmrz/obsidian-plugin-kit": {
"version": "0.1.2",
"resolved": "https://registry.npmjs.org/@vrtmrz/obsidian-plugin-kit/-/obsidian-plugin-kit-0.1.2.tgz",
"integrity": "sha512-LNNV8QCaN6FvRA+of96GiqI8atAbnMKQvLbqpi3nsepEe6tN8SmXO3P7pqBl3axNXEHhQfU5ggj5HAq9GJPgwQ==",
"version": "0.1.3",
"resolved": "https://registry.npmjs.org/@vrtmrz/obsidian-plugin-kit/-/obsidian-plugin-kit-0.1.3.tgz",
"integrity": "sha512-6fsKdhFZtBv6FXlZHtSmpqwROohFzDmres6q08nr2xYGVeh2ooBGU3zJS94WN/tOjDT+wa/Vr3yE42wmI0pIZA==",
"license": "MIT",
"dependencies": {
"@vrtmrz/ui-interactions": "0.1.1"
"@vrtmrz/ui-interactions": "0.1.2"
},
"peerDependencies": {
"obsidian": ">=1.8.7"
}
},
"node_modules/@vrtmrz/obsidian-test-session": {
"version": "0.2.4",
"resolved": "https://registry.npmjs.org/@vrtmrz/obsidian-test-session/-/obsidian-test-session-0.2.4.tgz",
"integrity": "sha512-fyb/6xHea/w9WwWi5u9ZJol1rBnVEk+fa1yM1RzUcf6VUAzmwn5zELWtOw8ZsFTdo9QpwVuAYlRAs35AdmUzqQ==",
"version": "0.2.6",
"resolved": "https://registry.npmjs.org/@vrtmrz/obsidian-test-session/-/obsidian-test-session-0.2.6.tgz",
"integrity": "sha512-7xDTmTW3igBxwQGgbbpoRZU0JvOBpGgdRT4qF2WVrKz03OtKMJdkfJILY04/HZ+n4tvn9Ze8phJBr0dx3wewcQ==",
"dev": true,
"license": "MIT",
"engines": {
@@ -4987,9 +4864,9 @@
}
},
"node_modules/@vrtmrz/ui-interactions": {
"version": "0.1.1",
"resolved": "https://registry.npmjs.org/@vrtmrz/ui-interactions/-/ui-interactions-0.1.1.tgz",
"integrity": "sha512-XpO5fQzC7jyOW3xVF7YtFHI7X1BPQ7hJW56uw8atx8mDR7C9ejG2YSn0XcZ1H5FOCPgJbmkh/FLQRKLqSK8jkw==",
"version": "0.1.2",
"resolved": "https://registry.npmjs.org/@vrtmrz/ui-interactions/-/ui-interactions-0.1.2.tgz",
"integrity": "sha512-njT2BSFh57imwGDxWXJOcGbZOYFVNGmeXMrS7/RUdBoAeOjWzJB9TxI7WcR1sgQIXKh8TE9hqALU8lue5b8dRw==",
"license": "MIT"
},
"node_modules/@wdio/config": {
@@ -6109,15 +5986,15 @@
"license": "MIT"
},
"node_modules/brace-expansion": {
"version": "5.0.7",
"resolved": "https://registry.npmjs.org/brace-expansion/-/brace-expansion-5.0.7.tgz",
"integrity": "sha512-7oFy703dxfY3/NLxC1fh2SUCQ0H9rmAY+5EpDVfXjUTTs+HEwR2nYaqLv+GWcTsumwxPfiz6CzCNkwXwBUwqCA==",
"version": "5.0.8",
"resolved": "https://registry.npmjs.org/brace-expansion/-/brace-expansion-5.0.8.tgz",
"integrity": "sha512-JZyDyq3D4AUifKTPOB7DELf6XsB3WdPuNxCtob1vFXPsSXhdAiHBWJ/tJ8HAc9aH84BK+5JFZLNkJKx3G9kzQg==",
"license": "MIT",
"dependencies": {
"balanced-match": "^4.0.2"
},
"engines": {
"node": "18 || 20 || >=22"
"node": "20 || >=22"
}
},
"node_modules/braces": {
@@ -6268,16 +6145,6 @@
"node": ">=6"
}
},
"node_modules/camelcase": {
"version": "5.3.1",
"resolved": "https://registry.npmjs.org/camelcase/-/camelcase-5.3.1.tgz",
"integrity": "sha512-L28STB170nwWS63UjtlEOE3dldQApaJXZkOI1uMFfzf3rRuPegHaHesyee+YxQ+W6SvRDQV6UrdOdRiR153wJg==",
"dev": true,
"license": "MIT",
"engines": {
"node": ">=6"
}
},
"node_modules/caniuse-lite": {
"version": "1.0.30001799",
"resolved": "https://registry.npmjs.org/caniuse-lite/-/caniuse-lite-1.0.30001799.tgz",
@@ -9025,16 +8892,6 @@
"url": "https://github.com/sponsors/ljharb"
}
},
"node_modules/get-package-type": {
"version": "0.1.0",
"resolved": "https://registry.npmjs.org/get-package-type/-/get-package-type-0.1.0.tgz",
"integrity": "sha512-pjzuKtY64GYfWizNAJ0fr9VqttZkNiK2iS430LtIHzjBEr6bX8Am2zm4sW4Ro5wjWW5cAlRL1qAMTcXbjNAO2Q==",
"dev": true,
"license": "MIT",
"engines": {
"node": ">=8.0.0"
}
},
"node_modules/get-port": {
"version": "7.2.0",
"resolved": "https://registry.npmjs.org/get-port/-/get-port-7.2.0.tgz",
@@ -9124,24 +8981,6 @@
"node": ">= 14"
}
},
"node_modules/glob": {
"version": "13.0.6",
"resolved": "https://registry.npmjs.org/glob/-/glob-13.0.6.tgz",
"integrity": "sha512-Wjlyrolmm8uDpm/ogGyXZXb1Z+Ca2B8NbJwqBVg0axK9GbBeoS7yGV6vjXnYdGm6X53iehEuxxbyiKp8QmN4Vw==",
"dev": true,
"license": "BlueOak-1.0.0",
"dependencies": {
"minimatch": "^10.2.2",
"minipass": "^7.1.3",
"path-scurry": "^2.0.2"
},
"engines": {
"node": "18 || 20 || >=22"
},
"funding": {
"url": "https://github.com/sponsors/isaacs"
}
},
"node_modules/glob-parent": {
"version": "6.0.2",
"resolved": "https://registry.npmjs.org/glob-parent/-/glob-parent-6.0.2.tgz",
@@ -10112,23 +9951,6 @@
"node": ">=8"
}
},
"node_modules/istanbul-lib-instrument": {
"version": "6.0.3",
"resolved": "https://registry.npmjs.org/istanbul-lib-instrument/-/istanbul-lib-instrument-6.0.3.tgz",
"integrity": "sha512-Vtgk7L/R2JHyyGW07spoFlB8/lpjiOLTjMdms6AFMraYt3BaJauod/NGrfnVG/y4Ix1JEuMRPDPEj2ua+zz1/Q==",
"dev": true,
"license": "BSD-3-Clause",
"dependencies": {
"@babel/core": "^7.23.9",
"@babel/parser": "^7.23.9",
"@istanbuljs/schema": "^0.1.3",
"istanbul-lib-coverage": "^3.2.0",
"semver": "^7.5.4"
},
"engines": {
"node": ">=10"
}
},
"node_modules/istanbul-lib-report": {
"version": "3.0.1",
"resolved": "https://registry.npmjs.org/istanbul-lib-report/-/istanbul-lib-report-3.0.1.tgz",
@@ -11176,16 +10998,6 @@
"dev": true,
"license": "MIT"
},
"node_modules/lru-cache": {
"version": "11.5.1",
"resolved": "https://registry.npmjs.org/lru-cache/-/lru-cache-11.5.1.tgz",
"integrity": "sha512-RPimw/7aMdv2oqRrxKwvZXcPfwBrn/JZ2xYcY9Hus/6LaS3VOAKVWKWgNLCFSiOm1ESXinjsDlidVU7JlnCN2A==",
"dev": true,
"license": "BlueOak-1.0.0",
"engines": {
"node": "20 || >=22"
}
},
"node_modules/ltgt": {
"version": "2.2.1",
"resolved": "https://registry.npmjs.org/ltgt/-/ltgt-2.2.1.tgz",
@@ -11720,9 +11532,9 @@
"license": "MIT"
},
"node_modules/octagonal-wheels": {
"version": "0.1.51",
"resolved": "https://registry.npmjs.org/octagonal-wheels/-/octagonal-wheels-0.1.51.tgz",
"integrity": "sha512-KTlfqKPjobHJg/t3A539srnFf+VHr1aXkHSmsNDDpiI5UFC7FamZ95dWpJfGE2EI/HULR5hveQDgkazmz8SAcg==",
"version": "0.1.52",
"resolved": "https://registry.npmjs.org/octagonal-wheels/-/octagonal-wheels-0.1.52.tgz",
"integrity": "sha512-9WJN2UveNh90Op1S07cIso1WyNrQbO/unibDLfUGnpomIcU4g6F+p8reZHW0Ed8sGzKF5qGnQp991Y9MB1TwNg==",
"license": "MIT",
"dependencies": {
"idb": "^8.0.3"
@@ -11991,23 +11803,6 @@
"dev": true,
"license": "MIT"
},
"node_modules/path-scurry": {
"version": "2.0.2",
"resolved": "https://registry.npmjs.org/path-scurry/-/path-scurry-2.0.2.tgz",
"integrity": "sha512-3O/iVVsJAPsOnpwWIeD+d6z/7PmqApyQePUtCndjatj/9I5LylHvt5qluFaBT3I5h3r1ejfR056c+FCv+NnNXg==",
"dev": true,
"license": "BlueOak-1.0.0",
"dependencies": {
"lru-cache": "^11.0.0",
"minipass": "^7.1.2"
},
"engines": {
"node": "18 || 20 || >=22"
},
"funding": {
"url": "https://github.com/sponsors/isaacs"
}
},
"node_modules/path-type": {
"version": "4.0.0",
"resolved": "https://registry.npmjs.org/path-type/-/path-type-4.0.0.tgz",
@@ -13634,13 +13429,6 @@
"node": ">= 10.x"
}
},
"node_modules/sprintf-js": {
"version": "1.0.3",
"resolved": "https://registry.npmjs.org/sprintf-js/-/sprintf-js-1.0.3.tgz",
"integrity": "sha512-D9cPgkvLlV3t3IzL0D0YLvGA9Ahk4PcvVwUbN0dSGr1aP0Nrt4AEnTUbuGvquEC0mA64Gqt1fzirlRs5ibXx8g==",
"dev": true,
"license": "BSD-3-Clause"
},
"node_modules/stackback": {
"version": "0.0.2",
"resolved": "https://registry.npmjs.org/stackback/-/stackback-0.0.2.tgz",
@@ -14286,21 +14074,6 @@
"node": ">=10"
}
},
"node_modules/test-exclude": {
"version": "8.0.0",
"resolved": "https://registry.npmjs.org/test-exclude/-/test-exclude-8.0.0.tgz",
"integrity": "sha512-ZOffsNrXYggvU1mDGHk54I96r26P8SyMjO5slMKSc7+IWmtB/MQKnEC2fP51imB3/pT6YK5cT5E8f+Dd9KdyOQ==",
"dev": true,
"license": "ISC",
"dependencies": {
"@istanbuljs/schema": "^0.1.2",
"glob": "^13.0.6",
"minimatch": "^10.2.2"
},
"engines": {
"node": "20 || >=22"
}
},
"node_modules/text-decoder": {
"version": "1.2.7",
"resolved": "https://registry.npmjs.org/text-decoder/-/text-decoder-1.2.7.tgz",
@@ -15155,67 +14928,6 @@
}
}
},
"node_modules/vite-plugin-istanbul": {
"version": "9.0.1",
"resolved": "https://registry.npmjs.org/vite-plugin-istanbul/-/vite-plugin-istanbul-9.0.1.tgz",
"integrity": "sha512-zgcdcqa4r3urX+xqhMQG2uLR2s6IdklQW+acBEtLw6fvUWhuvXswYqxjHAZZ5HCfHEgRs7RH7qIZCklTLRsKLg==",
"dev": true,
"license": "MIT",
"dependencies": {
"@babel/generator": "^7.29.7",
"@istanbuljs/load-nyc-config": "^1.1.0",
"@types/babel__generator": "7.27.0",
"espree": "^11.2.0",
"istanbul-lib-instrument": "^6.0.3",
"picocolors": "^1.1.1",
"source-map": "^0.7.6",
"test-exclude": "^8.0.0"
},
"peerDependencies": {
"vite": ">=7"
}
},
"node_modules/vite-plugin-istanbul/node_modules/eslint-visitor-keys": {
"version": "5.0.1",
"resolved": "https://registry.npmjs.org/eslint-visitor-keys/-/eslint-visitor-keys-5.0.1.tgz",
"integrity": "sha512-tD40eHxA35h0PEIZNeIjkHoDR4YjjJp34biM0mDvplBe//mB+IHCqHDGV7pxF+7MklTvighcCPPZC7ynWyjdTA==",
"dev": true,
"license": "Apache-2.0",
"engines": {
"node": "^20.19.0 || ^22.13.0 || >=24"
},
"funding": {
"url": "https://opencollective.com/eslint"
}
},
"node_modules/vite-plugin-istanbul/node_modules/espree": {
"version": "11.2.0",
"resolved": "https://registry.npmjs.org/espree/-/espree-11.2.0.tgz",
"integrity": "sha512-7p3DrVEIopW1B1avAGLuCSh1jubc01H2JHc8B4qqGblmg5gI9yumBgACjWo4JlIc04ufug4xJ3SQI8HkS/Rgzw==",
"dev": true,
"license": "BSD-2-Clause",
"dependencies": {
"acorn": "^8.16.0",
"acorn-jsx": "^5.3.2",
"eslint-visitor-keys": "^5.0.1"
},
"engines": {
"node": "^20.19.0 || ^22.13.0 || >=24"
},
"funding": {
"url": "https://opencollective.com/eslint"
}
},
"node_modules/vite-plugin-istanbul/node_modules/source-map": {
"version": "0.7.6",
"resolved": "https://registry.npmjs.org/source-map/-/source-map-0.7.6.tgz",
"integrity": "sha512-i5uvt8C3ikiWeNZSVZNWcfZPItFQOsYTUAOkcUPGd8DqDy1uOUikjt5dG+uRlwyvR108Fb9DOd4GvXfT0N2/uQ==",
"dev": true,
"license": "BSD-3-Clause",
"engines": {
"node": ">= 12"
}
},
"node_modules/vite/node_modules/fsevents": {
"version": "2.3.3",
"resolved": "https://registry.npmjs.org/fsevents/-/fsevents-2.3.3.tgz",
@@ -15270,6 +14982,7 @@
"integrity": "sha512-flY6ScbCIt9HThs+C5HS7jvGOB560DJtk/Z15IQROTA6zEy49Nh8T/dofWTQL+n3vswqn87sbJNiuqw1SDp5Ig==",
"dev": true,
"license": "MIT",
"peer": true,
"dependencies": {
"@vitest/expect": "4.1.8",
"@vitest/mocker": "4.1.8",
@@ -16211,11 +15924,11 @@
},
"src/apps/cli": {
"name": "self-hosted-livesync-cli",
"version": "1.0.0-beta.0-cli",
"version": "1.0.2-cli",
"dependencies": {
"chokidar": "^4.0.0",
"minimatch": "^10.2.5",
"octagonal-wheels": "^0.1.51",
"octagonal-wheels": "^0.1.52",
"pouchdb-adapter-http": "^9.0.0",
"pouchdb-adapter-leveldb": "^9.0.0",
"pouchdb-core": "^9.0.0",
@@ -16236,24 +15949,21 @@
},
"src/apps/webapp": {
"name": "livesync-webapp",
"version": "1.0.0-beta.0-webapp",
"version": "1.0.2-webapp",
"dependencies": {
"octagonal-wheels": "^0.1.51"
"octagonal-wheels": "^0.1.52"
},
"devDependencies": {
"@playwright/test": "^1.58.2",
"@sveltejs/vite-plugin-svelte": "^7.1.2",
"playwright": "^1.58.2",
"svelte": "5.56.3",
"typescript": "5.9.3",
"vite": "^8.0.16",
"vite-plugin-istanbul": "^9.0.1"
"vite": "^8.0.16"
}
},
"src/apps/webpeer": {
"version": "1.0.0-beta.0-webpeer",
"version": "1.0.2-webpeer",
"dependencies": {
"octagonal-wheels": "^0.1.51"
"octagonal-wheels": "^0.1.52"
},
"devDependencies": {
"@sveltejs/vite-plugin-svelte": "^7.1.2",
+26 -8
View File
@@ -1,6 +1,6 @@
{
"name": "obsidian-livesync",
"version": "1.0.0-beta.0",
"version": "1.0.2",
"description": "Reflect your vault changes to some other devices immediately. Please make sure to disable other synchronize solutions to avoid content corruption or duplication.",
"main": "main.js",
"type": "module",
@@ -12,6 +12,7 @@
"buildDev": "node esbuild.config.mjs dev",
"lint": "eslint --cache --cache-strategy content --concurrency off src",
"lint:community": "eslint --config eslint.community.config.mjs --concurrency off src",
"lint:community:tools": "eslint --config eslint.community.config.mjs --concurrency off --max-warnings 0 _tools",
"svelte-check": "svelte-check --tsconfig ./tsconfig.json --fail-on-warnings",
"tsc-check": "tsc --noEmit",
"tsc-check:apps": "tsc --noEmit -p src/apps/browser/tsconfig.json && tsc --noEmit -p src/apps/cli/tsconfig.json && tsc --noEmit -p src/apps/webapp/tsconfig.json && tsc --noEmit -p src/apps/webpeer/tsconfig.app.json && tsc --noEmit -p src/apps/webpeer/tsconfig.node.json",
@@ -22,14 +23,20 @@
"prettyNoWrite": "prettier --config ./.prettierrc.mjs \"**/*.js\" \"**/*.ts\" \"**/*.json\" ",
"precheck:compatibility": "npm run build",
"check:compatibility": "node utils/check-compatibility.js --file main.js --ios 15",
"check": "npm run tsc-check && npm run tsc-check:apps && npm run lint && npm run lint:community -- --quiet && npm run svelte-check && npm run check:compatibility",
"check": "npm run tsc-check && npm run tsc-check:apps && npm run lint && npm run lint:community -- --quiet && npm run lint:community:tools && npm run svelte-check && npm run check:compatibility",
"i18n:bake": "npm run i18n:yaml2json && npm run i18n:bakejson && npm run i18n:format",
"i18n:bakejson": "tsx _tools/bakei18n.ts",
"i18n:format": "prettier --config .prettierrc.mjs --write --log-level error 'src/common/messagesJson/*.json' 'src/common/messages/*.ts'",
"i18n:json2yaml": "tsx _tools/json2yaml.ts",
"i18n:yaml2json": "tsx _tools/yaml2json.ts",
"unittest": "deno test -A --no-check --coverage=cov_profile --v8-flags=--expose-gc --trace-leaks ./src/",
"test:unit": "vitest run --config vitest.config.unit.ts",
"build:browser-apps": "npm run build --workspace livesync-webapp --workspace webpeer",
"test:browser-apps": "npm run test:browser --workspace livesync-webapp && npm run test:browser --workspace webpeer",
"test:browser-apps:pages": "deno test -A --no-check --frozen --config test/browser-apps/deno.json --lock test/browser-apps/deno.lock test/browser-apps/pages/browser-smoke.test.ts",
"test:e2e:browser-apps": "npm run test:browser-apps",
"pretest:e2e:browser-apps:interop": "npm run build:browser-apps && npm run build --workspace self-hosted-livesync-cli",
"test:e2e:browser-apps:interop": "deno run -A --no-check --frozen --config test/browser-apps/deno.json --lock test/browser-apps/deno.lock test/browser-apps/run-compose-interop.ts",
"inspect:troubleshooting": "tsx _tools/inspect-troubleshooting-docs.ts",
"test:setup-tools": "bash utils/flyio/setenv.test.sh && deno test -A --config=utils/flyio/deno.jsonc --frozen --lock=utils/flyio/deno.lock utils/livesync-commonlib-version.test.ts utils/couchdb/provision.test.ts utils/setup/generate_setup_uri.test.ts utils/flyio/generate_setupuri.test.ts",
"test:contract:contexts": "vitest run --config vitest.config.unit.ts src/modules/services/ObsidianServiceContext.unit.spec.ts src/apps/cli/services/NodeServiceContext.unit.spec.ts src/apps/browser/createLiveSyncBrowserServiceHub.unit.spec.ts",
"test:contract:context:webapp": "vitest run --config vitest.config.unit.ts src/apps/browser/createLiveSyncBrowserServiceHub.unit.spec.ts",
@@ -37,6 +44,8 @@
"test:contract:context:obsidian": "npm run build && npm run test:e2e:obsidian:smoke",
"test:e2e:cli": "npm run test:e2e:ci --workspace self-hosted-livesync-cli",
"test:e2e:cli:p2p": "npm run test:e2e:p2p --workspace self-hosted-livesync-cli",
"test:e2e:cli:adaptive-s3": "npm run test:e2e:adaptive-s3 --workspace self-hosted-livesync-cli",
"test:e2e:cli:adaptive-webdav": "npm run test:e2e:adaptive-webdav --workspace self-hosted-livesync-cli",
"test:e2e:cli:all": "npm run test:e2e:all --workspace self-hosted-livesync-cli",
"test:integration": "npx dotenv-cli -e .env -e .test.env -- vitest run --config vitest.config.integration.ts",
"test:unit:coverage": "vitest run --config vitest.config.unit.ts --coverage",
@@ -50,18 +59,26 @@
"test:e2e:obsidian:onboarding-invitation": "tsx test/e2e-obsidian/scripts/onboarding-invitation.ts",
"test:e2e:obsidian:dialog-mounts": "tsx test/e2e-obsidian/scripts/dialog-mounts.ts",
"test:e2e:obsidian:conflict-dialog-policy": "tsx test/e2e-obsidian/scripts/conflict-dialog-policy.ts",
"test:e2e:obsidian:revision-repair": "tsx test/e2e-obsidian/scripts/revision-repair.ts",
"test:e2e:obsidian:document-history-nav": "tsx test/e2e-obsidian/scripts/document-history-nav.ts",
"test:e2e:obsidian:settings-ui": "tsx test/e2e-obsidian/scripts/settings-ui.ts",
"test:e2e:obsidian:review-harness": "tsx test/e2e-obsidian/scripts/review-harness.ts",
"test:e2e:obsidian:p2p-pane": "tsx test/e2e-obsidian/scripts/p2p-pane.ts",
"test:e2e:obsidian:vault-reflection": "tsx test/e2e-obsidian/scripts/vault-reflection.ts",
"test:e2e:obsidian:couchdb-upload": "tsx test/e2e-obsidian/scripts/couchdb-upload.ts",
"test:e2e:obsidian:couchdb-manual-setup-workflow": "tsx test/e2e-obsidian/scripts/couchdb-manual-setup-workflow.ts",
"test:e2e:obsidian:cli-to-obsidian-sync": "tsx test/e2e-obsidian/scripts/cli-to-obsidian-sync.ts",
"test:e2e:obsidian:minio-upload": "tsx test/e2e-obsidian/scripts/minio-upload.ts",
"test:e2e:obsidian:adaptive-s3": "tsx test/e2e-obsidian/scripts/minio-upload.ts --adaptive",
"test:e2e:obsidian:object-storage-setup-uri-workflow": "tsx test/e2e-obsidian/scripts/object-storage-setup-uri-workflow.ts",
"test:e2e:obsidian:p2p-setup-uri-workflow": "tsx test/e2e-obsidian/scripts/p2p-setup-uri-workflow.ts",
"pretest:e2e:obsidian:p2p-connection-check": "npm run build && npm run build --workspace webpeer",
"test:e2e:obsidian:p2p-connection-check": "tsx test/e2e-obsidian/scripts/p2p-connection-check.ts",
"test:e2e:obsidian:p2p-connection-check:services": "npm run test:e2e:obsidian:p2p-connection-check -- --manage-p2p",
"test:e2e:obsidian:startup-scan": "tsx test/e2e-obsidian/scripts/startup-scan.ts",
"test:e2e:obsidian:setup-uri-workflow": "tsx test/e2e-obsidian/scripts/setup-uri-workflow.ts",
"test:e2e:obsidian:two-vault-sync": "tsx test/e2e-obsidian/scripts/two-vault-sync.ts",
"test:e2e:obsidian:security-seed-reconnect": "tsx test/e2e-obsidian/scripts/security-seed-reconnect.ts",
"test:e2e:obsidian:hidden-file-snippet-sync": "tsx test/e2e-obsidian/scripts/hidden-file-snippet-sync.ts",
"test:e2e:obsidian:customisation-sync": "tsx test/e2e-obsidian/scripts/customisation-sync.ts",
"test:e2e:obsidian:setting-markdown-export": "tsx test/e2e-obsidian/scripts/setting-markdown-export.ts",
@@ -96,7 +113,6 @@
"@emnapi/core": "1.11.1",
"@emnapi/runtime": "1.11.1",
"@eslint/js": "^9.39.3",
"@playwright/test": "^1.58.2",
"@sveltejs/vite-plugin-svelte": "^7.1.2",
"@tsconfig/svelte": "^5.0.8",
"@types/deno": "^2.5.0",
@@ -114,7 +130,7 @@
"@types/transform-pouch": "^1.0.6",
"@typescript-eslint/parser": "8.56.1",
"@vitest/coverage-v8": "^4.1.8",
"@vrtmrz/obsidian-test-session": "0.2.4",
"@vrtmrz/obsidian-test-session": "0.2.6",
"dotenv-cli": "^11.0.0",
"esbuild": "0.28.1",
"esbuild-plugin-inline-worker": "^0.1.1",
@@ -163,15 +179,17 @@
"@smithy/querystring-builder": "^4.2.9",
"@smithy/types": "^4.14.3",
"@smithy/util-retry": "^4.4.5",
"@vrtmrz/livesync-commonlib": "0.1.0-rc.11",
"@vrtmrz/obsidian-plugin-kit": "0.1.2",
"@vrtmrz/browser-ui-kit": "0.1.0",
"@vrtmrz/livesync-commonlib": "0.1.0",
"@vrtmrz/obsidian-plugin-kit": "0.1.3",
"@vrtmrz/ui-interactions": "0.1.2",
"diff-match-patch": "^1.0.5",
"fflate": "^0.8.2",
"idb": "^8.0.3",
"markdown-it": "^14.2.0",
"minimatch": "^10.2.5",
"obsidian": "^1.13.1",
"octagonal-wheels": "^0.1.51",
"octagonal-wheels": "^0.1.52",
"qrcode-generator": "^1.4.4",
"xxhash-wasm-102": "npm:xxhash-wasm@^1.0.2"
},
+2 -2
View File
@@ -21,7 +21,7 @@ import { AbstractModule } from "./modules/AbstractModule";
import { ModulePeriodicProcess } from "./modules/core/ModulePeriodicProcess";
import { ModuleReplicator } from "./modules/core/ModuleReplicator";
import { ModuleReplicatorCouchDB } from "./modules/core/ModuleReplicatorCouchDB";
import { ModuleReplicatorMinIO } from "./modules/core/ModuleReplicatorMinIO";
import { ModuleReplicatorJournal } from "./modules/core/ModuleReplicatorJournal";
import { ModuleConflictChecker } from "./modules/coreFeatures/ModuleConflictChecker";
import { ModuleConflictResolver } from "./modules/coreFeatures/ModuleConflictResolver";
import { ModuleResolvingMismatchedTweaks } from "./modules/coreFeatures/ModuleResolveMismatchedTweaks";
@@ -139,7 +139,7 @@ export class LiveSyncBaseCore<
public registerModules(extraModules: AbstractModule[] = []) {
this._registerModule(new ModuleLiveSyncMain(this));
this._registerModule(new ModuleConflictChecker(this));
this._registerModule(new ModuleReplicatorMinIO(this));
this._registerModule(new ModuleReplicatorJournal(this));
this._registerModule(new ModuleReplicatorCouchDB(this));
this._registerModule(new ModuleReplicator(this));
this._registerModule(new ModuleConflictResolver(this));
+33 -130
View File
@@ -1,136 +1,39 @@
import type { Confirm, ConfirmActionLayout } from "@vrtmrz/livesync-commonlib/compat/interfaces/Confirm";
import { createNativeElement } from "@/apps/browserDom";
import MessageBox from "./ui/MessageBox.svelte";
import TextInputBox from "./ui/TextInputBox.svelte";
import { mount } from "svelte";
import { promiseWithResolvers } from "octagonal-wheels/promises";
import type { ServiceContext } from "@vrtmrz/livesync-commonlib/context";
import { _activeDocument, compatGlobal } from "@vrtmrz/livesync-commonlib/compat/common/coreEnvFunctions";
import { _activeDocument } from "@vrtmrz/livesync-commonlib/compat/common/coreEnvFunctions";
import { EVENT_PLUGIN_UNLOADED } from "@vrtmrz/livesync-commonlib/compat/events/coreEvents";
import { BrowserUiNotifications, createBrowserUi } from "@vrtmrz/browser-ui-kit";
function displayMessageBox<T, U extends string[]>(
message: string,
buttons: U,
title: string,
commit: (ret: U[number]) => T,
actionLayout: ConfirmActionLayout = "vertical"
): Promise<T> {
const el = createNativeElement(_activeDocument, "div");
const p = promiseWithResolvers<T>();
mount(MessageBox, {
target: el,
props: {
message,
buttons: buttons as string[],
title: title,
actionLayout,
commit: (action: U[number]) => {
const ret = commit(action);
p.resolve(ret);
},
},
});
_activeDocument.body.appendChild(el);
void p.promise.finally(() => {
el.remove();
});
return p.promise;
}
function promptForInput(
title: string,
key: string,
placeholder: string,
isPassword?: boolean
): Promise<string | false> {
const el = createNativeElement(_activeDocument, "div");
const p = promiseWithResolvers<string | false>();
mount(TextInputBox, {
target: el,
props: {
title,
message: key,
placeholder,
isPassword,
commit: (text: string | false) => {
p.resolve(text);
},
},
});
_activeDocument.body.appendChild(el);
void p.promise.finally(() => {
el.remove();
});
return p.promise;
}
import { createNativeElement } from "@/apps/browserDom";
import { renderMessageMarkdownInto } from "./ui/renderMessageMarkdown";
import { UiInteractionsConfirm } from "./UiInteractionsConfirm";
/**
* Compatibility facade consumed by Commonlib while browser presentation is
* implemented through Fancy Kit's neutral `UiInteractions` contract.
*/
export class BrowserConfirm<T extends ServiceContext> extends UiInteractionsConfirm {
readonly context: T;
export class BrowserConfirm<T extends ServiceContext> implements Confirm {
_context: T;
constructor(context: T) {
this._context = context;
}
askYesNo(message: string): Promise<"yes" | "no"> {
return displayMessageBox(message, ["Yes", "No"] as const, "Confirm", (action) =>
action == "Yes" ? "yes" : "no"
);
}
askString(title: string, key: string, placeholder: string, isPassword?: boolean): Promise<string | false> {
return promptForInput(title, key, placeholder, isPassword);
}
askYesNoDialog(
message: string,
opt: { title?: string; defaultOption?: "Yes" | "No"; timeout?: number }
): Promise<"yes" | "no"> {
return displayMessageBox(message, ["Yes", "No"] as const, opt.title ?? "Confirm", (action) =>
action == "Yes" ? "yes" : "no"
);
}
askSelectString(message: string, items: string[]): Promise<string> {
return displayMessageBox(message, [...items] as const, "Confirm", (action) => action);
}
askSelectStringDialogue<T extends readonly string[]>(
message: string,
buttons: T,
opt: { title?: string; defaultAction: T[number]; timeout?: number }
): Promise<T[number] | false> {
return displayMessageBox(message, [...buttons] as const, opt.title ?? "Confirm", (action) => action);
}
askInPopup(
key: string,
dialogText: string,
anchorCallback: (anchor: HTMLAnchorElement) => void,
durationMs: number = 20000
): void {
const existing = _activeDocument.querySelector(`[data-livesync-popup="${CSS.escape(key)}"]`);
existing?.remove();
const notice = createNativeElement(_activeDocument, "div");
notice.className = "livesync-browser-notice";
notice.dataset.livesyncPopup = key;
const [beforeText, afterText] = dialogText.split("{HERE}", 2);
notice.append(beforeText);
const anchor = createNativeElement(_activeDocument, "a");
anchor.href = "#";
anchorCallback(anchor);
anchor.addEventListener("click", () => notice.remove());
notice.append(anchor, afterText ?? "");
_activeDocument.body.appendChild(notice);
compatGlobal.setTimeout(() => notice.remove(), durationMs);
}
confirmWithMessage(
title: string,
contentMd: string,
buttons: string[],
defaultAction: (typeof buttons)[number],
timeout?: number,
actionLayout?: ConfirmActionLayout
): Promise<(typeof buttons)[number] | false> {
return displayMessageBox(
contentMd,
[...buttons] as const,
title ?? "Confirm",
(action) => action,
actionLayout
);
const dialogueController = new AbortController();
const notifications = new BrowserUiNotifications({
document: _activeDocument,
});
super({
ui: createBrowserUi({
document: _activeDocument,
signal: dialogueController.signal,
renderMarkdown: ({ container, markdown }) => {
renderMessageMarkdownInto(container, markdown);
},
}),
notifications,
createActionAnchor: () => createNativeElement(_activeDocument, "a"),
});
this.context = context;
context.events.onceEvent(EVENT_PLUGIN_UNLOADED, () => {
dialogueController.abort();
notifications.dispose();
});
}
}
+122
View File
@@ -0,0 +1,122 @@
import type {
KeyValueDatabase,
KeyValueDatabaseFactory,
} from "@vrtmrz/livesync-commonlib/compat/interfaces/KeyValueDatabase";
import { deleteDB, openDB, type IDBPDatabase } from "idb";
import { serialized } from "octagonal-wheels/concurrency/lock";
/** Creates an application-owned IndexedDB key-value factory for browser runtimes. */
export function createBrowserKeyValueDatabaseFactory(): KeyValueDatabaseFactory {
const cache = new Map<string, BrowserKeyValueDatabase>();
return async (databaseKey) =>
await serialized(`OpenBrowserKeyValueDatabase-${databaseKey}`, async () => {
const cached = cache.get(databaseKey);
if (cached && !cached.isDestroyed) {
return cached;
}
if (cached) {
await cached.ensuredDestroyed;
cache.delete(databaseKey);
}
const database = new BrowserKeyValueDatabase(databaseKey);
await database.getIsReady();
cache.set(databaseKey, database);
return database;
});
}
class BrowserKeyValueDatabase implements KeyValueDatabase {
private databasePromise?: Promise<IDBPDatabase<unknown>>;
private destroyed = false;
private destroyedPromise?: Promise<void>;
constructor(private readonly databaseKey: string) {}
get isDestroyed(): boolean {
return this.destroyed;
}
get ensuredDestroyed(): Promise<void> {
return this.destroyedPromise ?? Promise.resolve();
}
async getIsReady(): Promise<boolean> {
await this.ensureDatabase();
return !this.destroyed;
}
private ensureDatabase(): Promise<IDBPDatabase<unknown>> {
if (this.destroyed) {
throw new Error("Database is destroyed");
}
this.databasePromise ??= openDB(this.databaseKey, undefined, {
upgrade: (database) => {
if (!database.objectStoreNames.contains(this.databaseKey)) {
database.createObjectStore(this.databaseKey);
}
},
blocking: () => {
void this.closeDatabase();
},
terminated: () => {
this.databasePromise = undefined;
},
}).catch((error: unknown) => {
this.databasePromise = undefined;
throw error;
});
return this.databasePromise;
}
private get database(): Promise<IDBPDatabase<unknown>> {
return this.ensureDatabase();
}
private async closeDatabase(): Promise<void> {
const databasePromise = this.databasePromise;
this.databasePromise = undefined;
if (databasePromise) {
(await databasePromise).close();
}
}
async get<T>(key: IDBValidKey): Promise<T> {
return (await (await this.database).get(this.databaseKey, key)) as T;
}
async set<T>(key: IDBValidKey, value: T): Promise<IDBValidKey> {
await (await this.database).put(this.databaseKey, value, key);
return key;
}
async del(key: IDBValidKey): Promise<void> {
await (await this.database).delete(this.databaseKey, key);
}
async clear(): Promise<void> {
await (await this.database).clear(this.databaseKey);
}
async keys(query?: IDBValidKey | IDBKeyRange, count?: number): Promise<IDBValidKey[]> {
return await (await this.database).getAllKeys(this.databaseKey, query, count);
}
async close(): Promise<void> {
await this.closeDatabase();
}
async destroy(): Promise<void> {
if (this.destroyedPromise) {
await this.destroyedPromise;
return;
}
this.destroyed = true;
this.destroyedPromise = (async () => {
await this.closeDatabase();
await deleteDB(this.databaseKey);
})();
await this.destroyedPromise;
}
}
@@ -0,0 +1,133 @@
<script lang="ts">
import { onMount } from "svelte";
import type { P2PSyncSetting } from "@vrtmrz/livesync-commonlib/compat/common/types";
import type { P2PReplicatorPaneHost } from "@/features/P2PSync/P2PReplicator/P2PReplicatorPaneHost";
interface Props {
host: P2PReplicatorPaneHost;
}
let { host }: Props = $props();
const currentSettings = () => host.services.setting.currentSettings() as P2PSyncSetting;
const initialSettings = currentSettings();
let savedTurnServers = $state(initialSettings.P2P_turnServers);
let savedTurnUsername = $state(initialSettings.P2P_turnUsername);
let savedTurnCredential = $state(initialSettings.P2P_turnCredential);
let turnServers = $state(initialSettings.P2P_turnServers);
let turnUsername = $state(initialSettings.P2P_turnUsername);
let turnCredential = $state(initialSettings.P2P_turnCredential);
const isTurnServersModified = $derived(turnServers !== savedTurnServers);
const isTurnUsernameModified = $derived(turnUsername !== savedTurnUsername);
const isTurnCredentialModified = $derived(turnCredential !== savedTurnCredential);
const isModified = $derived(
isTurnServersModified || isTurnUsernameModified || isTurnCredentialModified
);
function loadSettings(settings: P2PSyncSetting): void {
savedTurnServers = settings.P2P_turnServers;
savedTurnUsername = settings.P2P_turnUsername;
savedTurnCredential = settings.P2P_turnCredential;
turnServers = savedTurnServers;
turnUsername = savedTurnUsername;
turnCredential = savedTurnCredential;
}
onMount(() =>
host.services.context.events.onEvent("setting-saved", (settings) => {
loadSettings(settings as P2PSyncSetting);
})
);
async function save(): Promise<void> {
await host.services.setting.applyPartial(
{
P2P_turnServers: turnServers,
P2P_turnUsername: turnUsername,
P2P_turnCredential: turnCredential,
},
true
);
loadSettings(currentSettings());
}
function revert(): void {
turnServers = savedTurnServers;
turnUsername = savedTurnUsername;
turnCredential = savedTurnCredential;
}
</script>
<section class="browser-p2p-transport-settings">
<details>
<summary>Optional TURN server settings</summary>
<p>
Configure TURN only when a direct peer-to-peer connection cannot be established.
</p>
<label class:is-dirty={isTurnServersModified}>
<span>TURN Server URLs (comma-separated)</span>
<input
type="text"
placeholder="turn:turn.example.com:3478"
bind:value={turnServers}
autocomplete="off"
spellcheck="false"
autocorrect="off"
/>
</label>
<label class:is-dirty={isTurnUsernameModified}>
<span>TURN Username</span>
<input
type="text"
placeholder="Enter TURN username"
bind:value={turnUsername}
autocomplete="off"
/>
</label>
<label class:is-dirty={isTurnCredentialModified}>
<span>TURN Credential</span>
<input
type="password"
placeholder="Enter TURN credential"
bind:value={turnCredential}
autocomplete="new-password"
/>
</label>
<div class="actions">
<button type="button" class="button mod-cta" disabled={!isModified} onclick={save}>
Save TURN settings
</button>
<button type="button" class="button" disabled={!isModified} onclick={revert}>
Revert TURN settings
</button>
</div>
</details>
</section>
<style>
.browser-p2p-transport-settings {
margin-bottom: 1rem;
}
p {
margin: 0.75rem 0;
}
label {
display: grid;
gap: 0.25rem;
margin: 0.75rem 0;
}
label.is-dirty {
background-color: var(--background-modifier-error);
}
input {
box-sizing: border-box;
width: 100%;
}
.actions {
display: flex;
flex-wrap: wrap;
gap: 0.5rem;
}
</style>
+24 -10
View File
@@ -1,15 +1,15 @@
import {
type ComponentHasResult,
SvelteDialogManagerBase,
SvelteDialogMixIn,
} from "@vrtmrz/livesync-commonlib/compat/services/implements/base/SvelteDialog";
import { createNativeElement } from "@/apps/browserDom";
import type { ServiceContext } from "@vrtmrz/livesync-commonlib/context";
import type { SvelteDialogManagerDependencies } from "@vrtmrz/livesync-commonlib/compat/services/implements/base/SvelteDialog";
import { _activeDocument } from "@vrtmrz/livesync-commonlib/compat/common/coreEnvFunctions";
import DialogHost from "@/modules/services/LiveSyncUI/DialogHost.svelte";
import { SvelteDialogSession } from "@/modules/services/SvelteDialogSession";
export class ShimModal {
export class BrowserModal {
contentEl: HTMLElement;
titleEl: HTMLElement;
modalEl: HTMLElement;
@@ -45,19 +45,14 @@ export class ShimModal {
}
onOpen() {}
onClose() {}
setPlaceholder(p: string) {}
setTitle(t: string) {
this.titleEl.textContent = t;
}
}
const BrowserSvelteDialogBase = SvelteDialogMixIn(ShimModal, DialogHost);
export class LiveSyncBrowserDialog<T, U, C extends ServiceContext = ServiceContext> extends BrowserModal {
private readonly session: SvelteDialogSession<T, U, C>;
export class LiveSyncBrowserDialog<T, U, C extends ServiceContext = ServiceContext> extends BrowserSvelteDialogBase<
T,
U,
C
> {
constructor(
context: C,
dependents: SvelteDialogManagerDependencies<C>,
@@ -65,7 +60,26 @@ export class LiveSyncBrowserDialog<T, U, C extends ServiceContext = ServiceConte
initialData?: U
) {
super();
this.initDialog(context, dependents, component, initialData);
this.session = new SvelteDialogSession({
surface: this,
context,
dependencies: dependents,
dialogHost: DialogHost,
component,
initialData,
});
}
override onOpen(): void {
this.session.onOpen();
}
override onClose(): void {
this.session.onClose();
}
waitForClose(): Promise<T | undefined> {
return this.session.waitForClose();
}
}
export class BrowserSvelteDialogManager<T extends ServiceContext> extends SvelteDialogManagerBase<T> {
@@ -0,0 +1,110 @@
import type { LOG_LEVEL } from "@vrtmrz/livesync-commonlib/compat/common/logger";
import type { Confirm } from "@vrtmrz/livesync-commonlib/compat/interfaces/Confirm";
import type { ICommandCompat } from "@vrtmrz/livesync-commonlib/compat/services/base/IService";
import type { ServiceContext } from "@vrtmrz/livesync-commonlib/context";
import { InjectableAPIService } from "@vrtmrz/livesync-commonlib/compat/services/implements/injectable/InjectableAPIService";
import { FetchHttpHandler } from "@smithy/fetch-http-handler";
import { _fetch } from "@vrtmrz/livesync-commonlib/compat/common/coreEnvFunctions";
declare const MANIFEST_VERSION: string | undefined;
declare const PACKAGE_VERSION: string | undefined;
export interface LiveSyncBrowserAPIServiceOptions {
confirm: Confirm;
getSystemVaultName(): string;
appId?: string;
isMobile?: () => boolean;
fetch?: typeof _fetch;
addLog?: (message: unknown, level: LOG_LEVEL, key?: string) => void;
addCommand?: <TCommand extends ICommandCompat>(command: TCommand) => TCommand;
showWindow?: (type: string) => Promise<void>;
registerWindow?: <T>(type: string, factory: (leaf: T) => unknown) => void;
addRibbonIcon?: (icon: string, title: string, callback: (event: MouseEvent) => unknown) => HTMLElement;
registerProtocolHandler?: (action: string, handler: (params: Record<string, string>) => unknown) => void;
addStatusBarItem?: () => HTMLElement | undefined;
}
/** Browser application implementation of Commonlib's injected host API contract. */
export class LiveSyncBrowserAPIService<T extends ServiceContext> extends InjectableAPIService<T> {
private readonly options: LiveSyncBrowserAPIServiceOptions;
constructor(context: T, options: LiveSyncBrowserAPIServiceOptions) {
super(context);
this.options = options;
this.addLog.setHandler((message, level, key) => {
options.addLog?.(message, level, key);
});
}
get confirm(): Confirm {
return this.options.confirm;
}
getCustomFetchHandler(): FetchHttpHandler {
return new FetchHttpHandler();
}
isMobile(): boolean {
return this.options.isMobile?.() ?? false;
}
showWindow(type: string): Promise<void> {
return this.options.showWindow?.(type) ?? Promise.resolve();
}
getAppID(): string {
return this.options.appId ?? this.options.getSystemVaultName();
}
getSystemVaultName(): string {
return this.options.getSystemVaultName();
}
override getPlatform(): string {
return "browser";
}
getAppVersion(): string {
return MANIFEST_VERSION ?? "0.0.0";
}
getPluginVersion(): string {
return PACKAGE_VERSION ?? "0.0.0";
}
addCommand<TCommand extends ICommandCompat>(command: TCommand): TCommand {
return this.options.addCommand?.(command) ?? command;
}
registerWindow<T>(type: string, factory: (leaf: T) => unknown): void {
this.options.registerWindow?.(type, factory);
}
addRibbonIcon(
icon: string,
title: string,
callback: (event: MouseEvent) => unknown
): HTMLElement {
const element = this.options.addRibbonIcon?.(icon, title, callback);
if (!element) {
throw new Error("Ribbon icons are not supported by this browser application");
}
return element;
}
registerProtocolHandler(
action: string,
handler: (params: Record<string, string>) => unknown
): void {
this.options.registerProtocolHandler?.(action, handler);
}
override nativeFetch(request: string | Request, options?: RequestInit): Promise<Response> {
const fetchImplementation = this.options.fetch ?? _fetch;
return fetchImplementation(request, options);
}
addStatusBarItem(): HTMLElement | undefined {
return this.options.addStatusBarItem?.();
}
}
+14 -2
View File
@@ -1,14 +1,26 @@
import type { BrowserServiceHostDependencies } from "@vrtmrz/livesync-commonlib/compat/services/BrowserServices";
import type { ServiceContext } from "@vrtmrz/livesync-commonlib/context";
import type { AppLifecycleService } from "@vrtmrz/livesync-commonlib/compat/services/base/AppLifecycleService";
import type { ConfigService } from "@vrtmrz/livesync-commonlib/compat/services/base/ConfigService";
import type { ControlService } from "@vrtmrz/livesync-commonlib/compat/services/base/ControlService";
import type { ReplicatorService } from "@vrtmrz/livesync-commonlib/compat/services/base/ReplicatorService";
import type { InjectableAPIService } from "@vrtmrz/livesync-commonlib/compat/services/implements/injectable/InjectableAPIService";
import { UIService } from "@vrtmrz/livesync-commonlib/compat/services/implements/base/UIService";
import DialogToCopy from "@/modules/services/LiveSyncUI/dialogues/DialogueToCopy.svelte";
import { BrowserSvelteDialogManager } from "./BrowserSvelteDialogManager";
export interface LiveSyncBrowserUIServiceDependencies<T extends ServiceContext> {
API: InjectableAPIService<T>;
appLifecycle: AppLifecycleService<T>;
config: ConfigService<T>;
control: ControlService<T>;
replicator: ReplicatorService<T>;
}
export class LiveSyncBrowserUIService<T extends ServiceContext> extends UIService<T> {
override get dialogToCopy() {
return DialogToCopy;
}
constructor(context: T, dependents: BrowserServiceHostDependencies<T>) {
constructor(context: T, dependents: LiveSyncBrowserUIServiceDependencies<T>) {
const browserConfirm = dependents.API.confirm;
const obsidianSvelteDialogManager = new BrowserSvelteDialogManager<T>(context, {
appLifecycle: dependents.appLifecycle,
+158
View File
@@ -0,0 +1,158 @@
import type {
Confirm,
ConfirmActionLayout,
} from "@vrtmrz/livesync-commonlib/compat/interfaces/Confirm";
import type { UiInteractions, UiNotifications } from "@vrtmrz/ui-interactions";
const DEFAULT_LABELS = {
confirmationTitle: "Confirmation",
selectionTitle: "Select",
yes: "Yes",
no: "No",
} as const;
export interface UiInteractionsConfirmOptions {
ui: UiInteractions;
notifications: UiNotifications;
createActionAnchor: () => HTMLAnchorElement;
}
function timeoutOption(timeoutSeconds?: number): { timeoutMs?: number } {
return timeoutSeconds !== undefined && timeoutSeconds > 0
? { timeoutMs: timeoutSeconds * 1_000 }
: {};
}
/** Adapts Commonlib's legacy confirmation contract to Fancy Kit capabilities. */
export class UiInteractionsConfirm implements Confirm {
readonly notifications: UiNotifications;
constructor(private readonly options: UiInteractionsConfirmOptions) {
this.notifications = options.notifications;
}
async askYesNo(message: string): Promise<"yes" | "no"> {
const result = await this.options.ui.confirmAction(
{
title: DEFAULT_LABELS.confirmationTitle,
message,
actions: ["yes", "no"],
labels: { yes: DEFAULT_LABELS.yes, no: DEFAULT_LABELS.no },
defaultAction: "no",
actionLayout: "vertical",
},
"legacy-confirm.ask-yes-no"
);
return result === "yes" ? "yes" : "no";
}
async askString(
title: string,
key: string,
placeholder: string,
isPassword = false
): Promise<string | false> {
const prompt = isPassword
? this.options.ui.promptPassword.bind(this.options.ui)
: this.options.ui.promptText.bind(this.options.ui);
const result = await prompt(
{
title,
label: key,
placeholder,
},
"legacy-confirm.ask-string"
);
return result ?? false;
}
async askYesNoDialog(
message: string,
opt: { title?: string; defaultOption?: "Yes" | "No"; timeout?: number } = {}
): Promise<"yes" | "no"> {
const result = await this.options.ui.confirmAction(
{
title: opt.title ?? DEFAULT_LABELS.confirmationTitle,
message,
actions: ["Yes", "No"],
labels: { Yes: DEFAULT_LABELS.yes, No: DEFAULT_LABELS.no },
defaultAction: opt.defaultOption ?? "No",
actionLayout: "vertical",
...timeoutOption(opt.timeout),
},
"legacy-confirm.ask-yes-no-dialog"
);
return result === "Yes" ? "yes" : "no";
}
async askSelectString(message: string, items: string[]): Promise<string> {
const result = await this.options.ui.pickOne(
{
items,
getText: (item) => item,
placeholder: message,
},
"legacy-confirm.ask-select-string"
);
return result ?? "";
}
async askSelectStringDialogue<T extends readonly string[]>(
message: string,
buttons: T,
opt: { title?: string; defaultAction: T[number]; timeout?: number }
): Promise<T[number] | false> {
const result = await this.options.ui.confirmAction(
{
title: opt.title ?? DEFAULT_LABELS.selectionTitle,
message,
actions: buttons,
defaultAction: opt.defaultAction,
actionLayout: "vertical",
...timeoutOption(opt.timeout),
},
"legacy-confirm.ask-select-string-dialogue"
);
return result ?? false;
}
askInPopup(
key: string,
dialogText: string,
anchorCallback: (anchor: HTMLAnchorElement) => void,
durationMs?: number
): void {
const anchor = this.options.createActionAnchor();
anchorCallback(anchor);
this.options.notifications.show(key, {
message: dialogText.replace("{HERE}", "").trim(),
action: {
label: anchor.textContent?.trim() || "Open",
onSelect: () => anchor.click(),
},
durationMs,
});
}
async confirmWithMessage(
title: string,
contentMd: string,
buttons: string[],
defaultAction: (typeof buttons)[number],
timeout?: number,
actionLayout?: ConfirmActionLayout
): Promise<(typeof buttons)[number] | false> {
const result = await this.options.ui.confirmAction(
{
title,
message: contentMd,
actions: buttons,
defaultAction,
actionLayout: actionLayout ?? "vertical",
...timeoutOption(timeout),
},
"legacy-confirm.confirm-with-message"
);
return result ?? false;
}
}
@@ -1,37 +1,215 @@
import { BrowserServiceHub, type BrowserServiceHost } from "@vrtmrz/livesync-commonlib/compat/services/BrowserServices";
import type { ObsidianLiveSyncSettings } from "@vrtmrz/livesync-commonlib/compat/common/types";
import type { KeyValueDatabaseFactory } from "@vrtmrz/livesync-commonlib/compat/interfaces/KeyValueDatabase";
import { PouchDB } from "@vrtmrz/livesync-commonlib/compat/pouchdb/pouchdb-browser";
import { ConfigService } from "@vrtmrz/livesync-commonlib/compat/services/base/ConfigService";
import { ControlService } from "@vrtmrz/livesync-commonlib/compat/services/base/ControlService";
import { DatabaseService } from "@vrtmrz/livesync-commonlib/compat/services/base/DatabaseService";
import { KeyValueDBService } from "@vrtmrz/livesync-commonlib/compat/services/base/KeyValueDBService";
import type { ISettingService } from "@vrtmrz/livesync-commonlib/compat/services/base/IService";
import { ServiceContext } from "@vrtmrz/livesync-commonlib/context";
import { BrowserAPIService } from "@vrtmrz/livesync-commonlib/compat/services/implements/browser/BrowserAPIService";
import { BrowserConfirm } from "./BrowserConfirm";
import { LiveSyncBrowserUIService } from "./LiveSyncBrowserUIService";
import { InjectableAppLifecycleService } from "@vrtmrz/livesync-commonlib/compat/services/implements/injectable/InjectableAppLifecycleService";
import { InjectableConflictService } from "@vrtmrz/livesync-commonlib/compat/services/implements/injectable/InjectableConflictService";
import { InjectableDatabaseEventService } from "@vrtmrz/livesync-commonlib/compat/services/implements/injectable/InjectableDatabaseEventService";
import { InjectableFileProcessingService } from "@vrtmrz/livesync-commonlib/compat/services/implements/injectable/InjectableFileProcessingService";
import { PathServiceCompat } from "@vrtmrz/livesync-commonlib/compat/services/implements/injectable/InjectablePathService";
import { InjectableRemoteService } from "@vrtmrz/livesync-commonlib/compat/services/implements/injectable/InjectableRemoteService";
import { InjectableReplicationService } from "@vrtmrz/livesync-commonlib/compat/services/implements/injectable/InjectableReplicationService";
import { InjectableReplicatorService } from "@vrtmrz/livesync-commonlib/compat/services/implements/injectable/InjectableReplicatorService";
import { InjectableServiceHub } from "@vrtmrz/livesync-commonlib/compat/services/implements/injectable/InjectableServiceHub";
import { InjectableSettingService } from "@vrtmrz/livesync-commonlib/compat/services/implements/injectable/InjectableSettingService";
import { InjectableTestService } from "@vrtmrz/livesync-commonlib/compat/services/implements/injectable/InjectableTestService";
import { InjectableTweakValueService } from "@vrtmrz/livesync-commonlib/compat/services/implements/injectable/InjectableTweakValueService";
import { InjectableVaultServiceCompat } from "@vrtmrz/livesync-commonlib/compat/services/implements/injectable/InjectableVaultService";
import { setLang, translateLiveSyncMessage } from "@/common/translation";
import { BrowserConfirm } from "./BrowserConfirm";
import { createBrowserKeyValueDatabaseFactory } from "./BrowserKeyValueDatabase";
import {
LiveSyncBrowserAPIService,
type LiveSyncBrowserAPIServiceOptions,
} from "./LiveSyncBrowserAPIService";
import { LiveSyncBrowserUIService } from "./LiveSyncBrowserUIService";
export type LiveSyncBrowserServiceHubOptions<T extends ServiceContext> = {
export interface LiveSyncBrowserSettingsPersistence {
load(): Promise<ObsidianLiveSyncSettings | undefined>;
save(settings: ObsidianLiveSyncSettings): Promise<void>;
}
export interface LiveSyncBrowserRestartPolicy {
schedule(): void;
perform?: () => void;
ask?: (message?: string) => void;
isScheduled?: () => boolean;
}
export interface LiveSyncBrowserServiceHubOptions<T extends ServiceContext> {
context?: T;
getSystemVaultName?: () => string;
settings?: LiveSyncBrowserSettingsPersistence;
restart?: LiveSyncBrowserRestartPolicy;
openKeyValueDatabase?: KeyValueDatabaseFactory;
};
API?: Omit<LiveSyncBrowserAPIServiceOptions, "confirm" | "getSystemVaultName">;
}
function createLiveSyncBrowserHost<T extends ServiceContext>(): BrowserServiceHost<T> {
return {
createAPI(context) {
return new BrowserAPIService(context, {
confirm: new BrowserConfirm(context),
});
},
createUI(context, dependencies) {
return new LiveSyncBrowserUIService(context, dependencies);
},
};
class LiveSyncBrowserAppLifecycleService<
T extends ServiceContext,
> extends InjectableAppLifecycleService<T> {}
class LiveSyncBrowserDatabaseService<T extends ServiceContext> extends DatabaseService<T> {}
class LiveSyncBrowserKeyValueDBService<T extends ServiceContext> extends KeyValueDBService<T> {}
class LiveSyncBrowserConfigService<T extends ServiceContext> extends ConfigService<T> {
constructor(
context: T,
private readonly setting: ISettingService
) {
super(context);
}
getSmallConfig(key: string): string | null {
return this.setting.getSmallConfig(key);
}
setSmallConfig(key: string, value: string): void {
this.setting.setSmallConfig(key, value);
}
deleteSmallConfig(key: string): void {
this.setting.deleteSmallConfig(key);
}
}
/** LiveSync-owned service composition shared by WebApp and WebPeer. */
export class LiveSyncBrowserServiceHub<T extends ServiceContext> extends InjectableServiceHub<T> {
constructor(options: LiveSyncBrowserServiceHubOptions<T>) {
const context =
options.context ??
(new ServiceContext({
translate: translateLiveSyncMessage,
}) as T);
const API = new LiveSyncBrowserAPIService(context, {
...options.API,
confirm: new BrowserConfirm(context),
getSystemVaultName: options.getSystemVaultName ?? (() => "livesync-browser"),
});
const conflict = new InjectableConflictService(context);
const fileProcessing = new InjectableFileProcessingService(context);
const setting = new InjectableSettingService(context, {
APIService: API,
onDisplayLanguageChanged: setLang,
});
const settingsPersistence = options.settings;
setting.loadData.setHandler(
settingsPersistence
? () => settingsPersistence.load()
: () => Promise.resolve(undefined)
);
setting.saveData.setHandler(
settingsPersistence
? (settings) => settingsPersistence.save(settings)
: () => Promise.resolve()
);
const appLifecycle = new LiveSyncBrowserAppLifecycleService(context, {
settingService: setting,
});
const restartPolicy = options.restart;
const scheduleRestart = restartPolicy ? () => restartPolicy.schedule() : () => {};
appLifecycle.scheduleRestart.setHandler(scheduleRestart);
appLifecycle.performRestart.setHandler(
restartPolicy?.perform ? () => restartPolicy.perform?.() : scheduleRestart
);
appLifecycle.askRestart.setHandler(
restartPolicy?.ask ? (message) => restartPolicy.ask?.(message) : scheduleRestart
);
appLifecycle.isReloadingScheduled.setHandler(
restartPolicy?.isScheduled ? () => restartPolicy.isScheduled?.() ?? false : () => false
);
const databaseEvents = new InjectableDatabaseEventService(context);
const path = new PathServiceCompat(context, {
settingService: setting,
});
const vault = new InjectableVaultServiceCompat(context, {
settingService: setting,
APIService: API,
});
const database = new LiveSyncBrowserDatabaseService(context, {
pouchDB: PouchDB,
path,
vault,
setting,
API,
});
const config = new LiveSyncBrowserConfigService(context, setting);
const replicator = new InjectableReplicatorService(context, {
settingService: setting,
appLifecycleService: appLifecycle,
databaseEventService: databaseEvents,
});
const remote = new InjectableRemoteService(context, {
pouchDB: PouchDB,
APIService: API,
appLifecycle,
setting,
});
const replication = new InjectableReplicationService(context, {
APIService: API,
appLifecycleService: appLifecycle,
replicatorService: replicator,
settingService: setting,
fileProcessingService: fileProcessing,
databaseService: database,
});
const keyValueDB = new LiveSyncBrowserKeyValueDBService(context, {
openKeyValueDatabase:
options.openKeyValueDatabase ?? createBrowserKeyValueDatabaseFactory(),
appLifecycle,
databaseEvents,
vault,
});
const control = new ControlService(context, {
appLifecycleService: appLifecycle,
databaseService: database,
fileProcessingService: fileProcessing,
settingService: setting,
APIService: API,
replicatorService: replicator,
});
const ui = new LiveSyncBrowserUIService(context, {
API,
appLifecycle,
config,
control,
replicator,
});
super(context, {
API,
appLifecycle,
conflict,
config,
control,
database,
databaseEvents,
fileProcessing,
keyValueDB,
path,
remote,
replication,
replicator,
setting,
test: new InjectableTestService(context),
tweakValue: new InjectableTweakValueService(context),
ui,
vault,
});
}
}
export function createLiveSyncBrowserServiceHub<T extends ServiceContext>(
options: LiveSyncBrowserServiceHubOptions<T> = {}
): BrowserServiceHub<T> {
const context = options.context ?? (new ServiceContext({ translate: translateLiveSyncMessage }) as T);
return new BrowserServiceHub<T>({
...options,
context,
onDisplayLanguageChanged: setLang,
host: createLiveSyncBrowserHost<T>(),
});
): LiveSyncBrowserServiceHub<T> {
return new LiveSyncBrowserServiceHub(options);
}
@@ -1,12 +1,20 @@
import { createServiceContext } from "@vrtmrz/livesync-commonlib/context";
import { describe, expect, it } from "vitest";
import { DEFAULT_SETTINGS } from "@vrtmrz/livesync-commonlib/compat/common/types";
import { afterEach, describe, expect, it, vi } from "vitest";
import {
observeServiceComposition,
observeServiceContext,
SERVICE_CONTEXT_MEMBERS,
} from "../../../test/contracts/serviceContext";
import { createLiveSyncBrowserServiceHub } from "./createLiveSyncBrowserServiceHub";
import {
createLiveSyncBrowserServiceHub,
type LiveSyncBrowserServiceHubOptions,
} from "./createLiveSyncBrowserServiceHub";
afterEach(() => {
vi.unstubAllGlobals();
});
describe("LiveSync browser service context contract", () => {
it("preserves one injected context and its API results throughout the Webapp composition", () => {
@@ -25,4 +33,57 @@ describe("LiveSync browser service context contract", () => {
[]
);
});
it("binds browser identity, persistence, restart policy, and native Fetch while composing the hub", async () => {
const deviceLocalSettings = new Map<string, string>();
vi.stubGlobal("localStorage", {
getItem: (key: string) => deviceLocalSettings.get(key) ?? null,
setItem: (key: string, value: string) => deviceLocalSettings.set(key, value),
removeItem: (key: string) => deviceLocalSettings.delete(key),
clear: () => deviceLocalSettings.clear(),
});
const savedSettings: (typeof DEFAULT_SETTINGS)[] = [];
const restartCalls: string[] = [];
const nativeFetch = vi.fn(async () => new Response("ok"));
const options: LiveSyncBrowserServiceHubOptions<ReturnType<typeof createServiceContext>> = {
getSystemVaultName: () => "browser-vault",
settings: {
load: async () => ({
...DEFAULT_SETTINGS,
couchDB_DBNAME: "browser-database",
}),
save: async (settings) => {
savedSettings.push(settings);
},
},
restart: {
schedule: () => restartCalls.push("schedule"),
perform: () => restartCalls.push("perform"),
ask: (message) => restartCalls.push(`ask:${message ?? ""}`),
isScheduled: () => true,
},
API: {
fetch: nativeFetch as typeof fetch,
},
};
const hub = createLiveSyncBrowserServiceHub(options);
expect(hub.API.getSystemVaultName()).toBe("browser-vault");
expect(hub.API.getPlatform()).toBe("browser");
const response = await hub.API.nativeFetch("https://example.invalid/");
expect(await response.text()).toBe("ok");
expect(nativeFetch).toHaveBeenCalledWith("https://example.invalid/", undefined);
await hub.setting.loadSettings();
expect(hub.setting.currentSettings().couchDB_DBNAME).toBe("browser-database");
savedSettings.length = 0;
await hub.setting.saveSettingData();
expect(savedSettings).toHaveLength(1);
hub.appLifecycle.scheduleRestart();
hub.appLifecycle.performRestart();
hub.appLifecycle.askRestart("restart now");
expect(hub.appLifecycle.isReloadingScheduled()).toBe(true);
expect(restartCalls).toEqual(["schedule", "perform", "ask:restart now"]);
});
});
-137
View File
@@ -1,137 +0,0 @@
<script lang="ts">
import { renderMessageMarkdown } from "./renderMessageMarkdown";
type Props = {
title: string;
message: string;
buttons: string[];
actionLayout?: ConfirmActionLayout;
commit: (button: string) => void;
};
type ConfirmActionLayout = "auto" | "vertical";
let { title, message, buttons, actionLayout, commit }: Props = $props();
const renderedMessage = $derived(renderMessageMarkdown(message));
function handleEsc(event: KeyboardEvent) {
if (event.key === "Escape") {
commit("");
}
}
</script>
<popup>
<header>{title}</header>
<article><div class="msg">{@html renderedMessage}</div></article>
<div class:vertical={actionLayout === "vertical"} class="buttons">
{#each buttons as button}
<button onclick={() => commit(button)}>{button}</button>
{/each}
</div>
</popup>
<div class="background" onclick={() => commit("")} onkeydown={handleEsc} role="none"></div>
<style>
popup {
z-index: 1000;
position: fixed;
background: rgba(255, 255, 255, 0.8);
max-width: 70vw;
max-height: 80vh;
margin: auto;
top: 0;
left: 0;
right: 0;
bottom: 0;
min-width: 50vw;
min-height: 50vh;
backdrop-filter: blur(5px);
display: flex;
flex-direction: column;
border-radius: 5px;
box-shadow: 0 0 10px rgba(0, 0, 0, 0.5);
border: 1px solid var(--background-primary-alt);
justify-content: space-between;
}
popup header {
background: var(--background-primary);
color: var(--text-normal);
padding: 1em;
border-bottom: 1px solid var(--background-primary-alt);
font: size 1.4em;
}
popup article {
padding: 1em;
display: flex;
justify-content: center;
overflow-y: auto;
}
popup article .msg {
width: 100%;
line-height: 1.5;
overflow-wrap: anywhere;
}
popup article .msg :global(:first-child) {
margin-top: 0;
}
popup article .msg :global(:last-child) {
margin-bottom: 0;
}
popup article .msg :global(pre) {
overflow-x: auto;
padding: 0.75em;
border-radius: 4px;
background: var(--background-secondary);
}
popup article .msg :global(code) {
font-family: var(--font-monospace);
}
popup article .msg :global(blockquote) {
margin: 0;
padding-left: 1em;
border-left: 3px solid var(--background-modifier-border);
color: var(--text-muted);
}
popup article .msg :global(ul),
popup article .msg :global(ol) {
padding-left: 1.5em;
}
popup article .msg :global(table) {
width: 100%;
border-collapse: collapse;
}
popup article .msg :global(th),
popup article .msg :global(td) {
padding: 0.4em 0.6em;
border: 1px solid var(--background-modifier-border);
}
popup article .msg :global(a) {
color: var(--text-accent);
}
popup .buttons {
border-top: 1px solid var(--background-primary-alt);
display: flex;
justify-content: center;
align-items: center;
padding: 1em;
}
popup .buttons button {
margin: 0 0.5em;
background-color: var(--background-primary-alt);
}
popup .buttons.vertical {
align-items: stretch;
flex-direction: column;
}
popup .buttons.vertical button {
margin: 0.25em 0;
width: 100%;
}
popup ~ .background {
position: fixed;
top: 0;
left: 0;
right: 0;
bottom: 0;
background: rgba(0, 0, 0, 0.125);
}
</style>
-126
View File
@@ -1,126 +0,0 @@
<script lang="ts">
type Props = {
title: string;
message: string;
initialText?: string;
placeholder?: string;
isPassword?: boolean;
commit: (text: string | false) => void;
};
const { title, message, commit, initialText, placeholder, isPassword }: Props = $props();
function initialTextSeed(): string {
return initialText ?? "";
}
let text = $state(initialTextSeed());
const type = $derived(isPassword ? "password" : "text");
function cancel() {
commit(false);
}
function handleKey(event: KeyboardEvent) {
if (event.key === "Escape") {
handleCancel(event);
} else if (event.key === "Enter") {
handleCommit(event);
}
}
function handleCancel(event: KeyboardEvent | MouseEvent) {
cancel();
event.preventDefault();
}
function handleCommit(event: KeyboardEvent | MouseEvent) {
commit(text);
event.preventDefault();
}
let textEl: HTMLInputElement;
$effect(() => {
textEl.focus();
});
</script>
<popup>
<header>{title}</header>
<article>
<div class="msg">{message}</div>
<div class="input">
<input
bind:this={textEl}
{type}
bind:value={text}
{placeholder}
onkeydown={handleKey}
onkeyup={handleKey}
/>
</div>
</article>
<div class="buttons">
<button onclick={handleCommit}>OK</button>
<button onclick={handleCancel}>Cancel</button>
</div>
</popup>
<div class="background" onclick={handleCancel} onkeydown={handleKey} role="none"></div>
<style>
popup {
z-index: 1000;
position: fixed;
background: rgba(255, 255, 255, 0.8);
max-width: 70vw;
max-height: 80vh;
margin: auto;
top: 0;
left: 0;
right: 0;
bottom: 0;
min-width: 50vw;
min-height: 50vh;
backdrop-filter: blur(5px);
display: flex;
flex-direction: column;
border-radius: 5px;
box-shadow: 0 0 10px rgba(0, 0, 0, 0.5);
border: 1px solid var(--background-primary-alt);
justify-content: space-between;
}
popup header {
background: var(--background-primary);
color: var(--text-normal);
padding: 1em;
border-bottom: 1px solid var(--background-primary-alt);
font: size 1.4em;
}
popup article {
align-items: center;
padding: 1em;
display: flex;
justify-content: center;
flex-direction: column;
}
popup article .msg {
overflow-y: auto;
white-space: pre-wrap;
}
popup .buttons {
border-top: 1px solid var(--background-primary-alt);
display: flex;
justify-content: center;
align-items: center;
padding: 1em;
}
popup .buttons button {
margin: 0 0.5em;
background-color: var(--background-primary-alt);
}
popup ~ .background {
position: fixed;
top: 0;
left: 0;
right: 0;
bottom: 0;
background: rgba(0, 0, 0, 0.125);
}
</style>
@@ -19,3 +19,13 @@ markdownRenderer.renderer.rules.link_open = (tokens, idx, options, env, self) =>
export function renderMessageMarkdown(message: string): string {
return markdownRenderer.render(message);
}
export function renderMessageMarkdownInto(container: HTMLElement, message: string): void {
const DOMParserConstructor = container.ownerDocument.defaultView?.DOMParser;
if (!DOMParserConstructor) {
container.textContent = message;
return;
}
const parsed = new DOMParserConstructor().parseFromString(renderMessageMarkdown(message), "text/html");
container.replaceChildren(...Array.from(parsed.body.childNodes));
}
+2 -2
View File
@@ -2,8 +2,8 @@
* Native DOM creation for browser applications which run outside Obsidian.
*
* Obsidian adds creation helpers to its own DOM environment. The standalone
* Webapp and WebPeer hosts do not own those prototype extensions, and the
* Webapp compatibility layer implements them on top of this native boundary.
* WebApp and WebPeer hosts use this native boundary instead of installing
* Obsidian-shaped prototype extensions.
*/
type NativeDocumentCreation = Pick<Document, "createElement" | "createDocumentFragment">;
+3 -3
View File
@@ -101,9 +101,9 @@ COPY --from=runtime-deps /deps/node_modules ./node_modules
# Copy the built CLI bundle from builder stage
COPY --from=builder /build/src/apps/cli/dist ./dist
# Install entrypoint wrapper
COPY src/apps/cli/docker-entrypoint.sh /usr/local/bin/livesync-cli
RUN chmod +x /usr/local/bin/livesync-cli
# Install the entrypoint wrapper with a deterministic mode, regardless of
# source checkout permissions.
COPY --chmod=755 src/apps/cli/docker-entrypoint.sh /usr/local/bin/livesync-cli
# Mount your vault / local database directory here
VOLUME ["/data"]
+17 -3
View File
@@ -6,7 +6,7 @@ import {
type FilePathWithPrefix,
type ObsidianLiveSyncSettings,
REMOTE_COUCHDB,
REMOTE_MINIO,
isJournalRemoteType,
type EntryMilestoneInfo,
type EntryDoc,
} from "@vrtmrz/livesync-commonlib/compat/common/types";
@@ -25,6 +25,8 @@ import { compatGlobal } from "@vrtmrz/livesync-commonlib/compat/common/coreEnvFu
import { fsPromises as fs, path } from "@vrtmrz/livesync-commonlib/node";
import type { LiveSyncCouchDBReplicator } from "@vrtmrz/livesync-commonlib/compat/replication/couchdb/LiveSyncReplicator";
import type { LiveSyncJournalReplicator } from "@vrtmrz/livesync-commonlib/compat/replication/journal/LiveSyncJournalReplicator";
import type { JournalSyncCore } from "@vrtmrz/livesync-commonlib/compat/replication/journal/JournalSyncCore";
import { journalProtocolConfigurationForSettings } from "@vrtmrz/livesync-commonlib/journal-storage";
import { writeStderrLine, writeStdoutLine } from "@/apps/cli/cliOutput";
function redactConnectionString(uri: string): string {
@@ -59,8 +61,20 @@ async function verifyRemoteState(
return false;
}
milestone = await dbRet.db.get(MILESTONE_DOCID);
} else if (settings.remoteType === REMOTE_MINIO) {
milestone = await (replicator as LiveSyncJournalReplicator).client.downloadJson("_00000000-milestone.json");
} else if (isJournalRemoteType(settings.remoteType)) {
const journalReplicator = replicator as LiveSyncJournalReplicator;
if (journalProtocolConfigurationForSettings(settings).journalFormat === "adaptive-v1") {
try {
await journalReplicator.client.ensureCheckpointCachesAreFresh();
standardIo.writeStderr("[Verification] Adaptive Journal repository is available.\n");
return true;
} catch (error) {
const message = error instanceof Error ? error.message : String(error);
standardIo.writeStderr(`[Verification] Failed to verify Adaptive Journal repository: ${message}\n`);
return false;
}
}
milestone = await (journalReplicator.client as JournalSyncCore).downloadJson("_00000000-milestone.json");
}
if (milestone) {
+61 -1
View File
@@ -2,7 +2,13 @@ import { fsPromises as fs, os, path } from "@vrtmrz/livesync-commonlib/node";
import * as processSetting from "@vrtmrz/livesync-commonlib/compat/API/processSetting";
import { ConnectionStringParser } from "@vrtmrz/livesync-commonlib/compat/common/ConnectionString";
import { configURIBase } from "@vrtmrz/livesync-commonlib/compat/common/models/shared.const";
import { DEFAULT_SETTINGS, REMOTE_COUCHDB, REMOTE_MINIO, REMOTE_P2P } from "@vrtmrz/livesync-commonlib/compat/common/types";
import {
DEFAULT_SETTINGS,
REMOTE_COUCHDB,
REMOTE_MINIO,
REMOTE_P2P,
REMOTE_WEBDAV,
} from "@vrtmrz/livesync-commonlib/compat/common/types";
import { describe, expect, it, vi, beforeEach, afterEach } from "vitest";
import { runCommand } from "./runCommand";
import type { CLIOptions } from "./types";
@@ -717,6 +723,60 @@ describe("runCommand abnormal cases", () => {
expect(core.services.control.applySettings).not.toHaveBeenCalled();
});
it("verifies an Adaptive Journal repository without reading the legacy milestone", async () => {
const core = createCoreMock();
const settings = core.services.setting.currentSettings();
settings.remoteType = REMOTE_MINIO;
settings.journalFormat = "adaptive-v1";
settings.packReadPolicy = "whole-pack";
const ensureCheckpointCachesAreFresh = vi.fn(async () => {});
core.services.replicator.getActiveReplicator.mockReturnValue({
nodeid: "test-node-id",
initializeDatabaseForReplication: vi.fn(async () => {}),
client: {
ensureCheckpointCachesAreFresh,
},
});
const result = await runCommand(makeOptions("mark-resolved", []), {
...context,
core,
});
expect(result).toBe(true);
expect(ensureCheckpointCachesAreFresh).toHaveBeenCalledTimes(1);
expect(core.services.context.standardIo.writeStderr).toHaveBeenCalledWith(
"[Verification] Adaptive Journal repository is available.\n"
);
});
it("uses the Adaptive verification path for a WebDAV remote", async () => {
const core = createCoreMock();
const settings = core.services.setting.currentSettings();
settings.remoteType = REMOTE_WEBDAV;
settings.webDAVactiveConnectionURI = "sls+webdav://dav.example/dav";
settings.journalFormat = "adaptive-v1";
settings.packReadPolicy = "whole-pack";
const ensureCheckpointCachesAreFresh = vi.fn(async () => {});
core.services.replicator.getActiveReplicator.mockReturnValue({
nodeid: "test-node-id",
initializeDatabaseForReplication: vi.fn(async () => {}),
client: {
ensureCheckpointCachesAreFresh,
},
});
const result = await runCommand(makeOptions("mark-resolved", []), {
...context,
core,
});
expect(result).toBe(true);
expect(ensureCheckpointCachesAreFresh).toHaveBeenCalledTimes(1);
});
it("mark-resolved with remote-id temporarily activates it and runs markResolved", async () => {
const core = createCoreMock();
const settings = core.services.setting.currentSettings();
+11
View File
@@ -0,0 +1,11 @@
import { readFileSync } from "node:fs";
import { describe, expect, it } from "vitest";
const dockerfile = readFileSync(new URL("./Dockerfile", import.meta.url), "utf8");
describe("CLI Docker image", () => {
it("sets a deterministic readable and executable entrypoint mode", () => {
expect(dockerfile).toContain("COPY --chmod=755 src/apps/cli/docker-entrypoint.sh /usr/local/bin/livesync-cli");
expect(dockerfile).not.toContain("RUN chmod +x /usr/local/bin/livesync-cli");
});
});
+27 -4
View File
@@ -37,6 +37,14 @@ defaultLoggerEnv.minLogLevel = LOG_LEVEL_DEBUG;
/** Injectable command boundary used by CLI integration probes. */
export type CliCommandRunner = (options: CLIOptions, context: CLICommandContext) => Promise<boolean>;
const SETTINGS_MANAGEMENT_COMMANDS: ReadonlySet<CLICommand> = new Set([
"setup",
"remote-add",
"remote-rm",
"remote-set",
"remote-activate",
]);
function printHelp(standardIo: StandardIo): void {
writeStdoutLine(
standardIo,
@@ -274,7 +282,10 @@ export async function main(
) {
const options = parseArgs(standardIo);
if (options.interval && options.command !== "daemon") {
writeStderrLine(standardIo, `Warning: --interval is only used in daemon mode, ignored for '${options.command}'`);
writeStderrLine(
standardIo,
`Warning: --interval is only used in daemon mode, ignored for '${options.command}'`
);
}
const avoidStdoutNoise =
options.command === "cat" ||
@@ -399,12 +410,15 @@ export async function main(
if (!options.verbose) return;
}
writeStderrLine(standardIo, prefix, message);
});
}, true);
// Prevent replication result from being processed automatically in non-daemon commands.
// In daemon mode the default handler must run so changes are applied to the filesystem.
if (options.command !== "daemon") {
serviceHubInstance.replication.processSynchroniseResult.addHandler(async () => {
writeStderrLine(standardIo, `[Info] Replication result received, but not processed automatically in CLI mode.`);
writeStderrLine(
standardIo,
`[Info] Replication result received, but not processed automatically in CLI mode.`
);
return await Promise.resolve(true);
}, -100);
}
@@ -506,7 +520,7 @@ export async function main(
// Save the settings file before any lifecycle events can mutate and persist them.
// suspendAllSync and other lifecycle hooks clobber sync settings in memory, and
// various code paths persist the clobbered state to disk. We restore on shutdown.
const settingsBackup = await fs.readFile(settingsPath, "utf-8").catch(() => null!);
let settingsBackup: string | null = await fs.readFile(settingsPath, "utf-8").catch(() => null);
// Restore settings file on any exit to undo lifecycle mutations.
// Write to a temp path first so a crash mid-write doesn't leave a truncated file.
@@ -576,6 +590,15 @@ export async function main(
settingsPath,
originalSyncSettings,
});
if (result && SETTINGS_MANAGEMENT_COMMANDS.has(options.command)) {
// Settings management is intentional, unlike the temporary changes made by suspendAllSync().
// setup replaces the complete configuration, while remote profile commands must retain the
// synchronisation mode which was active before the command lifecycle suspended it.
if (options.command !== "setup") {
await core.services.setting.applyPartial(originalSyncSettings, true);
}
settingsBackup = await fs.readFile(settingsPath, "utf-8");
}
if (!result) {
writeStderrLine(standardIo, `[Error] Command '${options.command}' failed`);
process.exitCode = 1;
+6 -2
View File
@@ -1,7 +1,7 @@
{
"name": "self-hosted-livesync-cli",
"private": true,
"version": "1.0.0-beta.0-cli",
"version": "1.0.2-cli",
"main": "dist/index.cjs",
"type": "module",
"scripts": {
@@ -22,6 +22,10 @@
"pretest:e2e:ci": "npm run build",
"test:e2e:ci": "deno task --cwd testdeno test:ci",
"test:e2e:p2p": "deno task --cwd testdeno test:p2p:compose",
"pretest:e2e:adaptive-s3": "npm run build",
"test:e2e:adaptive-s3": "deno task --cwd testdeno test:adaptive-journal-s3",
"pretest:e2e:adaptive-webdav": "npm run build",
"test:e2e:adaptive-webdav": "deno task --cwd testdeno test:adaptive-journal-webdav",
"test:e2e:mirror": "bash test/test-mirror-linux.sh",
"test:e2e:remote-commands": "bash test/test-remote-commands-linux.sh",
"pretest:e2e:all": "npm run build",
@@ -37,7 +41,7 @@
"dependencies": {
"chokidar": "^4.0.0",
"minimatch": "^10.2.5",
"octagonal-wheels": "^0.1.51",
"octagonal-wheels": "^0.1.52",
"pouchdb-adapter-http": "^9.0.0",
"pouchdb-adapter-leveldb": "^9.0.0",
"pouchdb-core": "^9.0.0",
@@ -25,6 +25,10 @@ type SerializableContainer =
| {
[NODE_KV_TYPED_KEY]: "ArrayBuffer";
[NODE_KV_VALUES_KEY]: number[];
}
| {
[NODE_KV_TYPED_KEY]: "BigInt";
[NODE_KV_VALUES_KEY]: string;
};
function isRecord(value: unknown): value is Record<string, unknown> {
@@ -32,6 +36,12 @@ function isRecord(value: unknown): value is Record<string, unknown> {
}
function serializeForNodeKV(value: unknown): unknown {
if (typeof value === "bigint") {
return {
[NODE_KV_TYPED_KEY]: "BigInt",
[NODE_KV_VALUES_KEY]: value.toString(10),
} satisfies SerializableContainer;
}
if (value instanceof Set) {
return {
[NODE_KV_TYPED_KEY]: "Set",
@@ -78,6 +88,9 @@ function deserializeFromNodeKV(value: unknown): unknown {
if (taggedType === "ArrayBuffer" && Array.isArray(taggedValues)) {
return Uint8Array.from(taggedValues).buffer;
}
if (taggedType === "BigInt" && typeof taggedValues === "string" && /^-?(?:0|[1-9]\d*)$/u.test(taggedValues)) {
return BigInt(taggedValues);
}
return Object.fromEntries(Object.entries(value).map(([k, v]) => [k, deserializeFromNodeKV(v)]));
}
@@ -1,8 +1,39 @@
import { describe, expect, it, vi } from "vitest";
import { createServiceContext } from "@vrtmrz/livesync-commonlib/compat/services/base/ServiceBase";
import { fsPromises as fs, os, path } from "@vrtmrz/livesync-commonlib/node";
import type { NodeKeyValueDBDependencies } from "./NodeKeyValueDBService";
import { NodeKeyValueDBService } from "./NodeKeyValueDBService";
function createInitialisableDependencies(): {
dependencies: NodeKeyValueDBDependencies;
initialise: () => Promise<boolean>;
} {
let initialise: (() => Promise<boolean>) | undefined;
const dependencies = {
appLifecycle: {
onSettingLoaded: {
addHandler: vi.fn((handler: () => Promise<boolean>) => {
initialise = handler;
}),
},
},
databaseEvents: {
onResetDatabase: { addHandler: vi.fn() },
onDatabaseInitialisation: { addHandler: vi.fn() },
onUnloadDatabase: { addHandler: vi.fn() },
onCloseDatabase: { addHandler: vi.fn() },
},
vault: {},
} as unknown as NodeKeyValueDBDependencies;
return {
dependencies,
initialise: async () => {
if (!initialise) throw new Error("Initialisation handler was not registered");
return await initialise();
},
};
}
describe("NodeKeyValueDBService.openSimpleStore", () => {
it("creates a namespaced store handle before the backing database is initialised", () => {
const dependencies = {
@@ -44,4 +75,29 @@ describe("NodeKeyValueDBService.openSimpleStore", () => {
await expect(store.get("key")).rejects.toThrow("KeyValueDB is not initialized yet");
});
it("preserves bigint values used by Adaptive Journal state", async () => {
const tempDir = await fs.mkdtemp(path.join(os.tmpdir(), "livesync-node-kv-bigint-"));
const filePath = path.join(tempDir, "keyvalue-db.json");
const writerState = {
lastCommittedSequence: 9007199254740993n,
pendingCommit: { sequence: 18446744073709551615n },
writerEpoch: "test-writer-epoch",
};
try {
const firstLifecycle = createInitialisableDependencies();
const first = new NodeKeyValueDBService(createServiceContext(), firstLifecycle.dependencies, filePath);
await expect(firstLifecycle.initialise()).resolves.toBe(true);
await first.openSimpleStore("adaptive").set("writer-state", writerState);
const secondLifecycle = createInitialisableDependencies();
const second = new NodeKeyValueDBService(createServiceContext(), secondLifecycle.dependencies, filePath);
await expect(secondLifecycle.initialise()).resolves.toBe(true);
await expect(second.openSimpleStore("adaptive").get("writer-state")).resolves.toEqual(writerState);
} finally {
await fs.rm(tempDir, { recursive: true, force: true });
}
});
});
@@ -0,0 +1,11 @@
import { readFileSync } from "node:fs";
import { describe, expect, it } from "vitest";
const setupPutCatHelper = readFileSync(new URL("./test/test-setup-put-cat-linux.sh", import.meta.url), "utf8");
describe("CLI setup URI E2E helper", () => {
it("evaluates Commonlib package imports as ESM", () => {
expect(setupPutCatHelper).toContain("node --input-type=module -e");
expect(setupPutCatHelper).not.toContain("npx tsx -e");
});
});
@@ -27,7 +27,7 @@ cli_test_init_settings_file "$SETTINGS_FILE"
echo "[INFO] creating setup URI from settings"
SETUP_URI="$(
SETTINGS_FILE="$SETTINGS_FILE" SETUP_PASSPHRASE="$SETUP_PASSPHRASE" npx tsx -e '
SETTINGS_FILE="$SETTINGS_FILE" SETUP_PASSPHRASE="$SETUP_PASSPHRASE" node --input-type=module -e '
import { fs } from "@vrtmrz/livesync-commonlib/node";
import { encodeSettingsToSetupURI } from "@vrtmrz/livesync-commonlib/compat/API/processSetting";
(async () => {
+3 -1
View File
@@ -34,7 +34,9 @@
"test:e2e-matrix:couchdb-enc0": "deno test --env-file=.test.env -A --no-check --filter='e2e matrix: COUCHDB-enc0' test-e2e-two-vaults-matrix.ts",
"test:e2e-matrix:couchdb-enc1": "deno test --env-file=.test.env -A --no-check --filter='e2e matrix: COUCHDB-enc1' test-e2e-two-vaults-matrix.ts",
"test:e2e-matrix:minio-enc0": "deno test --env-file=.test.env -A --no-check --filter='e2e matrix: MINIO-enc0' test-e2e-two-vaults-matrix.ts",
"test:e2e-matrix:minio-enc1": "deno test --env-file=.test.env -A --no-check --filter='e2e matrix: MINIO-enc1' test-e2e-two-vaults-matrix.ts"
"test:e2e-matrix:minio-enc1": "deno test --env-file=.test.env -A --no-check --filter='e2e matrix: MINIO-enc1' test-e2e-two-vaults-matrix.ts",
"test:adaptive-journal-s3": "deno test --env-file=.test.env -A --no-check test-adaptive-journal-s3.ts",
"test:adaptive-journal-webdav": "deno test --env-file=.test.env -A --no-check test-adaptive-journal-webdav.ts"
},
"imports": {
"@std/assert": "jsr:@std/assert@^1.0.13",
+8 -1
View File
@@ -23,6 +23,7 @@ export interface CliResult {
export const TEE_ENABLED = Deno.env.get("LIVESYNC_TEST_TEE") === "1";
const VERBOSE_ENABLED = Deno.env.get("LIVESYNC_CLI_VERBOSE") === "1";
const DEBUG_ENABLED = Deno.env.get("LIVESYNC_CLI_DEBUG") === "1";
const SETUP_URI_PREFIX = "obsidian://setuplivesync?settings=";
function sleep(ms: number): Promise<void> {
return new Promise((resolve) => setTimeout(resolve, ms));
@@ -40,7 +41,13 @@ function concatChunks(chunks: Uint8Array[]): Uint8Array {
}
export function formatTeeCommand(args: string[]): string {
return ["node", CLI_DIST, ...args].map((part) => JSON.stringify(part)).join(" ");
const redactArgument = (argument: string): string => {
if (argument.startsWith(SETUP_URI_PREFIX)) {
return `${SETUP_URI_PREFIX}<redacted>`;
}
return argument.replace(/^(sls\+[^:]+:\/\/)[^/?#@]*@/u, "$1<redacted>@");
};
return ["node", CLI_DIST, ...args.map(redactArgument)].map((part) => JSON.stringify(part)).join(" ");
}
export function createLineTeeWriter(
+204
View File
@@ -330,6 +330,39 @@ const MINIO_CONTAINER = "minio-test";
const MINIO_IMAGE = "minio/minio";
const MINIO_MC_IMAGE = "minio/mc";
const WEBDAV_CONTAINER = "webdav-test";
const WEBDAV_IMAGE = "httpd:2.4.68";
const WEBDAV_HTTPD_CONFIG = `ServerRoot "/usr/local/apache2"
Listen 80
LoadModule mpm_event_module modules/mod_mpm_event.so
LoadModule authn_core_module modules/mod_authn_core.so
LoadModule authz_core_module modules/mod_authz_core.so
LoadModule dav_module modules/mod_dav.so
LoadModule dav_fs_module modules/mod_dav_fs.so
LoadModule unixd_module modules/mod_unixd.so
User www-data
Group www-data
ServerName localhost
DocumentRoot "/usr/local/apache2/htdocs"
PidFile "/tmp/httpd.pid"
ErrorLog "/proc/self/fd/2"
LogLevel warn
DavLockDB "/usr/local/apache2/var/DavLock"
DavLockDiscovery Off
<Directory "/usr/local/apache2/htdocs">
AllowOverride None
Require all granted
</Directory>
<Directory "/usr/local/apache2/htdocs/dav">
Dav On
</Directory>
`;
export async function stopCouchdb(): Promise<void> {
await stopAndRemoveContainer(COUCHDB_CONTAINER);
untrackContainer(COUCHDB_CONTAINER);
@@ -466,6 +499,70 @@ export async function stopMinio(): Promise<void> {
untrackContainer(MINIO_CONTAINER);
}
export async function listMinioObjectKeys(
minioEndpoint: string,
accessKey: string,
secretKey: string,
bucket: string
): Promise<string[]> {
const cmd =
`mc alias set myminio ${shQuote(minioEndpoint)} ${shQuote(accessKey)} ${shQuote(secretKey)} >/dev/null 2>&1 && ` +
`mc ls --recursive --json myminio/${shQuote(bucket)}`;
const result = await docker(
"run",
"--rm",
"--network",
"host",
"--entrypoint",
"/bin/sh",
MINIO_MC_IMAGE,
"-c",
cmd
);
if (result.code !== 0) {
throw new Error(`Could not list MinIO objects: ${result.stderr.trim()}`);
}
return result.stdout
.split(/\r?\n/u)
.filter((line) => line.trim().length > 0)
.map((line) => JSON.parse(line) as { key?: unknown })
.map(({ key }) => {
if (typeof key !== "string") {
throw new Error("MinIO returned an object without a string key");
}
return key;
})
.sort();
}
export async function readMinioObjectText(
minioEndpoint: string,
accessKey: string,
secretKey: string,
bucket: string,
key: string
): Promise<string> {
const cmd =
`mc alias set myminio ${shQuote(minioEndpoint)} ${shQuote(accessKey)} ${shQuote(secretKey)} >/dev/null 2>&1 && ` +
`mc cat myminio/${shQuote(bucket)}/${shQuote(key)}`;
const result = await docker(
"run",
"--rm",
"--network",
"host",
"--entrypoint",
"/bin/sh",
MINIO_MC_IMAGE,
"-c",
cmd
);
if (result.code !== 0) {
throw new Error(`Could not read MinIO object ${key}: ${result.stderr.trim()}`);
}
return result.stdout;
}
async function initMinioBucket(
minioEndpoint: string,
accessKey: string,
@@ -561,6 +658,113 @@ export async function startMinio(
await waitForMinioBucket(minioEndpoint, accessKey, secretKey, bucket);
}
// ---------------------------------------------------------------------------
// WebDAV
// ---------------------------------------------------------------------------
export async function stopWebDAV(): Promise<void> {
await stopAndRemoveContainer(WEBDAV_CONTAINER);
untrackContainer(WEBDAV_CONTAINER);
}
async function waitForWebDAV(endpoint: string): Promise<void> {
for (let attempt = 0; attempt < 30; attempt += 1) {
try {
const response = await fetch(endpoint, {
method: "PROPFIND",
headers: { Depth: "0" },
signal: AbortSignal.timeout(3000),
});
await response.body?.cancel().catch(() => {});
if (response.status === 207) return;
} catch {
// The container is still starting.
}
await sleep(500);
}
throw new Error(`WebDAV collection did not become ready: ${endpoint}`);
}
export async function startWebDAV(endpoint: string): Promise<void> {
const url = new URL(endpoint);
if (url.protocol !== "http:" || (url.hostname !== "127.0.0.1" && url.hostname !== "localhost")) {
throw new Error(`Managed WebDAV requires a local HTTP endpoint, received: ${endpoint}`);
}
if (url.pathname.replace(/\/+$/u, "") !== "/dav") {
throw new Error(`Managed WebDAV requires the /dav collection, received: ${endpoint}`);
}
const hostPort = url.port || "80";
const encodedConfig = btoa(WEBDAV_HTTPD_CONFIG);
const startCommand = `printf '%s' '${encodedConfig}' | base64 -d > /tmp/httpd.conf && exec httpd -DFOREGROUND -f /tmp/httpd.conf`;
console.log("[INFO] stopping leftover WebDAV container if present");
await stopWebDAV().catch(() => {});
console.log("[INFO] starting Apache WebDAV test container");
await dockerOrFail(
"run",
"-d",
"--name",
WEBDAV_CONTAINER,
"-p",
`${hostPort}:80`,
"--tmpfs",
"/usr/local/apache2/htdocs/dav:mode=0777",
"--tmpfs",
"/usr/local/apache2/var:mode=0777",
"--entrypoint",
"/bin/sh",
WEBDAV_IMAGE,
"-c",
startCommand
);
trackContainer(WEBDAV_CONTAINER);
await waitForWebDAV(endpoint);
}
function directWebDAVObjectUrl(collectionEndpoint: string, key: string): string {
return `${collectionEndpoint.replace(/\/+$/u, "")}/${encodeURIComponent(key)}`;
}
function webDAVRequestHeaders(): HeadersInit {
const username = Deno.env.get("WEBDAV_USERNAME") ?? "";
const password = Deno.env.get("WEBDAV_PASSWORD") ?? "";
if (!username && !password) return {};
return { Authorization: `Basic ${btoa(`${username}:${password}`)}` };
}
export async function listWebDAVObjectKeys(collectionEndpoint: string): Promise<string[]> {
const collectionUrl = new URL(`${collectionEndpoint.replace(/\/+$/u, "")}/`);
const response = await fetch(collectionUrl, {
method: "PROPFIND",
headers: { ...webDAVRequestHeaders(), Depth: "1" },
});
if (response.status !== 207) {
throw new Error(`Could not list WebDAV objects: HTTP ${response.status}`);
}
const xml = await response.text();
const hrefs = [
...xml.matchAll(/<(?:[A-Za-z_][\w.-]*:)?href\b[^>]*>([\s\S]*?)<\/(?:[A-Za-z_][\w.-]*:)?href>/giu),
].map((match) => match[1].trim());
const basePath = decodeURIComponent(collectionUrl.pathname);
const keys = new Set<string>();
for (const href of hrefs) {
const path = decodeURIComponent(new URL(href, collectionUrl).pathname);
if (!path.startsWith(basePath)) continue;
const key = path.slice(basePath.length).replace(/\/$/u, "");
if (key && !key.includes("/")) keys.add(key);
}
return [...keys].sort();
}
export async function readWebDAVObjectText(collectionEndpoint: string, key: string): Promise<string> {
const response = await fetch(directWebDAVObjectUrl(collectionEndpoint, key), { headers: webDAVRequestHeaders() });
if (!response.ok) {
throw new Error(`Could not read WebDAV object ${key}: HTTP ${response.status}`);
}
return await response.text();
}
// ---------------------------------------------------------------------------
// P2P relay (strfry)
// ---------------------------------------------------------------------------
+45 -14
View File
@@ -13,7 +13,12 @@ export async function initSettingsFile(settingsFile: string): Promise<void> {
* Generate a full setup URI from a settings file via the Commonlib package API.
* Mirrors the bash flow in test-setup-put-cat-linux.sh.
*/
export async function generateSetupUriFromSettings(settingsFile: string, setupPassphrase: string): Promise<string> {
export async function generateSetupUriFromSettings(
settingsFile: string,
setupPassphrase: string,
preserveSettings = false,
runtimeVaultPassphrase?: string
): Promise<string> {
const script = [
"import { fs } from '@vrtmrz/livesync-commonlib/node';",
"import { encodeSettingsToSetupURI } from '@vrtmrz/livesync-commonlib/compat/API/processSetting';",
@@ -21,13 +26,18 @@ export async function generateSetupUriFromSettings(settingsFile: string, setupPa
" const settingsPath = process.env.SETTINGS_FILE;",
" const passphrase = process.env.SETUP_PASSPHRASE;",
" const settings = JSON.parse(fs.readFileSync(settingsPath, 'utf-8'));",
" settings.couchDB_DBNAME = 'setup-put-cat-db';",
" settings.couchDB_URI = 'http://127.0.0.1:5999';",
" settings.couchDB_USER = 'dummy';",
" settings.couchDB_PASSWORD = 'dummy';",
" settings.liveSync = false;",
" settings.syncOnStart = false;",
" settings.syncOnSave = false;",
" if (process.env.RUNTIME_VAULT_PASSPHRASE !== undefined) {",
" settings.passphrase = process.env.RUNTIME_VAULT_PASSPHRASE;",
" }",
" if (process.env.PRESERVE_SETTINGS !== 'true') {",
" settings.couchDB_DBNAME = 'setup-put-cat-db';",
" settings.couchDB_URI = 'http://127.0.0.1:5999';",
" settings.couchDB_USER = 'dummy';",
" settings.couchDB_PASSWORD = 'dummy';",
" settings.liveSync = false;",
" settings.syncOnStart = false;",
" settings.syncOnSave = false;",
" }",
" const uri = await encodeSettingsToSetupURI(settings, passphrase);",
" process.stdout.write(uri.trim());",
"})();",
@@ -41,13 +51,18 @@ export async function generateSetupUriFromSettings(settingsFile: string, setupPa
await Deno.writeTextFile(scriptPath, script);
try {
const env: Record<string, string> = {
SETTINGS_FILE: settingsFile,
SETUP_PASSPHRASE: setupPassphrase,
PRESERVE_SETTINGS: preserveSettings ? "true" : "false",
};
if (runtimeVaultPassphrase !== undefined) {
env.RUNTIME_VAULT_PASSPHRASE = runtimeVaultPassphrase;
}
const cmd = new Deno.Command("npx", {
args: ["tsx", scriptPath],
cwd: CLI_DIR,
env: {
SETTINGS_FILE: settingsFile,
SETUP_PASSPHRASE: setupPassphrase,
},
env,
stdin: "null",
stdout: "piped",
stderr: "piped",
@@ -112,7 +127,7 @@ export async function applyCouchdbSettings(
export async function applyRemoteSyncSettings(
settingsFile: string,
options: {
remoteType: "COUCHDB" | "MINIO";
remoteType: "COUCHDB" | "MINIO" | "WEBDAV";
couchdbUri?: string;
couchdbUser?: string;
couchdbPassword?: string;
@@ -121,10 +136,14 @@ export async function applyRemoteSyncSettings(
minioEndpoint?: string;
minioAccessKey?: string;
minioSecretKey?: string;
webDAVConnectionURI?: string;
encrypt?: boolean;
passphrase?: string;
enableCompression?: boolean;
usePathObfuscation?: boolean;
journalFormat?: "adaptive-v1" | "opaque-v1";
expectedRepositoryId?: string;
packReadPolicy?: "range" | "whole-pack";
}
): Promise<void> {
const data = JSON.parse(await Deno.readTextFile(settingsFile));
@@ -135,7 +154,7 @@ export async function applyRemoteSyncSettings(
data.couchDB_USER = options.couchdbUser;
data.couchDB_PASSWORD = options.couchdbPassword;
data.couchDB_DBNAME = options.couchdbDbname;
} else {
} else if (options.remoteType === "MINIO") {
data.remoteType = "MINIO";
data.bucket = options.minioBucket;
data.endpoint = options.minioEndpoint;
@@ -143,6 +162,18 @@ export async function applyRemoteSyncSettings(
data.secretKey = options.minioSecretKey;
data.region = "auto";
data.forcePathStyle = true;
} else {
data.remoteType = "WEBDAV";
data.webDAVactiveConnectionURI = options.webDAVConnectionURI;
}
if (options.journalFormat !== undefined) {
data.journalFormat = options.journalFormat;
}
if (options.expectedRepositoryId !== undefined) {
data.expectedRepositoryId = options.expectedRepositoryId;
}
if (options.packReadPolicy !== undefined) {
data.packReadPolicy = options.packReadPolicy;
}
data.liveSync = true;
+2
View File
@@ -11,6 +11,8 @@ const TASKS = [
"test:e2e-matrix:couchdb-enc1",
"test:e2e-matrix:minio-enc0",
"test:e2e-matrix:minio-enc1",
"test:adaptive-journal-s3",
"test:adaptive-journal-webdav",
] as const;
for (const [index, task] of TASKS.entries()) {
@@ -0,0 +1,146 @@
import { assert, assertEquals } from "@std/assert";
import { TempDir } from "./helpers/temp.ts";
import { assertFilesEqual, runCli, runCliOrFail, runCliWithInputOrFail, sanitiseCatStdout } from "./helpers/cli.ts";
import { applyRemoteSyncSettings, initSettingsFile } from "./helpers/settings.ts";
import { listMinioObjectKeys, readMinioObjectText, startMinio, stopMinio } from "./helpers/docker.ts";
const EXTERNAL_PACK_TEST_BYTES = 9 * 1024 * 1024;
function deterministicBytes(length: number, seed: number): Uint8Array {
const bytes = new Uint8Array(length);
let state = seed;
for (let index = 0; index < bytes.byteLength; index += 1) {
state ^= state << 13;
state ^= state >>> 17;
state ^= state << 5;
bytes[index] = state & 0xff;
}
return bytes;
}
function requireEnv(...keys: string[]): string {
for (const key of keys) {
const value = Deno.env.get(key)?.trim();
if (value) return value;
}
throw new Error(`Required environment variable is missing: ${keys.join(" or ")}`);
}
Deno.test("e2e: two CLI vaults synchronise through Adaptive Journal S3", async () => {
const suffix = `${Date.now()}-${Math.floor(Math.random() * 100000)}`;
const endpoint = requireEnv("MINIO_ENDPOINT", "minioEndpoint").replace(/\/$/u, "");
const accessKey = requireEnv("MINIO_ACCESS_KEY", "accessKey");
const secretKey = requireEnv("MINIO_SECRET_KEY", "secretKey");
const bucket = `${requireEnv("MINIO_BUCKET_NAME", "bucketName")}-${suffix}`;
const passphrase = "adaptive-journal-cli-e2e-passphrase";
await using workDir = await TempDir.create("livesync-cli-adaptive-journal-s3");
const vaultA = workDir.join("vault-a");
const vaultB = workDir.join("vault-b");
const settingsA = workDir.join("settings-a.json");
const settingsB = workDir.join("settings-b.json");
const binarySourceA = workDir.join("source-a.bin");
const binarySourceB = workDir.join("source-b.bin");
const binaryDestinationA = workDir.join("destination-a.bin");
const binaryDestinationB = workDir.join("destination-b.bin");
await Deno.mkdir(vaultA, { recursive: true });
await Deno.mkdir(vaultB, { recursive: true });
const keepDocker = Deno.env.get("LIVESYNC_DEBUG_KEEP_DOCKER") === "1";
await startMinio(endpoint, accessKey, secretKey, bucket);
try {
await initSettingsFile(settingsA);
await initSettingsFile(settingsB);
await applyRemoteSyncSettings(settingsA, {
remoteType: "MINIO",
minioBucket: bucket,
minioEndpoint: endpoint,
minioAccessKey: accessKey,
minioSecretKey: secretKey,
encrypt: true,
passphrase,
enableCompression: false,
journalFormat: "adaptive-v1",
packReadPolicy: "whole-pack",
});
await applyRemoteSyncSettings(settingsB, {
remoteType: "MINIO",
minioBucket: bucket,
minioEndpoint: endpoint,
minioAccessKey: accessKey,
minioSecretKey: secretKey,
encrypt: true,
passphrase,
enableCompression: false,
journalFormat: "adaptive-v1",
packReadPolicy: "range",
});
const textPath = "adaptive/text.md";
const binaryPath = "adaptive/data.bin";
await runCliWithInputOrFail(`created-by-a-${suffix}\n`, vaultA, "--settings", settingsA, "put", textPath);
await Deno.writeFile(binarySourceA, deterministicBytes(EXTERNAL_PACK_TEST_BYTES, 0x1a2b3c4d));
await runCliOrFail(vaultA, "--settings", settingsA, "push", binarySourceA, binaryPath);
await runCliOrFail(vaultA, "--settings", settingsA, "sync");
await runCliOrFail(vaultB, "--settings", settingsB, "sync");
assertEquals(
sanitiseCatStdout(await runCliOrFail(vaultB, "--settings", settingsB, "cat", textPath)).trimEnd(),
`created-by-a-${suffix}`
);
await runCliOrFail(vaultB, "--settings", settingsB, "pull", binaryPath, binaryDestinationB);
await assertFilesEqual(binarySourceA, binaryDestinationB, "Adaptive Journal Range transfer differs");
await runCliWithInputOrFail(`updated-by-b-${suffix}\n`, vaultB, "--settings", settingsB, "put", textPath);
await Deno.writeFile(binarySourceB, deterministicBytes(EXTERNAL_PACK_TEST_BYTES, 0x5e6f7788));
await runCliOrFail(vaultB, "--settings", settingsB, "push", binarySourceB, binaryPath);
await runCliOrFail(vaultB, "--settings", settingsB, "sync");
await runCliOrFail(vaultA, "--settings", settingsA, "sync");
assertEquals(
sanitiseCatStdout(await runCliOrFail(vaultA, "--settings", settingsA, "cat", textPath)).trimEnd(),
`updated-by-b-${suffix}`
);
await runCliOrFail(vaultA, "--settings", settingsA, "pull", binaryPath, binaryDestinationA);
await assertFilesEqual(binarySourceB, binaryDestinationA, "Adaptive Journal whole-Pack transfer differs");
await runCliOrFail(vaultA, "--settings", settingsA, "rm", binaryPath);
await runCliOrFail(vaultA, "--settings", settingsA, "sync");
await runCliOrFail(vaultB, "--settings", settingsB, "sync");
const deleted = await runCli(vaultB, "--settings", settingsB, "cat", binaryPath);
assert(deleted.code !== 0, `Deleted binary remained readable:\n${deleted.combined}`);
const objectKeys = await listMinioObjectKeys(endpoint, accessKey, secretKey, bucket);
assert(objectKeys.includes("a1~manifest.json"), `Adaptive manifest is missing:\n${objectKeys.join("\n")}`);
for (const prefix of ["a1~writer~", "a1~pack~", "a1~commit~"]) {
assert(
objectKeys.some((key) => key.startsWith(prefix)),
`Adaptive object with prefix ${prefix} is missing:\n${objectKeys.join("\n")}`
);
}
const packKeys = objectKeys.filter((key) => key.startsWith("a1~pack~"));
assert(packKeys.length >= 2, `Expected external Packs from both CLI writers:\n${objectKeys.join("\n")}`);
for (const legacyPrefix of ["a1~index~", "a1~delta~", "a1~metadata~"]) {
assert(
!objectKeys.some((key) => key.startsWith(legacyPrefix)),
`Legacy Adaptive object with prefix ${legacyPrefix} was written:\n${objectKeys.join("\n")}`
);
}
const manifest = JSON.parse(
await readMinioObjectText(endpoint, accessKey, secretKey, bucket, "a1~manifest.json")
) as { objectLayout?: unknown };
assertEquals(manifest.objectLayout, "commit-bundle-v1");
assert(
!objectKeys.some((key) => key.startsWith("a1~probe~")),
`Adaptive capability probe objects were not removed:\n${objectKeys.join("\n")}`
);
assert(
!objectKeys.includes("_00000000-milestone.json"),
`Legacy Journal milestone was written into the Adaptive repository:\n${objectKeys.join("\n")}`
);
} finally {
if (!keepDocker) {
await stopMinio().catch(() => {});
}
}
});
@@ -0,0 +1,215 @@
import { assert, assertEquals } from "@std/assert";
import { TempDir } from "./helpers/temp.ts";
import { assertFilesEqual, runCli, runCliOrFail, runCliWithInputOrFail, sanitiseCatStdout } from "./helpers/cli.ts";
import { applyRemoteSyncSettings, generateSetupUriFromSettings, initSettingsFile } from "./helpers/settings.ts";
import { listWebDAVObjectKeys, readWebDAVObjectText, startWebDAV, stopWebDAV } from "./helpers/docker.ts";
const EXTERNAL_PACK_TEST_BYTES = 9 * 1024 * 1024;
function deterministicBytes(length: number, seed: number): Uint8Array {
const bytes = new Uint8Array(length);
let state = seed;
for (let index = 0; index < bytes.byteLength; index += 1) {
state ^= state << 13;
state ^= state >>> 17;
state ^= state << 5;
bytes[index] = state & 0xff;
}
return bytes;
}
function webDAVConnectionURI(endpoint: string, prefix: string): string {
const endpointUrl = new URL(endpoint);
const proxyUrl = new URL(`https://${endpointUrl.host}${endpointUrl.pathname}`);
const username = Deno.env.get("WEBDAV_USERNAME") ?? "";
const password = Deno.env.get("WEBDAV_PASSWORD") ?? "";
proxyUrl.username = username;
proxyUrl.password = password;
if (endpointUrl.protocol === "http:") proxyUrl.searchParams.set("insecure", "true");
proxyUrl.searchParams.set("prefix", prefix);
return `sls+webdav:${proxyUrl.toString().slice("https:".length)}`;
}
function setPackReadPolicy(connectionURI: string, policy: "range" | "whole-pack"): string {
const url = new URL(connectionURI);
url.searchParams.set("packReadPolicy", policy);
return url.toString();
}
function selectAdaptiveJournal(connectionURI: string): string {
const url = new URL(connectionURI);
url.searchParams.set("journalFormat", "adaptive-v1");
return url.toString();
}
function remoteIdFromListing(listing: string): string {
const line = listing
.split(/\r?\n/u)
.find((candidate) => candidate.includes("\tWebDAV Remote\t") || candidate.includes("\tWebDAV "));
const id = line?.split("\t", 1)[0];
if (!id) throw new Error(`WebDAV remote profile was not listed:\n${listing}`);
return id;
}
Deno.test("e2e: two CLI vaults synchronise through Adaptive Journal WebDAV", async () => {
const suffix = `${Date.now()}-${Math.floor(Math.random() * 100000)}`;
const endpoint = (Deno.env.get("WEBDAV_ENDPOINT") ?? "http://127.0.0.1:8088/dav").replace(/\/+$/u, "");
const prefix = `adaptive-cli-${suffix}/`;
const collectionEndpoint = `${endpoint}/${prefix}`;
const connectionURI = webDAVConnectionURI(endpoint, prefix);
const adaptiveConnectionURI = selectAdaptiveJournal(connectionURI);
const vaultPassphrase = "adaptive-journal-webdav-cli-e2ee";
const setupPassphrase = "adaptive-journal-webdav-cli-setup";
await using workDir = await TempDir.create("livesync-cli-adaptive-journal-webdav");
const vaultA = workDir.join("vault-a");
const vaultB = workDir.join("vault-b");
const settingsA = workDir.join("settings-a.json");
const settingsB = workDir.join("settings-b.json");
const binarySourceA = workDir.join("source-a.bin");
const binarySourceB = workDir.join("source-b.bin");
const binaryDestinationA = workDir.join("destination-a.bin");
const binaryDestinationB = workDir.join("destination-b.bin");
await Deno.mkdir(vaultA, { recursive: true });
await Deno.mkdir(vaultB, { recursive: true });
const shouldStartDocker = Deno.env.get("LIVESYNC_START_DOCKER") !== "0";
const keepDocker = Deno.env.get("LIVESYNC_DEBUG_KEEP_DOCKER") === "1";
if (shouldStartDocker) await startWebDAV(endpoint);
try {
await initSettingsFile(settingsA);
await applyRemoteSyncSettings(settingsA, {
remoteType: "WEBDAV",
webDAVConnectionURI: connectionURI,
encrypt: true,
passphrase: vaultPassphrase,
enableCompression: false,
journalFormat: "adaptive-v1",
packReadPolicy: "whole-pack",
});
const addedRemote = await runCliOrFail(
vaultA,
"--settings",
settingsA,
"remote-add",
"WebDAV E2E",
adaptiveConnectionURI
);
const remoteId = addedRemote.trim().split("\t", 1)[0];
assert(remoteId, `remote-add did not return a profile ID:\n${addedRemote}`);
const settingsAfterRemoteAdd = JSON.parse(await Deno.readTextFile(settingsA)) as {
remoteConfigurations?: Record<string, unknown>;
liveSync?: boolean;
};
assert(
settingsAfterRemoteAdd.remoteConfigurations?.[remoteId],
`remote-add did not persist profile ${remoteId}: ${Object.keys(settingsAfterRemoteAdd.remoteConfigurations ?? {}).join(", ")}`
);
assertEquals(settingsAfterRemoteAdd.liveSync, true, "remote-add changed the persisted synchronisation mode");
await runCliOrFail(vaultA, "--settings", settingsA, "remote-activate", remoteId);
const textPath = "adaptive/text.md";
const binaryPath = "adaptive/data.bin";
await runCliWithInputOrFail(`created-by-a-${suffix}\n`, vaultA, "--settings", settingsA, "put", textPath);
await Deno.writeFile(binarySourceA, deterministicBytes(EXTERNAL_PACK_TEST_BYTES, 0x1a2b3c4d));
await runCliOrFail(vaultA, "--settings", settingsA, "push", binarySourceA, binaryPath);
await runCliOrFail(vaultA, "--settings", settingsA, "sync");
const remoteListing = await runCliOrFail(vaultA, "--settings", settingsA, "remote-ls");
assertEquals(remoteIdFromListing(remoteListing), remoteId);
assert(
remoteListing
.split(/\r?\n/u)
.some((line) => line.startsWith(`${remoteId}\t`) && line.includes("\tactive\t")),
`Activated WebDAV profile was not listed as active:\n${remoteListing}`
);
const exportedConnection = (
await runCliOrFail(vaultA, "--settings", settingsA, "remote-export", remoteId)
).trim();
assert(exportedConnection.startsWith("sls+webdav://"));
assert(exportedConnection.includes("journalFormat=adaptive-v1"));
assert(!exportedConnection.includes("packReadPolicy="));
const setupURI = await generateSetupUriFromSettings(settingsA, setupPassphrase, true, vaultPassphrase);
await initSettingsFile(settingsB);
await runCliWithInputOrFail(`${setupPassphrase}\n`, vaultB, "--settings", settingsB, "setup", setupURI);
const settingsAfterSetup = JSON.parse(await Deno.readTextFile(settingsB)) as {
encryptedPassphrase?: string;
};
assert(
typeof settingsAfterSetup.encryptedPassphrase === "string" &&
settingsAfterSetup.encryptedPassphrase.length > 0,
"setup did not persist the encrypted Vault passphrase"
);
await runCliOrFail(
vaultB,
"--settings",
settingsB,
"remote-set",
remoteId,
setPackReadPolicy(exportedConnection, "range")
);
const rangeConnection = (await runCliOrFail(vaultB, "--settings", settingsB, "remote-export", remoteId)).trim();
assert(rangeConnection.includes("packReadPolicy=range"));
await runCliOrFail(vaultB, "--settings", settingsB, "sync");
assertEquals(
sanitiseCatStdout(await runCliOrFail(vaultB, "--settings", settingsB, "cat", textPath)).trimEnd(),
`created-by-a-${suffix}`
);
await runCliOrFail(vaultB, "--settings", settingsB, "pull", binaryPath, binaryDestinationB);
await assertFilesEqual(binarySourceA, binaryDestinationB, "Adaptive Journal Range transfer differs");
await runCliWithInputOrFail(`updated-by-b-${suffix}\n`, vaultB, "--settings", settingsB, "put", textPath);
await Deno.writeFile(binarySourceB, deterministicBytes(EXTERNAL_PACK_TEST_BYTES, 0x5e6f7788));
await runCliOrFail(vaultB, "--settings", settingsB, "push", binarySourceB, binaryPath);
await runCliOrFail(vaultB, "--settings", settingsB, "sync");
await runCliOrFail(vaultA, "--settings", settingsA, "sync");
assertEquals(
sanitiseCatStdout(await runCliOrFail(vaultA, "--settings", settingsA, "cat", textPath)).trimEnd(),
`updated-by-b-${suffix}`
);
await runCliOrFail(vaultA, "--settings", settingsA, "pull", binaryPath, binaryDestinationA);
await assertFilesEqual(binarySourceB, binaryDestinationA, "Adaptive Journal whole-Pack transfer differs");
await runCliOrFail(vaultA, "--settings", settingsA, "rm", binaryPath);
await runCliOrFail(vaultA, "--settings", settingsA, "sync");
await runCliOrFail(vaultB, "--settings", settingsB, "sync");
const deleted = await runCli(vaultB, "--settings", settingsB, "cat", binaryPath);
assert(deleted.code !== 0, `Deleted binary remained readable:\n${deleted.combined}`);
const objectKeys = await listWebDAVObjectKeys(collectionEndpoint);
assert(objectKeys.includes("a1~manifest.json"), `Adaptive manifest is missing:\n${objectKeys.join("\n")}`);
for (const objectPrefix of ["a1~writer~", "a1~pack~", "a1~commit~"]) {
assert(
objectKeys.some((key) => key.startsWith(objectPrefix)),
`Adaptive object with prefix ${objectPrefix} is missing:\n${objectKeys.join("\n")}`
);
}
const packKeys = objectKeys.filter((key) => key.startsWith("a1~pack~"));
assert(packKeys.length >= 2, `Expected external Packs from both CLI writers:\n${objectKeys.join("\n")}`);
for (const legacyPrefix of ["a1~index~", "a1~delta~", "a1~metadata~"]) {
assert(
!objectKeys.some((key) => key.startsWith(legacyPrefix)),
`Legacy Adaptive object with prefix ${legacyPrefix} was written:\n${objectKeys.join("\n")}`
);
}
const manifest = JSON.parse(await readWebDAVObjectText(collectionEndpoint, "a1~manifest.json")) as {
objectLayout?: unknown;
};
assertEquals(manifest.objectLayout, "commit-bundle-v1");
assert(
!objectKeys.some((key) => key.startsWith("a1~probe~")),
`Adaptive capability probe objects were not removed:\n${objectKeys.join("\n")}`
);
assert(
!objectKeys.includes("_00000000-milestone.json"),
`Legacy Journal milestone was written into the Adaptive repository:\n${objectKeys.join("\n")}`
);
} finally {
if (shouldStartDocker && !keepDocker) {
await stopWebDAV().catch(() => {});
}
}
});

Some files were not shown because too many files have changed in this diff Show More