Compare commits

...

1152 Commits

Author SHA1 Message Date
vorotamoroz 52138bf7a5 Prefer focused Commonlib entry points 2026-07-18 18:16:16 +00:00
vorotamoroz ff01ca551a Harden lint validation and browser shims 2026-07-18 17:21:43 +00:00
vorotamoroz 81d27a932f Route application diagnostics through owned log paths 2026-07-18 16:53:20 +00:00
vorotamoroz 3e22c2eda8 Adopt the Commonlib-owned P2P transport 2026-07-18 16:05:01 +00:00
vorotamoroz 6cc06bf20f test: guard TypeScript modifier inspection 2026-07-17 19:40:43 +00:00
vorotamoroz d0be2968c2 ci: stabilise package boundary validation 2026-07-17 19:22:00 +00:00
vorotamoroz 2e999b9426 docs: align package boundary proof with published artefact 2026-07-17 18:20:29 +00:00
vorotamoroz 7b7801b029 test: verify the packaged Commonlib boundary 2026-07-17 17:50:56 +00:00
vorotamoroz f80137e364 docs: record package boundary verification 2026-07-17 16:08:49 +00:00
vorotamoroz 97964fddf1 refactor: inject CLI I/O and diagnostics 2026-07-17 16:01:48 +00:00
vorotamoroz 1b4a0d76dd fix: remove remaining Svelte state warnings 2026-07-17 15:37:20 +00:00
vorotamoroz f6b03dc811 refactor: make Svelte prop lifetimes explicit 2026-07-17 15:04:01 +00:00
vorotamoroz 6e00b8195a fix: remove ineffective trash setting toggle 2026-07-17 14:55:30 +00:00
vorotamoroz 326bf77183 refactor: align platform APIs with community review 2026-07-17 14:47:27 +00:00
vorotamoroz 298738fc67 fix: keep packaged dialogs inside mobile safe areas 2026-07-17 13:28:58 +00:00
vorotamoroz d5754f1f54 docs: keep package proof evidence durable 2026-07-17 11:58:08 +00:00
vorotamoroz 14da32cbab chore: refresh Commonlib package proof 2026-07-17 11:32:41 +00:00
vorotamoroz a721d3b602 refactor: consume Commonlib as a package 2026-07-17 11:13:16 +00:00
vorotamoroz 6475d32769 docs: define common library package boundary 2026-07-17 02:36:21 +00:00
vorotamoroz e114f66fb2 Harden release workflow validation and retries 2026-07-17 09:50:27 +09:00
vorotamoroz e31e761c5a Merge pull request #1028 from vrtmrz/0_25_83
Releasing 0.25.83
2026-07-16 22:41:26 +09:00
vorotamoroz 977d50566d docs: finalise 0.25.83 release notes 2026-07-16 13:01:32 +00:00
github-actions[bot] 85fc62d61c Releasing 0.25.83 2026-07-16 12:28:26 +00:00
Ouyang Xingyuan 8e0d5581f7 test: cover the non-bulk newest resolution entry point
Reason:
- The non-bulk notice regression test should exercise the real resolve-by-newest call path.

Changes:
- Resolve a mocked conflict through `_anyResolveConflictByNewest` and assert that its success notice remains visible.
2026-07-16 21:18:16 +09:00
Ouyang Xingyuan 32e992db93 fix: scope merge notice suppression to bulk resolution
Reason:
- Bulk newest resolution should not display one success notice per file, while non-bulk resolution must retain its existing notice.

Changes:
- Pass an explicit notice flag only through the bulk resolution path.
- Cover bulk suppression, non-bulk notices, and ten-file progress updates.
2026-07-16 21:18:16 +09:00
Ouyang Xingyuan 3103dc4f52 fix: suppress bulk conflict resolution notice spam
Bulk newest-revision conflict resolution created one notice per resolved file
and updated progress for nine out of every ten files.

Keep per-file bulk resolution success messages in the log without displaying
notices, update notice-level progress once every ten files, and cover both
behaviours with unit tests.
2026-07-16 21:18:16 +09:00
vorotamoroz 943b2a3bd4 Point at merged commonlib rename guard 2026-07-16 20:48:12 +09:00
vorotamoroz 69f4879589 test: cover case-only file renames in Obsidian 2026-07-16 20:48:12 +09:00
vorotamoroz 918d3b5547 fix: guard case-only rename recovery 2026-07-16 20:48:12 +09:00
metrovoc 869a893c19 fix: synchronise case-only file renames safely 2026-07-16 20:48:12 +09:00
vorotamoroz ba7ea27d0c feat: distinguish remote operation and request activity 2026-07-16 20:13:10 +09:00
vorotamoroz 9f9758bc12 fix: align chunk reads with remote delivery
Consume commonlib's delivery lifecycle, classify finite replication entry points, and document why the five-minute inactivity fuse is only a leak safety valve. Record the remote-activity counter correction and preserve continuous replication's unbounded live channel.
2026-07-16 20:13:10 +09:00
vorotamoroz f54d162ef9 Merge pull request #1024 from vrtmrz/0_25_82
Releasing 0.25.82
2026-07-16 00:07:44 +09:00
vorotamoroz 9d70ac8064 Synchronise workspace lock dependencies 2026-07-15 14:13:43 +00:00
vorotamoroz f44dc76087 Polish 0.25.82 release notes 2026-07-15 14:03:15 +00:00
github-actions[bot] 962589a40f Releasing 0.25.82 2026-07-15 14:00:29 +00:00
vorotamoroz d8bbab6bed Fix Deno setup for release preparation 2026-07-15 22:55:19 +09:00
vorotamoroz b475ea64e2 ci: publish CLI images for AMD64 and ARM64 2026-07-15 22:12:07 +09:00
vorotamoroz 908acc4a84 test: add CLI-to-Obsidian compatibility E2E 2026-07-15 22:12:07 +09:00
vorotamoroz 31e9186930 Protect bounded remote activity across app lifecycle 2026-07-15 20:53:27 +09:00
vorotamoroz 4cbccf85b4 Merge pull request #1019 from apple-ouyang/codex/refresh-security-seed-before-sync
Fix stale Security Seed after remote rebuild
2026-07-15 20:35:23 +09:00
Ouyang Xingyuan 45657ba39c fix: refresh Security Seed before replication
Reason:
- A client that remains open during a remote database rebuild can retain the previous Security Seed and upload documents encrypted with the wrong key.

Changes:
- Fetch the remote Security Seed at every replication preflight instead of reusing the process cache.
- Add a regression test and an unreleased change note for issue #1018.
2026-07-14 18:15:40 +08:00
vorotamoroz 056d89aa4d Merge pull request #1013 from vrtmrz/hold-release-merge-for-brat-validation
Hold release merges until BRAT validation
2026-07-14 08:52:41 +09:00
vorotamoroz 804103693f Hold release PR merge until BRAT validation 2026-07-14 08:45:23 +09:00
vorotamoroz eb3abc71b4 Merge pull request #1012 from vrtmrz/0_25_81
Releasing 0.25.81
2026-07-14 08:21:47 +09:00
vorotamoroz 77eacade10 Generate fallback types during releases 2026-07-14 03:15:42 +09:00
vorotamoroz d404f2ae54 Add the 0.25.81 release note 2026-07-14 03:04:10 +09:00
github-actions[bot] a7c488f377 Releasing 0.25.81 2026-07-13 18:02:25 +00:00
vorotamoroz 0f75cd92c0 Automate and validate the release flow (#998)
* improve release flow

* limit permission

* Test and harden the release workflow

* Keep workspace lockfile versions aligned

* Run release regression tests in CI
2026-07-14 03:01:22 +09:00
vorotamoroz a41c829c10 fix: preserve U+FEFF at chunk boundaries (#1010) 2026-07-14 02:15:57 +09:00
vorotamoroz ff12778301 Fix strfry tmpfs permissions in CI (#1011) 2026-07-14 02:12:54 +09:00
vorotamoroz e38084b901 Use cached ignore matchers from commonlib (#1008) 2026-07-14 00:29:11 +09:00
vorotamoroz 9265a74578 Merge pull request #1007 from apple-ouyang/codex/cache-cli-ignore-matchers
Compile CLI ignore patterns once
2026-07-13 13:26:21 +09:00
Ouyang Xingyuan f28d886603 perf(cli): compile ignore patterns once
Long-running CLI processes rebuilt the same minimatch ASTs for every checked path.

Compile patterns while loading ignore rules and reuse them until the rules are reloaded.

Refs #1006
2026-07-13 10:47:35 +08:00
vorotamoroz 9c375bd6fd Deploy QR aggregator with GitHub Pages 2026-07-12 21:06:56 +09:00
vorotamoroz 42a333a571 Record P2P validation infrastructure 2026-07-12 20:40:38 +09:00
vorotamoroz a6a5f7af53 Use merged storage capability contracts 2026-07-12 20:40:38 +09:00
vorotamoroz 4df87cc96a Explain portable storage path policy 2026-07-12 20:40:38 +09:00
vorotamoroz d16e66c67a Track unreleased storage contract changes 2026-07-12 20:40:38 +09:00
vorotamoroz 57e26f1079 Enforce rooted storage path contracts 2026-07-12 20:40:38 +09:00
vorotamoroz dbb8d2be22 test: establish storage adapter contracts 2026-07-12 20:40:38 +09:00
vorotamoroz a60932d9e4 Strengthen CLI benchmark verification 2026-07-12 20:02:29 +09:00
vorotamoroz fd3e8416b7 Move CLI E2E runner into CLI tests 2026-07-12 20:02:29 +09:00
vorotamoroz b9a27ffef9 Run CLI P2P E2E in Compose CI 2026-07-12 20:02:29 +09:00
vorotamoroz af72cac4e3 Add scoped CLI benchmark metadata 2026-07-12 20:02:29 +09:00
vorotamoroz 7d26f0ae35 Add P2P signalling-only netem benchmark 2026-07-12 20:02:29 +09:00
vorotamoroz 008a5ace06 Add P2P split-container netem stress fixture 2026-07-12 20:02:29 +09:00
vorotamoroz b1955703b9 Record P2P selected candidate paths 2026-07-12 20:02:29 +09:00
vorotamoroz 39065e15e3 Add CouchDB netem shim benchmark 2026-07-12 20:02:29 +09:00
vorotamoroz 7b480d4c1d Add netem smoke fixture for benchmark simulation 2026-07-12 20:02:29 +09:00
vorotamoroz 33cd6a3b51 Document CLI benchmark comparison model 2026-07-12 20:02:29 +09:00
vorotamoroz 870ac93cac Lower benchmark relay file descriptor limit 2026-07-12 20:02:29 +09:00
vorotamoroz 2fcdbbc4da Stabilise CLI P2P smoke readiness 2026-07-12 20:02:29 +09:00
vorotamoroz 18e8b239bf Run CLI network smoke on pull requests 2026-07-12 20:02:29 +09:00
vorotamoroz ef655a2976 Add Compose-based CLI network benchmarks 2026-07-12 20:02:29 +09:00
vorotamoroz 78eb6ec3bc build: use stable Obsidian test session 2026-07-12 15:10:49 +09:00
vorotamoroz c0780d19ba build: update Obsidian test session preview 2026-07-12 15:10:49 +09:00
vorotamoroz 5b61ea5009 test: share Obsidian E2E session infrastructure 2026-07-12 15:10:49 +09:00
vorotamoroz 6a9918677f Merge pull request #996 from vrtmrz/0_25_80
releasing 0.25.80
2026-07-07 21:21:34 +09:00
vorotamoroz 2d42b92a89 bump 2026-07-07 12:15:58 +00:00
vorotamoroz a2b794c520 Merge pull request #995 from vrtmrz/fix_conflict_issues
Fix conflict issues
2026-07-07 21:13:00 +09:00
vorotamoroz 6f9446f447 Merge remote-tracking branch 'origin/main' into fix_conflict_issues 2026-07-07 12:11:38 +00:00
vorotamoroz 3f9cd67b1c Merge pull request #992 from vrtmrz/fix_989
fix: enable hidden file sync before overwrite setup
2026-07-07 21:10:09 +09:00
vorotamoroz dbcbf2c5ca fix: document safer conflict preservation 2026-07-07 12:05:40 +00:00
vorotamoroz eed0fca8d3 fix: preserve ambiguous conflict candidates 2026-07-07 12:00:24 +00:00
vorotamoroz 05e031b90b update submodule pointer 2026-07-07 11:31:29 +00:00
vorotamoroz bec767c13f add notes 2026-07-07 11:30:36 +00:00
vorotamoroz 8046a777af fix: refine conflict merge policy 2026-07-07 11:29:51 +00:00
vorotamoroz 0c58b0c513 test: reproduce conflict issue reports 2026-07-07 11:07:09 +00:00
vorotamoroz b66e227a02 update doc
update submodule pointer
2026-07-07 10:54:49 +00:00
vorotamoroz af6df84b5d fix: enable hidden file sync before overwrite setup 2026-07-03 05:09:55 +00:00
vorotamoroz 06bba49aad Merge pull request #985 from vrtmrz/test/obsidian-real-e2e
Add real obsidian e2e
2026-07-01 19:18:58 +09:00
vorotamoroz f97888572b (build): use deno command for type generation 2026-06-30 09:37:54 +00:00
vorotamoroz 9d9b512f3f Merge remote-tracking branch 'origin/main' into test/obsidian-real-e2e 2026-06-30 09:18:08 +00:00
vorotamoroz 54e6a761e5 (test): add local Obsidian E2E suite 2026-06-30 08:36:29 +00:00
vorotamoroz d37ed51660 Merge pull request #984 from vrtmrz/0_25_79
bump and releasing 0.25.79
2026-06-30 01:06:43 +09:00
vorotamoroz 9fbdee47a6 Merge branch 'main' into 0_25_79 2026-06-29 12:30:28 +01:00
vorotamoroz b568c4b003 bump and releasing 0.25.79 2026-06-29 12:19:56 +01:00
vorotamoroz 505e3c5e8c Merge pull request #983 from vrtmrz/stream-inbox-patch
(refactor): genReplication to use StreamInbox and improve event handling
2026-06-29 20:02:59 +09:00
vorotamoroz 75cb9ece58 update the pointer 2026-06-29 12:02:33 +01:00
vorotamoroz 05877834dc update the pointer 2026-06-29 12:01:08 +01:00
vorotamoroz dae565129a Merge pull request #978 from apple-ouyang/codex/fast-fetch-interruptions
Remember Simple Fetch choices during interrupted initialisation
2026-06-29 19:56:17 +09:00
vorotamoroz 701520fe15 Merge pull request #981 from apple-ouyang/codex/maintenance-prereq-dialog
Ask before applying maintenance prerequisite settings
2026-06-29 19:52:56 +09:00
vorotamoroz e662e99641 Merge pull request #976 from kiraventom/main
Add nginx into table of contents
2026-06-29 19:42:24 +09:00
vorotamoroz 96396f77f5 (refactor): genReplication to use StreamInbox and improve event handling 2026-06-29 09:11:20 +00:00
vorotamoroz e916683b8d (test): customisation sync 2 2026-06-26 12:16:39 +00:00
vorotamoroz 7028a0857e (test): customisation sync 2026-06-26 12:02:29 +00:00
vorotamoroz 150f0700b0 (test): hidden file sync roundtrip and misc 2026-06-26 11:39:02 +00:00
vorotamoroz 36590ee762 (test): improve e2e CouchDB 2026-06-26 10:55:06 +00:00
vorotamoroz 4a33236c8f (test): improve e2e session helper 2026-06-26 10:46:45 +00:00
vorotamoroz be23fa51a1 (test): the E2E test on the real-Obsidian 2026-06-26 10:33:51 +00:00
Ouyang Xingyuan 7e59010824 Ask before applying maintenance prerequisite settings
Local database maintenance actions require local chunk revisions and fully replicated chunks, but previously failed with a notice when those settings were not already configured.

This adds a small prerequisite confirmation helper for the maintenance commands. When required settings are missing, the user can apply them and continue from the action they already started, or cancel without changing settings.

Covers both required maintenance settings:
- Compute revisions for chunks enabled
- Fetch chunks on demand disabled

Fixes #980
2026-06-25 14:41:14 +08:00
vorotamoroz 8e24739b96 Tag tidying 2026-06-24 05:55:05 +01:00
Ouyang Xingyuan b2ea810dde Update commonlib for Fast Fetch interruption retries
The plug-in needs the commonlib change that retries transient Fast Fetch failures from the latest checkpoint.

This updates the src/lib submodule pointer to the commonlib commit with retry and AbortError handling.
2026-06-24 10:22:03 +08:00
Ouyang Xingyuan bf8da52348 Remember pending Simple Fetch choices
Simple Fetch asked the same mode and deletion choices again after a failed or interrupted pending Fetch All run.

This stores the selected quick-flow choices in local small config until the operation completes, is cancelled, or is finalised.
2026-06-24 10:12:51 +08:00
Temirkanov Alikhan 31d01f039c Add nginx into table of contents 2026-06-23 17:59:34 +03:00
vorotamoroz 2d5b5da904 Merge pull request #974 from vrtmrz/0_25_78
Releasing v0.25.78
2026-06-23 20:19:31 +09:00
vorotamoroz 6d4ae6dccb bump 2026-06-23 12:10:05 +01:00
vorotamoroz 37ccbf249c Merge pull request #970 from vrtmrz/refactor_journalsync
Overhaul ObjectStorage Replicator
2026-06-23 20:02:02 +09:00
vorotamoroz 6e14f0210a Track the subrepo 2026-06-23 12:01:27 +01:00
vorotamoroz c626fb1783 Update submodule pointer,
Update release note
2026-06-23 11:58:57 +01:00
vorotamoroz 23af915604 Merge pull request #973 from apple-ouyang/codex/fast-fetch-resume
Update commonlib for resumable Fast Fetch
2026-06-23 19:18:43 +09:00
Ouyang Xingyuan 2257ab7647 Update commonlib for resumable Fast Fetch
Point src/lib at the commonlib commit that makes Fast Fetch resume from a persisted CouchDB sequence.
2026-06-23 17:50:57 +08:00
vorotamoroz 08bfec204f Merge pull request #971 from kiraventom/main
Add a section on reverse-proxying via Nginx
2026-06-23 18:46:46 +09:00
vorotamoroz 916e24e21e Fix for CI 2026-06-23 10:41:20 +01:00
vorotamoroz 3ee2c401d4 Correcting sloppy wording on the design doc.
Creating and revising tests
Amending the vagueness in/arises from the draft
2026-06-23 10:27:28 +01:00
vorotamoroz 78d9d66438 fix scripts to make sure building types 2026-06-23 09:38:48 +01:00
vorotamoroz 7298aca2e9 prettified 2026-06-23 09:36:40 +01:00
vorotamoroz 431a845f18 fixed
- Journal upload issue
- Types and Setting Tab issue
2026-06-23 09:36:32 +01:00
vorotamoroz 877d1b09f4 refine 2026-06-23 01:17:09 +09:00
Temirkanov Alikhan 34caf19e20 Add a section on reverse-proxying via Nginx 2026-06-22 12:29:24 +03:00
vorotamoroz 9004c194b3 Overhaul ObjectStorage Replicator 2026-06-22 13:38:31 +09:00
vorotamoroz c57b8a5f4e Write the minimal CoC. Welcome your feedback! 2026-06-19 12:13:23 +01:00
vorotamoroz f90a83ef1e Update docs 2026-06-19 11:33:38 +01:00
vorotamoroz 62f44e38c0 Releasing 0.25.77 (#968)
Squash commits
2026-06-19 17:45:37 +09:00
vorotamoroz c6c4044f3c Merge pull request #965 from vrtmrz/refactor_structure
Refactor structure (2)
2026-06-18 10:44:41 +09:00
vorotamoroz 2bd7832d71 Merge pull request #964 from mugenrei/main
Merge upstream
2026-06-18 10:43:08 +09:00
vorotamoroz 72033472f3 add dependency explicitly 2026-06-17 10:36:38 +01:00
vorotamoroz 93dc03e86f remove unused dependencies, update some dependencies 2026-06-17 10:05:45 +01:00
vorotamoroz dae8443fe8 update more deps 2026-06-17 06:35:14 +01:00
vorotamoroz 88a8bcbd5a barrel node specific modules to summarise the warnings 2026-06-17 06:15:25 +01:00
vorotamoroz 4a5283543d for automatic review 2026-06-17 06:10:30 +01:00
vorotamoroz 7895336189 revert deno test 2026-06-17 06:08:40 +01:00
vorotamoroz 2d5cdccf7d for automatic review 2026-06-17 05:51:01 +01:00
mugenrei 28cdf40fcd Merge branch 'main' into main 2026-06-17 01:44:26 -03:00
vorotamoroz 497fd04081 fix global references 2026-06-17 05:29:45 +01:00
vorotamoroz 38142aaf1c Merge pull request #963 from vrtmrz/refactor_structure
Refactor structure
2026-06-17 13:06:30 +09:00
vorotamoroz ae9c46f8f0 Fix import paths 2026-06-17 04:39:39 +01:00
vorotamoroz dcd10cd690 Fix dependency management 2026-06-17 04:24:55 +01:00
vorotamoroz 5a35b71339 define workspace 2026-06-17 04:01:45 +01:00
vorotamoroz 18a59219f7 add ignore paths 2026-06-15 12:25:50 +01:00
vorotamoroz c9095738e6 (chore): limit tags 2026-06-15 12:21:58 +01:00
vorotamoroz bed415fc6b (chore): add missing release date 2026-06-15 12:21:11 +01:00
vorotamoroz 4eeb93b9e4 Merge pull request #961 from vrtmrz/0_25_76
Releasing 0.25.76
2026-06-15 20:18:37 +09:00
vorotamoroz 0fc233686b prettify and bump 2026-06-15 11:54:41 +01:00
vorotamoroz def1d297f5 Merge pull request #958 from vrtmrz/fix_875
Fix S3 client with custom headers and strict server
2026-06-15 19:41:31 +09:00
vorotamoroz e7cf2a6fba Merge branch 'main' into fix_875 2026-06-15 19:41:18 +09:00
vorotamoroz 2da2fd7671 Merge pull request #960 from vrtmrz/fix_956
Fix configuration serialisation
2026-06-15 19:40:37 +09:00
vorotamoroz d5175969e7 Update notes. 2026-06-15 11:39:38 +01:00
vorotamoroz 46f9630999 track merged submodule 2026-06-15 11:39:13 +01:00
vorotamoroz 75adcf0ff0 track merged submodule 2026-06-15 11:38:50 +01:00
vorotamoroz 866dd49ba3 ### Fixed
- No longer connection information of the P2P synchronisation is broken on the specific platform (#956).
2026-06-15 11:34:25 +01:00
vorotamoroz d0a84d07aa (chore) typo 2026-06-15 10:43:27 +01:00
vorotamoroz 089a4c0f8b ### Fixed
- Now S3 connection with custom headers works properly.
2026-06-15 10:41:26 +01:00
vorotamoroz 1a1f816872 Merge pull request #957 from vrtmrz/v0_25_75
Releasing v0.25.75
2026-06-13 12:05:16 +09:00
vorotamoroz 9d86c2828b Update dependency, and bump 2026-06-13 11:58:40 +09:00
vorotamoroz 3b6d3beaa7 Merge pull request #955 from vrtmrz/fix_953
Fix an issue where using fast synchronisation caused a TypeError in some environment
2026-06-13 11:21:19 +09:00
vorotamoroz bb75b6ead8 Merge remote-tracking branch 'origin/main' into fix_953 2026-06-13 11:19:22 +09:00
vorotamoroz fccb2304f6 update submodule pointer 2026-06-13 11:18:03 +09:00
vorotamoroz f00ef5eaae update submodule pointer 2026-06-13 11:15:03 +09:00
vorotamoroz 4e7ee760de Merge pull request #949 from AutoraLabs/feat/keep-livesync-active-in-background
feat: opt-in desktop setting to keep replication active in the background
2026-06-13 11:14:04 +09:00
vorotamoroz c4faade30c Update submodule pointer 2026-06-13 11:13:35 +09:00
vorotamoroz 295dc1392a ### Fixed
- Fixed an issue where using fast synchronisation caused a TypeError in some environments (#953).
2026-06-12 11:33:07 +01:00
Miguel Ferreira 445a8c747c chore(submodule): bump src/lib to commonlib main (#51 merged as e98d929)
vrtmrz/livesync-commonlib#51 is merged into commonlib main as e98d929. Repoint
the gitlink from the PR branch commit to that merge commit so this PR builds
against upstream main.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-10 10:45:40 +01:00
Miguel Ferreira 292a6b9e1e refactor: detect platform via APIService.isMobile() instead of Platform.isDesktopApp
Address the maintainer review on #949: determine the platform through the
plugin's own service layer (services.API.isMobile()) rather than Obsidian's
Platform API directly, matching the existing call in ObsidianLiveSyncSettingTab.
Applies to both PR-introduced sites: the runtime guard (ModuleObsidianEvents)
and the settings-pane toggle (PaneSyncSettings).

The TFile import becomes type-only so deps.ts is no longer pulled at runtime;
the unit test drives the platform through the services.API.isMobile() mock.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-10 10:45:40 +01:00
vorotamoroz 0e04e7d31d Merge pull request #951 from vrtmrz/feat_docker_ci_build
feat: Docker CI workflow to enhance image tagging and push logic base…
2026-06-09 18:41:26 +09:00
vorotamoroz 4cf4acf7e9 feat: Docker CI workflow to enhance image tagging and push logic based on branch and event type 2026-06-09 09:00:43 +00:00
vorotamoroz 0856693aac Add instruction 2026-06-09 02:26:07 +01:00
vorotamoroz 39d78a04ac Enhance remote database management and add --vault option
Added new commands for remote database management and introduced --vault option for daemon and mirror commands.
2026-06-08 21:12:42 +09:00
vorotamoroz 0b8d73ccd8 Merge pull request #948 from vrtmrz/adjust_overwrite_prevention
Adjust overwrite prevention
2026-06-08 19:57:20 +09:00
vorotamoroz 5921a71227 Add CI 2026-06-08 10:52:00 +00:00
vorotamoroz a40929c9e4 fixed: enhance conflict handling by adding settings check for document writes 2026-06-08 10:47:34 +00:00
vorotamoroz 2d8a285201 Port new tests 2026-06-08 10:38:54 +00:00
vorotamoroz d9903bfe9e Merge pull request #947 from vrtmrz/0_25_74
Release: 0.25.74
2026-06-08 19:29:33 +09:00
vorotamoroz a6e7dddf7f Merge branch '0_25_74' into cli_test_deno 2026-06-08 11:29:06 +01:00
vorotamoroz 9a51c78011 tweak specificity 2026-06-08 11:18:35 +01:00
vorotamoroz 7b9c0b011f bump 2026-06-08 11:15:13 +01:00
vorotamoroz 26a050e6f6 Merge pull request #946 from vrtmrz/cli_vaultpath
feat:  decouple the database and vault directories
2026-06-08 19:12:11 +09:00
vorotamoroz 34162f747c fix corrupted test, update submodule 2026-06-08 11:10:37 +01:00
vorotamoroz 9e87ee4da1 Merge branch 'main' into cli_vaultpath 2026-06-08 19:07:02 +09:00
vorotamoroz ba5d4c434b Merge pull request #945 from vrtmrz/fix_941
fixed: ignore hidden files completely if hidden file sync disabled
2026-06-08 18:57:09 +09:00
vorotamoroz 8f44f4c61d Merge remote-tracking branch 'origin/main' into fix_941 2026-06-08 10:55:47 +01:00
vorotamoroz 23d8a1ecc3 update submodule 2026-06-08 10:51:08 +01:00
vorotamoroz a46abcad52 Merge branch 'main' into fix_941 2026-06-08 10:50:34 +01:00
vorotamoroz 347e1fcee5 Merge pull request #944 from vrtmrz/fix_911
fixed: fix automatic merging behaviour
2026-06-08 18:47:29 +09:00
vorotamoroz d85d8bc5e4 Merge branch 'main' into fix_911 2026-06-08 18:45:35 +09:00
vorotamoroz c493fb6f66 Merge pull request #942 from vrtmrz/feat_cli_database_commands
feat: Added new remote management commands
2026-06-08 18:44:32 +09:00
vorotamoroz cf173caf88 feat: decouple the database and vault directories 2026-06-08 10:43:10 +01:00
vorotamoroz a41c7b2f70 Merge pull request #928 from starskyzheng/feat/cli-daemon-vault-option
feat(cli): add --vault option for daemon and mirror commands
2026-06-08 18:03:25 +09:00
vorotamoroz 3acaa7f269 update readme 2026-06-08 09:26:43 +01:00
vorotamoroz 85787eddb3 fixed: ignore hidden files completely if hidden file sync disabled 2026-06-08 09:24:59 +01:00
vorotamoroz 8cbb233df7 fixed: fix automatic merging behaviour 2026-06-08 09:04:12 +01:00
vorotamoroz 5922186a0e feat: Added new remote management commands 2026-06-08 04:23:22 +00:00
vorotamoroz 60f21eb9d2 detect loopback and coturn option 2026-06-05 09:44:17 +01:00
vorotamoroz 6b7816d334 add coturn for test 2026-06-05 09:39:39 +01:00
vorotamoroz 369e62ee8d Improved: we can set empty for turnServer explicitly. 2026-06-05 09:27:19 +01:00
vorotamoroz 37593bbee6 Update CI to use deno 2026-06-05 09:07:38 +01:00
vorotamoroz baa51a66a7 Merge branch 'main' into cli_test_deno 2026-06-05 08:34:31 +01:00
郑泽宇 be979a3bf1 fix(cli): respect mirror positional arg before --vault flag
Per vrtmrz's review feedback, restore the mirror [vault-path] positional
argument support with correct priority order:
  mirror positional arg > --vault flag > databasePath

Also update --vault help text and CLI README with the new option.
2026-06-05 11:50:23 +08:00
Miguel Ferreira c78e583399 feat: opt-in desktop setting to keep replication active in the background
Replication is suspended when the Obsidian window becomes hidden (document.hidden),
so LiveSync and Periodic stop syncing while minimised until the window is focused.

Add keepReplicationActiveInBackground (default off, desktop only). When enabled, the
window-visibility handler no longer suspends on hide, so replication keeps running while
minimised. Becoming visible forces a teardown before reopen (LiveSync only) so a stalled,
half-open channel is always replaced.

Includes the setting definition (src/lib submodule), a desktop-only toggle in the Sync
pane shown for LiveSync and Periodic, a docs/settings.md entry, and unit tests for the
visibility handler.
2026-06-05 00:06:26 +01:00
vorotamoroz 31050c9cb8 update docs 2026-06-05 03:05:13 +09:00
vorotamoroz 1b44ab9f2b docs: update documentation 2026-06-05 01:57:03 +09:00
vorotamoroz 3a78c70539 (chore) update terms 2026-06-05 01:24:46 +09:00
vorotamoroz 0f08ad435f (chore): improve English 2026-06-05 01:08:54 +09:00
vorotamoroz 96abd930b3 Merge pull request #938 from vrtmrz/0_25_73
Release 0.25.73
2026-06-04 18:50:11 +09:00
vorotamoroz 4581db45c4 bump 2026-06-04 10:44:56 +01:00
vorotamoroz 05cab8ec66 Merge pull request #937 from vrtmrz/fix_926
fix: adjust CouchDB's database name checking to its specification
2026-06-04 18:39:38 +09:00
vorotamoroz b005625ef3 Merge branch 'main' into fix_926 2026-06-04 18:38:26 +09:00
vorotamoroz e5408b4dd7 Merge pull request #936 from vrtmrz/fix_path_handling_on_windows_cli
Fix: No longer path corruption on windows environment
2026-06-04 18:37:47 +09:00
vorotamoroz 95a9b1b41c Merge pull request #935 from vrtmrz/fix_first_fetch2
fixed: `Reset Syncronisation on This Device` for minio and P2P
2026-06-04 18:37:23 +09:00
vorotamoroz 2aa8bc1165 fix: adjust CouchDB's database name checking to its specification (#926). 2026-06-04 10:35:15 +01:00
vorotamoroz f8998d5441 (fixed): No longer path corruption on windows environment (at least, pass the check) on CLI 2026-06-04 10:17:34 +01:00
vorotamoroz 26f5f54f24 fixed: Reset Syncronisation on This Device for minio and P2P is now working properly. 2026-06-04 09:40:27 +01:00
vorotamoroz 99f3aca024 Merge pull request #933 from vrtmrz/0_25_71
0 25 72
2026-06-03 20:22:29 +09:00
vorotamoroz 194397dd94 bump again 2026-06-03 12:12:21 +01:00
vorotamoroz afa79d78dc bump 2026-06-03 11:18:25 +01:00
vorotamoroz 9a9440a768 Merge branch 'improve_first_fetch' into 0_25_71 2026-06-03 11:03:46 +01:00
vorotamoroz e375860af8 fix importing issue and unit test issue 2026-06-03 06:58:19 +01:00
vorotamoroz b57f34c15b Revise devs note 2026-06-03 01:13:50 +00:00
vorotamoroz b82fd9f04b Revise redflag.md option descriptions
Updated descriptions for redflag options in troubleshooting documentation.
2026-06-02 21:14:42 +09:00
vorotamoroz bb77426b7b update dependencies and bump 2026-06-02 12:50:46 +01:00
vorotamoroz 1bef5fbef3 Merge branch 'fix_warns' into improve_first_fetch 2026-06-02 12:36:32 +01:00
vorotamoroz 7d2ba1b0b9 ### Improved
- Database fetching (a.k.a. Reset Synchronisation on This Device) on the initialisation now supports streaming and is faster (CouchDB only)
- The database fetching process has been streamlined, and database operations are now suspended until it has been completed
- The initial synchronisation process has been simplified, making it easier to synchronise files with the remote server
- We can select the remote database to fetch from during the initialisation, when there are multiple remote databases configured (e.g. multiple CouchDBs or S3 remotes)
2026-06-02 12:34:46 +01:00
vorotamoroz ac6b9a4dad fix bad English 2026-06-01 12:13:01 +01:00
vorotamoroz 225e2c5096 bump 2026-06-01 12:10:56 +01:00
vorotamoroz 674d68b7d9 Fixed:
-  No longer the status element breaks other plugins' interaction (#930).
2026-06-01 12:05:31 +01:00
vorotamoroz 0e6dd300ef Merge branch 'fix_warns' into improve_first_fetch 2026-06-01 11:22:01 +01:00
vorotamoroz 8171db353a bump 2026-06-01 11:19:42 +01:00
vorotamoroz 6ab1556880 prettify 2026-06-01 11:19:32 +01:00
vorotamoroz cd2bff5fc7 Refactor types in svelte components. 2026-06-01 11:18:23 +01:00
vorotamoroz 5a280c7919 (feat): Bulk database fetching is now work in progress. This feature is expected to speed up rebuilds and setups. (WIP) 2026-06-01 10:41:48 +01:00
vorotamoroz c6697327d5 Fixed typings
Fixed wrong typing for serviceHub, svelte dialog
2026-06-01 06:20:33 +01:00
vorotamoroz 7c203a522a Aligned to eslint rules and fixed following things:
This commit may contains behavioural changes.

- Fix for the issue with corrupted log displays
- Wrap the activeDocument
- Reduced potential type errors and strengthened certain checks
- Made error handling more robust (by rewriting the error class)
2026-06-01 05:28:03 +01:00
vorotamoroz c80c294d93 satisfies
Address following rules
-
- @typescript-eslint/unbound-method
- obsidianmd/prefer-active-doc
- obsidianmd/prefer-window-timer

Improve typing
2026-06-01 04:20:47 +01:00
vorotamoroz 3e65ae932d Address following rules
- @typescript-eslint/no-redundant-type-constituents
2026-06-01 03:50:51 +01:00
vorotamoroz f710f03380 Address following rules
- @typescript-eslint/no-redundant-type-constituents
- @typescript-eslint/no-unnecessary-type-assertion
2026-06-01 03:37:55 +01:00
vorotamoroz b887269fc1 change eslint for more progressive 2026-06-01 02:37:07 +01:00
vorotamoroz 56a234e6d7 chore: wrap timer functions 2026-05-31 19:03:43 +09:00
vorotamoroz 39014b2294 use coreEnvVars for some vars 2026-05-30 23:59:36 +09:00
vorotamoroz 24e6c110a3 Remove unused imports 2026-05-30 23:53:19 +09:00
vorotamoroz f24d110552 Change type assertion 2026-05-30 23:49:32 +09:00
vorotamoroz 7189c1c05a Update dependency
set no-deprecated to warn
2026-05-30 23:42:32 +09:00
vorotamoroz 547afe9a86 remove unused eslint comment
use _fetch instead of fetch
add ignore list
2026-05-30 23:32:46 +09:00
vorotamoroz 7b5876037d Add ignores 2026-05-30 23:22:17 +09:00
vorotamoroz b714c00644 (chore): Improve type assertion, remove unused imports 2026-05-29 04:19:58 +01:00
郑泽宇 fff6df535f feat(cli): add --vault option for daemon and mirror commands
Allow specifying a separate vault directory for .md files, decoupled from
the database directory (where PouchDB data lives).

- Add --vault/-V flag to CLI options parser
- Use vaultPath (or fallback to databasePath) for file system operations
- Works with both daemon and mirror commands
- Log vault path alongside database path at startup
2026-05-28 16:44:34 +08:00
vorotamoroz e14e771bfb (chore): tidied tsconfig and eslint, and some incorrect imports 2026-05-28 04:42:03 +01:00
vorotamoroz 1130bbcee8 remove unmaintained tests 2026-05-26 11:33:14 +01:00
vorotamoroz 8841ef4619 fix css duplicated props 2026-05-26 11:27:31 +01:00
vorotamoroz 45fe0b3682 chore: lint: enable cache and concurrency 2026-05-26 11:24:41 +01:00
vorotamoroz 8d3825abc9 Merge pull request #925 from vrtmrz/v0_25_70
Releasing v0.25.70
2026-05-25 18:38:04 +09:00
vorotamoroz c5f9841b85 chore: fix grammatical error 2026-05-25 10:14:18 +01:00
vorotamoroz d36d176d99 bump 2026-05-25 10:10:46 +01:00
vorotamoroz 38b2cf73ed Update lib (forgot to include #942) 2026-05-25 09:55:11 +01:00
vorotamoroz 40b15a6950 Merge pull request #924 from vrtmrz/p2p_add_fix_and_diag
Improvements: More diagnostic information for P2P connections
2026-05-25 17:52:37 +09:00
vorotamoroz e312bb7640 Merge pull request #891 from SeleiXi/feat/conflict-diff-jump
feat: add diff navigation to conflict resolver
2026-05-25 17:51:49 +09:00
Ching Wing Kwok 852c0e6c13 Merge branch 'main' into feat/conflict-diff-jump 2026-05-25 15:38:21 +08:00
vorotamoroz 7c1bcf9e9b Merge pull request #889 from SeleiXi/diff-only-button
feat: add diff-only view button to document history
2026-05-25 14:00:11 +09:00
vorotamoroz 2b79bed085 Merge branch 'main' into pr/SeleiXi/889 2026-05-25 05:43:36 +01:00
vorotamoroz 6b1e0c4aa8 Merge pull request #890 from SeleiXi/feat/history-search
feat: Add document history search
2026-05-25 12:55:29 +09:00
vorotamoroz 3c3645eba4 Improved: More diagnostic information for P2P connections is now shown, including why a connection failure occurred and the current connection status. 2026-05-25 04:31:22 +01:00
SeleiXi 009cc3c87a Merge remote-tracking branch 'origin/main' into feat/conflict-diff-jump
# Conflicts:
#	src/modules/features/InteractiveConflictResolving/ConflictResolveModal.ts
2026-05-23 02:06:52 +08:00
SeleiXi fc5fd4be94 Merge remote-tracking branch 'origin/main' into feat/history-search
# Conflicts:
#	src/lib
#	src/modules/features/DocumentHistory/DocumentHistoryModal.ts
2026-05-23 02:05:33 +08:00
SeleiXi 8ed1acf79d Merge remote-tracking branch 'origin/main' into diff-only-button
# Conflicts:
#	src/modules/features/DocumentHistory/DocumentHistoryModal.ts
2026-05-23 02:04:48 +08:00
vorotamoroz c518223d21 Merge pull request #923 from vrtmrz/p2p_add_fix_and_diag
0.25.69: Fix P2P, add diag feature
2026-05-23 00:47:36 +09:00
vorotamoroz caaff618e9 fix grammar 2026-05-22 16:40:40 +01:00
vorotamoroz 148aa8505e bump 2026-05-22 16:38:44 +01:00
vorotamoroz f9a626a858 ### Fixed
- No longer the P2P passphrase mismatch causes a server shutdown.
- Settings related to P2P synchronisation are now correctly applied on start-up and no longer reverted.

### New features
- Diagnostic P2P connection stats are now available.
  - These stats indicate the number of connection trials, successes, and, failures.
2026-05-22 16:37:05 +01:00
vorotamoroz 2f10121d6c Merge remote-tracking branch 'origin/main' into cli_test_deno 2026-05-22 10:19:30 +00:00
vorotamoroz 1b8747115c Merge pull request #922 from vrtmrz/update_trystero
Releasing v0.25.68 (dependency and P2P update)
2026-05-22 19:17:22 +09:00
vorotamoroz e739302fb9 bump 2026-05-22 11:11:18 +01:00
vorotamoroz 8f20d53f55 Update trystero, typed well. fixed some potentially problems
Weaken terser for libraries
2026-05-22 11:01:51 +01:00
vorotamoroz 3ab80190d6 test fix ci (Redundant test) 2026-05-22 03:48:41 +00:00
vorotamoroz 8948bf2803 test cli:p2p use nonce for peername 2026-05-22 03:48:02 +00:00
vorotamoroz 486fd15c60 fix resouce handling 2026-05-22 03:46:56 +00:00
vorotamoroz 5fd85c71ca test: chore: prettify 2026-05-22 03:20:28 +00:00
vorotamoroz c1f41910c4 test: add actions / caching 2026-05-22 03:20:11 +00:00
vorotamoroz 3693d6a6b6 test: add port ready, container cleanup 2026-05-22 03:19:48 +00:00
vorotamoroz cc3c992b1d cli: add large-file-test and benchmark between couchdb and p2p 2026-05-22 03:05:44 +00:00
vorotamoroz df390ac456 test: fix deno test helpers 2026-05-22 03:02:11 +00:00
vorotamoroz fd84b0377b Update actions 2026-05-21 10:02:46 +01:00
vorotamoroz 340d416b76 Merge pull request #920 from vrtmrz/v0_25_66
Release v0.25.67
2026-05-20 19:44:41 +09:00
vorotamoroz 3034af8d69 bump again 2026-05-20 11:37:51 +01:00
vorotamoroz da3020bd45 fixed: fix auto-correction mismatch 2026-05-20 11:34:28 +01:00
vorotamoroz ce232c1002 bump 2026-05-20 11:14:41 +01:00
vorotamoroz 0e13926400 fixed: update lib
I should do this with #919
2026-05-20 11:10:43 +01:00
vorotamoroz fab7ec996a Merge pull request #919 from vrtmrz:feat_cli_remote_select
feat: add CLI commands to handle multiple remote configuration
2026-05-20 19:05:49 +09:00
vorotamoroz 88e22f99c5 Merge pull request #917 from vrtmrz:feat_tweak_auto_adjust
feat: implement auto-accept compatible tweak
2026-05-20 19:02:58 +09:00
vorotamoroz 83cbabf06f Merge branch 'main' into feat_tweak_auto_adjust 2026-05-20 11:01:51 +01:00
vorotamoroz 5e8d3b8f02 Update lib 2026-05-20 11:00:38 +01:00
vorotamoroz 1167b41340 feat: add CLI commands to handle multiple remote configuration 2026-05-20 05:10:42 +01:00
vorotamoroz 67da3964e5 Merge pull request #918 from vrtmrz/v0_25_65
V0.25.65
2026-05-19 20:27:34 +09:00
vorotamoroz 45ebc7eb6b Update the doc and issue report, and bump 2026-05-19 12:17:26 +01:00
vorotamoroz cc5ead68bc minor layout fix 2026-05-19 11:59:54 +01:00
vorotamoroz 9b9e4f22f3 Merge pull request #916 from vrtmrz:fix_866
fixed: Now Chunk Splitter: `V3: Fine Deduplication`
2026-05-19 19:46:08 +09:00
vorotamoroz 7823f46053 Merge branch 'main' into fix_866 2026-05-19 11:45:41 +01:00
vorotamoroz d6d8e548b3 Merge pull request #915 from vrtmrz/improve_log_feature
Improve log features
2026-05-19 19:40:28 +09:00
vorotamoroz 44b1ed7610 Merge branch 'main' into improve_log_feature 2026-05-19 11:40:14 +01:00
vorotamoroz 5786da5534 merged 2026-05-19 11:38:05 +01:00
vorotamoroz 042a80dd44 Merge pull request #914 from vrtmrz/fix_ios_resume
fix: Fix an issue about resuming from background on iOS (#888).
2026-05-19 19:34:49 +09:00
vorotamoroz ee30f6cd6d fixed: Now Chunk Splitter: V3: Fine Deduplication is working fine again (#866). 2026-05-19 11:26:35 +01:00
vorotamoroz 977a300808 fix grammatical error 2026-05-19 04:10:39 +01:00
vorotamoroz a392ccab6a - Improved an error verbosity on concurrent processing on start-up process.
- Now the `report` includes recent logs (of verbosity `verbose` even settings is not set to `verbose`).
- Updating logs is now debounced to avoid excessive updates during rapid log generation.
- Added a `Generate full report for opening the issue with debug info` command to the command palette, which generates a report without opening the settings dialogue.
2026-05-19 04:09:04 +01:00
vorotamoroz dfdfa5383b grammatical fix 2026-05-19 01:55:10 +01:00
vorotamoroz a08294ab16 fix: Fix an issue about resuming from background on iOS (#888). 2026-05-18 12:30:03 +01:00
vorotamoroz e8c33a0d6a feat: implement auto-accept compatible tweak setting and enhance mismatch resolution logic 2026-05-18 11:21:53 +01:00
vorotamoroz d6bf453a6d Merge branch 'main' of https://github.com/vrtmrz/obsidian-livesync 2026-05-18 11:07:43 +01:00
vorotamoroz e80cdc2dae remove unused files 2026-05-18 08:47:03 +01:00
vorotamoroz 60780678fd Merge pull request #903 from Joysimple/cli-docker
bugfix: Add package-lock.json into docker build
2026-05-18 12:38:12 +09:00
vorotamoroz 273e7a2b63 Merge pull request #909 from vrtmrz/p2p_config_selectable
P2p config selectable (a.k.a. v0.25.64)
2026-05-17 13:24:40 +09:00
vorotamoroz 2572c54744 Track main 2026-05-17 13:17:09 +09:00
vorotamoroz 6a7c987985 bump 2026-05-17 13:13:21 +09:00
vorotamoroz 6ef866a77c P2P: Enhance status pane and card with active remote selection and replication features
- Added active P2P remote selector and creation option in the status pane.
- Introduced immediate replication action for accepted peers.
- Updated status control icons for clarity.
- Display stable Room ID suffix above Peer ID in the status card.
- Implemented dedicated active remote configuration for P2P features.
- Added migration support for P2P active remote selection.
- Improved unit test coverage for P2P settings.
2026-05-17 13:08:51 +09:00
vorotamoroz eea26dee74 Update about P2P 2026-05-17 02:35:20 +09:00
vorotamoroz ee24fe8c24 Merge pull request #904 from vrtmrz/enhance_fix_p2p
Fixed: fixed P2P bugs and and implement new UI
2026-05-17 02:32:50 +09:00
vorotamoroz 9d9364af36 Fix updates.md 2026-05-17 02:24:58 +09:00
vorotamoroz 83228e2077 Fix P2P replicator creation and enhance error handling in synchronization functions 2026-05-17 02:23:58 +09:00
vorotamoroz a379b5bd78 bump 2026-05-17 01:40:50 +09:00
vorotamoroz 4ed1749652 Enhance P2P synchronization features and UI improvements 2026-05-17 01:36:09 +09:00
vorotamoroz 9a90256a8a Enhance P2P synchronization features and UI improvements 2026-05-16 23:50:08 +09:00
vorotamoroz f0628a0d2c Improve UI 2026-05-16 23:09:11 +09:00
vorotamoroz d5e2f57781 Fixed: fixed P2P bugs and and implement new UI 2026-05-15 10:18:53 +01:00
Nikolay Sokolov 02673a1631 bugfix: Add package-lock.json into docker build 2026-05-14 23:17:37 -07:00
vorotamoroz 91c9746886 Merge pull request #900 from vrtmrz/v0_25_62
Releasing v0.25.62
2026-05-14 20:15:02 +09:00
vorotamoroz 75b44b1636 bump 2026-05-14 09:40:12 +00:00
vorotamoroz f1fe48c1ee add version-bump 2026-05-14 09:35:53 +00:00
vorotamoroz 437e7c0d9c Fixed an issue where a connection could not be established when attempting to connect to a brand-new remote database without going through the set-up wizard or configuration checking 2026-05-14 09:32:34 +00:00
vorotamoroz 5ffa7ec7ee Merge pull request #897 from vrtmrz/v0_25_61
Releaseing v0.25.61
2026-05-13 23:06:57 +09:00
vorotamoroz a1859f5d2e bump 2026-05-13 14:44:38 +01:00
vorotamoroz 785af8cb8f Merge pull request #896 from vrtmrz/address_community_review
Address community review
2026-05-13 22:29:37 +09:00
vorotamoroz 06e1f4aa4a Update subrepo pointer 2026-05-13 14:25:49 +01:00
vorotamoroz 767f22ce9c Merge branch 'address_community_review' of https://github.com/vrtmrz/obsidian-livesync into address_community_review 2026-05-13 14:16:51 +01:00
vorotamoroz 6a9bba702c chore: ran prettier 2026-05-13 14:10:56 +01:00
vorotamoroz de2397dc3f Adding a rough DI 2026-05-13 14:10:55 +01:00
vorotamoroz daaad9212e Fix package-lock 2026-05-13 14:10:54 +01:00
vorotamoroz a6891374a1 chore: Package modernise, update linter 2026-05-13 14:10:01 +01:00
vorotamoroz b1cadf0549 prettify 2026-05-13 14:07:58 +01:00
vorotamoroz 95f40cc954 (chore): removing DOM Operation 2026-05-13 14:07:58 +01:00
vorotamoroz 8deaf123d6 Update eslint config to ignore file,
fix some type error on LiveSyncBaseCore
2026-05-13 14:06:51 +01:00
vorotamoroz 053813bffb Update for review once 2026-05-13 14:06:51 +01:00
vorotamoroz cc7af03618 chore: Package modernise, update linter 2026-05-13 14:06:51 +01:00
vorotamoroz a130e3700e prettify 2026-05-13 14:06:50 +01:00
vorotamoroz 0549e901b2 (chore): removing DOM Operation 2026-05-13 14:06:49 +01:00
vorotamoroz e9afe06968 Merge pull request #895 from vrtmrz/update_lib
Update lib to fix P2P problems and merging contributions
2026-05-13 22:01:17 +09:00
vorotamoroz c45aca4794 fixed: fixed subrepo pointer
I thought I’d rebased it, but it turns out everything had been merged.
2026-05-13 13:45:09 +01:00
vorotamoroz e2c54aaf43 Update lib to fix P2P problems 2026-05-13 13:31:11 +01:00
vorotamoroz 37715d4c9f chore: ran prettier 2026-05-13 11:12:40 +00:00
vorotamoroz 106367fa41 Adding a rough DI 2026-05-13 11:09:04 +00:00
vorotamoroz 538130aa91 Fix package-lock 2026-05-13 11:36:01 +01:00
vorotamoroz c9d0357fec Merge branch 'address_community_review' of https://github.com/vrtmrz/obsidian-livesync into address_community_review 2026-05-13 11:35:01 +01:00
vorotamoroz d05c76da36 Update eslint config to ignore file,
fix some type error on LiveSyncBaseCore
2026-05-13 11:33:46 +01:00
vorotamoroz d2eb6ecbaf Update for review once 2026-05-13 11:33:46 +01:00
vorotamoroz 25a6fde212 chore: Package modernise, update linter 2026-05-13 11:33:45 +01:00
vorotamoroz e8f8b680ef prettify 2026-05-13 11:33:03 +01:00
vorotamoroz 6c30f2b863 (chore): removing DOM Operation 2026-05-13 11:33:03 +01:00
vorotamoroz 8dda24a689 Merge pull request #882 from vrtmrz/p2p-rpc
feat: use new p2p-rpc wrapper
2026-05-13 19:24:26 +09:00
vorotamoroz fbbb63906a Merge branch 'main' into p2p-rpc 2026-05-13 19:22:03 +09:00
vorotamoroz 1e66a7f144 Merge pull request #894 from vrtmrz/fix_unexpected_error_on_startup
fixed: fixed unexpected error during startup
2026-05-13 19:16:18 +09:00
vorotamoroz df79d81475 fixed: fixed unexpected error during startup 2026-05-13 10:14:47 +00:00
vorotamoroz ad71355859 Merge pull request #893 from brian-spackman/fix-fractional-mtime-on-linux
fix: truncate sub-millisecond CLI mtimes to prevent mobile crash
2026-05-13 19:12:56 +09:00
vorotamoroz 95dc079fad Merge pull request #843 from andrewleech/daemon-sync
cli: implement continuous sync daemon mode
2026-05-13 18:53:59 +09:00
vorotamoroz 770d4af4a0 Update eslint config to ignore file,
fix some type error on LiveSyncBaseCore
2026-05-13 10:15:45 +01:00
vorotamoroz 3b311248cb Update for review once 2026-05-13 08:02:50 +01:00
Andrew Leech 67996f6d0a cli: fix stale stat.size in NodeVaultAdapter causing corrupted file errors
chokidar stats are captured at poll time and may not reflect the file's
final byte length by the time vault.read() is called. The downstream
integrity check compares stat.size to content length; a mismatch causes
other LiveSync clients to reject the file as corrupted.

Fix by updating file.stat.size from the actual content in read() and
readBinary().

Co-authored-by: Joysimple <Joysimple@users.noreply.github.com>
2026-05-13 16:56:08 +10:00
vorotamoroz 5772811a45 chore: Package modernise, update linter 2026-05-13 04:40:32 +01:00
vorotamoroz 55529cd71e prettify 2026-05-13 03:58:08 +01:00
vorotamoroz 2e9b8b7b62 (chore): removing DOM Operation 2026-05-13 03:55:11 +01:00
Andrew Leech 4ab2e41d18 cli daemon: set disableCheckingConfigMismatch for headless operation
The config mismatch dialog's defaultAction is "Dismiss" which blocks
replication. Since the daemon cannot resolve mismatches interactively,
skip the check entirely and accept the remote configuration as-is.
2026-05-13 11:21:06 +10:00
Andrew Leech c0ad8ee15a cli: add configurable ignore rules and deployment artifacts
IgnoreRules (src/apps/cli/serviceModules/IgnoreRules.ts):
- Reads .livesync/ignore for user-defined glob patterns
- Applies gitignore matchBase semantics: patterns without / get **/ prefix,
  patterns ending with / get ** appended for directory contents
- Supports `import: .gitignore` directive to merge gitignore patterns
- Rejects negation patterns with a warning (not fully supportable)
- Integrated into both daemon and mirror commands via isTargetFile handler

Wiring:
- IgnoreRules loaded before LiveSyncBaseCore construction so beginWatch()
  receives rules when it fires during onLoad/onFirstInitialise
- Passed through initialiseServiceModulesCLI -> StorageEventManagerCLI ->
  CLIStorageEventManagerAdapter -> CLIWatchAdapter

Deployment:
- src/apps/cli/deploy/livesync-cli.service - systemd unit template
- src/apps/cli/deploy/install.sh - user/system install script

Testing:
- src/apps/cli/test/test-daemon-linux.sh - e2e tests for ignore rules
- src/apps/cli/serviceModules/IgnoreRules.unit.spec.ts - 15 unit tests
- src/apps/cli/commands/daemonCommand.unit.spec.ts - 7 unit tests
2026-05-13 11:21:06 +10:00
Andrew Leech e6ae516493 cli: implement local→CouchDB file watching via chokidar
- Add chokidar ^4.0.0 as dependency (root package.json, runtime-package.json)
- Mark chokidar as external in vite.config.ts (not bundled, loaded at runtime)
- Implement CLIWatchAdapter.beginWatch() with chokidar:
  - ignoreInitial: true (startup files handled by mirror scan)
  - awaitWriteFinish to prevent partial-write events
  - Excludes dotfiles and .livesync/ directory at watcher level
  - Maps add/change/unlink/addDir/unlinkDir to IStorageEventWatchHandlers
  - Fatal error handler: logs clearly and releases watcher resources
- Add close() to CLIWatchAdapter, StorageEventManagerCLI for clean shutdown
- Register onUnload hook in CLIServiceModules to close watcher on shutdown
2026-05-13 11:21:06 +10:00
Andrew Leech a4d5ef4620 cli: implement daemon startup sequence and CouchDB→local sync
- Add daemon command to help text and --interval/-i flag for polling mode
- Capture original sync settings before suspendAllSync() clobbers them
- Implement daemon startup: mirror scan → restore settings → applySettings()
  which triggers the full suspend/resume lifecycle and starts the _changes feed
- Guard processSynchroniseResult no-op to non-daemon commands so default
  handler writes incoming CouchDB changes to the local filesystem
- Polling mode: restore settings + clearInterval-safe try/catch error handling
- Warn when both liveSync and syncOnStart are false after restore (no-op config)
- Fix: only block indefinitely if daemon startup succeeded
2026-05-13 11:21:06 +10:00
Brian Spackman 3f7bb047ac fix: floor sub-millisecond CLI mtimes to prevent mobile crash
On Linux, fs.Stats.mtimeMs and ctimeMs return floats with sub-millisecond
precision derived from the kernel's nanosecond filesystem mtime. Stored
raw, this produces document timestamps like 1778511180024.462 in CouchDB
rather than integer milliseconds.

Mobile clients running LiveSync 0.25.60 have been observed to crash when
processing change-feed updates carrying non-integer millisecond timestamps
from CLI-written documents. Desktop and mobile GUI plugins write integer
milliseconds, so the crash only manifests when the headless CLI on Linux
is the source. Whether the issue was introduced in 0.25.60 or had been
latent in earlier versions hasn't been investigated; 0.25.60 is the
version where the crash was confirmed and the fix verified.

Floor the values at every stat-read site (six across three adapters and
one command) so CLI-written documents carry integer-millisecond
timestamps consistent with the rest of the mesh.
2026-05-12 18:00:25 -06:00
SeleiXi 5454e1106f feat: add diff navigation to conflict resolver 2026-05-13 00:19:56 +08:00
SeleiXi 0d9397c8b9 fix: resolve UI alignment issue for diff navigation buttons 2026-05-12 00:52:20 +08:00
SeleiXi 429a3ff1fd feat: add diff-only view button to document history 2026-05-11 23:53:07 +08:00
SeleiXi bfff6ea7b8 feat: add document history search support 2026-05-11 22:45:42 +08:00
vorotamoroz b6b153c0de Merge pull request #887 from vrtmrz/add_ignore_to_eslint
chore: Change eslint config to ignore _tools
2026-05-11 21:01:47 +09:00
vorotamoroz eca6a6e0ba chore: Change eslint config to ignore _tools 2026-05-11 13:00:32 +01:00
vorotamoroz ca43d96c46 Merge pull request #886 from vrtmrz/fix_prettier
Fix prettier config
2026-05-11 20:34:22 +09:00
vorotamoroz 112e3c8b1d Fix prettier config 2026-05-11 12:33:32 +01:00
vorotamoroz d1eb105801 Merge pull request #872 from OriBoharon/make-cli-onboarding-easier
added documentaion and a hook build script to make onbaording easier when trying to build the cli app
2026-05-11 18:43:00 +09:00
vorotamoroz d5b93e89cd Change the default issue report label from 'bug' to 'uncategorised' 2026-05-11 17:55:31 +09:00
vorotamoroz e96fe7cde1 Merge pull request #885 from vrtmrz/tidy_file
(chore) remove obsoleted file
2026-05-11 17:52:22 +09:00
vorotamoroz 68e0610f1d (chore) remove obsoleted file 2026-05-11 09:49:32 +01:00
vorotamoroz a6be20695a feat: use new p2p-rpc wrapper 2026-05-11 03:49:35 +01:00
vorotamoroz 772b6ecf26 Merge pull request #871 from SeleiXi/feat/diff-navigation-buttons
feat: Add diff navigation buttons for Document History
2026-05-09 22:51:04 +09:00
SeleiXi 81dc7f604b feat: auto navigation to diff 2026-05-09 14:07:08 +08:00
vorotamoroz a9c87fa52e - Add default test environment
- Fixed to use environment by APIs
- Make test parallel
2026-05-08 03:04:14 +00:00
vorotamoroz e81f023943 Add default test env 2026-05-08 03:01:22 +00:00
vorotamoroz 2afe12ad2d fix pattern 2026-05-07 11:28:01 +01:00
vorotamoroz 4a9d6c1349 Add ci 2026-05-07 11:23:51 +01:00
vorotamoroz 279fc8876e feat(tests): enhance push/pull test with Docker integration and improved environment variable handling
style(test): format comment
2026-05-07 11:22:56 +01:00
vorotamoroz cc3d30dbcf feat(tests): add Deno-based tests for checking CLI functionality in the same-codebase between platforms. 2026-05-07 11:06:12 +01:00
vorotamoroz 39e82cc8a1 Fixed: Fix timing issue during test 2026-05-06 21:56:13 +09:00
bori 7a4b76a550 added documentaion and a hook build script to make onbaording easier 2026-05-02 18:51:07 +03:00
SeleiXi f9294446ba feat: add diff block navigation to Document History modal
Add prev/next buttons to jump between diff blocks in the
Document History view. Includes position indicator and
auto-scroll with visual focus highlighting.
2026-05-02 22:18:43 +08:00
vorotamoroz fa7ef62302 Fix: adjusting help
Co-authored-by: Copilot <copilot@github.com>
2026-04-29 18:42:54 +09:00
vorotamoroz 81d8224330 bump
Co-authored-by: Copilot <copilot@github.com>
2026-04-29 18:39:48 +09:00
vorotamoroz cc466a4b3c ### Fixed
- Now larger settings can be exported and imported via QR code without issues. (#595)

- Fixed some errors during serialisation and deserialisation of the settings, which caused issues in some cases when importing/exporting settings via QR code.

Co-authored-by: Copilot <copilot@github.com>
2026-04-29 18:37:44 +09:00
vorotamoroz ceebca7de9 Merge pull request #862 from fabiomanz/main
chore: remove obsolete `version` attribute from docker-compose.yml
2026-04-29 17:30:35 +09:00
Fabio c2f696d0a4 chore: attribute version is obsolete 2026-04-29 07:07:45 +00:00
vorotamoroz 1aa7c45794 Fix the readme
Co-authored-by: Copilot <copilot@github.com>
2026-04-29 12:55:34 +09:00
vorotamoroz faefa80cbd Fix again 2026-04-29 12:40:40 +09:00
vorotamoroz 3737eacffd Fix readme
Co-authored-by: Copilot <copilot@github.com>
2026-04-29 12:39:42 +09:00
vorotamoroz 4c0af0b608 Fixed(cli):
- `ls` and `mirror` commands now provide informative feedback when no documents are found or filters skip all files, resolving the issue where they would exit silently (#860).
- The command-line argument `vault` has been renamed to a more appropriate name, `databaseDir`.
- The `mirror` command now accepts a `vault` directory, which specifies the location where the actual files are stored. For compatibility reasons, the previous behaviour is still supported.

Co-authored-by: Copilot <copilot@github.com>
2026-04-29 12:22:00 +09:00
vorotamoroz bb69eb13e7 bump 2026-04-27 11:15:07 +09:00
vorotamoroz 7c9db6376f Fixed:
- No longer Setup-wizard drops username and password silently. (#865)
- Setup URI is now correctly imported (#859).
- now French translation is added.
2026-04-27 11:14:06 +09:00
vorotamoroz 4c04e4e676 Merge pull request #863 from koteitan/fix/859-strip-trailing-slash-from-uri
fix: strip trailing slash from couchDB_URI to avoid double-slash 401
2026-04-27 11:08:11 +09:00
koteitan 14ec35b257 fix: strip trailing slash from couchDB_URI to avoid double-slash 401
When couchDB_URI ends with a trailing slash (e.g. https://host/), the
database name concatenation produces a double-slash path
(https://host//obsidiannotes), which causes CouchDB to reject requests
with 401 "Name or password is incorrect".

Strip trailing slashes from couchDB_URI / baseUri at the path
concatenation sites in:
- src/common/utils.ts (_requestToCouchDBFetch, _requestToCouchDB)
- src/features/LocalDatabaseMainte/CmdLocalDatabaseMainte.ts

The companion fix for the replication path is in the livesync-commonlib
submodule.

Ref: #859
2026-04-27 00:12:57 +09:00
vorotamoroz b609e4973c Merge remote-tracking branch 'refs/remotes/origin/main' 2026-04-25 20:37:08 +09:00
vorotamoroz 354f0be9a3 bump
Co-authored-by: Copilot <copilot@github.com>
2026-04-25 20:16:50 +09:00
vorotamoroz 16804ed34c Merge pull request #842 from kdavh/patch-1
Update README.md, fix webpeer link
2026-04-25 19:07:56 +09:00
vorotamoroz 31bd270869 Fixed: Hidden file JSON conflicts no longer keep re-opening and dismissing the merge dialogue before we can act, which fixes persistent unresolvable data.json conflicts in plug-in settings sync (related: #850).
Co-authored-by: Copilot <copilot@github.com>
2026-04-25 17:22:25 +09:00
vorotamoroz b5d054f259 Fixed: Issue report generation now redacts remoteConfigurations connection strings and keeps only the scheme (e.g. sls+https://), so credentials are not exposed in reports.
Co-authored-by: Copilot <copilot@github.com>
2026-04-25 17:09:43 +09:00
vorotamoroz 1ef2955d00 - Fixed a worker-side recursion issue that could raise Maximum call stack size exceeded during chunk splitting (related: #855).
- Improved background worker crash cleanup so pending split/encryption tasks are released cleanly instead of being left in a waiting state (related: #855).
- On start-up, the selected remote configuration is now applied to runtime connection fields as well, reducing intermittent authentication failures caused by stale runtime settings (related: #855).

Co-authored-by: Copilot <copilot@github.com>
2026-04-25 16:51:37 +09:00
vorotamoroz 6ef56063b3 Fixed: No longer credentials are broken during object storage configuration (related: #852).
Co-authored-by: Copilot <copilot@github.com>
2026-04-25 15:03:38 +09:00
vorotamoroz a912585800 Improve issue template
Co-authored-by: Copilot <copilot@github.com>
2026-04-25 14:01:18 +09:00
vorotamoroz 7a863625bc bump 2026-04-09 04:32:34 +01:00
vorotamoroz 99b4037820 Fixed
- Packing a batch during the journal sync now continues even if the batch contains no items to upload.
2026-04-06 12:51:09 +01:00
vorotamoroz d59b5dc2f9 Fixed
- Remote configuration URIs are now correctly encrypted when saved after editing in the settings dialogue.
- Fixed an issue where devices could no longer upload after another device performed 'Fresh Start Wipe' and 'Overwrite remote' in Object Storage mode (#848).
2026-04-06 11:47:19 +01:00
vorotamoroz 4d0203e4ca Update: beta tagging 2026-04-06 11:46:51 +01:00
vorotamoroz 3e4db571cd Fixed
- Remote configuration URIs are now correctly encrypted when saved after editing in the settings dialogue.
- Fixed an issue where devices could no longer upload after another device performed 'Fresh Start Wipe' and 'Overwrite remote' in Object Storage mode (#848).
2026-04-06 11:45:26 +01:00
vorotamoroz b0a9bd84d6 bump 2026-04-05 18:21:38 +09:00
vorotamoroz 8c4e62e7c1 ### Fixed
- Now surely remote configurations are editable in the settings dialogue.
- We can fetch remote settings from the remote and apply them to the local settings for each remote configuration entry.
- No longer layout breaking occurs when the description of a remote configuration entry is too long.
2026-04-05 18:20:56 +09:00
vorotamoroz 3e03d1dbd5 bump 2026-04-05 17:48:00 +09:00
vorotamoroz 0dbf4cface bump 2026-04-05 17:47:45 +09:00
vorotamoroz bc22d61a3a Fixed: Now error messages are kept hidden if the show status inside the editor is disabled. 2026-04-05 17:43:29 +09:00
vorotamoroz d709bcc1d0 Add encryption for connection management 2026-04-05 16:24:34 +09:00
vorotamoroz d7088be8af Improved: remote management 2026-04-05 16:00:57 +09:00
vorotamoroz f17f1ecd93 ### Fixed
- No unexpected error (about a replicator) during early stage of initialisation.

### New features

- Now we can configure multiple Remote Databases of the same type, e.g, multiple CouchDBs or S3 remotes.
- We can switch between multiple Remote Databases in the settings dialogue.
2026-04-03 13:47:56 +01:00
vorotamoroz bf556bd9f4 Merge pull request #838 from chinhkrb113/contribai/docs/undocumented-test-environment-variables
📝 Docs: Undocumented test environment variables
2026-04-03 13:05:05 +09:00
vorotamoroz 8b40969fa3 Add ru locale 2026-04-02 10:28:58 +00:00
vorotamoroz 6cce931a88 Add test for CLI 2026-04-02 09:58:25 +00:00
vorotamoroz 216861f2c3 Prettified 2026-04-02 10:33:36 +01:00
vorotamoroz 6ce724afb4 Add dockerfiles to webapp and webpeer 2026-04-02 10:33:13 +01:00
vorotamoroz 2e3e106fb2 Fix dockerfile 2026-04-02 10:31:17 +01:00
vorotamoroz 00f2606a2f Added a bit for development on Windows. 2026-04-02 10:31:03 +01:00
vorotamoroz 3c94a44285 Fixed: Replication progress is now correctly saved and restored in the CLI. 2026-04-02 10:30:14 +01:00
vorotamoroz 4c0908acde Add CI Build for cli-docker image 2026-04-02 07:47:10 +01:00
vorotamoroz cda27fb7f8 - Update trystero to v0.23.0
- Add dockerfile for CLI
- Change relay image for testing on arm64
2026-03-31 07:17:51 +00:00
vorotamoroz 837a828cec Fix: fix update note... 2026-03-30 09:20:01 +01:00
vorotamoroz 4c8e13ccb9 bump 2026-03-30 09:14:49 +01:00
vorotamoroz 1ae4eaab02 Merge pull request #805 from L4z3x/patch-1
[docs] added changing docker compose data and etc folder ownership to user 5984.
2026-03-30 17:04:12 +09:00
chinhkrb113 b1efbf74c7 docs: clarify P2P_TEST_RELAY as Nostr relay 2026-03-30 02:18:25 +07:00
kdavh 12f04f6cf7 Update README.md, fix webpeer link 2026-03-28 12:47:28 -04:00
vorotamoroz a937feed3f Merge pull request #833 from rewse/fix/cli-sync-locked-error-message
fix(cli): show actionable error when sync fails due to locked remote DB
2026-03-28 23:58:34 +09:00
ChinhLee 2de9899a99 docs: undocumented test environment variables
The P2P test suite relies on several specific environment variables (e.g., `P2P_TEST_ROOM_ID`, `P2P_TEST_PASSPHRASE`, `P2P_TEST_RELAY`) loaded from `.env` or `.test.env`. Because these are not documented anywhere in the repository, new contributors will be unable to configure their local environment to run the P2P tests successfully.

Affected files: vitest.config.p2p.ts

Signed-off-by: ChinhLee <76194645+chinhkrb113@users.noreply.github.com>
2026-03-27 22:26:00 +07:00
vorotamoroz a0af6201a5 - No longer Peer-to-Peer Sync is not enabled. We cannot open a new connection. error occurs when we have not enabled P2P sync and are not expected to use it (#830). 2026-03-26 13:13:27 +01:00
vorotamoroz 9c7c6c8859 Merge pull request #831 from rewse/fix/cli-entrypoint-polyfill-default
fix(cli): handle incomplete localStorage in Node.js v25+
2026-03-26 20:36:46 +09:00
vorotamoroz 38d7cae1bc update some dependencies and ran npm-update. 2026-03-26 12:15:38 +01:00
vorotamoroz fee34f0dcb Update dependency: deduplicate 2026-03-26 11:55:06 +01:00
Shibata, Tats e01f7f4d92 test(cli): add TODO comment and locked-remote-DB test script
- Add inline TODO comment in runCommand.ts about standardising
  replication failure cause identification logic.
- Add test-sync-locked-remote-linux.sh that verifies:
  1. sync succeeds when the remote milestone is not locked.
  2. sync fails with an actionable error when the remote milestone
     has locked=true and accepted_nodes is empty.
2026-03-26 00:58:51 +09:00
Shibata, Tats 985004bc0e fix(cli): show actionable error when sync fails due to locked remote DB
When the remote database is locked and the CLI device is not in the
accepted_nodes list, openReplication returns false with no CLI-specific
guidance. The existing log message ('Fetch rebuilt DB, explicit
unlocking or chunk clean-up is required') is aimed at the Obsidian
plugin UI.

Check the replicator's remoteLockedAndDeviceNotAccepted flag after
sync failure and print a clear message directing the user to unlock
from the Obsidian plugin.

Ref: #832
2026-03-22 12:37:17 +09:00
Shibata, Tats 967a78d657 fix(cli): handle incomplete localStorage in Node.js v25+
Node.js v25 provides a built-in localStorage on globalThis, but without
`--localstorage-file` it is an empty object lacking getItem/setItem.
The existing check `!("localStorage" in globalThis)` passes, so the
polyfill is skipped and the CLI crashes with:

  TypeError: localStorage.getItem is not a function

Check for getItem as well so the polyfill is applied when the native
implementation is incomplete.
2026-03-22 11:57:47 +09:00
vorotamoroz 2ff60dd5ac Add missed files 2026-03-18 12:20:52 +01:00
vorotamoroz c3341da242 Fix english 2026-03-18 12:05:15 +01:00
vorotamoroz c2bfaeb5a9 Fixed: wrong import 2026-03-18 12:03:51 +01:00
vorotamoroz c454616e1c bump 2026-03-18 12:01:57 +01:00
vorotamoroz c88e73b7d3 Add note 2026-03-18 11:55:50 +01:00
vorotamoroz 3a29818612 - Delete items which are no longer used that might cause potential problems
- Fix Some Imports
- Fix floating promises on tests
2026-03-18 11:54:22 +01:00
vorotamoroz ee69085830 Fixed: Some buttons on the setting dialogue now respond correctly again (#827). 2026-03-18 11:51:52 +01:00
vorotamoroz 3963f7c971 Refactored: P2P replicator has been refactored to be a little roust and easier to understand. 2026-03-18 11:49:41 +01:00
vorotamoroz 602fcef949 - Fixed the issue where the detail level was not being applied in the log pane.
- Pop-ups are now shown.
- Add coverage for test.
- Pop-ups are now shown in the web app as well.
2026-03-18 11:48:31 +01:00
vorotamoroz 075d260fdd Fixed:
- Fixed the corrupted display of the help message.
- Remove some unnecessary codes.
2026-03-18 11:46:52 +01:00
vorotamoroz 0717093d81 update for npm ci 2026-03-17 20:09:28 +09:00
vorotamoroz 1f87a9fd3d port setupManager, setupProtocol to serviceFeature
remove styles on webapp UI, and add stylesheet
2026-03-17 19:58:12 +09:00
vorotamoroz fdd3a3aecb Add: vaultSelector (webapp) 2026-03-17 19:51:04 +09:00
vorotamoroz d8281390c4 bump 2026-03-17 10:34:29 +01:00
vorotamoroz 08b1712f39 bump 2026-03-16 00:49:54 +09:00
vorotamoroz 6c69547cef ### Fixed
- Fixed flaky timing issues in P2P synchronisation.
- Fixed more binary file handling issues in CLI.

### Tests

- Rewrite P2P end-to-end tests to use the CLI as host.
2026-03-16 00:48:22 +09:00
vorotamoroz 89bf0488c3 Refactor: More refactor P2P Replicator 2026-03-15 04:07:47 +09:00
vorotamoroz 653cf8dfbe Refactor: Refactor P2P Replicator 2026-03-15 03:33:03 +09:00
vorotamoroz 33338506cf bump 2026-03-14 17:02:16 +09:00
vorotamoroz 9dd479e597 Fix for an issue where conflicts cannot be resolved in Journal Sync
Remove unnecessary test calling in CLI
2026-03-14 16:51:30 +09:00
vorotamoroz 8cad4cdf80 Add workaround for my mac 2026-03-14 16:50:43 +09:00
vorotamoroz beced219c7 Fix: exit code 2026-03-14 16:13:14 +09:00
vorotamoroz dfe13b1abd Fixed:
- No longer unexpected `Unhandled Rejections` during P2P operations (waiting acceptance).
CLI new features
- P2P sync has been implemented.
2026-03-14 15:08:31 +09:00
vorotamoroz bf93bddbdd Fix: prevent transfer twice. 2026-03-13 23:34:38 +09:00
vorotamoroz 44890a34e8 remove conflicting option. 2026-03-13 23:08:05 +09:00
vorotamoroz a14aa201a8 Merge branch 'beta' of https://github.com/vrtmrz/obsidian-livesync into beta 2026-03-13 18:13:12 +09:00
vorotamoroz 338a9ba9fa Add: mirror command
Tidy: test
2026-03-13 18:01:38 +09:00
vorotamoroz 0c65b5add9 Add: mirror command 2026-03-13 12:55:46 +09:00
vorotamoroz 29ce9a5df4 remove todo 2026-03-12 12:45:39 +01:00
vorotamoroz 10f5cb8b42 add paths 2026-03-12 12:31:53 +01:00
vorotamoroz 8aad3716d4 fix grammatical errors 2026-03-12 12:29:43 +01:00
vorotamoroz d45f41500a Fix: no longer duplicated addLog setHandler 2026-03-12 12:27:47 +01:00
vorotamoroz 4cc0a11d86 Add ci cli-e2e 2026-03-12 12:23:13 +01:00
vorotamoroz ad0a6b458f bump 2026-03-12 12:16:00 +01:00
vorotamoroz 6ae1d5d6a5 update readme 2026-03-12 12:07:05 +01:00
vorotamoroz 84110aee97 update readme 2026-03-12 19:46:52 +09:00
vorotamoroz 4646577f35 Update 2026-03-12 19:42:10 +09:00
vorotamoroz 822d957976 Refactor: separate entrypoint and main,
Fix: readlng binary file
2026-03-12 19:41:10 +09:00
vorotamoroz d4aedf59f3 A- Add more tests.
- Object Storage support has also been confirmed (and fixed) in CLI.
2026-03-12 18:20:55 +09:00
vorotamoroz 5d80258a77 Add e2e-test (not passed yet)
Refine readme
2026-03-12 12:20:39 +09:00
vorotamoroz fa14531599 Add note 2026-03-12 03:03:03 +09:00
vorotamoroz 7992b3c2b9 Wrote the test (but untested) 2026-03-12 03:01:46 +09:00
vorotamoroz 5872cad1e5 Implement commands 2026-03-12 02:56:30 +09:00
vorotamoroz 16c0dfef4c Remove the grandiloquence from the note written in work in progress. 2026-03-11 23:51:35 +09:00
vorotamoroz 70c7624c7a Add note 2026-03-11 14:58:06 +01:00
vorotamoroz 4a0d5e99d0 Fix import path and add readme 2026-03-11 14:57:36 +01:00
vorotamoroz 0742773e1e Add self-hosted-livesync-cli to src/apps/cli as a headless, and a dedicated version. 2026-03-11 14:51:01 +01:00
vorotamoroz 2f8bc4fef2 - Now useOfflineScanner, useCheckRemoteSize, and useRedFlagFeatures are set from main.ts, instead of LiveSyncBaseCore. 2026-03-11 14:44:37 +01:00
vorotamoroz 0dfd42259d 11th March, 2026
Now, Self-hosted LiveSync has finally begun to be split into the Self-hosted LiveSync plugin for Obsidian, and a properly abstracted version of it.
This may not offer much benefit to Obsidian plugin users, or might even cause a slight inconvenience, but I believe it will certainly help improve testability and make the ecosystem better.
However, I do not see the point in putting something with little benefit into beta, so I am handling this on the alpha branch. I would actually preferred to create an R&D branch, but I was not keen on the ampersand, and I feel it will eventually become a proper beta anyway.

### Refactored

- Separated `ObsidianLiveSyncPlugin` into `ObsidianLiveSyncPlugin` and `LiveSyncBaseCore`.
- Now `LiveSyncCore` indicates the type specified version of `LiveSyncBaseCore`.
- Referencing `plugin.xxx` has been rewritten to referencing the corresponding service or `core.xxx`.

### Internal API changes

- Storage Access APIs are now yielding Promises. This is to allow more limited storage platforms to be supported.

### R&D

- Browser-version of Self-hosted LiveSync is now in development. This is not intended for public use now, but I will eventually make it available for testing.
- We can see the code in `src/apps/webapp` for the browser version.
2026-03-11 05:47:00 +01:00
vorotamoroz 9cf630320c Merge remote-tracking branch 'refs/remotes/origin/main' 2026-03-09 10:38:17 +09:00
vorotamoroz 584adc9296 bump 2026-03-09 10:25:19 +09:00
vorotamoroz f7dba6854f ### Fixed
- No longer unexpected deletion-propagation occurs when the parent directory is not empty (#813).

### Revert reversions
- Reverted the reversion of ModuleCheckRemoteSize. Now it is back to the service feature.
2026-03-09 10:24:49 +09:00
vorotamoroz 1d83e0ee31 manifest.json を更新 2026-03-08 17:24:54 +09:00
vorotamoroz d0244bd6d0 bump 2026-03-07 18:36:54 +09:00
vorotamoroz 79bb5e1c77 ### Reverted
- Reverted to ModuleRedFlag and ModuleInitializerFile to the previous version because of some unexpected issues. (#813)
2026-03-07 18:36:10 +09:00
vorotamoroz 3403712e24 Downgrade version from 0.25.50 to 0.25.48 for #813 2026-03-07 13:13:37 +09:00
vorotamoroz 8faa19629b bump 2026-03-03 13:39:13 +00:00
vorotamoroz 7ff9c666ce Fix: No more credentials logged 2026-03-03 13:34:18 +00:00
vorotamoroz d8bc2806e0 bump 2026-03-03 13:27:23 +00:00
vorotamoroz 62f78b4028 Modify unit-ci to upload coverage 2026-03-03 13:22:17 +00:00
vorotamoroz cf9d2720ce ### Fixed
- No longer deleted files are not clickable in the Global History pane.
- Diff view now uses more specific classes (#803).
- A message of configuration mismatching slightly added for better understanding.
    - Now it says `When replication is initiated manually via the command palette or ribbon, a dialogue box will open to address this.` to make it clear that the user can fix the issue by themselves.

### Refactored

- `ModuleRedFlag` has been refactored to `serviceFeatures/redFlag` and also tested.
- `ModuleInitializerFile` has been refactored to `lib/serviceFeatures/offlineScanner` and also tested.
2026-03-03 13:19:22 +00:00
vorotamoroz 09115dfe15 Fixed: Styles on the diff-dialogue has been qualified. 2026-03-02 10:52:14 +00:00
vorotamoroz 4cbb833e9d bump 2026-03-02 09:36:43 +00:00
vorotamoroz 7419d0d2a1 Add unit-ci hook to push 2026-03-02 09:15:29 +00:00
vorotamoroz f3e83d4045 Refactored: changed the implementation from using overrides to injecting an adapter. 2026-03-02 09:06:23 +00:00
vorotamoroz 28e06a21e4 bump 2026-02-27 11:15:23 +00:00
vorotamoroz e08fbbd223 ### Fixed and refactored
- Fixed the inexplicable behaviour when retrieving chunks from the network.
    - Chunk manager has been layered to responsible its own areas and duties. e.g., `DatabaseWriteLayer`, `DatabaseReadLayer`, `NetworkLayer`, `CacheLayer`,and `ArrivalWaitLayer`.
        - All layers have test now!
        - `LayeredChunkManager` has been implemented to manage these layers. Also tested.
    - `EntryManager` has been mostly rewritten, and also tested.

- Now we can configure `Never warn` for remote storage size notification, again.
2026-02-27 11:00:30 +00:00
vorotamoroz a1e331d452 bump 2026-02-26 11:34:14 +00:00
vorotamoroz 646f8af680 ### Fixed
- Unexpected errors no longer occurred when the plug-in was unloaded.
- Hidden File Sync now respects selectors.
- Registering protocol-handlers now works safely without causing unexpected errors.

### Refactored
- LiveSyncManagers has now explicit dependencies.
- LiveSyncLocalDB is now responsible for LiveSyncManagers, not accepting the managers as dependencies.
    - This is to avoid circular dependencies and clarify the ownership of the managers.
- ChangeManager has been refactored. This had a potential issue, so something had been fixed, possibly.
- Some tests have been ported from Deno's test runner to Vitest to accumulate coverage.
2026-02-26 11:30:57 +00:00
vorotamoroz 392f76fd36 ### Refactored
- `ModuleCheckRemoteSize` has been ported to a serviceFeature, and also tests have been added.
- Some unnecessary things have been removed.
2026-02-26 08:59:54 +00:00
vorotamoroz f61a3eb85b bump 2026-02-25 09:39:55 +00:00
vorotamoroz 19c03ec8d8 ### Refactored
- `ModuleTargetFilter`, which was responsible for checking if a file is a target file, has been ported to a serviceFeature.
  - And also tests have been added. The middleware-style-power.
- `ModuleObsidianAPI` has been removed and implemented in `APIService` and `RemoteService`.
- Now `APIService` is responsible for the network-online-status, not `databaseService.managers.networkManager`.
2026-02-25 09:38:31 +00:00
vorotamoroz be1642f1c1 Fix grammatical errors. 2026-02-24 11:20:59 +00:00
vorotamoroz c9a71e2076 fix: align dependency versions to main 2026-02-24 11:14:14 +00:00
vorotamoroz 2199c1ebd3 change: test:docker-all:xxx to be parallel. 2026-02-24 11:13:58 +00:00
vorotamoroz 278935f85d Fix mock for testing 2026-02-24 11:08:08 +00:00
vorotamoroz 010631f553 bump dependencies 2026-02-24 10:38:35 +00:00
vorotamoroz 8c0c65307a bump 2026-02-24 08:23:08 +00:00
vorotamoroz 988cb34d7c Update 2026-02-24 07:56:23 +00:00
vorotamoroz 6eec8117f5 Refactor: Constantising log-mark 2026-02-24 07:51:21 +00:00
vorotamoroz 9f6a909143 Merge remote-tracking branch 'origin/main' into beta to port #802 2026-02-24 07:44:18 +00:00
vorotamoroz 09f283721a Merge remote-tracking branch 'origin/main' into beta 2026-02-24 04:13:34 +00:00
vorotamoroz 235c702223 Merge pull request #804 from A-wry/reduce-fetch-error-visibility
Add connection warning style logic and settings UI dropdown
2026-02-24 13:07:29 +09:00
A-wry b923b43b6b update submodule pointer to latest commonlib changes 2026-02-23 22:29:46 -05:00
A-wry fdcf3be0f9 Tagged downstream network errors to respect networkWarningStyle setting 2026-02-23 22:10:25 -05:00
vorotamoroz 25dd907591 port PR #802 2026-02-24 01:57:08 +00:00
vorotamoroz 80c049d276 Merge pull request #802 from waspeer/fix/write-file-auto-md-file-already-exists
fix: handle "File already exists" for .md files in writeFileAuto
2026-02-24 10:50:37 +09:00
L4z3x 310496d0b8 added changing docker compose data and etc folder, to prevent permissions errors
while trying to follow the docker compose guide i created the data folders using the root user, and had this error when i run the stack:
`touch: cannot touch '/opt/couchdb/etc/local.d/docker.ini': Permission denied`

the problem was solved by changing the ownership of the folder to the user 5984, then one in the docker compose file.
2026-02-23 22:21:18 +01:00
vorotamoroz e961f01187 fix typos. 2026-02-21 14:17:28 +09:00
vorotamoroz 14b4c3cd50 prettified 2026-02-21 14:12:05 +09:00
vorotamoroz 4f987e7c2b ### Fixed
- Hidden file synchronisation now works!
- Now Hidden file synchronisation respects `.ignore` files.
- Replicator initialisation during rebuilding now works correctly.

### Refactored

- Some methods naming have been changed for better clarity, i.e., `_isTargetFileByLocalDB` is now `_isTargetAcceptedByLocalDB`.
2026-02-21 14:05:32 +09:00
A-wry f4d8c0a8db Add connection warning style logic and settings UI dropdown 2026-02-20 21:51:30 -05:00
vorotamoroz 556ce471f8 ## 0.25.43-patched-8 2026-02-20 14:28:28 +00:00
vorotamoroz 32b6717114 keep a note 2026-02-19 10:38:42 +00:00
vorotamoroz e0e72fae72 Fixed: saving device name 2026-02-19 10:37:19 +00:00
vorotamoroz 203dd17421 for 0.25.43-patched-7, please refer to the updates.md 2026-02-19 10:23:45 +00:00
vorotamoroz 1bde2b2ff1 Fixed an issue where the StorageEventManager
Build by Vite is now testing
2026-02-19 04:18:18 +00:00
mugenrei ebbac96b16 docs: add comprehensive Cloudflare Tunnel setup guide
- Step-by-step tunnel creation in Zero Trust dashboard
- Detailed public hostname configuration (couchdb:5984 vs localhost)
- Troubleshooting section for 404, 502, and 524 errors
- Obsidian plugin configuration instructions
- Emphasis on token-based tunnels ignoring local config file
2026-02-18 10:57:18 -03:00
mugenrei ed0e67e853 fix: CRLF line endings in couchdb-init.sh, add .gitattributes
Shell scripts with CRLF line endings crash inside Linux containers with
'set: illegal option -' and 'command not found' errors.

- Strip CRLF -> LF in couchdb-init.sh
- Add .gitattributes to enforce LF on .sh, .yml, .ini, .md etc.
  so this never regresses on Windows checkouts
2026-02-18 10:52:51 -03:00
mugenrei f99d7dc00f Add initial Docker setup and configuration for self-hosted LiveSync with CouchDB 2026-02-18 09:46:49 -03:00
vorotamoroz 2bf1c775ee ## 0.25.43-patched-6
### Fixed

- Unlocking the remote database after rebuilding has been fixed.

### Refactored
- Now `StorageEventManagerBase` is separated from `StorageEventManagerObsidian` following their concerns.
- Now `FileAccessBase` is separated from `FileAccessObsidian` following their concerns.
2026-02-18 12:13:05 +00:00
vorotamoroz 4658e3735d Fix Shim 2026-02-17 10:56:05 +00:00
vorotamoroz 627edc96bf bump for beta 2026-02-17 10:14:13 +00:00
vorotamoroz 0a1917e83c Refactor for 0.25.43-patched-5 (very long, please refer the updates.md) 2026-02-17 10:14:04 +00:00
Wannes Salomé 48b0d22da6 fix: handle "File already exists" for .md files in writeFileAuto
During concurrent initialisation (UPDATE STORAGE runs up to 10 ops in
parallel), getAbstractFileByPath can return null for .md files whose
vault index entry hasn't been populated yet, even though the file
already exists on disk. This causes vault.create() to throw "File
already exists."

The same root cause (stale in-memory index) was already identified for
non-.md files (see comment above) and handled via adapterWrite. Extend
that workaround to .md files by catching the "File already exists"
error and falling back to adapterWrite, consistent with the existing
approach.

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
2026-02-17 00:24:30 +01:00
vorotamoroz 3201399bdf bump 2026-02-16 11:51:09 +00:00
vorotamoroz 2ae70e8f07 Refactor: DatabaseService and Replicator 2026-02-16 11:51:03 +00:00
vorotamoroz 2b9bb1ed06 beta bump 2026-02-16 06:51:52 +00:00
vorotamoroz e63e3e6725 ### Refactor
- Module dependency refined. (For details, please refer to updates.md)
2026-02-16 06:50:31 +00:00
vorotamoroz 6e9ac6a9f9 - Application LifeCycle has now started in Main, not ServiceHub. 2026-02-14 15:21:00 +09:00
vorotamoroz fb59c4a723 Refactored, please refer updates.md 2026-02-13 12:02:31 +00:00
vorotamoroz 1b5ca9e52c Refactor (write notes later) 2026-02-12 08:56:30 +00:00
vorotamoroz 787627a156 Refactor: Move some functions from modules to services 2026-02-12 06:27:29 +00:00
vorotamoroz b1bba7685e Add note. 2026-02-12 03:39:56 +00:00
vorotamoroz cdfc0ccead wow, auditing... bump. 2026-02-05 12:15:35 +00:00
vorotamoroz 0635cad350 bake i18n 2026-02-05 12:13:12 +00:00
vorotamoroz 6fd1fa6313 Fix typos and make P2P sync not experimental 2026-02-05 12:07:29 +00:00
vorotamoroz 12b1f881dc ### Fixed
- Encryption/decryption issues when using Object Storage as remote have been fixed.
2026-02-05 11:47:01 +00:00
vorotamoroz bf3efab1af Merge pull request #733 from dayne/patch-2
Update regions setup-flyio-on-the-fly-v2.ipynb
2026-02-02 13:51:48 +09:00
vorotamoroz da72fda221 Merge pull request #777 from oenhu/patch-1
Update README_cn.md
2026-02-02 13:42:14 +09:00
vorotamoroz 665501f485 Merge pull request #778 from oenhu/patch-2
Create tech_info_cn.md
2026-02-02 13:40:17 +09:00
vorotamoroz 6ee332fff8 Merge pull request #779 from oenhu/main
refactor: replace hardcoded strings with i18n keys
2026-02-02 13:38:01 +09:00
vorotamoroz f66447cb59 Fix task 2026-02-02 13:00:28 +09:00
vorotamoroz eb3120a8fd Fix CI 2026-02-02 12:57:49 +09:00
vorotamoroz 5fa39b3c6e Fix typo 2026-02-02 12:28:26 +09:00
vorotamoroz 91c35a88dd separate CIs 2026-02-02 12:27:37 +09:00
vorotamoroz 49f4d79f4f Update dependencies (Mostly translations) 2026-02-02 12:27:15 +09:00
vorotamoroz abfd010467 Add the detail 2026-02-02 11:39:28 +09:00
vorotamoroz cde1013359 Reducing ambiguity 2026-02-02 11:32:52 +09:00
vorotamoroz 9c7f9e4316 Tidy 2026-02-02 11:31:40 +09:00
vorotamoroz aceda16c64 Add beta policy note 2026-02-02 11:31:06 +09:00
vorotamoroz 3656e5c725 Merge branch 'beta' 2026-02-02 11:00:12 +09:00
vorotamoroz 6915b160a2 Bump to stable 2026-02-02 10:57:15 +09:00
vorotamoroz f464623bf6 revert for release script 2026-01-30 09:36:39 +00:00
vorotamoroz b1f518071c add note 2026-01-30 09:27:31 +00:00
vorotamoroz 0e903c3520 Change: enabling PATH_TEST_INSTALL in production build 2026-01-30 09:25:12 +00:00
vorotamoroz edcdfd97c4 Reduce dynamic function binding from APIService from 2026-01-30 09:23:54 +00:00
vorotamoroz c2b7081215 Add some notes 2026-01-30 07:39:03 +00:00
vorotamoroz cf1954b10e Bump for beta 2026-01-29 07:15:22 +00:00
vorotamoroz 46546e121f Refactor: Move webpeer from lib to main repository. 2026-01-26 09:17:01 +00:00
vorotamoroz 28146eec2c Refactor: Migrate the outdated, unstable platform abstraction layer to Services 2026-01-26 09:13:40 +00:00
vorotamoroz 3cd9b9e06d bump 2026-01-24 16:48:51 +09:00
vorotamoroz 7c43c61b85 ### Fixed
- No longer `No available splitter for settings!!` errors occur after fetching old remote settings while rebuilding local database.

### Improved

- Boot sequence warning is now kept in the in-editor notification area. (#748)

### New feature

- We can now set the maximum modified time for reflect events in the settings. (for #754)

### Refactored
- Module to service refactoring has been started for better maintainability:
  - UI module has been moved to UI service.

### Behaviour change
- Default chunk splitter version has been changed to `Rabin-Karp` for new installations.
2026-01-24 16:25:04 +09:00
vorotamoroz 465af4f3aa Urg: 0.25.40 for fix wrong eventType 2026-01-23 11:55:30 +00:00
vorotamoroz 0a1e3dcd51 bump 2026-01-23 06:27:06 +00:00
vorotamoroz b97756d0cf add unittest 2026-01-23 05:45:53 +00:00
vorotamoroz acf4bc3737 prettified 2026-01-23 05:45:19 +00:00
vorotamoroz 88838872e7 refactor_serivces
- Rewrite the service's binding/handler assignment systems
- Removed loopholes that allowed traversal between services to clarify dependencies.
- Consolidated the hidden state-related state, the handler, and the addition of bindings to the handler into a single object.
  - Currently, functions that can have handlers added implement either addHandler or setHandler directly on the function itself.
I understand there are differing opinions on this, but for now, this is how it stands.
- Services now possess a Context. Please ensure each platform has a class that inherits from ServiceContext.
- To permit services to be dynamically bound, the services themselves are now defined by interfaces.
2026-01-23 05:44:14 +00:00
vorotamoroz 7d3827d335 bump 2026-01-17 14:36:47 +09:00
vorotamoroz 92d3a0cfa2 ### Fixed
- Fixed an issue where indexedDB would not close correctly on some environments, causing unexpected errors during database operations.
2026-01-17 14:32:44 +09:00
vorotamoroz bba26624ad bump 2026-01-15 04:12:19 +00:00
vorotamoroz b82f497cab Add development guide link 2026-01-15 04:04:49 +00:00
vorotamoroz 37f4d13e75 Merge branch 'main' of https://github.com/vrtmrz/obsidian-livesync 2026-01-15 03:59:53 +00:00
vorotamoroz 7965f5342c Fix type 2026-01-15 12:47:26 +09:00
vorotamoroz 9cdc14dda8 Fix: CI 2026-01-15 12:45:29 +09:00
vorotamoroz 4f46276ebf Fix for CI 2026-01-15 12:45:13 +09:00
vorotamoroz 931d360fb1 Add summary note 2026-01-14 09:41:16 +00:00
vorotamoroz f68c1855da Remove obsoleted file. 2026-01-14 08:07:40 +00:00
vorotamoroz dff654b6e5 Fixed:
- Fixed mishandling of removing  duplicated queue on processing
- Dependency importing
2026-01-14 08:07:22 +00:00
vorotamoroz 7e85bcbf08 harness-ci.yml mod ci 2026-01-10 12:20:17 +09:00
vorotamoroz 38a695ea12 Fix actions 2026-01-09 12:18:38 +00:00
vorotamoroz a502b0cd0c Split the tests 2026-01-09 12:15:57 +00:00
vorotamoroz 934f708753 Improve timing issue 2026-01-09 12:05:40 +00:00
vorotamoroz 0e574c6cb1 Flag executable 2026-01-09 11:50:23 +00:00
vorotamoroz 7375a85b07 Tests:
- More tests have been added.
2026-01-09 11:46:37 +00:00
vorotamoroz 4c3393d8b2 Fixed:
- Databases now correctly closed after rebuilding.
2026-01-09 11:45:00 +00:00
oenhu a9f1bbff9f refactor: replace hardcoded strings with i18n keys 2026-01-09 00:29:40 +08:00
oenhu f86815e420 Create tech_info_cn.md
Create Chinese version
2026-01-08 15:07:06 +08:00
oenhu fd16b166ef Update README_cn.md
Update the translation content based on the latest English REDME.md.
2026-01-08 01:41:07 +08:00
vorotamoroz 02aa9319c3 Remove chromiumSandbox for Actions 2026-01-07 09:15:31 +00:00
vorotamoroz 1a72e46d53 Fix excluded 2026-01-07 09:12:21 +00:00
vorotamoroz d755579968 Add excluded 2026-01-07 09:08:14 +00:00
vorotamoroz b74ee9df77 Fix to pulling submodules 2026-01-07 09:03:18 +00:00
vorotamoroz daa04bcea8 Add actions 2026-01-07 09:01:05 +00:00
vorotamoroz b96b2f24a6 Add some script and run npm install 2026-01-07 09:00:55 +00:00
vorotamoroz 5569ab62df Change Port 2026-01-07 09:00:16 +00:00
vorotamoroz d84b6c4f15 Flag executable 2026-01-07 08:41:43 +00:00
vorotamoroz 336f2c8a4d Add Test 2026-01-07 08:38:33 +00:00
vorotamoroz b52ceec36a Update submodule 2026-01-07 08:33:06 +00:00
vorotamoroz 1e6400cf79 Tidied:
- Add some note for corrupting notice
- Array function standardised.

Fixed:
- Remove obsoleted accessing to Obsidian's global.
- Avoiding errors in exceptional circumstances.
- Removal of several outdated and inefficient QueueProcessor implementations
  - Log output is now a bit efficient.
2026-01-07 04:45:19 +00:00
vorotamoroz 1ff1ac951b - Remove old comments
- Fix Importing paths
2026-01-07 04:37:30 +00:00
vorotamoroz aa6d771d17 bump 2025-12-25 10:30:04 +00:00
vorotamoroz 512c238415 ### Improved
- Now the garbage collector (V3) has been implemented. (Beta)
- Now the plug-in and device information is stored in the remote database.
2025-12-25 10:29:19 +00:00
vorotamoroz 55ffeeda10 bump 2025-12-24 03:38:40 +00:00
vorotamoroz 65f18b4160 ### Fixed
- Now the conflict resolution dialogue shows correctly which device only has older APIs (#764).
2025-12-24 03:37:41 +00:00
vorotamoroz b0c1d6a1bf bump 2025-12-10 09:48:25 +00:00
vorotamoroz ca19f2f2ed Modify build script and prettier config type (.prettierrc to .prettierrc.mjs 2025-12-10 09:47:31 +00:00
vorotamoroz ac0378ca4b ### Behaviour change
- The plug-in automatically fetches the missing chunks even if `Fetch chunks on demand` is disabled.
    - This change is to avoid loss of data when receiving a bulk of revisions.
    - This can be prevented by enabling `Use Only Local Chunks` in the settings.
- Storage application now saved during each event and restored on startup.
- Synchronisation result application is also now saved during each event and restored on startup.
    - These may avoid some unexpected loss of data when the editor crashes.

### Fixed

- Now the plug-in waits for the application of pended batch changes before the synchronisation starts.
    - This may avoid some unexpected loss or unexpected conflicts.
      Plug-in sends custom headers correctly when RequestAPI is used.
- No longer causing unexpected chunk creation during `Reset synchronisation on This Device` with bucket sync.

### Refactored

- Synchronisation result application process has been refactored.
- Storage application process has been refactored.
    - Please report if you find any unexpected behaviour after this update. A bit of large refactoring.
2025-12-10 09:45:57 +00:00
vorotamoroz f06f8d1eb6 bump 2025-12-05 11:08:48 +00:00
vorotamoroz 77074cb92f ### New feature
- We can analyse the local database with the `Analyse database usage` command.
- We can reset the notification threshold and check the remote usage at once with the `Reset notification threshold and check the remote database usage` command.

### Fixed

- Now the plug-in resets the remote size notification threshold after rebuild.
2025-12-05 11:07:59 +00:00
vorotamoroz 1274b6f683 Grammatical Fix 2025-12-02 11:28:56 +00:00
vorotamoroz c54ae58c0f bump 2025-12-02 11:24:31 +00:00
vorotamoroz 3f54921e90 ### Improved
- Now the plugin warns when we are on the several file-related situations that may cause unexpected behaviour (#300).
2025-12-02 11:20:24 +00:00
vorotamoroz 1b070c2dd4 bump 2025-11-18 08:54:41 +00:00
vorotamoroz 0e5846b670 add BUILD_MODE switching by environment variables. 2025-11-18 08:52:25 +00:00
vorotamoroz f81e71802b update dependency 2025-11-18 08:51:51 +00:00
vorotamoroz 4c761eebff ### Fixed
- Now fetching configuration from server can handle the empty remote correctly (reported on #756) by common-lib.
- No longer asking to switch adapters during rebuilding.
2025-11-18 08:51:32 +00:00
vorotamoroz bf754d6e07 Merge branch 'dev' 2025-11-17 23:37:39 +09:00
vorotamoroz 3cc70b985a bump 2025-11-17 23:30:53 +09:00
vorotamoroz 0e81ec2586 ### Fixed
- Now we can save settings correctly again (#756).
2025-11-17 23:30:16 +09:00
vorotamoroz 1c49acd5b5 Merge pull request #755 from vrtmrz/dev
v0.25.29
2025-11-17 13:32:32 +09:00
vorotamoroz bab66a64d7 bump again for release.. 2025-11-17 13:27:26 +09:00
vorotamoroz 477913456f OK. 2025-11-17 13:24:45 +09:00
vorotamoroz b0661cdbab bump 2025-11-17 13:20:16 +09:00
vorotamoroz 18f9a842b7 ### New feature
- We can now configure hidden file synchronisation to always overwrite with the latest version (#579).

### Fixed
- Timing dependency issues during initialisation have been mitigated (#714)

### Improved
- Error logs now contain stack-traces for better inspection.
2025-11-17 13:18:55 +09:00
vorotamoroz 5130bc5f2a I'm really sorry. Is there any way I can test this locally? 2025-11-17 13:16:50 +09:00
vorotamoroz ca8af80a27 again. 2025-11-17 13:15:56 +09:00
vorotamoroz df273d273b Sorry for broke exist release... 2025-11-17 13:14:56 +09:00
vorotamoroz 23aa0a82ca fix again.. 2025-11-17 13:09:59 +09:00
vorotamoroz 8f488b205b fix releaseing 2025-11-17 13:07:47 +09:00
vorotamoroz 893eac5c92 Move from deprecated 2025-11-17 13:06:26 +09:00
vorotamoroz cd6946bce2 node 22 to 24 2025-11-17 12:53:11 +09:00
vorotamoroz 174ca08954 Add production switch on environment vars 2025-11-17 12:52:52 +09:00
vorotamoroz 4af4d9c4bd bump 2025-11-12 09:23:49 +00:00
vorotamoroz 1b7a25598a ### Improved
- Now we can switch the database adapter between IndexedDB and IDB without rebuilding (#747).
- No longer checking for the adapter by `Doctor`.

### Changes

- The default adapter is reverted to IDB to avoid memory leaks (#747).

### Fixed (?)

- Reverted QR code library to v1.4.4 (To make sure #752).
2025-11-12 09:22:40 +00:00
vorotamoroz e2a01c14cc Merge branch 'main' of https://github.com/vrtmrz/obsidian-livesync 2025-11-07 09:56:53 +00:00
vorotamoroz a623b987c8 bump 2025-11-07 09:55:22 +00:00
vorotamoroz db28b9ec11 ### Improved
- Some JWT notes have been added to the setting dialogue (#742).

### Fixed

- No longer wrong values encoded into the QR code.
- We can acknowledge why the QR codes have not been generated.

### Refactored

- Some dependencies have been updated.
- Internal functions have been modularised into `octagonal-wheels` packages and are well tested.
- Fixed importing from the parent project in library codes. (#729).
2025-11-07 09:54:12 +00:00
vorotamoroz b2fbbb38f5 Fix tip formatting in jwt-on-couchdb.md
Updated tip formatting in JWT on CouchDB documentation.
2025-11-06 18:49:18 +09:00
vorotamoroz 33c01fdf1e bump 2025-11-06 09:43:58 +00:00
vorotamoroz 536c0426d6 Update lib for switch Refiner to Computed 2025-11-06 09:40:37 +00:00
vorotamoroz 2f848878c2 Add doc 2025-11-06 09:24:25 +00:00
vorotamoroz c4f2baef5e ### Fixed
#### JWT Authentication

- Now we can use JWT Authentication ES512 correctly (#742).
- Several misdirections in the Setting dialogues have been fixed (i.e., seconds and minutes confusion...).
- The key area in the Setting dialogue has been enlarged and accepts newlines correctly.
- Caching of JWT tokens now works correctly
    - Tokens are now cached and reused until they expire.
    - They will be kept until 10% of the expiration duration is remaining or 10 seconds, whichever is longer (but at a maximum of 1 minute).
- JWT settings are now correctly displayed on the Setting dialogue.

#### Other fixes

- Receiving non-latest revisions no longer causes unexpected overwrites.
    - On receiving revisions that made conflicting changes, we are still able to handle them.

### Improved

- No longer duplicated message notifications are shown when a connection to the remote server fails.
    - Instead, a single notification is shown, and it will be kept on the notification area inside the editor until the situation is resolved.
- The notification area is no longer imposing, distracting, and overwhelming.
    - With a pale background, but bordered and with icons.
2025-11-06 09:24:16 +00:00
vorotamoroz a5b88a8d47 Add tips 2025-11-06 02:10:12 +00:00
vorotamoroz 88e61fb41f Merge branch 'main' of https://github.com/vrtmrz/obsidian-livesync 2025-11-04 11:36:36 +00:00
vorotamoroz 9bf04332bb bump 2025-11-04 11:35:19 +00:00
vorotamoroz 5238dec3f2 fix: Now hidden file synchronisation respects the filters correctly (#631, #735) 2025-11-04 11:34:39 +00:00
vorotamoroz 2b7b411c52 Merge branch 'svelteui' 2025-11-04 11:09:22 +00:00
vorotamoroz aab0f7f034 Fix Backporting mis 2025-11-04 10:58:46 +00:00
vorotamoroz b3a0deb0e3 bump 2025-10-31 11:36:55 +01:00
vorotamoroz b9138d1395 ### Fixed
- We can enter the fields on some dialogue correctly on mobile devices now.

### New features

- We can use TURN server for P2P connections now.
2025-10-31 11:33:06 +01:00
vorotamoroz 04997b84c0 Merge pull request #740 from shaielc/_local
Default to _local when node not supplied
2025-10-30 17:58:24 +09:00
vorotamoroz 7eb9807aa5 Fix import path 2025-10-30 09:35:17 +01:00
vorotamoroz 91a4f234f1 bump 2025-10-30 09:30:38 +01:00
vorotamoroz 82f2860938 ### Fixed
- P2P Replication got more robust and stable.

### Breaking changes

- Send configuration via Peer-to-Peer connection is not compatible with older versions.
2025-10-30 09:29:51 +01:00
shyelc 41a112cd8a Default to _local when node not supplied 2025-10-30 07:27:52 +00:00
vorotamoroz 294ebf0c31 Add automatic node detection 2025-10-30 13:24:01 +09:00
vorotamoroz 5443317157 merged 2025-10-30 02:38:30 +01:00
vorotamoroz 47fe9d2af3 Adjust method name to actual behaviour 2025-10-30 02:28:18 +01:00
Dayne Broderson c76187c6d2 Update setup-flyio-on-the-fly-v2.ipynb
remove extranious newline
2025-10-29 17:07:34 -08:00
vorotamoroz 4c260a7d2b Merge pull request #723 from GrzybowskiBYD/patch-1
Fix an issue with CouchDB while using couchdb-init.sh
2025-10-29 10:35:00 +09:00
vorotamoroz 82f6fefd35 bump 2025-10-26 19:51:20 +09:00
vorotamoroz ada8001fcb ### Fixed
- We are now able to enable optional features correctly again (#732).
- No longer oversized files have been processed, furthermore.
  - Before creating a chunk, the file is verified as the target.
  - The behaviour upon receiving replication has been changed as follows:
    - If the remote file is oversized, it is ignored.
    - If not, but while the local file is oversized, it is also ignored.
2025-10-26 19:38:45 +09:00
Dayne Broderson b3b3ad843c Update regions setup-flyio-on-the-fly-v2.ipynb
Updating the regions list pulled from recent `fly platform regions`
2025-10-22 09:26:13 -08:00
vorotamoroz 8b81570035 Grammatically fixes 2025-10-22 14:02:52 +01:00
vorotamoroz d3e50421e4 Merge branch 'svelteui' of https://github.com/vrtmrz/obsidian-livesync into svelteui 2025-10-22 14:02:20 +01:00
vorotamoroz 12605f4604 Add updates 2025-10-22 14:02:00 +01:00
vorotamoroz 2c0dd82886 Add updates 2025-10-22 13:56:24 +01:00
vorotamoroz f5315aacb8 v0.25.23.beta1
### Fixed (This should be backported to 0.25.22 if the beta phase is prolonged)

- No longer larger files will not create a chunks during preparing `Reset Synchronisation on This Device`.

### Behaviour changes

- Setup wizard is now more `goal-oriented`. Brand-new screens are introduced.
- `Fetch everything` and `Rebuild everything` is now `Reset Synchronisation on This Device` and `Overwrite Server Data with This Device's Files`.
- Remote configuration and E2EE settings are now separated to each modal dialogue.
- Peer-to-Peer settings is also separated into its own modal dialogue.
- Setup-URI, and Report for the Issue are now not copied to clipboard automatically. Instead, there are copy dialogue and buttons to copy them explicitly.
- No longer optional features are introduced during the setup or `Reset Synchronisation on This Device`, `Overwrite Server Data with This Device's Files`.
- We cannot preform `Fetch everything` and `Rebuild everything` (Removed, so the old name) without restarting Obsidian now.

### Miscellaneous

- Setup QR Code generation is separated into a src/lib/src/API/processSetting.ts file. Please use it as a subrepository if you want to generate QR codes in your own application.
- Setup-URI is also separated into a src/lib/src/API/processSetting.ts
- Some direct access to web-APIs are now wrapped into the services layer.

### Dependency updates

- Many dependencies are updated. Please see `package.json`.
- As upgrading TypeScript, Fixed many UInt8Array<ArrayBuffer> and Uint8Array type mismatches.
2025-10-22 13:56:15 +01:00
vorotamoroz 5a93066870 bump 2025-10-15 01:02:24 +09:00
vorotamoroz 3a73073505 ### Fixed
- Fixed a bug that caused wrong event bindings and flag inversion (#727)
  - This caused following issues:
    - In some cases, settings changes were not applied or saved correctly.
    - Automatic synchronisation did not begin correctly.

### Improved
- Too large diffs are not shown in the file comparison view, due to performance reasons.
2025-10-15 01:00:24 +09:00
vorotamoroz ee0c0ee611 Add a note to prevent my forget 2025-10-13 11:27:03 +09:00
vorotamoroz d7ea30e304 Merge pull request #726 from vrtmrz/disenchant
Disenchant
2025-10-13 10:40:20 +09:00
vorotamoroz 2b9ded60f7 Add notes 2025-10-13 10:38:32 +09:00
vorotamoroz 40508822cf Bump and add readme 2025-10-13 10:13:45 +09:00
vorotamoroz 6f938d5f54 Update submodule commit reference 2025-10-13 09:48:25 +09:00
vorotamoroz 51dc44bfb0 bump 0.25.21.beta2 2025-10-08 05:01:07 +01:00
vorotamoroz 7c4f2bf78a ### Fixed
- Fixed wrong event type bindings (which caused some events not to be handled correctly).
- Fixed detected a timing issue in StorageEventManager
    - When multiple events for the same file are fired in quick succession, metadata has been kept older information. This induces unexpected wrong notifications and write prevention.
2025-10-08 05:00:42 +01:00
Marcin Grzybowski d82122de24 Update couchdb-init.sh
Fixed the {"error":"nodedown","reason":"nonode@nohost is down"} issue
2025-10-06 15:11:59 +02:00
vorotamoroz 67c9b4cf06 bump for beta 2025-10-06 10:45:59 +01:00
vorotamoroz 4808876968 - Rename methods for automatic binging checking.
- Add automatic binging checks.
2025-10-06 10:40:25 +01:00
vorotamoroz cccff21ecc Merge branch 'main' into disenchant and run prettier 2025-10-04 17:59:42 +09:00
vorotamoroz d8415a97e5 Move some dependency to devDependency 2025-10-04 17:49:02 +09:00
vorotamoroz 85e9aa2978 For convenience 2025-10-04 17:20:59 +09:00
vorotamoroz b4eb0e4868 Improved: copy a dev build to vault folder 2025-10-04 17:12:37 +09:00
vorotamoroz 3ea348f468 Merge pull request #716 from chriscross12324/bugfix/setting-panel-hierarchy
bugfix/setting-panel-hierarchy: Fixed visual inconsistencies
2025-10-04 17:07:25 +09:00
vorotamoroz 81362816d6 disenchanting and dispelling from the nightmarish implicit something
Indeed, even though if this changeset is mostly another nightmare. It might be in beta for a while.
2025-10-03 14:59:14 +01:00
Chris Coulthard d6efe4510f bugfix/setting-panel-hierarchy: Updated styling to improve consistency. Updated various setting panes (Fixed hierarchy issues that caused some titles to overlap rather then bump and whitespace formatting) 2025-09-28 19:27:14 -07:00
vorotamoroz ca5a7ae18c bump 2025-09-26 11:42:06 +01:00
vorotamoroz a27652ac34 ### Fixed
- Chunk fetching no longer reports errors when the fetched chunk could not be saved (#710).
    - Just using the fetched chunk temporarily.
- Chunk fetching reports errors when the fetched chunk is surely corrupted (#710, #712).
- It no longer detects files that the plug-in has modified.
    - It may reduce unnecessary file comparisons and unexpected file states.

### Improved

- Now checking the remote database configuration respecting the CouchDB version (#714).
2025-09-26 11:40:41 +01:00
vorotamoroz 29b89efc47 ## 0.25.19
### Improved
- Now encoding/decoding for chunk data and encryption/decryption are performed in native functions (if they were available).
2025-09-18 12:29:09 +01:00
vorotamoroz ef3eef2d08 Bump 2025-09-17 09:07:49 +01:00
vorotamoroz ffbbe32e36 ### Fixed
- Property encryption detection now works correctly (On Self-hosted LiveSync, it was not broken, but as a library, it was not working correctly).
- Initialising the chunk splitter is now surely performed.
- DirectFileManipulator now works fine (as a library)
    - Old `DirectFileManipulatorV1` is now removed.

### Refactored

- Removed some unnecessary intermediate files.
2025-09-17 09:05:16 +01:00
vorotamoroz 0a5371cdee bump 2025-09-16 10:47:00 +01:00
vorotamoroz 466bb142e2 Refactored: removed some unnecessary intermediate file 2025-09-16 10:45:14 +01:00
vorotamoroz d394a4ce7f ### Fixed
- No longer information-level logs have produced during toggling `Show only notifications` in the settings (#708).
- Ignoring filters for Hidden file sync now works correctly (#709).
2025-09-16 10:30:48 +01:00
vorotamoroz 71ce76e502 Merge pull request #704 from Gron-HD/main
Add Docker Compose troubleshooting for own server setup
2025-09-16 15:33:14 +09:00
vorotamoroz ae7a7dd456 Merge pull request #706 from abhith/patch-1
docs(README): update links for Customisation Sync and Hidden File Sync
2025-09-16 15:31:00 +09:00
vorotamoroz 4048186bb5 bump 2025-09-04 11:46:50 +01:00
vorotamoroz 2b94fd9139 ## Improved
- Improved connectivity for P2P connections
- The connection to the signalling server can now be disconnected while in the background or when explicitly disconnected.
  - These features use a patch that has not been incorporated upstream.
2025-09-04 11:44:49 +01:00
vorotamoroz ec72ece86d bump 2025-09-03 10:12:51 +01:00
vorotamoroz e394a994c5 Fix typo 2025-09-03 10:11:46 +01:00
vorotamoroz aa23b6a39a ### Improved
- Now we can configure `forcePathStyle` for bucket synchronisation (#707).
2025-09-03 10:08:49 +01:00
vorotamoroz 58e328a591 bump 2025-09-02 10:27:23 +01:00
vorotamoroz 1730c39d70 ### Fixed
- Opening IndexedDB handling has been ensured.
- Migration check of corrupted files detection has been fixed.
    - Now informs us about conflicted files as non-recoverable, but noted so.
    - No longer errors on not-found files.
2025-09-02 10:24:13 +01:00
Abhith Rajan dfeac201a2 docs(README): update links for Customisation Sync and Hidden File Sync sections 2025-09-01 21:54:11 +04:00
vorotamoroz b42152db5e bump 2025-09-01 12:28:01 +09:00
vorotamoroz 171cfc0a38 ### Fixed
- Conflict resolving dialogue now properly displays the changeset name instead of A or B (#691).
2025-09-01 12:23:38 +09:00
vorotamoroz d2787bdb6a Update older dependencies 2025-09-01 12:21:12 +09:00
Ron Gerber 44b022f003 Add Docker Compose troubleshooting for own server setup
Added another option for the init command that passes the variables directly. When i tried the command above i got the mentioned error.
2025-08-30 01:00:58 +02:00
vorotamoroz 58845276e7 bump 2025-08-29 11:48:33 +01:00
vorotamoroz a2cc093a9e ### Fixed
- Fixed an issue with automatic synchronisation starting (#702).
2025-08-29 11:46:11 +01:00
vorotamoroz fec203a751 bump 2025-08-28 10:27:31 +01:00
vorotamoroz 1a06837769 ### Fixed
- Automatic translation detection on the first launch now works correctly (#630).
- No errors are shown during synchronisations in offline (if not explicitly requested) (#699).
- Missing some checking during automatic-synchronisation now works correctly.
2025-08-28 10:26:17 +01:00
vorotamoroz 18d1ce8ec8 bump 2025-08-26 11:17:09 +01:00
vorotamoroz 2221d8c4e8 Update dependency 2025-08-26 11:14:30 +01:00
vorotamoroz 08548f8630 ### New experimental feature
- We can perform Garbage Collection (Beta2) without rebuilding the entire database, and also fetch the database.

### Fixed

- Resetting the bucket now properly clears all uploaded files.

### Refactored

- Some files have been moved to better reflect their purpose and improve maintainability.
- The extensive LiveSyncLocalDB has been split into separate files for each role.
2025-08-26 11:09:33 +01:00
vorotamoroz 5d24c3b984 bump 2025-08-20 10:36:47 +01:00
vorotamoroz de8fd43c8b ### Fixed
- CORS Checking messages now use replacements.
- Configuring CORS setting via the UI now respects the existing rules.
- Now startup-checking works correctly again, performs migration check serially and then it will also fix starting LiveSync or start-up sync. (#696)
- Statusline in editor now supported 'Bases'.
2025-08-20 10:36:22 +01:00
vorotamoroz ed88761eaa bump 2025-08-18 06:32:19 +01:00
vorotamoroz 4dcb37f5a2 ## 0.25.8
### New feature
- Insecure chunk detection has been implemented.

### Fixed
- Unexpected `Failed to obtain PBKDF2 salt` or similar errors during bucket-synchronisation no longer occur.
- Unexpected long delays for chunk-missing documents when using bucket-synchronisation have been resolved.
- Fetched remote chunks are now properly stored in the local database if `Fetch chunks on demand` is enabled.
- The 'fetch' dialogue's message has been refined.
- No longer overwriting any corrupted documents to the storage on boot-sequence.

### Refactored
- Type errors have been corrected.
2025-08-18 06:26:50 +01:00
vorotamoroz db0562eda1 An minor note added 2025-08-15 11:06:55 +01:00
vorotamoroz b610d5d959 bump 2025-08-15 11:04:34 +01:00
vorotamoroz 5abba74f3b ## 0.25.7
### Fixed

- Off-loaded chunking have been fixed to ensure proper functionality (#693).
- Chunk document ID assignment has been fixed.
- Replication prevention message during version up detection has been improved (#686).
- `Keep A` and `Keep B` on Conflict resolving dialogue has been renamed to `Use Base` and `Use Conflicted` (#691).

### Improved

- Metadata and content-size unmatched documents are now detected and reported, prevented to be applied to the storage.

### New Features

- `Scan for Broken files` has been implemented on `Hatch` -> `TroubleShooting`.

### Refactored

- Off-loaded processes have been refactored for the better maintainability.
- Removed unused code.
2025-08-15 10:51:39 +01:00
vorotamoroz 021c1fccfe Merge pull request #667 from h-exx/main
Edited the Docker Compose documentation to make it work
2025-08-14 14:24:14 +09:00
vorotamoroz 0a30af479f Fix Fetch chunks on demand 2025-08-09 02:17:50 +09:00
vorotamoroz a9c3f60fe7 bump 2025-08-09 01:51:17 +09:00
vorotamoroz f996e056af ### Fixed
- Storage scanning no longer occurs when `Suspend file watching` is enabled (including boot-sequence).

### Improved
- Saving notes and files now consumes less memory.
- Chunk caching is now more efficient.
- Both of them (may) are effective for #692, #680, and some more.

### Changed
- `Incubate Chunks in Document` (also known as `Eden`) is now fully sunset.
- The `Compute revisions for chunks` setting has also been removed.
- As mentioned, `Memory cache size (by total characters)` has been removed.

### Refactored
- A significant refactoring of the core codebase is underway (please refer the release-note).
2025-08-09 01:45:41 +09:00
vorotamoroz 1073ee9e30 bump 2025-07-29 12:50:25 +01:00
vorotamoroz f94653e60e ## 0.25.4
- The PBKDF2Salt is no longer corrupted when attempting replication while the device is offline (#686)
2025-07-29 12:45:28 +01:00
vorotamoroz 3dccf2076f Add draft design documents 2025-07-28 19:14:22 +09:00
vorotamoroz 3e78fe03e1 bump 2025-07-22 04:36:22 +01:00
vorotamoroz 4aa8fc3519 ## 0.25.3
### Fixed
- Now the `Doctor` at migration will save the configuration.
2025-07-22 04:35:40 +01:00
vorotamoroz ba3d2220e1 bump again 2025-07-19 18:08:27 +09:00
vorotamoroz 8057b516af bump 2025-07-19 17:51:53 +09:00
vorotamoroz f2b4431182 ## 0.25.1
19th July, 2025

### Refined and New Features
- Fetching the remote database on `RedFlag` now also retrieves remote configurations optionally.
- The setup wizard using Set-up URI and QR code has been improved.

### Changes
- The Set-up URI is now encrypted with a new encryption algorithm (mostly the same as `V2`).
2025-07-19 17:26:52 +09:00
vorotamoroz badec46d9a 0.25.0 released 2025-07-19 15:21:36 +09:00
vorotamoroz 355e41f488 bump for beta 2025-07-14 00:36:09 +09:00
vorotamoroz e0e7e1b5ca ### Fixed
- The encryption algorithm now uses HKDF with a master key.
- `Fetch everything from the remote` now works correctly.
- Extra log messages during QR code decoding have been removed.

### Changed
- Some settings have been moved to the `Patches` pane:

### Behavioural and API Changes
- `DirectFileManipulatorV2` now requires new settings (as you may already know, E2EEAlgorithm).
- The database version has been increased to `12` from `10`.
2025-07-14 00:33:40 +09:00
vorotamoroz ce4b61557a bump 2025-07-10 11:24:59 +01:00
vorotamoroz 52b02f3888 ## 0.24.31
### Fixed

- The description of `Enable Developers' Debug Tools.` has been refined.
- Automatic conflict checking and resolution has been improved.
- Resolving conflicts dialogue will not be shown for the multiple files at once.
2025-07-10 11:12:44 +01:00
vorotamoroz 7535999388 Update updates.md 2025-07-09 22:28:50 +09:00
vorotamoroz dccf8580b8 Update updates.md 2025-07-09 22:27:52 +09:00
vorotamoroz e3964f3c5d bump 2025-07-09 12:48:37 +01:00
vorotamoroz 375e7bde31 ### New Feature
- New chunking algorithm `V3: Fine deduplication` has been added, and will be recommended after updates.
- New language `ko` (Korean) has been added.
- Chinese (Simplified) translation has been updated.

### Fixed

- Numeric settings are now never lost the focus during the value changing.

### Improved
- All translations have rewritten into YAML format, to easier manage and contribution.
- Doctor recommendations have now shown in the user-friendly notation.

### Refactored

- Never ending `ObsidianLiveSyncSettingTag.ts` finally had separated into each pane's file.
- Some commented-out codes have been removed.
2025-07-09 12:15:59 +01:00
Jacques Faulkner 341f0ab12d Updated Table of Contents 2025-06-27 10:39:13 +01:00
Jacques Faulkner 39340c1e1b Sorry 1 more, reworded the creating directories comments 2025-06-26 14:45:04 +01:00
Jacques Faulkner 55cdc58857 Edited warning to make a bit more sense 2025-06-26 14:42:23 +01:00
Jacques Faulkner 4f1a9dc4e8 Forgot a # 2025-06-25 18:54:22 +01:00
Jacques Faulkner 013818b7d0 Update setup_own_server.md 2025-06-25 18:53:31 +01:00
vorotamoroz 1179438df8 bump 2025-06-20 12:43:39 +01:00
vorotamoroz 47ea8f6859 ## 0.24.29
### Fixed

- Synchronisation with buckets now works correctly, regardless of whether a prefix is set or the bucket has been (re-) initialised (#664).
- An information message is now displayed again, during any automatic synchronisation is enabled (#662).

### Tidied up

- Importing paths have been tidied up.
2025-06-20 12:43:15 +01:00
vorotamoroz 670fe16486 Merge branch 'main' of https://github.com/vrtmrz/obsidian-livesync 2025-06-16 02:53:30 +01:00
vorotamoroz 3f0093916c Add some note 2025-06-16 02:52:59 +01:00
vorotamoroz 9503474d06 bump 2025-06-15 18:49:31 +09:00
vorotamoroz ddf7b243e4 ## 0.24.28
### Fixed

- Batch Update is no longer available in LiveSync mode to avoid unexpected behaviour. (#653)
- Now compatible with Cloudflare R2 again for bucket synchronisation.
- Prevention of broken behaviour due to database connection failures added (#649).
2025-06-15 18:49:16 +09:00
vorotamoroz f37561c3c1 Update Library 2025-06-15 18:24:19 +09:00
vorotamoroz f01429decc Merge pull request #633 from jmarmstrong1207/patch-1
Add simple docker compose to the setup guide
2025-06-10 11:27:22 +09:00
vorotamoroz c0fcb66924 bump 2025-06-10 02:52:45 +01:00
vorotamoroz 5f76b9809b ## 0.24.27
### Improved

- We can use prefix for path for the Bucket synchronisation.
- The "Use Request API to avoid `inevitable` CORS problem" option is now promoted to the normal setting, not a niche patch.

### Fixed

- Now switching replicators applied immediately, without the need to restart Obsidian.

### Tidied up

- Some dependencies have been updated to the latest version.
2025-06-10 02:51:47 +01:00
vorotamoroz d61d6fec37 bump manifest 2025-05-14 13:55:42 +01:00
vorotamoroz 9fdd622824 bump 2025-05-14 13:55:17 +01:00
vorotamoroz 3b8d03a189 ## 0.24.26
### New Features

- Automatic display-language changing according to the Obsidian language
  setting.
- Now we can limit files to be synchronised even in the hidden files.
- "Use Request API to avoid `inevitable` CORS problem" has been implemented.
- `Show status icon instead of file warnings banner` has been implemented.

### Improved

- All regular expressions can be inverted by prefixing `!!` now.

### Fixed

- No longer unexpected files will be gathered during hidden file sync.
- No longer broken `\n` and new-line characters during the bucket
  synchronisation.
- We can purge the remote bucket again if we using MinIO instead of AWS S3 or
  Cloudflare R2.
- Purging the remote bucket is now more reliable.
- Some wrong messages have been fixed.

### Behaviour changed

- Entering into the deeper directories to gather the hidden files is now limited
  by `/` or `\/` prefixed ignore filters.

### Etcetera

- Some code has been tidied up.
- Trying less warning-suppressing and be more safer-coding.
- Dependent libraries have been updated to the latest version.
- Some build processes have been separated to `pre` and `post` processes.
2025-05-14 13:11:03 +01:00
James Armstrong 1f1a39e5a0 Add simple docker compose 2025-04-28 06:10:29 -07:00
vorotamoroz d0e92cff7a refine readme 2025-04-23 06:45:03 +01:00
vorotamoroz 5addddc792 Update doc 2025-04-23 05:51:51 +01:00
vorotamoroz d978892661 Update docs 2025-04-23 05:11:59 +01:00
vorotamoroz cfb061a6a2 add note to troubleshooting 2025-04-23 05:10:40 +01:00
vorotamoroz 381055fc93 bump 2025-04-22 11:29:42 +01:00
vorotamoroz 37d12916fc ## 0.24.25
### Improved

- Peer-to-peer synchronisation has been got more robust.

### Fixed

- No longer broken falsy values in settings during set-up by the QR code generation.

### Refactored

- Some `window` references now have pointed to `globalThis`.
- Some sloppy-import has been fixed.
- A server side implementation `Synchromesh` has been suffixed with `deno` instead of `server` now.
2025-04-22 11:28:55 +01:00
vorotamoroz 944aa846c4 bump 2025-04-15 11:10:37 +01:00
vorotamoroz abca808e29 ### Fixed
- No longer broken JSON files including `\n`, during the bucket synchronisation. (#623)
- Custom headers and JWT tokens are now correctly sent to the server during configuration checking. (#624)

### Improved

- Bucket synchronisation has been enhanced for better performance and reliability.
    - Now less duplicated chunks are sent to the server.
    - Fetching conflicted files from the server is now more reliable.
    - Dependent libraries have been updated to the latest version.
2025-04-15 11:09:49 +01:00
vorotamoroz 90bb610133 Update submodule 2025-04-14 03:23:24 +01:00
vorotamoroz 9c5e9fe63b Conclusion stated. 2025-04-11 14:13:22 +01:00
vorotamoroz 00dfae24d7 bump 2025-04-10 14:28:03 +01:00
vorotamoroz d8a41fe45d ### New Feature
- Now, we can send custom headers to the server.
- Authentication with JWT in CouchDB is now supported.

### Improved

- The QR Code for set-up can be shown also from the setting dialogue now.
- Conflict checking for preventing unexpected overwriting on the boot-up process has been quite faster.

### Fixed

- Some bugs on Dev and Testing modules have been fixed.
2025-04-10 14:24:33 +01:00
vorotamoroz 30467d1c25 Add link to P2P pseudo client. 2025-04-04 12:30:21 +01:00
vorotamoroz f8351f1d45 Merge branch 'main' of https://github.com/vrtmrz/obsidian-livesync 2025-04-04 11:52:12 +01:00
vorotamoroz 5924af98ab Update Library (Probably no effect) 2025-04-04 11:52:05 +01:00
vorotamoroz 2769b61da4 Update setup_own_server.md
Add note; #609
2025-04-04 18:24:13 +09:00
vorotamoroz bb4409221d Update README.md 2025-04-03 20:57:15 +09:00
vorotamoroz f398c14200 Bump for release-mistake. 2025-04-01 10:42:29 +01:00
vorotamoroz 27d58508dc Missed 2025-04-01 10:38:12 +01:00
vorotamoroz d4dea5b226 bump 2025-04-01 10:21:38 +01:00
vorotamoroz c79dc30cba ## 0.24.21
### Fixed

- No longer conflicted files are handled in the boot-up process. No more unexpected overwriting.
    - It ignores `Always overwrite with a newer file`, and always be prevented for the safety. Please pick it manually or open the file.
- Some log messages on conflict resolution has been corrected.
- Automatic merge notifications, displayed on the grounds of `same`, have been degraded to logs.

### Improved

- Now we can fetch the remote database with keeping local files completely intact.
    - In new option, all files are stored into the local database before the fetching, and will be merged automatically or detected as conflicts.
- The dialogue presenting options when performing `Fetch` are now more informative.

### Refactored

- Some class methods have been fixed its arguments to be more consistent.
- Types have been defined for some conditional results.
2025-04-01 10:20:21 +01:00
vorotamoroz b3119ee8a9 bump and update dependencies 2025-03-24 18:55:02 +09:00
vorotamoroz 2a1d71da5c Improved: show details of TypeError using Obsidian API. 2025-03-24 12:04:58 +09:00
vorotamoroz 24f31ed19e Merge branch 'main' of https://github.com/vrtmrz/obsidian-livesync 2025-03-19 04:43:06 +01:00
vorotamoroz a982629ae6 Update draft : possibly I should share cSpell dictionary. 2025-03-19 04:42:50 +01:00
vorotamoroz 85140aecab Update README.md 2025-03-05 20:40:04 +09:00
vorotamoroz 3f2e23ee88 bump 2025-03-05 11:13:58 +00:00
vorotamoroz 6049c19e8a ## 0.24.19
### New Feature

- Now we can generate a QR Code for transferring the configuration to another device.
2025-03-05 11:12:00 +00:00
vorotamoroz 65648683a3 bump 2025-02-28 12:02:45 +00:00
vorotamoroz 5d70f2c1e9 ## 0.24.18
### Fixed

- Now no chunk creation errors will be raised after switching `Compute revisions for chunks`.
- Some invisible file can be handled correctly (e.g., `writing-goals-history.csv`).
- Fetching configuration from the server is now saves the configuration immediately (if we are not in the wizard).

### Improved

- Mismatched configuration dialogue is now more informative, and rewritten to more user-friendly.
- Applying configuration mismatch is now without rebuilding (at our own risks).
- Now, rebuilding is decided more fine grained.

### Improved internally

- Translations can be nested. i.e., task:`Some procedure`, check: `%{task} checking`, checkfailed: `%{check} failed` produces `Some procedure checking failed`.
2025-02-28 11:58:15 +00:00
vorotamoroz cbcfdc453e update default vault and bump for release 2025-02-27 13:39:28 +00:00
vorotamoroz a4eb21593c bump 2025-02-27 13:24:51 +00:00
vorotamoroz 05eb2c8262 ## 0.24.16
### Improved

#### Peer-to-Peer

- Now peer-to-peer synchronisation checks the settings are compatible with each other.
- Peer-to-peer synchronisation now handles the platform and detects pseudo-clients.

#### General

- New migration method has been implemented, that called `Doctor`.

- The minimum interval for replication to be caused when an event occurs can now be configurable.
- Some detail note has been added and change nuance about the `Report` in the setting dialogue, which had less informative.

### Behaviour and default changed

- `Compute revisions for chunks` are backed into enabled again. it is necessary for garbage collection of chunks.

### Refactored

- Platform specific codes are more separated. No longer `node` modules were used in the browser and Obsidian.
2025-02-27 13:23:11 +00:00
vorotamoroz fecefa3631 ### Fixed
- Now, even without WeakRef, Polyfill is used and the whole thing works without error. However, if you can switch WebView Engine, it is recommended to switch to a WebView Engine that supports WeakRef.

And bumped.
2025-02-20 10:40:18 +00:00
vorotamoroz f8c4d5ccb0 Add a bit more. 2025-02-18 13:48:41 +00:00
vorotamoroz e63e79bc8e Add note of flag files. 2025-02-18 13:36:15 +00:00
vorotamoroz ed76125f3d bump 2025-02-18 13:02:54 +00:00
vorotamoroz 70f4e23474 ## 0.24.14
### Fixed

- Resolving conflicts of JSON files (and sensibly merging them) is now working fine, again!
    - And, failure logs are more informative.
- More robust to release the event listeners on unwatching the local database.

### Refactored

- JSON file conflict resolution dialogue has been rewritten into svelte v5.
- Upgrade eslint.
- Remove unnecessary pragma comments for eslint.
2025-02-18 12:59:18 +00:00
vorotamoroz f6d5b78cc8 bump 2025-02-17 11:35:34 +00:00
vorotamoroz 405624b51b ## 0.24.13
### Fixed
#### General Replication
- No longer unexpected errors occur when the replication is stopped during for some reason (e.g., network disconnection).
#### Peer-to-Peer Synchronisation
- Set-up process will not receive data from unexpected sources.
- No longer resource leaks while enabling the `broadcasting changes`
- Logs are less verbose.
- Received data is now correctly dispatched to other devices.
- `Timeout` error now more informative.
- No longer timeout error occurs for reporting the progress to other devices.
- Decision dialogues for the same thing are not shown multiply at the same time anymore.
- Disconnection of the peer-to-peer synchronisation is now more robust and less error-prone.
#### Webpeer
- Now we can toggle Peers' configuration.
### Refactored
- Cross-platform compatibility layer has been improved.
- Common events are moved to the common library.
- Displaying replication status of the peer-to-peer synchronisation is separated from the main-log-logic.
- Some file names have been changed to be more consistent.
2025-02-17 11:33:35 +00:00
vorotamoroz 90c0ff22b9 Add paths for future maintenance. 2025-02-17 11:30:42 +00:00
vorotamoroz 67568ea886 bump 2025-02-14 11:28:01 +00:00
vorotamoroz cc29b4058d 0.24.12
Fixed
-  No longer unnecessary acknowledgements are sent when starting peer-to-peer synchronisation.

Refactored
- Platform impedance-matching-layer has been improved.
- Some UIs have been get isomorphic among Obsidian and web applications (for `webpeer`).
2025-02-14 11:15:22 +00:00
vorotamoroz 4e8243b3d5 Update release.yml 2025-02-13 22:02:22 +09:00
vorotamoroz 4eb1787784 bump 2025-02-13 12:58:15 +00:00
vorotamoroz 1cd1465f2c 0.24.11
Improved

- New Translation: `es` (Spanish) by @zeedif (Thank you so much)!
- Now all of messages can be selectable and copyable, also on the iPhone, iPad, and Android devices. Now we can copy or share the messages easily.

New Feature

- Peer-to-Peer Synchronisation has been implemented!

Fixed

- No longer memory or resource leaks when the plug-in is disabled.
- Now deleted chunks are correctly detected on conflict resolution, and we are guided to resurrect them.
- Hanging issue during the initial synchronisation has been fixed.
- Some unnecessary logs have been removed.
- Now all modal dialogues are correctly closed when the plug-in is disabled.

Refactor

- Several interfaces have been moved to the separated library.
- Translations have been moved to each language file, and during the build, they are merged into one file.
- Non-mobile friendly code has been removed and replaced with the safer code.
- Started writing Platform impedance-matching-layer.
- Svelte has been updated to v5.
- Some function have got more robust type definitions.
- Terser optimisation has slightly improved.
- During the build, analysis meta-file of the bundled codes will be generated.
2025-02-13 12:48:00 +00:00
vorotamoroz 45ceca8bb6 run prettier and update lib 2025-02-12 03:44:46 +00:00
vorotamoroz 7b385aab9e Merge branch 'main' of https://github.com/vrtmrz/obsidian-livesync 2025-02-12 03:22:50 +00:00
vorotamoroz 98411e5f48 Merge pull request #573 from zeedif/main
Migrate UI strings to $tf for translation support
2025-02-12 12:21:44 +09:00
vorotamoroz b6687e2fb0 Adding bundle size analysis 2025-02-12 03:18:59 +00:00
Zeedif 658cbb7ded Merge branch 'main' into main 2025-02-06 20:46:26 -06:00
vorotamoroz 08a48154fa Add note 2025-02-05 01:35:17 +00:00
vorotamoroz 62501a5940 bump again 2025-02-05 01:33:31 +00:00
vorotamoroz ccb3dd52de Dep update 2025-02-05 01:10:03 +00:00
vorotamoroz 3e5f4c8946 bump 2025-02-05 01:04:21 +00:00
vorotamoroz 54e64c59a9 - Prettified some source.
- Fixed the issue which the filename is shown as `undefined`.
- Fixed the issue where files transferred at short intervals were not reflected.
- Updated dependency (including new translation)
2025-02-05 00:58:51 +00:00
CRuiz 588840ff8b Reemplazar $tf por $msg 2025-02-04 08:22:43 -06:00
CRuiz e6b8dfb279 Actualizar mensaje de recomendación de URI en la migración del módulo 2025-01-22 15:57:36 -06:00
CRuiz 73782c5389 Add translation ids 2025-01-22 13:41:18 -06:00
vorotamoroz f2b667d75e bump 2025-01-22 11:56:36 +00:00
vorotamoroz 9b1588a65b ## 0.24.8
### Fixed
-   Some parallel-processing tasks are now performed more safely.
-   Some error messages has been fixed.
### Improved
-   Synchronisation is now more efficient and faster.
-   Saving chunks is a bit more robust.
### New Feature
-   We can remove orphaned chunks again, now!
2025-01-22 11:55:56 +00:00
vorotamoroz 0629bc04bb Merge branch 'main' of https://github.com/vrtmrz/obsidian-livesync 2025-01-07 11:33:13 +00:00
vorotamoroz b0e97e6c96 Bump 2025-01-07 11:31:58 +00:00
vorotamoroz 7f853b0222 - 0.24.7
- Fixed (Security)
   - Assigning IDs to chunks has been corrected for more safety.
  - Fixed
    - Conflict resolution dialogue has been fixed
    - Resolving conflicts by timestamp has been fixed
  - Improved
    -  Notifications can be suppressed for the hidden files update now.
    -  No longer uses the old-xxhash and sha1 for generating the chunk ID.
2025-01-07 11:28:56 +00:00
vorotamoroz 3d4ad4a3b4 Update README.md
Add acknowledgements
2025-01-02 15:02:35 +09:00
vorotamoroz 4b1fff852a Merge pull request #544 from dcvdiego/main
make self-hosted docs slightly more readable
2024-12-23 12:28:37 +09:00
dcvdiego 08d7d24baf Update setup_own_server.md 2024-12-21 16:26:55 +00:00
vorotamoroz 9db3c3df0a bump 2024-12-14 02:32:07 +09:00
vorotamoroz 672940ad6f Fixed: No longer the empty-and-stilled log pane. 2024-12-14 02:31:31 +09:00
vorotamoroz 2338601fae 0.24.5 (by Library Fixing)
Fixed
- Fix some wrong behaviour during comparing JSON objects

Improved
- Reactive values are now shown more smoothly (Logs,a status line.
2024-12-13 10:59:05 +00:00
vorotamoroz 3e657b38a9 bump 2024-12-12 11:12:04 +00:00
vorotamoroz 21861d8c51 ## 0.24.4
### Fixed

-   Fixed so many inefficient and buggy modules inherited from the past.

### Improved

-   Tasks are now executed in an efficient asynchronous library.
-   On-demand chunk fetching is now more efficient and keeps the interval between requests.
    -   This will reduce the load on the server and the network.
    -   And, safe for the Cloudant.
2024-12-12 11:10:50 +00:00
vorotamoroz 3bb4aba395 Update dependencies 2024-12-12 11:10:29 +00:00
vorotamoroz 751de5a13e bump 2024-12-09 00:46:15 +00:00
vorotamoroz 29229f809b Update the doc 2024-12-09 00:40:15 +00:00
vorotamoroz 2d0dc2a389 Merge pull request #543 from Volkor3-16/main
Fix up wording of messages
2024-12-09 09:12:51 +09:00
Volkor 6cbe319b80 remove a unneeded "the" 2024-12-04 02:02:25 +11:00
Volkor e9fe58f818 run pretty - revert crlf to lf
since most modern ides handle lf on windows fine, and the entire project is in lf already, i'll set this back
2024-12-04 01:13:12 +11:00
Volkor 61524e1c44 FIX: manual setup button 2024-12-04 00:17:41 +11:00
vorotamoroz c25eaa09c9 fixed: Fixed font-family of in-editor status. 2024-12-03 10:15:53 +00:00
Volkor 71987e6814 Change a few more lines to more correctly reflect the settings 2024-11-27 23:39:47 +11:00
vorotamoroz b15d0710e5 bump 2024-11-21 11:40:37 +00:00
vorotamoroz 9d304b3233 v0.24.2
Rewritten

-   Hidden File Sync is now respects the file changes on the storage. Not simply comparing modified times.
    -   This makes hidden file sync more robust and reliable.

Fixed

-   `Scan hidden files before replication` is now configurable again.
-   Some unexpected errors are now handled more gracefully.
-   Meaningless event passing during boot sequence is now prevented.
-   Error handling for non-existing files has been fixed.
-   Hidden files will not be batched to avoid the potential error.
    -   This behaviour had been causing the error in the previous versions in specific situations.
-   The log which checking automatic conflict resolution is now in verbose level.
-   Replication log (skipping non-targetting files) shows the correct information.
-   The dialogue that asking enabling optional feature during `Rebuild Everything` now prevents to show the `overwrite` option.
    -   The rebuilding device is the first, meaningless.
-   Files with different modified time but identical content are no longer processed repeatedly.
-   Some unexpected errors which caused after terminating plug-in are now avoided.
-

Improved

-   JSON files are now more transferred efficiently.
    -   Now the JSON files are transferred in more fine chunks, which makes the transfer more efficient.
2024-11-21 11:40:15 +00:00
dcvdiego d062b13040 make docs slightly more readable 2024-11-19 16:28:55 +00:00
Volkor 7eceab59af Fix up wording of messages 2024-11-18 21:55:22 +11:00
vorotamoroz ed5cb3e043 Format utility 2024-11-12 01:09:07 +00:00
vorotamoroz 574fdf9202 Format submodule 2024-11-12 01:08:35 +00:00
vorotamoroz 9ec7b809a9 I have adapted the new-line-char to the codebase. But I am not sure if this is the right thing to do. 2024-11-12 01:04:46 +00:00
vorotamoroz 4d302aff9d Merge pull request #533 from doublethefish/chore/formatting
chore(formatting): adds prettier for consistent coding style
2024-11-12 09:53:50 +09:00
vorotamoroz b70009f4a9 Merge branch 'main' into chore/formatting 2024-11-12 09:37:52 +09:00
Frank Harrison c24ee32f37 chore(formatting): ignores generated code in pretty 2024-11-11 09:45:53 +00:00
Frank Harrison 012d0aa4df chore(format): also format json (no effect)
Adds json-file formating to the prettier command. Currently this has no
effect, but if the `tabWidth` is changed to 2 and the `printWidth`
reduced to 88, then it will also impact the json files and keep the
formatting consistent across code and data.
2024-11-11 09:41:56 +00:00
Frank Harrison 5c97e5b672 chore(format): no intentional behaviour change - runs pretty 2024-11-11 09:41:52 +00:00
Frank Harrison 6e1eb36f3b chore(format): adds prettier and commands to run it
... including ci/cd check-only commands.

The code is quite complex and missing a lot of dev-ops types checks and
standards. One of the key problems with codebases like this is
on-boarding new developers to the codebase (like myself). When there is
a consistent and enforced coding-style (irrespective of what the coding
style is) it makes it _significantly_ easier for collaborators and
maintainers to get on with the job in hand. It also, from a day-2-day
developer perspective, significantly reduces cognitive overhead re
reading code.
Finally this is a "trial balloon" PR, if this patch is accepted I will
likely do more work on testing and docs for the project.

- The new prettier config is a non-standard setup, but a close-match to
  how the code _currently_ looks.
- 120 col-width print width (instead of the better and more
  information-dense 88), this is so the diff after applying prettier to
  the code is less disruptive, whilst still showing the benefits of using
  a prettier.
- We use `tabWidth` setting of 4 as the code uses that more common
  setting instead of the more compact 2 spaces - note that 2 often leads
  to more readable and compact code.
- We enforce trailing commas, as that seems to be the norm in this
  code-base. We choose the `es5` standard here.
- We enforce tailing semi-colons (`semi`) as the majority of code used
  that flavour of `js`/`ts`.
- For now we only run on code and not json files.

This is designed such that `npm run pretty` re-formats the code for
development, and when integrated with ci/cd, `prettyCheck` will return
non-zero exit codes when formatting doesn't match the coding standards.
2024-11-11 09:41:35 +00:00
vorotamoroz 8809aee327 Add default values to a new SetupURI. 2024-11-11 08:05:10 +00:00
vorotamoroz fc04c557fc Merge branch 'main' of https://github.com/vrtmrz/obsidian-livesync 2024-11-11 01:23:07 +00:00
vorotamoroz 115a0d2d8a bump 2024-11-11 01:22:40 +00:00
vorotamoroz 2c97289ec8 Fixed
-   Vault History can show the correct information of match-or-not for each file and database even if it is a binary file.
-   `Sync settings via markdown` is now hidden during the setup wizard.
-   Verify and Fix will ignore the hidden files if the hidden file sync is disabled.

New feature
-   Now we can fetch the tweaks from the remote database while the setting dialogue and wizard are processing.

Improved
-   More things are moved to the modules.
    -   Includes the Main codebase. Now `main.ts` is almost stub.
-   EventHub is now more robust and typesafe.
2024-11-11 00:58:31 +00:00
vorotamoroz 6d472d17fd Fix output order 2 2024-11-09 10:31:34 +09:00
vorotamoroz 3a3aabfd11 Fix scripting 2024-11-09 10:21:05 +09:00
vorotamoroz 8b45dd1d24 .. 2024-11-08 11:12:35 +00:00
vorotamoroz a2b36ccf31 v0.24.0! 2024-10-28 11:18:29 +09:00
vorotamoroz 25e30fa09d Merge tag '0.24.0.dev-rc8' 2024-10-28 10:24:38 +09:00
vorotamoroz 8f5bc387b4 Update manifest-beta.json 2024-10-25 20:44:32 +09:00
vorotamoroz 5afe24c460 0.24.0.dev-rc8 2024-10-25 12:39:32 +01:00
vorotamoroz 658a09f1cc Update manifest-beta.json 2024-10-24 20:46:08 +09:00
vorotamoroz a9020a3aea 0.24.0.dev-rc7 2024-10-24 12:41:50 +01:00
vorotamoroz 293c731437 Update manifest-beta.json 2024-10-22 18:11:18 +09:00
vorotamoroz 5b4ae37030 0.24.0.dev-rc6 2024-10-22 10:08:20 +01:00
vorotamoroz 9e8d126259 Update manifest-beta.json 2024-10-21 17:52:28 +09:00
vorotamoroz 6d244a6e34 0.24.0.dev-rc5 2024-10-21 09:47:09 +01:00
vorotamoroz 1f0ad4eb1e Update manifest-beta.json 2024-10-18 19:23:48 +09:00
vorotamoroz e0e0ab0426 0.24.0.dev-rc4 2024-10-18 11:14:58 +01:00
vorotamoroz 5023d6da0b Merge pull request #474 from nyawox/per-file-sync-grammar
fix: per-file customization sync description grammar
2024-10-18 11:43:03 +09:00
vorotamoroz 49160c7d57 Merge branch 'main' into per-file-sync-grammar 2024-10-18 11:42:49 +09:00
vorotamoroz 4434224c29 Merge pull request #479 from fuhrysteve/patch-1
add username and password to setup URI instructions
2024-10-18 11:40:23 +09:00
vorotamoroz cf3b9e5522 Add manifest beta 2024-10-17 10:26:19 +01:00
vorotamoroz 7ca5ac5ac7 0.24.0.dev-rc3 2024-10-17 10:19:08 +01:00
vorotamoroz 095a3d20fb 0.24.0.dev-rc2 2024-10-17 09:57:42 +01:00
vorotamoroz 89e23b1bf4 Preparing v0.24.0 2024-10-16 12:44:07 +01:00
vorotamoroz 48315d657d bump 2024-09-24 14:02:53 +01:00
vorotamoroz b73ca73776 Refined:
- Setting dialogue very slightly refined.
  - The hodgepodge inside the `Hatch` pane has been sorted into more explicit categorised panes.
  - Applying the settings will now be more informative.

New features:
- Word-segmented chunk building on users language.

Fixed:
- Sending chunks on `Send chunk in bulk` are now buffered to avoid the out-of-memory error.
- `Send chunk in bulk` is back to default disabled.
- Merging conflicts of JSON files are now works fine even if it contains `null`.
Development:
- Implemented the logic for automatically generating the stub of document for the setting dialogue.
2024-09-24 14:00:44 +01:00
vorotamoroz 48e4d57278 bump 2024-09-08 17:58:10 +09:00
vorotamoroz 7eae25edd0 - Fixed:
- Case-insensitive file handling
    - Full-lower-case files are no longer created during database checking.
  - Bulk chunk transfer
    - The default value will automatically adjust to an acceptable size when using IBM Cloudant.
2024-09-08 17:55:04 +09:00
vorotamoroz 3285c1694b bump 2024-09-07 01:45:12 +09:00
vorotamoroz ede126d7d4 - 0.23.21:
- New Features:
    - Case-insensitive file handling
      - Files can now be handled case-insensitively.
      - This behaviour can be modified in the settings under `Handle files as Case-Sensitive` (Default: Prompt, Enabled for previous behaviour).
    - Improved chunk revision fixing
        - Revisions for chunks can now be fixed for faster chunk creation.
        - This can be adjusted in the settings under `Compute revisions for chunks` (Default: Prompt, Enabled for previous behaviour).
    - Bulk chunk transfer
      - Chunks can now be transferred in bulk during uploads.
      - This feature is enabled by default through `Send chunks in bulk`.
    - Creation of missing chunks without
      - Missing chunks can be created without storing notes, enhancing efficiency for first synchronisation or after prolonged periods without synchronisation.
  - Improvements:
    - File status scanning on the startup
      - Quite significant performance improvements.
      - No more missing scans of some files.
    - Status in editor enhancements
      - Significant performance improvements in the status display within the editor.
      - Notifications for files that will not be synchronised will now be properly communicated.
    - Encryption and Decryption
      - These processes are now performed in background threads to ensure fast and stable transfers.
    - Verify and repair all files
      - Got faster through parallel checking.
    - Migration on update
      - Migration messages and wizards have become more helpful.
  - Behavioural changes:
    - Chunk size adjustments
      - Large chunks will no longer be created for older, stable files, addressing storage consumption issues.
    - Flag file automation
      - Confirmation will be shown and we can cancel it.
  - Fixed:
    - Database File Scanning
      - All files in the database will now be enumerated correctly.
  - Miscellaneous
    - Dependency updated.
    - Now, tree shaking is left to terser, from esbuild.
2024-09-07 01:43:21 +09:00
Stephen J. Fuhry f778107727 add username and password to setup URI instructions 2024-08-07 14:43:38 -04:00
vorotamoroz 630889680e bump 2024-07-31 02:32:02 +01:00
vorotamoroz e46714e0f9 Fixed:
- Remote Storage Limit Notification dialogue has been fixed, now the chosen value is saved.
Improved:
- The Enlarging button on the enlarging threshold dialogue now displays the new value.
2024-07-31 02:31:13 +01:00
vorotamoroz 86d5582f37 bump 2024-07-31 02:14:11 +01:00
vorotamoroz 697ee1855b Fixed:
- Customisation Sync now checks the difference while storing or applying the configuration.
- Time difference in the dialogue has been fixed.
2024-07-31 02:13:25 +01:00
nyawox 12d825ea49 fix: per-file customization sync description grammar 2024-07-26 23:38:01 +09:00
vorotamoroz b8edc85528 bump 2024-07-25 13:37:34 +01:00
vorotamoroz e2740cbefe New feature:
- Per-file-saved customization sync has been shipped.
- Customisation sync has got beta3.
Improved:
- Start-up speed has been improved.
Fixed:
- On the customisation sync dialogue, buttons are kept within the screen.
- No more unnecessary entries on `data.json` for customisation sync.
- Selections are no longer lost while updating customisation items.
Tidied on source codes:
- Many typos have been fixed.
- Some unnecessary type casting removed.
2024-07-25 13:36:26 +01:00
vorotamoroz a96e4e4472 bump 2024-07-12 10:13:04 +01:00
vorotamoroz dd26bbfe64 Improved:
- Overall performance has been improved by using PouchDB 9.0.0.
- Configuration mismatch detection is refined. We can resolve mismatches more smoothly and naturally.
Fixed:
- Customisation Sync will be disabled when a corrupted configuration is detected.
New feature:
- We can get a notification about the storage usage of the remote database.
2024-07-12 10:11:16 +01:00
vorotamoroz 6b9bd473cf bump 2024-07-10 05:24:26 +01:00
vorotamoroz 4be4fa6cc7 Maintenance:
- Library refining (Phase 1 - step 2). There are no significant changes on the user side.
2024-07-10 05:23:34 +01:00
vorotamoroz a9745e850e Improved:
- The passphrase of the Setup URI is now automatically generated. (#426)
2024-07-01 11:05:33 +01:00
vorotamoroz 7b9515a47e bump 2024-07-01 06:18:52 +01:00
vorotamoroz 220dce51f2 Dependency Update 2024-07-01 06:16:04 +01:00
vorotamoroz a23fc866c0 Tidied:
- Thinning of this repository through the creation of a library of universal functions
2024-07-01 06:12:23 +01:00
vorotamoroz 5c86966d89 Bump 2024-06-14 12:36:18 +01:00
vorotamoroz 29ed4d2b95 Fixed:
- No longer batch-saving ignores editor inputs.
- The file-watching and serialisation processes have been changed to the one which is similar to previous implementations.
- We can configure the settings (Especially about text-boxes) even if we have configured the device name.
Improved:
- We can configure the delay of batch-saving.
  - Default: 5 seconds, the same as the previous hard-coded value. (Note: also, the previous behaviour was not correct).
- Also, we can configure the limit of delaying batch-saving.
- The performance of showing status indicators has been improved.
2024-06-14 12:35:56 +01:00
vorotamoroz 16c6c52128 bump 2024-06-04 11:35:36 +01:00
vorotamoroz 8b94a0b72e Fixed:
- No longer files have been trimmed even delimiters have been continuous.
- Fixed the toggle title to `Do not split chunks in the background` from `Do not split chunks in the foreground`.
- Non-configured item mismatches are no longer detected.
2024-06-04 11:34:42 +01:00
vorotamoroz c5ac76d916 bump 2024-05-30 10:53:48 +01:00
vorotamoroz b67a6db8a1 Improved:
- Now notes will be split into chunks in the background thread to improve smoothness.
  - Default enabled, to disable, toggle `Do not split chunks in the foreground` on `Hatch` -> `Compatibility`.
  - If you want to process very small notes in the foreground, please enable `Process small files in the foreground` on `Hatch` -> `Compatibility`.
- We can use a `splitting-limit-capped chunk splitter`; which performs more simple and make less amount of chunks.
  - Default disabled, to enable, toggle `Use splitting-limit-capped chunk splitter` on `Sync settings` -> `Performance tweaks`
Tidied
  - Some files have been separated into multiple files to make them more explicit in what they are responsible for.
2024-05-30 10:52:20 +01:00
vorotamoroz d4202161e8 bump 2024-05-28 12:27:30 +01:00
vorotamoroz 2a2b39009c Fixed:
- Now we *surely* can set the device name and enable customised synchronisation.
- Unnecessary dialogue update processes have been eliminated.
- Customisation sync no longer stores half-collected files.
- No longer hangs up when removing or renaming files with the `Sync on Save` toggle enabled.
Improved:
- Customisation sync now performs data deserialization more smoothly.
- New translations have been merged.
2024-05-28 12:26:23 +01:00
vorotamoroz bf3a6e7570 Add the documentation and new a build option (buildDev). 2024-05-28 08:56:26 +01:00
vorotamoroz 069b8513d1 bump 2024-05-27 12:21:08 +01:00
vorotamoroz 128b1843df Fixed: No longer configurations have been locked in the minimal setup. 2024-05-27 12:20:18 +01:00
vorotamoroz fd722b1fe5 bump 2024-05-27 12:05:41 +01:00
vorotamoroz 0bf087dba0 Fixed:
- No longer unexpected parallel replication is performed.
- Now we can set the device name and enable customised synchronisation again.
2024-05-27 12:04:19 +01:00
vorotamoroz 3a4b59b998 Update troubleshooting.md 2024-05-27 12:12:37 +09:00
vorotamoroz 8fc9d51c45 Add Note. 2024-05-27 04:11:44 +01:00
vorotamoroz 35feb5bf93 bump 2024-05-22 14:05:15 +01:00
vorotamoroz b3a85c5462 New feature:
- Now we are ready for i18n.
- The setting dialogue has been refined. Very controllable, clearly displayed disabled items, and ready to i18n.
Fixed:
- Many memory leaks have been rescued.
- Chunk caches now work well.
- Many trivial but potential bugs are fixed.
- No longer error messages will be shown on retrieving checkpoint or server information.
- Now we can check and correct tweak mismatch during the setup
Improved:
- Customisation synchronisation has got more smoother.
Tidied
- Practically unused functions have been removed or are being prepared for removal.
- Many of the type-errors and lint errors have been corrected.
- Unused files have been removed.
Note:
- From this version, some test files have been included. However, they are not enabled and released in the release build.
2024-05-22 14:04:22 +01:00
vorotamoroz 7b0ac22c3b Create terms.md 2024-05-13 14:04:02 +09:00
vorotamoroz dca8e4b2a4 bump 2024-05-10 11:38:03 +01:00
vorotamoroz 89de2dcc37 Fixed:
- No longer missing tasks which have queued as the same key (e.g., for the same operation to the same file).
- Some trivial issues have been fixed.
New feature:
- Reloading Obsidian can be scheduled until that file and database operations are stable.
2024-05-10 11:33:59 +01:00
vorotamoroz 172b08dbb3 bump 2024-05-08 23:57:19 +09:00
vorotamoroz d518a3fc1b Fixed:
- Now the remote chunks could be decrypted even if we are using `Incubate chunks in Document`. (The note of 0.23.6 has been fixed).
- Chunk retrieving with `Incubate chunks in document` got more efficiently.
- No longer task processor misses the completed tasks.
- Replication is no longer started automatically during changes in window visibility (e.g., task switching on the desktop) when off-focused.
2024-05-08 23:56:29 +09:00
vorotamoroz c6ed867498 bump 2024-05-07 12:59:55 +01:00
vorotamoroz 4f4923e977 New feature:
- Now we can check configuration mismatching between clients before synchronisation.
- Now we can perform remote database compaction from the `Maintenance` pane.
Fixed:
- We can detect the bucket could not be reachable.
2024-05-07 12:55:48 +01:00
vorotamoroz a5ebf29b3d Merge pull request #417 from MichaelBrunn3r/translation
fix: Grammar issues in settings page
2024-05-07 20:26:59 +09:00
vorotamoroz ee465184c8 bump 2024-05-05 23:46:16 +09:00
vorotamoroz d7d4f1e6f2 New feature:
- We can now use `Incubate Chunks in Document` to reduce non-well-formed chunks.
Fixed:
- No longer experimental configuration is shown on the Minimal Setup.
2024-05-05 23:45:45 +09:00
Michael Brunner cbf5023593 fix: Grammar issues in settings page 2024-05-04 12:34:53 +02:00
vorotamoroz 3925052f92 Add the design document of planned improving. 2024-05-04 02:59:24 +09:00
vorotamoroz 1934418258 Add the design document of planned improving. 2024-05-04 01:57:03 +09:00
vorotamoroz 2ae018b2bd Refactor:
- Files have been categorised for clarity. The deliverables are not affected.
2024-05-02 04:07:36 +01:00
vorotamoroz 8474497985 bump 2024-05-01 02:24:08 +09:00
vorotamoroz b5714cc83b Fixed:
- No longer unwanted `\f` in journal sync.
2024-05-01 02:22:30 +09:00
vorotamoroz 133f5a7109 bump 2024-04-30 11:49:16 +01:00
vorotamoroz daa3feebf1 Fixed:
- Journal Sync will not hang up during big replication, especially the initial one.
- All changes which have been replicated while rebuilding will not be postponed (Previous behaviour).
Improved:
- Now Journal Sync works efficiently in download and parse, or pack and upload.
- Less server storage and faster packing/unpacking usage by the new chunk format.
2024-04-30 11:48:27 +01:00
vorotamoroz 7b5f7d0fbf bump 2024-04-30 01:40:01 +09:00
vorotamoroz 29532193cb - Fixed:
- Now journal synchronisation considers untransferred each from sent and received.
  - Journal sync now handles retrying.
  - Journal synchronisation no longer considers the synchronisation of chunks as revision updates (Simply ignored).
  - Journal sync now splits the journal pack to prevent mobile device rebooting.
  - Maintenance menus which had been on the command palette are now back in the maintain pane on the setting dialogue.
- Improved:
  - Now all changes which have been replicated while rebuilding will be postponed.
2024-04-30 01:39:09 +09:00
vorotamoroz 5b4309c09d For the future. Because of a good opportunity. 2024-04-29 02:01:27 +09:00
vorotamoroz 16ef582453 Update: wrote about the new Remote Type. 2024-04-28 23:37:26 +09:00
vorotamoroz 3e22f70c7a Update README.md 2024-04-28 17:49:50 +09:00
vorotamoroz 0a8dbe097e bump 2024-04-27 03:35:32 +09:00
vorotamoroz 2c0fcf74d0 New feature: Object storage support 2024-04-27 03:33:59 +09:00
vorotamoroz a1ab1efd5d Update README.md 2024-04-20 21:45:21 +09:00
vorotamoroz c8fcf2d0d5 Bump 2024-04-19 12:06:09 +01:00
vorotamoroz c384e2f7fb Fixed:
- No longer data corrupting due to false BASE64 detections.
2024-04-19 12:04:14 +01:00
vorotamoroz 99c1c7dc1a bump 2024-04-18 12:37:49 +01:00
vorotamoroz 84adec4b1a New feature: Automatic data compression to reduce amount of traffic and the usage of remote database. 2024-04-18 12:30:29 +01:00
vorotamoroz f0b202bd91 bump 2024-04-12 01:32:03 +09:00
vorotamoroz d54b7e2d93 - Fixed:
- Error handling on booting now works fine.
  - Replication is now started automatically in LiveSync mode.
  - Batch database update is now disabled in LiveSync mode.
  - No longer automatically reconnection while off-focused.
  - Status saves are thinned out.
  - Now Self-hosted LiveSync waits for all files between the local database and storage to be surely checked.
- Improved:
  - The job scheduler is now more robust and stable.
  - The status indicator no longer flickers and keeps zero for a while.
  - No longer meaningless frequent updates of status indicators.
  - Now we can configure regular expression filters in handy UI. Thank you so much, @eth-p!
  - `Fetch` or `Rebuild everything` is now more safely performed.
- Minor things
  - Some utility function has been added.
  - Customisation sync now less wrong messages.
  - Digging the weeds for eradication of type errors.
2024-04-12 01:30:35 +09:00
vorotamoroz 6952ef37f5 Update quick_setup.md 2024-04-09 13:10:31 +09:00
vorotamoroz 9630bcbae8 bump 2024-03-22 10:50:03 +01:00
vorotamoroz c3f925ab9a Merge branch 'main' of https://github.com/vrtmrz/obsidian-livesync 2024-03-22 10:48:25 +01:00
vorotamoroz 034dc0538f - Fixed:
- Fixed the issue that binary files were sometimes corrupted.
  - Fixed customisation sync data could be corrupted.
- Improved:
  - Now the remote database costs lower memory.
    - This release requires a brief wait on the first synchronisation, to track the latest changeset again.
  - Description added for the `Device name`.
- Refactored:
  - Many type-errors have been resolved.
  - Obsolete file has been deleted.
2024-03-22 10:48:16 +01:00
vorotamoroz b6136df836 Update quick_setup.md 2024-03-22 14:27:34 +09:00
vorotamoroz 24aacdc2a1 bump 2024-03-22 04:07:17 +01:00
vorotamoroz f91109b1ad - Improved:
- Faster start-up by removing too many logs which indicates normality
  - By streamlined scanning of customised synchronisation extra phases have been deleted.
2024-03-22 04:07:07 +01:00
vorotamoroz e76e7ae8ea bump 2024-03-19 17:59:38 +01:00
vorotamoroz f7fbe85d65 - New feature:
- We can disable the status bar in the setting dialogue.
- Improved:
  - Now some files are handled as correct data type.
  - Customisation sync now uses the digest of each file for better performance.
  - The status in the Editor now works performant.
- Refactored:
  - Common functions have been ready and the codebase has been organised.
  - Stricter type checking following TypeScript updates.
  - Remove old iOS workaround for simplicity and performance.
2024-03-19 17:58:55 +01:00
vorotamoroz 0313443b29 Merge pull request #389 from Seeker0472/fix-command
Fixed docker-compose command in docs
2024-03-19 14:06:23 +09:00
seeker0472 755c30f468 fix docker-compose command 2024-03-17 14:30:35 +08:00
vorotamoroz b00b0cc5e5 bump 2024-03-15 10:37:15 +01:00
vorotamoroz d7985a6b41 Improved:
- Now using HTTP for the remote database URI warns of an error (on mobile) or notice (on desktop).
2024-03-15 10:36:00 +01:00
vorotamoroz 486e816902 Update dependencies 2024-03-15 10:35:41 +01:00
vorotamoroz ef9b19c24b bump 2024-03-04 04:07:51 +00:00
vorotamoroz 4ed9494176 Changed:
- The default settings has been changed.
Improved:
- Default and preferred settings are applied on completion of the wizard.
Fixed:
- Now Initialisation `Fetch` will be performed smoothly and there will be fewer conflicts.
- No longer stuck while Handling transferred or initialised documents.
2024-03-04 04:07:11 +00:00
vorotamoroz fcd56d59d5 bump 2024-03-01 08:33:37 +00:00
vorotamoroz 1cabfcfd19 Fixed:
- `Verify and repair all files` is no longer broken.
New feature::
- Now `Verify and repair all files` can restore or show history
Improved:
- Performance improved
2024-03-01 08:32:48 +00:00
vorotamoroz 37a18dbfef bump 2024-03-01 03:28:46 +00:00
vorotamoroz e7edf88713 Fixed
- No longer unchanged hidden files and customisations are saved and transferred now.
- File integrity of vault history indicates the integrity correctly.
Improved
- In the report, the schema of the remote database URI is now printed.
2024-03-01 03:28:06 +00:00
vorotamoroz 90ff75ab35 add notes. 2024-02-29 00:30:07 +00:00
vorotamoroz bff1d661f5 Update troubleshooting.md
Fix grammar
2024-02-29 00:42:28 +09:00
vorotamoroz 6b59c14774 Update doc 2024-02-28 08:29:06 +00:00
vorotamoroz 8249274eac bump 2024-02-28 08:28:07 +00:00
vorotamoroz 3c6dae7814 - Fixed:
- Fixed a bug on `fetch chunks on demand` that could not fetch the chunks on demand.
- Improved:
  - `fetch chunks on demand` works more smoothly.
  - Initialisation `Fetch` is now more efficient.
- Tidied:
  - Removed some meaningless codes.
2024-02-28 08:27:17 +00:00
vorotamoroz 60cf8fe640 bump 2024-02-27 08:36:37 +00:00
vorotamoroz 3d89b3863f Fixed:
- Now fetch and unlock the locked remote database works well again.
- No longer crash on symbolic links inside hidden folders.
Improved:
- Chunks are now created more efficiently.
- Better performance in saving notes.
- Network activities are indicated as an icon.
- Less memory used for binary processing.
Tidied:
- Cleaned unused functions up.
- Sorting out the codes that have become nonsense.
Changed:
- Now no longer `fetch chunks on demand` needs `Pacing replication`
2024-02-27 08:35:46 +00:00
vorotamoroz ee9364310d bump 2024-02-20 09:36:43 +00:00
vorotamoroz 86b9695bc2 Fixed:
- No longer deleted hidden files were ignored.
- The document history dialogue is now able to process the deleted revisions.
- Deletion of a hidden file is now surely performed even if the file is already conflicted.
2024-02-20 09:32:48 +00:00
vorotamoroz e05f8771b9 bump 2024-02-20 05:15:48 +00:00
vorotamoroz 65619c2478 Fixed:
- Fixed a problem with synchronisation taking a long time to start in some cases.
- Now we can disable E2EE encryption.
Improved:
- `Setup Wizard` is now more clear.
- `Minimal Setup` is now more simple.
- Self-hosted LiveSync now be able to use even if there are vaults with the same name.
- Now Self-hosted LiveSync waits until set-up is complete.
- Show reload prompts when possibly recommended while settings.
New feature:
- A guidance dialogue prompting for settings will be shown after the installation.
Changed
- Some setting names has been changed
2024-02-20 05:13:53 +00:00
vorotamoroz 1552fa9d9e Merge branch 'main' of https://github.com/vrtmrz/obsidian-livesync 2024-02-15 08:36:49 +00:00
vorotamoroz 767f12b52f Rewrite 2024-02-15 08:35:12 +00:00
vorotamoroz 4071ba120e Update troubleshooting.md 2024-02-15 13:56:15 +09:00
vorotamoroz 2c0e3ba01c update faq 2024-02-15 04:38:31 +00:00
vorotamoroz 90adf06830 Merge branch 'main' of https://github.com/vrtmrz/obsidian-livesync 2024-02-15 04:04:06 +00:00
vorotamoroz cf8e7ff6ca bump 2024-02-15 04:04:01 +00:00
vorotamoroz 95c3ff5043 Update common lib. 2024-02-15 04:02:28 +00:00
vorotamoroz 7ea3515801 Fixed:
- Some description of settings have been refined
New feature:
- TroubleShooting is now shown in the setting dialogue.
2024-02-15 04:02:20 +00:00
vorotamoroz f866981a8a Update troubleshooting.md 2024-02-14 17:38:52 +09:00
vorotamoroz 8f36d6f893 fix minor layout 2024-02-09 03:20:31 +00:00
vorotamoroz 6dd86e9392 modify minor layouts. 2024-02-09 03:16:33 +00:00
vorotamoroz d22716bef0 make more clear documents. 2024-02-09 03:13:03 +00:00
vorotamoroz 5d9baec5e4 update colab note 2024-02-07 10:17:45 +00:00
vorotamoroz 27d71ca2fb New utilities. 2024-02-06 11:03:51 +00:00
vorotamoroz c024ed13d3 Refining the content. 2024-02-06 11:02:48 +00:00
vorotamoroz b9527ccab0 bump 2024-01-30 17:31:52 +00:00
vorotamoroz fa3aa2702c Fixed:
- Now the result of conflict resolution could be surely written into the storage.
- Deleted files can be handled correctly again in the history dialogue and conflict dialogue.
- Some wrong log messages were fixed.
- Change handling now has become more stable.
- Some event handling became to be safer.
Improved:
- Dumping document information shows conflicts and revisions.
- The timestamp-only differences can be surely cached.
- Timestamp difference detection can be rounded by two seconds.
Refactored:
- A bit of organisation to write the test.
2024-01-30 17:31:02 +00:00
vorotamoroz 93e7cbb133 bump. 2024-01-29 08:41:03 +00:00
vorotamoroz 716ae32e02 Fixed:
- Deletion of files is now reliably synchronised.
2024-01-29 08:40:41 +00:00
vorotamoroz d6d8cbcf5a bump 2024-01-29 07:57:02 +00:00
vorotamoroz efd348b266 Fixed:
- No longer detects storage changes which have been caused by Self-hosted LiveSync itself.
- Setting sync file will be detected only if it has been configured now.
  - And its log will be shown only while the verbose log is enabled.
- Customisation file enumeration has got less blingy.
Fixed and improved:
- In-editor-status is now shown in the following areas:
  - Note editing pane (Source mode and live-preview mode).
  - New tab pane.
  - Canvas pane.
2024-01-29 07:56:02 +00:00
vorotamoroz 8969b1800a bump 2024-01-24 08:53:00 +00:00
vorotamoroz 2c8e026e29 Fixed:
- Now the results of resolving conflicts are surely synchronised.
Modified:
- Some setting items got new clear names.
New feature:
- We can limit the synchronising files by their size.
- Now the settings could be stored in a specific markdown file to synchronise or switch it
- Customisation of the obsoleted device is now able to be deleted at once.
2024-01-24 08:52:47 +00:00
vorotamoroz a6c27eab3d Merge pull request #367 from calvinbui/patch-1
Skip workspace-mobile.json for cross-platform sync
2024-01-24 15:55:50 +09:00
vorotamoroz 9b5c57d540 Merge pull request #336 from toon159/patch-1
Lower payload size limit and batch limit from 10 to 2
2024-01-22 13:03:29 +09:00
Calvin Bui c251c596e8 Skip workspace-mobile.json for cross-platform sync 2024-01-18 13:23:48 +11:00
vorotamoroz 61188cfaef bump 2024-01-16 08:36:37 +00:00
vorotamoroz 97d944fd75 New feature:
- We can perform automatic conflict resolution for inactive files, and postpone only manual ones by `Postpone manual resolution of inactive files`.
- Now we can see the image in the document history dialogue.
  - We can see the difference of the image, in the document history dialogue.
	- And also we can highlight differences.

Improved:
- Hidden file sync has been stabilised.
- Now automatically reloads the conflict-resolution dialogue when new conflicted revisions have arrived.

Fixed:
- No longer periodic process runs after unloading the plug-in.
- Now the modification of binary files is surely stored in the storage.
2024-01-16 08:32:43 +00:00
vorotamoroz d3dc1e7328 Minor fix and refine the readme 2024-01-12 10:29:18 +00:00
vorotamoroz 45304af369 bump 2024-01-12 09:38:57 +00:00
vorotamoroz 7f422d58f2 - Refined:
- Task scheduling logics has been rewritten.
  - Possibly many bugs and fragile behaviour has been fixed
- Fixed:
  - Remote-chunk-fetching now works with keeping request intervals
- New feature:
  - We can show only the icons in the editor.
2024-01-12 09:36:49 +00:00
vorotamoroz c2491fdfad bump 2023-12-11 12:55:01 +09:00
vorotamoroz 06a6e391e8 Fixed for change detection bug. 2023-12-11 12:53:50 +09:00
vorotamoroz f99475f6b7 bump 2023-12-11 12:46:23 +09:00
vorotamoroz 109fc00b9d Fixed
- Now ID of the documents is shown in the log with the first 8 letters.
2023-12-11 12:45:40 +09:00
vorotamoroz c071d822e1 - Improved:
- Now all revisions will be shown only its first a few letters.
- Fixed:
  - Check before modifying files has been implemented.
  - Content change detection has been improved.
2023-12-11 12:22:17 +09:00
vorotamoroz d2de5b4710 bump 2023-12-04 19:39:47 +09:00
vorotamoroz cf5ecd8922 Implemented:
- Now we can use SHA1 for hash function as fallback.
2023-12-04 19:39:04 +09:00
vorotamoroz b337a05b5a bump 2023-11-27 07:13:15 +00:00
vorotamoroz 9ea6bee9d1 - Fixed:
- No longer files are broken while rebuilding.
    - Now, Large binary files can be written correctly on a mobile platform.
    - Any decoding errors now make zero-byte files.
  - Modified:
    - All files are processed sequentially for each.
2023-11-27 06:55:55 +00:00
vorotamoroz 9747c26d50 bump 2023-11-25 02:22:26 +09:00
vorotamoroz bb4b764586 - Fixed:
- No more infinity loops on larger files.
    - Show message on decode error.
  - Refactored:
    - Fixed to avoid obsolete global variables.
2023-11-25 02:21:44 +09:00
vorotamoroz 279b4b41e5 bump 2023-11-24 10:32:46 +00:00
vorotamoroz b644fb791d - Changes and performance improvements:
- Now the saving files are processed by Blob.
    - The V2-Format has been reverted.
    - New encoding format has been enabled in default.
    - WARNING: Since this version, the compatibilities with older Filesystem LiveSync have been lost.
2023-11-24 10:31:58 +00:00
Vichaya Raksakunpanich 5802ed31be Update ObsidianLiveSyncSettingTab.ts
Lower payload size limit and batch limit to 2 due to IBM Cloudant read/write limitation. Hope it will fix the "Replication error".
2023-11-24 16:25:16 +07:00
vorotamoroz ac9428e96b Fixed
- To better replication, path obfuscation is now deterministic even if with E2EE.
2023-11-15 08:44:03 +00:00
vorotamoroz 280d9e1dd9 Fixed: Fixed the issue of TOML editing. 2023-11-07 01:07:58 +00:00
vorotamoroz f7209e566c bump 2023-10-24 10:07:29 +01:00
vorotamoroz 4a9ab2d1de Fixed:
- No longer enumerating file names is broken.
2023-10-24 10:07:17 +01:00
vorotamoroz cb74b5ee93 - Fixed
- Now empty file could be decoded.
    - Local files are no longer pre-saved before fetching from a remote database.
    - No longer deadlock while applying customisation sync.
    - Configuration with multiple files is now able to be applied correctly.
    - Deleting folder propagation now works without enabling the use of a trash bin.
2023-10-24 09:54:56 +01:00
vorotamoroz 60eecd7001 bump 2023-10-17 12:00:59 +09:00
vorotamoroz 4bd7b54bcd Fixed:
- Now the files which having digit or character prefixes in the path will not be ignored.
2023-10-17 12:00:19 +09:00
vorotamoroz 8923c73d1b bump 2023-10-14 23:08:34 +09:00
vorotamoroz 11e64b13e2 The text-input-dialogue is no longer broken. 2023-10-14 23:07:51 +09:00
vorotamoroz 983d9248ed bump 2023-10-13 04:23:59 +01:00
vorotamoroz 7240e84328 - New feature:
- We can launch Customization sync from the Ribbon if we enable it.
- Fixed:
  - Setup URI is now back to the previous spec; be encrypted by V1.
  - The Settings dialogue is now registered at the beginning of the start-up process.
- Improved:
  - Enumerating documents has been faster.
2023-10-13 04:22:24 +01:00
vorotamoroz 0d55ae2532 Merge pull request #298 from LiamSwayne/patch-1
Grammar fix
2023-10-04 13:07:42 +09:00
Liam Swayne dbd284f5dd grammar fix 2023-10-03 22:26:30 -04:00
vorotamoroz c000a02f4a bump 2023-10-02 10:39:54 +01:00
vorotamoroz 79754f48d6 New feature:
- We can delete all data of customization sync from the `Delete all customization sync data` on the `Hatch` pane.
Fixed:
- Prevent keep restarting on iOS by yielding microtasks.
2023-10-02 10:38:54 +01:00
vorotamoroz dd7a40630b bump 2023-10-02 09:54:56 +01:00
vorotamoroz 14406f8213 - Fixed:
- No more UI freezing and keep restarting on iOS.
  - Diff of Non-markdown documents are now shown correctly.
- Improved:
  - Performance has been a bit improved.
  - Customization sync has gotten faster.
    - However, We lost forward compatibility again (only for this feature). Please update all devices.
- Misc
  - Terser configuration has been more aggressive.
2023-10-02 09:53:41 +01:00
vorotamoroz 3bbd9c048d bump 2023-09-29 18:57:54 +09:00
vorotamoroz d91c4f50b4 - Improved:
- A New binary file handling implemented
  - A new encrypted format has been implemented
  - Now the chunk sizes will be adjusted for efficient sync
- Fixed:
  - levels of exception in some logs have been fixed
- Tidied:
  - Some Lint warnings have been suppressed.
2023-09-29 18:55:46 +09:00
vorotamoroz 395b7fbc42 bump 2023-09-22 18:03:09 +09:00
vorotamoroz 3773e57429 Improved:
- We can open the log pane also from the command palette now.
- Now, the hidden file scanning interval could be configured to 0.
- `Check database configuration` now points out that we do not have administrator permission.
2023-09-22 17:59:32 +09:00
vorotamoroz 4835fce62a bump 2023-09-21 09:44:50 +01:00
vorotamoroz ff814be4a0 Fixed:
- Now the synchronisation will begin without our interaction.
- No longer puts the configuration of the remote database into the log while checking configuration.
- Some outdated description notes have been removed.
- Options that are meaningless depending on other settings configured are now hidden.
2023-09-21 09:44:07 +01:00
vorotamoroz b271b63efa bump 2023-09-20 07:05:18 +01:00
vorotamoroz 23419e476a - Fixed:
- Hidden files are no longer handled in the initial replication.
  - Report from `Making report` fixed
2023-09-20 07:04:31 +01:00
vorotamoroz b9bd1f17b8 bump 2023-09-19 09:58:04 +01:00
vorotamoroz bcce277c36 New feature:
- `Sync on Editor save` has been implemented
- Now we can use the `Hidden file sync` and the `Customization sync` cooperatively.
- We can ignore specific plugins in Customization sync.
- Now the message of leftover conflicted files accepts our click.

Refactored:
- Parallelism functions made more explicit.
- Type errors have been reduced.

Fixed:
- Now documents would not be overwritten if they are conflicted.
- Some error messages have been fixed.
- Missing dialogue titles have been shown now.
2023-09-19 09:53:48 +01:00
vorotamoroz 5acbbe479e Merge pull request #276 from cgcel/main
Add Quick setup Chinese translation.
2023-09-13 17:13:55 +09:00
vorotamoroz c9f9d511e0 bump 2023-09-05 09:16:49 +01:00
vorotamoroz b8cb94c498 Fixed:
- Resolving conflicted revision has become more robust.
- LiveSync now try to keep local changes when fetching from the rebuilt remote database.
- Now, all files will be restored after performing `fetch` immediately.
2023-09-05 09:16:11 +01:00
cgcel 52c736f6b9 update: format markdown titles. 2023-09-01 11:01:06 +08:00
cgcel ebd1cb7777 add: Quick setup Chinese translation. 2023-09-01 10:58:07 +08:00
vorotamoroz 10decb7909 Merge pull request #259 from jt-wang/main
Upload colab notebook into the repo, and update setup_flyio.md to reference it
2023-08-24 13:04:02 +09:00
vorotamoroz e0aab8d69d bump 2023-08-24 04:03:03 +01:00
vorotamoroz 618600c753 Fixed:
- Now the empty (or deleted) file could be conflict-resolved.
2023-08-24 04:00:56 +01:00
Jingtao Wang d1aba87e37 Updated with latest changes 2023-08-23 19:22:07 -07:00
Jingtao Wang db889f635e Merge pull request #1 from jt-wang/move-colab-script-into-repo
Upload deploy_couchdb_to_flyio_v2_with_swap.ipynb into the repo, and reference it from fly.io setup doc.
2023-08-15 14:55:37 -07:00
Jingtao Wang dd80e634f5 Update setup_flyio.md to reference colab notebook within the repo 2023-08-15 14:54:43 -07:00
Jingtao Wang bec6fc1a74 Upload deploy_couchdb_to_flyio_v2_with_swap.ipynb
Upload https://gist.github.com/vrtmrz/37c3efd7842e49947aaaa7f665e5020a into the repo for unified management.
2023-08-15 14:49:24 -07:00
vorotamoroz 5c96c7f99b bump 2023-08-08 10:45:50 +01:00
vorotamoroz 7b9724f713 Fixed:
- Now nested ignore files could be parsed correctly.
- The unexpected deletion of hidden files in some cases has been corrected.
- Hidden file change is no longer reflected on the device which has made the change itself.
2023-08-08 10:42:07 +01:00
vorotamoroz 4cd12c85ed update the changelog. 2023-08-04 10:03:51 +01:00
vorotamoroz 90651540f9 Update release.yml 2023-08-04 17:52:13 +09:00
vorotamoroz 9e504d5002 bump 2023-08-04 09:45:14 +01:00
vorotamoroz faaa94423c New feature:
- (Beta) ignore files handling

Fixed:
- Buttons on lock-detected-dialogue now can be shown in narrow-width devices.

Improved:
- Some constant has been flattened to be evaluated.
- The usage of the deprecated API of obsidian has been reduced.
- Now the indexedDB adapter will be enabled while the importing configuration.

Misc:
- Compiler, framework, and dependencies have been upgraded.
- Due to standing for these impacts (especially in esbuild and svelte,) terser has been introduced.
  Feel free to notify your opinion to me! I do not like to obfuscate the code too.
2023-08-04 09:45:04 +01:00
vorotamoroz a7c179fc86 Merge pull request #247 from cgcel/patch-1
Update setup_own_server_cn.md
2023-07-31 12:49:50 +09:00
vorotamoroz ed1a670b9b bump 2023-07-26 17:35:06 +09:00
vorotamoroz 6c3c265bd6 Dependency updates 2023-07-26 17:32:07 +09:00
vorotamoroz 9d68025f2a Fixed:
- Now storing files after cleaning up is correct works.

Improved:
- Cleaning the local database up got incredibly fastened.
2023-07-26 17:31:55 +09:00
GC Chen e70972c8f9 Update setup_own_server_cn.md 2023-07-25 22:31:17 +08:00
vorotamoroz 7607be7729 bump 2023-07-25 19:20:18 +09:00
vorotamoroz db9d428ab4 Fixed:
- Internal documents are now ignored.
- Merge dialogue now respond immediately to button pressing.
- Periodic processing now works fine
- The checking interval of detecting conflicted has got shorter
- Replication is now cancelled while cleaning up
- The database locking by the cleaning up is now carefully unlocked
- Missing chunks message is correctly reported

New feature:
- Suspend database reflecting has been implemented
- Now fetch suspends the reflecting database and storage changes temporarily to improve the performance.
- We can choose the action when the remote database has been cleaned
- Merge dialogue now show `↲` before the new line.

Improved:
- Now progress is reported while the cleaning up and fetch process
- Cancelled replication is now detected
2023-07-25 19:16:39 +09:00
vorotamoroz 0a2caea3c7 bump 2023-07-25 00:30:17 +09:00
vorotamoroz b1d1ba0e6b - Implemented:
- Database clean-up is now in beta 2!
    We can shrink the remote database by deleting unused chunks, with keeping history.
    Note: Local database is not cleaned up totally. We have to `Fetch` again to let it done.
    **Note2**: Still in beta. Please back your vault up anything before.
- Fixed:
  - The log updates are not thinned out now.
2023-07-25 00:29:47 +09:00
vorotamoroz 5e844372cb Merge pull request #236 from bioluks/documentation-rproxy
CouchDB config corrections, reverse proxy documentation, other additions
2023-07-20 10:33:37 +09:00
Meriç Aşkın 99c6911e96 Merge branch 'vrtmrz:main' into documentation-rproxy 2023-07-18 04:25:53 +02:00
vorotamoroz dc880d7d4e Merge pull request #244 from samyarkd/patch-1
fix: separate languages
2023-07-18 09:24:04 +09:00
Samyar c157fef76c fix: separate languages
when i first saw it i thought it's one connected sentence
so, i thought it would be nice to separate them using a dash.
2023-07-14 22:14:51 +03:30
bioluks 2b2011dc49 Added docker-compose, table of contents, a new reverse proxies section populated with traefik for now 2023-07-04 01:52:48 +02:00
bioluks ae451e005e Moved enable_cors to the right section. Added explanation for difference of versions. Added bind_address for making sure the container uses all interfaces given. Added spaces between 'origins' and removed spaces between the 'methods' elements because it's like this in the official Documentation. Added a write permission warning since many newbies had this mistake with couchdb. 2023-07-04 00:15:00 +02:00
vorotamoroz 8a75d41cbb bump 2023-07-03 18:45:05 +09:00
vorotamoroz 5252cc0372 Improved:
- Boot-up performance has been improved.
- Customisation sync performance has been improved.
- Synchronising performance has been improved.
2023-07-03 18:44:02 +09:00
vorotamoroz 5022155317 Fix documentation 2023-06-19 16:31:52 +09:00
vorotamoroz d36f925c65 bump 2023-06-15 18:13:43 +09:00
vorotamoroz 3ae33e0500 Refactored: External dependency merged. 2023-06-15 18:07:11 +09:00
vorotamoroz 13e442a0c7 Improvements:
- Hashing ChunkID has been improved.
- Logging keeps 400 lines now.

Refactored:
- Import statement has been fixed about types.
2023-06-15 17:55:58 +09:00
vorotamoroz 6288716966 - Fixed
- Fixed the issue about fixing the database.
2023-06-09 19:28:29 +09:00
vorotamoroz 47d2cf9733 bump 2023-06-09 18:48:59 +09:00
vorotamoroz ae6a9ecee4 - New feature (For fixing a problem):
- We can fix the database obfuscated and plain paths that have been mixed up.
- Improvements
  - Customisation Sync performance has been improved.
2023-06-09 18:48:10 +09:00
vorotamoroz 2289bea8d9 bump 2023-06-07 17:32:28 +09:00
vorotamoroz cda90259c5 - New feature:
- Vault history: A tab has been implemented to give a birds-eye view of the changes that have occurred in the vault.
- Improved:
  - Log dialogue is now shown as one of tabs.
- Fixed:
  - Some minor issues has been fixed.
2023-06-07 17:29:53 +09:00
vorotamoroz 432a211f80 Merge pull request #224 from antoKeinanen/main
[Feature] Add password protection to askString function
2023-06-07 17:04:31 +09:00
antoKeinanen eaf8c4998e feat: add password protection for required inputs 2023-06-05 13:27:06 +03:00
antoKeinanen 55601f7910 feat: add option for password protection in askString function 2023-06-05 13:24:50 +03:00
vorotamoroz 13e70475d9 Add new documentation
Thanks for your discussion!!
2023-06-02 14:36:00 +09:00
vorotamoroz 2572177879 Merge pull request #222 from Hugo-Persson/add-troubleshooting-guide
Added troubleshooting guide
2023-06-02 09:25:18 +09:00
Hugo Persson e82a2560e4 Added troubleshooting guide 2023-06-01 19:54:42 +02:00
597 changed files with 95507 additions and 15581 deletions
+34
View File
@@ -0,0 +1,34 @@
# Git history
.git/
.gitignore
# Dependencies — re-installed inside Docker
node_modules/
src/apps/cli/node_modules/
# Pre-built CLI output — rebuilt inside Docker
src/apps/cli/dist/
# Obsidian plugin build outputs
main.js
main_org.js
pouchdb-browser.js
production/
# Test coverage and reports
coverage/
_testdata/
test/bench-network/bench-results/
src/apps/cli/testdeno/bench-results/
# Local environment / secrets
.env
*.env
.test.env
# local config files
*.local
# OS artefacts
.DS_Store
Thumbs.db
-4
View File
@@ -1,4 +0,0 @@
node_modules
build
.eslintrc.js.bak
src/lib/src/patches/pouchdb-utils
+27 -4
View File
@@ -2,7 +2,9 @@
"root": true,
"parser": "@typescript-eslint/parser",
"plugins": [
"@typescript-eslint"
"@typescript-eslint",
"eslint-plugin-svelte",
"eslint-plugin-import"
],
"extends": [
"eslint:recommended",
@@ -15,6 +17,17 @@
"tsconfig.json"
]
},
"ignorePatterns": [
"**/node_modules/*",
"**/jest.config.js",
"**/test.ts",
"**/tests.ts",
"**/**test.ts",
"**/**.test.ts",
"src/apps/**",
"esbuild.*.mjs",
"terser.*.mjs"
],
"rules": {
"no-unused-vars": "off",
"@typescript-eslint/no-unused-vars": [
@@ -23,12 +36,22 @@
"args": "none"
}
],
"no-unused-labels": "off",
"@typescript-eslint/ban-ts-comment": "off",
"no-prototype-builtins": "off",
"@typescript-eslint/no-empty-function": "off",
"require-await": "warn",
"no-async-promise-executor": "off",
"@typescript-eslint/require-await": "warn",
"@typescript-eslint/no-misused-promises": "warn",
"@typescript-eslint/no-floating-promises": "warn",
"no-async-promise-executor": "warn",
"@typescript-eslint/no-explicit-any": "off",
"@typescript-eslint/no-unnecessary-type-assertion": "error"
"@typescript-eslint/no-unnecessary-type-assertion": "error",
"no-constant-condition": [
"error",
{
"checkLoops": false
}
]
}
}
}
+24
View File
@@ -0,0 +1,24 @@
# Always checkout shell scripts with LF line endings (never CRLF)
*.sh text eol=lf
# Standard text files — auto normalize on checkout
*.md text eol=lf
*.yml text eol=lf
*.yaml text eol=lf
*.ini text eol=lf
*.env text eol=lf
*.json text eol=lf
*.ts text eol=lf
*.js text eol=lf
*.mjs text eol=lf
*.css text eol=lf
# Binary files — no line ending conversion
*.png binary
*.jpg binary
*.jpeg binary
*.gif binary
*.ico binary
*.woff2 binary
*.woff binary
*.sh text eol=lf
+47 -41
View File
@@ -2,77 +2,83 @@
name: Issue report
about: Create a report to help us improve
title: ''
labels: ''
labels: 'uncategorised'
assignees: ''
---
Thank you for taking the time to report this issue!
To improve the process, I would like to ask you to let me know the information in advance.
Before filling in this form, please read: [How to report an issue](../docs/to_issue_reporting.md).
All instructions and examples, and empty entries can be deleted.
Just for your information, a [filled example](https://docs.vrtmrz.net/LiveSync/hintandtrivia/Issue+example) is also written.
Issues with sufficient information will be prioritised.
## Abstract
The synchronisation hung up immediately after connecting.
---
## Expected behaviour
- Synchronisation ends with the message `Replication completed`
- Everything synchronised
## Required
## Actually happened
- Synchronisation has been cancelled with the message `TypeError ... ` (captured in the attached log, around LL.10-LL.12)
- No files synchronised
### Abstract
<!-- Briefly describe the problem in one or two sentences. -->
## Reproducing procedure
### Expected behaviour
<!-- What did you expect to happen? -->
1. Configure LiveSync as in the attached material.
2. Click the replication button on the ribbon.
3. Synchronising has begun.
4. About two or three seconds later, we got the error `TypeError ... `.
5. Replication has been stopped. No files synchronised.
### Actually happened
<!-- What actually happened? Include any error messages. -->
Note: If you do not catch the reproducing procedure, please let me know the frequency and signs.
## Report materials
If the information is not available, do not hesitate to report it as it is. You can also of course omit it if you think this is indeed unnecessary. If it is necessary, I will ask you.
### Report from the LiveSync
For more information, please refer to [Making the report](https://docs.vrtmrz.net/LiveSync/hintandtrivia/Making+the+report).
<details>
<summary>Report from hatch</summary>
```
<!-- paste here -->
```
</details>
### Reproducing procedure
<!-- Step-by-step instructions to reproduce the issue. If you cannot reproduce it reliably, please describe the frequency and any signs you noticed. -->
### Obsidian debug info
Please provide debug info for **each device involved**. The primary device (where the issue occurred) is required; others are strongly recommended. If your issue involves synchronisation between devices, debug info from relevant devices is very helpful.
To get it: open the command palette → "Show debug info".
<details>
<summary>Debug info</summary>
<summary>Device 1 (primary)</summary>
```
<!-- paste here -->
```
</details>
### Plug-in log
We can see the log by tapping the Document box icon. If you noticed something suspicious, please let me know.
Note: **Please enable `Verbose Log`**. For detail, refer to [Logging](https://docs.vrtmrz.net/LiveSync/hintandtrivia/Logging), please.
<details>
<summary>Plug-in log</summary>
<summary>Device 2 (if applicable)</summary>
```
<!-- paste here -->
```
</details>
### Network log
Network logs displayed in DevTools will possibly help with connection-related issues. To capture that, please refer to [DevTools](https://docs.vrtmrz.net/LiveSync/hintandtrivia/DevTools).
### LiveSync version
The hatch report (below) includes version information. If you cannot provide the report, please fill in the version here.
- Self-hosted LiveSync version: <!-- e.g. 0.23.0 — find it in Obsidian Settings → Community Plugins -->
### Report and Logs from LiveSync
Perform a `Generate full report for opening the issue with debug info` command and provide the generated report. This contains detailed information and recent 1000 log lines, which is very helpful for debugging. **PLEASE AMEND THE REPORT TO REMOVE ANY SENSITIVE INFORMATION BEFORE PASTING.**
If too large to paste here, upload to [Gist](https://gist.github.com/) and share the link.
<details>
<summary>Report and Logs (primary)</summary>
```
<!-- paste here or link to Gist -->
```
</details>
<details>
<summary>Report and Logs (if applicable)</summary>
```
<!-- paste here or link to Gist -->
```
</details>
---
## Optional
### Screenshots
If applicable, please add screenshots to help explain your problem.
### Other information, insights and intuition.
### Other information, insights and intuition
Please provide any additional context or information about the problem.
+182
View File
@@ -0,0 +1,182 @@
name: cli-deno-tests
on:
push:
branches:
- main
- beta
paths:
- '.github/workflows/cli-deno-tests.yml'
- 'src/apps/cli/**'
- 'test/bench-network/**'
- 'package.json'
- 'package-lock.json'
pull_request:
paths:
- '.github/workflows/cli-deno-tests.yml'
- 'src/apps/cli/**'
- 'test/bench-network/**'
- 'package.json'
- 'package-lock.json'
workflow_dispatch:
inputs:
test_task:
description: 'Deno test task to run'
type: choice
options:
- test:ci
- test:local
- test:e2e-matrix
default: test:ci
enable_debug:
description: 'Enable verbose and debug logging'
type: boolean
default: false
use_coturn:
description: 'Enable local coturn container for P2P tests'
type: boolean
default: false
permissions:
contents: read
jobs:
prepare:
runs-on: ubuntu-latest
outputs:
task_matrix: ${{ steps.select.outputs.task_matrix }}
steps:
- name: Select task matrix
id: select
shell: bash
run: |
set -euo pipefail
SELECTED_TASK="${{ github.event_name == 'workflow_dispatch' && inputs.test_task || 'test:ci' }}"
echo "[INFO] Selected task set: $SELECTED_TASK"
case "$SELECTED_TASK" in
test:ci)
TASK_MATRIX='["test:setup-put-cat","test:mirror","test:daemon","test:push-pull","test:decoupled-vault","test:sync-two-local","test:sync-locked-remote","test:remote-commands","test:e2e-matrix:couchdb-enc0","test:e2e-matrix:couchdb-enc1","test:e2e-matrix:minio-enc0","test:e2e-matrix:minio-enc1"]'
;;
test:local)
TASK_MATRIX='["test:setup-put-cat","test:mirror","test:daemon"]'
;;
test:e2e-matrix)
TASK_MATRIX='["test:e2e-matrix:couchdb-enc0","test:e2e-matrix:couchdb-enc1","test:e2e-matrix:minio-enc0","test:e2e-matrix:minio-enc1"]'
;;
*)
echo "[ERROR] Unknown task set: $SELECTED_TASK" >&2
exit 1
;;
esac
echo "task_matrix=$TASK_MATRIX" >> "$GITHUB_OUTPUT"
test:
needs: prepare
runs-on: ubuntu-latest
timeout-minutes: 60
env:
DENO_DIR: ~/.cache/deno
strategy:
fail-fast: false
matrix:
task: ${{ fromJson(needs.prepare.outputs.task_matrix) }}
steps:
- name: Checkout
uses: actions/checkout@v4
- name: Setup Node.js
uses: actions/setup-node@v4
with:
node-version: '24.x'
cache: 'npm'
cache-dependency-path: package-lock.json
- name: Setup Deno
uses: denoland/setup-deno@v2
with:
deno-version: v2.x
- name: Cache Deno dependencies
uses: actions/cache@v4
with:
path: ~/.cache/deno
key: ${{ runner.os }}-deno-${{ hashFiles('src/apps/cli/testdeno/deno.lock', 'src/apps/cli/testdeno/deno.json') }}
restore-keys: |
${{ runner.os }}-deno-
- name: Install dependencies
run: npm ci
- name: Build CLI
working-directory: src/apps/cli
run: npm run build
- name: Create .test.env
working-directory: src/apps/cli
run: |
cat <<EOF > .test.env
hostname=http://127.0.0.1:5989/
dbname=livesync-test-db-ci
username=admin
password=testpassword
minioEndpoint=http://127.0.0.1:9000
accessKey=minioadmin
secretKey=minioadmin
bucketName=livesync-test-bucket-ci
EOF
- name: Run Deno tests
working-directory: src/apps/cli/testdeno
env:
LIVESYNC_DOCKER_MODE: native
LIVESYNC_CLI_RETRY: 3
LIVESYNC_CLI_DEBUG: ${{ inputs.enable_debug == true && '1' || '0' }}
LIVESYNC_CLI_VERBOSE: ${{ inputs.enable_debug == true && '1' || '0' }}
LIVESYNC_USE_COTURN: ${{ inputs.use_coturn == true && '1' || '0' }}
run: |
TASK="${{ matrix.task }}"
echo "[INFO] Running Deno task: $TASK"
deno task "$TASK"
- name: Stop leftover containers
if: always()
run: |
docker stop couchdb-test minio-test relay-test coturn-test >/dev/null 2>&1 || true
docker rm couchdb-test minio-test relay-test coturn-test >/dev/null 2>&1 || true
compose-p2p-e2e:
runs-on: ubuntu-latest
timeout-minutes: 45
steps:
- name: Checkout
uses: actions/checkout@v4
- name: Show Docker versions
run: |
docker --version
docker compose version
- name: Run Compose CLI P2P E2E
env:
CLI_E2E_TASK: test:p2p-sync
RELAY: ws://nostr-relay:7777/
PEERS_TIMEOUT: '20'
SYNC_TIMEOUT: '60'
LIVESYNC_USE_COTURN: '0'
TURN_SERVERS: none
LIVESYNC_P2P_PEERS_RETRY: '1'
LIVESYNC_P2P_RELAY_READY_TIMEOUT_MS: '60000'
BENCH_LIVESYNC_TEST_TEE: '0'
run: docker compose -f test/bench-network/compose.yml run --build --rm bench-runner run-livesync-cli-e2e
- name: Show Compose diagnostics
if: failure()
run: |
docker compose -f test/bench-network/compose.yml ps
docker compose -f test/bench-network/compose.yml logs --no-color couchdb nostr-relay || true
- name: Stop Compose services
if: always()
run: docker compose -f test/bench-network/compose.yml down -v --remove-orphans
+134
View File
@@ -0,0 +1,134 @@
# Build and push the CLI Docker image to GitHub Container Registry (GHCR).#
# Image tag format: <manifest-version>-<unix-epoch>-cli
# Example: 0.25.56-1743500000-cli
#
# The image is also tagged 'latest' for convenience.
# Image name: ghcr.io/<owner>/livesync-cli
name: Build and Push CLI Docker Image
on:
push:
branches:
- main
tags:
- "*.*.*-cli"
paths-ignore:
- "docs/**"
- "*.md"
- "images/**"
- "assets/**"
- "instruction_images/**"
- "src/apps/webapp/**"
- "src/apps/webpeer/**"
- ".github/workflows/release.yml"
- ".github/workflows/unit-ci.yml"
- ".github/workflows/harness-ci.yml"
workflow_dispatch:
inputs:
dry_run:
description: Build only (do not push image to GHCR)
required: false
type: boolean
default: true
force:
description: Continue to build/push even if CLI E2E fails (workflow_dispatch only)
required: false
type: boolean
default: false
jobs:
build-and-push:
runs-on: ubuntu-latest
timeout-minutes: 90
permissions:
contents: read
packages: write
steps:
- name: Checkout
uses: actions/checkout@v4
- name: Derive image tag
id: meta
run: |
VERSION=$(jq -r '.version' manifest.json)
MAJOR_MINOR=$(echo "${VERSION}" | cut -d. -f1,2)
SHORT_SHA=$(git rev-parse --short HEAD)
IMAGE="ghcr.io/${{ github.repository_owner }}/livesync-cli"
# Build tag list based on the event and git ref
TAGS=""
if [[ "${{ github.ref }}" == refs/tags/* ]]; then
# Stable release builds
TAGS="${IMAGE}:${VERSION}-cli,${IMAGE}:${MAJOR_MINOR}-cli,${IMAGE}:latest,${IMAGE}:${VERSION}-sha-${SHORT_SHA}-cli"
elif [[ "${{ github.ref }}" == refs/heads/main ]]; then
# Bleeding-edge / nightly builds
TAGS="${IMAGE}:edge"
else
# Other branches / manual run fallback
TAGS="${IMAGE}:${VERSION}-dev-sha-${SHORT_SHA}-cli"
fi
# Determine if the image should be pushed
PUSH="true"
if [[ "${{ github.event_name }}" == "workflow_dispatch" ]]; then
if [[ "${{ inputs.dry_run }}" == "true" ]]; then
PUSH="false"
fi
fi
echo "tags=${TAGS}" >> $GITHUB_OUTPUT
echo "push=${PUSH}" >> $GITHUB_OUTPUT
- name: Log in to GitHub Container Registry
uses: docker/login-action@v3
with:
registry: ghcr.io
username: ${{ github.actor }}
password: ${{ secrets.GITHUB_TOKEN }}
- name: Set up QEMU
uses: docker/setup-qemu-action@v3
with:
platforms: arm64
- name: Set up Docker Buildx
uses: docker/setup-buildx-action@v3
- name: Setup Node.js
uses: actions/setup-node@v4
with:
node-version: "24.x"
cache: "npm"
- name: Install dependencies
run: npm ci
- name: Run CLI E2E (docker)
id: e2e
continue-on-error: ${{ github.event_name == 'workflow_dispatch' && inputs.force }}
working-directory: src/apps/cli
env:
CI: true
run: npm run test:e2e:docker:all
- name: Stop test containers (safety net)
if: always()
working-directory: src/apps/cli
run: |
# Keep this as a safety net for future suites/steps that may leave containers running.
bash ./util/couchdb-stop.sh >/dev/null 2>&1 || true
bash ./util/minio-stop.sh >/dev/null 2>&1 || true
bash ./util/p2p-stop.sh >/dev/null 2>&1 || true
- name: Build and push
if: ${{ steps.e2e.outcome == 'success' || (github.event_name == 'workflow_dispatch' && inputs.force) }}
uses: docker/build-push-action@v6
with:
context: .
file: src/apps/cli/Dockerfile
push: ${{ steps.meta.outputs.push }}
tags: ${{ steps.meta.outputs.tags }}
platforms: linux/amd64,linux/arm64
cache-from: type=gha
cache-to: type=gha,mode=max
+65
View File
@@ -0,0 +1,65 @@
# Run CLI E2E tests
name: cli-e2e
on:
workflow_dispatch:
inputs:
suite:
description: 'CLI E2E suite to run'
type: choice
options:
- two-vaults-matrix
- two-vaults-couchdb
- two-vaults-minio
default: two-vaults-matrix
permissions:
contents: read
jobs:
test:
runs-on: ubuntu-latest
timeout-minutes: 45
steps:
- name: Checkout
uses: actions/checkout@v4
- name: Setup Node.js
uses: actions/setup-node@v4
with:
node-version: '24.x'
cache: 'npm'
- name: Install dependencies
run: npm ci
- name: Run CLI E2E suite
working-directory: src/apps/cli
env:
CI: true
TEST_SUITE: ${{ github.event_name == 'workflow_dispatch' && inputs.suite || 'two-vaults-matrix' }}
run: |
set -euo pipefail
echo "[INFO] Running CLI E2E suite: $TEST_SUITE"
case "$TEST_SUITE" in
two-vaults-matrix)
npm run test:e2e:two-vaults:matrix
;;
two-vaults-couchdb)
REMOTE_TYPE=COUCHDB ENCRYPT=0 npm run test:e2e:two-vaults
;;
two-vaults-minio)
REMOTE_TYPE=MINIO ENCRYPT=0 npm run test:e2e:two-vaults
;;
*)
echo "[ERROR] Unknown suite: $TEST_SUITE" >&2
exit 1
;;
esac
- name: Stop test containers
if: always()
working-directory: src/apps/cli
run: |
bash ./util/couchdb-stop.sh >/dev/null 2>&1 || true
bash ./util/minio-stop.sh >/dev/null 2>&1 || true
@@ -0,0 +1,94 @@
# Run the Compose-packaged CLI P2P smoke benchmark.
#
# This workflow is intentionally manual-only. It exercises the local Compose
# package for CouchDB + Nostr relay + CLI runner, and uploads the benchmark JSON
# results for inspection without adding benchmark work to pull-request CI.
name: cli-p2p-compose-smoke
on:
workflow_dispatch:
inputs:
cases:
description: 'Comma-separated benchmark cases'
required: false
default: 'couchdb-baseline,p2p-direct-local'
signalling_cases:
description: 'Comma-separated signalling-shim P2P benchmark cases'
required: false
default: 'p2p-signalling-netem-home-wifi'
md_files:
description: 'Markdown file count'
required: false
default: '2'
bin_files:
description: 'Binary file count'
required: false
default: '1'
couchdb_rtt_ms:
description: 'Requested CouchDB RTT in milliseconds'
required: false
default: '20'
permissions:
contents: read
jobs:
smoke:
runs-on: ubuntu-latest
timeout-minutes: 45
steps:
- name: Checkout
uses: actions/checkout@v4
- name: Show Docker versions
run: |
docker --version
docker compose version
- name: Run Compose P2P smoke benchmark
env:
BENCH_CASES: ${{ inputs.cases || 'couchdb-baseline,p2p-direct-local' }}
BENCH_MD_FILE_COUNT: ${{ inputs.md_files || '2' }}
BENCH_MD_MIN_SIZE_BYTES: '128'
BENCH_MD_MAX_SIZE_BYTES: '256'
BENCH_BIN_FILE_COUNT: ${{ inputs.bin_files || '1' }}
BENCH_BIN_SIZE_BYTES: '512'
BENCH_COUCHDB_RTT_MS: ${{ inputs.couchdb_rtt_ms || '20' }}
BENCH_SYNC_TIMEOUT: '180'
BENCH_PEERS_TIMEOUT: '20'
LIVESYNC_P2P_RELAY_READY_TIMEOUT_MS: '60000'
BENCH_LIVESYNC_TEST_TEE: '0'
run: docker compose -f test/bench-network/compose.yml run --build --rm bench-runner
- name: Run Compose P2P signalling-shim smoke benchmark
env:
BENCH_CASES: ${{ inputs.signalling_cases || 'p2p-signalling-netem-home-wifi' }}
BENCH_MD_FILE_COUNT: ${{ inputs.md_files || '2' }}
BENCH_MD_MIN_SIZE_BYTES: '128'
BENCH_MD_MAX_SIZE_BYTES: '256'
BENCH_BIN_FILE_COUNT: ${{ inputs.bin_files || '1' }}
BENCH_BIN_SIZE_BYTES: '512'
BENCH_SYNC_TIMEOUT: '180'
BENCH_PEERS_TIMEOUT: '60'
LIVESYNC_P2P_RELAY_READY_TIMEOUT_MS: '60000'
BENCH_LIVESYNC_TEST_TEE: '0'
NETEM_PROFILE: 'home-wifi'
run: docker compose -f test/bench-network/compose.yml --profile signalling-shim run --build --rm bench-runner-signalling-shim
- name: Show Compose diagnostics
if: failure()
run: |
docker compose -f test/bench-network/compose.yml ps
docker compose -f test/bench-network/compose.yml --profile signalling-shim logs --no-color couchdb nostr-relay p2p-signalling-shim || true
- name: Upload benchmark results
if: always()
uses: actions/upload-artifact@v4
with:
name: cli-p2p-compose-smoke-results
path: test/bench-network/bench-results/**
if-no-files-found: warn
- name: Stop Compose services
if: always()
run: docker compose -f test/bench-network/compose.yml down -v --remove-orphans
+67
View File
@@ -0,0 +1,67 @@
name: Deploy GitHub Pages
on:
workflow_dispatch:
push:
branches:
- main
paths:
- 'aggregator.html'
- '.github/workflows/deploy-pages.yml'
pull_request:
paths:
- 'aggregator.html'
- '.github/workflows/deploy-pages.yml'
permissions:
contents: read
pages: write
id-token: write
concurrency:
group: pages
cancel-in-progress: false
jobs:
build:
name: Validate and package Pages site
runs-on: ubuntu-latest
steps:
- name: Checkout
uses: actions/checkout@v4
- name: Validate aggregator
run: |
test -s aggregator.html
grep -Fq '<!DOCTYPE html>' aggregator.html
grep -Fq 'obsidian://setuplivesync?settingsQR=' aggregator.html
sed -n '/<script>/,/<\/script>/p' aggregator.html | sed '1d;$d' > aggregator.js
node --check aggregator.js
rm aggregator.js
- name: Prepare Pages site
run: |
mkdir -p _site
cp aggregator.html _site/aggregator.html
touch _site/.nojekyll
- name: Configure GitHub Pages
uses: actions/configure-pages@v5
- name: Upload GitHub Pages artifact
uses: actions/upload-pages-artifact@v3
with:
path: _site
deploy:
name: Deploy GitHub Pages
if: github.event_name != 'pull_request'
needs: build
runs-on: ubuntu-latest
environment:
name: github-pages
url: ${{ steps.deployment.outputs.page_url }}
steps:
- name: Deploy GitHub Pages
id: deployment
uses: actions/deploy-pages@v4
+100
View File
@@ -0,0 +1,100 @@
name: Finalise Release Tags
on:
workflow_dispatch:
inputs:
version:
description: Release version, for example 0.25.81
required: true
type: string
release_branch:
description: Release PR branch. Defaults to the version with dots replaced by underscores.
required: false
type: string
expected_head_sha:
description: Full head commit SHA reviewed in the release PR
required: true
type: string
jobs:
finalise:
runs-on: ubuntu-latest
environment: release
permissions:
actions: write
contents: write
steps:
- name: Resolve release branch
id: branch
env:
VERSION: ${{ inputs.version }}
RELEASE_BRANCH_INPUT: ${{ inputs.release_branch }}
run: |
set -euo pipefail
BRANCH="${RELEASE_BRANCH_INPUT}"
if [[ -z "${BRANCH}" ]]; then
BRANCH="${VERSION//./_}"
fi
echo "name=${BRANCH}" >> "$GITHUB_OUTPUT"
- uses: actions/checkout@v4
with:
ref: ${{ steps.branch.outputs.name }}
fetch-depth: 0
- name: Use Node.js
uses: actions/setup-node@v4
with:
node-version: "24.x"
- name: Validate release head
env:
VERSION: ${{ inputs.version }}
EXPECTED_HEAD_SHA: ${{ inputs.expected_head_sha }}
run: |
set -euo pipefail
ACTUAL_HEAD_SHA="$(git rev-parse HEAD)"
if [[ "${ACTUAL_HEAD_SHA}" != "${EXPECTED_HEAD_SHA}" ]]; then
echo "Release branch head is ${ACTUAL_HEAD_SHA}, expected ${EXPECTED_HEAD_SHA}." >&2
exit 1
fi
node utils/release-notes.mjs validate "${VERSION}"
- name: Ensure and push release tags
env:
VERSION: ${{ inputs.version }}
EXPECTED_HEAD_SHA: ${{ inputs.expected_head_sha }}
run: |
set -euo pipefail
git fetch --tags --force
node utils/release-tags.mjs ensure "${VERSION}" "${EXPECTED_HEAD_SHA}"
git push --atomic origin "refs/tags/${VERSION}" "refs/tags/${VERSION}-cli"
- name: Dispatch release workflows
env:
GH_TOKEN: ${{ github.token }}
VERSION: ${{ inputs.version }}
run: |
set -euo pipefail
gh workflow run release.yml \
--ref "${VERSION}" \
--field tag="${VERSION}" \
--field draft=true \
--field prerelease=false
gh workflow run cli-docker.yml \
--ref "${VERSION}-cli" \
--field dry_run=false \
--field force=false
- name: Summarise next steps
env:
VERSION: ${{ inputs.version }}
run: |
{
echo "Ensured tags \`${VERSION}\` and \`${VERSION}-cli\` point to the reviewed release commit."
echo ""
echo "Dispatched the plug-in release workflow for \`${VERSION}\`. After approval for the release environment, it creates a draft GitHub Release."
echo "Dispatched the CLI Docker workflow for \`${VERSION}-cli\`. It publishes the version, major-minor, latest, and SHA-qualified image tags."
echo ""
echo "Keep the release pull request in draft after publishing the GitHub Release as the latest stable release. Mark it ready and merge it only after BRAT validation succeeds."
} >> "$GITHUB_STEP_SUMMARY"
+66
View File
@@ -0,0 +1,66 @@
# Run tests by Harnessed CI
name: harness-ci
on:
workflow_dispatch:
inputs:
testsuite:
description: 'Run specific test suite (leave empty to run all)'
type: choice
options:
- ''
- 'suite/'
- 'suitep2p/'
default: ''
permissions:
contents: read
jobs:
test:
runs-on: ubuntu-latest
timeout-minutes: 30
steps:
- name: Checkout
uses: actions/checkout@v4
- name: Setup Node.js
uses: actions/setup-node@v4
with:
node-version: '24.x'
cache: 'npm'
- name: Install dependencies
run: npm ci
- name: Install test dependencies (Playwright Chromium)
run: npm run test:install-dependencies
- name: Start test services (CouchDB)
run: npm run test:docker-couchdb:start
if: ${{ inputs.testsuite == '' || inputs.testsuite == 'suite/' }}
- name: Start test services (MinIO)
run: npm run test:docker-s3:start
if: ${{ inputs.testsuite == '' || inputs.testsuite == 'suite/' }}
- name: Start test services (Nostr Relay + WebPeer)
run: npm run test:docker-p2p:start
if: ${{ inputs.testsuite == '' || inputs.testsuite == 'suitep2p/' }}
- name: Run tests suite
if: ${{ inputs.testsuite == '' || inputs.testsuite == 'suite/' }}
env:
CI: true
run: npm run test suite/
- name: Run P2P tests suite
if: ${{ inputs.testsuite == '' || inputs.testsuite == 'suitep2p/' }}
env:
CI: true
run: npm run test:p2p
- name: Stop test services (CouchDB)
run: npm run test:docker-couchdb:stop
if: ${{ inputs.testsuite == '' || inputs.testsuite == 'suite/' }}
- name: Stop test services (MinIO)
run: npm run test:docker-s3:stop
if: ${{ inputs.testsuite == '' || inputs.testsuite == 'suite/' }}
- name: Stop test services (Nostr Relay + WebPeer)
run: npm run test:docker-p2p:stop
if: ${{ inputs.testsuite == '' || inputs.testsuite == 'suitep2p/' }}
+114
View File
@@ -0,0 +1,114 @@
name: Prepare Release PR
on:
workflow_dispatch:
inputs:
version:
description: Release version, for example 0.25.81
required: true
type: string
base_branch:
description: Base branch for the release PR
required: false
type: string
default: main
release_branch:
description: Release branch name. Defaults to the version with dots replaced by underscores.
required: false
type: string
release_date:
description: Release date in ordinal format. Defaults to the current UTC date.
required: false
type: string
allow_empty_updates:
description: Allow an empty Unreleased section
required: false
type: boolean
default: false
jobs:
prepare:
runs-on: ubuntu-latest
permissions:
contents: write
pull-requests: write
steps:
- uses: actions/checkout@v4
with:
ref: ${{ inputs.base_branch }}
fetch-depth: 0
- name: Use Node.js
uses: actions/setup-node@v4
with:
node-version: "24.x"
cache: npm
- name: Install dependencies
run: npm ci
- name: Prepare release changes
id: prepare
env:
VERSION: ${{ inputs.version }}
RELEASE_BRANCH_INPUT: ${{ inputs.release_branch }}
RELEASE_DATE: ${{ inputs.release_date }}
ALLOW_EMPTY_UPDATES: ${{ inputs.allow_empty_updates }}
run: |
set -euo pipefail
git config user.name "github-actions[bot]"
git config user.email "41898282+github-actions[bot]@users.noreply.github.com"
BRANCH="${RELEASE_BRANCH_INPUT}"
if [[ -z "${BRANCH}" ]]; then
BRANCH="${VERSION//./_}"
fi
if git ls-remote --exit-code --heads origin "${BRANCH}" >/dev/null 2>&1; then
echo "Release branch already exists: ${BRANCH}" >&2
exit 1
fi
git switch -c "${BRANCH}"
npm version "${VERSION}" --no-git-tag-version
node utils/release-notes.mjs prepare "${VERSION}"
git add package.json package-lock.json manifest.json versions.json updates.md src/apps/cli/package.json src/apps/webpeer/package.json src/apps/webapp/package.json
git diff --cached --check
git commit -m "Releasing ${VERSION}"
git push --set-upstream origin "${BRANCH}"
echo "branch=${BRANCH}" >> "$GITHUB_OUTPUT"
- name: Create draft release PR
env:
GH_TOKEN: ${{ github.token }}
VERSION: ${{ inputs.version }}
BASE_BRANCH: ${{ inputs.base_branch }}
RELEASE_BRANCH: ${{ steps.prepare.outputs.branch }}
run: |
cat > /tmp/release-pr-body.md <<EOF
> [!IMPORTANT]
> **Merge intentionally on hold**
>
> Publishing the GitHub Release does not unblock this pull request. Keep this pull request in draft, and leave \`main\` on the previous release, until the published build has passed BRAT validation.
## Release checklist
- [ ] Review and polish \`updates.md\`
- [ ] Confirm the release date
- [ ] Confirm \`manifest.json\`, \`versions.json\`, workspace package versions, and the locked Commonlib package version
- [ ] Confirm CI has passed
- [ ] Run the finalise release workflow with this PR's fixed head SHA
- [ ] Confirm the draft GitHub Release assets and the published CLI image
- [ ] Publish the GitHub Release as the latest stable release while keeping this pull request in draft
- [ ] Validate the published release with BRAT
- [ ] Mark this pull request ready and merge it with a merge commit
EOF
gh pr create \
--base "${BASE_BRANCH}" \
--head "${RELEASE_BRANCH}" \
--draft \
--title "Releasing ${VERSION}" \
--body-file /tmp/release-pr-body.md
+55 -69
View File
@@ -1,94 +1,80 @@
name: Release Obsidian Plugin
on:
push:
# Sequence of patterns matched against refs/tags
tags:
- '*' # Push events to matching any tag format, i.e. 1.0, 20.15.10
workflow_dispatch:
inputs:
tag:
description: Release tag to build
required: true
type: string
draft:
description: Create the GitHub Release as a draft
required: false
type: boolean
default: true
prerelease:
description: Mark the GitHub Release as a pre-release
required: false
type: boolean
default: false
jobs:
build:
runs-on: ubuntu-latest
environment: release
permissions:
contents: write
id-token: write
attestations: write
steps:
- uses: actions/checkout@v2
- uses: actions/checkout@v4
with:
fetch-depth: 0 # otherwise, you will failed to push refs to dest repo
submodules: recursive
fetch-depth: 0
ref: ${{ inputs.tag }}
- name: Use Node.js
uses: actions/setup-node@v1
uses: actions/setup-node@v4
with:
node-version: '14.x' # You might need to adjust this value to your own version
# Get the version number and put it in a variable
- name: Get Version
id: version
node-version: '24.x'
- name: Validate release
env:
TAG: ${{ inputs.tag }}
run: |
echo "::set-output name=tag::$(git describe --abbrev=0 --tags)"
set -euo pipefail
node utils/release-notes.mjs validate "${TAG}"
HEAD_SHA="$(git rev-parse HEAD)"
TAG_SHA="$(git rev-parse "refs/tags/${TAG}^{commit}")"
if [[ "${HEAD_SHA}" != "${TAG_SHA}" ]]; then
echo "Checked-out commit is ${HEAD_SHA}, but tag ${TAG} points to ${TAG_SHA}." >&2
exit 1
fi
# Build the plugin
- name: Build
id: build
run: |
npm ci
npm run build --if-present
# Attest
- name: Attest Plugin Artifacts
uses: actions/attest-build-provenance@v4
with:
subject-path: |
main.js
manifest.json
styles.css
# Package the required files into a zip
- name: Package
run: |
mkdir ${{ github.event.repository.name }}
cp main.js manifest.json styles.css README.md ${{ github.event.repository.name }}
zip -r ${{ github.event.repository.name }}.zip ${{ github.event.repository.name }}
# Create the release on github
- name: Create Release
id: create_release
uses: actions/create-release@v1
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
VERSION: ${{ github.ref }}
- name: Create Release and Upload Assets
uses: softprops/action-gh-release@v2
with:
tag_name: ${{ github.ref }}
release_name: ${{ github.ref }}
draft: true
prerelease: false
# Upload the packaged release file
- name: Upload zip file
id: upload-zip
uses: actions/upload-release-asset@v1
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
with:
upload_url: ${{ steps.create_release.outputs.upload_url }}
asset_path: ./${{ github.event.repository.name }}.zip
asset_name: ${{ github.event.repository.name }}-${{ steps.version.outputs.tag }}.zip
asset_content_type: application/zip
# Upload the main.js
- name: Upload main.js
id: upload-main
uses: actions/upload-release-asset@v1
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
with:
upload_url: ${{ steps.create_release.outputs.upload_url }}
asset_path: ./main.js
asset_name: main.js
asset_content_type: text/javascript
# Upload the manifest.json
- name: Upload manifest.json
id: upload-manifest
uses: actions/upload-release-asset@v1
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
with:
upload_url: ${{ steps.create_release.outputs.upload_url }}
asset_path: ./manifest.json
asset_name: manifest.json
asset_content_type: application/json
# Upload the style.css
- name: Upload styles.css
id: upload-css
uses: actions/upload-release-asset@v1
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
with:
upload_url: ${{ steps.create_release.outputs.upload_url }}
asset_path: ./styles.css
asset_name: styles.css
asset_content_type: text/css
# TODO: release notes???
files: |
${{ github.event.repository.name }}.zip
main.js
manifest.json
styles.css
name: ${{ inputs.tag }}
tag_name: ${{ inputs.tag }}
draft: ${{ inputs.draft }}
prerelease: ${{ inputs.prerelease }}
+154
View File
@@ -0,0 +1,154 @@
# Run Unit test without Harnesses
name: unit-ci
on:
workflow_dispatch:
push:
branches:
- main
- beta
paths:
- 'src/**'
- 'test/**'
- 'package.json'
- 'package-lock.json'
- 'tsconfig.json'
- 'vite.config.ts'
- 'vitest.config*.ts'
- 'esbuild.config.mjs'
- 'eslint.config.mjs'
- 'eslint.config.common.mjs'
- 'eslint.community.config.mjs'
- 'update-workspaces.mjs'
- 'version-bump.mjs'
- 'utils/release-*.mjs'
- 'utils/release-*.unit.spec.ts'
- '.github/workflows/prepare-release.yml'
- '.github/workflows/finalise-release.yml'
- '.github/workflows/release.yml'
- '.github/workflows/unit-ci.yml'
pull_request:
paths:
- 'src/**'
- 'test/**'
- 'package.json'
- 'package-lock.json'
- 'tsconfig.json'
- 'vite.config.ts'
- 'vitest.config*.ts'
- 'esbuild.config.mjs'
- 'eslint.config.mjs'
- 'eslint.config.common.mjs'
- 'eslint.community.config.mjs'
- 'update-workspaces.mjs'
- 'version-bump.mjs'
- 'utils/release-*.mjs'
- 'utils/release-*.unit.spec.ts'
- '.github/workflows/prepare-release.yml'
- '.github/workflows/finalise-release.yml'
- '.github/workflows/release.yml'
- '.github/workflows/unit-ci.yml'
permissions:
contents: read
jobs:
unit-test:
name: Unit Tests
runs-on: ubuntu-latest
timeout-minutes: 30
steps:
- name: Checkout
uses: actions/checkout@v4
- name: Setup Node.js
uses: actions/setup-node@v4
with:
node-version: '24.x'
cache: 'npm'
- name: Install dependencies
run: npm ci
- name: Run source checks
run: npm run check
- name: Run unit tests suite with coverage
run: npm run test:unit:coverage
- name: Upload coverage report
if: always()
uses: actions/upload-artifact@v4
with:
name: unit-coverage-report
path: coverage/**
integration-test:
name: Integration Tests
runs-on: ubuntu-latest
timeout-minutes: 30
steps:
- name: Checkout
uses: actions/checkout@v4
- name: Detect LiveSync-owned integration tests
id: integration_tests
shell: bash
run: |
git ls-files -- ':(glob)**/*.integration.spec.ts' ':(glob)**/*.integration.test.ts' > "$RUNNER_TEMP/livesync-integration-tests.txt"
if [[ -s "$RUNNER_TEMP/livesync-integration-tests.txt" ]]; then
echo 'present=true' >> "$GITHUB_OUTPUT"
else
echo 'present=false' >> "$GITHUB_OUTPUT"
fi
- name: Record delegated integration coverage
if: ${{ steps.integration_tests.outputs.present != 'true' }}
run: echo 'No LiveSync-owned integration tests are present. Commonlib integration tests run in the Commonlib package CI.' >> "$GITHUB_STEP_SUMMARY"
- name: Setup Node.js
if: ${{ steps.integration_tests.outputs.present == 'true' }}
uses: actions/setup-node@v4
with:
node-version: '24.x'
cache: 'npm'
- name: Install dependencies
if: ${{ steps.integration_tests.outputs.present == 'true' }}
run: npm ci
- name: Create environment configuration files
if: ${{ steps.integration_tests.outputs.present == 'true' }}
run: |
cat <<EOF > .env
BUILD_MODE=dev
PATHS_TEST_INSTALL=
EOF
cat <<EOF > .test.env
hostname=http://127.0.0.1:5989/
dbname=livesync-test-db2
username=admin
password=testpassword
minioEndpoint=http://127.0.0.1:9000
accessKey=minioadmin
secretKey=minioadmin
bucketName=livesync-test-bucket
EOF
- name: Start CouchDB container
if: ${{ steps.integration_tests.outputs.present == 'true' }}
run: npm run test:docker-couchdb:start
- name: Start MinIO container
if: ${{ steps.integration_tests.outputs.present == 'true' }}
run: npm run test:docker-s3:start
- name: Run integration tests
if: ${{ steps.integration_tests.outputs.present == 'true' }}
run: npm run test:integration
- name: Stop containers
if: ${{ always() && steps.integration_tests.outputs.present == 'true' }}
run: |
npm run test:docker-couchdb:stop || true
npm run test:docker-s3:stop || true
+21
View File
@@ -8,8 +8,29 @@ package-lock.json
# build
main.js
main_org.js
main_org_*.js
*.js.map
meta.json
meta-*.json
# obsidian
data.json
.vscode
# environment variables
.env
# local config files
*.local
cov_profile/**
coverage
src/apps/cli/dist/*
src/apps/webapp/playwright-report/
src/apps/webapp/test-results/
_testdata/**
utils/bench/splitResults.csv
.eslintcache
-3
View File
@@ -1,3 +0,0 @@
[submodule "src/lib"]
path = src/lib
url = https://github.com/vrtmrz/livesync-commonlib
+4
View File
@@ -0,0 +1,4 @@
pouchdb-browser.js
main_org.js
main.js
_types/**
+20
View File
@@ -0,0 +1,20 @@
import { readFileSync } from "fs";
let localPrettierConfig = {};
try {
const localConfig = readFileSync(".prettierrc.local", "utf-8");
localPrettierConfig = JSON.parse(localConfig);
console.log("Using local Prettier config from .prettierrc.local");
} catch (e) {
// no local config
}
const prettierConfig = {
trailingComma: "es5",
tabWidth: 4,
printWidth: 120,
semi: true,
endOfLine: "lf",
...localPrettierConfig,
};
export default prettierConfig;
+11
View File
@@ -0,0 +1,11 @@
hostname=http://localhost:5989/
dbname=livesync-test-db2
minioEndpoint=http://127.0.0.1:9000
username=admin
password=testpassword
accessKey=minioadmin
secretKey=minioadmin
bucketName=livesync-test-bucket
# ENABLE_DEBUGGER=true
# PRINT_LIVESYNC_LOGS=true
# ENABLE_UI=true
+68
View File
@@ -0,0 +1,68 @@
# AI Coding Assistant Instructions (AGENTS.md)
When working on this repository (writing code, comments, documentation, or commits), you MUST follow these guidelines to maintain consistency.
## Required Reference Files
Before making changes to documentation, user-facing text, or settings:
1. Read [docs/terms.md](docs/terms.md) for terminology, vocabulary conventions, and technical definitions.
2. Read [docs/settings.md](docs/settings.md) (and [docs/settings_ja.md](docs/settings_ja.md)) for UI settings and setting key mappings.
3. Read [docs/troubleshooting.md](docs/troubleshooting.md) for troubleshooting guidelines and common recovery steps (such as flag files and SCRAM state).
4. Read [devs.md](devs.md) for development workflows, module architecture, and testing infrastructure.
---
## Documentation and User-Facing Text Rules
Always adhere to the following stylistic and spelling rules:
1. **British English Spelling**:
- Write all documentation and user-facing messages in British English. If in doubt, the BBC News Styleguide may be useful as a reference.
- **Traditional Spelling (Trad-spelling)**: Use `-ise` and `-isation` suffixes instead of `-ize` and `-ization` (for example: 'initialisation', 'synchronisation', and 'organisation').
- **Oxford Comma**: Use the serial (Oxford) comma to separate items in lists of three or more (for example: 'settings, snippets, and themes' instead of 'settings, snippets and themes').
- **Logical Punctuation**: Place punctuation marks (such as commas and full stops) outside quotation marks unless they are part of the quoted text itself (for example: write 'dialogue', not 'dialogue,').
2. **No Contractions**:
- Do not use contractions in general text or documentation (for example: write "do not" instead of "don't", "cannot" instead of "can't", and "is not" instead of "isn't").
3. **Quotation Style**:
- Prefer single quotation marks (`'`) over double quotation marks (`"`) in general documentation text, unless the context requires double quotes (for example, inside JSON code blocks).
4. **Specific Terminology and Spelling**:
- Use **'dialogue'** in documentation, user-facing messages, and general text. Use **'dialog'** only inside source code (e.g. class names, methods).
- Use the hyphenated form **'plug-in'** in user-facing text. Use **'plugin'** only in codebase files, configuration settings, or technical contexts.
5. **User Communication Language**:
- Always reply to the user in the language in which they asked the question.
---
## Technical & Architecture Rules
1. **Database Structure**:
- Remember that Self-hosted LiveSync splits files into **Metadata** (file properties, size, paths) and **Chunks** (actual content). Do not store raw content in the metadata document directly.
2. **Setup and Recovery**:
- **Fast Setup (Simple Fetch)** is the preferred flow for initial replication on secondary devices. It utilises stream-based replication for high speed and delays local file reflection to suppress temporary synchronisation warnings.
- **Flag files** (such as `redflag.md`, `redflag2.md`, and `redflag3.md`) at the root of the vault control the boot-up sequence and trigger automated fetch/rebuild tasks.
3. **Subrepositories**:
- Treat `@vrtmrz/livesync-commonlib` as an external, authoritative package. Make Commonlib changes in its repository, validate the packed artefact and downstream LiveSync consumer, and update the exact dependency here. Do not recreate a `src/lib` source mirror or generated `_types` fallback.
4. **Application Directories**:
- The directory [src/apps](src/apps) contains independent application modules:
- `cli`: A Command Line Interface application. Tests specifically for the CLI (both unit and End-to-End tests) are located and executed within [src/apps/cli](src/apps/cli) using its local `package.json` scripts.
- `webapp`: A Web-based application.
- `webpeer`: A Web-based peer utility.
---
## Development & Verification Commands
Before submitting code, you should run verification scripts locally to ensure correct syntax and function.
1. **Lint and Type Checking**:
- Run `npm run check` to perform code verification. This runs type-checking (`tsc-check`), ESLint (`lint`), and Svelte checks (`svelte-check`).
2. **Unit Tests**:
- Run `npm run test:unit` to execute fast local unit tests.
- Run `npm run test` or `npm run test:full` for full testing suites (including dockerised services).
3. **Build**:
- Run `npm run build` to compile the production bundle (`main.js`).
- Run `npm run dev` for the development watch/build task.
+27
View File
@@ -0,0 +1,27 @@
# Code of Conduct
We wish to maintain an open, welcoming, and collaborative environment for all contributors.
## Our Standard
Our core principle is mutual respect. We encourage open discussion, diverse perspectives, and constructive feedback.
## The Limit of Tolerance
To preserve a tolerant and open community, we do not tolerate intolerance. Actions that aim to harass, exclude, or silence others are not welcome. Specifically, we do not accept personal attacks, breaches of privacy, or sustained disruption of discussions. We prioritise protecting the community's capacity for open, peaceful collaboration.
## Resolution
If any issue arises, the project maintainers will resolve it in a fair, minimal, and constructive manner, aiming to restore a cooperative environment. Depending on the nature of the behaviour, actions may range from a simple warning to temporary or permanent suspension of repository access.
## Contact
You can contact the project maintainer via email at `vrtmrz@proton.me` or via Nostr at `npub1azzj0dzw8evwtgyjeucyfz5cs8k0eg7rd0x4qvggcg3s7lx0dmaqv9sfka`.
## Criticism of the Maintainer
To ensure open and transparent governance, criticism of the maintainer will not be deleted as long as it is clearly framed as a constructive objection. However, spamming duplicate issues on the same topic or resorting to personal attacks will result in closure or removal.
## Revisions
This Code of Conduct is maintained by the project maintainers and may be updated to address new challenges. While the final decision rests with the maintainers, we welcome constructive suggestions and feedback through issues or pull requests.
+67
View File
@@ -0,0 +1,67 @@
# Contributing to Self-hosted LiveSync
Thank you for your interest in contributing to Self-hosted LiveSync! We welcome all contributions, including bug reports, feature requests, documentation improvements, translations, and pull requests.
## Getting Started
To set up the development environment, please follow these steps:
1. Clone the repository:
```bash
git clone https://github.com/vrtmrz/obsidian-livesync
```
2. Install the package dependencies:
```bash
npm ci
```
3. Build the plug-in:
```bash
npm run build
```
For a more comprehensive guide on development workflows, testing configurations, and the Commonlib dependency, please refer to [devs.md](devs.md).
## Guidelines for Contributions
### 1. Code Style and Verification
Before submitting a pull request, you must run verification scripts locally to ensure that there are no syntax, type, or linting errors:
- Run type checking and linting:
```bash
npm run check
```
This also type-checks the maintained CLI and browser applications, and applies the Community directory blocker rules. Run `npm run lint:community` separately to inspect its non-blocking recommendations.
- Run unit tests:
```bash
npm run test:unit
```
If you have the capability and a suitable environment (such as Linux and Docker), running the CLI End-to-End (E2E) tests is also highly appreciated. Instructions are detailed in [devs.md](devs.md). If you cannot run E2E tests locally, please explicitly ask to run the tests on the CI by stating 'Please run CI tests' in your pull request description.
### 2. Documentation and UI Text Style
To maintain consistency across the project, we ask that you follow the established writing style and conventions of the codebase when contributing documentation or user-facing messages:
- **Spelling**: Prioritise region-independent, neutral spelling if a suitable word exists. If there is no such word, please use British English spelling to align with the codebase's style (for example: preferring '-ise' and '-isation' suffixes over '-ize' and '-ization'). However, we do not treat alternative spellings as errors.
- **Oxford Comma**: Use the serial (Oxford) comma to separate items in lists of three or more (for example: 'settings, snippets, and themes').
- **Logical Punctuation**: Place punctuation marks outside quotation marks unless they are part of the quoted text itself (for example: write 'dialogue', not 'dialogue,').
- **No Contractions**: Avoid using contractions in general text or documentation (for example: write "do not" instead of "don't", and "cannot" instead of "can't").
- **Affirmative Phrasing**: Avoid asking questions using negative forms in user-facing dialogue. Use affirmative questions to prevent translation and interpretation discrepancies.
- **Specific Words**: Use 'dialogue' for documentation and user-facing messages (use 'dialog' only inside source code). Use the hyphenated form 'plug-in' in user-facing text (use 'plugin' only in configuration settings or technical contexts).
For a detailed list of vocabulary conventions and terms, please refer to [docs/terms.md](docs/terms.md).
### 3. Translations
To add or update translations, please refer to [docs/adding_translations.md](docs/adding_translations.md) for detailed instructions.
### 4. Commonlib changes
Shared synchronisation behaviour is provided by the `@vrtmrz/livesync-commonlib` package. If you wish to change that library, submit a separate pull request to the [livesync-commonlib repository](https://github.com/vrtmrz/livesync-commonlib), validate its packed artefact, then update the locked dependency in this repository. Do not add a source mirror or generated fallback declarations to this repository.
## License
By contributing, you agree that your contributions will be licensed under the MIT License.
+77 -69
View File
@@ -1,99 +1,107 @@
# Self-hosted LiveSync
[Japanese docs](./README_ja.md) - [Chinese docs](./README_cn.md).
[Japanese docs](./README_ja.md) [Chinese docs](./README_cn.md).
Self-hosted LiveSync is a community-implemented synchronization plugin.
A self-hosted or purchased CouchDB acts as the intermediate server. Available on every obsidian-compatible platform.
Self-hosted LiveSync is a community-developed synchronisation plug-in available on all Obsidian-compatible platforms. It leverages robust server solutions such as CouchDB or object storage systems (e.g., MinIO, S3, R2, etc.) to ensure reliable data synchronisation.
Note: It has no compatibility with the official "Obsidian Sync".
Additionally, it supports peer-to-peer synchronisation using WebRTC, enabling you to synchronise your notes directly between devices without relying on a server. Documentation is available for [Peer-to-Peer Synchronisation](./docs/p2p_sync_updates_2026.md).
![obsidian_live_sync_demo](https://user-images.githubusercontent.com/45774780/137355323-f57a8b09-abf2-4501-836c-8cb7d2ff24a3.gif)
Before installing or upgrading LiveSync, please back your vault up.
>[!IMPORTANT]
> This plug-in is not compatible with the official "Obsidian Sync" and cannot synchronise with it.
## Features
- Synchronise vaults efficiently with minimal traffic.
- Handle conflicting modifications effectively.
- Automatically merge simple conflicts.
- Use open-source solutions for the server.
- Compatible solutions are supported.
- Support end-to-end encryption.
- Synchronise settings, snippets, themes, and plug-ins via [Customisation Sync (Beta)](docs/settings.md#6-customization-sync-advanced) or [Hidden File Sync](docs/settings.md#7-hidden-files-advanced).
- Enable WebRTC peer-to-peer synchronisation without requiring a `host` (Experimental).
- This feature is still in the experimental stage. Please exercise caution when using it.
- WebRTC is a peer-to-peer synchronisation method, so **at least one device must be online to synchronise**.
- Instead of keeping your device online as a stable peer, you can use two pseudo-peers:
- [livesync-serverpeer](https://github.com/vrtmrz/livesync-serverpeer): A pseudo-client running on the server for receiving and sending data between devices.
- [webpeer](https://github.com/vrtmrz/obsidian-livesync/tree/main/src/apps/webpeer): A pseudo-client for receiving and sending data between devices.
- A pre-built instance is available at [fancy-syncing.vrtmrz.net/webpeer](https://fancy-syncing.vrtmrz.net/webpeer/) (hosted on the vrtmrz's blog site). This is also peer-to-peer. Feel free to use it.
- For more information, refer to the [English explanatory article](https://fancy-syncing.vrtmrz.net/blog/0034-p2p-sync-en.html) or the [Japanese explanatory article](https://fancy-syncing.vrtmrz.net/blog/0034-p2p-sync).
- Visual conflict resolver included.
- Bidirectional synchronization between devices nearly in real-time
- You can use CouchDB or its compatibles like IBM Cloudant.
- End-to-End encryption is supported.
- Plugin synchronization(Beta)
- Receive WebClip from [obsidian-livesync-webclip](https://chrome.google.com/webstore/detail/obsidian-livesync-webclip/jfpaflmpckblieefkegjncjoceapakdf) (End-to-End encryption will not be applicable.)
This plug-in may be particularly useful for researchers, engineers, and developers who need to keep their notes fully self-hosted for security reasons. It is also suitable for anyone seeking the peace of mind that comes with knowing their notes remain entirely private.
Useful for researchers, engineers and developers with a need to keep their notes fully self-hosted for security reasons. Or just anyone who would like the peace of mind of knowing that their notes are fully private.
>[!IMPORTANT]
> - Before installing or upgrading this plug-in, please back up your vault.
> - Do not enable this plug-in alongside another synchronisation solution (including iCloud and Obsidian Sync).
> - For backups, we also provide a plug-in called [Differential ZIP Backup](https://github.com/vrtmrz/diffzip).
## IMPORTANT NOTICE
## How to Use
- Do not enable this plugin with another synchronization solution at the same time (including iCloud and Obsidian Sync). Before enabling this plugin, make sure to disable all the other synchronization methods to avoid content corruption or duplication. If you want to synchronize to two or more services, do them one by one and never enable two synchronization methods at the same time.
This includes not putting your vault inside a cloud-synchronized folder (eg. an iCloud folder or Dropbox folder)
- This is a synchronization plugin. Not a backup solution. Do not rely on this for backup.
- If the device's storage runs out, database corruption may happen.
- Hidden files or any other invisible files wouldn't be kept in the database, and thus won't be synchronized. (**and may also get deleted**)
### 3-minute setup - CouchDB on fly.io
## How to use
**Recommended for beginners**
### Get your database ready.
[![LiveSync Setup onto Fly.io SpeedRun 2024 using Google Colab](https://img.youtube.com/vi/7sa_I1832Xc/0.jpg)](https://www.youtube.com/watch?v=7sa_I1832Xc)
First, get your database ready. IBM Cloudant is preferred for testing. Or you can use your own server with CouchDB. For more information, refer below:
1. [Setup IBM Cloudant](docs/setup_cloudant.md)
2. [Setup your CouchDB](docs/setup_own_server.md)
1. [Set up CouchDB on fly.io](docs/setup_flyio.md)
2. Configure plug-in in [Quick Setup](docs/quick_setup.md)
Note: More information about alternative hosting methods is needed! Currently, [using fly.io](https://github.com/vrtmrz/obsidian-livesync/discussions/85) is being discussed.
### Manual Setup
### Configure the plugin
1. Set up the server
1. [Set up CouchDB on fly.io](docs/setup_flyio.md)
2. [Set up your CouchDB](docs/setup_own_server.md)
2. Configure plug-in in [Quick Setup](docs/quick_setup.md)
> [!TIP]
> Fly.io is no longer free. Fortunately, we can still use IBM Cloudant despite some limitations. Refer to [Set up IBM Cloudant](docs/setup_cloudant.md).
> We can also use peer-to-peer synchronisation without a server. Alternatively, cheap object storage like Cloudflare R2 can be used for free.
> However, most importantly, we can use a server that we trust. Therefore, please set up your own server.
> CouchDB can also be run on a Raspberry Pi (please be mindful of your server's security).
See [Quick setup guide](doccs/../docs/quick_setup.md)
## Something looks corrupted...
## Information in the Status Bar
Please open the configuration link again and Answer below:
- If your local database looks corrupted (in other words, when your Obsidian getting weird even standalone.)
- Answer `No` to `Keep local DB?`
- If your remote database looks corrupted (in other words, when something happens while replicating)
- Answer `No` to `Keep remote DB?`
If you answered `No` to both, your databases will be rebuilt by the content on your device. And the remote database will lock out other devices. You have to synchronize all your devices again. (When this time, almost all your files should be synchronized with a timestamp. So you can use an existing vault).
## Test Server
~~Setting up an instance of Cloudant or local CouchDB is a little complicated, so I set up a [Tasting server for self-hosted-livesync](https://olstaste.vrtmrz.net/). Try it out for free!~~
Now (30 May 2023) suspending while the server transfer.
Note: Please read "Limitations" carefully. Do not send your private vault.
## Information in StatusBar
Synchronization status is shown in statusbar.
Synchronisation status is shown in the status bar with the following icons.
- Activity Indicator
- 📲 A finite remote operation is in progress
- 🌐N Approximate remote requests currently in progress
- Status
- ⏹️ Stopped
- 💤 LiveSync enabled. Waiting for changes.
- ⚡️ Synchronization in progress.
- ⚠ An error occurred.
- ↑ Uploaded chunks and metadata
- ↓ Downloaded chunks and metadata
- ⏳ Number of pending processes
- 🧩 Number of files waiting for their chunks.
If you have deleted or renamed files, please wait until ⏳ icon disappeared.
- 💤 LiveSync enabled. Waiting for changes
- ⚡️ Synchronisation in progress
- ⚠ An error occurred
- Statistical Indicators
- ↑ Uploaded chunks and metadata
- ↓ Downloaded chunks and metadata
- Progress Indicators
- 📥 Unprocessed transferred items
- 📄 Working database operation
- 💾 Working write storage processes
- ⏳ Working read storage processes
- 🛫 Pending read storage processes
- 📬 Batched read storage processes
- ⚙️ Working or pending storage processes for hidden files
- 🧩 Waiting chunks
- 🔌 Working customisation items (configuration, snippets, and plug-ins)
To prevent file and database corruption, please avoid closing Obsidian until all progress indicators have disappeared as much as possible (although the plug-in will attempt to resume if interrupted). This is especially important if you have deleted or renamed files.
## Hints
- If a folder becomes empty after a replication, it will be deleted by default. But you can toggle this behaviour. Check the [Settings](docs/settings.md).
- LiveSync mode drains more batteries in mobile devices. Periodic sync with some automatic sync is recommended.
- Mobile Obsidian can not connect to non-secure (HTTP) or locally-signed servers, even if the root certificate is installed on the device.
- There are no 'exclude_folders' like configurations.
- While synchronizing, files are compared by their modification time and the older ones will be overwritten by the newer ones. Then plugin checks for conflicts and if a merge is needed, a dialog will open.
- Rarely, a file in the database could be corrupted. The plugin will not write to local storage when a file looks corrupted. If a local version of the file is on your device, the corruption could be fixed by editing the local file and synchronizing it. But if the file does not exist on any of your devices, then it can not be rescued. In this case, you can delete these items from the settings dialog.
- To stop the boot-up sequence (eg. for fixing problems on databases), you can put a `redflag.md` file (or directory) at the root of your vault.
Tip for iOS: a redflag directory can be created at the root of the vault using the File application.
- Also, with `redflag2.md` placed, we can automatically rebuild both the local and the remote databases during the boot-up sequence. With `redflag3.md`, we can discard only the local database and fetch from the remote again.
- Q: The database is growing, how can I shrink it down?
A: each of the docs is saved with their past 100 revisions for detecting and resolving conflicts. Picturing that one device has been offline for a while, and comes online again. The device has to compare its notes with the remotely saved ones. If there exists a historic revision in which the note used to be identical, it could be updated safely (like git fast-forward). Even if that is not in revision histories, we only have to check the differences after the revision that both devices commonly have. This is like git's conflict-resolving method. So, We have to make the database again like an enlarged git repo if you want to solve the root of the problem.
- And more technical Information is in the [Technical Information](docs/tech_info.md)
- If you want to synchronize files without obsidian, you can use [filesystem-livesync](https://github.com/vrtmrz/filesystem-livesync).
- WebClipper is also available on Chrome Web Store:[obsidian-livesync-webclip](https://chrome.google.com/webstore/detail/obsidian-livesync-webclip/jfpaflmpckblieefkegjncjoceapakdf)
## Tips and Troubleshooting
- If you want a faster and simpler initial replication when setting up subsequent devices, see the [Fast Setup Guide](docs/tips/fast-setup.md).
- If you are having problems getting the plug-in working, see [Tips and Troubleshooting](docs/troubleshooting.md).
Repo is here: [obsidian-livesync-webclip](https://github.com/vrtmrz/obsidian-livesync-webclip). (Docs are a work in progress.)
## Acknowledgements
The project has been in continual progress and harmony thanks to the following:
- Many [Contributors](https://github.com/vrtmrz/obsidian-livesync/graphs/contributors).
- Many [GitHub Sponsors](https://github.com/sponsors/vrtmrz#sponsors).
- JetBrains Community Programs / Support for Open-Source Projects. <img src="https://resources.jetbrains.com/storage/products/company/brand/logos/jetbrains.png" alt="JetBrains logo" height="24">
May those who have contributed be honoured and remembered for their kindness and generosity.
## Development Guide
Please refer to the [Development Guide](devs.md) for development setup, testing infrastructure, code conventions, and more.
## License
The source code is licensed under the MIT License.
Licensed under the MIT License.
+75 -97
View File
@@ -1,9 +1,12 @@
# Self-hosted LiveSync
Self-hosted LiveSync (自搭建在线同步) 是一个社区实现的在线同步插件。
使用一个自搭建的或者购买的 CouchDB 作为中转服务器。兼容所有支持 Obsidian 的平台。
它利用诸如CouchDB或对象存储系统(例如MinIO、S3、R2等)等强大的服务器解决方案,以确保数据同步的可靠性。。兼容所有支持 Obsidian 的平台。
注意: 本插件与官方的 "Obsidian Sync" 服务不兼容
此外,它现在支持使用WebRTC进行点对点同步(实验性功能),使您无需依赖服务器即可直接在设备之间同步笔记
>[!IMPORTANT]
>本插件与官方的 "Obsidian Sync" 服务不兼容。
![obsidian_live_sync_demo](https://user-images.githubusercontent.com/45774780/137355323-f57a8b09-abf2-4501-836c-8cb7d2ff24a3.gif)
@@ -11,119 +14,94 @@ Self-hosted LiveSync (自搭建在线同步) 是一个社区实现的在线同
## 功能
- 可视化的冲突解决器
- 接近实时的多设备双向同步
- 可使用 CouchDB 以及兼容的服务,如 IBM Cloudant
- 支持端到端加密
- 插件同步 (Beta)
- 从 [obsidian-livesync-webclip](https://chrome.google.com/webstore/detail/obsidian-livesync-webclip/jfpaflmpckblieefkegjncjoceapakdf) 接收 WebClip (本功能不适用端到端加密)
- 以最少流量高效同步vault
- 有效处理冲突的修改。
- 自动合并简单冲突。
- 服务端使用开源的解决方案
- 支持兼容的解决方案。
- 支持端到端加密
- 同步设置、代码片段、主题和插件,通过 [Customisation Sync (Beta)](docs/settings.md#6-customization-sync-advanced) 或者 [Hidden File Sync](docs/settings.md#7-hidden-files-advanced).
- 启用 WebRTC 点对点同步,无需指定 `host`(实验性)。
- 此功能仍处于试验阶段。请在使用时务必谨慎。
- WebRTC 是一种点对点同步方法,因此**至少有一台设备必须在线才能进行同步**。
- 与其让您的设备作为稳定的对等节点保持在线,您可以使用两个 pseudo-peers:
- [livesync-serverpeer](https://github.com/vrtmrz/livesync-serverpeer): 在服务器上运行的 pseudo-client 用于在设备之间接收和发送数据。
- [webpeer](https://github.com/vrtmrz/livesync-commonlib/tree/main/apps/webpeer): 用于在设备之间接收和发送数据的pseudo-client。
- 一个预构建的实例现已上线,地址为 [fancy-syncing.vrtmrz.net/webpeer](https://fancy-syncing.vrtmrz.net/webpeer/) (托管于vrtmrz博客网站). 这也是一个点对点的实例。可自由使用。
- 欲了解更多信息,请参阅[英文说明文章](https://fancy-syncing.vrtmrz.net/blog/0034-p2p-sync-en.html)或[日文说明文章](https://fancy-syncing.vrtmrz.net/blog/0034-p2p-sync)。
适用于出于安全原因需要将笔记完全自托管的研究人员、工程师或开发人员,以及任何喜欢笔记完全私密所带来的安全感的人。
此插件适用于出于安全原因需要将笔记完全自托管的研究人员、工程师或开发人员,以及任何喜欢笔记完全私密所带来的安全感的人。
## 重要提醒
- 请勿与其同步解决方案(包括 iCloudObsidian Sync一起使用。在启用此插件之前,请确保禁用所有其他同步方法以避免内容损坏或重复。如果要同步到多个服务,请一一进行,切勿同时启用两种同步方法
这包括不能将您的保管库放在云同步文件夹中(例如 iCloud 文件夹或 Dropbox 文件夹)
- 这是一个同步插件,不是备份解决方案。不要依赖它进行备份。
- 如果设备的存储空间耗尽,可能会发生数据库损坏。
- 隐藏文件或任何其他不可见文件不会保存在数据库中,因此不会被同步。(**并且可能会被删除**)
>[!IMPORTANT]
> - 在安装或升级此插件之前,请务必备份您的保险库。
> - 请勿同时启用此插件与其同步方案(包括iCloudObsidian Sync)。
> - 对于备份,我们还提供了一款名为[Differential ZIP Backup](https://github.com/vrtmrz/diffzip)的插件。
## 如何使用
### 准备好你的数据库
### 3分钟搞定——在fly.io上部署CouchDB
首先,准备好你的数据库。IBM Cloudant 是用于测试的首选。或者,您也可以在自己的服务器上安装 CouchDB。有关更多信息,请参阅以下内容:
1. [Setup IBM Cloudant](docs/setup_cloudant.md)
2. [Setup your CouchDB](docs/setup_own_server_cn.md)
**推荐初学者第一次使用此方法**
[![LiveSync Setup onto Fly.io SpeedRun 2024 using Google Colab](https://img.youtube.com/vi/7sa_I1832Xc/0.jpg)](https://www.youtube.com/watch?v=7sa_I1832Xc)
Note: 正在征集更多搭建方法!目前在讨论的有 [使用 fly.io](https://github.com/vrtmrz/obsidian-livesync/discussions/85)
1. [Setup CouchDB on fly.io](docs/setup_flyio.md)
2. 在 [Quick Setup](docs/quick_setup.md) 中配置插件。
### 第一个设备
### 手动设置
1. 在您的设备上安装插件。
2. 配置远程数据库信息。
1. 将您的服务器信息填写到 `Remote Database configuration`(远程数据库配置)设置页中。
2. 建议启用 `End to End Encryption`(端到端加密)。输入密码后,单击“应用”。
3. 点击 `Test Database Connection` 并确保插件显示 `Connected to (你的数据库名称)`
4. 单击 `Check database configuration`(检查数据库配置)并确保所有测试均已通过。
3.`Sync Settings`(同步设置)选项卡中配置何时进行同步。(您也可以稍后再设置)
1. 如果要实时同步,请启用 `LiveSync`
2. 或者,根据您的需要设置同步方式。默认情况下,不会启用任何自动同步,这意味着您需要手动触发同步过程。
3. 其他配置也在这里。建议启用 `Use Trash for deleted files`(删除文件到回收站),但您也可以保持所有配置不变
4. 配置杂项功能。
1. 启用 `Show staus inside editor` 会在编辑器右上角显示状态。(推荐开启)
5. 回到编辑器。等待初始扫描完成。
6. 当状态不再变化并显示 ⏹️ 图标表示 COMPLETED(没有 ⏳ 和 🧩 图标)时,您就可以与服务器同步了。
7. 按功能区上的复制图标或从命令面板运行 `Replicate now`(立刻复制)。这会将您的所有数据发送到服务器。
8. 打开命令面板,运行 `Copy setup URI`(复制设置链接),并设置密码。这会将您的配置导出到剪贴板,作为您导入其他设备的链接。
1. 配置服务器
1. [在fly.io上快速搭建CouchDB](docs/setup_flyio.md)
2. [自行搭建CouchDB](docs/setup_own_server.md)
2. 在[快速设置](docs/quick_setup.md)中配置插件
> [!提示]
> Fly.io现已不再免费。不过,尽管存在一些问题,我们仍可使用IBM Cloudant。请参考[搭建IBM Cloudant](docs/setup_cloudant.md)。
> 此外,我们还可以采用点对点同步方式,无需搭建服务器;或者选用价格极低的对象存储——Cloudflare R2可免费使用
> 但最重要的是,我们可以选择自己信任的服务器。因此,建议您搭建自有服务器
> CouchDB可在树莓派上运行。(但请务必注意服务器的安全性)
**重要: 不要公开本链接,这个链接包含了你的所有认证信息!** (即使没有密码别人读不了)
### 后续设备
注意:如果要与非空的 vault 进行同步,文件的修改日期和时间必须互相匹配。否则,可能会发生额外的传输或文件可能会损坏。
为简单起见,我们强烈建议同步到一个全空的 vault。
## 状态栏中的信息
1. 安装插件
2. 打开您从第一台设备导出的链接。
3. 插件会询问您是否确定应用配置。 回答 `Yes`,然后按照以下说明进行操作:
1.`Keep local DB?` 回答 `Yes`
*注意:如果您希望保留本地现有 vault,则必须对此问题回答 `No`,并对 `Rebuild the database?` 回答 `No`。*
2.`Keep remote DB?` 回答 `Yes`
3.`Replicate once?` 回答 `Yes`
完成后,您的所有设置将会从第一台设备成功导入。
4. 你的笔记应该很快就会同步。
## 文件看起来有损坏...
请再次打开配置链接并回答如下:
- 如果您的本地数据库看起来已损坏(当你的本地 Obsidian 文件看起来很奇怪)
-`Keep local DB?` 回答 `No`
- 如果您的远程数据库看起来已损坏(当复制时发生中断)
-`Keep remote DB?` 回答 `No`
如果您对两者都回答“否”,您的数据库将根据您设备上的内容重建。并且远程数据库将锁定其他设备,您必须再次同步所有设备。(此时,几乎所有文件都会与时间戳同步。因此您可以安全地使用现有的 vault)。
## 测试服务器
设置 Cloudant 或本地 CouchDB 实例有点复杂,所以我搭建了一个 [self-hosted-livesync 尝鲜服务器](https://olstaste.vrtmrz.net/)。欢迎免费尝试!
注意:请仔细阅读“限制”条目。不要发送您的私人 vault。
## 状态栏信息
同步状态将显示在状态栏。
同步状态显示在状态栏中,采用以下图标
- 活动指示器
- 📲 网络请求
- 状态
- ⏹️ 就绪
- 💤 LiveSync 已启用,正在等待更改
- ⚡️ 同步中
-一个错误出现了。
- ↑ 上传的 chunk 和元数据数量
- ↓ 下载的 chunk 和元数据数量
- ⏳ 等待的过程的数量
- 🧩 正在等待 chunk 的文件数量
如果你删除或更名了文件,请等待 ⏳ 图标消失。
- ⏹️ 已停止
- 💤 LiveSync已启用,正在等待更改
- ⚡️ 同步中
-发生了错误
- 统计指标
- ↑ 上传的分块与元数据
- ↓ 下载的分块与元数据
- 进度指示器
- 📥 未处理的传输项
- 📄 正在进行的数据库操作
- 💾 正在进行的写入存储进程
- ⏳ 正在进行的读取存储进程
- 🛫 待处理的读取存储进程
- 📬 批量处理的读取存储进程
- ⚙️ 正在进行或待处理的隐藏文件存储进程
- 🧩 等待中的分块
- 🔌 正在进行的自定义项(配置、代码片段和插件)
为避免文件和数据库损坏,请等待所有进度指示器尽可能消失后再关闭 Obsidian(插件也会尝试恢复同步进度)。特别是在您已删除或重命名文件的情况下,请务必遵守此操作。
## 提示
- 如果文件夹在复制后变为空,则默认情况下该文件夹会被删除。您可以关闭此行为。检查 [设置](docs/settings.md)。
- LiveSync 模式在移动设备上可能导致耗电量增加。建议使用定期同步 + 条件自动同步。
- 移动平台上的 Obsidian 无法连接到非安全 (HTTP) 或本地签名的服务器,即使设备上安装了根证书。
- 没有类似“exclude_folders”的配置。
- 同步时,文件按修改时间进行比较,较旧的将被较新的文件覆盖。然后插件检查冲突,如果需要合并,将打开一个对话框。
- 数据库中的文件在罕见情况下可能会损坏。当接收到的文件看起来已损坏时,插件不会将其写入本地存储。如果您的设备上有文件的本地版本,则可以通过编辑本地文件并进行同步来覆盖损坏的版本。但是,如果您的任何设备上都不存在该文件,则无法挽救该文件。在这种情况下,您可以从设置对话框中删除这些损坏的文件。
- 要阻止插件的启动流程(例如,为了修复数据库问题),您可以在 vault 的根目录创建一个 "redflag.md" 文件。
- 问:数据库在增长,我该如何缩小它?
答:每个文档都保存了过去 100 次修订,用于检测和解决冲突。想象一台设备已经离线一段时间,然后再次上线。设备必须将其笔记与远程保存的笔记进行比较。如果存在曾经相同的历史修订,则可以安全地直接更新这个文件(和 git 的快进原理一样)。即使文件不在修订历史中,我们也只需检查两个设备上该文件的公有修订版本之后的差异。这就像 git 的冲突解决方法。所以,如果想从根本上解决数据库太大的问题,我们像构建一个扩大版的 git repo 一样去重新设计数据库。
- 更多技术信息在 [技术信息](docs/tech_info.md)
- 如果你想在没有黑曜石的情况下同步文件,你可以使用[filesystem-livesync](https://github.com/vrtmrz/filesystem-livesync)。
- WebClipper 也可在 Chrome Web Store 上使用:[obsidian-livesync-webclip](https://chrome.google.com/webstore/detail/obsidian-livesync-webclip/jfpaflmpckblieefkegjncjoceapakdf)
## 使用技巧与故障排除
如果您在配置插件时遇到问题,请参阅:[Tips and Troubleshooting](docs/troubleshooting.md).
仓库地址:[obsidian-livesync-webclip](https://github.com/vrtmrz/obsidian-livesync-webclip) (文档施工中)
## 致谢
本项目得以持续顺利推进,离不开以下各方的贡献:
- 众多[贡献者](https://github.com/vrtmrz/obsidian-livesync/graphs/contributors)。
- 许多[GitHub 赞助人](https://github.com/sponsors/vrtmrz#sponsors)。
- JetBrains 社区计划/对开源项目的支持。<img src="https://resources.jetbrains.com/storage/products/company/brand/logos/jetbrains.png" alt="JetBrains logo" height="24">
## License
愿所有作出贡献的人士因其善良与慷慨而受到尊敬与铭记。
The source code is licensed under the MIT License.
本源代码使用 MIT 协议授权。
## 许可协议
本项目采用 MIT 许可协议授权。
+93
View File
@@ -0,0 +1,93 @@
<!-- For translation: 20240227r0 -->
# Self-hosted LiveSync
[Documentación en inglés](./README_ja.md) - [Documentación en japonés](./README_ja.md) - [Documentación en chino](./README_cn.md).
Self-hosted LiveSync es un plugin de sincronización implementado por la comunidad, disponible en todas las plataformas compatibles con Obsidian y utiliza CouchDB o Almacenamiento de Objetos (por ejemplo, MinIO, S3, R2, etc.) como servidor.
![Demostración de Obsidian Live Sync](https://user-images.githubusercontent.com/45774780/137355323-f57a8b09-abf2-4501-836c-8cb7d2ff24a3.gif)
Nota: Este plugin no puede sincronizarse con el "Obsidian Sync" oficial.
## Características
- Sincroniza bóvedas de manera eficiente con menos tráfico.
- Buen manejo de modificaciones en conflicto.
- Fusión automática para conflictos simples.
- Uso de soluciones de código abierto para el servidor.
- Pueden usarse soluciones compatibles.
- Soporte de cifrado de extremo a extremo.
- Sincronización de configuraciones, fragmentos, temas y complementos a través de [Sincronización de personalización \(Beta\)](#customization-sync) o [Sincronización de archivos ocultos](#hiddenfilesync)
- WebClip de [obsidian-livesync-webclip](https://chrome.google.com/webstore/detail/obsidian-livesync-webclip/jfpaflmpckblieefkegjncjoceapakdf)
Este plugin puede ser útil para investigadores, ingenieros y desarrolladores que necesitan mantener sus notas totalmente autoalojadas por razones de seguridad, o para aquellos que deseen tener la tranquilidad de saber que sus notas son totalmente privadas.
>[!IMPORTANTE]
> - Antes de instalar o actualizar este plugin, realice un respaldo de su bóveda.
> - No active este plugin junto con otra solución de sincronización al mismo tiempo (incluyendo iCloud y Obsidian Sync).
> - Este es un plugin de sincronización, no una solución de respaldo. No confíe en él para realizar respaldos.
## Cómo usar
### Configuración en 3 minutos - CouchDB en fly.io
**Recomendado para principiantes**
[![Configuración de LiveSync en Fly.io 2024 usando Google Colab](https://img.youtube.com/vi/7sa_I1832Xc/0.jpg)](https://www.youtube.com/watch?v=7sa_I1832Xc)
1. [Configurar CouchDB en fly.io](docs/setup_flyio_es.md)
2. Configurar el plugin en [Configuración rápida](docs/quick_setup_es.md)
### Configuración manual
1. Configurar el servidor
1. [Configurar CouchDB en fly.io](docs/setup_flyio_es.md)
2. [Configurar su CouchDB](docs/setup_own_server_es.md)
2. Configura el plugin en [Configuración rápida](docs/quick_setup_es.md)
> [!CONSEJO]
> Actualmente, fly.io ya no es gratuito. Afortunadamente, aunque hay algunos problemas, aún podemos usar IBM Cloudant. Aquí está como [Configurar IBM Cloudant](docs/setup_cloudant.md). ¡Se actualizará pronto!
## Información en la barra de estado
El estado de sincronización se muestra en la barra de estado con los siguientes iconos.
- Indicador de actividad
- 📲 Solicitud de red
- Estado
- ⏹️ Detenido
- 💤 LiveSync activado. Esperando cambios
- ⚡️ Sincronización en progreso
- ⚠ Ocurrió un error
- Indicador estadístico
- ↑ Chunks y metadatos subidos
- ↓ Chunks y metadatos descargados
- Indicador de progreso
- 📥 Elementos transferidos sin procesar
- 📄 Operación de base de datos en curso
- 💾 Procesos de escritura en almacenamiento en curso
- ⏳ Procesos de lectura en almacenamiento en curso
- 🛫 Procesos de lectura en almacenamiento pendientes
- 📬 Procesos de lectura en almacenamiento por lotes
- ⚙️ Procesos de almacenamiento de archivos ocultos en curso o pendientes
- 🧩 Chunks en espera
- 🔌 Elementos de personalización en curso (Configuración, fragmentos y plugins)
Para prevenir la corrupción de archivos y bases de datos, antes de detener Obsidian espere hasta que todos los indicadores de progreso hayan desaparecido (el plugin también intentará reanudar, sin embargo). Especialmente en caso de que haya eliminado o renombrado archivos.
## Consejos y Solución de Problemas
Si tienes problemas para hacer funcionar el plugin, consulta: [Consejos y solución de problemas](docs/troubleshooting_es.md).
## Agradecimientos
El proyecto ha progresado y mantenido en armonía gracias a:
- Muchos [Colaboradores](https://github.com/vrtmrz/obsidian-livesync/graphs/contributors)
- Muchos [Patrocinadores de GitHub](https://github.com/sponsors/vrtmrz#sponsors)
- Programas comunitarios de JetBrains / Soporte para Proyectos de Código Abierto <img src="https://resources.jetbrains.com/storage/products/company/brand/logos/jetbrains.png" alt="JetBrains logo." height="24">
Que aquellos que han contribuido sean honrados y recordados por su amabilidad y generosidad.
## Licencia
Licenciado bajo la Licencia MIT.
+78 -76
View File
@@ -1,84 +1,86 @@
<!-- For translation: 20240227r0 -->
# Self-hosted LiveSync
[英語版ドキュメント](./README.md) - [中国語版ドキュメント](./README_cn.md).
**旧): obsidian-livesync**
Obsidianで利用可能なすべてのプラットフォームで使える、CouchDBをサーバに使用する、コミュニティ版の同期プラグイン
セルフホストしたデータベースを使って、双方向のライブシンクするObsidianのプラグイン。
**公式のSyncとは互換性はありません**
![obsidian_live_sync_demo](https://user-images.githubusercontent.com/45774780/137355323-f57a8b09-abf2-4501-836c-8cb7d2ff24a3.gif)
**インストールする前に、Vaultのバックアップを確実に取得してください**
[英語版](./README.md)
## こんなことができるプラグインです。
- Windows, Mac, iPad, iPhone, Android, Chromebookで動く
- セルフホストしたデータベースに同期して
- 複数端末で同時にその変更をほぼリアルタイムで配信し
- さらに、他の端末での変更も別の端末に配信する、双方向リアルタイムなLiveSyncを実現でき、
- 発生した変更の衝突はその場で解決できます。
- 同期先のホストにはCouchDBまたはその互換DBaaSのIBM Cloudantをサーバーに使用できます。あなたのデータは、あなたのものです。
- もちろんLiveではない同期もできます。
- 万が一のために、サーバーに送る内容を暗号化できます(betaです)。
- [Webクリッパー](https://chrome.google.com/webstore/detail/obsidian-livesync-webclip/jfpaflmpckblieefkegjncjoceapakdf) もあります(End-to-End暗号化対象外です)
NDAや類似の契約や義務、倫理を守る必要のある、研究者、設計者、開発者のような方に特にオススメです。
特にエンタープライズでは、たとえEnd to Endの暗号化が行われていても、管理下にあるサーバーにのみデータを格納することが求められる場合があります。
# 重要なお知らせ
- ❌ファイルの重複や破損を避けるため、複数の同期手段を同時に使用しないでください。
これは、Vaultをクラウド管理下のフォルダに置くことも含みます。(例えば、iCloudの管理フォルダ内に入れたり)。
- ⚠️このプラグインは、端末間でのノートの反映を目的として作成されました。バックアップ等が目的ではありません。そのため、バックアップは必ず別のソリューションで行うようにしてください。
- ストレージの空き容量が枯渇した場合、データベースが破損することがあります。
# このプラグインの使い方
1. Community Pluginsから、Self-holsted LiveSyncと検索しインストールするか、このリポジトリのReleasesから`main.js`, `manifest.json`, `style.css` をダウンロードしvaultの中の`.obsidian/plugins/obsidian-livesync`に入れて、Obsidianを再起動してください。
2. サーバーをセットアップします。IBM Cloudantがお手軽かつ堅牢で便利です。完全にセルフホストする際にはお持ちのサーバーにCouchDBをインストールする必要があります。詳しくは下記を参照してください
1. [IBM Cloudantのセットアップ](docs/setup_cloudant_ja.md)
2. [独自のCouchDBのセットアップ](docs/setup_own_server_ja.md)
備考: IBM Cloudantのアカウント登録が出来ないケースがあるようです。代替を探していて、今 [using fly.io](https://github.com/vrtmrz/obsidian-livesync/discussions/85)を検討しています。
1. [Quick setup](docs/quick_setup_ja.md)から、セットアップウィザード使ってセットアップしてください。
# テストサーバー
もし、CouchDBをインストールしたり、Cloudantのインスタンスをセットアップしたりするのに気が引ける場合、[Self-hosted LiveSyncのテストサーバー](https://olstaste.vrtmrz.net/)を作りましたので、使ってみてください。
備考: 制限事項をよく確認して使用してください。くれぐれも、本当に使用している自分のVaultを同期しないようにしてください。
# WebClipperあります
Self-hosted LiveSync用にWebClipperも作りました。Chrome Web Storeからダウンロードできます。
[obsidian-livesync-webclip](https://chrome.google.com/webstore/detail/obsidian-livesync-webclip/jfpaflmpckblieefkegjncjoceapakdf)
リポジトリはこちらです: [obsidian-livesync-webclip](https://github.com/vrtmrz/obsidian-livesync-webclip)。
相変わらずドキュメントは間に合っていません。
# ステータスバーの情報
右下のステータスバーに、同期の状態が表示されます
- 同期状態
- ⏹️ 同期は停止しています
- 💤 同期はLiveSync中で、なにか起こるのを待っています
- ⚡️ 同期中です
- ⚠ エラーが発生しています
- ↑ 送信したデータ数
- ↓ 受信したデータ数
- ⏳ 保留している処理の数です
ファイルを削除したりリネームした場合、この表示が消えるまでお待ちください。
# さらなる補足
- ファイルは同期された後、タイムスタンプを比較して新しければいったん新しい方で上書きされます。その後、衝突が発生したかによって、マージが行われます。
- まれにファイルが破損することがあります。破損したファイルに関してはディスクへの反映を試みないため、実際には使用しているデバイスには少し古いファイルが残っていることが多いです。そのファイルを再度更新してもらうと、データベースが更新されて問題なくなるケースがあります。ファイルがどの端末にも存在しない場合は、設定画面から、削除できます。
- データベースの復旧中に再起動した場合など、うまくローカルデータベースを修正できない際には、Vaultのトップに`redflag.md`というファイルを置いてください。起動時のシーケンスがスキップされます。
- データベースが大きくなってきてるんだけど、小さくできる?→各ノートは、それぞれの古い100リビジョンとともに保存されています。例えば、しばらくオフラインだったあるデバイスが、久しぶりに同期したと想定してみてください。そのとき、そのデバイスは最新とは少し異なるリビジョンを持ってるはずです。その場合でも、リモートのリビジョン履歴にリモートのものが存在した場合、安全にマージできます。もしリビジョン履歴に存在しなかった場合、確認しなければいけない差分も、対象を存在して持っている共通のリビジョン以降のみに絞れます。ちょうどGitのような方法で、衝突を解決している形になるのです。そのため、肥大化したリポジトリの解消と同様に、本質的にデータベースを小さくしたい場合は、データベースの作り直しが必要です。
- その他の技術的なお話は、[技術的な内容](docs/tech_info_ja.md)に書いてあります。
※公式のSyncと同期することはできません。
# ライセンス
## 機能
- 高効率・低トラフィックでVault同士を同期
- 競合解決がいい感じ
- 単純な競合なら自動マージします
- OSSソリューションを同期サーバに使用
- 互換ソリューションも使用可能です
- End-to-End暗号化実装済み
- 設定・スニペット・テーマ、プラグインの同期が可能
- [Webクリッパー](https://chrome.google.com/webstore/detail/obsidian-livesync-webclip/jfpaflmpckblieefkegjncjoceapakdf) もあります
The source code is licensed MIT.
NDAや類似の契約や義務、倫理を守る必要のある、研究者、設計者、開発者のような方に特にオススメです。
>[!IMPORTANT]
> - インストール・アップデート前には必ずVaultをバックアップしてください
> - 複数の同期ソリューションを同時に有効にしないでください(これはiCloudや公式のSyncも含みます)
> - このプラグインは同期プラグインです。バックアップとして使用しないでください
## このプラグインの使い方
### 3分セットアップ - CouchDB on fly.io
**はじめての方におすすめ**
[![LiveSync Setup onto Fly.io SpeedRun 2024 using Google Colab](https://img.youtube.com/vi/7sa_I1832Xc/0.jpg)](https://www.youtube.com/watch?v=7sa_I1832Xc)
1. [Fly.ioにCouchDBをセットアップする](docs/setup_flyio.md)
2. [Quick Setup](docs/quick_setup_ja.md)でプラグインを設定する
### Manually Setup
1. サーバのセットアップ
1. [Fly.ioにCouchDBをセットアップする](docs/setup_flyio.md)
2. [CouchDBをセットアップする](docs/setup_own_server_ja.md)
2. [Quick Setup](docs/quick_setup_ja.md)でプラグインを設定する
> [!TIP]
> IBM Cloudantもまだ使用できますが、いくつかの理由で現在はおすすめしていません。[IBM Cloudantのセットアップ](docs/setup_cloudant_ja.md)はまだあります。
## ステータスバーの説明
同期ステータスはステータスバーに、下記のアイコンとともに表示されます
- アクティビティー
- 📲 ネットワーク接続中
- 同期ステータス
- ⏹️ 停止中
- 💤 変更待ち(LiveSync中)
- ⚡️ 同期の進行中
- ⚠ エラー
- 統計情報
- ↑ アップロードしたチャンクとメタデータ数
- ↓ ダウンロードしたチャンクとメタデータ数
- 進捗情報
- 📥 転送後、未処理の項目数
- 📄 稼働中データベース操作数
- 💾 稼働中のストレージ書き込み数操作数
- ⏳ 稼働中のストレージ読み込み数操作数
- 🛫 待機中のストレージ読み込み数操作数
- ⚙️ 隠しファイルの操作数(待機・稼働中合計)
- 🧩 取得待ちを行っているチャンク数
- 🔌 設定同期関連の操作数
データベースやファイルの破損を避けるため、Obsidianの終了は進捗情報が表示されなくなるまで待ってください(プラグインも復帰を試みますが)。特にファイルを削除やリネームした場合は気をつけてください。
## Tips and Troubleshooting
- 2台目以降のセットアップ時に、初期同期をより迅速かつ簡単に行うには、[ファストセットアップガイド](docs/tips/fast-setup_ja.md)をご参照ください。
- 何かこまったら、[Tips and Troubleshooting](docs/troubleshooting.md)をご参照ください。
## License
Licensed under the MIT License.
+92
View File
@@ -0,0 +1,92 @@
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="UTF-8">
<meta name="viewport" content="width=device-width, initial-scale=1.0">
<title>Self-hosted LiveSync Setup QR Aggregator</title>
<style>
body { font-family: sans-serif; display: flex; flex-direction: column; align-items: center; justify-content: center; height: 100vh; margin: 0; background-color: #f4f4f9; color: #333; }
.container { background: white; padding: 2rem; border-radius: 8px; box-shadow: 0 4px 6px rgba(0,0,0,0.1); text-align: center; max-width: 90%; }
.progress { margin: 20px 0; font-size: 1.2rem; font-weight: bold; }
.status { margin-bottom: 20px; color: #666; }
.btn { display: inline-block; padding: 12px 24px; background-color: #7c4dff; color: white; text-decoration: none; border-radius: 4px; font-weight: bold; transition: background-color 0.2s; border: none; cursor: pointer; }
.btn:hover { background-color: #651fff; }
.btn:disabled { background-color: #ccc; cursor: not-allowed; }
.grid { display: grid; grid-template-columns: repeat(auto-fit, minmax(40px, 1fr)); gap: 8px; margin: 20px 0; }
.tile { width: 40px; height: 40px; border: 2px solid #ddd; border-radius: 4px; display: flex; align-items: center; justify-content: center; font-size: 0.8rem; }
.tile.filled { background-color: #7c4dff; color: white; border-color: #7c4dff; }
</style>
</head>
<body>
<div class="container">
<h1>LiveSync Setup</h1>
<div id="app">
<p>Checking hash data...</p>
</div>
</div>
<script>
function updateUI() {
const hash = window.location.hash.substring(1);
const params = new URLSearchParams(hash);
const id = params.get('id');
const total = parseInt(params.get('n') || '0');
const index = parseInt(params.get('i') || '-1');
const data = params.get('d');
const app = document.getElementById('app');
if (!id || total <= 0 || index === -1 || !data) {
app.innerHTML = '<p class="status">Invalid setup URL. Please scan the QR code correctly.</p>';
return;
}
// Get session data
const storageKey = 'ls_agg_' + id;
let session = JSON.parse(localStorage.getItem(storageKey) || '{}');
// Save current data
session[index] = data;
localStorage.setItem(storageKey, JSON.stringify(session));
const receivedIndexes = Object.keys(session).map(Number);
const count = receivedIndexes.length;
let html = `
<div class="status">Session ID: ${id}</div>
<div class="progress">${count} / ${total} Loaded</div>
<div class="grid">
`;
for (let i = 0; i < total; i++) {
const isFilled = session[i] !== undefined;
html += `<div class="tile ${isFilled ? 'filled' : ''}">${i + 1}</div>`;
}
html += `</div>`;
if (count === total) {
const sortedData = Array.from({length: total}, (_, i) => session[i]).join('');
// Use the correct protocol for settings
const obsidianUri = `obsidian://setuplivesync?settingsQR=${sortedData}`;
html += `
<p>All parts have been collected!</p>
<a href="${obsidianUri}" class="btn">Open Obsidian to complete setup</a>
<p style="margin-top:20px; font-size:0.8rem; color: #999;">Note: If the button does not respond, please ensure you are opening this in a browser that can trigger Obsidian.</p>
`;
} else {
html += `
<p class="status">Please scan the next QR code.</p>
<button class="btn" disabled>Waiting...</button>
`;
}
app.innerHTML = html;
}
window.addEventListener('hashchange', updateUI);
updateUI();
</script>
</body>
</html>
+308
View File
@@ -0,0 +1,308 @@
# Self-hosted LiveSync Development Guide
## Project Overview
Self-hosted LiveSync is an Obsidian plugin for synchronising vaults across devices using CouchDB, MinIO/S3, or peer-to-peer WebRTC. The codebase uses a modular architecture with TypeScript, Svelte, and PouchDB.
## Build & Development Workflow
### Environment Setup
#### First-time Setup
```bash
git clone https://github.com/vrtmrz/obsidian-livesync
cd obsidian-livesync
npm ci
npm run build
```
#### Branch switching
When switching branches, reinstall dependencies when the lockfile changes.
```bash
git checkout 0.25.70-patch1 # tag or branch name
npm ci
npm run build
```
### Commands
```bash
npm run test:unit # Run unit tests with vitest (or `npm run test:unit:coverage` for coverage)
npm run check # TypeScript and svelte type checking
npm run dev # Development build with auto-rebuild (uses .env for test vault paths)
npm run build # Production build
npm run buildDev # Development build (one-time)
npm run test:unit # Run unit tests only (no Docker services required)
npm test # Run Harness based vitest tests (requires Docker services), not recommended, unstable.
```
### Tips
Use CLI E2E tests or real Obsidian E2E scripts instead of `npm test` when the behaviour can be verified outside the browser harness.
### Unreleased change notes
Keep changes that may belong in a future release under `## Unreleased` at the top of `updates.md` when they do not justify an immediate release. Do not add a date to this virtual version. Move relevant entries under the real version and ordinal release date when preparing that release, then leave an empty `## Unreleased` section for subsequent work.
Use this section for durable release-note candidates, including compatibility-relevant internal maintenance, rather than tasks, local diagnostics, or implementation journals. Categorise user-visible behaviour separately from internal changes and testing.
### Auto-copy to test vaults
To facilitate development and testing, the build process can automatically copy the built plugin to specified test vault
- Create `.env` file with `PATHS_TEST_INSTALL` pointing to test vault plug-in directories (`:` separated on Unix, `;` on Windows)
- Development builds auto-copy to these paths on build whilst `npm run dev` is running (watch mode)
### Testing Infrastructure
- ~~**Deno Tests**: Unit tests for platform-independent code (e.g., `HashManager.test.ts`)~~
- This is now obsolete, migrated to vitest.
- **Vitest**:
- **Unit Tests** (`vitest.config.unit.ts`): Unit tests run in Node.js (excluding harnesses and integration tests). Unit tests should be `*.unit.spec.ts` and placed alongside the implementation file (e.g., `ChunkFetcher.unit.spec.ts`). Executed via `npm run test:unit`.
- **Integration Tests** (`vitest.config.integration.ts`): Tests run in Node.js against a real CouchDB instance. Integration tests should be `*.integration.spec.ts` or `*.integration.test.ts` and placed alongside the implementation file (e.g., `StreamingFetch.integration.spec.ts`). Executed via `npm run test:integration`.
- If you add a feature that interacts with the remote database (e.g., replication changes, custom changes feed parameters, or custom HTTP queries), you strongly expected to write an integration test to verify the behaviour against a real CouchDB server.
- **Browser Harness Tests** (`vitest.config.ts`): Transitional browser-based harness tests using Playwright/Chromium. Executed via `npm run test`. This layer is no longer the preferred destination for new broad E2E coverage because `test/harness` can drift from real Obsidian behaviour.
- **P2P Tests** (`vitest.config.p2p.ts`): Browser-based Peer-to-Peer replication tests. Executed via `npm run test:p2p`.
- **Commonlib Tests**: Commonlib owns unit and package tests for shared RPC, storage, replication, and platform contracts. LiveSync CI verifies the exact packed dependency as a downstream consumer.
- **Real Obsidian E2E** (`test/e2e-obsidian/`): Local-first scripts that launch real Obsidian with temporary vaults and the built Self-hosted LiveSync plug-in. Use these for boot-up sequence, vault reflection, RedFlag flows, Fast Setup (Simple Fetch), settings dialogues, restart-sensitive workflows, Object Storage regressions, and other behaviour that depends on Obsidian itself. Run focused scripts such as `npm run test:e2e:obsidian:two-vault-sync`, or use `npm run test:e2e:obsidian:local-suite:services` to run the broader local suite with CouchDB and MinIO fixtures managed by the wrapper.
- **Docker Services**: Tests require CouchDB, MinIO (S3), and P2P services:
```bash
npm run test:docker-all:start # Start all test services
npm run test:full # Run tests with coverage
npm run test:docker-all:stop # Stop services
```
If some services are not needed, start only required ones (e.g., `test:docker-couchdb:start`).
Note that if services are already running, starting script will fail. Please stop them first.
- **Test Structure**:
- `test/e2e-obsidian/` - Real Obsidian E2E scripts for local verification
- `test/suite/` - Transitional browser harness tests for sync operations
- `test/unit/` - Unit tests (via vitest, as harness is browser-based)
- `test/harness/` - Transitional mock implementations (e.g., `obsidian-mock.ts`). Avoid adding broad new E2E coverage here unless it is explicitly a compatibility bridge.
### Import Path Normalisation
The codebase uses the `@/` alias for source owned by this repository. Commonlib imports use explicit `@vrtmrz/livesync-commonlib` package subpaths. To normalise LiveSync-owned imports and exports, use the following utility script:
```bash
npm run pretty:importpath
```
Under the hood, this runs Deno with the script [utilsdeno/normalise-imports.ts](utilsdeno/normalise-imports.ts). You can pass additional flags to this script if required (by running it via Deno directly from the `utilsdeno` directory):
- `--run`: Applies the changes (the script runs in dry-run mode by default).
- `--all-alias`: Normalises sibling/child relative imports starting with `./` to use aliases.
### Commonlib dependency
Shared synchronisation code is compiled and typed by the `@vrtmrz/livesync-commonlib` package. `npm ci` installs the exact artefact recorded by the lockfile; this repository does not compile Commonlib source or commit fallback declarations.
Changes spanning both repositories must first produce a packed Commonlib artefact which passes its standalone package checks. Install that exact artefact in LiveSync, then run the LiveSync type checks, unit tests, application builds, CLI E2E, and any focused real-Obsidian E2E required by the changed boundary. Replace the temporary artefact reference with the reviewed immutable package version before release.
## Architecture
### Module System
The plugin uses a dynamic module system to reduce coupling and improve maintainability:
- **Service Hub**: Central registry for services using dependency injection
- Services are registered, and accessed via `this.services` (in most modules)
- **Module Loading**: All modules extend `AbstractModule` or `AbstractObsidianModule` (which extends `AbstractModule`). These modules are loaded in main.ts and some modules.
- **Module Categories** (by directory):
- `core/` - Platform-independent core functionality
- `coreObsidian/` - Obsidian-specific core (e.g., `ModuleFileAccessObsidian`)
- `essential/` - Required modules (e.g., `ModuleMigration`, `ModuleKeyValueDB`)
- `features/` - Optional features (e.g., `ModuleLog`, `ModuleObsidianSettings`)
- `extras/` - Development/testing tools (e.g., `ModuleDev`, ~~`ModuleIntegratedTest`~~)
- **Services**: Core services (e.g., `database`, `replicator`, `storageAccess`) are registered in `ServiceHub` and accessed by modules. They provide an extension point for add new behaviour without modifying existing code.
- For example, checks before the replication can be added to the `replication.onBeforeReplicate` handler, and the handlers can be return `false` to prevent replication-starting. `vault.isTargetFile` also can be used to prevent processing specific files.
- **ServiceModule**: A new type of module that directly depends on services.
#### Note on Module vs Service
After v0.25.44 refactoring, the Service will henceforth, as a rule, cease to use setHandler, that is to say, simple lazy binding. - They will be implemented directly in the service. - However, not everything will be middlewarised. Modules that maintain state or make decisions based on the results of multiple handlers are permitted.
Hence, the new feature should be implemented as follows:
- If it is a simple extension point (e.g., adding a check before replication), it should be implemented as a handler in the service (e.g., `replication.onBeforeReplicate`).
- If it requires maintaining state or making decisions based on multiple handlers, it should be implemented as a serviceModule dependent on the relevant services explicitly.
- If you have to implement a new feature without much modification, you can extent existing modules, but it is recommended to implement a new module or serviceModule for better maintainability.
- Refactoring existing modules to services is also always welcome!
- Please write tests for new features, you will notice that the simple handler approach is quite testable.
### Key Architectural Components
- **LiveSyncLocalDB** (`@vrtmrz/livesync-commonlib/compat/pouchdb/LiveSyncLocalDB`): Local PouchDB database wrapper
- **Replicators** (`@vrtmrz/livesync-commonlib/compat/replication/*`): CouchDB, Journal, and P2P synchronisation engines
- **Service Hub** (`src/modules/services/`): Central service registry using dependency injection
- **Common Library** (`@vrtmrz/livesync-commonlib`): Platform-independent synchronisation logic, shared with the CLI, Webapp, WebPeer, and external tools
### Conflict Merge Policy
Markdown conflict auto-merge should behave like a conservative three-way merge. The guiding rule is to merge changes when they touch non-overlapping regions, and to keep a manual conflict when the edits overlap semantically.
When in doubt, prefer the safer outcome: preserve data, keep the conflict visible, and ask the user rather than silently discarding content or choosing one side.
- If one side deletes a line and the other side leaves that same line unchanged, treat it as a safe deletion. The deleted line must not be reintroduced into the merged result.
- If one side inserts new content in a different region while the other side deletes an unchanged old region, preserve the insertion and the deletion.
- If one side deletes a line and the other side modifies that same line, keep the conflict for user resolution.
- If both sides insert different content at the same position, keep both insertions in a deterministic order unless the surrounding deletion context indicates that they are competing replacements.
- Avoid resolving conflicts by simply choosing the newest revision unless the user has explicitly selected that behaviour.
This policy is intentionally aligned with the conflict checkboxes and compatibility settings: automatic merge should remove avoidable prompts, but it must not silently choose between overlapping user intentions.
### File Structure Conventions
- **Platform-specific code**: Use `.platform.ts` suffix (replaced with `.obsidian.ts` in production builds via esbuild)
- **Development code**: Use `.dev.ts` suffix (replaced with `.prod.ts` in production)
- **Path aliases**: `@/*` maps to `src/*`; Commonlib uses package exports rather than a source alias
## Code Conventions
### Internationalisation (i18n)
- **Translation Workflow**:
1. Edit the human-readable YAML files in `src/common/messagesYAML/` in the `livesync-commonlib` repository
2. Run `npm run i18n:bake` in that repository to compile YAML → JSON → TypeScript constants
3. Use `$t()`, `$msg()` functions for translations
You can also use `$f` for formatted messages with Tagged Template Literals.
- **Usage**:
```typescript
$msg("dialog.someKey"); // Typed key with autocomplete
$t("Some message"); // Direct translation
$f`Hello, ${userName}`; // Formatted message
```
- **Supported languages**: `def` (English), `de`, `es`, `ja`, `ko`, `ru`, `zh`, `zh-tw`
### File Path Handling
- Use tagged types from `types.ts`: `FilePath`, `FilePathWithPrefix`, `DocumentID`
- Prefix constants: `CHeader` (chunks), `ICHeader`/`ICHeaderEnd` (internal data)
- Path utilities are supplied by the focused Commonlib compatibility path `@vrtmrz/livesync-commonlib/compat/string_and_binary/path`
### Logging & Debugging
- Use `this._log(msg, LOG_LEVEL_INFO)` in modules (automatically prefixes with module name)
- Log levels: `LOG_LEVEL_DEBUG`, `LOG_LEVEL_VERBOSE`, `LOG_LEVEL_INFO`, `LOG_LEVEL_NOTICE`, `LOG_LEVEL_URGENT`
- LOG_LEVEL_NOTICE and above are reported to the user via Obsidian notices
- LOG_LEVEL_DEBUG is for debug only and not shown in default builds
- Dev mode creates `ls-debug/` folder in `.obsidian/` for debug outputs (e.g., missing translations)
- This causes pretty significant performance overhead.
## Common Patterns
### Module Implementation (Now not recommended for new features, use services instead)
```typescript
export class ModuleExample extends AbstractObsidianModule {
async _everyOnloadStart(): Promise<boolean> {
/* ... */
}
onBindFunction(core: LiveSyncCore, services: typeof core.services): void {
services.appLifecycle.handleOnInitialise(this._everyOnloadStart.bind(this));
}
}
```
### Settings Management
- Settings are defined by Commonlib (`ObsidianLiveSyncSettings`)
- Configuration metadata is supplied by the Commonlib settings exports
- Use `this.services.setting.saveSettingData()` instead of using plugin methods directly
### Database Operations
- Local database operations through `LiveSyncLocalDB` (wraps PouchDB)
- Document types: `EntryDoc` (files), `EntryLeaf` (chunks), `PluginDataEntry` (plugin sync)
## Important Files
- [main.ts](src/main.ts) - Plugin entry point, module registration
- [esbuild.config.mjs](esbuild.config.mjs) - Build configuration with platform/dev file replacement
- [package.json](package.json) - Scripts reference and dependencies
## Beta Policy
- Beta versions are denoted by appending `-patchedN` to the base version number.
- `The base version` mostly corresponds to the stable release version.
- e.g., v0.25.41-patched1 is equivalent to v0.25.42-beta1.
- This notation is due to SemVer incompatibility of Obsidian's plugin system.
- Hence, this release is `0.25.41-patched1`.
- Each beta version may include larger changes, but bug fixes will often not be included.
- I think that in most cases, bug fixes will cause the stable releases.
- They will not be released per branch or backported; they will simply be released.
- Bug fixes for previous versions will be applied to the latest beta version.
This means, if xx.yy.02-patched1 exists and there is a defect in xx.yy.01, a fix is applied to xx.yy.02-patched1 and yields xx.yy.02-patched2.
If the fix is required immediately, it is released as xx.yy.02 (with xx.yy.01-patched1).
- This procedure remains unchanged from the current one.
- At the very least, I am using the latest beta.
- However, I will not be using a beta continuously for a week after it has been released. It is probably closer to an RC in nature.
In short, the situation remains unchanged for me, but it means you all become a little safer. Thank you for your understanding!
## Release Notes
- Keep the top section of `updates.md` as `## Unreleased` during normal development.
- When opening a feature or fix PR, update `## Unreleased` in the same PR if the change is user-facing.
- Add only user-facing changes that help users understand what they gain, what has changed, or what they may need to do after updating.
- Avoid listing purely internal refactors, maintenance chores, generated-file changes, and dependency updates unless they affect users; group and label them when they are included.
- When preparing a release, replace `## Unreleased` with the target version heading (for example, `## 0.25.81`) and add a fresh empty `## Unreleased` section above it for the next cycle.
- Review and polish the released section in the release PR before tagging, because the content is embedded into the plug-in and may be reused as the GitHub Release notes.
## Release Workflow
This workflow is for maintainers. Contributors should update `## Unreleased` for user-facing feature or fix PRs, but do not need to run the release workflows.
The `Finalise Release Tags` and `Release Obsidian Plugin` workflows use the `release` GitHub Environment. Configure Environment protection in the repository settings so tag creation and release publication require maintainer approval.
- Run the `Prepare Release PR` workflow with the target version. It creates the release branch, updates versions, confirms that Commonlib is locked to an immutable package version, moves the `## Unreleased` notes to the target version, commits the release preparation, pushes the branch, and opens a draft release PR.
- Do not tag the release branch when the PR is first created. Polish the release PR first, especially `updates.md`.
- Once the release PR head is fixed, run the `Finalise Release Tags` workflow with its full head commit SHA. It validates the release branch, ensures that both the plug-in tag (for example, `0.25.81`) and the CLI tag (for example, `0.25.81-cli`) point to that commit, and explicitly dispatches the plug-in and CLI publishing workflows. The workflow can be retried when existing tags already point to the reviewed commit, but stops if either tag points elsewhere.
- The plug-in publishing workflow is intentionally dispatch-only. Pushing a plug-in tag directly does not publish a GitHub Release; use `Finalise Release Tags`, or dispatch `Release Obsidian Plugin` explicitly for recovery or a pre-release. The CLI Docker workflow retains its documented branch, tag, and manual triggers.
- Approve the `Release Obsidian Plugin` workflow for the `release` environment, then inspect the generated draft GitHub Release. Confirm that the CLI workflow has published the fixed version tag, the major-minor moving tag (for example, `0.25-cli`), `latest`, and the SHA-qualified tag.
- Publish the draft GitHub Release as the latest stable release while keeping the release PR in draft and leaving `main` unchanged. Record the state in the PR with: 'Release `<version>` has been published as the latest stable release. This pull request intentionally remains in draft, and `main` has not yet been updated. Merge is on hold until BRAT validation is complete.'
- Validate the published release through BRAT. Confirm start-up, ordinary bidirectional synchronisation, and any regression scenario relevant to the release.
- After BRAT validation succeeds, mark the release PR ready and merge it with a merge commit. This keeps the tagged release commit in the `main` history.
- If BRAT validation fails, keep the release PR in draft. Do not move published tags; prepare and publish a new patch release instead.
- If a pre-release is needed, run the `Release Obsidian Plugin` workflow manually with the target tag, `draft=false`, and `prerelease=true`.
### Release Cheat Sheet
1. Before starting, add user-facing notes under `## Unreleased` in `updates.md`.
2. Run `Prepare Release PR` from GitHub Actions.
- `version`: the target version, for example `0.25.81`.
- `base_branch`: normally `main`.
- `release_branch`: leave blank to use the default branch name, for example `0_25_81`.
- `release_date`: use an ordinal date such as `14th July, 2026`, or leave blank to use the current UTC date.
- `allow_empty_updates`: leave disabled unless the release intentionally has no user-facing notes.
3. Review the generated draft PR.
- Polish `updates.md`.
- Confirm `package.json`, `manifest.json`, `versions.json`, workspace package versions, and the locked Commonlib package version.
- Confirm that `manifest.json` has the intended `minAppVersion`.
- Wait for the necessary CI checks.
4. When the PR head is fixed, run `Finalise Release Tags`.
- `version`: the same target version.
- `release_branch`: leave blank unless the release branch used a custom name.
- `expected_head_sha`: the full head commit SHA reviewed in the release PR.
5. Approve the `Release Obsidian Plugin` workflow for the `release` environment, then check the generated draft GitHub Release.
6. Confirm that the explicitly dispatched CLI Docker workflow has published the expected image tags.
7. Publish the draft GitHub Release as the latest stable release, but keep the release PR in draft and leave `main` unchanged.
8. Update the PR state message to say that the release is now the latest stable release and that merging is intentionally on hold until BRAT validation is complete.
9. Validate the published release through BRAT, including start-up, ordinary bidirectional synchronisation, and any release-specific regression scenario.
10. After BRAT validation succeeds, mark the release PR ready and merge it into `main` with a merge commit.
11. If validation fails, leave the PR in draft and prepare a new patch release without moving the published tags.
12. If the release should be a pre-release instead of a draft release, run `Release Obsidian Plugin` manually with the target `tag`, `draft=false`, and `prerelease=true`.
## Contribution Guidelines
- Follow existing code style and conventions
- Write integration tests (`*.integration.spec.ts` or `*.integration.test.ts`) when adding or modifying features that interact with the remote database, and ensure that they pass in the CI workflow.
- Please bump dependencies with care, check artifacts after updates, with diff-tools and only expected changes in the build output (to avoid unexpected vulnerabilities).
- When adding new features, please consider it has an OSS implementation, and avoid using proprietary services or APIs that may limit usage.
- For example, any functionality to connect to a new type of server is expected to either have an OSS implementation available for that server, or to be managed under some responsibilities and/or limitations without disrupting existing functionality, and scope for surveillance reduced by some means (e.g., by client-side encryption, auditing the server ourselves).
+45
View File
@@ -0,0 +1,45 @@
# For details and other explanations about this file refer to:
# https://github.com/vrtmrz/obsidian-livesync/blob/main/docs/setup_own_server.md#traefik
services:
couchdb:
image: couchdb:latest
container_name: obsidian-livesync
user: 1000:1000
environment:
- COUCHDB_USER=username
- COUCHDB_PASSWORD=password
volumes:
- ./data:/opt/couchdb/data
- ./local.ini:/opt/couchdb/etc/local.ini
# Ports not needed when already passed to Traefik
#ports:
# - 5984:5984
restart: unless-stopped
networks:
- proxy
labels:
- "traefik.enable=true"
# The Traefik Network
- "traefik.docker.network=proxy"
# Don't forget to replace 'obsidian-livesync.example.org' with your own domain
- "traefik.http.routers.obsidian-livesync.rule=Host(`obsidian-livesync.example.org`)"
# The 'websecure' entryPoint is basically your HTTPS entrypoint. Check the next code snippet if you are encountering problems only; you probably have a working traefik configuration if this is not your first container you are reverse proxying.
- "traefik.http.routers.obsidian-livesync.entrypoints=websecure"
- "traefik.http.routers.obsidian-livesync.service=obsidian-livesync"
- "traefik.http.services.obsidian-livesync.loadbalancer.server.port=5984"
- "traefik.http.routers.obsidian-livesync.tls=true"
# Replace the string 'letsencrypt' with your own certificate resolver
- "traefik.http.routers.obsidian-livesync.tls.certresolver=letsencrypt"
- "traefik.http.routers.obsidian-livesync.middlewares=obsidiancors"
# The part needed for CORS to work on Traefik 2.x starts here
- "traefik.http.middlewares.obsidiancors.headers.accesscontrolallowmethods=GET,PUT,POST,HEAD,DELETE"
- "traefik.http.middlewares.obsidiancors.headers.accesscontrolallowheaders=accept,authorization,content-type,origin,referer"
- "traefik.http.middlewares.obsidiancors.headers.accesscontrolalloworiginlist=app://obsidian.md,capacitor://localhost,http://localhost"
- "traefik.http.middlewares.obsidiancors.headers.accesscontrolmaxage=3600"
- "traefik.http.middlewares.obsidiancors.headers.addvaryheader=true"
- "traefik.http.middlewares.obsidiancors.headers.accessControlAllowCredentials=true"
networks:
proxy:
external: true
+52
View File
@@ -0,0 +1,52 @@
# Self-hosted LiveSync — Environment Variables
# Copy this file to .env and fill in your values.
# NEVER commit .env to version control.
# =============================================================================
# REQUIRED — CouchDB credentials
# =============================================================================
# Admin username for CouchDB
COUCHDB_USER=admin
# Admin password — use a strong random password (min 16 chars recommended)
COUCHDB_PASSWORD=change_me_use_a_strong_password
# Name of the database the Obsidian plugin will use
COUCHDB_DATABASE=obsidiannotes
# Host port CouchDB is exposed on (default: 5984)
# For tunnel-only deployments you can set this to 127.0.0.1:5984 to block external access
COUCHDB_PORT=5984
# =============================================================================
# PROFILE: caddy (--profile caddy)
# =============================================================================
# Your public domain pointing to this server (A record)
# Example: couchdb.yourdomain.com
COUCHDB_DOMAIN=couchdb.yourdomain.com
# Email for Let's Encrypt TLS certificate notifications
ACME_EMAIL=you@yourdomain.com
# =============================================================================
# PROFILE: tailscale (--profile tailscale)
# =============================================================================
# Tailscale OAuth key (not a regular auth key — must be OAuth for persistent use)
# Generate at: https://login.tailscale.com/admin/settings/oauth
# Scopes needed: devices:write
TS_AUTHKEY=tskey-auth-xxxxxxxxxxxxxxxx-xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
# Hostname this node will have on your tailnet (becomes <hostname>.<tailnet>.ts.net)
TS_HOSTNAME=livesync
# =============================================================================
# PROFILE: cloudflare (--profile cloudflare)
# =============================================================================
# Tunnel token from Cloudflare Zero Trust dashboard
# Create at: https://one.dash.cloudflare.com/ → Networks → Tunnels → Create tunnel
# Copy the token from the "Install connector" step
CF_TUNNEL_TOKEN=eyJhIjoixxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
+349
View File
@@ -0,0 +1,349 @@
# Self-hosted LiveSync — Docker Setup
A fully self-hosted CouchDB stack for the [obsidian-livesync](https://github.com/vrtmrz/obsidian-livesync) plugin.
**No fly.io. No IBM Cloudant. No cloud accounts required for basic use.**
> ✅ **Tested on Docker Desktop for Windows (Docker 29.2, Compose v5, WSL2 backend)** — full init, CORS, auth, and idempotent restart verified.
---
## Architecture
```
Obsidian (desktop / iOS / Android)
│ CouchDB Replication Protocol (HTTPS)
[ Reverse Proxy / Tunnel ] ◄── Choose ONE profile below
[ CouchDB container ] ◄── The only required service
│ initialized once by couchdb-init container
[ Named Docker Volume ] ◄── All vault data stored here
```
---
## Quick Start
### 1. Prerequisites
- [Docker Desktop](https://docs.docker.com/desktop/) (Windows/Mac/Linux) or Docker Engine + Compose plugin
- A machine that Obsidian devices can reach over HTTPS (see profiles below)
### 2. Configure
```bash
cd docker/
cp .env.example .env
# Edit .env — at minimum set COUCHDB_USER and a strong COUCHDB_PASSWORD
```
### 3. Launch
```bash
# Default: CouchDB only (LAN / localhost, no TLS)
docker compose up -d
# With Caddy (public domain + auto Let's Encrypt)
docker compose --profile caddy up -d
# With Tailscale (no domain needed, private mesh or public Funnel)
docker compose --profile tailscale up -d
# With Cloudflare Tunnel (Cloudflare account required)
docker compose --profile cloudflare up -d
```
### 4. Verify
```bash
# Should return {"status":"ok"}
curl -u admin:yourpassword http://localhost:5984/_up
# Check CORS headers
curl -v -H "Origin: app://obsidian.md" \
-u admin:yourpassword \
http://localhost:5984/
```
### 5. Connect Obsidian
In the Obsidian plugin settings (**Self-hosted LiveSync**):
| Field | Value |
|---|---|
| URI | `https://your-domain-or-ts-hostname:5984` (or `http://localhost:5984` for LAN-only) |
| Username | value of `COUCHDB_USER` |
| Password | value of `COUCHDB_PASSWORD` |
| Database name | value of `COUCHDB_DATABASE` (default: `obsidiannotes`) |
| End-to-end passphrase | *your own chosen passphrase — never stored server-side* |
---
## Profile Details
### Default (no profile) — LAN / Localhost only
CouchDB is exposed on `http://localhost:5984` (or LAN IP).
**Desktop Obsidian works over HTTP.** Mobile Obsidian requires HTTPS — use a tunnel profile.
### `--profile caddy` — Public Domain + Auto TLS
**Requires**:
- A domain with an A record pointing to this server's public IP
- Ports 80 and 443 open in your firewall/router
**Set in `.env`**:
```
COUCHDB_DOMAIN=couchdb.yourdomain.com
ACME_EMAIL=you@example.com
```
Caddy automatically issues a Let's Encrypt certificate. No manual cert management.
### `--profile tailscale` — No Domain Required ✅ Recommended for privacy
**Requires**:
- Free [Tailscale account](https://login.tailscale.com/)
- Install the Tailscale app on all your Obsidian devices
- Generate an **OAuth key** at: https://login.tailscale.com/admin/settings/oauth
(Scopes: `devices:write`)
**Set in `.env`**:
```
TS_AUTHKEY=tskey-auth-...
TS_HOSTNAME=livesync
```
**Two sub-modes**:
- **VPN mode** (default): CouchDB accessible only to devices on your Tailnet at
`https://livesync.<tailnet>.ts.net` — completely private
- **Funnel mode**: public HTTPS at `https://livesync.<tailnet>.ts.net` — no domain purchase
Enable in your [Tailscale ACL](https://login.tailscale.com/admin/acls):
```json
"nodeAttrs": [{"target": ["tag:container"], "attr": ["funnel"]}]
```
> **Note on Windows Docker Desktop**: If `/dev/net/tun` is unavailable, add `TS_USERSPACE=true`
> to the tailscale service environment in `docker-compose.yml`.
### `--profile cloudflare` — Cloudflare Tunnel
**Requires**:
- Free [Cloudflare account](https://www.cloudflare.com/)
- A domain managed by Cloudflare DNS (can transfer existing domain for free)
- Cloudflare Zero Trust account (free)
#### Step 1: Create a Cloudflare Tunnel
1. Log in to [Cloudflare Zero Trust](https://one.dash.cloudflare.com/)
2. Navigate to **Networks → Tunnels**
3. Click **Create a tunnel**
4. Choose **Cloudflared** as tunnel type
5. Name your tunnel (e.g., `obsidian-livesync`)
6. Click **Save tunnel**
7. **Copy the tunnel token** — it looks like `eyJhIjoiZX...` (very long, ~400 characters)
#### Step 2: Configure Environment
Edit `docker/.env`:
```env
CF_TUNNEL_TOKEN=eyJhIjoiZX... # Paste the full token from Step 1
COUCHDB_DOMAIN=sync.yourdomain.com # Must be a domain managed by Cloudflare
```
#### Step 3: Add Public Hostname Route
🚨 **CRITICAL**: Token-based tunnels ignore the local `cloudflared.yml` config file. All routing is controlled from the dashboard.
Back in the Zero Trust dashboard, **in the same tunnel creation flow** (or edit your tunnel later):
1. Go to the **Public Hostname** tab
2. Click **Add a public hostname**
3. Configure:
- **Subdomain**: `sync` (or your preferred subdomain)
- **Domain**: Select your Cloudflare domain from dropdown
- **Type**: `HTTP`
- **URL**: `couchdb:5984` ← **Do NOT use `localhost`!**
**Why `couchdb:5984` not `localhost:5984`?**
- The `cloudflared` container runs inside Docker on the same network as `couchdb`
- Docker's internal DNS resolves `couchdb` to the correct container
- Using `localhost` would look inside the `cloudflared` container (nothing there)
4. Under **Additional application settings** (expand):
- **No TLS Verify**: Leave **OFF** (CouchDB uses plain HTTP internally, that's fine)
- Leave other settings at defaults
5. Click **Save hostname**
#### Step 4: Start the Stack
```bash
cd docker/
docker compose --profile cloudflare up -d
```
Verify containers are running:
```bash
docker ps --filter "name=livesync"
```
You should see:
- `livesync-couchdb` — Status: Up (healthy)
- `livesync-cloudflared` — Status: Up
- `livesync-init` — Status: Exited (0)
#### Step 5: Test the Connection
```bash
# Should return 401 Unauthorized (proves CouchDB auth is working)
curl -I https://sync.yourdomain.com
# Should return {"couchdb":"Welcome",...}
curl -u admin:yourpassword https://sync.yourdomain.com
```
If you get **404**, see Troubleshooting below.
#### Step 6: Configure Obsidian Plugin
In Obsidian → Settings → **Self-hosted LiveSync**:
| Field | Value |
|---|---|
| URI | `https://sync.yourdomain.com` |
| Username | value of `COUCHDB_USER` from `.env` |
| Password | value of `COUCHDB_PASSWORD` from `.env` |
| Database name | value of `COUCHDB_DATABASE` from `.env` (default: `obsidiannotes`) |
| End-to-end passphrase | *Choose your own* — never stored server-side |
Under **Remote Database Configuration → Advanced**:
- Enable: ✅ **Use Request API to avoid inevitable CORS problem**
(See "Known Issue" below for why this is critical)
---
#### 🔧 Troubleshooting Cloudflare Tunnel
**Problem: 404 Error / Cloud flare Generic Error Page**
**Diagnosis**:
```bash
# Check if cloudflared is running
docker logs livesync-cloudflared --tail 20
# Look for: "Registered tunnel connection"
# If you see the connector ID, the tunnel is connected but routing is wrong
```
**Fix**: The public hostname rule is missing or incorrect.
1. Go to Zero Trust → Networks → Tunnels → your tunnel → **Edit**
2. Click **Public Hostname** tab
3. Verify a hostname exists with:
- Service Type: `HTTP`
- URL: `couchdb:5984` (NOT `localhost:5984`)
4. If no hostname exists, add it (see Step 3 above)
5. Wait 30 seconds for changes to propagate, then test again
**Problem: Connection immediately closes / 502 Bad Gateway**
**Diagnosis**: CouchDB is not healthy or not on the same Docker network as cloudflared.
```bash
docker ps --filter "name=livesync-couchdb"
# Status should be: Up (healthy)
docker inspect livesync-couchdb -f '{{.NetworkSettings.Networks}}'
# Should show: livesync-net
docker inspect livesync-cloudflared -f '{{.NetworkSettings.Networks}}'
# Should also show: livesync-net
```
**Fix**: If CouchDB is unhealthy, check logs:
```bash
docker logs livesync-couchdb --tail 50
```
**Problem: 524 Timeout Errors During Sync**
**Root cause**: Cloudflare's proxy has a **100-second idle timeout**. CouchDB's replication protocol uses long-polling on the `_changes` feed, which can idle for longer during quiet periods.
**Fix**: Switch to short-polling mode in the Obsidian plugin:
1. Obsidian → Settings → Self-hosted LiveSync
2. **Remote Database Configuration → Advanced**
3. Enable: ✅ **Use Request API to avoid inevitable CORS problem**
4. Save and restart sync
This keeps all requests under 100 seconds.
**Alternative**: Use Tailscale or Caddy profiles instead — neither has aggressive timeouts.
---
## Data & Backup
All vault data lives in the `couchdb-data` Docker named volume.
```bash
# Backup
docker run --rm -v obsidian-livesync_couchdb-data:/data \
-v $(pwd)/backup:/backup alpine \
tar czf /backup/couchdb-backup-$(date +%Y%m%d).tar.gz -C /data .
# Restore
docker run --rm -v obsidian-livesync_couchdb-data:/data \
-v $(pwd)/backup:/backup alpine \
tar xzf /backup/couchdb-backup-20260218.tar.gz -C /data
```
---
## Security Notes
- CouchDB requires authentication for **all** requests (configured by `livesync.ini`)
- Enable **End-to-End Encryption** passphrase in the Obsidian plugin — vault data is
encrypted before it ever leaves your device
- The init container runs once and exits — it has no persistent access
- Never expose CouchDB's admin interface (`/_utils`) to the public internet;
use a firewall rule or the path-based obfuscation trick from
[self-hosted-livesync-server](https://github.com/vrtmrz/self-hosted-livesync-server)
---
## Useful Commands
```bash
# View logs
docker compose logs -f couchdb
docker compose logs couchdb-init
# Re-run init (e.g. after changing credentials)
docker compose restart couchdb-init
# Stop without removing data
docker compose down
# Stop AND remove all data volumes (DESTRUCTIVE)
docker compose down -v
# Open CouchDB admin UI (Fauxton) in browser
open http://localhost:5984/_utils
```
---
## Troubleshooting
| Problem | Solution |
|---|---|
| Init container keeps restarting | CouchDB not healthy yet — wait 30s, check `docker compose logs couchdb` |
| `curl: (52) Empty reply` | CouchDB not fully started — the healthcheck should gate this |
| Mobile can't connect | Needs HTTPS — use tailscale or caddy profile |
| 524 errors with Cloudflare | Enable "Use Request API" toggle in Obsidian plugin |
| `Permission denied` on volumes | Run `docker compose down -v` and retry — first-run volume ownership issue |
| CORS errors in browser | Confirm CouchDB headers: `curl -v -H "Origin: app://obsidian.md" http://localhost:5984/` |
| CouchDB exits immediately, zero logs (Windows) | **Do not add `:ro`** to the `livesync.ini` volume mount. CouchDB's entrypoint runs `chmod 0644` on all files in `/opt/couchdb/etc` — read-only bind mounts cause a silent EPERM crash on Docker Desktop for Windows (WSL2). The compose file is already correct; do not modify it. |
| Settings in `livesync.ini` seem ignored | Settings requiring restart (e.g. bind_address) load at start. Runtime-only settings (require_valid_user, enable_cors) are set by the init container via REST API and take effect immediately without restart. |
+26
View File
@@ -0,0 +1,26 @@
# Caddy config for Self-hosted LiveSync CouchDB
# =============================================================================
# IMPORTANT: CouchDB handles CORS itself.
# Do NOT add CORS headers here — they will conflict with CouchDB's own headers.
# Do NOT intercept OPTIONS requests.
# =============================================================================
{
# Email used for Let's Encrypt certificate notifications
email {$ACME_EMAIL}
}
{$COUCHDB_DOMAIN} {
# Forward all traffic to CouchDB, preserving Host and forwarded-for headers
reverse_proxy couchdb:5984 {
header_up Host {host}
header_up X-Forwarded-For {remote_host}
header_up X-Forwarded-Proto {scheme}
}
# Logging
log {
output stdout
level WARN
}
}
+31
View File
@@ -0,0 +1,31 @@
# cloudflared tunnel configuration for Self-hosted LiveSync
# =============================================================================
#
# Prerequisites:
# 1. Create a tunnel in Cloudflare Zero Trust → Networks → Tunnels
# 2. Copy the tunnel token to CF_TUNNEL_TOKEN in your .env
# 3. Add a public hostname in the tunnel config:
# Hostname : couchdb.yourdomain.com (or whatever you set COUCHDB_DOMAIN to)
# Service : http://couchdb:5984
#
# Known issue: Cloudflare's 100-second proxy timeout can interrupt CouchDB's
# long-polling replication change feed, causing 524 errors.
# MITIGATION: In the Obsidian plugin settings, enable:
# "Use Request API to avoid inevitable CORS problem"
# This switches from long-poll to short-poll mode.
#
# =============================================================================
tunnel: ${CF_TUNNEL_ID}
credentials-file: /etc/cloudflared/credentials.json
ingress:
- hostname: ${COUCHDB_DOMAIN}
service: http://couchdb:5984
originRequest:
# Increase timeouts for CouchDB replication streams
connectTimeout: 30s
keepAliveTimeout: 90s
keepAliveConnections: 100
noTLSVerify: false
- service: http_status:404
+30
View File
@@ -0,0 +1,30 @@
; CouchDB local configuration for Self-hosted LiveSync
; This file is volume-mounted into /opt/couchdb/etc/local.d/livesync.ini
;
; IMPORTANT: Do NOT set require_valid_user here.
; CouchDB needs to start without auth to complete its first-run cluster setup
; (_users, _replicator databases must be created first).
; The couchdb-init service applies auth lockdown via REST API after first-run.
[couchdb]
; Max size per document (50MB). Large enough for binary attachments.
max_document_size = 50000000
[chttpd]
; Bind on all interfaces.
bind_address = 0.0.0.0
port = 5984
; 4 GB max request (handles very large vaults)
max_http_request_size = 4294967296
[httpd]
WWW-Authenticate = Basic realm="couchdb"
[cors]
; These are the exact app origins Obsidian uses on desktop + mobile
credentials = true
origins = app://obsidian.md,capacitor://localhost,http://localhost
[log]
; Reduce noise in Docker logs — set to "debug" if troubleshooting
level = warning
+19
View File
@@ -0,0 +1,19 @@
{
"TCP": {
"443": {
"HTTPS": true
}
},
"Web": {
"${TS_CERT_DOMAIN}:443": {
"Handlers": {
"/": {
"Proxy": "http://127.0.0.1:5984"
}
}
}
},
"AllowFunnel": {
"${TS_CERT_DOMAIN}:443": true
}
}
+187
View File
@@ -0,0 +1,187 @@
# Self-hosted LiveSync — Docker Compose
# =============================================================================
# PROFILES
# --------
# (default) CouchDB only — LAN/localhost access, no TLS
# Suitable for desktop-only use or testing.
#
# --profile caddy CouchDB + Caddy reverse proxy
# Auto TLS via Let's Encrypt. Needs public domain + ports 80/443.
#
# --profile tailscale CouchDB + Tailscale sidecar
# No domain required. HTTPS via *.ts.net PKI.
# Needs a Tailscale account (free tier works).
#
# --profile cloudflare CouchDB + cloudflared tunnel daemon
# Free public HTTPS via Cloudflare. Needs a CF account + tunnel token.
# NOTE: Enable "Use Request API" in the Obsidian plugin to avoid 524 timeouts.
#
# QUICK START (local test):
# cp .env.example .env && edit .env
# docker compose up -d
# curl -u admin:yourpassword http://localhost:5984/_up
# =============================================================================
name: obsidian-livesync
services:
# ---------------------------------------------------------------------------
# CouchDB — the only required service
# ---------------------------------------------------------------------------
couchdb:
image: couchdb:latest
container_name: livesync-couchdb
restart: unless-stopped
# NOTE: Do NOT set user: here — the CouchDB entrypoint starts as root to
# write docker.ini (from env vars), then drops to uid 5984 automatically.
environment:
COUCHDB_USER: ${COUCHDB_USER:?Set COUCHDB_USER in .env}
COUCHDB_PASSWORD: ${COUCHDB_PASSWORD:?Set COUCHDB_PASSWORD in .env}
volumes:
- couchdb-data:/opt/couchdb/data
# Mount to /opt/couchdb/etc/local.ini (NOT into local.d/).
# Do NOT use :ro — the CouchDB entrypoint runs chmod on this file at startup
# and will crash with EPERM if the file is read-only. The file is only read
# at startup; runtime changes go via the REST API into local.d/docker.ini.
- ./config/livesync.ini:/opt/couchdb/etc/local.ini
ports:
# Exposes CouchDB on the host for LAN/localhost access.
# The tunnel profiles (caddy/tailscale/cloudflare) provide HTTPS on top.
# You can remove this port mapping once a tunnel profile is in use.
- "${COUCHDB_PORT:-5984}:5984"
healthcheck:
# Test with admin credentials — ensures both CouchDB is up AND auth is ready.
# ${COUCHDB_USER} / ${COUCHDB_PASSWORD} are expanded by Docker Compose here.
test:
- "CMD-SHELL"
- "curl -sf -u ${COUCHDB_USER}:${COUCHDB_PASSWORD} http://localhost:5984/_session | grep -q ok || exit 1"
interval: 5s
timeout: 5s
retries: 24
start_period: 20s
networks:
- livesync-net
# ---------------------------------------------------------------------------
# One-shot init container — runs couchdb-init.sh after CouchDB is healthy.
# Sets single-node cluster, auth requirements, CORS, size limits, creates DB.
# Restarts on failure (e.g. race at first boot) but won't re-run if already done.
# ---------------------------------------------------------------------------
couchdb-init:
image: curlimages/curl:latest
container_name: livesync-init
restart: on-failure
depends_on:
couchdb:
condition: service_healthy
environment:
COUCHDB_INTERNAL_URL: http://couchdb:5984
COUCHDB_USER: ${COUCHDB_USER}
COUCHDB_PASSWORD: ${COUCHDB_PASSWORD}
COUCHDB_DATABASE: ${COUCHDB_DATABASE:-obsidiannotes}
volumes:
- ./scripts/couchdb-init.sh:/couchdb-init.sh:ro
entrypoint: ["sh", "/couchdb-init.sh"]
networks:
- livesync-net
# ---------------------------------------------------------------------------
# PROFILE: caddy — Caddy reverse proxy with automatic Let's Encrypt TLS
# Requirements: public domain, ports 80 + 443 open to internet
# Usage: docker compose --profile caddy up -d
# ---------------------------------------------------------------------------
caddy:
image: caddy:latest
container_name: livesync-caddy
profiles: [caddy]
restart: unless-stopped
depends_on:
couchdb:
condition: service_healthy
environment:
COUCHDB_DOMAIN: ${COUCHDB_DOMAIN:?Set COUCHDB_DOMAIN in .env for caddy profile}
ACME_EMAIL: ${ACME_EMAIL:?Set ACME_EMAIL in .env for caddy profile}
ports:
- "80:80"
- "443:443"
volumes:
- ./config/Caddyfile:/etc/caddy/Caddyfile:ro
- caddy-data:/data
- caddy-config:/config
networks:
- livesync-net
# ---------------------------------------------------------------------------
# PROFILE: tailscale — Tailscale sidecar for mesh VPN + optional Funnel
# Requirements: Tailscale account (free), OAuth key, Funnel enabled in ACL
# Usage: docker compose --profile tailscale up -d
# The CouchDB port mapping above can be removed for tailscale-only deployments.
# ---------------------------------------------------------------------------
tailscale:
image: tailscale/tailscale:latest
container_name: livesync-tailscale
profiles: [tailscale]
restart: unless-stopped
hostname: ${TS_HOSTNAME:-livesync}
environment:
TS_AUTHKEY: ${TS_AUTHKEY:?Set TS_AUTHKEY in .env for tailscale profile}
TS_STATE_DIR: /var/lib/tailscale
TS_SERVE_CONFIG: /config/serve.json
TS_USERSPACE: "false"
TS_ACCEPT_DNS: "false"
TS_EXTRA_ARGS: ""
volumes:
- tailscale-state:/var/lib/tailscale
- ./config/ts-serve.json:/config/serve.json:ro
- /dev/net/tun:/dev/net/tun
cap_add:
- NET_ADMIN
- SYS_MODULE
# Share CouchDB's network namespace so Tailscale can reach it on localhost
network_mode: service:couchdb
depends_on:
- couchdb
# ---------------------------------------------------------------------------
# PROFILE: cloudflare — cloudflared tunnel daemon
# Requirements: Cloudflare account, tunnel token from CF Zero Trust dashboard
# Usage: docker compose --profile cloudflare up -d
# NOTE: Enable "Use Request API" toggle in the Obsidian LiveSync plugin settings
# to avoid Cloudflare's 100-second proxy timeout (524 errors).
# ---------------------------------------------------------------------------
cloudflared:
image: cloudflare/cloudflared:latest
container_name: livesync-cloudflared
profiles: [cloudflare]
restart: unless-stopped
command: tunnel --no-autoupdate run
environment:
TUNNEL_TOKEN: ${CF_TUNNEL_TOKEN:?Set CF_TUNNEL_TOKEN in .env for cloudflare profile}
volumes:
- ./config/cloudflared.yml:/etc/cloudflared/config.yml:ro
depends_on:
couchdb:
condition: service_healthy
networks:
- livesync-net
# =============================================================================
# Volumes
# =============================================================================
volumes:
couchdb-data:
driver: local
caddy-data:
driver: local
caddy-config:
driver: local
tailscale-state:
driver: local
# =============================================================================
# Networks
# =============================================================================
networks:
livesync-net:
driver: bridge
+79
View File
@@ -0,0 +1,79 @@
#!/bin/sh
# Self-hosted LiveSync — CouchDB Initialization Script
# Runs once on first startup via the couchdb-init service.
# Configures single-node cluster, auth, CORS, and size limits.
set -e
hostname="${COUCHDB_INTERNAL_URL:-http://couchdb:5984}"
username="${COUCHDB_USER:?COUCHDB_USER is required}"
password="${COUCHDB_PASSWORD:?COUCHDB_PASSWORD is required}"
node="${COUCHDB_NODE:-_local}"
echo "==> Waiting for CouchDB at ${hostname} ..."
# _up is publicly accessible (no auth required) — safe pre-auth wait
until curl -sf "${hostname}/_up" 2>/dev/null | grep -q '"status":"ok"'; do
printf '.'
sleep 2
done
echo ""
echo "==> CouchDB is up. Initializing..."
# 1. Enable single-node cluster
curl -sf -X POST "${hostname}/_cluster_setup" \
-H "Content-Type: application/json" \
-d "{\"action\":\"enable_single_node\",\"username\":\"${username}\",\"password\":\"${password}\",\"bind_address\":\"0.0.0.0\",\"port\":5984,\"singlenode\":true}" \
--user "${username}:${password}" && echo "[OK] cluster_setup"
# 2. Require valid user on both http interfaces
curl -sf -X PUT "${hostname}/_node/${node}/_config/chttpd/require_valid_user" \
-H "Content-Type: application/json" -d '"true"' --user "${username}:${password}" && echo "[OK] chttpd/require_valid_user"
curl -sf -X PUT "${hostname}/_node/${node}/_config/chttpd_auth/require_valid_user" \
-H "Content-Type: application/json" -d '"true"' --user "${username}:${password}" && echo "[OK] chttpd_auth/require_valid_user"
# 3. HTTP auth challenge header
curl -sf -X PUT "${hostname}/_node/${node}/_config/httpd/WWW-Authenticate" \
-H "Content-Type: application/json" -d '"Basic realm=\"couchdb\""' --user "${username}:${password}" && echo "[OK] httpd/WWW-Authenticate"
# 4. Enable CORS on both http listeners
curl -sf -X PUT "${hostname}/_node/${node}/_config/httpd/enable_cors" \
-H "Content-Type: application/json" -d '"true"' --user "${username}:${password}" && echo "[OK] httpd/enable_cors"
curl -sf -X PUT "${hostname}/_node/${node}/_config/chttpd/enable_cors" \
-H "Content-Type: application/json" -d '"true"' --user "${username}:${password}" && echo "[OK] chttpd/enable_cors"
# 5. Increase size limits for large vaults
curl -sf -X PUT "${hostname}/_node/${node}/_config/chttpd/max_http_request_size" \
-H "Content-Type: application/json" -d '"4294967296"' --user "${username}:${password}" && echo "[OK] chttpd/max_http_request_size"
curl -sf -X PUT "${hostname}/_node/${node}/_config/couchdb/max_document_size" \
-H "Content-Type: application/json" -d '"50000000"' --user "${username}:${password}" && echo "[OK] couchdb/max_document_size"
# 6. CORS configuration — allow Obsidian app origins
curl -sf -X PUT "${hostname}/_node/${node}/_config/cors/credentials" \
-H "Content-Type: application/json" -d '"true"' --user "${username}:${password}" && echo "[OK] cors/credentials"
curl -sf -X PUT "${hostname}/_node/${node}/_config/cors/origins" \
-H "Content-Type: application/json" \
-d '"app://obsidian.md,capacitor://localhost,http://localhost"' \
--user "${username}:${password}" && echo "[OK] cors/origins"
# 7. Create the vault database if it doesn't exist
db="${COUCHDB_DATABASE:-obsidiannotes}"
set +e
status=$(curl -sf -o /dev/null -w "%{http_code}" --user "${username}:${password}" "${hostname}/${db}" 2>/dev/null)
curl_exit=$?
set -e
if [ "$status" = "200" ]; then
echo "[OK] database '${db}' already exists"
else
curl -sf -X PUT "${hostname}/${db}" --user "${username}:${password}" && echo "[OK] database '${db}' created" || echo "[WARN] database creation returned non-200 — may already exist"
fi
echo ""
echo "==> CouchDB initialization complete!"
echo " URL : ${hostname}"
echo " Database : ${db}"
echo " Username : ${username}"
+34
View File
@@ -0,0 +1,34 @@
# How to add translations
## Getting ready
1. Clone the Commonlib repository, which owns the translation catalogue.
```sh
git clone https://github.com/vrtmrz/livesync-commonlib
```
2. Make `ls-debug` folder under your vault's `.obsidian` folder (as like `.../dev/.obsidian/ls-debug`).
## Add translations for already defined terms
1. Install dependencies, and build the plug-in as dev. build.
```sh
cd livesync-commonlib
npm ci
npm run i18n:bake
```
2. Install the resulting packed Commonlib artefact in a Self-hosted LiveSync development checkout, build the plug-in, copy `main.js` to `.obsidian/plugins/obsidian-livesync` in your test Vault, and run Self-hosted LiveSync.
3. You will get the `missing-translation-yyyy-mm-dd.jsonl`, please fill in new translations.
4. Build the plug-in again, and confirm that displayed things were expected.
5. Add the translations to the Commonlib catalogue, run `npm run i18n:bake`, and make the pull request to `https://github.com/vrtmrz/livesync-commonlib`.
## Make messages to be translated
1. Find the message that you want to be translated.
2. Change the literal to use `$tf`, like below.
```diff
- Logger("Could not determine passphrase to save data.json! You probably make the configuration sure again!", LOG_LEVEL_URGENT);
+ Logger($tf('someKeyForPassphraseError'), LOG_LEVEL_URGENT);
```
3. Make the PR to `https://github.com/vrtmrz/obsidian-livesync`.
4. Follow the steps of "Add translations for already defined terms" to add the translations.
+251
View File
@@ -0,0 +1,251 @@
# Architectural Decision Record: Real Obsidian End-to-End Test Runner
## Status
Proposed / Spike Implemented
## Release
Not yet. Planned after the serviceFeature refactoring branch is reviewed.
## Context
The current end-to-end tests run through Vitest browser mode and a mocked Obsidian environment in `test/harness`. This has been useful for exercising synchronisation flows without launching Obsidian, but it is no longer a reliable final signal for plug-in behaviour.
The main issues are:
- The harness reimplements a large part of the Obsidian API surface, including vault files, workspace events, settings, and lifecycle behaviour. This mock can drift from real Obsidian behaviour without failing.
- The tests run inside a browser-style environment, while the desktop plug-in runs inside Obsidian's Electron environment with its own application lifecycle, storage paths, command registry, and event ordering.
- Several high-value regressions are about integration boundaries: boot-up sequence timing, real vault file reflection, Obsidian command registration, settings persistence, restart prompts, and file watcher behaviour. These are precisely the areas where a mock harness gives weak confidence.
- Maintaining the harness competes with maintaining the plug-in. Adding behaviour to the plug-in often requires teaching the mock another Obsidian detail before the actual regression can be tested.
The current harness should therefore stop being treated as the primary E2E layer.
## Decision
Introduce a new E2E layer that launches real Obsidian with temporary vaults and the built Self-hosted LiveSync plug-in installed into those vaults.
The long-term test pyramid should be:
1. Unit tests for deterministic operations and serviceFeature boundaries.
2. Integration tests for CouchDB, Object Storage, P2P services, database operations, and replication protocols.
3. Real Obsidian E2E tests for boot-up sequence, vault reflection, command registration, settings dialogues, restart scheduling, and user-visible workflows.
The existing `test/harness` should be demoted to a transitional compatibility layer. It may remain temporarily while the real Obsidian runner reaches parity for critical flows, but new high-level E2E coverage should target the real runner.
## Non-Goals
- Do not replace unit or integration tests with slow UI tests.
- Do not keep extending the Obsidian mock to cover new Obsidian APIs unless a short-term compatibility bridge is required.
- Do not require real Obsidian E2E for every pull request initially. The first CI integration should be opt-in or nightly until stability is proven.
- Do not test every setting dialogue through UI clicks if the behaviour is already covered by unit or integration tests. Use UI automation only for workflows whose risk is in real Obsidian integration.
## Proposed Architecture
### Runner
Create a dedicated runner under `test/e2e-obsidian/`.
The runner should:
- Create one or more temporary vault directories.
- Build the plug-in once with `npm run build` or a narrower production build command.
- Install `main.js`, `manifest.json`, and `styles.css` when present into `.obsidian/plugins/obsidian-livesync/`.
- Prepare `.obsidian/community-plugins.json` and `.obsidian/plugins/obsidian-livesync/data.json` as needed.
- Launch Obsidian against the temporary vault.
- Wait until the plug-in reports readiness through a deterministic probe.
- Drive assertions through a narrow control channel rather than fragile visual selectors wherever possible.
- Dispose of Obsidian and temporary vaults after each scenario.
### Obsidian Launch
The preferred desktop target is the installed Obsidian application. The launch mechanism should be platform-specific but hidden behind a small adapter:
- Linux: launch the Obsidian executable with a vault path or Obsidian URI, depending on what is most reliable. If an AppImage is used and FUSE is not available, extract it with `--appimage-extract` and launch the extracted `squashfs-root/obsidian` binary.
- macOS: launch the app bundle through `open` or the executable inside the bundle.
- Windows: launch the installed executable or the registered application protocol.
The first implementation can support Linux only if that is the local and CI target. Cross-platform support can be added after the runner contract is stable.
In headless Linux environments, launch through `xvfb-run`, pass Electron flags such as `--no-sandbox` and `--disable-gpu`, and isolate `HOME`, `XDG_CONFIG_HOME`, and `--user-data-dir` per temporary vault.
### Control Channel
The runner needs a stable way to observe readiness and issue test commands. Prefer a test-only plug-in bridge compiled only in test builds or enabled only by an environment variable.
Possible bridge options:
- The official Obsidian CLI, using the installed `obsidian-cli` helper to open vaults, reload the plug-in, run `eval`, and call developer commands.
- A local HTTP/WebSocket bridge bound to `127.0.0.1` with a random port and token.
- A file-based bridge in the vault, where Obsidian writes status files and consumes command files.
- A DevTools protocol bridge if Obsidian exposes a stable debugging port in the test environment.
The first implementation uses Obsidian's CLI for orchestration and readiness checks. The CLI handles vault opening through `obsidian://open?path=...`, enables community plug-ins through `app.plugins.setEnable(true)`, reloads Self-hosted LiveSync through `plugin:reload id=obsidian-livesync`, and verifies that `app.plugins.plugins['obsidian-livesync']` is loaded.
This keeps E2E-only behaviour out of the production plug-in bundle. The runner should not require Self-hosted LiveSync to write marker files or expose a test server merely to prove that Obsidian loaded it.
The DevTools protocol remains useful for diagnostics. Obsidian's CLI exposes developer commands such as `dev:cdp`, `dev:errors`, and `dev:console`, so the runner should prefer the CLI path first and fall back to direct DevTools attachment only if the CLI cannot provide the required signal.
### Test Data and Services
Keep the existing Docker scripts for CouchDB, MinIO, and P2P services. The real Obsidian runner should reuse these service fixtures instead of creating another service orchestration stack.
Each test should use unique database names, bucket prefixes, vault names, and P2P room IDs. This prevents tests from depending on cleanup and makes interrupted runs less harmful.
## Migration Plan
### Phase 0: Discovery
- Confirm how Obsidian can be launched reliably on the local development environment.
- Confirm whether Obsidian accepts a vault path directly, requires an Obsidian URI, or needs a pre-existing vault registry.
- Identify where Obsidian stores per-user state in the test environment and decide how to isolate it.
- Decide whether the first bridge is file-based or HTTP/WebSocket.
Initial discovery on Linux ARM64 found that:
- `Obsidian-1.12.7-arm64.AppImage` requires `libfuse.so.2` for direct AppImage execution.
- Extracting the AppImage with `--appimage-extract` works without FUSE.
- Launching the extracted `squashfs-root/obsidian` binary under `xvfb-run` with isolated user data stays alive for the smoke timeout.
- No missing shared libraries were reported by `ldd` for the extracted binary in the tested environment.
- Obsidian's CLI is disabled unless the global `obsidian.json` contains `cli: true`.
- Passing only `.obsidian/community-plugins.json` is not enough to load community plug-ins on Obsidian 1.12. The runner also has to enable the global community plug-in switch through `app.plugins.setEnable(true)`.
- The reliable launch sequence is: start Obsidian, send `obsidian://open?path=...` through `obsidian-cli`, wait until the vault-side CLI exposes the plug-in catalogue, enable community plug-ins, reload Self-hosted LiveSync, and verify plug-in readiness through `obsidian-cli eval`.
### Phase 1: Smoke Runner
- Add `test/e2e-obsidian/runner` utilities for temporary vault creation, plug-in installation, launch, readiness wait, and cleanup.
- Add one smoke test:
- launch Obsidian with an empty vault,
- load Self-hosted LiveSync,
- wait for the boot-up sequence to become ready,
- read the plug-in version or status through the control channel,
- close Obsidian cleanly.
- Add an npm script such as `test:e2e:obsidian`.
Current implementation status:
- Added `test/e2e-obsidian/runner` helpers for Obsidian discovery, CLI discovery, temporary vault creation, plug-in installation, process launch, CLI execution, and readiness polling.
- Added `test:e2e:obsidian:discover`, `test:e2e:obsidian:cli-help`, `test:e2e:obsidian:smoke`, `test:e2e:obsidian:vault-reflection`, `test:e2e:obsidian:couchdb-upload`, `test:e2e:obsidian:minio-upload`, `test:e2e:obsidian:startup-scan`, `test:e2e:obsidian:two-vault-sync`, `test:e2e:obsidian:hidden-file-snippet-sync`, `test:e2e:obsidian:customisation-sync`, `test:e2e:obsidian:setting-markdown-export`, `test:e2e:obsidian:local-suite`, `test:e2e:obsidian:local-suite:services`, and `test:e2e:obsidian:install-appimage`.
- Added `startObsidianLiveSyncSession()` so future workflows can reuse the launch, trusted temporary vault state, vault open, community plug-in reload, and readiness sequence without duplicating smoke runner code.
- Added CouchDB runner utilities that reuse `.test.env`/process environment values, create unique temporary databases, query uploaded documents directly, and clean up the database unless `E2E_OBSIDIAN_KEEP_COUCHDB=true` is set.
- Added a manual AppImage installer that downloads Obsidian `1.12.7` for `arm64` or `x86_64`, stores it under `_testdata/obsidian`, and extracts it for FUSE-free execution.
- Confirmed the smoke runner on Linux ARM64 with the extracted Obsidian `1.12.7` AppImage, `xvfb-run`, and the built Self-hosted LiveSync bundle.
- Confirmed the runner can enable the Obsidian CLI through isolated `obsidian.json` state, pre-seed the temporary Chromium local storage so the generated vault ID is trusted for community plug-ins, open the temporary vault through `obsidian-cli`, reload Self-hosted LiveSync, and verify readiness through `obsidian-cli eval`.
- Removed the first test-only ready-marker bridge from the plug-in bundle. The current runner observes readiness from outside the plug-in through Obsidian's own CLI, so normal user vaults do not receive E2E marker files.
Current verification:
- `npm run tsc-check` passes.
- `npm run build` passes with existing Svelte warnings.
- `npm run test:e2e:obsidian:discover` finds `_testdata/obsidian/squashfs-root/obsidian` when the extracted AppImage is present.
- `E2E_OBSIDIAN_SMOKE_TIMEOUT_MS=1000 npm run test:e2e:obsidian:smoke` passes locally.
- `npm run test:e2e:obsidian:vault-reflection` creates a note through Obsidian's vault API, verifies the reflected file on disk, and reads it back through Obsidian.
- `npm run test:e2e:obsidian:couchdb-upload` configures a unique CouchDB database, creates a note through Obsidian, commits it into the local database, runs one-shot synchronisation, and verifies that CouchDB contains the metadata document and all referenced chunk documents.
- `npm run test:e2e:obsidian:minio-upload` configures a unique Object Storage bucket prefix, creates a note through Obsidian, runs one-shot Journal Sync, and verifies through the AWS SDK that objects were written to the S3-compatible bucket.
- `npm run test:e2e:obsidian:startup-scan` verifies that a file written while Obsidian is stopped is picked up during the next real Obsidian boot and uploaded to CouchDB after one-shot synchronisation.
- `npm run test:e2e:obsidian:two-vault-sync` verifies two-vault note synchronisation: creation, update, rename, deletion, per-device target-filter differences, and a separate encrypted round-trip with Path Obfuscation enabled. The experimental Markdown conflict automatic merge check is available with `E2E_OBSIDIAN_INCLUDE_MARKDOWN_CONFLICT=true` but is not part of the default local suite.
- `npm run test:e2e:obsidian:hidden-file-snippet-sync` verifies hidden file synchronisation as a two-vault round-trip: creation, deletion, automatic JSON conflict merging with the merged result propagated by a second synchronisation, manual JSON Resolve dialogue application through Obsidian's UI, and per-device target-pattern differences.
- `npm run test:e2e:obsidian:customisation-sync` verifies a two-vault Customisation Sync workflow: scan a real snippet CSS file, config JSON file, and sample plug-in fixture into per-file Customisation Sync data, synchronise them through CouchDB, apply them on the second vault, assert the resulting `.obsidian` files, propagate a snippet update, and verify deletion of the source-vault snippet sync data without confusing it with the target vault's own applied copy.
- `npm run test:e2e:obsidian:setting-markdown-export` verifies that setting Markdown export creates a vault file and omits credentials when credential export is disabled.
- `npm run test:e2e:obsidian:install-appimage` reuses the existing AppImage and extracted binary when they are already present.
- `npm run test:e2e:obsidian:local-suite` runs the local verification sequence for the real Obsidian runner after CouchDB and MinIO have been started.
- `npm run test:e2e:obsidian:local-suite:services` stops leftover CouchDB and MinIO fixtures, starts fresh fixtures, runs the local suite, and stops the fixtures again.
- `npm run test:e2e:obsidian:local-suite:services` has been verified locally with real Obsidian, CouchDB, and MinIO. The run completed discovery, smoke, vault reflection, CouchDB upload, Object Storage upload, startup scan, two-vault synchronisation, Hidden File Sync, Customisation Sync, and setting Markdown export. The build step still emits existing Svelte warnings.
Known limits:
- The smoke runner currently proves only one-vault launch and plug-in load readiness. Broader workflows are covered by separate real Obsidian scripts, including CouchDB upload, startup scan, two-vault note synchronisation, Hidden File Sync, Customisation Sync, and setting Markdown export.
- Cross-platform support is still discovery-level. The working path has been validated on Linux ARM64. macOS and Windows should be validated in their own environments as follow-up work.
- CI wiring is intentionally not implemented. The runner depends on a licensed desktop application and is treated as a local verification tool.
### Phase 2: First Real Workflow
- Add a one-vault local workflow:
- configure a temporary CouchDB database,
- create a note in the real vault,
- wait for metadata and chunks to be stored,
- restart Obsidian,
- verify that the plug-in loads and the note remains consistent.
This validates real boot-up, settings persistence, vault file access, database writes, and restart-sensitive state.
Current implementation status:
- Added a pre-CouchDB workflow that creates a note through Obsidian's vault API, confirms the note is reflected as a real vault file, and reads the same note back through Obsidian. This covers the vault reflection part of the Phase 2 path before remote database setup is introduced.
- Added a first CouchDB-backed upload workflow, modelled after the CLI Deno tests: reuse the standard CouchDB environment variables, create a unique remote database, apply CouchDB settings through the plug-in's setting service, commit the note through the real Obsidian vault path, run one-shot synchronisation, and assert that remote metadata and chunks exist.
- Added an Object Storage-backed upload workflow against MinIO to exercise Journal Sync and the AWS SDK path from real Obsidian.
- Added Obsidian-specific workflows for boot-time vault scanning, two-vault note synchronisation, hidden `.obsidian/snippets` file round-tripping, hidden JSON conflict resolution, Customisation Sync application for snippets, config JSON files, and plug-in fixtures, per-device target-filter differences, and setting Markdown export. These scenarios assert against CouchDB documents, vault files, or real Obsidian UI outcomes instead of internal service state.
### Phase 3: Two-Vault Synchronisation
- Launch two Obsidian instances with two temporary vaults.
- Configure both against the same temporary remote database.
- Create, modify, rename, and delete notes in one vault.
- Verify reflection in the other vault.
- Cover encrypted and non-encrypted configurations separately.
Current implementation status:
- `test:e2e:obsidian:two-vault-sync` covers creation, update, rename, deletion, and per-device target-filter behaviour for a non-encrypted CouchDB configuration. Markdown conflict automatic merging remains an optional check because it needs a dedicated, less timing-sensitive fixture.
- The same script creates a separate temporary CouchDB database and temporary vault pair for an encrypted two-vault round-trip with Path Obfuscation enabled.
### Phase 4: Harness Retirement
- Mark `test/harness` as deprecated in documentation.
- Stop adding new tests to `test/suite` unless they are explicitly transitional.
- Do not mechanically port `test/suite` into real Obsidian E2E. Scenarios that can already be exercised and asserted through the CLI test layer should stay there or move to lower-level integration tests.
- Prioritise real Obsidian coverage for behaviours that the CLI cannot prove well, especially RedFlag flag-file recovery flows, Fast Setup (Simple Fetch), boot-up sequencing, restart-sensitive initial synchronisation, and user-visible recovery dialogues.
- Remove the harness only after the new runner covers the critical boot-up and synchronisation workflows.
Current implementation status:
- `test/harness` is now documented as a transitional compatibility layer.
- New broad E2E work should target `test/e2e-obsidian/` when real Obsidian behaviour is the risk being tested.
- The next high-value scenarios are RedFlag variants and Fast Setup (Simple Fetch) variants, not a line-by-line migration of `test/suite`.
## Local Verification Strategy
Real Obsidian E2E is a local verification layer. It should not be wired into the default CI gate.
- Keep the scripts individually runnable for focused local debugging.
- Provide `test:e2e:obsidian:local-suite` for a broader local pass after the CouchDB and MinIO fixtures have been started.
- Provide `test:e2e:obsidian:local-suite:services` for a broader local pass that manages the CouchDB and MinIO fixtures itself.
- Use `OBSIDIAN_BINARY` when testing against an installed desktop application.
- Use `test:e2e:obsidian:install-appimage` on Linux when a local AppImage copy is needed, and reuse the extracted `_testdata/obsidian/squashfs-root` directory between local runs.
- Capture Obsidian logs, plug-in logs, vault snapshots, and service logs manually when investigating failures.
- Fail fast on launch failures, readiness timeouts, and cleanup failures with clear diagnostics.
## Risks and Mitigations
- **Obsidian licensing and installation**: Keep the runner local-first and capable of using `OBSIDIAN_BINARY`.
- **Flakiness from UI timing**: Prefer a control channel and service-level probes over visual selectors.
- **Multiple instances**: Obsidian may not support multiple independent instances cleanly on all platforms. Start with one-instance smoke tests, then validate two-instance behaviour on Linux before expanding scope.
- **State leakage**: Isolate vault directories, Obsidian user data, remote database names, and bridge tokens per test.
- **Security of E2E controls**: Keep readiness and control outside the production plug-in bundle. Prefer Obsidian CLI probes over E2E-only plug-in code.
- **Runtime cost**: Keep real Obsidian E2E out of the default PR gate. Use focused scripts or the local suite when a change touches real Obsidian integration.
## Open Questions
- Which launch mechanism is most reliable for Obsidian on each supported desktop platform?
- Can two Obsidian instances run with isolated user data at the same time?
- Do future scenarios need a richer control channel than Obsidian CLI, or can CLI `eval` and developer commands cover the required workflows?
- Should any future E2E-only plug-in code live in a separate test build, or should the production bundle remain free of E2E controls?
- Which RedFlag and Fast Setup (Simple Fetch) variants should be added first?
## Initial Implementation Checklist
1. Add an Obsidian launch discovery script that prints the detected executable, version, and launch mode.
2. Add temporary vault and plug-in installation helpers.
3. Add CLI-based plug-in readiness polling.
4. Add `test:e2e:obsidian:smoke` for one-vault plug-in load.
5. Document required local environment variables, especially `OBSIDIAN_BINARY`.
6. Port one CouchDB-backed workflow after the smoke test is stable.
7. Mark `test/harness` as transitional and block new broad E2E work from targeting it.
8. Add the local suite script for broader local verification.
## Consequences
- Real Obsidian E2E becomes the source of truth for plug-in lifecycle and vault integration.
- Unit and integration tests remain the primary fast feedback loops.
- The old browser harness can be deleted once the new runner covers the critical workflows.
- The project will gain slower but higher-confidence tests for the behaviours most likely to differ between mocks and Obsidian itself.
+122
View File
@@ -0,0 +1,122 @@
# Architectural Decision Record: Bounded Remote Activity
## Status
Accepted
## Context
Self-hosted LiveSync performs remote work through more than one path:
- finite, or bounded, replication starts for manual, event-driven, periodic, and start-up synchronisation;
- long-running rebuild uploads, standard fetches, and fast fetches;
- continuous replication keeps a channel open until the application lifecycle stops it; and
- remote chunk fetching can occur independently of replication, for example while reading history.
The existing API request and response counters do not cover every one of these paths. They therefore cannot, by themselves, provide an accurate remote-work indicator.
Long-running finite operations can also be interrupted by platform lifecycle behaviour. A mobile or desktop display may sleep while an operation is in progress. Changing Obsidian to a hidden or minimised state normally invokes the suspension lifecycle, which closes the active replicator. The existing desktop-only `keepReplicationActiveInBackground` setting deliberately provides a broader, opt-in policy for continuous and periodic replication, and should not be made a prerequisite for completing a finite operation.
Screen wake lock and background execution are related platform effects, but they are not equivalent. A screen wake lock prevents display sleep only while the platform and document visibility permit it. It does not guarantee background execution, prevent operating-system suspension, or override an explicit system sleep action.
## Decision
`ReplicatorService` owns a reactive `boundedRemoteActivityCount` and a `runBoundedRemoteActivity` closure boundary.
The boundary has the following contract:
- increment the count immediately before running a finite remote task;
- run the task through an optional host-provided activity runner;
- decrement the count in `finally`, including when the task rejects;
- allow overlapping tasks, so transitions may be `0 → 1 → 2 → 1 → 0`; and
- count logical operations, not physical connections, sockets, HTTP requests, queued work, or retry delays outside the bounded task.
`ReplicatorService` also owns a narrower `finiteReplicationActivityCount`. Callers enter it through the typed `runFiniteReplicationActivity` method; diagnostic labels do not determine behaviour. The narrower count describes operations which may still place replicated documents in the local database; it excludes rebuilds, chunk-fetch claims, and other bounded work which cannot satisfy an arbitrary missing-chunk read. It is a delivery-lifecycle capability, not a second Wake Lock policy or a connection counter.
The Obsidian host injects the screen wake-lock manager from the `octagonal-wheels` package in the Fancy Kit monorepo as the activity runner on both mobile and desktop. Unsupported or rejected wake-lock requests remain best effort and do not prevent the remote task from running. The manager is disposed when the plug-in unloads.
Finite replication enters both counts only after readiness checks have succeeded and leaves them after `openReplication(..., continuous: false, ...)` settles. A successful completion has reached the latest sequence in that operation's scope and is therefore an authoritative quiescence boundary for chunk retrieval. A failed operation does not prove latest state, but can no longer deliver documents from that attempt. Failure handling runs afterwards so a mismatch or recovery dialogue does not retain the activity. This includes the direct start-up synchronisation path as well as manual, event-driven, and periodic calls through `ReplicationService`. The unbounded continuous channel does not enter either boundary, but its finite initial pull-only catch-up does; the one-shot parameter fallback chain remains inside that boundary.
Manual P2P commands which bypass `ReplicationService` enter the broad boundary. Direct P2P pull and push entry points are therefore both protected as finite remote work, covering the Obsidian panes, CLI, and Webapp. A pull or bidirectional synchronisation also enters the narrower finite-replication boundary because it can place documents in the local database. A push-only request remains broad-only: it cannot satisfy a local missing-chunk read and must not present itself as a delivery source. Automatic synchronisation on peer discovery, a pull requested by a remote peer, and a watched pull following a peer progress notification enter both boundaries because each can deliver local documents. A normal P2P peer-selection dialogue represents one broad finite session: it remains inside the boundary while waiting for a peer and while the person may perform repeated synchronisations, then settles when the dialogue closes and any in-flight synchronisation has finished. Closing without synchronising returns a failed result and releases the boundary. The 'Start Sync & Close' action completes its synchronisation before closing. This deliberately protects peer discovery and selection, because display sleep can interrupt discovery or connection establishment and require the person to start detection again. It may therefore retain a Wake Lock longer than the network transfer alone. A transfer performed inside that session temporarily adds a nested activity; the count remains a logical-operation count rather than a connection total.
`ChunkFetcher` enters the broad boundary synchronously when it accepts newly missing chunk identifiers, but it does not increment the finite-replication count. A typed per-identifier claim keeps the broad boundary active through queue waiting, interval throttling, `fetchRemoteChunks`, validation, local persistence, and terminal event delivery. Duplicate requests share the existing claim. Explicit absence, failure, cancellation, or a conservative five-minute period without fetcher-observable progress settles the affected claim. The five-minute value is only a last-resort leak fuse: it prevents a never-settling integration from retaining the per-identifier claim and waiter indefinitely and, once the activity runner has entered the claim task, lets the associated Wake Lock, lifecycle deferral, and indicator finish. It is not an arrival estimate, proof of remote absence, or a transport deadline. This scope is defined in detail by the chunk-arrival-quiescence ADR.
Rebuild operations use the same boundary at their destructive or remote phase:
- remote rebuild covers settings application, remote reset, and both upload passes, but releases before the completion dialogue;
- rebuild everything covers local and remote reset and both upload passes, but releases before the completion dialogue;
- standard fetch starts after any user confirmation and covers local reset, both download passes, and automatic reflection resumption; and
- fast fetch starts after remote-type selection and covers reset, resumable download retries, reflection resumption when requested, and checkpoint removal. A non-CouchDB fallback enters only the standard-fetch boundary.
Rebuilder-owned confirmation before destructive work and completion dialogues remain outside the boundary so a person cannot hold a Wake Lock indefinitely merely by leaving a dialogue open. P2P peer discovery and selection during a rebuild deliberately remain inside the boundary. They occur after destructive work has begun, and releasing protection at that point would both leave a partial rebuild unprotected and allow display sleep to interrupt peer detection, forcing the person to repeat it. The protected selection period is therefore part of the rebuild operation rather than an incidental confirmation dialogue.
The deprecated database-clean-up workflow also places its connection, one-shot replication, balancing, and remote resolution inside one `database-cleanup` boundary after the user's choice. Its preliminary count and choice dialogue remain outside.
On every platform, a visibility change to hidden defers the normal suspension lifecycle while the bounded count is non-zero. When the last bounded operation ends, the deferred suspension runs if the document is still hidden and the existing desktop background-replication setting does not apply. This does not bypass mobile operating-system restrictions: a hidden document loses its Screen Wake Lock, and the operating system may still pause or terminate the application. LiveSync merely avoids aborting the operation itself while it may still be able to finish.
Fetch rebuilds temporarily suspend file watching. Their visibility event still records the observed hidden state and a pending lifecycle suspension while bounded activity is in progress, without committing or processing file events. A hidden application is suspended after the rebuild boundary ends and can therefore resume normally when it becomes visible. If it becomes visible before the boundary ends, the pending suspension is cancelled and no unmatched resume lifecycle is emitted.
If the desktop background setting applies, its existing continuous or periodic policy remains authoritative. When a Desktop LiveSync window becomes visible during bounded activity, the normal continuous-channel teardown and resume sequence is also deferred until that activity ends, so recovery does not abort the finite operation.
The status bar separates the two meanings. `📲` is shown while the bounded remote activity count is non-zero. It therefore reports a finite logical operation, including periods such as P2P peer selection or chunk-fetch queueing when no request is currently crossing the network. An adjacent `🌐N` reports the approximate number of tracked physical-request units currently in progress. The icon values and the physical indicator's 150 ms minimum display time are named constants so their presentation can be revised without changing the activity contract.
Each physical HTTP attempt owns one balanced counter pair. The request counter is incremented immediately before invoking the selected fetch implementation, and the response counter is incremented in `finally`, whether the attempt returns or rejects. A web-fetch failure followed by the native fallback is two physical attempts and therefore contributes two balanced pairs. Callers must not add another pair around `performFetch`, because duplicated or missing increments leave the status indicator permanently active.
Object Storage contributes one approximate unit for each AWS SDK command issued by its adapter, including upload, download, listing, deletion, availability, and usage requests. A download remains active until its response body has been consumed. This boundary is above the request-handler choice, so it covers both the standard SDK handler and Obsidian's internal request API without double counting. SDK-internal retries remain within one reported command. The displayed value is therefore intentionally approximate and is not an exact count of sockets, HTTP exchanges, or bytes transferred.
P2P does not yet contribute to the physical-request count because it does not have a request unit comparable with CouchDB HTTP attempts or Object Storage SDK commands. Its finite operations remain visible through `📲`. A future P2P transfer metric should be added only when it has a stable meaning, rather than being inferred from the broad logical-operation count.
## Ownership
`ReplicatorService` is the shared ownership point because both `ReplicationService` and `ChunkFetcher` already depend on it. It owns the broad activity count and classifies the semantic subset which represents finite replication. Placing either activity state in `ReplicationService` would make chunk fetching depend in the opposite direction and risk a service dependency cycle. Adding another Service Hub service would introduce a wider capability surface without a distinct lifecycle owner.
The platform activity runner remains injected. Common library and headless consumers can omit it while retaining the same bounded activity count and operation semantics.
## Non-Goals
- Do not count continuous replication as a bounded activity.
- Do not reinterpret the count as an exact number of network connections or HTTP requests.
- Do not use either diagnostic count for replication completion, throttling, protocol correctness, or power-policy decisions.
- Do not claim or implement privileged mobile background execution.
- Do not guarantee protection against operating-system suspension, closing a laptop lid, forced termination, network loss, or a user-initiated sleep action.
- Do not add a lifecycle timeout which would abort an unusually slow rebuild. A genuinely stalled operation may postpone LiveSync's visibility suspension until it settles, but the platform may still suspend or terminate background work.
- Do not broaden `keepReplicationActiveInBackground`; it remains an opt-in desktop policy for continuous and periodic operation after finite work has ended.
- Do not include local storage reflection in this boundary. It is re-entrant and offline-capable, and needs a separate decision if activity reporting or power policy is added later.
## Verification
Unit tests cover:
- overlapping bounded activities and their reactive count transitions;
- count cleanup after rejection;
- entry into the boundary only after replication readiness succeeds;
- replication failure handling occurring after the finite activity ends;
- start-up one-shot replication and continuous start-up's finite pull-only catch-up entering the boundary, while the unbounded live channel does not;
- start-up readiness failure avoiding the boundary;
- direct P2P commands entering the boundary;
- direct P2P pull and push entry points entering the broad boundary, while only pull and bidirectional operations enter the finite-delivery boundary;
- automatic synchronisation on peer discovery, remote pull requests, and watched peer progress entering the boundary;
- P2P peer-selection sessions settling on close, including cancellation, repeated synchronisation, and a close during in-flight work;
- remote chunk fetching remaining inside the shared boundary from synchronous queue acceptance through local persistence and terminal notification;
- missing-chunk waiters rechecking local storage when observed per-identifier claims and finite replication have settled;
- the finite-replication count excluding other bounded work;
- standard, fast, remote, and combined rebuild activity boundaries;
- Rebuilder-owned confirmation and completion dialogues remaining outside rebuild activity;
- fallback from fast fetch avoiding a nested activity boundary;
- fast-fetch reflection resumption and checkpoint removal remaining inside the activity;
- visibility suspension being deferred while bounded activity is in progress on desktop and mobile;
- rebuild-time file-watching suspension preserving the deferred lifecycle action;
- deferred suspension after the final activity ends while the document remains hidden;
- hidden-to-visible transitions before the final activity avoiding an unmatched resume; and
- continuous-channel recovery being deferred when a desktop window becomes visible during bounded activity.
Additional tests cover balanced physical-request counters after success and rejection, each Object Storage command boundary, response-body consumption for downloads, the split `📲` and `🌐N` status labels, and a deterministic real-Obsidian CouchDB request held while the physical indicator is observed. The Object Storage integration and real-Obsidian MinIO workflows verify that actual AWS SDK operations advance and rebalance the shared counters.
The exact Fancy Kit screen wake-lock behaviour is covered by its package and Harness tests. A real Obsidian smoke test remains appropriate when changing the platform adapter or lifecycle integration, but is not required for changes confined to the already-tested injected activity-runner contract.
## Consequences
- One finite activity definition drives Wake Lock, lifecycle protection, and status UI without coupling common library code to Obsidian or browser globals.
- Callers can observe accurate logical activity even in CLI and Webapp hosts which do not inject a Wake Lock implementation.
- Rebuild operations now retain Wake Lock and lifecycle protection across their longest interruption-sensitive phases without retaining them for Rebuilder-owned pre-operation or completion dialogues. Post-reset P2P discovery and selection remain protected as an intentional part of completing the rebuild.
- Users can now distinguish the lifetime of a finite remote operation from approximate request activity within it.
@@ -0,0 +1,144 @@
# Architectural Decision Record: Chunk Arrival Quiescence
## Status
Accepted
## Context
A file metadata document refers to one or more content-addressed chunk documents. LiveSync normally writes the chunks before writing the metadata, but those writes are separate database operations and are not replicated as one atomic unit. A local reader can consequently observe metadata before every referenced chunk is locally readable. This is expected when CouchDB on-demand chunk fetching is enabled, and can also occur around conflict resolution, replication event processing, and historical data.
Before this decision, a missing-chunk waiter used a fixed 5-second or 30-second timeout. Its budget included queueing, throttling, network transfer, validation, local persistence, and event delivery. A healthy operation could therefore time out immediately before it delivered the requested chunk. Conversely, when no operation was capable of delivering the chunk, the timer merely delayed the same unavailable result.
Finite replication provides a stronger boundary than elapsed wall-clock time. In the absence of an error, a finite replication does not complete until it has reached the latest sequence in its scope. Once it completes, that replication cannot deliver another chunk. An on-demand fetch is a separate finite delivery path and needs its own per-identifier boundary.
## Decision
Wait only for a delivery lifecycle which is observable when the local miss is handled. Do not guess how long an unobserved producer might take.
Two lifecycle signals are relevant:
- `finiteReplicationActivityCount` records finite replication operations which can still place database documents in the local database; and
- a `ChunkDeliveryCoordinator` claim records the complete lifetime of an accepted on-demand request for each missing chunk identifier.
The finite replication count is distinct from the broader `boundedRemoteActivityCount`. Rebuilds and other bounded remote work may need Wake Lock and lifecycle protection without being able to satisfy an arbitrary missing-chunk read. A P2P push-only operation is broad for the same reason: it sends documents away but cannot place one in the local database. P2P pull and bidirectional operations are finite-delivery sources. `ReplicatorService` updates the narrower count only through the typed `runFiniteReplicationActivity` boundary; the diagnostic activity label is not used as a behavioural discriminator.
The waiting layer follows these rules:
1. Register the waiter before requesting on-demand delivery.
2. Dispatch `missingChunks` synchronously when direct fetch is permitted. `ChunkFetcher` must claim accepted identifiers before dispatch returns, closing the scheduling gap without a timer.
3. If a matching claim or finite replication is active, wait for that observable producer.
4. A valid chunk arrival resolves the waiter immediately.
5. An explicit remote-missing result resolves it as missing immediately.
6. Once every observed producer completes, read the requested identifier from the local database once more, bypassing the cache.
7. Return the rechecked chunk, or return unavailable. Do not add a fixed grace period after the producer has stopped.
8. If no producer is observable after synchronous dispatch, return unavailable immediately. There is no operation for a duration to represent.
If new relevant activity starts while the final database recheck is pending, that result is stale. The waiter remains active until the newer producer completes and a current recheck finishes.
A successful finite replication completion is the authoritative latest boundary. A failed operation does not prove that the remote lacks the chunk, but it is still terminal for that attempt: it can no longer deliver data. The same local recheck preserves any documents received before the failure, after which normal replication error and retry handling remains responsible for recovery. Missing-chunk code must not misreport that case as an explicit remote absence.
### On-demand fetch boundary
`ChunkFetcher` claims newly requested identifiers synchronously while handling the `missingChunks` event. The claim remains active through:
- queueing and concurrency scheduling;
- configured interval throttling;
- entry into the injected bounded-activity runner;
- `fetchRemoteChunks`;
- response validation;
- local database persistence; and
- fetched or missing event delivery.
The claim settles on every terminal path, including explicit absence, no active replicator, rejection, invalid results, destruction, and cancellation. Its completion Promise is the task passed to the bounded `chunk-fetch` activity, keeping Wake Lock, application lifecycle deferral, the remote-work indicator, and missing-chunk delivery aligned.
### Five-minute leak fuse
An accepted on-demand claim has a separate five-minute inactivity fuse. This is a last-resort leak safety valve, not a chunk-arrival budget or a remote-request timeout.
Its purpose is to prevent a faulty integration, a never-settling Promise, or a stalled transport from retaining logical ownership indefinitely. When it fires, the coordinator releases the per-identifier claim and its waiter. If the bounded activity callback has been entered, resolving the claim also allows the associated Wake Lock, application-lifecycle deferral, and remote-work indicator to be released. `ChunkFetcher` refreshes the fuse only at observable progress points, such as entering the activity boundary, beginning and completing throttling or transfer, and completing persistence.
Five minutes is deliberately a conservative operational limit, not a value derived from a network protocol, a benchmark, or evidence that a missing chunk will arrive within that period. Firing the fuse neither proves remote absence nor makes the underlying request safe to abort. The current `fetchRemoteChunks` contract has no `AbortSignal`, so a physical request may still complete after its logical claim has been released. A future cancellable transport contract should add transport-specific deadlines and explicit cancellation without changing the lifecycle-based wait rule.
### Continuous replication
The unbounded live channel is not a quiescence gate because it has no natural end. Its initial pull-only catch-up is finite, however, and must enter `runFiniteReplicationActivity`. This includes the one-shot parameter fallback chain: every retry remains within the catch-up boundary until it succeeds or stops. If continuous replication later restarts with adjusted parameters, the new initial catch-up enters a new finite boundary.
Once the live channel has begun, a chunk delivered through it still resolves an existing waiter immediately, but the channel itself does not keep a new waiter open. CouchDB on-demand fetching supplies its own per-identifier claim. If a future defect demonstrates a delivery race inside a live batch, that batch lifecycle should be exposed explicitly rather than approximated with another elapsed delay.
## Ownership
`ReplicatorService` owns both the broad bounded-operation count and the narrower finite-replication count. It is the common lifecycle owner for CouchDB, sequential, and P2P replicators.
`LayeredChunkManager` owns one `ChunkDeliveryCoordinator` and supplies it to its arrival layer and `ChunkFetcher`. `ArrivalWaitLayer` owns waiter resolution and the final local database recheck. It depends on the narrow coordinator capability rather than on `ReplicatorService` itself.
`ChunkFetcher` owns per-identifier claims because it knows when each identifier enters its queue and reaches a terminal result.
## Compatibility
- Preserve immediate reads when `waitForDelivery` is false or the deprecated call-site `timeout` is zero or negative.
- Treat a positive deprecated `timeout` only as source-compatible opt-in to lifecycle waiting. Its numeric value no longer represents an arrival duration.
- Preserve `preventRemoteRequest`: no on-demand request is dispatched, although an already-active finite replication may satisfy the waiter.
- Preserve Promise sharing for concurrent reads of the same chunk identifier.
- Preserve immediate explicit remote-missing results.
- Do not change which remote types support direct on-demand fetching.
## Historical Evidence and Scope
This decision addresses the lifecycle-race class rather than treating every Load failed report as a timeout:
- [Issue #166](https://github.com/vrtmrz/obsidian-livesync/issues/166) contained logs where chunk collection failed shortly before related chunk writes appeared. It is evidence for the timing class, although that issue's hidden-file start-up path was repaired separately and is not claimed as a direct regression test here.
- The 2021 timing fixes in [commit `39e2eab0`](https://github.com/vrtmrz/obsidian-livesync/commit/39e2eab0238d9c37e3653cdec884cbeed543fc23) and the extended leaf timeout in [commit `9facb577`](https://github.com/vrtmrz/obsidian-livesync/commit/9facb577601d8aceff7df547cd2a6f9357fdaa29) show that elapsed timeout values have historically been used to absorb the same ordering uncertainty. They do not provide a protocol basis for retaining 5-second or 30-second delays.
- Replication pacing introduced by [commit `8d66c372`](https://github.com/vrtmrz/obsidian-livesync/commit/8d66c372e15c43a2de84a223c6385077b7724eec) and commonlib [commit `051b50c`](https://github.com/vrtmrz/livesync-commonlib/commit/051b50ca38ec4c05a11e8216ac259b4488b825f0) is a direct precedent for preventing replication progress from outrunning chunk collection. The present design expresses that dependency as an explicit lifecycle and completion recheck.
- [Issue #505](https://github.com/vrtmrz/obsidian-livesync/issues/505) was traced to chunks which were genuinely absent after the former bulk-send option broke the chunks-before-metadata guarantee. Waiting cannot recreate missing data, so this decision does not claim to fix it.
- [Issue #771](https://github.com/vrtmrz/obsidian-livesync/issues/771) and [Issue #986](https://github.com/vrtmrz/obsidian-livesync/issues/986) contain ambiguous or version-dependent `Load failed` reports. They remain unclaimed until the original writer and database state can be reproduced.
The detailed setting and replicator matrix is recorded in [Chunk Retrieval and Waiting](../design_docs/chunk_retrieval_and_waiting.md).
## Alternatives Rejected
### Increase the fixed timeout constants
Any fixed total duration can still expire immediately before a queued or progressing operation reports its result. Larger values also make genuine failures slower without defining what the system is waiting for.
### Start another grace period after finite completion
A successful finite replication has already reached its latest sequence, and the completion recheck observes documents persisted without a waiter event. Waiting an additional 5 or 30 seconds has no identified producer to wait for and merely retains the historical approximation.
### Observe all bounded remote activity
The broad count includes operations which cannot provide the requested chunk. Using it as the delivery gate lets unrelated work delay a read and makes its completion semantically meaningless. A separate finite-replication count avoids that leak.
### Keep a fallback timer for unobserved delivery
An unobserved producer has no defined start, progress, or completion semantics. A timer would therefore be a guess rather than a safety property. Relevant delivery paths must claim their work synchronously or expose a finite replication boundary; otherwise the read returns unavailable.
### Remove every timer
The arrival wait has no elapsed timer, but an implementation fault can leave a delivery claim unresolved forever. The five-minute inactivity fuse bounds that leaked logical state without being used as a successful delivery condition.
## Verification
Unit tests use deterministic clocks and deferred Promises to cover:
- a finite replication which lasts well beyond the former arrival values;
- successful finite completion causing a cache-bypassing local database recheck;
- immediate unavailability when no producer is observable;
- per-identifier claims covering queueing, throttling, remote fetch, validation, persistence, and event delivery;
- explicit missing, no-replicator, rejection, invalid response, cancellation, runner rejection, and teardown paths;
- overlapping claims and finite replications;
- a runner which never enters the task and a request which never settles;
- the five-minute fuse being refreshed by observable progress;
- continuous replication's finite initial catch-up, including its parameter fallback path; and
- the setting and replicator decision matrix.
Integration-style unit tests exercise `LayeredChunkManager`, `ChunkFetcher`, a memory-backed PouchDB database, and a deferred fake replicator together. A real Obsidian test is not required because the change remains behind the existing database, service, and event boundaries and does not alter platform UI or an adapter contract.
## Consequences
- A healthy finite replication or on-demand request no longer loses a race against an unrelated wall-clock estimate.
- Successful finite replication completion provides a precise latest boundary for missing-chunk reads.
- The local recheck closes event-delivery and cache timing gaps without extending the wait after completion.
- Reads no longer pause for 5 or 30 seconds when no observable operation can deliver the chunk.
- Relevant producers must expose a lifecycle and must continue to prove cleanup on every exceptional path.
- The five-minute fuse bounds leaked logical activity, but it neither establishes remote absence nor cancels a physical request.
@@ -0,0 +1,336 @@
# Architectural Decision Record: Package the Common Library Behind Explicit Host Boundaries
## Status
Proposed — the implementation proof is complete, `@vrtmrz/livesync-commonlib@0.1.0-rc.2` has been published and verified as the exact downstream dependency, and the Community directory scanner preview confirms the intended package boundary.
## Context
Self-hosted LiveSync currently consumes `livesync-commonlib` as the `src/lib` Git submodule. TypeScript, Vite, Vitest, the CLI, Webapp, and WebPeer resolve `@lib/*` directly to the submodule's TypeScript source. The common-library repository has no package manifest, standalone build, export map, or self-contained test command. Its tests and dependency versions are consequently supplied by the Self-hosted LiveSync repository.
Source archives do not populate a Git submodule. Self-hosted LiveSync therefore also commits generated declarations under `_types` and resolves `@lib/*` to those declarations as a fallback. This makes one logical dependency appear in the plug-in repository twice: once as a submodule checkout and once as generated declarations. Release preparation must regenerate and commit the fallback, and repository scanners can report generated lint directives as if they were maintained plug-in source.
The Obsidian community scanner currently inspects repository source beyond the plug-in entry point. Reports include `src/apps/cli`, `src/apps/webapp`, `src/apps/webpeer`, and generated `_types`. Moving the common library from `src/lib` to another source directory or a workspace package inside the Self-hosted LiveSync repository would therefore preserve the scan surface. It could remove the submodule fallback, but it would not create the same dependency boundary as consuming a published npm package.
The common library is already a substantial shared domain layer. At the time of this decision it contains 270 non-test TypeScript or Svelte source files. At Self-hosted LiveSync revision `e114f66fb2b7c6f3fec2d53701f6638d2557e606`, source outside `src/lib` uses 180 distinct raw `@lib/*` specifiers across plug-in, application, and test source. Normalising optional TypeScript source suffixes leaves 140 migration paths. The plug-in contributes most of those imports, but the CLI, Webapp, and WebPeer also depend on the library. Treating every existing deep path as a permanent public API would make future refactoring impractical.
The intended external use is not new:
- [commonlib issue #1](https://github.com/vrtmrz/livesync-commonlib/issues/1) has requested an npm package and an established API since 2022;
- [Self-hosted LiveSync issue #87](https://github.com/vrtmrz/obsidian-livesync/issues/87) asks for a client with list, get, submit, and delete operations;
- [commonlib issue #10](https://github.com/vrtmrz/livesync-commonlib/issues/10) asks for an API which lets another application add Markdown files to the CouchDB data model; and
- [commonlib issue #13](https://github.com/vrtmrz/livesync-commonlib/issues/13) records the integration cost of consuming a Git submodule and compiling its source.
Existing external consumers use custom TypeScript aliases, loaders, and host stubs to reach `DirectFileManipulator` and other deep modules. The present source shape therefore imposes real integration work without supplying a stable contract.
The library is close to being self-contained, but it crosses its boundary in a small number of important places:
- four files depend on Self-hosted LiveSync's event hub or event identifiers;
- `KeyValueDBService` imports the host's concrete database-opening function;
- `ObsidianServiceContext` imports the plug-in class and Obsidian types from the parent repository; and
- `coreEnvFunctions` imports an Obsidian function type even though the module is intended to be host-neutral.
The library also mixes domain logic with host presentation:
- Svelte dialogue mounting depends on Svelte, LiveSync service context, application-lifecycle events, translation, and cancellation policy in one implementation;
- Obsidian context and setup helpers remain below the common-library directory;
- browser dialogue shims and LiveSync Svelte components are shipped alongside headless replication code; and
- the translation implementation statically imports the complete generated catalogue. The generated message modules and JSON catalogues account for approximately 1.5 MB of source and generated data, even when a headless consumer does not require them.
The root `src/index.ts` currently exports only `DirectFileManipulator` and its options. `DirectFileManipulator` is useful evidence for a future SDK, but it is not yet a sufficient stable façade: initialisation starts from its constructor, enumeration remains unfinished, watch ownership and failure semantics are not documented, and conflict and concurrency semantics are not defined as a public guarantee.
The present process model also assumes one main LiveSync instance. Event dispatch, translation state, environment configuration, offline-scan state, worker pools, synchronisation-parameter handlers, diagnostic counters, and some compatibility caches are module-scoped. Some of these are safe process-wide facilities, while others can allow two client instances to influence each other. Publishing a package makes concurrent clients in one process a supported possibility, so this state must be classified rather than carried across accidentally.
## Decision
Maintain `livesync-commonlib` as the authoritative independent repository and publish its compiled output as the scoped, pre-1.0 npm package `@vrtmrz/livesync-commonlib`.
Self-hosted LiveSync, its CLI, Webapp, WebPeer, and external tools will consume the compiled package and declarations through package export maps. They must not compile the common library's source through a path alias. The package lock records the exact resolved artefact used to build a plug-in release.
The common-library repository may become a small workspace if independently useful artefacts emerge, but it does not move into the Fancy Kit repository or the Self-hosted LiveSync plug-in repository. This preserves its domain ownership, existing history, issues, and independent release cadence while making the plug-in repository a package consumer.
The first package release is an infrastructure and compatibility release, not a declaration that every internal service is stable. It exposes:
- a small documented root API;
- named, task-oriented public subpaths;
- explicitly marked compatibility subpaths required to migrate current Self-hosted LiveSync imports; and
- no unrestricted source-directory wildcard as a permanent contract.
Compatibility subpaths may be exported during the migration, but they remain pre-1.0 and are documented as internal. New external integrations should use the high-level client façade rather than reproduce Self-hosted LiveSync's internal Service Hub composition.
### Domain ownership
The common library continues to own behaviour which defines the Self-hosted LiveSync data and synchronisation model:
- document, metadata, chunk, setting, and protocol types;
- path and identifier encoding;
- chunk splitting, hashing, compression, encryption, and content reconstruction;
- PouchDB-facing data access and replication primitives;
- CouchDB, Object Storage, and P2P replication domain logic;
- conflict, chunk-delivery, storage-event, and replication managers;
- platform-neutral storage and service contracts;
- headless composition; and
- the future high-level client façade.
The Self-hosted LiveSync repository owns plug-in and product integration:
- `ObsidianServiceContext` and every reference to the plug-in class, `App`, `Plugin`, or Obsidian lifecycle;
- Setup Wizard components and LiveSync-specific presentation policy;
- Obsidian menus, notices, settings panes, and dialogue composition;
- plug-in event wiring which is not part of the replication protocol; and
- the concrete initialisation of injected environment, translation, storage, key-value database, and UI capabilities.
Browser-only implementations may remain in the common-library repository only when they implement a documented host-neutral contract used by more than one browser consumer. LiveSync-specific Webapp and WebPeer composition remains application code even if it later moves out of the plug-in repository.
Neutral utilities which do not express LiveSync data, storage, replication, or UI policy may move to `octagonal-wheels` after more than one consumer proves the abstraction. Reusable Obsidian adapters may move to `@vrtmrz/obsidian-plugin-kit`. Neither package becomes an owner of LiveSync domain behaviour.
### Dependency inversion
The package must contain no `@/` import and no direct Obsidian import. The current reverse dependencies are removed as follows:
- common protocol and service events live with their common-library contracts;
- host-only event reactions are registered by Self-hosted LiveSync at its composition root;
- `KeyValueDBService` receives an `openKeyValueDatabase` factory through constructor or service dependencies;
- `ObsidianServiceContext` moves to Self-hosted LiveSync;
- the language getter uses a local function type rather than importing the Obsidian declaration; and
- browser globals, fetch, crypto, timers, storage, and document access are obtained through explicit host capabilities where behaviour varies by runtime.
Temporary package-level configuration is acceptable where converting every call site in one change would be unsafe, but configuration must be instance-scoped wherever multiple clients can coexist. Importing the package must not patch `HTMLElement`, `SVGElement`, or another global prototype. Webapp compatibility patches belong in Webapp bootstrap code.
Every mutable module-level value is classified as one of:
- immutable or safely shared process infrastructure;
- an explicitly keyed cache with bounded ownership and disposal; or
- client state which moves behind an instance-owned service or dependency.
In particular, event subscriptions, translation selection, offline-scan maps and timers, synchronisation-parameter handlers, and database transformation policy must not leak between independent client instances. A high-level client owns an explicit asynchronous disposal path which removes subscriptions, cancels work it owns, and releases its instance state.
### Translation resources
Translation lookup is separated from the generated catalogue. Core services receive a narrow translator capability, or a service-context equivalent, instead of importing a process-global `$msg` implementation which statically owns every language.
The first migration keeps language resources in the common-library repository and exposes them temporarily through the focused compatibility subpath `@vrtmrz/livesync-commonlib/compat/common/i18n`. Importing the root or `context` entry does not load the catalogue. Self-hosted LiveSync imports the catalogue and installs the translator at its composition root. A dedicated language subpath remains the preferred stable replacement once its contract has been narrowed.
This is initially one versioned npm package rather than two independently versioned packages. Message identifiers and their generated catalogue change together, so an immediate package split would add version coordination before the dependency direction is proven. A later `@vrtmrz/livesync-language` package is permitted when independent use or release cadence justifies it. Its dependency must point towards message contracts or core, never from core towards the catalogue.
### Svelte dialogue hosting
The present Svelte dialogue implementation is split into three responsibilities:
1. an Obsidian `Modal` host which owns content, title, close, one-shot settlement, and disposal;
2. a Svelte adapter which mounts and unmounts a component in that host; and
3. Self-hosted LiveSync policy which supplies service context, reacts to plug-in unload, requires explicit cancellation in selected workflows, translates messages, and styles Setup Wizard content.
Fancy Kit is an appropriate owner for the first responsibility when the contract is useful beyond this migration. The Kit API should be framework-neutral: it accepts a mount callback, exposes typed `resolve`, `cancel`, `close`, and `setTitle` controls, and accepts a disposer returned by the callback. It returns a `Promise<TResult | null>` which settles once, and it owns safe-area, viewport, focus, and Obsidian Modal lifecycle guarantees.
Fancy Kit does not add Svelte to its core dependencies for this migration. Self-hosted LiveSync initially owns the small Svelte adapter which calls Svelte's `mount` and `unmount` through the framework-neutral host. The browser dialogue host also remains outside the Obsidian plug-in kit. If a second Obsidian plug-in needs the same Svelte adapter, it can be promoted to an optional Kit subpath with an optional Svelte peer dependency, or to a separate `@vrtmrz/obsidian-svelte-kit` package. That promotion must preserve the existing `sideEffects: false` and focused-import guarantees for consumers which do not use Svelte.
Self-hosted LiveSync's `openWithExplicitCancel` retry policy, application service context, and Setup Wizard components do not move to Fancy Kit.
Arbitrary component mounting is not added to the neutral `UiInteractions` contract. A component instance is a framework and host integration detail rather than a portable interaction request which a generic driver can serialise or answer. A LiveSync workflow which needs a hosted component defines and receives its own narrow, typed dialogue capability; Self-hosted LiveSync composes that capability from the Kit Modal host and its local Svelte adapter.
### Package artefacts
The common-library package publishes compiled ESM and generated declaration files. Consumers do not receive raw TypeScript as the runtime entry point. The package must provide:
- an explicit `exports` map;
- declaration maps where they remain useful for debugging;
- browser and Node entry points where implementations differ;
- accurate `sideEffects` metadata;
- explicit runtime, peer, and optional dependencies;
- no unresolved `@lib/*`, `@/*`, Vite query, or source `.ts` import in published JavaScript or declarations;
- an `npm pack` contents check; and
- clean-install consumer fixtures for Node, a browser bundle, and Self-hosted LiveSync.
Svelte source, worker query imports, AWS SDK adapters, PouchDB adapters, and Node-only crypto must not leak through the root entry point. Optional feature subpaths may carry their own heavier dependency surface.
### Platform host entries
Platform-specific host access is explicit rather than inferred. `@vrtmrz/livesync-commonlib/node` supplies Node-only capabilities and `createNodeStorage({ rootPath })`. `@vrtmrz/livesync-commonlib/browser` supplies `createFileSystemAccessStorage({ rootHandle })` for the browser File System Access API. Browser bundles cannot resolve the Node implementation through the browser entry.
Both storage factories receive an existing root from their host. Commonlib does not choose a process directory, present a browser directory picker, request browser permission, or persist a `FileSystemDirectoryHandle`. The CLI owns path selection and configuration. The Webapp owns user activation, permission, handle persistence, and re-authorisation. The adapters own only path containment and storage operations below the injected root.
Here, the browser capability means the browser File System Access API, not Node's `fs` API. The package proof deliberately moves the rooted `IStorageAdapter` implementation first. The Webapp retains its LiveSync-specific `IFileSystemAdapter` composition, file-object cache, Vault semantics, picker flow, and storage-event policy. Those responsibilities can move later only with their own documented contracts; they are not implied by the low-level browser entry point.
A later composition may place the constructed storage contract on a host-specific `ServiceContext` subtype, so services depend only on the injected capability while each platform owns context initialisation. This proof does not add storage or raw platform objects to the base `ServiceContext`: doing so would make an optional application capability mandatory for Obsidian, CLI, Webapp, WebPeer, and test contexts at the same time. A future change should inject the narrow `IStorageAdapter` contract rather than expose `FileSystemDirectoryHandle`, Node `fs`, or environment detection through the shared context.
The paired adapters run against the same contract suite for metadata, text and binary access, append, listing, removal, missing paths, parent creation, empty-root handling, and traversal rejection. Timestamp fidelity remains platform-specific because the File System Access API does not provide the same creation-time and timestamp-setting facilities as Node.
The Node entry also centralises direct Node built-in access needed by trusted headless application code. This is a package and scanner boundary, not an assertion that Node and browser APIs are interchangeable. Cross-platform behaviour belongs in a shared contract with separate implementations, as demonstrated by rooted storage.
### Standard input and output
Commonlib's `context` entry exposes a narrow `StandardIo` contract for command-line composition. It reads UTF-8 standard input, asks one line-oriented question, and writes text or binary chunks to standard output and standard error without adding delimiters. Commonlib's `node` entry supplies `createNodeStandardIo()`, which binds that contract to host-selected Node streams and defaults to the current process streams.
Self-hosted LiveSync constructs the Node implementation at the CLI composition root and places the exact instance on `NodeServiceContext`. `NodeServiceHub` must receive that Context rather than silently constructing one. Command handlers reach input, prompts, and protocol output through the Context, so unit tests can inject memory I/O without replacing process globals. Obsidian and browser Contexts do not acquire a fictitious terminal capability, and the base `ServiceContextContract` remains limited to capabilities required by every composition.
Standard I/O does not own command-line arguments, exit codes, signals, stream lifecycle, log levels, or log persistence. Diagnostic logging remains the responsibility of the existing API and Logger composition. CLI adapters receive a narrow diagnostic callback wired to the service logging API instead of calling `console` directly. A CLI host may render selected logs on standard error, but Logger is not added to `StandardIo` or made mandatory on the base Context.
Commonlib verifies memory injection, split UTF-8 decoding, object-mode rejection, text and binary output, and prompting against injected Node streams. The downstream CLI verifies Context identity and exact injected I/O in unit tests, then runs the built Node artefact through its Deno command E2E.
### Context and result compatibility
`ServiceContextContract` is the minimum host-neutral composition contract. It supplies an event channel and message translator selected by the host. Obsidian and CLI contexts extend the default implementation with their own capabilities; the Webapp currently uses the default implementation. Every Service Hub and every service in one composition must retain the exact Context object supplied by that host.
Compatibility is established through observable results, not only structural TypeScript compatibility. A shared probe verifies event delivery, translation results, and Context identity across the public Service Hub surface. Commonlib separately verifies that default contexts isolate their event channels and translators. Real Obsidian runs the same invariants against the loaded plug-in through `obsidian-cli eval`; the CLI runs its composition contract in unit tests and then exercises the built Node artefact through Deno E2E; and the Webapp composition runs the shared contract directly without depending on its currently stale Playwright workflow.
The same rule applies as more APIs move: define the shared result set first, run the same cases against each implementation, and document platform-specific behaviour outside that result set. Matching method names or return types alone does not establish behavioural compatibility. The contract runner remains test support during this proof rather than becoming a new public package API.
### Barrels and export surfaces
The migration does not prohibit every barrel. A small root entry point or task-oriented subpath entry point is an intentional package contract when it has one documented responsibility, uses explicit named exports, and does not load unrelated implementations or optional dependencies. The root client entry point, a focused RPC entry point, and type-only storage-adapter entry points are examples which may remain after review.
An existing barrel or forwarding façade is removed when the migration provides a clearer import and the barrel:
- aggregates unrelated domains or exposes implementation layout as API;
- hides a platform, UI framework, worker, database adapter, or another optional dependency;
- makes side effects or tree-shaking behaviour difficult to determine;
- merely re-exports another package without adding a LiveSync-owned contract; or
- exists only to preserve the present `@lib/*` source alias or Service Hub composition.
In particular, the broad `common/types` barrel is retained only as a temporary compatibility path while focused settings, model, protocol, and path contracts are extracted. The `InjectableServices` forwarding barrel is removed in favour of explicit compatibility imports. Forwarding exports for `octagonal-wheels` facilities are removed where consumers can import the owning package directly; mixed modules such as conversion and utility modules are split when this prevents neutral dependencies from being presented as LiveSync-owned API.
Every retained barrel must correspond to an explicit `exports` entry, list named exports rather than use an unrestricted wildcard, and have a packed-consumer or bundle test proving that unrelated optional code is not loaded. Removing a barrel is not sufficient reason to expose every underlying file as a public subpath.
## Migration Plan
### Phase 0: Record and enforce the boundary
- Add a package-boundary check which rejects `@/` and direct Obsidian imports in common-library production source.
- Record the current Self-hosted LiveSync `@lib/*` import inventory and classify each path as public, compatibility-only, host-owned, or obsolete.
- Inventory mutable module-level state and add a two-client isolation test before promising a public client API.
- Add standalone test and type-check configuration to the common-library repository while retaining downstream Self-hosted LiveSync CI.
- Add a packed-consumer fixture before changing Self-hosted LiveSync resolution.
### Phase 1: Remove host leaks
- Move `ObsidianServiceContext` and `setupObsidian` presentation code to Self-hosted LiveSync.
- Inject the key-value database factory and host event reactions.
- Replace the Obsidian language type import with a local contract.
- Move global DOM compatibility mutation to the browser application bootstrap.
- Introduce translator injection and detach core imports from the full catalogue.
- Split the generic dialogue host, Svelte adapter, and LiveSync policy without changing visible behaviour.
### Phase 2: Build the package
- Add the package manifest, compiled ESM build, declarations, export map, and package documentation.
- Keep the root export deliberately small.
- Remove accidental and forwarding barrels where focused imports are clearer; retain only reviewed package entry points with explicit named exports.
- Add temporary compatibility subpaths required by the classified 140-path migration inventory.
- Test Node, browser, worker, PouchDB, Object Storage, P2P, and headless entry points independently.
- Verify that installing a core entry does not pull Svelte UI or language catalogue code into a representative bundle.
### Phase 3: Publish and validate a pre-release
- Publish an immutable pre-1.0 version to a pre-release npm dist-tag.
- Verify its package name, provenance, checksum, export map, and packed files.
- Run common-library tests against the packed artefact rather than the source checkout.
- Run downstream Self-hosted LiveSync type checks, unit tests, integration tests, CLI tests, Webapp tests, WebPeer tests, production builds, and focused real-Obsidian E2E against the exact package version.
- Validate at least one external consumer which currently uses a submodule or custom loader.
### Phase 4: Convert Self-hosted LiveSync
- Replace `@lib/*` source aliases with package imports.
- Remove the `src/lib` submodule, `_types`, `tsconfig.types.json`, `generate-types.mjs`, and release-workflow steps which regenerate fallback declarations.
- Move common-library i18n tooling out of Self-hosted LiveSync release preparation.
- Keep application-owned source under `src/apps` until a separate decision moves an application.
- Run the community scanner's branch or commit preview before releasing the converted plug-in.
### Phase 5: Stabilise the SDK
- Design a high-level asynchronous client around explicit `create`, `list`, `get`, `put`, `delete`, `watch`, and `close` lifecycles.
- Specify path normalisation, encryption negotiation, conflict handling, conditional writes, deletion, history, and resource disposal before declaring the façade stable.
- Adapt `DirectFileManipulator` behind that façade or deprecate it; do not treat its current constructor and deep dependencies as the final API.
- Narrow or remove compatibility-only export paths as Self-hosted LiveSync imports migrate to documented package modules.
## Implementation Proof
The package proof builds Commonlib as one compiled ESM package with a small root, `context`, `settings`, `remote-configurations`, `browser`, `node`, and `rpc` entries, plus the explicit compatibility exports required by the reviewed downstream revision from which it was built. It publishes neither raw TypeScript nor Svelte source. The immutable `@vrtmrz/livesync-commonlib@0.1.0-rc.2` registry artefact can be installed into a clean consumer, imported in Node, type-checked from declarations, and bundled independently for browser context, browser storage, browser services, and workers. Its registry version and checksum are recorded by release validation.
The published export map contains 126 explicitly named entries, including 118 `compat/*` entries and no unrestricted wildcard. Every compatibility entry was referenced by the reviewed Self-hosted LiveSync source used to prepare the artefact. A final consumer audit then moved imports already covered by the focused `context`, `settings`, and `remote-configurations` entries. The current branch consequently uses 115 compatibility paths; `compat/common/models/setting.const.defaults`, `compat/common/models/setting.const.preferred`, and `compat/hub/hub` remain only as an immutable surplus in `rc.2`. They are candidates for removal from the next reviewed compatibility inventory rather than grounds for changing the published artefact. Remaining compatibility imports still expose migration-only service composition, broader model types, replication implementations, or another contract which the focused entries do not yet replace.
The proof found and fixed three boundary defects which source-alias consumption had hidden: compiled JSON imports required explicit output extensions, precompiled Svelte output could not safely be treated as source by the downstream Svelte pipeline, and Vite's default client conditions selected Commonlib's browser worker while building the Node CLI. Packed-consumer regressions cover the first two. The CLI now uses Vite's server conditions and treats every Node built-in reported by Commonlib's Node entry as external; the built CLI is exercised through Deno E2E. Importing root or context also no longer patches DOM prototypes, translator injection prevents the context entry from loading the complete language catalogue, and standard input and protocol output are supplied by the package-owned host contract rather than direct stream access in command handlers.
Self-hosted LiveSync, its CLI, Webapp, WebPeer, plug-in source, and tests compile against that exact registry artefact without `@lib/*`. Focused downstream storage tests pass against the package-owned Node and File System Access API implementations. Commonlib also owns the Trystero implementation and version; the host retains no direct Trystero dependency, preventing two transport generations from entering one application graph. The old `src/lib` Git submodule, generated `_types` fallback, type-generation scripts, and source aliases are removed by the proof branch.
The Commonlib contract suite passes 21 tests covering Context results, both platform storage implementations, and standard I/O; its complete suite passes 1,123 tests across 62 files. Self-hosted LiveSync's three host-composition contract tests and complete 342-test unit suite across 38 files pass against the registry artefact. The plug-in, CLI, Webapp, and WebPeer production builds also pass. The CLI contract command completes its nine-step Deno workflow against the built Node artefact.
The package-owned Trystero transport also completes the canonical Compose P2P synchronisation workflow with a local relay and two isolated CLI peers. In real Obsidian, the plug-in starts with one consistent `ObsidianServiceContext`, the representative server-selection and Setup URI Svelte dialogues mount and close through their normal controls, their mobile variants satisfy viewport, safe-area, and touch-target assertions, and the settings pane exposes only the effective deletion controls. These runtime checks complement the package tests without making Webapp maintenance the primary release gate.
The Community directory scanner preview completes with no source-code errors. The former findings attributed to generated `_types`, raw `src/lib` source, Node built-ins, forbidden rule suppressions, unsupported Obsidian APIs, and undescribed directive comments are absent. This confirms that the registry dependency is recognised as a package boundary. The preview identified behaviour-neutral redundant CLI candidate types and Webapp File System Access API assertions; these are corrected in the host source rather than carried as known warnings.
The remaining source warnings belong to application code. Browser-app DOM creation requires a framework-neutral browser presentation boundary rather than Obsidian-only helpers, and the Obsidian setting tab does not yet expose declarative setting definitions. Browser dialogue visibility now uses DOM state instead of inline static styling, so the earlier styling warning is absent. Direct diagnostic output was resolved at its existing ownership boundaries: Webapp components use an injected log function backed by `BrowserAPIService`, WebPeer retains output in its Svelte log store, Obsidian modules use the established Logger path, and duplicate console emission was removed from `ModuleLog`. The later Webapp and WebPeer recomposition around maintained Context and serviceFeature APIs should preserve these explicit output paths. Declarative settings adoption and deprecated Obsidian interfaces still require separate, focused changes.
WebPeer's production build still reports that Vite externalises the guarded Node `crypto` fallback reached through a compatibility path. Browser execution selects `globalThis.crypto`, and the focused root, `context`, and `browser` bundle checks do not include the Node fallback, so this is not a leak in the reviewed public browser entries. Removing the compatibility-build warning requires a focused crypto-capability contract or a platform-specific implementation split and remains part of compatibility-surface narrowing.
The dependency preview also reports `uuid`, but the installed and locked graph resolves PouchDB's UUID dependency to the patched `uuid@11.1.1` through the repository override, and `npm audit` does not report the UUID advisory. The scanner appears to infer the older declared PouchDB range rather than the resolved override, so this warning is treated as a scanner false positive unless a packed-artefact inspection shows otherwise. A separate dependency review remains necessary for the pre-existing CLI `werift`/`ip` advisory, for which npm currently offers no automatic fix, and the development-only Istanbul `js-yaml` advisory.
The local real-Obsidian suite verifies the actual loaded `ObsidianServiceContext`, all 18 services, Vault reflection, CouchDB and Object Storage transfer, remote-activity accounting, CLI-to-Obsidian encrypted synchronisation, startup scanning, two-Vault create, update, delete, ordinary rename, case-only rename, target mismatch, Hidden File Sync, Customisation Sync, and setting Markdown export. These checks establish observable results and host composition rather than relying on declaration compatibility alone.
The current browser Harness has a pre-existing settings-migration initialisation timeout which reproduces on the untouched baseline. The Webapp Playwright workflow also currently reuses an unrelated process on its configured port and reaches a `Not Found` page. Neither failure is evidence for or against the package boundary. The Webapp production build and direct composition contract pass, while Webapp browser E2E remains supplementary until its runner is repaired. Current acceptance therefore relies primarily on Commonlib owner and packed-consumer tests, LiveSync unit and integration tests, Deno CLI E2E, application production builds, and the focused local real-Obsidian suite. The unrelated Harness and Playwright defects should be tracked independently.
## Scanner and Repository Consequences
Consuming the npm package removes the common-library source and generated `_types` from the plug-in repository's Community directory scan, because package dependencies are treated as dependencies rather than maintained plug-in source. The branch preview confirms this expected boundary.
The change does not hide or resolve warnings in Self-hosted LiveSync-owned source. The scanner will continue to inspect the plug-in, CLI, Webapp, and WebPeer while those applications remain in the repository. Moving those applications to a LiveSync-family application repository may be considered after the package boundary is stable, but it is not part of this decision because the CLI and Webapp still import shared plug-in composition code.
Package extraction also does not resolve release-asset attestation verification, unsupported release assets, declarative settings migration, or other scanner findings which are independent of source ownership.
## Alternatives Rejected
### Move the common library into the Self-hosted LiveSync monorepo
This would permit atomic source changes and remove the Git submodule, but the community scanner already examines non-plug-in application source. A source workspace would remain in scope, and every library directive and platform dependency would be attributed to the plug-in repository. It would also make independent consumers depend on the plug-in repository's release cadence.
### Move the common library into Fancy Kit
Fancy Kit owns reusable framework-neutral interactions, Obsidian adapters, test infrastructure, and neutral utilities. Replication protocols, chunk and metadata formats, PouchDB composition, conflict rules, and LiveSync storage policy are a different domain. Moving them would obscure ownership and make general plug-in tooling carry Self-hosted LiveSync release concerns.
### Publish the current source tree without changing boundaries
This would expose accidental deep imports, global mutations, Svelte and browser code, Node-only modules, and parent-repository imports. Consumers would still need source aliases and bundler-specific behaviour, while maintainers would be unable to distinguish public API from implementation.
### Split every concern into a separate npm package immediately
Core, language, UI, browser, Node, P2P, Object Storage, and SDK packages could make dependency graphs precise, but the present source has not yet proven those release boundaries. Starting with one compiled package and explicit optional subpaths allows measurement without creating a coordinated release matrix. Additional packages remain possible after their contracts and consumers are demonstrated.
## Verification
The package-boundary conversion is ready for acceptance only when:
- common-library production source has no parent-repository or Obsidian dependency;
- the package builds and tests from a standalone clean checkout;
- `npm pack` contains only intended compiled artefacts and documentation;
- packed Node and browser consumers resolve only exported paths;
- a headless client does not bundle Svelte or the full language catalogue;
- retained entry-point barrels do not load unrelated optional implementations, and no removed barrel is replaced by unrestricted deep exports;
- Self-hosted LiveSync verifies its local Svelte adapter and workflow policy through injected tests and a focused composition smoke test;
- Self-hosted LiveSync no longer has `src/lib`, `_types`, or `@lib/*` source aliases;
- the Commonlib owner and packed-consumer suites, Self-hosted LiveSync unit and integration tests, Deno CLI E2E, plug-in and application production builds, and focused real-Obsidian E2E pass against the reviewed package artefact;
- common-library downstream CI records and tests the exact Self-hosted LiveSync ref;
- a Community directory scanner preview confirms the expected change in source findings; and
- compatibility exports are explicitly enumerated from a reviewed downstream inventory rather than exposed through a wildcard.
The following are later SDK-stabilisation gates, not blockers for accepting the package boundary:
- two clients with different database, encryption, language, and lifecycle settings can coexist without sharing mutable client state;
- an external consumer can replace its submodule or custom loader with the documented high-level package API; and
- if the framework-neutral Modal host is promoted to Fancy Kit, Kit-owned lifecycle, viewport, safe-area, and touch-target guarantees replace duplicated device tests in LiveSync.
## Consequences
- The common library gains an independently consumable and testable artefact while retaining its domain ownership and history.
- Self-hosted LiveSync release archives no longer need generated fallback declarations for an absent submodule.
- Community directory scanning distinguishes plug-in source from the reviewed dependency.
- Changes which span the package and plug-in require coordinated package and downstream validation rather than one atomic source commit.
- Temporary compatibility exports increase the first package surface, but their pre-1.0 status and explicit classification prevent them from becoming silent permanent contracts.
- Translation, Svelte, and platform dependencies become optional composition concerns rather than root-package side effects.
- Fancy Kit may gain a generally useful typed Obsidian Modal host without acquiring LiveSync policy or a mandatory Svelte dependency.
## Open Questions Before Acceptance
- Define the focused stable language entry point which will replace the temporary compatibility subpath.
- Define the minimum conflict and conditional-write guarantees required by the first public high-level client.
- Continue narrowing the broad model, service-composition, and replication compatibility paths as focused result contracts become available.
- Confirm whether another Fancy Kit consumer needs the framework-neutral Modal host before it is added to the Kit, or whether LiveSync should pilot the contract first.
Binary file not shown.

After

Width:  |  Height:  |  Size: 7.3 KiB

+168
View File
@@ -0,0 +1,168 @@
# Data Structures of Self-Hosted LiveSync
## Overview
Self-hosted LiveSync uses the following types of documents:
- Metadata
- Legacy Metadata
- Binary Metadata
- Plain Metadata
- Chunk
- Versioning
- Synchronise Information
- Synchronise Parameters
- Milestone Information
## Description of Each Data Structure
All documents inherit from the `DatabaseEntry` interface. This is necessary for conflict resolution and deletion flags.
```ts
export interface DatabaseEntry {
_id: DocumentID;
_rev?: string;
_deleted?: boolean;
}
```
### Versioning Document
This document stores version information for Self-hosted LiveSync.
The ID is fixed as `obsydian_livesync_version` [VERSIONING_DOCID]. Yes, the typo has become a curse.
When Self-hosted LiveSync detects changes to this document via Replication, it reads the version information and checks compatibility.
In that case, if there are major changes, synchronisation may be stopped.
Please refer to negotiation.ts.
### Synchronise Information Document
This document stores information that should be verified in synchronisation settings.
The ID is fixed as `syncinfo` [SYNCINFO_ID].
The information stored in this document is only the conditions necessary for synchronisation to succeed, and as of v0.25.43, only a random string is stored.
This document is only used during rebuilds from the settings screen for CouchDB-based synchronisation, making it like an appendix. It may be removed in the future.
### Synchronise Parameters Document
This document stores synchronisation parameters.
Synchronisation parameters include the protocol version and salt used for encryption, but do not include chunking settings.
The ID is fixed as `_local/obsidian_livesync_sync_parameters` [DOCID_SYNC_PARAMETERS] or `_obsidian_livesync_journal_sync_parameters.json` [DOCID_JOURNAL_SYNC_PARAMETERS].
This document exists only on the remote and not locally.
This document stores the following information.
It is read each time before connecting and is used to verify that E2EE settings match.
This mismatch cannot be ignored and synchronisation will be stopped.
```ts
export interface SyncParameters extends DatabaseEntry {
_id: typeof DOCID_SYNC_PARAMETERS;
type: (typeof EntryTypes)["SYNC_PARAMETERS"];
protocolVersion: ProtocolVersion;
pbkdf2salt: string;
}
```
#### protocolVersion
This field indicates the protocol version used by the remote. Mostly, this value should be `2` (ProtocolVersions.ADVANCED_E2EE), which indicates safer E2EE support.
#### pbkdf2salt
This field stores the salt used for PBKDF2 key derivation on the remote. This salt and the passphrase provides E2EE encryption keys.
### Milestone Information Document
This document stores information about how the remote accepts and recognises clients.
The ID is fixed as `_local/obsidian_livesync_milestone` [MILESTONE_DOCID].
This document exists only on the remote and not locally.
This document is used to indicate synchronisation progress and includes the version range of accepted chunks for each node and adjustment values for each node.
Tweak Mismatched is determined based on the information in this document.
For details, please refer to LiveSyncReplicator.ts, LiveSyncJournalReplicator.ts, and LiveSyncDBFunctions.ts.
```ts
export interface EntryMilestoneInfo extends DatabaseEntry {
_id: typeof MILESTONE_DOCID;
type: EntryTypes["MILESTONE_INFO"];
created: number;
accepted_nodes: string[];
node_info: { [key: NodeKey]: NodeData };
locked: boolean;
cleaned?: boolean;
node_chunk_info: { [key: NodeKey]: ChunkVersionRange };
tweak_values: { [key: NodeKey]: TweakValues };
}
```
### locked
If the remote has been requested to lock out from any client, this is set to true.
When set to true, clients will stop synchronisation unless they are included in accepted_nodes.
### cleaned
If the remote has been cleaned up from any client, this is set to true.
In this case, clients will stop synchronisation as they need to rebuild again.
### Metadata Document
Metadata documents store metadata for Obsidian notes.
```ts
export interface MetadataDocument extends DatabaseEntry {
_id: DocumentID;
ctime: number;
mtime: number;
size: number;
deleted?: boolean;
eden: Record<string, EdenChunk>; // Obsolete
path: FilePathWithPrefix;
children: string[];
type: EntryTypes["NOTE_LEGACY" | "NOTE_BINARY" | "NOTE_PLAIN"];
}
```
### type
This field indicates the type of Metadata document.
By convention, Self-hosted LiveSync does not save the mime type of the file, but distinguishes them with this field. Please note this.
Possible values are as follows:
- NOTE_LEGACY: Legacy metadata document
- Please do not use
- NOTE_BINARY: Binary metadata document (newnote)
- NOTE_PLAIN: Plain metadata document (plain)
#### children
This field stores an array of Chunk Document IDs.
#### \_id, path
\_id is generated based on the path of the Obsidian note.
- If the path starts with `_`, it is converted to `/_` for convenience.
- If Case Sensitive is disabled, it is converted to lowercase.
When Obfuscation is enabled, the path field contains `f:{obfuscated path}`.
The path field stores the path as is. However, when Obfuscation is enabled, the obfuscated path is stored.
When Property Encryption is enabled, the path field stores all properties including children, mtime, ctime, and size in an encrypted state. Please refer to encryption.ts.
### Chunk Document
```ts
export type EntryLeaf = DatabaseEntry & {
_id: DocumentID;
type: EntryTypes["CHUNK"];
data: string;
};
```
Chunk documents store parts of note content.
- The type field is always `[CHUNK]`, `leaf`.
- The data field stores the chunk content.
- The \_id field is generated based on a hash of the content and the passphrase.
Hash functions used include xxHash and SHA-1, depending on settings.
Chunking methods used include Contextual Chunking and Rabin-Karp Chunking, depending on settings.
@@ -0,0 +1,122 @@
# [WITHDRAWN] Chunk Aggregation by Prefix
## Goal
To address the "document explosion" and storage bloat issues caused by the current chunking mechanism, while preserving the benefits of content-addressable storage and efficient delta synchronisation. This design aims to significantly reduce the number of documents in the database and simplify Garbage Collection (GC).
## Motivation
Our current synchronisation solution splits files into content-defined chunks, with each chunk stored as a separate document in CouchDB, identified by its hash. This architecture effectively leverages CouchDB's replication for automatic deduplication and efficient transfer.
However, this approach faces significant challenges as the number of files and edits increases:
1. **Document Explosion:** A large vault can generate millions of chunk documents, severely degrading CouchDB's performance, particularly during view building and replication.
2. **Storage Bloat & GC Difficulty:** Obsolete chunks generated during edits are difficult to identify and remove. Since CouchDB's deletion (`_deleted: true`) is a soft delete, and compaction is a heavy, space-intensive operation, unused chunks perpetually consume storage, making GC impractical for many users.
3. **The "Eden" Problem:** A previous attempt, "Keep newborn chunks in Eden", aimed to mitigate this by embedding volatile chunks within the parent document. While it reduced the number of standalone chunks, it introduced a new issue: the parent document's history (`_revs_info`) became excessively large, causing its own form of database bloat and making compaction equally necessary but difficult to manage.
This new design addresses the root cause—the sheer number of documents—by aggregating chunks into sets.
## Prerequisites
- The new implementation must maintain the core benefit of deduplication to ensure efficient synchronisation.
- The solution must not introduce a single point of bottleneck and should handle concurrent writes from multiple clients gracefully.
- The system must provide a clear and feasible strategy for Garbage Collection.
- The design should be forward-compatible, allowing for a smooth migration path for existing users.
## Outlined Methods and Implementation Plans
### Abstract
This design introduces a two-tiered document structure to manage chunks: **Index Documents** and **Data Documents**. Chunks are no longer stored as individual documents. Instead, they are grouped into `Data Documents` based on a common hash prefix. The existence and location of each chunk are tracked by `Index Documents`, which are also grouped by the same prefix. This approach dramatically reduces the total document count.
### Detailed Implementation
**1. Document Structure:**
- **Index Document:** Maps chunk hashes to their corresponding Data Document ID. Identified by a prefix of the chunk hash.
- `_id`: `idx:{prefix}` (e.g., `idx:a9f1b`)
- Content:
```json
{
"_id": "idx:a9f1b",
"_rev": "...",
"chunks": {
"a9f1b12...": "dat:a9f1b-001",
"a9f1b34...": "dat:a9f1b-001",
"a9f1b56...": "dat:a9f1b-002"
}
}
```
- **Data Document:** Contains the actual chunk data as base64-encoded strings. Identified by a prefix and a sequential number.
- `_id`: `dat:{prefix}-{sequence}` (e.g., `dat:a9f1b-001`)
- Content:
```json
{
"_id": "dat:a9f1b-001",
"_rev": "...",
"chunks": {
"a9f1b12...": "...", // base64 data
"a9f1b34...": "..." // base64 data
}
}
```
**2. Configuration:**
- `chunk_prefix_length`: The number of characters from the start of a chunk hash to use as a prefix (e.g., `5`). This determines the granularity of aggregation.
- `data_doc_size_limit`: The maximum size for a single Data Document to prevent it from becoming too large (e.g., 1MB). When this limit is reached, a new Data Document with an incremented sequence number is created.
**3. Write/Save Operation Flow:**
When a client creates new chunks:
1. For each new chunk, determine its hash prefix.
2. Read the corresponding `Index Document` (e.g., `idx:a9f1b`).
3. From the index, determine which of the new chunks already exist in the database.
4. For the **truly new chunks only**:
a. Read the last `Data Document` for that prefix (e.g., `dat:a9f1b-005`).
b. If it is nearing its size limit, create a new one (`dat:a9f1b-006`).
c. Add the new chunk data to the Data Document and save it.
5. Update the `Index Document` with the locations of the newly added chunks.
**4. Handling Write Conflicts:**
Concurrent writes to the same `Index Document` or `Data Document` from multiple clients will cause conflicts (409 Conflict). This is expected and must be handled gracefully. Since additions are incremental, the client application must implement a **retry-and-merge loop**:
1. Attempt to save the document.
2. On a conflict, re-fetch the latest version of the document from the server.
3. Merge its own changes into the latest version.
4. Attempt to save again.
5. Repeat until successful or a retry limit is reached.
**5. Garbage Collection (GC):**
GC becomes a manageable, periodic batch process:
1. Scan all file metadata documents to build a master set of all *currently referenced* chunk hashes.
2. Iterate through all `Index Documents`. For each chunk listed:
a. If the chunk hash is not in the master reference set, it is garbage.
b. Remove the garbage entry from the `Index Document`.
c. Remove the corresponding data from its `Data Document`.
3. If a `Data Document` becomes empty after this process, it can be deleted.
## Test Strategy
1. **Unit Tests:** Implement tests for the conflict resolution logic (retry-and-merge loop) to ensure robustness.
2. **Integration Tests:**
- Verify that concurrent writes from multiple simulated clients result in a consistent, merged state without data loss.
- Run a full synchronisation scenario and confirm the resulting database has a significantly lower document count compared to the previous implementation.
3. **GC Test:** Simulate a scenario where files are deleted, run the GC process, and verify that orphaned chunks are correctly removed from both Index and Data documents, and that storage is reclaimed after compaction.
4. **Migration Test:** Develop and test a "rebuild" process for existing users, which migrates their chunk data into the new aggregated structure.
## Documentation Strategy
- This design document will be published to explain the new architecture.
- The configuration options (`chunk_prefix_length`, etc.) will be documented for advanced users.
- A guide for the migration/rebuild process will be provided.
## Future Work
The separation of index and data opens up a powerful possibility. While this design initially implements both within CouchDB, the `Data Documents` could be offloaded to a dedicated object storage service such as **S3, MinIO, or Cloudflare R2**.
In such a hybrid model, CouchDB would handle only the lightweight `Index Documents` and file metadata, serving as a high-speed synchronisation and coordination layer. The bulky chunk data would reside in a more cost-effective and scalable blob store. This would represent the ultimate evolution of this architecture, combining the best of both worlds.
## Consideration and Conclusion
This design directly addresses the scalability limitations of the original chunk-per-document model. By aggregating chunks into sets, it significantly reduces the document count, which in turn improves database performance and makes maintenance feasible. The explicit handling of write conflicts and a clear strategy for garbage collection make this a robust and sustainable long-term solution. It effectively resolves the problems identified in previous approaches, including the "Eden" experiment, by tackling the root cause of database bloat. This architecture provides a solid foundation for future growth and scalability.
@@ -0,0 +1,118 @@
# Chunk Retrieval and Waiting
## Purpose
This document records how LiveSync retrieves chunks after file metadata has been found, which operation provides each terminal condition, and what the remaining time value means. It is an implementation specification for developers; it is not a user configuration guide.
The architectural decision and historical rationale are in [Chunk Arrival Quiescence](../adr/2026_07_chunk_arrival_quiescence.md).
## Invariants and Sources of Apparent Reordering
A normal local save creates and persists the chunks before it writes the metadata document which refers to them. LiveSync must preserve this invariant: publishing metadata first can expose a reference which no client can satisfy.
This ordering is not an atomic transaction across documents. A reader may still see metadata before a referenced chunk for these reasons:
- CouchDB replication transfers individual documents and does not expose the chunk and metadata writes as one atomic unit.
- With `readChunksOnline` enabled, CouchDB pull replication deliberately excludes chunk documents. Seeing metadata first is then the intended design, and the chunk is fetched by identifier.
- A winning metadata conflict revision may refer to chunks created by another revision or client which have not yet reached the local database.
- Replication persists documents before every downstream change callback and file-reflection task has necessarily observed them.
- Historical versions and removed transfer modes may have produced data which did not preserve the normal ordering invariant.
Waiting may resolve temporary visibility and processing gaps only when a known operation can still deliver the chunk. It cannot repair a chunk which is absent from every available source.
## Retrieval Capabilities
Direct on-demand fetch is currently available only for CouchDB when `useOnlyLocalChunk` is false. This capability deliberately does not depend on `readChunksOnline`:
- when `readChunksOnline` is true, direct fetch is the normal way to obtain a chunk omitted from pull replication; and
- when `readChunksOnline` is false, direct fetch remains a recovery path if a normally replicated chunk is locally absent.
MinIO's sequential replicator and P2P do not implement direct `fetchRemoteChunks` delivery through this path. Their chunks must arrive through a finite replication operation. A P2P pull or bidirectional synchronisation is such a producer. A push-only P2P request remains broad remote activity for Wake Lock and lifecycle reporting, but it is deliberately excluded from `finiteReplicationActivityCount` because it cannot deliver a local document.
`waitForReady` is a call-site policy, not a persisted user setting. `true` permits waiting for an already-observable producer. `false` normally requests an immediate local result, except that CouchDB on-demand delivery still waits for the claim which synchronous dispatch creates.
## Policy Matrix
The matrix selects whether lifecycle waiting is permitted and whether the waiter may dispatch a direct request. It does not assign elapsed arrival budgets.
| Remote | `waitForReady` | `useOnlyLocalChunk` | Direct fetch | Wait for observed producer | Intended behaviour |
| ------- | -------------: | ------------------: | -----------: | -------------------------: | --------------------------------------------------------------------------------------- |
| CouchDB | `false` | `false` | Yes | Yes | Dispatch on-demand fetch and finish at its per-identifier claim boundary. |
| CouchDB | `true` | `false` | Yes | Yes | Accept an active finite replication or dispatch direct fetch. |
| CouchDB | `false` | `true` | No | No | Return immediately after the local miss. |
| CouchDB | `true` | `true` | No | Yes | Wait for an already-active finite replication; otherwise return unavailable. |
| MinIO | `false` | Either | No | No | Return immediately after the local miss. |
| MinIO | `true` | Either | No | Yes | Wait for an already-active finite sequential replication; otherwise return unavailable. |
| P2P | `false` | Either | No | No | Return immediately after the local miss. |
| P2P | `true` | Either | No | Yes | Wait for an already-active finite P2P replication; otherwise return unavailable. |
For CouchDB, `readChunksOnline` changes what normal replication includes, not the direct-fetch capability or this matrix:
| `readChunksOnline` | CouchDB pull contains chunks | Role of direct fetch |
| -----------------: | ---------------------------: | ------------------------------------------------------------------------ |
| `true` | No | Primary chunk delivery after metadata arrives. |
| `false` | Yes | Recovery fallback for a chunk which is unexpectedly unavailable locally. |
`concurrencyOfReadChunksOnline` and `minimumIntervalOfReadChunksOnline` affect only the scheduling of CouchDB on-demand requests. They do not change whether a request may be dispatched or which lifecycle a reader observes. Accepted identifiers remain claimed while they wait for a concurrency slot and while the configured interval is applied. A minimum interval of five minutes or more is an exceptional value: the inactivity fuse may release the logical claim before that deliberate pause completes. This safety precedence does not abort the delayed physical request.
## Wait State Machine
1. Read the cache and local database.
2. If every requested chunk is present, return it without entering a wait.
3. Register one shared waiter per missing identifier.
4. If policy permits direct fetch, emit `missingChunks`. `ChunkFetcher` synchronously creates the per-identifier claim before the event dispatch returns.
5. Observe both the matching claim and `finiteReplicationActivityCount`.
6. Resolve immediately if a valid chunk or explicit remote-missing event arrives.
7. If an observed producer remains active, do not charge elapsed time against an arrival budget.
8. When all observed producers end, bypass the cache and read the identifiers from the local database once.
9. Return the rechecked chunk, or return unavailable. Do not add another fixed grace after the authoritative boundary.
10. If no producer is observable after synchronous dispatch, return unavailable immediately.
If new relevant activity starts while the final database recheck is pending, that result becomes stale. The waiter remains active until the newer producer completes and a current recheck finishes.
## Meaning of Finite Replication Completion
Finite replication enters the typed `runFiniteReplicationActivity` boundary and is represented by the narrower `finiteReplicationActivityCount`. The optional `replication` label remains diagnostic and does not control this behaviour.
For a successful finite operation, completion means that its replicator has reached the latest sequence in the operation's scope and processed its replication change callbacks. No more database documents can arrive from that operation. This is the primary semantic cutoff.
If the operation fails, it has not proved remote absence or latest state. It has nevertheless stopped being a producer. The waiting layer rechecks documents which may have arrived before the failure and then returns unavailable; the replication error and retry workflow owns further recovery.
Overlapping finite replications keep the count above zero until the final operation settles. The local recheck therefore occurs only after every observed finite producer is quiescent.
The continuous live channel is intentionally excluded because it has no completion boundary and would otherwise make a chunk read unbounded. The pull-only catch-up run before opening that channel is finite and enters the same typed boundary. Its one-shot batch-size fallback remains inside that boundary. A live-channel fallback starts another continuous attempt and therefore another bounded initial catch-up.
## Meaning of an On-demand Claim
An accepted identifier remains claimed from synchronous queue acceptance through throttling, physical fetch, validation, local persistence, and terminal event delivery. The claim is identifier-scoped because a global remote-work count cannot say whether unrelated work can provide this chunk.
The claim finishes when the fetcher has recorded an outcome for the identifier. A transport error, missing active replicator, or invalid result releases the claim without emitting an explicit remote-missing result unless the remote actually supplied that information.
## Meaning of the Five-minute Value
The five-minute value is an inactivity leak fuse for an accepted on-demand claim. It is the only elapsed duration in this state machine, and it is not a normal terminal condition.
The fuse bounds retention if a faulty activity runner never enters its task, a Promise never settles, or a transport stops making observable progress. It prevents the per-identifier claim and waiter from remaining live forever. Once the bounded activity callback has entered, releasing the claim also allows Wake Lock, application-lifecycle deferral, and the remote-work indicator associated with that callback to finish. Observable progress rearms the fuse.
Five minutes is a conservative operational ceiling rather than a measured chunk-arrival expectation. It must not be used to infer that the remote lacks a chunk, and it does not abort the physical request. `fetchRemoteChunks` does not yet accept an `AbortSignal`, so the request may complete after the logical state has been released. Transport cancellation and transport-specific deadlines are separate future work.
The old 5-second and 30-second constants remain exported for source compatibility only. A positive deprecated `ChunkReadOptions.timeout` opts into lifecycle waiting, but its numeric value is ignored. Zero or a negative value still requests an immediate result. New code uses `waitForDelivery` explicitly.
## Test Obligations
Changes to this behaviour must keep automated coverage for:
- chunks-before-metadata save ordering;
- every row in the retrieval policy matrix;
- a finite replication which remains active well beyond the former 5-second and 30-second values;
- successful completion with the chunk already persisted but no arrival event delivered;
- immediate unavailability when no producer is observable;
- overlapping finite operations and overlapping per-identifier claims;
- activity restarting while a local recheck is pending;
- direct fetch queueing, throttling, persistence, and terminal notification;
- explicit remote absence versus transport or replicator failure;
- runner rejection, cancellation, teardown, and an operation which never enters its task;
- leak-fuse refresh at observable progress points; and
- continuous replication's finite initial catch-up and parameter fallback.
The service, database, and event boundaries are testable with memory-backed PouchDB and injected activity sources. A real Obsidian test is required only when a change crosses into the platform adapter, application lifecycle, or visible UI rather than for this retrieval state machine alone.
+127
View File
@@ -0,0 +1,127 @@
# [WIP] The design intent explanation for using metadata and chunks
## Abstract
## Goal
- To explain the following:
- What metadata and chunks are
- The design intent of using metadata and chunks
## Background and Motivation
We are using PouchDB and CouchDB for storing files and synchronising them. PouchDB is a JavaScript database that stores data on the device (browser, and of course, Obsidian), while CouchDB is a NoSQL database that stores data on the server. The two databases can be synchronised to keep data consistent across devices via the CouchDB replication protocol. This is a powerful and flexible way to store and synchronise data, including conflict management, but it is not well suited for files. Therefore, we needed to manage how to store files and synchronise them.
## Terminology
- Password:
- A string used to authenticate the user.
- Passphrase:
- A string used to encrypt and decrypt data.
- This is not a password.
- Encrypt:
- To convert data into a format that is unreadable to anyone.
- Can be decrypted by the user who has the passphrase.
- Should be 1:n, containing random data to ensure that even the same data, when encrypted, results in different outputs.
- Obfuscate:
- To convert data into a format that is not easily readable.
- Can be decrypted by the user who has the passphrase.
- Should be 1:1, containing no random data, and the same data is always obfuscated to the same result. It is necessarily unreadable.
- Hash:
- To convert data into a fixed-length string that is not easily readable.
- Cannot be decrypted.
- Should be 1:1, containing no random data, and the same data is always hashed to the same result.
## Designs
### Principles
- To synchronise and handle conflicts, we should keep the history of modifications.
- No data should be lost. Even though some extra data may be stored, it should be removed later, safely.
- Each stored data item should be as small as possible to transfer efficiently, but not so small as to be inefficient.
- Any type of file should be supported, including binary files.
- Encryption should be supported efficiently.
- This method should not depart too far from the PouchDB/CouchDB philosophy. It needs to leave room for other `remote`s, to benefit from custom replicators.
As a result, we have adopted the following design.
- Files are stored as one metadata entry and multiple chunks.
- Chunks are content-addressable, and the metadata contains the ids of the chunks.
- Chunks may be referenced from multiple metadata entries. They should be efficiently managed to avoid redundancy.
### Metadata Design
The metadata contains the following information:
| Field | Type | Description | Note |
| -------- | -------------------- | ---------------------------- | ----------------------------------------------------------------------------------------------------- |
| _id | string | The id of the metadata | It is created from the file path |
| _rev | string | The revision of the metadata | It is created by PouchDB |
| children | [string] | The ids of the chunks | |
| path | string | The path of the file | If Obfuscate path has been enabled, it has been encrypted |
| size | number | The size of the metadata | Not respected; for troubleshooting |
| ctime | string | The creation timestamp | This is not used to compare files, but when writing to storage, it will be used |
| mtime | string | The modification timestamp | This will be used to compare files, and will be written to storage |
| type | `plain` \| `newnote` | The type of the file | Children of type `plain` will not be base64 encoded, while `newnote` will be |
| e_ | boolean | The file is encrypted | Encryption is processed during transfer to the remote. In local storage, this property does not exist |
#### Decision Rule for `_id` of Metadata
```ts
// Note: This is pseudo code.
let _id = PATH;
if (!HANDLE_FILES_AS_CASE_SENSITIVE) {
_id = _id.toLowerCase();
}
if (_id.startsWith("_")) {
_id = "/" + _id;
}
if (OBFUSCATE_PATH) {
_id = `f:${OBFUSCATE_PATH(_id, E2EE_PASSPHRASE)}`;
}
return _id;
```
#### Expected Questions
- Why do we need to handle files as case-sensitive?
- Some filesystems are case-sensitive, while others are not. For example, Windows is not case-sensitive, while Linux is. Therefore, we need to handle files as case-sensitive to manage conflicts.
- The trade-off is that you will not be able to manage files with different cases, so this can be disabled if you only have case-sensitive terminals.
- Why obfuscate the path?
- E2EE only encrypts the content of the file, not metadata. Hence, E2EE alone is not enough to protect the vault completely. The path is also part of the metadata, so it should be obfuscated. This is a trade-off between security and performance. However, if you title a note with sensitive information, you should obfuscate the path.
- What is `f:`?
- It is a prefix to indicate that the path is obfuscated. It is used to distinguish between normal paths and obfuscated paths. Due to file enumeration, Self-hosted LiveSync should scan the files to find the metadata, excluding chunks and other information.
- Why does an unobfuscated path not start with `f:`?
- For compatibility. Self-hosted LiveSync, by its nature, must also be able to handle files created with newer versions as far as possible.
### Chunk Design
#### Chunk Structure
The chunk contains the following information:
| Field | Type | Description | Note |
| ----- | ------------ | ------------------------- | ----------------------------------------------------------------------------------------------------- |
| _id | `h:{string}` | The id of the chunk | It is created from the hash of the chunk content |
| _rev | string | The revision of the chunk | It is created by PouchDB |
| data | string | The content of the chunk | |
| type | `leaf` | Fixed | |
| e_ | boolean | The chunk is encrypted | Encryption is processed during transfer to the remote. In local storage, this property does not exist |
**SORRY, TO BE WRITTEN, BUT WE HAVE IMPLEMENTED `v2`, WHICH REQUIRES MORE INFORMATION.**
### How they are unified
## Deduplication and Optimisation
## Synchronisation Strategy
## Performance Considerations
## Security and Privacy
## Edge Cases
+117
View File
@@ -0,0 +1,117 @@
# [IN DESIGN] Tiered Chunk Storage with Live Compaction
** VERY IMPORTANT NOTE: This design must be used with the new journal synchronisation method. Otherwise, we risk introducing the bloat of changes from hot-pack into the Bucket. (CouchDB/PouchDB can synchronise only the most recent changes, or resolve conflicts.) Previous Journal Sync **IS NOT**. Please proceed with caution. **
## Goal
To establish a highly efficient, robust, and scalable synchronisation architecture by introducing a tiered storage system inspired by Log-Structured Merge-Trees (LSM-Trees). This design aims to address the challenges of real-time synchronisation, specifically the massive generation of transient data, while minimising storage bloat and ensuring high performance.
## Motivation
Our previous designs, including "Chunk Aggregation by Prefix", successfully addressed the "document explosion" problem. However, the introduction of real-time editor synchronisation exposed a new, critical challenge: the constant generation of short-lived "garbage" chunks during user input. This "garbage storm" places immense pressure on storage, I/O, and the Garbage Collection (GC) process.
A simple aggregation strategy is insufficient because it treats all data equally, mixing valuable, stable chunks with transient, garbage chunks in permanent storage. This leads to storage bloat and inefficient compaction. We require a system that can intelligently distinguish between "hot" (volatile) and "cold" (stable) data, processing them in the most efficient manner possible.
## Outlined Methods and Implementation Plans
### Abstract
This design implements a two-tiered storage system within CouchDB.
1. **Level 0 Hot Storage:** A set of "Hot-Packs", one for each active client. These act as fast, append-only logs for all newly created chunks. They serve as a temporary staging area, absorbing the "garbage storm" of real-time editing.
2. **Level 1 Cold Storage:** The permanent, immutable storage for stable chunks, consisting of **Index Documents** for fast lookups and **Data Documents (Cold-Packs)** for storing chunk data.
A background "Compaction" process continuously promotes stable chunks from Hot Storage to Cold Storage, while automatically discarding garbage. This keeps the permanent storage clean and highly optimised.
### Detailed Implementation
**1. Document Structure:**
- **Hot-Pack Document (Level 0):** A per-client, append-only log.
- `_id`: `hotpack:{client_id}` (`client_id` could be the same as the `deviceNodeID` used in the `accepted_nodes` in MILESTONE_DOC; enables database 'lockout' for safe synchronisation)
- Content: A log of chunk creation events.
```json
{
"_id": "hotpack:a9f1b12...",
"_rev": "...",
"log": [
{ "hash": "abc...", "data": "...", "ts": ..., "file_id": "file1" },
{ "hash": "def...", "data": "...", "ts": ..., "file_id": "file2" }
]
}
```
- **Index Document (Level 1):** A fast, prefix-based lookup table for stable chunks.
- `_id`: `idx:{prefix}` (e.g., `idx:a9f1b`)
- Content: Maps a chunk hash to the ID of the Cold-Pack it resides in.
```json
{
"_id": "idx:a9f1b",
"chunks": { "a9f1b12...": "dat:1678886400" }
}
```
- **Cold-Pack Document (Level 1):** An immutable data block created by the compaction process.
- `_id`: `dat:{timestamp_or_uuid}` (e.g., `dat:1678886400123`)
- Content: A collection of stable chunks.
```json
{
"_id": "dat:1678886400123",
"chunks": { "a9f1b12...": "...", "c3d4e5f...": "..." }
}
```
- **Hot-Pack List Document:** A central registry of all active Hot-Packs. This might be a computed document that clients maintain in memory on startup.
- `_id`: `hotpack_list`
- Content: `{"active_clients": ["hotpack:a9f1b12...", "hotpack:c3d4e5f..."]}`
**2. Write/Save Operation Flow (Real-time Editing):**
1. A client generates a new chunk.
2. It **immediately appends** the chunk object (`{hash, data, ts, file_id}`) to its **own** Hot-Pack document's `log` array within its local PouchDB. This operation is extremely fast.
3. The PouchDB synchronisation process replicates this change to the remote CouchDB and other clients in the background. No other Hot-Packs are consulted during this write operation.
**3. Read/Load Operation Flow:**
To find a chunk's data:
1. The client first consults its in-memory list of active Hot-Pack IDs (see section 5).
2. It searches for the chunk hash in all **Hot-Pack documents**, starting from its own, then others. It reads them in reverse log order (newest first).
3. If not found, it consults the appropriate **Index Document (`idx:...`)** to get the ID of the Cold-Pack.
4. It then reads the chunk data from the corresponding **Cold-Pack document (`dat:...`)**.
**4. Compaction & Promotion Process (The "GC"):**
This is a background task run periodically by clients, or triggered when the number of unprocessed log entries exceeds a threshold (to maintain the ability to synchronise with the remote database, which has a limited document size).
1. The client takes its own Hot-Pack (`hotpack:{client_id}`) and scans its `log` array from the beginning (oldest first).
2. For each chunk in the log, it checks if the chunk is still referenced in the latest revision of any file.
- **If not referenced (Garbage):** The log entry is simply discarded.
- **If referenced (Stable):** The chunk is added to a "promotion batch".
3. After scanning a certain number of log entries, the client takes the "promotion batch".
4. It creates one or more new, immutable **Cold-Pack (`dat:...`)** documents to store the chunk data from the batch.
5. It updates the corresponding **Index (`idx:...`)** documents to point to the new Cold-Pack(s).
6. Once the promotion is successfully saved to the database, it **removes the processed entries from its Hot-Pack's `log` array**. This is a critical step to prevent reprocessing and keep the Hot-Pack small.
**5. Hot-Pack List Management:**
To know which Hot-Packs to read, clients will:
1. On startup, load the `hotpack_list` document into memory.
2. Use PouchDB's live `changes` feed to monitor the creation of new `hotpack:*` documents.
3. Upon detecting an unknown Hot-Pack, the client updates its in-memory list and attempts to update the central `hotpack_list` document (on a best-effort basis, with conflict resolution).
## Planned Test Strategy
1. **Unit Tests:** Test the Compaction/Promotion logic extensively. Ensure garbage is correctly identified and stable chunks are promoted correctly.
2. **Integration Tests:** Simulate a multi-client real-time editing session.
- Verify that writes are fast and responsive.
- Confirm that transient garbage chunks do not pollute the Cold Storage.
- Confirm that after a period of inactivity, compaction runs and the Hot-Packs shrink.
3. **Stress Tests:** Simulate many clients joining and leaving to test the robustness of the `hotpack_list` management.
## Documentation Strategy
- This design document will serve as the core architectural reference.
- The roles of each document type (Hot-Pack, Index, Cold-Pack, List) will be clearly explained for future developers.
- The logic of the Compaction/Promotion process will be detailed.
## Consideration and Conclusion
This tiered storage design is a direct evolution, born from the lessons of previous architectures. It embraces the ephemeral nature of data in real-time applications. By creating a "staging area" (Hot-Packs) for volatile data, it protects the integrity and performance of the permanent "cold" storage. The Compaction process acts as a self-cleaning mechanism, ensuring that only valuable, stable data is retained long-term. This is not just an optimisation; it is a fundamental shift that enables robust, high-performance, and scalable real-time synchronisation on top of CouchDB.
@@ -0,0 +1,97 @@
# [IN DESIGN] Tiered Chunk Storage for Bucket Sync
## Goal
To evolve the "Journal Sync" mechanism by integrating the Tiered Storage architecture. This design aims to drastically reduce the size and number of sync packs, minimise storage consumption on the backend bucket, and establish a clear, efficient process for Garbage Collection, all while remaining protocol-agnostic.
## Motivation
The original "Journal Sync" liberates us from CouchDB's protocol, but it still packages and transfers entire document changes, including bulky and often transient chunk data. In a real-time or frequent-editing scenario, this results in:
1. **Bloated Sync Packs:** Packs become large with redundant or short-lived chunk data, increasing upload and download times.
2. **Inefficient Storage:** The backend bucket stores numerous packs containing overlapping and obsolete chunk data, wasting space.
3. **Impractical Garbage Collection:** Identifying and purging obsolete *chunk data* from within the pack-based journal history is extremely difficult.
This new design addresses these problems by fundamentally changing *what* is synchronised in the journal packs. We will synchronise lightweight metadata and logs, while handling bulk data separately.
## Outlined methods and implementation plans
### Abstract
This design adapts the Tiered Storage model for a bucket-based backend. The backend bucket is partitioned into distinct areas for different data types. The "Journal Sync" process is now responsible for synchronising only the "hot" volatile data and lightweight metadata. A separate, asynchronous "Compaction" process, which can be run by any client, is responsible for migrating stable data into permanent, deduplicated "cold" storage.
### Detailed Implementation
**1. Bucket Structure:**
The backend bucket will have four distinct logical areas (prefixes):
- `packs/`: For "Journal Sync" packs, containing the journal of metadata and Hot-Log changes.
- `hot_logs/`: A dedicated area for each client's "Hot-Log," containing newly created, volatile chunks.
- `indices/`: For prefix-based Index files, mapping chunk hashes to their permanent location in Cold Storage.
- `cold_chunks/`: For deduplicated, stable chunk data, stored by content hash.
**2. Data Structures (Client-side PouchDB & Backend Bucket):**
- **Client Metadata:** Standard file metadata documents, kept in the client's PouchDB.
- **Hot-Log (in `hot_logs/`):** A per-client, append-only log file on the bucket.
- Path: `hot_logs/{client_id}.jsonlog`
- Content: A sequence of JSON objects, one per line, representing chunk creation events. `{"hash": "...", "data": "...", "ts": ..., "file_id": "..."}`
- **Index File (in `indices/`):** A JSON file for a given hash prefix.
- Path: `indices/{prefix}.json`
- Content: Maps a chunk hash to its content hash (which is its key in `cold_chunks/`). `{"hash_abc...": true, "hash_def...": true}`
- **Cold Chunk (in `cold_chunks/`):** The raw, immutable, deduplicated chunk data.
- Path: `cold_chunks/{chunk_hash}`
**3. "Journal Sync" - Send/Receive Operation (Not Live):**
This process is now extremely lightweight.
1. **Send:**
a. The client takes all newly generated chunks and **appends them to its own Hot-Log file (`hot_logs/{client_id}.jsonlog`)** on the bucket.
b. The client updates its local file metadata in PouchDB.
c. It then creates a "Journal Sync" pack containing **only the PouchDB journal of the file metadata changes.** This pack is very small as it contains no chunk data.
d. The pack is uploaded to `packs/`.
2. **Receive:**
a. The client downloads new packs from `packs/` and applies the metadata journal to its local PouchDB.
b. It downloads the latest versions of all **other clients' Hot-Log files** from `hot_logs/`.
c. Now the client has a complete, up-to-date view of all metadata and all "hot" chunks.
**4. Read/Load Operation Flow:**
To find a chunk's data:
1. The client searches for the chunk hash in its local copy of all **Hot-Logs**.
2. If not found, it downloads and consults the appropriate **Index file (`indices/{prefix}.json`)**.
3. If the index confirms existence, it downloads the data from **`cold_chunks/{chunk_hash}`**.
**5. Compaction & Promotion Process (Asynchronous "GC"):**
This is a deliberate, offline-capable process that any client can choose to run.
1. The client "leases" its own Hot-Log for compaction.
2. It reads its entire `hot_logs/{client_id}.jsonlog`.
3. For each chunk in the log, it checks if the chunk is referenced in the *current, latest state* of the file metadata.
- **If not referenced (Garbage):** The log entry is discarded.
- **If referenced (Stable):** The chunk is added to a "promotion batch."
4. For each chunk in the promotion batch:
a. It checks the corresponding `indices/{prefix}.json` to see if the chunk already exists in Cold Storage.
b. If it does not exist, it **uploads the chunk data to `cold_chunks/{chunk_hash}`** and updates the `indices/{prefix}.json` file.
5. Once the entire Hot-Log has been processed, the client **deletes its `hot_logs/{client_id}.jsonlog` file** (or truncates it to empty), effectively completing the cycle.
## Test strategy
1. **Component Tests:** Test the Compaction process independently. Ensure it correctly identifies stable versus garbage chunks and populates the `cold_chunks/` and `indices/` areas correctly.
2. **Integration Tests:**
- Simulate a multi-client sync cycle. Verify that sync packs in `packs/` are small.
- Confirm that `hot_logs/` are correctly created and updated.
- Run the Compaction process and verify that data migrates correctly to cold storage and the hot log is cleared.
3. **Conflict Tests:** Simulate two clients trying to compact the same index file simultaneously and ensure the outcome is consistent (for example, via a locking mechanism or last-write-wins).
## Documentation strategy
- This design document will be the primary reference for the bucket-based architecture.
- The structure of the backend bucket (`packs/`, `hot_logs/`, etc.) will be clearly defined.
- A detailed description of how to run the Compaction process will be provided to users.
## Consideration and Conclusion
By applying the Tiered Storage model to "Journal Sync", we transform it into a remarkably efficient system. The synchronisation of everyday changes becomes extremely fast and lightweight, as only metadata journals are exchanged. The heavy lifting of data deduplication and permanent storage is offloaded to a separate, asynchronous Compaction process. This clear separation of concerns makes the system highly scalable, minimises storage costs, and finally provides a practical, robust solution for Garbage Collection in a protocol-agnostic, bucket-based environment.
+50
View File
@@ -0,0 +1,50 @@
## The design document of the journal sync
Original title: Synchronise without CouchDB
### Goal
- Synchronise vaults without CouchDB
### Motivation
- Serving CouchDB is not pretty easy.
- Full spec DBaaS (Paid IBM Cloudant) is a bit expensive and lacking of alternatives.
- Securing alternatives, from just one protocol.
### Prerequisite
- We should have multiple implementations of the server software.
- We should also be able to use SaaS, with a choice of options.
- We should require them a reasonable sense of cost, ideally free of charge for trials.
- We should be able to serve some instance of the server software, as OSS — with transparency, availability of auditing, and the fact that they actually took place.
### Methods and implementations
Ordinarily, local pouchDB and the remote CouchDB are synchronised by sending each missing document through several conversations in their replication protocol. However, to achieve this plan, we cannot rely on CouchDB and its protocols. This limitation is so harsh. However, Overcoming this means gaining new possibilities. After some trials, It was concluded that synchronisation could be completed even if the actions that could be performed were limited to uploading, downloading, and retrieving the list. This means we can use any old-fashioned WebDAV server, and sophisticated 'object storages' such as Self-hosted MinIO, S3, and R2, or any we like. This is realised by sharing and complementing the differences of the journal by each client. Therefore, The focus is therefore on how to identify which are the differences and send them without dynamic communication.
All clients manage their data in PouchDB. I know this is probably known information, but it has its own journal.
First, all clients should record to what point in the journal they sent themselves last time. The client then packs from the previous point to the latest when sending and also updates their record. This pack is uploaded to the server with the name starting with the timestamp of its creation. This is the send operation.
Conversely, when receiving, the packs uploaded to the server that have not yet been received are received in order. This is easy as their names are in date order. When the process is successfully completed, the names of the files received are recorded. The journals from this pack are then reflected in their own database. Conflict resolution is left to PouchDB, so the client only needs to do the work of applying any differences. And here is the key: the client records the ID and revision of the document that was in the journal and applied.
This key works when creating a pack. When creating a pack, the client omits this 'document recorded as received and used'. This is because received and applied means that it has already been sent by another client and exists on the server. This ensures that unnecessary transmissions do not take place.
Synchronisation is then always started by receiving. This is a little trick to avoid including unnecessary documents in the pack.
These behaviours allow clients to voluntarily send and receive only the missing parts of the journal that are not stored on the server, without having to communicate with each other, and still keep a single, consistent journal on the server.
Source codes actually implemented this is already committed into the repository.
### Test strategy
This implementation replaces the synchronisation performed by CouchDB. Therefore, testing was simply done by comparing the same changes to the same vault, replicated in CouchDB, with those done by this implementation.
### Documentation strategy
- Documentation should be done in a quick setup, at least.
- As several server implementations can be selected, the description is omitted with regard to specific configuration values.
- A MinIO set-up might be nice to have. However, it is not considered essential.
- It would be a good opportunity to also publish these design documents.
### Consideration and Conclusion
This design offers a novel approach to journal synchronisation without relying on CouchDB. It leverages PouchDB's journaling capabilities and leverages simple server-side storage for efficient data exchange. Hence, the new design could be said to have gotten a broader outlook.
+94
View File
@@ -0,0 +1,94 @@
## The design document of the Journal Replicator 2nd Edition
### Goal
- Build a robust and memory-efficient replication foundation that decouples the physical storage layer by leveraging the Web Streams API.
- Maintain strict compliance with the data consistency and replication protocols of CouchDB/PouchDB.
- Introduce the `IJournalStorage` abstraction to ensure easy extensibility. This allows the core to seamlessly interact with Object Storages (MinIO, S3, R2, etc.) while opening the door for entirely new Storage Engines and Mocks for testing.
### Motivation
- The original Journal Replicator used a custom queue mechanism called `Trench` to manage backpressure, which had limitations regarding memory efficiency when dealing with a massive number of files.
- The storage operation logic was tightly coupled with `JournalSyncAbstract`, making it difficult to swap out the physical storage layer (e.g., S3 and WebDAV).
- The transfer of revision trees (`_revisions`) conforming to PouchDB's replication protocol was implicitly managed. There was a need for a stricter, more deterministic application of document histories.
### Differences from v1 (Original)
The overall architecture and mechanisms have been drastically modernised from the first version. Here are the key differences:
| Feature / Mechanism | v1 (Original) | v2 (2nd Edition) | Key Benefits in v2 |
| :--- | :--- | :--- | :--- |
| **Backpressure / Queueing** | Custom `Trench` mechanism | Native **Web Streams API** | Prevents memory exhaustion during massive transfers; extremely stable sustained throughput. |
| **Storage Architecture** | Tightly coupled in `JournalSyncAbstract` | Abstracted via **`IJournalStorage`** | Easy to plug in new Storage Engines (WebDAV, etc.) and testing Mocks without altering the core logic. |
| **Document Application** | Sometimes evaluated as new local edits | Strict `bulkDocs` with **`new_edits: false`** | Drastically faster insertions; prevents redundant conflict branches and "echo" network traffic. |
#### Class Structure Changes (Diff from v1)
Looking at the Git diff from the `main` branch, the class structure has undergone a significant refactoring to achieve the aforementioned decoupling:
- **`JournalSyncAbstract.ts` -> `JournalSyncCore.ts`**: The core logic was renamed. Instead of being an abstract base class for specific storages, it is now a concrete core class that manages the Web Streams API pipelines.
- **`JournalSyncMinio.ts` -> `MinioStorageAdapter.ts`**: The MinIO-specific implementation was decoupled from the core logic and converted into a dedicated storage adapter.
- **`IJournalStorage` (New)**: Introduced in `JournalStorageAdapter.ts` to define the interface that all storage adapters must implement.
### Methods and implementations
#### Pipeline Construction using Web Streams API
We replaced `Trench` with standard Web Streams APIs (`ReadableStream`, `TransformStream`, and `WritableStream`) to build the sending and receiving pipelines.
- **Sending Pipeline**: Reads documents from the PouchDB changes stream, passes them through a compression `TransformStream`, and pipes them to an upload `WritableStream`. This enables automatic backpressure, keeping memory consumption stable even during large-scale synchronisation.
- **Receiving Pipeline**: Processes storage file listing, downloading/decompression, and bulk application to PouchDB in a streamlined manner.
#### Decoupling the Physical Layer via IJournalStorage
To detach the storage operations from the core synchronisation logic (`JournalSyncCore`), we introduced the `IJournalStorage` interface.
This ensures extensibility not only to Object Storages (MinIO, S3, R2, etc., handled via `MinioStorageAdapter` and Connection Strings) but also to entirely new Storage Engines (e.g., WebDAV, Google Drive) and Mocks for testing. When adding a new backend, developers only need to add an Adapter that implements this interface, without modifying the core replicator.
```mermaid
classDiagram
class LiveSyncAbstractReplicator {
<<abstract>>
}
class LiveSyncJournalReplicator {
+setupJournalSyncClient()
}
class JournalSyncCore {
-IJournalStorage storage
+sendLocalJournal()
+receiveRemoteJournal()
}
class IJournalStorage {
<<interface>>
+upload()
+download()
}
class MinioStorageAdapter {
+upload()
+download()
}
LiveSyncAbstractReplicator <|-- LiveSyncJournalReplicator : Extends
LiveSyncJournalReplicator *-- JournalSyncCore : Instantiates & delegates
LiveSyncJournalReplicator ..> MinioStorageAdapter : Creates
JournalSyncCore --> IJournalStorage : Uses for backend I/O
IJournalStorage <|.. MinioStorageAdapter : Implements
```
#### Strict Application of PouchDB Replication Protocols
To synchronise precisely according to the CouchDB/PouchDB protocol, the following steps were optimised:
1. **Transferring History**: Using `bulkGet({ revs: true })`, the replicator transfers not only the latest revision of a document but its entire history tree (`_revisions`) alongside the deletion flag (`_deleted`).
2. **Applying History**: On the receiving end, the replicator uses `revsDiff` to identify which incoming revisions are missing locally. It then applies them using `bulkDocs(saveDocs, { new_edits: false })`.
By specifying `new_edits: false`, PouchDB integrates the received history exactly as it is without treating them as new local edits. This prevents unexpected conflicts and redundant branching of the revision tree.
### Performance and Speed Characteristics
By migrating from the previous `Trench` architecture to the Web Streams API and strict PouchDB protocol compliance, the replication speed characteristics have changed in the following ways:
1. **On-Demand Generation and Consistent Throughput**:
In the previous `Trench` architecture, the system would eagerly generate or download all 'Changes' in bulk before processing them. This batch processing became a significant bottleneck and caused massive memory spikes (Even though, some of them have been stored into the idb temporally). The Web Streams API fundamentally shifts this to **on-demand (lazy) generation**. Data is pulled and processed only as much as the next pipeline stage (Compress -> Upload/Write) can handle. While this on-demand approach might appear slightly slower in terms of peak burst speed compared to in-memory batching, it completely eliminates the 'create-everything-at-once' bottleneck. This makes the **sustained throughput far more stable** and prevents out-of-memory crashes on mobile devices.
2. **Faster Receive-Side Application (`new_edits: false`)**:
In the previous version, incoming documents were sometimes evaluated as new local edits. By utilising PouchDB's `bulkDocs({ new_edits: false })` alongside the proper `_revisions` tree, we bypass unnecessary conflict generation and local revision hashing. This drastically **speeds up the document insertion process** on the receiving end.
3. **Optimised Network Traffic**:
Because conflicts are resolved deterministically and revision trees are replicated exactly as they exist, the system avoids generating 'echoes' (redundant synchronisations triggered by a device misunderstanding a history tree). This reduces unnecessary background traffic significantly.
### Consideration and Conclusion
The Journal Replicator 2nd Edition achieves robust and scalable storage synchronisation through enhanced memory efficiency (via Web Streams), decoupled extensibility (via IJournalStorage), and strict protocol compliance (via `new_edits: false`).
Moving forward, this foundation will make it much easier to officially support a wider variety of backend storages.
+106
View File
@@ -0,0 +1,106 @@
# Keep newborn chunks in Eden
Notice: deprecated. please refer to the result section of this document.
## Goal
Reduce the number of chunks which in volatile, and reduce the usage of storage of the remote database in middle or long term.
## Motivation
- In the current implementation, Self-hosted LiveSync splits documents into metadata and multiple chunks. In particular, chunks are split so that they do not exceed a certain length.
- This is to optimise the transfer and take advantage of the properties of CouchDB. This also complies with the restriction of IBM Cloudant on the size of a single document.
- However, creating chunks halfway through each editing operation increases the number of unnecessary chunks.
- Chunks are shared by several documents. For this reason, it is not clear whether these chunks are needed or not unless all revisions of all documents are checked. This makes it difficult to remove unnecessary data.
- On the other hand, chunks are done in units that can be neatly divided as markdown to ensure relatively accurate de-duplication, even if they are created simultaneously on multiple terminals. Therefore, it is unlikely that the data in the editing process will be reused.
- For this reason, we have made features such as Batch save available, but they are not a fundamental solution.
- As a result, there is a large amount of data that cannot be erased and is probably unused. Therefore, `Fetch chunks on demand` is currently performed for optimal communication.
- If the generation of unnecessary chunks is sufficiently reduced, this function will become unnecessary.
- The problem is that this unnecessary chunking slows down both local and remote operations.
## Prerequisite
- The implementation must be able to control the size of the document appropriately so that it does not become non-transferable (1).
- The implementation must be such that data corruption can be avoided even if forward compatibility is not maintained; due to the nature of Self-hosted LiveSync, backward version connexions are expected.
- Viewed as a feature:
- This feature should be disabled for migration users.
- This feature should be enabled for new users and after rebuilds of migrated users.
- Therefore, back into the implementation view, Ideally, the implementation should be such that data recovery can be achieved by immediately upgrading after replication.
## Outlined methods and implementation plans
### Abstract
To store and transfer only stable chunks independently and share them from multiple documents after stabilisation, new chunks, i.e. chunks that are considered non-stable, are modified to be stored in the document and transferred with the document. In this case, care should be taken not to exceed prerequisite (1).
If this is achieved, the non-leaf document will not be transferred, and even if it is, the chunk will be stored in the document, so that the size can be reduced by the compaction.
Details are given below.
1. The document will henceforth have the property eden.
```typescript
// Paritally Type
type EntryWithEden = {
eden: {
[key: DocumentID]: {
data: string;
epoch: number; // The document revision which this chunk has been born.
};
};
};
```
2. The following configuration items are added:
Note: These configurations should be shared as `Tweaks value` between each client.
- useEden : boolean
- Max chunks in eden : number
- Max Total chunk lengths in eden: number
- Max age while in eden: number
3. In the document saving operation, chunks are added to Eden within each document, having the revision number of the existing document. And if some chunks in eden are not used in the operating revision, they would be removed.
Then after being so chosen, a few chunks are also chosen to be graduated as an independent `chunk` in following rules, and they would be left the eden:
- Those that have already been confirmed to exist as independent chunks.
- This confirmation of existence may ideally be determined by a fast first-order determination, e.g. by a Bloom filter.
- Those whose length exceeds the configured maximum length.
- Those have aged over the configured value, since epoch at the operating revision.
- Those whose total length, when added up when they are arranged in reverse order of the revision in which they were generated, is after the point at which they exceed the max length in the configuration. Or, those after the configured maximum items.
4. In the document loading operation, chunks are firstly read from these eden.
5. In End-to-End Encryption, property `eden` of documents will also be encrypted.
### Note
- When this feature has been enabled, forward compatibility is temporarily lost. However, it is detected as missing chunks, and this data is not reflected in the storage in the old version. Therefore, no data loss will occur.
## Test strategy
1. Confirm that synchronisation with the previous version is possible with this feature disabled.
2. With this feature enabled, connect from the previous version and confirm that errors are detected in the previous version but the files are not corrupted.
3. Ensure that the two versions with this feature enabled can withstand normal use.
## Documentation strategy
- This document is published, and will be referred from the release note.
- Indeed, we lack a fulfilled configuration table. Efforts will be made and, if they can be produced, this document will then be referenced. But not required while in the experimental or beta feature.
- However, this might be an essential feature. Further efforts are desired.
## Results from actual operation
After implementing this feature, we have been using it for a while. The following results were obtained.
- Drawbacks were thought not to be a problem, but they were actually a problem:
- A document with `Eden` has a quite larger history compared to a document without `Eden`.
- Self-hosted LiveSync does not perform compaction aggressively, which results in the remote database becoming partially bloated.
- Compaction of the Remote Database (CouchDB) requires the same amount of free space as the size of the database. Therefore, it is not possible to perform compaction on a remote database if we reached to the maximum size of the database. It means that when we detect it, it is too late.
- We have mentioned that `We need compaction` in previous sections. However, but it was so hard to be determined whether the compaction is required or not, until the database is bloated. (Of course, it requires some time to compact the database, and, literally, some document loses its history. It is not a good idea to perform frequently and meaninglessly. We need manual decision, but indeed difficult to normal users).
### Consideration and Conclusion
This feature results in two aspects:
- For the users who are familiar with the CouchDB, this feature is a bit useful. They can watch and handle the database by themselves.
- For the users who are not familiar with the CouchDB, i.e., normal users, this feature is not so useful, either. They are not familiar with the database, and they do not know how to handle it. Therefore, they cannot decide whether the compaction is required or not.
Hence, this feature would be kept as an experimental feature, but it is not enabled by default. In addition to that, it is marked as deprecated. Detailed notice will be noisy for the users who are not familiar with the CouchDB. Details would be kept in this document, for the future.
It is not recommended to use this feature, unless the person who is familiar with the CouchDB and the database management.
Vorotamoroz has written this document. Bias: I am the first author of this plug-in, familiar with the CouchDB.
Research and development has been frozen on 2025-04-11. But, bugs will be fixed if they are found. Please feel free to report them.
@@ -0,0 +1,206 @@
# The design document of remote configuration management
## Goal
- Allow us to manage multiple remote connections in a single vault.
- Keep the existing synchronisation implementations working without requiring a large rewrite.
- Provide a safe migration path from the previous single-remote configuration model.
- Allow connections to be imported and exported in a compact and reusable format.
## Motivation
Historically, Self-hosted LiveSync stored one effective remote configuration directly in the main settings. This was simple, but it had several limitations.
- We could only keep one CouchDB, one bucket, or one Peer-to-Peer target as the effective configuration at a time.
- Switching between same-type-remotes required manually rewriting the active settings.
- Setup URI, QR code, CLI setup, and similar entry points all restored settings differently, which made migration logic easy to miss.
- The internal settings shape had gradually become a mix of user-facing settings, transport-specific credentials, and compatibility-oriented values.
Once multiple remotes of the same type became desirable, the previous model no longer scaled well enough. We therefore needed a structure that could store many remotes, still expose one effective remote to the replication logic, and keep migration and import behaviour consistent.
## Prerequisite
- Existing synchronisation features must continue to read an effective remote configuration from the current settings.
- Existing vaults must continue to work without requiring manual reconfiguration.
- Setup URI, QR code, CLI setup, protocol handlers, and other imported settings must be normalised in the same way.
- Import and export must be compact enough to be shared easily.
- We must be explicit that exported connection strings may contain credentials or secrets.
## Outlined methods and implementation plans
### Abstract
The current settings now have two layers for remote configuration.
1. A stored collection of named remotes.
2. One active remote projected into the legacy flat settings fields.
This means the replication and database layers can continue to read the effective remote from the existing settings fields, while the settings dialogue and migration logic can manage many stored remotes.
In short, the list is the source of truth for saved remotes, and the legacy fields remain the runtime compatibility layer.
### Data model
The main settings now contain the following properties.
```typescript
type RemoteConfiguration = {
id: string;
name: string;
uri: string;
isEncrypted: boolean;
};
type RemoteConfigurations = {
remoteConfigurations: Record<string, RemoteConfiguration>;
activeConfigurationId: string;
};
```
Each entry stores a connection string in `uri`.
- `sls+http://` or `sls+https://` for CouchDB-compatible remotes
- `sls+s3://` for bucket-style remotes
- `sls+p2p://` for Peer-to-Peer remotes
This structure allows multiple remotes of the same type to be stored without adding a large number of duplicated settings fields.
### Runtime compatibility
The replication logic still reads the effective remote from legacy flat settings such as the following.
- `remoteType`
- `couchDB_URI`, `couchDB_USER`, `couchDB_PASSWORD`, `couchDB_DBNAME`
- `endpoint`, `bucket`, `accessKey`, `secretKey`, and related bucket fields
- `P2P_roomID`, `P2P_passphrase`, and related Peer-to-Peer fields
When a remote is activated, its connection string is parsed and projected into these legacy fields. Therefore, existing services do not need to know whether the remote came from an old vault, a Setup URI, or the new remote list.
This projection is intentionally one-way at runtime. The stored remote list is the persistent catalogue, while the flat fields describe the remote currently in use.
### Connection string format
The connection string is the transport-neutral storage format for a remote entry.
Benefits:
- It is compact enough for clipboard-based workflows.
- It can be used for import and export in the settings dialogue.
- It avoids introducing a separate serialisation format only for the remote list.
- It can be parsed into the legacy settings shape whenever the active remote changes.
This is not equivalent to Setup URI.
- Setup URI represents a broader settings transfer workflow.
- A remote connection string represents one remote only.
### Import and export
The settings dialogue now supports the following workflows.
- Add connection: create a new remote by using the remote setup dialogues.
- Import connection: paste a connection string, validate it, and save it as a named remote.
- Export: copy a stored remote connection string to the clipboard.
Import normalises the string by parsing and serialising it again before saving. This ensures that equivalent but differently formatted URIs are saved in a canonical form.
Export is intentionally simple. It copies the connection string itself, because this is the most direct representation of one remote entry.
### Security note
Connection strings may include credentials, secrets, JWT-related values, or Peer-to-Peer passphrases.
Therefore:
- Export is a deliberate clipboard operation.
- Import trusts the supplied connection string as-is after parsing.
- We should regard exported connection strings as sensitive information, much like Setup URI or a credentials-bearing configuration file.
The `isEncrypted` field is currently reserved for future expansion. At present, the connection string itself is stored plainly inside the settings data, in the same sense that the effective runtime configuration can contain usable remote credentials.
### Migration strategy
Older vaults store only one effective remote in the flat settings fields. The migration creates a first remote list from those values.
Rules:
- If no remote list exists and the legacy fields contain a CouchDB configuration, create `legacy-couchdb`.
- If no remote list exists and the legacy fields contain a bucket configuration, create `legacy-s3`.
- If no remote list exists and the legacy fields contain a Peer-to-Peer configuration, create `legacy-p2p`.
- If more than one legacy remote is populated, create all possible entries and select the active one according to `remoteType`.
This migration is intentionally additive. It does not remove the flat fields because they remain necessary as the active runtime projection.
### Normalisation and application paths
One important design lesson from this work is that migration cannot rely only on loading `data.json`.
Settings may enter the system from several routes:
- normal settings load
- Setup URI
- QR code
- protocol handler
- CLI setup
- Peer-to-Peer remote configuration retrieval
- red flag based remote adjustment
- settings markdown import
To keep behaviour consistent, normalisation is centralised in the settings service.
- `adjustSettings` is responsible for in-place normalisation and migration of a settings object.
- `applyExternalSettings` is responsible for applying imported or externally supplied settings after passing them through the same normalisation flow.
This ensures that imported settings can migrate to the current remote list model even if they never passed through the ordinary `loadSettings` path.
### Why not store only the remote list
It would be possible to let all consumers parse the active remote every time and stop using the flat fields entirely. However, this would require broader changes across replication, diagnostics, and compatibility layers.
The current design keeps the change set limited.
- The remote list improves storage and UX.
- The flat fields preserve compatibility and reduce migration risk.
This is a pragmatic transitional architecture, not an accidental duplication.
## Test strategy
The feature should be tested from four viewpoints.
1. Migration from old settings.
- A vault with only legacy flat remote settings should gain a remote list automatically.
- The correct active remote should be selected according to `remoteType`.
2. Runtime activation.
- Activating a stored remote should correctly project its values into the effective flat settings.
3. External import paths.
- Setup URI, QR code, CLI setup, Peer-to-Peer remote config, red flag adjustment, and settings markdown import should all pass through the same normalisation path.
4. Import and export.
- Imported connection strings should be parsed, canonicalised, named, and stored correctly.
- Export should copy the exact saved connection string.
## Documentation strategy
- This document explains the design and compatibility model of remote configuration management.
- User-facing setup documents should explain only how to add, import, export, and activate remotes.
- Release notes may refer to this document when changes in remote handling are significant.
## Outlook
Import/export configuration strings should also be encrypted in the future, but this is a separate feature that can be added on top of the current design.
## Consideration and conclusion
The remote configuration list solves the practical need to manage multiple remotes without forcing the whole codebase to abandon the previous effective-settings model at once.
Its core idea is modest but effective.
- Store named remotes as connection strings.
- Select one active remote.
- Project it into the legacy settings for runtime use.
- Normalise every imported settings object through the same path.
This keeps the implementation understandable and migration-friendly. It also opens the door for future work, such as encrypted per-remote storage, richer remote metadata, or remote-scoped options, without forcing another large redesign of how remotes are represented.
@@ -0,0 +1,55 @@
# Sharing `Tweak values`
NOTE: This is the planned feature design document. This is planned, but not be implemented now (v0.23.3). This has not reached the design freeze and will be added to from time to time.
## Goal
Share `Tweak values` between clients to match the chunk lengths, and match per-server configurations for better performance.
## Motivation
- In the current implementation, Self-hosted LiveSync splits documents into metadata and multiple chunks. In particular, chunks are split so that they do not exceed a certain length.
- This is to optimise the transfer and take advantage of the properties of CouchDB. This also complies with the restriction of IBM Cloudant on the size of a single document.
- The length of this chunk is adjusted according to a configured factor. Therefore, if this is inconsistent between clients, de-duplication will not work. This is because, in fact, they point to the same content in total, but are split in different places. This results in unnecessary transfers or storage consumption.
- The same applies to hash algorithms.
- There are more configurations which `preferred to be matched`, even if it is not required. such as the maximum size of files to be handled and the interval between requests to the remote database, unless there are specific circumstances.
- To avoid the tragedy of "Too many toggles", "Unexpected transfer amount", or "Poor performance" at once, the plug-in should know these problems or potential problems and be able to let us know.
## Prerequisite
- We must be informed of a discrepancy in a configured value that is required to be absolutely consistent and be able to make a decision on the spot.
- We should be able to see on the configuration dialogue, that there is a discrepancy between configured values that should be matched, and it should be possible to adjust them to a specific one of them (or default).
- We must not be exposed to unexpected; such as leaking credentials or their secrets.
## Outlined methods and implementation plans
### Abstract
- In the current implementation, each client checks the remote database for the existence of their node information, to detect whether the remote database accepts them.
- This is what 'Lock' is all about.
- To achieve this feature, the client will also send each configuration value. However, the configuration contains credentials and/or secret values. Hence we cannot send all of them.
- With a favourable prediction, Self-hosted LiveSync will continue to increase in feature. Each time this happens, the number of configuration values to be kept secret will also increase. Therefore, they must be handled by an allow-list.
- This allow-listed configuration are the `Tweak values`.
- If the plug-in detects mismatched `Tweak values` on checking the remote database, the plug-in will ask us to decide which is win (Mine, or theirs).
- Node information is one of the documents. Therefore, it will be replicated and saved locally. While showing dialogue, show the notice on each `Match preferred` configuration.
## Note
This feature should be mostly harmless. We will not be able to disable this.
## Test strategy
A: During synchronisation.
1. No message shall be displayed with all settings matched.
2. Message shall be displayed when there are mismatched, required match items.
1. The setting values can be changed according to the message.
2. The message can be ignored.
3. The message shall not be displayed even if there are mismatched items which is recommended to be matched.
B: On the setting dialogue.
1. All mismatched items shall be highlighted in some way.
## Documentation strategy
- This document is published, and will be referred from the release note.
- Indeed, we lack a fulfilled configuration table. Efforts will be made and, if they can be produced, this document will then be referenced. But not required while in the experimental or beta feature.
- However, this might be an essential feature. Further efforts are desired.
### Consideration and Conclusion
To be described after implemented, tested, and, released.
+62
View File
@@ -0,0 +1,62 @@
# User Guide: Peer-to-Peer Synchronisation (2026 Edition)
Peer-to-Peer (P2P) synchronisation has evolved significantly. This guide covers the essential setup and the new features introduced in the 2026 updates.
## 1. Core Concept: Server-less Freedom
P2P synchronisation allows your devices to talk directly to each other using WebRTC. A central server is not required for data storage, ensuring maximum privacy and "freedom."
## 2. Setting Up via P2P Status Pane
You no longer need to navigate through complex menus. Simply open the **P2P Status** (via the ribbon icon or command palette) and click the **⚙ (Cog)** icon.
This opens the **P2P Setup** dialogue where you can configure the essentials:
- **Room ID:** A unique identifier for your synchronisation group.
- **Passphrase:** Your encryption key. Ensure all your devices use the exact same passphrase.
- **Device Name:** A recognisable name for the current device (e.g., `iphone-16`).
Once you have saved the settings, return to the **P2P Status Pane** and click the **Connect** button to join the network.
*Tip: You can also toggle **Auto Connect** in the setup dialogue to automatically join the network whenever Obsidian starts.*
## 3. Real-time Control
The status pane in the right sidebar provides granular control over your synchronisation:
- **Active P2P Remote (new):** P2P now has its own active remote selection, separate from the normal active remote for database replication. Use the combo box next to the cog icon to choose which P2P remote configuration is active for P2P features.
- **Create P2P Remote (new):** Use the **+** button to open the P2P setup dialogue and create a dedicated P2P remote configuration. This is recommended when no P2P active remote has been selected yet.
- **Selection required (new):** If no P2P active remote is selected, the pane asks for selection before P2P target-related changes are saved.
- **Signalling Status:** Shows if you are connected to the relay (🟢 Online).
- **Live-push (Broadcast):** Toggle "Broadcast changes" to notify other peers whenever you make an edit.
- **Replicate now (🔄):** Start immediate bidirectional replication with a visible peer (Pull, then Push).
- **Watch (🔔/🔕):** Enable "Watch" on specific peers to automatically pull changes when they broadcast. This creates a "LiveSync-like" experience.
- **Sync target (🔗/⛓️‍💥):** Mark specific peers as **sync targets**. Peers marked here will be included when you run the **"P2P: Sync with targets"** command (see section 5). Click the button next to a peer to toggle it on (🔗, highlighted) or off (⛓️‍💥). This setting is persisted in your configuration.
## 4. Replication Dialogue
If you want to synchronise with a specific peer manually, use the **Replication** command or button. This opens the **Replication Dialogue** listing available devices.
Inside the dialogue, the **Server Status** card at the top confirms you are still connected while performing the sync.
The status card now shows a stable **Room ID suffix** above **Peer ID**. The Room ID suffix is better for identifying your P2P group, while Peer ID may change between connections.
Two actions are available per peer:
- **Sync** — Starts a bidirectional synchronisation (Pull then Push) and keeps the dialogue open so you can monitor progress or sync with additional peers.
- **Start Sync & Close** — Runs the same bidirectional synchronisation, waits for it to settle, then closes the dialogue. After a successful synchronisation, it also closes the signalling connection.
On supported mobile and desktop devices, LiveSync keeps the screen awake while this peer-selection dialogue is open and while its synchronisations finish. This is intentional: display sleep can interrupt peer discovery or connection establishment and require detection to start again. Wake Lock support remains best effort, does not keep a hidden application running in the background, and does not override operating-system sleep or suspension.
During a P2P rebuild, peer discovery and selection remain protected after the rebuild has started. Keep Obsidian visible until a peer is selected and the transfer finishes, because mobile platform restrictions can still pause or terminate a hidden application.
## 5. Syncing with Registered Targets via Command Palette
You can now trigger a synchronisation with all your pre-registered target peers in one step, without opening any UI.
1. Open the **Command Palette** (`Ctrl/Cmd + P`).
2. Run **"P2P: Sync with targets"**.
This command synchronises with every peer whose **SYNC** toggle is enabled in the **Detected Peers** list. If no targets are registered, or if the P2P server is not running, the command will notify you accordingly.
*Tip: Pair this command with a hotkey for a quick, keyboard-driven sync workflow.*
## 6. Technical Improvements in 2026
- **Decoupled Architecture:** The UI is now strictly separated from the core logic, making the plug-in more stable across different platforms (Mobile, Desktop, and Web).
- **Svelte 5 UI:** The interface has been rebuilt for better responsiveness and clearer status indicators.
- **Security:** All data remains end-to-end encrypted. Even the signalling relay never sees your actual notes.
+84 -39
View File
@@ -1,46 +1,82 @@
# Quick setup
The plugin has so many configuration options to deal with different circumstances. However, there are not so many settings that are actually used. Therefore, `The Setup wizard` has been implemented to simplify the initial setup.
Note: Subsequent devices are recommended to be set up using the `Copy setup URI` and `Open setup URI`.
[Japanese docs](./quick_setup_ja.md) - [Chinese docs](./quick_setup_cn.md).
## The Setup wizard
Open the `🧙‍♂️ Setup wizard` in the settings dialogue. If the plugin has not been configured before, it should already be open.
The plug-in has so many configuration options to deal with different circumstances. However, only a few settings are required in the normal cases. Therefore, `The Setup wizard` has been implemented to simplify the setup.
![](../images/quick_setup_1.png)
- Discard the existing configuration and set up
If you have changed any settings, this button allows you to discard all changes before setting up.
There are three methods to set up Self-hosted LiveSync.
- Do not discard the existing configuration and set up
Simply reconfigure. Be careful. In wizard mode, you cannot see all configuration items, even if they have been configured.
1. [Using setup URIs](#1-using-setup-uris) *(Recommended)*
2. [Minimal setup](#2-minimal-setup)
3. [Fully manual setup and enabling on this dialogue](#3-manually-setup)
Pressing `Next` on one of the above options will put the configuration dialog into wizard mode.
## At the first device
### Wizard mode
### 1. Using setup URIs
> [!TIP]
> What is the setup URI? Why is it required?
> The setup URI is the encrypted representation of Self-hosted LiveSync configuration as a URI. This starts `obsidian://setuplivesync?settings=`. This is encrypted with a passphrase, so that it can be shared relatively securely between devices. It is a bit long, but it is one line. This allows a series of settings to be set at once without any inconsistencies.
>
> If you have configured the remote database by [Automated setup on Fly.io](./setup_flyio.md#a-very-automated-setup) or [set up your server with the tool](./setup_own_server.md#1-generate-the-setup-uri-on-a-desktop-device-or-server), **you should have one of them**
In this procedure, [this video](https://youtu.be/7sa_I1832Xc?t=146) may help us.
1. Click the `Use` button (or launch the `Use the copied setup URI (Formerly Open setup URI)` command from the command palette).
2. Paste the Setup URI into the dialogue
3. Type the passphrase of the Setup URI
4. Answer `yes` for `Importing LiveSync's conf, OK?`.
5. Answer `Set it up as secondary or subsequent device` for `How would you like to set it up?`.
6. Initialisation will begin, please hold a while.
7. You will asked about the hidden file synchronisation, answer as you like.
1. If you are new to Self-hosted LiveSync, we can configure it later so leave it once.
8. Synchronisation has been started! `Reload app without saving` is recommended after the indicators of Self-hosted LiveSync disappear.
OK, we can proceed the [next step](#).
### 2. Minimal setup
If you do not have any setup URI, Press the `start` button. The setting dialogue turns into the wizard mode and will display only minimal items.
>[!TIP]
> We can generate the setup URI with the tool in any time. Please use [this tool](./setup_own_server.md#1-generate-the-setup-uri-on-a-desktop-device-or-server).
![](../images/quick_setup_2.png)
Let's see how to use it step-by-step.
## Remote Database configuration
#### Select the remote type
### Remote database configuration
1. Select the Remote Type from dropdown list.
We now have a choice between CouchDB (and its compatibles) and object storage (MinIO, S3, R2). CouchDB is the first choice and is also recommended. And supporting Object Storage is an experimental feature.
Enter the information for the database we have set up.
#### Remote configuration
##### CouchDB
Enter the information for the database we have set up.
![](../images/quick_setup_3.png)
##### Object Storage
#### Test database connection and Check database configuration
1. Enter the information for the S3 API and bucket.
![](../images/quick_setup_3b.png)
Note 1: if you use S3, you can leave the Endpoint URL empty.
Note 2: if your Object Storage cannot configure the CORS setting fully, you may able to connect to the server by enabling the `Use Custom HTTP Handler` toggle.
2. Press `Test` of `Test Connection` once and ensure you can connect to the Object Storage.
#### Only CouchDB: Test database connection and Check database configuration
We can check the connectivity to the database, and the database settings.
![](../images/quick_setup_5.png)
#### Test Database Connection
Check whether we can connect to the database. If it fails, there are several possible reasons, but first attempt the `Check database configuration` check to see if it fails there too.
#### Check database configuration
#### Only CouchDB: Check and Fix database configuration
Check the database settings and fix any problems on the spot.
@@ -49,40 +85,49 @@ Check the database settings and fix any problems on the spot.
This item may vary depending on the connection. In the above case, press all three Fix buttons.
If the Fix buttons disappear and all become check marks, we are done.
### Confidentiality configuration
#### Confidentiality configuration (Optional but very preferred)
![](../images/quick_setup_4.png)
Encrypt your database in case of unintended database exposure; enable End to End encryption and the contents of your notes will be encrypted at the moment it leaves the device. We strongly recommend enabling it. And `Path Obfuscation` also obfuscates filenames. Now stable and recommended.
Encryption is based on 256-bit AES-GCM.
Enable End-to-end encryption and the contents of your notes will be encrypted at the moment it leaves the device. We strongly recommend enabling it. And `Path Obfuscation` also obfuscates filenames. Now stable and recommended.
These setting can be disabled if you are inside a closed network and it is clear that you will not be accessed by third parties.
![](../images/quick_setup_7.png)
> [!TIP]
> Encryption is based on 256-bit AES-GCM.
#### Next
Go to the Sync Settings.
We should proceed to the Next step.
#### Discard existing database and proceed
Discard the contents of the Remote database and go to the Sync Settings.
### Sync Settings
#### Sync Settings
Finally, finish the wizard by selecting a preset for synchronisation.
Note: If you are going to use Object Storage, you cannot select `LiveSync`.
![](../images/quick_setup_9_1.png)
Select any synchronisation methods we want to use and `Apply` to initialise and build the local and remote databases as required. If `All done!` is displayed, we are done. Automatically, `Copy setup URI` will open and we will be asked for a passphrase to encrypt the `Setup URI`.
Select any synchronisation methods we want to use and `Apply`. If database initialisation is required, it will be performed at this time. When `All done!` is displayed, we are ready to synchronise.
The dialogue of `Copy current settings as a new setup URI` will open automatically. Please input a passphrase to encrypt the new `Setup URI`. (This passphrase is to encrypt the setup URI, not the vault).
![](../images/quick_setup_10.png)
Set the passphrase as you like.
The Setup URI will be copied to the clipboard, which you can then transfer to the second and subsequent devices in some way.
The Setup URI will be copied to the clipboard, please make a note(Not in Obsidian) of this.
# How to set up the second and subsequent units
After installing Self-hosted LiveSync on the first device, select `Open setup URI` from the command palette and enter the setup URI you transferred. Afterwards, enter your passphrase and a setup wizard will open.
Answer the following.
>[!TIP]
We can copy this at any time by running the "Copy settings as a new setup URI" command from the command palette (or clicking the "Copy the current settings to a Setup URI" button in the settings UI).
- `Yes` to `Importing LiveSync's conf, OK?`
- `Set it up as secondary or subsequent device` to `How would you like to set it up?`.
### 3. Manually setup
Then, The configuration will take effect and replication will start. Your files will be synchronised soon! You may need to close the settings dialog and reopen it to see the settings fields populated properly, but they will be set.
It is strongly recommended to perform a "minimal set-up" first and set up the other contents after making sure has been synchronised.
However, if you have some specific reasons to configure it manually, please click the `Enable` button of `Enable LiveSync on this device as the set-up was completed manually`.
And, please copy the setup URI by running the "Copy settings as a new setup URI" command (or using the "Copy the current settings to a Setup URI" button) and make a note(Not in Obsidian) of this.
## At the subsequent device
After installing Self-hosted LiveSync on the first device, we should have a setup URI. **The first choice is to use it**. Please share it with the device you want to setup.
It is completely same as [Using setup URIs on the first device](#1-using-setup-uris). Please refer it.
> [!TIP]
> **Fast Setup (Simple Fetch)**
> In recent versions, when you import a Setup URI or trigger a Fetch All, the plug-in boots in scheduled fetch mode and runs a simplified **Fast Setup** process. This allows you to choose your sync strategy with a single dialogue and performs initial synchronisation in one step. Refer to the [Fast Setup Guide](./tips/fast-setup.md) for more details.
+93
View File
@@ -0,0 +1,93 @@
# 快速配置 (Quick setup)
该插件有较多配置项, 可以应对不同的情况. 不过, 实际使用的设置并不多. 因此, 我们采用了 "设置向导 (The Setup wizard)" 来简化初始设置.
Note: 建议使用 `Copy setup URI` and `Open setup URI` 来设置后续设备.
## 设置向导 (The Setup wizard)
在设置对话框中打开 `🧙‍♂️ Setup wizard`. 如果之前未配置插件, 则会自动打开该页面.
![quick_setup_1](../images/quick_setup_1.png)
- 放弃现有配置并进行设置
如果您先前有过任何设置, 此按钮允许您在设置前放弃所有更改.
- 保留现有配置和设置
快速重新配置. 请注意, 在向导模式下, 您无法看到所有已经配置过的配置项.
在上述选项中按下 `Next`, 配置对话框将进入向导模式 (wizard mode).
### 向导模式 (Wizard mode)
![quick_setup_2](../images/quick_setup_2.png)
接下来将介绍如何逐步使用向导模式.
## 配置远程数据库
### 开始配置远程数据库
输入已部署好的数据库的信息.
![quick_setup_3](../images/quick_setup_3.png)
#### 测试数据库连接并检查数据库配置
我们可以检查数据库的连接性和数据库设置.
![quick_setup_5](../images/quick_setup_5.png)
#### 测试数据库连接
检查是否能成功连接数据库. 如果连接失败, 可能是多种原因导致的, 但请先点击 `Check database configuration` 来检查数据库配置是否有问题.
#### 检查数据库配置
检查数据库设置并修复问题.
![quick_setup_6](../images/quick_setup_6.png)
Config check 的显示内容可能因不同连接而异. 在上图情况下, 按下所有三个修复按钮.
如果修复按钮消失, 全部变为复选标记, 则表示修复完成.
### 加密配置
![quick_setup_4](../images/quick_setup_4.png)
为您的数据库加密, 以防数据库意外曝光; 启用端到端加密后, 笔记内容在离开设备时就会被加密. 我们强烈建议启用该功能. `路径混淆 (Path Obfuscation)` 还能混淆文件名. 现已稳定并推荐使用.
加密基于 256 位 AES-GCM.
如果你在一个封闭的网络中, 而且很明显第三方不会访问你的文件, 则可以禁用这些设置.
![quick_setup_7](../images/quick_setup_7.png)
#### Next
转到同步设置.
#### 放弃现有数据库并继续
清除远程数据库的内容, 然后转到同步设置.
### 同步设置
最后, 选择一个同步预设完成向导.
![quick_setup_9_1](../images/quick_setup_9_1.png)
选择我们要使用的任何同步方法, 然后 `Apply` 初始化并按要求建立本地和远程数据库. 如果显示 `All done!`, 我们就完成了. `Copy setup URI` 将自动打开,并要求我们输入密码以加密 `Setup URI`.
![quick_setup_10](../images/quick_setup_10.png)
根据需要设置密码。.
设置 URI (Setup URI) 将被复制到剪贴板, 然后您可以通过某种方式将其传输到第二个及后续设备.
## 如何设置第二单元和后续单元 (the second and subsequent units)
在第一台设备上安装 Self-hosted LiveSync 后, 从命令面板上选择 `Open setup URI`, 然后输入您传输的设置 URI (Setup URI). 然后输入密码,安装向导就会打开.
在弹窗中选择以下内容.
- `Importing LiveSync's conf, OK?` 选择 `Yes`
- `How would you like to set it up?`. 选择 `Set it up as secondary or subsequent device`
然后, 配置将生效并开始复制. 您的文件很快就会同步! 您可能需要关闭设置对话框并重新打开, 才能看到设置字段正确填充, 但它们都将设置好.
+9 -4
View File
@@ -1,6 +1,6 @@
# Quick setup
このプラグインには、いろいろな状況に対応するための非常に多くの設定オプションがあります。しかし、実際に使用する設定項目はそれほど多くはありません。そこで、初期設定を簡略化するために、「セットアップウィザード」を実装しています。
※なお、次のデバイスからは、`Copy setup URI``Open setup URI`を使ってセットアップしてください。
※なお、次のデバイスからは、`現在の設定をセットアップURIにコピー``セットアップURIで接続`を使ってセットアップしてください。
## Wizardの使い方
@@ -71,7 +71,8 @@ Fixボタンがなくなり、すべてチェックマークになれば完了
![](../images/quick_setup_9_1.png)
Presetsから、いずれかの同期方法を選び`Apply`を行うと、必要に応じてローカル・リモートのデータベースを初期化・構築します。
All done! と表示されれば完了です。自動的に、`Copy setup URI`が開き、`Setup URI`を暗号化するパスフレーズを聞かれます。
All done!」(日本語環境では「完了!」)と表示されれば完了です。自動的に、「現在の設定をセットアップURIにコピー」のダイアログが開き、Setup URIを暗号化するためのパスフレーズを求められます(このパスフレーズはSetup URIを暗号化するためのもので、Vault自体の暗号化キーではありません)。
パスフレーズを入力すると、クリップボードにSetup URIが保存されますので、これを2台目以降のデバイスに何らかの方法で転送してください。
![](../images/quick_setup_10.png)
@@ -79,10 +80,14 @@ All done! と表示されれば完了です。自動的に、`Copy setup URI`が
クリップボードにSetup URIが保存されますので、これを2台目以降のデバイスに何らかの方法で転送してください。
# 2台目以降の設定方法
2台目の端末にSelf-hosted LiveSyncをインストールしたあと、コマンドパレットから`Open setup URI`を選択し、転送したsetup URIを入力します。その後、パスフレーズを入力するとセットアップ用のウィザードが開きます。
2台目の端末にSelf-hosted LiveSyncをインストールしたあと、コマンドパレットから`Use the copied setup URI (Formerly Open setup URI)`を選択し、転送したsetup URIを入力します。その後、パスフレーズを入力するとセットアップ用のウィザードが開きます。
下記のように答えてください。
- `Importing LiveSync's conf, OK?``Yes`
- `How would you like to set it up?``Set it up as secondary or subsequent device`
これで設定が反映され、レプリケーションが開始されます。
これで設定が反映され、レプリケーションが開始されます。
> [!TIP]
> **ファストセットアップ (Fast Setup)**
> 近年のバージョンでは、セットアップURIの読み込みやデータの全取得(Fetch All)を実行した際、より簡単に同期戦略を選択して即座に初期同期を完了できる **ファストセットアップ (Simple Fetch)** フローが利用できます。詳細は [ファストセットアップガイド](./tips/fast-setup_ja.md) をご参照ください。
+907 -181
View File
File diff suppressed because it is too large Load Diff
+305 -28
View File
@@ -3,23 +3,133 @@
# このプラグインの設定項目
## Remote Database Configurations
同期先のデータベース設定を行います。何らかの同期が有効になっている場合は編集できないため、同期を解除してから行ってください
同期先のデータベース設定Remote Serverを行います。
### URI
CouchDBのURIを入力します。Cloudantの場合は「External Endpoint(preferred)」になります。
**スラッシュで終わってはいけません。**
こちらにデータベース名を含めてもかまいません。
現在のバージョンでは、複数のリモート接続設定(接続プロファイル)を登録・管理し、切り替えて使用することが可能です(「Remote Databases」リスト)。
### Username
ユーザー名を入力します。このユーザーは管理者権限があることが望ましいです。
- ** 新規接続を追加 (Add new connection)**: 新しい接続設定を作成し、各セットアップダイアログを起動します。
- **📥 接続をインポート (Import connection)**: 接続文字列(`sls+https://...``sls+s3://...``sls+p2p://...`など)を貼り付けてインポートします。
- **🔧 設定 (Configure)**: セットアップダイアログを開き、選択した接続プロファイルの設定を編集します。
- **✅ 有効化 (Activate)**: 選択したプロファイルをアクティブな同期先として有効化します。
- **🗑️ 削除 (Delete)**: 接続プロファイルを一覧から削除します。
### Password
パスワードを入力します。
これらの接続プロファイルを追加・編集する際、選択したデータベースの種類(CouchDB、S3互換オブジェクトストレージ、P2Pなど)に応じたセットアップダイアログが開きます。
### Database Name
同期するデータベース名を入力します。
⚠️存在しない場合は、テストや接続を行った際、自動的に作成されます[^1]。
[^1]:権限がない場合は自動作成には失敗します。
何らかの同期が有効になっている場合は編集できないため、同期を解除してから行ってください。
### CouchDB の設定
CouchDBの各設定項目は、接続プロファイルを追加 (➕) または設定 (🔧) する際に開く **CouchDB セットアップダイアログ** 内で設定します。
#### URI
設定キー: couchDB_URI
CouchDBの接続先URIです。ダイアログ内では **URL** と表記されます。Cloudantの場合は「External Endpoint (preferred)」になります。
注意: Obsidian Mobileではセキュア接続 (HTTPS) のみが使用可能です。また、末尾にスラッシュ(`/`)を付けてはいけません。
#### Username
設定キー: couchDB_USER
CouchDBのログインユーザー名です。ダイアログ内では **Username** と表記されます。このユーザーには管理者権限があることが望ましいです。
#### Password
設定キー: couchDB_PASSWORD
CouchDBのログインパスワードです。ダイアログ内では **Password** と表記されます。
#### Database Name
設定キー: couchDB_DBNAME
同期先のデータベース名です。ダイアログ内では **Database Name** と表記されます。
注意: データベース名には大文字、スペース、および一部の特殊文字(`_$()+/-` 以外)は使用できません。また、アンダースコア(`_`)から始めることはできません。存在しない場合は、接続テスト時または設定適用時に自動作成されます(作成権限が必要です)。
#### CORS回避のためにRequest APIを使用する
設定キー: useRequestAPI
この項目はセットアップダイアログ内では **Use Internal API** と表記されます。有効な場合、不可避なCORS問題を回避するためにObsidianの内部Request APIを使用します。これはWeb標準に準拠していない回避策であり、すべての環境での動作を保証するものではありません。安全性が低下する可能性がある点にご注意ください。将来のObsidianのアップデートによって動作しなくなる可能性があります。
#### カスタムヘッダー
設定キー: couchDB_CustomHeaders
CouchDBサーバーに送信するすべてのリクエストに含めるカスタムHTTPヘッダーを設定します。ダイアログ内では **Custom Headers** と表記されます。`ヘッダー名: 値` の形式で、1行に1つずつ入力してください。
#### JWT認証の使用 (実験的機能)
設定キー: useJWT
CouchDBでのJSON Web Token (JWT) 認証を有効にします。ダイアログ内では **Use JWT Authentication** と表記されます。十分に検証されていない実験的機能であるため、ご注意ください。
#### JWTアルゴリズム
設定キー: jwtAlgorithm
JWTの署名に使用するアルゴリズムを選択します。ダイアログ内では **JWT Algorithm** と表記されます。対応アルゴリズム: `HS256`, `HS512`, `ES256`, `ES512`
#### JWT有効期限 (分)
設定キー: jwtExpDuration
トークンの有効期限を分単位で指定します。ダイアログ内では **JWT Expiration Duration (minutes)** と表記されます。`0` を指定すると有効期限は無効になります。
#### JWTキー
設定キー: jwtKey
JWTの署名に使用する秘密鍵またはプライベートキーを指定します。ダイアログ内では **JWT Key** と表記されます。`HS256/HS512` の場合は共通鍵を、`ES256/ES512` の場合は pkcs8 PEM形式の秘密鍵を入力してください。
#### JWTキーID (kid)
設定キー: jwtKid
JWTヘッダーに含めるキーIDを指定します。ダイアログ内では **JWT Key ID (kid)** と表記されます。
#### JWTサブジェクト (sub)
設定キー: jwtSub
JWTのサブジェクト (CouchDBユーザー名) を指定します。ダイアログ内では **JWT Subject (sub)** と表記されます。
### Object Storage (Minio, S3, R2) の設定
Object Storageの各設定項目は、接続プロファイルを追加 (➕) または設定 (🔧) する際に開く **S3/MinIO/R2 セットアップダイアログ** 内で設定します。
#### エンドポイントURL
設定キー: endpoint
S3互換ストレージのエンドポイントURLです。ダイアログ内では **Endpoint URL** と表記されます。
注意: Obsidian Mobileではセキュア接続 (HTTPS) のみが使用可能です。
#### アクセスキー ID
設定キー: accessKey
認証に使用するアクセスキーIDです。ダイアログ内では **Access Key ID** と表記されます。
#### シークレットアクセスキー
設定キー: secretKey
認証に使用するシークレットアクセスキーです。ダイアログ内では **Secret Access Key** と表記されます。
#### リージョン
設定キー: region
ストレージのリージョンを指定します(例: `us-east-1`、Cloudflare R2の場合は `auto`)。ダイアログ内では **Region** と表記されます。
#### バケット名
設定キー: bucket
同期データを保存するバケット名です。ダイアログ内では **Bucket Name** と表記されます。
#### カスタムHTTPハンドラーを使用する
設定キー: useCustomRequestHandler
この項目はセットアップダイアログ内では **Use internal API** と表記されます。オブジェクトストレージがCORSをサポートしていない場合に有効にします。Obsidianの内部APIを使用してS3サーバーと通信することでCORS制約を回避します。Web標準には準拠していないため、将来のObsidianのアップデートによって動作しなくなる可能性があります。
#### バケット内のファイルプレフィックス
設定キー: bucketPrefix
この項目はセットアップダイアログ内では **Folder Prefix** と表記されます。実質的なディレクトリ指定です。末尾は `/` である必要があります(例:`vault-name/`)。バケットのルートに保存する場合は空欄のままにしてください。
#### forcePathStyleを有効にする
設定キー: forcePathStyle
この項目はセットアップダイアログ内では **Use Path-Style Access** と表記されます。有効な場合、バケット操作でforcePathStyleオプションを使用します。
#### カスタムヘッダー
設定キー: bucketCustomHeaders
オブジェクトストレージバケットに送信するすべてのリクエストに含めるカスタムHTTPヘッダーを設定します。ダイアログ内では **Custom Headers** と表記されます。`ヘッダー名: 値` の形式で、1行に1つずつ入力してください。
@@ -30,6 +140,18 @@ CouchDBのURIを入力します。Cloudantの場合は「External Endpoint(prefe
### Passphrase
暗号化を行う際に使用するパスフレーズです。充分に長いものを使用してください。
### パスの難読化
設定キー: usePathObfuscation
ダイアログ内では **Obfuscate Properties** と表記されます。有効な場合、リモートサーバー上でのファイルパスやフォルダ名を難読化(暗号化)します。これによりプライバシーが向上しますが、パフォーマンスがわずかに低下する可能性があります。
### 暗号化アルゴリズム
設定キー: E2EEAlgorithm
ダイアログ内では **Encryption Algorithm** と表記されます。エンドツーエンド暗号化に使用する暗号化アルゴリズムのバージョンを選択します。
- `v2` (V2: AES-256-GCM With HKDF): 推奨されるデフォルトのバージョンです。
- `forceV1` または `""` (V1: Legacy): レガシーな暗号化バージョンです。古いバージョンで暗号化された既存の保管庫(Vault)を同期する場合にのみ使用してください。
### Apply
End to End 暗号化を行うに当たって、異なるパスフレーズで暗号化された同一の内容を入手されることは避けるべきです。また、Self-hosted LiveSyncはコンテンツのcrc32を重複回避に使用しているため、その点でも攻撃が有効になってしまいます。
@@ -53,12 +175,66 @@ End to End 暗号化を行うに当たって、異なるパスフレーズで暗
どちらのオペレーションも、実行するとすべての同期設定が無効化されます。
### Test Database connection
上記の設定でデータベースに接続できるか確認します。
### Check database configuration
ここから直接CouchDBの設定を確認・変更できます。
### Peer-to-Peer (P2P) 同期の設定
#### P2P同期を有効にする
設定キー: P2P_Enabled
WebRTCを介したデバイス間での直接的なP2P同期を有効にします。ダイアログ内では **Enabled** と表記されます。
#### リレーサーバーのURL
設定キー: P2P_relays
WebRTCによるP2P接続を仲介・調整するためのWebSocketリレーサーバーのURLを指定します。ダイアログ内では **Relay URL** と表記されます。複数のURLを指定する場合はカンマで区切ります。ダイアログ内のボタンをクリックすると、デフォルトのリレーサーバーを設定できます。
#### グループID
設定キー: P2P_roomID
同期するデバイス群を識別するためのルームIDまたはグループIDを指定します。ダイアログ内では **Group ID** と表記されます。同期させたいすべてのデバイスで同じグループIDを指定する必要があります。任意のカスタム文字列を入力するか、ランダム生成ボタンで生成できます。
#### パスフレーズ
設定キー: P2P_passphrase
P2P通信の認証および暗号化に使用するパスワード(パスフレーズ)を指定します。ダイアログ内では **Passphrase** と表記されます。同期するすべてのデバイスで同じパスフレーズを指定する必要があります。
#### デバイス名
設定キー: P2P_DevicePeerName
P2Pネットワーク上でこのデバイスを識別するための名前を指定します。ダイアログ内では **Device Peer ID** と表記されます。グループ内のデバイス間で重複しない一意の値を設定してください。
#### 起動時のP2P自動接続開始
設定キー: P2P_AutoStart
有効な場合、プラグインの起動時に自動的にP2P接続を開始します。ダイアログ内では **Auto Start P2P Connection** と表記されます。
#### 接続済みピアへの変更の自動ブロードキャスト
設定キー: P2P_AutoBroadcast
有効な場合、ローカルでの変更が接続済みのピアに自動的にブロードキャストされます。ダイアログ内では **Auto Broadcast Changes** と表記されます。通知されたピアは変更の取得を開始します。
#### TURNサーバーのURL (カンマ区切り)
設定キー: P2P_turnServers
ダイアログ内では **TURN Server URLs (comma-separated)** と表記されます。厳しいNATやファイアウォールがある環境で、WebRTCの直接接続が確立できない場合にP2P接続を中継するためのTURN/STUNサーバーのURLをカンマ区切りで指定します。通常は空欄のままで問題ありません。
#### TURNユーザー名
設定キー: P2P_turnUsername
TURNサーバーでの認証に使用するユーザー名を設定します。ダイアログ内では **TURN Username** と表記されます。
#### TURNパスワード
設定キー: P2P_turnCredential
TURNサーバーでの認証に使用するパスワード(クレデンシャル)を設定します。ダイアログ内では **TURN Credential** と表記されます。
## Local Database Configurations
端末内に作成されるデータベースの設定です。
@@ -71,7 +247,8 @@ End to End 暗号化を行うに当たって、異なるパスフレーズで暗
このオプションはLiveSyncと同時には使用できません。
### minimum chunk size と LongLine threshold
チャンクの分割についての設定です。
チャンクの分割についての設定です。※現在これらの項目はUIから直接設定することはできません(デフォルト値で自動処理されます)。
Self-hosted LiveSyncは一つのチャンクのサイズを最低minimum chunk size文字確保した上で、できるだけ効率的に同期できるよう、ノートを分割してチャンクを作成します。
これは、同期を行う際に、一定の文字数で分割した場合、先頭の方を編集すると、その後の分割位置がすべてずれ、結果としてほぼまるごとのファイルのファイル送受信を行うことになっていた問題を避けるために実装されました。
具体的には、先頭から順に直近の下記の箇所を検索し、一番長く切れたものを一つのチャンクとします。
@@ -88,6 +265,11 @@ Self-hosted LiveSyncは一つのチャンクのサイズを最低minimum chunk s
改行文字と#を除き、すべて●に置換しても、アルゴリズムは有効に働きます。
デフォルトは20文字と、250文字です。
### チャンクスプリッター
設定キー: chunkSplitterVersion
チャンク分割アルゴリズムを選択します。V3が最も効率的です。問題が発生した場合はDefaultまたはLegacyに設定してください。
## General Settings
一般的な設定です。
@@ -97,18 +279,35 @@ Self-hosted LiveSyncは一つのチャンクのサイズを最低minimum chunk s
### Vervose log
詳細なログをログに出力します。
### ファイル警告バナーの代わりにステータスアイコンを表示
設定キー: hideFileWarningNotice
有効な場合、ファイル警告バナーの代わりにステータス表示内に ⛔ アイコンが表示されます(詳細情報は非表示になります)。
### ネットワーク警告のスタイル
設定キー: networkWarningStyle
同期サーバーに接続できない場合のネットワークエラーの表示方法。
## Sync setting
同期に関する設定です。
### LiveSync
LiveSyncを行います。
他の同期方法では、同期の順序が「バージョン確認を行い、ロックが行われていないか確認した後、リモートの変更を受信した後、デバイスの変更を送信する」という挙動になります。
### 同期モード (Sync Mode)
設定キー: syncMode
### Periodic Sync
定期的に同期を行います。
同期処理を実行するトリガーとなる条件を設定します。
- **LiveSync** (`LIVESYNC`): リアルタイムかつ継続的な双方向同期を行います。
注意: このモードには CouchDB または WebRTC P2P リモートサーバーが必要です。S3互換オブジェクトストレージではサポートされていません。
- **Periodic Sync** (`PERIODIC`): **Periodic Sync Interval** で指定した一定の間隔ごとに同期処理を実行します。
- **On Events** (`ONEVENTS`): ファイルの保存、ファイルを開く、起動時など、特定のイベントが発生した際に同期をトリガーします(詳細は下部の設定スイッチで制御します)。
### Periodic Sync Interval
定期的に同期を行う場合の間隔です。
定期的に同期を行う場合の間隔(秒単位)です。
### 同期の最小間隔
設定キー: syncMinimumInterval
イベント時の自動同期の最小間隔(ミリ秒)。
### Sync on Save
ファイルが保存されたときに同期を行います。
@@ -146,6 +345,11 @@ Self-hosted LiveSyncは通常、フォルダ内のファイルがすべて削除
- Scan hidden files periodicaly.
このオプションを有効にすると、n秒おきに隠しファイルをスキャンします。
#### 非表示ファイルの変更通知を抑制
設定キー: suppressNotifyHiddenFilesChange
有効な場合、非表示ファイルの変更に関する通知を抑制します。
隠しファイルは能動的に検出されないため、スキャンが必要です。
スキャンでは、ファイルと共にファイルの変更時刻を保存します。もしファイルが消された場合は、その事実も保存します。このファイルを記録したエントリーがレプリケーションされた際、ストレージよりも新しい場合はストレージに反映されます。
@@ -176,6 +380,45 @@ Self-hosted LiveSyncはPouchDBを使用し、リモートと[このプロトコ
### Batch limit
一度に処理するBatchの数です。デフォルトは40です。
### 1回のリクエストで送信するチャンクの最大サイズ
設定キー: sendChunksBulkMaxSize
メガバイト(MB)単位で指定します。
## Customisation Sync (カスタマイズ同期)
プラグイン、ホットキー、テーマ、スニペットなどのObsidianのカスタマイズ設定を同期する機能です(以前は **Plugin Sync** と呼ばれていました)。
### デバイス名 (Device name)
設定キー: deviceAndVaultName
同期するすべてのデバイス間で一意となるデバイス名です。この設定を編集するには、一度カスタマイズ同期を無効にする必要があります。
### ファイル保存ごとのカスタマイズ同期 (Per-file-saved customisation sync)
設定キー: usePluginSyncV2
有効な場合、ファイルごとの効率的なカスタマイズ同期が使用されます。有効にする際には簡単な移行作業が必要であり、すべてのデバイスを v0.23.18 以降にアップデートする必要があります。この機能を有効にすると、古いバージョンとの互換性が失われます。
### カスタマイズ同期を有効にする (Enable customisation sync)
設定キー: usePluginSync
テーマ、スニペット、ホットキー、プラグイン設定などの同期を有効にします。
注意: 安全上の理由から、この機能を使用するにはエンドツーエンド暗号化(End-to-End Encryption)が有効になっている必要があります。
### カスタマイズの自動スキャン (Scan customisation automatically)
設定キー: autoSweepPlugins
レプリケーション(同期処理)を実行する前に、カスタマイズ設定の変更をスキャンします。
### 定期的なカスタマイズのスキャン (Scan customisation periodically)
設定キー: autoSweepPluginsPeriodic
1分ごとにカスタマイズ設定の変更を定期的にスキャンします。
### カスタマイズ更新の通知 (Notify customised)
設定キー: notifyPluginOrSettingUpdated
他のデバイスで新しくカスタマイズ設定が更新されたときに通知を表示します。
## Miscellaneous
その他の設定です
### Show status inside editor
@@ -195,8 +438,8 @@ Self-hosted LiveSyncはPouchDBを使用し、リモートと[このプロトコ
![CorruptedData](../images/lock_pattern1.png)
データベースがロックされていて、端末が「解決済み」とマークされていない場合、警告が表示されます。
他のデバイスで、End to End暗号化を有効にしたか、Drop Historyを行った等、他の端末がそのまま同期を行ってはいない状態に陥った場合表示されます。
暗号化を有効化した場合は、パスフレーズを設定してApply and recieve、Drop Historyを行った場合は、Drop and recieveを行うと自動的に解除されます。
手動でこのロックを解除する場合は「mark this device as resolved」をクリックしてください。
暗号化を有効化した場合は、パスフレーズを設定して「このデバイスの同期状態をリセット」、または「このデバイスのファイルでサーバーデータを上書き」を行うと自動的に解除されます。
手動でこのロックを解除する場合は「I've made a backup, mark this device 'resolved'」をクリックしてください。
- パターン2
![CorruptedData](../images/lock_pattern2.png)
@@ -207,18 +450,52 @@ Self-hosted LiveSyncはPouchDBを使用し、リモートと[このプロトコ
### Verify and repair all files
Vault内のファイルを全て読み込み直し、もし差分があったり、データベースから正常に読み込めなかったものに関して、データベースに反映します。
- Drop and send
デバイスとリモートのデータベースを破棄し、ロックしてからデバイスのファイルでデータベースを構築後、リモートに上書きします。
- Drop and receive
デバイスのデータベースを破棄した後、リモートから、操作しているデバイスに関してロックを解除し、データを受信して再構築します。
- このデバイスの同期状態をリセット (Reset Synchronisation on This Device)
ローカルのデータベースを破棄し、リモートのデータから再構築します。
- このデバイスのファイルでサーバーデータを上書き (Overwrite Server Data with This Device's Files)
ローカルおよびリモートのデータベースをこのデバイス上のファイルで再構築(上書き)します。
### Lock remote database
リモートのデータベースをロックし、他の端末で同期を行おうとしてもエラーとともに同期がキャンセルされるように設定します。これは、データベースの再構築を行った場合、自動的に設定されるものと同じものです。
万が一同期に不具合が発生していて、使用しているデバイスのデータ+サーバーのデータを保護する場合などに、緊急避難的に使用してください。
### Suspend file watching
ファイルの更新の監視を止めます。
### Scram スイッチ (Scram Switches)
データベースの破損や予期しないデータ喪失を防ぐために、同期処理を緊急停止するためのスイッチです。重大な設定不一致や同期エラーが発生した場合、プラグインは自動的に Scram 状態に移行し、同期動作を一時停止することがあります。
#### ファイルの更新監視を一時停止 (Suspend file watching)
設定キー: suspendFileWatching
ローカルファイル変更の監視と検知を停止します。
#### データベース反映を一時停止 (Suspend database reflecting)
設定キー: suspendParseReplicationResult
データベースでの変更をストレージファイル(Vault内のファイル)へ書き戻す処理を停止します。
### 互換性(メタデータ)(Compatibility (Metadata))
#### 削除済みファイルのメタデータを保持しない (Do not keep metadata of deleted files.)
設定キー: deleteMetadataOfDeletedFiles
ファイルを削除した際に、そのファイルの同期履歴メタデータも即座にデータベースから削除し、保持しないようにします。
#### 削除済みデータのメタデータをクリーンナップする (Delete old metadata of deleted files on start-up)
設定キー: automaticallyDeleteMetadataOfDeletedFiles
ファイルを削除した際のメタデータを保持する期間(日数)を設定します。指定した日数を経過した古い削除済みファイルのメタデータは、プラグイン起動時にデータベースから自動的に削除(クリーンナップ)されます。`0` を指定すると自動削除は無効になります。
### 破損している可能性があるファイルも処理する
設定キー: processSizeMismatchedFiles
サイズ不一致のあるファイルを処理します。特定のAPIや外部連携によって作成されたファイルを同期する際に役立ちます。
### Remediation
#### イベント反映時の最大ファイル更新日時
設定キー: maxMTimeForReflectEvents
この値(Unixエポックからの秒数)より新しい更新日時を持つファイルについては、イベントの反映を無視します。0を指定すると制限が無効になります。
### Corrupted data
![CorruptedData](../images/corrupted_data.png)
+9 -9
View File
@@ -13,7 +13,7 @@ In these instructions, create IBM Cloudant Instance for trial.
1. You can choose "Lite plan" for free.
![step 3](../instruction_images/cloudant_3.png)
1. Select Multitenant(it's the default) and the region as you like.
1. Select Multitenant (it is the default) and the region as you like.
![step 4](../instruction_images/cloudant_4.png)
1. Be sure to select "IAM and Legacy credentials" for "Authentication Method".
@@ -28,20 +28,20 @@ In these instructions, create IBM Cloudant Instance for trial.
1. When all of the above steps have been done, open "Resource list" on the left pane. you can see the Cloudant instance in the "Service and software". Click it.
![step 8](../instruction_images/cloudant_8.png)
1. In resource details, there's information to connect from Self-hosted LiveSync.
Copy the "External Endpoint(preferred)" address. <sup>(\*1)</sup>. We use this address later, with the database name.
1. In resource details, there is information to connect from Self-hosted LiveSync.
Copy the "External Endpoint (preferred)" address. <sup>(\*1)</sup>. We use this address later, with the database name.
![step 9](../instruction_images/cloudant_9.png)
## Database setup
1. Hit the "Launch Dashboard" button, Cloudant dashboard will be shown.
Yes, it's almost CouchDB's fauxton.
Yes, it is almost CouchDB's fauxton.
![step 1](../instruction_images/couchdb_1.png)
1. First, you have to enable the CORS option.
Hit the Account menu and open the "CORS" tab.
Initially, "Origin Domains" is set to "Restrict to specific domains"., so set to "All domains(\*)"
_NOTE: of course We want to set "app://obsidian.md" but it's not acceptable on Cloudant._
_NOTE: of course We want to set "app://obsidian.md" but it is not acceptable on Cloudant._
![step 2](../instruction_images/couchdb_2.png)
1. Next, Open the "Databases" tab and hit the "Create Database" button.
@@ -55,10 +55,10 @@ In these instructions, create IBM Cloudant Instance for trial.
### Credentials Setup
1. Back into IBM Cloud, Open the "Service credentials". You'll get an empty list, hit the "New credential" button.
1. Back into IBM Cloud, Open the "Service credentials". You will get an empty list, hit the "New credential" button.
![step 1](../instruction_images/credentials_1.png)
1. The dialog to create a credential will be shown.
1. The dialogue to create a credential will be shown.
type any name or leave it default, hit the "Add" button.
![step 2](../instruction_images/credentials_2.png)
_NOTE: This "name" is not related to your username that uses in Self-hosted LiveSync._
@@ -68,14 +68,14 @@ In these instructions, create IBM Cloudant Instance for trial.
![step 3](../instruction_images/credentials_3.png)
The username and password pair is inside this JSON.
"username" and "password" are so.
follow the figure, it's
follow the figure, it is
"apikey-v2-2unu15184f7o8emr90xlqgkm2ncwhbltml6tgnjl9sd5"<sup>(\*3)</sup> and "c2c11651d75497fa3d3c486e4c8bdf27"<sup>(\*4)</sup>
## Self-hosted LiveSync settings
![Setting](../images/remote_db_setting.png)
The Setting should be as below:
The settings should be as follows:
| Items | Value | example |
| ------------- | ----- | ----------------------------------------------------------------- |
+252
View File
@@ -0,0 +1,252 @@
<!-- For translation: 20240209r0 -->
# Setup CouchDB on fly.io
This is how to configure fly.io and CouchDB on it for Self-hosted LiveSync.
> [!WARNING]
> It is **your** instance. In Obsidian, we have files locally. Hence, do not hesitate to destroy the remote database if you feel something have got weird. We can launch and switch to the new CouchDB instance anytime[^1].
>
[^1]: Actually, I am always building the database for reproduction of the issue like so.
> [!NOTE]
> **What and why is the Fly.io?**
> At some point, we started to experience problems related to our IBM Cloudant account. At the same time, Self-hosted LiveSync started to improve its functionality, requiring CouchDB in a more natural state to use all its features.
>
> Then we found Fly.io. Fly.io is the PaaS Platform, which can be useable for a very reasonable price. It generally falls within the `Free Allowances` range in most cases.
## Required materials
- A valid credit or debit card.
## Setup CouchDB instance
### A. Very automated setup
[![LiveSync Setup onto Fly.io SpeedRun 2024 using Google Colab](https://img.youtube.com/vi/7sa_I1832Xc/0.jpg)](https://www.youtube.com/watch?v=7sa_I1832Xc)
1. Open [setup-flyio-on-the-fly-v2.ipynb](../setup-flyio-on-the-fly-v2.ipynb).
2. Press the `Open in Colab` button.
3. Choose a region and run all blocks (Refer to video).
1. If you do not have the account yet, the sign-up page will be shown, please follow the instructions. The [Official document is here](https://fly.io/docs/hands-on/sign-up/).
4. Copy the Setup-URI and Use it in the Obsidian.
5. You have been synchronised. Use the Setup-URI in subsequent devices.
Steps 4 and 5 are detailed in the [Quick Setup](./quick_setup.md#1-using-setup-uris).
> [!NOTE]
> Your automatically configured configurations will be shown on the result in the Colab note like below, and **it will not be saved**. Please make a note of it somewhere.
> ```
> -- YOUR CONFIGURATION --
> URL : https://billowing-dawn-6619.fly.dev
> username: billowing-cherry-22580
> password: misty-dew-13571
> region : nrt
> ```
### B. Scripted Setup
Please refer to the document of [deploy-server.sh](../utils/readme.md#deploy-serversh).
### C. Manual Setup
| Used in the text | Meaning and where to use | Memo |
| ---------------- | --------------------------- | ------------------------------------------------------------------------ |
| campanella | Username | It is less likely to fail if it consists only of letters and numbers. |
| dfusiuada9suy | Password | |
| nrt | Region to make the instance | We can use any [region](https://fly.io/docs/reference/regions/) near us. |
#### 1. Install flyctl
- Mac or Linux
```sh
$ curl -L https://fly.io/install.sh | sh
```
- Windows
```powershell
$ iwr https://fly.io/install.ps1 -useb | iex
```
#### 2. Sign up or Sign in to fly.io
- Sign up
```bash
$ fly auth signup
```
- Sign in
```bash
$ fly auth login
```
For more information, please refer to [Sign up](https://fly.io/docs/hands-on/sign-up/) and [Sign in](https://fly.io/docs/hands-on/sign-in/).
#### 3. Make a configuration file
1. Make `fly.toml` from template `fly.template.toml`.
We can simply copy and rename the file. The template is on [utils/flyio/fly.template.toml](../utils/flyio/fly.template.toml)
2. Decide the instance name, initialize the App, and set credentials.
>[!TIP]
> - The name `billowing-dawn-6619` is randomly decided name, and it will be a part of the CouchDB URL. It should be globally unique. Therefore, it is recommended to use something random for this name.
> - Explicit naming is very good for humans. However, we do not often get the chance to actually enter this manually (have designed so). This database may contain important information for you. The needle should be hidden in the haystack.
```bash
$ fly launch --name=billowing-dawn-6619 --env="COUCHDB_USER=campanella" --copy-config=true --detach --no-deploy --region nrt --yes
$ fly secrets set COUCHDB_PASSWORD=dfusiuada9suy
```
#### 4. Deploy
```
$ flyctl deploy
An existing fly.toml file was found
Using build strategies '[the "couchdb:latest" docker image]'. Remove [build] from fly.toml to force a rescan
Creating app in /home/vorotamoroz/dev/obsidian-livesync/utils/flyio
We're about to launch your app on Fly.io. Here's what you're getting:
Organization: vorotamoroz (fly launch defaults to the personal org)
Name: billowing-dawn-6619 (specified on the command line)
Region: Tokyo, Japan (specified on the command line)
App Machines: shared-cpu-1x, 256MB RAM (specified on the command line)
Postgres: <none> (not requested)
Redis: <none> (not requested)
Created app 'billowing-dawn-6619' in organization 'personal'
Admin URL: https://fly.io/apps/billowing-dawn-6619
Hostname: billowing-dawn-6619.fly.dev
Wrote config file fly.toml
Validating /home/vorotamoroz/dev/obsidian-livesync/utils/flyio/fly.toml
Platform: machines
✓ Configuration is valid
Your app is ready! Deploy with `flyctl deploy`
Secrets are staged for the first deployment
==> Verifying app config
Validating /home/vorotamoroz/dev/obsidian-livesync/utils/flyio/fly.toml
Platform: machines
✓ Configuration is valid
--> Verified app config
==> Building image
Searching for image 'couchdb:latest' remotely...
image found: img_ox20prk63084j1zq
Watch your deployment at https://fly.io/apps/billowing-dawn-6619/monitoring
Provisioning ips for billowing-dawn-6619
Dedicated ipv6: 2a09:8280:1::37:fde9
Shared ipv4: 66.241.124.163
Add a dedicated ipv4 with: fly ips allocate-v4
Creating a 1 GB volume named 'couchdata' for process group 'app'. Use 'fly vol extend' to increase its size
This deployment will:
* create 1 "app" machine
No machines in group app, launching a new machine
WARNING The app is not listening on the expected address and will not be reachable by fly-proxy.
You can fix this by configuring your app to listen on the following addresses:
- 0.0.0.0:5984
Found these processes inside the machine with open listening sockets:
PROCESS | ADDRESSES
-----------------*---------------------------------------
/.fly/hallpass | [fdaa:0:73b9:a7b:22e:3851:7f28:2]:22
Finished launching new machines
NOTE: The machines for [app] have services with 'auto_stop_machines = true' that will be stopped when idling
-------
Checking DNS configuration for billowing-dawn-6619.fly.dev
Visit your newly deployed app at https://billowing-dawn-6619.fly.dev/
```
#### 5. Apply CouchDB configuration
After the initial setup, CouchDB needs some more customisations to be used from Self-hosted LiveSync. It can be configured in browsers or by HTTP-REST APIs.
This section is set up using the REST API.
1. Prepare environment variables.
- Mac or Linux:
```bash
export couchHost=https://billowing-dawn-6619.fly.dev
export couchUser=campanella
export couchPwd=dfusiuada9suy
```
- Windows
```powershell
set couchHost https://billowing-dawn-6619.fly.dev
set couchUser campanella
set couchPwd dfusiuada9suy
$creds = [System.Convert]::ToBase64String([System.Text.Encoding]::UTF8.GetBytes("${couchUser}:${couchPwd}"))
```
2. Perform cluster setup
- Mac or Linux
```bash
curl -X POST "${couchHost}/_cluster_setup" -H "Content-Type: application/json" -d "{\"action\":\"enable_single_node\",\"username\":\"${couchUser}\",\"password\":\"${couchPwd}\",\"bind_address\":\"0.0.0.0\",\"port\":5984,\"singlenode\":true}" --user "${couchUser}:${couchPwd}"
```
- Windows
```powershell
iwr -UseBasicParsing -Method 'POST' -ContentType 'application/json; charset=utf-8' -Headers @{ 'Authorization' = 'Basic ' + $creds } "${couchHost}/_cluster_setup" -Body "{""action"":""enable_single_node"",""username"":""${couchUser}"",""password"":""${couchPwd}"",""bind_address"":""0.0.0.0"",""port"":5984,""singlenode"":true}"
```
Note: if the response code is not 200. We have to retry the request once again.
If you run the request several times and it does not result in 200, something is wrong. Please report it.
3. Configure parameters
- Mac or Linux
```bash
curl -X PUT "${couchHost}/_node/nonode@nohost/_config/chttpd/require_valid_user" -H "Content-Type: application/json" -d '"true"' --user "${couchUser}:${couchPwd}"
curl -X PUT "${couchHost}/_node/nonode@nohost/_config/chttpd_auth/require_valid_user" -H "Content-Type: application/json" -d '"true"' --user "${couchUser}:${couchPwd}"
curl -X PUT "${couchHost}/_node/nonode@nohost/_config/httpd/WWW-Authenticate" -H "Content-Type: application/json" -d '"Basic realm=\"couchdb\""' --user "${couchUser}:${couchPwd}"
curl -X PUT "${couchHost}/_node/nonode@nohost/_config/httpd/enable_cors" -H "Content-Type: application/json" -d '"true"' --user "${couchUser}:${couchPwd}"
curl -X PUT "${couchHost}/_node/nonode@nohost/_config/chttpd/enable_cors" -H "Content-Type: application/json" -d '"true"' --user "${couchUser}:${couchPwd}"
curl -X PUT "${couchHost}/_node/nonode@nohost/_config/chttpd/max_http_request_size" -H "Content-Type: application/json" -d '"4294967296"' --user "${couchUser}:${couchPwd}"
curl -X PUT "${couchHost}/_node/nonode@nohost/_config/couchdb/max_document_size" -H "Content-Type: application/json" -d '"50000000"' --user "${couchUser}:${couchPwd}"
curl -X PUT "${couchHost}/_node/nonode@nohost/_config/cors/credentials" -H "Content-Type: application/json" -d '"true"' --user "${couchUser}:${couchPwd}"
curl -X PUT "${couchHost}/_node/nonode@nohost/_config/cors/origins" -H "Content-Type: application/json" -d '"app://obsidian.md,capacitor://localhost,http://localhost"' --user "${couchUser}:${couchPwd}"
```
- Windows
```powershell
iwr -UseBasicParsing -Method 'PUT' -ContentType 'application/json; charset=utf-8' -Headers @{ 'Authorization' = 'Basic ' + $creds } "${couchHost}/_node/nonode@nohost/_config/chttpd/require_valid_user" -Body '"true"'
iwr -UseBasicParsing -Method 'PUT' -ContentType 'application/json; charset=utf-8' -Headers @{ 'Authorization' = 'Basic ' + $creds } "${couchHost}/_node/nonode@nohost/_config/chttpd_auth/require_valid_user" -Body '"true"'
iwr -UseBasicParsing -Method 'PUT' -ContentType 'application/json; charset=utf-8' -Headers @{ 'Authorization' = 'Basic ' + $creds } "${couchHost}/_node/nonode@nohost/_config/httpd/WWW-Authenticate" -Body '"Basic realm=\"couchdb\""'
iwr -UseBasicParsing -Method 'PUT' -ContentType 'application/json; charset=utf-8' -Headers @{ 'Authorization' = 'Basic ' + $creds } "${couchHost}/_node/nonode@nohost/_config/httpd/enable_cors" -Body '"true"'
iwr -UseBasicParsing -Method 'PUT' -ContentType 'application/json; charset=utf-8' -Headers @{ 'Authorization' = 'Basic ' + $creds } "${couchHost}/_node/nonode@nohost/_config/chttpd/enable_cors" -Body '"true"'
iwr -UseBasicParsing -Method 'PUT' -ContentType 'application/json; charset=utf-8' -Headers @{ 'Authorization' = 'Basic ' + $creds } "${couchHost}/_node/nonode@nohost/_config/chttpd/max_http_request_size" -Body '"4294967296"'
iwr -UseBasicParsing -Method 'PUT' -ContentType 'application/json; charset=utf-8' -Headers @{ 'Authorization' = 'Basic ' + $creds } "${couchHost}/_node/nonode@nohost/_config/couchdb/max_document_size" -Body '"50000000"'
iwr -UseBasicParsing -Method 'PUT' -ContentType 'application/json; charset=utf-8' -Headers @{ 'Authorization' = 'Basic ' + $creds } "${couchHost}/_node/nonode@nohost/_config/cors/credentials" -Body '"true"'
iwr -UseBasicParsing -Method 'PUT' -ContentType 'application/json; charset=utf-8' -Headers @{ 'Authorization' = 'Basic ' + $creds } "${couchHost}/_node/nonode@nohost/_config/cors/origins" -Body '"app://obsidian.md,capacitor://localhost,http://localhost"'
```
Note: Each of these should also be repeated until finished in 200.
#### 6. Use it from Self-hosted LiveSync
Now the CouchDB is ready to use from Self-hosted LiveSync. We can use `https://billowing-dawn-6619.fly.dev` in URI, `campanella` in `Username` and `dfusiuada9suy` in `Password` on Self-hosted LiveSync. The `Database name` could be anything you want.
Please refer to the [Minimal Setup of the Quick Setup](./quick_setup.md#2-minimal-setup).
## Delete the Instance
If you want to delete the CouchDB instance, you can do that in [fly.io Dashboard](https://fly.io/dashboard/personal)
If you have done with [B. Scripted Setup](#b-scripted-setup), we can use [delete-server.sh](../utils/readme.md#delete-serversh).
+354 -92
View File
@@ -1,125 +1,217 @@
# Setup a CouchDB server
## Configure
## Table of Contents
The easiest way to set up a CouchDB instance is using the official [docker image](https://hub.docker.com/_/couchdb).
- [Setup a CouchDB server](#setup-a-couchdb-server)
- [Table of Contents](#table-of-contents)
- [1. Prepare CouchDB](#1-prepare-couchdb)
- [A. Using Docker](#a-using-docker)
- [1. Prepare](#1-prepare)
- [2. Run docker container](#2-run-docker-container)
- [B. Using Docker Compose](#b-using-docker-compose)
- [1. Prepare](#1-prepare-1)
- [2. Creating Compose file](#2-create-a-docker-composeyml-file-with-the-following-added-to-it)
- [3. Boot check](#3-run-the-docker-compose-file-to-boot-check)
- [4. Starting Docker Compose in background](#4-run-the-docker-compose-file-in-the-background)
- [C. Install CouchDB directly](#c-install-couchdb-directly)
- [2. Run couchdb-init.sh for initialise](#2-run-couchdb-initsh-for-initialise)
- [3. Expose CouchDB to the Internet](#3-expose-couchdb-to-the-internet)
- [4. Client Setup](#4-client-setup)
- [1. Generate the setup URI on a desktop device or server](#1-generate-the-setup-uri-on-a-desktop-device-or-server)
- [2. Setup Self-hosted LiveSync to Obsidian](#2-setup-self-hosted-livesync-to-obsidian)
- [Manual setup information](#manual-setup-information)
- [Setting up your domain](#setting-up-your-domain)
- [Reverse Proxies](#reverse-proxies)
- [Traefik](#traefik)
- [Nginx](#nginx)
---
Some initial configuration is required. Create a `local.ini` to use Self-hosted LiveSync as follows:
## 1. Prepare CouchDB
### A. Using Docker
```ini
[couchdb]
single_node=true
max_document_size = 50000000
#### 1. Prepare
```bash
[chttpd]
require_valid_user = true
max_http_request_size = 4294967296
# Adding environment variables.
export hostname=localhost:5984
export username=goojdasjdas #Please change as you like.
export password=kpkdasdosakpdsa #Please change as you like
[chttpd_auth]
require_valid_user = true
authentication_redirect = /_utils/session.html
[httpd]
WWW-Authenticate = Basic realm="couchdb"
enable_cors = true
[cors]
origins = app://obsidian.md,capacitor://localhost,http://localhost
credentials = true
headers = accept, authorization, content-type, origin, referer
methods = GET, PUT, POST, HEAD, DELETE
max_age = 3600
# Creating the save data & configuration directories.
mkdir couchdb-data
mkdir couchdb-etc
```
## Run
#### 2. Run docker container
1. Boot Check.
```
$ docker run --name couchdb-for-ols --rm -it -e COUCHDB_USER=${username} -e COUCHDB_PASSWORD=${password} -v ${PWD}/couchdb-data:/opt/couchdb/data -v ${PWD}/couchdb-etc:/opt/couchdb/etc/local.d -p 5984:5984 couchdb
```
> [!WARNING]
> If your container threw an error or exited unexpectedly, please check the permission of couchdb-data, and couchdb-etc.
> Once CouchDB starts, these directories will be owned by uid:`5984`. Please chown it for that uid again.
### Docker CLI
You can launch CouchDB using your `local.ini` like this:
2. Enable it in the background
```
$ docker run --rm -it -e COUCHDB_USER=admin -e COUCHDB_PASSWORD=password -v /path/to/local.ini:/opt/couchdb/etc/local.ini -p 5984:5984 couchdb
```
*Remember to replace the path with the path to your local.ini*
Run in detached mode:
```
$ docker run -d --restart always -e COUCHDB_USER=admin -e COUCHDB_PASSWORD=password -v /path/to/local.ini:/opt/couchdb/etc/local.ini -p 5984:5984 couchdb
```
*Remember to replace the path with the path to your local.ini*
### Docker Compose
Create a directory, place your `local.ini` within it, and create a `docker-compose.yml` alongside it. The directory structure should look similar to this:
```
obsidian-livesync
├── docker-compose.yml
└── local.ini
$ docker run --name couchdb-for-ols -d --restart always -e COUCHDB_USER=${username} -e COUCHDB_PASSWORD=${password} -v ${PWD}/couchdb-data:/opt/couchdb/data -v ${PWD}/couchdb-etc:/opt/couchdb/etc/local.d -p 5984:5984 couchdb
```
A good place to start for `docker-compose.yml`:
```yaml
version: "2.1"
Congrats, move on to [step 2](#2-run-couchdb-initsh-for-initialise)
### B. Using Docker Compose
#### 1. Prepare
```
# Creating the save data & configuration directories.
mkdir couchdb-data
mkdir couchdb-etc
# Changing perms to user 5984.
chown -R 5984:5984 ./couchdb-data
chown -R 5984:5984 ./couchdb-etc
```
#### 2. Create a `docker-compose.yml` file with the following added to it
```
services:
couchdb:
image: couchdb
container_name: obsidian-livesync
user: 1000:1000
image: couchdb:latest
container_name: couchdb-for-ols
user: 5984:5984
environment:
- COUCHDB_USER=admin
- COUCHDB_PASSWORD=password
- COUCHDB_USER=<INSERT USERNAME HERE> #Please change as you like.
- COUCHDB_PASSWORD=<INSERT PASSWORD HERE> #Please change as you like.
volumes:
- ./data:/opt/couchdb/data
- ./local.ini:/opt/couchdb/etc/local.ini
- ./couchdb-data:/opt/couchdb/data
- ./couchdb-etc:/opt/couchdb/etc/local.d
ports:
- 5984:5984
restart: unless-stopped
```
And finally launch the container
#### 3. Run the Docker Compose file to boot check
```
docker compose up
# Or if using the old version
docker-compose up
```
> [!WARNING]
> If your container threw an error or exited unexpectedly, please check the permission of couchdb-data, and couchdb-etc.
> Once CouchDB starts, these directories will be owned by uid:`5984`. Please chown it for that uid again.
#### 4. Run the Docker Compose file in the background
If all went well and didn't throw any errors, `CTRL+C` out of it, and then run this command
```
# -d will launch detached so the container runs in background
docker compose up -d
# Or if using the old version
docker-compose up -d
```
## Access from a mobile device
If you want to access Self-hosted LiveSync from mobile devices, you need a valid SSL certificate.
Congrats, move on to [step 2](#2-run-couchdb-initsh-for-initialise)
### Testing from a mobile
In the testing phase, [localhost.run](https://localhost.run/) or something like services is very useful.
example using localhost.run:
### C. Install CouchDB directly
Please refer to the [official document](https://docs.couchdb.org/en/stable/install/index.html). However, we do not have to configure it fully. Just the administrator needs to be configured.
## 2. Run couchdb-init.sh for initialise
```
$ ssh -R 80:localhost:5984 nokey@localhost.run
Warning: Permanently added the RSA host key for IP address '35.171.254.69' to the list of known hosts.
===============================================================================
Welcome to localhost.run!
Follow your favourite reverse tunnel at [https://twitter.com/localhost_run].
**You need a SSH key to access this service.**
If you get a permission denied follow Gitlab's most excellent howto:
https://docs.gitlab.com/ee/ssh/
*Only rsa and ed25519 keys are supported*
To set up and manage custom domains go to https://admin.localhost.run/
More details on custom domains (and how to enable subdomains of your custom
domain) at https://localhost.run/docs/custom-domains
To explore using localhost.run visit the documentation site:
https://localhost.run/docs/
===============================================================================
** your connection id is xxxxxxxxxxxxxxxxxxxxxxxxxxxx, please mention it if you send me a message about an issue. **
xxxxxxxx.localhost.run tunneled with tls termination, https://xxxxxxxx.localhost.run
Connection to localhost.run closed by remote host.
Connection to localhost.run closed.
curl -s https://raw.githubusercontent.com/vrtmrz/obsidian-livesync/main/utils/couchdb/couchdb-init.sh | bash
```
https://xxxxxxxx.localhost.run is the temporary server address.
If it results like the following:
```
-- Configuring CouchDB by REST APIs... -->
{"ok":true}
""
""
""
""
""
""
""
""
""
<-- Configuring CouchDB by REST APIs Done!
```
Your CouchDB has been initialised successfully. If you want this manually, please read the script.
If you are using Docker Compose and the above command does not work or displays `ERROR: Hostname missing`, you can try running the following command, replacing the placeholders with your own values:
```
curl -s https://raw.githubusercontent.com/vrtmrz/obsidian-livesync/main/utils/couchdb/couchdb-init.sh | hostname=http://<YOUR SERVER IP>:5984 username=<INSERT USERNAME HERE> password=<INSERT PASSWORD HERE> bash
```
## 3. Expose CouchDB to the Internet
- You can skip this instruction if you using only in intranet and only with desktop devices.
- For mobile devices, Obsidian requires a valid SSL certificate. Usually, it needs exposing the internet.
Whatever solutions we can use. For simplicity, the following sample uses Cloudflare Zero Trust for testing.
```
cloudflared tunnel --url http://localhost:5984
```
You will then get the following output:
```
2024-02-14T10:35:25Z INF Thank you for trying Cloudflare Tunnel. Doing so, without a Cloudflare account, is a quick way to experiment and try it out. However, be aware that these account-less Tunnels have no uptime guarantee. If you intend to use Tunnels in production you should use a pre-created named tunnel by following: https://developers.cloudflare.com/cloudflare-one/connections/connect-apps
2024-02-14T10:35:25Z INF Requesting new quick Tunnel on trycloudflare.com...
2024-02-14T10:35:26Z INF +--------------------------------------------------------------------------------------------+
2024-02-14T10:35:26Z INF | Your quick Tunnel has been created! Visit it at (it may take some time to be reachable): |
2024-02-14T10:35:26Z INF | https://tiles-photograph-routine-groundwater.trycloudflare.com |
2024-02-14T10:35:26Z INF +--------------------------------------------------------------------------------------------+
:
:
:
```
Now `https://tiles-photograph-routine-groundwater.trycloudflare.com` is our server. Make it into the background once, please.
## 4. Client Setup
> [!TIP]
> Now manual configuration is not recommended for some reasons. However, if you want to do so, please use `Setup wizard`. The recommended extra configurations will be also set.
### 1. Generate the setup URI on a desktop device or server
```bash
export hostname=https://tiles-photograph-routine-groundwater.trycloudflare.com #Point to your vault
export database=obsidiannotes #Please change as you like
export passphrase=dfsapkdjaskdjasdas #Please change as you like
export username=johndoe
export password=abc123
deno run -A https://raw.githubusercontent.com/vrtmrz/obsidian-livesync/main/utils/flyio/generate_setupuri.ts
```
> [!TIP]
> What is the `passphrase`? Is it different from `uri_passphrase`?
> Yes, the `passphrase` we have exported now is for an End-to-End Encryption passphrase.
> And, `uri_passphrase` that used in the `generate_setupuri.ts` is a different one; for decrypting Set-up URI at using that.
> Why: I (vorotamoroz) think that the passphrase of the Setup-URI should be different from the E2EE passphrase to prevent exposure caused by operational errors or the possibility of evil in our environment. On top of that, I believe that it is desirable for the Setup-URI to be random. Setup-URI is inevitably long, so it goes through the clipboard. I think that its passphrase should not go through the same path, so it should essentially be typed manually.
> Hence, if we keep empty for uri_passphrase, generate_setupuri.ts generates an adjective-noun-randomnumber passphrase so that we can remember it without going through the clipboard.
You will then get the following output:
```bash
obsidian://setuplivesync?settings=%5B%22tm2DpsOE74nJAryprZO2M93wF%2Fvg.......4b26ed33230729%22%5D
Your passphrase of Setup-URI is: patient-haze
This passphrase is never shown again, so please note it in a safe place.
```
Please keep your passphrase of Setup-URI.
### 2. Setup Self-hosted LiveSync to Obsidian
[This video](https://youtu.be/7sa_I1832Xc?t=146) may help us.
1. Install Self-hosted LiveSync
2. Choose `Use the copied setup URI` from the command palette and paste the setup URI. (obsidian://setuplivesync?settings=.....).
3. Type the previously displayed passphrase (`patient-haze`) for setup-uri passphrase.
4. Answer `yes` and `Set it up...`, and finish the first dialogue with `Keep them disabled`.
5. `Reload app without save` once.
---
## Manual setup information
### Setting up your domain
@@ -127,6 +219,176 @@ Set the A record of your domain to point to your server, and host reverse proxy
Note: Mounting CouchDB on the top directory is not recommended.
Using Caddy is a handy way to serve the server with SSL automatically.
I have published [docker-compose.yml and ini files](https://github.com/vrtmrz/self-hosted-livesync-server) that launch Caddy and CouchDB at once. Please try it out.
I have published [docker-compose.yml and ini files](https://github.com/vrtmrz/self-hosted-livesync-server) that launch Caddy and CouchDB at once. If you are using Traefik you can check the [Reverse Proxies](#reverse-proxies) section below.
And, be sure to check the server log and be careful of malicious access.
## Reverse Proxies
### Traefik
If you are using Traefik, this [docker-compose.yml](https://github.com/vrtmrz/obsidian-livesync/blob/main/docker-compose.traefik.yml) file (also pasted below) has all the right CORS parameters set. It assumes you have an external network called `proxy`.
```yaml
services:
couchdb:
image: couchdb:latest
container_name: obsidian-livesync
user: 1000:1000
environment:
- COUCHDB_USER=username
- COUCHDB_PASSWORD=password
volumes:
- ./data:/opt/couchdb/data
- ./local.ini:/opt/couchdb/etc/local.ini
# Ports not needed when already passed to Traefik
#ports:
# - 5984:5984
restart: unless-stopped
networks:
- proxy
labels:
- "traefik.enable=true"
# The Traefik Network
- "traefik.docker.network=proxy"
# Don't forget to replace 'obsidian-livesync.example.org' with your own domain
- "traefik.http.routers.obsidian-livesync.rule=Host(`obsidian-livesync.example.org`)"
# The 'websecure' entryPoint is basically your HTTPS entrypoint. Check the next code snippet if you are encountering problems only; you probably have a working traefik configuration if this is not your first container you are reverse proxying.
- "traefik.http.routers.obsidian-livesync.entrypoints=websecure"
- "traefik.http.routers.obsidian-livesync.service=obsidian-livesync"
- "traefik.http.services.obsidian-livesync.loadbalancer.server.port=5984"
- "traefik.http.routers.obsidian-livesync.tls=true"
# Replace the string 'letsencrypt' with your own certificate resolver
- "traefik.http.routers.obsidian-livesync.tls.certresolver=letsencrypt"
- "traefik.http.routers.obsidian-livesync.middlewares=obsidiancors"
# The part needed for CORS to work on Traefik 2.x starts here
- "traefik.http.middlewares.obsidiancors.headers.accesscontrolallowmethods=GET,PUT,POST,HEAD,DELETE"
- "traefik.http.middlewares.obsidiancors.headers.accesscontrolallowheaders=accept,authorization,content-type,origin,referer"
- "traefik.http.middlewares.obsidiancors.headers.accesscontrolalloworiginlist=app://obsidian.md,capacitor://localhost,http://localhost"
- "traefik.http.middlewares.obsidiancors.headers.accesscontrolmaxage=3600"
- "traefik.http.middlewares.obsidiancors.headers.addvaryheader=true"
- "traefik.http.middlewares.obsidiancors.headers.accessControlAllowCredentials=true"
networks:
proxy:
external: true
```
Partial `traefik.yml` config file mentioned in above:
```yml
...
entryPoints:
web:
address: ":80"
http:
redirections:
entryPoint:
to: "websecure"
scheme: "https"
websecure:
address: ":443"
...
```
### Nginx
When configuring nginx as a reverse-proxy for CouchDB, note the common mistakes:
1. If fast fetch progress stalls and seems to freeze indefinitely, make sure you disabled `proxy_buffering`:
```nginx
location / {
proxy_pass http://127.0.0.1:5984;
proxy_buffering off;
...
}
```
2. If you get the "413 Entity too large" error, increase the `client_max_body_size`:
```nginx
location / {
proxy_pass http://127.0.0.1:5984;
client_max_body_size 50M; # Tweak the value to your needs
...
}
```
3. If you get the "404 Database not found" error, make sure you placed CouchDB at the root location (recommended):
```nginx
server {
server_name couchdb.domain.com
location / {
proxy_pass http://127.0.0.1:5984;
...
}
}
```
It is possible to place CouchDB into the subdirectory, however, the config should be modified respectively:
```nginx
server {
server_name domain.com
location /couchdb {
rewrite ^ $request_uri;
rewrite ^/couchdb/(.*) /$1 break;
proxy_pass http://127.0.0.1:5984$uri;
proxy_set_header Host $host;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
...
}
location /_session {
proxy_pass http://127.0.0.1:5984/_session;
proxy_set_header Host $host;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
...
}
}
```
4. If you added custom HTTP headers in database connection advanced settings, make sure to update both nginx and CouchDB configurations:
Nginx:
```nginx
location / {
set $pass 1;
# Example of handling custom HTTP header
if ($http_x_custom_header != 'foo'){
set $pass 0;
}
# Important: OPTIONS requests don't carry headers, so they should always be proxied to the CouchDB
if ($request_method = 'OPTIONS') {
set $pass 1;
}
if ($pass = 0) {
return 403;
}
proxy_pass http://127.0.0.1:5984;
...
}
```
couchdb-etc/docker.ini:
```ini
...
[cors]
credentials = true
origins = app://obsidian.md,capacitor://localhost,http://localhost
;Make sure to add your custom header to the list so CORS won't break
headers = accept, authorization, content-type, origin, referer, x-custom-header
```
+63 -8
View File
@@ -1,8 +1,19 @@
# 在你自己的服务器上设置 CouchDB
## 目录
- [配置 CouchDB](#配置-CouchDB)
- [运行 CouchDB](#运行-CouchDB)
- [Docker CLI](#docker-cli)
- [Docker Compose](#docker-compose)
- [创建数据库](#创建数据库)
- [从移动设备访问](#从移动设备访问)
- [移动设备测试](#移动设备测试)
- [设置你的域名](#设置你的域名)
---
> 注:提供了 [docker-compose.yml 和 ini 文件](https://github.com/vrtmrz/self-hosted-livesync-server) 可以同时启动 Caddy 和 CouchDB。推荐直接使用该 docker-compose 配置进行搭建。(若使用,请查阅链接中的文档,而不是这个文档)
## 安装 CouchDB 并从 PC 或 Mac 上访问
## 配置 CouchDB
设置 CouchDB 的最简单方法是使用 [CouchDB docker image]((https://hub.docker.com/_/couchdb)).
@@ -33,17 +44,61 @@ methods = GET, PUT, POST, HEAD, DELETE
max_age = 3600
```
创建 `local.ini` 并用如下指令启动 CouchDB
```
$ docker run --rm -it -e COUCHDB_USER=admin -e COUCHDB_PASSWORD=password -v .local.ini:/opt/couchdb/etc/local.ini -p 5984:5984 couchdb
```
Note: 此时 local.ini 的文件所有者会变成 5984:5984。这是 docker 镜像的限制,请修改文件所有者后再编辑 local.ini。
## 运行 CouchDB
在确定 Self-hosted LiveSync 可以和服务器同步后,可以后台启动 docker 镜像:
### Docker CLI
你可以通过指定 `local.ini` 配置运行 CouchDB:
```
$ docker run -d --restart always -e COUCHDB_USER=admin -e COUCHDB_PASSWORD=password -v .local.ini:/opt/couchdb/etc/local.ini -p 5984:5984 couchdb
$ docker run --rm -it -e COUCHDB_USER=admin -e COUCHDB_PASSWORD=password -v /path/to/local.ini:/opt/couchdb/etc/local.ini -p 5984:5984 couchdb
```
*记得将上述命令中的 local.ini 挂载路径替换成实际的存放路径*
后台运行:
```
$ docker run -d --restart always -e COUCHDB_USER=admin -e COUCHDB_PASSWORD=password -v /path/to/local.ini:/opt/couchdb/etc/local.ini -p 5984:5984 couchdb
```
*记得将上述命令中的 local.ini 挂载路径替换成实际的存放路径*
### Docker Compose
创建一个文件夹, 将你的 `local.ini` 放在文件夹内, 然后在文件夹内创建 `docker-compose.yml`. 请确保对 `local.ini` 有读写权限并且确保在容器运行后能创建 `data` 文件夹. 文件夹结构大概如下:
```
obsidian-livesync
├── docker-compose.yml
└── local.ini
```
可以参照以下内容编辑 `docker-compose.yml`:
```yaml
services:
couchdb:
image: couchdb
container_name: obsidian-livesync
user: 1000:1000
environment:
- COUCHDB_USER=admin
- COUCHDB_PASSWORD=password
volumes:
- ./data:/opt/couchdb/data
- ./local.ini:/opt/couchdb/etc/local.ini
ports:
- 5984:5984
restart: unless-stopped
```
最后, 创建并启动容器:
```
# -d will launch detached so the container runs in background
docker-compose up -d
```
## 创建数据库
CouchDB 部署成功后, 需要手动创建一个数据库, 方便插件连接并同步.
1. 访问 `http://localhost:5984/_utils`, 输入帐号密码后进入管理页面
2. 点击 Create Database, 然后根据个人喜好创建数据库
## 从移动设备访问
如果你想要从移动设备访问 Self-hosted LiveSync,你需要一个合法的 SSL 证书。
+1 -1
View File
@@ -12,10 +12,10 @@ max_document_size = 50000000
[chttpd]
require_valid_user = true
max_http_request_size = 4294967296
[chttpd_auth]
require_valid_user = true
max_http_request_size = 4294967296
authentication_redirect = /_utils/session.html
[httpd]
+2 -2
View File
@@ -1,8 +1,8 @@
# Designed architecture
## How does this plugin synchronize.
## How does this plug-in synchronise.
![Synchronization](../images/1.png)
![Synchronisation](../images/1.png)
1. When notes are created or modified, Obsidian raises some events. Self-hosted LiveSync catches these events and reflects changes into Local PouchDB.
2. PouchDB automatically or manually replicates changes to remote CouchDB.
+16
View File
@@ -0,0 +1,16 @@
# 架构设计
## 这个插件是怎么实现同步的.
![Synchronisation](../images/1.png)
1. 当笔记创建或修改时,Obsidian会触发事件。Self-hosted LiveSync捕获这些事件,并将变更同步至本地PouchDB
2. PouchDB通过自动或手动方式将变更同步至远程CouchDB
3. 其他设备监听远程CouchDB的变更,从而获取最新更新
4. Self-hosted LiveSync 将同步的变更集反映到Obsidian存储库中。
注:图示为简化演示,仅展示两个设备间的单向同步。实际为多设备间同时进行的双向同步。
## 降低带宽消耗的技术方案。
![dedupe](../images/2.png)
+1 -1
View File
@@ -2,7 +2,7 @@
## 同期
![Synchronization](../images/1.png)
![Synchronisation](../images/1.png)
1. ノートが更新された際、Obsidianがイベントを発報します。Obsidian-LiveSyncはそれをハンドリングして、ローカルのPouchDBに変更を反映します。
2. PouchDBは、リモートのCouchDBに差分をレプリケーションします。
+102
View File
@@ -0,0 +1,102 @@
# Notes on Terminology, Spelling, Vocabulary Conventions
## Spelling and Vocabulary conventions
All guidelines and conventions listed below are disclosed and maintained solely for the sake of documentation `consistency`.
1. Almost all of the English words are written in British English. This convention originated from the author's personal preference.
- **Traditional Spelling (Trad-spelling)**: We prefer traditional British English spellings. In particular, we use `-ise` and `-isation` suffixes rather than the Oxford spelling `-ize` and `-ization` (for example, 'initialisation', 'synchronisation', and 'organisation').
- **Oxford Comma**: We use the serial (Oxford) comma to separate items in lists of three or more (for example, 'settings, snippets, and themes' instead of 'settings, snippets and themes').
- **Logical Punctuation**: We place punctuation marks (such as commas and full stops) outside quotation marks, unless the punctuation mark is part of the quoted text itself. For example, we write 'dialogue', not 'dialogue,'.
- **BBC News Styleguide**: If in wonder, the BBC News Styleguide may be useful as a reference.
2. Idiomatic terms, such as those used in HTML, CSS, and JavaScript, are usually aligned with the language used in the technology. For example, "color" instead of "colour", "program" instead of "programme", etc. Especially, terms which are used for attributes, properties, and methods are notable.
3. We use `dialogue` in documentation for consistency. While `dialog` may appear in source code, particularly in class names, method names, and attributes (following technical conventions in No. 2), we consistently use `dialogue` for user-facing messages and general documentation text. This approach balances No. 1 with No. 2.
4. Contractions are not used. For example, "do not" instead of "don't", "cannot" instead of "can't", etc., especially `'d`.
- We may encounter difficulties with tenses.
5. However, try using affirmative forms, `Discard` instead of `Do not keep`, `Continue` instead of `Do not stop`, etc.
- Some languages, such as Japanese, have a different meaning for `yes` and `no` between affirmative and negative questions.
6. Single quotation marks (`'`) are preferred over double quotation marks (`"`) in general documentation text, unless the context requires double quotes (for example, inside JSON code blocks).
### Terminology
- Boot-up sequence (boot-sequence)
- The initialisation process of the plug-in when Obsidian starts. It starts with the loading of the plug-in, setting up core services, loading saved settings, and opening the local database. Once the layout is ready, the plug-in checks for the presence of flag files, runs configuration diagnostics, connects to the remote database, and begins file watching. The sequence finishes once the plug-in is fully ready and operational.
- Broken files (Size mismatch)
- A state where a file's metadata and the actual content stored in its chunks do not match, causing file retrieval or synchronisation failures. These mismatches can be detected and resolved by running validation tools such as `Verify and repair all files` on the Hatch pane.
- Chunk / Chunks
- Divided units of data stored in the database or object storage to facilitate efficient synchronisation.
- Compaction
- A database maintenance procedure that discards old historical document revisions to shrink the remote database size.
- Custom HTTP Handler / Use Internal API (CORS Bypass Settings)
- Settings used to bypass CORS restrictions by routing requests through Obsidian's native request APIs. There are two distinct settings under the hood depending on the remote server type:
- **For S3-compatible Object Storage (useCustomRequestHandler)**: Labeled as **"Use Custom HTTP Handler"** in the standard settings tab, **"Use internal API"** in the Svelte-based Setup Wizard dialogue, and represented as `useProxy` in the Setup URI's query parameters due to an unfortunate misunderstanding during development.
- **For CouchDB (useRequestAPI)**: Labeled as **"Use Request API to avoid `inevitable` CORS problem"** in the standard settings tab, **"Use Internal API"** in the Svelte-based Setup Wizard dialogue, and represented as `useRequestAPI` in the Setup URI's query parameters.
- Customisation Sync
- The feature that synchronises settings, snippets, themes, and plug-ins. Write with an "s" in documentation (`Customisation`), though technical configurations and links may use `customization`.
- Database Adapter (IDB vs. IndexedDB)
- The local database storage interface used by PouchDB. The `IDB` adapter is recommended since the older `IndexedDB` adapter is obsolete and known to cause memory leaks in `LiveSync` mode. Users can switch between these adapters without a full database rebuild, although a local data migration and an Obsidian restart are required.
- Database Suffix (additionalSuffixOfDatabaseName)
- A unique suffix appended to the database name to allow synchronising multiple vaults with the same name on the same remote server.
- E2EE Algorithm
- The cryptographic algorithm version used for end-to-end encryption. All devices in the synchronisation group must be configured with a compatible version (such as `V2` or `V1`).
- Eden (Eden Chunks)
- A performance optimisation where newly created chunks are held within the document until they stabilise, before graduating to independent chunks.
- Fast Setup (Simple Fetch)
- A simplified, automated initial synchronisation flow triggered when setting up subsequent devices or recovering a database. It bypasses the detailed step-by-step setup wizard dialogues, prompting the user with high-level data processing decisions and completing the initial download and local file scan in one continuous process.
- Flag files (redflag.md, redflag2.md, redflag3.md)
- Special Markdown files (or directories) placed at the root of the vault to stop the boot-up sequence or trigger recovery tasks. For instance, `redflag.md` suspends all processes, while `redflag2.md` (`flag_rebuild.md`) triggers a full database rebuild and `redflag3.md` (`flag_fetch.md`) discards the local database to fetch it again from the remote.
- Garbage Collection (GC)
- The process of identifying and purging unreferenced chunks (unused data) from local and remote databases to reclaim storage space.
- Hatch (Hatch pane)
- A dedicated troubleshooting and maintenance section in the plug-in settings, typically hidden behind a warning-labeled collapsible panel to prevent accidental misconfiguration. It contains diagnostic utilities, database reset controls, status reports, and advanced edge-case patches.
- Hidden File Sync
- The feature that synchronises files located in hidden directories (like `.obsidian`).
- JWT Authentication
- An experimental authentication option for CouchDB allowing secure token-based authentication instead of standard credentials. It requires a configured private key/secret, algorithm, expiration duration, subject, and key ID.
- LiveSync
- A very confusing term.
- As a shortened form of `Self-hosted LiveSync`.
- As the name of a synchronisation mode. This should be changed to `Continuous`, in contrast to `Periodic`.
- livesync-serverpeer / webpeer
- Pseudo-clients that assist in WebRTC peer-to-peer communication.
- Metadata (File metadata)
- A database document that stores properties of a file, including its filename, path, size, modification time, and references (hashes) of the chunks that comprise the file's content. Conflict state is carried by the surrounding PouchDB/CouchDB revision metadata rather than by a separate history field inside the file metadata document. In Self-hosted LiveSync, file metadata is stored separately from the actual file content to enable efficient synchronisation and versioning.
- OneShot Sync
- A single, immediate bidirectional synchronisation (pull then push) triggered on demand or on specific events, as opposed to continuous (live) replication.
- Overwrite Server Data with This Device's Files
- A maintenance operation (formerly known as `Rebuild everything`) that discards the remote database and reconstructs it by uploading all current local files as a fresh database, overwriting any remote changes.
- Path Obfuscation
- A privacy option that encrypts file paths and folder names on the remote server.
- plug-in
- We use the hyphenated form `plug-in` in user-facing messages and general documentation, while `plugin` may appear in codebase files, configuration settings, or technical contexts.
- Relay Server (P2P relays)
- A WebSocket-based coordination server used to establish direct WebRTC peer-to-peer connections. The default relay is provided by the plug-in author.
- Remediation (maxMTimeForReflectEvents)
- A recovery setting that restricts the propagation of changes from the database to local storage, ignoring any file events (such as accidental mass deletions) that occurred after a specified date and time.
- Reset Synchronisation on This Device
- A maintenance operation (formerly known as `Fetch everything`) that discards the local database and reconstructs it by downloading all data from the remote server.
- Scram (Scram Switches)
- Emergency controls in the settings that allow users to suspend file watching or database writes to prevent corruption.
- Segmenter (Segmented-splitter)
- A chunking method that divides files on semantic boundaries (such as paragraphs or sections) rather than arbitrary byte boundaries.
- Self-hosted LiveSync
- The name of this plug-in. `Self-hosted` is one word.
- Setting Doctor (Config Doctor)
- A diagnostic utility that checks for mismatches or suboptimal configurations, presenting users with ideal values and recommendation reasons to easily resolve issues during migration, configuration import, or general troubleshooting.
- Setup URI
- An encrypted representation of the plug-in's settings containing server configuration, which allows users to clone their configuration across devices securely using a passphrase.
- Streaming replication (Stream-based replication)
- A data transfer method that downloads database documents as a continuous stream of events. It is significantly faster than traditional chunk-by-chunk HTTP requests and is used during Fast Setup to retrieve remote metadata quickly.
- Sync Mode
- The replication trigger mechanism. Users can select from `On Events` (synchronising on local file changes), `Periodic and Events` (synchronising at fixed intervals as well as on events), or `LiveSync` (continuous, real-time synchronisation).
- TURN Server (WebRTC P2P)
- A server type (Traversal Using Relays around NAT) used as a fallback to relay traffic when direct WebRTC peer-to-peer connection is blocked by strict NAT or firewalls.
- Update Thinning (Batch database update)
- An optimisation that groups multiple local file edits together over a short delay before committing them to the local database, reducing the number of database write operations.
- WebRTC P2P (Peer-to-Peer)
- A synchronisation method enabling direct communication between devices without a central server database.
+65
View File
@@ -0,0 +1,65 @@
# Fast Setup (Simple Fetch)
Fast Setup is a streamlined, user-friendly data retrieval and initialisation flow designed to simplify setting up secondary devices or recovering databases.
Instead of guiding the user through the detailed multi-step setup wizard dialogues, Fast Setup prompts the user with high-level sync decisions and automates database download and local storage scanning in one continuous process.
---
## How It Works
When you import a **Setup URI** on a secondary device, or when a **Fetch All** operation is triggered (such as by placing a `redflag3.md` / `flag_fetch.md` flag file at the root of the vault), the plug-in schedules remote data retrieval.
On the next startup, the plug-in boots in scheduled fetch mode and opens a simplified dialogue: **"Data retrieval scheduled"**.
---
## Technical Characteristics
Fast Setup leverages several backend optimisations to make the retrieval fast, safe, and clean:
1. **Stream-based Replication for Speed**
- It fetches all remote metadata via stream reception, which is significantly faster than traditional chunk-by-chunk retrieval.
2. **Delayed File Reflection to Prevent Corrupted Warnings**
- By suspending file reflection during the download phase, it prevents the plug-in from raising temporary or false "corrupted data synchronisation" or "size mismatch" warnings that can occur during the chunk download process.
3. **Time-Based Comparison is Generally Sufficient**
- Since the vault is entering a fresh synchronisation or recovery state, comparing files based on their modification timestamps (newer-wins) is highly reliable and sufficient to reconcile files without needing complex manual conflict resolution.
---
## Step-by-Step Guide
### Step 1: Choose Data Processing Method
You will be prompted to choose how the retrieved remote data will interact with your existing local files:
1. **Compare time and take newer (newer-wins)**
- Compares the modified time of files and accepts the newer version.
- **Recommended if:** You have been using Self-hosted LiveSync and have made changes on multiple devices that you want to merge.
2. **Overwrite all with remote files (remote-wins)**
- Remote data is treated as the source of truth.
- **Recommended if:** You are setting up a brand new device with an empty or clean vault.
- *Warning: This will overwrite local files with remote files. Please ensure you have a backup of your local vault before proceeding.*
3. **Use the detailed flow (legacy)**
- Switches back to the detailed, traditional setup wizard dialogues.
- **Recommended if:** You want full control over the step-by-step database setup options.
### Step 2: Configure Conflict & Deletion Rules
Depending on your choice in Step 1, you will configure how to handle mismatches:
#### If you chose "Compare time and take newer":
- **Delete local files if they were deleted on remote**
- Keeps your local vault clean by removing files that have already been deleted on other devices.
- **Recreate remote files even if they were deleted on remote**
- Preserves local files and uploads them back to the remote database, even if they were deleted on other devices.
#### If you chose "Overwrite all with remote files":
- **Delete local files if not on remote**
- Removes local-only files so that your local vault matches the remote database exactly.
- **Keep local files even if not on remote**
- Retains all existing local-only files, although this may result in duplicates that you will need to clean up manually after synchronisation.
### Step 3: Automated Synchronisation
Once you confirm your choices:
1. The plug-in performs a fast download of the remote database (`fetchLocalDBFast`).
2. It automatically runs a full scan (`synchroniseAllFilesBetweenDBandStorage`) in the foreground to reflect database changes in your local vault files immediately.
3. The plug-in finalises the process and resumes normal operational status.
+66
View File
@@ -0,0 +1,66 @@
# ファストセットアップ (Fast Setup / Simple Fetch)
ファストセットアップは、2台目以降のデバイスのセットアップやデータベース再構築時のデータ取得・初期化処理を、直感的かつ迅速に行うための簡略化された同期フローです。
従来のセットアップウィザードにおける複数の詳細なステップを踏むことなく、同期の基本方針を選択するだけで、データベースのダウンロードとローカルファイルのスキャン・反映を一連のプロセスとして自動的に実行します。
---
## 仕組み
2台目以降のデバイスで **セットアップURI** をインポートした場合や、手動でデータの再フェッチ(Vaultルートに `redflag3.md` / `flag_fetch.md` を配置する等)が予約された場合、プラグインはリモートデータベースからのデータ取得スケジュールを設定します。
その後の起動時、プラグインはデータフェッチ予約モードで立ち上がり、**「Data retrieval scheduled(データ取得のスケジュール)」** という簡略化されたダイアログを表示します。
---
## 技術的な特徴
ファストセットアップは、高速かつ安全でクリーンな処理を実現するために、以下の最適化を行っています。
1. **高速なストリーム受信**
- 全データを取得しますが、ストリーム受信によるレプリケーションを使用するため、処理が非常に高速です。
2. **ストレージ反映の遅延による不要な警告の抑制**
- データのダウンロード中にローカルファイル(ストレージ)への書き出し(反映)処理をあえて遅延させることで、同期途中で発生しがちな「破損データ同期」や「サイズ不一致」などの一時的なエラー警告を抑え込みます。
- すべてのデータダウンロードが完了した後に一括してストレージへ書き出すため、不必要な警告画面でユーザーを混乱させません。
3. **時刻ベース比較の妥当性**
- 初期セットアップやリカバリーの段階(この状態に移行した直後)においては、概ねファイル更新時刻(タイムスタンプ)ベースでの単純比較を行うことで、十分かつ妥当な同期結果を得ることができます。これにより複雑な競合解決の手間を省きます。
---
## 設定手順
### ステップ 1: データの反映方法の選択
取得したリモートデータを、既存のローカルファイルとどのように統合するかを選択します。
1. **Compare time and take newer (newer-wins)**
- ファイルの更新日時を比較し、より新しい方を採用します。
- **推奨されるケース:** すでに Self-hosted LiveSync を使用しており、複数のデバイスで編集した変更内容をタイムスタンプに基づいて統合したい場合。
2. **Overwrite all with remote files (remote-wins)**
- リモートデータベースの内容を正(Source of Truth)として扱います。
- **推奨されるケース:** まったく新しいデバイスをセットアップする場合(空のVaultなど)。
- *警告: ローカルにあるすべてのファイルがリモートの内容で上書きされます。重要なデータがある場合は、事前にバックアップを取得してください。*
3. **Use the detailed flow (legacy)**
- 従来の詳細なセットアップウィザードダイアログに戻ります。
- **推奨されるケース:** データベースの構成オプションをステップバイステップで細かく制御・確認したい場合。
### ステップ 2: 競合および削除ルールの構成
ステップ 1 での選択内容に応じて、ローカルとリモートの不一致をどう処理するかを設定します。
#### 「Compare time and take newer」を選択した場合:
- **Delete local files if they were deleted on remote**
- 他のデバイスで削除済みのファイルをローカルからも削除し、Vaultを同期・クリーンな状態に保ちます。
- **Recreate remote files even if they were deleted on remote**
- 他のデバイスで削除されたファイルであっても、ローカルファイルを維持し、リモートデータベースに再度アップロードします。
#### 「Overwrite all with remote files」を選択した場合:
- **Delete local files if not on remote**
- リモートに存在しないローカル専用ファイルを削除し、ローカルのVaultをリモートデータベースと完全に一致させます。
- **Keep local files even if not on remote**
- リモートに存在しないローカルファイルをそのまま残します。ただし、同期後に重複ファイルが発生する可能性があるため、その場合は手動でクリーンアップを行ってください。
### ステップ 3: 自動同期の実行
選択を確定すると、以下の処理が順次実行されます。
1. リモートデータベースの高速ダウンロードを実行します (`fetchLocalDBFast`)。
2. ローカルファイルへの変更反映のため、フォアグラウンドでフルスキャン (`synchroniseAllFilesBetweenDBandStorage`) を自動的に実行します。
3. 処理が完了すると、プラグインは通常の動作状態へ復帰します。
+81
View File
@@ -0,0 +1,81 @@
---
title: "JWT Authentication on CouchDB"
livesync-version: 0.25.24
tags:
- tips
- CouchDB
- JWT
authors:
- vorotamoroz
---
# JWT Authentication on CouchDB
When using CouchDB as a backend for Self-hosted LiveSync, it is possible to enhance security by employing JWT (JSON Web Token) Authentication. In particular, using asymmetric keys (ES256 and ES512) provides greater security against token interception.
## Setting up JWT Authentication (Asymmetrical Key Example)
### 1. Generate a key pair
We can use `openssl` to generate an EC key pair as follows:
```bash
# Generate private key
# ES512 for secp521r1 curve, we can also use ES256 for prime256v1 curve
openssl ecparam -name secp521r1 -genkey -noout | openssl pkcs8 -topk8 -inform PEM -nocrypt -out private_key.pem
# openssl ecparam -name prime256v1 -genkey -noout | openssl pkcs8 -topk8 -inform PEM -nocrypt -out private_key.pem
# Generate public key in SPKI format
openssl ec -in private_key.pem -pubout -outform PEM -out public_key.pem
```
> [!TIP]
> A key generator will be provided again in a future version of the user interface.
### 2. Configure CouchDB to accept JWT tokens
The following configuration is required:
| Key | Value | Note |
| ------------------------------ | ----------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| chttpd/authentication_handlers | {chttpd_auth, jwt_authentication_handler} | In total, it may be `{chttpd_auth, jwt_authentication_handler}, {chttpd_auth, cookie_authentication_handler}, {chttpd_auth, default_authentication_handler}`, or something similar. |
| jwt_auth/required_claims | "exp" | |
| jwt_keys/ec:your_key_id | Your public key in PEM (SPKI) format | Replace `your_key_id` with your actual key ID. You can decide as you like. Note that you can add multiple keys if needed. If you want to use HSxxx, you should set `jwt_keys/hmac:your_key_id` with your HMAC secret. |
Note: When configuring CouchDB via web interface (Fauxton), new-lines on the public key should be replaced with `\n` for header and footer lines (So wired, but true I have tested). as follows:
```
-----BEGIN PUBLIC KEY-----
\nMIGbMBAGByqGSM49AgEGBSuBBAAjA4GGAAQBq0irb/+K0Qzo7ayIHj0Xtthcntjz
r665J5UYdEQMiTtku5rnp95RuN97uA2pPOJOacMBAoiVUnZ1pqEBz9xH9yoAixji
Ju...........................................................gTt
/xtqrJRwrEy986oRZRQ=
\n-----END PUBLIC KEY-----
```
For detailed information, please refer to the [CouchDB JWT Authentication Documentation](https://docs.couchdb.org/en/stable/api/server/authn.html#jwt-authentication).
### 3. Configure Self-hosted LiveSync to use JWT Authentication
| Setting | Description |
| ----------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------- |
| Use JWT Authentication | Enable this option to use JWT Authentication. |
| JWT Algorithm | Select the JWT signing algorithm (e.g., ES256, ES512) that matches your key pair. |
| JWT Key | Paste your private key in PEM (pkcs8) format. |
| JWT Expiration Duration | Set the token expiration time in minutes. Locally cached tokens are also invalidated after this duration. |
| JWT Key ID (kid) | Enter the key ID that you used when configuring CouchDB, i.e., the one that replaced `your_key_id`. |
| JWT Subject (sub) | Set your user ID; this overrides the original `Username` setting. If you have detected access with `Username`, you have failed to authorise with JWT. |
> [!IMPORTANT]
> Self-hosted LiveSync requests to CouchDB treat the user as `_admin`. If you want to restrict access, configure `jwt_auth/roles_claim_name` to a custom claim name. (Self-hosted LiveSync always sets `_couchdb.roles` with the value `["_admin"]`).
### 4. Test the configuration
Just try to `Test Settings and Continue` in the remote setup dialogue. If you have successfully authenticated, you are all set.
## Additional Notes
This feature is still experimental. Please ensure to test thoroughly in your environment before deploying to production.
However, we think that this is a great step towards enhancing security when using CouchDB with Self-hosted LiveSync. We shall enable this setting by default in future releases.
We would love to hear your feedback and any issues you encounter.
+29
View File
@@ -0,0 +1,29 @@
---
title: "Peer-to-Peer Synchronisation Tips"
livesync-version: 0.25.24
tags:
- tips
- p2p
authors:
- vorotamoroz
---
# Peer-to-Peer Synchronisation Tips
> [!IMPORTANT]
> Peer-to-peer synchronisation is still an experimental feature. Although we have made every effort to ensure its reliability, it may not function correctly in all environments.
## Difficulties with Peer-to-Peer Synchronisation
It is often the case that peer-to-peer connections do not function correctly, for instance, when using mobile data services.
In such circumstances, we recommend connecting all devices to a single Virtual Private Network (VPN). It is advisable to select a service, such as Tailscale, which facilitates direct communication between peers wherever possible.
Should one be in an environment where even Tailscale is unable to connect, or where it cannot be lawfully installed, please continue reading.
## A More Detailed Explanation
The failure of a Peer-to-Peer connection via WebRTC can be attributed to several factors. These may include an unsuccessful UDP hole-punching attempt, or an intermediary gateway intentionally terminating the connection. Troubleshooting this matter is not a simple undertaking. Furthermore, and rather unfortunately, gateway administrators are typically aware of this type of network behaviour. Whilst a legitimate purpose for such traffic can be cited, such as for web conferencing, this is often insufficient to prevent it from being blocked.
This situation, however, is the primary reason that our project does not provide a TURN server. Although it is said that a TURN server within WebRTC does not decrypt communications, the project holds the view that the risk of a malicious party impersonating a TURN server must be avoided. Consequently, configuring a TURN server for relay communication is not currently possible through the user interface. Furthermore, there is no official project TURN server, which is to say, one that could be monitored by a third party.
We request that you provide your own server, using your own Fully Qualified Domain Name (FQDN), and subsequently enter its details into the advanced settings.
For testing purposes, Cloudflare's Real-Time TURN Service is exceedingly convenient and offers a generous amount of free data. However, it must be noted that because it is a well-known destination, such traffic is highly conspicuous. There is also a significant possibility that it may be blocked by default. We advise proceeding with caution.
+145
View File
@@ -0,0 +1,145 @@
# How to report an issue
Thank you for helping improve Self-hosted LiveSync!
This document explains how to collect the information needed for an issue report. Issues with sufficient information will be prioritised.
---
## Filled example
Here is an example of a well-filled report for reference.
### Abstract
The synchronisation hung up immediately after connecting.
### Expected behaviour
- Synchronisation ends with the message `Replication completed`
- Everything synchronised
### Actually happened
- Synchronisation was cancelled with the message `TypeError: Failed to fetch` (visible in the plug-in log around lines 1012)
- No files synchronised
### Reproducing procedure
1. Configure LiveSync with the settings shown in the attached report.
2. Click the sync button on the ribbon.
3. Synchronisation begins.
4. About two or three seconds later, the error `TypeError: Failed to fetch` appears.
5. Replication stops. No files synchronised.
### Obsidian debug info (Device 1 — Windows desktop)
```
SYSTEM INFO:
Obsidian version: v1.2.8
Installer version: v1.1.15
Operating system: Windows 10 Pro 10.0.19044
Login status: logged in
Catalyst license: supporter
Insider build toggle: off
Community theme: Minimal v6.1.11
Snippets enabled: 3
Restricted mode: off
Plugins installed: 35
Plugins enabled: 11
1: Self-hosted LiveSync v0.19.4
...
```
### Report from LiveSync
```
----remote config----
cors:
credentials: "true"
...
---- Plug-in config ---
couchDB_URI: self-hosted
couchDB_USER: 𝑅𝐸𝐷𝐴𝐶𝑇𝐸𝐷
...
```
### Plug-in log
```
2023/5/24 10:50:33->HTTP:GET to:/ -> failed
2023/5/24 10:50:33->TypeError:Failed to fetch
2023/5/24 10:50:33->could not connect to https://example.com/ : your vault
(TypeError:Failed to fetch)
```
---
## How to collect each piece of information
### Obsidian debug info
Open the command palette (`Ctrl/Cmd + P`) and run **"Show debug info"**. Copy the output and paste it into the issue.
If multiple devices are involved in the problem (e.g., sync between a phone and a desktop), please provide the debug info for each device. The device where the issue occurred is required; information from other devices is strongly recommended.
### Report from LiveSync (hatch report)
1. Open LiveSync settings.
2. Go to the **Hatch** pane.
3. Press the **Make report** button.
The report will be copied to your clipboard. It contains your LiveSync configuration and the remote server configuration, with credentials automatically redacted.
**Tip:** For large reports, consider uploading to [GitHub Gist](https://gist.github.com/) and sharing the link instead of pasting directly into the issue. This makes it easier to manage, and if you accidentally leave sensitive data in, a Gist can be deleted.
If you paste directly, wrap it in a `<details>` tag to keep the issue readable:
```
<details>
<summary>Report from hatch</summary>
```
----remote config----
:
```
</details>
```
### Plug-in log
The plug-in log is volatile by default (not saved to disk) and shown only in the log dialogue, which can be opened by tapping the **document box icon** in the ribbon.
#### Enable verbose log
Before reproducing the issue, enable **Verbose Log** in LiveSync's **General Settings** pane. Without this, many diagnostic messages will be suppressed.
#### Persist the log to a file (optional)
If you need to capture a log across a restart, enable **"Write logs into the file"** in General Settings. Note that log files may contain sensitive information — use this option only for troubleshooting, and disable it afterwards.
As with the hatch report, consider uploading large logs to [GitHub Gist](https://gist.github.com/).
### Network log (for connection-related issues only)
If the issue is related to network connectivity (e.g., cannot connect to the server, authentication errors), a network log captured from browser DevTools can be very helpful. You do not need to include this for non-connection issues.
#### Opening DevTools
| Platform | Shortcut |
|----------|----------|
| Windows / Linux | `Ctrl + Shift + I` |
| macOS | `Cmd + Shift + I` |
| Android | Use [Chrome remote debugging](https://developer.chrome.com/docs/devtools/remote-debugging/) |
| iOS | Use [Safari Web Inspector](https://developer.apple.com/documentation/safari-developer-tools/inspecting-ios) on a Mac |
#### What to capture
1. Open the **Network** pane in DevTools.
2. Reproduce the issue.
3. Look for requests marked in red.
4. Capture screenshots of the **Headers**, **Payload**, and **Response** tabs for those requests.
**Important — redact before sharing:**
- Headers: conceal the request URL path, Remote Address, `authority`, and `authorisation` values.
- Payload / Response: the `_id` field contains your file paths — redact if needed.
+452
View File
@@ -0,0 +1,452 @@
# Tips and Troubleshooting
- [Tips and Troubleshooting](#tips-and-troubleshooting)
- [Tips](#tips)
- [CORS avoidance](#cors-avoidance)
- [CORS configuration with reverse proxy](#cors-configuration-with-reverse-proxy)
- [Nginx](#nginx)
- [Nginx and subdirectory](#nginx-and-subdirectory)
- [Caddy](#caddy)
- [Caddy and subdirectory](#caddy-and-subdirectory)
- [Apache](#apache)
- [Show all setting panes](#show-all-setting-panes)
- [How to resolve `Tweaks Mismatched of Changed`](#how-to-resolve-tweaks-mismatched-of-changed)
- [Notable bugs and fixes](#notable-bugs-and-fixes)
- [Binary files get bigger on iOS](#binary-files-get-bigger-on-ios)
- [Some setting name has been changed](#some-setting-name-has-been-changed)
- [Questions and Answers](#questions-and-answers)
- [How should I share the settings between multiple devices?](#how-should-i-share-the-settings-between-multiple-devices)
- [What should I enter for the passphrase of Setup-URI?](#what-should-i-enter-for-the-passphrase-of-setup-uri)
- [Why the settings of Self-hosted LiveSync itself is disabled in default?](#why-the-settings-of-self-hosted-livesync-itself-is-disabled-in-default)
- [The plug-in says `something went wrong`.](#the-plug-in-says-something-went-wrong)
- [A large number of files were deleted, and were synchronised!](#a-large-number-of-files-were-deleted-and-were-synchronised)
- [Why `Use an old adapter for compatibility` is somehow enabled in my vault?](#why-use-an-old-adapter-for-compatibility-is-somehow-enabled-in-my-vault)
- [ZIP (or any extensions) files were not synchronised. Why?](#zip-or-any-extensions-files-were-not-synchronised-why)
- [I hope to report the issue, but you said you needs `Report`. How to make it?](#i-hope-to-report-the-issue-but-you-said-you-needs-report-how-to-make-it)
- [Where can I check the log?](#where-can-i-check-the-log)
- [Why are the logs volatile and ephemeral?](#why-are-the-logs-volatile-and-ephemeral)
- [Some network logs are not written into the file.](#some-network-logs-are-not-written-into-the-file)
- [If a file were deleted or trimmed, the capacity of the database should be reduced, right?](#if-a-file-were-deleted-or-trimmed-the-capacity-of-the-database-should-be-reduced-right)
- [How to launch the DevTools](#how-to-launch-the-devtools)
- [On Desktop Devices](#on-desktop-devices)
- [On Android](#on-android)
- [On iOS, iPadOS devices](#on-ios-ipados-devices)
- [How can I use the DevTools?](#how-can-i-use-the-devtools)
- [Checking the network log](#checking-the-network-log)
- [Troubleshooting](#troubleshooting)
- [While using Cloudflare Tunnels, often Obsidian API fallback and `524` error occurs.](#while-using-cloudflare-tunnels-often-obsidian-api-fallback-and-524-error-occurs)
- [On the mobile device, cannot synchronise on the local network!](#on-the-mobile-device-cannot-synchronise-on-the-local-network)
- [I think that something bad happening on the vault...](#i-think-that-something-bad-happening-on-the-vault)
- [Flag Files](#flag-files)
- [Old tips](#old-tips)
<!-- - -->
## Tips
### CORS avoidance
If we are unable to configure CORS properly for any reason (for example, if we cannot configure non-administered network devices), we may choose to ignore CORS.
To use the Obsidian API (also known as the Non-Native API) to bypass CORS, we can enable the toggle ``Use Request API to avoid `inevitable` CORS problem``.
<!-- Add **Long explanation of CORS** here for integrity -->
### CORS configuration with reverse proxy
- IMPORTANT: CouchDB handles CORS by itself. Do not process CORS on the reverse
proxy.
- Do not process `Option` requests on the reverse proxy!
- Make sure `host` and `X-Forwarded-For` headers are forwarded to the CouchDB.
- If you are using a subdirectory, make sure to handle it properly. More
detailed information is in the
[CouchDB documentation](https://docs.couchdb.org/en/stable/best-practices/reverse-proxies.html).
Minimal configurations are as follows:
#### Nginx
```nginx
location / {
proxy_pass http://localhost:5984;
proxy_redirect off;
proxy_buffering off;
proxy_set_header Host $host;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
}
```
#### Nginx and subdirectory
```nginx
location /couchdb {
rewrite ^ $request_uri;
rewrite ^/couchdb/(.*) /$1 break;
proxy_pass http://localhost:5984$uri;
proxy_redirect off;
proxy_buffering off;
proxy_set_header Host $host;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
}
location /_session {
proxy_pass http://localhost:5984/_session;
proxy_redirect off;
proxy_buffering off;
proxy_set_header Host $host;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
}
```
#### Caddy
```caddyfile
domain.com {
reverse_proxy localhost:5984
}
```
#### Caddy and subdirectory
```caddyfile
domain.com {
reverse_proxy /couchdb/* localhost:5984
reverse_proxy /_session/* localhost:5984/_session
}
```
#### Apache
Sorry, Apache is not recommended for CouchDB. Omit the configuration from here.
Please refer to the
[Official documentation](https://docs.couchdb.org/en/stable/best-practices/reverse-proxies.html#reverse-proxying-with-apache-http-server).
### Show all setting panes
Full pane is not shown by default. To show all panes, please toggle all in
`🧙‍♂️ Wizard` -> `Enable extra and advanced features`.
For your information, the all panes are as follows:
![All Panes](all_toggles.png)
### How to resolve `Tweaks Mismatched of Changed`
(Since v0.23.17)
If you have changed some configurations or tweaks which should be unified
between the devices, you will be asked how to reflect (or not) other devices at
the next synchronisation. It also occurs on the device itself, where changes are
made, to prevent unexpected configuration changes from unwanted propagation.\
(We may thank this behaviour if we have synchronised or backed up and restored
Self-hosted LiveSync. At least, for me so).
Following dialogue will be shown: ![Dialogue](tweak_mismatch_dialogue.png)
- If we want to propagate the setting of the device, we should choose
`Update with mine`.
- On other devices, we should choose `Use configured` to accept and use the
configured configuration.
- `Dismiss` can postpone a decision. However, we cannot synchronise until we
have decided.
Rest assured that in most cases we can choose `Use configured`. (Unless you are
certain that you have not changed the configuration).
If we see it for the first time, it reflects the settings of the device that has
been synchronised with the remote for the first time since the upgrade.
Probably, we can accept that.
<!-- Add here -->
## Notable bugs and fixes
### Binary files get bigger on iOS
- Reported at: v0.20.x
- Fixed at: v0.21.2 (Fixed but not reviewed)
- Required action: larger files will not be fixed automatically, please perform
`Verify and repair all files`. If our local database and storage are not
matched, we will be asked to apply which one.
### Some setting name has been changed
- Fixed at: v0.22.6
| Previous name | New name |
| ---------------------------- | ---------------------------------------- |
| Open setup URI | Use the copied setup URI |
| Copy setup URI | Copy current settings as a new setup URI |
| Setup Wizard | Minimal Setup |
| Check database configuration | Check and Fix database configuration |
## Questions and Answers
### How should I share the settings between multiple devices?
- Device setup:
- Using `Setup URI` is the most straightforward way.
- Setting changes during use:
- Use `Sync settings via Markdown files` on the `🔄️ Sync settings` pane.
### What should I enter for the passphrase of Setup-URI?
- Anything you like is OK. However, the recommendation is as follows:
- Include the vault (group) information.
- Include the date of operation.
- Anything random for your security.
- For example, `MyVault-20240901-r4nd0mStr1ng`.
- Why?
- The Setup-URI is encoded; that means it cannot indicate the actual settings. Hence, if you use the same passphrase for multiple vaults, you may accidentally mix up vaults.
### Why the settings of Self-hosted LiveSync itself is disabled in default?
Basically, if we configure all `additionalSuffixOfDatabaseName` the same, we can synchronise this file between multiple devices.
(`additionalSuffixOfDatabaseName` should be unique in each device, not in the synchronised vaults).
However, if we synchronise the settings of Self-hosted LiveSync itself, we may encounter some unexpected behaviours.
For example, if a setting that 'let Self-hosted LiveSync setting be excluded' is synced, it is very unlikely that things will recover automatically after this, and there is little chance we will even notice this. Even if we change our minds and change the settings back on other devices. It could get even worse if incompatible changes are automatically reflected; everything will break.
### The plug-in says `something went wrong`.
There are many cases where this is really unclear. One possibility is that the chunk fetch did not go well.
1. Restarting Obsidian sometimes helps (fetch-order problem).
2. If actually there are no chunks, please perform `Recreate missing chunks for all files` on the `🧰 Hatch` pane at the other devices. And synchronise again. (also restart Obsidian may effect).
3. If the problem persists, please perform `Verify and repair all files` on the `🧰 Hatch` pane. If our local database and storage are not matched, we will be asked to apply which one.
### A large number of files were deleted, and were synchronised!
1. Backup everything important.
- Your local vault.
- Your CouchDB database (this can be done by replicating to another database).
2. Prepare the empty vault
3. Place `redflag.md` at the top of the vault.
4. Apply the settings **BUT DO NOT PROCEED TO RESTORE YET**.
- You can use `Setup URI`, QR Code, or manually apply the settings.
5. Set `Maximum file modification time for reflected file events` in `Remediation` on the `🩹 Patches` pane.
- If you know when the files were deleted, set the time a bit before that.
- If not, bisecting may help us.
6. Delete `redflag.md`.
7. Perform `Reset Synchronisation on This Device` on the `🎛️ Maintenance` pane.
This mode is very fragile. Please be careful.
### Why `Use an old adapter for compatibility` is somehow enabled in my vault?
Because you are a compassionate and experienced user. Before v0.17.16, we used
an old adapter for the local database. At that time, current default adapter has
not been stable. The new adapter has better performance and has a new feature
like purging. Therefore, we should use new adapters and current default is so.
However, when switching from an old adapter to a new adapter, some converting or
local database rebuilding is required, and it takes some time. It was a long
time ago now, but we once inconvenienced everyone in a hurry when we changed the
format of our database. For these reasons, this toggle is automatically on if we
have upgraded from vault which using an old adapter.
When you overwrite server data with this device's files or reset synchronisation on this device again, you will be asked to
switch this.
Therefore, experienced users (especially those stable enough not to have to
overwrite server data) may have this toggle enabled in their Vault. Please
disable it when you have enough time.
### ZIP (or any extensions) files were not synchronised. Why?
It depends on Obsidian detects. May toggling `Detect all extensions` of
`File and links` (setting of Obsidian) will help us.
### I hope to report the issue, but you said you needs `Report`. How to make it?
We can copy the report to the clipboard, by performing
`Generate full report for opening the issue with debug info` command!
### Where can I check the log?
We can launch the log pane by `Show log` on the command palette. And if you have
troubled something, please enable the `Verbose Log` on the `General Setting`
pane.
`Generate full report for opening the issue with debug info` command also contains
the recent 1000 log lines, which is very helpful for debugging. Full-report is
already set to the verbose level, so it contains all the logs without enabling the
`Verbose Log` toggle.
Let me note that please be sure to remove any sensitive information before sharing the report.
However, the logs would not be kept so long and cleared when restarted. If you
want to check the logs, please enable `Write logs into the file` temporarily.
![ScreenShot](../images/write_logs_into_the_file.png)
> [!IMPORTANT]
>
> - Writing logs into the file will impact the performance.
> - Please make sure that you have erased all your confidential information
> before reporting issue.
### Why are the logs volatile and ephemeral?
To avoid unexpected exposure to our confidential things.
### Some network logs are not written into the file.
Especially the CORS error will be reported as a general error to the plug-in for
security reasons. So we cannot detect and log it. We are only able to
investigate them by [Checking the network log](#checking-the-network-log).
### If a file were deleted or trimmed, the capacity of the database should be reduced, right?
No, even though if files were deleted, chunks were not deleted. Self-hosted
LiveSync splits the files into multiple chunks and transfers only newly created.
This behaviour enables us to less traffic. And, the chunks will be shared
between the files to reduce the total usage of the database.
And one more thing, we can handle the conflicts on any device even though it has
happened on other devices. This means that conflicts will happen in the past,
after the time we have synchronised. Hence we cannot collect and delete the
unused chunks even though if we are not currently referenced.
To shrink the database size, `Overwrite Server Data with This Device's Files` is the only reliable and effective way.
But do not worry, if we have synchronised well. We have the actual and real
files. Only it takes a bit of time and traffic.
### How to launch the DevTools
#### On Desktop Devices
We can launch the DevTools by pressing `ctrl`+`shift`+`i` (`Command`+`shift`+`i` on Mac).
#### On Android
Please refer to [Remote debug Android devices](https://developer.chrome.com/docs/devtools/remote-debugging/).
Once the DevTools have been launched, everything operates the same as on a PC.
#### On iOS, iPadOS devices
If we have a Mac, we can inspect from Safari on the Mac. Please refer to [Inspecting iOS and iPadOS](https://developer.apple.com/documentation/safari-developer-tools/inspecting-ios).
### How can I use the DevTools?
#### Checking the network log
1. Open the network pane.
2. Find the requests marked in red.\
![Errored](../images/devtools1.png)
3. Capture the `Headers`, `Payload`, and, `Response`. **Please be sure to keep
important information confidential**. If the `Response` contains secrets, you
can omitted that. Note: Headers contains a some credentials. **The path of
the request URL, Remote Address, authority, and authorization must be
concealed.**\
![Concealed sample](../images/devtools2.png)
## Troubleshooting
<!-- Add here -->
### While using Cloudflare Tunnels, often Obsidian API fallback and `524` error occurs.
A `524` error occurs when the request to the server is not completed within a
`specified time`. This is a timeout error from Cloudflare. From the reported
issue, it seems to be 100 seconds. (#627).
Therefore, this error returns from Cloudflare, not from the server. Hence, the
result contains no CORS field. It means that this response makes the Obsidian
API fallback.
However, even if the Obsidian API fallback occurs, the request is still not
completed within the `specified time`, 100 seconds.
To solve this issue, we need to configure the timeout settings.
Please enable the toggle in `💪 Power users` -> `CouchDB Connection Tweak` ->
`Use timeouts instead of heartbeats`.
### On the mobile device, cannot synchronise on the local network!
Obsidian mobile is not able to connect to the non-secure end-point, such as
starting with `http://`. Make sure your URI of CouchDB. Also not able to use a
self-signed certificate.
### I think that something bad happening on the vault...
Place the [flag file](#flag-files) on top of the vault, and restart Obsidian. The most simple
way is to create a new note and rename it to `redflag`. Of course, we can put it
without Obsidian.
For example, if there is `redflag.md`, Self-hosted LiveSync suspends all database and storage
processes.
### Scram State and Flag Files (SCRAM Warning Loop)
The plug-in uses a **Scram state** (emergency suspension of all synchronisation processes) to prevent database corruption when severe errors or conflicts are detected. This state is often triggered or persisted by **flag files** placed at the root of the vault.
If you encounter a warning saying **"Scram detected, all sync operations are suspended per SCRAM"** or get caught in an infinite loop where the warning persists even after clicking "Resume", it is likely due to a flag file in your vault.
#### Flag Files
A flag file is a simple Markdown file located at the root of your vault. Its very existence is significant; it may be left blank or contain any text. These files are used so they can be easily placed or deleted from outside Obsidian (e.g., when Obsidian fails to launch).
| Filename | Human-Friendly Name | Description |
| ------------- | ------------------- | --------------------------------------------------------------------------------------- |
| `redflag.md` | - | Suspends all processes (activates Scram). |
| `redflag2.md` | `flag_rebuild.md` | Suspends all processes, and overwrites server data with this device's files. |
| `redflag3.md` | `flag_fetch.md` | Suspends all processes, discards the local database, and resets synchronisation on this device. |
When resetting synchronisation on this device or overwriting server data, restarting Obsidian is performed once for safety reasons. At that time, Self-hosted LiveSync uses these files to determine whether the process should be carried out. (This mechanism is especially useful on mobile devices to force cancellation if the database rebuilding fails). These files are not subject to synchronisation.
#### How to Resolve the Scram Loop
If you cannot disable Scram, please follow these steps:
1. Close Obsidian completely.
2. Open your system's file manager and check the root directory of your vault.
3. Locate and delete any flag files (such as `redflag.md`, `redflag2.md`, or `redflag3.md`).
4. Launch Obsidian.
5. Go to the settings dialogue -> **Hatch** -> **Scram Switches**, and manually toggle **Suspend file watching** and **Suspend database reflecting** to `false` (disabled) if they have not been reset automatically.
> [!TIP]
> This is the reason why flag files are standard `.md` files: it allows you to manage them externally. On mobile devices, you can use system file manager applications (such as the native **Files** app on iOS/iPadOS or **Files by Google** on Android) to find and delete these files to resolve a lock, or conversely, create/place a new `redflag.md` file (or directory) at the root of your vault to force-suspend synchronisation and stop Obsidian's boot-up sequence if you need to fix a database issue.
### JWT Authentication Errors
#### DataError when configuring JWT authentication
If you encounter a `DataError:` with no additional information in the logs when configuring JWT authentication, this usually indicates a private key formatting issue.
Self-hosted LiveSync requires the private key (for ES256/ES512 algorithms) to be in the **PKCS#8 PEM** format. Standard SEC1 EC private keys (which begin with `-----BEGIN EC PRIVATE KEY-----`) will trigger this error.
To resolve this, convert your private key to PKCS#8 format using the following `openssl` command:
```bash
openssl pkcs8 -topk8 -inform PEM -nocrypt -in private.key -out pkcs8.key
```
Then paste the contents of `pkcs8.key` (which begins with `-----BEGIN PRIVATE KEY-----`) into the JWT Key field.
### Old tips
- Rarely, a file in the database could be corrupted. The plug-in will not write
to local storage when a file looks corrupted. If a local version of the file
is on your device, the corruption could be fixed by editing the local file and
synchronising it. But if the file does not exist on any of your devices, then
it can not be rescued. In this case, you can delete these items from the
settings dialogue.
- To stop the boot-up sequence (eg. for fixing problems on databases), you can
put a `redflag.md` file (or directory) at the root of your vault. Tip for iOS:
a redflag directory can be created at the root of the vault using the File
application.
- Also, with `redflag2.md` placed, we can automatically overwrite server data with this device's files during the boot-up sequence. With `redflag3.md`, we
can discard only the local database and reset synchronisation on this device.
- Q: The database is growing, how can I shrink it down? A: each of the docs is
saved with their past 100 revisions for detecting and resolving conflicts.
Picturing that one device has been offline for a while, and comes online
again. The device has to compare its notes with the remotely saved ones. If
there exists a historic revision in which the note used to be identical, it
could be updated safely (like git fast-forward). Even if that is not in
revision histories, we only have to check the differences after the revision
that both devices commonly have. This is like git's conflict-resolving method.
So, We have to make the database again like an enlarged git repo if you want
to solve the root of the problem.
- And more technical Information is in the [Technical Information](tech_info.md)
- If you want to synchronise files without obsidian, you can use
[filesystem-livesync](https://github.com/vrtmrz/filesystem-livesync).
- WebClipper is also available on Chrome Web
Store:[obsidian-livesync-webclip](https://chrome.google.com/webstore/detail/obsidian-livesync-webclip/jfpaflmpckblieefkegjncjoceapakdf)
Repo is here:
[obsidian-livesync-webclip](https://github.com/vrtmrz/obsidian-livesync-webclip).
(Docs are a work in progress.)
Binary file not shown.

After

Width:  |  Height:  |  Size: 46 KiB

+219 -39
View File
@@ -1,46 +1,226 @@
//@ts-check
import esbuild from "esbuild";
import process from "process";
import builtins from "builtin-modules";
import sveltePlugin from "esbuild-svelte";
import sveltePreprocess from "svelte-preprocess";
import { sveltePreprocess } from "svelte-preprocess";
import fs from "node:fs";
const banner = `/*
THIS IS A GENERATED/BUNDLED FILE BY ESBUILD
if you want to view the source, please visit the github repository of this plugin
*/
`;
// import terser from "terser";
import { minify } from "terser";
import inlineWorkerPlugin from "esbuild-plugin-inline-worker";
import { terserOption } from "./terser.config.mjs";
import path from "node:path";
const prod = process.argv[2] === "production";
const manifestJson = JSON.parse(fs.readFileSync("./manifest.json"));
const packageJson = JSON.parse(fs.readFileSync("./package.json"));
const prod = process.argv[2] === "production" || process.env?.BUILD_MODE === "production";
const keepTest = true; //!prod;
const manifestJson = JSON.parse(fs.readFileSync("./manifest.json") + "");
const packageJson = JSON.parse(fs.readFileSync("./package.json") + "");
const updateInfo = JSON.stringify(fs.readFileSync("./updates.md") + "");
esbuild
.build({
banner: {
js: banner,
const PATHS_TEST_INSTALL = process.env?.PATHS_TEST_INSTALL || "";
const PATH_TEST_INSTALL = PATHS_TEST_INSTALL.split(path.delimiter)
.map((p) => p.trim())
.filter((p) => p.length);
if (PATH_TEST_INSTALL) {
console.log(`Built files will be copied to ${PATH_TEST_INSTALL}`);
} else {
console.log(
"Development build: You can install the plug-in to Obsidian for testing by exporting the PATHS_TEST_INSTALL environment variable with the paths to your vault plugins directories separated by your system path delimiter (':' on Unix, ';' on Windows)."
);
}
const moduleAliasPlugin = {
name: "module-alias",
setup(build) {
build.onResolve({ filter: /.(dev)(.ts|)$/ }, (args) => {
// console.log(args.path);
if (prod) {
let prodTs = args.path.replace(".dev", ".prod");
const statFile = prodTs.endsWith(".ts") ? prodTs : prodTs + ".ts";
const realPath = path.join(args.resolveDir, statFile);
console.log(`Checking ${statFile}`);
if (fs.existsSync(realPath)) {
console.log(`Replaced ${args.path} with ${prodTs}`);
return {
path: realPath,
namespace: "file",
};
}
}
return null;
});
build.onResolve({ filter: /.(platform)(.ts|)$/ }, (args) => {
// console.log(args.path);
if (prod) {
let prodTs = args.path.replace(".platform", ".obsidian");
const statFile = prodTs.endsWith(".ts") ? prodTs : prodTs + ".ts";
const realPath = path.join(args.resolveDir, statFile);
console.log(`Checking ${statFile}`);
if (fs.existsSync(realPath)) {
console.log(`Replaced ${args.path} with ${prodTs}`);
return {
path: realPath,
namespace: "file",
};
}
}
return null;
});
},
};
const removePragmaCommentsPlugin = {
name: "remove-pragma-comments",
setup(build) {
const sourceRoot = `${path.resolve("src")}${path.sep}`;
// Filter target extensions (e.g., JavaScript and TypeScript)
build.onLoad({ filter: /\.[jt]s?$/ }, async (args) => {
// Dependencies are already compiled and may contain comment-like text inside strings.
// Only maintained plug-in source needs its local suppression directives removed.
if (!args.path.startsWith(sourceRoot)) return;
const source = await fs.promises.readFile(args.path, "utf8");
// Regex targeting both single-line and multi-line comments
// This regex looks for:
// - /* eslint ... */ (multi-line)
// const esLintPragmaRegexBlock = /\/\*[\s\S]*?eslint[\s\S]*?\*\/|([^\\:]|^)\/\/.*eslint.*$/gm;
// - // eslint-disable-next-line
let cleanedSource = source;
const tsIgnoreRegex = /\/\*\s*@ts-ignore\s*\*\/|([^\\:]|^)\/\/.*?@ts-ignore.*$/gm;
const esLintPragmaRegexLine = /([^\\:]|^)\/\/.*?eslint-.*$/gm;
const exps = [tsIgnoreRegex, esLintPragmaRegexLine];
for (const exp of exps) {
cleanedSource = cleanedSource.replace(exp, "$1");
}
return {
contents: cleanedSource,
loader: args.path.endsWith("ts") ? "ts" : "js",
};
});
},
};
/** @type esbuild.Plugin[] */
const plugins = [
{
name: "my-plugin",
setup(build) {
let count = 0;
build.onEnd(async (result) => {
if (count++ === 0) {
console.log("first build:");
if (prod) {
console.log("MetaFile:");
if (result.metafile) {
fs.writeFileSync("meta.json", JSON.stringify(result.metafile));
let text = await esbuild.analyzeMetafile(result.metafile, {
verbose: true,
});
// console.log(text);
}
}
} else {
console.log("subsequent build:");
}
const filename = `meta-${prod ? "prod" : "dev"}.json`;
await fs.promises.writeFile(filename, JSON.stringify(result.metafile, null, 2));
if (prod) {
console.log("Performing terser");
const src = fs.readFileSync("./main_org.js").toString();
// @ts-ignore
const ret = await minify(src, terserOption);
if (ret && ret.code) {
fs.writeFileSync("./main.js", ret.code);
}
console.log("Finished terser");
} else {
fs.copyFileSync("./main_org.js", "./main.js");
}
if (PATH_TEST_INSTALL) {
for (const installPath of PATH_TEST_INSTALL) {
const realPath = path.resolve(installPath);
console.log(`Copying built files to ${realPath}`);
if (!fs.existsSync(realPath)) {
console.warn(`Test install path ${installPath} does not exist`);
continue;
}
const manifestX = JSON.parse(fs.readFileSync("./manifest.json") + "");
manifestX.version = manifestJson.version + "." + Date.now();
fs.writeFileSync(path.join(installPath, "manifest.json"), JSON.stringify(manifestX, null, 2));
fs.copyFileSync("./main.js", path.join(installPath, "main.js"));
fs.copyFileSync("./styles.css", path.join(installPath, "styles.css"));
}
}
});
},
entryPoints: ["src/main.ts"],
bundle: true,
define: {
"MANIFEST_VERSION": `"${manifestJson.version}"`,
"PACKAGE_VERSION": `"${packageJson.version}"`,
"UPDATE_INFO": `${updateInfo}`,
"global":"window",
},
external: ["obsidian", "electron", "crypto"],
format: "cjs",
watch: !prod,
target: "es2018",
logLevel: "info",
sourcemap: prod ? false : "inline",
treeShaking: true,
platform: "browser",
plugins: [
sveltePlugin({
preprocess: sveltePreprocess(),
compilerOptions: { css: true },
}),
],
outfile: "main.js",
})
.catch(() => process.exit(1));
},
];
const externals = [
"obsidian",
"electron",
"crypto",
"@codemirror/autocomplete",
"@codemirror/collab",
"@codemirror/commands",
"@codemirror/language",
"@codemirror/lint",
"@codemirror/search",
"@codemirror/state",
"@codemirror/view",
"@lezer/common",
"@lezer/highlight",
"@lezer/lr",
];
const context = await esbuild.context({
banner: {
js: "// Leave it all to terser",
},
entryPoints: ["src/main.ts"],
bundle: true,
define: {
MANIFEST_VERSION: `"${manifestJson.version}"`,
PACKAGE_VERSION: `"${packageJson.version}"`,
UPDATE_INFO: `${updateInfo}`,
global: "window",
},
external: externals,
// minifyWhitespace: true,
format: "cjs",
target: "es2018",
logLevel: "info",
platform: "browser",
metafile: true,
sourcemap: prod ? false : "inline",
treeShaking: false,
outfile: "main_org.js",
mainFields: ["browser", "module", "main"],
minifyWhitespace: false,
minifySyntax: false,
minifyIdentifiers: false,
minify: false,
dropLabels: prod && !keepTest ? ["TEST", "DEV"] : [],
// keepNames: true,
plugins: [
moduleAliasPlugin,
inlineWorkerPlugin({
external: externals,
treeShaking: true,
}),
sveltePlugin({
preprocess: sveltePreprocess(),
compilerOptions: { css: "injected", preserveComments: false },
}),
removePragmaCommentsPlugin,
...plugins,
],
});
if (prod) {
await context.rebuild();
process.exit(0);
} else {
await context.watch();
}
+81
View File
@@ -0,0 +1,81 @@
import obsidianmd from "eslint-plugin-obsidianmd";
import globals from "globals";
import { defineConfig, globalIgnores } from "eslint/config";
export default defineConfig(
globalIgnores([
"node_modules",
"**/dist",
"build",
"coverage",
"main.js",
"main_org.js",
"pouchdb-browser.js",
"package.json",
"package-lock.json",
"versions.json",
// Svelte is covered by the project lint and svelte-check; the directory report currently analyses TypeScript.
"**/*.svelte",
"**/svelte.config.js",
"**/*.unit.spec.ts",
"**/test/**",
"src/apps/_test/**",
"src/apps/cli/testdeno/**",
]),
{
languageOptions: {
globals: {
...globals.browser,
...globals.node,
MANIFEST_VERSION: "readonly",
PACKAGE_VERSION: "readonly",
UPDATE_INFO: "readonly",
hostPlatform: "readonly",
},
parserOptions: {
project: [
"./tsconfig.json",
"./src/apps/browser/tsconfig.json",
"./src/apps/cli/tsconfig.json",
"./src/apps/webapp/tsconfig.json",
"./src/apps/webpeer/tsconfig.app.json",
"./src/apps/webpeer/tsconfig.node.json",
],
tsconfigRootDir: import.meta.dirname,
},
},
},
...obsidianmd.configs.recommended,
{
rules: {
// The directory review reports console usage as guidance rather than a release blocker.
"obsidianmd/rule-custom-message": "off",
"no-console": "warn",
"obsidianmd/no-unsupported-api": "error",
// Keep legacy type-safety debt visible while reserving errors for directory-review blockers.
"@typescript-eslint/no-unsafe-argument": "warn",
"@typescript-eslint/no-unsafe-assignment": "warn",
"@typescript-eslint/no-unsafe-call": "warn",
"@typescript-eslint/no-unsafe-member-access": "warn",
"@typescript-eslint/no-unsafe-return": "warn",
"@typescript-eslint/no-base-to-string": "warn",
"@typescript-eslint/no-redundant-type-constituents": "warn",
"@typescript-eslint/no-unnecessary-type-assertion": "warn",
},
},
{
files: ["src/apps/**/*.{ts,js,mjs}"],
rules: {
// These applications are inspected by the directory review but accept external command and file data.
"@typescript-eslint/restrict-template-expressions": "warn",
},
},
{
files: ["src/apps/browser/**/*.ts", "src/apps/webapp/**/*.ts"],
rules: {
"obsidianmd/no-static-styles-assignment": "off",
"obsidianmd/prefer-create-el": "off",
"obsidianmd/prefer-active-doc": "off",
},
}
);
+142
View File
@@ -0,0 +1,142 @@
const restrictedGlobalsOptions = [
{
name: "app",
message: "Avoid using the global app object. Instead use the reference provided by your plugin instance.",
},
"warn",
{
name: "fetch",
message: "Use the built-in `requestUrl` function instead of `fetch` for network requests in Obsidian.",
},
{
name: "localStorage",
message:
"Prefer `App#saveLocalStorage` / `App#loadLocalStorage` functions to write / read localStorage data that's unique to a vault.",
},
];
const restrictedImportsOptions = [
{
name: "axios",
message: "Use the built-in `requestUrl` function instead of `axios`.",
},
{
name: "superagent",
message: "Use the built-in `requestUrl` function instead of `superagent`.",
},
{
name: "got",
message: "Use the built-in `requestUrl` function instead of `got`.",
},
{
name: "ofetch",
message: "Use the built-in `requestUrl` function instead of `ofetch`.",
},
{
name: "ky",
message: "Use the built-in `requestUrl` function instead of `ky`.",
},
{
name: "node-fetch",
message: "Use the built-in `requestUrl` function instead of `node-fetch`.",
},
{
name: "moment",
message: "The 'moment' package is bundled with Obsidian. Please import it from 'obsidian' instead.",
},
];
const warnWhileDev = "off";
/**
* @type {import("eslint").Linter.RulesRecord}
*/
export const baseRules = {
// -- Base rules (turned off in favour of TS specific versions or explicitly disabled).
"no-unused-vars": "off",
"no-unused-labels": "off",
"no-prototype-builtins": "off",
"require-await": "off",
// -- TypeScript specific rules (Gradual adoption of stricter rules, currently set to 'warn' for a while).
"@typescript-eslint/no-explicit-any": "warn",
"@typescript-eslint/no-redundant-type-constituents": "warn",
// -- TypeScript specific rules
// @typescript-eslint/no-unsafe-* rules and @typescript-eslint/no-explicit-any:
// This project contains a lot of library-sh code where the use of `any` is often necessary and justified.
// Rules is now set to 'off' for a while.
"@typescript-eslint/no-unsafe-argument": "off",
"@typescript-eslint/no-unsafe-call": "off",
"@typescript-eslint/no-unsafe-member-access": "off",
"@typescript-eslint/no-unsafe-return": "off",
"@typescript-eslint/no-unsafe-assignment": "off",
// -- Reasonable rules.
"@typescript-eslint/no-deprecated": warnWhileDev,
"@typescript-eslint/no-unused-vars": ["error", { args: "none" }],
"@typescript-eslint/ban-ts-comment": "off",
"@typescript-eslint/no-empty-function": "off",
"@typescript-eslint/require-await": "error",
"@typescript-eslint/no-misused-promises": "error",
"@typescript-eslint/no-floating-promises": "error",
"@typescript-eslint/no-unnecessary-type-assertion": "error",
// -- General rules
"no-async-promise-executor": warnWhileDev,
"no-constant-condition": ["error", { checkLoops: false }],
// -- Disabled rules
// no-undef: This option breaks the global declarations for the library files and is not worth the effort to fix at this time.
"no-undef": "off",
};
/**
* @type {import("eslint").Linter.RulesRecord}
*/
export const obsidianRules = {
// -- Obsidian rules
// Runtime fallbacks must use requireApiVersion guards recognised by the official rule.
"obsidianmd/no-unsupported-api": "error",
// -- Plugin specific overrides
"obsidianmd/rule-custom-message": "off",
"obsidianmd/ui/sentence-case": "off",
"obsidianmd/no-plugin-as-component": "warn",
// -- Temporary overrides for migration
"obsidianmd/no-static-styles-assignment": "off",
};
/**
* @type {(base:string) => import("eslint").Linter.RulesRecord}
*/
export const ImportAliasRules = (base) => ({
"@dword-design/import-alias/prefer-alias": [
"error",
{
aliasForSubpaths: true,
alias: {
"@": `${base}/src`,
},
},
],
});
/**
* @type {import("eslint").Linter.RulesRecord}
*/
export const CommunityReviewRecommendedRules = {
"no-unused-vars": "off",
"no-prototype-bultins": "off",
"no-self-compare": "warn",
"no-eval": "error",
"no-implied-eval": "error",
"prefer-const": "off",
"no-implicit-globals": "error",
"no-console": "off", // overridden by obsidianmd/rule-custom-message
"no-restricted-globals": ["error", ...restrictedGlobalsOptions],
"no-restricted-imports": ["error", ...restrictedImportsOptions],
"no-alert": "error",
"no-undef": "error",
"@typescript-eslint/ban-ts-comment": "off",
"@typescript-eslint/no-deprecated": "error",
"@typescript-eslint/no-unused-vars": ["warn", { args: "none" }],
"@typescript-eslint/require-await": "off",
"@typescript-eslint/no-explicit-any": ["error", { fixToUnknown: true }],
// "import/no-nodejs-modules": "off",
// "import/no-extraneous-dependencies": "error",
};
+121
View File
@@ -0,0 +1,121 @@
import tsParser from "@typescript-eslint/parser";
import obsidianmd from "eslint-plugin-obsidianmd";
import globals from "globals";
import { defineConfig, globalIgnores } from "eslint/config";
import * as sveltePlugin from "eslint-plugin-svelte";
import svelteParser from "svelte-eslint-parser";
import importAlias from "@dword-design/eslint-plugin-import-alias";
import { baseRules, ImportAliasRules, obsidianRules } from "./eslint.config.common.mjs";
const warnWhileDev = "off"; // Change to "warn" to enable warnings for rules that are currently disabled.
const lintProjects = [
"./tsconfig.json",
"./src/apps/browser/tsconfig.json",
"./src/apps/cli/tsconfig.json",
"./src/apps/webapp/tsconfig.json",
"./src/apps/webpeer/tsconfig.app.json",
"./src/apps/webpeer/tsconfig.node.json",
];
export default defineConfig([
globalIgnores([
// Build outputs and legacy files
"**/build",
"**/dist/**",
"coverage",
"**/main.js",
"main_org.js",
"pouchdb-browser.js",
"version-bump.mjs",
"package.json",
"**/*.json",
"**/.eslintrc.js.bak",
// Files from linked dependencies (those files should not exist for most people).
"modules/octagonal-wheels/dist",
// Sub-project tooling with its own environment
"utils",
// Config files and build scripts
"**/jest.config.js",
"**/rollup.config.js",
"**/esbuild.config.mjs",
"**/terser.*.mjs",
".prettierrc.*.mjs",
".prettierrc.mjs",
"*.config.mjs",
"vite.*",
"vitest.*",
// Testing files (Simplified patterns)
"test/**",
"**/test/**",
"src/apps/_test/**",
"src/apps/cli/testdeno/**",
"**/*.test.ts",
"**/*.unit.spec.ts",
"**/test.ts",
"**/tests.ts",
]),
...sveltePlugin.configs["flat/base"],
...obsidianmd.configs.recommended,
importAlias.configs.recommended,
{
files: ["**/*.ts"],
languageOptions: {
globals: { ...globals.browser, PouchDB: "readonly" },
parser: tsParser,
parserOptions: {
project: lintProjects,
tsconfigRootDir: import.meta.dirname,
},
},
linterOptions: {
reportUnusedDisableDirectives: false,
},
rules: {
...baseRules,
...obsidianRules,
// -- Project specific rules
...ImportAliasRules("."),
},
},
{
files: ["**/*.svelte"],
languageOptions: {
globals: { ...globals.browser, PouchDB: "readonly" },
parser: svelteParser,
parserOptions: {
parser: tsParser,
project: lintProjects,
tsconfigRootDir: import.meta.dirname,
extraFileExtensions: [".svelte"],
},
},
rules: {
// no-unused-vars:
// Svelte template's declarations have a lot of false positives and the rule is not worth the effort to fix at this time.
// it may improve in the future with some options as like ["error", { argsIgnorePattern: "^_", varsIgnorePattern: "^_" }],]
"no-unused-vars": "off",
...obsidianRules,
"obsidianmd/no-plugin-as-component": "off",
...ImportAliasRules("."),
},
},
{
files: ["src/apps/**/*.ts"],
rules: {
// Platform adapters implement asynchronous contracts even when a local operation is synchronous.
"@typescript-eslint/require-await": "off",
// Keep existing application code visible without making gradual type tightening a release blocker.
"@typescript-eslint/no-base-to-string": "warn",
"@typescript-eslint/no-unnecessary-type-assertion": "warn",
"@typescript-eslint/restrict-template-expressions": "warn",
},
},
{
files: ["src/apps/browser/**/*.{ts,svelte}", "src/apps/webapp/**/*.ts"],
rules: {
// Browser applications use the DOM rather than Obsidian's DOM extensions.
"obsidianmd/prefer-create-el": "off",
"obsidianmd/prefer-active-doc": "off",
},
},
]);
+1
View File
@@ -0,0 +1 @@
PATHS_TEST_INSTALL=your-vault-plugin-path:and-another-path
Binary file not shown.

After

Width:  |  Height:  |  Size: 11 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 41 KiB

BIN
View File
Binary file not shown.

After

Width:  |  Height:  |  Size: 13 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 25 KiB

After

Width:  |  Height:  |  Size: 20 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 74 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 36 KiB

After

Width:  |  Height:  |  Size: 5.7 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 10 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 17 KiB

+3 -3
View File
@@ -1,10 +1,10 @@
{
"id": "obsidian-livesync",
"name": "Self-hosted LiveSync",
"version": "0.19.7",
"minAppVersion": "0.9.12",
"version": "0.25.83",
"minAppVersion": "1.7.2",
"description": "Community implementation of self-hosted livesync. Reflect your vault changes to some other devices immediately. Please make sure to disable other synchronize solutions to avoid content corruption or duplication.",
"author": "vorotamoroz",
"authorUrl": "https://github.com/vrtmrz",
"isDesktopOnly": false
}
}
+14962 -5658
View File
File diff suppressed because it is too large Load Diff
+167 -34
View File
@@ -1,52 +1,185 @@
{
"name": "obsidian-livesync",
"version": "0.19.7",
"version": "0.25.83",
"description": "Reflect your vault changes to some other devices immediately. Please make sure to disable other synchronize solutions to avoid content corruption or duplication.",
"main": "main.js",
"type": "module",
"scripts": {
"dev": "node esbuild.config.mjs",
"dev": "node --env-file=.env esbuild.config.mjs",
"build": "node esbuild.config.mjs production",
"lint": "eslint src"
"buildVite": "npx dotenv-cli -e .env -- vite build --mode production",
"buildViteOriginal": "npx dotenv-cli -e .env -- vite build --mode original",
"buildDev": "node esbuild.config.mjs dev",
"lint": "eslint --cache --cache-strategy content --concurrency auto src",
"lint:community": "eslint --config eslint.community.config.mjs --concurrency auto src",
"svelte-check": "svelte-check --tsconfig ./tsconfig.json --fail-on-warnings",
"tsc-check": "tsc --noEmit",
"tsc-check:apps": "tsc --noEmit -p src/apps/browser/tsconfig.json && tsc --noEmit -p src/apps/cli/tsconfig.json && tsc --noEmit -p src/apps/webapp/tsconfig.json && tsc --noEmit -p src/apps/webpeer/tsconfig.app.json && tsc --noEmit -p src/apps/webpeer/tsconfig.node.json",
"pretty:importpath": "cd utilsdeno && deno run -A ./normalise-imports.ts",
"pretty:json": "prettier --config ./.prettierrc.mjs \"**/*.json\" --write --log-level error",
"pretty": "npm run prettyNoWrite -- --write --log-level error",
"prettyCheck": "npm run prettyNoWrite -- --check",
"prettyNoWrite": "prettier --config ./.prettierrc.mjs \"**/*.js\" \"**/*.ts\" \"**/*.json\" ",
"precheck:compatibility": "npm run build",
"check:compatibility": "node utils/check-compatibility.js --file main.js --ios 15",
"check": "npm run tsc-check && npm run tsc-check:apps && npm run lint && npm run lint:community -- --quiet && npm run svelte-check && npm run check:compatibility",
"unittest": "deno test -A --no-check --coverage=cov_profile --v8-flags=--expose-gc --trace-leaks ./src/",
"test": "vitest run",
"test:unit": "vitest run --config vitest.config.unit.ts",
"test:contract:contexts": "vitest run --config vitest.config.unit.ts src/modules/services/ObsidianServiceContext.unit.spec.ts src/apps/cli/services/NodeServiceContext.unit.spec.ts src/apps/browser/createLiveSyncBrowserServiceHub.unit.spec.ts",
"test:contract:context:webapp": "vitest run --config vitest.config.unit.ts src/apps/browser/createLiveSyncBrowserServiceHub.unit.spec.ts",
"test:contract:context:cli": "npm run build --workspace self-hosted-livesync-cli && deno task --cwd src/apps/cli/testdeno test:setup-put-cat",
"test:contract:context:obsidian": "npm run build && npm run test:e2e:obsidian:smoke",
"test:e2e:cli": "npm run test:e2e:ci --workspace self-hosted-livesync-cli",
"test:e2e:cli:p2p": "npm run test:e2e:p2p --workspace self-hosted-livesync-cli",
"test:e2e:cli:all": "npm run test:e2e:all --workspace self-hosted-livesync-cli",
"test:integration": "npx dotenv-cli -e .env -e .test.env -- vitest run --config vitest.config.integration.ts",
"test:unit:coverage": "vitest run --config vitest.config.unit.ts --coverage",
"test:install-playwright": "npx playwright install chromium",
"test:install-dependencies": "npm run test:install-playwright",
"test:e2e:obsidian:install-appimage": "tsx test/e2e-obsidian/scripts/install-appimage.ts",
"test:e2e:obsidian:discover": "tsx test/e2e-obsidian/scripts/discover.ts",
"test:e2e:obsidian:cli-help": "tsx test/e2e-obsidian/scripts/cli-help.ts",
"test:e2e:obsidian:debug-ui": "tsx test/e2e-obsidian/scripts/debug-ui.ts",
"test:e2e:obsidian:smoke": "tsx test/e2e-obsidian/scripts/smoke.ts",
"test:e2e:obsidian:dialog-mounts": "tsx test/e2e-obsidian/scripts/dialog-mounts.ts",
"test:e2e:obsidian:settings-ui": "tsx test/e2e-obsidian/scripts/settings-ui.ts",
"test:e2e:obsidian:vault-reflection": "tsx test/e2e-obsidian/scripts/vault-reflection.ts",
"test:e2e:obsidian:couchdb-upload": "tsx test/e2e-obsidian/scripts/couchdb-upload.ts",
"test:e2e:obsidian:cli-to-obsidian-sync": "tsx test/e2e-obsidian/scripts/cli-to-obsidian-sync.ts",
"test:e2e:obsidian:minio-upload": "tsx test/e2e-obsidian/scripts/minio-upload.ts",
"test:e2e:obsidian:startup-scan": "tsx test/e2e-obsidian/scripts/startup-scan.ts",
"test:e2e:obsidian:two-vault-sync": "tsx test/e2e-obsidian/scripts/two-vault-sync.ts",
"test:e2e:obsidian:hidden-file-snippet-sync": "tsx test/e2e-obsidian/scripts/hidden-file-snippet-sync.ts",
"test:e2e:obsidian:customisation-sync": "tsx test/e2e-obsidian/scripts/customisation-sync.ts",
"test:e2e:obsidian:setting-markdown-export": "tsx test/e2e-obsidian/scripts/setting-markdown-export.ts",
"test:e2e:obsidian:local-suite": "tsx test/e2e-obsidian/scripts/local-suite.ts",
"test:e2e:obsidian:local-suite:services": "tsx test/e2e-obsidian/scripts/local-suite.ts --manage-services",
"test:coverage": "vitest run --coverage",
"test:docker-couchdb:up": "npx dotenv-cli -e .env -e .test.env -- ./test/shell/couchdb-start.sh",
"test:docker-couchdb:init": "npx dotenv-cli -e .env -e .test.env -- ./test/shell/couchdb-init.sh",
"test:docker-couchdb:start": "npm run test:docker-couchdb:up && sleep 5 && npm run test:docker-couchdb:init",
"test:docker-couchdb:down": "npx dotenv-cli -e .env -e .test.env -- ./test/shell/couchdb-stop.sh",
"test:docker-couchdb:stop": "npm run test:docker-couchdb:down",
"test:docker-s3:up": "npx dotenv-cli -e .env -e .test.env -- ./test/shell/minio-start.sh",
"test:docker-s3:init": "npx dotenv-cli -e .env -e .test.env -- ./test/shell/minio-init.sh",
"test:docker-s3:start": "npm run test:docker-s3:up && sleep 3 && npm run test:docker-s3:init",
"test:docker-s3:down": "npx dotenv-cli -e .env -e .test.env -- ./test/shell/minio-stop.sh",
"test:docker-s3:stop": "npm run test:docker-s3:down",
"test:docker-p2p:up": "npx dotenv-cli -e .env -e .test.env -- ./test/shell/p2p-start.sh",
"test:docker-p2p:init": "npx dotenv-cli -e .env -e .test.env -- ./test/shell/p2p-init.sh",
"test:docker-p2p:start": "npm run test:docker-p2p:up && sleep 3 && npm run test:docker-p2p:init",
"test:docker-p2p:down": "npx dotenv-cli -e .env -e .test.env -- ./test/shell/p2p-stop.sh",
"test:docker-p2p:stop": "npm run test:docker-p2p:down",
"test:docker-all:up": "npm run test:docker-couchdb:up ; npm run test:docker-s3:up ; npm run test:docker-p2p:up",
"test:docker-all:init": "npm run test:docker-couchdb:init ; npm run test:docker-s3:init ; npm run test:docker-p2p:init",
"test:docker-all:down": "npm run test:docker-couchdb:down ; npm run test:docker-s3:down ; npm run test:docker-p2p:down",
"test:docker-all:start": "npm run test:docker-all:up && sleep 5 && npm run test:docker-all:init",
"test:docker-all:stop": "npm run test:docker-all:down",
"test:full": "npm run test:docker-all:start && vitest run --coverage && npm run test:docker-all:stop",
"test:p2p": "bash test/suitep2p/run-p2p-tests.sh",
"update-workspaces": "node update-workspaces.mjs",
"version": "node version-bump.mjs && node update-workspaces.mjs && npm run pretty:json && git add manifest.json versions.json src/apps/cli/package.json src/apps/webpeer/package.json src/apps/webapp/package.json"
},
"keywords": [],
"author": "vorotamoroz",
"license": "MIT",
"devDependencies": {
"@tsconfig/svelte": "^4.0.1",
"@types/diff-match-patch": "^1.0.32",
"@types/node": "^20.2.5",
"@types/pouchdb": "^6.4.0",
"@types/pouchdb-browser": "^6.1.3",
"@typescript-eslint/eslint-plugin": "^5.54.0",
"@typescript-eslint/parser": "^5.54.0",
"builtin-modules": "^3.3.0",
"esbuild": "0.15.15",
"esbuild-svelte": "^0.7.3",
"eslint": "^8.35.0",
"eslint-config-airbnb-base": "^15.0.0",
"eslint-plugin-import": "^2.27.5",
"@dword-design/eslint-plugin-import-alias": "^8.1.8",
"@emnapi/core": "1.11.1",
"@emnapi/runtime": "1.11.1",
"@eslint/js": "^9.39.3",
"@playwright/test": "^1.58.2",
"@sveltejs/vite-plugin-svelte": "^7.1.2",
"@tsconfig/svelte": "^5.0.8",
"@types/deno": "^2.5.0",
"@types/diff-match-patch": "^1.0.36",
"@types/markdown-it": "^14.1.2",
"@types/micromatch": "^4.0.10",
"@types/node": "^24.10.13",
"@types/pouchdb": "^6.4.2",
"@types/pouchdb-adapter-http": "^6.1.6",
"@types/pouchdb-adapter-idb": "^6.1.7",
"@types/pouchdb-browser": "^6.1.5",
"@types/pouchdb-core": "^7.0.15",
"@types/pouchdb-mapreduce": "^6.1.10",
"@types/pouchdb-replication": "^6.4.7",
"@types/transform-pouch": "^1.0.6",
"@typescript-eslint/parser": "8.56.1",
"@vitest/browser": "^4.1.8",
"@vitest/browser-playwright": "^4.1.8",
"@vitest/coverage-v8": "^4.1.8",
"@vrtmrz/obsidian-test-session": "0.2.0",
"dotenv-cli": "^11.0.0",
"esbuild": "0.28.1",
"esbuild-plugin-inline-worker": "^0.1.1",
"esbuild-svelte": "^0.9.4",
"eslint": "^9.39.3",
"eslint-plugin-obsidianmd": "^0.4.1",
"eslint-plugin-svelte": "^3.19.0",
"events": "^3.3.0",
"obsidian": "^1.1.1",
"postcss": "^8.4.21",
"postcss-load-config": "^4.0.1",
"pouchdb-adapter-http": "^8.0.1",
"pouchdb-adapter-idb": "^8.0.1",
"pouchdb-adapter-indexeddb": "^8.0.1",
"pouchdb-core": "^8.0.1",
"pouchdb-find": "^8.0.1",
"pouchdb-mapreduce": "^8.0.1",
"pouchdb-replication": "^8.0.1",
"pouchdb-utils": "^8.0.1",
"svelte": "^3.59.1",
"svelte-preprocess": "^5.0.3",
"globals": "^14.0.0",
"playwright": "^1.58.2",
"postcss": "^8.5.6",
"pouchdb-adapter-http": "^9.0.0",
"pouchdb-adapter-idb": "^9.0.0",
"pouchdb-adapter-indexeddb": "^9.0.0",
"pouchdb-adapter-memory": "^9.0.0",
"pouchdb-core": "^9.0.0",
"pouchdb-errors": "^9.0.0",
"pouchdb-find": "^9.0.0",
"pouchdb-mapreduce": "^9.0.0",
"pouchdb-merge": "^9.0.0",
"pouchdb-replication": "^9.0.0",
"pouchdb-utils": "^9.0.0",
"prettier": "3.8.1",
"rollup-plugin-copy": "^3.5.0",
"svelte": "5.56.3",
"svelte-check": "^4.6.0",
"svelte-eslint-parser": "^1.8.0",
"svelte-preprocess": "^6.0.3",
"terser": "^5.39.0",
"tinyglobby": "^0.2.15",
"transform-pouch": "^2.0.0",
"tslib": "^2.5.0",
"typescript": "^5.0.4"
"tsx": "^4.21.0",
"typescript": "5.9.3",
"typescript-eslint": "^8.61.0",
"vite": "^8.0.16",
"vitest": "^4.1.8",
"webdriverio": "^9.27.0",
"yaml": "^2.8.2"
},
"dependencies": {
"@aws-sdk/client-s3": "^3.808.0",
"@smithy/fetch-http-handler": "^5.3.10",
"@smithy/md5-js": "^4.2.9",
"@smithy/middleware-apply-body-checksum": "^4.3.9",
"@smithy/protocol-http": "^5.3.9",
"@smithy/querystring-builder": "^4.2.9",
"@smithy/types": "^4.14.3",
"@smithy/util-retry": "^4.4.5",
"@vrtmrz/livesync-commonlib": "0.1.0-rc.2",
"diff-match-patch": "^1.0.5",
"idb": "^7.1.1",
"xxhash-wasm": "^0.4.2"
}
"fflate": "^0.8.2",
"idb": "^8.0.3",
"markdown-it": "^14.2.0",
"minimatch": "^10.2.5",
"obsidian": "^1.13.1",
"octagonal-wheels": "^0.1.51",
"qrcode-generator": "^1.4.4",
"xxhash-wasm-102": "npm:xxhash-wasm@^1.0.2"
},
"overrides": {
"pouchdb-core": {
"uuid": "11.1.1"
},
"pouchdb-utils": {
"uuid": "11.1.1"
}
},
"workspaces": [
"src/apps/cli",
"src/apps/webpeer",
"src/apps/webapp"
]
}
+151
View File
@@ -0,0 +1,151 @@
{
"cells": [
{
"cell_type": "markdown",
"metadata": {
"colab_type": "text",
"id": "view-in-github"
},
"source": [
"<a href=\"https://colab.research.google.com/gist/vrtmrz/9402b101746e08e969b1a4f5f0deb465/setup-flyio-on-the-fly-v2.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "AzLlAcLFRO5A"
},
"source": [
"- Initial version 7th Feb. 2024"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "z1x8DQpa9opC"
},
"outputs": [],
"source": [
"# Install prerequesties\n",
"!curl -L https://fly.io/install.sh | sh\n",
"!curl -fsSL https://deno.land/x/install/install.sh | sh\n",
"!apt update && apt -y install jq\n",
"import os\n",
"%env PATH=/root/.fly/bin:/root/.deno/bin/:{os.environ[\"PATH\"]}\n",
"!git clone --recursive https://github.com/vrtmrz/obsidian-livesync"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "mGN08BaFDviy"
},
"outputs": [],
"source": [
"# Login up sign up\n",
"!flyctl auth signup"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "BBFTFOP6vA8m"
},
"source": [
"Select a region and execute the block."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "TNl0A603EF9E"
},
"outputs": [],
"source": [
"# see https://fly.io/docs/reference/regions/\n",
"region = \"nrt/Tokyo, Japan\" #@param [\"jnb/Johannesburg, South Africa\",\"bom/Mumbai, India\",\"sin/Singapore, Singapore\",\"syd/Sydney, Australia\",\"nrt/Tokyo, Japan\",\"ams/Amsterdam, Netherlands\",\"fra/Frankfurt, Germany\",\"lhr/London, United Kingdom\",\"cdg/Paris, France\",\"arn/Stockholm, Sweden\",\"iad/Ashburn, Virginia (US)\",\"ord/Chicago, Illinois (US)\",\"dfw/Dallas, Texas (US)\",\"lax/Los Angeles, California (US)\",\"sjc/San Jose, California (US)\",\"ewr/Secaucus, NJ (US)\",\"yyz/Toronto, Canada\",\"gru/Sao Paulo, Brazil\"] {allow-input: true}\n",
"%env region={region.split(\"/\")[0]}\n",
"#%env appame=\n",
"#%env username=\n",
"#%env password=\n",
"#%env database=\n",
"#%env passphrase=\n",
"\n",
"# automatic setup leave it -->\n",
"%cd obsidian-livesync/utils/flyio\n",
"!./deploy-server.sh | tee deploy-result.txt\n",
"\n",
"## Show result button\n",
"from IPython.display import HTML\n",
"last_line=\"\"\n",
"with open('deploy-result.txt', 'r') as f:\n",
" last_line = f.readlines()[-1]\n",
" last_line = str.strip(last_line)\n",
"\n",
"if last_line.startswith(\"obsidian://\"):\n",
" result = HTML(f\"Copy your setup-URI with this button! -&gt; <button onclick=\\\"navigator.clipboard.writeText('{last_line}')\\\">Copy setup uri</button><br>Importing passphrase is displayed one. <br>If you want to synchronise in live mode, please apply a preset after ensuring the imported configuration works.\")\n",
"else:\n",
" result = \"Failed to encrypt the setup URI\"\n",
"result"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "oeIzExnEKhFp"
},
"source": [
"If you see the `Copy setup URI` button, Congratulations! Your CouchDB is ready to use! Please click the button. And open this on Obsidian.\n",
"\n",
"And, you should keep the output to your secret memo.\n",
"\n"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "sdQrqOjERN3K"
},
"source": [
"\n",
"\n",
"---\n",
"\n",
"\n",
"If you want to delete this CouchDB instance, you can do it by executing next cell. \n",
"If your fly.toml has been gone, access https://fly.io/dashboard and check the existing app."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "7JMSkNvVIIfg"
},
"outputs": [],
"source": [
"!./delete-server.sh"
]
}
],
"metadata": {
"colab": {
"authorship_tag": "ABX9TyMexQ5pErH5LBG2tENtEVWf",
"include_colab_link": true,
"private_outputs": true,
"provenance": []
},
"kernelspec": {
"display_name": "Python 3",
"name": "python3"
},
"language_info": {
"name": "python"
}
},
"nbformat": 4,
"nbformat_minor": 0
}
-687
View File
@@ -1,687 +0,0 @@
import { writable } from 'svelte/store';
import { Notice, type PluginManifest, parseYaml } from "./deps";
import type { EntryDoc, LoadedEntry, InternalFileEntry, FilePathWithPrefix, FilePath, DocumentID, AnyEntry } from "./lib/src/types";
import { LOG_LEVEL } from "./lib/src/types";
import { ICXHeader, PERIODIC_PLUGIN_SWEEP, } from "./types";
import { Parallels, delay, getDocData } from "./lib/src/utils";
import { Logger } from "./lib/src/logger";
import { WrappedNotice } from "./lib/src/wrapper";
import { base64ToArrayBuffer, arrayBufferToBase64, readString, uint8ArrayToHexString } from "./lib/src/strbin";
import { runWithLock } from "./lib/src/lock";
import { LiveSyncCommands } from "./LiveSyncCommands";
import { stripAllPrefixes } from "./lib/src/path";
import { PeriodicProcessor, askYesNo, disposeMemoObject, memoIfNotExist, memoObject, retrieveMemoObject, scheduleTask } from "./utils";
import { Semaphore } from "./lib/src/semaphore";
import { PluginDialogModal } from "./dialogs";
import { JsonResolveModal } from "./JsonResolveModal";
function serialize<T>(obj: T): string {
return JSON.stringify(obj, null, 1);
}
function deserialize<T>(str: string, def: T) {
try {
return JSON.parse(str) as T;
} catch (ex) {
try {
return parseYaml(str);
} catch (ex) {
return def;
}
}
}
export const pluginList = writable([] as PluginDataExDisplay[]);
export const pluginIsEnumerating = writable(false);
const encoder = new TextEncoder();
const hashString = (async (key: string) => {
// const buff = writeString(key);
const buff = encoder.encode(key);
const digest = await crypto.subtle.digest('SHA-256', buff);
return uint8ArrayToHexString(new Uint8Array(digest));
})
export type PluginDataExFile = {
filename: string,
data?: string[],
mtime: number,
size: number,
version?: string,
displayName?: string,
}
export type PluginDataExDisplay = {
documentPath: FilePathWithPrefix,
category: string,
name: string,
term: string,
displayName?: string,
files: PluginDataExFile[],
version?: string,
mtime: number,
}
export type PluginDataEx = {
documentPath?: FilePathWithPrefix,
category: string,
name: string,
displayName?: string,
term: string,
files: PluginDataExFile[],
version?: string,
mtime: number,
};
export class ConfigSync extends LiveSyncCommands {
confirmPopup: WrappedNotice = null;
get kvDB() {
return this.plugin.kvDB;
}
ensureDirectoryEx(fullPath: string) {
return this.plugin.ensureDirectoryEx(fullPath);
}
pluginDialog: PluginDialogModal = null;
periodicPluginSweepProcessor = new PeriodicProcessor(this.plugin, async () => await this.scanAllConfigFiles(false));
pluginList: PluginDataExDisplay[] = [];
showPluginSyncModal() {
if (!this.settings.usePluginSync) {
return;
}
if (this.pluginDialog != null) {
this.pluginDialog.open();
} else {
this.pluginDialog = new PluginDialogModal(this.app, this.plugin);
this.pluginDialog.open();
}
}
hidePluginSyncModal() {
if (this.pluginDialog != null) {
this.pluginDialog.close();
this.pluginDialog = null;
}
}
onunload() {
this.hidePluginSyncModal();
this.periodicPluginSweepProcessor?.disable();
}
onload() {
this.plugin.addCommand({
id: "livesync-plugin-dialog-ex",
name: "Show customization sync dialog",
callback: () => {
this.showPluginSyncModal();
},
});
}
getFileCategory(filePath: string): "CONFIG" | "THEME" | "SNIPPET" | "PLUGIN_MAIN" | "PLUGIN_ETC" | "PLUGIN_DATA" | "" {
if (filePath.split("/").length == 2 && filePath.endsWith(".json")) return "CONFIG";
if (filePath.split("/").length == 4 && filePath.startsWith(`${this.app.vault.configDir}/themes/`)) return "THEME";
if (filePath.startsWith(`${this.app.vault.configDir}/snippets/`) && filePath.endsWith(".css")) return "SNIPPET";
if (filePath.startsWith(`${this.app.vault.configDir}/plugins/`)) {
if (filePath.endsWith("/styles.css") || filePath.endsWith("/manifest.json") || filePath.endsWith("/main.js")) {
return "PLUGIN_MAIN";
} else if (filePath.endsWith("/data.json")) {
return "PLUGIN_DATA";
} else {
//TODO: to be configurable.
// With algorithm which implemented at v0.19.0, is too heavy.
return "";
// return "PLUGIN_ETC";
}
// return "PLUGIN";
}
return "";
}
isTargetPath(filePath: string): boolean {
if (!filePath.startsWith(this.app.vault.configDir)) return false;
// Idea non-filter option?
return this.getFileCategory(filePath) != "";
}
async onInitializeDatabase(showNotice: boolean) {
if (this.settings.usePluginSync) {
try {
Logger("Scanning customizations...");
await this.scanAllConfigFiles(showNotice);
Logger("Scanning customizations : done");
} catch (ex) {
Logger("Scanning customizations : failed");
Logger(ex, LOG_LEVEL.VERBOSE);
}
}
}
async beforeReplicate(showNotice: boolean) {
if (this.settings.autoSweepPlugins && this.settings.usePluginSync) {
await this.scanAllConfigFiles(showNotice);
}
}
async onResume() {
if (this.plugin.suspended) {
return;
}
if (this.settings.autoSweepPlugins && this.settings.usePluginSync) {
await this.scanAllConfigFiles(false);
}
this.periodicPluginSweepProcessor.enable(this.settings.autoSweepPluginsPeriodic && !this.settings.watchInternalFileChanges ? (PERIODIC_PLUGIN_SWEEP * 1000) : 0);
}
async reloadPluginList(showMessage: boolean) {
this.pluginList = [];
pluginList.set(this.pluginList)
await this.updatePluginList(showMessage);
}
async updatePluginList(showMessage: boolean, updatedDocumentPath?: FilePathWithPrefix): Promise<void> {
const logLevel = showMessage ? LOG_LEVEL.NOTICE : LOG_LEVEL.INFO;
// pluginList.set([]);
if (!this.settings.usePluginSync) {
this.pluginList = [];
pluginList.set(this.pluginList)
return;
}
await runWithLock("update-plugin-list", false, async () => {
// if (updatedDocumentPath != "") pluginList.update(e => e.filter(ee => ee.documentPath != updatedDocumentPath));
// const work: Record<string, Record<string, Record<string, Record<string, PluginDataEntryEx>>>> = {};
const entries = [] as PluginDataExDisplay[]
const plugins = this.localDatabase.findEntries(ICXHeader + "", `${ICXHeader}\u{10ffff}`, { include_docs: true });
const semaphore = Semaphore(4);
const para = Parallels();
let count = 0;
pluginIsEnumerating.set(true);
let processed = false;
try {
for await (const plugin of plugins) {
const path = plugin.path || this.getPath(plugin);
if (updatedDocumentPath && updatedDocumentPath != path) {
continue;
}
processed = true;
const oldEntry = (this.pluginList.find(e => e.documentPath == path));
if (oldEntry && oldEntry.mtime == plugin.mtime) continue;
await para.wait(5);
para.add((async (v) => {
const release = await semaphore.acquire(1);
try {
count++;
if (count % 10 == 0) Logger(`Enumerating files... ${count}`, logLevel, "get-plugins");
Logger(`plugin-${path}`, LOG_LEVEL.VERBOSE);
const wx = await this.localDatabase.getDBEntry(path, null, false, false);
if (wx) {
const data = deserialize(getDocData(wx.data), {}) as PluginDataEx;
const xFiles = [] as PluginDataExFile[];
for (const file of data.files) {
const work = { ...file };
const tempStr = getDocData(work.data);
work.data = [await hashString(tempStr)];
xFiles.push(work);
}
entries.push({
...data,
documentPath: this.getPath(wx),
files: xFiles
});
}
} catch (ex) {
//TODO
Logger(`Something happened at enumerating customization :${v.path}`, LOG_LEVEL.NOTICE);
console.warn(ex);
} finally {
release();
}
}
)(plugin));
}
await para.all();
let newList = [...this.pluginList];
for (const item of entries) {
newList = newList.filter(x => x.documentPath != item.documentPath);
newList.push(item)
}
if (updatedDocumentPath != "" && !processed) newList = newList.filter(e => e.documentPath != updatedDocumentPath);
this.pluginList = newList;
pluginList.set(newList);
Logger(`All files enumerated`, logLevel, "get-plugins");
} finally {
pluginIsEnumerating.set(false);
}
});
// return entries;
}
async compareUsingDisplayData(dataA: PluginDataExDisplay, dataB: PluginDataExDisplay) {
const docA = await this.localDatabase.getDBEntry(dataA.documentPath);
const docB = await this.localDatabase.getDBEntry(dataB.documentPath);
if (docA && docB) {
const pluginDataA = deserialize(getDocData(docA.data), {}) as PluginDataEx;
pluginDataA.documentPath = dataA.documentPath;
const pluginDataB = deserialize(getDocData(docB.data), {}) as PluginDataEx;
pluginDataB.documentPath = dataB.documentPath;
// Use outer structure to wrap each data.
return await this.showJSONMergeDialogAndMerge(docA, docB, pluginDataA, pluginDataB);
}
return false;
}
showJSONMergeDialogAndMerge(docA: LoadedEntry, docB: LoadedEntry, pluginDataA: PluginDataEx, pluginDataB: PluginDataEx): Promise<boolean> {
const fileA = { ...pluginDataA.files[0], ctime: pluginDataA.files[0].mtime, _id: `${pluginDataA.documentPath}` as DocumentID };
const fileB = pluginDataB.files[0];
const docAx = { ...docA, ...fileA } as LoadedEntry, docBx = { ...docB, ...fileB } as LoadedEntry
return runWithLock("config:merge-data", false, () => new Promise((res) => {
Logger("Opening data-merging dialog", LOG_LEVEL.VERBOSE);
// const docs = [docA, docB];
const path = stripAllPrefixes(docAx.path.split("/").slice(-1).join("/") as FilePath);
const modal = new JsonResolveModal(this.app, path, [docAx, docBx], async (keep, result) => {
if (result == null) return res(false);
try {
res(await this.applyData(pluginDataA, result));
} catch (ex) {
Logger("Could not apply merged file");
Logger(ex, LOG_LEVEL.VERBOSE);
res(false);
}
}, "📡", "🛰️", "B");
modal.open();
}));
}
async applyData(data: PluginDataEx, content?: string): Promise<boolean> {
Logger(`Applying ${data.displayName || data.name}..`);
const baseDir = this.app.vault.configDir;
try {
if (!data.documentPath) throw "InternalError: Document path not exist";
const dx = await this.localDatabase.getDBEntry(data.documentPath);
if (dx == false) {
throw "Not found on database"
}
const loadedData = deserialize(getDocData(dx.data), {}) as PluginDataEx;
for (const f of loadedData.files) {
Logger(`Applying ${f.filename} of ${data.displayName || data.name}..`);
try {
// console.dir(f);
const path = `${baseDir}/${f.filename}`;
await this.ensureDirectoryEx(path);
if (!content) {
const dt = base64ToArrayBuffer(f.data);
await this.app.vault.adapter.writeBinary(path, dt);
} else {
await this.app.vault.adapter.write(path, content);
}
Logger(`Applying ${f.filename} of ${data.displayName || data.name}.. Done`);
} catch (ex) {
Logger(`Applying ${f.filename} of ${data.displayName || data.name}.. Failed`);
Logger(ex, LOG_LEVEL.VERBOSE);
}
}
const uPath = `${baseDir}/${loadedData.files[0].filename}` as FilePath;
await this.storeCustomizationFiles(uPath);
await this.updatePluginList(true, uPath);
await delay(100);
Logger(`Config ${data.displayName || data.name} has been applied`, LOG_LEVEL.NOTICE);
if (data.category == "PLUGIN_DATA" || data.category == "PLUGIN_MAIN") {
//@ts-ignore
const manifests = Object.values(this.app.plugins.manifests) as any as PluginManifest[];
//@ts-ignore
const enabledPlugins = this.app.plugins.enabledPlugins as Set<string>;
const pluginManifest = manifests.find((manifest) => enabledPlugins.has(manifest.id) && manifest.dir == `${baseDir}/plugins/${data.name}`);
if (pluginManifest) {
Logger(`Unloading plugin: ${pluginManifest.name}`, LOG_LEVEL.NOTICE, "plugin-reload-" + pluginManifest.id);
// @ts-ignore
await this.app.plugins.unloadPlugin(pluginManifest.id);
// @ts-ignore
await this.app.plugins.loadPlugin(pluginManifest.id);
Logger(`Plugin reloaded: ${pluginManifest.name}`, LOG_LEVEL.NOTICE, "plugin-reload-" + pluginManifest.id);
}
} else if (data.category == "CONFIG") {
scheduleTask("configReload", 250, async () => {
if (await askYesNo(this.app, "Do you want to restart and reload Obsidian now?") == "yes") {
// @ts-ignore
this.app.commands.executeCommandById("app:reload")
}
})
}
return true;
} catch (ex) {
Logger(`Applying ${data.displayName || data.name}.. Failed`);
Logger(ex, LOG_LEVEL.VERBOSE);
return false;
}
}
async deleteData(data: PluginDataEx): Promise<boolean> {
try {
if (data.documentPath) {
await this.deleteConfigOnDatabase(data.documentPath);
await this.updatePluginList(false, data.documentPath);
Logger(`Delete: ${data.documentPath}`, LOG_LEVEL.NOTICE);
}
return true;
} catch (ex) {
Logger(`Failed to delete: ${data.documentPath}`, LOG_LEVEL.NOTICE);
return false;
}
}
async parseReplicationResultItem(docs: PouchDB.Core.ExistingDocument<EntryDoc>) {
if (docs._id.startsWith(ICXHeader)) {
if (this.plugin.settings.usePluginSync) {
await this.updatePluginList(false, (docs as AnyEntry).path ? (docs as AnyEntry).path : this.getPath((docs as AnyEntry)));
}
if (this.plugin.settings.usePluginSync && this.plugin.settings.notifyPluginOrSettingUpdated) {
if (!this.pluginDialog || (this.pluginDialog && !this.pluginDialog.isOpened())) {
const fragment = createFragment((doc) => {
doc.createEl("span", null, (a) => {
a.appendText(`Some configuration has been arrived, Press `);
a.appendChild(a.createEl("a", null, (anchor) => {
anchor.text = "HERE";
anchor.addEventListener("click", () => {
this.showPluginSyncModal();
});
}));
a.appendText(` to open the config sync dialog , or press elsewhere to dismiss this message.`);
});
});
const updatedPluginKey = "popupUpdated-plugins";
scheduleTask(updatedPluginKey, 1000, async () => {
const popup = await memoIfNotExist(updatedPluginKey, () => new Notice(fragment, 0));
//@ts-ignore
const isShown = popup?.noticeEl?.isShown();
if (!isShown) {
memoObject(updatedPluginKey, new Notice(fragment, 0));
}
scheduleTask(updatedPluginKey + "-close", 20000, () => {
const popup = retrieveMemoObject<Notice>(updatedPluginKey);
if (!popup)
return;
//@ts-ignore
if (popup?.noticeEl?.isShown()) {
popup.hide();
}
disposeMemoObject(updatedPluginKey);
});
});
}
}
return true;
}
return false;
}
async realizeSettingSyncMode(): Promise<void> {
this.periodicPluginSweepProcessor?.disable();
if (this.plugin.suspended)
return;
if (!this.settings.usePluginSync) {
return;
}
if (this.settings.autoSweepPlugins) {
await this.scanAllConfigFiles(false);
}
this.periodicPluginSweepProcessor.enable(this.settings.autoSweepPluginsPeriodic && !this.settings.watchInternalFileChanges ? (PERIODIC_PLUGIN_SWEEP * 1000) : 0);
return;
}
recentProcessedInternalFiles = [] as string[];
async makeEntryFromFile(path: FilePath): Promise<false | PluginDataExFile> {
const stat = await this.app.vault.adapter.stat(path);
let version: string | undefined;
let displayName: string | undefined;
if (!stat) {
return false;
}
const contentBin = await this.app.vault.adapter.readBinary(path);
let content: string[];
try {
content = await arrayBufferToBase64(contentBin);
if (path.toLowerCase().endsWith("/manifest.json")) {
const v = readString(new Uint8Array(contentBin));
try {
const json = JSON.parse(v);
if ("version" in json) {
version = `${json.version}`;
}
if ("name" in json) {
displayName = `${json.name}`;
}
} catch (ex) {
Logger(`Configuration sync data: ${path} looks like manifest, but could not read the version`, LOG_LEVEL.INFO);
}
}
} catch (ex) {
Logger(`The file ${path} could not be encoded`);
Logger(ex, LOG_LEVEL.VERBOSE);
return false;
}
const mtime = stat.mtime;
return {
filename: path.substring(this.app.vault.configDir.length + 1),
data: content,
mtime,
size: stat.size,
version,
displayName: displayName,
}
}
filenameToUnifiedKey(path: string, termOverRide?: string) {
const term = termOverRide || this.plugin.deviceAndVaultName;
const category = this.getFileCategory(path);
const name = (category == "CONFIG" || category == "SNIPPET") ?
(path.split("/").slice(-1)[0]) :
(category == "PLUGIN_ETC" ?
path.split("/").slice(-2).join("/") :
path.split("/").slice(-2)[0]);
return `${ICXHeader}${term}/${category}/${name}.md` as FilePathWithPrefix
}
async storeCustomizationFiles(path: FilePath, termOverRide?: string) {
const term = termOverRide || this.plugin.deviceAndVaultName;
const vf = this.filenameToUnifiedKey(path, term);
return await runWithLock(`plugin-${vf}`, false, async () => {
const category = this.getFileCategory(path);
let mtime = 0;
let fileTargets = [] as FilePath[];
// let savePath = "";
const name = (category == "CONFIG" || category == "SNIPPET") ?
(path.split("/").reverse()[0]) :
(path.split("/").reverse()[1]);
const parentPath = path.split("/").slice(0, -1).join("/");
const prefixedFileName = this.filenameToUnifiedKey(path, term);
const id = await this.path2id(prefixedFileName);
const dt: PluginDataEx = {
category: category,
files: [],
name: name,
mtime: 0,
term: term
}
// let scheduleKey = "";
if (category == "CONFIG" || category == "SNIPPET" || category == "PLUGIN_ETC" || category == "PLUGIN_DATA") {
fileTargets = [path];
if (category == "PLUGIN_ETC") {
dt.displayName = path.split("/").slice(-1).join("/");
}
} else if (category == "PLUGIN_MAIN") {
fileTargets = ["manifest.json", "main.js", "styles.css"].map(e => `${parentPath}/${e}` as FilePath);
} else if (category == "THEME") {
fileTargets = ["manifest.json", "theme.css"].map(e => `${parentPath}/${e}` as FilePath);
}
for (const target of fileTargets) {
const data = await this.makeEntryFromFile(target);
if (data == false) {
// Logger(`Config: skipped: ${target} `, LOG_LEVEL.VERBOSE);
continue;
}
if (data.version) {
dt.version = data.version;
}
if (data.displayName) {
dt.displayName = data.displayName;
}
// Use average for total modified time.
mtime = mtime == 0 ? data.mtime : ((data.mtime + mtime) / 2);
dt.files.push(data);
}
dt.mtime = mtime;
// Logger(`Configuration saving: ${prefixedFileName}`);
if (dt.files.length == 0) {
Logger(`Nothing left: deleting.. ${path}`);
await this.deleteConfigOnDatabase(prefixedFileName);
await this.updatePluginList(false, prefixedFileName);
return
}
const content = serialize(dt);
try {
const old = await this.localDatabase.getDBEntryMeta(prefixedFileName, null, false);
let saveData: LoadedEntry;
if (old === false) {
saveData = {
_id: id,
path: prefixedFileName,
data: content,
mtime,
ctime: mtime,
datatype: "newnote",
size: content.length,
children: [],
deleted: false,
type: "newnote",
};
} else {
if (old.mtime == mtime) {
// Logger(`STORAGE --> DB:${file.path}: (hidden) Not changed`, LOG_LEVEL.VERBOSE);
return true;
}
saveData =
{
...old,
data: content,
mtime,
size: content.length,
datatype: "newnote",
children: [],
deleted: false,
type: "newnote",
};
}
const ret = await this.localDatabase.putDBEntry(saveData);
await this.updatePluginList(false, saveData.path);
Logger(`STORAGE --> DB:${prefixedFileName}: (config) Done`);
return ret;
} catch (ex) {
Logger(`STORAGE --> DB:${prefixedFileName}: (config) Failed`);
Logger(ex, LOG_LEVEL.VERBOSE);
return false;
}
})
}
async watchVaultRawEventsAsync(path: FilePath) {
if (!this.settings.usePluginSync) return false;
if (!this.isTargetPath(path)) return false;
const stat = await this.app.vault.adapter.stat(path);
// Make sure that target is a file.
if (stat && stat.type != "file")
return false;
const storageMTime = ~~((stat && stat.mtime || 0) / 1000);
const key = `${path}-${storageMTime}`;
if (this.recentProcessedInternalFiles.contains(key)) {
// If recently processed, it may caused by self.
return true;
}
this.recentProcessedInternalFiles = [key, ...this.recentProcessedInternalFiles].slice(0, 100);
this.storeCustomizationFiles(path).then(() => {/* Fire and forget */ });
}
async scanAllConfigFiles(showMessage: boolean) {
const logLevel = showMessage ? LOG_LEVEL.NOTICE : LOG_LEVEL.INFO;
Logger("Scanning customizing files.", logLevel, "scan-all-config");
const term = this.plugin.deviceAndVaultName;
if (term == "") {
Logger("We have to configure the device name", LOG_LEVEL.NOTICE);
return;
}
const filesAll = await this.scanInternalFiles();
const files = filesAll.filter(e => this.isTargetPath(e)).map(e => ({ key: this.filenameToUnifiedKey(e), file: e }));
const virtualPathsOfLocalFiles = [...new Set(files.map(e => e.key))];
const filesOnDB = ((await this.localDatabase.allDocsRaw({ startkey: ICXHeader + "", endkey: `${ICXHeader}\u{10ffff}`, include_docs: true })).rows.map(e => e.doc) as InternalFileEntry[]).filter(e => !e.deleted);
let deleteCandidate = filesOnDB.map(e => this.getPath(e)).filter(e => e.startsWith(`${ICXHeader}${term}/`));
for (const vp of virtualPathsOfLocalFiles) {
const p = files.find(e => e.key == vp).file;
await this.storeCustomizationFiles(p);
deleteCandidate = deleteCandidate.filter(e => e != vp);
}
for (const vp of deleteCandidate) {
await this.deleteConfigOnDatabase(vp);
}
this.updatePluginList(false).then(/* fire and forget */);
}
async deleteConfigOnDatabase(prefixedFileName: FilePathWithPrefix, forceWrite = false) {
// const id = await this.path2id(prefixedFileName);
const mtime = new Date().getTime();
await runWithLock("file-x-" + prefixedFileName, false, async () => {
try {
const old = await this.localDatabase.getDBEntryMeta(prefixedFileName, null, false) as InternalFileEntry | false;
let saveData: InternalFileEntry;
if (old === false) {
Logger(`STORAGE -x> DB:${prefixedFileName}: (config) already deleted (Not found on database)`);
} else {
if (old.deleted) {
Logger(`STORAGE -x> DB:${prefixedFileName}: (config) already deleted`);
return;
}
saveData =
{
...old,
mtime,
size: 0,
children: [],
deleted: true,
type: "newnote",
};
}
await this.localDatabase.putRaw(saveData);
await this.updatePluginList(false, prefixedFileName);
Logger(`STORAGE -x> DB:${prefixedFileName}: (config) Done`);
} catch (ex) {
Logger(`STORAGE -x> DB:${prefixedFileName}: (config) Failed`);
Logger(ex, LOG_LEVEL.VERBOSE);
return false;
}
});
}
async scanInternalFiles(): Promise<FilePath[]> {
const filenames = (await this.getFiles(this.app.vault.configDir, 2)).filter(e => e.startsWith(".")).filter(e => !e.startsWith(".trash"));
return filenames as FilePath[];
}
async getFiles(
path: string,
lastDepth: number
) {
if (lastDepth == -1) return [];
const w = await this.app.vault.adapter.list(path);
let files = [
...w.files
];
for (const v of w.folders) {
files = files.concat(await this.getFiles(v, lastDepth - 1));
}
return files;
}
}
-721
View File
@@ -1,721 +0,0 @@
import { Notice, normalizePath, PluginManifest } from "./deps";
import { EntryDoc, LoadedEntry, LOG_LEVEL, InternalFileEntry, FilePathWithPrefix, FilePath } from "./lib/src/types";
import { InternalFileInfo, ICHeader, ICHeaderEnd } from "./types";
import { Parallels, delay, isDocContentSame } from "./lib/src/utils";
import { Logger } from "./lib/src/logger";
import { PouchDB } from "./lib/src/pouchdb-browser.js";
import { disposeMemoObject, memoIfNotExist, memoObject, retrieveMemoObject, scheduleTask, isInternalMetadata, PeriodicProcessor } from "./utils";
import { WrappedNotice } from "./lib/src/wrapper";
import { base64ToArrayBuffer, arrayBufferToBase64 } from "./lib/src/strbin";
import { runWithLock } from "./lib/src/lock";
import { JsonResolveModal } from "./JsonResolveModal";
import { LiveSyncCommands } from "./LiveSyncCommands";
import { addPrefix, stripAllPrefixes } from "./lib/src/path";
export class HiddenFileSync extends LiveSyncCommands {
periodicInternalFileScanProcessor: PeriodicProcessor = new PeriodicProcessor(this.plugin, async () => this.settings.syncInternalFiles && this.localDatabase.isReady && await this.syncInternalFilesAndDatabase("push", false));
confirmPopup: WrappedNotice = null;
get kvDB() {
return this.plugin.kvDB;
}
ensureDirectoryEx(fullPath: string) {
return this.plugin.ensureDirectoryEx(fullPath);
}
getConflictedDoc(path: FilePathWithPrefix, rev: string) {
return this.plugin.getConflictedDoc(path, rev);
}
onunload() {
this.periodicInternalFileScanProcessor?.disable();
}
onload() {
this.plugin.addCommand({
id: "livesync-scaninternal",
name: "Sync hidden files",
callback: () => {
this.syncInternalFilesAndDatabase("safe", true);
},
});
}
async onInitializeDatabase(showNotice: boolean) {
if (this.settings.syncInternalFiles) {
try {
Logger("Synchronizing hidden files...");
await this.syncInternalFilesAndDatabase("push", showNotice);
Logger("Synchronizing hidden files done");
} catch (ex) {
Logger("Synchronizing hidden files failed");
Logger(ex, LOG_LEVEL.VERBOSE);
}
}
}
async beforeReplicate(showNotice: boolean) {
if (this.localDatabase.isReady && this.settings.syncInternalFiles && this.settings.syncInternalFilesBeforeReplication && !this.settings.watchInternalFileChanges) {
await this.syncInternalFilesAndDatabase("push", showNotice);
}
}
async onResume() {
this.periodicInternalFileScanProcessor?.disable();
if (this.plugin.suspended)
return;
if (this.settings.syncInternalFiles) {
await this.syncInternalFilesAndDatabase("safe", false);
}
this.periodicInternalFileScanProcessor.enable(this.settings.syncInternalFiles && this.settings.syncInternalFilesInterval ? (this.settings.syncInternalFilesInterval * 1000) : 0);
}
parseReplicationResultItem(docs: PouchDB.Core.ExistingDocument<EntryDoc>) {
return false;
}
realizeSettingSyncMode(): Promise<void> {
this.periodicInternalFileScanProcessor?.disable();
if (this.plugin.suspended)
return;
if (!this.plugin.isReady)
return;
this.periodicInternalFileScanProcessor.enable(this.settings.syncInternalFiles && this.settings.syncInternalFilesInterval ? (this.settings.syncInternalFilesInterval * 1000) : 0);
return;
}
procInternalFiles: string[] = [];
async execInternalFile() {
await runWithLock("execInternal", false, async () => {
const w = [...this.procInternalFiles];
this.procInternalFiles = [];
Logger(`Applying hidden ${w.length} files change...`);
await this.syncInternalFilesAndDatabase("pull", false, false, w);
Logger(`Applying hidden ${w.length} files changed`);
});
}
procInternalFile(filename: string) {
this.procInternalFiles.push(filename);
scheduleTask("procInternal", 500, async () => {
await this.execInternalFile();
});
}
recentProcessedInternalFiles = [] as string[];
async watchVaultRawEventsAsync(path: FilePath) {
if (!this.settings.syncInternalFiles) return;
const stat = await this.app.vault.adapter.stat(path);
// sometimes folder is coming.
if (stat && stat.type != "file")
return;
const storageMTime = ~~((stat && stat.mtime || 0) / 1000);
const key = `${path}-${storageMTime}`;
if (this.recentProcessedInternalFiles.contains(key)) {
//If recently processed, it may caused by self.
return;
}
this.recentProcessedInternalFiles = [key, ...this.recentProcessedInternalFiles].slice(0, 100);
// const id = await this.path2id(path, ICHeader);
const prefixedFileName = addPrefix(path, ICHeader);
const filesOnDB = await this.localDatabase.getDBEntryMeta(prefixedFileName);
const dbMTime = ~~((filesOnDB && filesOnDB.mtime || 0) / 1000);
// Skip unchanged file.
if (dbMTime == storageMTime) {
// Logger(`STORAGE --> DB:${path}: (hidden) Nothing changed`);
return;
}
// Do not compare timestamp. Always local data should be preferred except this plugin wrote one.
if (storageMTime == 0) {
await this.deleteInternalFileOnDatabase(path);
} else {
await this.storeInternalFileToDatabase({ path: path, ...stat });
}
}
async resolveConflictOnInternalFiles() {
// Scan all conflicted internal files
const conflicted = this.localDatabase.findEntries(ICHeader, ICHeaderEnd, { conflicts: true });
for await (const doc of conflicted) {
if (!("_conflicts" in doc))
continue;
if (isInternalMetadata(doc._id)) {
await this.resolveConflictOnInternalFile(doc.path);
}
}
}
async resolveConflictOnInternalFile(path: FilePathWithPrefix): Promise<boolean> {
try {
// Retrieve data
const id = await this.path2id(path, ICHeader);
const doc = await this.localDatabase.getRaw(id, { conflicts: true });
// If there is no conflict, return with false.
if (!("_conflicts" in doc))
return false;
if (doc._conflicts.length == 0)
return false;
Logger(`Hidden file conflicted:${path}`);
const conflicts = doc._conflicts.sort((a, b) => Number(a.split("-")[0]) - Number(b.split("-")[0]));
const revA = doc._rev;
const revB = conflicts[0];
if (path.endsWith(".json")) {
const conflictedRev = conflicts[0];
const conflictedRevNo = Number(conflictedRev.split("-")[0]);
//Search
const revFrom = (await this.localDatabase.getRaw<EntryDoc>(id, { revs_info: true }));
const commonBase = revFrom._revs_info.filter(e => e.status == "available" && Number(e.rev.split("-")[0]) < conflictedRevNo).first()?.rev ?? "";
const result = await this.plugin.mergeObject(path, commonBase, doc._rev, conflictedRev);
if (result) {
Logger(`Object merge:${path}`, LOG_LEVEL.INFO);
const filename = stripAllPrefixes(path);
const isExists = await this.app.vault.adapter.exists(filename);
if (!isExists) {
await this.ensureDirectoryEx(filename);
}
await this.app.vault.adapter.write(filename, result);
const stat = await this.app.vault.adapter.stat(filename);
await this.storeInternalFileToDatabase({ path: filename, ...stat });
await this.extractInternalFileFromDatabase(filename);
await this.localDatabase.removeRaw(id, revB);
return this.resolveConflictOnInternalFile(path);
} else {
Logger(`Object merge is not applicable.`, LOG_LEVEL.VERBOSE);
}
const docAMerge = await this.localDatabase.getDBEntry(path, { rev: revA });
const docBMerge = await this.localDatabase.getDBEntry(path, { rev: revB });
if (docAMerge != false && docBMerge != false) {
if (await this.showJSONMergeDialogAndMerge(docAMerge, docBMerge)) {
await delay(200);
// Again for other conflicted revisions.
return this.resolveConflictOnInternalFile(path);
}
return false;
}
}
const revBDoc = await this.localDatabase.getRaw(id, { rev: revB });
// determine which revision should been deleted.
// simply check modified time
const mtimeA = ("mtime" in doc && doc.mtime) || 0;
const mtimeB = ("mtime" in revBDoc && revBDoc.mtime) || 0;
// Logger(`Revisions:${new Date(mtimeA).toLocaleString} and ${new Date(mtimeB).toLocaleString}`);
// console.log(`mtime:${mtimeA} - ${mtimeB}`);
const delRev = mtimeA < mtimeB ? revA : revB;
// delete older one.
await this.localDatabase.removeRaw(id, delRev);
Logger(`Older one has been deleted:${path}`);
// check the file again
return this.resolveConflictOnInternalFile(path);
} catch (ex) {
Logger(`Failed to resolve conflict (Hidden): ${path}`);
Logger(ex, LOG_LEVEL.VERBOSE);
return false;
}
}
//TODO: Tidy up. Even though it is experimental feature, So dirty...
async syncInternalFilesAndDatabase(direction: "push" | "pull" | "safe" | "pullForce" | "pushForce", showMessage: boolean, files: InternalFileInfo[] | false = false, targetFiles: string[] | false = false) {
await this.resolveConflictOnInternalFiles();
const logLevel = showMessage ? LOG_LEVEL.NOTICE : LOG_LEVEL.INFO;
Logger("Scanning hidden files.", logLevel, "sync_internal");
const ignorePatterns = this.settings.syncInternalFilesIgnorePatterns
.replace(/\n| /g, "")
.split(",").filter(e => e).map(e => new RegExp(e, "i"));
if (!files)
files = await this.scanInternalFiles();
const filesOnDB = ((await this.localDatabase.allDocsRaw({ startkey: ICHeader, endkey: ICHeaderEnd, include_docs: true })).rows.map(e => e.doc) as InternalFileEntry[]).filter(e => !e.deleted);
const allFileNamesSrc = [...new Set([...files.map(e => normalizePath(e.path)), ...filesOnDB.map(e => stripAllPrefixes(this.getPath(e)))])];
const allFileNames = allFileNamesSrc.filter(filename => !targetFiles || (targetFiles && targetFiles.indexOf(filename) !== -1));
function compareMTime(a: number, b: number) {
const wa = ~~(a / 1000);
const wb = ~~(b / 1000);
const diff = wa - wb;
return diff;
}
const fileCount = allFileNames.length;
let processed = 0;
let filesChanged = 0;
// count updated files up as like this below:
// .obsidian: 2
// .obsidian/workspace: 1
// .obsidian/plugins: 1
// .obsidian/plugins/recent-files-obsidian: 1
// .obsidian/plugins/recent-files-obsidian/data.json: 1
const updatedFolders: { [key: string]: number; } = {};
const countUpdatedFolder = (path: string) => {
const pieces = path.split("/");
let c = pieces.shift();
let pathPieces = "";
filesChanged++;
while (c) {
pathPieces += (pathPieces != "" ? "/" : "") + c;
pathPieces = normalizePath(pathPieces);
if (!(pathPieces in updatedFolders)) {
updatedFolders[pathPieces] = 0;
}
updatedFolders[pathPieces]++;
c = pieces.shift();
}
};
// Cache update time information for files which have already been processed (mainly for files that were skipped due to the same content)
let caches: { [key: string]: { storageMtime: number; docMtime: number; }; } = {};
caches = await this.kvDB.get<{ [key: string]: { storageMtime: number; docMtime: number; }; }>("diff-caches-internal") || {};
const filesMap = files.reduce((acc, cur) => {
acc[cur.path] = cur;
return acc;
}, {} as { [key: string]: InternalFileInfo; });
const filesOnDBMap = filesOnDB.reduce((acc, cur) => {
acc[stripAllPrefixes(this.getPath(cur))] = cur;
return acc;
}, {} as { [key: string]: InternalFileEntry; });
const para = Parallels();
for (const filename of allFileNames) {
processed++;
if (processed % 100 == 0) {
Logger(`Hidden file: ${processed}/${fileCount}`, logLevel, "sync_internal");
}
if (!filename) continue;
if (ignorePatterns.some(e => filename.match(e)))
continue;
const fileOnStorage = filename in filesMap ? filesMap[filename] : undefined;
const fileOnDatabase = filename in filesOnDBMap ? filesOnDBMap[filename] : undefined;
const cache = filename in caches ? caches[filename] : { storageMtime: 0, docMtime: 0 };
await para.wait(5);
const proc = (async (xFileOnStorage: InternalFileInfo, xFileOnDatabase: InternalFileEntry) => {
if (xFileOnStorage && xFileOnDatabase) {
// Both => Synchronize
if ((direction != "pullForce" && direction != "pushForce") && xFileOnDatabase.mtime == cache.docMtime && xFileOnStorage.mtime == cache.storageMtime) {
return;
}
const nw = compareMTime(xFileOnStorage.mtime, xFileOnDatabase.mtime);
if (nw > 0 || direction == "pushForce") {
await this.storeInternalFileToDatabase(xFileOnStorage);
}
if (nw < 0 || direction == "pullForce") {
// skip if not extraction performed.
if (!await this.extractInternalFileFromDatabase(filename))
return;
}
// If process successfully updated or file contents are same, update cache.
cache.docMtime = xFileOnDatabase.mtime;
cache.storageMtime = xFileOnStorage.mtime;
caches[filename] = cache;
countUpdatedFolder(filename);
} else if (!xFileOnStorage && xFileOnDatabase) {
if (direction == "push" || direction == "pushForce") {
if (xFileOnDatabase.deleted)
return;
await this.deleteInternalFileOnDatabase(filename, false);
} else if (direction == "pull" || direction == "pullForce") {
if (await this.extractInternalFileFromDatabase(filename)) {
countUpdatedFolder(filename);
}
} else if (direction == "safe") {
if (xFileOnDatabase.deleted)
return;
if (await this.extractInternalFileFromDatabase(filename)) {
countUpdatedFolder(filename);
}
}
} else if (xFileOnStorage && !xFileOnDatabase) {
await this.storeInternalFileToDatabase(xFileOnStorage);
} else {
throw new Error("Invalid state on hidden file sync");
// Something corrupted?
}
});
para.add(proc(fileOnStorage, fileOnDatabase))
}
await para.all();
await this.kvDB.set("diff-caches-internal", caches);
// When files has been retrieved from the database. they must be reloaded.
if ((direction == "pull" || direction == "pullForce") && filesChanged != 0) {
const configDir = normalizePath(this.app.vault.configDir);
// Show notification to restart obsidian when something has been changed in configDir.
if (configDir in updatedFolders) {
// Numbers of updated files that is below of configDir.
let updatedCount = updatedFolders[configDir];
try {
//@ts-ignore
const manifests = Object.values(this.app.plugins.manifests) as any as PluginManifest[];
//@ts-ignore
const enabledPlugins = this.app.plugins.enabledPlugins as Set<string>;
const enabledPluginManifests = manifests.filter(e => enabledPlugins.has(e.id));
for (const manifest of enabledPluginManifests) {
if (manifest.dir in updatedFolders) {
// If notified about plug-ins, reloading Obsidian may not be necessary.
updatedCount -= updatedFolders[manifest.dir];
const updatePluginId = manifest.id;
const updatePluginName = manifest.name;
const fragment = createFragment((doc) => {
doc.createEl("span", null, (a) => {
a.appendText(`Files in ${updatePluginName} has been updated, Press `);
a.appendChild(a.createEl("a", null, (anchor) => {
anchor.text = "HERE";
anchor.addEventListener("click", async () => {
Logger(`Unloading plugin: ${updatePluginName}`, LOG_LEVEL.NOTICE, "plugin-reload-" + updatePluginId);
// @ts-ignore
await this.app.plugins.unloadPlugin(updatePluginId);
// @ts-ignore
await this.app.plugins.loadPlugin(updatePluginId);
Logger(`Plugin reloaded: ${updatePluginName}`, LOG_LEVEL.NOTICE, "plugin-reload-" + updatePluginId);
});
}));
a.appendText(` to reload ${updatePluginName}, or press elsewhere to dismiss this message.`);
});
});
const updatedPluginKey = "popupUpdated-" + updatePluginId;
scheduleTask(updatedPluginKey, 1000, async () => {
const popup = await memoIfNotExist(updatedPluginKey, () => new Notice(fragment, 0));
//@ts-ignore
const isShown = popup?.noticeEl?.isShown();
if (!isShown) {
memoObject(updatedPluginKey, new Notice(fragment, 0));
}
scheduleTask(updatedPluginKey + "-close", 20000, () => {
const popup = retrieveMemoObject<Notice>(updatedPluginKey);
if (!popup)
return;
//@ts-ignore
if (popup?.noticeEl?.isShown()) {
popup.hide();
}
disposeMemoObject(updatedPluginKey);
});
});
}
}
} catch (ex) {
Logger("Error on checking plugin status.");
Logger(ex, LOG_LEVEL.VERBOSE);
}
// If something changes left, notify for reloading Obsidian.
if (updatedCount != 0) {
const fragment = createFragment((doc) => {
doc.createEl("span", null, (a) => {
a.appendText(`Hidden files have been synchronized, Press `);
a.appendChild(a.createEl("a", null, (anchor) => {
anchor.text = "HERE";
anchor.addEventListener("click", () => {
// @ts-ignore
this.app.commands.executeCommandById("app:reload");
});
}));
a.appendText(` to reload obsidian, or press elsewhere to dismiss this message.`);
});
});
scheduleTask("popupUpdated-" + configDir, 1000, () => {
//@ts-ignore
const isShown = this.confirmPopup?.noticeEl?.isShown();
if (!isShown) {
this.confirmPopup = new Notice(fragment, 0);
}
scheduleTask("popupClose" + configDir, 20000, () => {
this.confirmPopup?.hide();
this.confirmPopup = null;
});
});
}
}
}
Logger(`Hidden files scanned: ${filesChanged} files had been modified`, logLevel, "sync_internal");
}
async storeInternalFileToDatabase(file: InternalFileInfo, forceWrite = false) {
const id = await this.path2id(file.path, ICHeader);
const prefixedFileName = addPrefix(file.path, ICHeader);
const contentBin = await this.app.vault.adapter.readBinary(file.path);
let content: string[];
try {
content = await arrayBufferToBase64(contentBin);
} catch (ex) {
Logger(`The file ${file.path} could not be encoded`);
Logger(ex, LOG_LEVEL.VERBOSE);
return false;
}
const mtime = file.mtime;
return await runWithLock("file-" + prefixedFileName, false, async () => {
try {
const old = await this.localDatabase.getDBEntry(prefixedFileName, null, false, false);
let saveData: LoadedEntry;
if (old === false) {
saveData = {
_id: id,
path: prefixedFileName,
data: content,
mtime,
ctime: mtime,
datatype: "newnote",
size: file.size,
children: [],
deleted: false,
type: "newnote",
};
} else {
if (isDocContentSame(old.data, content) && !forceWrite) {
// Logger(`STORAGE --> DB:${file.path}: (hidden) Not changed`, LOG_LEVEL.VERBOSE);
return;
}
saveData =
{
...old,
data: content,
mtime,
size: file.size,
datatype: "newnote",
children: [],
deleted: false,
type: "newnote",
};
}
const ret = await this.localDatabase.putDBEntry(saveData, true);
Logger(`STORAGE --> DB:${file.path}: (hidden) Done`);
return ret;
} catch (ex) {
Logger(`STORAGE --> DB:${file.path}: (hidden) Failed`);
Logger(ex, LOG_LEVEL.VERBOSE);
return false;
}
});
}
async deleteInternalFileOnDatabase(filename: FilePath, forceWrite = false) {
const id = await this.path2id(filename, ICHeader);
const prefixedFileName = addPrefix(filename, ICHeader);
const mtime = new Date().getTime();
await runWithLock("file-" + prefixedFileName, false, async () => {
try {
const old = await this.localDatabase.getDBEntryMeta(prefixedFileName, null, true) as InternalFileEntry | false;
let saveData: InternalFileEntry;
if (old === false) {
saveData = {
_id: id,
path: prefixedFileName,
mtime,
ctime: mtime,
size: 0,
children: [],
deleted: true,
type: "newnote",
};
} else {
if (old.deleted) {
Logger(`STORAGE -x> DB:${filename}: (hidden) already deleted`);
return;
}
saveData =
{
...old,
mtime,
size: 0,
children: [],
deleted: true,
type: "newnote",
};
}
await this.localDatabase.putRaw(saveData);
Logger(`STORAGE -x> DB:${filename}: (hidden) Done`);
} catch (ex) {
Logger(`STORAGE -x> DB:${filename}: (hidden) Failed`);
Logger(ex, LOG_LEVEL.VERBOSE);
return false;
}
});
}
async extractInternalFileFromDatabase(filename: FilePath, force = false) {
const isExists = await this.app.vault.adapter.exists(filename);
const prefixedFileName = addPrefix(filename, ICHeader);
return await runWithLock("file-" + prefixedFileName, false, async () => {
try {
// Check conflicted status
//TODO option
const fileOnDB = await this.localDatabase.getDBEntry(prefixedFileName, { conflicts: true }, false, true);
if (fileOnDB === false)
throw new Error(`File not found on database.:${filename}`);
// Prevent overwrite for Prevent overwriting while some conflicted revision exists.
if (fileOnDB?._conflicts?.length) {
Logger(`Hidden file ${filename} has conflicted revisions, to keep in safe, writing to storage has been prevented`, LOG_LEVEL.INFO);
return;
}
const deleted = "deleted" in fileOnDB ? fileOnDB.deleted : false;
if (deleted) {
if (!isExists) {
Logger(`STORAGE <x- DB:${filename}: deleted (hidden) Deleted on DB, but the file is already not found on storage.`);
} else {
Logger(`STORAGE <x- DB:${filename}: deleted (hidden).`);
await this.app.vault.adapter.remove(filename);
try {
//@ts-ignore internalAPI
await app.vault.adapter.reconcileInternalFile(filename);
} catch (ex) {
Logger("Failed to call internal API(reconcileInternalFile)", LOG_LEVEL.VERBOSE);
Logger(ex, LOG_LEVEL.VERBOSE);
}
}
return true;
}
if (!isExists) {
await this.ensureDirectoryEx(filename);
await this.app.vault.adapter.writeBinary(filename, base64ToArrayBuffer(fileOnDB.data), { mtime: fileOnDB.mtime, ctime: fileOnDB.ctime });
try {
//@ts-ignore internalAPI
await app.vault.adapter.reconcileInternalFile(filename);
} catch (ex) {
Logger("Failed to call internal API(reconcileInternalFile)", LOG_LEVEL.VERBOSE);
Logger(ex, LOG_LEVEL.VERBOSE);
}
Logger(`STORAGE <-- DB:${filename}: written (hidden,new${force ? ", force" : ""})`);
return true;
} else {
const contentBin = await this.app.vault.adapter.readBinary(filename);
const content = await arrayBufferToBase64(contentBin);
if (content == fileOnDB.data && !force) {
// Logger(`STORAGE <-- DB:${filename}: skipped (hidden) Not changed`, LOG_LEVEL.VERBOSE);
return true;
}
await this.app.vault.adapter.writeBinary(filename, base64ToArrayBuffer(fileOnDB.data), { mtime: fileOnDB.mtime, ctime: fileOnDB.ctime });
try {
//@ts-ignore internalAPI
await app.vault.adapter.reconcileInternalFile(filename);
} catch (ex) {
Logger("Failed to call internal API(reconcileInternalFile)", LOG_LEVEL.VERBOSE);
Logger(ex, LOG_LEVEL.VERBOSE);
}
Logger(`STORAGE <-- DB:${filename}: written (hidden, overwrite${force ? ", force" : ""})`);
return true;
}
} catch (ex) {
Logger(`STORAGE <-- DB:${filename}: written (hidden, overwrite${force ? ", force" : ""}) Failed`);
Logger(ex, LOG_LEVEL.VERBOSE);
return false;
}
});
}
showJSONMergeDialogAndMerge(docA: LoadedEntry, docB: LoadedEntry): Promise<boolean> {
return runWithLock("conflict:merge-data", false, () => new Promise((res) => {
Logger("Opening data-merging dialog", LOG_LEVEL.VERBOSE);
const docs = [docA, docB];
const path = stripAllPrefixes(docA.path);
const modal = new JsonResolveModal(this.app, path, [docA, docB], async (keep, result) => {
// modal.close();
try {
const filename = path;
let needFlush = false;
if (!result && !keep) {
Logger(`Skipped merging: ${filename}`);
res(false);
return;
}
//Delete old revisions
if (result || keep) {
for (const doc of docs) {
if (doc._rev != keep) {
if (await this.localDatabase.deleteDBEntry(this.getPath(doc), { rev: doc._rev })) {
Logger(`Conflicted revision has been deleted: ${filename}`);
needFlush = true;
}
}
}
}
if (!keep && result) {
const isExists = await this.app.vault.adapter.exists(filename);
if (!isExists) {
await this.ensureDirectoryEx(filename);
}
await this.app.vault.adapter.write(filename, result);
const stat = await this.app.vault.adapter.stat(filename);
await this.storeInternalFileToDatabase({ path: filename, ...stat }, true);
try {
//@ts-ignore internalAPI
await app.vault.adapter.reconcileInternalFile(filename);
} catch (ex) {
Logger("Failed to call internal API(reconcileInternalFile)", LOG_LEVEL.VERBOSE);
Logger(ex, LOG_LEVEL.VERBOSE);
}
Logger(`STORAGE <-- DB:${filename}: written (hidden,merged)`);
}
if (needFlush) {
await this.extractInternalFileFromDatabase(filename, false);
Logger(`STORAGE --> DB:${filename}: extracted (hidden,merged)`);
}
res(true);
} catch (ex) {
Logger("Could not merge conflicted json");
Logger(ex, LOG_LEVEL.VERBOSE);
res(false);
}
});
modal.open();
}));
}
async scanInternalFiles(): Promise<InternalFileInfo[]> {
const ignoreFilter = this.settings.syncInternalFilesIgnorePatterns
.replace(/\n| /g, "")
.split(",").filter(e => e).map(e => new RegExp(e, "i"));
const root = this.app.vault.getRoot();
const findRoot = root.path;
const filenames = (await this.getFiles(findRoot, [], null, ignoreFilter)).filter(e => e.startsWith(".")).filter(e => !e.startsWith(".trash"));
const files = filenames.map(async (e) => {
return {
path: e as FilePath,
stat: await this.app.vault.adapter.stat(e)
};
});
const result: InternalFileInfo[] = [];
for (const f of files) {
const w = await f;
result.push({
...w,
...w.stat
});
}
return result;
}
async getFiles(
path: string,
ignoreList: string[],
filter: RegExp[],
ignoreFilter: RegExp[]
) {
const w = await this.app.vault.adapter.list(path);
let files = [
...w.files
.filter((e) => !ignoreList.some((ee) => e.endsWith(ee)))
.filter((e) => !filter || filter.some((ee) => e.match(ee)))
.filter((e) => !ignoreFilter || ignoreFilter.every((ee) => !e.match(ee))),
];
L1: for (const v of w.folders) {
for (const ignore of ignoreList) {
if (v.endsWith(ignore)) {
continue L1;
}
}
if (ignoreFilter && ignoreFilter.some(e => v.match(e))) {
continue L1;
}
files = files.concat(await this.getFiles(v, ignoreList, filter, ignoreFilter));
}
return files;
}
}
-316
View File
@@ -1,316 +0,0 @@
import { normalizePath, type PluginManifest } from "./deps";
import type { DocumentID, EntryDoc, FilePathWithPrefix, LoadedEntry } from "./lib/src/types";
import { LOG_LEVEL } from "./lib/src/types";
import { type PluginDataEntry, PERIODIC_PLUGIN_SWEEP, type PluginList, type DevicePluginList, PSCHeader, PSCHeaderEnd } from "./types";
import { getDocData, isDocContentSame } from "./lib/src/utils";
import { Logger } from "./lib/src/logger";
import { PouchDB } from "./lib/src/pouchdb-browser.js";
import { isPluginMetadata, PeriodicProcessor } from "./utils";
import { PluginDialogModal } from "./dialogs";
import { NewNotice } from "./lib/src/wrapper";
import { versionNumberString2Number } from "./lib/src/strbin";
import { runWithLock } from "./lib/src/lock";
import { LiveSyncCommands } from "./LiveSyncCommands";
export class PluginAndTheirSettings extends LiveSyncCommands {
get deviceAndVaultName() {
return this.plugin.deviceAndVaultName;
}
pluginDialog: PluginDialogModal = null;
periodicPluginSweepProcessor = new PeriodicProcessor(this.plugin, async () => await this.sweepPlugin(false));
showPluginSyncModal() {
if (this.pluginDialog != null) {
this.pluginDialog.open();
} else {
this.pluginDialog = new PluginDialogModal(this.app, this.plugin);
this.pluginDialog.open();
}
}
hidePluginSyncModal() {
if (this.pluginDialog != null) {
this.pluginDialog.close();
this.pluginDialog = null;
}
}
onload(): void | Promise<void> {
this.plugin.addCommand({
id: "livesync-plugin-dialog",
name: "Show Plugins and their settings",
callback: () => {
this.showPluginSyncModal();
},
});
this.showPluginSyncModal();
}
onunload() {
this.hidePluginSyncModal();
this.periodicPluginSweepProcessor?.disable();
}
parseReplicationResultItem(doc: PouchDB.Core.ExistingDocument<EntryDoc>) {
if (isPluginMetadata(doc._id)) {
if (this.settings.notifyPluginOrSettingUpdated) {
this.triggerCheckPluginUpdate();
return true;
}
}
return false;
}
async beforeReplicate(showMessage: boolean) {
if (this.settings.autoSweepPlugins) {
await this.sweepPlugin(showMessage);
}
}
async onResume() {
if (this.plugin.suspended)
return;
if (this.settings.autoSweepPlugins) {
await this.sweepPlugin(false);
}
this.periodicPluginSweepProcessor.enable(this.settings.autoSweepPluginsPeriodic && !this.settings.watchInternalFileChanges ? (PERIODIC_PLUGIN_SWEEP * 1000) : 0);
}
async onInitializeDatabase(showNotice: boolean) {
if (this.settings.usePluginSync) {
try {
Logger("Scanning plugins...");
await this.sweepPlugin(showNotice);
Logger("Scanning plugins done");
} catch (ex) {
Logger("Scanning plugins failed");
Logger(ex, LOG_LEVEL.VERBOSE);
}
}
}
async realizeSettingSyncMode() {
this.periodicPluginSweepProcessor?.disable();
if (this.plugin.suspended)
return;
if (this.settings.autoSweepPlugins) {
await this.sweepPlugin(false);
}
this.periodicPluginSweepProcessor.enable(this.settings.autoSweepPluginsPeriodic && !this.settings.watchInternalFileChanges ? (PERIODIC_PLUGIN_SWEEP * 1000) : 0);
}
triggerCheckPluginUpdate() {
(async () => await this.checkPluginUpdate())();
}
async getPluginList(): Promise<{ plugins: PluginList; allPlugins: DevicePluginList; thisDevicePlugins: DevicePluginList; }> {
const docList = await this.localDatabase.allDocsRaw<PluginDataEntry>({ startkey: PSCHeader, endkey: PSCHeaderEnd, include_docs: false });
const oldDocs: PluginDataEntry[] = ((await Promise.all(docList.rows.map(async (e) => await this.localDatabase.getDBEntry(e.id as FilePathWithPrefix /* WARN!! THIS SHOULD BE WRAPPED */)))).filter((e) => e !== false) as LoadedEntry[]).map((e) => JSON.parse(getDocData(e.data)));
const plugins: { [key: string]: PluginDataEntry[]; } = {};
const allPlugins: { [key: string]: PluginDataEntry; } = {};
const thisDevicePlugins: { [key: string]: PluginDataEntry; } = {};
for (const v of oldDocs) {
if (typeof plugins[v.deviceVaultName] === "undefined") {
plugins[v.deviceVaultName] = [];
}
plugins[v.deviceVaultName].push(v);
allPlugins[v._id] = v;
if (v.deviceVaultName == this.deviceAndVaultName) {
thisDevicePlugins[v.manifest.id] = v;
}
}
return { plugins, allPlugins, thisDevicePlugins };
}
async checkPluginUpdate() {
if (!this.plugin.settings.usePluginSync)
return;
await this.sweepPlugin(false);
const { allPlugins, thisDevicePlugins } = await this.getPluginList();
const arrPlugins = Object.values(allPlugins);
let updateFound = false;
for (const plugin of arrPlugins) {
const ownPlugin = thisDevicePlugins[plugin.manifest.id];
if (ownPlugin) {
const remoteVersion = versionNumberString2Number(plugin.manifest.version);
const ownVersion = versionNumberString2Number(ownPlugin.manifest.version);
if (remoteVersion > ownVersion) {
updateFound = true;
}
if (((plugin.mtime / 1000) | 0) > ((ownPlugin.mtime / 1000) | 0) && (plugin.dataJson ?? "") != (ownPlugin.dataJson ?? "")) {
updateFound = true;
}
}
}
if (updateFound) {
const fragment = createFragment((doc) => {
doc.createEl("a", null, (a) => {
a.text = "There're some new plugins or their settings";
a.addEventListener("click", () => this.showPluginSyncModal());
});
});
NewNotice(fragment, 10000);
} else {
Logger("Everything is up to date.", LOG_LEVEL.NOTICE);
}
}
async sweepPlugin(showMessage = false, specificPluginPath = "") {
if (!this.settings.usePluginSync)
return;
if (!this.localDatabase.isReady)
return;
// @ts-ignore
const pl = this.app.plugins;
const manifests: PluginManifest[] = Object.values(pl.manifests);
let specificPlugin = "";
if (specificPluginPath != "") {
specificPlugin = manifests.find(e => e.dir.endsWith("/" + specificPluginPath))?.id ?? "";
}
await runWithLock("sweepplugin", true, async () => {
const logLevel = showMessage ? LOG_LEVEL.NOTICE : LOG_LEVEL.INFO;
if (!this.deviceAndVaultName) {
Logger("You have to set your device name.", LOG_LEVEL.NOTICE);
return;
}
Logger("Scanning plugins", logLevel);
const oldDocs = await this.localDatabase.allDocsRaw<EntryDoc>({
startkey: `ps:${this.deviceAndVaultName}-${specificPlugin}`,
endkey: `ps:${this.deviceAndVaultName}-${specificPlugin}\u{10ffff}`,
include_docs: true,
});
// Logger("OLD DOCS.", LOG_LEVEL.VERBOSE);
// sweep current plugin.
const procs = manifests.map(async (m) => {
const pluginDataEntryID = `ps:${this.deviceAndVaultName}-${m.id}` as DocumentID;
try {
if (specificPlugin && m.id != specificPlugin) {
return;
}
Logger(`Reading plugin:${m.name}(${m.id})`, LOG_LEVEL.VERBOSE);
const path = normalizePath(m.dir) + "/";
const adapter = this.app.vault.adapter;
const files = ["manifest.json", "main.js", "styles.css", "data.json"];
const pluginData: { [key: string]: string; } = {};
for (const file of files) {
const thePath = path + file;
if (await adapter.exists(thePath)) {
pluginData[file] = await adapter.read(thePath);
}
}
let mtime = 0;
if (await adapter.exists(path + "/data.json")) {
mtime = (await adapter.stat(path + "/data.json")).mtime;
}
const p: PluginDataEntry = {
_id: pluginDataEntryID,
dataJson: pluginData["data.json"],
deviceVaultName: this.deviceAndVaultName,
mainJs: pluginData["main.js"],
styleCss: pluginData["styles.css"],
manifest: m,
manifestJson: pluginData["manifest.json"],
mtime: mtime,
type: "plugin",
};
const d: LoadedEntry = {
_id: p._id,
path: p._id as string as FilePathWithPrefix,
data: JSON.stringify(p),
ctime: mtime,
mtime: mtime,
size: 0,
children: [],
datatype: "plain",
type: "plain"
};
Logger(`check diff:${m.name}(${m.id})`, LOG_LEVEL.VERBOSE);
await runWithLock("plugin-" + m.id, false, async () => {
const old = await this.localDatabase.getDBEntry(p._id as string as FilePathWithPrefix /* This also should be explained */, null, false, false);
if (old !== false) {
const oldData = { data: old.data, deleted: old._deleted };
const newData = { data: d.data, deleted: d._deleted };
if (isDocContentSame(oldData.data, newData.data) && oldData.deleted == newData.deleted) {
Logger(`Nothing changed:${m.name}`);
return;
}
}
await this.localDatabase.putDBEntry(d);
Logger(`Plugin saved:${m.name}`, logLevel);
});
} catch (ex) {
Logger(`Plugin save failed:${m.name}`, LOG_LEVEL.NOTICE);
} finally {
oldDocs.rows = oldDocs.rows.filter((e) => e.id != pluginDataEntryID);
}
//remove saved plugin data.
}
);
await Promise.all(procs);
const delDocs = oldDocs.rows.map((e) => {
// e.doc._deleted = true;
if (e.doc.type == "newnote" || e.doc.type == "plain") {
e.doc.deleted = true;
if (this.settings.deleteMetadataOfDeletedFiles) {
e.doc._deleted = true;
}
} else {
e.doc._deleted = true;
}
return e.doc;
});
Logger(`Deleting old plugin:(${delDocs.length})`, LOG_LEVEL.VERBOSE);
await this.localDatabase.bulkDocsRaw(delDocs);
Logger(`Scan plugin done.`, logLevel);
});
}
async applyPluginData(plugin: PluginDataEntry) {
await runWithLock("plugin-" + plugin.manifest.id, false, async () => {
const pluginTargetFolderPath = normalizePath(plugin.manifest.dir) + "/";
const adapter = this.app.vault.adapter;
// @ts-ignore
const stat = this.app.plugins.enabledPlugins.has(plugin.manifest.id) == true;
if (stat) {
// @ts-ignore
await this.app.plugins.unloadPlugin(plugin.manifest.id);
Logger(`Unload plugin:${plugin.manifest.id}`, LOG_LEVEL.NOTICE);
}
if (plugin.dataJson)
await adapter.write(pluginTargetFolderPath + "data.json", plugin.dataJson);
Logger("wrote:" + pluginTargetFolderPath + "data.json", LOG_LEVEL.NOTICE);
if (stat) {
// @ts-ignore
await this.app.plugins.loadPlugin(plugin.manifest.id);
Logger(`Load plugin:${plugin.manifest.id}`, LOG_LEVEL.NOTICE);
}
});
}
async applyPlugin(plugin: PluginDataEntry) {
await runWithLock("plugin-" + plugin.manifest.id, false, async () => {
// @ts-ignore
const stat = this.app.plugins.enabledPlugins.has(plugin.manifest.id) == true;
if (stat) {
// @ts-ignore
await this.app.plugins.unloadPlugin(plugin.manifest.id);
Logger(`Unload plugin:${plugin.manifest.id}`, LOG_LEVEL.NOTICE);
}
const pluginTargetFolderPath = normalizePath(plugin.manifest.dir) + "/";
const adapter = this.app.vault.adapter;
if ((await adapter.exists(pluginTargetFolderPath)) === false) {
await adapter.mkdir(pluginTargetFolderPath);
}
await adapter.write(pluginTargetFolderPath + "main.js", plugin.mainJs);
await adapter.write(pluginTargetFolderPath + "manifest.json", plugin.manifestJson);
if (plugin.styleCss)
await adapter.write(pluginTargetFolderPath + "styles.css", plugin.styleCss);
if (stat) {
// @ts-ignore
await this.app.plugins.loadPlugin(plugin.manifest.id);
Logger(`Load plugin:${plugin.manifest.id}`, LOG_LEVEL.NOTICE);
}
});
}
}
-346
View File
@@ -1,346 +0,0 @@
import { type EntryDoc, type ObsidianLiveSyncSettings, LOG_LEVEL, DEFAULT_SETTINGS } from "./lib/src/types";
import { configURIBase } from "./types";
import { Logger } from "./lib/src/logger";
import { PouchDB } from "./lib/src/pouchdb-browser.js";
import { askSelectString, askYesNo, askString } from "./utils";
import { decrypt, encrypt } from "./lib/src/e2ee_v2";
import { LiveSyncCommands } from "./LiveSyncCommands";
import { delay } from "./lib/src/utils";
import { confirmWithMessage } from "./dialogs";
import { Platform } from "./deps";
export class SetupLiveSync extends LiveSyncCommands {
onunload() { }
onload(): void | Promise<void> {
this.plugin.registerObsidianProtocolHandler("setuplivesync", async (conf: any) => await this.setupWizard(conf.settings));
this.plugin.addCommand({
id: "livesync-copysetupuri",
name: "Copy the setup URI",
callback: this.command_copySetupURI.bind(this),
});
this.plugin.addCommand({
id: "livesync-copysetupurifull",
name: "Copy the setup URI (Full)",
callback: this.command_copySetupURIFull.bind(this),
});
this.plugin.addCommand({
id: "livesync-opensetupuri",
name: "Open the setup URI",
callback: this.command_openSetupURI.bind(this),
});
}
onInitializeDatabase(showNotice: boolean) { }
beforeReplicate(showNotice: boolean) { }
onResume() { }
parseReplicationResultItem(docs: PouchDB.Core.ExistingDocument<EntryDoc>): boolean | Promise<boolean> {
return false;
}
async realizeSettingSyncMode() { }
async command_copySetupURI() {
const encryptingPassphrase = await askString(this.app, "Encrypt your settings", "The passphrase to encrypt the setup URI", "");
if (encryptingPassphrase === false)
return;
const setting = { ...this.settings, configPassphraseStore: "", encryptedCouchDBConnection: "", encryptedPassphrase: "" };
const keys = Object.keys(setting) as (keyof ObsidianLiveSyncSettings)[];
for (const k of keys) {
if (JSON.stringify(k in setting ? setting[k] : "") == JSON.stringify(k in DEFAULT_SETTINGS ? DEFAULT_SETTINGS[k] : "*")) {
delete setting[k];
}
}
const encryptedSetting = encodeURIComponent(await encrypt(JSON.stringify(setting), encryptingPassphrase, false));
const uri = `${configURIBase}${encryptedSetting}`;
await navigator.clipboard.writeText(uri);
Logger("Setup URI copied to clipboard", LOG_LEVEL.NOTICE);
}
async command_copySetupURIFull() {
const encryptingPassphrase = await askString(this.app, "Encrypt your settings", "The passphrase to encrypt the setup URI", "");
if (encryptingPassphrase === false)
return;
const setting = { ...this.settings, configPassphraseStore: "", encryptedCouchDBConnection: "", encryptedPassphrase: "" };
const encryptedSetting = encodeURIComponent(await encrypt(JSON.stringify(setting), encryptingPassphrase, false));
const uri = `${configURIBase}${encryptedSetting}`;
await navigator.clipboard.writeText(uri);
Logger("Setup URI copied to clipboard", LOG_LEVEL.NOTICE);
}
async command_openSetupURI() {
const setupURI = await askString(this.app, "Easy setup", "Set up URI", `${configURIBase}aaaaa`);
if (setupURI === false)
return;
if (!setupURI.startsWith(`${configURIBase}`)) {
Logger("Set up URI looks wrong.", LOG_LEVEL.NOTICE);
return;
}
const config = decodeURIComponent(setupURI.substring(configURIBase.length));
console.dir(config);
await this.setupWizard(config);
}
async setupWizard(confString: string) {
try {
const oldConf = JSON.parse(JSON.stringify(this.settings));
const encryptingPassphrase = await askString(this.app, "Passphrase", "The passphrase to decrypt your setup URI", "");
if (encryptingPassphrase === false)
return;
const newConf = await JSON.parse(await decrypt(confString, encryptingPassphrase, false));
if (newConf) {
const result = await askYesNo(this.app, "Importing LiveSync's conf, OK?");
if (result == "yes") {
const newSettingW = Object.assign({}, DEFAULT_SETTINGS, newConf) as ObsidianLiveSyncSettings;
this.plugin.replicator.closeReplication();
this.settings.suspendFileWatching = true;
console.dir(newSettingW);
// Back into the default method once.
newSettingW.configPassphraseStore = "";
newSettingW.encryptedPassphrase = "";
newSettingW.encryptedCouchDBConnection = "";
const setupJustImport = "Just import setting";
const setupAsNew = "Set it up as secondary or subsequent device";
const setupAgain = "Reconfigure and reconstitute the data";
const setupManually = "Leave everything to me";
newSettingW.syncInternalFiles = false;
newSettingW.usePluginSync = false;
const setupType = await askSelectString(this.app, "How would you like to set it up?", [setupAsNew, setupAgain, setupJustImport, setupManually]);
if (setupType == setupJustImport) {
this.plugin.settings = newSettingW;
this.plugin.usedPassphrase = "";
await this.plugin.saveSettings();
} else if (setupType == setupAsNew) {
this.plugin.settings = newSettingW;
this.plugin.usedPassphrase = "";
await this.fetchLocal();
} else if (setupType == setupAgain) {
const confirm = "I know this operation will rebuild all my databases with files on this device, and files that are on the remote database and I didn't synchronize to any other devices will be lost and want to proceed indeed.";
if (await askSelectString(this.app, "Do you really want to do this?", ["Cancel", confirm]) != confirm) {
return;
}
this.plugin.settings = newSettingW;
this.plugin.usedPassphrase = "";
await this.rebuildEverything();
} else if (setupType == setupManually) {
const keepLocalDB = await askYesNo(this.app, "Keep local DB?");
const keepRemoteDB = await askYesNo(this.app, "Keep remote DB?");
if (keepLocalDB == "yes" && keepRemoteDB == "yes") {
// nothing to do. so peaceful.
this.plugin.settings = newSettingW;
this.plugin.usedPassphrase = "";
this.suspendAllSync();
this.suspendExtraSync();
await this.plugin.saveSettings();
const replicate = await askYesNo(this.app, "Unlock and replicate?");
if (replicate == "yes") {
await this.plugin.replicate(true);
await this.plugin.markRemoteUnlocked();
}
Logger("Configuration loaded.", LOG_LEVEL.NOTICE);
return;
}
if (keepLocalDB == "no" && keepRemoteDB == "no") {
const reset = await askYesNo(this.app, "Drop everything?");
if (reset != "yes") {
Logger("Cancelled", LOG_LEVEL.NOTICE);
this.plugin.settings = oldConf;
return;
}
}
let initDB;
this.plugin.settings = newSettingW;
this.plugin.usedPassphrase = "";
await this.plugin.saveSettings();
if (keepLocalDB == "no") {
await this.plugin.resetLocalDatabase();
await this.plugin.localDatabase.initializeDatabase();
const rebuild = await askYesNo(this.app, "Rebuild the database?");
if (rebuild == "yes") {
initDB = this.plugin.initializeDatabase(true);
} else {
await this.plugin.markRemoteResolved();
}
}
if (keepRemoteDB == "no") {
await this.plugin.tryResetRemoteDatabase();
await this.plugin.markRemoteLocked();
}
if (keepLocalDB == "no" || keepRemoteDB == "no") {
const replicate = await askYesNo(this.app, "Replicate once?");
if (replicate == "yes") {
if (initDB != null) {
await initDB;
}
await this.plugin.replicate(true);
}
}
}
}
Logger("Configuration loaded.", LOG_LEVEL.NOTICE);
} else {
Logger("Cancelled.", LOG_LEVEL.NOTICE);
}
} catch (ex) {
Logger("Couldn't parse or decrypt configuration uri.", LOG_LEVEL.NOTICE);
}
}
suspendExtraSync() {
Logger("Hidden files and plugin synchronization have been temporarily disabled. Please enable them after the fetching, if you need them.", LOG_LEVEL.NOTICE)
this.plugin.settings.syncInternalFiles = false;
this.plugin.settings.usePluginSync = false;
this.plugin.settings.autoSweepPlugins = false;
}
async askHiddenFileConfiguration(opt: { enableFetch?: boolean, enableOverwrite?: boolean }) {
this.plugin.addOnSetup.suspendExtraSync();
const message = `Would you like to enable \`Hidden File Synchronization\` or \`Customization sync\`?
${opt.enableFetch ? " - Fetch: Use files stored from other devices. \n" : ""}${opt.enableOverwrite ? "- Overwrite: Use files from this device. \n" : ""}- Custom: Synchronize only customization files with a dedicated interface.
- Keep them disabled: Do not use hidden file synchronization.
Of course, we are able to disable these features.`
const CHOICE_FETCH = "Fetch";
const CHOICE_OVERWRITE = "Overwrite";
const CHOICE_CUSTOMIZE = "Custom";
const CHOICE_DISMISS = "keep them disabled";
const choices = [];
if (opt?.enableFetch) {
choices.push(CHOICE_FETCH);
}
if (opt?.enableOverwrite) {
choices.push(CHOICE_OVERWRITE);
}
choices.push(CHOICE_CUSTOMIZE);
choices.push(CHOICE_DISMISS);
const ret = await confirmWithMessage(this.plugin, "Hidden file sync", message, choices, CHOICE_DISMISS, 40);
if (ret == CHOICE_FETCH) {
await this.configureHiddenFileSync("FETCH");
} else if (ret == CHOICE_OVERWRITE) {
await this.configureHiddenFileSync("OVERWRITE");
} else if (ret == CHOICE_DISMISS) {
await this.configureHiddenFileSync("DISABLE");
} else if (ret == CHOICE_CUSTOMIZE) {
await this.configureHiddenFileSync("CUSTOMIZE");
}
}
async configureHiddenFileSync(mode: "FETCH" | "OVERWRITE" | "MERGE" | "DISABLE" | "CUSTOMIZE") {
this.plugin.addOnSetup.suspendExtraSync();
if (mode == "DISABLE") {
this.plugin.settings.syncInternalFiles = false;
this.plugin.settings.usePluginSync = false;
await this.plugin.saveSettings();
return;
}
if (mode != "CUSTOMIZE") {
Logger("Gathering files for enabling Hidden File Sync", LOG_LEVEL.NOTICE);
if (mode == "FETCH") {
await this.plugin.addOnHiddenFileSync.syncInternalFilesAndDatabase("pullForce", true);
} else if (mode == "OVERWRITE") {
await this.plugin.addOnHiddenFileSync.syncInternalFilesAndDatabase("pushForce", true);
} else if (mode == "MERGE") {
await this.plugin.addOnHiddenFileSync.syncInternalFilesAndDatabase("safe", true);
}
this.plugin.settings.syncInternalFiles = true;
await this.plugin.saveSettings();
Logger(`Done! Restarting the app is strongly recommended!`, LOG_LEVEL.NOTICE);
} else if (mode == "CUSTOMIZE") {
if (!this.plugin.deviceAndVaultName) {
let name = await askString(this.app, "Device name", "Please set this device name", `desktop`);
if (!name) {
if (Platform.isAndroidApp) {
name = "android-app"
} else if (Platform.isIosApp) {
name = "ios"
} else if (Platform.isMacOS) {
name = "macos"
} else if (Platform.isMobileApp) {
name = "mobile-app"
} else if (Platform.isMobile) {
name = "mobile"
} else if (Platform.isSafari) {
name = "safari"
} else if (Platform.isDesktop) {
name = "desktop"
} else if (Platform.isDesktopApp) {
name = "desktop-app"
} else {
name = "unknown"
}
name = name + Math.random().toString(36).slice(-4);
}
this.plugin.deviceAndVaultName = name;
}
this.plugin.settings.usePluginSync = true;
await this.plugin.saveSettings();
await this.plugin.addOnConfigSync.scanAllConfigFiles(true);
}
}
suspendAllSync() {
this.plugin.settings.liveSync = false;
this.plugin.settings.periodicReplication = false;
this.plugin.settings.syncOnSave = false;
this.plugin.settings.syncOnStart = false;
this.plugin.settings.syncOnFileOpen = false;
this.plugin.settings.syncAfterMerge = false;
//this.suspendExtraSync();
}
async askUseNewAdapter() {
if (!this.plugin.settings.useIndexedDBAdapter) {
const message = `Now this plugin has been configured to use the old database adapter for keeping compatibility. Do you want to deactivate it?`;
const CHOICE_YES = "Yes, disable and use latest";
const CHOICE_NO = "No, keep compatibility";
const choices = [CHOICE_YES, CHOICE_NO];
const ret = await confirmWithMessage(this.plugin, "Database adapter", message, choices, CHOICE_YES, 10);
if (ret == CHOICE_YES) {
this.plugin.settings.useIndexedDBAdapter = false;
}
}
}
async fetchLocal() {
this.suspendExtraSync();
this.askUseNewAdapter();
await this.plugin.realizeSettingSyncMode();
await this.plugin.resetLocalDatabase();
await delay(1000);
await this.plugin.markRemoteResolved();
await this.plugin.openDatabase();
this.plugin.isReady = true;
await delay(500);
await this.plugin.replicateAllFromServer(true);
await delay(1000);
await this.plugin.replicateAllFromServer(true);
await this.askHiddenFileConfiguration({ enableFetch: true });
}
async rebuildRemote() {
this.suspendExtraSync();
await this.plugin.realizeSettingSyncMode();
await this.plugin.markRemoteLocked();
await this.plugin.tryResetRemoteDatabase();
await this.plugin.markRemoteLocked();
await delay(500);
await this.askHiddenFileConfiguration({ enableOverwrite: true });
await delay(1000);
await this.plugin.replicateAllToServer(true);
await delay(1000);
await this.plugin.replicateAllToServer(true);
}
async rebuildEverything() {
this.suspendExtraSync();
this.askUseNewAdapter();
await this.plugin.realizeSettingSyncMode();
await this.plugin.resetLocalDatabase();
await delay(1000);
await this.plugin.initializeDatabase(true);
await this.plugin.markRemoteLocked();
await this.plugin.tryResetRemoteDatabase();
await this.plugin.markRemoteLocked();
await delay(500);
await this.askHiddenFileConfiguration({ enableOverwrite: true });
await delay(1000);
await this.plugin.replicateAllToServer(true);
await delay(1000);
await this.plugin.replicateAllToServer(true);
}
}
-84
View File
@@ -1,84 +0,0 @@
import { App, Modal } from "./deps";
import { DIFF_DELETE, DIFF_EQUAL, DIFF_INSERT } from "diff-match-patch";
import { diff_result } from "./lib/src/types";
import { escapeStringToHTML } from "./lib/src/strbin";
export class ConflictResolveModal extends Modal {
// result: Array<[number, string]>;
result: diff_result;
filename: string;
callback: (remove_rev: string) => Promise<void>;
constructor(app: App, filename: string, diff: diff_result, callback: (remove_rev: string) => Promise<void>) {
super(app);
this.result = diff;
this.callback = callback;
this.filename = filename;
}
onOpen() {
const { contentEl } = this;
contentEl.empty();
contentEl.createEl("h2", { text: "This document has conflicted changes." });
contentEl.createEl("span", { text: this.filename });
const div = contentEl.createDiv("");
div.addClass("op-scrollable");
let diff = "";
for (const v of this.result.diff) {
const x1 = v[0];
const x2 = v[1];
if (x1 == DIFF_DELETE) {
diff += "<span class='deleted'>" + escapeStringToHTML(x2) + "</span>";
} else if (x1 == DIFF_EQUAL) {
diff += "<span class='normal'>" + escapeStringToHTML(x2) + "</span>";
} else if (x1 == DIFF_INSERT) {
diff += "<span class='added'>" + escapeStringToHTML(x2) + "</span>";
}
}
diff = diff.replace(/\n/g, "<br>");
div.innerHTML = diff;
const div2 = contentEl.createDiv("");
const date1 = new Date(this.result.left.mtime).toLocaleString() + (this.result.left.deleted ? " (Deleted)" : "");
const date2 = new Date(this.result.right.mtime).toLocaleString() + (this.result.right.deleted ? " (Deleted)" : "");
div2.innerHTML = `
<span class='deleted'>A:${date1}</span><br /><span class='added'>B:${date2}</span><br>
`;
contentEl.createEl("button", { text: "Keep A" }, (e) => {
e.addEventListener("click", async () => {
await this.callback(this.result.right.rev);
this.callback = null;
this.close();
});
});
contentEl.createEl("button", { text: "Keep B" }, (e) => {
e.addEventListener("click", async () => {
await this.callback(this.result.left.rev);
this.callback = null;
this.close();
});
});
contentEl.createEl("button", { text: "Concat both" }, (e) => {
e.addEventListener("click", async () => {
await this.callback("");
this.callback = null;
this.close();
});
});
contentEl.createEl("button", { text: "Not now" }, (e) => {
e.addEventListener("click", () => {
this.close();
});
});
}
onClose() {
const { contentEl } = this;
contentEl.empty();
if (this.callback != null) {
this.callback(null);
}
}
}
-212
View File
@@ -1,212 +0,0 @@
import { TFile, Modal, App } from "./deps";
import { getPathFromTFile, isValidPath } from "./utils";
import { base64ToArrayBuffer, base64ToString, escapeStringToHTML } from "./lib/src/strbin";
import ObsidianLiveSyncPlugin from "./main";
import { DIFF_DELETE, DIFF_EQUAL, DIFF_INSERT, diff_match_patch } from "diff-match-patch";
import { DocumentID, FilePathWithPrefix, LoadedEntry, LOG_LEVEL } from "./lib/src/types";
import { Logger } from "./lib/src/logger";
import { isErrorOfMissingDoc } from "./lib/src/utils_couchdb";
import { getDocData } from "./lib/src/utils";
import { stripPrefix } from "./lib/src/path";
export class DocumentHistoryModal extends Modal {
plugin: ObsidianLiveSyncPlugin;
range: HTMLInputElement;
contentView: HTMLDivElement;
info: HTMLDivElement;
fileInfo: HTMLDivElement;
showDiff = false;
id: DocumentID;
file: FilePathWithPrefix;
revs_info: PouchDB.Core.RevisionInfo[] = [];
currentDoc: LoadedEntry;
currentText = "";
currentDeleted = false;
constructor(app: App, plugin: ObsidianLiveSyncPlugin, file: TFile | FilePathWithPrefix, id: DocumentID) {
super(app);
this.plugin = plugin;
this.file = (file instanceof TFile) ? getPathFromTFile(file) : file;
this.id = id;
if (!file) {
this.file = this.plugin.id2path(id, null);
}
if (localStorage.getItem("ols-history-highlightdiff") == "1") {
this.showDiff = true;
}
}
async loadFile() {
if (!this.id) {
this.id = await this.plugin.path2id(this.file);
}
const db = this.plugin.localDatabase;
try {
const w = await db.localDatabase.get(this.id, { revs_info: true });
this.revs_info = w._revs_info.filter((e) => e?.status == "available");
this.range.max = `${this.revs_info.length - 1}`;
this.range.value = this.range.max;
this.fileInfo.setText(`${this.file} / ${this.revs_info.length} revisions`);
await this.loadRevs();
} catch (ex) {
if (isErrorOfMissingDoc(ex)) {
this.range.max = "0";
this.range.value = "";
this.range.disabled = true;
this.showDiff
this.contentView.setText(`History of this file was not recorded.`);
} else {
this.contentView.setText(`Error occurred.`);
Logger(ex, LOG_LEVEL.VERBOSE);
}
}
}
async loadRevs() {
if (this.revs_info.length == 0) return;
const db = this.plugin.localDatabase;
const index = this.revs_info.length - 1 - (this.range.value as any) / 1;
const rev = this.revs_info[index];
const w = await db.getDBEntry(this.file, { rev: rev.rev }, false, false, true);
this.currentText = "";
this.currentDeleted = false;
if (w === false) {
this.currentDeleted = true;
this.info.innerHTML = "";
this.contentView.innerHTML = `Could not read this revision<br>(${rev.rev})`;
} else {
this.currentDoc = w;
this.info.innerHTML = `Modified:${new Date(w.mtime).toLocaleString()}`;
let result = "";
const w1data = w.datatype == "plain" ? getDocData(w.data) : base64ToString(w.data);
this.currentDeleted = w.deleted;
this.currentText = w1data;
if (this.showDiff) {
const prevRevIdx = this.revs_info.length - 1 - ((this.range.value as any) / 1 - 1);
if (prevRevIdx >= 0 && prevRevIdx < this.revs_info.length) {
const oldRev = this.revs_info[prevRevIdx].rev;
const w2 = await db.getDBEntry(this.file, { rev: oldRev }, false, false, true);
if (w2 != false) {
const dmp = new diff_match_patch();
const w2data = w2.datatype == "plain" ? getDocData(w2.data) : base64ToString(w2.data);
const diff = dmp.diff_main(w2data, w1data);
dmp.diff_cleanupSemantic(diff);
for (const v of diff) {
const x1 = v[0];
const x2 = v[1];
if (x1 == DIFF_DELETE) {
result += "<span class='history-deleted'>" + escapeStringToHTML(x2) + "</span>";
} else if (x1 == DIFF_EQUAL) {
result += "<span class='history-normal'>" + escapeStringToHTML(x2) + "</span>";
} else if (x1 == DIFF_INSERT) {
result += "<span class='history-added'>" + escapeStringToHTML(x2) + "</span>";
}
}
result = result.replace(/\n/g, "<br>");
} else {
result = escapeStringToHTML(w1data);
}
} else {
result = escapeStringToHTML(w1data);
}
} else {
result = escapeStringToHTML(w1data);
}
this.contentView.innerHTML = (this.currentDeleted ? "(At this revision, the file has been deleted)\n" : "") + result;
}
}
onOpen() {
const { contentEl } = this;
contentEl.empty();
contentEl.createEl("h2", { text: "Document History" });
this.fileInfo = contentEl.createDiv("");
this.fileInfo.addClass("op-info");
const divView = contentEl.createDiv("");
divView.addClass("op-flex");
divView.createEl("input", { type: "range" }, (e) => {
this.range = e;
e.addEventListener("change", (e) => {
this.loadRevs();
});
e.addEventListener("input", (e) => {
this.loadRevs();
});
});
contentEl
.createDiv("", (e) => {
e.createEl("label", {}, (label) => {
label.appendChild(
createEl("input", { type: "checkbox" }, (checkbox) => {
if (this.showDiff) {
checkbox.checked = true;
}
checkbox.addEventListener("input", (evt: any) => {
this.showDiff = checkbox.checked;
localStorage.setItem("ols-history-highlightdiff", this.showDiff == true ? "1" : "");
this.loadRevs();
});
})
);
label.appendText("Highlight diff");
});
})
.addClass("op-info");
this.info = contentEl.createDiv("");
this.info.addClass("op-info");
this.loadFile();
const div = contentEl.createDiv({ text: "Loading old revisions..." });
this.contentView = div;
div.addClass("op-scrollable");
div.addClass("op-pre");
const buttons = contentEl.createDiv("");
buttons.createEl("button", { text: "Copy to clipboard" }, (e) => {
e.addClass("mod-cta");
e.addEventListener("click", async () => {
await navigator.clipboard.writeText(this.currentText);
Logger(`Old content copied to clipboard`, LOG_LEVEL.NOTICE);
});
});
async function focusFile(path: string) {
const targetFile = app.vault
.getFiles()
.find((f) => f.path === path);
if (targetFile) {
const leaf = app.workspace.getLeaf(false);
await leaf.openFile(targetFile);
} else {
Logger("The file could not view on the editor", LOG_LEVEL.NOTICE)
}
}
buttons.createEl("button", { text: "Back to this revision" }, (e) => {
e.addClass("mod-cta");
e.addEventListener("click", async () => {
// const pathToWrite = this.plugin.id2path(this.id, true);
const pathToWrite = stripPrefix(this.file);
if (!isValidPath(pathToWrite)) {
Logger("Path is not valid to write content.", LOG_LEVEL.INFO);
}
if (this.currentDoc?.datatype == "plain") {
await this.app.vault.adapter.write(pathToWrite, getDocData(this.currentDoc.data));
await focusFile(pathToWrite);
this.close();
} else if (this.currentDoc?.datatype == "newnote") {
await this.app.vault.adapter.writeBinary(pathToWrite, base64ToArrayBuffer(this.currentDoc.data));
await focusFile(pathToWrite);
this.close();
} else {
Logger(`Could not parse entry`, LOG_LEVEL.NOTICE);
}
});
});
}
onClose() {
const { contentEl } = this;
contentEl.empty();
}
}

Some files were not shown because too many files have changed in this diff Show More