public/obsidian-livesync
1
0
Fork 0
mirror of https://github.com/vrtmrz/obsidian-livesync.git synced 2026-02-23 12:38:47 +00:00
Code Issues Packages Projects Releases Wiki Activity

Compare commits

base: public:0.13.3
Branches Tags
public:main public:beta public:snyk-upgrade-7fd31e6f474568014502e7205847f257 public:snyk-fix-90f3b71450f7543a1c61724d8afa0e71 public:refactor_services_2 public:refactor_services public:dev public:snyk-upgrade-fe9625385aa4be849c7157033fd2abb0 public:snyk-upgrade-99a31e70c822894595544b0c9813b11a public:svelteui public:disenchant public:snyk-fix-61f76ef0bce2a276df9552c2fc80a14f public:snyk-upgrade-2baa7938ba318464c3bad5c3fd5cbda5 public:snyk-upgrade-7af62a575eab20e8fb4e9565bf1adcfc public:snyk-upgrade-b32f281b31a81483f126b8fb6c54a957 public:snyk-upgrade-bd61b1103442d3faecc9fc726c34e9a4 public:snyk-upgrade-5b2d6584430c061b5187611a5d7fb64a public:snyk-fix-0e9fbbcb374fcb8d5e8cb027d8414fb7 public:snyk-upgrade-5242daf91e706d1ae500c9ea6545636e public:snyk-upgrade-70434a6822fbad83424135dfad9df909 public:snyk-upgrade-8fa01ccad0352125f1866d41e57d35e1 public:snyk-upgrade-39eb97b4655e70091f718a88fccd57c1 public:snyk-upgrade-2b99d54d19e207887e36c3071b796925 public:forprivacy public:snyk-upgrade-dad9088d0a333677e0157e3deef7fc38 public:snyk-upgrade-b9a76d74b3e5723759af5b3319aabb80 public:snyk-upgrade-7da6459d8ff5d293d80de9d80061d912 public:snyk-upgrade-6a3786fc4cbe394542e9d1b6058c46e7 public:snyk-upgrade-8fc9e62771dc41b0f23dc0efefb5c949
public:0.25.43-patched-9 public:0.25.43-patched-8 public:0.25.43-patched-7 public:0.25.43-patched-6 public:0.25.43-patched-5 public:0.25.43-patched-4 public:0.25.43-patched-3 public:0.25.43-patched-2 public:0.25.43-patched-1 public:0.25.43 public:0.25.42 public:0.25.41-patched-3 public:0.25.41-patched-2 public:0.25.41-patched-1 public:0.25.41 public:0.25.40 public:0.25.39 public:0.25.38 public:0.25.37 public:0.25.36 public:0.25.35 public:0.25.34 public:0.25.33 public:0.25.32 public:0.25.31 public:0.25.30 public:0.25.30-dev public:0.25.29 public:0.25.28 public:0.25.27 public:0.25.26 public:0.25.25 public:0.25.24 public:0.25.24.beta3 public:0.25.24.beta2 public:0.25.24.beta1 public:0.25.23 public:0.25.23.beta1 public:0.25.22 public:0.25.21 public:0.25.21.beta2 public:0.25.21.beta1 public:0.25.20 public:0.25.19 public:0.25.18 public:0.25.17 public:0.25.16 public:0.25.15 public:0.25.14 public:0.25.13 public:0.25.12 public:0.25.11 public:0.25.10 public:0.25.9 public:0.25.8 public:0.25.7 public:0.25.6 public:0.25.5 public:0.25.4 public:0.25.3 public:0.25.2 public:0.25.1 public:0.25.0 public:0.25.0-beta1 public:0.24.31 public:0.24.30 public:0.24.29 public:0.24.28 public:0.24.27 public:0.24.26 public:0.24.25 public:0.24.24 public:0.24.23 public:0.24.22 public:0.24.21 public:0.24.20 public:0.24.19 public:0.24.18 public:0.24.17 public:0.24.16 public:0.24.15 public:0.24.14 public:0.24.13 public:0.24.12 public:0.24.11 public:0.24.10 public:0.24.9 public:0.24.8 public:0.24.7 public:0.24.6 public:0.24.5 public:0.24.4 public:0.24.3 public:0.24.2 public:0.24.1 public:0.24.0 public:0.24.0.dev-rc8 public:0.24.0.dev-rc7 public:0.24.0.dev-rc6 public:0.24.0.dev-rc5 public:0.24.0.dev-rc4 public:0.24.0.dev-rc3 public:0.24.0.dev-rc2 public:0.23.23 public:0.23.22 public:0.23.21 public:0.23.20 public:0.23.19 public:0.23.18 public:0.23.17 public:0.23.16 public:0.23.15 public:0.23.14 public:0.23.13 public:0.23.12 public:0.23.11 public:0.23.10 public:0.23.9 public:0.23.8 public:0.23.7 public:0.23.6 public:0.23.5 public:0.23.4 public:0.23.3 public:0.23.2 public:0.23.1 public:0.23.0 public:0.22.19 public:0.22.18 public:0.22.17 public:0.22.16 public:0.22.15 public:0.22.14 public:0.22.13 public:0.22.12 public:0.22.11 public:0.22.10 public:0.22.9 public:0.22.8 public:0.22.7 public:0.22.6 public:0.22.5 public:0.22.4 public:0.22.3 public:0.22.2 public:0.22.1 public:0.22.0 public:0.21.5 public:0.21.4 public:0.21.3 public:0.21.2 public:0.21.1 public:0.21.0 public:0.20.7 public:0.20.6 public:0.20.5 public:0.20.4 public:0.20.3 public:0.20.2 public:0.20.1 public:0.20.0 public:0.19.23 public:0.19.22 public:0.19.21 public:0.19.20 public:0.19.19 public:0.19.18 public:0.19.17 public:0.19.16 public:0.19.15 public:0.19.14 public:0.19.13 public:0.19.12 public:0.19.11 public:0.19.10 public:0.19.9 public:0.19.8 public:0.19.7 public:0.19.6 public:0.19.5 public:0.19.4 public:0.19.3 public:0.19.2 public:0.19.1 public:0.19.0 public:0.18.6 public:0.18.5 public:0.18.4 public:0.18.3 public:0.18.2 public:0.18.1 public:0.18.0 public:0.17.34 public:0.17.33 public:0.17.32 public:0.17.31 public:0.17.30 public:0.17.29 public:0.17.28 public:0.17.27 public:0.17.26 public:0.17.25 public:0.17.24 public:0.17.23 public:0.17.22 public:0.17.21 public:0.17.20 public:0.17.19 public:0.17.18 public:0.17.17 public:0.17.16 public:0.17.15 public:0.17.14 public:0.17.13 public:0.17.12 public:0.17.11 public:0.17.10 public:0.17.9 public:0.17.8 public:0.17.7 public:0.17.6 public:0.17.5 public:0.17.4 public:0.17.3 public:0.17.2 public:0.17.1 public:0.17.0 public:0.16.8 public:0.16.7 public:0.16.6 public:0.16.5 public:0.16.4 public:0.16.3 public:0.16.2 public:0.16.1 public:0.16.0 public:0.15.10 public:0.15.9 public:0.15.8 public:0.15.7 public:0.15.6 public:0.15.5 public:0.15.4 public:0.15.3 public:0.15.2 public:0.15.1 public:0.15.0 public:0.14.8 public:0.14.7 public:0.14.6 public:0.14.5 public:0.14.4 public:0.14.3 public:0.14.2 public:0.14.1 public:0.14.0 public:0.13.4 public:0.13.3 public:0.13.2 public:0.13.1 public:0.13.0 public:0.12.5 public:0.12.4 public:0.12.3 public:0.12.2 public:0.12.1 public:0.12.0 public:0.11.10 public:0.11.9 public:0.11.8 public:0.11.7 public:0.11.6 public:0.11.5 public:0.11.4 public:0.11.3 public:0.11.2 public:0.11.1 public:0.11.0 public:0.10.1 public:0.10.0 public:0.9.0 public:0.8.8 public:0.8.7 public:0.8.6 public:0.8.5 public:0.8.3 public:0.8.2 public:0.8.1 public:0.8.0 public:0.7.2 public:0.7.1 public:0.7.0 public:0.6.1 public:0.6.0 public:0.5.0 public:0.4.1 public:0.4.0 public:0.3.1 public:0.3.0 public:0.2.1 public:0.2.0 public:0.1.26 public:0.1.25 public:0.1.24 public:0.1.23 public:0.1.22 public:0.1.21 public:0.1.20 public:0.1.19 public:0.1.18 public:0.1.17 public:0.1.16 public:0.1.15 public:0.1.14 public:0.1.13 public:0.1.12 public:0.1.11 public:0.1.10 public:0.1.9 public:0.1.8 public:0.1.7 public:0.1.6 public:0.1.5 public:0.1.4 public:0.1.3 public:0.1.2 public:0.1.1 public:0.1.0 public:0.0.9 public:0.0.8 public:0.0.7 public:0.0.6
...
compare: public:0.14.7
Branches Tags
public:beta public:main public:snyk-upgrade-7fd31e6f474568014502e7205847f257 public:snyk-fix-90f3b71450f7543a1c61724d8afa0e71 public:refactor_services_2 public:refactor_services public:dev public:snyk-upgrade-fe9625385aa4be849c7157033fd2abb0 public:snyk-upgrade-99a31e70c822894595544b0c9813b11a public:svelteui public:disenchant public:snyk-fix-61f76ef0bce2a276df9552c2fc80a14f public:snyk-upgrade-2baa7938ba318464c3bad5c3fd5cbda5 public:snyk-upgrade-7af62a575eab20e8fb4e9565bf1adcfc public:snyk-upgrade-b32f281b31a81483f126b8fb6c54a957 public:snyk-upgrade-bd61b1103442d3faecc9fc726c34e9a4 public:snyk-upgrade-5b2d6584430c061b5187611a5d7fb64a public:snyk-fix-0e9fbbcb374fcb8d5e8cb027d8414fb7 public:snyk-upgrade-5242daf91e706d1ae500c9ea6545636e public:snyk-upgrade-70434a6822fbad83424135dfad9df909 public:snyk-upgrade-8fa01ccad0352125f1866d41e57d35e1 public:snyk-upgrade-39eb97b4655e70091f718a88fccd57c1 public:snyk-upgrade-2b99d54d19e207887e36c3071b796925 public:forprivacy public:snyk-upgrade-dad9088d0a333677e0157e3deef7fc38 public:snyk-upgrade-b9a76d74b3e5723759af5b3319aabb80 public:snyk-upgrade-7da6459d8ff5d293d80de9d80061d912 public:snyk-upgrade-6a3786fc4cbe394542e9d1b6058c46e7 public:snyk-upgrade-8fc9e62771dc41b0f23dc0efefb5c949
public:0.25.43-patched-9 public:0.25.43-patched-8 public:0.25.43-patched-7 public:0.25.43-patched-6 public:0.25.43-patched-5 public:0.25.43-patched-4 public:0.25.43-patched-3 public:0.25.43-patched-2 public:0.25.43-patched-1 public:0.25.43 public:0.25.42 public:0.25.41-patched-3 public:0.25.41-patched-2 public:0.25.41-patched-1 public:0.25.41 public:0.25.40 public:0.25.39 public:0.25.38 public:0.25.37 public:0.25.36 public:0.25.35 public:0.25.34 public:0.25.33 public:0.25.32 public:0.25.31 public:0.25.30 public:0.25.30-dev public:0.25.29 public:0.25.28 public:0.25.27 public:0.25.26 public:0.25.25 public:0.25.24 public:0.25.24.beta3 public:0.25.24.beta2 public:0.25.24.beta1 public:0.25.23 public:0.25.23.beta1 public:0.25.22 public:0.25.21 public:0.25.21.beta2 public:0.25.21.beta1 public:0.25.20 public:0.25.19 public:0.25.18 public:0.25.17 public:0.25.16 public:0.25.15 public:0.25.14 public:0.25.13 public:0.25.12 public:0.25.11 public:0.25.10 public:0.25.9 public:0.25.8 public:0.25.7 public:0.25.6 public:0.25.5 public:0.25.4 public:0.25.3 public:0.25.2 public:0.25.1 public:0.25.0 public:0.25.0-beta1 public:0.24.31 public:0.24.30 public:0.24.29 public:0.24.28 public:0.24.27 public:0.24.26 public:0.24.25 public:0.24.24 public:0.24.23 public:0.24.22 public:0.24.21 public:0.24.20 public:0.24.19 public:0.24.18 public:0.24.17 public:0.24.16 public:0.24.15 public:0.24.14 public:0.24.13 public:0.24.12 public:0.24.11 public:0.24.10 public:0.24.9 public:0.24.8 public:0.24.7 public:0.24.6 public:0.24.5 public:0.24.4 public:0.24.3 public:0.24.2 public:0.24.1 public:0.24.0 public:0.24.0.dev-rc8 public:0.24.0.dev-rc7 public:0.24.0.dev-rc6 public:0.24.0.dev-rc5 public:0.24.0.dev-rc4 public:0.24.0.dev-rc3 public:0.24.0.dev-rc2 public:0.23.23 public:0.23.22 public:0.23.21 public:0.23.20 public:0.23.19 public:0.23.18 public:0.23.17 public:0.23.16 public:0.23.15 public:0.23.14 public:0.23.13 public:0.23.12 public:0.23.11 public:0.23.10 public:0.23.9 public:0.23.8 public:0.23.7 public:0.23.6 public:0.23.5 public:0.23.4 public:0.23.3 public:0.23.2 public:0.23.1 public:0.23.0 public:0.22.19 public:0.22.18 public:0.22.17 public:0.22.16 public:0.22.15 public:0.22.14 public:0.22.13 public:0.22.12 public:0.22.11 public:0.22.10 public:0.22.9 public:0.22.8 public:0.22.7 public:0.22.6 public:0.22.5 public:0.22.4 public:0.22.3 public:0.22.2 public:0.22.1 public:0.22.0 public:0.21.5 public:0.21.4 public:0.21.3 public:0.21.2 public:0.21.1 public:0.21.0 public:0.20.7 public:0.20.6 public:0.20.5 public:0.20.4 public:0.20.3 public:0.20.2 public:0.20.1 public:0.20.0 public:0.19.23 public:0.19.22 public:0.19.21 public:0.19.20 public:0.19.19 public:0.19.18 public:0.19.17 public:0.19.16 public:0.19.15 public:0.19.14 public:0.19.13 public:0.19.12 public:0.19.11 public:0.19.10 public:0.19.9 public:0.19.8 public:0.19.7 public:0.19.6 public:0.19.5 public:0.19.4 public:0.19.3 public:0.19.2 public:0.19.1 public:0.19.0 public:0.18.6 public:0.18.5 public:0.18.4 public:0.18.3 public:0.18.2 public:0.18.1 public:0.18.0 public:0.17.34 public:0.17.33 public:0.17.32 public:0.17.31 public:0.17.30 public:0.17.29 public:0.17.28 public:0.17.27 public:0.17.26 public:0.17.25 public:0.17.24 public:0.17.23 public:0.17.22 public:0.17.21 public:0.17.20 public:0.17.19 public:0.17.18 public:0.17.17 public:0.17.16 public:0.17.15 public:0.17.14 public:0.17.13 public:0.17.12 public:0.17.11 public:0.17.10 public:0.17.9 public:0.17.8 public:0.17.7 public:0.17.6 public:0.17.5 public:0.17.4 public:0.17.3 public:0.17.2 public:0.17.1 public:0.17.0 public:0.16.8 public:0.16.7 public:0.16.6 public:0.16.5 public:0.16.4 public:0.16.3 public:0.16.2 public:0.16.1 public:0.16.0 public:0.15.10 public:0.15.9 public:0.15.8 public:0.15.7 public:0.15.6 public:0.15.5 public:0.15.4 public:0.15.3 public:0.15.2 public:0.15.1 public:0.15.0 public:0.14.8 public:0.14.7 public:0.14.6 public:0.14.5 public:0.14.4 public:0.14.3 public:0.14.2 public:0.14.1 public:0.14.0 public:0.13.4 public:0.13.3 public:0.13.2 public:0.13.1 public:0.13.0 public:0.12.5 public:0.12.4 public:0.12.3 public:0.12.2 public:0.12.1 public:0.12.0 public:0.11.10 public:0.11.9 public:0.11.8 public:0.11.7 public:0.11.6 public:0.11.5 public:0.11.4 public:0.11.3 public:0.11.2 public:0.11.1 public:0.11.0 public:0.10.1 public:0.10.0 public:0.9.0 public:0.8.8 public:0.8.7 public:0.8.6 public:0.8.5 public:0.8.3 public:0.8.2 public:0.8.1 public:0.8.0 public:0.7.2 public:0.7.1 public:0.7.0 public:0.6.1 public:0.6.0 public:0.5.0 public:0.4.1 public:0.4.0 public:0.3.1 public:0.3.0 public:0.2.1 public:0.2.0 public:0.1.26 public:0.1.25 public:0.1.24 public:0.1.23 public:0.1.22 public:0.1.21 public:0.1.20 public:0.1.19 public:0.1.18 public:0.1.17 public:0.1.16 public:0.1.15 public:0.1.14 public:0.1.13 public:0.1.12 public:0.1.11 public:0.1.10 public:0.1.9 public:0.1.8 public:0.1.7 public:0.1.6 public:0.1.5 public:0.1.4 public:0.1.3 public:0.1.2 public:0.1.1 public:0.1.0 public:0.0.9 public:0.0.8 public:0.0.7 public:0.0.6

23 Commits
0.13.3 ... 0.14.7

Author SHA1 Message Date
vorotamoroz
b14ecdb205 Bump 2022-09-06 17:02:58 +09:00
vorotamoroz
21362adb5b Typos 2022-09-06 14:32:09 +09:00
vorotamoroz
f8c1474700 Refactored 2022-09-06 13:42:12 +09:00
vorotamoroz
b35052a485 bump 2022-09-05 16:55:35 +09:00
vorotamoroz
c367d35e09 Target ES2018 2022-09-05 16:55:29 +09:00
vorotamoroz
2a5078cdbb bump 2022-09-05 16:54:06 +09:00
vorotamoroz
8112a07210 Implemented: 
- Auto chunk size adjusting.
  Now our large files are processed more efficiently
- These configuration has been removed.

Improved
- Remote chunk retrieving logic has been speeded up.

Fixed
- Fixed process handling of boot sequence
 2022-09-05 16:53:22 +09:00
vorotamoroz
c9daa1b47d Fixed issue of importing configurations. 2022-09-04 01:16:29 +09:00
vorotamoroz
73ac93e8c5 bump 2022-09-04 01:08:09 +09:00
vorotamoroz
8d2b9eff37 Improved: 
- New test items have been added to `Check database configuration`
 2022-09-04 01:08:02 +09:00
vorotamoroz
0ee32a2147 bump 2022-09-03 16:44:51 +09:00
vorotamoroz
ac3c78e198 Fixed 
- Could not retrieve files if synchronisation has been interrupted or failed
 2022-09-03 16:43:59 +09:00
vorotamoroz
0da1e3d9c8 bump 2022-08-30 15:24:38 +09:00
vorotamoroz
8f021a3c93 Improved: 
- Use local chunks in preference to remote them if present.
 2022-08-30 15:24:26 +09:00
vorotamoroz
6db0743096 Update release.yml 2022-08-29 16:50:53 +09:00
vorotamoroz
0e300a0a6b bump 2022-08-29 16:48:35 +09:00
vorotamoroz
9d0ffd1848 Implemented: 
- The target selecting filter was implemented.
- We can configure size of chunks.
- Read chunks online.

Fixed:
- Typos
 2022-08-29 16:32:14 +09:00
vorotamoroz
e7f4d8c9c2 Add error handling for loading the document 2022-08-29 16:13:54 +09:00
vorotamoroz
ca36e1b663 Merge branch 'main' of https://github.com/vrtmrz/obsidian-livesync 2022-08-09 17:10:19 +09:00
vorotamoroz
8f583e3680 Fixed: 
- Now, we can synchronise hidden files that conflicted on each devices.
Enhanced:
- We can search for conflicting docs.
- Pending processes can now be run at any time.
- Performance improved on synchronising large numbers of files at once.
 2022-08-09 17:10:08 +09:00
vorotamoroz
98407cf72f Merge pull request #101 from Miigon/chinese-doc 
Add Chinese documentations
 2022-08-09 17:06:39 +09:00
Miigon
1f377cdf67 add setup_own_server_cn.md 2022-08-09 13:06:35 +08:00
Miigon
3a965e74da add chinese README_cn.md 2022-08-09 12:38:43 +08:00
18 changed files with 1080 additions and 445 deletions
Expand all files Collapse all files

2
.github/workflows/release.yml vendored
View File

@@ -22,7 +22,7 @@ jobs:
- name: Get Version
id: version
run: |
echo "::set-output name=tag::$(git describe --abbrev=0)"
echo "::set-output name=tag::$(git describe --abbrev=0 --tags)"
# Build the plugin
- name: Build
id: build

1
.gitignore vendored
View File

@@ -12,3 +12,4 @@ main.js
# obsidian
data.json
.vscode

8
README.md
View File

@@ -1,6 +1,6 @@
# Self-hosted LiveSync
[Japanese docs](./README_ja.md).
[Japanese docs](./README_ja.md) [Chinese docs](./README_cn.md).
Self-hosted LiveSync is a community implemented synchronization plugin.
A self-hosted or purchased CouchDB acts as the intermediate server. Available on every obsidian-compatible platform.
@@ -43,7 +43,7 @@ Note: More information about alternative hosting methods needed! Currently, [usi
### First device
1. Install the plugin on your device.
2. Configure remote database infomation.
2. Configure remote database information.
1. Fill your server's information into the `Remote Database configuration` pane.
2. Enabling `End to End Encryption` is recommended. After entering a passphrase, click `Apply`.
3. Click `Test Database Connection` and make sure that the plugin says `Connected to (your-database-name)`.
@@ -53,7 +53,7 @@ Note: More information about alternative hosting methods needed! Currently, [usi
2. Or, set up the synchronization as you like. By default, none of the settings are enabled, meaning you would need to manually trigger the synchronization process.
3. Additional configurations are also here. I recommend enabling `Use Trash for deleted files`, but you can also leave all configurations as-is.
4. Configure miscellaneous features.
1. Enabling `Show staus inside editor` shows status at the top-right corner of the editor while in editing mode. (Recommended)
1. Enabling `Show status inside editor` shows status at the top-right corner of the editor while in editing mode. (Recommended)
5. Go back to the editor. Wait for the initial scan to complete.
6. When the status no longer changes and shows a ⏹️ for COMPLETED (No ⏳ and 🧩 icons), you are ready to synchronize with the server.
7. Press the replicate icon on the Ribbon or run `Replicate now` from the command palette. This will send all your data to the server.
@@ -115,7 +115,7 @@ If you have deleted or renamed files, please wait until ⏳ icon disappeared.
- While synchronizing, files are compared by their modification time and the older ones will be overwritten by the newer ones. Then plugin checks for conflicts and if a merge is needed, a dialog will open.
- Rarely, a file in the database could be corrupted. The plugin will not write to local storage when a file looks corrupted. If a local version of the file is on your device, the corruption could be fixed by editing the local file and synchronizing it. But if the file does not exist on any of your devices, then it can not be rescued. In this case you can delete these items from the settings dialog.
- If your database looks corrupted, try "Drop History". Usually, It is the easiest way.
- To stop the bootup sequence (eg. for fixing problems on databases), you can put a `redflag.md` file at the root of your vault.
- To stop the boot up sequence (eg. for fixing problems on databases), you can put a `redflag.md` file at the root of your vault.
- Q: Database is growing, how can I shrink it down?
A: each of the docs is saved with their past 100 revisions for detecting and resolving conflicts. Picturing that one device has been offline for a while, and comes online again. The device has to compare its notes with the remotely saved ones. If there exists a historic revision in which the note used to be identical, it could be updated safely (like git fast-forward). Even if that is not in revision histories, we only have to check the differences after the revision that both devices commonly have. This is like git's conflict resolving method. So, We have to make the database again like an enlarged git repo if you want to solve the root of the problem.
- And more technical Information are in the [Technical Information](docs/tech_info.md)

130
README_cn.md Normal file
View File

@@ -0,0 +1,130 @@
# Self-hosted LiveSync
Self-hosted LiveSync (自搭建在线同步) 是一个社区实现的在线同步插件。
使用一个自搭建的或者购买的 CouchDB 作为中转服务器。兼容所有支持 Obsidian 的平台。
注意: 本插件与官方的 "Obsidian Sync" 服务不兼容。
![obsidian_live_sync_demo](https://user-images.githubusercontent.com/45774780/137355323-f57a8b09-abf2-4501-836c-8cb7d2ff24a3.gif)
安装或升级 LiveSync 之前，请备份你的 vault。
## 功能
- 可视化的冲突解决器
- 接近实时的多设备双向同步
- 可使用 CouchDB 以及兼容的服务，如 IBM Cloudant
- 支持端到端加密
- 插件同步 (Beta)
- 从 [obsidian-livesync-webclip](https://chrome.google.com/webstore/detail/obsidian-livesync-webclip/jfpaflmpckblieefkegjncjoceapakdf) 接收 WebClip (本功能不适用端到端加密)
适用于出于安全原因需要将笔记完全自托管的研究人员、工程师或开发人员，以及任何喜欢笔记完全私密所带来的安全感的人。
## 重要提醒
- 请勿与其他同步解决方案（包括 iCloud、Obsidian Sync）一起使用。在启用此插件之前，请确保禁用所有其他同步方法以避免内容损坏或重复。如果要同步到多个服务，请一一进行，切勿同时启用两种同步方法。
这包括不能将您的保管库放在云同步文件夹中（例如 iCloud 文件夹或 Dropbox 文件夹）
- 这是一个同步插件，不是备份解决方案。不要依赖它进行备份。
- 如果设备的存储空间耗尽，可能会发生数据库损坏。
- 隐藏文件或任何其他不可见文件不会保存在数据库中，因此不会被同步。（**并且可能会被删除**）
## 如何使用
### 准备好你的数据库
首先，准备好你的数据库。IBM Cloudant 是用于测试的首选。或者，您也可以在自己的服务器上安装 CouchDB。有关更多信息，请参阅以下内容：
1. [Setup IBM Cloudant](docs/setup_cloudant.md)
2. [Setup your CouchDB](docs/setup_own_server_cn.md)
Note: 正在征集更多搭建方法！目前在讨论的有 [使用 fly.io](https://github.com/vrtmrz/obsidian-livesync/discussions/85)。
### 第一个设备
1. 在您的设备上安装插件。
2. 配置远程数据库信息。
1. 将您的服务器信息填写到 `Remote Database configuration`（远程数据库配置）设置页中。
2. 建议启用 `End to End Encryption`（端到端加密）。输入密码后，单击“应用”。
3. 点击 `Test Database Connection` 并确保插件显示 `Connected to (你的数据库名称)`。
4. 单击 `Check database configuration`（检查数据库配置）并确保所有测试均已通过。
3. 在 `Sync Settings`（同步设置）选项卡中配置何时进行同步。（您也可以稍后再设置）
1. 如果要实时同步，请启用 `LiveSync`。
2. 或者，根据您的需要设置同步方式。默认情况下，不会启用任何自动同步，这意味着您需要手动触发同步过程。
3. 其他配置也在这里。建议启用 `Use Trash for deleted files`（删除文件到回收站），但您也可以保持所有配置不变。
4. 配置杂项功能。
1. 启用 `Show staus inside editor` 会在编辑器右上角显示状态。（推荐开启）
5. 回到编辑器。等待初始扫描完成。
6. 当状态不再变化并显示 ⏹️ 图标表示 COMPLETED（没有 ⏳ 和 🧩 图标）时，您就可以与服务器同步了。
7. 按功能区上的复制图标或从命令面板运行 `Replicate now`（立刻复制）。这会将您的所有数据发送到服务器。
8. 打开命令面板，运行 `Copy setup URI`（复制设置链接），并设置密码。这会将您的配置导出到剪贴板，作为您导入其他设备的链接。
**重要: 不要公开本链接，这个链接包含了你的所有认证信息！** (即使没有密码别人读不了)
### 后续设备
注意：如果要与非空的 vault 进行同步，文件的修改日期和时间必须互相匹配。否则，可能会发生额外的传输或文件可能会损坏。
为简单起见，我们强烈建议同步到一个全空的 vault。
1. 安装插件。
2. 打开您从第一台设备导出的链接。
3. 插件会询问您是否确定应用配置。 回答 `Yes`，然后按照以下说明进行操作：
1. 对 `Keep local DB?` 回答 `Yes`。
*注意：如果您希望保留本地现有 vault，则必须对此问题回答 `No`，并对 `Rebuild the database?` 回答 `No`。*
2. 对 `Keep remote DB?` 回答 `Yes`。
3. 对 `Replicate once?` 回答 `Yes`。
完成后，您的所有设置将会从第一台设备成功导入。
4. 你的笔记应该很快就会同步。
## 文件看起来有损坏...
请再次打开配置链接并回答如下：
- 如果您的本地数据库看起来已损坏（当你的本地 Obsidian 文件看起来很奇怪）
- 对 `Keep local DB?` 回答 `No`
- 如果您的远程数据库看起来已损坏（当复制时发生中断）
- 对 `Keep remote DB?` 回答 `No`
如果您对两者都回答“否”，您的数据库将根据您设备上的内容重建。并且远程数据库将锁定其他设备，您必须再次同步所有设备。（此时，几乎所有文件都会与时间戳同步。因此您可以安全地使用现有的 vault）。
## 测试服务器
设置 Cloudant 或本地 CouchDB 实例有点复杂，所以我搭建了一个 [self-hosted-livesync 尝鲜服务器](https://olstaste.vrtmrz.net/)。欢迎免费尝试！
注意：请仔细阅读“限制”条目。不要发送您的私人 vault。
## 状态栏信息
同步状态将显示在状态栏。
- 状态
- ⏹️ 就绪
- 💤 LiveSync 已启用，正在等待更改。
- ⚡️ 同步中。
- ⚠ 一个错误出现了。
- ↑ 上传的 chunk 和元数据数量
- ↓ 下载的 chunk 和元数据数量
- ⏳ 等待的过程的数量
- 🧩 正在等待 chunk 的文件数量
如果你删除或更名了文件，请等待 ⏳ 图标消失。
## 提示
- 如果文件夹在复制后变为空，则默认情况下该文件夹会被删除。您可以关闭此行为。检查 [设置](docs/settings.md)。
- LiveSync 模式在移动设备上可能导致耗电量增加。建议使用定期同步 + 条件自动同步。
- 移动平台上的 Obsidian 无法连接到非安全 (HTTP) 或本地签名的服务器，即使设备上安装了根证书。
- 没有类似“exclude_folders”的配置。
- 同步时，文件按修改时间进行比较，较旧的将被较新的文件覆盖。然后插件检查冲突，如果需要合并，将打开一个对话框。
- 数据库中的文件在罕见情况下可能会损坏。当接收到的文件看起来已损坏时，插件不会将其写入本地存储。如果您的设备上有文件的本地版本，则可以通过编辑本地文件并进行同步来覆盖损坏的版本。但是，如果您的任何设备上都不存在该文件，则无法挽救该文件。在这种情况下，您可以从设置对话框中删除这些损坏的文件。
- 如果您的数据库看起来已损坏，请尝试 "Drop History"（“删除历史记录”）。通常，这是最简单的方法。
- 要阻止插件的启动流程（例如，为了修复数据库问题），您可以在 vault 的根目录创建一个 "redflag.md" 文件。
- 问：数据库在增长，我该如何缩小它？
答：每个文档都保存了过去 100 次修订，用于检测和解决冲突。想象一台设备已经离线一段时间，然后再次上线。设备必须将其笔记与远程保存的笔记进行比较。如果存在曾经相同的历史修订，则可以安全地直接更新这个文件（和 git 的快进原理一样）。即使文件不在修订历史中，我们也只需检查两个设备上该文件的公有修订版本之后的差异。这就像 git 的冲突解决方法。所以，如果想从根本上解决数据库太大的问题，我们像构建一个扩大版的 git repo 一样去重新设计数据库。
- 更多技术信息在 [技术信息](docs/tech_info.md)
- 如果你想在没有黑曜石的情况下同步文件，你可以使用[filesystem-livesync](https://github.com/vrtmrz/filesystem-livesync)。
- WebClipper 也可在 Chrome Web Store 上使用：[obsidian-livesync-webclip](https://chrome.google.com/webstore/detail/obsidian-livesync-webclip/jfpaflmpckblieefkegjncjoceapakdf)
仓库地址：[obsidian-livesync-webclip](https://github.com/vrtmrz/obsidian-livesync-webclip) （文档施工中）
## License
The source code is licensed under the MIT License.
本源代码使用 MIT 协议授权。

95
docs/setup_own_server_cn.md Normal file
View File

@@ -0,0 +1,95 @@
# 在你自己的服务器上设置 CouchDB
> 注：提供了 [docker-compose.yml 和 ini 文件](https://github.com/vrtmrz/self-hosted-livesync-server) 可以同时启动 Caddy 和 CouchDB。推荐直接使用该 docker-compose 配置进行搭建。（若使用，请查阅链接中的文档，而不是这个文档）
## 安装 CouchDB 并从 PC 或 Mac 上访问
设置 CouchDB 的最简单方法是使用 [CouchDB docker image]((https://hub.docker.com/_/couchdb)).
需要修改一些 `local.ini` 中的配置，以让它可以用于 Self-hosted LiveSync，如下：
```
[couchdb]
single_node=true
[chttpd]
require_valid_user = true
[chttpd_auth]
require_valid_user = true
authentication_redirect = /_utils/session.html
[httpd]
WWW-Authenticate = Basic realm="couchdb"
enable_cors = true
[cors]
origins = app://obsidian.md,capacitor://localhost,http://localhost
credentials = true
headers = accept, authorization, content-type, origin, referer
methods = GET, PUT, POST, HEAD, DELETE
max_age = 3600
```
创建 `local.ini` 并用如下指令启动 CouchDB：
```
$ docker run --rm -it -e COUCHDB_USER=admin -e COUCHDB_PASSWORD=password -v .local.ini:/opt/couchdb/etc/local.ini -p 5984:5984 couchdb
```
Note: 此时 local.ini 的文件所有者会变成 5984:5984。这是 docker 镜像的限制，请修改文件所有者后再编辑 local.ini。
在确定 Self-hosted LiveSync 可以和服务器同步后，可以后台启动 docker 镜像：
```
$ docker run -d --restart always -e COUCHDB_USER=admin -e COUCHDB_PASSWORD=password -v .local.ini:/opt/couchdb/etc/local.ini -p 5984:5984 couchdb
```
## 从移动设备访问
如果你想要从移动设备访问 Self-hosted LiveSync，你需要一个合法的 SSL 证书。
### 移动设备测试
测试时，[localhost.run](http://localhost.run/) 这一类的反向隧道服务很实用。（非必须，只是用于终端设备不方便 ssh 的时候的备选方案）
```
$ ssh -R 80:localhost:5984 nokey@localhost.run
Warning: Permanently added the RSA host key for IP address '35.171.254.69' to the list of known hosts.
===============================================================================
Welcome to localhost.run!
Follow your favourite reverse tunnel at [https://twitter.com/localhost_run].
**You need a SSH key to access this service.**
If you get a permission denied follow Gitlab's most excellent howto:
https://docs.gitlab.com/ee/ssh/
*Only rsa and ed25519 keys are supported*
To set up and manage custom domains go to https://admin.localhost.run/
More details on custom domains (and how to enable subdomains of your custom
domain) at https://localhost.run/docs/custom-domains
To explore using localhost.run visit the documentation site:
https://localhost.run/docs/
===============================================================================
** your connection id is xxxxxxxxxxxxxxxxxxxxxxxxxxxx, please mention it if you send me a message about an issue. **
xxxxxxxx.localhost.run tunneled with tls termination, https://xxxxxxxx.localhost.run
Connection to localhost.run closed by remote host.
Connection to localhost.run closed.
```
https://xxxxxxxx.localhost.run 即为临时服务器地址。
### 设置你的域名
设置一个指向你服务器的 A 记录，并根据需要设置反向代理。
Note: 不推荐将 CouchDB 挂载到根目录
可以使用 Caddy 很方便的给服务器加上 SSL 功能
提供了 [docker-compose.yml 和 ini 文件](https://github.com/vrtmrz/self-hosted-livesync-server) 可以同时启动 Caddy 和 CouchDB。
注意检查服务器日志，当心恶意访问。

2
esbuild.config.mjs
View File

@@ -29,7 +29,7 @@ esbuild
external: ["obsidian", "electron", ...builtins],
format: "cjs",
watch: !prod,
target: "es2015",
target: "es2018",
logLevel: "info",
sourcemap: prod ? false : "inline",
treeShaking: true,

2
manifest.json
View File

@@ -1,7 +1,7 @@
{
"id": "obsidian-livesync",
"name": "Self-hosted LiveSync",
"version": "0.13.3",
"version": "0.14.7",
"minAppVersion": "0.9.12",
"description": "Community implementation of self-hosted livesync. Reflect your vault changes to some other devices immediately. Please make sure to disable other synchronize solutions to avoid content corruption or duplication.",
"author": "vorotamoroz",

4
package-lock.json generated
View File

@@ -1,12 +1,12 @@
{
"name": "obsidian-livesync",
"version": "0.13.3",
"version": "0.14.7",
"lockfileVersion": 2,
"requires": true,
"packages": {
"": {
"name": "obsidian-livesync",
"version": "0.13.3",
"version": "0.14.7",
"license": "MIT",
"dependencies": {
"diff-match-patch": "^1.0.5",

2
package.json
View File

@@ -1,6 +1,6 @@
{
"name": "obsidian-livesync",
"version": "0.13.3",
"version": "0.14.7",
"description": "Reflect your vault changes to some other devices immediately. Please make sure to disable other synchronize solutions to avoid content corruption or duplication.",
"main": "main.js",
"type": "module",

27
src/DocumentHistoryModal.ts
View File

@@ -31,14 +31,25 @@ export class DocumentHistoryModal extends Modal {
}
async loadFile() {
const db = this.plugin.localDatabase;
const w = await db.localDatabase.get(path2id(this.file), { revs_info: true });
this.revs_info = w._revs_info.filter((e) => e.status == "available");
this.range.max = `${this.revs_info.length - 1}`;
this.range.value = this.range.max;
this.fileInfo.setText(`${this.file} / ${this.revs_info.length} revisions`);
await this.loadRevs();
try {
const w = await db.localDatabase.get(path2id(this.file), { revs_info: true });
this.revs_info = w._revs_info.filter((e) => e.status == "available");
this.range.max = `${this.revs_info.length - 1}`;
this.range.value = this.range.max;
this.fileInfo.setText(`${this.file} / ${this.revs_info.length} revisions`);
await this.loadRevs();
} catch (ex) {
if (ex.status && ex.status == 404) {
this.range.max = "0";
this.range.value = "";
this.range.disabled = true;
this.showDiff
this.contentView.setText(`History of this file was not recorded.`);
}
}
}
async loadRevs() {
if (this.revs_info.length == 0) return;
const db = this.plugin.localDatabase;
const index = this.revs_info.length - 1 - (this.range.value as any) / 1;
const rev = this.revs_info[index];
@@ -154,7 +165,7 @@ export class DocumentHistoryModal extends Modal {
const leaf = app.workspace.getLeaf(false);
await leaf.openFile(targetFile);
} else {
Logger("The file cound not view on the editor", LOG_LEVEL.NOTICE)
Logger("The file could not view on the editor", LOG_LEVEL.NOTICE)
}
}
buttons.createEl("button", { text: "Back to this revision" }, (e) => {
@@ -162,7 +173,7 @@ export class DocumentHistoryModal extends Modal {
e.addEventListener("click", async () => {
const pathToWrite = this.file.startsWith("i:") ? this.file.substring("i:".length) : this.file;
if (!isValidPath(pathToWrite)) {
Logger("Path is not vaild to write content.", LOG_LEVEL.INFO);
Logger("Path is not valid to write content.", LOG_LEVEL.INFO);
}
if (this.currentDoc?.datatype == "plain") {
await this.app.vault.adapter.write(pathToWrite, this.currentDoc.data);

302
src/LocalPouchDB.ts
View File

@@ -24,14 +24,14 @@ import {
} from "./lib/src/types";
import { RemoteDBSettings } from "./lib/src/types";
import { resolveWithIgnoreKnownError, runWithLock, shouldSplitAsPlainText, splitPieces2, enableEncryption } from "./lib/src/utils";
import { path2id } from "./utils";
import { id2path, path2id } from "./utils";
import { Logger } from "./lib/src/logger";
import { checkRemoteVersion, connectRemoteCouchDBWithSetting, getLastPostFailedBySize } from "./utils_couchdb";
import { checkRemoteVersion, connectRemoteCouchDBWithSetting, getLastPostFailedBySize, putDesignDocuments } from "./utils_couchdb";
import { KeyValueDatabase, OpenKeyValueDatabase } from "./KeyValueDB";
import { LRUCache } from "./lib/src/LRUCache";
// when replicated, LiveSync checks chunk versions that every node used.
// If all minumum version of every devices were up, that means we can convert database automatically.
// If all minimum version of every devices were up, that means we can convert database automatically.
const currentVersionRange: ChunkVersionRange = {
min: 0,
@@ -72,6 +72,7 @@ export class LocalPouchDB {
chunkVersion = -1;
maxChunkVersion = -1;
minChunkVersion = -1;
needScanning = false;
cancelHandler<T extends PouchDB.Core.Changes<EntryDoc> | PouchDB.Replication.Sync<EntryDoc> | PouchDB.Replication.Replication<EntryDoc>>(handler: T): T {
if (handler != null) {
@@ -160,8 +161,9 @@ export class LocalPouchDB {
this.localDatabase.removeAllListeners();
});
this.nodeid = nodeinfo.nodeid;
await putDesignDocuments(this.localDatabase);
// Traceing the leaf id
// Tracings the leaf id
const changes = this.localDatabase
.changes({
since: "now",
@@ -186,7 +188,7 @@ export class LocalPouchDB {
const oi = await old.info();
if (oi.doc_count == 0) {
Logger("Old database is empty, proceed to next step", LOG_LEVEL.VERBOSE);
// aleady converted.
// already converted.
return nextSeq();
}
//
@@ -292,13 +294,17 @@ export class LocalPouchDB {
throw new Error(`Chunk was not found: ${id}`);
}
} else {
Logger(`Something went wrong on retriving chunk`);
Logger(`Something went wrong while retrieving chunks`);
throw ex;
}
}
}
async getDBEntryMeta(path: string, opt?: PouchDB.Core.GetOptions, includeDeleted = false): Promise<false | LoadedEntry> {
// safety valve
if (!this.isTargetFile(path)) {
return false;
}
const id = path2id(path);
try {
let obj: EntryDocResponse = null;
@@ -348,6 +354,10 @@ export class LocalPouchDB {
return false;
}
async getDBEntry(path: string, opt?: PouchDB.Core.GetOptions, dump = false, waitForReady = true, includeDeleted = false): Promise<false | LoadedEntry> {
// safety valve
if (!this.isTargetFile(path)) {
return false;
}
const id = path2id(path);
try {
let obj: EntryDocResponse = null;
@@ -392,26 +402,51 @@ export class LocalPouchDB {
// simple note
}
if (obj.type == "newnote" || obj.type == "plain") {
// search childrens
// search children
try {
if (dump) {
Logger(`Enhanced doc`);
Logger(obj);
}
let childrens: string[];
try {
childrens = await Promise.all(obj.children.map((e) => this.getDBLeaf(e, waitForReady)));
if (dump) {
Logger(`Chunks:`);
Logger(childrens);
let children: string[] = [];
if (this.settings.readChunksOnline) {
const items = await this.CollectChunks(obj.children);
if (items) {
for (const v of items) {
if (v && v.type == "leaf") {
children.push(v.data);
} else {
if (!opt) {
Logger(`Chunks of ${obj._id} are not valid.`, LOG_LEVEL.NOTICE);
this.needScanning = true;
this.corruptedEntries[obj._id] = obj;
}
return false;
}
}
} else {
if (opt) {
Logger(`Could not retrieve chunks of ${obj._id}. we have to `, LOG_LEVEL.NOTICE);
this.needScanning = true;
}
return false;
}
} else {
try {
children = await Promise.all(obj.children.map((e) => this.getDBLeaf(e, waitForReady)));
if (dump) {
Logger(`Chunks:`);
Logger(children);
}
} catch (ex) {
Logger(`Something went wrong on reading chunks of ${obj._id} from database, see verbose info for detail.`, LOG_LEVEL.NOTICE);
Logger(ex, LOG_LEVEL.VERBOSE);
this.corruptedEntries[obj._id] = obj;
return false;
}
} catch (ex) {
Logger(`Something went wrong on reading chunks of ${obj._id} from database, see verbose info for detail.`, LOG_LEVEL.NOTICE);
Logger(ex, LOG_LEVEL.VERBOSE);
this.corruptedEntries[obj._id] = obj;
return false;
}
const data = childrens.join("");
const data = children.join("");
const doc: LoadedEntry & PouchDB.Core.IdMeta & PouchDB.Core.GetMeta = {
data: data,
_id: obj._id,
@@ -452,6 +487,10 @@ export class LocalPouchDB {
return false;
}
async deleteDBEntry(path: string, opt?: PouchDB.Core.GetOptions): Promise<boolean> {
// safety valve
if (!this.isTargetFile(path)) {
return false;
}
const id = path2id(path);
try {
@@ -521,7 +560,7 @@ export class LocalPouchDB {
for (const v of result.rows) {
// let doc = v.doc;
if (v.id.startsWith(prefix) || v.id.startsWith("/" + prefix)) {
delDocs.push(v.id);
if (this.isTargetFile(id2path(v.id))) delDocs.push(v.id);
// console.log("!" + v.id);
} else {
if (!v.id.startsWith("h:")) {
@@ -566,37 +605,34 @@ export class LocalPouchDB {
return true;
}
async putDBEntry(note: LoadedEntry, saveAsBigChunk?: boolean) {
//safety valve
if (!this.isTargetFile(id2path(note._id))) {
return;
}
// let leftData = note.data;
const savenNotes = [];
const savedNotes = [];
let processed = 0;
let made = 0;
let skiped = 0;
let pieceSize = MAX_DOC_SIZE_BIN;
let skipped = 0;
const maxChunkSize = MAX_DOC_SIZE_BIN * Math.max(this.settings.customChunkSize, 1);
let pieceSize = maxChunkSize;
let plainSplit = false;
let cacheUsed = 0;
const userpasswordHash = this.h32Raw(new TextEncoder().encode(this.settings.passphrase));
const userPasswordHash = this.h32Raw(new TextEncoder().encode(this.settings.passphrase));
if (!saveAsBigChunk && shouldSplitAsPlainText(note._id)) {
pieceSize = MAX_DOC_SIZE;
plainSplit = true;
}
const minimumChunkSize = Math.min(Math.max(40, ~~(note.data.length / 100)), maxChunkSize);
if (pieceSize < minimumChunkSize) pieceSize = minimumChunkSize;
const newLeafs: EntryLeaf[] = [];
// To keep low bandwith and database size,
// Dedup pieces on database.
// from 0.1.10, for best performance. we use markdown delimiters
// 1. \n[^\n]{longLineThreshold}[^\n]*\n -> long sentence shuld break.
// 2. \n\n shold break
// 3. \r\n\r\n should break
// 4. \n# should break.
let minimumChunkSize = this.settings.minimumChunkSize;
if (minimumChunkSize < 10) minimumChunkSize = 10;
let longLineThreshold = this.settings.longLineThreshold;
if (longLineThreshold < 100) longLineThreshold = 100;
const pieces = splitPieces2(note.data, pieceSize, plainSplit, minimumChunkSize, longLineThreshold);
const pieces = splitPieces2(note.data, pieceSize, plainSplit, minimumChunkSize, 0);
for (const piece of pieces()) {
processed++;
let leafid = "";
let leafId = "";
// Get hash of piece.
let hashedPiece = "";
let hashQ = 0; // if hash collided, **IF**, count it up.
@@ -605,40 +641,40 @@ export class LocalPouchDB {
const cache = this.hashCaches.get(piece);
if (cache) {
hashedPiece = "";
leafid = cache;
leafId = cache;
needMake = false;
skiped++;
skipped++;
cacheUsed++;
} else {
if (this.settings.encrypt) {
// When encryption has been enabled, make hash to be different between each passphrase to avoid inferring password.
hashedPiece = "+" + (this.h32Raw(new TextEncoder().encode(piece)) ^ userpasswordHash).toString(16);
hashedPiece = "+" + (this.h32Raw(new TextEncoder().encode(piece)) ^ userPasswordHash).toString(16);
} else {
hashedPiece = this.h32(piece);
}
leafid = "h:" + hashedPiece;
leafId = "h:" + hashedPiece;
do {
let nleafid = leafid;
let newLeafId = leafId;
try {
nleafid = `${leafid}${hashQ}`;
const pieceData = await this.localDatabase.get<EntryLeaf>(nleafid);
newLeafId = `${leafId}${hashQ}`;
const pieceData = await this.localDatabase.get<EntryLeaf>(newLeafId);
if (pieceData.type == "leaf" && pieceData.data == piece) {
leafid = nleafid;
leafId = newLeafId;
needMake = false;
tryNextHash = false;
this.hashCaches.set(piece, leafid);
this.hashCaches.set(piece, leafId);
} else if (pieceData.type == "leaf") {
Logger("hash:collision!!");
hashQ++;
tryNextHash = true;
} else {
leafid = nleafid;
leafId = newLeafId;
tryNextHash = false;
}
} catch (ex) {
if (ex.status && ex.status == 404) {
//not found, we can use it.
leafid = nleafid;
leafId = newLeafId;
needMake = true;
tryNextHash = false;
} else {
@@ -653,18 +689,18 @@ export class LocalPouchDB {
const savePiece = piece;
const d: EntryLeaf = {
_id: leafid,
_id: leafId,
data: savePiece,
type: "leaf",
};
newLeafs.push(d);
this.hashCaches.set(piece, leafid);
this.hashCaches.set(piece, leafId);
made++;
} else {
skiped++;
skipped++;
}
}
savenNotes.push(leafid);
savedNotes.push(leafId);
}
let saved = true;
if (newLeafs.length > 0) {
@@ -673,7 +709,7 @@ export class LocalPouchDB {
for (const item of result) {
if (!(item as any).ok) {
if ((item as any).status && (item as any).status == 409) {
// conflicted, but it would be ok in childrens.
// conflicted, but it would be ok in children.
} else {
Logger(`Save failed:id:${item.id} rev:${item.rev}`, LOG_LEVEL.NOTICE);
Logger(item);
@@ -688,9 +724,9 @@ export class LocalPouchDB {
}
}
if (saved) {
Logger(`Content saved:${note._id} ,pieces:${processed} (new:${made}, skip:${skiped}, cache:${cacheUsed})`);
Logger(`Content saved:${note._id} ,pieces:${processed} (new:${made}, skip:${skipped}, cache:${cacheUsed})`);
const newDoc: PlainEntry | NewEntry = {
children: savenNotes,
children: savedNotes,
_id: note._id,
ctime: note.ctime,
mtime: note.mtime,
@@ -727,12 +763,12 @@ export class LocalPouchDB {
}
});
} else {
Logger(`note coud not saved:${note._id}`);
Logger(`note could not saved:${note._id}`);
}
}
updateInfo: () => void = () => {
console.log("default updinfo");
console.log("Update Info default implement");
};
// eslint-disable-next-line require-await
async migrate(from: number, to: number): Promise<boolean> {
@@ -772,14 +808,15 @@ export class LocalPouchDB {
return false;
}
const dbret = await connectRemoteCouchDBWithSetting(setting, this.isMobile);
if (typeof dbret === "string") {
Logger(`could not connect to ${uri}: ${dbret}`, showResult ? LOG_LEVEL.NOTICE : LOG_LEVEL.INFO);
const dbRet = await connectRemoteCouchDBWithSetting(setting, this.isMobile);
if (typeof dbRet === "string") {
Logger(`could not connect to ${uri}: ${dbRet}`, showResult ? LOG_LEVEL.NOTICE : LOG_LEVEL.INFO);
return false;
}
if (!skipCheck) {
if (!(await checkRemoteVersion(dbret.db, this.migrate.bind(this), VER))) {
await putDesignDocuments(dbRet.db);
if (!(await checkRemoteVersion(dbRet.db, this.migrate.bind(this), VER))) {
Logger("Remote database is newer or corrupted, make sure to latest version of self-hosted-livesync installed", LOG_LEVEL.NOTICE);
return false;
}
@@ -793,7 +830,7 @@ export class LocalPouchDB {
node_chunk_info: { [this.nodeid]: currentVersionRange }
};
const remoteMilestone: EntryMilestoneInfo = { ...defMilestonePoint, ...(await resolveWithIgnoreKnownError(dbret.db.get(MILSTONE_DOCID), defMilestonePoint)) };
const remoteMilestone: EntryMilestoneInfo = { ...defMilestonePoint, ...(await resolveWithIgnoreKnownError(dbRet.db.get(MILSTONE_DOCID), defMilestonePoint)) };
remoteMilestone.node_chunk_info = { ...defMilestonePoint.node_chunk_info, ...remoteMilestone.node_chunk_info };
this.remoteLocked = remoteMilestone.locked;
this.remoteLockedAndDeviceNotAccepted = remoteMilestone.locked && remoteMilestone.accepted_nodes.indexOf(this.nodeid) == -1;
@@ -807,7 +844,7 @@ export class LocalPouchDB {
if (writeMilestone) {
remoteMilestone.node_chunk_info[this.nodeid].min = currentVersionRange.min;
remoteMilestone.node_chunk_info[this.nodeid].max = currentVersionRange.max;
await dbret.db.put(remoteMilestone);
await dbRet.db.put(remoteMilestone);
}
// Check compatibility and make sure available version
@@ -850,9 +887,13 @@ export class LocalPouchDB {
batches_limit: setting.batches_limit,
batch_size: setting.batch_size,
};
if (setting.readChunksOnline) {
syncOptionBase.push = { filter: 'replicate/push' };
syncOptionBase.pull = { filter: 'replicate/pull' };
}
const syncOption: PouchDB.Replication.SyncOptions = keepAlive ? { live: true, retry: true, heartbeat: 30000, ...syncOptionBase } : { ...syncOptionBase };
return { db: dbret.db, info: dbret.info, syncOptionBase, syncOption };
return { db: dbRet.db, info: dbRet.info, syncOptionBase, syncOption };
}
openReplication(setting: RemoteDBSettings, keepAlive: boolean, showResult: boolean, callback: (e: PouchDB.Core.ExistingDocument<EntryDoc>[]) => Promise<void>) {
@@ -891,7 +932,7 @@ export class LocalPouchDB {
Logger("Replication completed", showResult ? LOG_LEVEL.NOTICE : LOG_LEVEL.INFO, showResult ? "sync" : "");
this.syncHandler = this.cancelHandler(this.syncHandler);
}
replicationDeniend(e: any) {
replicationDenied(e: any) {
this.syncStatus = "ERRORED";
this.updateInfo();
this.syncHandler = this.cancelHandler(this.syncHandler);
@@ -902,6 +943,8 @@ export class LocalPouchDB {
this.syncStatus = "ERRORED";
this.syncHandler = this.cancelHandler(this.syncHandler);
this.updateInfo();
Logger("Replication error", LOG_LEVEL.NOTICE, "sync");
Logger(e);
}
replicationPaused() {
this.syncStatus = "PAUSED";
@@ -915,13 +958,13 @@ export class LocalPouchDB {
callback: (e: PouchDB.Core.ExistingDocument<EntryDoc>[]) => Promise<void>,
retrying: boolean,
callbackDone: (e: boolean | any) => void,
syncmode: "sync" | "pullOnly" | "pushOnly"
syncMode: "sync" | "pullOnly" | "pushOnly"
): Promise<boolean> {
if (this.syncHandler != null) {
Logger("Replication is already in progress.", showResult ? LOG_LEVEL.NOTICE : LOG_LEVEL.INFO, "sync");
return;
}
Logger(`Oneshot Sync begin... (${syncmode})`);
Logger(`Oneshot Sync begin... (${syncMode})`);
let thisCallback = callbackDone;
const ret = await this.checkReplicationConnectivity(setting, true, retrying, showResult);
if (ret === false) {
@@ -941,17 +984,17 @@ export class LocalPouchDB {
this.originalSetting = setting;
}
this.syncHandler = this.cancelHandler(this.syncHandler);
if (syncmode == "sync") {
if (syncMode == "sync") {
this.syncHandler = this.localDatabase.sync(db, { checkpoint: "target", ...syncOptionBase });
this.syncHandler
.on("change", async (e) => {
await this.replicationChangeDetected(e, showResult, docSentOnStart, docArrivedOnStart, callback);
if (retrying) {
if (this.docSent - docSentOnStart + (this.docArrived - docArrivedOnStart) > this.originalSetting.batch_size * 2) {
// restore configration.
// restore configuration.
Logger("Back into original settings once.");
this.syncHandler = this.cancelHandler(this.syncHandler);
this.openOneshotReplication(this.originalSetting, showResult, callback, false, callbackDone, syncmode);
this.openOneshotReplication(this.originalSetting, showResult, callback, false, callbackDone, syncMode);
}
}
})
@@ -961,17 +1004,17 @@ export class LocalPouchDB {
thisCallback(true);
}
});
} else if (syncmode == "pullOnly") {
this.syncHandler = this.localDatabase.replicate.from(db, { checkpoint: "target", ...syncOptionBase });
} else if (syncMode == "pullOnly") {
this.syncHandler = this.localDatabase.replicate.from(db, { checkpoint: "target", ...syncOptionBase, ...(this.settings.readChunksOnline ? { filter: "replicate/pull" } : {}) });
this.syncHandler
.on("change", async (e) => {
await this.replicationChangeDetected({ direction: "pull", change: e }, showResult, docSentOnStart, docArrivedOnStart, callback);
if (retrying) {
if (this.docSent - docSentOnStart + (this.docArrived - docArrivedOnStart) > this.originalSetting.batch_size * 2) {
// restore configration.
// restore configuration.
Logger("Back into original settings once.");
this.syncHandler = this.cancelHandler(this.syncHandler);
this.openOneshotReplication(this.originalSetting, showResult, callback, false, callbackDone, syncmode);
this.openOneshotReplication(this.originalSetting, showResult, callback, false, callbackDone, syncMode);
}
}
})
@@ -981,16 +1024,16 @@ export class LocalPouchDB {
thisCallback(true);
}
});
} else if (syncmode == "pushOnly") {
this.syncHandler = this.localDatabase.replicate.to(db, { checkpoint: "target", ...syncOptionBase });
} else if (syncMode == "pushOnly") {
this.syncHandler = this.localDatabase.replicate.to(db, { checkpoint: "target", ...syncOptionBase, ...(this.settings.readChunksOnline ? { filter: "replicate/push" } : {}) });
this.syncHandler.on("change", async (e) => {
await this.replicationChangeDetected({ direction: "push", change: e }, showResult, docSentOnStart, docArrivedOnStart, callback);
if (retrying) {
if (this.docSent - docSentOnStart + (this.docArrived - docArrivedOnStart) > this.originalSetting.batch_size * 2) {
// restore configration.
// restore configuration.
Logger("Back into original settings once.");
this.syncHandler = this.cancelHandler(this.syncHandler);
this.openOneshotReplication(this.originalSetting, showResult, callback, false, callbackDone, syncmode);
this.openOneshotReplication(this.originalSetting, showResult, callback, false, callbackDone, syncMode);
}
}
})
@@ -1005,7 +1048,7 @@ export class LocalPouchDB {
this.syncHandler
.on("active", () => this.replicationActivated(showResult))
.on("denied", (e) => {
this.replicationDeniend(e);
this.replicationDenied(e);
if (thisCallback != null) {
thisCallback(e);
}
@@ -1015,15 +1058,15 @@ export class LocalPouchDB {
Logger("Replication stopped.", showResult ? LOG_LEVEL.NOTICE : LOG_LEVEL.INFO, "sync");
if (getLastPostFailedBySize()) {
// Duplicate settings for smaller batch.
const xsetting: RemoteDBSettings = JSON.parse(JSON.stringify(setting));
xsetting.batch_size = Math.ceil(xsetting.batch_size / 2) + 2;
xsetting.batches_limit = Math.ceil(xsetting.batches_limit / 2) + 2;
if (xsetting.batch_size <= 5 && xsetting.batches_limit <= 5) {
const tempSetting: RemoteDBSettings = JSON.parse(JSON.stringify(setting));
tempSetting.batch_size = Math.ceil(tempSetting.batch_size / 2) + 2;
tempSetting.batches_limit = Math.ceil(tempSetting.batches_limit / 2) + 2;
if (tempSetting.batch_size <= 5 && tempSetting.batches_limit <= 5) {
Logger("We can't replicate more lower value.", showResult ? LOG_LEVEL.NOTICE : LOG_LEVEL.INFO);
} else {
Logger(`Retry with lower batch size:${xsetting.batch_size}/${xsetting.batches_limit}`, showResult ? LOG_LEVEL.NOTICE : LOG_LEVEL.INFO);
Logger(`Retry with lower batch size:${tempSetting.batch_size}/${tempSetting.batches_limit}`, showResult ? LOG_LEVEL.NOTICE : LOG_LEVEL.INFO);
thisCallback = null;
this.openOneshotReplication(xsetting, showResult, callback, true, callbackDone, syncmode);
this.openOneshotReplication(tempSetting, showResult, callback, true, callbackDone, syncMode);
}
} else {
Logger("Replication error", LOG_LEVEL.NOTICE, "sync");
@@ -1065,7 +1108,7 @@ export class LocalPouchDB {
const docArrivedOnStart = this.docArrived;
const docSentOnStart = this.docSent;
if (!retrying) {
//TODO if successfly saven, roll back org setting.
//TODO if successfully saved, roll back org setting.
this.originalSetting = setting;
}
this.syncHandler = this.cancelHandler(this.syncHandler);
@@ -1092,7 +1135,7 @@ export class LocalPouchDB {
}
})
.on("complete", (e) => this.replicationCompleted(showResult))
.on("denied", (e) => this.replicationDeniend(e))
.on("denied", (e) => this.replicationDenied(e))
.on("error", (e) => {
this.replicationErrored(e);
Logger("Replication stopped.", LOG_LEVEL.NOTICE, "sync");
@@ -1141,7 +1184,7 @@ export class LocalPouchDB {
Logger("Remote Database Destroyed", LOG_LEVEL.NOTICE);
await this.tryCreateRemoteDatabase(setting);
} catch (ex) {
Logger("Something happened on Remote Database Destory:", LOG_LEVEL.NOTICE);
Logger("Something happened on Remote Database Destroy:", LOG_LEVEL.NOTICE);
Logger(ex, LOG_LEVEL.NOTICE);
}
}
@@ -1154,13 +1197,13 @@ export class LocalPouchDB {
}
async markRemoteLocked(setting: RemoteDBSettings, locked: boolean) {
const uri = setting.couchDB_URI + (setting.couchDB_DBNAME == "" ? "" : "/" + setting.couchDB_DBNAME);
const dbret = await connectRemoteCouchDBWithSetting(setting, this.isMobile);
if (typeof dbret === "string") {
Logger(`could not connect to ${uri}:${dbret}`, LOG_LEVEL.NOTICE);
const dbRet = await connectRemoteCouchDBWithSetting(setting, this.isMobile);
if (typeof dbRet === "string") {
Logger(`could not connect to ${uri}:${dbRet}`, LOG_LEVEL.NOTICE);
return;
}
if (!(await checkRemoteVersion(dbret.db, this.migrate.bind(this), VER))) {
if (!(await checkRemoteVersion(dbRet.db, this.migrate.bind(this), VER))) {
Logger("Remote database is newer or corrupted, make sure to latest version of self-hosted-livesync installed", LOG_LEVEL.NOTICE);
return;
}
@@ -1173,7 +1216,7 @@ export class LocalPouchDB {
node_chunk_info: { [this.nodeid]: currentVersionRange }
};
const remoteMilestone: EntryMilestoneInfo = { ...defInitPoint, ...await resolveWithIgnoreKnownError(dbret.db.get(MILSTONE_DOCID), defInitPoint) };
const remoteMilestone: EntryMilestoneInfo = { ...defInitPoint, ...await resolveWithIgnoreKnownError(dbRet.db.get(MILSTONE_DOCID), defInitPoint) };
remoteMilestone.node_chunk_info = { ...defInitPoint.node_chunk_info, ...remoteMilestone.node_chunk_info };
remoteMilestone.accepted_nodes = [this.nodeid];
remoteMilestone.locked = locked;
@@ -1182,17 +1225,17 @@ export class LocalPouchDB {
} else {
Logger("Unlock remote database to prevent data corruption", LOG_LEVEL.NOTICE);
}
await dbret.db.put(remoteMilestone);
await dbRet.db.put(remoteMilestone);
}
async markRemoteResolved(setting: RemoteDBSettings) {
const uri = setting.couchDB_URI + (setting.couchDB_DBNAME == "" ? "" : "/" + setting.couchDB_DBNAME);
const dbret = await connectRemoteCouchDBWithSetting(setting, this.isMobile);
if (typeof dbret === "string") {
Logger(`could not connect to ${uri}:${dbret}`, LOG_LEVEL.NOTICE);
const dbRet = await connectRemoteCouchDBWithSetting(setting, this.isMobile);
if (typeof dbRet === "string") {
Logger(`could not connect to ${uri}:${dbRet}`, LOG_LEVEL.NOTICE);
return;
}
if (!(await checkRemoteVersion(dbret.db, this.migrate.bind(this), VER))) {
if (!(await checkRemoteVersion(dbRet.db, this.migrate.bind(this), VER))) {
Logger("Remote database is newer or corrupted, make sure to latest version of self-hosted-livesync installed", LOG_LEVEL.NOTICE);
return;
}
@@ -1205,11 +1248,11 @@ export class LocalPouchDB {
node_chunk_info: { [this.nodeid]: currentVersionRange }
};
// check local database hash status and remote replicate hash status
const remoteMilestone: EntryMilestoneInfo = { ...defInitPoint, ...await resolveWithIgnoreKnownError(dbret.db.get(MILSTONE_DOCID), defInitPoint) };
const remoteMilestone: EntryMilestoneInfo = { ...defInitPoint, ...await resolveWithIgnoreKnownError(dbRet.db.get(MILSTONE_DOCID), defInitPoint) };
remoteMilestone.node_chunk_info = { ...defInitPoint.node_chunk_info, ...remoteMilestone.node_chunk_info };
remoteMilestone.accepted_nodes = Array.from(new Set([...remoteMilestone.accepted_nodes, this.nodeid]));
Logger("Mark this device as 'resolved'.", LOG_LEVEL.NOTICE);
await dbret.db.put(remoteMilestone);
await dbRet.db.put(remoteMilestone);
}
async sanCheck(entry: EntryDoc): Promise<boolean> {
if (entry.type == "plain" || entry.type == "newnote") {
@@ -1272,16 +1315,16 @@ export class LocalPouchDB {
// console.dir(chunks);
let alive = 0;
let nonref = 0;
let unreachable = 0;
for (const chunk of chunks) {
const items = chunk[1];
if (items.size == 0) {
nonref++;
unreachable++;
} else {
alive++;
}
}
Logger(`Garbage checking completed, documents:${docNum}. Used chunks:${alive}, Retained chunks:${nonref}. Retained chunks will be reused, but you can rebuild database if you feel there are too much.`, LOG_LEVEL.NOTICE, "gc");
Logger(`Garbage checking completed, documents:${docNum}. Used chunks:${alive}, Retained chunks:${unreachable}. Retained chunks will be reused, but you can rebuild database if you feel there are too much.`, LOG_LEVEL.NOTICE, "gc");
});
return;
}
@@ -1293,4 +1336,57 @@ export class LocalPouchDB {
if (this.minChunkVersion > 0 && this.minChunkVersion > ver) return false;
return true;
}
isTargetFile(file: string) {
if (file.includes(":")) return true;
if (this.settings.syncOnlyRegEx) {
const syncOnly = new RegExp(this.settings.syncOnlyRegEx);
if (!file.match(syncOnly)) return false;
}
if (this.settings.syncIgnoreRegEx) {
const syncIgnore = new RegExp(this.settings.syncIgnoreRegEx);
if (file.match(syncIgnore)) return false;
}
return true;
}
// Collect chunks from both local and remote.
async CollectChunks(ids: string[], showResult = false) {
// Fetch local chunks.
const localChunks = await this.localDatabase.allDocs({ keys: ids, include_docs: true });
const missingChunks = localChunks.rows.filter(e => "error" in e).map(e => e.key);
// If we have enough chunks, return them.
if (missingChunks.length == 0) {
return localChunks.rows.map(e => e.doc);
}
// Fetching remote chunks.
const ret = await connectRemoteCouchDBWithSetting(this.settings, this.isMobile);
if (typeof (ret) === "string") {
Logger(`Could not connect to server.${ret} `, showResult ? LOG_LEVEL.NOTICE : LOG_LEVEL.INFO, "fetch");
return false;
}
const remoteChunks = await ret.db.allDocs({ keys: missingChunks, include_docs: true });
if (remoteChunks.rows.some(e => "error" in e)) {
return false;
}
const remoteChunkItems = remoteChunks.rows.map(e => e.doc);
const max = remoteChunkItems.length;
let last = 0;
// Chunks should be ordered by as we requested.
function findChunk(key: string) {
const offset = last;
for (let i = 0; i < max; i++) {
const idx = (offset + i) % max;
last = i;
if (remoteChunkItems[idx]._id == key) return remoteChunkItems[idx];
}
throw Error("Chunk collecting error");
}
// Merge them
return localChunks.rows.map(e => ("error" in e) ? (findChunk(e.key)) : e.doc);
}
}

181
src/ObsidianLiveSyncSettingTab.ts
View File

@@ -49,7 +49,7 @@ export class ObsidianLiveSyncSettingTab extends PluginSettingTab {
<label class='sls-setting-label'><input type='radio' name='disp' value='60' class='sls-setting-tab' ><div class='sls-setting-menu-btn'>🔌</div></label>
<label class='sls-setting-label'><input type='radio' name='disp' value='70' class='sls-setting-tab' ><div class='sls-setting-menu-btn'>🚑</div></label>
`;
const menutabs = w.querySelectorAll(".sls-setting-label");
const menuTabs = w.querySelectorAll(".sls-setting-label");
const changeDisplay = (screen: string) => {
for (const k in screenElements) {
if (k == screen) {
@@ -59,11 +59,11 @@ export class ObsidianLiveSyncSettingTab extends PluginSettingTab {
}
}
};
menutabs.forEach((element) => {
menuTabs.forEach((element) => {
const e = element.querySelector(".sls-setting-tab");
if (!e) return;
e.addEventListener("change", (event) => {
menutabs.forEach((element) => element.removeClass("selected"));
menuTabs.forEach((element) => element.removeClass("selected"));
changeDisplay((event.currentTarget as HTMLInputElement).value);
element.addClass("selected");
});
@@ -115,12 +115,12 @@ export class ObsidianLiveSyncSettingTab extends PluginSettingTab {
};
const applyDisplayEnabled = () => {
if (isAnySyncEnabled()) {
dbsettings.forEach((e) => {
dbSettings.forEach((e) => {
e.setDisabled(true).setTooltip("Could not change this while any synchronization options are enabled.");
});
syncWarn.removeClass("sls-hidden");
} else {
dbsettings.forEach((e) => {
dbSettings.forEach((e) => {
e.setDisabled(false).setTooltip("");
});
syncWarn.addClass("sls-hidden");
@@ -149,8 +149,8 @@ export class ObsidianLiveSyncSettingTab extends PluginSettingTab {
}
};
const dbsettings: Setting[] = [];
dbsettings.push(
const dbSettings: Setting[] = [];
dbSettings.push(
new Setting(containerRemoteDatabaseEl).setName("URI").addText((text) =>
text
.setPlaceholder("https://........")
@@ -201,11 +201,11 @@ export class ObsidianLiveSyncSettingTab extends PluginSettingTab {
.addToggle((toggle) =>
toggle.setValue(this.plugin.settings.workingEncrypt).onChange(async (value) => {
this.plugin.settings.workingEncrypt = value;
phasspharase.setDisabled(!value);
passphrase.setDisabled(!value);
await this.plugin.saveSettings();
})
);
const phasspharase = new Setting(containerRemoteDatabaseEl)
const passphrase = new Setting(containerRemoteDatabaseEl)
.setName("Passphrase")
.setDesc("Encrypting passphrase. If you change the passphrase of a existing database, overwriting the remote database is strongly recommended.")
.addText((text) => {
@@ -217,7 +217,7 @@ export class ObsidianLiveSyncSettingTab extends PluginSettingTab {
});
text.inputEl.setAttribute("type", "password");
});
phasspharase.setDisabled(!this.plugin.settings.workingEncrypt);
passphrase.setDisabled(!this.plugin.settings.workingEncrypt);
const checkWorkingPassphrase = async (): Promise<boolean> => {
const settingForCheck: RemoteDBSettings = {
...this.plugin.settings,
@@ -417,7 +417,7 @@ export class ObsidianLiveSyncSettingTab extends PluginSettingTab {
const res = await requestToCouchDB(this.plugin.settings.couchDB_URI, this.plugin.settings.couchDB_USER, this.plugin.settings.couchDB_PASSWORD, undefined, key, value);
console.dir(res);
if (res.status == 200) {
Logger(`${title} successfly updated`, LOG_LEVEL.NOTICE);
Logger(`${title} successfully updated`, LOG_LEVEL.NOTICE);
checkResultDiv.removeChild(x);
checkConfig();
} else {
@@ -469,6 +469,22 @@ export class ObsidianLiveSyncSettingTab extends PluginSettingTab {
} else {
addResult("✔ httpd.enable_cors is ok.");
}
// If the server is not cloudant, configure request size
if (!this.plugin.settings.couchDB_URI.contains(".cloudantnosqldb.")) {
// REQUEST SIZE
if (Number(responseConfig?.chttpd?.max_http_request_size ?? 0) < 4294967296) {
addResult("❗ chttpd.max_http_request_size is low)");
addConfigFixButton("Set chttpd.max_http_request_size", "chttpd/max_http_request_size", "4294967296");
} else {
addResult("✔ chttpd.max_http_request_size is ok.");
}
if (Number(responseConfig?.couchdb?.max_document_size ?? 0) < 50000000) {
addResult("❗ couchdb.max_document_size is low)");
addConfigFixButton("Set couchdb.max_document_size", "couchdb/max_document_size", "50000000");
} else {
addResult("✔ couchdb.max_document_size is ok.");
}
}
// CORS check
// checking connectivity for mobile
if (responseConfig?.cors?.credentials != "true") {
@@ -515,10 +531,10 @@ export class ObsidianLiveSyncSettingTab extends PluginSettingTab {
addResult("✔ CORS origin OK");
}
}
addResult("--Done--", ["ob-btn-config-haed"]);
addResult("--Done--", ["ob-btn-config-head"]);
addResult("If you have some trouble with Connection-check even though all Config-check has been passed, Please check your reverse proxy's configuration.", ["ob-btn-config-info"]);
} catch (ex) {
Logger(`Checking configration failed`);
Logger(`Checking configuration failed`);
Logger(ex);
}
};
@@ -583,43 +599,7 @@ export class ObsidianLiveSyncSettingTab extends PluginSettingTab {
})
)
containerLocalDatabaseEl.createEl("div", {
text: sanitizeHTMLToDom(`Advanced settings<br>
Configuration of how LiveSync makes chunks from the file.`),
});
new Setting(containerLocalDatabaseEl)
.setName("Minimum chunk size")
.setDesc("(letters), minimum chunk size.")
.addText((text) => {
text.setPlaceholder("")
.setValue(this.plugin.settings.minimumChunkSize + "")
.onChange(async (value) => {
let v = Number(value);
if (isNaN(v) || v < 10 || v > 1000) {
v = 10;
}
this.plugin.settings.minimumChunkSize = v;
await this.plugin.saveSettings();
});
text.inputEl.setAttribute("type", "number");
});
new Setting(containerLocalDatabaseEl)
.setName("LongLine Threshold")
.setDesc("(letters), If the line is longer than this, make the line to chunk")
.addText((text) => {
text.setPlaceholder("")
.setValue(this.plugin.settings.longLineThreshold + "")
.onChange(async (value) => {
let v = Number(value);
if (isNaN(v) || v < 10 || v > 1000) {
v = 10;
}
this.plugin.settings.longLineThreshold = v;
await this.plugin.saveSettings();
});
text.inputEl.setAttribute("type", "number");
});
let newDatabaseName = this.plugin.settings.additionalSuffixOfDatabaseName + "";
new Setting(containerLocalDatabaseEl)
.setName("Database suffix")
@@ -652,7 +632,7 @@ export class ObsidianLiveSyncSettingTab extends PluginSettingTab {
new Setting(containerGeneralSettingsEl)
.setName("Do not show low-priority Log")
.setDesc("Reduce log infomations")
.setDesc("Reduce log information")
.addToggle((toggle) =>
toggle.setValue(this.plugin.settings.lessInformationInLog).onChange(async (value) => {
this.plugin.settings.lessInformationInLog = value;
@@ -661,7 +641,7 @@ export class ObsidianLiveSyncSettingTab extends PluginSettingTab {
);
new Setting(containerGeneralSettingsEl)
.setName("Verbose Log")
.setDesc("Show verbose log ")
.setDesc("Show verbose log")
.addToggle((toggle) =>
toggle.setValue(this.plugin.settings.showVerboseLog).onChange(async (value) => {
this.plugin.settings.showVerboseLog = value;
@@ -810,15 +790,6 @@ export class ObsidianLiveSyncSettingTab extends PluginSettingTab {
})
);
// new Setting(containerSyncSettingEl)
// .setName("Skip old files on sync")
// .setDesc("Skip old incoming if incoming changes older than storage.")
// .addToggle((toggle) =>
// toggle.setValue(this.plugin.settings.skipOlderFilesOnSync).onChange(async (value) => {
// this.plugin.settings.skipOlderFilesOnSync = value;
// await this.plugin.saveSettings();
// })
// );
new Setting(containerSyncSettingEl)
.setName("Check conflict only on opened files")
.setDesc("Do not check conflict for replication")
@@ -829,9 +800,7 @@ export class ObsidianLiveSyncSettingTab extends PluginSettingTab {
})
);
containerSyncSettingEl.createEl("h3", {
text: sanitizeHTMLToDom(`Experimental`),
});
new Setting(containerSyncSettingEl)
.setName("Sync hidden files")
.addToggle((toggle) =>
@@ -926,6 +895,86 @@ export class ObsidianLiveSyncSettingTab extends PluginSettingTab {
})
)
containerSyncSettingEl.createEl("h3", {
text: sanitizeHTMLToDom(`Experimental`),
});
new Setting(containerSyncSettingEl)
.setName("Regular expression to ignore files")
.setDesc("If this is set, any changes to local and remote files that match this will be skipped.")
.addTextArea((text) => {
text
.setValue(this.plugin.settings.syncIgnoreRegEx)
.setPlaceholder("\\.pdf$")
.onChange(async (value) => {
let isValidRegExp = false;
try {
new RegExp(value);
isValidRegExp = true;
} catch (_) {
// NO OP.
}
if (isValidRegExp || value.trim() == "") {
this.plugin.settings.syncIgnoreRegEx = value;
await this.plugin.saveSettings();
}
})
return text;
}
);
new Setting(containerSyncSettingEl)
.setName("Regular expression for restricting synchronization targets")
.setDesc("If this is set, changes to local and remote files that only match this will be processed.")
.addTextArea((text) => {
text
.setValue(this.plugin.settings.syncOnlyRegEx)
.setPlaceholder("\\.md$|\\.txt")
.onChange(async (value) => {
let isValidRegExp = false;
try {
new RegExp(value);
isValidRegExp = true;
} catch (_) {
// NO OP.
}
if (isValidRegExp || value.trim() == "") {
this.plugin.settings.syncOnlyRegEx = value;
await this.plugin.saveSettings();
}
})
return text;
}
);
new Setting(containerSyncSettingEl)
.setName("Chunk size")
.setDesc("Customize chunk size for binary files (0.1MBytes). This cannot be increased when using IBM Cloudant.")
.addText((text) => {
text.setPlaceholder("")
.setValue(this.plugin.settings.customChunkSize + "")
.onChange(async (value) => {
let v = Number(value);
if (isNaN(v) || v < 100) {
v = 100;
}
this.plugin.settings.customChunkSize = v;
await this.plugin.saveSettings();
});
text.inputEl.setAttribute("type", "number");
});
new Setting(containerSyncSettingEl)
.setName("Read chunks online.")
.setDesc("If this option is enabled, LiveSync reads chunks online directly instead of replicating them locally. Increasing Custom chunk size is recommended.")
.addToggle((toggle) => {
toggle
.setValue(this.plugin.settings.readChunksOnline)
.onChange(async (value) => {
this.plugin.settings.readChunksOnline = value;
await this.plugin.saveSettings();
})
return toggle;
}
);
containerSyncSettingEl.createEl("h3", {
text: sanitizeHTMLToDom(`Advanced settings`),
});
@@ -1064,7 +1113,7 @@ export class ObsidianLiveSyncSettingTab extends PluginSettingTab {
c.addClass("op-warn");
}
}
const hatchWarn = containerHatchEl.createEl("div", { text: `To stop the bootup sequence for fixing problems on databases, you can put redflag.md on top of your vault (Rebooting obsidian is required).` });
const hatchWarn = containerHatchEl.createEl("div", { text: `To stop the boot up sequence for fixing problems on databases, you can put redflag.md on top of your vault (Rebooting obsidian is required).` });
hatchWarn.addClass("op-warn-info");
new Setting(containerHatchEl)
@@ -1179,7 +1228,7 @@ export class ObsidianLiveSyncSettingTab extends PluginSettingTab {
new Setting(containerHatchEl)
.setName("Drop old encrypted database")
.setDesc("WARNING: Please use this button only when you have failed on converting old-style localdatabase at v0.10.0.")
.setDesc("WARNING: Please use this button only when you have failed on converting old-style local database at v0.10.0.")
.addButton((button) =>
button
.setButtonText("Drop")
@@ -1193,7 +1242,7 @@ export class ObsidianLiveSyncSettingTab extends PluginSettingTab {
addScreenElement("50", containerHatchEl);
// With great respect, thank you TfTHacker!
// refered: https://github.com/TfTHacker/obsidian42-brat/blob/main/src/features/BetaPlugins.ts
// Refer: https://github.com/TfTHacker/obsidian42-brat/blob/main/src/features/BetaPlugins.ts
const containerPluginSettings = containerEl.createDiv();
containerPluginSettings.createEl("h3", { text: "Plugins and settings (beta)" });

2
src/lib

Submodule src/lib updated: a49a096a6a...aacfa353a9

629
src/main.ts
View File

File diff suppressed because it is too large Load Diff

4
src/utils.ts
View File

@@ -3,7 +3,7 @@ import { normalizePath } from "obsidian";
import { path2id_base, id2path_base } from "./lib/src/utils";
// For backward compatibility, using the path for determining id.
// Only CouchDB nonacceptable ID (that starts with an underscore) has been prefixed with "/".
// Only CouchDB unacceptable ID (that starts with an underscore) has been prefixed with "/".
// The first slash will be deleted when the path is normalized.
export function path2id(filename: string): string {
const x = normalizePath(filename);
@@ -63,7 +63,7 @@ export async function memoIfNotExist<T>(key: string, func: () => T | Promise<T>)
}
return memos[key] as T;
}
export function retriveMemoObject<T>(key: string): T | false {
export function retrieveMemoObject<T>(key: string): T | false {
if (key in memos) {
return memos[key];
} else {

80
src/utils_couchdb.ts
View File

@@ -9,9 +9,9 @@ export const isValidRemoteCouchDBURI = (uri: string): boolean => {
if (uri.startsWith("http://")) return true;
return false;
};
let last_post_successed = false;
let last_successful_post = false;
export const getLastPostFailedBySize = () => {
return !last_post_successed;
return !last_successful_post;
};
const fetchByAPI = async (request: RequestUrlParam): Promise<RequestUrlResponse> => {
const ret = await requestUrl(request);
@@ -40,8 +40,8 @@ export const connectRemoteCouchDBWithSetting = (settings: RemoteDBSettings, isMo
const connectRemoteCouchDB = async (uri: string, auth: { username: string; password: string }, disableRequestURI: boolean, passphrase: string | boolean): Promise<string | { db: PouchDB.Database<EntryDoc>; info: PouchDB.Core.DatabaseInfo }> => {
if (!isValidRemoteCouchDBURI(uri)) return "Remote URI is not valid";
if (uri.toLowerCase() != uri) return "Remote URI and database name cound not contain capital letters.";
if (uri.indexOf(" ") !== -1) return "Remote URI and database name cound not contain spaces.";
if (uri.toLowerCase() != uri) return "Remote URI and database name could not contain capital letters.";
if (uri.indexOf(" ") !== -1) return "Remote URI and database name could not contain spaces.";
let authHeader = "";
if (auth.username && auth.password) {
const utf8str = String.fromCharCode.apply(null, new TextEncoder().encode(`${auth.username}:${auth.password}`));
@@ -62,7 +62,7 @@ const connectRemoteCouchDB = async (uri: string, auth: { username: string; passw
if (opts_length > 1024 * 1024 * 10) {
// over 10MB
if (uri.contains(".cloudantnosqldb.")) {
last_post_successed = false;
last_successful_post = false;
Logger("This request should fail on IBM Cloudant.", LOG_LEVEL.VERBOSE);
throw new Error("This request should fail on IBM Cloudant.");
}
@@ -91,9 +91,9 @@ const connectRemoteCouchDB = async (uri: string, auth: { username: string; passw
try {
const r = await fetchByAPI(requestParam);
if (method == "POST" || method == "PUT") {
last_post_successed = r.status - (r.status % 100) == 200;
last_successful_post = r.status - (r.status % 100) == 200;
} else {
last_post_successed = true;
last_successful_post = true;
}
Logger(`HTTP:${method}${size} to:${localURL} -> ${r.status}`, LOG_LEVEL.DEBUG);
@@ -106,7 +106,7 @@ const connectRemoteCouchDB = async (uri: string, auth: { username: string; passw
Logger(`HTTP:${method}${size} to:${localURL} -> failed`, LOG_LEVEL.VERBOSE);
// limit only in bulk_docs.
if (url.toString().indexOf("_bulk_docs") !== -1) {
last_post_successed = false;
last_successful_post = false;
}
Logger(ex);
throw ex;
@@ -116,19 +116,19 @@ const connectRemoteCouchDB = async (uri: string, auth: { username: string; passw
// -old implementation
try {
const responce: Response = await fetch(url, opts);
const response: Response = await fetch(url, opts);
if (method == "POST" || method == "PUT") {
last_post_successed = responce.ok;
last_successful_post = response.ok;
} else {
last_post_successed = true;
last_successful_post = true;
}
Logger(`HTTP:${method}${size} to:${localURL} -> ${responce.status}`, LOG_LEVEL.DEBUG);
return responce;
Logger(`HTTP:${method}${size} to:${localURL} -> ${response.status}`, LOG_LEVEL.DEBUG);
return response;
} catch (ex) {
Logger(`HTTP:${method}${size} to:${localURL} -> failed`, LOG_LEVEL.VERBOSE);
// limit only in bulk_docs.
if (url.toString().indexOf("_bulk_docs") !== -1) {
last_post_successed = false;
last_successful_post = false;
}
Logger(ex);
throw ex;
@@ -225,3 +225,55 @@ export const checkSyncInfo = async (db: PouchDB.Database): Promise<boolean> => {
}
}
};
export async function putDesignDocuments(db: PouchDB.Database) {
type DesignDoc = {
_id: string;
_rev: string;
ver: number;
filters: {
default: string,
push: string,
pull: string,
};
}
const design: DesignDoc = {
"_id": "_design/replicate",
"_rev": undefined as string | undefined,
"ver": 2,
"filters": {
"default": function (doc: any, req: any) {
return !("remote" in doc && doc.remote);
}.toString(),
"push": function (doc: any, req: any) {
return true;
}.toString(),
"pull": function (doc: any, req: any) {
return !(doc.type && doc.type == "leaf")
}.toString(),
}
}
// We can use the filter on replication : filter: 'replicate/default',
try {
const w = await db.get<DesignDoc>(design._id);
if (w.ver < design.ver) {
design._rev = w._rev;
//@ts-ignore
await db.put(design);
return true;
}
} catch (ex) {
if (ex.status && ex.status == 404) {
delete design._rev;
//@ts-ignore
await db.put(design);
return true;
} else {
Logger("Could not make design documents", LOG_LEVEL.INFO);
}
}
return false;
}

23
tsconfig.json
View File

@@ -2,15 +2,26 @@
"compilerOptions": {
"baseUrl": ".",
"module": "ESNext",
"target": "ES6",
"target": "ES2018",
"allowJs": true,
"noImplicitAny": true,
"moduleResolution": "node",
// "importsNotUsedAsValues": "error",
"importHelpers": true,
"importHelpers": false,
"alwaysStrict": true,
"lib": ["es2018", "DOM", "ES5", "ES6", "ES7"]
"lib": [
"es2018",
"DOM",
"ES5",
"ES6",
"ES7",
"es2019.array"
]
},
"include": ["**/*.ts"],
"exclude": ["pouchdb-browser-webpack"]
}
"include": [
"**/*.ts"
],
"exclude": [
"pouchdb-browser-webpack"
]
}

31
updates.md
View File

@@ -1,3 +1,27 @@
### 0.14.1
- The target selecting filter was implemented.
Now we can set what files are synchronised by regular expression.
- We can configure the size of chunks.
We can use larger chunks to improve performance.
(This feature can not be used with IBM Cloudant)
- Read chunks online.
Now we can synchronise only metadata and retrieve chunks on demand. It reduces local database size and time for replication.
- Added this note.
- Use local chunks in preference to remote them if present,
#### Recommended configuration for Self-hosted CouchDB
- Set chunk size to around 100 to 250 (10MB - 25MB per chunk)
- *Set batch size to 100 and batch limit to 20 (0.14.2)*
- Be sure to `Read chunks online` checked.
#### Minors
- 0.14.2 Fixed issue about retrieving files if synchronisation has been interrupted or failed
- 0.14.3 New test items have been added to `Check database configuration`.
- 0.14.4 Fixed issue of importing configurations.
- 0.14.5 Auto chunk size adjusting implemented.
- 0.14.6 Change Target to ES2018
- 0.14.7 Refactor and fix typos.
### 0.13.0
- The metadata of the deleted files will be kept on the database by default. If you want to delete this as the previous version, please turn on `Delete metadata of deleted files.`. And, if you have upgraded from the older version, please ensure every device has been upgraded.
@@ -9,4 +33,9 @@
#### Minors
- 0.13.1 Fixed on conflict resolution.
- 0.13.2 Fixed file deletion failures.
- 0.13.2 Fixed file deletion failures.
- 0.13.4
- Now, we can synchronise hidden files that conflicted on each devices.
- We can search for conflicting docs.
- Pending processes can now be run at any time.
- Performance improved on synchronising large numbers of files at once.