public/obsidian-livesync
1
0
Fork 0
mirror of https://github.com/vrtmrz/obsidian-livesync.git synced 2026-05-07 08:11:50 +00:00
Code Issues Packages Projects Releases Wiki Activity

Compare commits

base: public:0.18.3
Branches Tags
public:main public:auto_conf public:beta public:alpha public:snyk-fix-34b8981b68f7111ada8b22377113a98c public:snyk-upgrade-7fd31e6f474568014502e7205847f257 public:snyk-fix-90f3b71450f7543a1c61724d8afa0e71 public:refactor_services_2 public:refactor_services public:dev public:snyk-upgrade-fe9625385aa4be849c7157033fd2abb0 public:snyk-upgrade-99a31e70c822894595544b0c9813b11a public:svelteui public:disenchant public:snyk-fix-61f76ef0bce2a276df9552c2fc80a14f public:snyk-upgrade-2baa7938ba318464c3bad5c3fd5cbda5 public:snyk-upgrade-7af62a575eab20e8fb4e9565bf1adcfc public:snyk-upgrade-b32f281b31a81483f126b8fb6c54a957 public:snyk-upgrade-bd61b1103442d3faecc9fc726c34e9a4 public:snyk-upgrade-5b2d6584430c061b5187611a5d7fb64a public:snyk-fix-0e9fbbcb374fcb8d5e8cb027d8414fb7 public:snyk-upgrade-5242daf91e706d1ae500c9ea6545636e public:snyk-upgrade-70434a6822fbad83424135dfad9df909 public:snyk-upgrade-8fa01ccad0352125f1866d41e57d35e1 public:snyk-upgrade-39eb97b4655e70091f718a88fccd57c1 public:snyk-upgrade-2b99d54d19e207887e36c3071b796925 public:forprivacy public:snyk-upgrade-dad9088d0a333677e0157e3deef7fc38 public:snyk-upgrade-b9a76d74b3e5723759af5b3319aabb80 public:snyk-upgrade-7da6459d8ff5d293d80de9d80061d912 public:snyk-upgrade-6a3786fc4cbe394542e9d1b6058c46e7 public:snyk-upgrade-8fc9e62771dc41b0f23dc0efefb5c949
public:0.25.60 public:0.25.59 public:0.25.58 public:0.25.57 public:0.25.56+patched5 public:0.25.56+patched4 public:0.25.56+patched3 public:0.25.56+patched2 public:0.25.56+patched1 public:0.25.56 public:0.25.55 public:0.25.54 public:0.25.53 public:0.25.52-patched-3 public:0.25.52-patched-2 public:0.25.52-patched-1 public:0.25.52 public:0.25.51 public:0.25.50 public:0.25.49 public:0.25.48 public:0.25.47 public:0.25.46 public:0.25.45 public:0.25.44 public:0.25.43-patched-9 public:0.25.43-patched-8 public:0.25.43-patched-7 public:0.25.43-patched-6 public:0.25.43-patched-5 public:0.25.43-patched-4 public:0.25.43-patched-3 public:0.25.43-patched-2 public:0.25.43-patched-1 public:0.25.43 public:0.25.42 public:0.25.41-patched-3 public:0.25.41-patched-2 public:0.25.41-patched-1 public:0.25.41 public:0.25.40 public:0.25.39 public:0.25.38 public:0.25.37 public:0.25.36 public:0.25.35 public:0.25.34 public:0.25.33 public:0.25.32 public:0.25.31 public:0.25.30 public:0.25.30-dev public:0.25.29 public:0.25.28 public:0.25.27 public:0.25.26 public:0.25.25 public:0.25.24 public:0.25.24.beta3 public:0.25.24.beta2 public:0.25.24.beta1 public:0.25.23 public:0.25.23.beta1 public:0.25.22 public:0.25.21 public:0.25.21.beta2 public:0.25.21.beta1 public:0.25.20 public:0.25.19 public:0.25.18 public:0.25.17 public:0.25.16 public:0.25.15 public:0.25.14 public:0.25.13 public:0.25.12 public:0.25.11 public:0.25.10 public:0.25.9 public:0.25.8 public:0.25.7 public:0.25.6 public:0.25.5 public:0.25.4 public:0.25.3 public:0.25.2 public:0.25.1 public:0.25.0 public:0.25.0-beta1 public:0.24.31 public:0.24.30 public:0.24.29 public:0.24.28 public:0.24.27 public:0.24.26 public:0.24.25 public:0.24.24 public:0.24.23 public:0.24.22 public:0.24.21 public:0.24.20 public:0.24.19 public:0.24.18 public:0.24.17 public:0.24.16 public:0.24.15 public:0.24.14 public:0.24.13 public:0.24.12 public:0.24.11 public:0.24.10 public:0.24.9 public:0.24.8 public:0.24.7 public:0.24.6 public:0.24.5 public:0.24.4 public:0.24.3 public:0.24.2 public:0.24.1 public:0.24.0 public:0.24.0.dev-rc8 public:0.24.0.dev-rc7 public:0.24.0.dev-rc6 public:0.24.0.dev-rc5 public:0.24.0.dev-rc4 public:0.24.0.dev-rc3 public:0.24.0.dev-rc2 public:0.23.23 public:0.23.22 public:0.23.21 public:0.23.20 public:0.23.19 public:0.23.18 public:0.23.17 public:0.23.16 public:0.23.15 public:0.23.14 public:0.23.13 public:0.23.12 public:0.23.11 public:0.23.10 public:0.23.9 public:0.23.8 public:0.23.7 public:0.23.6 public:0.23.5 public:0.23.4 public:0.23.3 public:0.23.2 public:0.23.1 public:0.23.0 public:0.22.19 public:0.22.18 public:0.22.17 public:0.22.16 public:0.22.15 public:0.22.14 public:0.22.13 public:0.22.12 public:0.22.11 public:0.22.10 public:0.22.9 public:0.22.8 public:0.22.7 public:0.22.6 public:0.22.5 public:0.22.4 public:0.22.3 public:0.22.2 public:0.22.1 public:0.22.0 public:0.21.5 public:0.21.4 public:0.21.3 public:0.21.2 public:0.21.1 public:0.21.0 public:0.20.7 public:0.20.6 public:0.20.5 public:0.20.4 public:0.20.3 public:0.20.2 public:0.20.1 public:0.20.0 public:0.19.23 public:0.19.22 public:0.19.21 public:0.19.20 public:0.19.19 public:0.19.18 public:0.19.17 public:0.19.16 public:0.19.15 public:0.19.14 public:0.19.13 public:0.19.12 public:0.19.11 public:0.19.10 public:0.19.9 public:0.19.8 public:0.19.7 public:0.19.6 public:0.19.5 public:0.19.4 public:0.19.3 public:0.19.2 public:0.19.1 public:0.19.0 public:0.18.6 public:0.18.5 public:0.18.4 public:0.18.3 public:0.18.2 public:0.18.1 public:0.18.0 public:0.17.34 public:0.17.33 public:0.17.32 public:0.17.31 public:0.17.30 public:0.17.29 public:0.17.28 public:0.17.27 public:0.17.26 public:0.17.25 public:0.17.24 public:0.17.23 public:0.17.22 public:0.17.21 public:0.17.20 public:0.17.19 public:0.17.18 public:0.17.17 public:0.17.16 public:0.17.15 public:0.17.14 public:0.17.13 public:0.17.12 public:0.17.11 public:0.17.10 public:0.17.9 public:0.17.8 public:0.17.7 public:0.17.6 public:0.17.5 public:0.17.4 public:0.17.3 public:0.17.2 public:0.17.1 public:0.17.0 public:0.16.8 public:0.16.7 public:0.16.6 public:0.16.5 public:0.16.4 public:0.16.3 public:0.16.2 public:0.16.1 public:0.16.0 public:0.15.10 public:0.15.9 public:0.15.8 public:0.15.7 public:0.15.6 public:0.15.5 public:0.15.4 public:0.15.3 public:0.15.2 public:0.15.1 public:0.15.0 public:0.14.8 public:0.14.7 public:0.14.6 public:0.14.5 public:0.14.4 public:0.14.3 public:0.14.2 public:0.14.1 public:0.14.0 public:0.13.4 public:0.13.3 public:0.13.2 public:0.13.1 public:0.13.0 public:0.12.5 public:0.12.4 public:0.12.3 public:0.12.2 public:0.12.1 public:0.12.0 public:0.11.10 public:0.11.9 public:0.11.8 public:0.11.7 public:0.11.6 public:0.11.5 public:0.11.4 public:0.11.3 public:0.11.2 public:0.11.1 public:0.11.0 public:0.10.1 public:0.10.0 public:0.9.0 public:0.8.8 public:0.8.7 public:0.8.6 public:0.8.5 public:0.8.3 public:0.8.2 public:0.8.1 public:0.8.0 public:0.7.2 public:0.7.1 public:0.7.0 public:0.6.1 public:0.6.0 public:0.5.0 public:0.4.1 public:0.4.0 public:0.3.1 public:0.3.0 public:0.2.1 public:0.2.0 public:0.1.26 public:0.1.25 public:0.1.24 public:0.1.23 public:0.1.22 public:0.1.21 public:0.1.20 public:0.1.19 public:0.1.18 public:0.1.17 public:0.1.16 public:0.1.15 public:0.1.14 public:0.1.13 public:0.1.12 public:0.1.11 public:0.1.10 public:0.1.9 public:0.1.8 public:0.1.7 public:0.1.6 public:0.1.5 public:0.1.4 public:0.1.3 public:0.1.2 public:0.1.1 public:0.1.0 public:0.0.9 public:0.0.8 public:0.0.7 public:0.0.6
...
compare: public:0.19.9
Branches Tags
public:main public:auto_conf public:beta public:alpha public:snyk-fix-34b8981b68f7111ada8b22377113a98c public:snyk-upgrade-7fd31e6f474568014502e7205847f257 public:snyk-fix-90f3b71450f7543a1c61724d8afa0e71 public:refactor_services_2 public:refactor_services public:dev public:snyk-upgrade-fe9625385aa4be849c7157033fd2abb0 public:snyk-upgrade-99a31e70c822894595544b0c9813b11a public:svelteui public:disenchant public:snyk-fix-61f76ef0bce2a276df9552c2fc80a14f public:snyk-upgrade-2baa7938ba318464c3bad5c3fd5cbda5 public:snyk-upgrade-7af62a575eab20e8fb4e9565bf1adcfc public:snyk-upgrade-b32f281b31a81483f126b8fb6c54a957 public:snyk-upgrade-bd61b1103442d3faecc9fc726c34e9a4 public:snyk-upgrade-5b2d6584430c061b5187611a5d7fb64a public:snyk-fix-0e9fbbcb374fcb8d5e8cb027d8414fb7 public:snyk-upgrade-5242daf91e706d1ae500c9ea6545636e public:snyk-upgrade-70434a6822fbad83424135dfad9df909 public:snyk-upgrade-8fa01ccad0352125f1866d41e57d35e1 public:snyk-upgrade-39eb97b4655e70091f718a88fccd57c1 public:snyk-upgrade-2b99d54d19e207887e36c3071b796925 public:forprivacy public:snyk-upgrade-dad9088d0a333677e0157e3deef7fc38 public:snyk-upgrade-b9a76d74b3e5723759af5b3319aabb80 public:snyk-upgrade-7da6459d8ff5d293d80de9d80061d912 public:snyk-upgrade-6a3786fc4cbe394542e9d1b6058c46e7 public:snyk-upgrade-8fc9e62771dc41b0f23dc0efefb5c949
public:0.25.60 public:0.25.59 public:0.25.58 public:0.25.57 public:0.25.56+patched5 public:0.25.56+patched4 public:0.25.56+patched3 public:0.25.56+patched2 public:0.25.56+patched1 public:0.25.56 public:0.25.55 public:0.25.54 public:0.25.53 public:0.25.52-patched-3 public:0.25.52-patched-2 public:0.25.52-patched-1 public:0.25.52 public:0.25.51 public:0.25.50 public:0.25.49 public:0.25.48 public:0.25.47 public:0.25.46 public:0.25.45 public:0.25.44 public:0.25.43-patched-9 public:0.25.43-patched-8 public:0.25.43-patched-7 public:0.25.43-patched-6 public:0.25.43-patched-5 public:0.25.43-patched-4 public:0.25.43-patched-3 public:0.25.43-patched-2 public:0.25.43-patched-1 public:0.25.43 public:0.25.42 public:0.25.41-patched-3 public:0.25.41-patched-2 public:0.25.41-patched-1 public:0.25.41 public:0.25.40 public:0.25.39 public:0.25.38 public:0.25.37 public:0.25.36 public:0.25.35 public:0.25.34 public:0.25.33 public:0.25.32 public:0.25.31 public:0.25.30 public:0.25.30-dev public:0.25.29 public:0.25.28 public:0.25.27 public:0.25.26 public:0.25.25 public:0.25.24 public:0.25.24.beta3 public:0.25.24.beta2 public:0.25.24.beta1 public:0.25.23 public:0.25.23.beta1 public:0.25.22 public:0.25.21 public:0.25.21.beta2 public:0.25.21.beta1 public:0.25.20 public:0.25.19 public:0.25.18 public:0.25.17 public:0.25.16 public:0.25.15 public:0.25.14 public:0.25.13 public:0.25.12 public:0.25.11 public:0.25.10 public:0.25.9 public:0.25.8 public:0.25.7 public:0.25.6 public:0.25.5 public:0.25.4 public:0.25.3 public:0.25.2 public:0.25.1 public:0.25.0 public:0.25.0-beta1 public:0.24.31 public:0.24.30 public:0.24.29 public:0.24.28 public:0.24.27 public:0.24.26 public:0.24.25 public:0.24.24 public:0.24.23 public:0.24.22 public:0.24.21 public:0.24.20 public:0.24.19 public:0.24.18 public:0.24.17 public:0.24.16 public:0.24.15 public:0.24.14 public:0.24.13 public:0.24.12 public:0.24.11 public:0.24.10 public:0.24.9 public:0.24.8 public:0.24.7 public:0.24.6 public:0.24.5 public:0.24.4 public:0.24.3 public:0.24.2 public:0.24.1 public:0.24.0 public:0.24.0.dev-rc8 public:0.24.0.dev-rc7 public:0.24.0.dev-rc6 public:0.24.0.dev-rc5 public:0.24.0.dev-rc4 public:0.24.0.dev-rc3 public:0.24.0.dev-rc2 public:0.23.23 public:0.23.22 public:0.23.21 public:0.23.20 public:0.23.19 public:0.23.18 public:0.23.17 public:0.23.16 public:0.23.15 public:0.23.14 public:0.23.13 public:0.23.12 public:0.23.11 public:0.23.10 public:0.23.9 public:0.23.8 public:0.23.7 public:0.23.6 public:0.23.5 public:0.23.4 public:0.23.3 public:0.23.2 public:0.23.1 public:0.23.0 public:0.22.19 public:0.22.18 public:0.22.17 public:0.22.16 public:0.22.15 public:0.22.14 public:0.22.13 public:0.22.12 public:0.22.11 public:0.22.10 public:0.22.9 public:0.22.8 public:0.22.7 public:0.22.6 public:0.22.5 public:0.22.4 public:0.22.3 public:0.22.2 public:0.22.1 public:0.22.0 public:0.21.5 public:0.21.4 public:0.21.3 public:0.21.2 public:0.21.1 public:0.21.0 public:0.20.7 public:0.20.6 public:0.20.5 public:0.20.4 public:0.20.3 public:0.20.2 public:0.20.1 public:0.20.0 public:0.19.23 public:0.19.22 public:0.19.21 public:0.19.20 public:0.19.19 public:0.19.18 public:0.19.17 public:0.19.16 public:0.19.15 public:0.19.14 public:0.19.13 public:0.19.12 public:0.19.11 public:0.19.10 public:0.19.9 public:0.19.8 public:0.19.7 public:0.19.6 public:0.19.5 public:0.19.4 public:0.19.3 public:0.19.2 public:0.19.1 public:0.19.0 public:0.18.6 public:0.18.5 public:0.18.4 public:0.18.3 public:0.18.2 public:0.18.1 public:0.18.0 public:0.17.34 public:0.17.33 public:0.17.32 public:0.17.31 public:0.17.30 public:0.17.29 public:0.17.28 public:0.17.27 public:0.17.26 public:0.17.25 public:0.17.24 public:0.17.23 public:0.17.22 public:0.17.21 public:0.17.20 public:0.17.19 public:0.17.18 public:0.17.17 public:0.17.16 public:0.17.15 public:0.17.14 public:0.17.13 public:0.17.12 public:0.17.11 public:0.17.10 public:0.17.9 public:0.17.8 public:0.17.7 public:0.17.6 public:0.17.5 public:0.17.4 public:0.17.3 public:0.17.2 public:0.17.1 public:0.17.0 public:0.16.8 public:0.16.7 public:0.16.6 public:0.16.5 public:0.16.4 public:0.16.3 public:0.16.2 public:0.16.1 public:0.16.0 public:0.15.10 public:0.15.9 public:0.15.8 public:0.15.7 public:0.15.6 public:0.15.5 public:0.15.4 public:0.15.3 public:0.15.2 public:0.15.1 public:0.15.0 public:0.14.8 public:0.14.7 public:0.14.6 public:0.14.5 public:0.14.4 public:0.14.3 public:0.14.2 public:0.14.1 public:0.14.0 public:0.13.4 public:0.13.3 public:0.13.2 public:0.13.1 public:0.13.0 public:0.12.5 public:0.12.4 public:0.12.3 public:0.12.2 public:0.12.1 public:0.12.0 public:0.11.10 public:0.11.9 public:0.11.8 public:0.11.7 public:0.11.6 public:0.11.5 public:0.11.4 public:0.11.3 public:0.11.2 public:0.11.1 public:0.11.0 public:0.10.1 public:0.10.0 public:0.9.0 public:0.8.8 public:0.8.7 public:0.8.6 public:0.8.5 public:0.8.3 public:0.8.2 public:0.8.1 public:0.8.0 public:0.7.2 public:0.7.1 public:0.7.0 public:0.6.1 public:0.6.0 public:0.5.0 public:0.4.1 public:0.4.0 public:0.3.1 public:0.3.0 public:0.2.1 public:0.2.0 public:0.1.26 public:0.1.25 public:0.1.24 public:0.1.23 public:0.1.22 public:0.1.21 public:0.1.20 public:0.1.19 public:0.1.18 public:0.1.17 public:0.1.16 public:0.1.15 public:0.1.14 public:0.1.13 public:0.1.12 public:0.1.11 public:0.1.10 public:0.1.9 public:0.1.8 public:0.1.7 public:0.1.6 public:0.1.5 public:0.1.4 public:0.1.3 public:0.1.2 public:0.1.1 public:0.1.0 public:0.0.9 public:0.0.8 public:0.0.7 public:0.0.6

41 Commits
0.18.3 ... 0.19.9

Author SHA1 Message Date
vorotamoroz
47d2cf9733 bump 2023-06-09 18:48:59 +09:00
vorotamoroz
ae6a9ecee4 - New feature (For fixing a problem): 
- We can fix the database obfuscated and plain paths that have been mixed up.
- Improvements
  - Customisation Sync performance has been improved.
 2023-06-09 18:48:10 +09:00
vorotamoroz
2289bea8d9 bump 2023-06-07 17:32:28 +09:00
vorotamoroz
cda90259c5 - New feature: 
- Vault history: A tab has been implemented to give a birds-eye view of the changes that have occurred in the vault.
- Improved:
  - Log dialogue is now shown as one of tabs.
- Fixed:
  - Some minor issues has been fixed.
 2023-06-07 17:29:53 +09:00
vorotamoroz
432a211f80 Merge pull request #224 from antoKeinanen/main 
[Feature] Add password protection to askString function
 2023-06-07 17:04:31 +09:00
antoKeinanen
eaf8c4998e feat: add password protection for required inputs 2023-06-05 13:27:06 +03:00
antoKeinanen
55601f7910 feat: add option for password protection in askString function 2023-06-05 13:24:50 +03:00
vorotamoroz
13e70475d9 Add new documentation 
Thanks for your discussion!!
 2023-06-02 14:36:00 +09:00
vorotamoroz
2572177879 Merge pull request #222 from Hugo-Persson/add-troubleshooting-guide 
Added troubleshooting guide
 2023-06-02 09:25:18 +09:00
Hugo Persson
e82a2560e4 Added troubleshooting guide 2023-06-01 19:54:42 +02:00
vorotamoroz
09146591eb bump 2023-06-01 17:06:23 +09:00
vorotamoroz
69c6e57df3 Fix: 
- Fixed Setup wizard
- Set initial pane to General settings.
 2023-06-01 17:01:42 +09:00
vorotamoroz
5e181a8ec4 Update docs 2023-06-01 16:19:14 +09:00
vorotamoroz
4354cc3054 bump 2023-06-01 13:05:29 +09:00
vorotamoroz
0664427c63 Refined: 
- Configuration dialogue refined.
 2023-06-01 13:02:56 +09:00
vorotamoroz
49c4736d69 Improved: 
- Confirmation for new adapters while rebuilding.
- Batched file is now shown in digits.

Fixed:
- Some framework have been upgraded.
 2023-06-01 12:47:41 +09:00
vorotamoroz
f0ce8f0e05 Fixed: 
- Import declarations
- Logging has been tweaked
Improved:
 2023-06-01 12:36:10 +09:00
vorotamoroz
0a70afc5a3 Update issue templates 2023-05-24 11:51:53 +09:00
vorotamoroz
431239a736 Merge pull request #218 from garlic-hub/garlic-hub-patch-1 
Update setup_own_server.md
 2023-05-23 17:46:35 +09:00
vorotamoroz
1ceb671683 bump 2023-05-23 17:40:47 +09:00
vorotamoroz
ea40e5918c Fixed: 
- Now hidden file synchronisation would not be hanged, even if so many files exist.

Improved:
- Customisation sync works more smoothly.
 2023-05-23 17:39:02 +09:00
garlic-hub
64681729ff Update setup_own_server.md 2023-05-23 04:49:50 +00:00
vorotamoroz
830f2f25d1 update a dependency. 2023-05-17 16:27:35 +09:00
vorotamoroz
05f0abebf0 bump 2023-05-17 16:26:46 +09:00
vorotamoroz
842da980d7 Improved: 
- Reduced remote database checking to improve speed and reduce bandwidth.

Fixed:
- Chunks which previously misinterpreted are now interpreted correctly.
- Deleted file detection on hidden file synchronising now works fine.
- Now the Customisation sync is surely quiet while it has been disabled.
 2023-05-17 16:20:07 +09:00
vorotamoroz
d8ecbb593b bump 2023-05-09 18:03:57 +09:00
vorotamoroz
8d66c372e1 Improved: 
- Now replication will be paced by collecting chunks.
 2023-05-09 17:49:40 +09:00
vorotamoroz
7c06750d93 bump 2023-05-02 18:00:55 +09:00
vorotamoroz
808fdc0944 Fixed: 
- Fixed garbage collection error while unreferenced chunks exist many.
- Fixed filename validation on Linux.

Improved:
- Showing status is now thinned for performance.
- Enhance caching while collecting chunks.
 2023-05-02 17:59:58 +09:00
vorotamoroz
ce25eee74b bump 2023-04-30 11:31:09 +09:00
vorotamoroz
146c170dec Fixed: 
- Fixed hidden file handling on Linux

Improved:
- Now customization sync works more smoothly.
 2023-04-30 11:28:39 +09:00
vorotamoroz
cf06f878db bump 2023-04-28 14:24:37 +09:00
vorotamoroz
e77031f1cd Implemented: 
- New feature `Customization sync` has replaced `Plugin and their settings`
 2023-04-28 13:32:58 +09:00
vorotamoroz
3f2224c3a6 Merge pull request #203 from garlic-hub/garlic-hub-patch-1 
Update quick_setup.md
 2023-04-21 17:15:31 +09:00
garlic-hub
2322b5bc34 Update quick_setup.md 2023-04-20 21:30:56 +00:00
vorotamoroz
83ac5e7086 bump 2023-04-14 17:39:37 +09:00
vorotamoroz
09f35a2af4 New features: 
- Now remote database cleaning-up will be detected automatically.
- A solution selection dialogue will be shown if synchronisation is rejected after cleaning or rebuilding the remote database.
- During fetching or rebuilding, we can configure `Hidden file synchronisation` on the spot.
 2023-04-14 17:39:09 +09:00
vorotamoroz
fae0a9d76a bump 2023-04-13 17:33:28 +09:00
vorotamoroz
9a27c9bfe5 - Actions for maintaining databases moved to the 🎛️Maintain databases. 
- Clean-up of unreferenced chunks has been implemented on an **experimental**.
 2023-04-13 17:33:17 +09:00
vorotamoroz
5e75917b8d bump 2023-04-12 12:08:35 +09:00
vorotamoroz
3322d13b55 - Fixed: 
- `Fetch` and `Rebuild database` will work more safely.
- Case-sensitive renaming now works fine.
  Revoked the logic which was made at #130, however, looks fine now.
 2023-04-12 12:08:08 +09:00
39 changed files with 4473 additions and 1831 deletions
Expand all files Collapse all files

78
.github/ISSUE_TEMPLATE/issue-report.md vendored Normal file
View File

@@ -0,0 +1,78 @@
---
name: Issue report
about: Create a report to help us improve
title: ''
labels: ''
assignees: ''
---
Thank you for taking the time to report this issue!
To improve the process, I would like to ask you to let me know the information in advance.
All instructions and examples, and empty entries can be deleted.
Just for your information, a [filled example](https://docs.vrtmrz.net/LiveSync/hintandtrivia/Issue+example) is also written.
## Abstract
The synchronisation hung up immediately after connecting.
## Expected behaviour
- Synchronisation ends with the message `Replication completed`
- Everything synchronised
## Actually happened
- Synchronisation has been cancelled with the message `TypeError ... ` (captured in the attached log, around LL.10-LL.12)
- No files synchronised
## Reproducing procedure
1. Configure LiveSync as in the attached material.
2. Click the replication button on the ribbon.
3. Synchronising has begun.
4. About two or three seconds later, we got the error `TypeError ... `.
5. Replication has been stopped. No files synchronised.
Note: If you do not catch the reproducing procedure, please let me know the frequency and signs.
## Report materials
If the information is not available, do not hesitate to report it as it is. You can also of course omit it if you think this is indeed unnecessary. If it is necessary, I will ask you.
### Report from the LiveSync
For more information, please refer to [Making the report](https://docs.vrtmrz.net/LiveSync/hintandtrivia/Making+the+report).
<details>
<summary>Report from hatch</summary>
```
<!-- paste here -->
```
</details>
### Obsidian debug info
<details>
<summary>Debug info</summary>
```
<!-- paste here -->
```
</details>
### Plug-in log
We can see the log by tapping the Document box icon. If you noticed something suspicious, please let me know.
Note: **Please enable `Verbose Log`**. For detail, refer to [Logging](https://docs.vrtmrz.net/LiveSync/hintandtrivia/Logging), please.
<details>
<summary>Plug-in log</summary>
```
<!-- paste here -->
```
</details>
### Network log
Network logs displayed in DevTools will possibly help with connection-related issues. To capture that, please refer to [DevTools](https://docs.vrtmrz.net/LiveSync/hintandtrivia/DevTools).
### Screenshots
If applicable, please add screenshots to help explain your problem.
### Other information, insights and intuition.
Please provide any additional context or information about the problem.

18
README.md
View File

@@ -34,11 +34,10 @@ Useful for researchers, engineers and developers with a need to keep their notes
### Get your database ready. ### Get your database ready.
First, get your database ready. IBM Cloudant is preferred for testing. Or you can use your own server with CouchDB. For more information, refer below: First, get your database ready. fly.io is preferred for testing. Or you can use your own server with CouchDB. For more information, refer below:
1. [Setup IBM Cloudant](docs/setup_cloudant.md) 1. [Setup fly.io](docs/setup_flyio.md)
2. [Setup your CouchDB](docs/setup_own_server.md) 2. [Setup IBM Cloudant](docs/setup_cloudant.md)
3. [Setup your CouchDB](docs/setup_own_server.md)
Note: More information about alternative hosting methods is needed! Currently, [using fly.io](https://github.com/vrtmrz/obsidian-livesync/discussions/85) is being discussed.
### Configure the plugin ### Configure the plugin
@@ -54,12 +53,6 @@ Please open the configuration link again and Answer below:
If you answered `No` to both, your databases will be rebuilt by the content on your device. And the remote database will lock out other devices. You have to synchronize all your devices again. (When this time, almost all your files should be synchronized with a timestamp. So you can use an existing vault). If you answered `No` to both, your databases will be rebuilt by the content on your device. And the remote database will lock out other devices. You have to synchronize all your devices again. (When this time, almost all your files should be synchronized with a timestamp. So you can use an existing vault).
## Test Server
~~Setting up an instance of Cloudant or local CouchDB is a little complicated, so I set up a [Tasting server for self-hosted-livesync](https://olstaste.vrtmrz.net/). Try it out for free!~~
Now (30 May 2023) suspending while the server transfer.
Note: Please read "Limitations" carefully. Do not send your private vault.
## Information in StatusBar ## Information in StatusBar
Synchronization status is shown in statusbar. Synchronization status is shown in statusbar.
@@ -94,6 +87,9 @@ If you have deleted or renamed files, please wait until ⏳ icon disappeared.
Repo is here: [obsidian-livesync-webclip](https://github.com/vrtmrz/obsidian-livesync-webclip). (Docs are a work in progress.) Repo is here: [obsidian-livesync-webclip](https://github.com/vrtmrz/obsidian-livesync-webclip). (Docs are a work in progress.)
## Troubleshooting
If you are having problems getting the plugin working see: [Troubleshooting](docs/troubleshooting.md)
## License ## License
The source code is licensed under the MIT License. The source code is licensed under the MIT License.

73
docs/quick_setup.md
View File

@@ -1,86 +1,77 @@
# Quick setup # Quick setup
The Setup wizard has been implemented since v0.15.0. This simplifies the initial setup. The plugin has so many configuration options to deal with different circumstances. However, there are not so many settings that are actually used. Therefore, `The Setup wizard` has been implemented to simplify the initial setup.
Note: The subsequent devices should be set up using the `Copy setup URI` and `Open setup URI`. Note: Subsequent devices are recommended to be set up using the `Copy setup URI` and `Open setup URI`.
## How to open and use wizard ## The Setup wizard
Open from `🪄 Setup wizard` in the setting dialogue. If there is no configuration or no synchronisation settings have been activated, it should already be open. Open the `🧙‍♂️ Setup wizard` in the settings dialogue. If the plugin has not been configured before, it should already be open.
![](../images/quick_setup_1.png) ![](../images/quick_setup_1.png)
### Discard the existing configuration and set up - Discard the existing configuration and set up
If you have made any settings, this button allows you to discard them all before setting up. If you have changed any settings, this button allows you to discard all changes before setting up.
### Do not discard the existing configuration and set up - Do not discard the existing configuration and set up
Simply reconfigure. Be careful. In wizard mode, you cannot see all configuration items, even if they have been configured. Simply reconfigure. Be careful. In wizard mode, you cannot see all configuration items, even if they have been configured.
Pressing `Next` on any of these will put the configuration dialog into wizard mode. Pressing `Next` on one of the above options will put the configuration dialog into wizard mode.
### Wizard mode ### Wizard mode
![](../images/quick_setup_2.png) ![](../images/quick_setup_2.png)
We can set it up step by step. Let's see how to use it step-by-step.
## Remote Database configuration ## Remote Database configuration
### Remote database configuration ### Remote database configuration
Enter the information in the database we have set up. Enter the information for the database we have set up.
![](../images/quick_setup_3.png) ![](../images/quick_setup_3.png)
### End to End Encryption
![](../images/quick_setup_4.png) #### Test database connection and Check database configuration
If End to End encryption is enabled, the possibility of a third party who does not know the Passphrase being able to read the contents of the Remote database if they are leaked is reduced. So we strongly recommend enabling it. We can check the connectivity to the database, and the database settings.
Encryption is based on 256-bit AES-GCM.
This setting can be disabled if you are inside a closed network and it is clear that you will not be accessed by third parties.
### Test database connection and Check database configuration
Here we can check the status of the connection to the database and the database settings.
![](../images/quick_setup_5.png) ![](../images/quick_setup_5.png)
#### Test Database Connection #### Test Database Connection
Check whether we can connect to the database. If it fails, there are several reasons, but once you have done the `Check database configuration`, check if it fails there too. Check whether we can connect to the database. If it fails, there are several possible reasons, but first attempt the `Check database configuration` check to see if it fails there too.
#### Check database configuration #### Check database configuration
Check the database settings and fix any deficiencies on the spot. Check the database settings and fix any problems on the spot.
![](../images/quick_setup_6.png) ![](../images/quick_setup_6.png)
This item may vary depending on the connection. In the above case, press all three Fix buttons. This item may vary depending on the connection. In the above case, press all three Fix buttons.
If the Fix buttons disappear and all become check marks, we are done. If the Fix buttons disappear and all become check marks, we are done.
### Confidentiality configuration
![](../images/quick_setup_4.png)
Encrypt your database in case of unintended database exposure; enable End to End encryption and the contents of your notes will be encrypted at the moment it leaves the device. We strongly recommend enabling it. And `Path Obfuscation` also obfuscates filenames. Now stable and recommended.
Encryption is based on 256-bit AES-GCM.
These setting can be disabled if you are inside a closed network and it is clear that you will not be accessed by third parties.
![](../images/quick_setup_7.png) ![](../images/quick_setup_7.png)
### Next #### Next
Go to the Local Database configuration. Go to the Sync Settings.
### Discard exist database and proceed #### Discard existing database and proceed
Discard the contents of the Remote database and go to the Local Database configuration. Discard the contents of the Remote database and go to the Sync Settings.
## Local Database configuration ### Sync Settings
Finally, finish the wizard by selecting a preset for synchronisation.
![](../images/quick_setup_8.png)
Configure the local database. If we already have a Vaults with Self-hosted LiveSync installed and having the same directory name as currently we are setting up, please specify a different suffix than the Vault you have already set up here.
## Miscellaneous
Finally, finish the miscellaneous configurations and select a preset for synchronisation.
![](../images/quick_setup_9_1.png) ![](../images/quick_setup_9_1.png)
The `Show status inside editor` can be enabled to your liking. If enabled, the status is displayed in the top right-hand corner of the editor. Select any synchronisation methods we want to use and `Apply` to initialise and build the local and remote databases as required. If `All done!` is displayed, we are done. Automatically, `Copy setup URI` will open and we will be asked for a passphrase to encrypt the `Setup URI`.
![](../images/quick_setup_9_2.png)
From Presets, select the synchronisation method we want to use and `Apply` to initialise and build the local and remote databases as required.
If `All done!' is displayed, we are done. Automatically, `Copy setup URI` will open and we will be asked for a passphrase to encrypt the `Setup URI`.
![](../images/quick_setup_10.png) ![](../images/quick_setup_10.png)
@@ -88,10 +79,10 @@ Set the passphrase as you like.
The Setup URI will be copied to the clipboard, which you can then transfer to the second and subsequent devices in some way. The Setup URI will be copied to the clipboard, which you can then transfer to the second and subsequent devices in some way.
# How to set up the second and subsequent units # How to set up the second and subsequent units
After installing Self-hosted LiveSync on the device, select `Open setup URI` from the command palette and enter the setup URI you transferred. Afterwards, enter your passphrase and a setup wizard will open. After installing Self-hosted LiveSync on the first device, select `Open setup URI` from the command palette and enter the setup URI you transferred. Afterwards, enter your passphrase and a setup wizard will open.
Answer the following. Answer the following.
- `Yes` to `Importing LiveSync's conf, OK?` - `Yes` to `Importing LiveSync's conf, OK?`
- `Set it up as secondary or subsequent device` to `How would you like to set it up?`. - `Set it up as secondary or subsequent device` to `How would you like to set it up?`.
Then, The configuration will now take effect and replication will start. Your files will be synchronised soon! Then, The configuration will take effect and replication will start. Your files will be synchronised soon! You may need to close the settings dialog and reopen it to see the settings fields populated properly, but they will be set.

38
docs/quick_setup_ja.md
View File

@@ -1,10 +1,10 @@
# Quick setup # Quick setup
v0.15.0からSetup wizardが実装されました。これで、初回セットアップがシンプルになります。 このプラグインには、いろいろな状況に対応するための非常に多くの設定オプションがあります。しかし、実際に使用する設定項目はそれほど多くはありません。そこで、初期設定を簡略化するために、「セットアップウィザード」を実装しています。
※なお、次のデバイスからは、`Copy setup URI`と`Open setup URI`を使ってセットアップしてください。 ※なお、次のデバイスからは、`Copy setup URI`と`Open setup URI`を使ってセットアップしてください。
## Wizardの使い方 ## Wizardの使い方
`🪄 Setup wizard` から開きます。もしセットアップされていなかったり、同期設定が何も有効になっていない場合はデフォルトで開いています。 `🧙‍♂️ Setup wizard` から開きます。もしセットアップされていなかったり、同期設定が何も有効になっていない場合はデフォルトで開いています。
![](../images/quick_setup_1.png) ![](../images/quick_setup_1.png)
@@ -32,20 +32,12 @@ v0.15.0からSetup wizardが実装されました。これで、初回セット
これらはデータベースをセットアップした際に決めた情報です。 これらはデータベースをセットアップした際に決めた情報です。
### End to End暗号化の設定 ### Test database connectionとCheck database configuration
![](../images/quick_setup_4.png)
End to End暗号化を有効にした場合、万が一Remote databaseの内容が流出してもPassphraseを知らない第三者にそれを読まれる可能性が低くなります。そのため、有効化を強く推奨します。
暗号化は256bitのAES-GCMを採用しています。
この設定は、あなたが閉じたネットワークの内側にいて、かつ第三者からアクセスされない事が明確な場合には無効にできます。
### Test database connectionとCheck database configuraion
ここで、データベースへの接続状況と、データベース設定を確認します。 ここで、データベースへの接続状況と、データベース設定を確認します。
![](../images/quick_setup_5.png) ![](../images/quick_setup_5.png)
#### Test Database Connection #### Test Database Connection
データベースに接続出来るか自体を確認します。失敗する場合はいくつか理由がありますが、一度Check database configurationを行ってそちらでも失敗するか確認してください。 データベースに接続できるか自体を確認します。失敗する場合はいくつか理由がありますが、一度Check database configurationを行ってそちらでも失敗するか確認してください。
#### Check database configuration #### Check database configuration
データベースの設定を確認し、不備がある場合はその場で修正します。 データベースの設定を確認し、不備がある場合はその場で修正します。
@@ -55,6 +47,15 @@ End to End暗号化を有効にした場合、万が一Remote databaseの内容
この項目は接続先によって異なる場合があります。上記の場合、みっつのFixボタンを順にすべて押してください。 この項目は接続先によって異なる場合があります。上記の場合、みっつのFixボタンを順にすべて押してください。
Fixボタンがなくなり、すべてチェックマークになれば完了です。 Fixボタンがなくなり、すべてチェックマークになれば完了です。
### 機密性設定
![](../images/quick_setup_4.png)
意図しないデータベースの暴露に備えて、End to End Encryptionを有効にします。この項目を有効にした場合、デバイスを出る瞬間にノートの内容が暗号化されます。`Path Obfuscation`を有効にすると、ファイル名も難読化されます。現在は安定しているため、こちらも推奨されます。
暗号化には256bitのAES-GCMを採用しています。
これらの設定は、あなたが閉じたネットワークの内側にいて、かつ第三者からアクセスされない事が明確な場合には無効にできます。
![](../images/quick_setup_7.png) ![](../images/quick_setup_7.png)
### Next ### Next
@@ -63,20 +64,13 @@ Fixボタンがなくなり、すべてチェックマークになれば完了
### Discard exist database and proceed ### Discard exist database and proceed
すでにRemote databaseがある場合、Remote databaseの内容を破棄してから次へ進みます すでにRemote databaseがある場合、Remote databaseの内容を破棄してから次へ進みます
## Local Database confiuration
![](../images/quick_setup_8.png)
ローカルのデータベースを設定します。もし、すでにSelf-hosted LiveSyncをインストールしたVaultがあり、そのVaultと同じデータベース名を使用している場合は、ここですでに設定したVaultとは異なるsuffixを指定してください。
## Miscellaneous ## Sync Settings
最後にその他の設定を行います。 最後に同期方法の設定を行います。
![](../images/quick_setup_9_1.png) ![](../images/quick_setup_9_1.png)
`Show status inside editor`はお好みで有効化してください。有効にするとエディターの右上にステータスが表示されます。 Presetsから、いずれかの同期方法を選び`Apply`を行うと、必要に応じてローカル・リモートのデータベースを初期化・構築します。
![](../images/quick_setup_9_2.png)
Presetsから、使用する同期方法を選び`Apply`を行うと、必要に応じてローカル・リモートのデータベースを初期化・構築します。
All done! と表示されれば完了です。自動的に、`Copy setup URI`が開き、`Setup URI`を暗号化するパスフレーズを聞かれます。 All done! と表示されれば完了です。自動的に、`Copy setup URI`が開き、`Setup URI`を暗号化するパスフレーズを聞かれます。
![](../images/quick_setup_10.png) ![](../images/quick_setup_10.png)

287
docs/setup_flyio.md Normal file
View File

@@ -0,0 +1,287 @@
# Setup CouchDB on fly.io
In some cases, the use of IBM Cloudant was found to be hard. We looked for alternatives, but there were no services available. Therefore, we have to build our own servers, which is quite a challenge. In this situation, with fly.io, most of the processes are simplified and only CouchDB needs to be configured.
This is how to configure fly.io and CouchDB on it for Self-hosted LiveSync.
It generally falls within the `Free Allowances` range in most cases.
**[Automatic setup using Colaboratory](#automatic-setup-using-colaboratory) is recommended, after reading this document through. It is reproducible and hard to fail.**
## Required materials
- A valid credit or debit card.
## Warning
- It will be `your` instance. Check the log regularly.
## Prerequisites
For simplicity, the following description assumes that the settings are as shown in the table below. Please read it in accordance with the actual settings you wish to make.
| Used in the text | Meaning and where to use | Memo |
| ---------------- | --------------------------- | ------------------------------------------------------------------------ |
| campanella | Username | It is less likely to fail if it consists only of letters and numbers. |
| dfusiuada9suy | Password | |
| nrt | Region to make the instance | We can use any [region](https://fly.io/docs/reference/regions/) near us. |
## Steps with your computer
If you want to avoid installing anything, please skip to [Automatic setup using Colaboratory](#automatic-setup-using-colaboratory).
### 1. Install flyctl
- Mac or Linux
```sh
$ curl -L https://fly.io/install.sh | sh
```
- Windows
```powershell
$ iwr https://fly.io/install.ps1 -useb | iex
```
### 2. Sign up or Sign in to fly.io
- Sign up
```bash
$ fly auth signup
```
- Sign in
```bash
$ fly auth login
```
For more information, please refer [Sign up](https://fly.io/docs/hands-on/sign-up/) and [Sign in](https://fly.io/docs/hands-on/sign-in/).
### 3. Make configuration files
Please be careful, `nrt` is the region where near to Japan. Please use your preferred region.
1. Make fly.toml
```
$ flyctl launch --generate-name --detach --no-deploy --region nrt
Creating app in /home/vrtmrz/dev/fly/demo
Scanning source code
Could not find a Dockerfile, nor detect a runtime or framework from source code. Continuing with a blank app.
automatically selected personal organization: vorotamoroz
App will use 'nrt' region as primary
Created app 'billowing-dawn-6619' in organization 'personal'
Admin URL: https://fly.io/apps/billowing-dawn-6619
Hostname: billowing-dawn-6619.fly.dev
Wrote config file fly.toml
```
`billowing-dawn-6619` is an automatically generated name. It is used as the hostname. Please note it in something.
Note: we can specify this without `--generate-name`, but does not recommend in the trial phases.
1. Make volume
```
$ flyctl volumes create --region nrt couchdata --size 2 --yes
ID: vol_g67340kxgmmvydxw
Name: couchdata
App: billowing-dawn-6619
Region: nrt
Zone: 35b7
Size GB: 2
Encrypted: true
Created at: 02 Jun 23 01:19 UTC
```
3. Edit fly.toml
Changes:
- Change exposing port from `8080` to `5984`
- Mounting the volume `couchdata` created in step 2 under `/opt/couchdb/data`
- Set `campanella` for the administrator of CouchDB
- Customise CouchDB to use persistent ini-file; which is located under the data directory.
- To use Dockerfile
```diff
# fly.toml app configuration file generated for billowing-dawn-6619 on 2023-06-02T10:18:59+09:00
#
# See https://fly.io/docs/reference/configuration/ for information about how to use this file.
#
app = "billowing-dawn-6619"
primary_region = "nrt"
[http_service]
- internal_port = 8080
+ internal_port = 5984
force_https = true
auto_stop_machines = true
auto_start_machines = true
min_machines_running = 0
+[mounts]
+ source="couchdata"
+ destination="/opt/couchdb/data"
+
+[env]
+ COUCHDB_USER = "campanella"
+ ERL_FLAGS="-couch_ini /opt/couchdb/etc/default.ini /opt/couchdb/etc/default.d/ /opt/couchdb/etc/local.d /opt/couchdb/etc/local.ini /opt/couchdb/data/persistence.ini"
+
+[build]
+ dockerfile = "./Dockerfile"
```
4. Make `Dockerfile`
Create a Dockerfile that patches the start-up script to fix ini file permissions.
```dockerfile
FROM couchdb:latest
RUN sed -i '2itouch /opt/couchdb/data/persistence.ini && chmod +w /opt/couchdb/data/persistence.ini' /docker-entrypoint.sh
```
5. Set credential
```
flyctl secrets set COUCHDB_PASSWORD=dfusiuada9suy
```
### 4. Deploy
```
$ flyctl deploy --detach --remote-only
==> Verifying app config
Validating /home/vrtmrz/dev/fly/demo/fly.toml
Platform: machines
✓ Configuration is valid
--> Verified app config
==> Building image
Remote builder fly-builder-bold-sky-4515 ready
==> Creating build context
--> Creating build context done
==> Building image with Docker
--> docker host: 20.10.12 linux x86_64
-------------:SNIPPED:-------------
Watch your app at https://fly.io/apps/billowing-dawn-6619/monitoring
Provisioning ips for billowing-dawn-6619
Dedicated ipv6: 2a09:8280:1::2d:240f
Shared ipv4: 66.241.125.213
Add a dedicated ipv4 with: fly ips allocate-v4
This deployment will:
* create 1 "app" machine
No machines in group app, launching a new machine
Machine e7845d1f297183 [app] has state: started
Finished launching new machines
NOTE: The machines for [app] have services with 'auto_stop_machines = true' that will be stopped when idling
```
Now your CouchDB has been launched. (Do not forget to delete it if no longer need).
If failed, please check by `flyctl doctor`. Failure of remote build may be resolved by `flyctl` wireguard reset` or something.
```
$ flyctl status
App
Name = billowing-dawn-6619
Owner = personal
Hostname = billowing-dawn-6619.fly.dev
Image = billowing-dawn-6619:deployment-01H1WWB3CK5Z9ZX71KHBSDGHF1
Platform = machines
Machines
PROCESS ID VERSION REGION STATE CHECKS LAST UPDATED
app e7845d1f297183 1 nrt started 2023-06-02T01:43:34Z
```
### 5. Apply CouchDB configuration
After the initial setup, CouchDB needs some more customisations to be used from Self-hosted LiveSync. It can be configured in browsers or by HTTP-REST APIs.
This section is set up using the REST API.
1. Prepare environment variables.
- Mac or Linux:
```bash
export couchHost=https://billowing-dawn-6619.fly.dev
export couchUser=campanella
export couchPwd=dfusiuada9suy
```
- Windows
```powershell
set couchHost https://billowing-dawn-6619.fly.dev
set couchUser campanella
set couchPwd dfusiuada9suy
$creds = [System.Convert]::ToBase64String([System.Text.Encoding]::UTF8.GetBytes("${couchUser}:${couchPwd}"))
```
2. Perform cluster setup
- Mac or Linux
```bash
curl -X POST "${couchHost}/_cluster_setup" -H "Content-Type: application/json" -d "{\"action\":\"enable_single_node\",\"username\":\"${couchUser}\",\"password\":\"${couchPwd}\",\"bind_address\":\"0.0.0.0\",\"port\":5984,\"singlenode\":true}" --user "${couchUser}:${couchPwd}"
```
- Windows
```powershell
iwr -UseBasicParsing -Method 'POST' -ContentType 'application/json; charset=utf-8' -Headers @{ 'Authorization' = 'Basic ' + $creds } "${couchHost}/_cluster_setup" -Body "{""action"":""enable_single_node"",""username"":""${couchUser}"",""password"":""${couchPwd}"",""bind_address"":""0.0.0.0"",""port"":5984,""singlenode"":true}"
```
Note: if the response code is not 200. We have to retry the request once again.
If you run the request several times and it does not result in 200, something is wrong. Please report it.
3. Configure parameters
- Mac or Linux
```bash
curl -X PUT "${couchHost}/_node/nonode@nohost/_config/chttpd/require_valid_user" -H "Content-Type: application/json" -d '"true"' --user "${couchUser}:${couchPwd}"
curl -X PUT "${couchHost}/_node/nonode@nohost/_config/chttpd_auth/require_valid_user" -H "Content-Type: application/json" -d '"true"' --user "${couchUser}:${couchPwd}"
curl -X PUT "${couchHost}/_node/nonode@nohost/_config/httpd/WWW-Authenticate" -H "Content-Type: application/json" -d '"Basic realm=\"couchdb\""' --user "${couchUser}:${couchPwd}"
curl -X PUT "${couchHost}/_node/nonode@nohost/_config/httpd/enable_cors" -H "Content-Type: application/json" -d '"true"' --user "${couchUser}:${couchPwd}"
curl -X PUT "${couchHost}/_node/nonode@nohost/_config/chttpd/enable_cors" -H "Content-Type: application/json" -d '"true"' --user "${couchUser}:${couchPwd}"
curl -X PUT "${couchHost}/_node/nonode@nohost/_config/chttpd/max_http_request_size" -H "Content-Type: application/json" -d '"4294967296"' --user "${couchUser}:${couchPwd}"
curl -X PUT "${couchHost}/_node/nonode@nohost/_config/couchdb/max_document_size" -H "Content-Type: application/json" -d '"50000000"' --user "${couchUser}:${couchPwd}"
curl -X PUT "${couchHost}/_node/nonode@nohost/_config/cors/credentials" -H "Content-Type: application/json" -d '"true"' --user "${couchUser}:${couchPwd}"
curl -X PUT "${couchHost}/_node/nonode@nohost/_config/cors/origins" -H "Content-Type: application/json" -d '"app://obsidian.md,capacitor://localhost,http://localhost"' --user "${couchUser}:${couchPwd}"
```
- Windows
```powershell
iwr -UseBasicParsing -Method 'PUT' -ContentType 'application/json; charset=utf-8' -Headers @{ 'Authorization' = 'Basic ' + $creds } "${couchHost}/_node/nonode@nohost/_config/chttpd/require_valid_user" -Body '"true"'
iwr -UseBasicParsing -Method 'PUT' -ContentType 'application/json; charset=utf-8' -Headers @{ 'Authorization' = 'Basic ' + $creds } "${couchHost}/_node/nonode@nohost/_config/chttpd_auth/require_valid_user" -Body '"true"'
iwr -UseBasicParsing -Method 'PUT' -ContentType 'application/json; charset=utf-8' -Headers @{ 'Authorization' = 'Basic ' + $creds } "${couchHost}/_node/nonode@nohost/_config/httpd/WWW-Authenticate" -Body '"Basic realm=\"couchdb\""'
iwr -UseBasicParsing -Method 'PUT' -ContentType 'application/json; charset=utf-8' -Headers @{ 'Authorization' = 'Basic ' + $creds } "${couchHost}/_node/nonode@nohost/_config/httpd/enable_cors" -Body '"true"'
iwr -UseBasicParsing -Method 'PUT' -ContentType 'application/json; charset=utf-8' -Headers @{ 'Authorization' = 'Basic ' + $creds } "${couchHost}/_node/nonode@nohost/_config/chttpd/enable_cors" -Body '"true"'
iwr -UseBasicParsing -Method 'PUT' -ContentType 'application/json; charset=utf-8' -Headers @{ 'Authorization' = 'Basic ' + $creds } "${couchHost}/_node/nonode@nohost/_config/chttpd/max_http_request_size" -Body '"4294967296"'
iwr -UseBasicParsing -Method 'PUT' -ContentType 'application/json; charset=utf-8' -Headers @{ 'Authorization' = 'Basic ' + $creds } "${couchHost}/_node/nonode@nohost/_config/couchdb/max_document_size" -Body '"50000000"'
iwr -UseBasicParsing -Method 'PUT' -ContentType 'application/json; charset=utf-8' -Headers @{ 'Authorization' = 'Basic ' + $creds } "${couchHost}/_node/nonode@nohost/_config/cors/credentials" -Body '"true"'
iwr -UseBasicParsing -Method 'PUT' -ContentType 'application/json; charset=utf-8' -Headers @{ 'Authorization' = 'Basic ' + $creds } "${couchHost}/_node/nonode@nohost/_config/cors/origins" -Body '"app://obsidian.md,capacitor://localhost,http://localhost"'
```
Note: Each of these should also be repeated until finished in 200.
### 6. Use it from Self-hosted LiveSync
Now the CouchDB is ready to use from Self-hosted LiveSync. We can use `https://billowing-dawn-6619.fly.dev` in URI, `campanella` in `Username` and `dfusiuada9suy` in `Password` on Self-hosted LiveSync. `Database name` could be anything you want.
`Enhance chunk size` could be up to around `100`.
## Automatic setup using Colaboratory
We can perform all these steps by using [this Colaboratory notebook](https://gist.github.com/vrtmrz/b437a539af25ef191bd452aae369242f) without installing anything.
## After testing / before creating a new instance
**Be sure to delete the instance. We can check instances on the [Dashboard](https://fly.io/dashboard/personal)**

59
docs/setup_own_server.md
View File

@@ -1,13 +1,12 @@
# Setup CouchDB to your server # Setup a CouchDB server
## Configure
## Install CouchDB and access from a PC or Mac The easiest way to set up a CouchDB instance is using the official [docker image](https://hub.docker.com/_/couchdb).
The easiest way to set up the CouchDB is using the [docker image]((https://hub.docker.com/_/couchdb)). Some initial configuration is required. Create a `local.ini` to use Self-hosted LiveSync as follows:
But some additional configurations are required in `local.ini` to use from Self-hosted LiveSync, like below: ```ini
```
[couchdb] [couchdb]
single_node=true single_node=true
max_document_size = 50000000 max_document_size = 50000000
@@ -32,28 +31,62 @@ methods = GET, PUT, POST, HEAD, DELETE
max_age = 3600 max_age = 3600
``` ```
Make `local.ini` and run with docker run like this, you can launch the CouchDB. ## Run
### Docker CLI
You can launch CouchDB using your `local.ini` like this:
``` ```
$ docker run --rm -it -e COUCHDB_USER=admin -e COUCHDB_PASSWORD=password -v /path/to/local.ini:/opt/couchdb/etc/local.ini -p 5984:5984 couchdb $ docker run --rm -it -e COUCHDB_USER=admin -e COUCHDB_PASSWORD=password -v /path/to/local.ini:/opt/couchdb/etc/local.ini -p 5984:5984 couchdb
``` ```
*Remember to replace the path with the path to your local.ini* *Remember to replace the path with the path to your local.ini*
Note: At this time, the file owner of local.ini became 5984:5984. It's the limitation docker image. please change the owner before editing local.ini again.
If you could confirm that Self-hosted LiveSync can sync with the server, launch the docker image as a background as you like. Run in detached mode:
Example to run docker in detached mode:
``` ```
$ docker run -d --restart always -e COUCHDB_USER=admin -e COUCHDB_PASSWORD=password -v /path/to/local.ini:/opt/couchdb/etc/local.ini -p 5984:5984 couchdb $ docker run -d --restart always -e COUCHDB_USER=admin -e COUCHDB_PASSWORD=password -v /path/to/local.ini:/opt/couchdb/etc/local.ini -p 5984:5984 couchdb
``` ```
*Remember to replace the path with the path to your local.ini* *Remember to replace the path with the path to your local.ini*
### Docker Compose
Create a directory, place your `local.ini` within it, and create a `docker-compose.yml` alongside it. The directory structure should look similar to this:
```
obsidian-livesync
├── docker-compose.yml
└── local.ini
```
A good place to start for `docker-compose.yml`:
```yaml
version: "2.1"
services:
couchdb:
image: couchdb
container_name: obsidian-livesync
user: 1000:1000
environment:
- COUCHDB_USER=admin
- COUCHDB_PASSWORD=password
volumes:
- ./data:/opt/couchdb/data
- ./local.ini:/opt/couchdb/etc/local.ini
ports:
- 5984:5984
restart: unless-stopped
```
And finally launch the container
```
# -d will launch detached so the container runs in background
docker compose up -d
```
## Access from a mobile device ## Access from a mobile device
If you want to access Self-hosted LiveSync from mobile devices, you need a valid SSL certificate. If you want to access Self-hosted LiveSync from mobile devices, you need a valid SSL certificate.
### Testing from a mobile ### Testing from a mobile
In the testing phase, [localhost.run](http://localhost.run/) or something like services is very useful. In the testing phase, [localhost.run](https://localhost.run/) or something like services is very useful.
example on using localhost.run) example using localhost.run:
``` ```
$ ssh -R 80:localhost:5984 nokey@localhost.run $ ssh -R 80:localhost:5984 nokey@localhost.run
Warning: Permanently added the RSA host key for IP address '35.171.254.69' to the list of known hosts. Warning: Permanently added the RSA host key for IP address '35.171.254.69' to the list of known hosts.

12
docs/troubleshooting.md Normal file
View File

@@ -0,0 +1,12 @@
# Info
In this document some tips for solving issues will be given.
# Issue with syncing with mobile device
- If you are getting problem like could not sync files when doing initial sync. Try lower batch size
- Open setting
- Open Live Sync settings
- Go to sync settings
- Go down to Advanced settings.
- Lower "Batch size" and "Batch limit" and try to sync again.
- If you keep getting error keep lowering until you find the sweet spot.

BIN
images/quick_setup_1.png
View File

Binary file not shown.
Side by Side Swipe Overlay

Before

Width:  |  Height:  |  Size: 39 KiB

After

Width:  |  Height:  |  Size: 25 KiB

BIN
images/quick_setup_2.png
View File

Binary file not shown.
Side by Side Swipe Overlay

Before

Width:  |  Height:  |  Size: 16 KiB

After

Width:  |  Height:  |  Size: 6.2 KiB

BIN
images/quick_setup_4.png
View File

Binary file not shown.
Side by Side Swipe Overlay

Before

Width:  |  Height:  |  Size: 31 KiB

After

Width:  |  Height:  |  Size: 20 KiB

BIN
images/quick_setup_9_1.png
View File

Binary file not shown.
Side by Side Swipe Overlay

Before

Width:  |  Height:  |  Size: 35 KiB

After

Width:  |  Height:  |  Size: 6.4 KiB

2
manifest.json
View File

@@ -1,7 +1,7 @@
{ {
"id": "obsidian-livesync", "id": "obsidian-livesync",
"name": "Self-hosted LiveSync", "name": "Self-hosted LiveSync",
"version": "0.18.3", "version": "0.19.9",
"minAppVersion": "0.9.12", "minAppVersion": "0.9.12",
"description": "Community implementation of self-hosted livesync. Reflect your vault changes to some other devices immediately. Please make sure to disable other synchronize solutions to avoid content corruption or duplication.", "description": "Community implementation of self-hosted livesync. Reflect your vault changes to some other devices immediately. Please make sure to disable other synchronize solutions to avoid content corruption or duplication.",
"author": "vorotamoroz", "author": "vorotamoroz",

112
package-lock.json generated
View File

@@ -1,22 +1,22 @@
{ {
"name": "obsidian-livesync", "name": "obsidian-livesync",
"version": "0.18.3", "version": "0.19.9",
"lockfileVersion": 2, "lockfileVersion": 2,
"requires": true, "requires": true,
"packages": { "packages": {
"": { "": {
"name": "obsidian-livesync", "name": "obsidian-livesync",
"version": "0.18.3", "version": "0.19.9",
"license": "MIT", "license": "MIT",
"dependencies": { "dependencies": {
"diff-match-patch": "^1.0.5", "diff-match-patch": "^1.0.5",
"esbuild": "0.15.15",
"esbuild-svelte": "^0.7.3",
"idb": "^7.1.1", "idb": "^7.1.1",
"xxhash-wasm": "^0.4.2" "xxhash-wasm": "^0.4.2"
}, },
"devDependencies": { "devDependencies": {
"@tsconfig/svelte": "^4.0.1",
"@types/diff-match-patch": "^1.0.32", "@types/diff-match-patch": "^1.0.32",
"@types/node": "^20.2.5",
"@types/pouchdb": "^6.4.0", "@types/pouchdb": "^6.4.0",
"@types/pouchdb-browser": "^6.1.3", "@types/pouchdb-browser": "^6.1.3",
"@typescript-eslint/eslint-plugin": "^5.54.0", "@typescript-eslint/eslint-plugin": "^5.54.0",
@@ -39,11 +39,11 @@
"pouchdb-mapreduce": "^8.0.1", "pouchdb-mapreduce": "^8.0.1",
"pouchdb-replication": "^8.0.1", "pouchdb-replication": "^8.0.1",
"pouchdb-utils": "^8.0.1", "pouchdb-utils": "^8.0.1",
"svelte": "^3.55.1", "svelte": "^3.59.1",
"svelte-preprocess": "^5.0.1", "svelte-preprocess": "^5.0.3",
"transform-pouch": "^2.0.0", "transform-pouch": "^2.0.0",
"tslib": "^2.5.0", "tslib": "^2.5.0",
"typescript": "^4.9.5" "typescript": "^5.0.4"
} }
}, },
"node_modules/@codemirror/state": { "node_modules/@codemirror/state": {
@@ -203,6 +203,12 @@
"node": ">= 8" "node": ">= 8"
} }
}, },
"node_modules/@tsconfig/svelte": {
"version": "4.0.1",
"resolved": "https://registry.npmjs.org/@tsconfig/svelte/-/svelte-4.0.1.tgz",
"integrity": "sha512-B+XlGpmuAQzJqDoBATNCvEPqQg0HkO7S8pM14QDI5NsmtymzRexQ1N+nX2H6RTtFbuFgaZD4I8AAi8voGg0GLg==",
"dev": true
},
"node_modules/@types/codemirror": { "node_modules/@types/codemirror": {
"version": "0.0.108", "version": "0.0.108",
"resolved": "https://registry.npmjs.org/@types/codemirror/-/codemirror-0.0.108.tgz", "resolved": "https://registry.npmjs.org/@types/codemirror/-/codemirror-0.0.108.tgz",
@@ -252,9 +258,9 @@
"dev": true "dev": true
}, },
"node_modules/@types/node": { "node_modules/@types/node": {
"version": "14.17.21", "version": "20.2.5",
"resolved": "https://registry.npmjs.org/@types/node/-/node-14.17.21.tgz", "resolved": "https://registry.npmjs.org/@types/node/-/node-20.2.5.tgz",
"integrity": "sha512-zv8ukKci1mrILYiQOwGSV4FpkZhyxQtuFWGya2GujWg+zVAeRQ4qbaMmWp9vb9889CFA8JECH7lkwCL6Ygg8kA==", "integrity": "sha512-JJulVEQXmiY9Px5axXHeYGLSjhkZEnD+MDPDGbCbIAbMslkKwmygtZFy1X6s/075Yo94sf8GuSlFfPzysQrWZQ==",
"dev": true "dev": true
}, },
"node_modules/@types/pouchdb": { "node_modules/@types/pouchdb": {
@@ -443,15 +449,6 @@
"integrity": "sha512-SnHmG9wN1UVmagJOnyo/qkk0Z7gejYxOYYmaAwr5u2yFYfsupN3sg10kyzN8Hep/2zbHxCnsumxOoRIRMBwKCg==", "integrity": "sha512-SnHmG9wN1UVmagJOnyo/qkk0Z7gejYxOYYmaAwr5u2yFYfsupN3sg10kyzN8Hep/2zbHxCnsumxOoRIRMBwKCg==",
"dev": true "dev": true
}, },
"node_modules/@types/sass": {
"version": "1.43.1",
"resolved": "https://registry.npmjs.org/@types/sass/-/sass-1.43.1.tgz",
"integrity": "sha512-BPdoIt1lfJ6B7rw35ncdwBZrAssjcwzI5LByIrYs+tpXlj/CAkuVdRsgZDdP4lq5EjyWzwxZCqAoFyHKFwp32g==",
"dev": true,
"dependencies": {
"@types/node": "*"
}
},
"node_modules/@types/semver": { "node_modules/@types/semver": {
"version": "7.3.13", "version": "7.3.13",
"resolved": "https://registry.npmjs.org/@types/semver/-/semver-7.3.13.tgz", "resolved": "https://registry.npmjs.org/@types/semver/-/semver-7.3.13.tgz",
@@ -3719,23 +3716,22 @@
} }
}, },
"node_modules/svelte": { "node_modules/svelte": {
"version": "3.55.1", "version": "3.59.1",
"resolved": "https://registry.npmjs.org/svelte/-/svelte-3.55.1.tgz", "resolved": "https://registry.npmjs.org/svelte/-/svelte-3.59.1.tgz",
"integrity": "sha512-S+87/P0Ve67HxKkEV23iCdAh/SX1xiSfjF1HOglno/YTbSTW7RniICMCofWGdJJbdjw3S+0PfFb1JtGfTXE0oQ==", "integrity": "sha512-pKj8fEBmqf6mq3/NfrB9SLtcJcUvjYSWyePlfCqN9gujLB25RitWK8PvFzlwim6hD/We35KbPlRteuA6rnPGcQ==",
"dev": true, "dev": true,
"engines": { "engines": {
"node": ">= 8" "node": ">= 8"
} }
}, },
"node_modules/svelte-preprocess": { "node_modules/svelte-preprocess": {
"version": "5.0.1", "version": "5.0.4",
"resolved": "https://registry.npmjs.org/svelte-preprocess/-/svelte-preprocess-5.0.1.tgz", "resolved": "https://registry.npmjs.org/svelte-preprocess/-/svelte-preprocess-5.0.4.tgz",
"integrity": "sha512-0HXyhCoc9rsW4zGOgtInylC6qj259E1hpFnJMJWTf+aIfeqh4O/QHT31KT2hvPEqQfdjmqBR/kO2JDkkciBLrQ==", "integrity": "sha512-ABia2QegosxOGsVlsSBJvoWeXy1wUKSfF7SWJdTjLAbx/Y3SrVevvvbFNQqrSJw89+lNSsM58SipmZJ5SRi5iw==",
"dev": true, "dev": true,
"hasInstallScript": true, "hasInstallScript": true,
"dependencies": { "dependencies": {
"@types/pug": "^2.0.6", "@types/pug": "^2.0.6",
"@types/sass": "^1.43.1",
"detect-indent": "^6.1.0", "detect-indent": "^6.1.0",
"magic-string": "^0.27.0", "magic-string": "^0.27.0",
"sorcery": "^0.11.0", "sorcery": "^0.11.0",
@@ -3754,8 +3750,8 @@
"sass": "^1.26.8", "sass": "^1.26.8",
"stylus": "^0.55.0", "stylus": "^0.55.0",
"sugarss": "^2.0.0 || ^3.0.0 || ^4.0.0", "sugarss": "^2.0.0 || ^3.0.0 || ^4.0.0",
"svelte": "^3.23.0", "svelte": "^3.23.0 || ^4.0.0-next.0 || ^4.0.0",
"typescript": "^3.9.5 || ^4.0.0" "typescript": ">=3.9.5 || ^4.0.0 || ^5.0.0"
}, },
"peerDependenciesMeta": { "peerDependenciesMeta": {
"@babel/core": { "@babel/core": {
@@ -3916,16 +3912,16 @@
} }
}, },
"node_modules/typescript": { "node_modules/typescript": {
"version": "4.9.5", "version": "5.0.4",
"resolved": "https://registry.npmjs.org/typescript/-/typescript-4.9.5.tgz", "resolved": "https://registry.npmjs.org/typescript/-/typescript-5.0.4.tgz",
"integrity": "sha512-1FXk9E2Hm+QzZQ7z+McJiHL4NW1F2EzMu9Nq9i3zAaGqibafqYwCVU6WyWAuyQRRzOlxou8xZSyXLEN8oKj24g==", "integrity": "sha512-cW9T5W9xY37cc+jfEnaUvX91foxtHkza3Nw3wkoF4sSlKn0MONdkdEndig/qPBWXNkmplh3NzayQzCiHM4/hqw==",
"dev": true, "dev": true,
"bin": { "bin": {
"tsc": "bin/tsc", "tsc": "bin/tsc",
"tsserver": "bin/tsserver" "tsserver": "bin/tsserver"
}, },
"engines": { "engines": {
"node": ">=4.2.0" "node": ">=12.20"
} }
}, },
"node_modules/unbox-primitive": { "node_modules/unbox-primitive": {
@@ -4087,9 +4083,9 @@
"dev": true "dev": true
}, },
"node_modules/yaml": { "node_modules/yaml": {
"version": "2.1.3", "version": "2.2.2",
"resolved": "https://registry.npmjs.org/yaml/-/yaml-2.1.3.tgz", "resolved": "https://registry.npmjs.org/yaml/-/yaml-2.2.2.tgz",
"integrity": "sha512-AacA8nRULjKMX2DvWvOAdBZMOfQlypSFkjcOcu9FalllIDJ1kvlREzcdIZmidQUqqeMv7jorHjq2HlLv/+c2lg==", "integrity": "sha512-CBKFWExMn46Foo4cldiChEzn7S7SRV+wqiluAb6xmueD/fGyRHIhX8m14vVGgeFWjN540nKCNVj6P21eQjgTuA==",
"dev": true, "dev": true,
"engines": { "engines": {
"node": ">= 14" "node": ">= 14"
@@ -4233,6 +4229,12 @@
"fastq": "^1.6.0" "fastq": "^1.6.0"
} }
}, },
"@tsconfig/svelte": {
"version": "4.0.1",
"resolved": "https://registry.npmjs.org/@tsconfig/svelte/-/svelte-4.0.1.tgz",
"integrity": "sha512-B+XlGpmuAQzJqDoBATNCvEPqQg0HkO7S8pM14QDI5NsmtymzRexQ1N+nX2H6RTtFbuFgaZD4I8AAi8voGg0GLg==",
"dev": true
},
"@types/codemirror": { "@types/codemirror": {
"version": "0.0.108", "version": "0.0.108",
"resolved": "https://registry.npmjs.org/@types/codemirror/-/codemirror-0.0.108.tgz", "resolved": "https://registry.npmjs.org/@types/codemirror/-/codemirror-0.0.108.tgz",
@@ -4282,9 +4284,9 @@
"dev": true "dev": true
}, },
"@types/node": { "@types/node": {
"version": "14.17.21", "version": "20.2.5",
"resolved": "https://registry.npmjs.org/@types/node/-/node-14.17.21.tgz", "resolved": "https://registry.npmjs.org/@types/node/-/node-20.2.5.tgz",
"integrity": "sha512-zv8ukKci1mrILYiQOwGSV4FpkZhyxQtuFWGya2GujWg+zVAeRQ4qbaMmWp9vb9889CFA8JECH7lkwCL6Ygg8kA==", "integrity": "sha512-JJulVEQXmiY9Px5axXHeYGLSjhkZEnD+MDPDGbCbIAbMslkKwmygtZFy1X6s/075Yo94sf8GuSlFfPzysQrWZQ==",
"dev": true "dev": true
}, },
"@types/pouchdb": { "@types/pouchdb": {
@@ -4473,15 +4475,6 @@
"integrity": "sha512-SnHmG9wN1UVmagJOnyo/qkk0Z7gejYxOYYmaAwr5u2yFYfsupN3sg10kyzN8Hep/2zbHxCnsumxOoRIRMBwKCg==", "integrity": "sha512-SnHmG9wN1UVmagJOnyo/qkk0Z7gejYxOYYmaAwr5u2yFYfsupN3sg10kyzN8Hep/2zbHxCnsumxOoRIRMBwKCg==",
"dev": true "dev": true
}, },
"@types/sass": {
"version": "1.43.1",
"resolved": "https://registry.npmjs.org/@types/sass/-/sass-1.43.1.tgz",
"integrity": "sha512-BPdoIt1lfJ6B7rw35ncdwBZrAssjcwzI5LByIrYs+tpXlj/CAkuVdRsgZDdP4lq5EjyWzwxZCqAoFyHKFwp32g==",
"dev": true,
"requires": {
"@types/node": "*"
}
},
"@types/semver": { "@types/semver": {
"version": "7.3.13", "version": "7.3.13",
"resolved": "https://registry.npmjs.org/@types/semver/-/semver-7.3.13.tgz", "resolved": "https://registry.npmjs.org/@types/semver/-/semver-7.3.13.tgz",
@@ -6787,19 +6780,18 @@
"dev": true "dev": true
}, },
"svelte": { "svelte": {
"version": "3.55.1", "version": "3.59.1",
"resolved": "https://registry.npmjs.org/svelte/-/svelte-3.55.1.tgz", "resolved": "https://registry.npmjs.org/svelte/-/svelte-3.59.1.tgz",
"integrity": "sha512-S+87/P0Ve67HxKkEV23iCdAh/SX1xiSfjF1HOglno/YTbSTW7RniICMCofWGdJJbdjw3S+0PfFb1JtGfTXE0oQ==", "integrity": "sha512-pKj8fEBmqf6mq3/NfrB9SLtcJcUvjYSWyePlfCqN9gujLB25RitWK8PvFzlwim6hD/We35KbPlRteuA6rnPGcQ==",
"dev": true "dev": true
}, },
"svelte-preprocess": { "svelte-preprocess": {
"version": "5.0.1", "version": "5.0.4",
"resolved": "https://registry.npmjs.org/svelte-preprocess/-/svelte-preprocess-5.0.1.tgz", "resolved": "https://registry.npmjs.org/svelte-preprocess/-/svelte-preprocess-5.0.4.tgz",
"integrity": "sha512-0HXyhCoc9rsW4zGOgtInylC6qj259E1hpFnJMJWTf+aIfeqh4O/QHT31KT2hvPEqQfdjmqBR/kO2JDkkciBLrQ==", "integrity": "sha512-ABia2QegosxOGsVlsSBJvoWeXy1wUKSfF7SWJdTjLAbx/Y3SrVevvvbFNQqrSJw89+lNSsM58SipmZJ5SRi5iw==",
"dev": true, "dev": true,
"requires": { "requires": {
"@types/pug": "^2.0.6", "@types/pug": "^2.0.6",
"@types/sass": "^1.43.1",
"detect-indent": "^6.1.0", "detect-indent": "^6.1.0",
"magic-string": "^0.27.0", "magic-string": "^0.27.0",
"sorcery": "^0.11.0", "sorcery": "^0.11.0",
@@ -6910,9 +6902,9 @@
} }
}, },
"typescript": { "typescript": {
"version": "4.9.5", "version": "5.0.4",
"resolved": "https://registry.npmjs.org/typescript/-/typescript-4.9.5.tgz", "resolved": "https://registry.npmjs.org/typescript/-/typescript-5.0.4.tgz",
"integrity": "sha512-1FXk9E2Hm+QzZQ7z+McJiHL4NW1F2EzMu9Nq9i3zAaGqibafqYwCVU6WyWAuyQRRzOlxou8xZSyXLEN8oKj24g==", "integrity": "sha512-cW9T5W9xY37cc+jfEnaUvX91foxtHkza3Nw3wkoF4sSlKn0MONdkdEndig/qPBWXNkmplh3NzayQzCiHM4/hqw==",
"dev": true "dev": true
}, },
"unbox-primitive": { "unbox-primitive": {
@@ -7047,9 +7039,9 @@
"dev": true "dev": true
}, },
"yaml": { "yaml": {
"version": "2.1.3", "version": "2.2.2",
"resolved": "https://registry.npmjs.org/yaml/-/yaml-2.1.3.tgz", "resolved": "https://registry.npmjs.org/yaml/-/yaml-2.2.2.tgz",
"integrity": "sha512-AacA8nRULjKMX2DvWvOAdBZMOfQlypSFkjcOcu9FalllIDJ1kvlREzcdIZmidQUqqeMv7jorHjq2HlLv/+c2lg==", "integrity": "sha512-CBKFWExMn46Foo4cldiChEzn7S7SRV+wqiluAb6xmueD/fGyRHIhX8m14vVGgeFWjN540nKCNVj6P21eQjgTuA==",
"dev": true "dev": true
}, },
"yocto-queue": { "yocto-queue": {

14
package.json
View File

@@ -1,6 +1,6 @@
{ {
"name": "obsidian-livesync", "name": "obsidian-livesync",
"version": "0.18.3", "version": "0.19.9",
"description": "Reflect your vault changes to some other devices immediately. Please make sure to disable other synchronize solutions to avoid content corruption or duplication.", "description": "Reflect your vault changes to some other devices immediately. Please make sure to disable other synchronize solutions to avoid content corruption or duplication.",
"main": "main.js", "main": "main.js",
"type": "module", "type": "module",
@@ -13,7 +13,9 @@
"author": "vorotamoroz", "author": "vorotamoroz",
"license": "MIT", "license": "MIT",
"devDependencies": { "devDependencies": {
"@tsconfig/svelte": "^4.0.1",
"@types/diff-match-patch": "^1.0.32", "@types/diff-match-patch": "^1.0.32",
"@types/node": "^20.2.5",
"@types/pouchdb": "^6.4.0", "@types/pouchdb": "^6.4.0",
"@types/pouchdb-browser": "^6.1.3", "@types/pouchdb-browser": "^6.1.3",
"@typescript-eslint/eslint-plugin": "^5.54.0", "@typescript-eslint/eslint-plugin": "^5.54.0",
@@ -36,17 +38,15 @@
"pouchdb-mapreduce": "^8.0.1", "pouchdb-mapreduce": "^8.0.1",
"pouchdb-replication": "^8.0.1", "pouchdb-replication": "^8.0.1",
"pouchdb-utils": "^8.0.1", "pouchdb-utils": "^8.0.1",
"svelte": "^3.55.1", "svelte": "^3.59.1",
"svelte-preprocess": "^5.0.1", "svelte-preprocess": "^5.0.3",
"transform-pouch": "^2.0.0", "transform-pouch": "^2.0.0",
"tslib": "^2.5.0", "tslib": "^2.5.0",
"typescript": "^4.9.5" "typescript": "^5.0.4"
}, },
"dependencies": { "dependencies": {
"diff-match-patch": "^1.0.5", "diff-match-patch": "^1.0.5",
"esbuild": "0.15.15",
"esbuild-svelte": "^0.7.3",
"idb": "^7.1.1", "idb": "^7.1.1",
"xxhash-wasm": "^0.4.2" "xxhash-wasm": "^0.4.2"
} }
} }

683
src/CmdConfigSync.ts Normal file
View File

@@ -0,0 +1,683 @@
import { writable } from 'svelte/store';
import { Notice, type PluginManifest, parseYaml } from "./deps";
import type { EntryDoc, LoadedEntry, InternalFileEntry, FilePathWithPrefix, FilePath, DocumentID, AnyEntry } from "./lib/src/types";
import { LOG_LEVEL } from "./lib/src/types";
import { ICXHeader, PERIODIC_PLUGIN_SWEEP, } from "./types";
import { Parallels, delay, getDocData } from "./lib/src/utils";
import { Logger } from "./lib/src/logger";
import { WrappedNotice } from "./lib/src/wrapper";
import { base64ToArrayBuffer, arrayBufferToBase64, readString, uint8ArrayToHexString } from "./lib/src/strbin";
import { runWithLock } from "./lib/src/lock";
import { LiveSyncCommands } from "./LiveSyncCommands";
import { stripAllPrefixes } from "./lib/src/path";
import { PeriodicProcessor, askYesNo, disposeMemoObject, memoIfNotExist, memoObject, retrieveMemoObject, scheduleTask } from "./utils";
import { PluginDialogModal } from "./dialogs";
import { JsonResolveModal } from "./JsonResolveModal";
function serialize<T>(obj: T): string {
return JSON.stringify(obj, null, 1);
}
function deserialize<T>(str: string, def: T) {
try {
return JSON.parse(str) as T;
} catch (ex) {
try {
return parseYaml(str);
} catch (ex) {
return def;
}
}
}
export const pluginList = writable([] as PluginDataExDisplay[]);
export const pluginIsEnumerating = writable(false);
const encoder = new TextEncoder();
const hashString = (async (key: string) => {
// const buff = writeString(key);
const buff = encoder.encode(key);
const digest = await crypto.subtle.digest('SHA-256', buff);
return uint8ArrayToHexString(new Uint8Array(digest));
})
export type PluginDataExFile = {
filename: string,
data?: string[],
mtime: number,
size: number,
version?: string,
displayName?: string,
}
export type PluginDataExDisplay = {
documentPath: FilePathWithPrefix,
category: string,
name: string,
term: string,
displayName?: string,
files: PluginDataExFile[],
version?: string,
mtime: number,
}
export type PluginDataEx = {
documentPath?: FilePathWithPrefix,
category: string,
name: string,
displayName?: string,
term: string,
files: PluginDataExFile[],
version?: string,
mtime: number,
};
export class ConfigSync extends LiveSyncCommands {
confirmPopup: WrappedNotice = null;
get kvDB() {
return this.plugin.kvDB;
}
ensureDirectoryEx(fullPath: string) {
return this.plugin.ensureDirectoryEx(fullPath);
}
pluginDialog: PluginDialogModal = null;
periodicPluginSweepProcessor = new PeriodicProcessor(this.plugin, async () => await this.scanAllConfigFiles(false));
pluginList: PluginDataExDisplay[] = [];
showPluginSyncModal() {
if (!this.settings.usePluginSync) {
return;
}
if (this.pluginDialog != null) {
this.pluginDialog.open();
} else {
this.pluginDialog = new PluginDialogModal(this.app, this.plugin);
this.pluginDialog.open();
}
}
hidePluginSyncModal() {
if (this.pluginDialog != null) {
this.pluginDialog.close();
this.pluginDialog = null;
}
}
onunload() {
this.hidePluginSyncModal();
this.periodicPluginSweepProcessor?.disable();
}
onload() {
this.plugin.addCommand({
id: "livesync-plugin-dialog-ex",
name: "Show customization sync dialog",
callback: () => {
this.showPluginSyncModal();
},
});
}
getFileCategory(filePath: string): "CONFIG" | "THEME" | "SNIPPET" | "PLUGIN_MAIN" | "PLUGIN_ETC" | "PLUGIN_DATA" | "" {
if (filePath.split("/").length == 2 && filePath.endsWith(".json")) return "CONFIG";
if (filePath.split("/").length == 4 && filePath.startsWith(`${this.app.vault.configDir}/themes/`)) return "THEME";
if (filePath.startsWith(`${this.app.vault.configDir}/snippets/`) && filePath.endsWith(".css")) return "SNIPPET";
if (filePath.startsWith(`${this.app.vault.configDir}/plugins/`)) {
if (filePath.endsWith("/styles.css") || filePath.endsWith("/manifest.json") || filePath.endsWith("/main.js")) {
return "PLUGIN_MAIN";
} else if (filePath.endsWith("/data.json")) {
return "PLUGIN_DATA";
} else {
//TODO: to be configurable.
// With algorithm which implemented at v0.19.0, is too heavy.
return "";
// return "PLUGIN_ETC";
}
// return "PLUGIN";
}
return "";
}
isTargetPath(filePath: string): boolean {
if (!filePath.startsWith(this.app.vault.configDir)) return false;
// Idea non-filter option?
return this.getFileCategory(filePath) != "";
}
async onInitializeDatabase(showNotice: boolean) {
if (this.settings.usePluginSync) {
try {
Logger("Scanning customizations...");
await this.scanAllConfigFiles(showNotice);
Logger("Scanning customizations : done");
} catch (ex) {
Logger("Scanning customizations : failed");
Logger(ex, LOG_LEVEL.VERBOSE);
}
}
}
async beforeReplicate(showNotice: boolean) {
if (this.settings.autoSweepPlugins && this.settings.usePluginSync) {
await this.scanAllConfigFiles(showNotice);
}
}
async onResume() {
if (this.plugin.suspended) {
return;
}
if (this.settings.autoSweepPlugins && this.settings.usePluginSync) {
await this.scanAllConfigFiles(false);
}
this.periodicPluginSweepProcessor.enable(this.settings.autoSweepPluginsPeriodic && !this.settings.watchInternalFileChanges ? (PERIODIC_PLUGIN_SWEEP * 1000) : 0);
}
async reloadPluginList(showMessage: boolean) {
this.pluginList = [];
pluginList.set(this.pluginList)
await this.updatePluginList(showMessage);
}
async updatePluginList(showMessage: boolean, updatedDocumentPath?: FilePathWithPrefix): Promise<void> {
const logLevel = showMessage ? LOG_LEVEL.NOTICE : LOG_LEVEL.INFO;
// pluginList.set([]);
if (!this.settings.usePluginSync) {
this.pluginList = [];
pluginList.set(this.pluginList)
return;
}
scheduleTask("update-plugin-list-task", 200, async () => {
await runWithLock("update-plugin-list", false, async () => {
const entries = [] as PluginDataExDisplay[]
const plugins = this.localDatabase.findEntries(ICXHeader + "", `${ICXHeader}\u{10ffff}`, { include_docs: true });
const para = Parallels();
let count = 0;
pluginIsEnumerating.set(true);
let processed = false;
try {
for await (const plugin of plugins) {
const path = plugin.path || this.getPath(plugin);
if (updatedDocumentPath && updatedDocumentPath != path) {
continue;
}
processed = true;
const oldEntry = (this.pluginList.find(e => e.documentPath == path));
if (oldEntry && oldEntry.mtime == plugin.mtime) continue;
await para.wait(15);
para.add((async (v) => {
try {
count++;
if (count % 10 == 0) Logger(`Enumerating files... ${count}`, logLevel, "get-plugins");
Logger(`plugin-${path}`, LOG_LEVEL.VERBOSE);
const wx = await this.localDatabase.getDBEntry(path, null, false, false);
if (wx) {
const data = deserialize(getDocData(wx.data), {}) as PluginDataEx;
const xFiles = [] as PluginDataExFile[];
for (const file of data.files) {
const work = { ...file };
const tempStr = getDocData(work.data);
work.data = [await hashString(tempStr)];
xFiles.push(work);
}
entries.push({
...data,
documentPath: this.getPath(wx),
files: xFiles
});
}
} catch (ex) {
//TODO
Logger(`Something happened at enumerating customization :${v.path}`, LOG_LEVEL.NOTICE);
console.warn(ex);
}
}
)(plugin));
}
await para.all();
let newList = [...this.pluginList];
for (const item of entries) {
newList = newList.filter(x => x.documentPath != item.documentPath);
newList.push(item)
}
if (updatedDocumentPath != "" && !processed) newList = newList.filter(e => e.documentPath != updatedDocumentPath);
this.pluginList = newList;
pluginList.set(newList);
Logger(`All files enumerated`, logLevel, "get-plugins");
} finally {
pluginIsEnumerating.set(false);
}
});
});
// return entries;
}
async compareUsingDisplayData(dataA: PluginDataExDisplay, dataB: PluginDataExDisplay) {
const docA = await this.localDatabase.getDBEntry(dataA.documentPath);
const docB = await this.localDatabase.getDBEntry(dataB.documentPath);
if (docA && docB) {
const pluginDataA = deserialize(getDocData(docA.data), {}) as PluginDataEx;
pluginDataA.documentPath = dataA.documentPath;
const pluginDataB = deserialize(getDocData(docB.data), {}) as PluginDataEx;
pluginDataB.documentPath = dataB.documentPath;
// Use outer structure to wrap each data.
return await this.showJSONMergeDialogAndMerge(docA, docB, pluginDataA, pluginDataB);
}
return false;
}
showJSONMergeDialogAndMerge(docA: LoadedEntry, docB: LoadedEntry, pluginDataA: PluginDataEx, pluginDataB: PluginDataEx): Promise<boolean> {
const fileA = { ...pluginDataA.files[0], ctime: pluginDataA.files[0].mtime, _id: `${pluginDataA.documentPath}` as DocumentID };
const fileB = pluginDataB.files[0];
const docAx = { ...docA, ...fileA } as LoadedEntry, docBx = { ...docB, ...fileB } as LoadedEntry
return runWithLock("config:merge-data", false, () => new Promise((res) => {
Logger("Opening data-merging dialog", LOG_LEVEL.VERBOSE);
// const docs = [docA, docB];
const path = stripAllPrefixes(docAx.path.split("/").slice(-1).join("/") as FilePath);
const modal = new JsonResolveModal(this.app, path, [docAx, docBx], async (keep, result) => {
if (result == null) return res(false);
try {
res(await this.applyData(pluginDataA, result));
} catch (ex) {
Logger("Could not apply merged file");
Logger(ex, LOG_LEVEL.VERBOSE);
res(false);
}
}, "📡", "🛰️", "B");
modal.open();
}));
}
async applyData(data: PluginDataEx, content?: string): Promise<boolean> {
Logger(`Applying ${data.displayName || data.name}..`);
const baseDir = this.app.vault.configDir;
try {
if (!data.documentPath) throw "InternalError: Document path not exist";
const dx = await this.localDatabase.getDBEntry(data.documentPath);
if (dx == false) {
throw "Not found on database"
}
const loadedData = deserialize(getDocData(dx.data), {}) as PluginDataEx;
for (const f of loadedData.files) {
Logger(`Applying ${f.filename} of ${data.displayName || data.name}..`);
try {
// console.dir(f);
const path = `${baseDir}/${f.filename}`;
await this.ensureDirectoryEx(path);
if (!content) {
const dt = base64ToArrayBuffer(f.data);
await this.app.vault.adapter.writeBinary(path, dt);
} else {
await this.app.vault.adapter.write(path, content);
}
Logger(`Applying ${f.filename} of ${data.displayName || data.name}.. Done`);
} catch (ex) {
Logger(`Applying ${f.filename} of ${data.displayName || data.name}.. Failed`);
Logger(ex, LOG_LEVEL.VERBOSE);
}
}
const uPath = `${baseDir}/${loadedData.files[0].filename}` as FilePath;
await this.storeCustomizationFiles(uPath);
await this.updatePluginList(true, uPath);
await delay(100);
Logger(`Config ${data.displayName || data.name} has been applied`, LOG_LEVEL.NOTICE);
if (data.category == "PLUGIN_DATA" || data.category == "PLUGIN_MAIN") {
//@ts-ignore
const manifests = Object.values(this.app.plugins.manifests) as any as PluginManifest[];
//@ts-ignore
const enabledPlugins = this.app.plugins.enabledPlugins as Set<string>;
const pluginManifest = manifests.find((manifest) => enabledPlugins.has(manifest.id) && manifest.dir == `${baseDir}/plugins/${data.name}`);
if (pluginManifest) {
Logger(`Unloading plugin: ${pluginManifest.name}`, LOG_LEVEL.NOTICE, "plugin-reload-" + pluginManifest.id);
// @ts-ignore
await this.app.plugins.unloadPlugin(pluginManifest.id);
// @ts-ignore
await this.app.plugins.loadPlugin(pluginManifest.id);
Logger(`Plugin reloaded: ${pluginManifest.name}`, LOG_LEVEL.NOTICE, "plugin-reload-" + pluginManifest.id);
}
} else if (data.category == "CONFIG") {
scheduleTask("configReload", 250, async () => {
if (await askYesNo(this.app, "Do you want to restart and reload Obsidian now?") == "yes") {
// @ts-ignore
this.app.commands.executeCommandById("app:reload")
}
})
}
return true;
} catch (ex) {
Logger(`Applying ${data.displayName || data.name}.. Failed`);
Logger(ex, LOG_LEVEL.VERBOSE);
return false;
}
}
async deleteData(data: PluginDataEx): Promise<boolean> {
try {
if (data.documentPath) {
await this.deleteConfigOnDatabase(data.documentPath);
await this.updatePluginList(false, data.documentPath);
Logger(`Delete: ${data.documentPath}`, LOG_LEVEL.NOTICE);
}
return true;
} catch (ex) {
Logger(`Failed to delete: ${data.documentPath}`, LOG_LEVEL.NOTICE);
return false;
}
}
async parseReplicationResultItem(docs: PouchDB.Core.ExistingDocument<EntryDoc>) {
if (docs._id.startsWith(ICXHeader)) {
if (this.plugin.settings.usePluginSync) {
await this.updatePluginList(false, (docs as AnyEntry).path ? (docs as AnyEntry).path : this.getPath((docs as AnyEntry)));
}
if (this.plugin.settings.usePluginSync && this.plugin.settings.notifyPluginOrSettingUpdated) {
if (!this.pluginDialog || (this.pluginDialog && !this.pluginDialog.isOpened())) {
const fragment = createFragment((doc) => {
doc.createEl("span", null, (a) => {
a.appendText(`Some configuration has been arrived, Press `);
a.appendChild(a.createEl("a", null, (anchor) => {
anchor.text = "HERE";
anchor.addEventListener("click", () => {
this.showPluginSyncModal();
});
}));
a.appendText(` to open the config sync dialog , or press elsewhere to dismiss this message.`);
});
});
const updatedPluginKey = "popupUpdated-plugins";
scheduleTask(updatedPluginKey, 1000, async () => {
const popup = await memoIfNotExist(updatedPluginKey, () => new Notice(fragment, 0));
//@ts-ignore
const isShown = popup?.noticeEl?.isShown();
if (!isShown) {
memoObject(updatedPluginKey, new Notice(fragment, 0));
}
scheduleTask(updatedPluginKey + "-close", 20000, () => {
const popup = retrieveMemoObject<Notice>(updatedPluginKey);
if (!popup)
return;
//@ts-ignore
if (popup?.noticeEl?.isShown()) {
popup.hide();
}
disposeMemoObject(updatedPluginKey);
});
});
}
}
return true;
}
return false;
}
async realizeSettingSyncMode(): Promise<void> {
this.periodicPluginSweepProcessor?.disable();
if (this.plugin.suspended)
return;
if (!this.settings.usePluginSync) {
return;
}
if (this.settings.autoSweepPlugins) {
await this.scanAllConfigFiles(false);
}
this.periodicPluginSweepProcessor.enable(this.settings.autoSweepPluginsPeriodic && !this.settings.watchInternalFileChanges ? (PERIODIC_PLUGIN_SWEEP * 1000) : 0);
return;
}
recentProcessedInternalFiles = [] as string[];
async makeEntryFromFile(path: FilePath): Promise<false | PluginDataExFile> {
const stat = await this.app.vault.adapter.stat(path);
let version: string | undefined;
let displayName: string | undefined;
if (!stat) {
return false;
}
const contentBin = await this.app.vault.adapter.readBinary(path);
let content: string[];
try {
content = await arrayBufferToBase64(contentBin);
if (path.toLowerCase().endsWith("/manifest.json")) {
const v = readString(new Uint8Array(contentBin));
try {
const json = JSON.parse(v);
if ("version" in json) {
version = `${json.version}`;
}
if ("name" in json) {
displayName = `${json.name}`;
}
} catch (ex) {
Logger(`Configuration sync data: ${path} looks like manifest, but could not read the version`, LOG_LEVEL.INFO);
}
}
} catch (ex) {
Logger(`The file ${path} could not be encoded`);
Logger(ex, LOG_LEVEL.VERBOSE);
return false;
}
const mtime = stat.mtime;
return {
filename: path.substring(this.app.vault.configDir.length + 1),
data: content,
mtime,
size: stat.size,
version,
displayName: displayName,
}
}
filenameToUnifiedKey(path: string, termOverRide?: string) {
const term = termOverRide || this.plugin.deviceAndVaultName;
const category = this.getFileCategory(path);
const name = (category == "CONFIG" || category == "SNIPPET") ?
(path.split("/").slice(-1)[0]) :
(category == "PLUGIN_ETC" ?
path.split("/").slice(-2).join("/") :
path.split("/").slice(-2)[0]);
return `${ICXHeader}${term}/${category}/${name}.md` as FilePathWithPrefix
}
async storeCustomizationFiles(path: FilePath, termOverRide?: string) {
const term = termOverRide || this.plugin.deviceAndVaultName;
if (term == "") {
Logger("We have to configure the device name", LOG_LEVEL.NOTICE);
return;
}
const vf = this.filenameToUnifiedKey(path, term);
return await runWithLock(`plugin-${vf}`, false, async () => {
const category = this.getFileCategory(path);
let mtime = 0;
let fileTargets = [] as FilePath[];
// let savePath = "";
const name = (category == "CONFIG" || category == "SNIPPET") ?
(path.split("/").reverse()[0]) :
(path.split("/").reverse()[1]);
const parentPath = path.split("/").slice(0, -1).join("/");
const prefixedFileName = this.filenameToUnifiedKey(path, term);
const id = await this.path2id(prefixedFileName);
const dt: PluginDataEx = {
category: category,
files: [],
name: name,
mtime: 0,
term: term
}
// let scheduleKey = "";
if (category == "CONFIG" || category == "SNIPPET" || category == "PLUGIN_ETC" || category == "PLUGIN_DATA") {
fileTargets = [path];
if (category == "PLUGIN_ETC") {
dt.displayName = path.split("/").slice(-1).join("/");
}
} else if (category == "PLUGIN_MAIN") {
fileTargets = ["manifest.json", "main.js", "styles.css"].map(e => `${parentPath}/${e}` as FilePath);
} else if (category == "THEME") {
fileTargets = ["manifest.json", "theme.css"].map(e => `${parentPath}/${e}` as FilePath);
}
for (const target of fileTargets) {
const data = await this.makeEntryFromFile(target);
if (data == false) {
// Logger(`Config: skipped: ${target} `, LOG_LEVEL.VERBOSE);
continue;
}
if (data.version) {
dt.version = data.version;
}
if (data.displayName) {
dt.displayName = data.displayName;
}
// Use average for total modified time.
mtime = mtime == 0 ? data.mtime : ((data.mtime + mtime) / 2);
dt.files.push(data);
}
dt.mtime = mtime;
// Logger(`Configuration saving: ${prefixedFileName}`);
if (dt.files.length == 0) {
Logger(`Nothing left: deleting.. ${path}`);
await this.deleteConfigOnDatabase(prefixedFileName);
await this.updatePluginList(false, prefixedFileName);
return
}
const content = serialize(dt);
try {
const old = await this.localDatabase.getDBEntryMeta(prefixedFileName, null, false);
let saveData: LoadedEntry;
if (old === false) {
saveData = {
_id: id,
path: prefixedFileName,
data: content,
mtime,
ctime: mtime,
datatype: "newnote",
size: content.length,
children: [],
deleted: false,
type: "newnote",
};
} else {
if (old.mtime == mtime) {
// Logger(`STORAGE --> DB:${file.path}: (hidden) Not changed`, LOG_LEVEL.VERBOSE);
return true;
}
saveData =
{
...old,
data: content,
mtime,
size: content.length,
datatype: "newnote",
children: [],
deleted: false,
type: "newnote",
};
}
const ret = await this.localDatabase.putDBEntry(saveData);
await this.updatePluginList(false, saveData.path);
Logger(`STORAGE --> DB:${prefixedFileName}: (config) Done`);
return ret;
} catch (ex) {
Logger(`STORAGE --> DB:${prefixedFileName}: (config) Failed`);
Logger(ex, LOG_LEVEL.VERBOSE);
return false;
}
})
}
async watchVaultRawEventsAsync(path: FilePath) {
if (!this.settings.usePluginSync) return false;
if (!this.isTargetPath(path)) return false;
const stat = await this.app.vault.adapter.stat(path);
// Make sure that target is a file.
if (stat && stat.type != "file")
return false;
const storageMTime = ~~((stat && stat.mtime || 0) / 1000);
const key = `${path}-${storageMTime}`;
if (this.recentProcessedInternalFiles.contains(key)) {
// If recently processed, it may caused by self.
return true;
}
this.recentProcessedInternalFiles = [key, ...this.recentProcessedInternalFiles].slice(0, 100);
this.storeCustomizationFiles(path).then(() => {/* Fire and forget */ });
}
async scanAllConfigFiles(showMessage: boolean) {
const logLevel = showMessage ? LOG_LEVEL.NOTICE : LOG_LEVEL.INFO;
Logger("Scanning customizing files.", logLevel, "scan-all-config");
const term = this.plugin.deviceAndVaultName;
if (term == "") {
Logger("We have to configure the device name", LOG_LEVEL.NOTICE);
return;
}
const filesAll = await this.scanInternalFiles();
const files = filesAll.filter(e => this.isTargetPath(e)).map(e => ({ key: this.filenameToUnifiedKey(e), file: e }));
const virtualPathsOfLocalFiles = [...new Set(files.map(e => e.key))];
const filesOnDB = ((await this.localDatabase.allDocsRaw({ startkey: ICXHeader + "", endkey: `${ICXHeader}\u{10ffff}`, include_docs: true })).rows.map(e => e.doc) as InternalFileEntry[]).filter(e => !e.deleted);
let deleteCandidate = filesOnDB.map(e => this.getPath(e)).filter(e => e.startsWith(`${ICXHeader}${term}/`));
for (const vp of virtualPathsOfLocalFiles) {
const p = files.find(e => e.key == vp).file;
await this.storeCustomizationFiles(p);
deleteCandidate = deleteCandidate.filter(e => e != vp);
}
for (const vp of deleteCandidate) {
await this.deleteConfigOnDatabase(vp);
}
this.updatePluginList(false).then(/* fire and forget */);
}
async deleteConfigOnDatabase(prefixedFileName: FilePathWithPrefix, forceWrite = false) {
// const id = await this.path2id(prefixedFileName);
const mtime = new Date().getTime();
await runWithLock("file-x-" + prefixedFileName, false, async () => {
try {
const old = await this.localDatabase.getDBEntryMeta(prefixedFileName, null, false) as InternalFileEntry | false;
let saveData: InternalFileEntry;
if (old === false) {
Logger(`STORAGE -x> DB:${prefixedFileName}: (config) already deleted (Not found on database)`);
} else {
if (old.deleted) {
Logger(`STORAGE -x> DB:${prefixedFileName}: (config) already deleted`);
return;
}
saveData =
{
...old,
mtime,
size: 0,
children: [],
deleted: true,
type: "newnote",
};
}
await this.localDatabase.putRaw(saveData);
await this.updatePluginList(false, prefixedFileName);
Logger(`STORAGE -x> DB:${prefixedFileName}: (config) Done`);
} catch (ex) {
Logger(`STORAGE -x> DB:${prefixedFileName}: (config) Failed`);
Logger(ex, LOG_LEVEL.VERBOSE);
return false;
}
});
}
async scanInternalFiles(): Promise<FilePath[]> {
const filenames = (await this.getFiles(this.app.vault.configDir, 2)).filter(e => e.startsWith(".")).filter(e => !e.startsWith(".trash"));
return filenames as FilePath[];
}
async getFiles(
path: string,
lastDepth: number
) {
if (lastDepth == -1) return [];
const w = await this.app.vault.adapter.list(path);
let files = [
...w.files
];
for (const v of w.folders) {
files = files.concat(await this.getFiles(v, lastDepth - 1));
}
return files;
}
}

72
src/CmdHiddenFileSync.ts
View File

@@ -1,14 +1,13 @@
import { Notice, normalizePath, PluginManifest } from "./deps"; import { Notice, normalizePath, PluginManifest } from "./deps";
import { EntryDoc, LoadedEntry, LOG_LEVEL, InternalFileEntry, FilePathWithPrefix, FilePath } from "./lib/src/types"; import { EntryDoc, LoadedEntry, LOG_LEVEL, InternalFileEntry, FilePathWithPrefix, FilePath } from "./lib/src/types";
import { InternalFileInfo, ICHeader, ICHeaderEnd } from "./types"; import { InternalFileInfo, ICHeader, ICHeaderEnd } from "./types";
import { delay, isDocContentSame } from "./lib/src/utils"; import { Parallels, delay, isDocContentSame } from "./lib/src/utils";
import { Logger } from "./lib/src/logger"; import { Logger } from "./lib/src/logger";
import { PouchDB } from "./lib/src/pouchdb-browser.js"; import { PouchDB } from "./lib/src/pouchdb-browser.js";
import { disposeMemoObject, memoIfNotExist, memoObject, retrieveMemoObject, scheduleTask, trimPrefix, isIdOfInternalMetadata, PeriodicProcessor } from "./utils"; import { disposeMemoObject, memoIfNotExist, memoObject, retrieveMemoObject, scheduleTask, isInternalMetadata, PeriodicProcessor } from "./utils";
import { WrappedNotice } from "./lib/src/wrapper"; import { WrappedNotice } from "./lib/src/wrapper";
import { base64ToArrayBuffer, arrayBufferToBase64 } from "./lib/src/strbin"; import { base64ToArrayBuffer, arrayBufferToBase64 } from "./lib/src/strbin";
import { runWithLock } from "./lib/src/lock"; import { runWithLock } from "./lib/src/lock";
import { Semaphore } from "./lib/src/semaphore";
import { JsonResolveModal } from "./JsonResolveModal"; import { JsonResolveModal } from "./JsonResolveModal";
import { LiveSyncCommands } from "./LiveSyncCommands"; import { LiveSyncCommands } from "./LiveSyncCommands";
import { addPrefix, stripAllPrefixes } from "./lib/src/path"; import { addPrefix, stripAllPrefixes } from "./lib/src/path";
@@ -28,7 +27,7 @@ export class HiddenFileSync extends LiveSyncCommands {
onunload() { onunload() {
this.periodicInternalFileScanProcessor?.disable(); this.periodicInternalFileScanProcessor?.disable();
} }
onload(): void | Promise<void> { onload() {
this.plugin.addCommand({ this.plugin.addCommand({
id: "livesync-scaninternal", id: "livesync-scaninternal",
name: "Sync hidden files", name: "Sync hidden files",
@@ -78,7 +77,7 @@ export class HiddenFileSync extends LiveSyncCommands {
procInternalFiles: string[] = []; procInternalFiles: string[] = [];
async execInternalFile() { async execInternalFile() {
await runWithLock("execinternal", false, async () => { await runWithLock("execInternal", false, async () => {
const w = [...this.procInternalFiles]; const w = [...this.procInternalFiles];
this.procInternalFiles = []; this.procInternalFiles = [];
Logger(`Applying hidden ${w.length} files change...`); Logger(`Applying hidden ${w.length} files change...`);
@@ -95,6 +94,7 @@ export class HiddenFileSync extends LiveSyncCommands {
recentProcessedInternalFiles = [] as string[]; recentProcessedInternalFiles = [] as string[];
async watchVaultRawEventsAsync(path: FilePath) { async watchVaultRawEventsAsync(path: FilePath) {
if (!this.settings.syncInternalFiles) return;
const stat = await this.app.vault.adapter.stat(path); const stat = await this.app.vault.adapter.stat(path);
// sometimes folder is coming. // sometimes folder is coming.
if (stat && stat.type != "file") if (stat && stat.type != "file")
@@ -122,12 +122,6 @@ export class HiddenFileSync extends LiveSyncCommands {
await this.deleteInternalFileOnDatabase(path); await this.deleteInternalFileOnDatabase(path);
} else { } else {
await this.storeInternalFileToDatabase({ path: path, ...stat }); await this.storeInternalFileToDatabase({ path: path, ...stat });
const pluginDir = this.app.vault.configDir + "/plugins/";
const pluginFiles = ["manifest.json", "data.json", "style.css", "main.js"];
if (path.startsWith(pluginDir) && pluginFiles.some(e => path.endsWith(e)) && this.settings.usePluginSync) {
const pluginName = trimPrefix(path, pluginDir).split("/")[0];
await this.plugin.addOnPluginAndTheirSettings.sweepPlugin(false, pluginName);
}
} }
} }
@@ -138,7 +132,7 @@ export class HiddenFileSync extends LiveSyncCommands {
for await (const doc of conflicted) { for await (const doc of conflicted) {
if (!("_conflicts" in doc)) if (!("_conflicts" in doc))
continue; continue;
if (isIdOfInternalMetadata(doc._id)) { if (isInternalMetadata(doc._id)) {
await this.resolveConflictOnInternalFile(doc.path); await this.resolveConflictOnInternalFile(doc.path);
} }
} }
@@ -259,37 +253,35 @@ export class HiddenFileSync extends LiveSyncCommands {
c = pieces.shift(); c = pieces.shift();
} }
}; };
const p = [] as Promise<void>[];
const semaphore = Semaphore(10);
// Cache update time information for files which have already been processed (mainly for files that were skipped due to the same content) // Cache update time information for files which have already been processed (mainly for files that were skipped due to the same content)
let caches: { [key: string]: { storageMtime: number; docMtime: number; }; } = {}; let caches: { [key: string]: { storageMtime: number; docMtime: number; }; } = {};
caches = await this.kvDB.get<{ [key: string]: { storageMtime: number; docMtime: number; }; }>("diff-caches-internal") || {}; caches = await this.kvDB.get<{ [key: string]: { storageMtime: number; docMtime: number; }; }>("diff-caches-internal") || {};
const filesMap = files.reduce((acc, cur) => {
acc[cur.path] = cur;
return acc;
}, {} as { [key: string]: InternalFileInfo; });
const filesOnDBMap = filesOnDB.reduce((acc, cur) => {
acc[stripAllPrefixes(this.getPath(cur))] = cur;
return acc;
}, {} as { [key: string]: InternalFileEntry; });
const para = Parallels();
for (const filename of allFileNames) { for (const filename of allFileNames) {
if (!filename) continue;
processed++; processed++;
if (processed % 100 == 0) if (processed % 100 == 0) {
Logger(`Hidden file: ${processed}/${fileCount}`, logLevel, "sync_internal"); Logger(`Hidden file: ${processed}/${fileCount}`, logLevel, "sync_internal");
}
if (!filename) continue;
if (ignorePatterns.some(e => filename.match(e))) if (ignorePatterns.some(e => filename.match(e)))
continue; continue;
const fileOnStorage = files.find(e => e.path == filename); const fileOnStorage = filename in filesMap ? filesMap[filename] : undefined;
const fileOnDatabase = filesOnDB.find(e => stripAllPrefixes(this.getPath(e)) == filename); const fileOnDatabase = filename in filesOnDBMap ? filesOnDBMap[filename] : undefined;
const addProc = async (p: () => Promise<void>): Promise<void> => {
const releaser = await semaphore.acquire(1);
try {
return p();
} catch (ex) {
Logger("Some process failed", logLevel);
Logger(ex);
} finally {
releaser();
}
};
const cache = filename in caches ? caches[filename] : { storageMtime: 0, docMtime: 0 }; const cache = filename in caches ? caches[filename] : { storageMtime: 0, docMtime: 0 };
p.push(addProc(async () => { await para.wait(5);
const xFileOnStorage = fileOnStorage; const proc = (async (xFileOnStorage: InternalFileInfo, xFileOnDatabase: InternalFileEntry) => {
const xFileOnDatabase = fileOnDatabase;
if (xFileOnStorage && xFileOnDatabase) { if (xFileOnStorage && xFileOnDatabase) {
// Both => Synchronize // Both => Synchronize
if ((direction != "pullForce" && direction != "pushForce") && xFileOnDatabase.mtime == cache.docMtime && xFileOnStorage.mtime == cache.storageMtime) { if ((direction != "pullForce" && direction != "pushForce") && xFileOnDatabase.mtime == cache.docMtime && xFileOnStorage.mtime == cache.storageMtime) {
@@ -331,9 +323,11 @@ export class HiddenFileSync extends LiveSyncCommands {
throw new Error("Invalid state on hidden file sync"); throw new Error("Invalid state on hidden file sync");
// Something corrupted? // Something corrupted?
} }
}));
});
para.add(proc(fileOnStorage, fileOnDatabase))
} }
await Promise.all(p); await para.all();
await this.kvDB.set("diff-caches-internal", caches); await this.kvDB.set("diff-caches-internal", caches);
// When files has been retrieved from the database. they must be reloaded. // When files has been retrieved from the database. they must be reloaded.
@@ -500,7 +494,7 @@ export class HiddenFileSync extends LiveSyncCommands {
const mtime = new Date().getTime(); const mtime = new Date().getTime();
await runWithLock("file-" + prefixedFileName, false, async () => { await runWithLock("file-" + prefixedFileName, false, async () => {
try { try {
const old = await this.localDatabase.getDBEntry(prefixedFileName, null, false, false) as InternalFileEntry | false; const old = await this.localDatabase.getDBEntryMeta(prefixedFileName, null, true) as InternalFileEntry | false;
let saveData: InternalFileEntry; let saveData: InternalFileEntry;
if (old === false) { if (old === false) {
saveData = { saveData = {
@@ -546,7 +540,7 @@ export class HiddenFileSync extends LiveSyncCommands {
try { try {
// Check conflicted status // Check conflicted status
//TODO option //TODO option
const fileOnDB = await this.localDatabase.getDBEntry(prefixedFileName, { conflicts: true }, false, false); const fileOnDB = await this.localDatabase.getDBEntry(prefixedFileName, { conflicts: true }, false, true);
if (fileOnDB === false) if (fileOnDB === false)
throw new Error(`File not found on database.:${filename}`); throw new Error(`File not found on database.:${filename}`);
// Prevent overwrite for Prevent overwriting while some conflicted revision exists. // Prevent overwrite for Prevent overwriting while some conflicted revision exists.
@@ -613,7 +607,7 @@ export class HiddenFileSync extends LiveSyncCommands {
showJSONMergeDialogAndMerge(docA: LoadedEntry, docB: LoadedEntry): Promise<boolean> { showJSONMergeDialogAndMerge(docA: LoadedEntry, docB: LoadedEntry): Promise<boolean> {
return new Promise((res) => { return runWithLock("conflict:merge-data", false, () => new Promise((res) => {
Logger("Opening data-merging dialog", LOG_LEVEL.VERBOSE); Logger("Opening data-merging dialog", LOG_LEVEL.VERBOSE);
const docs = [docA, docB]; const docs = [docA, docB];
const path = stripAllPrefixes(docA.path); const path = stripAllPrefixes(docA.path);
@@ -624,6 +618,8 @@ export class HiddenFileSync extends LiveSyncCommands {
let needFlush = false; let needFlush = false;
if (!result && !keep) { if (!result && !keep) {
Logger(`Skipped merging: ${filename}`); Logger(`Skipped merging: ${filename}`);
res(false);
return;
} }
//Delete old revisions //Delete old revisions
if (result || keep) { if (result || keep) {
@@ -665,7 +661,7 @@ export class HiddenFileSync extends LiveSyncCommands {
} }
}); });
modal.open(); modal.open();
}); }));
} }
async scanInternalFiles(): Promise<InternalFileInfo[]> { async scanInternalFiles(): Promise<InternalFileInfo[]> {

10
src/CmdPluginAndTheirSettings.ts
View File

@@ -1,6 +1,7 @@
import { normalizePath, PluginManifest } from "./deps"; import { normalizePath, type PluginManifest } from "./deps";
import { DocumentID, EntryDoc, FilePathWithPrefix, LoadedEntry, LOG_LEVEL } from "./lib/src/types"; import type { DocumentID, EntryDoc, FilePathWithPrefix, LoadedEntry } from "./lib/src/types";
import { PluginDataEntry, PERIODIC_PLUGIN_SWEEP, PluginList, DevicePluginList, PSCHeader, PSCHeaderEnd } from "./types"; import { LOG_LEVEL } from "./lib/src/types";
import { type PluginDataEntry, PERIODIC_PLUGIN_SWEEP, type PluginList, type DevicePluginList, PSCHeader, PSCHeaderEnd } from "./types";
import { getDocData, isDocContentSame } from "./lib/src/utils"; import { getDocData, isDocContentSame } from "./lib/src/utils";
import { Logger } from "./lib/src/logger"; import { Logger } from "./lib/src/logger";
import { PouchDB } from "./lib/src/pouchdb-browser.js"; import { PouchDB } from "./lib/src/pouchdb-browser.js";
@@ -42,6 +43,7 @@ export class PluginAndTheirSettings extends LiveSyncCommands {
this.showPluginSyncModal(); this.showPluginSyncModal();
}, },
}); });
this.showPluginSyncModal();
} }
onunload() { onunload() {
this.hidePluginSyncModal(); this.hidePluginSyncModal();
@@ -165,7 +167,7 @@ export class PluginAndTheirSettings extends LiveSyncCommands {
await runWithLock("sweepplugin", true, async () => { await runWithLock("sweepplugin", true, async () => {
const logLevel = showMessage ? LOG_LEVEL.NOTICE : LOG_LEVEL.INFO; const logLevel = showMessage ? LOG_LEVEL.NOTICE : LOG_LEVEL.INFO;
if (!this.deviceAndVaultName) { if (!this.deviceAndVaultName) {
Logger("You have to set your device and vault name.", LOG_LEVEL.NOTICE); Logger("You have to set your device name.", LOG_LEVEL.NOTICE);
return; return;
} }
Logger("Scanning plugins", logLevel); Logger("Scanning plugins", logLevel);

192
src/CmdSetupLiveSync.ts
View File

@@ -1,10 +1,13 @@
import { EntryDoc, ObsidianLiveSyncSettings, LOG_LEVEL, DEFAULT_SETTINGS } from "./lib/src/types"; import { type EntryDoc, type ObsidianLiveSyncSettings, LOG_LEVEL, DEFAULT_SETTINGS } from "./lib/src/types";
import { configURIBase } from "./types"; import { configURIBase } from "./types";
import { Logger } from "./lib/src/logger"; import { Logger } from "./lib/src/logger";
import { PouchDB } from "./lib/src/pouchdb-browser.js"; import { PouchDB } from "./lib/src/pouchdb-browser.js";
import { askSelectString, askYesNo, askString } from "./utils"; import { askSelectString, askYesNo, askString } from "./utils";
import { decrypt, encrypt } from "./lib/src/e2ee_v2"; import { decrypt, encrypt } from "./lib/src/e2ee_v2";
import { LiveSyncCommands } from "./LiveSyncCommands"; import { LiveSyncCommands } from "./LiveSyncCommands";
import { delay } from "./lib/src/utils";
import { confirmWithMessage } from "./dialogs";
import { Platform } from "./deps";
export class SetupLiveSync extends LiveSyncCommands { export class SetupLiveSync extends LiveSyncCommands {
onunload() { } onunload() { }
@@ -38,7 +41,7 @@ export class SetupLiveSync extends LiveSyncCommands {
async realizeSettingSyncMode() { } async realizeSettingSyncMode() { }
async command_copySetupURI() { async command_copySetupURI() {
const encryptingPassphrase = await askString(this.app, "Encrypt your settings", "The passphrase to encrypt the setup URI", ""); const encryptingPassphrase = await askString(this.app, "Encrypt your settings", "The passphrase to encrypt the setup URI", "", true);
if (encryptingPassphrase === false) if (encryptingPassphrase === false)
return; return;
const setting = { ...this.settings, configPassphraseStore: "", encryptedCouchDBConnection: "", encryptedPassphrase: "" }; const setting = { ...this.settings, configPassphraseStore: "", encryptedCouchDBConnection: "", encryptedPassphrase: "" };
@@ -54,7 +57,7 @@ export class SetupLiveSync extends LiveSyncCommands {
Logger("Setup URI copied to clipboard", LOG_LEVEL.NOTICE); Logger("Setup URI copied to clipboard", LOG_LEVEL.NOTICE);
} }
async command_copySetupURIFull() { async command_copySetupURIFull() {
const encryptingPassphrase = await askString(this.app, "Encrypt your settings", "The passphrase to encrypt the setup URI", ""); const encryptingPassphrase = await askString(this.app, "Encrypt your settings", "The passphrase to encrypt the setup URI", "", true);
if (encryptingPassphrase === false) if (encryptingPassphrase === false)
return; return;
const setting = { ...this.settings, configPassphraseStore: "", encryptedCouchDBConnection: "", encryptedPassphrase: "" }; const setting = { ...this.settings, configPassphraseStore: "", encryptedCouchDBConnection: "", encryptedPassphrase: "" };
@@ -78,7 +81,7 @@ export class SetupLiveSync extends LiveSyncCommands {
async setupWizard(confString: string) { async setupWizard(confString: string) {
try { try {
const oldConf = JSON.parse(JSON.stringify(this.settings)); const oldConf = JSON.parse(JSON.stringify(this.settings));
const encryptingPassphrase = await askString(this.app, "Passphrase", "The passphrase to decrypt your setup URI", ""); const encryptingPassphrase = await askString(this.app, "Passphrase", "The passphrase to decrypt your setup URI", "", true);
if (encryptingPassphrase === false) if (encryptingPassphrase === false)
return; return;
const newConf = await JSON.parse(await decrypt(confString, encryptingPassphrase, false)); const newConf = await JSON.parse(await decrypt(confString, encryptingPassphrase, false));
@@ -97,7 +100,8 @@ export class SetupLiveSync extends LiveSyncCommands {
const setupAsNew = "Set it up as secondary or subsequent device"; const setupAsNew = "Set it up as secondary or subsequent device";
const setupAgain = "Reconfigure and reconstitute the data"; const setupAgain = "Reconfigure and reconstitute the data";
const setupManually = "Leave everything to me"; const setupManually = "Leave everything to me";
newSettingW.syncInternalFiles = false;
newSettingW.usePluginSync = false;
const setupType = await askSelectString(this.app, "How would you like to set it up?", [setupAsNew, setupAgain, setupJustImport, setupManually]); const setupType = await askSelectString(this.app, "How would you like to set it up?", [setupAsNew, setupAgain, setupJustImport, setupManually]);
if (setupType == setupJustImport) { if (setupType == setupJustImport) {
this.plugin.settings = newSettingW; this.plugin.settings = newSettingW;
@@ -106,11 +110,7 @@ export class SetupLiveSync extends LiveSyncCommands {
} else if (setupType == setupAsNew) { } else if (setupType == setupAsNew) {
this.plugin.settings = newSettingW; this.plugin.settings = newSettingW;
this.plugin.usedPassphrase = ""; this.plugin.usedPassphrase = "";
await this.plugin.saveSettings(); await this.fetchLocal();
await this.plugin.resetLocalDatabase();
await this.plugin.localDatabase.initializeDatabase();
await this.plugin.markRemoteResolved();
await this.plugin.replicate(true);
} else if (setupType == setupAgain) { } else if (setupType == setupAgain) {
const confirm = "I know this operation will rebuild all my databases with files on this device, and files that are on the remote database and I didn't synchronize to any other devices will be lost and want to proceed indeed."; const confirm = "I know this operation will rebuild all my databases with files on this device, and files that are on the remote database and I didn't synchronize to any other devices will be lost and want to proceed indeed.";
if (await askSelectString(this.app, "Do you really want to do this?", ["Cancel", confirm]) != confirm) { if (await askSelectString(this.app, "Do you really want to do this?", ["Cancel", confirm]) != confirm) {
@@ -118,15 +118,7 @@ export class SetupLiveSync extends LiveSyncCommands {
} }
this.plugin.settings = newSettingW; this.plugin.settings = newSettingW;
this.plugin.usedPassphrase = ""; this.plugin.usedPassphrase = "";
await this.plugin.saveSettings(); await this.rebuildEverything();
await this.plugin.resetLocalDatabase();
await this.plugin.localDatabase.initializeDatabase();
await this.plugin.initializeDatabase(true);
await this.plugin.tryResetRemoteDatabase();
await this.plugin.markRemoteLocked();
await this.plugin.markRemoteResolved();
await this.plugin.replicate(true);
} else if (setupType == setupManually) { } else if (setupType == setupManually) {
const keepLocalDB = await askYesNo(this.app, "Keep local DB?"); const keepLocalDB = await askYesNo(this.app, "Keep local DB?");
const keepRemoteDB = await askYesNo(this.app, "Keep remote DB?"); const keepRemoteDB = await askYesNo(this.app, "Keep remote DB?");
@@ -134,6 +126,8 @@ export class SetupLiveSync extends LiveSyncCommands {
// nothing to do. so peaceful. // nothing to do. so peaceful.
this.plugin.settings = newSettingW; this.plugin.settings = newSettingW;
this.plugin.usedPassphrase = ""; this.plugin.usedPassphrase = "";
this.suspendAllSync();
this.suspendExtraSync();
await this.plugin.saveSettings(); await this.plugin.saveSettings();
const replicate = await askYesNo(this.app, "Unlock and replicate?"); const replicate = await askYesNo(this.app, "Unlock and replicate?");
if (replicate == "yes") { if (replicate == "yes") {
@@ -189,4 +183,164 @@ export class SetupLiveSync extends LiveSyncCommands {
Logger("Couldn't parse or decrypt configuration uri.", LOG_LEVEL.NOTICE); Logger("Couldn't parse or decrypt configuration uri.", LOG_LEVEL.NOTICE);
} }
} }
suspendExtraSync() {
Logger("Hidden files and plugin synchronization have been temporarily disabled. Please enable them after the fetching, if you need them.", LOG_LEVEL.NOTICE)
this.plugin.settings.syncInternalFiles = false;
this.plugin.settings.usePluginSync = false;
this.plugin.settings.autoSweepPlugins = false;
}
async askHiddenFileConfiguration(opt: { enableFetch?: boolean, enableOverwrite?: boolean }) {
this.plugin.addOnSetup.suspendExtraSync();
const message = `Would you like to enable \`Hidden File Synchronization\` or \`Customization sync\`?
${opt.enableFetch ? " - Fetch: Use files stored from other devices. \n" : ""}${opt.enableOverwrite ? "- Overwrite: Use files from this device. \n" : ""}- Custom: Synchronize only customization files with a dedicated interface.
- Keep them disabled: Do not use hidden file synchronization.
Of course, we are able to disable these features.`
const CHOICE_FETCH = "Fetch";
const CHOICE_OVERWRITE = "Overwrite";
const CHOICE_CUSTOMIZE = "Custom";
const CHOICE_DISMISS = "keep them disabled";
const choices = [];
if (opt?.enableFetch) {
choices.push(CHOICE_FETCH);
}
if (opt?.enableOverwrite) {
choices.push(CHOICE_OVERWRITE);
}
choices.push(CHOICE_CUSTOMIZE);
choices.push(CHOICE_DISMISS);
const ret = await confirmWithMessage(this.plugin, "Hidden file sync", message, choices, CHOICE_DISMISS, 40);
if (ret == CHOICE_FETCH) {
await this.configureHiddenFileSync("FETCH");
} else if (ret == CHOICE_OVERWRITE) {
await this.configureHiddenFileSync("OVERWRITE");
} else if (ret == CHOICE_DISMISS) {
await this.configureHiddenFileSync("DISABLE");
} else if (ret == CHOICE_CUSTOMIZE) {
await this.configureHiddenFileSync("CUSTOMIZE");
}
}
async configureHiddenFileSync(mode: "FETCH" | "OVERWRITE" | "MERGE" | "DISABLE" | "CUSTOMIZE") {
this.plugin.addOnSetup.suspendExtraSync();
if (mode == "DISABLE") {
this.plugin.settings.syncInternalFiles = false;
this.plugin.settings.usePluginSync = false;
await this.plugin.saveSettings();
return;
}
if (mode != "CUSTOMIZE") {
Logger("Gathering files for enabling Hidden File Sync", LOG_LEVEL.NOTICE);
if (mode == "FETCH") {
await this.plugin.addOnHiddenFileSync.syncInternalFilesAndDatabase("pullForce", true);
} else if (mode == "OVERWRITE") {
await this.plugin.addOnHiddenFileSync.syncInternalFilesAndDatabase("pushForce", true);
} else if (mode == "MERGE") {
await this.plugin.addOnHiddenFileSync.syncInternalFilesAndDatabase("safe", true);
}
this.plugin.settings.syncInternalFiles = true;
await this.plugin.saveSettings();
Logger(`Done! Restarting the app is strongly recommended!`, LOG_LEVEL.NOTICE);
} else if (mode == "CUSTOMIZE") {
if (!this.plugin.deviceAndVaultName) {
let name = await askString(this.app, "Device name", "Please set this device name", `desktop`);
if (!name) {
if (Platform.isAndroidApp) {
name = "android-app"
} else if (Platform.isIosApp) {
name = "ios"
} else if (Platform.isMacOS) {
name = "macos"
} else if (Platform.isMobileApp) {
name = "mobile-app"
} else if (Platform.isMobile) {
name = "mobile"
} else if (Platform.isSafari) {
name = "safari"
} else if (Platform.isDesktop) {
name = "desktop"
} else if (Platform.isDesktopApp) {
name = "desktop-app"
} else {
name = "unknown"
}
name = name + Math.random().toString(36).slice(-4);
}
this.plugin.deviceAndVaultName = name;
}
this.plugin.settings.usePluginSync = true;
await this.plugin.saveSettings();
await this.plugin.addOnConfigSync.scanAllConfigFiles(true);
}
}
suspendAllSync() {
this.plugin.settings.liveSync = false;
this.plugin.settings.periodicReplication = false;
this.plugin.settings.syncOnSave = false;
this.plugin.settings.syncOnStart = false;
this.plugin.settings.syncOnFileOpen = false;
this.plugin.settings.syncAfterMerge = false;
//this.suspendExtraSync();
}
async askUseNewAdapter() {
if (!this.plugin.settings.useIndexedDBAdapter) {
const message = `Now this plugin has been configured to use the old database adapter for keeping compatibility. Do you want to deactivate it?`;
const CHOICE_YES = "Yes, disable and use latest";
const CHOICE_NO = "No, keep compatibility";
const choices = [CHOICE_YES, CHOICE_NO];
const ret = await confirmWithMessage(this.plugin, "Database adapter", message, choices, CHOICE_YES, 10);
if (ret == CHOICE_YES) {
this.plugin.settings.useIndexedDBAdapter = false;
}
}
}
async fetchLocal() {
this.suspendExtraSync();
this.askUseNewAdapter();
await this.plugin.realizeSettingSyncMode();
await this.plugin.resetLocalDatabase();
await delay(1000);
await this.plugin.markRemoteResolved();
await this.plugin.openDatabase();
this.plugin.isReady = true;
await delay(500);
await this.plugin.replicateAllFromServer(true);
await delay(1000);
await this.plugin.replicateAllFromServer(true);
await this.askHiddenFileConfiguration({ enableFetch: true });
}
async rebuildRemote() {
this.suspendExtraSync();
await this.plugin.realizeSettingSyncMode();
await this.plugin.markRemoteLocked();
await this.plugin.tryResetRemoteDatabase();
await this.plugin.markRemoteLocked();
await delay(500);
await this.askHiddenFileConfiguration({ enableOverwrite: true });
await delay(1000);
await this.plugin.replicateAllToServer(true);
await delay(1000);
await this.plugin.replicateAllToServer(true);
}
async rebuildEverything() {
this.suspendExtraSync();
this.askUseNewAdapter();
await this.plugin.realizeSettingSyncMode();
await this.plugin.resetLocalDatabase();
await delay(1000);
await this.plugin.initializeDatabase(true);
await this.plugin.markRemoteLocked();
await this.plugin.tryResetRemoteDatabase();
await this.plugin.markRemoteLocked();
await delay(500);
await this.askHiddenFileConfiguration({ enableOverwrite: true });
await delay(1000);
await this.plugin.replicateAllToServer(true);
await delay(1000);
await this.plugin.replicateAllToServer(true);
}
} }

29
src/DocumentHistoryModal.ts
View File

@@ -3,7 +3,7 @@ import { getPathFromTFile, isValidPath } from "./utils";
import { base64ToArrayBuffer, base64ToString, escapeStringToHTML } from "./lib/src/strbin"; import { base64ToArrayBuffer, base64ToString, escapeStringToHTML } from "./lib/src/strbin";
import ObsidianLiveSyncPlugin from "./main"; import ObsidianLiveSyncPlugin from "./main";
import { DIFF_DELETE, DIFF_EQUAL, DIFF_INSERT, diff_match_patch } from "diff-match-patch"; import { DIFF_DELETE, DIFF_EQUAL, DIFF_INSERT, diff_match_patch } from "diff-match-patch";
import { DocumentID, FilePathWithPrefix, LoadedEntry, LOG_LEVEL } from "./lib/src/types"; import { type DocumentID, type FilePathWithPrefix, type LoadedEntry, LOG_LEVEL } from "./lib/src/types";
import { Logger } from "./lib/src/logger"; import { Logger } from "./lib/src/logger";
import { isErrorOfMissingDoc } from "./lib/src/utils_couchdb"; import { isErrorOfMissingDoc } from "./lib/src/utils_couchdb";
import { getDocData } from "./lib/src/utils"; import { getDocData } from "./lib/src/utils";
@@ -24,12 +24,14 @@ export class DocumentHistoryModal extends Modal {
currentDoc: LoadedEntry; currentDoc: LoadedEntry;
currentText = ""; currentText = "";
currentDeleted = false; currentDeleted = false;
initialRev: string;
constructor(app: App, plugin: ObsidianLiveSyncPlugin, file: TFile | FilePathWithPrefix, id: DocumentID) { constructor(app: App, plugin: ObsidianLiveSyncPlugin, file: TFile | FilePathWithPrefix, id: DocumentID, revision?: string) {
super(app); super(app);
this.plugin = plugin; this.plugin = plugin;
this.file = (file instanceof TFile) ? getPathFromTFile(file) : file; this.file = (file instanceof TFile) ? getPathFromTFile(file) : file;
this.id = id; this.id = id;
this.initialRev = revision;
if (!file) { if (!file) {
this.file = this.plugin.id2path(id, null); this.file = this.plugin.id2path(id, null);
} }
@@ -38,7 +40,7 @@ export class DocumentHistoryModal extends Modal {
} }
} }
async loadFile() { async loadFile(initialRev: string) {
if (!this.id) { if (!this.id) {
this.id = await this.plugin.path2id(this.file); this.id = await this.plugin.path2id(this.file);
} }
@@ -49,7 +51,7 @@ export class DocumentHistoryModal extends Modal {
this.range.max = `${this.revs_info.length - 1}`; this.range.max = `${this.revs_info.length - 1}`;
this.range.value = this.range.max; this.range.value = this.range.max;
this.fileInfo.setText(`${this.file} / ${this.revs_info.length} revisions`); this.fileInfo.setText(`${this.file} / ${this.revs_info.length} revisions`);
await this.loadRevs(); await this.loadRevs(initialRev);
} catch (ex) { } catch (ex) {
if (isErrorOfMissingDoc(ex)) { if (isErrorOfMissingDoc(ex)) {
this.range.max = "0"; this.range.max = "0";
@@ -63,18 +65,27 @@ export class DocumentHistoryModal extends Modal {
} }
} }
} }
async loadRevs() { async loadRevs(initialRev?: string) {
if (this.revs_info.length == 0) return; if (this.revs_info.length == 0) return;
const db = this.plugin.localDatabase; if (initialRev) {
const rIndex = this.revs_info.findIndex(e => e.rev == initialRev);
if (rIndex >= 0) {
this.range.value = `${this.revs_info.length - 1 - rIndex}`;
}
}
const index = this.revs_info.length - 1 - (this.range.value as any) / 1; const index = this.revs_info.length - 1 - (this.range.value as any) / 1;
const rev = this.revs_info[index]; const rev = this.revs_info[index];
const w = await db.getDBEntry(this.file, { rev: rev.rev }, false, false, true); await this.showExactRev(rev.rev);
}
async showExactRev(rev: string) {
const db = this.plugin.localDatabase;
const w = await db.getDBEntry(this.file, { rev: rev }, false, false, true);
this.currentText = ""; this.currentText = "";
this.currentDeleted = false; this.currentDeleted = false;
if (w === false) { if (w === false) {
this.currentDeleted = true; this.currentDeleted = true;
this.info.innerHTML = ""; this.info.innerHTML = "";
this.contentView.innerHTML = `Could not read this revision<br>(${rev.rev})`; this.contentView.innerHTML = `Could not read this revision<br>(${rev})`;
} else { } else {
this.currentDoc = w; this.currentDoc = w;
this.info.innerHTML = `Modified:${new Date(w.mtime).toLocaleString()}`; this.info.innerHTML = `Modified:${new Date(w.mtime).toLocaleString()}`;
@@ -158,7 +169,7 @@ export class DocumentHistoryModal extends Modal {
.addClass("op-info"); .addClass("op-info");
this.info = contentEl.createDiv(""); this.info = contentEl.createDiv("");
this.info.addClass("op-info"); this.info.addClass("op-info");
this.loadFile(); this.loadFile(this.initialRev);
const div = contentEl.createDiv({ text: "Loading old revisions..." }); const div = contentEl.createDiv({ text: "Loading old revisions..." });
this.contentView = div; this.contentView = div;
div.addClass("op-scrollable"); div.addClass("op-scrollable");

328
src/GlobalHistory.svelte Normal file
View File

@@ -0,0 +1,328 @@
<script lang="ts">
import ObsidianLiveSyncPlugin from "./main";
import { onDestroy, onMount } from "svelte";
import type { AnyEntry, FilePathWithPrefix } from "./lib/src/types";
import { getDocData, isDocContentSame } from "./lib/src/utils";
import { diff_match_patch } from "diff-match-patch";
import { DocumentHistoryModal } from "./DocumentHistoryModal";
import { isPlainText, stripAllPrefixes } from "./lib/src/path";
import { TFile } from "./deps";
import { arrayBufferToBase64 } from "./lib/src/strbin";
export let plugin: ObsidianLiveSyncPlugin;
let showDiffInfo = false;
let showChunkCorrected = false;
let checkStorageDiff = false;
let range_from_epoch = Date.now() - 3600000 * 24 * 7;
let range_to_epoch = Date.now() + 3600000 * 24 * 2;
const timezoneOffset = new Date().getTimezoneOffset();
let dispDateFrom = new Date(range_from_epoch - timezoneOffset).toISOString().split("T")[0];
let dispDateTo = new Date(range_to_epoch - timezoneOffset).toISOString().split("T")[0];
$: {
range_from_epoch = new Date(dispDateFrom).getTime() + timezoneOffset;
range_to_epoch = new Date(dispDateTo).getTime() + timezoneOffset;
getHistory(showDiffInfo, showChunkCorrected, checkStorageDiff);
}
function mtimeToDate(mtime: number) {
return new Date(mtime).toLocaleString();
}
type HistoryData = {
id: string;
rev: string;
path: string;
dirname: string;
filename: string;
mtime: number;
mtimeDisp: string;
isDeleted: boolean;
size: number;
changes: string;
chunks: string;
isPlain: boolean;
};
let history = [] as HistoryData[];
let loading = false;
async function fetchChanges(): Promise<HistoryData[]> {
try {
const db = plugin.localDatabase;
let result = [] as typeof history;
for await (const docA of db.findAllNormalDocs()) {
if (docA.mtime < range_from_epoch) {
continue;
}
if (docA.type != "newnote" && docA.type != "plain") continue;
const path = plugin.getPath(docA as AnyEntry);
const isPlain = isPlainText(docA.path);
const revs = await db.getRaw(docA._id, { revs_info: true });
let p: string = undefined;
const reversedRevs = revs._revs_info.reverse();
const DIFF_DELETE = -1;
const DIFF_EQUAL = 0;
const DIFF_INSERT = 1;
for (const revInfo of reversedRevs) {
if (revInfo.status == "available") {
const doc =
(!isPlain && showDiffInfo) || (checkStorageDiff && revInfo.rev == docA._rev)
? await db.getDBEntry(path, { rev: revInfo.rev }, false, false, true)
: await db.getDBEntryMeta(path, { rev: revInfo.rev }, true);
if (doc === false) continue;
const rev = revInfo.rev;
const mtime = "mtime" in doc ? doc.mtime : 0;
if (range_from_epoch > mtime) {
continue;
}
if (range_to_epoch < mtime) {
continue;
}
let diffDetail = "";
if (showDiffInfo && !isPlain) {
const data = getDocData(doc.data);
if (p === undefined) {
p = data;
}
if (p != data) {
const dmp = new diff_match_patch();
const diff = dmp.diff_main(p, data);
dmp.diff_cleanupSemantic(diff);
p = data;
const pxinit = {
[DIFF_DELETE]: 0,
[DIFF_EQUAL]: 0,
[DIFF_INSERT]: 0,
} as { [key: number]: number };
const px = diff.reduce((p, c) => ({ ...p, [c[0]]: (p[c[0]] ?? 0) + c[1].length }), pxinit);
diffDetail = `-${px[DIFF_DELETE]}, +${px[DIFF_INSERT]}`;
}
}
const isDeleted = doc._deleted || (doc as any)?.deleted || false;
if (isDeleted) {
diffDetail += " 🗑️";
}
if (rev == docA._rev) {
if (checkStorageDiff) {
const abs = plugin.app.vault.getAbstractFileByPath(stripAllPrefixes(plugin.getPath(docA)));
if (abs instanceof TFile) {
let result = false;
if (isPlainText(docA.path)) {
const data = await plugin.app.vault.read(abs);
result = isDocContentSame(data, doc.data);
} else {
const data = await plugin.app.vault.readBinary(abs);
const dataEEncoded = await arrayBufferToBase64(data);
result = isDocContentSame(dataEEncoded, doc.data);
}
if (result) {
diffDetail += " ⚖️";
} else {
diffDetail += " ⚠️";
}
}
}
}
const docPath = plugin.getPath(doc as AnyEntry);
const [filename, ...pathItems] = docPath.split("/").reverse();
let chunksStatus = "";
if (showChunkCorrected) {
const chunks = (doc as any)?.children ?? [];
const loadedChunks = await db.allDocsRaw({ keys: [...chunks] });
const totalCount = loadedChunks.rows.length;
const errorCount = loadedChunks.rows.filter((e) => "error" in e).length;
if (errorCount == 0) {
chunksStatus = `✅ ${totalCount}`;
} else {
chunksStatus = `🔎 ${errorCount} ✅ ${totalCount}`;
}
}
result.push({
id: doc._id,
rev: doc._rev,
path: docPath,
dirname: pathItems.reverse().join("/"),
filename: filename,
mtime: mtime,
mtimeDisp: mtimeToDate(mtime),
size: (doc as any)?.size ?? 0,
isDeleted: isDeleted,
changes: diffDetail,
chunks: chunksStatus,
isPlain: isPlain,
});
}
}
}
return [...result].sort((a, b) => b.mtime - a.mtime);
} finally {
loading = false;
}
}
async function getHistory(showDiffInfo: boolean, showChunkCorrected: boolean, checkStorageDiff: boolean) {
loading = true;
const newDisplay = [];
const page = await fetchChanges();
newDisplay.push(...page);
history = [...newDisplay];
}
function nextWeek() {
dispDateTo = new Date(range_to_epoch - timezoneOffset + 3600 * 1000 * 24 * 7).toISOString().split("T")[0];
}
function prevWeek() {
dispDateFrom = new Date(range_from_epoch - timezoneOffset - 3600 * 1000 * 24 * 7).toISOString().split("T")[0];
}
onMount(async () => {
await getHistory(showDiffInfo, showChunkCorrected, checkStorageDiff);
});
onDestroy(() => {});
function showHistory(file: string, rev: string) {
new DocumentHistoryModal(plugin.app, plugin, file as unknown as FilePathWithPrefix, null, rev).open();
}
function openFile(file: string) {
plugin.app.workspace.openLinkText(file, file);
}
</script>
<div class="globalhistory">
<h1>Vault history</h1>
<div class="control">
<div class="row"><label for="">From:</label><input type="date" bind:value={dispDateFrom} disabled={loading} /></div>
<div class="row"><label for="">To:</label><input type="date" bind:value={dispDateTo} disabled={loading} /></div>
<div class="row">
<label for="">Info:</label>
<label><input type="checkbox" bind:checked={showDiffInfo} disabled={loading} /><span>Diff</span></label>
<label><input type="checkbox" bind:checked={showChunkCorrected} disabled={loading} /><span>Chunks</span></label>
<label><input type="checkbox" bind:checked={checkStorageDiff} disabled={loading} /><span>File integrity</span></label>
</div>
</div>
{#if loading}
<div class="">Gathering information...</div>
{/if}
<table>
<tr>
<th> Date </th>
<th> Path </th>
<th> Rev </th>
<th> Stat </th>
{#if showChunkCorrected}
<th> Chunks </th>
{/if}
</tr>
<tr>
<td colspan="5" class="more">
{#if loading}
<div class="" />
{:else}
<div><button on:click={() => nextWeek()}>+1 week</button></div>
{/if}
</td>
</tr>
{#each history as entry}
<tr>
<td class="mtime">
{entry.mtimeDisp}
</td>
<td class="path">
<div class="filenames">
<span class="path">/{entry.dirname.split("/").join(`​/`)}</span>
<span class="filename"><a on:click={() => openFile(entry.path)}>{entry.filename}</a></span>
</div>
</td>
<td>
<span class="rev">
{#if entry.isPlain}
<a on:click={() => showHistory(entry.path, entry.rev)}>{entry.rev}</a>
{:else}
{entry.rev}
{/if}
</span>
</td>
<td>
{entry.changes}
</td>
{#if showChunkCorrected}
<td>
{entry.chunks}
</td>
{/if}
</tr>
{/each}
<tr>
<td colspan="5" class="more">
{#if loading}
<div class="" />
{:else}
<div><button on:click={() => prevWeek()}>+1 week</button></div>
{/if}
</td>
</tr>
</table>
</div>
<style>
* {
box-sizing: border-box;
}
.globalhistory {
margin-bottom: 2em;
}
table {
width: 100%;
}
.more > div {
display: flex;
}
.more > div > button {
flex-grow: 1;
}
th {
position: sticky;
top: 0;
backdrop-filter: blur(10px);
}
td.mtime {
white-space: break-spaces;
}
td.path {
word-break: break-word;
}
.row {
display: flex;
flex-direction: row;
flex-wrap: wrap;
}
.row > label {
display: flex;
align-items: center;
min-width: 5em;
}
.row > input {
flex-grow: 1;
}
.filenames {
display: flex;
flex-direction: column;
}
.filenames > .path {
font-size: 70%;
}
.rev {
text-overflow: ellipsis;
max-width: 3em;
display: inline-block;
overflow: hidden;
white-space: nowrap;
}
</style>

47
src/GlobalHistoryView.ts Normal file
View File

@@ -0,0 +1,47 @@
import {
ItemView,
WorkspaceLeaf
} from "./deps";
import GlobalHistoryComponent from "./GlobalHistory.svelte";
import type ObsidianLiveSyncPlugin from "./main";
export const VIEW_TYPE_GLOBAL_HISTORY = "global-history";
export class GlobalHistoryView extends ItemView {
component: GlobalHistoryComponent;
plugin: ObsidianLiveSyncPlugin;
icon: "clock";
title: string;
navigation: true;
getIcon(): string {
return "clock";
}
constructor(leaf: WorkspaceLeaf, plugin: ObsidianLiveSyncPlugin) {
super(leaf);
this.plugin = plugin;
}
getViewType() {
return VIEW_TYPE_GLOBAL_HISTORY;
}
getDisplayText() {
return "Vault history";
}
async onOpen() {
this.component = new GlobalHistoryComponent({
target: this.contentEl,
props: {
plugin: this.plugin,
},
});
}
async onClose() {
this.component.$destroy();
}
}

11
src/JsonResolveModal.ts
View File

@@ -8,12 +8,18 @@ export class JsonResolveModal extends Modal {
callback: (keepRev: string, mergedStr?: string) => Promise<void>; callback: (keepRev: string, mergedStr?: string) => Promise<void>;
docs: LoadedEntry[]; docs: LoadedEntry[];
component: JsonResolvePane; component: JsonResolvePane;
nameA: string;
nameB: string;
defaultSelect: string;
constructor(app: App, filename: FilePath, docs: LoadedEntry[], callback: (keepRev: string, mergedStr?: string) => Promise<void>) { constructor(app: App, filename: FilePath, docs: LoadedEntry[], callback: (keepRev: string, mergedStr?: string) => Promise<void>, nameA?: string, nameB?: string, defaultSelect?: string) {
super(app); super(app);
this.callback = callback; this.callback = callback;
this.filename = filename; this.filename = filename;
this.docs = docs; this.docs = docs;
this.nameA = nameA;
this.nameB = nameB;
this.defaultSelect = defaultSelect;
} }
async UICallback(keepRev: string, mergedStr?: string) { async UICallback(keepRev: string, mergedStr?: string) {
this.close(); this.close();
@@ -32,6 +38,9 @@ export class JsonResolveModal extends Modal {
props: { props: {
docs: this.docs, docs: this.docs,
filename: this.filename, filename: this.filename,
nameA: this.nameA,
nameB: this.nameB,
defaultSelect: this.defaultSelect,
callback: (keepRev, mergedStr) => this.UICallback(keepRev, mergedStr), callback: (keepRev, mergedStr) => this.UICallback(keepRev, mergedStr),
}, },
}); });

42
src/JsonResolvePane.svelte
View File

@@ -1,5 +1,5 @@
<script lang="ts"> <script lang="ts">
import { Diff, DIFF_DELETE, DIFF_INSERT, diff_match_patch } from "diff-match-patch"; import { type Diff, DIFF_DELETE, DIFF_INSERT, diff_match_patch } from "diff-match-patch";
import type { FilePath, LoadedEntry } from "./lib/src/types"; import type { FilePath, LoadedEntry } from "./lib/src/types";
import { base64ToString } from "./lib/src/strbin"; import { base64ToString } from "./lib/src/strbin";
import { getDocData } from "./lib/src/utils"; import { getDocData } from "./lib/src/utils";
@@ -10,7 +10,9 @@
Promise.resolve(); Promise.resolve();
}; };
export let filename: FilePath = "" as FilePath; export let filename: FilePath = "" as FilePath;
export let nameA: string = "A";
export let nameB: string = "B";
export let defaultSelect: string = "";
let docA: LoadedEntry = undefined; let docA: LoadedEntry = undefined;
let docB: LoadedEntry = undefined; let docB: LoadedEntry = undefined;
let docAContent = ""; let docAContent = "";
@@ -20,14 +22,8 @@
let objAB: any = {}; let objAB: any = {};
let objBA: any = {}; let objBA: any = {};
let diffs: Diff[]; let diffs: Diff[];
const modes = [ type SelectModes = "" | "A" | "B" | "AB" | "BA";
["", "Not now"], let mode: SelectModes = defaultSelect as SelectModes;
["A", "A"],
["B", "B"],
["AB", "A + B"],
["BA", "B + A"],
] as ["" | "A" | "B" | "AB" | "BA", string][];
let mode: "" | "A" | "B" | "AB" | "BA" = "";
function docToString(doc: LoadedEntry) { function docToString(doc: LoadedEntry) {
return doc.datatype == "plain" ? getDocData(doc.data) : base64ToString(doc.data); return doc.datatype == "plain" ? getDocData(doc.data) : base64ToString(doc.data);
@@ -47,8 +43,13 @@
return getDiff(JSON.stringify(a, null, 2), JSON.stringify(b, null, 2)); return getDiff(JSON.stringify(a, null, 2), JSON.stringify(b, null, 2));
} }
function apply() { function apply() {
if (mode == "A") return callback(docA._rev, null); if (docA._id == docB._id) {
if (mode == "B") return callback(docB._rev, null); if (mode == "A") return callback(docA._rev, null);
if (mode == "B") return callback(docB._rev, null);
} else {
if (mode == "A") return callback(null, docToString(docA));
if (mode == "B") return callback(null, docToString(docB));
}
if (mode == "BA") return callback(null, JSON.stringify(objBA, null, 2)); if (mode == "BA") return callback(null, JSON.stringify(objBA, null, 2));
if (mode == "AB") return callback(null, JSON.stringify(objAB, null, 2)); if (mode == "AB") return callback(null, JSON.stringify(objAB, null, 2));
callback(null, null); callback(null, null);
@@ -92,12 +93,19 @@
$: selectedObj = mode in mergedObjs ? mergedObjs[mode] : {}; $: selectedObj = mode in mergedObjs ? mergedObjs[mode] : {};
$: { $: {
diffs = getJsonDiff(objA, selectedObj); diffs = getJsonDiff(objA, selectedObj);
console.dir(selectedObj);
} }
$: modes = [
["", "Not now"],
["A", nameA || "A"],
["B", nameB || "B"],
["AB", `${nameA || "A"} + ${nameB || "B"}`],
["BA", `${nameB || "B"} + ${nameA || "A"}`],
] as ["" | "A" | "B" | "AB" | "BA", string][];
</script> </script>
<h1>Conflicted settings</h1> <h1>Conflicted settings</h1>
<div><span>{filename}</span></div> <h2>{filename}</h2>
{#if !docA || !docB} {#if !docA || !docB}
<div class="message">Just for a minute, please!</div> <div class="message">Just for a minute, please!</div>
<div class="buttons"> <div class="buttons">
@@ -125,12 +133,14 @@
NO PREVIEW NO PREVIEW
{/if} {/if}
<div> <div>
A Rev:{revStringToRevNumber(docA._rev)} ,{new Date(docA.mtime).toLocaleString()} {nameA}
{#if docA._id == docB._id} Rev:{revStringToRevNumber(docA._rev)} {/if} ,{new Date(docA.mtime).toLocaleString()}
{docAContent.length} letters {docAContent.length} letters
</div> </div>
<div> <div>
B Rev:{revStringToRevNumber(docB._rev)} ,{new Date(docB.mtime).toLocaleString()} {nameB}
{#if docA._id == docB._id} Rev:{revStringToRevNumber(docB._rev)} {/if} ,{new Date(docB.mtime).toLocaleString()}
{docBContent.length} letters {docBContent.length} letters
</div> </div>

81
src/LogPane.svelte Normal file
View File

@@ -0,0 +1,81 @@
<script lang="ts">
import { onDestroy, onMount } from "svelte";
import { logMessageStore } from "./lib/src/stores";
let unsubscribe: () => void;
let messages = [] as string[];
let wrapRight = false;
let autoScroll = true;
let suspended = false;
onMount(async () => {
unsubscribe = logMessageStore.observe((e) => {
if (!suspended) {
messages = [...e];
if (autoScroll) {
if (scroll) scroll.scrollTop = scroll.scrollHeight;
}
}
});
logMessageStore.invalidate();
setTimeout(() => {
if (scroll) scroll.scrollTop = scroll.scrollHeight;
}, 100);
});
onDestroy(() => {
if (unsubscribe) unsubscribe();
});
let scroll: HTMLDivElement;
</script>
<div class="logpane">
<!-- <h1>Self-hosted LiveSync Log</h1> -->
<div class="control">
<div class="row">
<label><input type="checkbox" bind:checked={wrapRight} /><span>Wrap</span></label>
<label><input type="checkbox" bind:checked={autoScroll} /><span>Auto scroll</span></label>
<label><input type="checkbox" bind:checked={suspended} /><span>Pause</span></label>
</div>
</div>
<div class="log" bind:this={scroll}>
{#each messages as line}
<pre class:wrap-right={wrapRight}>{line}</pre>
{/each}
</div>
</div>
<style>
* {
box-sizing: border-box;
}
.logpane {
display: flex;
height: 100%;
flex-direction: column;
}
.log {
overflow-y: scroll;
user-select: text;
padding-bottom: 2em;
}
.log > pre {
margin: 0;
}
.log > pre.wrap-right {
word-break: break-all;
max-width: 100%;
width: 100%;
white-space: normal;
}
.row {
display: flex;
flex-direction: row;
justify-content: flex-end;
}
.row > label {
display: flex;
align-items: center;
min-width: 5em;
margin-right: 1em;
}
</style>

46
src/LogPaneView.ts Normal file
View File

@@ -0,0 +1,46 @@
import {
ItemView,
WorkspaceLeaf
} from "obsidian";
import LogPaneComponent from "./LogPane.svelte";
import type ObsidianLiveSyncPlugin from "./main";
export const VIEW_TYPE_LOG = "log-log";
// Show notes as like scroll.
export class LogPaneView extends ItemView {
component: LogPaneComponent;
plugin: ObsidianLiveSyncPlugin;
icon: "view-log";
title: string;
navigation: true;
getIcon(): string {
return "view-log";
}
constructor(leaf: WorkspaceLeaf, plugin: ObsidianLiveSyncPlugin) {
super(leaf);
this.plugin = plugin;
}
getViewType() {
return VIEW_TYPE_LOG;
}
getDisplayText() {
return "Self-hosted LiveSync Log";
}
async onOpen() {
this.component = new LogPaneComponent({
target: this.contentEl,
props: {
},
});
}
async onClose() {
this.component.$destroy();
}
}

2209
src/ObsidianLiveSyncSettingTab.ts
View File

File diff suppressed because it is too large Load Diff

318
src/PluginCombo.svelte Normal file
View File

@@ -0,0 +1,318 @@
<script lang="ts">
import type { PluginDataExDisplay } from "./CmdConfigSync";
import { Logger } from "./lib/src/logger";
import { versionNumberString2Number } from "./lib/src/strbin";
import { type FilePath, LOG_LEVEL } from "./lib/src/types";
import { getDocData } from "./lib/src/utils";
import type ObsidianLiveSyncPlugin from "./main";
import { askString, scheduleTask } from "./utils";
export let list: PluginDataExDisplay[] = [];
export let thisTerm = "";
export let hideNotApplicable = false;
export let selectNewest = 0;
export let applyAllPluse = 0;
export let applyData: (data: PluginDataExDisplay) => Promise<boolean>;
export let compareData: (dataA: PluginDataExDisplay, dataB: PluginDataExDisplay) => Promise<boolean>;
export let deleteData: (data: PluginDataExDisplay) => Promise<boolean>;
export let hidden: boolean;
export let plugin: ObsidianLiveSyncPlugin;
export let isMaintenanceMode: boolean = false;
const addOn = plugin.addOnConfigSync;
let selected = "";
let freshness = "";
let equivalency = "";
let version = "";
let canApply: boolean = false;
let canCompare: boolean = false;
let currentSelectNewest = 0;
let currentApplyAll = 0;
// Selectable terminals
let terms = [] as string[];
async function comparePlugin(local: PluginDataExDisplay, remote: PluginDataExDisplay) {
let freshness = "";
let equivalency = "";
let version = "";
let contentCheck = false;
let canApply: boolean = false;
let canCompare = false;
if (!local && !remote) {
// NO OP. whats happened?
freshness = "";
} else if (local && !remote) {
freshness = "⚠ Local only";
} else if (remote && !local) {
freshness = "✓ Remote only";
canApply = true;
} else {
const dtDiff = (local?.mtime ?? 0) - (remote?.mtime ?? 0);
if (dtDiff / 1000 < -10) {
freshness = "✓ Newer";
canApply = true;
contentCheck = true;
} else if (dtDiff / 1000 > 10) {
freshness = "⚠ Older";
canApply = true;
contentCheck = true;
} else {
freshness = "⚖️ Same old";
canApply = false;
contentCheck = true;
}
}
const localVersionStr = local?.version || "0.0.0";
const remoteVersionStr = remote?.version || "0.0.0";
if (local?.version || remote?.version) {
const localVersion = versionNumberString2Number(localVersionStr);
const remoteVersion = versionNumberString2Number(remoteVersionStr);
if (localVersion == remoteVersion) {
version = "⚖️ Same ver.";
} else if (localVersion > remoteVersion) {
version = `⚠ Lower ${localVersionStr} > ${remoteVersionStr}`;
} else if (localVersion < remoteVersion) {
version = `✓ Higher ${localVersionStr} < ${remoteVersionStr}`;
}
}
if (contentCheck) {
const { canApply, equivalency, canCompare } = await checkEquivalency(local, remote);
return { canApply, freshness, equivalency, version, canCompare };
}
return { canApply, freshness, equivalency, version, canCompare };
}
async function checkEquivalency(local: PluginDataExDisplay, remote: PluginDataExDisplay) {
let equivalency = "";
let canApply = false;
let canCompare = false;
const filenames = [...new Set([...local.files.map((e) => e.filename), ...remote.files.map((e) => e.filename)])];
const matchingStatus = filenames
.map((filename) => {
const localFile = local.files.find((e) => e.filename == filename);
const remoteFile = remote.files.find((e) => e.filename == filename);
if (!localFile && !remoteFile) {
return 0b0000000;
} else if (localFile && !remoteFile) {
return 0b0000010; //"LOCAL_ONLY";
} else if (!localFile && remoteFile) {
return 0b0001000; //"REMOTE ONLY"
} else {
if (getDocData(localFile.data) == getDocData(remoteFile.data)) {
return 0b0000100; //"EVEN"
} else {
return 0b0010000; //"DIFFERENT";
}
}
})
.reduce((p, c) => p | c, 0);
if (matchingStatus == 0b0000100) {
equivalency = "⚖️ Same";
canApply = false;
} else if (matchingStatus <= 0b0000100) {
equivalency = "Same or local only";
canApply = false;
} else if (matchingStatus == 0b0010000) {
canApply = true;
canCompare = true;
equivalency = "≠ Different";
} else {
canApply = true;
canCompare = true;
equivalency = "≠ Different";
}
return { equivalency, canApply, canCompare };
}
async function performCompare(local: PluginDataExDisplay, remote: PluginDataExDisplay) {
const result = await comparePlugin(local, remote);
canApply = result.canApply;
freshness = result.freshness;
equivalency = result.equivalency;
version = result.version;
canCompare = result.canCompare;
if (local?.files.length != 1 || !local?.files?.first()?.filename?.endsWith(".json")) {
canCompare = false;
}
}
async function updateTerms(list: PluginDataExDisplay[], selectNewest: boolean, isMaintenanceMode: boolean) {
const local = list.find((e) => e.term == thisTerm);
selected = "";
if (isMaintenanceMode) {
terms = [...new Set(list.map((e) => e.term))];
} else if (hideNotApplicable) {
const termsTmp = [];
const wk = [...new Set(list.map((e) => e.term))];
for (const termName of wk) {
const remote = list.find((e) => e.term == termName);
if ((await comparePlugin(local, remote)).canApply) {
termsTmp.push(termName);
}
}
terms = [...termsTmp];
} else {
terms = [...new Set(list.map((e) => e.term))].filter((e) => e != thisTerm);
}
let newest: PluginDataExDisplay = local;
if (selectNewest) {
for (const term of terms) {
const remote = list.find((e) => e.term == term);
if (remote && remote.mtime && (newest?.mtime || 0) < remote.mtime) {
newest = remote;
}
}
if (newest && newest.term != thisTerm) {
selected = newest.term;
}
// selectNewest = false;
}
}
$: {
// React pulse and select
const doSelectNewest = selectNewest != currentSelectNewest;
currentSelectNewest = selectNewest;
updateTerms(list, doSelectNewest, isMaintenanceMode);
}
$: {
// React pulse and apply
const doApply = applyAllPluse != currentApplyAll;
currentApplyAll = applyAllPluse;
if (doApply && selected) {
if (!hidden) {
applySelected();
}
}
}
$: {
freshness = "";
equivalency = "";
version = "";
canApply = false;
if (selected == "") {
// NO OP.
} else if (selected == thisTerm) {
freshness = "This device";
canApply = false;
} else {
const local = list.find((e) => e.term == thisTerm);
const remote = list.find((e) => e.term == selected);
performCompare(local, remote);
}
}
async function applySelected() {
const local = list.find((e) => e.term == thisTerm);
const selectedItem = list.find((e) => e.term == selected);
if (selectedItem && (await applyData(selectedItem))) {
scheduleTask("update-plugin-list", 250, () => addOn.updatePluginList(true, local.documentPath));
}
}
async function compareSelected() {
const local = list.find((e) => e.term == thisTerm);
const selectedItem = list.find((e) => e.term == selected);
if (local && selectedItem && (await compareData(local, selectedItem))) {
scheduleTask("update-plugin-list", 250, () => addOn.updatePluginList(true, local.documentPath));
}
}
async function deleteSelected() {
const selectedItem = list.find((e) => e.term == selected);
// const deletedPath = selectedItem.documentPath;
if (selectedItem && (await deleteData(selectedItem))) {
scheduleTask("update-plugin-list", 250, () => addOn.reloadPluginList(true));
}
}
async function duplicateItem() {
const local = list.find((e) => e.term == thisTerm);
const duplicateTermName = await askString(plugin.app, "Duplicate", "device name", "");
if (duplicateTermName) {
if (duplicateTermName.contains("/")) {
Logger(`We can not use "/" to the device name`, LOG_LEVEL.NOTICE);
return;
}
const key = `${plugin.app.vault.configDir}/${local.files[0].filename}`;
await addOn.storeCustomizationFiles(key as FilePath, duplicateTermName);
await addOn.updatePluginList(false, addOn.filenameToUnifiedKey(key, duplicateTermName));
}
}
</script>
{#if terms.length > 0}
<span class="spacer" />
{#if !hidden}
<span class="messages">
<span class="message">{freshness}</span>
<span class="message">{equivalency}</span>
<span class="message">{version}</span>
</span>
<select bind:value={selected}>
<option value={""}>-</option>
{#each terms as term}
<option value={term}>{term}</option>
{/each}
</select>
{#if canApply || (isMaintenanceMode && selected != "")}
{#if canCompare}
<button on:click={compareSelected}>🔍</button>
{:else}
<button disabled />
{/if}
<button on:click={applySelected}>✓</button>
{:else}
<button disabled />
<button disabled />
{/if}
{#if isMaintenanceMode}
{#if selected != ""}
<button on:click={deleteSelected}>🗑️</button>
{:else}
<button on:click={duplicateItem}>📑</button>
{/if}
{/if}
{/if}
{:else}
<span class="spacer" />
<span class="message even">All devices are even</span>
<button disabled />
<button disabled />
{/if}
<style>
.spacer {
min-width: 1px;
flex-grow: 1;
}
button {
margin: 2px 4px;
min-width: 3em;
max-width: 4em;
}
button:disabled {
border: none;
box-shadow: none;
background-color: transparent;
visibility: collapse;
}
button:disabled:hover {
border: none;
box-shadow: none;
background-color: transparent;
visibility: collapse;
}
span.message {
color: var(--text-muted);
font-size: var(--font-ui-smaller);
padding: 0 1em;
line-height: var(--line-height-tight);
}
span.messages {
display: flex;
flex-direction: column;
align-items: center;
}
:global(.is-mobile) .spacer {
margin-left: auto;
}
</style>

461
src/PluginPane.svelte
View File

@@ -1,309 +1,230 @@
<script lang="ts"> <script lang="ts">
import { onMount } from "svelte"; import { onMount } from "svelte";
import { DevicePluginList, PluginDataEntry } from "./types";
import { versionNumberString2Number } from "./lib/src/strbin";
import ObsidianLiveSyncPlugin from "./main"; import ObsidianLiveSyncPlugin from "./main";
import { PluginAndTheirSettings } from "./CmdPluginAndTheirSettings"; import { type PluginDataExDisplay, pluginIsEnumerating, pluginList } from "./CmdConfigSync";
import PluginCombo from "./PluginCombo.svelte";
type JudgeResult = "" | "NEWER" | "EVEN" | "EVEN_BUT_DIFFERENT" | "OLDER" | "REMOTE_ONLY";
interface PluginDataEntryDisp extends PluginDataEntry {
versionInfo: string;
mtimeInfo: string;
mtimeFlag: JudgeResult;
versionFlag: JudgeResult;
}
export let plugin: ObsidianLiveSyncPlugin; export let plugin: ObsidianLiveSyncPlugin;
let plugins: PluginDataEntry[] = [];
let deviceAndPlugins: { [key: string]: PluginDataEntryDisp[] } = {};
let devicePluginList: [string, PluginDataEntryDisp[]][] = null;
let ownPlugins: DevicePluginList = null;
let showOwnPlugins = false;
let targetList: { [key: string]: boolean } = {};
let addOn: PluginAndTheirSettings; $: hideNotApplicable = true;
$: { $: thisTerm = plugin.deviceAndVaultName;
const f = plugin.addOns.filter((e) => e instanceof PluginAndTheirSettings);
if (f && f.length > 0) { const addOn = plugin.addOnConfigSync;
addOn = f[0] as PluginAndTheirSettings;
} let list: PluginDataExDisplay[] = [];
let selectNewestPulse = 0;
let hideEven = true;
let loading = false;
let applyAllPluse = 0;
let isMaintenanceMode = false;
async function requestUpdate() {
await addOn.updatePluginList(true);
} }
function saveTargetList() { async function requestReload() {
window.localStorage.setItem("ols-plugin-targetlist", JSON.stringify(targetList)); await addOn.reloadPluginList(true);
} }
pluginList.subscribe((e) => {
function loadTargetList() { list = e;
let e = window.localStorage.getItem("ols-plugin-targetlist") || "{}"; });
try { pluginIsEnumerating.subscribe((e) => {
targetList = JSON.parse(e); loading = e;
} catch (_) { });
// NO OP.
}
}
function clearSelection() {
targetList = {};
}
async function updateList() {
let x = await addOn.getPluginList();
ownPlugins = x.thisDevicePlugins;
plugins = Object.values(x.allPlugins);
let targetListItems = Array.from(new Set(plugins.map((e) => e.deviceVaultName + "---" + e.manifest.id)));
let newTargetList: { [key: string]: boolean } = {};
for (const id of targetListItems) {
for (const tag of ["---plugin", "---setting"]) {
newTargetList[id + tag] = id + tag in targetList && targetList[id + tag];
}
}
targetList = newTargetList;
saveTargetList();
}
$: {
deviceAndPlugins = {};
for (const p of plugins) {
if (p.deviceVaultName == plugin.deviceAndVaultName && !showOwnPlugins) {
continue;
}
if (!(p.deviceVaultName in deviceAndPlugins)) {
deviceAndPlugins[p.deviceVaultName] = [];
}
let dispInfo: PluginDataEntryDisp = {
...p,
versionInfo: "",
mtimeInfo: "",
versionFlag: "",
mtimeFlag: "",
};
dispInfo.versionInfo = p.manifest.version;
let x = new Date().getTime() / 1000;
let mtime = p.mtime / 1000;
let diff = (x - mtime) / 60;
if (p.mtime == 0) {
dispInfo.mtimeInfo = `-`;
} else if (diff < 60) {
dispInfo.mtimeInfo = `${diff | 0} Mins ago`;
} else if (diff < 60 * 24) {
dispInfo.mtimeInfo = `${(diff / 60) | 0} Hours ago`;
} else if (diff < 60 * 24 * 10) {
dispInfo.mtimeInfo = `${(diff / (60 * 24)) | 0} Days ago`;
} else {
dispInfo.mtimeInfo = new Date(dispInfo.mtime).toLocaleString();
}
// compare with own plugin
let id = p.manifest.id;
if (id in ownPlugins) {
// Which we have.
const ownPlugin = ownPlugins[id];
let localVer = versionNumberString2Number(ownPlugin.manifest.version);
let pluginVer = versionNumberString2Number(p.manifest.version);
if (localVer > pluginVer) {
dispInfo.versionFlag = "OLDER";
} else if (localVer == pluginVer) {
if (ownPlugin.manifestJson + (ownPlugin.styleCss ?? "") + ownPlugin.mainJs != p.manifestJson + (p.styleCss ?? "") + p.mainJs) {
dispInfo.versionFlag = "EVEN_BUT_DIFFERENT";
} else {
dispInfo.versionFlag = "EVEN";
}
} else if (localVer < pluginVer) {
dispInfo.versionFlag = "NEWER";
}
if ((ownPlugin.dataJson ?? "") == (p.dataJson ?? "")) {
if (ownPlugin.mtime == 0 && p.mtime == 0) {
dispInfo.mtimeFlag = "";
} else {
dispInfo.mtimeFlag = "EVEN";
}
} else {
if (((ownPlugin.mtime / 1000) | 0) > ((p.mtime / 1000) | 0)) {
dispInfo.mtimeFlag = "OLDER";
} else if (((ownPlugin.mtime / 1000) | 0) == ((p.mtime / 1000) | 0)) {
dispInfo.mtimeFlag = "EVEN_BUT_DIFFERENT";
} else if (((ownPlugin.mtime / 1000) | 0) < ((p.mtime / 1000) | 0)) {
dispInfo.mtimeFlag = "NEWER";
}
}
} else {
dispInfo.versionFlag = "REMOTE_ONLY";
dispInfo.mtimeFlag = "REMOTE_ONLY";
}
deviceAndPlugins[p.deviceVaultName].push(dispInfo);
}
devicePluginList = Object.entries(deviceAndPlugins);
}
function getDispString(stat: JudgeResult): string {
if (stat == "") return "";
if (stat == "NEWER") return " (Newer)";
if (stat == "OLDER") return " (Older)";
if (stat == "EVEN") return " (Even)";
if (stat == "EVEN_BUT_DIFFERENT") return " (Even but different)";
if (stat == "REMOTE_ONLY") return " (Remote Only)";
return "";
}
onMount(async () => { onMount(async () => {
loadTargetList(); requestUpdate();
await updateList();
}); });
function toggleShowOwnPlugins() { function filterList(list: PluginDataExDisplay[], categories: string[]) {
showOwnPlugins = !showOwnPlugins; const w = list.filter((e) => categories.indexOf(e.category) !== -1);
return w.sort((a, b) => `${a.category}-${a.name}`.localeCompare(`${b.category}-${b.name}`));
} }
function toggleTarget(key: string) { function groupBy(items: PluginDataExDisplay[], key: string) {
targetList[key] = !targetList[key]; let ret = {} as Record<string, PluginDataExDisplay[]>;
saveTargetList(); for (const v of items) {
} //@ts-ignore
const k = (key in v ? v[key] : "") as string;
function toggleAll(devicename: string) { ret[k] = ret[k] || [];
for (const c in targetList) { ret[k].push(v);
if (c.startsWith(devicename)) {
targetList[c] = true;
}
} }
} for (const k in ret) {
ret[k] = ret[k].sort((a, b) => `${a.category}-${a.name}`.localeCompare(`${b.category}-${b.name}`));
async function sweepPlugins() {
//@ts-ignore
await plugin.app.plugins.loadManifests();
await addOn.sweepPlugin(true);
updateList();
}
async function applyPlugins() {
for (const c in targetList) {
if (targetList[c] == true) {
const [deviceAndVault, id, opt] = c.split("---");
if (deviceAndVault in deviceAndPlugins) {
const entry = deviceAndPlugins[deviceAndVault].find((e) => e.manifest.id == id);
if (entry) {
if (opt == "plugin") {
if (entry.versionFlag != "EVEN") await addOn.applyPlugin(entry);
} else if (opt == "setting") {
if (entry.mtimeFlag != "EVEN") await addOn.applyPluginData(entry);
}
}
}
}
} }
//@ts-ignore const w = Object.entries(ret);
await plugin.app.plugins.loadManifests(); return w.sort(([a], [b]) => `${a}`.localeCompare(`${b}`));
await addOn.sweepPlugin(true);
updateList();
} }
async function checkUpdates() { const displays = {
await addOn.checkPluginUpdate(); CONFIG: "Configuration",
THEME: "Themes",
SNIPPET: "Snippets",
};
async function scanAgain() {
await addOn.scanAllConfigFiles(true);
await requestUpdate();
} }
async function replicateAndRefresh() { async function replicate() {
await plugin.replicate(true); await plugin.replicate(true);
updateList();
} }
function selectAllNewest() {
selectNewestPulse++;
}
function applyAll() {
applyAllPluse++;
}
async function applyData(data: PluginDataExDisplay): Promise<boolean> {
return await addOn.applyData(data);
}
async function compareData(docA: PluginDataExDisplay, docB: PluginDataExDisplay): Promise<boolean> {
return await addOn.compareUsingDisplayData(docA, docB);
}
async function deleteData(data: PluginDataExDisplay): Promise<boolean> {
return await addOn.deleteData(data);
}
$: options = {
thisTerm,
hideNotApplicable,
selectNewest: selectNewestPulse,
applyAllPluse,
applyData,
compareData,
deleteData,
plugin,
isMaintenanceMode,
};
</script> </script>
<div> <div>
<h1>Plugins and their settings</h1> <div>
<div class="ols-plugins-div-buttons"> <h1>Customization sync</h1>
Show own items <div class="buttons">
<div class="checkbox-container" class:is-enabled={showOwnPlugins} on:click={toggleShowOwnPlugins} /> <button on:click={() => scanAgain()}>Scan changes</button>
</div> <button on:click={() => replicate()}>Sync once</button>
<div class="sls-plugins-wrap"> <button on:click={() => requestUpdate()}>Refresh</button>
<table class="sls-plugins-tbl"> {#if isMaintenanceMode}
<tr style="position:sticky"> <button on:click={() => requestReload()}>Reload</button>
<th class="sls-plugins-tbl-device-head">Name</th>
<th class="sls-plugins-tbl-device-head">Info</th>
<th class="sls-plugins-tbl-device-head">Target</th>
</tr>
{#if !devicePluginList}
<tr>
<td colspan="3" class="sls-table-tail tcenter"> Retrieving... </td>
</tr>
{:else if devicePluginList.length == 0}
<tr>
<td colspan="3" class="sls-table-tail tcenter"> No plugins found. </td>
</tr>
{:else}
{#each devicePluginList as [deviceName, devicePlugins]}
<tr>
<th colspan="2" class="sls-plugins-tbl-device-head">{deviceName}</th>
<th class="sls-plugins-tbl-device-head">
<button class="mod-cta" on:click={() => toggleAll(deviceName)}>✔</button>
</th>
</tr>
{#each devicePlugins as plugin}
<tr>
<td class="sls-table-head">{plugin.manifest.name}</td>
<td class="sls-table-tail tcenter">{plugin.versionInfo}{getDispString(plugin.versionFlag)}</td>
<td class="sls-table-tail tcenter">
{#if plugin.versionFlag === "EVEN" || plugin.versionFlag === ""}
-
{:else}
<div class="wrapToggle">
<div
class="checkbox-container"
class:is-enabled={targetList[plugin.deviceVaultName + "---" + plugin.manifest.id + "---plugin"]}
on:click={() => toggleTarget(plugin.deviceVaultName + "---" + plugin.manifest.id + "---plugin")}
/>
</div>
{/if}
</td>
</tr>
<tr>
<td class="sls-table-head">Settings</td>
<td class="sls-table-tail tcenter">{plugin.mtimeInfo}{getDispString(plugin.mtimeFlag)}</td>
<td class="sls-table-tail tcenter">
{#if plugin.mtimeFlag === "EVEN" || plugin.mtimeFlag === ""}
-
{:else}
<div class="wrapToggle">
<div
class="checkbox-container"
class:is-enabled={targetList[plugin.deviceVaultName + "---" + plugin.manifest.id + "---setting"]}
on:click={() => toggleTarget(plugin.deviceVaultName + "---" + plugin.manifest.id + "---setting")}
/>
</div>
{/if}
</td>
</tr>
<tr class="divider">
<th colspan="3" />
</tr>
{/each}
{/each}
{/if} {/if}
</table> <button on:click={() => selectAllNewest()}>Select All Shiny</button>
</div>
<div class="buttons">
<button on:click={() => applyAll()}>Apply All</button>
</div>
</div> </div>
<div class="ols-plugins-div-buttons"> {#if loading}
<button class="" on:click={replicateAndRefresh}>Replicate and refresh</button> <div>
<button class="" on:click={clearSelection}>Clear Selection</button> <span>Updating list...</span>
</div>
{/if}
<div class="list">
{#if list.length == 0}
<div class="center">No Items.</div>
{:else}
{#each Object.entries(displays) as [key, label]}
<div>
<h3>{label}</h3>
{#each groupBy(filterList(list, [key]), "name") as [name, listX]}
<div class="labelrow {hideEven ? 'hideeven' : ''}">
<div class="title">
{name}
</div>
<PluginCombo {...options} list={listX} hidden={false} />
</div>
{/each}
</div>
{/each}
<div>
<h3>Plugins</h3>
{#each groupBy(filterList(list, ["PLUGIN_MAIN", "PLUGIN_DATA", "PLUGIN_ETC"]), "name") as [name, listX]}
<div class="labelrow {hideEven ? 'hideeven' : ''}">
<div class="title">
{name}
</div>
<PluginCombo {...options} list={listX} hidden={true} />
</div>
<div class="filerow {hideEven ? 'hideeven' : ''}">
<div class="filetitle">Main</div>
<PluginCombo {...options} list={filterList(listX, ["PLUGIN_MAIN"])} hidden={false} />
</div>
<div class="filerow {hideEven ? 'hideeven' : ''}">
<div class="filetitle">Data</div>
<PluginCombo {...options} list={filterList(listX, ["PLUGIN_DATA"])} hidden={false} />
</div>
{/each}
</div>
{/if}
</div> </div>
<div class="buttons">
<div class="ols-plugins-div-buttons"> <label><span>Hide not applicable items</span><input type="checkbox" bind:checked={hideEven} /></label>
<button class="mod-cta" on:click={checkUpdates}>Check Updates</button> </div>
<button class="mod-cta" on:click={sweepPlugins}>Scan installed</button> <div class="buttons">
<button class="mod-cta" on:click={applyPlugins}>Apply all</button> <label><span>Maintenance mode</span><input type="checkbox" bind:checked={isMaintenanceMode} /></label>
</div> </div>
<!-- <div class="ols-plugins-div-buttons">-->
<!-- <button class="mod-warning" on:click={applyPlugins}>Delete all selected</button>-->
<!-- </div>-->
</div> </div>
<style> <style>
.ols-plugins-div-buttons { .labelrow {
margin-left: 0.4em;
display: flex;
justify-content: flex-start;
align-items: center;
border-top: 1px solid var(--background-modifier-border);
padding: 4px;
flex-wrap: wrap;
}
.filerow {
margin-left: 1.25em;
display: flex;
justify-content: flex-start;
align-items: center;
padding-right: 4px;
flex-wrap: wrap;
}
.filerow.hideeven:has(.even),
.labelrow.hideeven:has(.even) {
display: none;
}
.title {
color: var(--text-normal);
font-size: var(--font-ui-medium);
line-height: var(--line-height-tight);
margin-right: auto;
}
.filetitle {
color: var(--text-normal);
font-size: var(--font-ui-medium);
line-height: var(--line-height-tight);
margin-right: auto;
}
.buttons {
display: flex; display: flex;
flex-direction: row; flex-direction: row;
justify-content: flex-end; justify-content: flex-end;
margin-top: 8px; margin-top: 8px;
flex-wrap: wrap;
}
.buttons > button {
margin-left: 4px;
width: auto;
} }
.wrapToggle { label {
display: flex; display: flex;
justify-content: center; justify-content: center;
align-content: center; align-items: center;
}
label > span {
margin-right: 0.25em;
}
:global(.is-mobile) .title,
:global(.is-mobile) .filetitle {
width: 100%;
}
.center {
display: flex;
justify-content: center;
align-items: center;
min-height: 3em;
} }
</style> </style>

6
src/StorageEventManager.ts
View File

@@ -57,14 +57,14 @@ export class StorageEventManagerObsidian extends StorageEventManager {
watchVaultRename(file: TAbstractFile, oldFile: string, ctx?: any) { watchVaultRename(file: TAbstractFile, oldFile: string, ctx?: any) {
if (file instanceof TFile) { if (file instanceof TFile) {
this.appendWatchEvent([ this.appendWatchEvent([
{ type: "CREATE", file },
{ type: "DELETE", file: { path: oldFile as FilePath, mtime: file.stat.mtime, ctime: file.stat.ctime, size: file.stat.size, deleted: true } }, { type: "DELETE", file: { path: oldFile as FilePath, mtime: file.stat.mtime, ctime: file.stat.ctime, size: file.stat.size, deleted: true } },
{ type: "CREATE", file },
], ctx); ], ctx);
} }
} }
// Watch raw events (Internal API) // Watch raw events (Internal API)
watchVaultRawEvents(path: FilePath) { watchVaultRawEvents(path: FilePath) {
if (!this.plugin.settings.syncInternalFiles) return; if (!this.plugin.settings.syncInternalFiles && !this.plugin.settings.usePluginSync) return;
if (!this.plugin.settings.watchInternalFileChanges) return; if (!this.plugin.settings.watchInternalFileChanges) return;
if (!path.startsWith(app.vault.configDir)) return; if (!path.startsWith(app.vault.configDir)) return;
const ignorePatterns = this.plugin.settings.syncInternalFilesIgnorePatterns const ignorePatterns = this.plugin.settings.syncInternalFilesIgnorePatterns
@@ -112,7 +112,7 @@ export class StorageEventManagerObsidian extends StorageEventManager {
} }
if (this.plugin.settings.batchSave) { if (this.plugin.settings.batchSave && !this.plugin.settings.liveSync) {
// if the latest event is the same type, omit that // if the latest event is the same type, omit that
// a.md MODIFY <- this should be cancelled when a.md MODIFIED // a.md MODIFY <- this should be cancelled when a.md MODIFIED
// b.md MODIFY <- this should be cancelled when b.md MODIFIED // b.md MODIFY <- this should be cancelled when b.md MODIFIED

7
src/deps.ts
View File

@@ -1,9 +1,10 @@
import { FilePath } from "./lib/src/types"; import { type FilePath } from "./lib/src/types";
export { export {
addIcon, App, DataWriteOptions, debounce, Editor, FuzzySuggestModal, MarkdownRenderer, MarkdownView, Modal, Notice, Platform, Plugin, PluginManifest, addIcon, App, debounce, Editor, FuzzySuggestModal, MarkdownRenderer, MarkdownView, Modal, Notice, Platform, Plugin, PluginSettingTab, Plugin_2, requestUrl, sanitizeHTMLToDom, Setting, stringifyYaml, TAbstractFile, TextAreaComponent, TFile, TFolder,
PluginSettingTab, Plugin_2, requestUrl, RequestUrlParam, RequestUrlResponse, sanitizeHTMLToDom, Setting, stringifyYaml, TAbstractFile, TextAreaComponent, TFile, TFolder, parseYaml, ItemView, WorkspaceLeaf
} from "obsidian"; } from "obsidian";
export type { DataWriteOptions, PluginManifest, RequestUrlParam, RequestUrlResponse } from "obsidian";
import { import {
normalizePath as normalizePath_ normalizePath as normalizePath_
} from "obsidian"; } from "obsidian";

113
src/dialogs.ts
View File

@@ -1,4 +1,5 @@
import { App, FuzzySuggestModal, Modal, Setting } from "./deps"; import { ButtonComponent } from "obsidian";
import { App, FuzzySuggestModal, MarkdownRenderer, Modal, Plugin, Setting } from "./deps";
import ObsidianLiveSyncPlugin from "./main"; import ObsidianLiveSyncPlugin from "./main";
//@ts-ignore //@ts-ignore
@@ -8,6 +9,9 @@ export class PluginDialogModal extends Modal {
plugin: ObsidianLiveSyncPlugin; plugin: ObsidianLiveSyncPlugin;
logEl: HTMLDivElement; logEl: HTMLDivElement;
component: PluginPane = null; component: PluginPane = null;
isOpened() {
return this.component != null;
}
constructor(app: App, plugin: ObsidianLiveSyncPlugin) { constructor(app: App, plugin: ObsidianLiveSyncPlugin) {
super(app); super(app);
@@ -39,13 +43,15 @@ export class InputStringDialog extends Modal {
key: string; key: string;
placeholder: string; placeholder: string;
isManuallyClosed = false; isManuallyClosed = false;
isPassword = false;
constructor(app: App, title: string, key: string, placeholder: string, onSubmit: (result: string | false) => void) { constructor(app: App, title: string, key: string, placeholder: string, isPassword: boolean, onSubmit: (result: string | false) => void) {
super(app); super(app);
this.onSubmit = onSubmit; this.onSubmit = onSubmit;
this.title = title; this.title = title;
this.placeholder = placeholder; this.placeholder = placeholder;
this.key = key; this.key = key;
this.isPassword = isPassword;
} }
onOpen() { onOpen() {
@@ -54,7 +60,7 @@ export class InputStringDialog extends Modal {
contentEl.createEl("h1", { text: this.title }); contentEl.createEl("h1", { text: this.title });
// For enter to submit // For enter to submit
const formEl = contentEl.createEl("form"); const formEl = contentEl.createEl("form");
new Setting(formEl).setName(this.key).addText((text) => new Setting(formEl).setName(this.key).setClass(this.isPassword ? "password-input" : "normal-input").addText((text) =>
text.onChange((value) => { text.onChange((value) => {
this.result = value; this.result = value;
}) })
@@ -123,4 +129,103 @@ export class PopoverSelectString extends FuzzySuggestModal<string> {
} }
}, 100); }, 100);
} }
} }
export class MessageBox extends Modal {
plugin: Plugin;
title: string;
contentMd: string;
buttons: string[];
result: string;
isManuallyClosed = false;
defaultAction: string | undefined;
timeout: number | undefined;
timer: ReturnType<typeof setInterval> = undefined;
defaultButtonComponent: ButtonComponent | undefined;
onSubmit: (result: string | boolean) => void;
constructor(plugin: Plugin, title: string, contentMd: string, buttons: string[], defaultAction: (typeof buttons)[number], timeout: number, onSubmit: (result: (typeof buttons)[number] | false) => void) {
super(plugin.app);
this.plugin = plugin;
this.title = title;
this.contentMd = contentMd;
this.buttons = buttons;
this.onSubmit = onSubmit;
this.defaultAction = defaultAction;
this.timeout = timeout;
if (this.timeout) {
this.timer = setInterval(() => {
this.timeout--;
if (this.timeout < 0) {
if (this.timer) {
clearInterval(this.timer);
this.timer = undefined;
}
this.result = defaultAction;
this.isManuallyClosed = true;
this.close();
} else {
this.defaultButtonComponent.setButtonText(`( ${this.timeout} ) ${defaultAction}`);
}
}, 1000);
}
}
onOpen() {
const { contentEl } = this;
contentEl.addEventListener("click", () => {
if (this.timer) {
clearInterval(this.timer);
this.timer = undefined;
}
})
contentEl.createEl("h1", { text: this.title });
const div = contentEl.createDiv();
MarkdownRenderer.renderMarkdown(this.contentMd, div, "/", null);
const buttonSetting = new Setting(contentEl);
for (const button of this.buttons) {
buttonSetting.addButton((btn) => {
btn
.setButtonText(button)
.onClick(() => {
this.isManuallyClosed = true;
this.result = button;
if (this.timer) {
clearInterval(this.timer);
this.timer = undefined;
}
this.close();
})
if (button == this.defaultAction) {
this.defaultButtonComponent = btn;
}
return btn;
}
)
}
}
onClose() {
const { contentEl } = this;
contentEl.empty();
if (this.timer) {
clearInterval(this.timer);
this.timer = undefined;
}
if (this.isManuallyClosed) {
this.onSubmit(this.result);
} else {
this.onSubmit(false);
}
}
}
export function confirmWithMessage(plugin: Plugin, title: string, contentMd: string, buttons: string[], defaultAction?: (typeof buttons)[number], timeout?: number): Promise<(typeof buttons)[number] | false> {
return new Promise((res) => {
const dialog = new MessageBox(plugin, title, contentMd, buttons, defaultAction, timeout, (result) => res(result));
dialog.open();
});
}

2
src/lib

Submodule src/lib updated: f5db618612...63fa0074fe

408
src/main.ts
View File

@@ -1,17 +1,16 @@
const isDebug = false; const isDebug = false;
import { Diff, DIFF_DELETE, DIFF_EQUAL, DIFF_INSERT, diff_match_patch } from "diff-match-patch"; import { type Diff, DIFF_DELETE, DIFF_EQUAL, DIFF_INSERT, diff_match_patch } from "diff-match-patch";
import { debounce, Notice, Plugin, TFile, addIcon, TFolder, normalizePath, TAbstractFile, Editor, MarkdownView, RequestUrlParam, RequestUrlResponse, requestUrl } from "./deps"; import { debounce, Notice, Plugin, TFile, addIcon, TFolder, normalizePath, TAbstractFile, Editor, MarkdownView, type RequestUrlParam, type RequestUrlResponse, requestUrl } from "./deps";
import { EntryDoc, LoadedEntry, ObsidianLiveSyncSettings, diff_check_result, diff_result_leaf, EntryBody, LOG_LEVEL, VER, DEFAULT_SETTINGS, diff_result, FLAGMD_REDFLAG, SYNCINFO_ID, SALT_OF_PASSPHRASE, ConfigPassphraseStore, CouchDBConnection, FLAGMD_REDFLAG2, FLAGMD_REDFLAG3, PREFIXMD_LOGFILE, DatabaseConnectingStatus, EntryHasPath, DocumentID, FilePathWithPrefix, FilePath, AnyEntry } from "./lib/src/types"; import { type EntryDoc, type LoadedEntry, type ObsidianLiveSyncSettings, type diff_check_result, type diff_result_leaf, type EntryBody, LOG_LEVEL, VER, DEFAULT_SETTINGS, type diff_result, FLAGMD_REDFLAG, SYNCINFO_ID, SALT_OF_PASSPHRASE, type ConfigPassphraseStore, type CouchDBConnection, FLAGMD_REDFLAG2, FLAGMD_REDFLAG3, PREFIXMD_LOGFILE, type DatabaseConnectingStatus, type EntryHasPath, type DocumentID, type FilePathWithPrefix, type FilePath, type AnyEntry } from "./lib/src/types";
import { InternalFileInfo, queueItem, CacheData, FileEventItem, FileWatchEventQueueMax } from "./types"; import { type InternalFileInfo, type queueItem, type CacheData, type FileEventItem, FileWatchEventQueueMax } from "./types";
import { delay, getDocData, isDocContentSame } from "./lib/src/utils"; import { getDocData, isDocContentSame, Parallels } from "./lib/src/utils";
import { Logger } from "./lib/src/logger"; import { Logger } from "./lib/src/logger";
import { PouchDB } from "./lib/src/pouchdb-browser.js"; import { PouchDB } from "./lib/src/pouchdb-browser.js";
import { LogDisplayModal } from "./LogDisplayModal";
import { ConflictResolveModal } from "./ConflictResolveModal"; import { ConflictResolveModal } from "./ConflictResolveModal";
import { ObsidianLiveSyncSettingTab } from "./ObsidianLiveSyncSettingTab"; import { ObsidianLiveSyncSettingTab } from "./ObsidianLiveSyncSettingTab";
import { DocumentHistoryModal } from "./DocumentHistoryModal"; import { DocumentHistoryModal } from "./DocumentHistoryModal";
import { applyPatch, cancelAllPeriodicTask, cancelAllTasks, cancelTask, generatePatchObj, id2path, isObjectMargeApplicable, isSensibleMargeApplicable, flattenObject, path2id, scheduleTask, tryParseJSON, createFile, modifyFile, isValidPath, getAbstractFileByPath, touch, recentlyTouched, isIdOfInternalMetadata, isPluginMetadata, stripInternalMetadataPrefix, isChunk, askSelectString, askYesNo, askString, PeriodicProcessor, clearTouched, getPath, getPathWithoutPrefix, getPathFromTFile } from "./utils"; import { applyPatch, cancelAllPeriodicTask, cancelAllTasks, cancelTask, generatePatchObj, id2path, isObjectMargeApplicable, isSensibleMargeApplicable, flattenObject, path2id, scheduleTask, tryParseJSON, createFile, modifyFile, isValidPath, getAbstractFileByPath, touch, recentlyTouched, isInternalMetadata, isPluginMetadata, stripInternalMetadataPrefix, isChunk, askSelectString, askYesNo, askString, PeriodicProcessor, clearTouched, getPath, getPathWithoutPrefix, getPathFromTFile, localDatabaseCleanUp, balanceChunks, performRebuildDB } from "./utils";
import { encrypt, tryDecrypt } from "./lib/src/e2ee_v2"; import { encrypt, tryDecrypt } from "./lib/src/e2ee_v2";
import { enableEncryption, isCloudantURI, isErrorOfMissingDoc, isValidRemoteCouchDBURI } from "./lib/src/utils_couchdb"; import { enableEncryption, isCloudantURI, isErrorOfMissingDoc, isValidRemoteCouchDBURI } from "./lib/src/utils_couchdb";
import { getGlobalStore, ObservableStore, observeStores } from "./lib/src/store"; import { getGlobalStore, ObservableStore, observeStores } from "./lib/src/store";
@@ -22,13 +21,16 @@ import { addPrefix, isPlainText, shouldBeIgnored, stripAllPrefixes } from "./lib
import { runWithLock } from "./lib/src/lock"; import { runWithLock } from "./lib/src/lock";
import { Semaphore } from "./lib/src/semaphore"; import { Semaphore } from "./lib/src/semaphore";
import { StorageEventManager, StorageEventManagerObsidian } from "./StorageEventManager"; import { StorageEventManager, StorageEventManagerObsidian } from "./StorageEventManager";
import { LiveSyncLocalDB, LiveSyncLocalDBEnv } from "./lib/src/LiveSyncLocalDB"; import { LiveSyncLocalDB, type LiveSyncLocalDBEnv } from "./lib/src/LiveSyncLocalDB";
import { LiveSyncDBReplicator, LiveSyncReplicatorEnv } from "./lib/src/LiveSyncReplicator"; import { LiveSyncDBReplicator, type LiveSyncReplicatorEnv } from "./lib/src/LiveSyncReplicator";
import { KeyValueDatabase, OpenKeyValueDatabase } from "./KeyValueDB"; import { type KeyValueDatabase, OpenKeyValueDatabase } from "./KeyValueDB";
import { LiveSyncCommands } from "./LiveSyncCommands"; import { LiveSyncCommands } from "./LiveSyncCommands";
import { PluginAndTheirSettings } from "./CmdPluginAndTheirSettings";
import { HiddenFileSync } from "./CmdHiddenFileSync"; import { HiddenFileSync } from "./CmdHiddenFileSync";
import { SetupLiveSync } from "./CmdSetupLiveSync"; import { SetupLiveSync } from "./CmdSetupLiveSync";
import { ConfigSync } from "./CmdConfigSync";
import { confirmWithMessage } from "./dialogs";
import { GlobalHistoryView, VIEW_TYPE_GLOBAL_HISTORY } from "./GlobalHistoryView";
import { LogPaneView, VIEW_TYPE_LOG } from "./LogPaneView";
setNoticeClass(Notice); setNoticeClass(Notice);
@@ -47,10 +49,11 @@ export default class ObsidianLiveSyncPlugin extends Plugin
packageVersion = ""; packageVersion = "";
manifestVersion = ""; manifestVersion = "";
addOnPluginAndTheirSettings = new PluginAndTheirSettings(this); // addOnPluginAndTheirSettings = new PluginAndTheirSettings(this);
addOnHiddenFileSync = new HiddenFileSync(this); addOnHiddenFileSync = new HiddenFileSync(this);
addOnSetup = new SetupLiveSync(this); addOnSetup = new SetupLiveSync(this);
addOns = [this.addOnPluginAndTheirSettings, this.addOnHiddenFileSync, this.addOnSetup] as LiveSyncCommands[]; addOnConfigSync = new ConfigSync(this);
addOns = [this.addOnHiddenFileSync, this.addOnSetup, this.addOnConfigSync] as LiveSyncCommands[];
periodicSyncProcessor = new PeriodicProcessor(this, async () => await this.replicate()); periodicSyncProcessor = new PeriodicProcessor(this, async () => await this.replicate());
@@ -85,7 +88,7 @@ export default class ObsidianLiveSyncPlugin extends Plugin
} }
processReplication = (e: PouchDB.Core.ExistingDocument<EntryDoc>[]) => this.parseReplicationResult(e); processReplication = (e: PouchDB.Core.ExistingDocument<EntryDoc>[]) => this.parseReplicationResult(e);
async connectRemoteCouchDB(uri: string, auth: { username: string; password: string }, disableRequestURI: boolean, passphrase: string | false, useDynamicIterationCount: boolean): Promise<string | { db: PouchDB.Database<EntryDoc>; info: PouchDB.Core.DatabaseInfo }> { async connectRemoteCouchDB(uri: string, auth: { username: string; password: string }, disableRequestURI: boolean, passphrase: string | false, useDynamicIterationCount: boolean, performSetup: boolean, skipInfo: boolean): Promise<string | { db: PouchDB.Database<EntryDoc>; info: PouchDB.Core.DatabaseInfo }> {
if (!isValidRemoteCouchDBURI(uri)) return "Remote URI is not valid"; if (!isValidRemoteCouchDBURI(uri)) return "Remote URI is not valid";
if (uri.toLowerCase() != uri) return "Remote URI and database name could not contain capital letters."; if (uri.toLowerCase() != uri) return "Remote URI and database name could not contain capital letters.";
if (uri.indexOf(" ") !== -1) return "Remote URI and database name could not contain spaces."; if (uri.indexOf(" ") !== -1) return "Remote URI and database name could not contain spaces.";
@@ -102,6 +105,7 @@ export default class ObsidianLiveSyncPlugin extends Plugin
const conf: PouchDB.HttpAdapter.HttpAdapterConfiguration = { const conf: PouchDB.HttpAdapter.HttpAdapterConfiguration = {
adapter: "http", adapter: "http",
auth, auth,
skip_setup: !performSetup,
fetch: async (url: string | Request, opts: RequestInit) => { fetch: async (url: string | Request, opts: RequestInit) => {
let size = ""; let size = "";
const localURL = url.toString().substring(uri.length); const localURL = url.toString().substring(uri.length);
@@ -190,6 +194,9 @@ export default class ObsidianLiveSyncPlugin extends Plugin
if (passphrase !== "false" && typeof passphrase === "string") { if (passphrase !== "false" && typeof passphrase === "string") {
enableEncryption(db, passphrase, useDynamicIterationCount); enableEncryption(db, passphrase, useDynamicIterationCount);
} }
if (skipInfo) {
return { db: db, info: { db_name: "", doc_count: 0, update_seq: "" } };
}
try { try {
const info = await db.info(); const info = await db.info();
return { db: db, info: info }; return { db: db, info: info };
@@ -205,7 +212,7 @@ export default class ObsidianLiveSyncPlugin extends Plugin
id2path(id: DocumentID, entry: EntryHasPath, stripPrefix?: boolean): FilePathWithPrefix { id2path(id: DocumentID, entry: EntryHasPath, stripPrefix?: boolean): FilePathWithPrefix {
const tempId = id2path(id, entry); const tempId = id2path(id, entry);
if (stripPrefix && isIdOfInternalMetadata(tempId)) { if (stripPrefix && isInternalMetadata(tempId)) {
const out = stripInternalMetadataPrefix(tempId); const out = stripInternalMetadataPrefix(tempId);
return out; return out;
} }
@@ -341,7 +348,7 @@ export default class ObsidianLiveSyncPlugin extends Plugin
} }
async resolveConflicted(target: FilePathWithPrefix) { async resolveConflicted(target: FilePathWithPrefix) {
if (isIdOfInternalMetadata(target)) { if (isInternalMetadata(target)) {
await this.addOnHiddenFileSync.resolveConflictOnInternalFile(target); await this.addOnHiddenFileSync.resolveConflictOnInternalFile(target);
} else if (isPluginMetadata(target)) { } else if (isPluginMetadata(target)) {
await this.resolveConflictByNewerEntry(target); await this.resolveConflictByNewerEntry(target);
@@ -387,25 +394,13 @@ export default class ObsidianLiveSyncPlugin extends Plugin
try { try {
if (this.isRedFlagRaised() || this.isRedFlag2Raised() || this.isRedFlag3Raised()) { if (this.isRedFlagRaised() || this.isRedFlag2Raised() || this.isRedFlag3Raised()) {
this.settings.batchSave = false; this.settings.batchSave = false;
this.settings.liveSync = false; this.addOnSetup.suspendAllSync();
this.settings.periodicReplication = false; this.addOnSetup.suspendExtraSync();
this.settings.syncOnSave = false;
this.settings.syncOnStart = false;
this.settings.syncOnFileOpen = false;
this.settings.syncAfterMerge = false;
this.settings.autoSweepPlugins = false;
this.settings.usePluginSync = false;
this.settings.suspendFileWatching = true; this.settings.suspendFileWatching = true;
this.settings.syncInternalFiles = false;
await this.saveSettings(); await this.saveSettings();
if (this.isRedFlag2Raised()) { if (this.isRedFlag2Raised()) {
Logger(`${FLAGMD_REDFLAG2} has been detected! Self-hosted LiveSync suspends all sync and rebuild everything.`, LOG_LEVEL.NOTICE); Logger(`${FLAGMD_REDFLAG2} has been detected! Self-hosted LiveSync suspends all sync and rebuild everything.`, LOG_LEVEL.NOTICE);
await this.resetLocalDatabase(); await this.addOnSetup.rebuildEverything();
await this.initializeDatabase(true);
await this.markRemoteLocked();
await this.tryResetRemoteDatabase();
await this.markRemoteLocked();
await this.replicateAllToServer(true);
await this.deleteRedFlag2(); await this.deleteRedFlag2();
if (await askYesNo(this.app, "Do you want to disable Suspend file watching and restart obsidian now?") == "yes") { if (await askYesNo(this.app, "Do you want to disable Suspend file watching and restart obsidian now?") == "yes") {
this.settings.suspendFileWatching = false; this.settings.suspendFileWatching = false;
@@ -415,12 +410,7 @@ export default class ObsidianLiveSyncPlugin extends Plugin
} }
} else if (this.isRedFlag3Raised()) { } else if (this.isRedFlag3Raised()) {
Logger(`${FLAGMD_REDFLAG3} has been detected! Self-hosted LiveSync will discard the local database and fetch everything from the remote once again.`, LOG_LEVEL.NOTICE); Logger(`${FLAGMD_REDFLAG3} has been detected! Self-hosted LiveSync will discard the local database and fetch everything from the remote once again.`, LOG_LEVEL.NOTICE);
await this.resetLocalDatabase(); await this.addOnSetup.fetchLocal();
await delay(1000);
await this.markRemoteResolved();
await this.openDatabase();
this.isReady = true;
await this.replicateAllFromServer(true);
await this.deleteRedFlag3(); await this.deleteRedFlag3();
if (await askYesNo(this.app, "Do you want to disable Suspend file watching and restart obsidian now?") == "yes") { if (await askYesNo(this.app, "Do you want to disable Suspend file watching and restart obsidian now?") == "yes") {
this.settings.suspendFileWatching = false; this.settings.suspendFileWatching = false;
@@ -550,7 +540,7 @@ export default class ObsidianLiveSyncPlugin extends Plugin
}); });
this.addRibbonIcon("view-log", "Show log", () => { this.addRibbonIcon("view-log", "Show log", () => {
new LogDisplayModal(this.app, this).open(); this.showView(VIEW_TYPE_LOG);
}); });
this.addSettingTab(new ObsidianLiveSyncSettingTab(this.app, this)); this.addSettingTab(new ObsidianLiveSyncSettingTab(this.app, this));
@@ -661,8 +651,44 @@ export default class ObsidianLiveSyncPlugin extends Plugin
}, },
}) })
this.registerView(
VIEW_TYPE_GLOBAL_HISTORY,
(leaf) => new GlobalHistoryView(leaf, this)
);
this.registerView(
VIEW_TYPE_LOG,
(leaf) => new LogPaneView(leaf, this)
);
this.addCommand({
id: "livesync-global-history",
name: "Show vault history",
callback: () => {
this.showGlobalHistory()
}
})
}
async showView(viewType: string) {
const leaves = this.app.workspace.getLeavesOfType(viewType);
if (leaves.length == 0) {
await this.app.workspace.getLeaf(true).setViewState({
type: viewType,
active: true,
});
} else {
leaves[0].setViewState({
type: viewType,
active: true,
})
}
if (leaves.length > 0) {
this.app.workspace.revealLeaf(
leaves[0]
);
}
}
showGlobalHistory() {
this.showView(VIEW_TYPE_GLOBAL_HISTORY);
} }
onunload() { onunload() {
for (const addOn of this.addOns) { for (const addOn of this.addOns) {
@@ -893,7 +919,7 @@ export default class ObsidianLiveSyncPlugin extends Plugin
async procFileEvent(applyBatch?: boolean) { async procFileEvent(applyBatch?: boolean) {
if (!this.isReady) return; if (!this.isReady) return;
if (this.settings.batchSave) { if (this.settings.batchSave && !this.settings.liveSync) {
if (!applyBatch && this.vaultManager.getQueueLength() < FileWatchEventQueueMax) { if (!applyBatch && this.vaultManager.getQueueLength() < FileWatchEventQueueMax) {
// Defer till applying batch save or queue has been grown enough. // Defer till applying batch save or queue has been grown enough.
// or 30 seconds after. // or 30 seconds after.
@@ -912,10 +938,17 @@ export default class ObsidianLiveSyncPlugin extends Plugin
const file = queue.args.file; const file = queue.args.file;
const key = `file-last-proc-${queue.type}-${file.path}`; const key = `file-last-proc-${queue.type}-${file.path}`;
const last = Number(await this.kvDB.get(key) || 0); const last = Number(await this.kvDB.get(key) || 0);
let mtime = file.mtime;
if (queue.type == "DELETE") { if (queue.type == "DELETE") {
await this.deleteFromDBbyPath(file.path); await this.deleteFromDBbyPath(file.path);
mtime = file.mtime - 1;
const keyD1 = `file-last-proc-CREATE-${file.path}`;
const keyD2 = `file-last-proc-CHANGED-${file.path}`;
await this.kvDB.set(keyD1, mtime);
await this.kvDB.set(keyD2, mtime);
} else if (queue.type == "INTERNAL") { } else if (queue.type == "INTERNAL") {
await this.addOnHiddenFileSync.watchVaultRawEventsAsync(file.path); await this.addOnHiddenFileSync.watchVaultRawEventsAsync(file.path);
await this.addOnConfigSync.watchVaultRawEventsAsync(file.path);
} else { } else {
const targetFile = this.app.vault.getAbstractFileByPath(file.path); const targetFile = this.app.vault.getAbstractFileByPath(file.path);
if (!(targetFile instanceof TFile)) { if (!(targetFile instanceof TFile)) {
@@ -930,6 +963,8 @@ export default class ObsidianLiveSyncPlugin extends Plugin
const cache = queue.args.cache; const cache = queue.args.cache;
if (queue.type == "CREATE" || queue.type == "CHANGED") { if (queue.type == "CREATE" || queue.type == "CHANGED") {
const keyD1 = `file-last-proc-DELETED-${file.path}`;
await this.kvDB.set(keyD1, mtime);
if (!await this.updateIntoDB(targetFile, false, cache)) { if (!await this.updateIntoDB(targetFile, false, cache)) {
Logger(`DB -> STORAGE: failed, cancel the relative operations: ${targetFile.path}`, LOG_LEVEL.INFO); Logger(`DB -> STORAGE: failed, cancel the relative operations: ${targetFile.path}`, LOG_LEVEL.INFO);
// cancel running queues and remove one of atomic operation // cancel running queues and remove one of atomic operation
@@ -942,7 +977,7 @@ export default class ObsidianLiveSyncPlugin extends Plugin
await this.watchVaultRenameAsync(targetFile, queue.args.oldPath); await this.watchVaultRenameAsync(targetFile, queue.args.oldPath);
} }
} }
await this.kvDB.set(key, file.mtime); await this.kvDB.set(key, mtime);
} while (this.vaultManager.getQueueLength() > 0); } while (this.vaultManager.getQueueLength() > 0);
return true; return true;
}) })
@@ -970,7 +1005,7 @@ export default class ObsidianLiveSyncPlugin extends Plugin
} }
async applyBatchChange() { async applyBatchChange() {
if (this.settings.batchSave) { if (this.settings.batchSave && !this.settings.liveSync) {
return await this.procFileEvent(true); return await this.procFileEvent(true);
} }
} }
@@ -1005,7 +1040,7 @@ export default class ObsidianLiveSyncPlugin extends Plugin
} }
//--> Basic document Functions //--> Basic document Functions
notifies: { [key: string]: { notice: Notice; timer: NodeJS.Timeout; count: number } } = {}; notifies: { [key: string]: { notice: Notice; timer: ReturnType<typeof setTimeout>; count: number } } = {};
lastLog = ""; lastLog = "";
// eslint-disable-next-line require-await // eslint-disable-next-line require-await
@@ -1114,13 +1149,16 @@ export default class ObsidianLiveSyncPlugin extends Plugin
// This occurs not only when files are deleted, but also when conflicts are resolved. // This occurs not only when files are deleted, but also when conflicts are resolved.
// We have to check no other revisions are left. // We have to check no other revisions are left.
const lastDocs = await this.localDatabase.getDBEntry(path); const lastDocs = await this.localDatabase.getDBEntry(path);
if (path != file.path) {
Logger(`delete skipped: ${file.path} :Not exactly matched`, LOG_LEVEL.VERBOSE);
}
if (lastDocs === false) { if (lastDocs === false) {
await this.deleteVaultItem(file); await this.deleteVaultItem(file);
} else { } else {
// it perhaps delete some revisions. // it perhaps delete some revisions.
// may be we have to reload this // may be we have to reload this
await this.pullFile(path, null, true); await this.pullFile(path, null, true);
Logger(`delete skipped:${lastDocs._id}`, LOG_LEVEL.VERBOSE); Logger(`delete skipped:${file.path}`, LOG_LEVEL.VERBOSE);
} }
return; return;
} }
@@ -1288,7 +1326,7 @@ export default class ObsidianLiveSyncPlugin extends Plugin
const now = new Date().getTime(); const now = new Date().getTime();
if (queue.missingChildren.length == 0) { if (queue.missingChildren.length == 0) {
queue.done = true; queue.done = true;
if (isIdOfInternalMetadata(queue.entry._id)) { if (isInternalMetadata(queue.entry._id)) {
//system file //system file
const filename = this.getPathWithoutPrefix(queue.entry); const filename = this.getPathWithoutPrefix(queue.entry);
this.addOnHiddenFileSync.procInternalFile(filename); this.addOnHiddenFileSync.procInternalFile(filename);
@@ -1333,7 +1371,7 @@ export default class ObsidianLiveSyncPlugin extends Plugin
if (!this.isTargetFile(path)) return; if (!this.isTargetFile(path)) return;
const skipOldFile = this.settings.skipOlderFilesOnSync && false; //patched temporary. const skipOldFile = this.settings.skipOlderFilesOnSync && false; //patched temporary.
// Do not handle internal files if the feature has not been enabled. // Do not handle internal files if the feature has not been enabled.
if (isIdOfInternalMetadata(doc._id) && !this.settings.syncInternalFiles) return; if (isInternalMetadata(doc._id) && !this.settings.syncInternalFiles) return;
// It is better for your own safety, not to handle the following files // It is better for your own safety, not to handle the following files
const ignoreFiles = [ const ignoreFiles = [
"_design/replicate", "_design/replicate",
@@ -1341,11 +1379,11 @@ export default class ObsidianLiveSyncPlugin extends Plugin
FLAGMD_REDFLAG2, FLAGMD_REDFLAG2,
FLAGMD_REDFLAG3 FLAGMD_REDFLAG3
]; ];
if (!isIdOfInternalMetadata(doc._id) && ignoreFiles.contains(path)) { if (!isInternalMetadata(doc._id) && ignoreFiles.contains(path)) {
return; return;
} }
if ((!isIdOfInternalMetadata(doc._id)) && skipOldFile) { if ((!isInternalMetadata(doc._id)) && skipOldFile) {
const info = getAbstractFileByPath(stripAllPrefixes(path)); const info = getAbstractFileByPath(stripAllPrefixes(path));
if (info && info instanceof TFile) { if (info && info instanceof TFile) {
@@ -1353,7 +1391,7 @@ export default class ObsidianLiveSyncPlugin extends Plugin
const docMtime = ~~(doc.mtime / 1000); const docMtime = ~~(doc.mtime / 1000);
//TODO: some margin required. //TODO: some margin required.
if (localMtime >= docMtime) { if (localMtime >= docMtime) {
Logger(`${doc._id} Skipped, older than storage.`, LOG_LEVEL.VERBOSE); Logger(`${path} (${doc._id}, ${doc._rev}) Skipped, older than storage.`, LOG_LEVEL.VERBOSE);
return; return;
} }
} }
@@ -1364,12 +1402,12 @@ export default class ObsidianLiveSyncPlugin extends Plugin
missingChildren: [] as string[], missingChildren: [] as string[],
timeout: now + this.chunkWaitTimeout, timeout: now + this.chunkWaitTimeout,
}; };
// If `Read chunks online` is enabled, retrieve chunks from the remote CouchDB directly. // If `Read chunks online` is disabled, chunks should be transferred before here.
// However, in some cases, chunks are after that. So, if missing chunks exist, we have to wait for them.
if ((!this.settings.readChunksOnline) && "children" in doc) { if ((!this.settings.readChunksOnline) && "children" in doc) {
const c = await this.localDatabase.allDocsRaw<EntryDoc>({ keys: doc.children, include_docs: false }); const c = await this.localDatabase.collectChunksWithCache(doc.children as DocumentID[]);
const missing = c.rows.filter((e) => "error" in e).map((e) => e.key); const missing = c.filter((e) => e.chunk === false).map((e) => e.id);
// fetch from remote if (missing.length > 0) Logger(`${path} (${doc._id}, ${doc._rev}) Queued (waiting ${missing.length} items)`, LOG_LEVEL.VERBOSE);
if (missing.length > 0) Logger(`${doc._id}(${doc._rev}) Queued (waiting ${missing.length} items)`, LOG_LEVEL.VERBOSE);
newQueue.missingChildren = missing; newQueue.missingChildren = missing;
this.queuedFiles.push(newQueue); this.queuedFiles.push(newQueue);
} else { } else {
@@ -1381,17 +1419,18 @@ export default class ObsidianLiveSyncPlugin extends Plugin
//---> Sync //---> Sync
async parseReplicationResult(docs: Array<PouchDB.Core.ExistingDocument<EntryDoc>>): Promise<void> { async parseReplicationResult(docs: Array<PouchDB.Core.ExistingDocument<EntryDoc>>): Promise<void> {
const docsSorted = docs.sort((a: any, b: any) => b?.mtime ?? 0 - a?.mtime ?? 0);
L1: L1:
for (const change of docs) { for (const change of docsSorted) {
if (isChunk(change._id)) {
await this.parseIncomingChunk(change);
continue;
}
for (const proc of this.addOns) { for (const proc of this.addOns) {
if (await proc.parseReplicationResultItem(change)) { if (await proc.parseReplicationResultItem(change)) {
continue L1; continue L1;
} }
} }
if (isChunk(change._id)) {
await this.parseIncomingChunk(change);
continue;
}
if (change._id == SYNCINFO_ID) { if (change._id == SYNCINFO_ID) {
continue; continue;
} }
@@ -1468,9 +1507,11 @@ export default class ObsidianLiveSyncPlugin extends Plugin
} }
this.statusBar.title = e.syncStatus; this.statusBar.title = e.syncStatus;
let waiting = ""; let waiting = "";
if (this.settings.batchSave) { if (this.settings.batchSave && !this.settings.liveSync) {
waiting = " " + "🛫".repeat(this.vaultManager.getQueueLength()); const len = this.vaultManager?.getQueueLength();
waiting = waiting.replace(/(🛫){10}/g, "🚀"); if (len != 0) {
waiting = ` 🛫${len}`;
}
} }
let queued = ""; let queued = "";
const queue = Object.entries(e.queuedItems).filter((e) => !e[1].warned); const queue = Object.entries(e.queuedItems).filter((e) => !e[1].warned);
@@ -1478,7 +1519,7 @@ export default class ObsidianLiveSyncPlugin extends Plugin
if (queuedCount) { if (queuedCount) {
const pieces = queue.map((e) => e[1].missingChildren).reduce((prev, cur) => prev + cur.length, 0); const pieces = queue.map((e) => e[1].missingChildren).reduce((prev, cur) => prev + cur.length, 0);
queued = ` 🧩 ${queuedCount} (${pieces})`; queued = ` 🧩${queuedCount} (${pieces})`;
} }
const processes = e.count; const processes = e.count;
const processesDisp = processes == 0 ? "" : ` ⏳${processes}`; const processesDisp = processes == 0 ? "" : ` ⏳${processes}`;
@@ -1490,19 +1531,20 @@ export default class ObsidianLiveSyncPlugin extends Plugin
} }
return proc.substring(0, p); return proc.substring(0, p);
} }
const pendingTask = e.pending.length const pendingTask = e.pending.length
? "\nPending: " + ? e.pending.length < 10 ? ("\nPending: " +
Object.entries(e.pending.reduce((p, c) => ({ ...p, [getProcKind(c)]: (p[getProcKind(c)] ?? 0) + 1 }), {} as { [key: string]: number })) Object.entries(e.pending.reduce((p, c) => ({ ...p, [getProcKind(c)]: (p[getProcKind(c)] ?? 0) + 1 }), {} as { [key: string]: number }))
.map((e) => `${e[0]}${e[1] == 1 ? "" : `(${e[1]})`}`) .map((e) => `${e[0]}${e[1] == 1 ? "" : `(${e[1]})`}`)
.join(", ") .join(", ")
: ""; ) : `\n Pending: ${e.pending.length}` : "";
const runningTask = e.running.length const runningTask = e.running.length
? "\nRunning: " + ? e.running.length < 10 ? ("\nRunning: " +
Object.entries(e.running.reduce((p, c) => ({ ...p, [getProcKind(c)]: (p[getProcKind(c)] ?? 0) + 1 }), {} as { [key: string]: number })) Object.entries(e.running.reduce((p, c) => ({ ...p, [getProcKind(c)]: (p[getProcKind(c)] ?? 0) + 1 }), {} as { [key: string]: number }))
.map((e) => `${e[0]}${e[1] == 1 ? "" : `(${e[1]})`}`) .map((e) => `${e[0]}${e[1] == 1 ? "" : `(${e[1]})`}`)
.join(", ") .join(", ")
: ""; ) : `\n Running: ${e.running.length}` : "";
this.setStatusBarText(message + pendingTask + runningTask); this.setStatusBarText(message + pendingTask + runningTask);
}) })
} }
@@ -1511,27 +1553,25 @@ export default class ObsidianLiveSyncPlugin extends Plugin
return; return;
} }
logHideTimer: NodeJS.Timeout = null;
setStatusBarText(message: string = null, log: string = null) { setStatusBarText(message: string = null, log: string = null) {
if (!this.statusBar) return; if (!this.statusBar) return;
const newMsg = typeof message == "string" ? message : this.lastMessage; const newMsg = typeof message == "string" ? message : this.lastMessage;
const newLog = typeof log == "string" ? log : this.lastLog; const newLog = typeof log == "string" ? log : this.lastLog;
if (`${this.lastMessage}-${this.lastLog}` != `${newMsg}-${newLog}`) { if (`${this.lastMessage}-${this.lastLog}` != `${newMsg}-${newLog}`) {
this.statusBar.setText(newMsg.split("\n")[0]); scheduleTask("update-display", 50, () => {
this.statusBar.setText(newMsg.split("\n")[0]);
if (this.settings.showStatusOnEditor) { if (this.settings.showStatusOnEditor) {
const root = activeDocument.documentElement; const root = activeDocument.documentElement;
const q = root.querySelectorAll(`.CodeMirror-wrap,.cm-s-obsidian>.cm-editor,.canvas-wrapper`); const q = root.querySelectorAll(`.CodeMirror-wrap,.cm-s-obsidian>.cm-editor,.canvas-wrapper`);
q.forEach(e => e.setAttr("data-log", '' + (newMsg + "\n" + newLog) + '')) q.forEach(e => e.setAttr("data-log", '' + (newMsg + "\n" + newLog) + ''))
} else { } else {
const root = activeDocument.documentElement; const root = activeDocument.documentElement;
const q = root.querySelectorAll(`.CodeMirror-wrap,.cm-s-obsidian>.cm-editor,.canvas-wrapper`); const q = root.querySelectorAll(`.CodeMirror-wrap,.cm-s-obsidian>.cm-editor,.canvas-wrapper`);
q.forEach(e => e.setAttr("data-log", '')) q.forEach(e => e.setAttr("data-log", ''))
} }
if (this.logHideTimer != null) { }, true);
clearTimeout(this.logHideTimer); scheduleTask("log-hide", 3000, () => this.setStatusBarText(null, ""));
}
this.logHideTimer = setTimeout(() => this.setStatusBarText(null, ""), 3000);
this.lastMessage = newMsg; this.lastMessage = newMsg;
this.lastLog = newLog; this.lastLog = newLog;
} }
@@ -1547,7 +1587,43 @@ export default class ObsidianLiveSyncPlugin extends Plugin
await this.applyBatchChange(); await this.applyBatchChange();
await Promise.all(this.addOns.map(e => e.beforeReplicate(showMessage))); await Promise.all(this.addOns.map(e => e.beforeReplicate(showMessage)));
await this.loadQueuedFiles(); await this.loadQueuedFiles();
return await this.replicator.openReplication(this.settings, false, showMessage); const ret = await this.replicator.openReplication(this.settings, false, showMessage);
if (!ret) {
if (this.replicator.remoteLockedAndDeviceNotAccepted) {
if (this.replicator.remoteCleaned) {
const message = `
The remote database has been cleaned up.
To synchronize, this device must also be cleaned up or fetch everything again once.
Fetching may takes some time. Cleaning up is not stable yet but fast.
`
const CHOICE_CLEANUP = "Clean up";
const CHOICE_FETCH = "Fetch again";
const CHOICE_DISMISS = "Dismiss";
const ret = await confirmWithMessage(this, "Locked", message, [CHOICE_CLEANUP, CHOICE_FETCH, CHOICE_DISMISS], CHOICE_DISMISS, 10);
if (ret == CHOICE_CLEANUP) {
await localDatabaseCleanUp(this, true, false);
await balanceChunks(this, false);
}
if (ret == CHOICE_FETCH) {
await performRebuildDB(this, "localOnly");
}
} else {
const message = `
The remote database has been rebuilt.
To synchronize, this device must fetch everything again once.
Or if you are sure know what had been happened, we can unlock the database from the setting dialog.
`
const CHOICE_FETCH = "Fetch again";
const CHOICE_DISMISS = "Dismiss";
const ret = await confirmWithMessage(this, "Locked", message, [CHOICE_FETCH, CHOICE_DISMISS], CHOICE_DISMISS, 10);
if (ret == CHOICE_FETCH) {
await performRebuildDB(this, "localOnly");
}
}
}
}
return ret;
} }
async initializeDatabase(showingNotice?: boolean, reopenDatabase = true) { async initializeDatabase(showingNotice?: boolean, reopenDatabase = true) {
@@ -1578,12 +1654,12 @@ export default class ObsidianLiveSyncPlugin extends Plugin
return await this.replicator.replicateAllFromServer(this.settings, showingNotice); return await this.replicator.replicateAllFromServer(this.settings, showingNotice);
} }
async markRemoteLocked() { async markRemoteLocked(lockByClean?: boolean) {
return await this.replicator.markRemoteLocked(this.settings, true); return await this.replicator.markRemoteLocked(this.settings, true, lockByClean);
} }
async markRemoteUnlocked() { async markRemoteUnlocked() {
return await this.replicator.markRemoteLocked(this.settings, false); return await this.replicator.markRemoteLocked(this.settings, false, false);
} }
async markRemoteResolved() { async markRemoteResolved() {
@@ -1606,13 +1682,15 @@ export default class ObsidianLiveSyncPlugin extends Plugin
const filesStorageName = filesStorage.map((e) => e.path); const filesStorageName = filesStorage.map((e) => e.path);
Logger("Collecting local files on the DB", LOG_LEVEL.VERBOSE); Logger("Collecting local files on the DB", LOG_LEVEL.VERBOSE);
const filesDatabase = [] as FilePathWithPrefix[] const filesDatabase = [] as FilePathWithPrefix[]
let count = 0;
for await (const doc of this.localDatabase.findAllNormalDocs()) { for await (const doc of this.localDatabase.findAllNormalDocs()) {
count++;
if (count % 25 == 0) Logger(`Collecting local files on the DB: ${count}`, showingNotice ? LOG_LEVEL.NOTICE : LOG_LEVEL.INFO, "syncAll");
const path = getPath(doc); const path = getPath(doc);
if (isValidPath(path) && this.isTargetFile(path)) { if (isValidPath(path) && this.isTargetFile(path)) {
filesDatabase.push(path); filesDatabase.push(path);
} }
} }
Logger("Opening the key-value database", LOG_LEVEL.VERBOSE); Logger("Opening the key-value database", LOG_LEVEL.VERBOSE);
const isInitialized = await (this.kvDB.get<boolean>("initialized")) || false; const isInitialized = await (this.kvDB.get<boolean>("initialized")) || false;
// Make chunk bigger if it is the initial scan. There must be non-active docs. // Make chunk bigger if it is the initial scan. There must be non-active docs.
@@ -1631,25 +1709,23 @@ export default class ObsidianLiveSyncPlugin extends Plugin
const runAll = async<T>(procedureName: string, objects: T[], callback: (arg: T) => Promise<void>) => { const runAll = async<T>(procedureName: string, objects: T[], callback: (arg: T) => Promise<void>) => {
Logger(procedureName); Logger(procedureName);
const semaphore = Semaphore(25);
if (!this.localDatabase.isReady) throw Error("Database is not ready!"); if (!this.localDatabase.isReady) throw Error("Database is not ready!");
const processes = objects.map(e => (async (v) => { const para = Parallels();
const releaser = await semaphore.acquire(1, procedureName); for (const v of objects) {
await para.wait(10);
para.add((async (v) => {
try {
await callback(v);
} catch (ex) {
Logger(`Error while ${procedureName}`, LOG_LEVEL.NOTICE);
Logger(ex);
}
})(v));
try {
await callback(v);
} catch (ex) {
Logger(`Error while ${procedureName}`, LOG_LEVEL.NOTICE);
Logger(ex);
} finally {
releaser();
}
} }
)(e)); await para.all();
await Promise.all(processes);
Logger(`${procedureName} done.`); Logger(`${procedureName} done.`);
}; }
await runAll("UPDATE DATABASE", onlyInStorage, async (e) => { await runAll("UPDATE DATABASE", onlyInStorage, async (e) => {
Logger(`UPDATE DATABASE ${e.path}`); Logger(`UPDATE DATABASE ${e.path}`);
@@ -2043,74 +2119,70 @@ export default class ObsidianLiveSyncPlugin extends Plugin
} }
showMergeDialog(filename: FilePathWithPrefix, conflictCheckResult: diff_result): Promise<boolean> { showMergeDialog(filename: FilePathWithPrefix, conflictCheckResult: diff_result): Promise<boolean> {
return new Promise((res, rej) => { return runWithLock("resolve-conflict:" + filename, false, () =>
Logger("open conflict dialog", LOG_LEVEL.VERBOSE); new Promise((res, rej) => {
new ConflictResolveModal(this.app, filename, conflictCheckResult, async (selected) => { Logger("open conflict dialog", LOG_LEVEL.VERBOSE);
const testDoc = await this.localDatabase.getDBEntry(filename, { conflicts: true }, false, false, true); new ConflictResolveModal(this.app, filename, conflictCheckResult, async (selected) => {
if (testDoc === false) { const testDoc = await this.localDatabase.getDBEntry(filename, { conflicts: true }, false, false, true);
Logger("Missing file..", LOG_LEVEL.VERBOSE); if (testDoc === false) {
return res(true); Logger("Missing file..", LOG_LEVEL.VERBOSE);
} return res(true);
if (!testDoc._conflicts) { }
Logger("Nothing have to do with this conflict", LOG_LEVEL.VERBOSE); if (!testDoc._conflicts) {
return res(true); Logger("Nothing have to do with this conflict", LOG_LEVEL.VERBOSE);
} return res(true);
const toDelete = selected; }
const toKeep = conflictCheckResult.left.rev != toDelete ? conflictCheckResult.left.rev : conflictCheckResult.right.rev; const toDelete = selected;
if (toDelete == "") { const toKeep = conflictCheckResult.left.rev != toDelete ? conflictCheckResult.left.rev : conflictCheckResult.right.rev;
// concat both, if (toDelete == "") {
// delete conflicted revision and write a new file, store it again. // concat both,
const p = conflictCheckResult.diff.map((e) => e[1]).join(""); // delete conflicted revision and write a new file, store it again.
await this.localDatabase.deleteDBEntry(filename, { rev: testDoc._conflicts[0] }); const p = conflictCheckResult.diff.map((e) => e[1]).join("");
const file = getAbstractFileByPath(stripAllPrefixes(filename)) as TFile; await this.localDatabase.deleteDBEntry(filename, { rev: testDoc._conflicts[0] });
if (file) { const file = getAbstractFileByPath(stripAllPrefixes(filename)) as TFile;
await this.app.vault.modify(file, p); if (file) {
await this.updateIntoDB(file); await this.app.vault.modify(file, p);
await this.updateIntoDB(file);
} else {
const newFile = await this.app.vault.create(filename, p);
await this.updateIntoDB(newFile);
}
await this.pullFile(filename);
Logger("concat both file");
if (this.settings.syncAfterMerge && !this.suspended) {
await this.replicate();
}
setTimeout(() => {
//resolved, check again.
this.showIfConflicted(filename);
}, 500);
} else if (toDelete == null) {
Logger("Leave it still conflicted");
} else { } else {
const newFile = await this.app.vault.create(filename, p); await this.localDatabase.deleteDBEntry(filename, { rev: toDelete });
await this.updateIntoDB(newFile); await this.pullFile(filename, null, true, toKeep);
Logger(`Conflict resolved:${filename}`);
if (this.settings.syncAfterMerge && !this.suspended) {
await this.replicate();
}
setTimeout(() => {
//resolved, check again.
this.showIfConflicted(filename);
}, 500);
} }
await this.pullFile(filename);
Logger("concat both file");
if (this.settings.syncAfterMerge && !this.suspended) {
await this.replicate();
}
setTimeout(() => {
//resolved, check again.
this.showIfConflicted(filename);
}, 500);
} else if (toDelete == null) {
Logger("Leave it still conflicted");
} else {
await this.localDatabase.deleteDBEntry(filename, { rev: toDelete });
await this.pullFile(filename, null, true, toKeep);
Logger(`Conflict resolved:${filename}`);
if (this.settings.syncAfterMerge && !this.suspended) {
await this.replicate();
}
setTimeout(() => {
//resolved, check again.
this.showIfConflicted(filename);
}, 500);
}
return res(true); return res(true);
}).open(); }).open();
}); })
);
} }
conflictedCheckFiles: FilePath[] = []; conflictedCheckFiles: FilePath[] = [];
// queueing the conflicted file check // queueing the conflicted file check
conflictedCheckTimer: number;
queueConflictedCheck(file: TFile) { queueConflictedCheck(file: TFile) {
this.conflictedCheckFiles = this.conflictedCheckFiles.filter((e) => e != file.path); this.conflictedCheckFiles = this.conflictedCheckFiles.filter((e) => e != file.path);
this.conflictedCheckFiles.push(getPathFromTFile(file)); this.conflictedCheckFiles.push(getPathFromTFile(file));
if (this.conflictedCheckTimer != null) { scheduleTask("check-conflict", 100, async () => {
window.clearTimeout(this.conflictedCheckTimer);
}
this.conflictedCheckTimer = window.setTimeout(async () => {
this.conflictedCheckTimer = null;
const checkFiles = JSON.parse(JSON.stringify(this.conflictedCheckFiles)) as FilePath[]; const checkFiles = JSON.parse(JSON.stringify(this.conflictedCheckFiles)) as FilePath[];
for (const filename of checkFiles) { for (const filename of checkFiles) {
try { try {
@@ -2122,7 +2194,7 @@ export default class ObsidianLiveSyncPlugin extends Plugin
Logger(ex); Logger(ex);
} }
} }
}, 100); });
} }
async showIfConflicted(filename: FilePathWithPrefix) { async showIfConflicted(filename: FilePathWithPrefix) {
@@ -2193,7 +2265,7 @@ export default class ObsidianLiveSyncPlugin extends Plugin
const dK = `${file.path}-diff`; const dK = `${file.path}-diff`;
const isLastDiff = dK in caches ? caches[dK] : { storageMtime: 0, docMtime: 0 }; const isLastDiff = dK in caches ? caches[dK] : { storageMtime: 0, docMtime: 0 };
if (isLastDiff.docMtime == docMtime && isLastDiff.storageMtime == storageMtime) { if (isLastDiff.docMtime == docMtime && isLastDiff.storageMtime == storageMtime) {
Logger("STORAGE .. DB :" + file.path, LOG_LEVEL.VERBOSE); // Logger("STORAGE .. DB :" + file.path, LOG_LEVEL.VERBOSE);
caches[dK] = { storageMtime, docMtime }; caches[dK] = { storageMtime, docMtime };
return caches; return caches;
} }

8
src/types.ts
View File

@@ -62,13 +62,21 @@ export type FileEventItem = {
key: string, key: string,
} }
// Hidden items (Now means `chunk`)
export const CHeader = "h:"; export const CHeader = "h:";
// Plug-in Stored Container (Obsolete)
export const PSCHeader = "ps:"; export const PSCHeader = "ps:";
export const PSCHeaderEnd = "ps;"; export const PSCHeaderEnd = "ps;";
// Internal data Container
export const ICHeader = "i:"; export const ICHeader = "i:";
export const ICHeaderEnd = "i;"; export const ICHeaderEnd = "i;";
export const ICHeaderLength = ICHeader.length; export const ICHeaderLength = ICHeader.length;
// Internal data Container (eXtended)
export const ICXHeader = "ix:";
export const FileWatchEventQueueMax = 10; export const FileWatchEventQueueMax = 10;
export const configURIBase = "obsidian://setuplivesync?settings="; export const configURIBase = "obsidian://setuplivesync?settings=";

344
src/utils.ts
View File

@@ -1,10 +1,12 @@
import { DataWriteOptions, normalizePath, TFile, Platform, TAbstractFile, App, Plugin_2 } from "./deps"; import { type DataWriteOptions, normalizePath, TFile, Platform, TAbstractFile, App, Plugin_2, type RequestUrlParam, requestUrl } from "./deps";
import { path2id_base, id2path_base, isValidFilenameInLinux, isValidFilenameInDarwin, isValidFilenameInWidows, isValidFilenameInAndroid, stripAllPrefixes } from "./lib/src/path"; import { path2id_base, id2path_base, isValidFilenameInLinux, isValidFilenameInDarwin, isValidFilenameInWidows, isValidFilenameInAndroid, stripAllPrefixes } from "./lib/src/path";
import { Logger } from "./lib/src/logger"; import { Logger } from "./lib/src/logger";
import { AnyEntry, DocumentID, EntryHasPath, FilePath, FilePathWithPrefix, LOG_LEVEL } from "./lib/src/types"; import { type AnyEntry, type DocumentID, type EntryDoc, type EntryHasPath, type FilePath, type FilePathWithPrefix, LOG_LEVEL, type NewEntry } from "./lib/src/types";
import { CHeader, ICHeader, ICHeaderLength, PSCHeader } from "./types"; import { CHeader, ICHeader, ICHeaderLength, PSCHeader } from "./types";
import { InputStringDialog, PopoverSelectString } from "./dialogs"; import { InputStringDialog, PopoverSelectString } from "./dialogs";
import ObsidianLiveSyncPlugin from "./main";
import { runWithLock } from "./lib/src/lock";
// For backward compatibility, using the path for determining id. // For backward compatibility, using the path for determining id.
// Only CouchDB unacceptable ID (that starts with an underscore) has been prefixed with "/". // Only CouchDB unacceptable ID (that starts with an underscore) has been prefixed with "/".
@@ -42,7 +44,10 @@ export function getPathFromTFile(file: TAbstractFile) {
} }
const tasks: { [key: string]: ReturnType<typeof setTimeout> } = {}; const tasks: { [key: string]: ReturnType<typeof setTimeout> } = {};
export function scheduleTask(key: string, timeout: number, proc: (() => Promise<any> | void)) { export function scheduleTask(key: string, timeout: number, proc: (() => Promise<any> | void), skipIfTaskExist?: boolean) {
if (skipIfTaskExist && key in tasks) {
return;
}
cancelTask(key); cancelTask(key);
tasks[key] = setTimeout(async () => { tasks[key] = setTimeout(async () => {
delete tasks[key]; delete tasks[key];
@@ -232,8 +237,8 @@ export function applyPatch(from: Record<string | number | symbol, any>, patch: R
} }
export function mergeObject( export function mergeObject(
objA: Record<string | number | symbol, any>, objA: Record<string | number | symbol, any> | [any],
objB: Record<string | number | symbol, any> objB: Record<string | number | symbol, any> | [any]
) { ) {
const newEntries = Object.entries(objB); const newEntries = Object.entries(objB);
const ret: any = { ...objA }; const ret: any = { ...objA };
@@ -276,6 +281,11 @@ export function mergeObject(
ret[key] = v; ret[key] = v;
} }
} }
if (Array.isArray(objA) && Array.isArray(objB)) {
return Object.values(Object.entries(ret)
.sort()
.reduce((p, [key, value]) => ({ ...p, [key]: value }), {}));
}
return Object.entries(ret) return Object.entries(ret)
.sort() .sort()
.reduce((p, [key, value]) => ({ ...p, [key]: value }), {}); .reduce((p, [key, value]) => ({ ...p, [key]: value }), {});
@@ -325,14 +335,16 @@ export function isValidPath(filename: string) {
let touchedFiles: string[] = []; let touchedFiles: string[] = [];
export function getAbstractFileByPath(path: FilePath): TAbstractFile | null { export function getAbstractFileByPath(path: FilePath): TAbstractFile | null {
// Hidden API but so useful. // Disabled temporary.
// @ts-ignore return app.vault.getAbstractFileByPath(path);
if ("getAbstractFileByPathInsensitive" in app.vault && (app.vault.adapter?.insensitive ?? false)) { // // Hidden API but so useful.
// @ts-ignore // // @ts-ignore
return app.vault.getAbstractFileByPathInsensitive(path); // if ("getAbstractFileByPathInsensitive" in app.vault && (app.vault.adapter?.insensitive ?? false)) {
} else { // // @ts-ignore
return app.vault.getAbstractFileByPath(path); // return app.vault.getAbstractFileByPathInsensitive(path);
} // } else {
// return app.vault.getAbstractFileByPath(path);
// }
} }
export function trimPrefix(target: string, prefix: string) { export function trimPrefix(target: string, prefix: string) {
return target.startsWith(prefix) ? target.substring(prefix.length) : target; return target.startsWith(prefix) ? target.substring(prefix.length) : target;
@@ -358,7 +370,7 @@ export function clearTouched() {
* @param id ID * @param id ID
* @returns * @returns
*/ */
export function isIdOfInternalMetadata(id: FilePath | FilePathWithPrefix | DocumentID): boolean { export function isInternalMetadata(id: FilePath | FilePathWithPrefix | DocumentID): boolean {
return id.startsWith(ICHeader); return id.startsWith(ICHeader);
} }
export function stripInternalMetadataPrefix<T extends FilePath | FilePathWithPrefix | DocumentID>(id: T): T { export function stripInternalMetadataPrefix<T extends FilePath | FilePathWithPrefix | DocumentID>(id: T): T {
@@ -393,9 +405,9 @@ export const askSelectString = (app: App, message: string, items: string[]): Pro
}; };
export const askString = (app: App, title: string, key: string, placeholder: string): Promise<string | false> => { export const askString = (app: App, title: string, key: string, placeholder: string, isPassword?: boolean): Promise<string | false> => {
return new Promise((res) => { return new Promise((res) => {
const dialog = new InputStringDialog(app, title, key, placeholder, (result) => res(result)); const dialog = new InputStringDialog(app, title, key, placeholder, isPassword, (result) => res(result));
dialog.open(); dialog.open();
}); });
}; };
@@ -426,3 +438,303 @@ export class PeriodicProcessor {
if (this._timer) clearInterval(this._timer); if (this._timer) clearInterval(this._timer);
} }
} }
function sizeToHumanReadable(size: number | undefined) {
if (!size) return "-";
const i = Math.floor(Math.log(size) / Math.log(1024));
return Number.parseInt((size / Math.pow(1024, i)).toFixed(2)) + ' ' + ['B', 'kB', 'MB', 'GB', 'TB'][i];
}
export const _requestToCouchDBFetch = async (baseUri: string, username: string, password: string, path?: string, body?: string | any, method?: string) => {
const utf8str = String.fromCharCode.apply(null, new TextEncoder().encode(`${username}:${password}`));
const encoded = window.btoa(utf8str);
const authHeader = "Basic " + encoded;
const transformedHeaders: Record<string, string> = { authorization: authHeader, "content-type": "application/json" };
const uri = `${baseUri}/${path}`;
const requestParam = {
url: uri,
method: method || (body ? "PUT" : "GET"),
headers: new Headers(transformedHeaders),
contentType: "application/json",
body: JSON.stringify(body),
};
return await fetch(uri, requestParam);
}
export const _requestToCouchDB = async (baseUri: string, username: string, password: string, origin: string, path?: string, body?: any, method?: string) => {
const utf8str = String.fromCharCode.apply(null, new TextEncoder().encode(`${username}:${password}`));
const encoded = window.btoa(utf8str);
const authHeader = "Basic " + encoded;
const transformedHeaders: Record<string, string> = { authorization: authHeader, origin: origin };
const uri = `${baseUri}/${path}`;
const requestParam: RequestUrlParam = {
url: uri,
method: method || (body ? "PUT" : "GET"),
headers: transformedHeaders,
contentType: "application/json",
body: body ? JSON.stringify(body) : undefined,
};
return await requestUrl(requestParam);
}
export const requestToCouchDB = async (baseUri: string, username: string, password: string, origin: string, key?: string, body?: string, method?: string) => {
const uri = `_node/_local/_config${key ? "/" + key : ""}`;
return await _requestToCouchDB(baseUri, username, password, origin, uri, body, method);
};
export async function performRebuildDB(plugin: ObsidianLiveSyncPlugin, method: "localOnly" | "remoteOnly" | "rebuildBothByThisDevice") {
if (method == "localOnly") {
await plugin.addOnSetup.fetchLocal();
}
if (method == "remoteOnly") {
await plugin.addOnSetup.rebuildRemote();
}
if (method == "rebuildBothByThisDevice") {
await plugin.addOnSetup.rebuildEverything();
}
}
export const gatherChunkUsage = async (db: PouchDB.Database<EntryDoc>) => {
const used = new Map();
const unreferenced = new Map();
const removed = new Map();
const missing = new Map();
const xx = await db.allDocs({ startkey: "h:", endkey: `h:\u{10ffff}` });
for (const xxd of xx.rows) {
const chunk = xxd.id
unreferenced.set(chunk, xxd.value.rev);
}
const x = await db.find({ limit: 999999999, selector: { children: { $exists: true, $type: "array" } }, fields: ["_id", "path", "mtime", "children"] });
for (const temp of x.docs) {
for (const chunk of (temp as NewEntry).children) {
used.set(chunk, (used.has(chunk) ? used.get(chunk) : 0) + 1);
if (unreferenced.has(chunk)) {
removed.set(chunk, unreferenced.get(chunk));
unreferenced.delete(chunk);
} else {
if (!removed.has(chunk)) {
if (!missing.has(temp._id)) {
missing.set(temp._id, []);
}
missing.get(temp._id).push(chunk);
}
}
}
}
return { used, unreferenced, missing };
}
export const localDatabaseCleanUp = async (plugin: ObsidianLiveSyncPlugin, force: boolean, dryRun: boolean) => {
await runWithLock("clean-up:local", true, async () => {
const db = plugin.localDatabase.localDatabase;
if ((db as any)?.adapter != "indexeddb") {
if (force && !dryRun) {
Logger("Fetch from the remote database", LOG_LEVEL.NOTICE, "clean-up-db");
await performRebuildDB(plugin, "localOnly");
return;
} else {
Logger("This feature requires disabling `Use an old adapter for compatibility`.", LOG_LEVEL.NOTICE, "clean-up-db");
return;
}
}
Logger(`The remote database has been locked for garbage collection`, LOG_LEVEL.NOTICE, "clean-up-db");
Logger(`Gathering chunk usage information`, LOG_LEVEL.NOTICE, "clean-up-db");
const { unreferenced, missing } = await gatherChunkUsage(db);
if (missing.size != 0) {
Logger(`Some chunks are not found! We have to rescue`, LOG_LEVEL.NOTICE);
Logger(missing, LOG_LEVEL.VERBOSE);
} else {
Logger(`All chunks are OK`, LOG_LEVEL.NOTICE);
}
const payload = {} as Record<string, string[]>;
for (const [id, rev] of unreferenced) {
payload[id] = [rev];
}
const removeItems = Object.keys(payload).length;
if (removeItems == 0) {
Logger(`No unreferenced chunks found (Local)`, LOG_LEVEL.NOTICE);
await plugin.markRemoteResolved();
}
if (dryRun) {
Logger(`There are ${removeItems} unreferenced chunks (Local)`, LOG_LEVEL.NOTICE);
return;
}
Logger(`Deleting unreferenced chunks: ${removeItems}`, LOG_LEVEL.NOTICE, "clean-up-db");
for (const [id, rev] of unreferenced) {
//@ts-ignore
const ret = await db.purge(id, rev);
Logger(ret, LOG_LEVEL.VERBOSE);
}
plugin.localDatabase.refreshSettings();
Logger(`Compacting local database...`, LOG_LEVEL.NOTICE, "clean-up-db");
await db.compact();
await plugin.markRemoteResolved();
Logger("Done!", LOG_LEVEL.NOTICE, "clean-up-db");
})
}
export const balanceChunks = async (plugin: ObsidianLiveSyncPlugin, dryRun: boolean) => {
await runWithLock("clean-up:balance", true, async () => {
const localDB = plugin.localDatabase.localDatabase;
Logger(`Gathering chunk usage information`, LOG_LEVEL.NOTICE, "clean-up-db");
const ret = await plugin.replicator.connectRemoteCouchDBWithSetting(plugin.settings, plugin.isMobile);
if (typeof ret === "string") {
Logger(`Connect error: ${ret}`, LOG_LEVEL.NOTICE, "clean-up-db");
return;
}
const localChunks = new Map<string, string>();
const xx = await localDB.allDocs({ startkey: "h:", endkey: `h:\u{10ffff}` });
for (const xxd of xx.rows) {
const chunk = xxd.id
localChunks.set(chunk, xxd.value.rev);
}
// const info = ret.info;
const remoteDB = ret.db;
const remoteChunks = new Map<string, string>();
const xxr = await remoteDB.allDocs({ startkey: "h:", endkey: `h:\u{10ffff}` });
for (const xxd of xxr.rows) {
const chunk = xxd.id
remoteChunks.set(chunk, xxd.value.rev);
}
const localToRemote = new Map<string, string>([...localChunks]);
const remoteToLocal = new Map<string, string>([...remoteChunks]);
for (const id of new Set([...localChunks.keys(), ...remoteChunks.keys()])) {
if (remoteChunks.has(id)) {
localToRemote.delete(id);
}
if (localChunks.has(id)) {
remoteToLocal.delete(id);
}
}
function arrayToChunkedArray<T>(src: T[], size = 25) {
const ret = [] as T[][];
let i = 0;
while (i < src.length) {
ret.push(src.slice(i, i += size));
}
return ret;
}
if (localToRemote.size == 0) {
Logger(`No chunks need to be sent`, LOG_LEVEL.NOTICE);
} else {
Logger(`${localToRemote.size} chunks need to be sent`, LOG_LEVEL.NOTICE);
if (!dryRun) {
const w = arrayToChunkedArray([...localToRemote]);
for (const chunk of w) {
for (const [id,] of chunk) {
const queryRet = await localDB.allDocs({ keys: [id], include_docs: true });
const docs = queryRet.rows.filter(e => !("error" in e)).map(x => x.doc);
const ret = await remoteDB.bulkDocs(docs, { new_edits: false });
Logger(ret, LOG_LEVEL.VERBOSE);
}
}
Logger(`Done! ${remoteToLocal.size} chunks have been sent`, LOG_LEVEL.NOTICE);
}
}
if (remoteToLocal.size == 0) {
Logger(`No chunks need to be retrieved`, LOG_LEVEL.NOTICE);
} else {
Logger(`${remoteToLocal.size} chunks need to be retrieved`, LOG_LEVEL.NOTICE);
if (!dryRun) {
const w = arrayToChunkedArray([...remoteToLocal]);
for (const chunk of w) {
for (const [id,] of chunk) {
const queryRet = await remoteDB.allDocs({ keys: [id], include_docs: true });
const docs = queryRet.rows.filter(e => !("error" in e)).map(x => x.doc);
const ret = await localDB.bulkDocs(docs, { new_edits: false });
Logger(ret, LOG_LEVEL.VERBOSE);
}
}
Logger(`Done! ${remoteToLocal.size} chunks have been retrieved`, LOG_LEVEL.NOTICE);
}
}
})
}
export const remoteDatabaseCleanup = async (plugin: ObsidianLiveSyncPlugin, dryRun: boolean) => {
const getSize = function (info: PouchDB.Core.DatabaseInfo, key: "active" | "external" | "file") {
return Number.parseInt((info as any)?.sizes?.[key] ?? 0);
}
await runWithLock("clean-up:remote", true, async () => {
const CHUNK_SIZE = 100;
function makeChunkedArrayFromArray<T>(items: T[]): T[][] {
const chunked = [];
for (let i = 0; i < items.length; i += CHUNK_SIZE) {
chunked.push(items.slice(i, i + CHUNK_SIZE));
}
return chunked;
}
try {
const ret = await plugin.replicator.connectRemoteCouchDBWithSetting(plugin.settings, plugin.isMobile);
if (typeof ret === "string") {
Logger(`Connect error: ${ret}`, LOG_LEVEL.NOTICE, "clean-up-db");
return;
}
const info = ret.info;
Logger(JSON.stringify(info), LOG_LEVEL.VERBOSE, "clean-up-db");
Logger(`Database active-size: ${sizeToHumanReadable(getSize(info, "active"))}, external-size:${sizeToHumanReadable(getSize(info, "external"))}, file-size: ${sizeToHumanReadable(getSize(info, "file"))}`, LOG_LEVEL.NOTICE);
if (!dryRun) {
Logger(`The remote database has been locked for garbage collection`, LOG_LEVEL.NOTICE, "clean-up-db");
await plugin.markRemoteLocked(true);
}
Logger(`Gathering chunk usage information`, LOG_LEVEL.NOTICE, "clean-up-db");
const db = ret.db;
const { unreferenced, missing } = await gatherChunkUsage(db);
if (missing.size != 0) {
Logger(`Some chunks are not found! We have to rescue`, LOG_LEVEL.NOTICE);
Logger(missing, LOG_LEVEL.VERBOSE);
} else {
Logger(`All chunks are OK`, LOG_LEVEL.NOTICE);
}
const payload = {} as Record<string, string[]>;
for (const [id, rev] of unreferenced) {
payload[id] = [rev];
}
const removeItems = Object.keys(payload).length;
if (removeItems == 0) {
Logger(`No unreferenced chunk found (Remote)`, LOG_LEVEL.NOTICE);
return;
}
if (dryRun) {
Logger(`There are ${removeItems} unreferenced chunks (Remote)`, LOG_LEVEL.NOTICE);
return;
}
Logger(`Deleting unreferenced chunks: ${removeItems}`, LOG_LEVEL.NOTICE, "clean-up-db");
const buffer = makeChunkedArrayFromArray(Object.entries(payload));
for (const chunkedPayload of buffer) {
const rets = await _requestToCouchDBFetch(
`${plugin.settings.couchDB_URI}/${plugin.settings.couchDB_DBNAME}`,
plugin.settings.couchDB_USER,
plugin.settings.couchDB_PASSWORD,
"_purge",
chunkedPayload.reduce((p, c) => ({ ...p, [c[0]]: c[1] }), {}), "POST");
// const result = await rets();
Logger(JSON.stringify(await rets.json()), LOG_LEVEL.VERBOSE);
}
Logger(`Compacting database...`, LOG_LEVEL.NOTICE, "clean-up-db");
await db.compact();
const endInfo = await db.info();
Logger(`Processed database active-size: ${sizeToHumanReadable(getSize(endInfo, "active"))}, external-size:${sizeToHumanReadable(getSize(endInfo, "external"))}, file-size: ${sizeToHumanReadable(getSize(endInfo, "file"))}`, LOG_LEVEL.NOTICE);
Logger(`Reduced sizes: active-size: ${sizeToHumanReadable(getSize(info, "active") - getSize(endInfo, "active"))}, external-size:${sizeToHumanReadable(getSize(info, "external") - getSize(endInfo, "external"))}, file-size: ${sizeToHumanReadable(getSize(info, "file") - getSize(endInfo, "file"))}`, LOG_LEVEL.NOTICE);
Logger(JSON.stringify(endInfo), LOG_LEVEL.VERBOSE, "clean-up-db");
Logger(`Local database cleaning up...`);
await localDatabaseCleanUp(plugin, true, false);
} catch (ex) {
Logger("Failed to clean up db.")
Logger(ex, LOG_LEVEL.VERBOSE);
}
});
}

10
styles.css
View File

@@ -251,4 +251,12 @@ div.sls-setting-menu-btn {
.sls-item-dirty::before { .sls-item-dirty::before {
content: "✏"; content: "✏";
} }
.sls-setting-hidden {
display: none;
}
.password-input > .setting-item-control >input {
-webkit-text-security: disc;
}

6
tsconfig.json
View File

@@ -1,4 +1,6 @@
{ {
"extends": "@tsconfig/svelte/tsconfig.json",
"inlineSourceMap": true,
"compilerOptions": { "compilerOptions": {
"baseUrl": ".", "baseUrl": ".",
"module": "ESNext", "module": "ESNext",
@@ -6,6 +8,10 @@
"allowJs": true, "allowJs": true,
"noImplicitAny": true, "noImplicitAny": true,
"moduleResolution": "node", "moduleResolution": "node",
"types": [
"svelte",
"node"
],
// "importsNotUsedAsValues": "error", // "importsNotUsedAsValues": "error",
"importHelpers": false, "importHelpers": false,
"alwaysStrict": true, "alwaysStrict": true,

115
updates.md
View File

@@ -1,56 +1,79 @@
### 0.18.0 ### 0.19.0
#### Now, paths of files in the database can now be obfuscated. (Experimental Feature) #### Customization sync
At before v0.18.0, Self-hosted LiveSync used the path of files, to detect and resolve conflicts. In naive. The ID of the document stored in the CouchDB was naturally the filename.
However, it means a sort of lacking confidentiality. If the credentials of the database have been leaked, the attacker (or an innocent bystander) can read the path of files. So we could not use confidential things in the filename in some environments.
Since v0.18.0, they can be obfuscated. so it is no longer possible to decipher the path from the ID. Instead of that, it costs a bit CPU load than before, and the data structure has been changed a bit.
We can configure the `Path Obfuscation` in the `Remote database configuration` pane. Since `Plugin and their settings` have been broken, so I tried to fix it, not just fix it, but fix it the way it should be.
Note: **When changing this configuration, we need to rebuild both of the local and the remote databases**.
#### Minors Now, we have `Customization sync`.
- 0.18.1
- Fixed:
- Some messages are fixed (Typo)
- File type detection now works fine!
- 0.18.2
- Improved:
- The setting pane has been refined.
- We can enable `hidden files sync` with several initial behaviours; `Merge`, `Fetch` remote, and `Overwrite` remote.
- No longer `Touch hidden files`.
- 0.18.3
- Fixed Pop-up is now correctly shown after hidden file synchronisation.
### 0.17.0 It is a real shame that the compatibility between these features has been broken. However, this new feature is surely useful and I believe that worth getting over the pain.
- 0.17.0 has no surfaced changes but the design of saving chunks has been changed. They have compatibility but changing files after upgrading makes different chunks than before 0.16.x. We can use the new feature with the same configuration. Only the menu on the command palette has been changed. The dialog can be opened by `Show customization sync dialog`.
Please rebuild databases once if you have been worried about storage usage.
- Improved: I hope you will give it a try.
- Splitting markdown
- Saving chunks
- Changed:
- Chunk ID numbering rules
#### Minors #### Minors
- __0.17.1 to 0.17.30 has been moved into `update_old.md`__
- 0.17.31 - 0.19.1
- Fixed: Fixed hidden file handling on Linux
- Improved: Now customization sync works more smoothly.
- 0.19.2
- Fixed: - Fixed:
- Now `redflag3` can be run surely. - Fixed garbage collection error while unreferenced chunks exist many.
- Synchronisation can now be aborted. - Fixed filename validation on Linux.
- Note: The synchronisation flow has been rewritten drastically. Please do not haste to inform me if you have noticed anything. - Improved:
- 0.17.32 - Showing status is now thinned for performance.
- Enhance caching while collecting chunks.
- 0.19.3
- Improved:
- Now replication will be paced by collecting chunks. If synchronisation has been deadlocked, please enable `Do not pace synchronization` once.
- 0.19.4
- Improved:
- Reduced remote database checking to improve speed and reduce bandwidth.
- Fixed: - Fixed:
- Now periodic internal file scanning works well. - Chunks which previously misinterpreted are now interpreted correctly.
- The handler of Window-visibility-changed has been fixed. - No more missing chunks which not be found forever, except if it has been actually missing.
- And minor fixes possibly included. - Deleted file detection on hidden file synchronising now works fine.
- Refactored: - Now the Customisation sync is surely quiet while it has been disabled.
- Unused logic has been removed. - 0.19.5
- Some utility functions have been moved into suitable files. - Fixed:
- Function names have been renamed. - Now hidden file synchronisation would not be hanged, even if so many files exist.
- 0.17.33 - Improved:
- Maintenance update: Refactored; the responsibilities that `LocalDatabase` had were shared. (Hoping) No changes in behaviour. - Customisation sync works more smoothly.
- 0.17.34 - Note: Concurrent processing has been rollbacked into the original implementation. As a result, the total number of processes is no longer shown next to the hourglass icon. However, only the processes that are running concurrently are shown.
- Fixed: The `Fetch` that was broken at 0.17.33 has been fixed. - 0.19.6
- Refactored again: Internal file sync, plug-in sync and Set up URI have been moved into each file. - Fixed:
... To continue on to `updates_old.md`. - Logging has been tweaked.
- No more too many planes and rockets.
- The batch database update now surely only works in non-live mode.
- Internal things:
- Some frameworks has been upgraded.
- Import declaration has been fixed.
- Improved:
- The plug-in now asks to enable a new adaptor, when rebuilding, if it is not enabled yet.
- The setting dialogue refined.
- Configurations for compatibilities have been moved under the hatch.
- Made it clear that disabled is the default.
- Ambiguous names configuration have been renamed.
- Items that have no meaning in the settings are no longer displayed.
- Some items have been reordered for clarity.
- Each configuration has been grouped.
- 0.19.7
- Fixed:
- The initial pane of Setting dialogue is now changed to General Settings.
- The Setup Wizard is now able to flush existing settings and get into the mode again.
- 0.19.8
- New feature:
- Vault history: A tab has been implemented to give a birds-eye view of the changes that have occurred in the vault.
- Improved:
- Now the passphrases on the dialogue masked out. Thank you @antoKeinanen!
- Log dialogue is now shown as one of tabs.
- Fixed:
- Some minor issues has been fixed.
- 0.19.9
- New feature (For fixing a problem):
- We can fix the database obfuscated and plain paths that have been mixed up.
- Improvements
- Customisation Sync performance has been improved.
... To continue on to `updates_old.md`.

63
updates_old.md
View File

@@ -1,3 +1,46 @@
### 0.18.0
#### Now, paths of files in the database can now be obfuscated. (Experimental Feature)
At before v0.18.0, Self-hosted LiveSync used the path of files, to detect and resolve conflicts. In naive. The ID of the document stored in the CouchDB was naturally the filename.
However, it means a sort of lacking confidentiality. If the credentials of the database have been leaked, the attacker (or an innocent bystander) can read the path of files. So we could not use confidential things in the filename in some environments.
Since v0.18.0, they can be obfuscated. so it is no longer possible to decipher the path from the ID. Instead of that, it costs a bit CPU load than before, and the data structure has been changed a bit.
We can configure the `Path Obfuscation` in the `Remote database configuration` pane.
Note: **When changing this configuration, we need to rebuild both of the local and the remote databases**.
#### Minors
- 0.18.1
- Fixed:
- Some messages are fixed (Typo)
- File type detection now works fine!
- 0.18.2
- Improved:
- The setting pane has been refined.
- We can enable `hidden files sync` with several initial behaviours; `Merge`, `Fetch` remote, and `Overwrite` remote.
- No longer `Touch hidden files`.
- 0.18.3
- Fixed Pop-up is now correctly shown after hidden file synchronisation.
- 0.18.4
- Fixed:
- `Fetch` and `Rebuild database` will work more safely.
- Case-sensitive renaming now works fine.
Revoked the logic which was made at #130, however, looks fine now.
- 0.18.5
- Improved:
- Actions for maintaining databases moved to the `🎛️Maintain databases`.
- Clean-up of unreferenced chunks has been implemented on an **experimental**.
- This feature requires enabling `Use new adapter`.
- Be sure to fully all devices synchronised before perform it.
- After cleaning up the remote, all devices will be locked out. If we are sure had it be synchronised, we can perform only cleaning-up locally. If not, we have to perform `Fetch`.
- 0.18.6
- New features:
- Now remote database cleaning-up will be detected automatically.
- A solution selection dialogue will be shown if synchronisation is rejected after cleaning or rebuilding the remote database.
- During fetching or rebuilding, we can configure `Hidden file synchronisation` on the spot.
- It let us free from conflict resolution on initial synchronising.
### 0.17.0 ### 0.17.0
- 0.17.0 has no surfaced changes but the design of saving chunks has been changed. They have compatibility but changing files after upgrading makes different chunks than before 0.16.x. - 0.17.0 has no surfaced changes but the design of saving chunks has been changed. They have compatibility but changing files after upgrading makes different chunks than before 0.16.x.
Please rebuild databases once if you have been worried about storage usage. Please rebuild databases once if you have been worried about storage usage.
@@ -155,6 +198,26 @@
- Fixed a problem about reading chunks online when a file has more chunks than the concurrency limit. - Fixed a problem about reading chunks online when a file has more chunks than the concurrency limit.
- Rollbacked: - Rollbacked:
- Logs are kept only for 100 lines, again. - Logs are kept only for 100 lines, again.
- 0.17.31
- Fixed:
- Now `redflag3` can be run surely.
- Synchronisation can now be aborted.
- Note: The synchronisation flow has been rewritten drastically. Please do not haste to inform me if you have noticed anything.
- 0.17.32
- Fixed:
- Now periodic internal file scanning works well.
- The handler of Window-visibility-changed has been fixed.
- And minor fixes possibly included.
- Refactored:
- Unused logic has been removed.
- Some utility functions have been moved into suitable files.
- Function names have been renamed.
- 0.17.33
- Maintenance update: Refactored; the responsibilities that `LocalDatabase` had were shared. (Hoping) No changes in behaviour.
- 0.17.34
- Fixed: The `Fetch` that was broken at 0.17.33 has been fixed.
- Refactored again: Internal file sync, plug-in sync and Set up URI have been moved into each file.
### 0.16.0 ### 0.16.0
- Now hidden files need not be scanned. Changes will be detected automatically. - Now hidden files need not be scanned. Changes will be detected automatically.
- If you want it to back to its previous behaviour, please disable `Monitor changes to internal files`. - If you want it to back to its previous behaviour, please disable `Monitor changes to internal files`.