public/obsidian-livesync
1
0
Fork 0
mirror of https://github.com/vrtmrz/obsidian-livesync.git synced 2026-02-22 20:18:48 +00:00
Code Issues Packages Projects Releases Wiki Activity

Compare commits

base: public:0.11.2
Branches Tags
public:main public:beta public:snyk-upgrade-7fd31e6f474568014502e7205847f257 public:snyk-fix-90f3b71450f7543a1c61724d8afa0e71 public:refactor_services_2 public:refactor_services public:dev public:snyk-upgrade-fe9625385aa4be849c7157033fd2abb0 public:snyk-upgrade-99a31e70c822894595544b0c9813b11a public:svelteui public:disenchant public:snyk-fix-61f76ef0bce2a276df9552c2fc80a14f public:snyk-upgrade-2baa7938ba318464c3bad5c3fd5cbda5 public:snyk-upgrade-7af62a575eab20e8fb4e9565bf1adcfc public:snyk-upgrade-b32f281b31a81483f126b8fb6c54a957 public:snyk-upgrade-bd61b1103442d3faecc9fc726c34e9a4 public:snyk-upgrade-5b2d6584430c061b5187611a5d7fb64a public:snyk-fix-0e9fbbcb374fcb8d5e8cb027d8414fb7 public:snyk-upgrade-5242daf91e706d1ae500c9ea6545636e public:snyk-upgrade-70434a6822fbad83424135dfad9df909 public:snyk-upgrade-8fa01ccad0352125f1866d41e57d35e1 public:snyk-upgrade-39eb97b4655e70091f718a88fccd57c1 public:snyk-upgrade-2b99d54d19e207887e36c3071b796925 public:forprivacy public:snyk-upgrade-dad9088d0a333677e0157e3deef7fc38 public:snyk-upgrade-b9a76d74b3e5723759af5b3319aabb80 public:snyk-upgrade-7da6459d8ff5d293d80de9d80061d912 public:snyk-upgrade-6a3786fc4cbe394542e9d1b6058c46e7 public:snyk-upgrade-8fc9e62771dc41b0f23dc0efefb5c949
public:0.25.43-patched-9 public:0.25.43-patched-8 public:0.25.43-patched-7 public:0.25.43-patched-6 public:0.25.43-patched-5 public:0.25.43-patched-4 public:0.25.43-patched-3 public:0.25.43-patched-2 public:0.25.43-patched-1 public:0.25.43 public:0.25.42 public:0.25.41-patched-3 public:0.25.41-patched-2 public:0.25.41-patched-1 public:0.25.41 public:0.25.40 public:0.25.39 public:0.25.38 public:0.25.37 public:0.25.36 public:0.25.35 public:0.25.34 public:0.25.33 public:0.25.32 public:0.25.31 public:0.25.30 public:0.25.30-dev public:0.25.29 public:0.25.28 public:0.25.27 public:0.25.26 public:0.25.25 public:0.25.24 public:0.25.24.beta3 public:0.25.24.beta2 public:0.25.24.beta1 public:0.25.23 public:0.25.23.beta1 public:0.25.22 public:0.25.21 public:0.25.21.beta2 public:0.25.21.beta1 public:0.25.20 public:0.25.19 public:0.25.18 public:0.25.17 public:0.25.16 public:0.25.15 public:0.25.14 public:0.25.13 public:0.25.12 public:0.25.11 public:0.25.10 public:0.25.9 public:0.25.8 public:0.25.7 public:0.25.6 public:0.25.5 public:0.25.4 public:0.25.3 public:0.25.2 public:0.25.1 public:0.25.0 public:0.25.0-beta1 public:0.24.31 public:0.24.30 public:0.24.29 public:0.24.28 public:0.24.27 public:0.24.26 public:0.24.25 public:0.24.24 public:0.24.23 public:0.24.22 public:0.24.21 public:0.24.20 public:0.24.19 public:0.24.18 public:0.24.17 public:0.24.16 public:0.24.15 public:0.24.14 public:0.24.13 public:0.24.12 public:0.24.11 public:0.24.10 public:0.24.9 public:0.24.8 public:0.24.7 public:0.24.6 public:0.24.5 public:0.24.4 public:0.24.3 public:0.24.2 public:0.24.1 public:0.24.0 public:0.24.0.dev-rc8 public:0.24.0.dev-rc7 public:0.24.0.dev-rc6 public:0.24.0.dev-rc5 public:0.24.0.dev-rc4 public:0.24.0.dev-rc3 public:0.24.0.dev-rc2 public:0.23.23 public:0.23.22 public:0.23.21 public:0.23.20 public:0.23.19 public:0.23.18 public:0.23.17 public:0.23.16 public:0.23.15 public:0.23.14 public:0.23.13 public:0.23.12 public:0.23.11 public:0.23.10 public:0.23.9 public:0.23.8 public:0.23.7 public:0.23.6 public:0.23.5 public:0.23.4 public:0.23.3 public:0.23.2 public:0.23.1 public:0.23.0 public:0.22.19 public:0.22.18 public:0.22.17 public:0.22.16 public:0.22.15 public:0.22.14 public:0.22.13 public:0.22.12 public:0.22.11 public:0.22.10 public:0.22.9 public:0.22.8 public:0.22.7 public:0.22.6 public:0.22.5 public:0.22.4 public:0.22.3 public:0.22.2 public:0.22.1 public:0.22.0 public:0.21.5 public:0.21.4 public:0.21.3 public:0.21.2 public:0.21.1 public:0.21.0 public:0.20.7 public:0.20.6 public:0.20.5 public:0.20.4 public:0.20.3 public:0.20.2 public:0.20.1 public:0.20.0 public:0.19.23 public:0.19.22 public:0.19.21 public:0.19.20 public:0.19.19 public:0.19.18 public:0.19.17 public:0.19.16 public:0.19.15 public:0.19.14 public:0.19.13 public:0.19.12 public:0.19.11 public:0.19.10 public:0.19.9 public:0.19.8 public:0.19.7 public:0.19.6 public:0.19.5 public:0.19.4 public:0.19.3 public:0.19.2 public:0.19.1 public:0.19.0 public:0.18.6 public:0.18.5 public:0.18.4 public:0.18.3 public:0.18.2 public:0.18.1 public:0.18.0 public:0.17.34 public:0.17.33 public:0.17.32 public:0.17.31 public:0.17.30 public:0.17.29 public:0.17.28 public:0.17.27 public:0.17.26 public:0.17.25 public:0.17.24 public:0.17.23 public:0.17.22 public:0.17.21 public:0.17.20 public:0.17.19 public:0.17.18 public:0.17.17 public:0.17.16 public:0.17.15 public:0.17.14 public:0.17.13 public:0.17.12 public:0.17.11 public:0.17.10 public:0.17.9 public:0.17.8 public:0.17.7 public:0.17.6 public:0.17.5 public:0.17.4 public:0.17.3 public:0.17.2 public:0.17.1 public:0.17.0 public:0.16.8 public:0.16.7 public:0.16.6 public:0.16.5 public:0.16.4 public:0.16.3 public:0.16.2 public:0.16.1 public:0.16.0 public:0.15.10 public:0.15.9 public:0.15.8 public:0.15.7 public:0.15.6 public:0.15.5 public:0.15.4 public:0.15.3 public:0.15.2 public:0.15.1 public:0.15.0 public:0.14.8 public:0.14.7 public:0.14.6 public:0.14.5 public:0.14.4 public:0.14.3 public:0.14.2 public:0.14.1 public:0.14.0 public:0.13.4 public:0.13.3 public:0.13.2 public:0.13.1 public:0.13.0 public:0.12.5 public:0.12.4 public:0.12.3 public:0.12.2 public:0.12.1 public:0.12.0 public:0.11.10 public:0.11.9 public:0.11.8 public:0.11.7 public:0.11.6 public:0.11.5 public:0.11.4 public:0.11.3 public:0.11.2 public:0.11.1 public:0.11.0 public:0.10.1 public:0.10.0 public:0.9.0 public:0.8.8 public:0.8.7 public:0.8.6 public:0.8.5 public:0.8.3 public:0.8.2 public:0.8.1 public:0.8.0 public:0.7.2 public:0.7.1 public:0.7.0 public:0.6.1 public:0.6.0 public:0.5.0 public:0.4.1 public:0.4.0 public:0.3.1 public:0.3.0 public:0.2.1 public:0.2.0 public:0.1.26 public:0.1.25 public:0.1.24 public:0.1.23 public:0.1.22 public:0.1.21 public:0.1.20 public:0.1.19 public:0.1.18 public:0.1.17 public:0.1.16 public:0.1.15 public:0.1.14 public:0.1.13 public:0.1.12 public:0.1.11 public:0.1.10 public:0.1.9 public:0.1.8 public:0.1.7 public:0.1.6 public:0.1.5 public:0.1.4 public:0.1.3 public:0.1.2 public:0.1.1 public:0.1.0 public:0.0.9 public:0.0.8 public:0.0.7 public:0.0.6
...
compare: public:0.19.0
Branches Tags
public:beta public:main public:snyk-upgrade-7fd31e6f474568014502e7205847f257 public:snyk-fix-90f3b71450f7543a1c61724d8afa0e71 public:refactor_services_2 public:refactor_services public:dev public:snyk-upgrade-fe9625385aa4be849c7157033fd2abb0 public:snyk-upgrade-99a31e70c822894595544b0c9813b11a public:svelteui public:disenchant public:snyk-fix-61f76ef0bce2a276df9552c2fc80a14f public:snyk-upgrade-2baa7938ba318464c3bad5c3fd5cbda5 public:snyk-upgrade-7af62a575eab20e8fb4e9565bf1adcfc public:snyk-upgrade-b32f281b31a81483f126b8fb6c54a957 public:snyk-upgrade-bd61b1103442d3faecc9fc726c34e9a4 public:snyk-upgrade-5b2d6584430c061b5187611a5d7fb64a public:snyk-fix-0e9fbbcb374fcb8d5e8cb027d8414fb7 public:snyk-upgrade-5242daf91e706d1ae500c9ea6545636e public:snyk-upgrade-70434a6822fbad83424135dfad9df909 public:snyk-upgrade-8fa01ccad0352125f1866d41e57d35e1 public:snyk-upgrade-39eb97b4655e70091f718a88fccd57c1 public:snyk-upgrade-2b99d54d19e207887e36c3071b796925 public:forprivacy public:snyk-upgrade-dad9088d0a333677e0157e3deef7fc38 public:snyk-upgrade-b9a76d74b3e5723759af5b3319aabb80 public:snyk-upgrade-7da6459d8ff5d293d80de9d80061d912 public:snyk-upgrade-6a3786fc4cbe394542e9d1b6058c46e7 public:snyk-upgrade-8fc9e62771dc41b0f23dc0efefb5c949
public:0.25.43-patched-9 public:0.25.43-patched-8 public:0.25.43-patched-7 public:0.25.43-patched-6 public:0.25.43-patched-5 public:0.25.43-patched-4 public:0.25.43-patched-3 public:0.25.43-patched-2 public:0.25.43-patched-1 public:0.25.43 public:0.25.42 public:0.25.41-patched-3 public:0.25.41-patched-2 public:0.25.41-patched-1 public:0.25.41 public:0.25.40 public:0.25.39 public:0.25.38 public:0.25.37 public:0.25.36 public:0.25.35 public:0.25.34 public:0.25.33 public:0.25.32 public:0.25.31 public:0.25.30 public:0.25.30-dev public:0.25.29 public:0.25.28 public:0.25.27 public:0.25.26 public:0.25.25 public:0.25.24 public:0.25.24.beta3 public:0.25.24.beta2 public:0.25.24.beta1 public:0.25.23 public:0.25.23.beta1 public:0.25.22 public:0.25.21 public:0.25.21.beta2 public:0.25.21.beta1 public:0.25.20 public:0.25.19 public:0.25.18 public:0.25.17 public:0.25.16 public:0.25.15 public:0.25.14 public:0.25.13 public:0.25.12 public:0.25.11 public:0.25.10 public:0.25.9 public:0.25.8 public:0.25.7 public:0.25.6 public:0.25.5 public:0.25.4 public:0.25.3 public:0.25.2 public:0.25.1 public:0.25.0 public:0.25.0-beta1 public:0.24.31 public:0.24.30 public:0.24.29 public:0.24.28 public:0.24.27 public:0.24.26 public:0.24.25 public:0.24.24 public:0.24.23 public:0.24.22 public:0.24.21 public:0.24.20 public:0.24.19 public:0.24.18 public:0.24.17 public:0.24.16 public:0.24.15 public:0.24.14 public:0.24.13 public:0.24.12 public:0.24.11 public:0.24.10 public:0.24.9 public:0.24.8 public:0.24.7 public:0.24.6 public:0.24.5 public:0.24.4 public:0.24.3 public:0.24.2 public:0.24.1 public:0.24.0 public:0.24.0.dev-rc8 public:0.24.0.dev-rc7 public:0.24.0.dev-rc6 public:0.24.0.dev-rc5 public:0.24.0.dev-rc4 public:0.24.0.dev-rc3 public:0.24.0.dev-rc2 public:0.23.23 public:0.23.22 public:0.23.21 public:0.23.20 public:0.23.19 public:0.23.18 public:0.23.17 public:0.23.16 public:0.23.15 public:0.23.14 public:0.23.13 public:0.23.12 public:0.23.11 public:0.23.10 public:0.23.9 public:0.23.8 public:0.23.7 public:0.23.6 public:0.23.5 public:0.23.4 public:0.23.3 public:0.23.2 public:0.23.1 public:0.23.0 public:0.22.19 public:0.22.18 public:0.22.17 public:0.22.16 public:0.22.15 public:0.22.14 public:0.22.13 public:0.22.12 public:0.22.11 public:0.22.10 public:0.22.9 public:0.22.8 public:0.22.7 public:0.22.6 public:0.22.5 public:0.22.4 public:0.22.3 public:0.22.2 public:0.22.1 public:0.22.0 public:0.21.5 public:0.21.4 public:0.21.3 public:0.21.2 public:0.21.1 public:0.21.0 public:0.20.7 public:0.20.6 public:0.20.5 public:0.20.4 public:0.20.3 public:0.20.2 public:0.20.1 public:0.20.0 public:0.19.23 public:0.19.22 public:0.19.21 public:0.19.20 public:0.19.19 public:0.19.18 public:0.19.17 public:0.19.16 public:0.19.15 public:0.19.14 public:0.19.13 public:0.19.12 public:0.19.11 public:0.19.10 public:0.19.9 public:0.19.8 public:0.19.7 public:0.19.6 public:0.19.5 public:0.19.4 public:0.19.3 public:0.19.2 public:0.19.1 public:0.19.0 public:0.18.6 public:0.18.5 public:0.18.4 public:0.18.3 public:0.18.2 public:0.18.1 public:0.18.0 public:0.17.34 public:0.17.33 public:0.17.32 public:0.17.31 public:0.17.30 public:0.17.29 public:0.17.28 public:0.17.27 public:0.17.26 public:0.17.25 public:0.17.24 public:0.17.23 public:0.17.22 public:0.17.21 public:0.17.20 public:0.17.19 public:0.17.18 public:0.17.17 public:0.17.16 public:0.17.15 public:0.17.14 public:0.17.13 public:0.17.12 public:0.17.11 public:0.17.10 public:0.17.9 public:0.17.8 public:0.17.7 public:0.17.6 public:0.17.5 public:0.17.4 public:0.17.3 public:0.17.2 public:0.17.1 public:0.17.0 public:0.16.8 public:0.16.7 public:0.16.6 public:0.16.5 public:0.16.4 public:0.16.3 public:0.16.2 public:0.16.1 public:0.16.0 public:0.15.10 public:0.15.9 public:0.15.8 public:0.15.7 public:0.15.6 public:0.15.5 public:0.15.4 public:0.15.3 public:0.15.2 public:0.15.1 public:0.15.0 public:0.14.8 public:0.14.7 public:0.14.6 public:0.14.5 public:0.14.4 public:0.14.3 public:0.14.2 public:0.14.1 public:0.14.0 public:0.13.4 public:0.13.3 public:0.13.2 public:0.13.1 public:0.13.0 public:0.12.5 public:0.12.4 public:0.12.3 public:0.12.2 public:0.12.1 public:0.12.0 public:0.11.10 public:0.11.9 public:0.11.8 public:0.11.7 public:0.11.6 public:0.11.5 public:0.11.4 public:0.11.3 public:0.11.2 public:0.11.1 public:0.11.0 public:0.10.1 public:0.10.0 public:0.9.0 public:0.8.8 public:0.8.7 public:0.8.6 public:0.8.5 public:0.8.3 public:0.8.2 public:0.8.1 public:0.8.0 public:0.7.2 public:0.7.1 public:0.7.0 public:0.6.1 public:0.6.0 public:0.5.0 public:0.4.1 public:0.4.0 public:0.3.1 public:0.3.0 public:0.2.1 public:0.2.0 public:0.1.26 public:0.1.25 public:0.1.24 public:0.1.23 public:0.1.22 public:0.1.21 public:0.1.20 public:0.1.19 public:0.1.18 public:0.1.17 public:0.1.16 public:0.1.15 public:0.1.14 public:0.1.13 public:0.1.12 public:0.1.11 public:0.1.10 public:0.1.9 public:0.1.8 public:0.1.7 public:0.1.6 public:0.1.5 public:0.1.4 public:0.1.3 public:0.1.2 public:0.1.1 public:0.1.0 public:0.0.9 public:0.0.8 public:0.0.7 public:0.0.6

221 Commits
0.11.2 ... 0.19.0

Author SHA1 Message Date
vorotamoroz
cf06f878db bump 2023-04-28 14:24:37 +09:00
vorotamoroz
e77031f1cd Implemented: 
- New feature `Customization sync` has replaced `Plugin and their settings`
 2023-04-28 13:32:58 +09:00
vorotamoroz
3f2224c3a6 Merge pull request #203 from garlic-hub/garlic-hub-patch-1 
Update quick_setup.md
 2023-04-21 17:15:31 +09:00
garlic-hub
2322b5bc34 Update quick_setup.md 2023-04-20 21:30:56 +00:00
vorotamoroz
83ac5e7086 bump 2023-04-14 17:39:37 +09:00
vorotamoroz
09f35a2af4 New features: 
- Now remote database cleaning-up will be detected automatically.
- A solution selection dialogue will be shown if synchronisation is rejected after cleaning or rebuilding the remote database.
- During fetching or rebuilding, we can configure `Hidden file synchronisation` on the spot.
 2023-04-14 17:39:09 +09:00
vorotamoroz
fae0a9d76a bump 2023-04-13 17:33:28 +09:00
vorotamoroz
9a27c9bfe5 - Actions for maintaining databases moved to the 🎛️Maintain databases. 
- Clean-up of unreferenced chunks has been implemented on an **experimental**.
 2023-04-13 17:33:17 +09:00
vorotamoroz
5e75917b8d bump 2023-04-12 12:08:35 +09:00
vorotamoroz
3322d13b55 - Fixed: 
- `Fetch` and `Rebuild database` will work more safely.
- Case-sensitive renaming now works fine.
  Revoked the logic which was made at #130, however, looks fine now.
 2023-04-12 12:08:08 +09:00
vorotamoroz
851c9f8a71 Pop-up is now correctly shown after hidden file synchronisation. 2023-04-11 12:54:20 +09:00
vorotamoroz
b02596dfa1 bump 2023-04-11 12:45:40 +09:00
vorotamoroz
02c69b202e Improved: 
- The setting pane refined.
- We can enable `hidden files sync` with several initial behaviours: `Merge`, `Fetch` remote, and `Overwrite` remote.
- No longer `Touch hidden files`
 2023-04-11 12:45:24 +09:00
vorotamoroz
6b2c7b56a5 add note. 2023-04-10 15:18:32 +09:00
vorotamoroz
820168a5ab bump. 2023-04-10 15:15:20 +09:00
vorotamoroz
40015642e4 Fixed 
- fixed type annotation
- update lib
 2023-04-10 15:14:47 +09:00
vorotamoroz
7a5cffb6a8 bump 2023-04-10 12:07:03 +09:00
vorotamoroz
e395e53248 Implemented: 
- Explicit types
- Path obfuscation.
- ... and minor changes.
 2023-04-10 12:04:30 +09:00
vorotamoroz
97f91b1eb0 Update README.md 2023-03-30 18:23:52 +09:00
vorotamoroz
2f4159182e Update README.md 2023-03-30 18:23:30 +09:00
vorotamoroz
302a4024a8 Update README.md 2023-03-30 18:22:49 +09:00
vorotamoroz
bc17f4f70d bump 2023-03-23 16:48:42 +09:00
vorotamoroz
6f33d23088 - Fixed: The Fetch that was broken at 0.17.33 has been fixed. 
- Refactored again: Internal file sync, plug-in sync and Set up URI have been moved into each file.
 2023-03-23 16:48:30 +09:00
vorotamoroz
4998e2ef0b bump 2023-03-22 15:04:57 +09:00
vorotamoroz
f5e0b826a6 Refactored 
- the responsibilities that `LocalDatabase` had were shared.
 2023-03-22 15:04:26 +09:00
vorotamoroz
3a3f79bb99 bump 2023-03-17 17:50:53 +09:00
vorotamoroz
9efb6ed0c1 Fixed: 
- Now periodic internal file scanning works well.
- The handler of Window-visibility-changed has been fixed.
- And minor fixes possibly included.
Refactored:
- Unused logic has been removed.
- Some utility functions have been moved into suitable files.
- Function names have been renamed.
 2023-03-17 17:48:24 +09:00
vorotamoroz
6b7956ab67 bump 2023-03-14 19:03:28 +09:00
vorotamoroz
58196c2423 Fixed: 
- Now `redflag3` can be run surely.
- Synchronisation can now be aborted.
 2023-03-14 19:02:57 +09:00
vorotamoroz
3940260d42 bump 2023-03-02 12:56:59 +09:00
vorotamoroz
b16333c604 Implemented: 
- `Resolve all conflicted files` has been implemented.
Fixed:
- Fixed a problem about reading chunks online when a file has more chunks than the concurrency limit.
Rollbacked:
- Logs are kept only for 100 lines, again.
 2023-03-02 12:54:41 +09:00
vorotamoroz
7bf6d1f663 update dependencies 2023-03-02 12:51:46 +09:00
vorotamoroz
7046928068 bump 2023-03-01 12:59:48 +09:00
vorotamoroz
333fcbaaeb - Fixed: 
- Requests of reading chunks online are now split into a reasonable(and configurable) size.
    - No longer error message will be shown on Linux devices with hidden file synchronisation.
  - Improved:
    - The interval of reading chunks online is now configurable.
    - Boot sequence has been speeded up, more.
  - Misc:
    - Messages on the boot sequence will now be more detailed. If you want to see them, please enable the verbose log.
    - Logs became be kept for 1000 lines while the verbose log is enabled.
 2023-03-01 12:58:29 +09:00
vorotamoroz
009f92c307 bump 2023-02-28 17:25:46 +09:00
vorotamoroz
3e541bd061 Fixed: 
- Some messages have been refined.
- Boot sequence has been speeded up.
- Opening the local database multiple times in a short duration has been suppressed.
 2023-02-28 17:15:43 +09:00
vorotamoroz
52d08301cc bump 2023-02-27 17:57:37 +09:00
vorotamoroz
49d4c239f2 Improved: 
- Now, the filename of the conflicted settings will be shown on the merging dialogue
- The plugin data can be resolved when conflicted.
- The semaphore status display has been changed to count only.
- Applying to the storage will be concurrent with a few files.
 2023-02-27 17:57:05 +09:00
vorotamoroz
748d031b36 bump 2023-02-21 09:13:19 +09:00
vorotamoroz
dbe77718c8 Urgent: 
- The modified document will be reflected in the storage now.
 2023-02-21 09:12:14 +09:00
vorotamoroz
f334974cc3 bump 2023-02-20 17:58:14 +09:00
vorotamoroz
8f2ae437c6 Fixed: 
- Now reading error will be reported.
 2023-02-20 17:54:57 +09:00
vorotamoroz
a0efda9e71 bump 2023-02-17 17:37:15 +09:00
vorotamoroz
be3d61c1c7 - New feature: 
- If any conflicted files have been left, they will be reported.
- Fixed:
  - Now the name of the conflicting file is shown on the conflict-resolving dialogue.
  - Hidden files are now able to be merged again.
  - No longer error caused at plug-in being loaded.
- Improved:
  - Caching chunks are now limited in total size of cached chunks.
 2023-02-17 17:35:06 +09:00
vorotamoroz
b24c4ef55b bump 2023-02-13 15:17:23 +09:00
vorotamoroz
ff850b48ca - Improved: 
- Now we can preserve the logs into the file.
    - Note: This option will be enabled automatically also when we flagging a red flag.
  - File names can now be made platform-appropriate.
- Refactored:
  - Some redundant implementations have been sorted out.
 2023-02-13 15:17:09 +09:00
vorotamoroz
ad3860ac40 bump 2023-02-09 17:24:57 +09:00
vorotamoroz
437b7ebae1 - Fixed: 
- Now hidden files will not be synchronised while we are not configured.
  - Some processes could start without waiting for synchronisation to complete, but now they will wait for.
- Improved
  - Now, by placing `redflag3.md`, we can discard the local database and fetch again.
 2023-02-09 17:22:12 +09:00
vorotamoroz
e3305c24e1 Merge pull request #170 from hilsonp/main 
add tip : redflag.md can be a directory which is easier to create on iOS
 2023-02-09 15:45:00 +09:00
Pierre Hilson
d008e2a1d0 add tip : redflag.md can be a directory which is easier to create on iOS 2023-02-03 18:58:54 +01:00
vorotamoroz
bb8c7eb043 bump 2023-02-03 18:36:33 +09:00
vorotamoroz
e61bebd3ee - Fixed: Skip patterns now handle capital letters. 
- Improved
  - New configuration to avoid exceeding throttle capacity.
  - The conflicted `data.json` is no longer merged automatically.
 2023-02-03 18:30:40 +09:00
vorotamoroz
99594fe517 Merge pull request #169 from karasevm/feat-timeout 
Allow switching CouchDB long polling from heartbeat to timeout
 2023-02-03 17:41:54 +09:00
Maksim Karasev
972d208af4 Expose useTimeouts option in the settings tab 2023-02-02 02:24:32 +03:00
vorotamoroz
2c36ec497c bump 2023-01-30 19:56:46 +09:00
vorotamoroz
677895547c Ensure that Obsidian will be notified when hidden files have been changed 2023-01-30 19:48:09 +09:00
vorotamoroz
aec0b2986b bump 2023-01-28 21:24:03 +09:00
vorotamoroz
e6025b92d8 Ensure logging. 2023-01-28 21:20:26 +09:00
vorotamoroz
fad9fed5ca bump 2023-01-27 21:47:04 +09:00
vorotamoroz
e46246cd63 Fixed: 
- Fixed lack of error handling.
 2023-01-27 17:49:53 +09:00
vorotamoroz
0f3be19dd7 bump 2023-01-25 22:39:50 +09:00
vorotamoroz
bc568ff479 Fixed: 
- Now we can merge JSON files even if they have entries which cannot be compared.
 2023-01-25 22:37:55 +09:00
vorotamoroz
2fdc7669f3 bump 2023-01-25 20:54:20 +09:00
vorotamoroz
ec8d9785ed - Improved: 
- Plugins and their settings no longer need scanning if changes are monitored.
  - Now synchronising plugins and their settings are performed parallelly and faster.
  - We can place `redflag2.md` to rebuild the database automatically while the boot sequence.
- Experimental:
  - We can use a new adapter on PouchDB. This will make us smoother.
    - Note: Not compatible with the older version.
- Fixed:
  - The default batch size is smaller again.
  - Plugins and their setting can be synchronised again.
  - Hidden files and plugins are correctly scanned while rebuilding.
  - Files with the name started `_` are also being performed conflict-checking.
 2023-01-25 20:53:20 +09:00
vorotamoroz
71a80cacc3 bump again 2023-01-19 19:05:16 +09:00
vorotamoroz
38daeca89f fixed leaked logging 2023-01-19 19:03:57 +09:00
vorotamoroz
7ec64a6a93 bump 2023-01-19 18:55:18 +09:00
vorotamoroz
c5c6deb742 Improved: 
- Confidential information has no longer stored in data.json as is.
- Synchronising progress has been shown in the notification.
- We can commit passphrases with a keyboard.
- Configuration which had not been saved yet is marked now.

Fixed:
- Hidden files have been synchronised again.

And, minor changes have been included.
 2023-01-19 18:50:06 +09:00
vorotamoroz
ef57fbfdda Fixed: 
- Now the filename is shown on the Conflict resolving dialog
- Rename of files has been improved again.
 2023-01-19 13:11:30 +09:00
vorotamoroz
bc158e9f2b bump 2023-01-17 17:46:06 +09:00
vorotamoroz
6513c53c7e Fixed: 
- Document history is now displayed again.

Reorganised:
- Many files have been refactored.
 2023-01-17 17:39:26 +09:00
vorotamoroz
5d1074065c bump 2023-01-16 17:33:31 +09:00
vorotamoroz
b444082b0c Fixed: 
- Performance improvement
- Now `Chunk size` can be set to under one hundred.

New feature:
- The number of transfers required before replication stabilises is now displayed.
 2023-01-16 17:31:37 +09:00
vorotamoroz
d5e6419504 bump 2023-01-15 11:17:08 +09:00
vorotamoroz
1bf1e1540d Fix diff check. 2023-01-15 11:09:23 +09:00
vorotamoroz
be1e6b11ac Fixed 
- Large files addressed.
 2023-01-13 19:43:39 +09:00
vorotamoroz
a486788572 bump 2023-01-06 16:33:54 +09:00
vorotamoroz
e5784a1da6 Fixed: 
- Conflict merge of internal files is no longer broken.
Improved:
- Smoother status display inside the editor.
 2023-01-06 16:27:39 +09:00
vorotamoroz
2100e22276 bump 2022-12-27 18:12:00 +09:00
vorotamoroz
ec08dc5fe8 Improved: 
- Performance improved
  Prebuilt PouchDB is no longer used.

Fixed:
- Merging hidden files is also fixed.

New Feature:
- Now we can synchronise automatically after merging conflicts.
 2022-12-27 18:09:51 +09:00
vorotamoroz
c92e94e552 fix eslint configuration 2022-12-27 17:58:42 +09:00
vorotamoroz
c7db8592c6 bump 2022-12-26 16:16:46 +09:00
vorotamoroz
fc3617d9f9 Fixed: 
- Fixed merging issues.
- Fixed button styling.

Changed:
- Default behaviour of conflict checking on synchronising.
 2022-12-26 16:12:57 +09:00
vorotamoroz
34c1b040db bump 2022-12-24 21:05:32 +09:00
vorotamoroz
6b85aecafe Fixed: 
- Now our renamed/deleted files have been surely deleted again.
 2022-12-24 21:05:09 +09:00
vorotamoroz
4dabadd5ea A little better. 2022-12-22 18:02:12 +09:00
vorotamoroz
0619c96c48 bump 2022-12-22 17:59:30 +09:00
vorotamoroz
b0f612b61c New feature: 
- Use dynamic iteration count
Fixed:
- Read chunks online will fetch the remote chunks correctly.
- Read chunks online will save fetched chunks to the local database.
 2022-12-22 17:58:52 +09:00
vorotamoroz
81caad8602 Canvases are now treated as a sort of plain text file. 2022-12-21 15:09:10 +09:00
vorotamoroz
f5e28b5e1c bump 2022-12-21 14:51:59 +09:00
vorotamoroz
0c206226b1 New feature 
- JSON merging for data and canvas.
 2022-12-21 14:51:39 +09:00
vorotamoroz
1ad5dcc1cc bump 2022-12-16 18:56:28 +09:00
vorotamoroz
a512566e5b New feature 
- We can merge conflicted documents automatically if sensible.

Fixed:
- Writing to the storage will be pended while they have conflicts after replication.

Minor changes included.
 2022-12-16 18:55:04 +09:00
vorotamoroz
02de82af46 bump 2022-12-06 18:03:31 +09:00
vorotamoroz
840e03a2d3 Fixed: 
- Now we can verify and repair database again.
 2022-12-06 17:59:48 +09:00
vorotamoroz
96b676caf3 bump 2022-12-05 19:53:24 +09:00
vorotamoroz
a8219de375 Improved: 
- Splitting markdown
- Saving chunks

Changed:
- Chunk ID numbering rules

Fixed:
- Just weed.
 2022-12-05 19:37:24 +09:00
vorotamoroz
db3eb7e1a0 bump 2022-11-24 14:14:27 +09:00
vorotamoroz
50f51393fc upgrade lib. 2022-11-24 14:14:17 +09:00
vorotamoroz
8a04e332d6 Fix check warning for max_document_size, max_http_request_size as like as #145 2022-11-23 15:41:09 +09:00
vorotamoroz
12ae17aa2f Merge pull request #145 from Bpazy/patch-1 
Fix check warning for max_document_size, max_http_request_size
 2022-11-23 15:37:32 +09:00
Ziyuan Han
657f12f966 Fix check warning 2022-11-23 14:12:50 +08:00
Ziyuan Han
15a7bed448 Fix check warning 2022-11-23 14:11:44 +08:00
vorotamoroz
420c3b94df bump 2022-11-23 10:34:04 +09:00
vorotamoroz
239c087132 framework and dependency upgraded. 2022-11-23 10:27:12 +09:00
vorotamoroz
d1a633c799 bump 2022-11-07 17:26:40 +09:00
vorotamoroz
1c07cd92fc - Fixed 
- Automatic (temporary) batch size adjustment has been restored to work correctly.
  - Chunk splitting has been backed to the previous behaviour for saving them correctly.
- Improved
  - Corrupted chunks will be detected automatically.
  - Now on the case-insensitive system, `aaa.md` and `AAA.md` will be treated as the same file or path at applying changesets.
 2022-11-07 17:26:33 +09:00
vorotamoroz
adc84d53b1 bump again 2022-10-27 17:43:00 +09:00
vorotamoroz
c3a762ceed bump 2022-10-27 17:42:15 +09:00
vorotamoroz
5945638633 Fixed: 
- Conflict detection and merging of deleted files.
- Fixed wrong logs.
- Fix redundant logs.

Implemented
- Automatically deletion of old metadata.
 2022-10-27 17:41:26 +09:00
vorotamoroz
331acd463d bump 2022-10-25 11:48:21 +09:00
vorotamoroz
9d4f41bbf9 Fixed failure of detection 2022-10-25 11:47:02 +09:00
vorotamoroz
8831165965 bump 2022-10-25 11:34:09 +09:00
vorotamoroz
ed62e9331b Implemented: 
- A configuration information reporting tools has been implemented.

Improved:
- Fixed detection of IBM Cloudant
 2022-10-25 11:33:37 +09:00
vorotamoroz
799e604eb2 bump 2022-10-21 18:22:02 +09:00
vorotamoroz
d9b69d9a1b Fixed: 
- Fixed the Infinity loop
 2022-10-21 18:20:03 +09:00
vorotamoroz
c18b5c24b4 bump 2022-10-14 17:39:16 +09:00
vorotamoroz
07f16e3d7d Added missing log updates. 2022-10-14 17:37:25 +09:00
vorotamoroz
486f1aa4a0 Bump 2022-10-05 17:14:52 +09:00
vorotamoroz
075c6beb68 New feature: 
- Monitor hidden files, Now we can use internal file sync without scan.
Fixed:
- Periodic synchronisation sometimes failed.
- Status-display had not been cleared in some cases.
- `Skip patterns default` has been changed to more clear name.
 2022-10-05 17:14:32 +09:00
vorotamoroz
d6121b0c1e bump 2022-10-03 10:57:39 +09:00
vorotamoroz
3292a48054 Fixed 
- The boot sequence has been corrected and now boots smoothly.
- Auto applying of batch save will be processed earlier than before.
 2022-10-03 10:52:31 +09:00
vorotamoroz
ee37764040 bump 2022-10-02 01:48:50 +09:00
vorotamoroz
b6f7fced22 Use new library for batching the chunk retrieving 2022-10-02 01:45:44 +09:00
vorotamoroz
13456c0854 Fixed: deleted debug message 2022-10-02 01:44:39 +09:00
vorotamoroz
2663a52fd7 bump 2022-09-29 16:58:53 +09:00
vorotamoroz
d4bbf79514 Fixed: 
- Fixed a bug about deleting empty directory
- Weird behaviour on boot-sequence on mobile devices.
 2022-09-29 16:58:39 +09:00
vorotamoroz
5f96cc6b82 bump 2022-09-28 17:57:23 +09:00
vorotamoroz
8c8f5d045f Fixed: 
- Fixed bug about renaming file
 2022-09-28 17:56:34 +09:00
vorotamoroz
40cf8be890 Bump 2022-09-28 16:17:11 +09:00
vorotamoroz
6b03dbbe75 Fixed: 
- File tracking logic has been refined.
 2022-09-28 16:17:04 +09:00
vorotamoroz
74425f75d2 bump 2022-09-27 17:59:05 +09:00
vorotamoroz
ac7c622466 Fixed docs. 2022-09-27 17:58:31 +09:00
vorotamoroz
4b32365694 Implemented: 
- Add new features for setting Self-hosted LiveSync up more easier.
 2022-09-27 17:58:13 +09:00
vorotamoroz
728edac283 Merge pull request #114 from JEndler/main 
Fixed Docker command in docs.
 2022-09-15 17:45:38 +09:00
Jakob Endler
ab9c0190bb Fixed Docker command in docs. 2022-09-12 18:36:50 +02:00
vorotamoroz
5a7610d411 bump 2022-09-12 11:16:41 +09:00
vorotamoroz
4691ae1463 Fixed: 
- Now we can detect hidden files changes and morethings again.
 2022-09-12 11:03:28 +09:00
vorotamoroz
0923ac3d85 Bump 2022-09-11 14:22:24 +09:00
vorotamoroz
ca100d6d9d Fixed: 
- Fixed the issue about lock/unlock remote database while rebuilding in wizard
 2022-09-11 14:21:02 +09:00
vorotamoroz
bc373d4359 Bump 2022-09-11 10:38:38 +09:00
vorotamoroz
4038b683fe Documentated! 2022-09-11 10:38:20 +09:00
vorotamoroz
5e7b44d35a Wizard behaviour has been improved 2022-09-11 10:37:46 +09:00
vorotamoroz
d04be6813b bump again. 2022-09-11 03:02:44 +09:00
vorotamoroz
8e578e2100 missed the stylesheet 2022-09-11 03:02:34 +09:00
vorotamoroz
55fcdfe18f Bump 2022-09-11 02:56:41 +09:00
vorotamoroz
66f2fea2f4 Remove outdated configuration items. 2022-09-11 02:51:17 +09:00
vorotamoroz
beb7bf6fb9 Implemented 
- Configuration wizard.
Fixed
- Remove outdated configuration items.
 2022-09-11 02:50:51 +09:00
vorotamoroz
34791114e5 Fixed 
- Fix file extension
- Remove GC.
- Remove obsoluted methods.
 2022-09-11 02:46:29 +09:00
vorotamoroz
de5cdf507d bump 2022-09-06 17:13:53 +09:00
vorotamoroz
83209f3923 Refactored: 
- PouchDB handling moved into Common lib.
 2022-09-06 17:11:43 +09:00
vorotamoroz
b14ecdb205 Bump 2022-09-06 17:02:58 +09:00
vorotamoroz
21362adb5b Typos 2022-09-06 14:32:09 +09:00
vorotamoroz
f8c1474700 Refactored 2022-09-06 13:42:12 +09:00
vorotamoroz
b35052a485 bump 2022-09-05 16:55:35 +09:00
vorotamoroz
c367d35e09 Target ES2018 2022-09-05 16:55:29 +09:00
vorotamoroz
2a5078cdbb bump 2022-09-05 16:54:06 +09:00
vorotamoroz
8112a07210 Implemented: 
- Auto chunk size adjusting.
  Now our large files are processed more efficiently
- These configuration has been removed.

Improved
- Remote chunk retrieving logic has been speeded up.

Fixed
- Fixed process handling of boot sequence
 2022-09-05 16:53:22 +09:00
vorotamoroz
c9daa1b47d Fixed issue of importing configurations. 2022-09-04 01:16:29 +09:00
vorotamoroz
73ac93e8c5 bump 2022-09-04 01:08:09 +09:00
vorotamoroz
8d2b9eff37 Improved: 
- New test items have been added to `Check database configuration`
 2022-09-04 01:08:02 +09:00
vorotamoroz
0ee32a2147 bump 2022-09-03 16:44:51 +09:00
vorotamoroz
ac3c78e198 Fixed 
- Could not retrieve files if synchronisation has been interrupted or failed
 2022-09-03 16:43:59 +09:00
vorotamoroz
0da1e3d9c8 bump 2022-08-30 15:24:38 +09:00
vorotamoroz
8f021a3c93 Improved: 
- Use local chunks in preference to remote them if present.
 2022-08-30 15:24:26 +09:00
vorotamoroz
6db0743096 Update release.yml 2022-08-29 16:50:53 +09:00
vorotamoroz
0e300a0a6b bump 2022-08-29 16:48:35 +09:00
vorotamoroz
9d0ffd1848 Implemented: 
- The target selecting filter was implemented.
- We can configure size of chunks.
- Read chunks online.

Fixed:
- Typos
 2022-08-29 16:32:14 +09:00
vorotamoroz
e7f4d8c9c2 Add error handling for loading the document 2022-08-29 16:13:54 +09:00
vorotamoroz
ca36e1b663 Merge branch 'main' of https://github.com/vrtmrz/obsidian-livesync 2022-08-09 17:10:19 +09:00
vorotamoroz
8f583e3680 Fixed: 
- Now, we can synchronise hidden files that conflicted on each devices.
Enhanced:
- We can search for conflicting docs.
- Pending processes can now be run at any time.
- Performance improved on synchronising large numbers of files at once.
 2022-08-09 17:10:08 +09:00
vorotamoroz
98407cf72f Merge pull request #101 from Miigon/chinese-doc 
Add Chinese documentations
 2022-08-09 17:06:39 +09:00
Miigon
1f377cdf67 add setup_own_server_cn.md 2022-08-09 13:06:35 +08:00
Miigon
3a965e74da add chinese README_cn.md 2022-08-09 12:38:43 +08:00
vorotamoroz
640c2a91f7 bumped 2022-08-08 09:57:53 +09:00
vorotamoroz
df9f9d9bc0 fixed the ambiguous wording and some minor changes 2022-08-08 09:53:17 +09:00
vorotamoroz
787bf37906 Merge pull request #98 from Miigon/docfix 
improved grammar & fixed typos in docs and plugin texts
 2022-08-08 09:30:18 +09:00
Miigon
63abeb7f6b improved grammar & fixed typos in docs and plugin texts 2022-08-04 14:34:45 +08:00
vorotamoroz
f0ffb0620e Fixed file deletion failures. 2022-07-29 13:33:33 +09:00
vorotamoroz
c88c939cd9 bumped 2022-07-29 09:42:22 +09:00
vorotamoroz
05b53eb2cf Fixes: 
- Now conflict resolution back well
 2022-07-29 09:38:23 +09:00
vorotamoroz
61b65b0461 Bumped 2022-07-28 19:20:59 +09:00
vorotamoroz
ac9be937b4 New feature: 
- The metadata of the deleted files will be kept on the database by default.
- We can see the history of deleted files.
- Update information became to be shown on the major upgrade.

Fixed:
- `Pick file to show` was renamed to `Pick a file to show.
- Files in the `Pick a file to show` are now ordered by their modified date descent.
 2022-07-28 19:19:37 +09:00
vorotamoroz
c610284cab bumped again 2022-07-25 13:50:42 +09:00
vorotamoroz
2e6ed4777c Added postcss for ci 2022-07-25 13:22:37 +09:00
vorotamoroz
ab6ff01f1a Bumped 2022-07-25 13:16:49 +09:00
vorotamoroz
c836953fa9 Fixed: 
- Leaked prototyping code.

    New feature:
    - `Pick file to show history` shows a file list and you can choose one for showing history.
    - `Back to this revision` implemented on the Document history dialog.
 2022-07-25 13:14:59 +09:00
vorotamoroz
e69371ff24 Degraded! 2022-07-25 13:05:00 +09:00
vorotamoroz
d324add086 Fixed: 
- Leaked prototyping code.

New feature:
- `Pick file to show history` shows a file list and you can choose one for showing history.
- `Back to this revision` implemented on the Document history dialog.
 2022-07-25 12:48:48 +09:00
vorotamoroz
0caf330f39 Merge remote-tracking branch 'origin/snyk-upgrade-52349e72cdcadcd0ba41dd967c715aa7' 2022-07-21 16:40:18 +09:00
vorotamoroz
3a147ca427 Merge branch 'snyk-upgrade-6a3786fc4cbe394542e9d1b6058c46e7' 2022-07-21 16:26:54 +09:00
vorotamoroz
8266cfba40 Merge remote-tracking branch 'origin/snyk-upgrade-d6d88a887d6bc2d451a6e9c7a7759727' 2022-07-21 16:15:35 +09:00
vorotamoroz
e2f06181fa bumped 2022-07-21 13:06:57 +09:00
vorotamoroz
bb6d787607 Imprinting version numbers to boot log. 2022-07-21 13:06:42 +09:00
vorotamoroz
cb406e2db6 New feature. 
- Local database name can now be customized.
- Buttons to back skip-patterns of Hidden file sync to default.
 2022-07-21 13:05:35 +09:00
vorotamoroz
0a1248c5fc Fixed: 
- Now Notification is less noisy.
- Some synchronization won't be missed.
- Scanning speed improved.

Implemented:

- Implemented notifications to reload the plugin.
- Rescue button to updating all hidden files for overwriting them on other vaults.
 2022-07-20 16:57:21 +09:00
vorotamoroz
7b9b934c61 Tidied up. 2022-07-19 18:41:39 +09:00
vorotamoroz
27505f3024 Add the doc 2022-07-19 18:41:27 +09:00
vorotamoroz
1cddcf8b95 bumped 2022-07-19 18:00:25 +09:00
vorotamoroz
fddc466b0f New Feature 
- Hidden file sync.
 2022-07-19 17:57:29 +09:00
vorotamoroz
0e6a6dcd2a bumped 2022-07-14 18:35:04 +09:00
vorotamoroz
f3a47b904f - fix for the pop-out window. 
- fix file handling in boot sequence.
 2022-07-14 18:34:55 +09:00
vorotamoroz
6563481501 add type annotation 2022-07-14 18:32:59 +09:00
vorotamoroz
b5e8ee691a Merge pull request #89 from MohamedBassem/patch-1 
Removing the step to enable history from the README
 2022-07-13 10:03:40 +09:00
Mohamed Bassem
22a428f216 Removing the step to enable history from the README 
While following the README, it told me to enable "Use History" though I couldn't find this setting anywhere. Turned out, it became the default as of 89de551fd7 with no way to be configured. So this PR, removes this step from the readme.
 2022-07-12 12:20:24 +01:00
snyk-bot
d5a95d43dd fix: upgrade idb from 7.0.1 to 7.0.2 
Snyk has created this PR to upgrade idb from 7.0.1 to 7.0.2.

See this package in npm:
https://www.npmjs.com/package/idb

See this project in Snyk:
https://app.snyk.io/org/vrtmrz/project/d2c9b72d-6e38-433f-bbad-725719c0fa4d?utm_source=github&utm_medium=referral&page=upgrade-pr
 2022-07-09 05:02:39 +00:00
vorotamoroz
7d6b83a1cb Fixed 
- Saving notes with wrong type.
 2022-07-07 17:21:23 +09:00
vorotamoroz
41034d7d92 Create release.yml 2022-06-30 18:18:31 +09:00
vorotamoroz
2455ff6ee1 bump 2022-06-30 18:17:09 +09:00
vorotamoroz
89de551fd7 Fixed: 
- Unexpected massive palallel running of file checking in boot sequence is solved.
- Batch file change is not  missing changes now.
- Ignore changes caused by the plug-ins themselves.
- Garbage collection is completely disabled.
- Fixed sometimes fails initial replication after dropping local DB.
Improved:
- a bit more understandable messages
- Save the file into the big chunk on initial scan.
- Use history is always enabled.
- Boot sequence got faster.
 2022-06-30 17:46:42 +09:00
vorotamoroz
124a49b80f Fixed and implemented 
- Readme tidied.
- More faster e2ee.
- URI and databasename check improved.
 2022-06-23 18:26:43 +09:00
vorotamoroz
3e76292aa7 Implemented: 
- Encrypting setup URI by passphrse.
(Note: You have to make the setup URI again)

Fixed:
- Setup procedure fixed.
- Status text fixed.
- Documentation fixed.
 2022-06-19 15:36:36 +09:00
vorotamoroz
4634ab73b1 - Automatic garbage collection disabled 
- Fixed database unloading problem
 2022-06-19 14:09:11 +09:00
vorotamoroz
359c10f1d7 Correction of wording 2022-06-15 17:59:10 +09:00
vorotamoroz
59ebac3efc Correction of wording 2022-06-15 17:54:35 +09:00
vorotamoroz
b4edca3a99 Fixed format. 2022-06-15 17:51:07 +09:00
vorotamoroz
4b76b10a6f Add some note. 2022-06-15 17:45:46 +09:00
vorotamoroz
d4b53280e3 Fixed status message and lag on boot time scan. 2022-06-15 17:45:37 +09:00
vorotamoroz
dbd9b17b20 Fixed: 
- Fixed ignoring changes on replicating.
- Disabled `Skip old files on sync` temporary (so fragile between multiple devices)
 2022-06-14 19:49:21 +09:00
snyk-bot
7da930a8bb fix: upgrade svelte-preprocess from 4.10.3 to 4.10.5 
Snyk has created this PR to upgrade svelte-preprocess from 4.10.3 to 4.10.5.

See this package in npm:
https://www.npmjs.com/package/svelte-preprocess

See this project in Snyk:
https://app.snyk.io/org/vrtmrz/project/d2c9b72d-6e38-433f-bbad-725719c0fa4d?utm_source=github&utm_medium=referral&page=upgrade-pr
 2022-04-29 03:54:29 +00:00
snyk-bot
a632b79726 fix: upgrade esbuild-svelte from 0.6.2 to 0.7.0 
Snyk has created this PR to upgrade esbuild-svelte from 0.6.2 to 0.7.0.

See this package in npm:
https://www.npmjs.com/package/esbuild-svelte

See this project in Snyk:
https://app.snyk.io/org/vrtmrz/project/d2c9b72d-6e38-433f-bbad-725719c0fa4d?utm_source=github&utm_medium=referral&page=upgrade-pr
 2022-04-29 03:54:25 +00:00
66 changed files with 11127 additions and 5916 deletions
Expand all files Collapse all files

5
.eslintignore
View File

@@ -1,3 +1,4 @@
npm node_modules
node_modules
build
.eslintrc.js.bak
.eslintrc.js.bak
src/lib/src/patches/pouchdb-utils

27
.eslintrc
View File

@@ -1,19 +1,34 @@
{
"root": true,
"parser": "@typescript-eslint/parser",
"plugins": ["@typescript-eslint"],
"extends": ["eslint:recommended", "plugin:@typescript-eslint/eslint-recommended", "plugin:@typescript-eslint/recommended"],
"plugins": [
"@typescript-eslint"
],
"extends": [
"eslint:recommended",
"plugin:@typescript-eslint/eslint-recommended",
"plugin:@typescript-eslint/recommended"
],
"parserOptions": {
"sourceType": "module"
"sourceType": "module",
"project": [
"tsconfig.json"
]
},
"rules": {
"no-unused-vars": "off",
"@typescript-eslint/no-unused-vars": ["error", { "args": "none" }],
"@typescript-eslint/no-unused-vars": [
"error",
{
"args": "none"
}
],
"@typescript-eslint/ban-ts-comment": "off",
"no-prototype-builtins": "off",
"@typescript-eslint/no-empty-function": "off",
"require-await": "warn",
"no-async-promise-executor": "off",
"@typescript-eslint/no-explicit-any": "off"
"@typescript-eslint/no-explicit-any": "off",
"@typescript-eslint/no-unnecessary-type-assertion": "error"
}
}
}

94
.github/workflows/release.yml vendored Normal file
View File

@@ -0,0 +1,94 @@
name: Release Obsidian Plugin
on:
push:
# Sequence of patterns matched against refs/tags
tags:
- '*' # Push events to matching any tag format, i.e. 1.0, 20.15.10
workflow_dispatch:
jobs:
build:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v2
with:
fetch-depth: 0 # otherwise, you will failed to push refs to dest repo
submodules: recursive
- name: Use Node.js
uses: actions/setup-node@v1
with:
node-version: '14.x' # You might need to adjust this value to your own version
# Get the version number and put it in a variable
- name: Get Version
id: version
run: |
echo "::set-output name=tag::$(git describe --abbrev=0 --tags)"
# Build the plugin
- name: Build
id: build
run: |
npm ci
npm run build --if-present
# Package the required files into a zip
- name: Package
run: |
mkdir ${{ github.event.repository.name }}
cp main.js manifest.json styles.css README.md ${{ github.event.repository.name }}
zip -r ${{ github.event.repository.name }}.zip ${{ github.event.repository.name }}
# Create the release on github
- name: Create Release
id: create_release
uses: actions/create-release@v1
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
VERSION: ${{ github.ref }}
with:
tag_name: ${{ github.ref }}
release_name: ${{ github.ref }}
draft: true
prerelease: false
# Upload the packaged release file
- name: Upload zip file
id: upload-zip
uses: actions/upload-release-asset@v1
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
with:
upload_url: ${{ steps.create_release.outputs.upload_url }}
asset_path: ./${{ github.event.repository.name }}.zip
asset_name: ${{ github.event.repository.name }}-${{ steps.version.outputs.tag }}.zip
asset_content_type: application/zip
# Upload the main.js
- name: Upload main.js
id: upload-main
uses: actions/upload-release-asset@v1
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
with:
upload_url: ${{ steps.create_release.outputs.upload_url }}
asset_path: ./main.js
asset_name: main.js
asset_content_type: text/javascript
# Upload the manifest.json
- name: Upload manifest.json
id: upload-manifest
uses: actions/upload-release-asset@v1
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
with:
upload_url: ${{ steps.create_release.outputs.upload_url }}
asset_path: ./manifest.json
asset_name: manifest.json
asset_content_type: application/json
# Upload the style.css
- name: Upload styles.css
id: upload-css
uses: actions/upload-release-asset@v1
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
with:
upload_url: ${{ steps.create_release.outputs.upload_url }}
asset_path: ./styles.css
asset_name: styles.css
asset_content_type: text/css
# TODO: release notes???

1
.gitignore vendored
View File

@@ -12,3 +12,4 @@ main.js
# obsidian
data.json
.vscode

130
README.md
View File

@@ -1,93 +1,99 @@
# Self-hosted LiveSync
Sorry for late! [Japanese docs](./README_ja.md) is also coming up.
[Japanese docs](./README_ja.md) [Chinese docs](./README_cn.md).
**Renamed from: obsidian-livesync**
Self-hosted LiveSync is a community-implemented synchronization plugin.
A self-hosted or purchased CouchDB acts as the intermediate server. Available on every obsidian-compatible platform.
Using a self-hosted database, live-sync to multi-devices bidirectionally.
Runs in Mac, Android, Windows, and iOS. Perhaps available on Linux too.
Community implementation, not compatible with official "Sync".
Note: It has no compatibility with the official "Obsidian Sync".
![obsidian_live_sync_demo](https://user-images.githubusercontent.com/45774780/137355323-f57a8b09-abf2-4501-836c-8cb7d2ff24a3.gif)
**It's getting almost stable now, But Please make sure to back your vault up!**
Before installing or upgrading LiveSync, please back your vault up.
Limitations: ~~Folder deletion handling is not completed.~~ **It would work now.**
## Features
## This plugin enables...
- Runs in Windows, Mac, iPad, iPhone, Android, Chromebook
- Synchronize to Self-hosted Database
- Replicate to/from other devices bidirectionally near-real-time
- Resolving synchronizing conflicts in the Obsidian.
- You can use CouchDB or its compatibles like IBM Cloudant. CouchDB is OSS, and IBM Cloudant has the terms and certificates about security. Your notes are yours.
- Off-line sync is also available.
- End-to-End encryption is available (beta).
- Visual conflict resolver included.
- Bidirectional synchronization between devices nearly in real-time
- You can use CouchDB or its compatibles like IBM Cloudant.
- End-to-End encryption is supported.
- Plugin synchronization(Beta)
- Receive WebClip from [obsidian-livesync-webclip](https://chrome.google.com/webstore/detail/obsidian-livesync-webclip/jfpaflmpckblieefkegjncjoceapakdf) (End-to-End encryption will not be applicable.)
It must be useful for the Researcher, Engineer, Developer who has to keep NDA or something like agreement.
Especially, in some companies, people have to store all data to their fully controlled host, even End-to-End encryption applied.
Useful for researchers, engineers and developers with a need to keep their notes fully self-hosted for security reasons. Or just anyone who would like the peace of mind of knowing that their notes are fully private.
## IMPORTANT NOTICE
- Do not use with other synchronize solutions. Before enabling this plugin, make sure to disable other synchronize solutions, to avoid content corruption or duplication. If you want to synchronize to both backend, sync one by one, please.
This includes making your vault on the cloud-controlled folder(e.g., Inside the iCloud folder).
- This is the synchronization plugin. Not backup solutions. Do not rely on this for backup.
- When the device's storage has been run out, Database corruption may happen.
- When editing hidden files or any other invisible files from obsidian, the file wouldn't be kept in the database. (**Or be deleted.**)
## Supplements
- When the file has been deleted, the deletion of the file is replicated to other devices.
- When the folder became empty by replication, The folder will be deleted in the default setting. But you can change this behaivour. Check the [Settings](docs/settings.md).
- LiveSync drains many batteries in mobile devices.
- Mobile Obsidian can not connect to the non-secure(HTTP) or local CA-signed servers, even though the certificate is stored in the device store.
- There are no 'exclude_folders' like configurations.
- Do not enable this plugin with another synchronization solution at the same time (including iCloud and Obsidian Sync). Before enabling this plugin, make sure to disable all the other synchronization methods to avoid content corruption or duplication. If you want to synchronize to two or more services, do them one by one and never enable two synchronization methods at the same time.
This includes not putting your vault inside a cloud-synchronized folder (eg. an iCloud folder or Dropbox folder)
- This is a synchronization plugin. Not a backup solution. Do not rely on this for backup.
- If the device's storage runs out, database corruption may happen.
- Hidden files or any other invisible files wouldn't be kept in the database, and thus won't be synchronized. (**and may also get deleted**)
## How to use
1. Install from Obsidian, or download from this repo's releases, copy `main.js`, `styles.css` and `manifest.json` into `[your-vault]/.obsidian/plugins/`
2. Get your database. IBM Cloudant is preferred for testing. Or you can use your own server with CouchDB.
For more information, refer below:
1. [Setup IBM Cloudant](docs/setup_cloudant.md)
2. [Setup your CouchDB](docs/setup_own_server.md)
3. Enter connection information to Plugin's setting dialog. In details, refer [Settings of Self-hosted LiveSync](docs/settings.md)
4. Enable LiveSync or other Synchronize method as you like.
### Get your database ready.
First, get your database ready. IBM Cloudant is preferred for testing. Or you can use your own server with CouchDB. For more information, refer below:
1. [Setup IBM Cloudant](docs/setup_cloudant.md)
2. [Setup your CouchDB](docs/setup_own_server.md)
Note: More information about alternative hosting methods is needed! Currently, [using fly.io](https://github.com/vrtmrz/obsidian-livesync/discussions/85) is being discussed.
### Configure the plugin
See [Quick setup guide](doccs/../docs/quick_setup.md)
## Something looks corrupted...
Please open the configuration link again and Answer below:
- If your local database looks corrupted (in other words, when your Obsidian getting weird even standalone.)
- Answer `No` to `Keep local DB?`
- If your remote database looks corrupted (in other words, when something happens while replicating)
- Answer `No` to `Keep remote DB?`
If you answered `No` to both, your databases will be rebuilt by the content on your device. And the remote database will lock out other devices. You have to synchronize all your devices again. (When this time, almost all your files should be synchronized with a timestamp. So you can use an existing vault).
## Test Server
Setting up an instance of Cloudant or local CouchDB is a little complicated, so I made the [Tasting server of self-hosted-livesync](https://olstaste.vrtmrz.net/) up. Try free!
~~Setting up an instance of Cloudant or local CouchDB is a little complicated, so I set up a [Tasting server for self-hosted-livesync](https://olstaste.vrtmrz.net/). Try it out for free!~~
Now (30 May 2023) suspending while the server transfer.
Note: Please read "Limitations" carefully. Do not send your private vault.
## WebClipper is also available.
Available from on Chrome Web Store:[obsidian-livesync-webclip](https://chrome.google.com/webstore/detail/obsidian-livesync-webclip/jfpaflmpckblieefkegjncjoceapakdf)
Repo is here: [obsidian-livesync-webclip](https://github.com/vrtmrz/obsidian-livesync-webclip). (Docs are work in progress.)
# Information in StatusBar
## Information in StatusBar
Synchronization status is shown in statusbar.
- Status
- ⏹️ Stopped
- 💤 LiveSync is enabled. Waiting for changes.
- ⚡️ Synchronize is now in progress.
- ⚠ Error occurred.
- ↑ Uploaded pieces
- ↓ Downloaded pieces
- ⏳ Count of the pending process
If you have deleted or renamed files, please wait until this disappears.
- 💤 LiveSync enabled. Waiting for changes.
- ⚡️ Synchronization in progress.
- ⚠ An error occurred.
- ↑ Uploaded chunks and metadata
- ↓ Downloaded chunks and metadata
- ⏳ Number of pending processes
- 🧩 Number of files waiting for their chunks.
If you have deleted or renamed files, please wait until ⏳ icon disappeared.
# More supplements
- When synchronized, files are compared by their modified times and overwritten by the newer ones once. Then plugin checks the conflicts and if a merge is needed, the dialog will open.
- Rarely, the file in the database would be broken. The plugin will not write storage when it looks broken, so some old files must be on your device. If you edit the file, it will be cured. But if the file does not exist on any device, can not rescue it. So you can delete these items from the setting dialog.
- If your database looks corrupted, try "Drop History". Usually, It is the easiest way.
- To stop the bootup sequence for fixing problems on databases, you can put `redflag.md` on top of your vault.
- Q: Database is growing, how can I shrink it up?
A: each of the docs is saved with their old 100 revisions to detect and resolve confliction. Picture yourself that one device has been off the line for a while, and joined again. The device has to check his note and remote saved note. If exists in revision histories of remote notes even though the device's note is a little different from the latest one, it could be merged safely. Even if that is not in revision histories, we only have to check differences after the revision that both devices commonly have. This is like The git's conflict resolving method. So, We have to make the database again like an enlarged git repo if you want to solve the root of the problem.
- And more technical Information are in the [Technical Information](docs/tech_info.md)
## Hints
- If a folder becomes empty after a replication, it will be deleted by default. But you can toggle this behaviour. Check the [Settings](docs/settings.md).
- LiveSync mode drains more batteries in mobile devices. Periodic sync with some automatic sync is recommended.
- Mobile Obsidian can not connect to non-secure (HTTP) or locally-signed servers, even if the root certificate is installed on the device.
- There are no 'exclude_folders' like configurations.
- While synchronizing, files are compared by their modification time and the older ones will be overwritten by the newer ones. Then plugin checks for conflicts and if a merge is needed, a dialog will open.
- Rarely, a file in the database could be corrupted. The plugin will not write to local storage when a file looks corrupted. If a local version of the file is on your device, the corruption could be fixed by editing the local file and synchronizing it. But if the file does not exist on any of your devices, then it can not be rescued. In this case, you can delete these items from the settings dialog.
- To stop the boot-up sequence (eg. for fixing problems on databases), you can put a `redflag.md` file (or directory) at the root of your vault.
Tip for iOS: a redflag directory can be created at the root of the vault using the File application.
- Also, with `redflag2.md` placed, we can automatically rebuild both the local and the remote databases during the boot-up sequence. With `redflag3.md`, we can discard only the local database and fetch from the remote again.
- Q: The database is growing, how can I shrink it down?
A: each of the docs is saved with their past 100 revisions for detecting and resolving conflicts. Picturing that one device has been offline for a while, and comes online again. The device has to compare its notes with the remotely saved ones. If there exists a historic revision in which the note used to be identical, it could be updated safely (like git fast-forward). Even if that is not in revision histories, we only have to check the differences after the revision that both devices commonly have. This is like git's conflict-resolving method. So, We have to make the database again like an enlarged git repo if you want to solve the root of the problem.
- And more technical Information is in the [Technical Information](docs/tech_info.md)
- If you want to synchronize files without obsidian, you can use [filesystem-livesync](https://github.com/vrtmrz/filesystem-livesync).
- WebClipper is also available on Chrome Web Store:[obsidian-livesync-webclip](https://chrome.google.com/webstore/detail/obsidian-livesync-webclip/jfpaflmpckblieefkegjncjoceapakdf)
# License
Repo is here: [obsidian-livesync-webclip](https://github.com/vrtmrz/obsidian-livesync-webclip). (Docs are a work in progress.)
The source code is licensed MIT.
## License
The source code is licensed under the MIT License.

129
README_cn.md Normal file
View File

@@ -0,0 +1,129 @@
# Self-hosted LiveSync
Self-hosted LiveSync (自搭建在线同步) 是一个社区实现的在线同步插件。
使用一个自搭建的或者购买的 CouchDB 作为中转服务器。兼容所有支持 Obsidian 的平台。
注意: 本插件与官方的 "Obsidian Sync" 服务不兼容。
![obsidian_live_sync_demo](https://user-images.githubusercontent.com/45774780/137355323-f57a8b09-abf2-4501-836c-8cb7d2ff24a3.gif)
安装或升级 LiveSync 之前，请备份你的 vault。
## 功能
- 可视化的冲突解决器
- 接近实时的多设备双向同步
- 可使用 CouchDB 以及兼容的服务，如 IBM Cloudant
- 支持端到端加密
- 插件同步 (Beta)
- 从 [obsidian-livesync-webclip](https://chrome.google.com/webstore/detail/obsidian-livesync-webclip/jfpaflmpckblieefkegjncjoceapakdf) 接收 WebClip (本功能不适用端到端加密)
适用于出于安全原因需要将笔记完全自托管的研究人员、工程师或开发人员，以及任何喜欢笔记完全私密所带来的安全感的人。
## 重要提醒
- 请勿与其他同步解决方案（包括 iCloud、Obsidian Sync）一起使用。在启用此插件之前，请确保禁用所有其他同步方法以避免内容损坏或重复。如果要同步到多个服务，请一一进行，切勿同时启用两种同步方法。
这包括不能将您的保管库放在云同步文件夹中（例如 iCloud 文件夹或 Dropbox 文件夹）
- 这是一个同步插件，不是备份解决方案。不要依赖它进行备份。
- 如果设备的存储空间耗尽，可能会发生数据库损坏。
- 隐藏文件或任何其他不可见文件不会保存在数据库中，因此不会被同步。（**并且可能会被删除**）
## 如何使用
### 准备好你的数据库
首先，准备好你的数据库。IBM Cloudant 是用于测试的首选。或者，您也可以在自己的服务器上安装 CouchDB。有关更多信息，请参阅以下内容：
1. [Setup IBM Cloudant](docs/setup_cloudant.md)
2. [Setup your CouchDB](docs/setup_own_server_cn.md)
Note: 正在征集更多搭建方法！目前在讨论的有 [使用 fly.io](https://github.com/vrtmrz/obsidian-livesync/discussions/85)。
### 第一个设备
1. 在您的设备上安装插件。
2. 配置远程数据库信息。
1. 将您的服务器信息填写到 `Remote Database configuration`（远程数据库配置）设置页中。
2. 建议启用 `End to End Encryption`（端到端加密）。输入密码后，单击“应用”。
3. 点击 `Test Database Connection` 并确保插件显示 `Connected to (你的数据库名称)`。
4. 单击 `Check database configuration`（检查数据库配置）并确保所有测试均已通过。
3. 在 `Sync Settings`（同步设置）选项卡中配置何时进行同步。（您也可以稍后再设置）
1. 如果要实时同步，请启用 `LiveSync`。
2. 或者，根据您的需要设置同步方式。默认情况下，不会启用任何自动同步，这意味着您需要手动触发同步过程。
3. 其他配置也在这里。建议启用 `Use Trash for deleted files`（删除文件到回收站），但您也可以保持所有配置不变。
4. 配置杂项功能。
1. 启用 `Show staus inside editor` 会在编辑器右上角显示状态。（推荐开启）
5. 回到编辑器。等待初始扫描完成。
6. 当状态不再变化并显示 ⏹️ 图标表示 COMPLETED（没有 ⏳ 和 🧩 图标）时，您就可以与服务器同步了。
7. 按功能区上的复制图标或从命令面板运行 `Replicate now`（立刻复制）。这会将您的所有数据发送到服务器。
8. 打开命令面板，运行 `Copy setup URI`（复制设置链接），并设置密码。这会将您的配置导出到剪贴板，作为您导入其他设备的链接。
**重要: 不要公开本链接，这个链接包含了你的所有认证信息！** (即使没有密码别人读不了)
### 后续设备
注意：如果要与非空的 vault 进行同步，文件的修改日期和时间必须互相匹配。否则，可能会发生额外的传输或文件可能会损坏。
为简单起见，我们强烈建议同步到一个全空的 vault。
1. 安装插件。
2. 打开您从第一台设备导出的链接。
3. 插件会询问您是否确定应用配置。 回答 `Yes`，然后按照以下说明进行操作：
1. 对 `Keep local DB?` 回答 `Yes`。
*注意：如果您希望保留本地现有 vault，则必须对此问题回答 `No`，并对 `Rebuild the database?` 回答 `No`。*
2. 对 `Keep remote DB?` 回答 `Yes`。
3. 对 `Replicate once?` 回答 `Yes`。
完成后，您的所有设置将会从第一台设备成功导入。
4. 你的笔记应该很快就会同步。
## 文件看起来有损坏...
请再次打开配置链接并回答如下：
- 如果您的本地数据库看起来已损坏（当你的本地 Obsidian 文件看起来很奇怪）
- 对 `Keep local DB?` 回答 `No`
- 如果您的远程数据库看起来已损坏（当复制时发生中断）
- 对 `Keep remote DB?` 回答 `No`
如果您对两者都回答“否”，您的数据库将根据您设备上的内容重建。并且远程数据库将锁定其他设备，您必须再次同步所有设备。（此时，几乎所有文件都会与时间戳同步。因此您可以安全地使用现有的 vault）。
## 测试服务器
设置 Cloudant 或本地 CouchDB 实例有点复杂，所以我搭建了一个 [self-hosted-livesync 尝鲜服务器](https://olstaste.vrtmrz.net/)。欢迎免费尝试！
注意：请仔细阅读“限制”条目。不要发送您的私人 vault。
## 状态栏信息
同步状态将显示在状态栏。
- 状态
- ⏹️ 就绪
- 💤 LiveSync 已启用，正在等待更改。
- ⚡️ 同步中。
- ⚠ 一个错误出现了。
- ↑ 上传的 chunk 和元数据数量
- ↓ 下载的 chunk 和元数据数量
- ⏳ 等待的过程的数量
- 🧩 正在等待 chunk 的文件数量
如果你删除或更名了文件，请等待 ⏳ 图标消失。
## 提示
- 如果文件夹在复制后变为空，则默认情况下该文件夹会被删除。您可以关闭此行为。检查 [设置](docs/settings.md)。
- LiveSync 模式在移动设备上可能导致耗电量增加。建议使用定期同步 + 条件自动同步。
- 移动平台上的 Obsidian 无法连接到非安全 (HTTP) 或本地签名的服务器，即使设备上安装了根证书。
- 没有类似“exclude_folders”的配置。
- 同步时，文件按修改时间进行比较，较旧的将被较新的文件覆盖。然后插件检查冲突，如果需要合并，将打开一个对话框。
- 数据库中的文件在罕见情况下可能会损坏。当接收到的文件看起来已损坏时，插件不会将其写入本地存储。如果您的设备上有文件的本地版本，则可以通过编辑本地文件并进行同步来覆盖损坏的版本。但是，如果您的任何设备上都不存在该文件，则无法挽救该文件。在这种情况下，您可以从设置对话框中删除这些损坏的文件。
- 要阻止插件的启动流程（例如，为了修复数据库问题），您可以在 vault 的根目录创建一个 "redflag.md" 文件。
- 问：数据库在增长，我该如何缩小它？
答：每个文档都保存了过去 100 次修订，用于检测和解决冲突。想象一台设备已经离线一段时间，然后再次上线。设备必须将其笔记与远程保存的笔记进行比较。如果存在曾经相同的历史修订，则可以安全地直接更新这个文件（和 git 的快进原理一样）。即使文件不在修订历史中，我们也只需检查两个设备上该文件的公有修订版本之后的差异。这就像 git 的冲突解决方法。所以，如果想从根本上解决数据库太大的问题，我们像构建一个扩大版的 git repo 一样去重新设计数据库。
- 更多技术信息在 [技术信息](docs/tech_info.md)
- 如果你想在没有黑曜石的情况下同步文件，你可以使用[filesystem-livesync](https://github.com/vrtmrz/filesystem-livesync)。
- WebClipper 也可在 Chrome Web Store 上使用：[obsidian-livesync-webclip](https://chrome.google.com/webstore/detail/obsidian-livesync-webclip/jfpaflmpckblieefkegjncjoceapakdf)
仓库地址：[obsidian-livesync-webclip](https://github.com/vrtmrz/obsidian-livesync-webclip) （文档施工中）
## License
The source code is licensed under the MIT License.
本源代码使用 MIT 协议授权。

22
README_ja.md
View File

@@ -6,7 +6,7 @@
**公式のSyncとは互換性はありません**
![obsidian_live_sync_demo](https://user-images.githubusercontent.com/45774780/137355323-f57a8b09-abf2-4501-836c-8cb7d2ff24a3.gif)
**ほぼ動くようになってきましたが、Vaultのバックアップは確実に取得してください**
**インストールする前に、Vaultのバックアップを確実に取得してください**
[英語版](./README.md)
@@ -30,28 +30,17 @@ NDAや類似の契約や義務、倫理を守る必要のある、研究者、
これは、Vaultをクラウド管理下のフォルダに置くことも含みます。(例えば、iCloudの管理フォルダ内に入れたり)。
- ⚠️このプラグインは、端末間でのノートの反映を目的として作成されました。バックアップ等が目的ではありません。そのため、バックアップは必ず別のソリューションで行うようにしてください。
- ストレージの空き容量が枯渇した場合、データベースが破損することがあります。
- 隠しファイルやObsidisanが認識できないファイルを編集した場合、そのファイルは削除されることがあります。
# 補足
- レプリケーションなどでファイルがリモートデバイスから削除された場合、受信したデバイスでも、ファイルの削除が反映されます。
- その際、Self-hosted LiveSyncは、フォルダが空になった際に、フォルダをデフォルトでは残しません。残す場合はオプションから設定してください。
- LiveSyncはモバイルではバッテリーをかなり消費します。
- モバイル端末からは、非httpsのエンドポイント、または独自CAが発行した証明書でホストされているhttpsのサーバーには接続できません。
- 除外フォルダのような設定はありません。
# このプラグインの使い方
1. Community Pluginsから、Self-holsted LiveSyncと検索しインストールするか、このリポジトリのReleasesから`main.js`, `manifest.json`, `style.css` をダウンロードしvaultの中の`.obsidian/plugins/obsidian-livesync`に入れて、Obsidianを再起動してください。
2. サーバーを確保します。IBM Cloudantがお手軽かつ堅牢で便利です。完全にセルフホストする際にはお持ちのサーバーにCouchDBをインストールする必要があります。詳しくは下記を参照してください
2. サーバーをセットアップします。IBM Cloudantがお手軽かつ堅牢で便利です。完全にセルフホストする際にはお持ちのサーバーにCouchDBをインストールする必要があります。詳しくは下記を参照してください
1. [IBM Cloudantのセットアップ](docs/setup_cloudant_ja.md)
2. [独自のCouchDBのセットアップ](docs/setup_own_server_ja.md)
3. サーバー情報を入力します。初回のみ、Obsidianを再起動することをオススメします。
設定内容の詳細は[このプラグインの設定](docs/settings_ja.md)を参照してください。
4. お好きな同期方法を選んで、利用を開始してください。
備考： IBM Cloudantのアカウント登録が出来ないケースがあるようです。代替を探していて、今 [using fly.io](https://github.com/vrtmrz/obsidian-livesync/discussions/85)を検討しています。
1. [Quick setup](docs/quick_setup_ja.md)から、セットアップウィザード使ってセットアップしてください。
# テストサーバー
@@ -84,7 +73,6 @@ Self-hosted LiveSync用にWebClipperも作りました。Chrome Web Storeから
# さらなる補足
- ファイルは同期された後、タイムスタンプを比較して新しければいったん新しい方で上書きされます。その後、衝突が発生したかによって、マージが行われます。
- まれにファイルが破損することがあります。破損したファイルに関してはディスクへの反映を試みないため、実際には使用しているデバイスには少し古いファイルが残っていることが多いです。そのファイルを再度更新してもらうと、データベースが更新されて問題なくなるケースがあります。ファイルがどの端末にも存在しない場合は、設定画面から、削除できます。
- データベースが変。そういうときは、いったんデータベースをDrop Historyのapply and sendで再初期化してみてください。だいたい直ります。
- データベースの復旧中に再起動した場合など、うまくローカルデータベースを修正できない際には、Vaultのトップに`redflag.md`というファイルを置いてください。起動時のシーケンスがスキップされます。
- データベースが大きくなってきてるんだけど、小さくできる？→各ノートは、それぞれの古い100リビジョンとともに保存されています。例えば、しばらくオフラインだったあるデバイスが、久しぶりに同期したと想定してみてください。そのとき、そのデバイスは最新とは少し異なるリビジョンを持ってるはずです。その場合でも、リモートのリビジョン履歴にリモートのものが存在した場合、安全にマージできます。もしリビジョン履歴に存在しなかった場合、確認しなければいけない差分も、対象を存在して持っている共通のリビジョン以降のみに絞れます。ちょうどGitのような方法で、衝突を解決している形になるのです。そのため、肥大化したリポジトリの解消と同様に、本質的にデータベースを小さくしたい場合は、データベースの作り直しが必要です。
- その他の技術的なお話は、[技術的な内容](docs/tech_info_ja.md)に書いてあります。

97
docs/quick_setup.md Normal file
View File

@@ -0,0 +1,97 @@
# Quick setup
The Setup wizard has been implemented since v0.15.0 simplifying the initial setup.
Note: Subsequent devices should be set up using the `Copy setup URI` and `Open setup URI` functionality.
## How to open and use the wizard
Open the `🪄 Setup wizard` in the settings dialogue. If the plugin has not been configured before, it should already be open.
![](../images/quick_setup_1.png)
### Discard the existing configuration and set up
If you have changed any settings, this button allows you to discard all changes before setting up.
### Do not discard the existing configuration and set up
Simply reconfigure. Be careful. In wizard mode, you cannot see all configuration items, even if they have been configured.
Pressing `Next` on one of the above options will put the configuration dialog into wizard mode.
### Wizard mode
![](../images/quick_setup_2.png)
Let's see how to use it step-by-step.
## Remote Database configuration
### Remote database configuration
Enter the information for the database we have set up.
![](../images/quick_setup_3.png)
### End to End Encryption
![](../images/quick_setup_4.png)
When End to End encryption is enabled, a third party will be less likely to be able to read your Remote database in the event of a data breach/leak (assuming they do not know the Passphrase). We strongly recommend enabling it.
Encryption is based on 256-bit AES-GCM.
This setting can be disabled if you are inside a closed network and it is clear that you will not be accessed by third parties.
### Test database connection and Check database configuration
Here we can check the status of the connection to the database and the database settings.
![](../images/quick_setup_5.png)
#### Test Database Connection
Check whether we can connect to the database. If it fails, there are several possible reasons, but first attempt the `Check database configuration` check to see if it fails there too.
#### Check database configuration
Check the database settings and fix any deficiencies on the spot.
![](../images/quick_setup_6.png)
This item may vary depending on the connection. In the above case, press all three Fix buttons.
If the Fix buttons disappear and all become check marks, we are done.
![](../images/quick_setup_7.png)
### Next
Go to the Local Database configuration.
### Discard existing database and proceed
Discard the contents of the Remote database and go to the Local Database configuration.
## Local Database configuration
![](../images/quick_setup_8.png)
Configure the local database. If we already have a Vault with Self-hosted LiveSync installed which has the same directory name as the one we are currently setting up, please specify a different suffix than the Vault you have already set up here.
## Miscellaneous
Finally, finish the miscellaneous configurations and select a preset for synchronisation.
![](../images/quick_setup_9_1.png)
The `Show status inside editor` can be enabled to your liking. If enabled, the status is displayed in the top right-hand corner of the editor. Learn how to read the status bar [here](/README.md#information-in-statusbar).
![](../images/quick_setup_9_2.png)
From Presets, select the synchronisation method we want to use and `Apply` to initialise and build the local and remote databases as required.
If `All done!' is displayed, we are done. Automatically, `Copy setup URI` will open and we will be asked for a passphrase to encrypt the `Setup URI`.
![](../images/quick_setup_10.png)
Set the passphrase as you like.
The Setup URI will be copied to the clipboard, which you can then transfer to the second and subsequent devices in some way.
# How to set up the second and subsequent units
After installing Self-hosted LiveSync on the first device, select `Open setup URI` from the command palette and enter the setup URI you transferred. Afterwards, enter your passphrase and a setup wizard will open.
Answer the following.
- `Yes` to `Importing LiveSync's conf, OK?`
- `Set it up as secondary or subsequent device` to `How would you like to set it up?`.
Then, The configuration will take effect and replication will start. Your files will be synchronised soon! You may need to close the settings dialog and reopen it to see the settings fields populated properly, but they will be set.

94
docs/quick_setup_ja.md Normal file
View File

@@ -0,0 +1,94 @@
# Quick setup
v0.15.0からSetup wizardが実装されました。これで、初回セットアップがシンプルになります。
※なお、次のデバイスからは、`Copy setup URI`と`Open setup URI`を使ってセットアップしてください。
## Wizardの使い方
`🪄 Setup wizard` から開きます。もしセットアップされていなかったり、同期設定が何も有効になっていない場合はデフォルトで開いています。
![](../images/quick_setup_1.png)
### Discard the existing configuration and set up
今設定されている内容をいったん全部消してから、ウィザードを始めます。
### Do not discard the existing configuration and set up
今の設定を消さずにウィザードを開始します。
たとえ設定されていたとしても、ウィザードモードではすべての設定を見ることができません。
いずれかのNextを押すと、設定画面がウィザードモードになります。
### Wizardモード
![](../images/quick_setup_2.png)
順番に設定を行っていきます。
## Remote Database configuration
### Remote databaseの設定
セットアップしたデータベースの情報を入力していきます。
![](../images/quick_setup_3.png)
これらはデータベースをセットアップした際に決めた情報です。
### End to End暗号化の設定
![](../images/quick_setup_4.png)
End to End暗号化を有効にした場合、万が一Remote databaseの内容が流出してもPassphraseを知らない第三者にそれを読まれる可能性が低くなります。そのため、有効化を強く推奨します。
暗号化は256bitのAES-GCMを採用しています。
この設定は、あなたが閉じたネットワークの内側にいて、かつ第三者からアクセスされない事が明確な場合には無効にできます。
### Test database connectionとCheck database configuraion
ここで、データベースへの接続状況と、データベース設定を確認します。
![](../images/quick_setup_5.png)
#### Test Database Connection
データベースに接続出来るか自体を確認します。失敗する場合はいくつか理由がありますが、一度Check database configurationを行ってそちらでも失敗するか確認してください。
#### Check database configuration
データベースの設定を確認し、不備がある場合はその場で修正します。
![](../images/quick_setup_6.png)
この項目は接続先によって異なる場合があります。上記の場合、みっつのFixボタンを順にすべて押してください。
Fixボタンがなくなり、すべてチェックマークになれば完了です。
![](../images/quick_setup_7.png)
### Next
次へ進みます
### Discard exist database and proceed
すでにRemote databaseがある場合、Remote databaseの内容を破棄してから次へ進みます
## Local Database confiuration
![](../images/quick_setup_8.png)
ローカルのデータベースを設定します。もし、すでにSelf-hosted LiveSyncをインストールしたVaultがあり、そのVaultと同じデータベース名を使用している場合は、ここですでに設定したVaultとは異なるsuffixを指定してください。
## Miscellaneous
最後にその他の設定を行います。
![](../images/quick_setup_9_1.png)
`Show status inside editor`はお好みで有効化してください。有効にするとエディターの右上にステータスが表示されます。
![](../images/quick_setup_9_2.png)
Presetsから、使用する同期方法を選び`Apply`を行うと、必要に応じてローカル・リモートのデータベースを初期化・構築します。
All done! と表示されれば完了です。自動的に、`Copy setup URI`が開き、`Setup URI`を暗号化するパスフレーズを聞かれます。
![](../images/quick_setup_10.png)
お好みのパスフレーズを設定してください。
クリップボードにSetup URIが保存されますので、これを2台目以降のデバイスに何らかの方法で転送してください。
# 2台目以降の設定方法
2台目の端末にSelf-hosted LiveSyncをインストールしたあと、コマンドパレットから`Open setup URI`を選択し、転送したsetup URIを入力します。その後、パスフレーズを入力するとセットアップ用のウィザードが開きます。
下記のように答えてください。
- `Importing LiveSync's conf, OK?` に `Yes`
- `How would you like to set it up?` に `Set it up as secondary or subsequent device`
これで設定が反映され、レプリケーションが開始されます。

190
docs/settings.md
View File

@@ -2,15 +2,29 @@ NOTE: This document surely became outdated. I'll improve this doc in a while. bu
# Settings of this plugin
The settings dialog has been quite long, so I split each configuration into tabs.
If you feel something, please feel free to inform me.
| icon | description |
| :---: | ----------------------------------------------------------------- |
| 🛰️ | [Remote Database Configurations](#remote-database-configurations) |
| 📦 | [Local Database Configurations](#local-database-configurations) |
| ⚙️ | [General Settings](#general-settings) |
| 🔁 | [Sync Settings](#sync-settings) |
| 🔧 | [Miscellaneous](#miscellaneous) |
| 🧰 | [Hatch](#miscellaneous) |
| 🔌 | [Plugin and its settings](#plugin-and-its-settings) |
| 🚑 | [Corrupted data](#corrupted-data) |
## Remote Database Configurations
Configure settings of synchronize server. If any synchronization is enabled, you can't edit this section. Please disable all synchronization to change.
Configure the settings of synchronize server. If any synchronization is enabled, you can't edit this section. Please disable all synchronization to change.
### URI
URI of CouchDB. In the case of Cloudant, It's "External Endpoint(preferred)".
**Do not end it up with a slash** when it doesn't contain the database name.
### Username
Your CouchDB's Username. With administrator's privilege is preferred.
Your CouchDB's Username. Administrator's privilege is preferred.
### Password
Your CouchDB's Password.
@@ -21,12 +35,6 @@ The Database name to synchronize.
⚠️If not exist, created automatically.
### Use the old connecting method
This option has been removed at v0.10.0
### End to End Encryption
Encrypt your database. It affects only the database, your files are left as plain.
@@ -38,83 +46,58 @@ Note: If you want to use "Plugins and their settings", you have to enable this.
The passphrase to used as the key of encryption. Please use the long text.
### Apply
To enable End-to-End encryption, there must be no items of the same content encrypted with different passphrases to avoid attackers guessing passphrases. Self-hosted LiveSync uses crc32 of the chunks, It is really a must.
Set the End to End encryption enabled and its passphrase for use in replication.
If you change the passphrase of an existing database, overwriting the remote database is strongly recommended.
So, this plugin completely deletes everything from both local and remote databases before enabling it and then synchronizing again.
To enable, "Apply and send" from the most powerful device.
If you want to synchronize an existing database, set passphrase and press "Just apply".
### Overwrite remote database
Overwrite the remote database with the local database using the passphrase you applied.
- Apply and send
1. Initialize the Local Database and set (or clear) passphrase, put all files into the database again.
2. Initialize the Remote Database.
3. Lock the Remote Database.
4. Send it all.
This process is simply heavy. Using a PC or Mac is preferred.
- Apply and receive
1. Initialize the Local Database and set (or clear) the passphrase.
2. Unlock the Remote Database.
3. Retrieve all and decrypt to file.
When running these operations, every synchronization settings is disabled.
### Rebuild
Rebuild remote and local databases with local files. It will delete all document history and retained chunks, and shrink the database.
### Test Database connection
You can check the connection by clicking this button.
### Check database configuration
You can check and modify your CouchDB's configuration from here directly.
You can check and modify your CouchDB configuration from here directly.
### Lock remote database.
Other devices are banned from the database when you have locked the database.
If you have something troubled with other devices, you can protect the vault and remote database with your device.
## Local Database Configurations
"Local Database" is created inside your obsidian.
### Batch database update
Delay database update until raise replication, open another file, window visibility changed, or file events except for file modification.
Delay database update until raise replication, open another file, window visibility changes, or file events except for file modification.
This option can not be used with LiveSync at the same time.
### Auto Garbage Collection delay
When the note has been modified, Self-hosted LiveSync splits the note into some chunks by parsing the markdown structure. And saving only file information and modified chunks into the Local Database. At this time, do not delete old chunks.
So, Self-hosted LiveSync has to delete old chunks somewhen.
However, the chunk is represented as the crc32 of their contents and shared between all notes. In this way, Self-hosted LiveSync dedupes the entries and keeps low bandwidth and low transfer amounts.
In addition to this, when we edit notes, sometimes back to the previous expression. So It cannot be said that it will be unnecessary immediately.
Therefore, the plugin deletes unused chunks at once when you leave Obsidian for a while (after this setting seconds).
This process is called "Garbage Collection"
As a result, Obsidian's behavior is temporarily slowed down.
Default is 300 seconds.
If you are an early adopter, maybe this value is left as 30 seconds. Please change this value to larger values.
Note: If you want to use "Use history", this vault must be set to 0.
### Manual Garbage Collect
Run "Garbage Collection" manually.
### Fetch rebuilt DB.
If one device rebuilds or locks the remote database, every other device will be locked out from the remote database until it fetches rebuilt DB.
### minimum chunk size and LongLine threshold
The configuration of chunk splitting.
Self-hosted LiveSync splits the note into chunks for efficient synchronization. This chunk should be longer than "Minimum chunk size".
Self-hosted LiveSync splits the note into chunks for efficient synchronization. This chunk should be longer than the "Minimum chunk size".
Specifically, the length of the chunk is determined by the following orders.
1. Find the nearest newline character, and if it is farther than LongLineThreshold, this piece becomes an independent chunk.
2. If not, find nearest to these items.
1. Newline character
2. Empty line (Windows style)
3. Empty line (non-Windows style)
3. Compare the farther in these 3 positions and next "\[newline\]#" position, pick a shorter piece to as chunk.
2. If not, find the nearest to these items.
1. A newline character
2. An empty line (Windows style)
3. An empty line (non-Windows style)
3. Compare the farther in these 3 positions and the next "newline\]#" position, and pick a shorter piece as a chunk.
This rule was made empirically from my dataset. If this rule acts as badly on your data. Please give me the information.
You can dump saved note structure to `Dump informations of this doc`. Replace every character to x except newline and "#" when sending information to me.
You can dump saved note structure to `Dump informations of this doc`. Replace every character with x except newline and "#" when sending information to me.
Default values are 20 letters and 250 letters.
The default values are 20 letters and 250 letters.
## General Settings
@@ -123,7 +106,7 @@ If you enable this option, log only the entries with the popup.
### Verbose log
## Sync setting
## Sync Settings
### LiveSync
Do LiveSync.
@@ -160,6 +143,24 @@ Self-hosted LiveSync will delete the folder when the folder becomes empty. If th
### Use newer file if conflicted (beta)
Always use the newer file to resolve and overwrite when conflict has occurred.
### Experimental.
### Sync hidden files
Synchronize hidden files.
- Scan hidden files before replication.
If you enable this option, all hidden files are scanned once before replication.
- Scan hidden files periodicaly.
If you enable this option, all hidden files will be scanned each [n] seconds.
Hidden files are not actively detected, so we need scanning.
Each scan stores the file with their modification time. And if the file has been disappeared, the fact is also stored. Then, When the entry of the hidden file has been replicated, it will be reflected in the storage if the entry is newer than storage.
Therefore, the clock must be adjusted. If the modification time is determined to be older, the changeset will be skipped or cancelled (It means, **deleted**), even if the file spawned in a hidden folder.
### Advanced settings
Self-hosted LiveSync using PouchDB and synchronizes with the remote by [this protocol](https://docs.couchdb.org/en/stable/replication/protocol.html).
So, it splits every entry into chunks to be acceptable by the database with limited payload size and document size.
@@ -212,28 +213,6 @@ You can set synchronization method at once as these pattern:
- Sync on File Open : disabled
- Sync on Start : disabled
### Use history
If you enable this option, you can keep document histories in your database.
(Not all intermediate changes are synchronized.)
You can check the changes caused by your edit and/or replication.
### Enable plugin synchronization
If you want to use this feature, you have to activate this feature by this switch.
### Sweep plugins automatically
Plugin sweep will run before replication automatically.
### Sweep plugins periodically
Plugin sweep will run each 1 minute.
### Notify updates
When replication is complete, a message will be notified if a newer version of the plugin applied to this device is configured on another device.
### Device and Vault name
To save the plugins, you have to set a unique name every each device.
### Open
Open the "Plugins and their settings" dialog.
## Hatch
From here, everything is under the hood. Please handle it with care.
@@ -255,36 +234,6 @@ But, there's no problem even if you leave it as it is.
### Verify and repair all files
read all files in the vault, and update them into the database if there's diff or could not read from the database.
### Sanity check
Make sure that all the files on the local database have all chunks.
### Drop history
Drop all histories on the local database and the remote database, and initialize When synchronization time has been prolonged to the new device or new vault, or database size became to be much larger. Try this.
Note: When CouchDB deletes entries, to merge confliction, there left old entries as deleted data before compaction. After compaction has been run, deleted data are become "tombstone". "tombstone" uses less disk, But still use some.
It's the specification, to shrink database size from the root, re-initialization is required, even it's explicit or implicit.
Same as a setting passphrase, database locking is also performed.
- Drop and send (Same as "Apply and send")
1. Initialize the Local Database and set (or clear) passphrase, put all files into the database again.
2. Initialize the Remote Database.
3. Lock the Remote Database.
4. Send it all.
- Drop and receive (Same as "Apply and receive")
1. Initialize the Local Database and set (or clear) the passphrase.
2. Unlock the Remote Database.
3. Retrieve all and decrypt to file.
### Lock remote database
Lock the remote database to ban out other devices for synchronization. It is the same as the database lock that happened in dropping databases or applying passphrases.
Use it as an emergency evacuation method to protect local or remote databases when synchronization has been broken.
### Suspend file watching
If enable this option, Self-hosted LiveSync dismisses every file change or deletes the event.
@@ -292,18 +241,27 @@ From here, these commands are used inside applying encryption passphrases or dro
Usually, doesn't use it so much. But sometimes it could be handy.
### Reset remote database
Discard the data stored in the remote database.
## Plugins and settings (beta)
### Reset local database
Discard the data stored in the local database.
### Enable plugin synchronization
If you want to use this feature, you have to activate this feature by this switch.
### Initialize local database again
Discard the data stored in the local database and initialize and create the database from the files on storage.
### Sweep plugins automatically
Plugin sweep will run before replication automatically.
### Corrupted data
### Sweep plugins periodically
Plugin sweep will run each 1 minute.
### Notify updates
When replication is complete, a message will be notified if a newer version of the plugin applied to this device is configured on another device.
### Device and Vault name
To save the plugins, you have to set a unique name every each device.
### Open
Open the "Plugins and their settings" dialog.
### Corrupted or missing data
![CorruptedData](../images/corrupted_data.png)
When Self-hosted LiveSync could not write to the file on the storage, the files are shown here. If you have the old data in your vault, change it once, it will be cured. Or you can use the "File History" plugin.
But if you don't, sorry, we can't rescue the file, and error messages are shown frequently, and you have to delete the file from here.

64
docs/settings_ja.md
View File

@@ -70,22 +70,6 @@ End to End 暗号化を行うに当たって、異なるパスフレーズで暗
- ファイルの修正以外のファイル関連イベント
このオプションはLiveSyncと同時には使用できません。
### Auto Garbage Collection delay
Self-hosted LiveSyncはノートの変更時、ノートをmarkdownの構造を鑑みてチャンクに分割し、ファイルの情報と更新があったチャンクのみ保存していきます。この際、古いチャンクの削除は行いません。
そのため、使わなくなったチャンクをどこかのタイミングで消去する必要があります。
ただし、このチャンクはチャンクの内容から作成されるため、同一の内容からは同一のチャンクが作成され、同じノートだけではなく、すべてのノートから共有されます。これによってデータベースの使用容量とデバイス‐サーバー間での転送量を削減しています。
執筆を繰り返す上で、元の文書に戻したりすることもあるため、一概に「すぐに不要になる」とは言い切れません。そこで、プラグインはObsidianを開いたまま操作しなくなってからこの設定値秒後、まとめて使用していないチャンクを削除します。
この処理をGarbage Collectionと呼んでいます。
この作業はすべてのファイル変更の反映を停止して一気に行う必要があります。そのため、一時的にObsidianの動作がかなり重くなります。
Obsidianでのファイル操作が終わってから指定秒数が経過した際に実行されます。
デフォルト値は300秒です。
※ごく初期は30秒でした。初期から使用されている方は、是非300秒ぐらいまで伸ばしてください。ストレスが違います。
### Manual Garbage Collect
上記のGarbage Collectionを手動で行います。
### minimum chunk size と LongLine threshold
チャンクの分割についての設定です。
Self-hosted LiveSyncは一つのチャンクのサイズを最低minimum chunk size文字確保した上で、できるだけ効率的に同期できるよう、ノートを分割してチャンクを作成します。
@@ -150,6 +134,30 @@ Self-hosted LiveSyncは通常、フォルダ内のファイルがすべて削除
### Use newer file if conflicted (beta)
競合が発生したとき、常に新しいファイルを使用して競合を自動的に解決します。
### Experimental.
### Sync hidden files
隠しファイルを同期します
- Scan hidden files before replication.
このオプション有効にすると、レプリケーションを実行する前に隠しファイルをスキャンします。
- Scan hidden files periodicaly.
このオプションを有効にすると、n秒おきに隠しファイルをスキャンします。
隠しファイルは能動的に検出されないため、スキャンが必要です。
スキャンでは、ファイルと共にファイルの変更時刻を保存します。もしファイルが消された場合は、その事実も保存します。このファイルを記録したエントリーがレプリケーションされた際、ストレージよりも新しい場合はストレージに反映されます。
そのため、端末のクロックは時刻合わせされている必要があります。ファイルが隠しフォルダに生成された場合でも、もし変更時刻が古いと判断された場合はスキップされるかキャンセル（つまり、削除）されます。
Each scan stores the file with their modification time. And if the file has been disappeared, the fact is also stored. Then, When the entry of the hidden file has been replicated, it will be reflected in the storage if the entry is newer than storage.
Therefore, the clock must be adjusted. If the modification time is old, the changeset will be skipped or cancelled (It means, **deleted**), even if the file spawned in a hidden folder.
### Advanced settings
Self-hosted LiveSyncはPouchDBを使用し、リモートと[このプロトコル](https://docs.couchdb.org/en/stable/replication/protocol.html)で同期しています。
そのため、全てのノートなどはデータベースが許容するペイロードサイズやドキュメントサイズに併せてチャンクに分割されています。
@@ -199,16 +207,6 @@ Self-hosted LiveSyncはPouchDBを使用し、リモートと[このプロトコ
### Verify and repair all files
Vault内のファイルを全て読み込み直し、もし差分があったり、データベースから正常に読み込めなかったものに関して、データベースに反映します。
### Sanity check
ローカルデータベースに保存されている全てのファイルが正しくチャンクを持っていることを確認します。
### Drop history
データベースに記録されている履歴を削除し、データベースを初期化します。
新しい端末や新しいVaultへの同期にやたらと時間がかかったり、データベースサイズが肥大化したりしてきた際に使用してください。
備考:CouchDBは、データを削除する際、衝突の解決のために、削除した痕跡を保存します。そのため、Garbage Collectしていたとしても、データは必ず増え続けます。
パスフレーズ設定と同様に、完全に同期されているのであれば、データを失う可能性は低いです。
また、同様にデータベースのロック等の処理も行われます。
- Drop and send
デバイスとリモートのデータベースを破棄し、ロックしてからデバイスのファイルでデータベースを構築後、リモートに上書きします。
- Drop and receive
@@ -222,20 +220,6 @@ Vault内のファイルを全て読み込み直し、もし差分があったり
### Suspend file watching
ファイルの更新の監視を止めます。
これ以降の操作は、暗号化設定のApplyや、Drop Historyで行われる処理を手動で行うためのオプションです。
あまり使用することはありませんがいざというときに使用します。
### reset remote database
リモートのデータベースを破棄します。
### reset local database
ローカルのデータベースを破棄します。
### initialize local database again
デバイスのデータベースを破棄し、実ファイルから再度データベースを構築します。
### Corrupted data
![CorruptedData](../images/corrupted_data.png)

2
docs/setup_cloudant.md
View File

@@ -71,7 +71,7 @@ In these instructions, create IBM Cloudant Instance for trial.
follow the figure, it's
"apikey-v2-2unu15184f7o8emr90xlqgkm2ncwhbltml6tgnjl9sd5"<sup>(\*3)</sup> and "c2c11651d75497fa3d3c486e4c8bdf27"<sup>(\*4)</sup>
## Self-hosted LiveSync setting
## Self-hosted LiveSync settings
![Setting](../images/remote_db_setting.png)

22
docs/setup_own_server.md
View File

@@ -1,7 +1,7 @@
# Setup CouchDB to your server
## Install CouchDB and access from PC or Mac
## Install CouchDB and access from a PC or Mac
The easiest way to set up the CouchDB is using the [docker image]((https://hub.docker.com/_/couchdb)).
@@ -10,9 +10,11 @@ But some additional configurations are required in `local.ini` to use from Self-
```
[couchdb]
single_node=true
max_document_size = 50000000
[chttpd]
require_valid_user = true
max_http_request_size = 4294967296
[chttpd_auth]
require_valid_user = true
@@ -32,21 +34,23 @@ max_age = 3600
Make `local.ini` and run with docker run like this, you can launch the CouchDB.
```
$ docker run --rm -it -e COUCHDB_USER=admin -e COUCHDB_PASSWORD=password -v .local.ini:/opt/couchdb/etc/local.ini -p 5984:5984 couchdb
$ docker run --rm -it -e COUCHDB_USER=admin -e COUCHDB_PASSWORD=password -v /path/to/local.ini:/opt/couchdb/etc/local.ini -p 5984:5984 couchdb
```
*Remember to replace the path with the path to your local.ini*
Note: At this time, the file owner of local.ini became 5984:5984. It's the limitation docker image. please change the owner before editing local.ini again.
If you could confirm that Self-hosted LiveSync can sync with the server, launch docker image as background as you like.
If you could confirm that Self-hosted LiveSync can sync with the server, launch the docker image as a background as you like.
example)
Example to run docker in detached mode:
```
$ docker run -d --restart always -e COUCHDB_USER=admin -e COUCHDB_PASSWORD=password -v .local.ini:/opt/couchdb/etc/local.ini -p 5984:5984 couchdb
$ docker run -d --restart always -e COUCHDB_USER=admin -e COUCHDB_PASSWORD=password -v /path/to/local.ini:/opt/couchdb/etc/local.ini -p 5984:5984 couchdb
```
*Remember to replace the path with the path to your local.ini*
## Access from mobile device
## Access from a mobile device
If you want to access Self-hosted LiveSync from mobile devices, you need a valid SSL certificate.
### Testing from mobile
### Testing from a mobile
In the testing phase, [localhost.run](http://localhost.run/) or something like services is very useful.
example on using localhost.run)
@@ -90,6 +94,6 @@ Set the A record of your domain to point to your server, and host reverse proxy
Note: Mounting CouchDB on the top directory is not recommended.
Using Caddy is a handy way to serve the server with SSL automatically.
I have published [docker-compose.yml and ini files](https://github.com/vrtmrz/self-hosted-livesync-server) that launches Caddy and CouchDB at once. Please try it out.
I have published [docker-compose.yml and ini files](https://github.com/vrtmrz/self-hosted-livesync-server) that launch Caddy and CouchDB at once. Please try it out.
And, be sure to check the server log and be careful of malicious access.
And, be sure to check the server log and be careful of malicious access.

97
docs/setup_own_server_cn.md Normal file
View File

@@ -0,0 +1,97 @@
# 在你自己的服务器上设置 CouchDB
> 注：提供了 [docker-compose.yml 和 ini 文件](https://github.com/vrtmrz/self-hosted-livesync-server) 可以同时启动 Caddy 和 CouchDB。推荐直接使用该 docker-compose 配置进行搭建。（若使用，请查阅链接中的文档，而不是这个文档）
## 安装 CouchDB 并从 PC 或 Mac 上访问
设置 CouchDB 的最简单方法是使用 [CouchDB docker image]((https://hub.docker.com/_/couchdb)).
需要修改一些 `local.ini` 中的配置，以让它可以用于 Self-hosted LiveSync，如下：
```
[couchdb]
single_node=true
max_document_size = 50000000
[chttpd]
require_valid_user = true
max_http_request_size = 4294967296
[chttpd_auth]
require_valid_user = true
authentication_redirect = /_utils/session.html
[httpd]
WWW-Authenticate = Basic realm="couchdb"
enable_cors = true
[cors]
origins = app://obsidian.md,capacitor://localhost,http://localhost
credentials = true
headers = accept, authorization, content-type, origin, referer
methods = GET, PUT, POST, HEAD, DELETE
max_age = 3600
```
创建 `local.ini` 并用如下指令启动 CouchDB：
```
$ docker run --rm -it -e COUCHDB_USER=admin -e COUCHDB_PASSWORD=password -v .local.ini:/opt/couchdb/etc/local.ini -p 5984:5984 couchdb
```
Note: 此时 local.ini 的文件所有者会变成 5984:5984。这是 docker 镜像的限制，请修改文件所有者后再编辑 local.ini。
在确定 Self-hosted LiveSync 可以和服务器同步后，可以后台启动 docker 镜像：
```
$ docker run -d --restart always -e COUCHDB_USER=admin -e COUCHDB_PASSWORD=password -v .local.ini:/opt/couchdb/etc/local.ini -p 5984:5984 couchdb
```
## 从移动设备访问
如果你想要从移动设备访问 Self-hosted LiveSync，你需要一个合法的 SSL 证书。
### 移动设备测试
测试时，[localhost.run](http://localhost.run/) 这一类的反向隧道服务很实用。（非必须，只是用于终端设备不方便 ssh 的时候的备选方案）
```
$ ssh -R 80:localhost:5984 nokey@localhost.run
Warning: Permanently added the RSA host key for IP address '35.171.254.69' to the list of known hosts.
===============================================================================
Welcome to localhost.run!
Follow your favourite reverse tunnel at [https://twitter.com/localhost_run].
**You need a SSH key to access this service.**
If you get a permission denied follow Gitlab's most excellent howto:
https://docs.gitlab.com/ee/ssh/
*Only rsa and ed25519 keys are supported*
To set up and manage custom domains go to https://admin.localhost.run/
More details on custom domains (and how to enable subdomains of your custom
domain) at https://localhost.run/docs/custom-domains
To explore using localhost.run visit the documentation site:
https://localhost.run/docs/
===============================================================================
** your connection id is xxxxxxxxxxxxxxxxxxxxxxxxxxxx, please mention it if you send me a message about an issue. **
xxxxxxxx.localhost.run tunneled with tls termination, https://xxxxxxxx.localhost.run
Connection to localhost.run closed by remote host.
Connection to localhost.run closed.
```
https://xxxxxxxx.localhost.run 即为临时服务器地址。
### 设置你的域名
设置一个指向你服务器的 A 记录，并根据需要设置反向代理。
Note: 不推荐将 CouchDB 挂载到根目录
可以使用 Caddy 很方便的给服务器加上 SSL 功能
提供了 [docker-compose.yml 和 ini 文件](https://github.com/vrtmrz/self-hosted-livesync-server) 可以同时启动 Caddy 和 CouchDB。
注意检查服务器日志，当心恶意访问。

2
docs/setup_own_server_ja.md
View File

@@ -8,12 +8,14 @@ CouchDBを構築するには、[Dockerのイメージ](https://hub.docker.com/_/
```
[couchdb]
single_node=true
max_document_size = 50000000
[chttpd]
require_valid_user = true
[chttpd_auth]
require_valid_user = true
max_http_request_size = 4294967296
authentication_redirect = /_utils/session.html
[httpd]

4
docs/tech_info.md
View File

@@ -9,8 +9,8 @@
3. Another device is watching remote CouchDB's changes, so retrieve new changes.
4. Self-hosted LiveSync reflects replicated changeset into Obsidian's vault.
Note: The figure is drawn as single-directional, between two devices. But everything occurs bi-directionally between many devices at once in real.
Note: The figure is drawn as single-directional, between two devices for demonstration purposes. Everything actually occurs bi-directionally between many devices at the same time.
## Techniques to keep bandwidth low.
## Techniques to keep bandwidth consumption low.
![dedupe](../images/2.png)

17
esbuild.config.mjs
View File

@@ -3,7 +3,7 @@ import process from "process";
import builtins from "builtin-modules";
import sveltePlugin from "esbuild-svelte";
import sveltePreprocess from "svelte-preprocess";
import fs from "node:fs";
const banner = `/*
THIS IS A GENERATED/BUNDLED FILE BY ESBUILD
if you want to view the source, please visit the github repository of this plugin
@@ -11,7 +11,9 @@ if you want to view the source, please visit the github repository of this plugi
`;
const prod = process.argv[2] === "production";
const manifestJson = JSON.parse(fs.readFileSync("./manifest.json"));
const packageJson = JSON.parse(fs.readFileSync("./package.json"));
const updateInfo = JSON.stringify(fs.readFileSync("./updates.md") + "");
esbuild
.build({
banner: {
@@ -19,13 +21,20 @@ esbuild
},
entryPoints: ["src/main.ts"],
bundle: true,
external: ["obsidian", "electron", ...builtins],
define: {
"MANIFEST_VERSION": `"${manifestJson.version}"`,
"PACKAGE_VERSION": `"${packageJson.version}"`,
"UPDATE_INFO": `${updateInfo}`,
"global":"window",
},
external: ["obsidian", "electron", "crypto"],
format: "cjs",
watch: !prod,
target: "es2015",
target: "es2018",
logLevel: "info",
sourcemap: prod ? false : "inline",
treeShaking: true,
platform: "browser",
plugins: [
sveltePlugin({
preprocess: sveltePreprocess(),

BIN
images/quick_setup_1.png Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 39 KiB

BIN
images/quick_setup_10.png Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 13 KiB

BIN
images/quick_setup_2.png Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 16 KiB

BIN
images/quick_setup_3.png Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 24 KiB

BIN
images/quick_setup_4.png Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 31 KiB

BIN
images/quick_setup_5.png Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 36 KiB

BIN
images/quick_setup_6.png Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 163 KiB

BIN
images/quick_setup_7.png Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 10 KiB

BIN
images/quick_setup_8.png Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 32 KiB

BIN
images/quick_setup_9_1.png Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 35 KiB

BIN
images/quick_setup_9_2.png Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 1.7 KiB

4
manifest.json
View File

@@ -1,10 +1,10 @@
{
"id": "obsidian-livesync",
"name": "Self-hosted LiveSync",
"version": "0.11.2",
"version": "0.19.0",
"minAppVersion": "0.9.12",
"description": "Community implementation of self-hosted livesync. Reflect your vault changes to some other devices immediately. Please make sure to disable other synchronize solutions to avoid content corruption or duplication.",
"author": "vorotamoroz",
"authorUrl": "https://github.com/vrtmrz",
"isDesktopOnly": false
}
}

4810
package-lock.json generated
View File

File diff suppressed because it is too large Load Diff

52
package.json
View File

@@ -1,6 +1,6 @@
{
"name": "obsidian-livesync",
"version": "0.11.2",
"version": "0.19.0",
"description": "Reflect your vault changes to some other devices immediately. Please make sure to disable other synchronize solutions to avoid content corruption or duplication.",
"main": "main.js",
"type": "module",
@@ -13,32 +13,40 @@
"author": "vorotamoroz",
"license": "MIT",
"devDependencies": {
"@rollup/plugin-commonjs": "^18.0.0",
"@rollup/plugin-node-resolve": "^11.2.1",
"@rollup/plugin-typescript": "^8.2.1",
"@types/diff-match-patch": "^1.0.32",
"@types/pouchdb": "^6.4.0",
"@types/pouchdb-browser": "^6.1.3",
"@typescript-eslint/eslint-plugin": "^5.7.0",
"@typescript-eslint/parser": "^5.0.0",
"builtin-modules": "^3.2.0",
"esbuild": "0.13.12",
"esbuild-svelte": "^0.6.0",
"eslint": "^7.32.0",
"eslint-config-airbnb-base": "^14.2.1",
"eslint-plugin-import": "^2.25.2",
"obsidian": "^0.14.6",
"rollup": "^2.32.1",
"svelte-preprocess": "^4.10.2",
"tslib": "^2.2.0",
"typescript": "^4.2.4"
"@typescript-eslint/eslint-plugin": "^5.54.0",
"@typescript-eslint/parser": "^5.54.0",
"builtin-modules": "^3.3.0",
"esbuild": "0.15.15",
"esbuild-svelte": "^0.7.3",
"eslint": "^8.35.0",
"eslint-config-airbnb-base": "^15.0.0",
"eslint-plugin-import": "^2.27.5",
"events": "^3.3.0",
"obsidian": "^1.1.1",
"postcss": "^8.4.21",
"postcss-load-config": "^4.0.1",
"pouchdb-adapter-http": "^8.0.1",
"pouchdb-adapter-idb": "^8.0.1",
"pouchdb-adapter-indexeddb": "^8.0.1",
"pouchdb-core": "^8.0.1",
"pouchdb-find": "^8.0.1",
"pouchdb-mapreduce": "^8.0.1",
"pouchdb-replication": "^8.0.1",
"pouchdb-utils": "^8.0.1",
"svelte": "^3.55.1",
"svelte-preprocess": "^5.0.1",
"transform-pouch": "^2.0.0",
"tslib": "^2.5.0",
"typescript": "^4.9.5"
},
"dependencies": {
"diff-match-patch": "^1.0.5",
"esbuild": "0.13.12",
"esbuild-svelte": "^0.6.0",
"idb": "^7.0.1",
"svelte-preprocess": "^4.10.2",
"esbuild": "0.15.15",
"esbuild-svelte": "^0.7.3",
"idb": "^7.1.1",
"xxhash-wasm": "^0.4.2"
}
}
}

1
pouchdb-browser-webpack/.gitignore vendored
View File

@@ -1 +0,0 @@
node_modules

4
pouchdb-browser-webpack/README.md
View File

@@ -1,4 +0,0 @@
# PouchDB-browser
Just webpacked.
(Rollup couldn't pack pouchdb-browser into browser bundle)

1
pouchdb-browser-webpack/dist/pouchdb-browser.js vendored
View File

File diff suppressed because one or more lines are too long

25
pouchdb-browser-webpack/package.json
View File

@@ -1,25 +0,0 @@
{
"name": "pouchdb-browser-webpack",
"version": "1.0.0",
"description": "pouchdb-browser webpack",
"main": "index.js",
"scripts": {
"test": "echo \"Error: no test specified\" && exit 1",
"build": "webpack --mode=production --node-env=production",
"build:dev": "webpack --mode=development",
"build:prod": "webpack --mode=production --node-env=production",
"watch": "webpack --watch"
},
"keywords": [],
"author": "",
"license": "ISC",
"dependencies": {
"pouchdb-browser": "^7.3.0",
"transform-pouch": "^2.0.0",
"pouchdb-find": "^7.3.0"
},
"devDependencies": {
"webpack": "^5.58.1",
"webpack-cli": "^4.9.0"
}
}

10
pouchdb-browser-webpack/src/index.js
View File

@@ -1,10 +0,0 @@
/* eslint-disable no-undef */
/* eslint-disable @typescript-eslint/no-var-requires */
// This module just webpacks pouchdb-browser
// import * as PouchDB_src from "pouchdb-browser";
const pouch = require("pouchdb-browser").default;
const find = require("pouchdb-find").default;
const transform = require("transform-pouch");
const PouchDB = pouch.plugin(find).plugin(transform);
export { PouchDB };

30
pouchdb-browser-webpack/webpack.config.js
View File

@@ -1,30 +0,0 @@
// Generated using webpack-cli https://github.com/webpack/webpack-cli
const path = require("path");
const isProduction = process.env.NODE_ENV == "production";
const config = {
entry: "./src/index.js",
output: {
filename: "pouchdb-browser.js",
path: path.resolve(__dirname, "dist"),
library: {
type: "module",
},
},
experiments: {
outputModule: true,
},
plugins: [],
module: {},
};
module.exports = () => {
if (isProduction) {
config.mode = "production";
} else {
config.mode = "development";
}
return config;
};

31
rollup.config.js
View File

@@ -1,31 +0,0 @@
import typescript from "@rollup/plugin-typescript";
import { nodeResolve } from "@rollup/plugin-node-resolve";
import commonjs from "@rollup/plugin-commonjs";
const isProd = process.env.BUILD === "production";
const banner = `/*
THIS IS A GENERATED/BUNDLED FILE BY ROLLUP
if you want to view the source visit the plugins github repository
*/
`;
export default {
input: "./src/main.ts",
output: {
dir: ".",
sourcemap: "inline",
sourcemapExcludeSources: isProd,
format: "cjs",
exports: "default",
banner,
},
external: ["obsidian"],
plugins: [
typescript({ exclude: ["pouchdb-browser.js", "pouchdb-browser-webpack"] }),
nodeResolve({
browser: true,
}),
commonjs(),
],
};

651
src/CmdConfigSync.ts Normal file
View File

@@ -0,0 +1,651 @@
import { writable } from 'svelte/store';
import { Notice, PluginManifest, stringifyYaml, parseYaml } from "./deps";
import { EntryDoc, LoadedEntry, LOG_LEVEL, InternalFileEntry, FilePathWithPrefix, FilePath, DocumentID } from "./lib/src/types";
import { ICXHeader, PERIODIC_PLUGIN_SWEEP, } from "./types";
import { delay, getDocData } from "./lib/src/utils";
import { Logger } from "./lib/src/logger";
import { PouchDB } from "./lib/src/pouchdb-browser.js";
import { WrappedNotice } from "./lib/src/wrapper";
import { base64ToArrayBuffer, arrayBufferToBase64, readString, writeString, uint8ArrayToHexString } from "./lib/src/strbin";
import { runWithLock } from "./lib/src/lock";
import { LiveSyncCommands } from "./LiveSyncCommands";
import { addPrefix, stripAllPrefixes } from "./lib/src/path";
import { PeriodicProcessor, askYesNo, disposeMemoObject, memoIfNotExist, memoObject, retrieveMemoObject, scheduleTask } from "./utils";
import { Semaphore } from "./lib/src/semaphore";
import { PluginDialogModal } from "./dialogs";
import { JsonResolveModal } from "./JsonResolveModal";
export const pluginList = writable([] as PluginDataExDisplay[]);
export const pluginIsEnumerating = writable(false);
const hashString = (async (key: string) => {
const buff = writeString(key);
const digest = await crypto.subtle.digest('SHA-256', buff);
return uint8ArrayToHexString(new Uint8Array(digest));
})
export type PluginDataExFile = {
filename: string,
data?: string[],
mtime: number,
size: number,
version?: string,
displayName?: string,
}
export type PluginDataExDisplay = {
documentPath: FilePathWithPrefix,
category: string,
name: string,
term: string,
displayName?: string,
files: PluginDataExFile[],
version?: string,
mtime: number,
}
export type PluginDataEx = {
documentPath?: FilePathWithPrefix,
category: string,
name: string,
displayName?: string,
term: string,
files: PluginDataExFile[],
version?: string,
mtime: number,
};
export class ConfigSync extends LiveSyncCommands {
confirmPopup: WrappedNotice = null;
get kvDB() {
return this.plugin.kvDB;
}
ensureDirectoryEx(fullPath: string) {
return this.plugin.ensureDirectoryEx(fullPath);
}
pluginDialog: PluginDialogModal = null;
periodicPluginSweepProcessor = new PeriodicProcessor(this.plugin, async () => await this.scanAllConfigFiles(false));
showPluginSyncModal() {
if (!this.settings.usePluginSync) {
return;
}
if (this.pluginDialog != null) {
this.pluginDialog.open();
} else {
this.pluginDialog = new PluginDialogModal(this.app, this.plugin);
this.pluginDialog.open();
}
}
hidePluginSyncModal() {
if (this.pluginDialog != null) {
this.pluginDialog.close();
this.pluginDialog = null;
}
}
onunload() {
this.hidePluginSyncModal();
this.periodicPluginSweepProcessor?.disable();
}
onload() {
this.plugin.addCommand({
id: "livesync-plugin-dialog-ex",
name: "Show customization sync dialog",
callback: () => {
this.showPluginSyncModal();
},
});
}
getFileCategory(filePath: string): "CONFIG" | "THEME" | "SNIPPET" | "PLUGIN_MAIN" | "PLUGIN_ETC" | "PLUGIN_DATA" | "" {
if (filePath.split("/").length == 2 && filePath.endsWith(".json")) return "CONFIG";
if (filePath.split("/").length == 4 && filePath.startsWith(`${this.app.vault.configDir}/themes/`)) return "THEME";
if (filePath.startsWith(`${this.app.vault.configDir}/snippets/`) && filePath.endsWith(".css")) return "SNIPPET";
if (filePath.startsWith(`${this.app.vault.configDir}/plugins/`)) {
if (filePath.endsWith("/styles.css") || filePath.endsWith("/manifest.json") || filePath.endsWith("/main.js")) {
return "PLUGIN_MAIN";
} else if (filePath.endsWith("/data.json")) {
return "PLUGIN_DATA";
} else {
//TODO: to be configurable.
// With algorithm which implemented at v0.19.0, is too heavy.
return "";
// return "PLUGIN_ETC";
}
// return "PLUGIN";
}
return "";
}
isTargetPath(filePath: string): boolean {
if (!filePath.startsWith(this.app.vault.configDir)) return false;
// Idea non-filter option?
return this.getFileCategory(filePath) != "";
}
async onInitializeDatabase(showNotice: boolean) {
if (this.settings.usePluginSync) {
try {
Logger("Scanning customizations...");
await this.scanAllConfigFiles(showNotice);
Logger("Scanning customizations : done");
} catch (ex) {
Logger("Scanning customizations : failed");
Logger(ex, LOG_LEVEL.VERBOSE);
}
}
}
async beforeReplicate(showNotice: boolean) {
if (this.settings.autoSweepPlugins && this.settings.usePluginSync) {
await this.scanAllConfigFiles(showNotice);
}
}
async onResume() {
if (this.plugin.suspended)
return;
if (this.settings.autoSweepPlugins) {
await this.scanAllConfigFiles(true);
}
this.periodicPluginSweepProcessor.enable(this.settings.autoSweepPluginsPeriodic && !this.settings.watchInternalFileChanges ? (PERIODIC_PLUGIN_SWEEP * 1000) : 0);
}
async reloadPluginList() {
pluginList.set([])
await this.updatePluginList();
}
async updatePluginList(updatedDocumentPath?: FilePathWithPrefix): Promise<void> {
// pluginList.set([]);
if (!this.settings.usePluginSync) {
pluginList.set([]);
return;
}
await runWithLock("update-plugin-list", false, async () => {
if (updatedDocumentPath != "") pluginList.update(e => e.filter(ee => ee.documentPath != updatedDocumentPath));
// const work: Record<string, Record<string, Record<string, Record<string, PluginDataEntryEx>>>> = {};
const entries = [] as PluginDataExDisplay[]
const plugins = this.localDatabase.findEntries(ICXHeader + "", `${ICXHeader}\u{10ffff}`, { include_docs: true });
const semaphore = Semaphore(4);
const processes = [] as Promise<void>[];
let count = 0;
pluginIsEnumerating.set(true);
try {
for await (const plugin of plugins) {
const path = plugin.path || this.getPath(plugin);
if (updatedDocumentPath && updatedDocumentPath != path) {
continue;
}
processes.push((async (v) => {
const release = await semaphore.acquire(1);
try {
Logger(`Enumerating files... ${count++}`, LOG_LEVEL.NOTICE, "get-plugins");
Logger(`plugin-${path}`, LOG_LEVEL.VERBOSE);
const wx = await this.localDatabase.getDBEntry(path, null, false, false);
if (wx) {
const data = parseYaml(getDocData(wx.data)) as PluginDataEx;
const xFiles = [] as PluginDataExFile[];
for (const file of data.files) {
const work = { ...file };
const tempStr = getDocData(work.data);
work.data = [await hashString(tempStr)];
xFiles.push(work);
}
entries.push({
...data,
documentPath: this.getPath(wx),
files: xFiles
});
}
} catch (ex) {
//TODO
Logger(`Something happened at enumerating customization :${v.path}`);
console.warn(ex);
} finally {
release();
}
}
)(plugin));
}
await Promise.all(processes);
pluginList.update(e => {
let newList = [...e];
for (const item of entries) {
console.log(item.documentPath);
newList = newList.filter(x => x.documentPath != item.documentPath);
newList.push(item)
}
return newList;
})
Logger(`All files enumerated`, LOG_LEVEL.NOTICE, "get-plugins");
} finally {
pluginIsEnumerating.set(false);
}
});
// return entries;
}
async compareUsingDisplayData(dataA: PluginDataExDisplay, dataB: PluginDataExDisplay) {
const docA = await this.localDatabase.getDBEntry(dataA.documentPath);
const docB = await this.localDatabase.getDBEntry(dataB.documentPath);
if (docA && docB) {
const pluginDataA = parseYaml(getDocData(docA.data)) as PluginDataEx;
pluginDataA.documentPath = dataA.documentPath;
const pluginDataB = parseYaml(getDocData(docB.data)) as PluginDataEx;
pluginDataB.documentPath = dataB.documentPath;
// Use outer structure to wrap each data.
return await this.showJSONMergeDialogAndMerge(docA, docB, pluginDataA, pluginDataB);
}
return false;
}
showJSONMergeDialogAndMerge(docA: LoadedEntry, docB: LoadedEntry, pluginDataA: PluginDataEx, pluginDataB: PluginDataEx): Promise<boolean> {
const fileA = { ...pluginDataA.files[0], ctime: pluginDataA.files[0].mtime, _id: `${pluginDataA.documentPath}` as DocumentID };
const fileB = pluginDataB.files[0];
const docAx = { ...docA, ...fileA } as LoadedEntry, docBx = { ...docB, ...fileB } as LoadedEntry
return runWithLock("config:merge-data", false, () => new Promise((res) => {
Logger("Opening data-merging dialog", LOG_LEVEL.VERBOSE);
// const docs = [docA, docB];
const path = stripAllPrefixes(docAx.path.split("/").slice(-1).join("/") as FilePath);
const modal = new JsonResolveModal(this.app, path, [docAx, docBx], async (keep, result) => {
if (result == null) return res(false);
try {
res(await this.applyData(pluginDataA, result));
} catch (ex) {
Logger("Could not apply merged file");
Logger(ex, LOG_LEVEL.VERBOSE);
res(false);
}
}, "📡", "🛰️", "B");
modal.open();
}));
}
async applyData(data: PluginDataEx, content?: string): Promise<boolean> {
Logger(`Applying ${data.displayName || data.name}..`);
const baseDir = this.app.vault.configDir;
try {
if (!data.documentPath) throw "InternalError: Document path not exist";
const dx = await this.localDatabase.getDBEntry(data.documentPath);
if (dx == false) {
throw "Not found on database"
}
const loadedData = parseYaml(getDocData(dx.data)) as PluginDataEx;
for (const f of loadedData.files) {
Logger(`Applying ${f.filename} of ${data.displayName || data.name}..`);
try {
// console.dir(f);
const path = `${baseDir}/${f.filename}`;
await this.ensureDirectoryEx(path);
if (!content) {
const dt = base64ToArrayBuffer(f.data);
await this.app.vault.adapter.writeBinary(path, dt);
} else {
await this.app.vault.adapter.write(path, content);
}
Logger(`Applying ${f.filename} of ${data.displayName || data.name}.. Done`);
} catch (ex) {
Logger(`Applying ${f.filename} of ${data.displayName || data.name}.. Failed`);
Logger(ex, LOG_LEVEL.VERBOSE);
}
}
const uPath = `${baseDir}/${loadedData.files[0].filename}` as FilePath;
await this.storeCustomizationFiles(uPath);
await this.updatePluginList(uPath);
await delay(100);
Logger(`Config ${data.displayName || data.name} has been applied`, LOG_LEVEL.NOTICE);
if (data.category == "PLUGIN_DATA" || data.category == "PLUGIN_MAIN") {
//@ts-ignore
const manifests = Object.values(this.app.plugins.manifests) as any as PluginManifest[];
//@ts-ignore
const enabledPlugins = this.app.plugins.enabledPlugins as Set<string>;
const pluginManifest = manifests.find((manifest) => enabledPlugins.has(manifest.id) && manifest.dir == `${baseDir}/plugins/${data.name}`);
if (pluginManifest) {
Logger(`Unloading plugin: ${pluginManifest.name}`, LOG_LEVEL.NOTICE, "plugin-reload-" + pluginManifest.id);
// @ts-ignore
await this.app.plugins.unloadPlugin(pluginManifest.id);
// @ts-ignore
await this.app.plugins.loadPlugin(pluginManifest.id);
Logger(`Plugin reloaded: ${pluginManifest.name}`, LOG_LEVEL.NOTICE, "plugin-reload-" + pluginManifest.id);
}
} else if (data.category == "CONFIG") {
scheduleTask("configReload", 250, async () => {
if (await askYesNo(this.app, "Do you want to restart and reload Obsidian now?") == "yes") {
// @ts-ignore
this.app.commands.executeCommandById("app:reload")
}
})
}
return true;
} catch (ex) {
Logger(`Applying ${data.displayName || data.name}.. Failed`);
Logger(ex, LOG_LEVEL.VERBOSE);
return false;
}
}
async deleteData(data: PluginDataEx): Promise<boolean> {
try {
if (data.documentPath) {
await this.deleteConfigOnDatabase(data.documentPath);
Logger(`Delete: ${data.documentPath}`, LOG_LEVEL.NOTICE);
}
return true;
} catch (ex) {
Logger(`Failed to delete: ${data.documentPath}`, LOG_LEVEL.NOTICE);
return false;
}
}
parseReplicationResultItem(docs: PouchDB.Core.ExistingDocument<EntryDoc>) {
if (docs._id.startsWith(ICXHeader)) {
if (this.plugin.settings.usePluginSync && this.plugin.settings.notifyPluginOrSettingUpdated) {
if (!this.pluginDialog || (this.pluginDialog && !this.pluginDialog.isOpened())) {
const fragment = createFragment((doc) => {
doc.createEl("span", null, (a) => {
a.appendText(`Some configuration has been arrived, Press `);
a.appendChild(a.createEl("a", null, (anchor) => {
anchor.text = "HERE";
anchor.addEventListener("click", async () => {
this.showPluginSyncModal();
});
}));
a.appendText(` to open the config sync dialog , or press elsewhere to dismiss this message.`);
});
});
const updatedPluginKey = "popupUpdated-plugins";
scheduleTask(updatedPluginKey, 1000, async () => {
const popup = await memoIfNotExist(updatedPluginKey, () => new Notice(fragment, 0));
//@ts-ignore
const isShown = popup?.noticeEl?.isShown();
if (!isShown) {
memoObject(updatedPluginKey, new Notice(fragment, 0));
}
scheduleTask(updatedPluginKey + "-close", 20000, () => {
const popup = retrieveMemoObject<Notice>(updatedPluginKey);
if (!popup)
return;
//@ts-ignore
if (popup?.noticeEl?.isShown()) {
popup.hide();
}
disposeMemoObject(updatedPluginKey);
});
});
} else {
this.updatePluginList(docs.path ? docs.path : this.getPath(docs));
}
}
return true;
}
return false;
}
async realizeSettingSyncMode(): Promise<void> {
this.periodicPluginSweepProcessor?.disable();
if (this.plugin.suspended)
return;
if (!this.settings.usePluginSync) {
return;
}
if (this.settings.autoSweepPlugins) {
await this.scanAllConfigFiles(false);
}
this.periodicPluginSweepProcessor.enable(this.settings.autoSweepPluginsPeriodic && !this.settings.watchInternalFileChanges ? (PERIODIC_PLUGIN_SWEEP * 1000) : 0);
return;
}
recentProcessedInternalFiles = [] as string[];
async makeEntryFromFile(path: FilePath): Promise<false | PluginDataExFile> {
const stat = await this.app.vault.adapter.stat(path);
let version: string | undefined;
let displayName: string | undefined;
if (!stat) {
return false;
}
const contentBin = await this.app.vault.adapter.readBinary(path);
let content: string[];
try {
content = await arrayBufferToBase64(contentBin);
if (path.toLowerCase().endsWith("/manifest.json")) {
const v = readString(new Uint8Array(contentBin));
try {
const json = JSON.parse(v);
if ("version" in json) {
version = `${json.version}`;
}
if ("name" in json) {
displayName = `${json.name}`;
}
} catch (ex) {
Logger(`Configuration sync data: ${path} looks like manifest, but could not read the version`, LOG_LEVEL.INFO);
}
}
} catch (ex) {
Logger(`The file ${path} could not be encoded`);
Logger(ex, LOG_LEVEL.VERBOSE);
return false;
}
const mtime = stat.mtime;
return {
filename: path.substring(this.app.vault.configDir.length + 1),
data: content,
mtime,
size: stat.size,
version,
displayName: displayName,
}
}
filenameToUnifiedKey(path: string, termOverRide?: string) {
const term = termOverRide || this.plugin.deviceAndVaultName;
const category = this.getFileCategory(path);
const name = (category == "CONFIG" || category == "SNIPPET") ?
(path.split("/").slice(-1)[0]) :
(category == "PLUGIN_ETC" ?
path.split("/").slice(-2).join("/") :
path.split("/").slice(-2)[0]);
return `${ICXHeader}${term}/${category}/${name}.md` as FilePathWithPrefix
}
async storeCustomizationFiles(path: FilePath, termOverRide?: string) {
const term = termOverRide || this.plugin.deviceAndVaultName;
const vf = this.filenameToUnifiedKey(path, term);
return await runWithLock(`plugin-${vf}`, false, async () => {
const category = this.getFileCategory(path);
let mtime = 0;
let fileTargets = [] as FilePath[];
// let savePath = "";
const name = (category == "CONFIG" || category == "SNIPPET") ?
(path.split("/").reverse()[0]) :
(path.split("/").reverse()[1]);
const parentPath = path.split("/").slice(0, -1).join("/");
const prefixedFileName = this.filenameToUnifiedKey(path, term);
const id = await this.path2id(prefixedFileName);
const dt: PluginDataEx = {
category: category,
files: [],
name: name,
mtime: 0,
term: term
}
// let scheduleKey = "";
if (category == "CONFIG" || category == "SNIPPET" || category == "PLUGIN_ETC" || category == "PLUGIN_DATA") {
fileTargets = [path];
if (category == "PLUGIN_ETC") {
dt.displayName = path.split("/").slice(-1).join("/");
}
} else if (category == "PLUGIN_MAIN") {
fileTargets = ["manifest.json", "main.js", "styles.css"].map(e => `${parentPath}/${e}` as FilePath);
} else if (category == "THEME") {
fileTargets = ["manifest.json", "theme.css"].map(e => `${parentPath}/${e}` as FilePath);
}
for (const target of fileTargets) {
const data = await this.makeEntryFromFile(target);
if (data == false) {
Logger(`Config: skipped: ${target} `, LOG_LEVEL.VERBOSE);
continue;
}
if (data.version) {
dt.version = data.version;
}
if (data.displayName) {
dt.displayName = data.displayName;
}
// Use average for total modified time.
mtime = mtime == 0 ? data.mtime : ((data.mtime + mtime) / 2);
dt.files.push(data);
}
dt.mtime = mtime;
// Logger(`Configuration saving: ${prefixedFileName}`);
if (dt.files.length == 0) {
Logger(`Nothing left: deleting.. ${path}`);
return await this.deleteConfigOnDatabase(prefixedFileName);
}
const content = stringifyYaml(dt);
try {
const old = await this.localDatabase.getDBEntryMeta(prefixedFileName, null, false);
let saveData: LoadedEntry;
if (old === false) {
saveData = {
_id: id,
path: prefixedFileName,
data: content,
mtime,
ctime: mtime,
datatype: "newnote",
size: content.length,
children: [],
deleted: false,
type: "newnote",
};
} else {
if (old.mtime == mtime) {
// Logger(`STORAGE --> DB:${file.path}: (hidden) Not changed`, LOG_LEVEL.VERBOSE);
return true;
}
saveData =
{
...old,
data: content,
mtime,
size: content.length,
datatype: "newnote",
children: [],
deleted: false,
type: "newnote",
};
}
const ret = await this.localDatabase.putDBEntry(saveData);
Logger(`STORAGE --> DB:${prefixedFileName}: (config) Done`);
return ret;
} catch (ex) {
Logger(`STORAGE --> DB:${prefixedFileName}: (config) Failed`);
Logger(ex, LOG_LEVEL.VERBOSE);
return false;
}
})
// })
}
async watchVaultRawEventsAsync(path: FilePath) {
if (!this.isTargetPath(path)) return false;
const stat = await this.app.vault.adapter.stat(path);
// Make sure that target is a file.
if (stat && stat.type != "file")
return false;
const storageMTime = ~~((stat && stat.mtime || 0) / 1000);
const key = `${path}-${storageMTime}`;
if (this.recentProcessedInternalFiles.contains(key)) {
// If recently processed, it may caused by self.
return true;
}
this.recentProcessedInternalFiles = [key, ...this.recentProcessedInternalFiles].slice(0, 100);
this.storeCustomizationFiles(path).then(() => {/* Fire and forget */ });
}
async scanAllConfigFiles(showMessage: boolean) {
const logLevel = showMessage ? LOG_LEVEL.NOTICE : LOG_LEVEL.INFO;
Logger("Scanning customizing files.", logLevel, "scan-all-config");
const term = this.plugin.deviceAndVaultName;
if (term == "") {
Logger("We have to configure the device name", LOG_LEVEL.NOTICE);
return;
}
const filesAll = await this.scanInternalFiles();
const files = filesAll.filter(e => this.isTargetPath(e)).map(e => ({ key: this.filenameToUnifiedKey(e), file: e }));
const virtualPathsOfLocalFiles = [...new Set(files.map(e => e.key))];
const filesOnDB = ((await this.localDatabase.allDocsRaw({ startkey: ICXHeader + "", endkey: `${ICXHeader}\u{10ffff}`, include_docs: true })).rows.map(e => e.doc) as InternalFileEntry[]).filter(e => !e.deleted);
let deleteCandidate = filesOnDB.map(e => this.getPath(e)).filter(e => e.startsWith(`${ICXHeader}${term}/`));
for (const vp of virtualPathsOfLocalFiles) {
const p = files.find(e => e.key == vp).file;
await this.storeCustomizationFiles(p);
deleteCandidate = deleteCandidate.filter(e => e != vp);
}
for (const vp of deleteCandidate) {
await this.deleteConfigOnDatabase(vp);
}
}
async deleteConfigOnDatabase(prefixedFileName: FilePathWithPrefix, forceWrite = false) {
// const id = await this.path2id(prefixedFileName);
const mtime = new Date().getTime();
await runWithLock("file-x-" + prefixedFileName, false, async () => {
try {
const old = await this.localDatabase.getDBEntryMeta(prefixedFileName, null, false) as InternalFileEntry | false;
let saveData: InternalFileEntry;
if (old === false) {
Logger(`STORAGE -x> DB:${prefixedFileName}: (config) already deleted (Not found on database)`);
} else {
if (old.deleted) {
Logger(`STORAGE -x> DB:${prefixedFileName}: (config) already deleted`);
return;
}
saveData =
{
...old,
mtime,
size: 0,
children: [],
deleted: true,
type: "newnote",
};
}
await this.localDatabase.putRaw(saveData);
Logger(`STORAGE -x> DB:${prefixedFileName}: (config) Done`);
} catch (ex) {
Logger(`STORAGE -x> DB:${prefixedFileName}: (config) Failed`);
Logger(ex, LOG_LEVEL.VERBOSE);
return false;
}
});
}
async scanInternalFiles(): Promise<FilePath[]> {
const filenames = (await this.getFiles(this.app.vault.configDir, 2)).filter(e => e.startsWith(".")).filter(e => !e.startsWith(".trash"));
return filenames as FilePath[];
}
async getFiles(
path: string,
lastDepth: number
) {
if (lastDepth == -1) return [];
const w = await this.app.vault.adapter.list(path);
let files = [
...w.files
];
for (const v of w.folders) {
files = files.concat(await this.getFiles(v, lastDepth - 1));
}
return files;
}
}

722
src/CmdHiddenFileSync.ts Normal file
View File

@@ -0,0 +1,722 @@
import { Notice, normalizePath, PluginManifest } from "./deps";
import { EntryDoc, LoadedEntry, LOG_LEVEL, InternalFileEntry, FilePathWithPrefix, FilePath } from "./lib/src/types";
import { InternalFileInfo, ICHeader, ICHeaderEnd } from "./types";
import { delay, isDocContentSame } from "./lib/src/utils";
import { Logger } from "./lib/src/logger";
import { PouchDB } from "./lib/src/pouchdb-browser.js";
import { disposeMemoObject, memoIfNotExist, memoObject, retrieveMemoObject, scheduleTask, isInternalMetadata, PeriodicProcessor } from "./utils";
import { WrappedNotice } from "./lib/src/wrapper";
import { base64ToArrayBuffer, arrayBufferToBase64 } from "./lib/src/strbin";
import { runWithLock } from "./lib/src/lock";
import { Semaphore } from "./lib/src/semaphore";
import { JsonResolveModal } from "./JsonResolveModal";
import { LiveSyncCommands } from "./LiveSyncCommands";
import { addPrefix, stripAllPrefixes } from "./lib/src/path";
export class HiddenFileSync extends LiveSyncCommands {
periodicInternalFileScanProcessor: PeriodicProcessor = new PeriodicProcessor(this.plugin, async () => this.settings.syncInternalFiles && this.localDatabase.isReady && await this.syncInternalFilesAndDatabase("push", false));
confirmPopup: WrappedNotice = null;
get kvDB() {
return this.plugin.kvDB;
}
ensureDirectoryEx(fullPath: string) {
return this.plugin.ensureDirectoryEx(fullPath);
}
getConflictedDoc(path: FilePathWithPrefix, rev: string) {
return this.plugin.getConflictedDoc(path, rev);
}
onunload() {
this.periodicInternalFileScanProcessor?.disable();
}
onload() {
this.plugin.addCommand({
id: "livesync-scaninternal",
name: "Sync hidden files",
callback: () => {
this.syncInternalFilesAndDatabase("safe", true);
},
});
}
async onInitializeDatabase(showNotice: boolean) {
if (this.settings.syncInternalFiles) {
try {
Logger("Synchronizing hidden files...");
await this.syncInternalFilesAndDatabase("push", showNotice);
Logger("Synchronizing hidden files done");
} catch (ex) {
Logger("Synchronizing hidden files failed");
Logger(ex, LOG_LEVEL.VERBOSE);
}
}
}
async beforeReplicate(showNotice: boolean) {
if (this.localDatabase.isReady && this.settings.syncInternalFiles && this.settings.syncInternalFilesBeforeReplication && !this.settings.watchInternalFileChanges) {
await this.syncInternalFilesAndDatabase("push", showNotice);
}
}
async onResume() {
this.periodicInternalFileScanProcessor?.disable();
if (this.plugin.suspended)
return;
if (this.settings.syncInternalFiles) {
await this.syncInternalFilesAndDatabase("safe", false);
}
this.periodicInternalFileScanProcessor.enable(this.settings.syncInternalFiles && this.settings.syncInternalFilesInterval ? (this.settings.syncInternalFilesInterval * 1000) : 0);
}
parseReplicationResultItem(docs: PouchDB.Core.ExistingDocument<EntryDoc>) {
return false;
}
realizeSettingSyncMode(): Promise<void> {
this.periodicInternalFileScanProcessor?.disable();
if (this.plugin.suspended)
return;
if (!this.plugin.isReady)
return;
this.periodicInternalFileScanProcessor.enable(this.settings.syncInternalFiles && this.settings.syncInternalFilesInterval ? (this.settings.syncInternalFilesInterval * 1000) : 0);
return;
}
procInternalFiles: string[] = [];
async execInternalFile() {
await runWithLock("execInternal", false, async () => {
const w = [...this.procInternalFiles];
this.procInternalFiles = [];
Logger(`Applying hidden ${w.length} files change...`);
await this.syncInternalFilesAndDatabase("pull", false, false, w);
Logger(`Applying hidden ${w.length} files changed`);
});
}
procInternalFile(filename: string) {
this.procInternalFiles.push(filename);
scheduleTask("procInternal", 500, async () => {
await this.execInternalFile();
});
}
recentProcessedInternalFiles = [] as string[];
async watchVaultRawEventsAsync(path: FilePath) {
if (!this.settings.syncInternalFiles) return;
const stat = await this.app.vault.adapter.stat(path);
// sometimes folder is coming.
if (stat && stat.type != "file")
return;
const storageMTime = ~~((stat && stat.mtime || 0) / 1000);
const key = `${path}-${storageMTime}`;
if (this.recentProcessedInternalFiles.contains(key)) {
//If recently processed, it may caused by self.
return;
}
this.recentProcessedInternalFiles = [key, ...this.recentProcessedInternalFiles].slice(0, 100);
// const id = await this.path2id(path, ICHeader);
const prefixedFileName = addPrefix(path, ICHeader);
const filesOnDB = await this.localDatabase.getDBEntryMeta(prefixedFileName);
const dbMTime = ~~((filesOnDB && filesOnDB.mtime || 0) / 1000);
// Skip unchanged file.
if (dbMTime == storageMTime) {
// Logger(`STORAGE --> DB:${path}: (hidden) Nothing changed`);
return;
}
// Do not compare timestamp. Always local data should be preferred except this plugin wrote one.
if (storageMTime == 0) {
await this.deleteInternalFileOnDatabase(path);
} else {
await this.storeInternalFileToDatabase({ path: path, ...stat });
}
}
async resolveConflictOnInternalFiles() {
// Scan all conflicted internal files
const conflicted = this.localDatabase.findEntries(ICHeader, ICHeaderEnd, { conflicts: true });
for await (const doc of conflicted) {
if (!("_conflicts" in doc))
continue;
if (isInternalMetadata(doc._id)) {
await this.resolveConflictOnInternalFile(doc.path);
}
}
}
async resolveConflictOnInternalFile(path: FilePathWithPrefix): Promise<boolean> {
try {
// Retrieve data
const id = await this.path2id(path, ICHeader);
const doc = await this.localDatabase.getRaw(id, { conflicts: true });
// If there is no conflict, return with false.
if (!("_conflicts" in doc))
return false;
if (doc._conflicts.length == 0)
return false;
Logger(`Hidden file conflicted:${path}`);
const conflicts = doc._conflicts.sort((a, b) => Number(a.split("-")[0]) - Number(b.split("-")[0]));
const revA = doc._rev;
const revB = conflicts[0];
if (path.endsWith(".json")) {
const conflictedRev = conflicts[0];
const conflictedRevNo = Number(conflictedRev.split("-")[0]);
//Search
const revFrom = (await this.localDatabase.getRaw<EntryDoc>(id, { revs_info: true }));
const commonBase = revFrom._revs_info.filter(e => e.status == "available" && Number(e.rev.split("-")[0]) < conflictedRevNo).first()?.rev ?? "";
const result = await this.plugin.mergeObject(path, commonBase, doc._rev, conflictedRev);
if (result) {
Logger(`Object merge:${path}`, LOG_LEVEL.INFO);
const filename = stripAllPrefixes(path);
const isExists = await this.app.vault.adapter.exists(filename);
if (!isExists) {
await this.ensureDirectoryEx(filename);
}
await this.app.vault.adapter.write(filename, result);
const stat = await this.app.vault.adapter.stat(filename);
await this.storeInternalFileToDatabase({ path: filename, ...stat });
await this.extractInternalFileFromDatabase(filename);
await this.localDatabase.removeRaw(id, revB);
return this.resolveConflictOnInternalFile(path);
} else {
Logger(`Object merge is not applicable.`, LOG_LEVEL.VERBOSE);
}
const docAMerge = await this.localDatabase.getDBEntry(path, { rev: revA });
const docBMerge = await this.localDatabase.getDBEntry(path, { rev: revB });
if (docAMerge != false && docBMerge != false) {
if (await this.showJSONMergeDialogAndMerge(docAMerge, docBMerge)) {
await delay(200);
// Again for other conflicted revisions.
return this.resolveConflictOnInternalFile(path);
}
return false;
}
}
const revBDoc = await this.localDatabase.getRaw(id, { rev: revB });
// determine which revision should been deleted.
// simply check modified time
const mtimeA = ("mtime" in doc && doc.mtime) || 0;
const mtimeB = ("mtime" in revBDoc && revBDoc.mtime) || 0;
// Logger(`Revisions:${new Date(mtimeA).toLocaleString} and ${new Date(mtimeB).toLocaleString}`);
// console.log(`mtime:${mtimeA} - ${mtimeB}`);
const delRev = mtimeA < mtimeB ? revA : revB;
// delete older one.
await this.localDatabase.removeRaw(id, delRev);
Logger(`Older one has been deleted:${path}`);
// check the file again
return this.resolveConflictOnInternalFile(path);
} catch (ex) {
Logger(`Failed to resolve conflict (Hidden): ${path}`);
Logger(ex, LOG_LEVEL.VERBOSE);
return false;
}
}
//TODO: Tidy up. Even though it is experimental feature, So dirty...
async syncInternalFilesAndDatabase(direction: "push" | "pull" | "safe" | "pullForce" | "pushForce", showMessage: boolean, files: InternalFileInfo[] | false = false, targetFiles: string[] | false = false) {
await this.resolveConflictOnInternalFiles();
const logLevel = showMessage ? LOG_LEVEL.NOTICE : LOG_LEVEL.INFO;
Logger("Scanning hidden files.", logLevel, "sync_internal");
const ignorePatterns = this.settings.syncInternalFilesIgnorePatterns
.replace(/\n| /g, "")
.split(",").filter(e => e).map(e => new RegExp(e, "i"));
if (!files)
files = await this.scanInternalFiles();
const filesOnDB = ((await this.localDatabase.allDocsRaw({ startkey: ICHeader, endkey: ICHeaderEnd, include_docs: true })).rows.map(e => e.doc) as InternalFileEntry[]).filter(e => !e.deleted);
const allFileNamesSrc = [...new Set([...files.map(e => normalizePath(e.path)), ...filesOnDB.map(e => stripAllPrefixes(this.getPath(e)))])];
const allFileNames = allFileNamesSrc.filter(filename => !targetFiles || (targetFiles && targetFiles.indexOf(filename) !== -1));
function compareMTime(a: number, b: number) {
const wa = ~~(a / 1000);
const wb = ~~(b / 1000);
const diff = wa - wb;
return diff;
}
const fileCount = allFileNames.length;
let processed = 0;
let filesChanged = 0;
// count updated files up as like this below:
// .obsidian: 2
// .obsidian/workspace: 1
// .obsidian/plugins: 1
// .obsidian/plugins/recent-files-obsidian: 1
// .obsidian/plugins/recent-files-obsidian/data.json: 1
const updatedFolders: { [key: string]: number; } = {};
const countUpdatedFolder = (path: string) => {
const pieces = path.split("/");
let c = pieces.shift();
let pathPieces = "";
filesChanged++;
while (c) {
pathPieces += (pathPieces != "" ? "/" : "") + c;
pathPieces = normalizePath(pathPieces);
if (!(pathPieces in updatedFolders)) {
updatedFolders[pathPieces] = 0;
}
updatedFolders[pathPieces]++;
c = pieces.shift();
}
};
const p = [] as Promise<void>[];
const semaphore = Semaphore(10);
// Cache update time information for files which have already been processed (mainly for files that were skipped due to the same content)
let caches: { [key: string]: { storageMtime: number; docMtime: number; }; } = {};
caches = await this.kvDB.get<{ [key: string]: { storageMtime: number; docMtime: number; }; }>("diff-caches-internal") || {};
for (const filename of allFileNames) {
if (!filename) continue;
processed++;
if (processed % 100 == 0)
Logger(`Hidden file: ${processed}/${fileCount}`, logLevel, "sync_internal");
if (ignorePatterns.some(e => filename.match(e)))
continue;
const fileOnStorage = files.find(e => e.path == filename);
const fileOnDatabase = filesOnDB.find(e => stripAllPrefixes(this.getPath(e)) == filename);
const addProc = async (p: () => Promise<void>): Promise<void> => {
const releaser = await semaphore.acquire(1);
try {
return p();
} catch (ex) {
Logger("Some process failed", logLevel);
Logger(ex);
} finally {
releaser();
}
};
const cache = filename in caches ? caches[filename] : { storageMtime: 0, docMtime: 0 };
p.push(addProc(async () => {
const xFileOnStorage = fileOnStorage;
const xFileOnDatabase = fileOnDatabase;
if (xFileOnStorage && xFileOnDatabase) {
// Both => Synchronize
if ((direction != "pullForce" && direction != "pushForce") && xFileOnDatabase.mtime == cache.docMtime && xFileOnStorage.mtime == cache.storageMtime) {
return;
}
const nw = compareMTime(xFileOnStorage.mtime, xFileOnDatabase.mtime);
if (nw > 0 || direction == "pushForce") {
await this.storeInternalFileToDatabase(xFileOnStorage);
}
if (nw < 0 || direction == "pullForce") {
// skip if not extraction performed.
if (!await this.extractInternalFileFromDatabase(filename))
return;
}
// If process successfully updated or file contents are same, update cache.
cache.docMtime = xFileOnDatabase.mtime;
cache.storageMtime = xFileOnStorage.mtime;
caches[filename] = cache;
countUpdatedFolder(filename);
} else if (!xFileOnStorage && xFileOnDatabase) {
if (direction == "push" || direction == "pushForce") {
if (xFileOnDatabase.deleted)
return;
await this.deleteInternalFileOnDatabase(filename, false);
} else if (direction == "pull" || direction == "pullForce") {
if (await this.extractInternalFileFromDatabase(filename)) {
countUpdatedFolder(filename);
}
} else if (direction == "safe") {
if (xFileOnDatabase.deleted)
return;
if (await this.extractInternalFileFromDatabase(filename)) {
countUpdatedFolder(filename);
}
}
} else if (xFileOnStorage && !xFileOnDatabase) {
await this.storeInternalFileToDatabase(xFileOnStorage);
} else {
throw new Error("Invalid state on hidden file sync");
// Something corrupted?
}
}));
}
await Promise.all(p);
await this.kvDB.set("diff-caches-internal", caches);
// When files has been retrieved from the database. they must be reloaded.
if ((direction == "pull" || direction == "pullForce") && filesChanged != 0) {
const configDir = normalizePath(this.app.vault.configDir);
// Show notification to restart obsidian when something has been changed in configDir.
if (configDir in updatedFolders) {
// Numbers of updated files that is below of configDir.
let updatedCount = updatedFolders[configDir];
try {
//@ts-ignore
const manifests = Object.values(this.app.plugins.manifests) as any as PluginManifest[];
//@ts-ignore
const enabledPlugins = this.app.plugins.enabledPlugins as Set<string>;
const enabledPluginManifests = manifests.filter(e => enabledPlugins.has(e.id));
for (const manifest of enabledPluginManifests) {
if (manifest.dir in updatedFolders) {
// If notified about plug-ins, reloading Obsidian may not be necessary.
updatedCount -= updatedFolders[manifest.dir];
const updatePluginId = manifest.id;
const updatePluginName = manifest.name;
const fragment = createFragment((doc) => {
doc.createEl("span", null, (a) => {
a.appendText(`Files in ${updatePluginName} has been updated, Press `);
a.appendChild(a.createEl("a", null, (anchor) => {
anchor.text = "HERE";
anchor.addEventListener("click", async () => {
Logger(`Unloading plugin: ${updatePluginName}`, LOG_LEVEL.NOTICE, "plugin-reload-" + updatePluginId);
// @ts-ignore
await this.app.plugins.unloadPlugin(updatePluginId);
// @ts-ignore
await this.app.plugins.loadPlugin(updatePluginId);
Logger(`Plugin reloaded: ${updatePluginName}`, LOG_LEVEL.NOTICE, "plugin-reload-" + updatePluginId);
});
}));
a.appendText(` to reload ${updatePluginName}, or press elsewhere to dismiss this message.`);
});
});
const updatedPluginKey = "popupUpdated-" + updatePluginId;
scheduleTask(updatedPluginKey, 1000, async () => {
const popup = await memoIfNotExist(updatedPluginKey, () => new Notice(fragment, 0));
//@ts-ignore
const isShown = popup?.noticeEl?.isShown();
if (!isShown) {
memoObject(updatedPluginKey, new Notice(fragment, 0));
}
scheduleTask(updatedPluginKey + "-close", 20000, () => {
const popup = retrieveMemoObject<Notice>(updatedPluginKey);
if (!popup)
return;
//@ts-ignore
if (popup?.noticeEl?.isShown()) {
popup.hide();
}
disposeMemoObject(updatedPluginKey);
});
});
}
}
} catch (ex) {
Logger("Error on checking plugin status.");
Logger(ex, LOG_LEVEL.VERBOSE);
}
// If something changes left, notify for reloading Obsidian.
if (updatedCount != 0) {
const fragment = createFragment((doc) => {
doc.createEl("span", null, (a) => {
a.appendText(`Hidden files have been synchronized, Press `);
a.appendChild(a.createEl("a", null, (anchor) => {
anchor.text = "HERE";
anchor.addEventListener("click", () => {
// @ts-ignore
this.app.commands.executeCommandById("app:reload");
});
}));
a.appendText(` to reload obsidian, or press elsewhere to dismiss this message.`);
});
});
scheduleTask("popupUpdated-" + configDir, 1000, () => {
//@ts-ignore
const isShown = this.confirmPopup?.noticeEl?.isShown();
if (!isShown) {
this.confirmPopup = new Notice(fragment, 0);
}
scheduleTask("popupClose" + configDir, 20000, () => {
this.confirmPopup?.hide();
this.confirmPopup = null;
});
});
}
}
}
Logger(`Hidden files scanned: ${filesChanged} files had been modified`, logLevel, "sync_internal");
}
async storeInternalFileToDatabase(file: InternalFileInfo, forceWrite = false) {
const id = await this.path2id(file.path, ICHeader);
const prefixedFileName = addPrefix(file.path, ICHeader);
const contentBin = await this.app.vault.adapter.readBinary(file.path);
let content: string[];
try {
content = await arrayBufferToBase64(contentBin);
} catch (ex) {
Logger(`The file ${file.path} could not be encoded`);
Logger(ex, LOG_LEVEL.VERBOSE);
return false;
}
const mtime = file.mtime;
return await runWithLock("file-" + prefixedFileName, false, async () => {
try {
const old = await this.localDatabase.getDBEntry(prefixedFileName, null, false, false);
let saveData: LoadedEntry;
if (old === false) {
saveData = {
_id: id,
path: prefixedFileName,
data: content,
mtime,
ctime: mtime,
datatype: "newnote",
size: file.size,
children: [],
deleted: false,
type: "newnote",
};
} else {
if (isDocContentSame(old.data, content) && !forceWrite) {
// Logger(`STORAGE --> DB:${file.path}: (hidden) Not changed`, LOG_LEVEL.VERBOSE);
return;
}
saveData =
{
...old,
data: content,
mtime,
size: file.size,
datatype: "newnote",
children: [],
deleted: false,
type: "newnote",
};
}
const ret = await this.localDatabase.putDBEntry(saveData, true);
Logger(`STORAGE --> DB:${file.path}: (hidden) Done`);
return ret;
} catch (ex) {
Logger(`STORAGE --> DB:${file.path}: (hidden) Failed`);
Logger(ex, LOG_LEVEL.VERBOSE);
return false;
}
});
}
async deleteInternalFileOnDatabase(filename: FilePath, forceWrite = false) {
const id = await this.path2id(filename, ICHeader);
const prefixedFileName = addPrefix(filename, ICHeader);
const mtime = new Date().getTime();
await runWithLock("file-" + prefixedFileName, false, async () => {
try {
const old = await this.localDatabase.getDBEntry(prefixedFileName, null, false, false) as InternalFileEntry | false;
let saveData: InternalFileEntry;
if (old === false) {
saveData = {
_id: id,
path: prefixedFileName,
mtime,
ctime: mtime,
size: 0,
children: [],
deleted: true,
type: "newnote",
};
} else {
if (old.deleted) {
Logger(`STORAGE -x> DB:${filename}: (hidden) already deleted`);
return;
}
saveData =
{
...old,
mtime,
size: 0,
children: [],
deleted: true,
type: "newnote",
};
}
await this.localDatabase.putRaw(saveData);
Logger(`STORAGE -x> DB:${filename}: (hidden) Done`);
} catch (ex) {
Logger(`STORAGE -x> DB:${filename}: (hidden) Failed`);
Logger(ex, LOG_LEVEL.VERBOSE);
return false;
}
});
}
async extractInternalFileFromDatabase(filename: FilePath, force = false) {
const isExists = await this.app.vault.adapter.exists(filename);
const prefixedFileName = addPrefix(filename, ICHeader);
return await runWithLock("file-" + prefixedFileName, false, async () => {
try {
// Check conflicted status
//TODO option
const fileOnDB = await this.localDatabase.getDBEntry(prefixedFileName, { conflicts: true }, false, false);
if (fileOnDB === false)
throw new Error(`File not found on database.:${filename}`);
// Prevent overwrite for Prevent overwriting while some conflicted revision exists.
if (fileOnDB?._conflicts?.length) {
Logger(`Hidden file ${filename} has conflicted revisions, to keep in safe, writing to storage has been prevented`, LOG_LEVEL.INFO);
return;
}
const deleted = "deleted" in fileOnDB ? fileOnDB.deleted : false;
if (deleted) {
if (!isExists) {
Logger(`STORAGE <x- DB:${filename}: deleted (hidden) Deleted on DB, but the file is already not found on storage.`);
} else {
Logger(`STORAGE <x- DB:${filename}: deleted (hidden).`);
await this.app.vault.adapter.remove(filename);
try {
//@ts-ignore internalAPI
await app.vault.adapter.reconcileInternalFile(filename);
} catch (ex) {
Logger("Failed to call internal API(reconcileInternalFile)", LOG_LEVEL.VERBOSE);
Logger(ex, LOG_LEVEL.VERBOSE);
}
}
return true;
}
if (!isExists) {
await this.ensureDirectoryEx(filename);
await this.app.vault.adapter.writeBinary(filename, base64ToArrayBuffer(fileOnDB.data), { mtime: fileOnDB.mtime, ctime: fileOnDB.ctime });
try {
//@ts-ignore internalAPI
await app.vault.adapter.reconcileInternalFile(filename);
} catch (ex) {
Logger("Failed to call internal API(reconcileInternalFile)", LOG_LEVEL.VERBOSE);
Logger(ex, LOG_LEVEL.VERBOSE);
}
Logger(`STORAGE <-- DB:${filename}: written (hidden,new${force ? ", force" : ""})`);
return true;
} else {
const contentBin = await this.app.vault.adapter.readBinary(filename);
const content = await arrayBufferToBase64(contentBin);
if (content == fileOnDB.data && !force) {
// Logger(`STORAGE <-- DB:${filename}: skipped (hidden) Not changed`, LOG_LEVEL.VERBOSE);
return true;
}
await this.app.vault.adapter.writeBinary(filename, base64ToArrayBuffer(fileOnDB.data), { mtime: fileOnDB.mtime, ctime: fileOnDB.ctime });
try {
//@ts-ignore internalAPI
await app.vault.adapter.reconcileInternalFile(filename);
} catch (ex) {
Logger("Failed to call internal API(reconcileInternalFile)", LOG_LEVEL.VERBOSE);
Logger(ex, LOG_LEVEL.VERBOSE);
}
Logger(`STORAGE <-- DB:${filename}: written (hidden, overwrite${force ? ", force" : ""})`);
return true;
}
} catch (ex) {
Logger(`STORAGE <-- DB:${filename}: written (hidden, overwrite${force ? ", force" : ""}) Failed`);
Logger(ex, LOG_LEVEL.VERBOSE);
return false;
}
});
}
showJSONMergeDialogAndMerge(docA: LoadedEntry, docB: LoadedEntry): Promise<boolean> {
return runWithLock("conflict:merge-data", false, () => new Promise((res) => {
Logger("Opening data-merging dialog", LOG_LEVEL.VERBOSE);
const docs = [docA, docB];
const path = stripAllPrefixes(docA.path);
const modal = new JsonResolveModal(this.app, path, [docA, docB], async (keep, result) => {
// modal.close();
try {
const filename = path;
let needFlush = false;
if (!result && !keep) {
Logger(`Skipped merging: ${filename}`);
res(false);
return;
}
//Delete old revisions
if (result || keep) {
for (const doc of docs) {
if (doc._rev != keep) {
if (await this.localDatabase.deleteDBEntry(this.getPath(doc), { rev: doc._rev })) {
Logger(`Conflicted revision has been deleted: ${filename}`);
needFlush = true;
}
}
}
}
if (!keep && result) {
const isExists = await this.app.vault.adapter.exists(filename);
if (!isExists) {
await this.ensureDirectoryEx(filename);
}
await this.app.vault.adapter.write(filename, result);
const stat = await this.app.vault.adapter.stat(filename);
await this.storeInternalFileToDatabase({ path: filename, ...stat }, true);
try {
//@ts-ignore internalAPI
await app.vault.adapter.reconcileInternalFile(filename);
} catch (ex) {
Logger("Failed to call internal API(reconcileInternalFile)", LOG_LEVEL.VERBOSE);
Logger(ex, LOG_LEVEL.VERBOSE);
}
Logger(`STORAGE <-- DB:${filename}: written (hidden,merged)`);
}
if (needFlush) {
await this.extractInternalFileFromDatabase(filename, false);
Logger(`STORAGE --> DB:${filename}: extracted (hidden,merged)`);
}
res(true);
} catch (ex) {
Logger("Could not merge conflicted json");
Logger(ex, LOG_LEVEL.VERBOSE);
res(false);
}
});
modal.open();
}));
}
async scanInternalFiles(): Promise<InternalFileInfo[]> {
const ignoreFilter = this.settings.syncInternalFilesIgnorePatterns
.replace(/\n| /g, "")
.split(",").filter(e => e).map(e => new RegExp(e, "i"));
const root = this.app.vault.getRoot();
const findRoot = root.path;
const filenames = (await this.getFiles(findRoot, [], null, ignoreFilter)).filter(e => e.startsWith(".")).filter(e => !e.startsWith(".trash"));
const files = filenames.map(async (e) => {
return {
path: e as FilePath,
stat: await this.app.vault.adapter.stat(e)
};
});
const result: InternalFileInfo[] = [];
for (const f of files) {
const w = await f;
result.push({
...w,
...w.stat
});
}
return result;
}
async getFiles(
path: string,
ignoreList: string[],
filter: RegExp[],
ignoreFilter: RegExp[]
) {
const w = await this.app.vault.adapter.list(path);
let files = [
...w.files
.filter((e) => !ignoreList.some((ee) => e.endsWith(ee)))
.filter((e) => !filter || filter.some((ee) => e.match(ee)))
.filter((e) => !ignoreFilter || ignoreFilter.every((ee) => !e.match(ee))),
];
L1: for (const v of w.folders) {
for (const ignore of ignoreList) {
if (v.endsWith(ignore)) {
continue L1;
}
}
if (ignoreFilter && ignoreFilter.some(e => v.match(e))) {
continue L1;
}
files = files.concat(await this.getFiles(v, ignoreList, filter, ignoreFilter));
}
return files;
}
}

315
src/CmdPluginAndTheirSettings.ts Normal file
View File

@@ -0,0 +1,315 @@
import { normalizePath, PluginManifest } from "./deps";
import { DocumentID, EntryDoc, FilePathWithPrefix, LoadedEntry, LOG_LEVEL } from "./lib/src/types";
import { PluginDataEntry, PERIODIC_PLUGIN_SWEEP, PluginList, DevicePluginList, PSCHeader, PSCHeaderEnd } from "./types";
import { getDocData, isDocContentSame } from "./lib/src/utils";
import { Logger } from "./lib/src/logger";
import { PouchDB } from "./lib/src/pouchdb-browser.js";
import { isPluginMetadata, PeriodicProcessor } from "./utils";
import { PluginDialogModal } from "./dialogs";
import { NewNotice } from "./lib/src/wrapper";
import { versionNumberString2Number } from "./lib/src/strbin";
import { runWithLock } from "./lib/src/lock";
import { LiveSyncCommands } from "./LiveSyncCommands";
export class PluginAndTheirSettings extends LiveSyncCommands {
get deviceAndVaultName() {
return this.plugin.deviceAndVaultName;
}
pluginDialog: PluginDialogModal = null;
periodicPluginSweepProcessor = new PeriodicProcessor(this.plugin, async () => await this.sweepPlugin(false));
showPluginSyncModal() {
if (this.pluginDialog != null) {
this.pluginDialog.open();
} else {
this.pluginDialog = new PluginDialogModal(this.app, this.plugin);
this.pluginDialog.open();
}
}
hidePluginSyncModal() {
if (this.pluginDialog != null) {
this.pluginDialog.close();
this.pluginDialog = null;
}
}
onload(): void | Promise<void> {
this.plugin.addCommand({
id: "livesync-plugin-dialog",
name: "Show Plugins and their settings",
callback: () => {
this.showPluginSyncModal();
},
});
this.showPluginSyncModal();
}
onunload() {
this.hidePluginSyncModal();
this.periodicPluginSweepProcessor?.disable();
}
parseReplicationResultItem(doc: PouchDB.Core.ExistingDocument<EntryDoc>) {
if (isPluginMetadata(doc._id)) {
if (this.settings.notifyPluginOrSettingUpdated) {
this.triggerCheckPluginUpdate();
return true;
}
}
return false;
}
async beforeReplicate(showMessage: boolean) {
if (this.settings.autoSweepPlugins) {
await this.sweepPlugin(showMessage);
}
}
async onResume() {
if (this.plugin.suspended)
return;
if (this.settings.autoSweepPlugins) {
await this.sweepPlugin(false);
}
this.periodicPluginSweepProcessor.enable(this.settings.autoSweepPluginsPeriodic && !this.settings.watchInternalFileChanges ? (PERIODIC_PLUGIN_SWEEP * 1000) : 0);
}
async onInitializeDatabase(showNotice: boolean) {
if (this.settings.usePluginSync) {
try {
Logger("Scanning plugins...");
await this.sweepPlugin(showNotice);
Logger("Scanning plugins done");
} catch (ex) {
Logger("Scanning plugins failed");
Logger(ex, LOG_LEVEL.VERBOSE);
}
}
}
async realizeSettingSyncMode() {
this.periodicPluginSweepProcessor?.disable();
if (this.plugin.suspended)
return;
if (this.settings.autoSweepPlugins) {
await this.sweepPlugin(false);
}
this.periodicPluginSweepProcessor.enable(this.settings.autoSweepPluginsPeriodic && !this.settings.watchInternalFileChanges ? (PERIODIC_PLUGIN_SWEEP * 1000) : 0);
}
triggerCheckPluginUpdate() {
(async () => await this.checkPluginUpdate())();
}
async getPluginList(): Promise<{ plugins: PluginList; allPlugins: DevicePluginList; thisDevicePlugins: DevicePluginList; }> {
const docList = await this.localDatabase.allDocsRaw<PluginDataEntry>({ startkey: PSCHeader, endkey: PSCHeaderEnd, include_docs: false });
const oldDocs: PluginDataEntry[] = ((await Promise.all(docList.rows.map(async (e) => await this.localDatabase.getDBEntry(e.id as FilePathWithPrefix /* WARN!! THIS SHOULD BE WRAPPED */)))).filter((e) => e !== false) as LoadedEntry[]).map((e) => JSON.parse(getDocData(e.data)));
const plugins: { [key: string]: PluginDataEntry[]; } = {};
const allPlugins: { [key: string]: PluginDataEntry; } = {};
const thisDevicePlugins: { [key: string]: PluginDataEntry; } = {};
for (const v of oldDocs) {
if (typeof plugins[v.deviceVaultName] === "undefined") {
plugins[v.deviceVaultName] = [];
}
plugins[v.deviceVaultName].push(v);
allPlugins[v._id] = v;
if (v.deviceVaultName == this.deviceAndVaultName) {
thisDevicePlugins[v.manifest.id] = v;
}
}
return { plugins, allPlugins, thisDevicePlugins };
}
async checkPluginUpdate() {
if (!this.plugin.settings.usePluginSync)
return;
await this.sweepPlugin(false);
const { allPlugins, thisDevicePlugins } = await this.getPluginList();
const arrPlugins = Object.values(allPlugins);
let updateFound = false;
for (const plugin of arrPlugins) {
const ownPlugin = thisDevicePlugins[plugin.manifest.id];
if (ownPlugin) {
const remoteVersion = versionNumberString2Number(plugin.manifest.version);
const ownVersion = versionNumberString2Number(ownPlugin.manifest.version);
if (remoteVersion > ownVersion) {
updateFound = true;
}
if (((plugin.mtime / 1000) | 0) > ((ownPlugin.mtime / 1000) | 0) && (plugin.dataJson ?? "") != (ownPlugin.dataJson ?? "")) {
updateFound = true;
}
}
}
if (updateFound) {
const fragment = createFragment((doc) => {
doc.createEl("a", null, (a) => {
a.text = "There're some new plugins or their settings";
a.addEventListener("click", () => this.showPluginSyncModal());
});
});
NewNotice(fragment, 10000);
} else {
Logger("Everything is up to date.", LOG_LEVEL.NOTICE);
}
}
async sweepPlugin(showMessage = false, specificPluginPath = "") {
if (!this.settings.usePluginSync)
return;
if (!this.localDatabase.isReady)
return;
// @ts-ignore
const pl = this.app.plugins;
const manifests: PluginManifest[] = Object.values(pl.manifests);
let specificPlugin = "";
if (specificPluginPath != "") {
specificPlugin = manifests.find(e => e.dir.endsWith("/" + specificPluginPath))?.id ?? "";
}
await runWithLock("sweepplugin", true, async () => {
const logLevel = showMessage ? LOG_LEVEL.NOTICE : LOG_LEVEL.INFO;
if (!this.deviceAndVaultName) {
Logger("You have to set your device name.", LOG_LEVEL.NOTICE);
return;
}
Logger("Scanning plugins", logLevel);
const oldDocs = await this.localDatabase.allDocsRaw<EntryDoc>({
startkey: `ps:${this.deviceAndVaultName}-${specificPlugin}`,
endkey: `ps:${this.deviceAndVaultName}-${specificPlugin}\u{10ffff}`,
include_docs: true,
});
// Logger("OLD DOCS.", LOG_LEVEL.VERBOSE);
// sweep current plugin.
const procs = manifests.map(async (m) => {
const pluginDataEntryID = `ps:${this.deviceAndVaultName}-${m.id}` as DocumentID;
try {
if (specificPlugin && m.id != specificPlugin) {
return;
}
Logger(`Reading plugin:${m.name}(${m.id})`, LOG_LEVEL.VERBOSE);
const path = normalizePath(m.dir) + "/";
const adapter = this.app.vault.adapter;
const files = ["manifest.json", "main.js", "styles.css", "data.json"];
const pluginData: { [key: string]: string; } = {};
for (const file of files) {
const thePath = path + file;
if (await adapter.exists(thePath)) {
pluginData[file] = await adapter.read(thePath);
}
}
let mtime = 0;
if (await adapter.exists(path + "/data.json")) {
mtime = (await adapter.stat(path + "/data.json")).mtime;
}
const p: PluginDataEntry = {
_id: pluginDataEntryID,
dataJson: pluginData["data.json"],
deviceVaultName: this.deviceAndVaultName,
mainJs: pluginData["main.js"],
styleCss: pluginData["styles.css"],
manifest: m,
manifestJson: pluginData["manifest.json"],
mtime: mtime,
type: "plugin",
};
const d: LoadedEntry = {
_id: p._id,
path: p._id as string as FilePathWithPrefix,
data: JSON.stringify(p),
ctime: mtime,
mtime: mtime,
size: 0,
children: [],
datatype: "plain",
type: "plain"
};
Logger(`check diff:${m.name}(${m.id})`, LOG_LEVEL.VERBOSE);
await runWithLock("plugin-" + m.id, false, async () => {
const old = await this.localDatabase.getDBEntry(p._id as string as FilePathWithPrefix /* This also should be explained */, null, false, false);
if (old !== false) {
const oldData = { data: old.data, deleted: old._deleted };
const newData = { data: d.data, deleted: d._deleted };
if (isDocContentSame(oldData.data, newData.data) && oldData.deleted == newData.deleted) {
Logger(`Nothing changed:${m.name}`);
return;
}
}
await this.localDatabase.putDBEntry(d);
Logger(`Plugin saved:${m.name}`, logLevel);
});
} catch (ex) {
Logger(`Plugin save failed:${m.name}`, LOG_LEVEL.NOTICE);
} finally {
oldDocs.rows = oldDocs.rows.filter((e) => e.id != pluginDataEntryID);
}
//remove saved plugin data.
}
);
await Promise.all(procs);
const delDocs = oldDocs.rows.map((e) => {
// e.doc._deleted = true;
if (e.doc.type == "newnote" || e.doc.type == "plain") {
e.doc.deleted = true;
if (this.settings.deleteMetadataOfDeletedFiles) {
e.doc._deleted = true;
}
} else {
e.doc._deleted = true;
}
return e.doc;
});
Logger(`Deleting old plugin:(${delDocs.length})`, LOG_LEVEL.VERBOSE);
await this.localDatabase.bulkDocsRaw(delDocs);
Logger(`Scan plugin done.`, logLevel);
});
}
async applyPluginData(plugin: PluginDataEntry) {
await runWithLock("plugin-" + plugin.manifest.id, false, async () => {
const pluginTargetFolderPath = normalizePath(plugin.manifest.dir) + "/";
const adapter = this.app.vault.adapter;
// @ts-ignore
const stat = this.app.plugins.enabledPlugins.has(plugin.manifest.id) == true;
if (stat) {
// @ts-ignore
await this.app.plugins.unloadPlugin(plugin.manifest.id);
Logger(`Unload plugin:${plugin.manifest.id}`, LOG_LEVEL.NOTICE);
}
if (plugin.dataJson)
await adapter.write(pluginTargetFolderPath + "data.json", plugin.dataJson);
Logger("wrote:" + pluginTargetFolderPath + "data.json", LOG_LEVEL.NOTICE);
if (stat) {
// @ts-ignore
await this.app.plugins.loadPlugin(plugin.manifest.id);
Logger(`Load plugin:${plugin.manifest.id}`, LOG_LEVEL.NOTICE);
}
});
}
async applyPlugin(plugin: PluginDataEntry) {
await runWithLock("plugin-" + plugin.manifest.id, false, async () => {
// @ts-ignore
const stat = this.app.plugins.enabledPlugins.has(plugin.manifest.id) == true;
if (stat) {
// @ts-ignore
await this.app.plugins.unloadPlugin(plugin.manifest.id);
Logger(`Unload plugin:${plugin.manifest.id}`, LOG_LEVEL.NOTICE);
}
const pluginTargetFolderPath = normalizePath(plugin.manifest.dir) + "/";
const adapter = this.app.vault.adapter;
if ((await adapter.exists(pluginTargetFolderPath)) === false) {
await adapter.mkdir(pluginTargetFolderPath);
}
await adapter.write(pluginTargetFolderPath + "main.js", plugin.mainJs);
await adapter.write(pluginTargetFolderPath + "manifest.json", plugin.manifestJson);
if (plugin.styleCss)
await adapter.write(pluginTargetFolderPath + "styles.css", plugin.styleCss);
if (stat) {
// @ts-ignore
await this.app.plugins.loadPlugin(plugin.manifest.id);
Logger(`Load plugin:${plugin.manifest.id}`, LOG_LEVEL.NOTICE);
}
});
}
}

331
src/CmdSetupLiveSync.ts Normal file
View File

@@ -0,0 +1,331 @@
import { EntryDoc, ObsidianLiveSyncSettings, LOG_LEVEL, DEFAULT_SETTINGS } from "./lib/src/types";
import { configURIBase } from "./types";
import { Logger } from "./lib/src/logger";
import { PouchDB } from "./lib/src/pouchdb-browser.js";
import { askSelectString, askYesNo, askString } from "./utils";
import { decrypt, encrypt } from "./lib/src/e2ee_v2";
import { LiveSyncCommands } from "./LiveSyncCommands";
import { delay } from "./lib/src/utils";
import { confirmWithMessage } from "./dialogs";
import { Platform } from "./deps";
export class SetupLiveSync extends LiveSyncCommands {
onunload() { }
onload(): void | Promise<void> {
this.plugin.registerObsidianProtocolHandler("setuplivesync", async (conf: any) => await this.setupWizard(conf.settings));
this.plugin.addCommand({
id: "livesync-copysetupuri",
name: "Copy the setup URI",
callback: this.command_copySetupURI.bind(this),
});
this.plugin.addCommand({
id: "livesync-copysetupurifull",
name: "Copy the setup URI (Full)",
callback: this.command_copySetupURIFull.bind(this),
});
this.plugin.addCommand({
id: "livesync-opensetupuri",
name: "Open the setup URI",
callback: this.command_openSetupURI.bind(this),
});
}
onInitializeDatabase(showNotice: boolean) { }
beforeReplicate(showNotice: boolean) { }
onResume() { }
parseReplicationResultItem(docs: PouchDB.Core.ExistingDocument<EntryDoc>): boolean | Promise<boolean> {
return false;
}
async realizeSettingSyncMode() { }
async command_copySetupURI() {
const encryptingPassphrase = await askString(this.app, "Encrypt your settings", "The passphrase to encrypt the setup URI", "");
if (encryptingPassphrase === false)
return;
const setting = { ...this.settings, configPassphraseStore: "", encryptedCouchDBConnection: "", encryptedPassphrase: "" };
const keys = Object.keys(setting) as (keyof ObsidianLiveSyncSettings)[];
for (const k of keys) {
if (JSON.stringify(k in setting ? setting[k] : "") == JSON.stringify(k in DEFAULT_SETTINGS ? DEFAULT_SETTINGS[k] : "*")) {
delete setting[k];
}
}
const encryptedSetting = encodeURIComponent(await encrypt(JSON.stringify(setting), encryptingPassphrase, false));
const uri = `${configURIBase}${encryptedSetting}`;
await navigator.clipboard.writeText(uri);
Logger("Setup URI copied to clipboard", LOG_LEVEL.NOTICE);
}
async command_copySetupURIFull() {
const encryptingPassphrase = await askString(this.app, "Encrypt your settings", "The passphrase to encrypt the setup URI", "");
if (encryptingPassphrase === false)
return;
const setting = { ...this.settings, configPassphraseStore: "", encryptedCouchDBConnection: "", encryptedPassphrase: "" };
const encryptedSetting = encodeURIComponent(await encrypt(JSON.stringify(setting), encryptingPassphrase, false));
const uri = `${configURIBase}${encryptedSetting}`;
await navigator.clipboard.writeText(uri);
Logger("Setup URI copied to clipboard", LOG_LEVEL.NOTICE);
}
async command_openSetupURI() {
const setupURI = await askString(this.app, "Easy setup", "Set up URI", `${configURIBase}aaaaa`);
if (setupURI === false)
return;
if (!setupURI.startsWith(`${configURIBase}`)) {
Logger("Set up URI looks wrong.", LOG_LEVEL.NOTICE);
return;
}
const config = decodeURIComponent(setupURI.substring(configURIBase.length));
console.dir(config);
await this.setupWizard(config);
}
async setupWizard(confString: string) {
try {
const oldConf = JSON.parse(JSON.stringify(this.settings));
const encryptingPassphrase = await askString(this.app, "Passphrase", "The passphrase to decrypt your setup URI", "");
if (encryptingPassphrase === false)
return;
const newConf = await JSON.parse(await decrypt(confString, encryptingPassphrase, false));
if (newConf) {
const result = await askYesNo(this.app, "Importing LiveSync's conf, OK?");
if (result == "yes") {
const newSettingW = Object.assign({}, DEFAULT_SETTINGS, newConf) as ObsidianLiveSyncSettings;
this.plugin.replicator.closeReplication();
this.settings.suspendFileWatching = true;
console.dir(newSettingW);
// Back into the default method once.
newSettingW.configPassphraseStore = "";
newSettingW.encryptedPassphrase = "";
newSettingW.encryptedCouchDBConnection = "";
const setupJustImport = "Just import setting";
const setupAsNew = "Set it up as secondary or subsequent device";
const setupAgain = "Reconfigure and reconstitute the data";
const setupManually = "Leave everything to me";
newSettingW.syncInternalFiles = false;
newSettingW.usePluginSync = false;
const setupType = await askSelectString(this.app, "How would you like to set it up?", [setupAsNew, setupAgain, setupJustImport, setupManually]);
if (setupType == setupJustImport) {
this.plugin.settings = newSettingW;
this.plugin.usedPassphrase = "";
await this.plugin.saveSettings();
} else if (setupType == setupAsNew) {
this.plugin.settings = newSettingW;
this.plugin.usedPassphrase = "";
await this.fetchLocal();
} else if (setupType == setupAgain) {
const confirm = "I know this operation will rebuild all my databases with files on this device, and files that are on the remote database and I didn't synchronize to any other devices will be lost and want to proceed indeed.";
if (await askSelectString(this.app, "Do you really want to do this?", ["Cancel", confirm]) != confirm) {
return;
}
this.plugin.settings = newSettingW;
this.plugin.usedPassphrase = "";
await this.rebuildEverything();
} else if (setupType == setupManually) {
const keepLocalDB = await askYesNo(this.app, "Keep local DB?");
const keepRemoteDB = await askYesNo(this.app, "Keep remote DB?");
if (keepLocalDB == "yes" && keepRemoteDB == "yes") {
// nothing to do. so peaceful.
this.plugin.settings = newSettingW;
this.plugin.usedPassphrase = "";
this.suspendAllSync();
this.suspendExtraSync();
await this.plugin.saveSettings();
const replicate = await askYesNo(this.app, "Unlock and replicate?");
if (replicate == "yes") {
await this.plugin.replicate(true);
await this.plugin.markRemoteUnlocked();
}
Logger("Configuration loaded.", LOG_LEVEL.NOTICE);
return;
}
if (keepLocalDB == "no" && keepRemoteDB == "no") {
const reset = await askYesNo(this.app, "Drop everything?");
if (reset != "yes") {
Logger("Cancelled", LOG_LEVEL.NOTICE);
this.plugin.settings = oldConf;
return;
}
}
let initDB;
this.plugin.settings = newSettingW;
this.plugin.usedPassphrase = "";
await this.plugin.saveSettings();
if (keepLocalDB == "no") {
await this.plugin.resetLocalDatabase();
await this.plugin.localDatabase.initializeDatabase();
const rebuild = await askYesNo(this.app, "Rebuild the database?");
if (rebuild == "yes") {
initDB = this.plugin.initializeDatabase(true);
} else {
await this.plugin.markRemoteResolved();
}
}
if (keepRemoteDB == "no") {
await this.plugin.tryResetRemoteDatabase();
await this.plugin.markRemoteLocked();
}
if (keepLocalDB == "no" || keepRemoteDB == "no") {
const replicate = await askYesNo(this.app, "Replicate once?");
if (replicate == "yes") {
if (initDB != null) {
await initDB;
}
await this.plugin.replicate(true);
}
}
}
}
Logger("Configuration loaded.", LOG_LEVEL.NOTICE);
} else {
Logger("Cancelled.", LOG_LEVEL.NOTICE);
}
} catch (ex) {
Logger("Couldn't parse or decrypt configuration uri.", LOG_LEVEL.NOTICE);
}
}
suspendExtraSync() {
Logger("Hidden files and plugin synchronization have been temporarily disabled. Please enable them after the fetching, if you need them.", LOG_LEVEL.NOTICE)
this.plugin.settings.syncInternalFiles = false;
this.plugin.settings.usePluginSync = false;
this.plugin.settings.autoSweepPlugins = false;
}
async askHiddenFileConfiguration(opt: { enableFetch?: boolean, enableOverwrite?: boolean }) {
this.plugin.addOnSetup.suspendExtraSync();
const message = `Would you like to enable \`Hidden File Synchronization\` or \`Customization sync\`?
${opt.enableFetch ? " - Fetch: Use files stored from other devices. \n" : ""}${opt.enableOverwrite ? "- Overwrite: Use files from this device. \n" : ""}- Custom: Synchronize only customization files with a dedicated interface.
- Keep them disabled: Do not use hidden file synchronization.
Of course, we are able to disable these features.`
const CHOICE_FETCH = "Fetch";
const CHOICE_OVERWRITE = "Overwrite";
const CHOICE_CUSTOMIZE = "Custom";
const CHOICE_DISMISS = "keep them disabled";
const choices = [];
if (opt?.enableFetch) {
choices.push(CHOICE_FETCH);
}
if (opt?.enableOverwrite) {
choices.push(CHOICE_OVERWRITE);
}
choices.push(CHOICE_CUSTOMIZE);
choices.push(CHOICE_DISMISS);
const ret = await confirmWithMessage(this.plugin, "Hidden file sync", message, choices, CHOICE_DISMISS, 40);
if (ret == CHOICE_FETCH) {
await this.configureHiddenFileSync("FETCH");
} else if (ret == CHOICE_OVERWRITE) {
await this.configureHiddenFileSync("OVERWRITE");
} else if (ret == CHOICE_DISMISS) {
await this.configureHiddenFileSync("DISABLE");
} else if (ret == CHOICE_CUSTOMIZE) {
await this.configureHiddenFileSync("CUSTOMIZE");
}
}
async configureHiddenFileSync(mode: "FETCH" | "OVERWRITE" | "MERGE" | "DISABLE" | "CUSTOMIZE") {
this.plugin.addOnSetup.suspendExtraSync();
if (mode == "DISABLE") {
this.plugin.settings.syncInternalFiles = false;
this.plugin.settings.usePluginSync = false;
await this.plugin.saveSettings();
return;
}
if (mode != "CUSTOMIZE") {
Logger("Gathering files for enabling Hidden File Sync", LOG_LEVEL.NOTICE);
if (mode == "FETCH") {
await this.plugin.addOnHiddenFileSync.syncInternalFilesAndDatabase("pullForce", true);
} else if (mode == "OVERWRITE") {
await this.plugin.addOnHiddenFileSync.syncInternalFilesAndDatabase("pushForce", true);
} else if (mode == "MERGE") {
await this.plugin.addOnHiddenFileSync.syncInternalFilesAndDatabase("safe", true);
}
this.plugin.settings.syncInternalFiles = true;
await this.plugin.saveSettings();
Logger(`Done! Restarting the app is strongly recommended!`, LOG_LEVEL.NOTICE);
} else if (mode == "CUSTOMIZE") {
if (!this.plugin.deviceAndVaultName) {
let name = await askString(this.app, "Device name", "Please set this device name", `desktop`);
if (!name) {
if (Platform.isAndroidApp) {
name = "android-app"
} else if (Platform.isIosApp) {
name = "ios"
} else if (Platform.isMacOS) {
name = "macos"
} else if (Platform.isMobileApp) {
name = "mobile-app"
} else if (Platform.isMobile) {
name = "mobile"
} else if (Platform.isSafari) {
name = "safari"
} else if (Platform.isDesktop) {
name = "desktop"
} else if (Platform.isDesktopApp) {
name = "desktop-app"
} else {
name = "unknown"
}
name = name + Math.random().toString(36).slice(-4);
}
this.plugin.deviceAndVaultName = name;
}
this.plugin.settings.usePluginSync = true;
await this.plugin.saveSettings();
await this.plugin.addOnConfigSync.scanAllConfigFiles(true);
}
}
suspendAllSync() {
this.plugin.settings.liveSync = false;
this.plugin.settings.periodicReplication = false;
this.plugin.settings.syncOnSave = false;
this.plugin.settings.syncOnStart = false;
this.plugin.settings.syncOnFileOpen = false;
this.plugin.settings.syncAfterMerge = false;
//this.suspendExtraSync();
}
async fetchLocal() {
this.suspendExtraSync();
await this.plugin.realizeSettingSyncMode();
await this.plugin.resetLocalDatabase();
await delay(1000);
await this.plugin.markRemoteResolved();
await this.plugin.openDatabase();
this.plugin.isReady = true;
await delay(500);
await this.plugin.replicateAllFromServer(true);
await delay(1000);
await this.plugin.replicateAllFromServer(true);
await this.askHiddenFileConfiguration({ enableFetch: true });
}
async rebuildRemote() {
this.suspendExtraSync();
await this.plugin.realizeSettingSyncMode();
await this.plugin.markRemoteLocked();
await this.plugin.tryResetRemoteDatabase();
await this.plugin.markRemoteLocked();
await delay(500);
await this.askHiddenFileConfiguration({ enableOverwrite: true });
await delay(1000);
await this.plugin.replicateAllToServer(true);
await delay(1000);
await this.plugin.replicateAllToServer(true);
}
async rebuildEverything() {
this.suspendExtraSync();
await this.plugin.realizeSettingSyncMode();
await this.plugin.resetLocalDatabase();
await delay(1000);
await this.plugin.initializeDatabase(true);
await this.plugin.markRemoteLocked();
await this.plugin.tryResetRemoteDatabase();
await this.plugin.markRemoteLocked();
await delay(500);
await this.askHiddenFileConfiguration({ enableOverwrite: true });
await delay(1000);
await this.plugin.replicateAllToServer(true);
await delay(1000);
await this.plugin.replicateAllToServer(true);
}
}

13
src/ConflictResolveModal.ts
View File

@@ -1,17 +1,19 @@
import { App, Modal } from "obsidian";
import { App, Modal } from "./deps";
import { DIFF_DELETE, DIFF_EQUAL, DIFF_INSERT } from "diff-match-patch";
import { diff_result } from "./lib/src/types";
import { escapeStringToHTML } from "./lib/src/utils";
import { escapeStringToHTML } from "./lib/src/strbin";
export class ConflictResolveModal extends Modal {
// result: Array<[number, string]>;
result: diff_result;
filename: string;
callback: (remove_rev: string) => Promise<void>;
constructor(app: App, diff: diff_result, callback: (remove_rev: string) => Promise<void>) {
constructor(app: App, filename: string, diff: diff_result, callback: (remove_rev: string) => Promise<void>) {
super(app);
this.result = diff;
this.callback = callback;
this.filename = filename;
}
onOpen() {
@@ -20,6 +22,7 @@ export class ConflictResolveModal extends Modal {
contentEl.empty();
contentEl.createEl("h2", { text: "This document has conflicted changes." });
contentEl.createEl("span", { text: this.filename });
const div = contentEl.createDiv("");
div.addClass("op-scrollable");
let diff = "";
@@ -38,8 +41,8 @@ export class ConflictResolveModal extends Modal {
diff = diff.replace(/\n/g, "<br>");
div.innerHTML = diff;
const div2 = contentEl.createDiv("");
const date1 = new Date(this.result.left.mtime).toLocaleString();
const date2 = new Date(this.result.right.mtime).toLocaleString();
const date1 = new Date(this.result.left.mtime).toLocaleString() + (this.result.left.deleted ? " (Deleted)" : "");
const date2 = new Date(this.result.right.mtime).toLocaleString() + (this.result.right.deleted ? " (Deleted)" : "");
div2.innerHTML = `
<span class='deleted'>A:${date1}</span><br /><span class='added'>B:${date2}</span><br>
`;

110
src/DocumentHistoryModal.ts
View File

@@ -1,10 +1,13 @@
import { TFile, Modal, App } from "obsidian";
import { path2id } from "./utils";
import { escapeStringToHTML } from "./lib/src/utils";
import { TFile, Modal, App } from "./deps";
import { getPathFromTFile, isValidPath } from "./utils";
import { base64ToArrayBuffer, base64ToString, escapeStringToHTML } from "./lib/src/strbin";
import ObsidianLiveSyncPlugin from "./main";
import { DIFF_DELETE, DIFF_EQUAL, DIFF_INSERT, diff_match_patch } from "diff-match-patch";
import { LOG_LEVEL } from "./lib/src/types";
import { DocumentID, FilePathWithPrefix, LoadedEntry, LOG_LEVEL } from "./lib/src/types";
import { Logger } from "./lib/src/logger";
import { isErrorOfMissingDoc } from "./lib/src/utils_couchdb";
import { getDocData } from "./lib/src/utils";
import { stripPrefix } from "./lib/src/path";
export class DocumentHistoryModal extends Modal {
plugin: ObsidianLiveSyncPlugin;
@@ -13,51 +16,81 @@ export class DocumentHistoryModal extends Modal {
info: HTMLDivElement;
fileInfo: HTMLDivElement;
showDiff = false;
id: DocumentID;
file: string;
file: FilePathWithPrefix;
revs_info: PouchDB.Core.RevisionInfo[] = [];
currentDoc: LoadedEntry;
currentText = "";
currentDeleted = false;
constructor(app: App, plugin: ObsidianLiveSyncPlugin, file: TFile) {
constructor(app: App, plugin: ObsidianLiveSyncPlugin, file: TFile | FilePathWithPrefix, id: DocumentID) {
super(app);
this.plugin = plugin;
this.file = file.path;
this.file = (file instanceof TFile) ? getPathFromTFile(file) : file;
this.id = id;
if (!file) {
this.file = this.plugin.id2path(id, null);
}
if (localStorage.getItem("ols-history-highlightdiff") == "1") {
this.showDiff = true;
}
}
async loadFile() {
if (!this.id) {
this.id = await this.plugin.path2id(this.file);
}
const db = this.plugin.localDatabase;
const w = await db.localDatabase.get(path2id(this.file), { revs_info: true });
this.revs_info = w._revs_info.filter((e) => e.status == "available");
this.range.max = `${this.revs_info.length - 1}`;
this.range.value = this.range.max;
this.fileInfo.setText(`${this.file} / ${this.revs_info.length} revisions`);
await this.loadRevs();
try {
const w = await db.localDatabase.get(this.id, { revs_info: true });
this.revs_info = w._revs_info.filter((e) => e?.status == "available");
this.range.max = `${this.revs_info.length - 1}`;
this.range.value = this.range.max;
this.fileInfo.setText(`${this.file} / ${this.revs_info.length} revisions`);
await this.loadRevs();
} catch (ex) {
if (isErrorOfMissingDoc(ex)) {
this.range.max = "0";
this.range.value = "";
this.range.disabled = true;
this.showDiff
this.contentView.setText(`History of this file was not recorded.`);
} else {
this.contentView.setText(`Error occurred.`);
Logger(ex, LOG_LEVEL.VERBOSE);
}
}
}
async loadRevs() {
if (this.revs_info.length == 0) return;
const db = this.plugin.localDatabase;
const index = this.revs_info.length - 1 - (this.range.value as any) / 1;
const rev = this.revs_info[index];
const w = await db.getDBEntry(path2id(this.file), { rev: rev.rev }, false, false);
const w = await db.getDBEntry(this.file, { rev: rev.rev }, false, false, true);
this.currentText = "";
this.currentDeleted = false;
if (w === false) {
this.currentDeleted = true;
this.info.innerHTML = "";
this.contentView.innerHTML = `Could not read this revision<br>(${rev.rev})`;
} else {
this.currentDoc = w;
this.info.innerHTML = `Modified:${new Date(w.mtime).toLocaleString()}`;
let result = "";
this.currentText = w.data;
const w1data = w.datatype == "plain" ? getDocData(w.data) : base64ToString(w.data);
this.currentDeleted = w.deleted;
this.currentText = w1data;
if (this.showDiff) {
const prevRevIdx = this.revs_info.length - 1 - ((this.range.value as any) / 1 - 1);
if (prevRevIdx >= 0 && prevRevIdx < this.revs_info.length) {
const oldRev = this.revs_info[prevRevIdx].rev;
const w2 = await db.getDBEntry(path2id(this.file), { rev: oldRev }, false, false);
const w2 = await db.getDBEntry(this.file, { rev: oldRev }, false, false, true);
if (w2 != false) {
const dmp = new diff_match_patch();
const diff = dmp.diff_main(w2.data, w.data);
const w2data = w2.datatype == "plain" ? getDocData(w2.data) : base64ToString(w2.data);
const diff = dmp.diff_main(w2data, w1data);
dmp.diff_cleanupSemantic(diff);
for (const v of diff) {
const x1 = v[0];
@@ -73,15 +106,15 @@ export class DocumentHistoryModal extends Modal {
result = result.replace(/\n/g, "<br>");
} else {
result = escapeStringToHTML(w.data);
result = escapeStringToHTML(w1data);
}
} else {
result = escapeStringToHTML(w.data);
result = escapeStringToHTML(w1data);
}
} else {
result = escapeStringToHTML(w.data);
result = escapeStringToHTML(w1data);
}
this.contentView.innerHTML = result;
this.contentView.innerHTML = (this.currentDeleted ? "(At this revision, the file has been deleted)\n" : "") + result;
}
}
@@ -138,6 +171,39 @@ export class DocumentHistoryModal extends Modal {
Logger(`Old content copied to clipboard`, LOG_LEVEL.NOTICE);
});
});
async function focusFile(path: string) {
const targetFile = app.vault
.getFiles()
.find((f) => f.path === path);
if (targetFile) {
const leaf = app.workspace.getLeaf(false);
await leaf.openFile(targetFile);
} else {
Logger("The file could not view on the editor", LOG_LEVEL.NOTICE)
}
}
buttons.createEl("button", { text: "Back to this revision" }, (e) => {
e.addClass("mod-cta");
e.addEventListener("click", async () => {
// const pathToWrite = this.plugin.id2path(this.id, true);
const pathToWrite = stripPrefix(this.file);
if (!isValidPath(pathToWrite)) {
Logger("Path is not valid to write content.", LOG_LEVEL.INFO);
}
if (this.currentDoc?.datatype == "plain") {
await this.app.vault.adapter.write(pathToWrite, getDocData(this.currentDoc.data));
await focusFile(pathToWrite);
this.close();
} else if (this.currentDoc?.datatype == "newnote") {
await this.app.vault.adapter.writeBinary(pathToWrite, base64ToArrayBuffer(this.currentDoc.data));
await focusFile(pathToWrite);
this.close();
} else {
Logger(`Could not parse entry`, LOG_LEVEL.NOTICE);
}
});
});
}
onClose() {
const { contentEl } = this;

64
src/JsonResolveModal.ts Normal file
View File

@@ -0,0 +1,64 @@
import { App, Modal } from "./deps";
import { FilePath, LoadedEntry } from "./lib/src/types";
import JsonResolvePane from "./JsonResolvePane.svelte";
export class JsonResolveModal extends Modal {
// result: Array<[number, string]>;
filename: FilePath;
callback: (keepRev: string, mergedStr?: string) => Promise<void>;
docs: LoadedEntry[];
component: JsonResolvePane;
nameA: string;
nameB: string;
defaultSelect: string;
constructor(app: App, filename: FilePath, docs: LoadedEntry[], callback: (keepRev: string, mergedStr?: string) => Promise<void>, nameA?: string, nameB?: string, defaultSelect?: string) {
super(app);
this.callback = callback;
this.filename = filename;
this.docs = docs;
this.nameA = nameA;
this.nameB = nameB;
this.defaultSelect = defaultSelect;
}
async UICallback(keepRev: string, mergedStr?: string) {
this.close();
await this.callback(keepRev, mergedStr);
this.callback = null;
}
onOpen() {
const { contentEl } = this;
contentEl.empty();
if (this.component == null) {
this.component = new JsonResolvePane({
target: contentEl,
props: {
docs: this.docs,
filename: this.filename,
nameA: this.nameA,
nameB: this.nameB,
defaultSelect: this.defaultSelect,
callback: (keepRev, mergedStr) => this.UICallback(keepRev, mergedStr),
},
});
}
return;
}
onClose() {
const { contentEl } = this;
contentEl.empty();
// contentEl.empty();
if (this.callback != null) {
this.callback(null);
}
if (this.component != null) {
this.component.$destroy();
this.component = null;
}
}
}

174
src/JsonResolvePane.svelte Normal file
View File

@@ -0,0 +1,174 @@
<script lang="ts">
import { Diff, DIFF_DELETE, DIFF_INSERT, diff_match_patch } from "diff-match-patch";
import type { FilePath, LoadedEntry } from "./lib/src/types";
import { base64ToString } from "./lib/src/strbin";
import { getDocData } from "./lib/src/utils";
import { mergeObject } from "./utils";
export let docs: LoadedEntry[] = [];
export let callback: (keepRev: string, mergedStr?: string) => Promise<void> = async (_, __) => {
Promise.resolve();
};
export let filename: FilePath = "" as FilePath;
export let nameA: string = "A";
export let nameB: string = "B";
export let defaultSelect: string = "";
let docA: LoadedEntry = undefined;
let docB: LoadedEntry = undefined;
let docAContent = "";
let docBContent = "";
let objA: any = {};
let objB: any = {};
let objAB: any = {};
let objBA: any = {};
let diffs: Diff[];
type SelectModes = "" | "A" | "B" | "AB" | "BA";
let mode: SelectModes = defaultSelect as SelectModes;
function docToString(doc: LoadedEntry) {
return doc.datatype == "plain" ? getDocData(doc.data) : base64ToString(doc.data);
}
function revStringToRevNumber(rev: string) {
return rev.split("-")[0];
}
function getDiff(left: string, right: string) {
const dmp = new diff_match_patch();
const mapLeft = dmp.diff_linesToChars_(left, right);
const diffLeftSrc = dmp.diff_main(mapLeft.chars1, mapLeft.chars2, false);
dmp.diff_charsToLines_(diffLeftSrc, mapLeft.lineArray);
return diffLeftSrc;
}
function getJsonDiff(a: object, b: object) {
return getDiff(JSON.stringify(a, null, 2), JSON.stringify(b, null, 2));
}
function apply() {
if (docA._id == docB._id) {
if (mode == "A") return callback(docA._rev, null);
if (mode == "B") return callback(docB._rev, null);
} else {
if (mode == "A") return callback(null, docToString(docA));
if (mode == "B") return callback(null, docToString(docB));
}
if (mode == "BA") return callback(null, JSON.stringify(objBA, null, 2));
if (mode == "AB") return callback(null, JSON.stringify(objAB, null, 2));
callback(null, null);
}
$: {
if (docs && docs.length >= 1) {
if (docs[0].mtime < docs[1].mtime) {
docA = docs[0];
docB = docs[1];
} else {
docA = docs[1];
docB = docs[0];
}
docAContent = docToString(docA);
docBContent = docToString(docB);
try {
objA = false;
objB = false;
objA = JSON.parse(docAContent);
objB = JSON.parse(docBContent);
objAB = mergeObject(objA, objB);
objBA = mergeObject(objB, objA);
if (JSON.stringify(objAB) == JSON.stringify(objBA)) {
objBA = false;
}
} catch (ex) {
objBA = false;
objAB = false;
}
}
}
$: mergedObjs = {
"": false,
A: objA,
B: objB,
AB: objAB,
BA: objBA,
};
$: selectedObj = mode in mergedObjs ? mergedObjs[mode] : {};
$: {
diffs = getJsonDiff(objA, selectedObj);
}
$: modes = [
["", "Not now"],
["A", nameA || "A"],
["B", nameB || "B"],
["AB", `${nameA || "A"} + ${nameB || "B"}`],
["BA", `${nameB || "B"} + ${nameA || "A"}`],
] as ["" | "A" | "B" | "AB" | "BA", string][];
</script>
<h1>Conflicted settings</h1>
<h2>{filename}</h2>
{#if !docA || !docB}
<div class="message">Just for a minute, please!</div>
<div class="buttons">
<button on:click={apply}>Dismiss</button>
</div>
{:else}
<div class="options">
{#each modes as m}
{#if m[0] == "" || mergedObjs[m[0]] != false}
<label class={`sls-setting-label ${m[0] == mode ? "selected" : ""}`}
><input type="radio" name="disp" bind:group={mode} value={m[0]} class="sls-setting-tab" />
<div class="sls-setting-menu-btn">{m[1]}</div></label
>
{/if}
{/each}
</div>
{#if selectedObj != false}
<div class="op-scrollable json-source">
{#each diffs as diff}
<span class={diff[0] == DIFF_DELETE ? "deleted" : diff[0] == DIFF_INSERT ? "added" : "normal"}>{diff[1]}</span>
{/each}
</div>
{:else}
NO PREVIEW
{/if}
<div>
{nameA}
{#if docA._id == docB._id} Rev:{revStringToRevNumber(docA._rev)} {/if} ,{new Date(docA.mtime).toLocaleString()}
{docAContent.length} letters
</div>
<div>
{nameB}
{#if docA._id == docB._id} Rev:{revStringToRevNumber(docB._rev)} {/if} ,{new Date(docB.mtime).toLocaleString()}
{docBContent.length} letters
</div>
<div class="buttons">
<button on:click={apply}>Apply</button>
</div>
{/if}
<style>
.deleted {
text-decoration: line-through;
}
* {
box-sizing: border-box;
}
.scroller {
display: flex;
flex-direction: column;
overflow-y: scroll;
max-height: 60vh;
user-select: text;
}
.json-source {
white-space: pre;
height: auto;
overflow: auto;
min-height: var(--font-ui-medium);
flex-grow: 1;
}
</style>

52
src/KeyValueDB.ts Normal file
View File

@@ -0,0 +1,52 @@
import { deleteDB, IDBPDatabase, openDB } from "idb";
export interface KeyValueDatabase {
get<T>(key: string): Promise<T>;
set<T>(key: string, value: T): Promise<IDBValidKey>;
del(key: string): Promise<void>;
clear(): Promise<void>;
keys(query?: IDBValidKey | IDBKeyRange, count?: number): Promise<IDBValidKey[]>;
close(): void;
destroy(): void;
}
const databaseCache: { [key: string]: IDBPDatabase<any> } = {};
export const OpenKeyValueDatabase = async (dbKey: string): Promise<KeyValueDatabase> => {
if (dbKey in databaseCache) {
databaseCache[dbKey].close();
delete databaseCache[dbKey];
}
const storeKey = dbKey;
const dbPromise = openDB(dbKey, 1, {
upgrade(db) {
db.createObjectStore(storeKey);
},
});
let db: IDBPDatabase<any> = null;
db = await dbPromise;
databaseCache[dbKey] = db;
return {
get<T>(key: string): Promise<T> {
return db.get(storeKey, key);
},
set<T>(key: string, value: T) {
return db.put(storeKey, value, key);
},
del(key: string) {
return db.delete(storeKey, key);
},
clear() {
return db.clear(storeKey);
},
keys(query?: IDBValidKey | IDBKeyRange, count?: number) {
return db.getAllKeys(storeKey, query, count);
},
close() {
delete databaseCache[dbKey];
return db.close();
},
async destroy() {
delete databaseCache[dbKey];
db.close();
await deleteDB(dbKey);
},
};
};

37
src/LiveSyncCommands.ts Normal file
View File

@@ -0,0 +1,37 @@
import { AnyEntry, DocumentID, EntryDoc, EntryHasPath, FilePath, FilePathWithPrefix } from "./lib/src/types";
import { PouchDB } from "./lib/src/pouchdb-browser.js";
import type ObsidianLiveSyncPlugin from "./main";
export abstract class LiveSyncCommands {
plugin: ObsidianLiveSyncPlugin;
get app() {
return this.plugin.app;
}
get settings() {
return this.plugin.settings;
}
get localDatabase() {
return this.plugin.localDatabase;
}
id2path(id: DocumentID, entry?: EntryHasPath, stripPrefix?: boolean): FilePathWithPrefix {
return this.plugin.id2path(id, entry, stripPrefix);
}
async path2id(filename: FilePathWithPrefix | FilePath, prefix?: string): Promise<DocumentID> {
return await this.plugin.path2id(filename, prefix);
}
getPath(entry: AnyEntry): FilePathWithPrefix {
return this.plugin.getPath(entry);
}
constructor(plugin: ObsidianLiveSyncPlugin) {
this.plugin = plugin;
}
abstract onunload(): void;
abstract onload(): void | Promise<void>;
abstract onInitializeDatabase(showNotice: boolean): void | Promise<void>;
abstract beforeReplicate(showNotice: boolean): void | Promise<void>;
abstract onResume(): void | Promise<void>;
abstract parseReplicationResultItem(docs: PouchDB.Core.ExistingDocument<EntryDoc>): Promise<boolean> | boolean;
abstract realizeSettingSyncMode(): Promise<void>;
}

1373
src/LocalPouchDB.ts
View File

File diff suppressed because it is too large Load Diff

27
src/LogDisplayModal.ts
View File

@@ -1,21 +1,17 @@
import { App, Modal } from "obsidian";
import { escapeStringToHTML } from "./lib/src/utils";
import { App, Modal } from "./deps";
import { logMessageStore } from "./lib/src/stores";
import { escapeStringToHTML } from "./lib/src/strbin";
import ObsidianLiveSyncPlugin from "./main";
export class LogDisplayModal extends Modal {
plugin: ObsidianLiveSyncPlugin;
logEl: HTMLDivElement;
unsubscribe: () => void;
constructor(app: App, plugin: ObsidianLiveSyncPlugin) {
super(app);
this.plugin = plugin;
}
updateLog() {
let msg = "";
for (const v of this.plugin.logMessage) {
msg += escapeStringToHTML(v) + "<br>";
}
this.logEl.innerHTML = msg;
}
onOpen() {
const { contentEl } = this;
@@ -25,13 +21,18 @@ export class LogDisplayModal extends Modal {
div.addClass("op-scrollable");
div.addClass("op-pre");
this.logEl = div;
this.updateLog = this.updateLog.bind(this);
this.plugin.addLogHook = this.updateLog;
this.updateLog();
this.unsubscribe = logMessageStore.observe((e) => {
let msg = "";
for (const v of e) {
msg += escapeStringToHTML(v) + "<br>";
}
this.logEl.innerHTML = msg;
})
logMessageStore.invalidate();
}
onClose() {
const { contentEl } = this;
contentEl.empty();
this.plugin.addLogHook = null;
if (this.unsubscribe) this.unsubscribe();
}
}

1655
src/ObsidianLiveSyncSettingTab.ts
View File

File diff suppressed because it is too large Load Diff

314
src/PluginCombo.svelte Normal file
View File

@@ -0,0 +1,314 @@
<script lang="ts">
import type { PluginDataExDisplay } from "./CmdConfigSync";
import { Logger } from "./lib/src/logger";
import { versionNumberString2Number } from "./lib/src/strbin";
import { FilePath, LOG_LEVEL } from "./lib/src/types";
import { getDocData } from "./lib/src/utils";
import type ObsidianLiveSyncPlugin from "./main";
import { askString, scheduleTask } from "./utils";
export let list: PluginDataExDisplay[] = [];
export let thisTerm = "";
export let hideNotApplicable = false;
export let selectNewest = 0;
export let applyAllPluse = 0;
export let applyData: (data: PluginDataExDisplay) => Promise<boolean>;
export let compareData: (dataA: PluginDataExDisplay, dataB: PluginDataExDisplay) => Promise<boolean>;
export let deleteData: (data: PluginDataExDisplay) => Promise<boolean>;
export let hidden: boolean;
export let plugin: ObsidianLiveSyncPlugin;
export let isMaintenanceMode: boolean = false;
const addOn = plugin.addOnConfigSync;
let selected = "";
let freshness = "";
let equivalency = "";
let version = "";
let canApply: boolean = false;
let canCompare: boolean = false;
let currentSelectNewest = 0;
let currentApplyAll = 0;
// Selectable terminals
let terms = [] as string[];
async function comparePlugin(local: PluginDataExDisplay, remote: PluginDataExDisplay) {
let freshness = "";
let equivalency = "";
let version = "";
let contentCheck = false;
let canApply: boolean = false;
let canCompare = false;
if (!local && !remote) {
// NO OP. whats happened?
freshness = "";
} else if (local && !remote) {
freshness = "⚠ Local only";
} else if (remote && !local) {
freshness = "✓ Remote only";
canApply = true;
} else {
const dtDiff = (local?.mtime ?? 0) - (remote?.mtime ?? 0);
if (dtDiff / 1000 < -10) {
freshness = "✓ Newer";
canApply = true;
contentCheck = true;
} else if (dtDiff / 1000 > 10) {
freshness = "⚠ Older";
canApply = true;
contentCheck = true;
} else {
freshness = "⚖️ Same old";
canApply = false;
contentCheck = true;
}
}
const localVersionStr = local?.version || "0.0.0";
const remoteVersionStr = remote?.version || "0.0.0";
if (local?.version || remote?.version) {
const localVersion = versionNumberString2Number(localVersionStr);
const remoteVersion = versionNumberString2Number(remoteVersionStr);
if (localVersion == remoteVersion) {
version = "⚖️ Same ver.";
} else if (localVersion > remoteVersion) {
version = `⚠ Lower ${localVersionStr} > ${remoteVersionStr}`;
} else if (localVersion < remoteVersion) {
version = `✓ Higher ${localVersionStr} < ${remoteVersionStr}`;
}
}
if (contentCheck) {
const { canApply, equivalency, canCompare } = await checkEquivalency(local, remote);
return { canApply, freshness, equivalency, version, canCompare };
}
return { canApply, freshness, equivalency, version, canCompare };
}
async function checkEquivalency(local: PluginDataExDisplay, remote: PluginDataExDisplay) {
let equivalency = "";
let canApply = false;
let canCompare = false;
const filenames = [...new Set([...local.files.map((e) => e.filename), ...remote.files.map((e) => e.filename)])];
const matchingStatus = filenames
.map((filename) => {
const localFile = local.files.find((e) => e.filename == filename);
const remoteFile = remote.files.find((e) => e.filename == filename);
if (!localFile && !remoteFile) {
return 0b0000000;
} else if (localFile && !remoteFile) {
return 0b0000010; //"LOCAL_ONLY";
} else if (!localFile && remoteFile) {
return 0b0001000; //"REMOTE ONLY"
} else {
if (getDocData(localFile.data) == getDocData(remoteFile.data)) {
return 0b0000100; //"EVEN"
} else {
return 0b0010000; //"DIFFERENT";
}
}
})
.reduce((p, c) => p | c, 0);
if (matchingStatus == 0b0000100) {
equivalency = "⚖️ Same";
canApply = false;
} else if (matchingStatus <= 0b0000100) {
equivalency = "Same or local only";
canApply = false;
} else if (matchingStatus == 0b0010000) {
canApply = true;
canCompare = true;
equivalency = "≠ Different";
} else {
canApply = true;
canCompare = true;
equivalency = "≠ Different";
}
return { equivalency, canApply, canCompare };
}
async function performCompare(local: PluginDataExDisplay, remote: PluginDataExDisplay) {
const result = await comparePlugin(local, remote);
canApply = result.canApply;
freshness = result.freshness;
equivalency = result.equivalency;
version = result.version;
canCompare = result.canCompare;
if (local?.files.length != 1 || !local?.files?.first()?.filename?.endsWith(".json")) {
canCompare = false;
}
}
async function updateTerms(list: PluginDataExDisplay[], selectNewest: boolean, isMaintenanceMode: boolean) {
const local = list.find((e) => e.term == thisTerm);
selected = "";
if (isMaintenanceMode) {
terms = [...new Set(list.map((e) => e.term))];
} else if (hideNotApplicable) {
const termsTmp = [];
const wk = [...new Set(list.map((e) => e.term))];
for (const termName of wk) {
const remote = list.find((e) => e.term == termName);
if ((await comparePlugin(local, remote)).canApply) {
termsTmp.push(termName);
}
}
terms = [...termsTmp];
} else {
terms = [...new Set(list.map((e) => e.term))].filter((e) => e != thisTerm);
}
let newest: PluginDataExDisplay = local;
if (selectNewest) {
for (const term of terms) {
const remote = list.find((e) => e.term == term);
if (remote && remote.mtime && (newest?.mtime || 0) < remote.mtime) {
newest = remote;
}
}
if (newest && newest.term != thisTerm) {
selected = newest.term;
}
// selectNewest = false;
}
}
$: {
// React pulse and select
const doSelectNewest = selectNewest != currentSelectNewest;
currentSelectNewest = selectNewest;
updateTerms(list, doSelectNewest, isMaintenanceMode);
}
$: {
// React pulse and apply
const doApply = applyAllPluse != currentApplyAll;
currentApplyAll = applyAllPluse;
if (doApply && selected) {
if (!hidden) {
applySelected();
}
}
}
$: {
freshness = "";
equivalency = "";
version = "";
canApply = false;
if (selected == "") {
// NO OP.
} else if (selected == thisTerm) {
freshness = "This device";
canApply = false;
} else {
const local = list.find((e) => e.term == thisTerm);
const remote = list.find((e) => e.term == selected);
performCompare(local, remote);
}
}
async function applySelected() {
const local = list.find((e) => e.term == thisTerm);
const selectedItem = list.find((e) => e.term == selected);
if (selectedItem && (await applyData(selectedItem))) {
scheduleTask("update-plugin-list", 250, () => addOn.updatePluginList(local.documentPath));
}
}
async function compareSelected() {
const local = list.find((e) => e.term == thisTerm);
const selectedItem = list.find((e) => e.term == selected);
if (local && selectedItem && (await compareData(local, selectedItem))) {
scheduleTask("update-plugin-list", 250, () => addOn.updatePluginList(local.documentPath));
}
}
async function deleteSelected() {
const selectedItem = list.find((e) => e.term == selected);
const deletedPath = selectedItem.documentPath;
if (selectedItem && (await deleteData(selectedItem))) {
scheduleTask("update-plugin-list", 250, () => addOn.updatePluginList(deletedPath));
}
}
async function duplicateItem() {
const local = list.find((e) => e.term == thisTerm);
const duplicateTermName = await askString(plugin.app, "Duplicate", "device name", "");
if (duplicateTermName) {
if (duplicateTermName.contains("/")) {
Logger(`We can not use "/" to the device name`, LOG_LEVEL.NOTICE);
return;
}
const key = `${plugin.app.vault.configDir}/${local.files[0].filename}`;
await addOn.storeCustomizationFiles(key as FilePath, duplicateTermName);
await addOn.updatePluginList(addOn.filenameToUnifiedKey(key, duplicateTermName));
}
}
</script>
{#if terms.length > 0}
<span class="spacer" />
{#if !hidden}
<span class="messages">
<span class="message">{freshness}</span>
<span class="message">{equivalency}</span>
<span class="message">{version}</span>
</span>
<select bind:value={selected}>
<option value={""}>-</option>
{#each terms as term}
<option value={term}>{term}</option>
{/each}
</select>
{#if canApply || (isMaintenanceMode && selected != "")}
{#if canCompare}
<button on:click={compareSelected}>🔍</button>
{:else}
<button disabled />
{/if}
<button on:click={applySelected}>✓</button>
{:else}
<button disabled />
<button disabled />
{/if}
{#if isMaintenanceMode}
{#if selected != ""}
<button on:click={deleteSelected}>🗑️</button>
{:else}
<button on:click={duplicateItem}>📑</button>
{/if}
{/if}
{/if}
{:else}
<span class="spacer" />
<span class="message even">All devices are even</span>
<button disabled />
<button disabled />
{/if}
<style>
.spacer {
flex-grow: 1;
}
button {
margin: 2px 4px;
min-width: 3em;
max-width: 4em;
}
button:disabled {
border: none;
box-shadow: none;
background-color: transparent;
visibility: collapse;
}
button:disabled:hover {
border: none;
box-shadow: none;
background-color: transparent;
visibility: collapse;
}
span.message {
color: var(--text-muted);
font-size: var(--font-ui-smaller);
padding: 0 1em;
line-height: var(--line-height-tight);
}
span.messages {
display: flex;
flex-direction: column;
align-items: center;
}
</style>

440
src/PluginPane.svelte
View File

@@ -1,295 +1,221 @@
<script lang="ts">
import ObsidianLiveSyncPlugin from "./main";
import { onMount } from "svelte";
import { DevicePluginList, PluginDataEntry } from "./types";
import { versionNumberString2Number } from "./lib/src/utils";
type JudgeResult = "" | "NEWER" | "EVEN" | "EVEN_BUT_DIFFERENT" | "OLDER" | "REMOTE_ONLY";
interface PluginDataEntryDisp extends PluginDataEntry {
versionInfo: string;
mtimeInfo: string;
mtimeFlag: JudgeResult;
versionFlag: JudgeResult;
}
import ObsidianLiveSyncPlugin from "./main";
import { PluginDataExDisplay, pluginIsEnumerating, pluginList } from "./CmdConfigSync";
import PluginCombo from "./PluginCombo.svelte";
export let plugin: ObsidianLiveSyncPlugin;
let plugins: PluginDataEntry[] = [];
let deviceAndPlugins: { [key: string]: PluginDataEntryDisp[] } = {};
let devicePluginList: [string, PluginDataEntryDisp[]][] = null;
let ownPlugins: DevicePluginList = null;
let showOwnPlugins = false;
let targetList: { [key: string]: boolean } = {};
function saveTargetList() {
window.localStorage.setItem("ols-plugin-targetlist", JSON.stringify(targetList));
$: hideNotApplicable = true;
$: thisTerm = plugin.deviceAndVaultName;
const addOn = plugin.addOnConfigSync;
let list: PluginDataExDisplay[] = [];
let selectNewestPulse = 0;
let hideEven = true;
let loading = false;
let applyAllPluse = 0;
let isMaintenanceMode = false;
async function requestUpdate() {
await addOn.updatePluginList();
}
function loadTargetList() {
let e = window.localStorage.getItem("ols-plugin-targetlist") || "{}";
try {
targetList = JSON.parse(e);
} catch (_) {
// NO OP.
}
async function requestReload() {
await addOn.reloadPluginList();
}
function clearSelection() {
targetList = {};
}
async function updateList() {
let x = await plugin.getPluginList();
ownPlugins = x.thisDevicePlugins;
plugins = Object.values(x.allPlugins);
let targetListItems = Array.from(new Set(plugins.map((e) => e.deviceVaultName + "---" + e.manifest.id)));
let newTargetList: { [key: string]: boolean } = {};
for (const id of targetListItems) {
for (const tag of ["---plugin", "---setting"]) {
newTargetList[id + tag] = id + tag in targetList && targetList[id + tag];
}
}
targetList = newTargetList;
saveTargetList();
}
$: {
deviceAndPlugins = {};
for (const p of plugins) {
if (p.deviceVaultName == plugin.deviceAndVaultName && !showOwnPlugins) {
continue;
}
if (!(p.deviceVaultName in deviceAndPlugins)) {
deviceAndPlugins[p.deviceVaultName] = [];
}
let dispInfo: PluginDataEntryDisp = { ...p, versionInfo: "", mtimeInfo: "", versionFlag: "", mtimeFlag: "" };
dispInfo.versionInfo = p.manifest.version;
let x = new Date().getTime() / 1000;
let mtime = p.mtime / 1000;
let diff = (x - mtime) / 60;
if (p.mtime == 0) {
dispInfo.mtimeInfo = `-`;
} else if (diff < 60) {
dispInfo.mtimeInfo = `${diff | 0} Mins ago`;
} else if (diff < 60 * 24) {
dispInfo.mtimeInfo = `${(diff / 60) | 0} Hours ago`;
} else if (diff < 60 * 24 * 10) {
dispInfo.mtimeInfo = `${(diff / (60 * 24)) | 0} Days ago`;
} else {
dispInfo.mtimeInfo = new Date(dispInfo.mtime).toLocaleString();
}
// compare with own plugin
let id = p.manifest.id;
if (id in ownPlugins) {
// Which we have.
const ownPlugin = ownPlugins[id];
let localVer = versionNumberString2Number(ownPlugin.manifest.version);
let pluginVer = versionNumberString2Number(p.manifest.version);
if (localVer > pluginVer) {
dispInfo.versionFlag = "OLDER";
} else if (localVer == pluginVer) {
if (ownPlugin.manifestJson + (ownPlugin.styleCss ?? "") + ownPlugin.mainJs != p.manifestJson + (p.styleCss ?? "") + p.mainJs) {
dispInfo.versionFlag = "EVEN_BUT_DIFFERENT";
} else {
dispInfo.versionFlag = "EVEN";
}
} else if (localVer < pluginVer) {
dispInfo.versionFlag = "NEWER";
}
if ((ownPlugin.dataJson ?? "") == (p.dataJson ?? "")) {
if (ownPlugin.mtime == 0 && p.mtime == 0) {
dispInfo.mtimeFlag = "";
} else {
dispInfo.mtimeFlag = "EVEN";
}
} else {
if (((ownPlugin.mtime / 1000) | 0) > ((p.mtime / 1000) | 0)) {
dispInfo.mtimeFlag = "OLDER";
} else if (((ownPlugin.mtime / 1000) | 0) == ((p.mtime / 1000) | 0)) {
dispInfo.mtimeFlag = "EVEN_BUT_DIFFERENT";
} else if (((ownPlugin.mtime / 1000) | 0) < ((p.mtime / 1000) | 0)) {
dispInfo.mtimeFlag = "NEWER";
}
}
} else {
dispInfo.versionFlag = "REMOTE_ONLY";
dispInfo.mtimeFlag = "REMOTE_ONLY";
}
deviceAndPlugins[p.deviceVaultName].push(dispInfo);
}
devicePluginList = Object.entries(deviceAndPlugins);
}
function getDispString(stat: JudgeResult): string {
if (stat == "") return "";
if (stat == "NEWER") return " (Newer)";
if (stat == "OLDER") return " (Older)";
if (stat == "EVEN") return " (Even)";
if (stat == "EVEN_BUT_DIFFERENT") return " (Even but different)";
if (stat == "REMOTE_ONLY") return " (Remote Only)";
return "";
}
pluginList.subscribe((e) => {
list = e;
});
pluginIsEnumerating.subscribe((e) => {
loading = e;
});
onMount(async () => {
loadTargetList();
await updateList();
requestUpdate();
});
function toggleShowOwnPlugins() {
showOwnPlugins = !showOwnPlugins;
function filterList(list: PluginDataExDisplay[], categories: string[]) {
const w = list.filter((e) => categories.indexOf(e.category) !== -1);
return w.sort((a, b) => `${a.category}-${a.name}`.localeCompare(`${b.category}-${b.name}`));
}
function toggleTarget(key: string) {
targetList[key] = !targetList[key];
saveTargetList();
}
function toggleAll(devicename: string) {
for (const c in targetList) {
if (c.startsWith(devicename)) {
targetList[c] = true;
}
function groupBy(items: PluginDataExDisplay[], key: string) {
let ret = {} as Record<string, PluginDataExDisplay[]>;
for (const v of items) {
//@ts-ignore
const k = (key in v ? v[key] : "") as string;
ret[k] = ret[k] || [];
ret[k].push(v);
}
}
async function sweepPlugins() {
//@ts-ignore
await plugin.app.plugins.loadManifests();
await plugin.sweepPlugin(true);
updateList();
}
async function applyPlugins() {
for (const c in targetList) {
if (targetList[c] == true) {
const [deviceAndVault, id, opt] = c.split("---");
if (deviceAndVault in deviceAndPlugins) {
const entry = deviceAndPlugins[deviceAndVault].find((e) => e.manifest.id == id);
if (entry) {
if (opt == "plugin") {
if (entry.versionFlag != "EVEN") await plugin.applyPlugin(entry);
} else if (opt == "setting") {
if (entry.mtimeFlag != "EVEN") await plugin.applyPluginData(entry);
}
}
}
}
for (const k in ret) {
ret[k] = ret[k].sort((a, b) => `${a.category}-${a.name}`.localeCompare(`${b.category}-${b.name}`));
}
//@ts-ignore
await plugin.app.plugins.loadManifests();
await plugin.sweepPlugin(true);
updateList();
const w = Object.entries(ret);
return w.sort(([a], [b]) => `${a}`.localeCompare(`${b}`));
}
async function checkUpdates() {
await plugin.checkPluginUpdate();
const displays = {
CONFIG: "Configuration",
THEME: "Themes",
SNIPPET: "Snippets",
};
async function scanAgain() {
await addOn.scanAllConfigFiles(true);
await requestUpdate();
}
async function replicateAndRefresh() {
async function replicate() {
await plugin.replicate(true);
updateList();
}
function selectAllNewest() {
selectNewestPulse++;
}
function applyAll() {
applyAllPluse++;
}
async function applyData(data: PluginDataExDisplay): Promise<boolean> {
return await addOn.applyData(data);
}
async function compareData(docA: PluginDataExDisplay, docB: PluginDataExDisplay): Promise<boolean> {
return await addOn.compareUsingDisplayData(docA, docB);
}
async function deleteData(data: PluginDataExDisplay): Promise<boolean> {
return await addOn.deleteData(data);
}
$: options = {
thisTerm,
hideNotApplicable,
selectNewest: selectNewestPulse,
applyAllPluse,
applyData,
compareData,
deleteData,
plugin,
isMaintenanceMode,
};
</script>
<div>
<h1>Plugins and their settings</h1>
<div class="ols-plugins-div-buttons">
Show own items
<div class="checkbox-container" class:is-enabled={showOwnPlugins} on:click={toggleShowOwnPlugins} />
</div>
<div class="sls-plugins-wrap">
<table class="sls-plugins-tbl">
<tr style="position:sticky">
<th class="sls-plugins-tbl-device-head">Name</th>
<th class="sls-plugins-tbl-device-head">Info</th>
<th class="sls-plugins-tbl-device-head">Target</th>
</tr>
{#if !devicePluginList}
<tr>
<td colspan="3" class="sls-table-tail tcenter"> Retrieving... </td>
</tr>
{:else if devicePluginList.length == 0}
<tr>
<td colspan="3" class="sls-table-tail tcenter"> No plugins found. </td>
</tr>
{:else}
{#each devicePluginList as [deviceName, devicePlugins]}
<tr>
<th colspan="2" class="sls-plugins-tbl-device-head">{deviceName}</th>
<th class="sls-plugins-tbl-device-head">
<button class="mod-cta" on:click={() => toggleAll(deviceName)}>✔</button>
</th>
</tr>
{#each devicePlugins as plugin}
<tr>
<td class="sls-table-head">{plugin.manifest.name}</td>
<td class="sls-table-tail tcenter">{plugin.versionInfo}{getDispString(plugin.versionFlag)}</td>
<td class="sls-table-tail tcenter">
{#if plugin.versionFlag === "EVEN" || plugin.versionFlag === ""}
-
{:else}
<div class="wrapToggle">
<div
class="checkbox-container"
class:is-enabled={targetList[plugin.deviceVaultName + "---" + plugin.manifest.id + "---plugin"]}
on:click={() => toggleTarget(plugin.deviceVaultName + "---" + plugin.manifest.id + "---plugin")}
/>
</div>
{/if}
</td>
</tr>
<tr>
<td class="sls-table-head">Settings</td>
<td class="sls-table-tail tcenter">{plugin.mtimeInfo}{getDispString(plugin.mtimeFlag)}</td>
<td class="sls-table-tail tcenter">
{#if plugin.mtimeFlag === "EVEN" || plugin.mtimeFlag === ""}
-
{:else}
<div class="wrapToggle">
<div
class="checkbox-container"
class:is-enabled={targetList[plugin.deviceVaultName + "---" + plugin.manifest.id + "---setting"]}
on:click={() => toggleTarget(plugin.deviceVaultName + "---" + plugin.manifest.id + "---setting")}
/>
</div>
{/if}
</td>
</tr>
<tr class="divider">
<th colspan="3" />
</tr>
{/each}
{/each}
<div>
<h1>Customization sync</h1>
<div class="buttons">
<button on:click={() => scanAgain()}>Scan changes</button>
<button on:click={() => replicate()}>Sync once</button>
<button on:click={() => requestUpdate()}>Refresh</button>
{#if isMaintenanceMode}
<button on:click={() => requestReload()}>Reload</button>
{/if}
</table>
<button on:click={() => selectAllNewest()}>Select All Shiny</button>
</div>
<div class="buttons">
<button on:click={() => applyAll()}>Apply All</button>
</div>
</div>
<div class="ols-plugins-div-buttons">
<button class="" on:click={replicateAndRefresh}>Replicate and refresh</button>
<button class="" on:click={clearSelection}>Clear Selection</button>
{#if loading}
<div>
<span>Updating list...</span>
</div>
{/if}
<div class="list">
{#if list.length == 0}
<div class="center">No Items.</div>
{:else}
{#each Object.entries(displays) as [key, label]}
<div>
<h3>{label}</h3>
{#each groupBy(filterList(list, [key]), "name") as [name, listX]}
<div class="labelrow {hideEven ? 'hideeven' : ''}">
<div class="title">
{name}
</div>
<PluginCombo {...options} list={listX} hidden={false} />
</div>
{/each}
</div>
{/each}
<div>
<h3>Plugins</h3>
{#each groupBy(filterList(list, ["PLUGIN_MAIN", "PLUGIN_DATA", "PLUGIN_ETC"]), "name") as [name, listX]}
<div class="labelrow {hideEven ? 'hideeven' : ''}">
<div class="title">
{name}
</div>
<PluginCombo {...options} list={listX} hidden={true} />
</div>
<div class="filerow {hideEven ? 'hideeven' : ''}">
<div class="filetitle">Main</div>
<PluginCombo {...options} list={filterList(listX, ["PLUGIN_MAIN"])} hidden={false} />
</div>
<div class="filerow {hideEven ? 'hideeven' : ''}">
<div class="filetitle">Data</div>
<PluginCombo {...options} list={filterList(listX, ["PLUGIN_DATA"])} hidden={false} />
</div>
{/each}
</div>
{/if}
</div>
<div class="ols-plugins-div-buttons">
<button class="mod-cta" on:click={checkUpdates}>Check Updates</button>
<button class="mod-cta" on:click={sweepPlugins}>Scan installed</button>
<button class="mod-cta" on:click={applyPlugins}>Apply all</button>
<div class="buttons">
<label><span>Hide not applicable items</span><input type="checkbox" bind:checked={hideEven} /></label>
</div>
<div class="buttons">
<label><span>Maintenance mode</span><input type="checkbox" bind:checked={isMaintenanceMode} /></label>
</div>
<!-- <div class="ols-plugins-div-buttons">-->
<!-- <button class="mod-warning" on:click={applyPlugins}>Delete all selected</button>-->
<!-- </div>-->
</div>
<style>
.ols-plugins-div-buttons {
.labelrow {
margin-left: 0.4em;
display: flex;
justify-content: flex-start;
align-items: center;
border-top: 1px solid var(--background-modifier-border);
padding: 4px;
}
.filerow {
margin-left: 1.25em;
display: flex;
justify-content: flex-start;
align-items: center;
padding-right: 4px;
}
.filerow.hideeven:has(.even),
.labelrow.hideeven:has(.even) {
display: none;
}
.title {
color: var(--text-normal);
font-size: var(--font-ui-medium);
line-height: var(--line-height-tight);
margin-right: auto;
}
.filetitle {
color: var(--text-normal);
font-size: var(--font-ui-medium);
line-height: var(--line-height-tight);
margin-right: auto;
}
.buttons {
display: flex;
flex-direction: row;
justify-content: flex-end;
margin-top: 8px;
}
.buttons > button {
margin-left: 4px;
}
.wrapToggle {
label {
display: flex;
justify-content: center;
align-content: center;
align-items: center;
}
label > span {
margin-right: 0.25em;
}
.center {
display: flex;
justify-content: center;
align-items: center;
min-height: 3em;
}
</style>

172
src/StorageEventManager.ts Normal file
View File

@@ -0,0 +1,172 @@
import { Plugin_2, TAbstractFile, TFile, TFolder } from "./deps";
import { isPlainText, shouldBeIgnored } from "./lib/src/path";
import { getGlobalStore } from "./lib/src/store";
import { FilePath, ObsidianLiveSyncSettings } from "./lib/src/types";
import { FileEventItem, FileEventType, FileInfo, InternalFileInfo, queueItem } from "./types";
import { recentlyTouched } from "./utils";
export abstract class StorageEventManager {
abstract fetchEvent(): FileEventItem | false;
abstract cancelRelativeEvent(item: FileEventItem): void;
abstract getQueueLength(): number;
}
type LiveSyncForStorageEventManager = Plugin_2 &
{
settings: ObsidianLiveSyncSettings
} & {
isTargetFile: (file: string | TAbstractFile) => boolean,
procFileEvent: (applyBatch?: boolean) => Promise<boolean>
};
export class StorageEventManagerObsidian extends StorageEventManager {
plugin: LiveSyncForStorageEventManager;
queuedFilesStore = getGlobalStore("queuedFiles", { queuedItems: [] as queueItem[], fileEventItems: [] as FileEventItem[] });
watchedFileEventQueue = [] as FileEventItem[];
constructor(plugin: LiveSyncForStorageEventManager) {
super();
this.plugin = plugin;
this.watchVaultChange = this.watchVaultChange.bind(this);
this.watchVaultCreate = this.watchVaultCreate.bind(this);
this.watchVaultDelete = this.watchVaultDelete.bind(this);
this.watchVaultRename = this.watchVaultRename.bind(this);
this.watchVaultRawEvents = this.watchVaultRawEvents.bind(this);
plugin.registerEvent(app.vault.on("modify", this.watchVaultChange));
plugin.registerEvent(app.vault.on("delete", this.watchVaultDelete));
plugin.registerEvent(app.vault.on("rename", this.watchVaultRename));
plugin.registerEvent(app.vault.on("create", this.watchVaultCreate));
//@ts-ignore : Internal API
plugin.registerEvent(app.vault.on("raw", this.watchVaultRawEvents));
}
watchVaultCreate(file: TAbstractFile, ctx?: any) {
this.appendWatchEvent([{ type: "CREATE", file }], ctx);
}
watchVaultChange(file: TAbstractFile, ctx?: any) {
this.appendWatchEvent([{ type: "CHANGED", file }], ctx);
}
watchVaultDelete(file: TAbstractFile, ctx?: any) {
this.appendWatchEvent([{ type: "DELETE", file }], ctx);
}
watchVaultRename(file: TAbstractFile, oldFile: string, ctx?: any) {
if (file instanceof TFile) {
this.appendWatchEvent([
{ type: "DELETE", file: { path: oldFile as FilePath, mtime: file.stat.mtime, ctime: file.stat.ctime, size: file.stat.size, deleted: true } },
{ type: "CREATE", file },
], ctx);
}
}
// Watch raw events (Internal API)
watchVaultRawEvents(path: FilePath) {
if (!this.plugin.settings.syncInternalFiles && !this.plugin.settings.usePluginSync) return;
if (!this.plugin.settings.watchInternalFileChanges) return;
if (!path.startsWith(app.vault.configDir)) return;
const ignorePatterns = this.plugin.settings.syncInternalFilesIgnorePatterns
.replace(/\n| /g, "")
.split(",").filter(e => e).map(e => new RegExp(e, "i"));
if (ignorePatterns.some(e => path.match(e))) return;
this.appendWatchEvent(
[{
type: "INTERNAL",
file: { path, mtime: 0, ctime: 0, size: 0 }
}], null);
}
// Cache file and waiting to can be proceed.
async appendWatchEvent(params: { type: FileEventType, file: TAbstractFile | InternalFileInfo, oldPath?: string }[], ctx?: any) {
let forcePerform = false;
for (const param of params) {
if (shouldBeIgnored(param.file.path)) {
continue;
}
const atomicKey = [0, 0, 0, 0, 0, 0].map(e => `${Math.floor(Math.random() * 100000)}`).join("-");
const type = param.type;
const file = param.file;
const oldPath = param.oldPath;
if (file instanceof TFolder) continue;
if (!this.plugin.isTargetFile(file.path)) continue;
if (this.plugin.settings.suspendFileWatching) continue;
let cache: null | string | ArrayBuffer;
// new file or something changed, cache the changes.
if (file instanceof TFile && (type == "CREATE" || type == "CHANGED")) {
if (recentlyTouched(file)) {
continue;
}
if (!isPlainText(file.name)) {
cache = await app.vault.readBinary(file);
} else {
// cache = await this.app.vault.read(file);
cache = await app.vault.cachedRead(file);
if (!cache) cache = await app.vault.read(file);
}
}
if (type == "DELETE" || type == "RENAME") {
forcePerform = true;
}
if (this.plugin.settings.batchSave) {
// if the latest event is the same type, omit that
// a.md MODIFY <- this should be cancelled when a.md MODIFIED
// b.md MODIFY <- this should be cancelled when b.md MODIFIED
// a.md MODIFY
// a.md CREATE
// :
let i = this.watchedFileEventQueue.length;
L1:
while (i >= 0) {
i--;
if (i < 0) break L1;
if (this.watchedFileEventQueue[i].args.file.path != file.path) {
continue L1;
}
if (this.watchedFileEventQueue[i].type != type) break L1;
this.watchedFileEventQueue.remove(this.watchedFileEventQueue[i]);
//this.queuedFilesStore.set({ queuedItems: this.queuedFiles, fileEventItems: this.watchedFileEventQueue });
this.queuedFilesStore.apply((value) => ({ ...value, fileEventItems: this.watchedFileEventQueue }));
}
}
const fileInfo = file instanceof TFile ? {
ctime: file.stat.ctime,
mtime: file.stat.mtime,
file: file,
path: file.path,
size: file.stat.size
} as FileInfo : file as InternalFileInfo;
this.watchedFileEventQueue.push({
type,
args: {
file: fileInfo,
oldPath,
cache,
ctx
},
key: atomicKey
})
}
// this.queuedFilesStore.set({ queuedItems: this.queuedFiles, fileEventItems: this.watchedFileEventQueue });
this.queuedFilesStore.apply((value) => ({ ...value, fileEventItems: this.watchedFileEventQueue }));
this.plugin.procFileEvent(forcePerform);
}
fetchEvent(): FileEventItem | false {
if (this.watchedFileEventQueue.length == 0) return false;
const item = this.watchedFileEventQueue.shift();
this.queuedFilesStore.apply((value) => ({ ...value, fileEventItems: this.watchedFileEventQueue }));
return item;
}
cancelRelativeEvent(item: FileEventItem) {
this.watchedFileEventQueue = [...this.watchedFileEventQueue].filter(e => e.key != item.key);
this.queuedFilesStore.apply((value) => ({ ...value, fileEventItems: this.watchedFileEventQueue }));
}
getQueueLength() {
return this.watchedFileEventQueue.length;
}
}

12
src/deps.ts Normal file
View File

@@ -0,0 +1,12 @@
import { FilePath } from "./lib/src/types";
export {
addIcon, App, DataWriteOptions, debounce, Editor, FuzzySuggestModal, MarkdownRenderer, MarkdownView, Modal, Notice, Platform, Plugin, PluginManifest,
PluginSettingTab, Plugin_2, requestUrl, RequestUrlParam, RequestUrlResponse, sanitizeHTMLToDom, Setting, stringifyYaml, TAbstractFile, TextAreaComponent, TFile, TFolder,
parseYaml
} from "obsidian";
import {
normalizePath as normalizePath_
} from "obsidian";
const normalizePath = normalizePath_ as <T extends string | FilePath>(from: T) => T;
export { normalizePath }

229
src/dialogs.ts Normal file
View File

@@ -0,0 +1,229 @@
import { ButtonComponent } from "obsidian";
import { App, FuzzySuggestModal, MarkdownRenderer, Modal, Plugin, Setting } from "./deps";
import ObsidianLiveSyncPlugin from "./main";
//@ts-ignore
import PluginPane from "./PluginPane.svelte";
export class PluginDialogModal extends Modal {
plugin: ObsidianLiveSyncPlugin;
logEl: HTMLDivElement;
component: PluginPane = null;
isOpened() {
return this.component != null;
}
constructor(app: App, plugin: ObsidianLiveSyncPlugin) {
super(app);
this.plugin = plugin;
}
onOpen() {
const { contentEl } = this;
if (this.component == null) {
this.component = new PluginPane({
target: contentEl,
props: { plugin: this.plugin },
});
}
}
onClose() {
if (this.component != null) {
this.component.$destroy();
this.component = null;
}
}
}
export class InputStringDialog extends Modal {
result: string | false = false;
onSubmit: (result: string | boolean) => void;
title: string;
key: string;
placeholder: string;
isManuallyClosed = false;
constructor(app: App, title: string, key: string, placeholder: string, onSubmit: (result: string | false) => void) {
super(app);
this.onSubmit = onSubmit;
this.title = title;
this.placeholder = placeholder;
this.key = key;
}
onOpen() {
const { contentEl } = this;
contentEl.createEl("h1", { text: this.title });
// For enter to submit
const formEl = contentEl.createEl("form");
new Setting(formEl).setName(this.key).addText((text) =>
text.onChange((value) => {
this.result = value;
})
);
new Setting(formEl).addButton((btn) =>
btn
.setButtonText("Ok")
.setCta()
.onClick(() => {
this.isManuallyClosed = true;
this.close();
})
).addButton((btn) =>
btn
.setButtonText("Cancel")
.setCta()
.onClick(() => {
this.close();
})
);
}
onClose() {
const { contentEl } = this;
contentEl.empty();
if (this.isManuallyClosed) {
this.onSubmit(this.result);
} else {
this.onSubmit(false);
}
}
}
export class PopoverSelectString extends FuzzySuggestModal<string> {
app: App;
callback: (e: string) => void = () => { };
getItemsFun: () => string[] = () => {
return ["yes", "no"];
}
constructor(app: App, note: string, placeholder: string | null, getItemsFun: () => string[], callback: (e: string) => void) {
super(app);
this.app = app;
this.setPlaceholder((placeholder ?? "y/n) ") + note);
if (getItemsFun) this.getItemsFun = getItemsFun;
this.callback = callback;
}
getItems(): string[] {
return this.getItemsFun();
}
getItemText(item: string): string {
return item;
}
onChooseItem(item: string, evt: MouseEvent | KeyboardEvent): void {
// debugger;
this.callback(item);
this.callback = null;
}
onClose(): void {
setTimeout(() => {
if (this.callback != null) {
this.callback("");
}
}, 100);
}
}
export class MessageBox extends Modal {
plugin: Plugin;
title: string;
contentMd: string;
buttons: string[];
result: string;
isManuallyClosed = false;
defaultAction: string | undefined;
timeout: number | undefined;
timer: ReturnType<typeof setInterval> = undefined;
defaultButtonComponent: ButtonComponent | undefined;
onSubmit: (result: string | boolean) => void;
constructor(plugin: Plugin, title: string, contentMd: string, buttons: string[], defaultAction: (typeof buttons)[number], timeout: number, onSubmit: (result: (typeof buttons)[number] | false) => void) {
super(plugin.app);
this.plugin = plugin;
this.title = title;
this.contentMd = contentMd;
this.buttons = buttons;
this.onSubmit = onSubmit;
this.defaultAction = defaultAction;
this.timeout = timeout;
if (this.timeout) {
this.timer = setInterval(() => {
this.timeout--;
if (this.timeout < 0) {
if (this.timer) {
clearInterval(this.timer);
this.timer = undefined;
}
this.result = defaultAction;
this.isManuallyClosed = true;
this.close();
} else {
this.defaultButtonComponent.setButtonText(`( ${this.timeout} ) ${defaultAction}`);
}
}, 1000);
}
}
onOpen() {
const { contentEl } = this;
contentEl.addEventListener("click", () => {
if (this.timer) {
clearInterval(this.timer);
this.timer = undefined;
}
})
contentEl.createEl("h1", { text: this.title });
const div = contentEl.createDiv();
MarkdownRenderer.renderMarkdown(this.contentMd, div, "/", null);
const buttonSetting = new Setting(contentEl);
for (const button of this.buttons) {
buttonSetting.addButton((btn) => {
btn
.setButtonText(button)
.onClick(() => {
this.isManuallyClosed = true;
this.result = button;
if (this.timer) {
clearInterval(this.timer);
this.timer = undefined;
}
this.close();
})
if (button == this.defaultAction) {
this.defaultButtonComponent = btn;
}
return btn;
}
)
}
}
onClose() {
const { contentEl } = this;
contentEl.empty();
if (this.timer) {
clearInterval(this.timer);
this.timer = undefined;
}
if (this.isManuallyClosed) {
this.onSubmit(this.result);
} else {
this.onSubmit(false);
}
}
}
export function confirmWithMessage(plugin: Plugin, title: string, contentMd: string, buttons: string[], defaultAction?: (typeof buttons)[number], timeout?: number): Promise<(typeof buttons)[number] | false> {
return new Promise((res) => {
const dialog = new MessageBox(plugin, title, contentMd, buttons, defaultAction, timeout, (result) => res(result));
dialog.open();
});
}

2
src/lib

Submodule src/lib updated: 6451afd112...75f24a27b0

2945
src/main.ts
View File

File diff suppressed because it is too large Load Diff

4
src/pouchdb-browser.ts
View File

@@ -1,4 +0,0 @@
import { PouchDB as PouchDB_ } from "../pouchdb-browser-webpack/dist/pouchdb-browser.js";
const Pouch: PouchDB.Static = PouchDB_;
export { Pouch as PouchDB };

62
src/types.ts
View File

@@ -1,5 +1,5 @@
import { PluginManifest } from "obsidian";
import { DatabaseEntry } from "./lib/src/types";
import { PluginManifest, TFile } from "./deps";
import { DatabaseEntry, EntryBody, FilePath } from "./lib/src/types";
export interface PluginDataEntry extends DatabaseEntry {
deviceVaultName: string;
@@ -22,3 +22,61 @@ export interface DevicePluginList {
[key: string]: PluginDataEntry;
}
export const PERIODIC_PLUGIN_SWEEP = 60;
export interface InternalFileInfo {
path: FilePath;
mtime: number;
ctime: number;
size: number;
deleted?: boolean;
}
export interface FileInfo {
path: FilePath;
mtime: number;
ctime: number;
size: number;
deleted?: boolean;
file: TFile;
}
export type queueItem = {
entry: EntryBody;
missingChildren: string[];
timeout?: number;
done?: boolean;
warned?: boolean;
};
export type CacheData = string | ArrayBuffer;
export type FileEventType = "CREATE" | "DELETE" | "CHANGED" | "RENAME" | "INTERNAL";
export type FileEventArgs = {
file: FileInfo | InternalFileInfo;
cache?: CacheData;
oldPath?: string;
ctx?: any;
}
export type FileEventItem = {
type: FileEventType,
args: FileEventArgs,
key: string,
}
// Hidden items (Now means `chunk`)
export const CHeader = "h:";
// Plug-in Stored Container (Obsolete)
export const PSCHeader = "ps:";
export const PSCHeaderEnd = "ps;";
// Internal data Container
export const ICHeader = "i:";
export const ICHeaderEnd = "i;";
export const ICHeaderLength = ICHeader.length;
// Internal data Container (eXtended)
export const ICXHeader = "ix:";
export const FileWatchEventQueueMax = 10;
export const configURIBase = "obsidian://setuplivesync?settings=";

728
src/utils.ts
View File

@@ -1,14 +1,726 @@
import { normalizePath } from "obsidian";
import { DataWriteOptions, normalizePath, TFile, Platform, TAbstractFile, App, Plugin_2, RequestUrlParam, requestUrl } from "./deps";
import { path2id_base, id2path_base, isValidFilenameInLinux, isValidFilenameInDarwin, isValidFilenameInWidows, isValidFilenameInAndroid, stripAllPrefixes } from "./lib/src/path";
import { path2id_base, id2path_base } from "./lib/src/utils";
import { Logger } from "./lib/src/logger";
import { AnyEntry, DocumentID, EntryDoc, EntryHasPath, FilePath, FilePathWithPrefix, LOG_LEVEL, NewEntry } from "./lib/src/types";
import { CHeader, ICHeader, ICHeaderLength, PSCHeader } from "./types";
import { InputStringDialog, PopoverSelectString } from "./dialogs";
import ObsidianLiveSyncPlugin from "./main";
import { runWithLock } from "./lib/src/lock";
// For backward compatibility, using the path for determining id.
// Only CouchDB nonacceptable ID (that starts with an underscore) has been prefixed with "/".
// Only CouchDB unacceptable ID (that starts with an underscore) has been prefixed with "/".
// The first slash will be deleted when the path is normalized.
export function path2id(filename: string): string {
const x = normalizePath(filename);
return path2id_base(x);
export async function path2id(filename: FilePathWithPrefix | FilePath, obfuscatePassphrase: string | false): Promise<DocumentID> {
const temp = filename.split(":");
const path = temp.pop();
const normalizedPath = normalizePath(path as FilePath);
temp.push(normalizedPath);
const fixedPath = temp.join(":") as FilePathWithPrefix;
const out = await path2id_base(fixedPath, obfuscatePassphrase);
return out;
}
export function id2path(filename: string): string {
return id2path_base(normalizePath(filename));
export function id2path(id: DocumentID, entry?: EntryHasPath): FilePathWithPrefix {
const filename = id2path_base(id, entry);
const temp = filename.split(":");
const path = temp.pop();
const normalizedPath = normalizePath(path as FilePath);
temp.push(normalizedPath);
const fixedPath = temp.join(":") as FilePathWithPrefix;
return fixedPath;
}
export function getPath(entry: AnyEntry) {
return id2path(entry._id, entry);
}
export function getPathWithoutPrefix(entry: AnyEntry) {
const f = getPath(entry);
return stripAllPrefixes(f);
}
export function getPathFromTFile(file: TAbstractFile) {
return file.path as FilePath;
}
const tasks: { [key: string]: ReturnType<typeof setTimeout> } = {};
export function scheduleTask(key: string, timeout: number, proc: (() => Promise<any> | void)) {
cancelTask(key);
tasks[key] = setTimeout(async () => {
delete tasks[key];
await proc();
}, timeout);
}
export function cancelTask(key: string) {
if (key in tasks) {
clearTimeout(tasks[key]);
delete tasks[key];
}
}
export function cancelAllTasks() {
for (const v in tasks) {
clearTimeout(tasks[v]);
delete tasks[v];
}
}
const intervals: { [key: string]: ReturnType<typeof setInterval> } = {};
export function setPeriodicTask(key: string, timeout: number, proc: (() => Promise<any> | void)) {
cancelPeriodicTask(key);
intervals[key] = setInterval(async () => {
delete intervals[key];
await proc();
}, timeout);
}
export function cancelPeriodicTask(key: string) {
if (key in intervals) {
clearInterval(intervals[key]);
delete intervals[key];
}
}
export function cancelAllPeriodicTask() {
for (const v in intervals) {
clearInterval(intervals[v]);
delete intervals[v];
}
}
const memos: { [key: string]: any } = {};
export function memoObject<T>(key: string, obj: T): T {
memos[key] = obj;
return memos[key] as T;
}
export async function memoIfNotExist<T>(key: string, func: () => T | Promise<T>): Promise<T> {
if (!(key in memos)) {
const w = func();
const v = w instanceof Promise ? (await w) : w;
memos[key] = v;
}
return memos[key] as T;
}
export function retrieveMemoObject<T>(key: string): T | false {
if (key in memos) {
return memos[key];
} else {
return false;
}
}
export function disposeMemoObject(key: string) {
delete memos[key];
}
export function isSensibleMargeApplicable(path: string) {
if (path.endsWith(".md")) return true;
return false;
}
export function isObjectMargeApplicable(path: string) {
if (path.endsWith(".canvas")) return true;
if (path.endsWith(".json")) return true;
return false;
}
export function tryParseJSON(str: string, fallbackValue?: any) {
try {
return JSON.parse(str);
} catch (ex) {
return fallbackValue;
}
}
const MARK_OPERATOR = `\u{0001}`;
const MARK_DELETED = `${MARK_OPERATOR}__DELETED`;
const MARK_ISARRAY = `${MARK_OPERATOR}__ARRAY`;
const MARK_SWAPPED = `${MARK_OPERATOR}__SWAP`;
function unorderedArrayToObject(obj: Array<any>) {
return obj.map(e => ({ [e.id as string]: e })).reduce((p, c) => ({ ...p, ...c }), {})
}
function objectToUnorderedArray(obj: object) {
const entries = Object.entries(obj);
if (entries.some(e => e[0] != e[1]?.id)) throw new Error("Item looks like not unordered array")
return entries.map(e => e[1]);
}
function generatePatchUnorderedArray(from: Array<any>, to: Array<any>) {
if (from.every(e => typeof (e) == "object" && ("id" in e)) && to.every(e => typeof (e) == "object" && ("id" in e))) {
const fObj = unorderedArrayToObject(from);
const tObj = unorderedArrayToObject(to);
const diff = generatePatchObj(fObj, tObj);
if (Object.keys(diff).length > 0) {
return { [MARK_ISARRAY]: diff };
} else {
return {};
}
}
return { [MARK_SWAPPED]: to };
}
export function generatePatchObj(from: Record<string | number | symbol, any>, to: Record<string | number | symbol, any>) {
const entries = Object.entries(from);
const tempMap = new Map<string | number | symbol, any>(entries);
const ret = {} as Record<string | number | symbol, any>;
const newEntries = Object.entries(to);
for (const [key, value] of newEntries) {
if (!tempMap.has(key)) {
//New
ret[key] = value;
tempMap.delete(key);
} else {
//Exists
const v = tempMap.get(key);
if (typeof (v) !== typeof (value) || (Array.isArray(v) !== Array.isArray(value))) {
//if type is not match, replace completely.
ret[key] = { [MARK_SWAPPED]: value };
} else {
if (typeof (v) == "object" && typeof (value) == "object" && !Array.isArray(v) && !Array.isArray(value)) {
const wk = generatePatchObj(v, value);
if (Object.keys(wk).length > 0) ret[key] = wk;
} else if (typeof (v) == "object" && typeof (value) == "object" && Array.isArray(v) && Array.isArray(value)) {
const wk = generatePatchUnorderedArray(v, value);
if (Object.keys(wk).length > 0) ret[key] = wk;
} else if (typeof (v) != "object" && typeof (value) != "object") {
if (JSON.stringify(tempMap.get(key)) !== JSON.stringify(value)) {
ret[key] = value;
}
} else {
if (JSON.stringify(tempMap.get(key)) !== JSON.stringify(value)) {
ret[key] = { [MARK_SWAPPED]: value };
}
}
}
tempMap.delete(key);
}
}
//Not used item, means deleted one
for (const [key,] of tempMap) {
ret[key] = MARK_DELETED
}
return ret;
}
export function applyPatch(from: Record<string | number | symbol, any>, patch: Record<string | number | symbol, any>) {
const ret = from;
const patches = Object.entries(patch);
for (const [key, value] of patches) {
if (value == MARK_DELETED) {
delete ret[key];
continue;
}
if (typeof (value) == "object") {
if (MARK_SWAPPED in value) {
ret[key] = value[MARK_SWAPPED];
continue;
}
if (MARK_ISARRAY in value) {
if (!(key in ret)) ret[key] = [];
if (!Array.isArray(ret[key])) {
throw new Error("Patch target type is mismatched (array to something)");
}
const orgArrayObject = unorderedArrayToObject(ret[key]);
const appliedObject = applyPatch(orgArrayObject, value[MARK_ISARRAY]);
const appliedArray = objectToUnorderedArray(appliedObject);
ret[key] = [...appliedArray];
} else {
if (!(key in ret)) {
ret[key] = value;
continue;
}
ret[key] = applyPatch(ret[key], value);
}
} else {
ret[key] = value;
}
}
return ret;
}
export function mergeObject(
objA: Record<string | number | symbol, any> | [any],
objB: Record<string | number | symbol, any> | [any]
) {
const newEntries = Object.entries(objB);
const ret: any = { ...objA };
if (
typeof objA !== typeof objB ||
Array.isArray(objA) !== Array.isArray(objB)
) {
return objB;
}
for (const [key, v] of newEntries) {
if (key in ret) {
const value = ret[key];
if (
typeof v !== typeof value ||
Array.isArray(v) !== Array.isArray(value)
) {
//if type is not match, replace completely.
ret[key] = v;
} else {
if (
typeof v == "object" &&
typeof value == "object" &&
!Array.isArray(v) &&
!Array.isArray(value)
) {
ret[key] = mergeObject(v, value);
} else if (
typeof v == "object" &&
typeof value == "object" &&
Array.isArray(v) &&
Array.isArray(value)
) {
ret[key] = [...new Set([...v, ...value])];
} else {
ret[key] = v;
}
}
} else {
ret[key] = v;
}
}
if (Array.isArray(objA) && Array.isArray(objB)) {
return Object.values(Object.entries(ret)
.sort()
.reduce((p, [key, value]) => ({ ...p, [key]: value }), {}));
}
return Object.entries(ret)
.sort()
.reduce((p, [key, value]) => ({ ...p, [key]: value }), {});
}
export function flattenObject(obj: Record<string | number | symbol, any>, path: string[] = []): [string, any][] {
if (typeof (obj) != "object") return [[path.join("."), obj]];
if (Array.isArray(obj)) return [[path.join("."), JSON.stringify(obj)]];
const e = Object.entries(obj);
const ret = []
for (const [key, value] of e) {
const p = flattenObject(value, [...path, key]);
ret.push(...p);
}
return ret;
}
export function modifyFile(file: TFile, data: string | ArrayBuffer, options?: DataWriteOptions) {
if (typeof (data) === "string") {
return app.vault.modify(file, data, options);
} else {
return app.vault.modifyBinary(file, data, options);
}
}
export function createFile(path: string, data: string | ArrayBuffer, options?: DataWriteOptions): Promise<TFile> {
if (typeof (data) === "string") {
return app.vault.create(path, data, options);
} else {
return app.vault.createBinary(path, data, options);
}
}
export function isValidPath(filename: string) {
if (Platform.isDesktop) {
// if(Platform.isMacOS) return isValidFilenameInDarwin(filename);
if (process.platform == "darwin") return isValidFilenameInDarwin(filename);
if (process.platform == "linux") return isValidFilenameInLinux(filename);
return isValidFilenameInWidows(filename);
}
if (Platform.isAndroidApp) return isValidFilenameInAndroid(filename);
if (Platform.isIosApp) return isValidFilenameInDarwin(filename);
//Fallback
Logger("Could not determine platform for checking filename", LOG_LEVEL.VERBOSE);
return isValidFilenameInWidows(filename);
}
let touchedFiles: string[] = [];
export function getAbstractFileByPath(path: FilePath): TAbstractFile | null {
// Disabled temporary.
return app.vault.getAbstractFileByPath(path);
// // Hidden API but so useful.
// // @ts-ignore
// if ("getAbstractFileByPathInsensitive" in app.vault && (app.vault.adapter?.insensitive ?? false)) {
// // @ts-ignore
// return app.vault.getAbstractFileByPathInsensitive(path);
// } else {
// return app.vault.getAbstractFileByPath(path);
// }
}
export function trimPrefix(target: string, prefix: string) {
return target.startsWith(prefix) ? target.substring(prefix.length) : target;
}
export function touch(file: TFile | FilePath) {
const f = file instanceof TFile ? file : getAbstractFileByPath(file) as TFile;
const key = `${f.path}-${f.stat.mtime}-${f.stat.size}`;
touchedFiles.unshift(key);
touchedFiles = touchedFiles.slice(0, 100);
}
export function recentlyTouched(file: TFile) {
const key = `${file.path}-${file.stat.mtime}-${file.stat.size}`;
if (touchedFiles.indexOf(key) == -1) return false;
return true;
}
export function clearTouched() {
touchedFiles = [];
}
/**
* returns is internal chunk of file
* @param id ID
* @returns
*/
export function isInternalMetadata(id: FilePath | FilePathWithPrefix | DocumentID): boolean {
return id.startsWith(ICHeader);
}
export function stripInternalMetadataPrefix<T extends FilePath | FilePathWithPrefix | DocumentID>(id: T): T {
return id.substring(ICHeaderLength) as T;
}
export function id2InternalMetadataId(id: DocumentID): DocumentID {
return ICHeader + id as DocumentID;
}
// const CHeaderLength = CHeader.length;
export function isChunk(str: string): boolean {
return str.startsWith(CHeader);
}
export function isPluginMetadata(str: string): boolean {
return str.startsWith(PSCHeader);
}
export const askYesNo = (app: App, message: string): Promise<"yes" | "no"> => {
return new Promise((res) => {
const popover = new PopoverSelectString(app, message, null, null, (result) => res(result as "yes" | "no"));
popover.open();
});
};
export const askSelectString = (app: App, message: string, items: string[]): Promise<string> => {
const getItemsFun = () => items;
return new Promise((res) => {
const popover = new PopoverSelectString(app, message, "", getItemsFun, (result) => res(result));
popover.open();
});
};
export const askString = (app: App, title: string, key: string, placeholder: string): Promise<string | false> => {
return new Promise((res) => {
const dialog = new InputStringDialog(app, title, key, placeholder, (result) => res(result));
dialog.open();
});
};
export class PeriodicProcessor {
_process: () => Promise<any>;
_timer?: number;
_plugin: Plugin_2;
constructor(plugin: Plugin_2, process: () => Promise<any>) {
this._plugin = plugin;
this._process = process;
}
async process() {
try {
await this._process();
} catch (ex) {
Logger(ex);
}
}
enable(interval: number) {
this.disable();
if (interval == 0) return;
this._timer = window.setInterval(() => this._process().then(() => { }), interval);
this._plugin.registerInterval(this._timer);
}
disable() {
if (this._timer) clearInterval(this._timer);
}
}
function sizeToHumanReadable(size: number | undefined) {
if (!size) return "-";
const i = Math.floor(Math.log(size) / Math.log(1024));
return Number.parseInt((size / Math.pow(1024, i)).toFixed(2)) + ' ' + ['B', 'kB', 'MB', 'GB', 'TB'][i];
}
export const _requestToCouchDBFetch = async (baseUri: string, username: string, password: string, path?: string, body?: string | any, method?: string) => {
const utf8str = String.fromCharCode.apply(null, new TextEncoder().encode(`${username}:${password}`));
const encoded = window.btoa(utf8str);
const authHeader = "Basic " + encoded;
const transformedHeaders: Record<string, string> = { authorization: authHeader, "content-type": "application/json" };
const uri = `${baseUri}/${path}`;
const requestParam = {
url: uri,
method: method || (body ? "PUT" : "GET"),
headers: new Headers(transformedHeaders),
contentType: "application/json",
body: JSON.stringify(body),
};
return await fetch(uri, requestParam);
}
export const _requestToCouchDB = async (baseUri: string, username: string, password: string, origin: string, path?: string, body?: any, method?: string) => {
const utf8str = String.fromCharCode.apply(null, new TextEncoder().encode(`${username}:${password}`));
const encoded = window.btoa(utf8str);
const authHeader = "Basic " + encoded;
const transformedHeaders: Record<string, string> = { authorization: authHeader, origin: origin };
const uri = `${baseUri}/${path}`;
const requestParam: RequestUrlParam = {
url: uri,
method: method || (body ? "PUT" : "GET"),
headers: transformedHeaders,
contentType: "application/json",
body: body ? JSON.stringify(body) : undefined,
};
return await requestUrl(requestParam);
}
export const requestToCouchDB = async (baseUri: string, username: string, password: string, origin: string, key?: string, body?: string, method?: string) => {
const uri = `_node/_local/_config${key ? "/" + key : ""}`;
return await _requestToCouchDB(baseUri, username, password, origin, uri, body, method);
};
export async function performRebuildDB(plugin: ObsidianLiveSyncPlugin, method: "localOnly" | "remoteOnly" | "rebuildBothByThisDevice") {
if (method == "localOnly") {
await plugin.addOnSetup.fetchLocal();
}
if (method == "remoteOnly") {
await plugin.addOnSetup.rebuildRemote();
}
if (method == "rebuildBothByThisDevice") {
await plugin.addOnSetup.rebuildEverything();
}
}
export const gatherChunkUsage = async (db: PouchDB.Database<EntryDoc>) => {
const used = new Map();
const unreferenced = new Map();
const removed = new Map();
const missing = new Map();
const xx = await db.allDocs({ startkey: "h:", endkey: `h:\u{10ffff}` });
for (const xxd of xx.rows) {
const chunk = xxd.id
unreferenced.set(chunk, xxd.value.rev);
}
const x = await db.find({ limit: 999999999, selector: { children: { $exists: true, $type: "array" } }, fields: ["_id", "path", "mtime", "children"] });
for (const temp of x.docs) {
for (const chunk of (temp as NewEntry).children) {
used.set(chunk, (used.has(chunk) ? used.get(chunk) : 0) + 1);
if (unreferenced.has(chunk)) {
removed.set(chunk, unreferenced.get(chunk));
unreferenced.delete(chunk);
} else {
if (!removed.has(chunk)) {
if (!missing.has(temp._id)) {
missing.set(temp._id, []);
}
missing.get(temp._id).push(chunk);
}
}
}
}
return { used, unreferenced, missing };
}
export const localDatabaseCleanUp = async (plugin: ObsidianLiveSyncPlugin, force: boolean, dryRun: boolean) => {
await runWithLock("clean-up:local", true, async () => {
const db = plugin.localDatabase.localDatabase;
if ((db as any)?.adapter != "indexeddb") {
if (force && !dryRun) {
Logger("Fetch from the remote database", LOG_LEVEL.NOTICE, "clean-up-db");
await performRebuildDB(plugin, "localOnly");
return;
} else {
Logger("This feature requires enabling `Use new adapter`. Please enable it", LOG_LEVEL.NOTICE, "clean-up-db");
return;
}
}
Logger(`The remote database has been locked for garbage collection`, LOG_LEVEL.NOTICE, "clean-up-db");
Logger(`Gathering chunk usage information`, LOG_LEVEL.NOTICE, "clean-up-db");
const { unreferenced, missing } = await gatherChunkUsage(db);
if (missing.size != 0) {
Logger(`Some chunks are not found! We have to rescue`, LOG_LEVEL.NOTICE);
Logger(missing, LOG_LEVEL.VERBOSE);
} else {
Logger(`All chunks are OK`, LOG_LEVEL.NOTICE);
}
const payload = {} as Record<string, string[]>;
for (const [id, rev] of unreferenced) {
payload[id] = [rev];
}
const removeItems = Object.keys(payload).length;
if (removeItems == 0) {
Logger(`No unreferenced chunks found (Local)`, LOG_LEVEL.NOTICE);
await plugin.markRemoteResolved();
}
if (dryRun) {
Logger(`There are ${removeItems} unreferenced chunks (Local)`, LOG_LEVEL.NOTICE);
return;
}
Logger(`Deleting unreferenced chunks: ${removeItems}`, LOG_LEVEL.NOTICE, "clean-up-db");
for (const [id, rev] of unreferenced) {
//@ts-ignore
const ret = await db.purge(id, rev);
Logger(ret, LOG_LEVEL.VERBOSE);
}
plugin.localDatabase.refreshSettings();
Logger(`Compacting local database...`, LOG_LEVEL.NOTICE, "clean-up-db");
await db.compact();
await plugin.markRemoteResolved();
Logger("Done!", LOG_LEVEL.NOTICE, "clean-up-db");
})
}
export const balanceChunks = async (plugin: ObsidianLiveSyncPlugin, dryRun: boolean) => {
await runWithLock("clean-up:balance", true, async () => {
const localDB = plugin.localDatabase.localDatabase;
Logger(`Gathering chunk usage information`, LOG_LEVEL.NOTICE, "clean-up-db");
const ret = await plugin.replicator.connectRemoteCouchDBWithSetting(plugin.settings, plugin.isMobile);
if (typeof ret === "string") {
Logger(`Connect error: ${ret}`, LOG_LEVEL.NOTICE, "clean-up-db");
return;
}
const localChunks = new Map<string, string>();
const xx = await localDB.allDocs({ startkey: "h:", endkey: `h:\u{10ffff}` });
for (const xxd of xx.rows) {
const chunk = xxd.id
localChunks.set(chunk, xxd.value.rev);
}
// const info = ret.info;
const remoteDB = ret.db;
const remoteChunks = new Map<string, string>();
const xxr = await remoteDB.allDocs({ startkey: "h:", endkey: `h:\u{10ffff}` });
for (const xxd of xxr.rows) {
const chunk = xxd.id
remoteChunks.set(chunk, xxd.value.rev);
}
const localToRemote = new Map<string, string>([...localChunks]);
const remoteToLocal = new Map<string, string>([...remoteChunks]);
for (const id of new Set([...localChunks.keys(), ...remoteChunks.keys()])) {
if (remoteChunks.has(id)) {
localToRemote.delete(id);
}
if (localChunks.has(id)) {
remoteToLocal.delete(id);
}
}
function arrayToChunkedArray<T>(src: T[], size = 25) {
const ret = [] as T[][];
let i = 0;
while (i < src.length) {
ret.push(src.slice(i, i += size));
}
return ret;
}
if (localToRemote.size == 0) {
Logger(`No chunks need to be sent`, LOG_LEVEL.NOTICE);
} else {
Logger(`${localToRemote.size} chunks need to be sent`, LOG_LEVEL.NOTICE);
if (!dryRun) {
const w = arrayToChunkedArray([...localToRemote]);
for (const chunk of w) {
for (const [id,] of chunk) {
const queryRet = await localDB.allDocs({ keys: [id], include_docs: true });
const docs = queryRet.rows.filter(e => !("error" in e)).map(x => x.doc);
const ret = await remoteDB.bulkDocs(docs, { new_edits: false });
Logger(ret, LOG_LEVEL.VERBOSE);
}
}
Logger(`Done! ${remoteToLocal.size} chunks have been sent`, LOG_LEVEL.NOTICE);
}
}
if (remoteToLocal.size == 0) {
Logger(`No chunks need to be retrieved`, LOG_LEVEL.NOTICE);
} else {
Logger(`${remoteToLocal.size} chunks need to be retrieved`, LOG_LEVEL.NOTICE);
if (!dryRun) {
const w = arrayToChunkedArray([...remoteToLocal]);
for (const chunk of w) {
for (const [id,] of chunk) {
const queryRet = await remoteDB.allDocs({ keys: [id], include_docs: true });
const docs = queryRet.rows.filter(e => !("error" in e)).map(x => x.doc);
const ret = await localDB.bulkDocs(docs, { new_edits: false });
Logger(ret, LOG_LEVEL.VERBOSE);
}
}
Logger(`Done! ${remoteToLocal.size} chunks have been retrieved`, LOG_LEVEL.NOTICE);
}
}
})
}
export const remoteDatabaseCleanup = async (plugin: ObsidianLiveSyncPlugin, dryRun: boolean) => {
const getSize = function (info: PouchDB.Core.DatabaseInfo, key: "active" | "external" | "file") {
return Number.parseInt((info as any)?.sizes?.[key] ?? 0);
}
await runWithLock("clean-up:remote", true, async () => {
try {
const ret = await plugin.replicator.connectRemoteCouchDBWithSetting(plugin.settings, plugin.isMobile);
if (typeof ret === "string") {
Logger(`Connect error: ${ret}`, LOG_LEVEL.NOTICE, "clean-up-db");
return;
}
const info = ret.info;
Logger(JSON.stringify(info), LOG_LEVEL.VERBOSE, "clean-up-db");
Logger(`Database active-size: ${sizeToHumanReadable(getSize(info, "active"))}, external-size:${sizeToHumanReadable(getSize(info, "external"))}, file-size: ${sizeToHumanReadable(getSize(info, "file"))}`, LOG_LEVEL.NOTICE);
if (!dryRun) {
Logger(`The remote database has been locked for garbage collection`, LOG_LEVEL.NOTICE, "clean-up-db");
await plugin.markRemoteLocked(true);
}
Logger(`Gathering chunk usage information`, LOG_LEVEL.NOTICE, "clean-up-db");
const db = ret.db;
const { unreferenced, missing } = await gatherChunkUsage(db);
if (missing.size != 0) {
Logger(`Some chunks are not found! We have to rescue`, LOG_LEVEL.NOTICE);
Logger(missing, LOG_LEVEL.VERBOSE);
} else {
Logger(`All chunks are OK`, LOG_LEVEL.NOTICE);
}
const payload = {} as Record<string, string[]>;
for (const [id, rev] of unreferenced) {
payload[id] = [rev];
}
const removeItems = Object.keys(payload).length;
if (removeItems == 0) {
Logger(`No unreferenced chunk found (Remote)`, LOG_LEVEL.NOTICE);
return;
}
if (dryRun) {
Logger(`There are ${removeItems} unreferenced chunks (Remote)`, LOG_LEVEL.NOTICE);
return;
}
Logger(`Deleting unreferenced chunks: ${removeItems}`, LOG_LEVEL.NOTICE, "clean-up-db");
const rets = await _requestToCouchDBFetch(
`${plugin.settings.couchDB_URI}/${plugin.settings.couchDB_DBNAME}`,
plugin.settings.couchDB_USER,
plugin.settings.couchDB_PASSWORD,
"_purge",
payload, "POST");
// const result = await rets();
Logger(JSON.stringify(await rets.json()), LOG_LEVEL.VERBOSE);
Logger(`Compacting database...`, LOG_LEVEL.NOTICE, "clean-up-db");
await db.compact();
const endInfo = await db.info();
Logger(`Processed database active-size: ${sizeToHumanReadable(getSize(endInfo, "active"))}, external-size:${sizeToHumanReadable(getSize(endInfo, "external"))}, file-size: ${sizeToHumanReadable(getSize(endInfo, "file"))}`, LOG_LEVEL.NOTICE);
Logger(`Reduced sizes: active-size: ${sizeToHumanReadable(getSize(info, "active") - getSize(endInfo, "active"))}, external-size:${sizeToHumanReadable(getSize(info, "external") - getSize(endInfo, "external"))}, file-size: ${sizeToHumanReadable(getSize(info, "file") - getSize(endInfo, "file"))}`, LOG_LEVEL.NOTICE);
Logger(JSON.stringify(endInfo), LOG_LEVEL.VERBOSE, "clean-up-db");
Logger(`Local database cleaning up...`);
await localDatabaseCleanUp(plugin, true, false);
} catch (ex) {
Logger("Failed to clean up db.")
Logger(ex, LOG_LEVEL.VERBOSE);
}
});
}

225
src/utils_couchdb.ts
View File

@@ -1,225 +0,0 @@
import { Logger } from "./lib/src/logger";
import { LOG_LEVEL, VER, VERSIONINFO_DOCID, EntryVersionInfo, EntryDoc, RemoteDBSettings, SYNCINFO_ID, SyncInfo } from "./lib/src/types";
import { enableEncryption, resolveWithIgnoreKnownError } from "./lib/src/utils";
import { PouchDB } from "./pouchdb-browser";
import { requestUrl, RequestUrlParam, RequestUrlResponse } from "obsidian";
export const isValidRemoteCouchDBURI = (uri: string): boolean => {
if (uri.startsWith("https://")) return true;
if (uri.startsWith("http://")) return true;
return false;
};
let last_post_successed = false;
export const getLastPostFailedBySize = () => {
return !last_post_successed;
};
const fetchByAPI = async (request: RequestUrlParam): Promise<RequestUrlResponse> => {
const ret = await requestUrl(request);
if (ret.status - (ret.status % 100) !== 200) {
const er: Error & { status?: number } = new Error(`Request Error:${ret.status}`);
if (ret.json) {
er.message = ret.json.reason;
er.name = `${ret.json.error ?? ""}:${ret.json.message ?? ""}`;
}
er.status = ret.status;
throw er;
}
return ret;
};
export const connectRemoteCouchDBWithSetting = (settings: RemoteDBSettings, isMobile: boolean) =>
connectRemoteCouchDB(
settings.couchDB_URI + (settings.couchDB_DBNAME == "" ? "" : "/" + settings.couchDB_DBNAME),
{
username: settings.couchDB_USER,
password: settings.couchDB_PASSWORD,
},
settings.disableRequestURI || isMobile,
settings.encrypt ? settings.passphrase : settings.encrypt
);
const connectRemoteCouchDB = async (uri: string, auth: { username: string; password: string }, disableRequestURI: boolean, passphrase: string | boolean): Promise<string | { db: PouchDB.Database<EntryDoc>; info: PouchDB.Core.DatabaseInfo }> => {
if (!isValidRemoteCouchDBURI(uri)) return "Remote URI is not valid";
let authHeader = "";
if (auth.username && auth.password) {
const utf8str = String.fromCharCode.apply(null, new TextEncoder().encode(`${auth.username}:${auth.password}`));
const encoded = window.btoa(utf8str);
authHeader = "Basic " + encoded;
} else {
authHeader = "";
}
const conf: PouchDB.HttpAdapter.HttpAdapterConfiguration = {
adapter: "http",
auth,
fetch: async function (url: string | Request, opts: RequestInit) {
let size = "";
const localURL = url.toString().substring(uri.length);
const method = opts.method ?? "GET";
if (opts.body) {
const opts_length = opts.body.toString().length;
if (opts_length > 1024 * 1024 * 10) {
// over 10MB
if (uri.contains(".cloudantnosqldb.")) {
last_post_successed = false;
Logger("This request should fail on IBM Cloudant.", LOG_LEVEL.VERBOSE);
throw new Error("This request should fail on IBM Cloudant.");
}
}
size = ` (${opts_length})`;
}
if (!disableRequestURI && typeof url == "string" && typeof (opts.body ?? "") == "string") {
const body = opts.body as string;
const transformedHeaders = { ...(opts.headers as Record<string, string>) };
if (authHeader != "") transformedHeaders["authorization"] = authHeader;
delete transformedHeaders["host"];
delete transformedHeaders["Host"];
delete transformedHeaders["content-length"];
delete transformedHeaders["Content-Length"];
const requestParam: RequestUrlParam = {
url: url as string,
method: opts.method,
body: body,
headers: transformedHeaders,
contentType: "application/json",
// contentType: opts.headers,
};
try {
const r = await fetchByAPI(requestParam);
if (method == "POST" || method == "PUT") {
last_post_successed = r.status - (r.status % 100) == 200;
} else {
last_post_successed = true;
}
Logger(`HTTP:${method}${size} to:${localURL} -> ${r.status}`, LOG_LEVEL.DEBUG);
return new Response(r.arrayBuffer, {
headers: r.headers,
status: r.status,
statusText: `${r.status}`,
});
} catch (ex) {
Logger(`HTTP:${method}${size} to:${localURL} -> failed`, LOG_LEVEL.VERBOSE);
// limit only in bulk_docs.
if (url.toString().indexOf("_bulk_docs") !== -1) {
last_post_successed = false;
}
Logger(ex);
throw ex;
}
}
// -old implementation
try {
const responce: Response = await fetch(url, opts);
if (method == "POST" || method == "PUT") {
last_post_successed = responce.ok;
} else {
last_post_successed = true;
}
Logger(`HTTP:${method}${size} to:${localURL} -> ${responce.status}`, LOG_LEVEL.DEBUG);
return responce;
} catch (ex) {
Logger(`HTTP:${method}${size} to:${localURL} -> failed`, LOG_LEVEL.VERBOSE);
// limit only in bulk_docs.
if (url.toString().indexOf("_bulk_docs") !== -1) {
last_post_successed = false;
}
Logger(ex);
throw ex;
}
// return await fetch(url, opts);
},
};
const db: PouchDB.Database<EntryDoc> = new PouchDB<EntryDoc>(uri, conf);
if (passphrase && typeof passphrase === "string") {
enableEncryption(db, passphrase);
}
try {
const info = await db.info();
return { db: db, info: info };
} catch (ex) {
let msg = `${ex.name}:${ex.message}`;
if (ex.name == "TypeError" && ex.message == "Failed to fetch") {
msg += "\n**Note** This error caused by many reasons. The only sure thing is you didn't touch the server.\nTo check details, open inspector.";
}
Logger(ex, LOG_LEVEL.VERBOSE);
return msg;
}
};
// check the version of remote.
// if remote is higher than current(or specified) version, return false.
export const checkRemoteVersion = async (db: PouchDB.Database, migrate: (from: number, to: number) => Promise<boolean>, barrier: number = VER): Promise<boolean> => {
try {
const versionInfo = (await db.get(VERSIONINFO_DOCID)) as EntryVersionInfo;
if (versionInfo.type != "versioninfo") {
return false;
}
const version = versionInfo.version;
if (version < barrier) {
const versionUpResult = await migrate(version, barrier);
if (versionUpResult) {
await bumpRemoteVersion(db);
return true;
}
}
if (version == barrier) return true;
return false;
} catch (ex) {
if (ex.status && ex.status == 404) {
if (await bumpRemoteVersion(db)) {
return true;
}
return false;
}
throw ex;
}
};
export const bumpRemoteVersion = async (db: PouchDB.Database, barrier: number = VER): Promise<boolean> => {
const vi: EntryVersionInfo = {
_id: VERSIONINFO_DOCID,
version: barrier,
type: "versioninfo",
};
const versionInfo = (await resolveWithIgnoreKnownError(db.get(VERSIONINFO_DOCID), vi)) as EntryVersionInfo;
if (versionInfo.type != "versioninfo") {
return false;
}
vi._rev = versionInfo._rev;
await db.put(vi);
return true;
};
export const checkSyncInfo = async (db: PouchDB.Database): Promise<boolean> => {
try {
const syncinfo = (await db.get(SYNCINFO_ID)) as SyncInfo;
console.log(syncinfo);
// if we could decrypt the doc, it must be ok.
return true;
} catch (ex) {
if (ex.status && ex.status == 404) {
const randomStrSrc = "abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ";
const temp = [...Array(30)]
.map((e) => Math.floor(Math.random() * randomStrSrc.length))
.map((e) => randomStrSrc[e])
.join("");
const newSyncInfo: SyncInfo = {
_id: SYNCINFO_ID,
type: "syncinfo",
data: temp,
};
if (await db.put(newSyncInfo)) {
return true;
}
return false;
} else {
console.dir(ex);
return false;
}
}
};

65
styles.css
View File

@@ -2,28 +2,33 @@
color: var(--text-on-accent);
background-color: var(--text-accent);
}
.normal {
color: var(--text-normal);
}
.deleted {
color: var(--text-on-accent);
background-color: var(--text-muted);
text-decoration: line-through;
}
.op-scrollable {
overflow-y: scroll;
/* min-height: 280px; */
max-height: 280px;
user-select: text;
}
.op-pre {
white-space: pre-wrap;
}
.op-warn {
border: 1px solid salmon;
padding: 2px;
border-radius: 4px;
}
.op-warn::before {
content: "Warning";
font-weight: bold;
@@ -31,11 +36,13 @@
position: relative;
display: block;
}
.op-warn-info {
border: 1px solid rgb(255, 209, 81);
padding: 2px;
border-radius: 4px;
}
.op-warn-info::before {
content: "Notice";
font-weight: bold;
@@ -43,27 +50,33 @@
position: relative;
display: block;
}
.syncstatusbar {
-webkit-filter: grayscale(100%);
filter: grayscale(100%);
}
.tcenter {
text-align: center;
}
.sls-plugins-wrap {
display: flex;
flex-grow: 1;
max-height: 50vh;
overflow-y: scroll;
}
.sls-plugins-tbl {
border: 1px solid var(--background-modifier-border);
width: 100%;
max-height: 80%;
}
.divider th {
border-top: 1px solid var(--background-modifier-border);
}
/* .sls-table-head{
width:50%;
}
@@ -72,12 +85,11 @@
} */
.sls-btn-left {
padding-right: 4px;
}
.sls-btn-right {
padding-left: 4px;
.sls-header-button {
margin-left: 2em;
}
.sls-hidden {
display: none;
}
@@ -85,9 +97,11 @@
:root {
--slsmessage: "";
}
.CodeMirror-wrap::before,
.cm-s-obsidian > .cm-editor::before {
content: var(--slsmessage);
.cm-s-obsidian>.cm-editor::before,
.canvas-wrapper::before {
content: attr(data-log);
text-align: right;
white-space: pre-wrap;
position: absolute;
@@ -102,15 +116,22 @@
filter: grayscale(100%);
}
.canvas-wrapper::before {
right: 48px;
}
.CodeMirror-wrap::before {
right: 0px;
}
.cm-s-obsidian > .cm-editor::before {
.cm-s-obsidian>.cm-editor::before {
right: 16px;
}
.sls-setting-tab {
display: none;
}
div.sls-setting-menu-btn {
color: var(--text-normal);
background-color: var(--background-secondary-alt);
@@ -131,8 +152,9 @@ div.sls-setting-menu-btn {
flex-grow: 1;
/* width: 100%; */
}
.sls-setting-tab:hover ~ div.sls-setting-menu-btn,
.sls-setting-tab:checked ~ div.sls-setting-menu-btn {
.sls-setting-tab:hover~div.sls-setting-menu-btn,
.sls-setting-label.selected .sls-setting-tab:checked~div.sls-setting-menu-btn {
background-color: var(--interactive-accent);
color: var(--text-on-accent);
}
@@ -143,14 +165,17 @@ div.sls-setting-menu-btn {
/* flex-wrap: wrap; */
overflow-x: auto;
}
.sls-setting-label {
flex-grow: 1;
display: inline-flex;
justify-content: center;
}
.setting-collapsed {
display: none;
}
.sls-plugins-tbl-buttons {
text-align: right;
}
@@ -159,13 +184,16 @@ div.sls-setting-menu-btn {
flex-grow: 0;
padding: 6px 10px;
}
.sls-plugins-tbl-device-head {
background-color: var(--background-secondary-alt);
color: var(--text-accent);
}
.op-flex {
display: flex;
}
.op-flex input {
display: inline-flex;
flex-grow: 1;
@@ -185,9 +213,11 @@ div.sls-setting-menu-btn {
color: var(--text-on-accent);
background-color: var(--text-accent);
}
.history-normal {
color: var(--text-normal);
}
.history-deleted {
color: var(--text-on-accent);
background-color: var(--text-muted);
@@ -197,6 +227,7 @@ div.sls-setting-menu-btn {
.ob-btn-config-fix label {
margin-right: 40px;
}
.ob-btn-config-info {
border: 1px solid salmon;
padding: 2px;
@@ -209,3 +240,15 @@ div.sls-setting-menu-btn {
margin: 1px;
border-radius: 4px;
}
.isWizard .wizardHidden {
display: none;
}
.sls-setting:not(.isWizard) .wizardOnly {
display: none;
}
.sls-item-dirty::before {
content: "✏";
}

23
tsconfig.json
View File

@@ -2,15 +2,26 @@
"compilerOptions": {
"baseUrl": ".",
"module": "ESNext",
"target": "ES6",
"target": "ES2018",
"allowJs": true,
"noImplicitAny": true,
"moduleResolution": "node",
// "importsNotUsedAsValues": "error",
"importHelpers": true,
"importHelpers": false,
"alwaysStrict": true,
"lib": ["es2018", "DOM", "ES5", "ES6", "ES7"]
"lib": [
"es2018",
"DOM",
"ES5",
"ES6",
"ES7",
"es2019.array"
]
},
"include": ["**/*.ts"],
"exclude": ["pouchdb-browser-webpack"]
}
"include": [
"**/*.ts"
],
"exclude": [
"pouchdb-browser-webpack"
]
}

17
updates.md Normal file
View File

@@ -0,0 +1,17 @@
### 0.19.0
#### Customization sync
Since `Plugin and their settings` have been broken, so I tried to fix it, not just fix it, but fix it the way it should be.
Now, we have `Customization sync`.
It is a real shame that the compatibility between these features has been broken. However, this new feature is surely useful and I believe that worth getting over the pain.
We can use the new feature with the same configuration. Only the menu on the command palette has been changed. The dialog can be opened by `Show customization sync dialog`.
I hope you will give it a try.
#### Minors
... To continue on to `updates_old.md`.

316
updates_old.md Normal file
View File

@@ -0,0 +1,316 @@
### 0.18.0
#### Now, paths of files in the database can now be obfuscated. (Experimental Feature)
At before v0.18.0, Self-hosted LiveSync used the path of files, to detect and resolve conflicts. In naive. The ID of the document stored in the CouchDB was naturally the filename.
However, it means a sort of lacking confidentiality. If the credentials of the database have been leaked, the attacker (or an innocent bystander) can read the path of files. So we could not use confidential things in the filename in some environments.
Since v0.18.0, they can be obfuscated. so it is no longer possible to decipher the path from the ID. Instead of that, it costs a bit CPU load than before, and the data structure has been changed a bit.
We can configure the `Path Obfuscation` in the `Remote database configuration` pane.
Note: **When changing this configuration, we need to rebuild both of the local and the remote databases**.
#### Minors
- 0.18.1
- Fixed:
- Some messages are fixed (Typo)
- File type detection now works fine!
- 0.18.2
- Improved:
- The setting pane has been refined.
- We can enable `hidden files sync` with several initial behaviours; `Merge`, `Fetch` remote, and `Overwrite` remote.
- No longer `Touch hidden files`.
- 0.18.3
- Fixed Pop-up is now correctly shown after hidden file synchronisation.
- 0.18.4
- Fixed:
- `Fetch` and `Rebuild database` will work more safely.
- Case-sensitive renaming now works fine.
Revoked the logic which was made at #130, however, looks fine now.
- 0.18.5
- Improved:
- Actions for maintaining databases moved to the `🎛️Maintain databases`.
- Clean-up of unreferenced chunks has been implemented on an **experimental**.
- This feature requires enabling `Use new adapter`.
- Be sure to fully all devices synchronised before perform it.
- After cleaning up the remote, all devices will be locked out. If we are sure had it be synchronised, we can perform only cleaning-up locally. If not, we have to perform `Fetch`.
- 0.18.6
- New features:
- Now remote database cleaning-up will be detected automatically.
- A solution selection dialogue will be shown if synchronisation is rejected after cleaning or rebuilding the remote database.
- During fetching or rebuilding, we can configure `Hidden file synchronisation` on the spot.
- It let us free from conflict resolution on initial synchronising.
### 0.17.0
- 0.17.0 has no surfaced changes but the design of saving chunks has been changed. They have compatibility but changing files after upgrading makes different chunks than before 0.16.x.
Please rebuild databases once if you have been worried about storage usage.
- Improved:
- Splitting markdown
- Saving chunks
- Changed:
- Chunk ID numbering rules
#### Minors
- 0.17.1
- Fixed: Now we can verify and repair the database.
- Refactored inside.
- 0.17.2
- New feature
- We can merge conflicted documents automatically if sensible.
- Fixed
- Writing to the storage will be pended while they have conflicts after replication.
- 0.17.3
- Now we supported canvas! And conflicted JSON files are also synchronised with merging its content if they are obvious.
- 0.17.4
- Canvases are now treated as a sort of plain text file. now we transfer only the metadata and chunks that have differences.
- 0.17.5 Now `read chunks online` had been fixed, and a new feature: `Use dynamic iteration count` to reduce the load on encryption/decryption.
Note: `Use dynamic iteration count` is not compatible with earlier versions.
- 0.17.6 Now our renamed/deleted files have been surely deleted again.
- 0.17.7
- Fixed:
- Fixed merging issues.
- Fixed button styling.
- Changed:
- Conflict checking on synchronising has been enabled for every note in default.
- 0.17.8
- Improved: Performance improved. Prebuilt PouchDB is no longer used.
- Fixed: Merging hidden files is also fixed.
- New Feature: Now we can synchronise automatically after merging conflicts.
- 0.17.9
- Fixed: Conflict merge of internal files is no longer broken.
- Improved: Smoother status display inside the editor.
- 0.17.10
- Fixed: Large file synchronising has been now addressed!
Note: When synchronising large files, we have to set `Chunk size` to lower than 50, disable `Read chunks online`, `Batch size` should be set 50-100, and `Batch limit` could be around 20.
- 0.17.11
- Fixed:
- Performance improvement
- Now `Chunk size` can be set to under one hundred.
- New feature:
- The number of transfers required before replication stabilises is now displayed.
- 0.17.12: Skipped.
- 0.17.13
- Fixed: Document history is now displayed again.
- Reorganised: Many files have been refactored.
- 0.17.14: Skipped.
- 0.17.15
- Improved:
- Confidential information has no longer stored in data.json as is.
- Synchronising progress has been shown in the notification.
- We can commit passphrases with a keyboard.
- Configuration which had not been saved yet is marked now.
- Now the filename is shown on the Conflict resolving dialog
- Fixed:
- Hidden files have been synchronised again.
- Rename of files has been fixed again.
And, minor changes have been included.
- 0.17.16:
- Improved:
- Plugins and their settings no longer need scanning if changes are monitored.
- Now synchronising plugins and their settings are performed parallelly and faster.
- We can place `redflag2.md` to rebuild the database automatically while the boot sequence.
- Experimental:
- We can use a new adapter on PouchDB. This will make us smoother.
- Note: Not compatible with the older version.
- Fixed:
- The default batch size is smaller again.
- Plugins and their setting can be synchronised again.
- Hidden files and plugins are correctly scanned while rebuilding.
- Files with the name started `_` are also being performed conflict-checking.
- 0.17.17
- Fixed: Now we can merge JSON files even if we failed to compare items like null.
- 0.17.18
- Fixed: Fixed lack of error handling.
- 0.17.19
- Fixed: Error reporting has been ensured.
- 0.17.20
- Improved: Changes of hidden files will be notified to Obsidian.
- 0.17.21
- Fixed: Skip patterns now handle capital letters.
- Improved
- New configuration to avoid exceeding throttle capacity.
- We have been grateful to @karasevm!
- The conflicted `data.json` is no longer merged automatically.
- This behaviour is not configurable, unlike the `Use newer file if conflicted` of normal files.
- 0.17.22
- Fixed:
- Now hidden files will not be synchronised while we are not configured.
- Some processes could start without waiting for synchronisation to complete, but now they will wait for.
- Improved
- Now, by placing `redflag3.md`, we can discard the local database and fetch again.
- The document has been updated! Thanks to @hilsonp!
- 0.17.23
- Improved:
- Now we can preserve the logs into the file.
- Note: This option will be enabled automatically also when we flagging a red flag.
- File names can now be made platform-appropriate.
- Refactored:
- Some redundant implementations have been sorted out.
- 0.17.24
- New feature:
- If any conflicted files have been left, they will be reported.
- Fixed:
- Now the name of the conflicting file is shown on the conflict-resolving dialogue.
- Hidden files are now able to be merged again.
- No longer error caused at plug-in being loaded.
- Improved:
- Caching chunks are now limited in total size of cached chunks.
- 0.17.25
- Fixed:
- Now reading error will be reported.
- 0.17.26
- Fixed(Urgent):
- The modified document will be reflected in the storage now.
- 0.17.27
- Improved:
- Now, the filename of the conflicted settings will be shown on the merging dialogue
- The plugin data can be resolved when conflicted.
- The semaphore status display has been changed to count only.
- Applying to the storage will be concurrent with a few files.
- 0.17.28
-Fixed:
- Some messages have been refined.
- Boot sequence has been speeded up.
- Opening the local database multiple times in a short duration has been suppressed.
- Older migration logic.
- Note: If you have used 0.10.0 or lower and have not upgraded, you will need to run 0.17.27 or earlier once or reinstall Obsidian.
- 0.17.29
- Fixed:
- Requests of reading chunks online are now split into a reasonable(and configurable) size.
- No longer error message will be shown on Linux devices with hidden file synchronisation.
- Improved:
- The interval of reading chunks online is now configurable.
- Boot sequence has been speeded up, more.
- Misc:
- Messages on the boot sequence will now be more detailed. If you want to see them, please enable the verbose log.
- Logs became be kept for 1000 lines while the verbose log is enabled.
- 0.17.30
- Implemented:
- `Resolve all conflicted files` has been implemented.
- Fixed:
- Fixed a problem about reading chunks online when a file has more chunks than the concurrency limit.
- Rollbacked:
- Logs are kept only for 100 lines, again.
- 0.17.31
- Fixed:
- Now `redflag3` can be run surely.
- Synchronisation can now be aborted.
- Note: The synchronisation flow has been rewritten drastically. Please do not haste to inform me if you have noticed anything.
- 0.17.32
- Fixed:
- Now periodic internal file scanning works well.
- The handler of Window-visibility-changed has been fixed.
- And minor fixes possibly included.
- Refactored:
- Unused logic has been removed.
- Some utility functions have been moved into suitable files.
- Function names have been renamed.
- 0.17.33
- Maintenance update: Refactored; the responsibilities that `LocalDatabase` had were shared. (Hoping) No changes in behaviour.
- 0.17.34
- Fixed: The `Fetch` that was broken at 0.17.33 has been fixed.
- Refactored again: Internal file sync, plug-in sync and Set up URI have been moved into each file.
### 0.16.0
- Now hidden files need not be scanned. Changes will be detected automatically.
- If you want it to back to its previous behaviour, please disable `Monitor changes to internal files`.
- Due to using an internal API, this feature may become unusable with a major update. If this happens, please disable this once.
#### Minors
- 0.16.1 Added missing log updates.
- 0.16.2 Fixed many problems caused by combinations of `Sync On Save` and the tracking logic that changed at 0.15.6.
- 0.16.3
- Fixed detection of IBM Cloudant (And if there are some issues, be fixed automatically).
- A configuration information reporting tool has been implemented.
- 0.16.4 Fixed detection failure. Please set the `Chunk size` again when using a self-hosted database.
- 0.16.5
- Fixed
- Conflict detection and merging now be able to treat deleted files.
- Logs while the boot-up sequence has been tidied up.
- Fixed incorrect log entries.
- New Feature
- The feature of automatically deleting old expired metadata has been implemented.
We can configure it in `Delete old metadata of deleted files on start-up` in the `General Settings` pane.
- 0.16.6
- Fixed
- Automatic (temporary) batch size adjustment has been restored to work correctly.
- Chunk splitting has been backed to the previous behaviour for saving them correctly.
- Improved
- Corrupted chunks will be detected automatically.
- Now on the case-insensitive system, `aaa.md` and `AAA.md` will be treated as the same file or path at applying changesets.
- 0.16.7 Nothing has been changed except toolsets, framework library, and as like them. Please inform me if something had been getting strange!
- 0.16.8 Now we can synchronise without `bad_request:invalid UTF-8 JSON` even while end-to-end encryption has been disabled.
Note:
Before 0.16.5, LiveSync had some issues making chunks. In this case, synchronisation had became been always failing after a corrupted one should be made. After 0.16.6, the corrupted chunk is automatically detected. Sorry for troubling you but please do `rebuild everything` when this plug-in notified so.
### 0.15.0
- Outdated configuration items have been removed.
- Setup wizard has been implemented!
I appreciate for reviewing and giving me advice @Pouhon158!
#### Minors
- 0.15.1 Missed the stylesheet.
- 0.15.2 The wizard has been improved and documented!
- 0.15.3 Fixed the issue about locking/unlocking remote database while rebuilding in the wizard.
- 0.15.4 Fixed issues about asynchronous processing (e.g., Conflict check or hidden file detection)
- 0.15.5 Add new features for setting Self-hosted LiveSync up more easier.
- 0.15.6 File tracking logic has been refined.
- 0.15.7 Fixed bug about renaming file.
- 0.15.8 Fixed bug about deleting empty directory, weird behaviour on boot-sequence on mobile devices.
- 0.15.9 Improved chunk retrieving, now chunks are retrieved in batch on continuous requests.
- 0.15.10 Fixed:
- The boot sequence has been corrected and now boots smoothly.
- Auto applying of batch save will be processed earlier than before.
### 0.14.1
- The target selecting filter was implemented.
Now we can set what files are synchronised by regular expression.
- We can configure the size of chunks.
We can use larger chunks to improve performance.
(This feature can not be used with IBM Cloudant)
- Read chunks online.
Now we can synchronise only metadata and retrieve chunks on demand. It reduces local database size and time for replication.
- Added this note.
- Use local chunks in preference to remote them if present,
#### Recommended configuration for Self-hosted CouchDB
- Set chunk size to around 100 to 250 (10MB - 25MB per chunk)
- *Set batch size to 100 and batch limit to 20 (0.14.2)*
- Be sure to `Read chunks online` checked.
#### Minors
- 0.14.2 Fixed issue about retrieving files if synchronisation has been interrupted or failed
- 0.14.3 New test items have been added to `Check database configuration`.
- 0.14.4 Fixed issue of importing configurations.
- 0.14.5 Auto chunk size adjusting implemented.
- 0.14.6 Change Target to ES2018
- 0.14.7 Refactor and fix typos.
- 0.14.8 Refactored again. There should be no change in behaviour, but please let me know if there is any.
### 0.13.0
- The metadata of the deleted files will be kept on the database by default. If you want to delete this as the previous version, please turn on `Delete metadata of deleted files.`. And, if you have upgraded from the older version, please ensure every device has been upgraded.
- Please turn on `Delete metadata of deleted files.` if you are using livesync-classroom or filesystem-livesync.
- We can see the history of deleted files.
- `Pick file to show` was renamed to `Pick a file to show.
- Files in the `Pick a file to show` are now ordered by their modified date descent.
- Update information became to be shown on the major upgrade.
#### Minors
- 0.13.1 Fixed on conflict resolution.
- 0.13.2 Fixed file deletion failures.
- 0.13.4
- Now, we can synchronise hidden files that conflicted on each devices.
- We can search for conflicting docs.
- Pending processes can now be run at any time.
- Performance improved on synchronising large numbers of files at once.