public/obsidian-livesync
1
0
Fork 0
mirror of https://github.com/vrtmrz/obsidian-livesync.git synced 2026-02-24 13:08:48 +00:00
Code Issues Packages Projects Releases Wiki Activity

Compare commits

base: public:0.25.43-patched-2
Branches Tags
public:main public:beta public:snyk-upgrade-7fd31e6f474568014502e7205847f257 public:snyk-fix-90f3b71450f7543a1c61724d8afa0e71 public:refactor_services_2 public:refactor_services public:dev public:snyk-upgrade-fe9625385aa4be849c7157033fd2abb0 public:snyk-upgrade-99a31e70c822894595544b0c9813b11a public:svelteui public:disenchant public:snyk-fix-61f76ef0bce2a276df9552c2fc80a14f public:snyk-upgrade-2baa7938ba318464c3bad5c3fd5cbda5 public:snyk-upgrade-7af62a575eab20e8fb4e9565bf1adcfc public:snyk-upgrade-b32f281b31a81483f126b8fb6c54a957 public:snyk-upgrade-bd61b1103442d3faecc9fc726c34e9a4 public:snyk-upgrade-5b2d6584430c061b5187611a5d7fb64a public:snyk-fix-0e9fbbcb374fcb8d5e8cb027d8414fb7 public:snyk-upgrade-5242daf91e706d1ae500c9ea6545636e public:snyk-upgrade-70434a6822fbad83424135dfad9df909 public:snyk-upgrade-8fa01ccad0352125f1866d41e57d35e1 public:snyk-upgrade-39eb97b4655e70091f718a88fccd57c1 public:snyk-upgrade-2b99d54d19e207887e36c3071b796925 public:forprivacy public:snyk-upgrade-dad9088d0a333677e0157e3deef7fc38 public:snyk-upgrade-b9a76d74b3e5723759af5b3319aabb80 public:snyk-upgrade-7da6459d8ff5d293d80de9d80061d912 public:snyk-upgrade-6a3786fc4cbe394542e9d1b6058c46e7 public:snyk-upgrade-8fc9e62771dc41b0f23dc0efefb5c949
public:0.25.44 public:0.25.43-patched-9 public:0.25.43-patched-8 public:0.25.43-patched-7 public:0.25.43-patched-6 public:0.25.43-patched-5 public:0.25.43-patched-4 public:0.25.43-patched-3 public:0.25.43-patched-2 public:0.25.43-patched-1 public:0.25.43 public:0.25.42 public:0.25.41-patched-3 public:0.25.41-patched-2 public:0.25.41-patched-1 public:0.25.41 public:0.25.40 public:0.25.39 public:0.25.38 public:0.25.37 public:0.25.36 public:0.25.35 public:0.25.34 public:0.25.33 public:0.25.32 public:0.25.31 public:0.25.30 public:0.25.30-dev public:0.25.29 public:0.25.28 public:0.25.27 public:0.25.26 public:0.25.25 public:0.25.24 public:0.25.24.beta3 public:0.25.24.beta2 public:0.25.24.beta1 public:0.25.23 public:0.25.23.beta1 public:0.25.22 public:0.25.21 public:0.25.21.beta2 public:0.25.21.beta1 public:0.25.20 public:0.25.19 public:0.25.18 public:0.25.17 public:0.25.16 public:0.25.15 public:0.25.14 public:0.25.13 public:0.25.12 public:0.25.11 public:0.25.10 public:0.25.9 public:0.25.8 public:0.25.7 public:0.25.6 public:0.25.5 public:0.25.4 public:0.25.3 public:0.25.2 public:0.25.1 public:0.25.0 public:0.25.0-beta1 public:0.24.31 public:0.24.30 public:0.24.29 public:0.24.28 public:0.24.27 public:0.24.26 public:0.24.25 public:0.24.24 public:0.24.23 public:0.24.22 public:0.24.21 public:0.24.20 public:0.24.19 public:0.24.18 public:0.24.17 public:0.24.16 public:0.24.15 public:0.24.14 public:0.24.13 public:0.24.12 public:0.24.11 public:0.24.10 public:0.24.9 public:0.24.8 public:0.24.7 public:0.24.6 public:0.24.5 public:0.24.4 public:0.24.3 public:0.24.2 public:0.24.1 public:0.24.0 public:0.24.0.dev-rc8 public:0.24.0.dev-rc7 public:0.24.0.dev-rc6 public:0.24.0.dev-rc5 public:0.24.0.dev-rc4 public:0.24.0.dev-rc3 public:0.24.0.dev-rc2 public:0.23.23 public:0.23.22 public:0.23.21 public:0.23.20 public:0.23.19 public:0.23.18 public:0.23.17 public:0.23.16 public:0.23.15 public:0.23.14 public:0.23.13 public:0.23.12 public:0.23.11 public:0.23.10 public:0.23.9 public:0.23.8 public:0.23.7 public:0.23.6 public:0.23.5 public:0.23.4 public:0.23.3 public:0.23.2 public:0.23.1 public:0.23.0 public:0.22.19 public:0.22.18 public:0.22.17 public:0.22.16 public:0.22.15 public:0.22.14 public:0.22.13 public:0.22.12 public:0.22.11 public:0.22.10 public:0.22.9 public:0.22.8 public:0.22.7 public:0.22.6 public:0.22.5 public:0.22.4 public:0.22.3 public:0.22.2 public:0.22.1 public:0.22.0 public:0.21.5 public:0.21.4 public:0.21.3 public:0.21.2 public:0.21.1 public:0.21.0 public:0.20.7 public:0.20.6 public:0.20.5 public:0.20.4 public:0.20.3 public:0.20.2 public:0.20.1 public:0.20.0 public:0.19.23 public:0.19.22 public:0.19.21 public:0.19.20 public:0.19.19 public:0.19.18 public:0.19.17 public:0.19.16 public:0.19.15 public:0.19.14 public:0.19.13 public:0.19.12 public:0.19.11 public:0.19.10 public:0.19.9 public:0.19.8 public:0.19.7 public:0.19.6 public:0.19.5 public:0.19.4 public:0.19.3 public:0.19.2 public:0.19.1 public:0.19.0 public:0.18.6 public:0.18.5 public:0.18.4 public:0.18.3 public:0.18.2 public:0.18.1 public:0.18.0 public:0.17.34 public:0.17.33 public:0.17.32 public:0.17.31 public:0.17.30 public:0.17.29 public:0.17.28 public:0.17.27 public:0.17.26 public:0.17.25 public:0.17.24 public:0.17.23 public:0.17.22 public:0.17.21 public:0.17.20 public:0.17.19 public:0.17.18 public:0.17.17 public:0.17.16 public:0.17.15 public:0.17.14 public:0.17.13 public:0.17.12 public:0.17.11 public:0.17.10 public:0.17.9 public:0.17.8 public:0.17.7 public:0.17.6 public:0.17.5 public:0.17.4 public:0.17.3 public:0.17.2 public:0.17.1 public:0.17.0 public:0.16.8 public:0.16.7 public:0.16.6 public:0.16.5 public:0.16.4 public:0.16.3 public:0.16.2 public:0.16.1 public:0.16.0 public:0.15.10 public:0.15.9 public:0.15.8 public:0.15.7 public:0.15.6 public:0.15.5 public:0.15.4 public:0.15.3 public:0.15.2 public:0.15.1 public:0.15.0 public:0.14.8 public:0.14.7 public:0.14.6 public:0.14.5 public:0.14.4 public:0.14.3 public:0.14.2 public:0.14.1 public:0.14.0 public:0.13.4 public:0.13.3 public:0.13.2 public:0.13.1 public:0.13.0 public:0.12.5 public:0.12.4 public:0.12.3 public:0.12.2 public:0.12.1 public:0.12.0 public:0.11.10 public:0.11.9 public:0.11.8 public:0.11.7 public:0.11.6 public:0.11.5 public:0.11.4 public:0.11.3 public:0.11.2 public:0.11.1 public:0.11.0 public:0.10.1 public:0.10.0 public:0.9.0 public:0.8.8 public:0.8.7 public:0.8.6 public:0.8.5 public:0.8.3 public:0.8.2 public:0.8.1 public:0.8.0 public:0.7.2 public:0.7.1 public:0.7.0 public:0.6.1 public:0.6.0 public:0.5.0 public:0.4.1 public:0.4.0 public:0.3.1 public:0.3.0 public:0.2.1 public:0.2.0 public:0.1.26 public:0.1.25 public:0.1.24 public:0.1.23 public:0.1.22 public:0.1.21 public:0.1.20 public:0.1.19 public:0.1.18 public:0.1.17 public:0.1.16 public:0.1.15 public:0.1.14 public:0.1.13 public:0.1.12 public:0.1.11 public:0.1.10 public:0.1.9 public:0.1.8 public:0.1.7 public:0.1.6 public:0.1.5 public:0.1.4 public:0.1.3 public:0.1.2 public:0.1.1 public:0.1.0 public:0.0.9 public:0.0.8 public:0.0.7 public:0.0.6
...
compare: public:beta
Branches Tags
public:beta public:main public:snyk-upgrade-7fd31e6f474568014502e7205847f257 public:snyk-fix-90f3b71450f7543a1c61724d8afa0e71 public:refactor_services_2 public:refactor_services public:dev public:snyk-upgrade-fe9625385aa4be849c7157033fd2abb0 public:snyk-upgrade-99a31e70c822894595544b0c9813b11a public:svelteui public:disenchant public:snyk-fix-61f76ef0bce2a276df9552c2fc80a14f public:snyk-upgrade-2baa7938ba318464c3bad5c3fd5cbda5 public:snyk-upgrade-7af62a575eab20e8fb4e9565bf1adcfc public:snyk-upgrade-b32f281b31a81483f126b8fb6c54a957 public:snyk-upgrade-bd61b1103442d3faecc9fc726c34e9a4 public:snyk-upgrade-5b2d6584430c061b5187611a5d7fb64a public:snyk-fix-0e9fbbcb374fcb8d5e8cb027d8414fb7 public:snyk-upgrade-5242daf91e706d1ae500c9ea6545636e public:snyk-upgrade-70434a6822fbad83424135dfad9df909 public:snyk-upgrade-8fa01ccad0352125f1866d41e57d35e1 public:snyk-upgrade-39eb97b4655e70091f718a88fccd57c1 public:snyk-upgrade-2b99d54d19e207887e36c3071b796925 public:forprivacy public:snyk-upgrade-dad9088d0a333677e0157e3deef7fc38 public:snyk-upgrade-b9a76d74b3e5723759af5b3319aabb80 public:snyk-upgrade-7da6459d8ff5d293d80de9d80061d912 public:snyk-upgrade-6a3786fc4cbe394542e9d1b6058c46e7 public:snyk-upgrade-8fc9e62771dc41b0f23dc0efefb5c949
public:0.25.44 public:0.25.43-patched-9 public:0.25.43-patched-8 public:0.25.43-patched-7 public:0.25.43-patched-6 public:0.25.43-patched-5 public:0.25.43-patched-4 public:0.25.43-patched-3 public:0.25.43-patched-2 public:0.25.43-patched-1 public:0.25.43 public:0.25.42 public:0.25.41-patched-3 public:0.25.41-patched-2 public:0.25.41-patched-1 public:0.25.41 public:0.25.40 public:0.25.39 public:0.25.38 public:0.25.37 public:0.25.36 public:0.25.35 public:0.25.34 public:0.25.33 public:0.25.32 public:0.25.31 public:0.25.30 public:0.25.30-dev public:0.25.29 public:0.25.28 public:0.25.27 public:0.25.26 public:0.25.25 public:0.25.24 public:0.25.24.beta3 public:0.25.24.beta2 public:0.25.24.beta1 public:0.25.23 public:0.25.23.beta1 public:0.25.22 public:0.25.21 public:0.25.21.beta2 public:0.25.21.beta1 public:0.25.20 public:0.25.19 public:0.25.18 public:0.25.17 public:0.25.16 public:0.25.15 public:0.25.14 public:0.25.13 public:0.25.12 public:0.25.11 public:0.25.10 public:0.25.9 public:0.25.8 public:0.25.7 public:0.25.6 public:0.25.5 public:0.25.4 public:0.25.3 public:0.25.2 public:0.25.1 public:0.25.0 public:0.25.0-beta1 public:0.24.31 public:0.24.30 public:0.24.29 public:0.24.28 public:0.24.27 public:0.24.26 public:0.24.25 public:0.24.24 public:0.24.23 public:0.24.22 public:0.24.21 public:0.24.20 public:0.24.19 public:0.24.18 public:0.24.17 public:0.24.16 public:0.24.15 public:0.24.14 public:0.24.13 public:0.24.12 public:0.24.11 public:0.24.10 public:0.24.9 public:0.24.8 public:0.24.7 public:0.24.6 public:0.24.5 public:0.24.4 public:0.24.3 public:0.24.2 public:0.24.1 public:0.24.0 public:0.24.0.dev-rc8 public:0.24.0.dev-rc7 public:0.24.0.dev-rc6 public:0.24.0.dev-rc5 public:0.24.0.dev-rc4 public:0.24.0.dev-rc3 public:0.24.0.dev-rc2 public:0.23.23 public:0.23.22 public:0.23.21 public:0.23.20 public:0.23.19 public:0.23.18 public:0.23.17 public:0.23.16 public:0.23.15 public:0.23.14 public:0.23.13 public:0.23.12 public:0.23.11 public:0.23.10 public:0.23.9 public:0.23.8 public:0.23.7 public:0.23.6 public:0.23.5 public:0.23.4 public:0.23.3 public:0.23.2 public:0.23.1 public:0.23.0 public:0.22.19 public:0.22.18 public:0.22.17 public:0.22.16 public:0.22.15 public:0.22.14 public:0.22.13 public:0.22.12 public:0.22.11 public:0.22.10 public:0.22.9 public:0.22.8 public:0.22.7 public:0.22.6 public:0.22.5 public:0.22.4 public:0.22.3 public:0.22.2 public:0.22.1 public:0.22.0 public:0.21.5 public:0.21.4 public:0.21.3 public:0.21.2 public:0.21.1 public:0.21.0 public:0.20.7 public:0.20.6 public:0.20.5 public:0.20.4 public:0.20.3 public:0.20.2 public:0.20.1 public:0.20.0 public:0.19.23 public:0.19.22 public:0.19.21 public:0.19.20 public:0.19.19 public:0.19.18 public:0.19.17 public:0.19.16 public:0.19.15 public:0.19.14 public:0.19.13 public:0.19.12 public:0.19.11 public:0.19.10 public:0.19.9 public:0.19.8 public:0.19.7 public:0.19.6 public:0.19.5 public:0.19.4 public:0.19.3 public:0.19.2 public:0.19.1 public:0.19.0 public:0.18.6 public:0.18.5 public:0.18.4 public:0.18.3 public:0.18.2 public:0.18.1 public:0.18.0 public:0.17.34 public:0.17.33 public:0.17.32 public:0.17.31 public:0.17.30 public:0.17.29 public:0.17.28 public:0.17.27 public:0.17.26 public:0.17.25 public:0.17.24 public:0.17.23 public:0.17.22 public:0.17.21 public:0.17.20 public:0.17.19 public:0.17.18 public:0.17.17 public:0.17.16 public:0.17.15 public:0.17.14 public:0.17.13 public:0.17.12 public:0.17.11 public:0.17.10 public:0.17.9 public:0.17.8 public:0.17.7 public:0.17.6 public:0.17.5 public:0.17.4 public:0.17.3 public:0.17.2 public:0.17.1 public:0.17.0 public:0.16.8 public:0.16.7 public:0.16.6 public:0.16.5 public:0.16.4 public:0.16.3 public:0.16.2 public:0.16.1 public:0.16.0 public:0.15.10 public:0.15.9 public:0.15.8 public:0.15.7 public:0.15.6 public:0.15.5 public:0.15.4 public:0.15.3 public:0.15.2 public:0.15.1 public:0.15.0 public:0.14.8 public:0.14.7 public:0.14.6 public:0.14.5 public:0.14.4 public:0.14.3 public:0.14.2 public:0.14.1 public:0.14.0 public:0.13.4 public:0.13.3 public:0.13.2 public:0.13.1 public:0.13.0 public:0.12.5 public:0.12.4 public:0.12.3 public:0.12.2 public:0.12.1 public:0.12.0 public:0.11.10 public:0.11.9 public:0.11.8 public:0.11.7 public:0.11.6 public:0.11.5 public:0.11.4 public:0.11.3 public:0.11.2 public:0.11.1 public:0.11.0 public:0.10.1 public:0.10.0 public:0.9.0 public:0.8.8 public:0.8.7 public:0.8.6 public:0.8.5 public:0.8.3 public:0.8.2 public:0.8.1 public:0.8.0 public:0.7.2 public:0.7.1 public:0.7.0 public:0.6.1 public:0.6.0 public:0.5.0 public:0.4.1 public:0.4.0 public:0.3.1 public:0.3.0 public:0.2.1 public:0.2.0 public:0.1.26 public:0.1.25 public:0.1.24 public:0.1.23 public:0.1.22 public:0.1.21 public:0.1.20 public:0.1.19 public:0.1.18 public:0.1.17 public:0.1.16 public:0.1.15 public:0.1.14 public:0.1.13 public:0.1.12 public:0.1.11 public:0.1.10 public:0.1.9 public:0.1.8 public:0.1.7 public:0.1.6 public:0.1.5 public:0.1.4 public:0.1.3 public:0.1.2 public:0.1.1 public:0.1.0 public:0.0.9 public:0.0.8 public:0.0.7 public:0.0.6

33 Commits
0.25.43-pa ... beta

Author SHA1 Message Date
vorotamoroz
be1642f1c1 Fix grammatical errors. 2026-02-24 11:20:59 +00:00
vorotamoroz
c9a71e2076 fix: align dependency versions to main 2026-02-24 11:14:14 +00:00
vorotamoroz
2199c1ebd3 change: test:docker-all:xxx to be parallel. 2026-02-24 11:13:58 +00:00
vorotamoroz
278935f85d Fix mock for testing 2026-02-24 11:08:08 +00:00
vorotamoroz
010631f553 bump dependencies 2026-02-24 10:38:35 +00:00
vorotamoroz
8c0c65307a bump 2026-02-24 08:23:08 +00:00
vorotamoroz
988cb34d7c Update 2026-02-24 07:56:23 +00:00
vorotamoroz
6eec8117f5 Refactor: Constantising log-mark 2026-02-24 07:51:21 +00:00
vorotamoroz
9f6a909143 Merge remote-tracking branch 'origin/main' into beta to port #802 2026-02-24 07:44:18 +00:00
vorotamoroz
09f283721a Merge remote-tracking branch 'origin/main' into beta 2026-02-24 04:13:34 +00:00
vorotamoroz
235c702223 Merge pull request #804 from A-wry/reduce-fetch-error-visibility 
Add connection warning style logic and settings UI dropdown
 2026-02-24 13:07:29 +09:00
A-wry
b923b43b6b update submodule pointer to latest commonlib changes 2026-02-23 22:29:46 -05:00
A-wry
fdcf3be0f9 Tagged downstream network errors to respect networkWarningStyle setting 2026-02-23 22:10:25 -05:00
vorotamoroz
25dd907591 port PR #802 2026-02-24 01:57:08 +00:00
vorotamoroz
80c049d276 Merge pull request #802 from waspeer/fix/write-file-auto-md-file-already-exists 
fix: handle "File already exists" for .md files in writeFileAuto
 2026-02-24 10:50:37 +09:00
vorotamoroz
e961f01187 fix typos. 2026-02-21 14:17:28 +09:00
vorotamoroz
14b4c3cd50 prettified 2026-02-21 14:12:05 +09:00
vorotamoroz
4f987e7c2b ### Fixed 
- Hidden file synchronisation now works!
- Now Hidden file synchronisation respects `.ignore` files.
- Replicator initialisation during rebuilding now works correctly.

### Refactored

- Some methods naming have been changed for better clarity, i.e., `_isTargetFileByLocalDB` is now `_isTargetAcceptedByLocalDB`.
 2026-02-21 14:05:32 +09:00
A-wry
f4d8c0a8db Add connection warning style logic and settings UI dropdown 2026-02-20 21:51:30 -05:00
vorotamoroz
556ce471f8 ## 0.25.43-patched-8 2026-02-20 14:28:28 +00:00
vorotamoroz
32b6717114 keep a note 2026-02-19 10:38:42 +00:00
vorotamoroz
e0e72fae72 Fixed: saving device name 2026-02-19 10:37:19 +00:00
vorotamoroz
203dd17421 for 0.25.43-patched-7, please refer to the updates.md 2026-02-19 10:23:45 +00:00
vorotamoroz
1bde2b2ff1 Fixed an issue where the StorageEventManager 
Build by Vite is now testing
 2026-02-19 04:18:18 +00:00
vorotamoroz
2bf1c775ee ## 0.25.43-patched-6 
### Fixed

- Unlocking the remote database after rebuilding has been fixed.

### Refactored
- Now `StorageEventManagerBase` is separated from `StorageEventManagerObsidian` following their concerns.
- Now `FileAccessBase` is separated from `FileAccessObsidian` following their concerns.
 2026-02-18 12:13:05 +00:00
vorotamoroz
4658e3735d Fix Shim 2026-02-17 10:56:05 +00:00
vorotamoroz
627edc96bf bump for beta 2026-02-17 10:14:13 +00:00
vorotamoroz
0a1917e83c Refactor for 0.25.43-patched-5 (very long, please refer the updates.md) 2026-02-17 10:14:04 +00:00
Wannes Salomé
48b0d22da6 fix: handle "File already exists" for .md files in writeFileAuto 
During concurrent initialisation (UPDATE STORAGE runs up to 10 ops in
parallel), getAbstractFileByPath can return null for .md files whose
vault index entry hasn't been populated yet, even though the file
already exists on disk. This causes vault.create() to throw "File
already exists."

The same root cause (stale in-memory index) was already identified for
non-.md files (see comment above) and handled via adapterWrite. Extend
that workaround to .md files by catching the "File already exists"
error and falling back to adapterWrite, consistent with the existing
approach.

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
 2026-02-17 00:24:30 +01:00
vorotamoroz
3201399bdf bump 2026-02-16 11:51:09 +00:00
vorotamoroz
2ae70e8f07 Refactor: DatabaseService and Replicator 2026-02-16 11:51:03 +00:00
vorotamoroz
2b9bb1ed06 beta bump 2026-02-16 06:51:52 +00:00
vorotamoroz
e63e3e6725 ### Refactor 
- Module dependency refined. (For details, please refer to updates.md)
 2026-02-16 06:50:31 +00:00
101 changed files with 5294 additions and 5166 deletions
Expand all files Collapse all files

17
devs.md
View File

@@ -11,13 +11,28 @@ The plugin uses a dynamic module system to reduce coupling and improve maintaina
- **Service Hub**: Central registry for services using dependency injection
- Services are registered, and accessed via `this.services` (in most modules)
- **Module Loading**: All modules extend `AbstractModule` or `AbstractObsidianModule` (which extends `AbstractModule`). These modules are loaded in main.ts and some modules
- **Module Loading**: All modules extend `AbstractModule` or `AbstractObsidianModule` (which extends `AbstractModule`). These modules are loaded in main.ts and some modules.
- **Module Categories** (by directory):
- `core/` - Platform-independent core functionality
- `coreObsidian/` - Obsidian-specific core (e.g., `ModuleFileAccessObsidian`)
- `essential/` - Required modules (e.g., `ModuleMigration`, `ModuleKeyValueDB`)
- `features/` - Optional features (e.g., `ModuleLog`, `ModuleObsidianSettings`)
- `extras/` - Development/testing tools (e.g., `ModuleDev`, `ModuleIntegratedTest`)
- **Services**: Core services (e.g., `database`, `replicator`, `storageAccess`) are registered in `ServiceHub` and accessed by modules. They provide an extension point for add new behaviour without modifying existing code.
- For example, checks before the replication can be added to the `replication.onBeforeReplicate` handler, and the handlers can be return `false` to prevent replication-starting. `vault.isTargetFile` also can be used to prevent processing specific files.
- **ServiceModule**: A new type of module that directly depends on services.
#### Note on Module vs Service
After v0.25.44 refactoring, the Service will henceforth, as a rule, cease to use setHandler, that is to say, simple lazy binding. - They will be implemented directly in the service. - However, not everything will be middlewarised. Modules that maintain state or make decisions based on the results of multiple handlers are permitted.
Hence, the new feature should be implemented as follows:
- If it is a simple extension point (e.g., adding a check before replication), it should be implemented as a handler in the service (e.g., `replication.onBeforeReplicate`).
- If it requires maintaining state or making decisions based on multiple handlers, it should be implemented as a serviceModule dependent on the relevant services explicitly.
- If you have to implement a new feature without much modification, you can extent existing modules, but it is recommended to implement a new module or serviceModule for better maintainability.
- Refactoring existing modules to services is also always welcome!
- Please write tests for new features, you will notice that the simple handler approach is quite testable.
### Key Architectural Components

2
manifest.json
View File

@@ -1,7 +1,7 @@
{
"id": "obsidian-livesync",
"name": "Self-hosted LiveSync",
"version": "0.25.43-patched-2",
"version": "0.25.44",
"minAppVersion": "0.9.12",
"description": "Community implementation of self-hosted livesync. Reflect your vault changes to some other devices immediately. Please make sure to disable other synchronize solutions to avoid content corruption or duplication.",
"author": "vorotamoroz",

4100
package-lock.json generated
View File

File diff suppressed because it is too large Load Diff

71
package.json
View File

@@ -1,6 +1,6 @@
{
"name": "obsidian-livesync",
"version": "0.25.43-patched-2",
"version": "0.25.44",
"description": "Reflect your vault changes to some other devices immediately. Please make sure to disable other synchronize solutions to avoid content corruption or duplication.",
"main": "main.js",
"type": "module",
@@ -16,6 +16,8 @@
"dev": "node --env-file=.env esbuild.config.mjs",
"prebuild": "npm run bakei18n",
"build": "node esbuild.config.mjs production",
"buildVite": "npx dotenv-cli -e .env -- vite build --mode production",
"buildViteOriginal": "npx dotenv-cli -e .env -- vite build --mode original",
"buildDev": "node esbuild.config.mjs dev",
"lint": "eslint src",
"svelte-check": "svelte-check --tsconfig ./tsconfig.json",
@@ -46,9 +48,9 @@
"test:docker-p2p:start": "npm run test:docker-p2p:up && sleep 3 && npm run test:docker-p2p:init",
"test:docker-p2p:down": "npx dotenv-cli -e .env -e .test.env -- ./test/shell/p2p-stop.sh",
"test:docker-p2p:stop": "npm run test:docker-p2p:down",
"test:docker-all:up": "npm run test:docker-couchdb:up && npm run test:docker-s3:up && npm run test:docker-p2p:up",
"test:docker-all:init": "npm run test:docker-couchdb:init && npm run test:docker-s3:init && npm run test:docker-p2p:init",
"test:docker-all:down": "npm run test:docker-couchdb:down && npm run test:docker-s3:down && npm run test:docker-p2p:down",
"test:docker-all:up": "npm run test:docker-couchdb:up ; npm run test:docker-s3:up ; npm run test:docker-p2p:up",
"test:docker-all:init": "npm run test:docker-couchdb:init ; npm run test:docker-s3:init ; npm run test:docker-p2p:init",
"test:docker-all:down": "npm run test:docker-couchdb:down ; npm run test:docker-s3:down ; npm run test:docker-p2p:down",
"test:docker-all:start": "npm run test:docker-all:up && sleep 5 && npm run test:docker-all:init",
"test:docker-all:stop": "npm run test:docker-all:down",
"test:full": "npm run test:docker-all:start && vitest run --coverage && npm run test:docker-all:stop"
@@ -57,15 +59,15 @@
"author": "vorotamoroz",
"license": "MIT",
"devDependencies": {
"@chialab/esbuild-plugin-worker": "^0.18.1",
"@eslint/compat": "^1.2.7",
"@eslint/eslintrc": "^3.3.0",
"@eslint/js": "^9.21.0",
"@sveltejs/vite-plugin-svelte": "^6.2.1",
"@tsconfig/svelte": "^5.0.5",
"@types/deno": "^2.3.0",
"@chialab/esbuild-plugin-worker": "^0.19.0",
"@eslint/compat": "^2.0.2",
"@eslint/eslintrc": "^3.3.4",
"@eslint/js": "^9.39.3",
"@sveltejs/vite-plugin-svelte": "^6.2.4",
"@tsconfig/svelte": "^5.0.8",
"@types/deno": "^2.5.0",
"@types/diff-match-patch": "^1.0.36",
"@types/node": "^22.13.8",
"@types/node": "^24.10.13",
"@types/pouchdb": "^6.4.2",
"@types/pouchdb-adapter-http": "^6.1.6",
"@types/pouchdb-adapter-idb": "^6.1.7",
@@ -74,25 +76,25 @@
"@types/pouchdb-mapreduce": "^6.1.10",
"@types/pouchdb-replication": "^6.4.7",
"@types/transform-pouch": "^1.0.6",
"@typescript-eslint/eslint-plugin": "8.46.2",
"@typescript-eslint/parser": "8.46.2",
"@typescript-eslint/eslint-plugin": "8.56.1",
"@typescript-eslint/parser": "8.56.1",
"@vitest/browser": "^4.0.16",
"@vitest/browser-playwright": "^4.0.16",
"@vitest/coverage-v8": "^4.0.16",
"builtin-modules": "5.0.0",
"dotenv": "^17.2.3",
"dotenv": "^17.3.1",
"dotenv-cli": "^11.0.0",
"esbuild": "0.25.0",
"esbuild-plugin-inline-worker": "^0.1.1",
"esbuild-svelte": "^0.9.3",
"eslint": "^9.38.0",
"esbuild-svelte": "^0.9.4",
"eslint": "^9.39.3",
"eslint-plugin-import": "^2.32.0",
"eslint-plugin-svelte": "^3.12.4",
"eslint-plugin-svelte": "^3.15.0",
"events": "^3.3.0",
"glob": "^11.0.3",
"obsidian": "^1.8.7",
"playwright": "^1.57.0",
"postcss": "^8.5.3",
"glob": "^13.0.6",
"obsidian": "^1.12.3",
"playwright": "^1.58.2",
"postcss": "^8.5.6",
"postcss-load-config": "^6.0.1",
"pouchdb-adapter-http": "^9.0.0",
"pouchdb-adapter-idb": "^9.0.0",
@@ -105,31 +107,32 @@
"pouchdb-merge": "^9.0.0",
"pouchdb-replication": "^9.0.0",
"pouchdb-utils": "^9.0.0",
"prettier": "3.5.2",
"prettier": "3.8.1",
"rollup-plugin-copy": "^3.5.0",
"svelte": "5.41.1",
"svelte-check": "^4.3.3",
"svelte-check": "^4.4.3",
"svelte-preprocess": "^6.0.3",
"terser": "^5.39.0",
"transform-pouch": "^2.0.0",
"tslib": "^2.8.1",
"tsx": "^4.20.6",
"tsx": "^4.21.0",
"typescript": "5.9.3",
"vite": "^7.3.0",
"vite": "^7.3.1",
"vitest": "^4.0.16",
"webdriverio": "^9.23.0",
"yaml": "^2.8.0"
"webdriverio": "^9.24.0",
"yaml": "^2.8.2"
},
"dependencies": {
"@aws-sdk/client-s3": "^3.808.0",
"@smithy/fetch-http-handler": "^5.0.2",
"@smithy/md5-js": "^4.0.2",
"@smithy/middleware-apply-body-checksum": "^4.1.0",
"@smithy/protocol-http": "^5.1.0",
"@smithy/querystring-builder": "^4.0.2",
"@smithy/fetch-http-handler": "^5.3.10",
"@smithy/md5-js": "^4.2.9",
"@smithy/middleware-apply-body-checksum": "^4.3.9",
"@smithy/protocol-http": "^5.3.9",
"@smithy/querystring-builder": "^4.2.9",
"diff-match-patch": "^1.0.5",
"fflate": "^0.8.2",
"idb": "^8.0.3",
"minimatch": "^10.0.2",
"minimatch": "^10.2.2",
"octagonal-wheels": "^0.1.45",
"qrcode-generator": "^1.4.4",
"trystero": "^0.22.0",

10
src/apps/webpeer/package.json
View File

@@ -11,13 +11,13 @@
},
"dependencies": {},
"devDependencies": {
"eslint-plugin-svelte": "^3.12.4",
"@sveltejs/vite-plugin-svelte": "^6.2.1",
"@tsconfig/svelte": "^5.0.5",
"eslint-plugin-svelte": "^3.15.0",
"@sveltejs/vite-plugin-svelte": "^6.2.4",
"@tsconfig/svelte": "^5.0.8",
"svelte": "5.41.1",
"svelte-check": "^4.3.3",
"svelte-check": "^443.3",
"typescript": "5.9.3",
"vite": "^7.3.0"
"vite": "^7.3.1"
},
"imports": {
"../../src/worker/bgWorker.ts": "../../src/worker/bgWorker.mock.ts",

23
src/apps/webpeer/src/P2PReplicatorShim.ts
View File

@@ -34,8 +34,11 @@ import { TrysteroReplicator } from "@lib/replication/trystero/TrysteroReplicator
import { SETTING_KEY_P2P_DEVICE_NAME } from "@lib/common/types";
import { ServiceContext } from "@lib/services/base/ServiceBase";
import type { InjectableServiceHub } from "@lib/services/InjectableServices";
import { Menu } from "@/lib/src/services/implements/browser/Menu";
import type { InjectableVaultServiceCompat } from "@/lib/src/services/implements/injectable/InjectableVaultService";
import { Menu } from "@lib/services/implements/browser/Menu";
import type { InjectableVaultServiceCompat } from "@lib/services/implements/injectable/InjectableVaultService";
import { SimpleStoreIDBv2 } from "octagonal-wheels/databases/SimpleStoreIDBv2";
import type { InjectableAPIService } from "@/lib/src/services/implements/injectable/InjectableAPIService";
import type { BrowserAPIService } from "@/lib/src/services/implements/browser/BrowserAPIService";
function addToList(item: string, list: string) {
return unique(
@@ -80,13 +83,10 @@ export class P2PReplicatorShim implements P2PReplicatorBase, CommandShim {
constructor() {
const browserServiceHub = new BrowserServiceHub<ServiceContext>();
this.services = browserServiceHub;
(this.services.vault as InjectableVaultServiceCompat<ServiceContext>).vaultName.setHandler(
(this.services.API as BrowserAPIService<ServiceContext>).getSystemVaultName.setHandler(
() => "p2p-livesync-web-peer"
);
this.services.setting.currentSettings.setHandler(() => {
return this.settings as any;
});
}
async init() {
// const { simpleStoreAPI } = await getWrappedSynchromesh();
@@ -102,11 +102,10 @@ export class P2PReplicatorShim implements P2PReplicatorBase, CommandShim {
Logger(ex, LOG_LEVEL_VERBOSE);
}
}
const repStore = this.services.database.openSimpleStore<any>("p2p-livesync-web-peer");
const repStore = SimpleStoreIDBv2.open<any>("p2p-livesync-web-peer");
this._simpleStore = repStore;
let _settings = (await repStore.get("settings")) || ({ ...P2P_DEFAULT_SETTINGS } as P2PSyncSetting);
this.services.setting.settings = _settings as any;
this.plugin = {
saveSettings: async () => {
await repStore.set("settings", _settings);
@@ -148,9 +147,9 @@ export class P2PReplicatorShim implements P2PReplicatorBase, CommandShim {
simpleStore(): SimpleStore<any> {
return this._simpleStore;
}
handleReplicatedDocuments(docs: EntryDoc[]): Promise<void> {
handleReplicatedDocuments(docs: EntryDoc[]): Promise<boolean> {
// No op. This is a client and does not need to process the docs
return Promise.resolve();
return Promise.resolve(true);
}
getPluginShim() {

4
src/common/SvelteItemView.ts
View File

@@ -4,7 +4,7 @@ import { type mount, unmount } from "svelte";
export abstract class SvelteItemView extends ItemView {
abstract instantiateComponent(target: HTMLElement): ReturnType<typeof mount> | Promise<ReturnType<typeof mount>>;
component?: ReturnType<typeof mount>;
async onOpen() {
override async onOpen() {
await super.onOpen();
this.contentEl.empty();
await this._dismountComponent();
@@ -17,7 +17,7 @@ export abstract class SvelteItemView extends ItemView {
this.component = undefined;
}
}
async onClose() {
override async onClose() {
await super.onClose();
if (this.component) {
await unmount(this.component);

2
src/common/events.ts
View File

@@ -21,7 +21,6 @@ export const EVENT_REQUEST_CLOSE_P2P = "request-close-p2p";
export const EVENT_REQUEST_RUN_DOCTOR = "request-run-doctor";
export const EVENT_REQUEST_RUN_FIX_INCOMPLETE = "request-run-fix-incomplete";
export const EVENT_ON_UNRESOLVED_ERROR = "on-unresolved-error";
export const EVENT_ANALYSE_DB_USAGE = "analyse-db-usage";
export const EVENT_REQUEST_PERFORM_GC_V3 = "request-perform-gc-v3";
@@ -44,7 +43,6 @@ declare global {
[EVENT_REQUEST_SHOW_SETUP_QR]: undefined;
[EVENT_REQUEST_RUN_DOCTOR]: string;
[EVENT_REQUEST_RUN_FIX_INCOMPLETE]: undefined;
[EVENT_ON_UNRESOLVED_ERROR]: undefined;
[EVENT_ANALYSE_DB_USAGE]: undefined;
[EVENT_REQUEST_CHECK_REMOTE_SIZE]: undefined;
[EVENT_REQUEST_PERFORM_GC_V3]: undefined;

77
src/common/utils.ts
View File

@@ -23,7 +23,7 @@ import {
type UXFileInfo,
type UXFileInfoStub,
} from "../lib/src/common/types.ts";
import { ICHeader, ICXHeader } from "./types.ts";
export { ICHeader, ICXHeader } from "./types.ts";
import type ObsidianLiveSyncPlugin from "../main.ts";
import { writeString } from "../lib/src/string_and_binary/convert.ts";
import { fireAndForget } from "../lib/src/common/utils.ts";
@@ -31,7 +31,6 @@ import { sameChangePairs } from "./stores.ts";
import { scheduleTask } from "octagonal-wheels/concurrency/task";
import { EVENT_PLUGIN_UNLOADED, eventHub } from "./events.ts";
import { promiseWithResolver, type PromiseWithResolvers } from "octagonal-wheels/promises";
import { AuthorizationHeaderGenerator } from "../lib/src/replication/httplib.ts";
import type { KeyValueDatabase } from "../lib/src/interfaces/KeyValueDatabase.ts";
@@ -68,21 +67,13 @@ export function getPathFromTFile(file: TAbstractFile) {
return file.path as FilePath;
}
import { isInternalFile } from "@lib/common/typeUtils.ts";
export function getPathFromUXFileInfo(file: UXFileInfoStub | string | FilePathWithPrefix) {
if (typeof file == "string") return file as FilePathWithPrefix;
return file.path;
}
export function getStoragePathFromUXFileInfo(file: UXFileInfoStub | string | FilePathWithPrefix) {
if (typeof file == "string") return stripAllPrefixes(file as FilePathWithPrefix);
return stripAllPrefixes(file.path);
}
export function getDatabasePathFromUXFileInfo(file: UXFileInfoStub | string | FilePathWithPrefix) {
if (typeof file == "string" && file.startsWith(ICXHeader)) return file as FilePathWithPrefix;
const prefix = isInternalFile(file) ? ICHeader : "";
if (typeof file == "string") return (prefix + stripAllPrefixes(file as FilePathWithPrefix)) as FilePathWithPrefix;
return (prefix + stripAllPrefixes(file.path)) as FilePathWithPrefix;
}
import {
isInternalFile,
getPathFromUXFileInfo,
getStoragePathFromUXFileInfo,
getDatabasePathFromUXFileInfo,
} from "@lib/common/typeUtils.ts";
export { isInternalFile, getPathFromUXFileInfo, getStoragePathFromUXFileInfo, getDatabasePathFromUXFileInfo };
const memos: { [key: string]: any } = {};
export function memoObject<T>(key: string, obj: T): T {
@@ -160,7 +151,7 @@ export class PeriodicProcessor {
() =>
fireAndForget(async () => {
await this.process();
if (this._plugin.services?.appLifecycle?.hasUnloaded()) {
if (this._plugin.services?.control?.hasUnloaded()) {
this.disable();
}
}),
@@ -263,10 +254,8 @@ export function requestToCouchDBWithCredentials(
return _requestToCouchDB(baseUri, credentials, origin, uri, body, method, customHeaders);
}
export const BASE_IS_NEW = Symbol("base");
export const TARGET_IS_NEW = Symbol("target");
export const EVEN = Symbol("even");
import { BASE_IS_NEW, EVEN, TARGET_IS_NEW } from "@lib/common/models/shared.const.symbols.ts";
export { BASE_IS_NEW, EVEN, TARGET_IS_NEW };
// Why 2000? : ZIP FILE Does not have enough resolution.
const resolution = 2000;
export function compareMTime(
@@ -469,47 +458,3 @@ export function onlyInNTimes(n: number, proc: (progress: number) => any) {
}
};
}
const waitingTasks = {} as Record<string, { task?: PromiseWithResolvers<any>; previous: number; leastNext: number }>;
export function rateLimitedSharedExecution<T>(key: string, interval: number, proc: () => Promise<T>): Promise<T> {
if (!(key in waitingTasks)) {
waitingTasks[key] = { task: undefined, previous: 0, leastNext: 0 };
}
if (waitingTasks[key].task) {
// Extend the previous execution time.
waitingTasks[key].leastNext = Date.now() + interval;
return waitingTasks[key].task.promise;
}
const previous = waitingTasks[key].previous;
const delay = previous == 0 ? 0 : Math.max(interval - (Date.now() - previous), 0);
const task = promiseWithResolver<T>();
void task.promise.finally(() => {
if (waitingTasks[key].task === task) {
waitingTasks[key].task = undefined;
waitingTasks[key].previous = Math.max(Date.now(), waitingTasks[key].leastNext);
}
});
waitingTasks[key] = {
task,
previous: Date.now(),
leastNext: Date.now() + interval,
};
void scheduleTask("thin-out-" + key, delay, async () => {
try {
task.resolve(await proc());
} catch (ex) {
task.reject(ex);
}
});
return task.promise;
}
export function updatePreviousExecutionTime(key: string, timeDelta: number = 0) {
if (!(key in waitingTasks)) {
waitingTasks[key] = { task: undefined, previous: 0, leastNext: 0 };
}
waitingTasks[key].leastNext = Math.max(Date.now() + timeDelta, waitingTasks[key].leastNext);
}

1
src/deps.ts
View File

@@ -40,6 +40,7 @@ export type {
MarkdownFileInfo,
ListedFiles,
ValueComponent,
Stat,
} from "obsidian";
import { normalizePath as normalizePath_ } from "obsidian";
const normalizePath = normalizePath_ as <T extends string | FilePath>(from: T) => T;

2
src/features/ConfigSync/CmdConfigSync.ts
View File

@@ -1802,7 +1802,7 @@ export class ConfigSync extends LiveSyncCommands {
}
return files;
}
onBindFunction(core: LiveSyncCore, services: InjectableServiceHub): void {
override onBindFunction(core: LiveSyncCore, services: InjectableServiceHub): void {
services.fileProcessing.processOptionalFileEvent.addHandler(this._anyProcessOptionalFileEvent.bind(this));
services.conflict.getOptionalConflictCheckMethod.addHandler(this._anyGetOptionalConflictCheckMethod.bind(this));
services.replication.processVirtualDocument.addHandler(this._anyModuleParsedReplicationResultItem.bind(this));

4
src/features/ConfigSync/PluginDialogModal.ts
View File

@@ -14,7 +14,7 @@ export class PluginDialogModal extends Modal {
this.plugin = plugin;
}
onOpen() {
override onOpen() {
const { contentEl } = this;
this.contentEl.style.overflow = "auto";
this.contentEl.style.display = "flex";
@@ -28,7 +28,7 @@ export class PluginDialogModal extends Modal {
}
}
onClose() {
override onClose() {
if (this.component) {
void unmount(this.component);
this.component = undefined;

4
src/features/HiddenFileCommon/JsonResolveModal.ts
View File

@@ -50,7 +50,7 @@ export class JsonResolveModal extends Modal {
this.callback = undefined;
}
onOpen() {
override onOpen() {
const { contentEl } = this;
this.titleEl.setText(this.title);
contentEl.empty();
@@ -74,7 +74,7 @@ export class JsonResolveModal extends Modal {
return;
}
onClose() {
override onClose() {
const { contentEl } = this;
contentEl.empty();
// contentEl.empty();

2
src/features/HiddenFileSync/CmdHiddenFileSync.ts
View File

@@ -1934,7 +1934,7 @@ ${messageFetch}${messageOverwrite}${messageMerge}
*/
// <-- Local Storage SubFunctions
onBindFunction(core: LiveSyncCore, services: typeof core.services) {
override onBindFunction(core: LiveSyncCore, services: typeof core.services) {
// No longer needed on initialisation
// services.databaseEvents.handleOnDatabaseInitialisation(this._everyOnInitializeDatabase.bind(this));
services.appLifecycle.onSettingLoaded.addHandler(this._everyOnloadAfterLoadSettings.bind(this));

12
src/features/LiveSyncCommands.ts
View File

@@ -1,4 +1,4 @@
import { LOG_LEVEL_VERBOSE, Logger } from "octagonal-wheels/common/logger";
import { LOG_LEVEL_VERBOSE } from "octagonal-wheels/common/logger";
import {
LOG_LEVEL_INFO,
LOG_LEVEL_NOTICE,
@@ -12,6 +12,7 @@ import type ObsidianLiveSyncPlugin from "../main.ts";
import { MARK_DONE } from "../modules/features/ModuleLog.ts";
import type { LiveSyncCore } from "../main.ts";
import { __$checkInstanceBinding } from "../lib/src/dev/checks.ts";
import { createInstanceLogFunction } from "@/lib/src/services/lib/logUtils.ts";
let noticeIndex = 0;
export abstract class LiveSyncCommands {
@@ -43,6 +44,7 @@ export abstract class LiveSyncCommands {
constructor(plugin: ObsidianLiveSyncPlugin) {
this.plugin = plugin;
this.onBindFunction(plugin, plugin.services);
this._log = createInstanceLogFunction(this.constructor.name, this.services.API);
__$checkInstanceBinding(this);
}
abstract onunload(): void;
@@ -58,13 +60,7 @@ export abstract class LiveSyncCommands {
return this.services.database.isDatabaseReady();
}
_log = (msg: any, level: LOG_LEVEL = LOG_LEVEL_INFO, key?: string) => {
if (typeof msg === "string" && level !== LOG_LEVEL_NOTICE) {
msg = `[${this.constructor.name}]\u{200A} ${msg}`;
}
// console.log(msg);
Logger(msg, level, key);
};
_log: ReturnType<typeof createInstanceLogFunction>;
_verbose = (msg: any, key?: string) => {
this._log(msg, LOG_LEVEL_VERBOSE, key);

7
src/features/P2PSync/CmdP2PReplicator.ts
View File

@@ -40,9 +40,6 @@ export class P2PReplicator extends LiveSyncCommands implements P2PReplicatorBase
getSettings(): P2PSyncSetting {
return this.plugin.settings;
}
get settings() {
return this.plugin.settings;
}
getDB() {
return this.plugin.localDatabase.localDatabase;
}
@@ -65,7 +62,7 @@ export class P2PReplicator extends LiveSyncCommands implements P2PReplicatorBase
// this.onBindFunction(plugin, plugin.services);
}
async handleReplicatedDocuments(docs: EntryDoc[]): Promise<void> {
async handleReplicatedDocuments(docs: EntryDoc[]): Promise<boolean> {
// console.log("Processing Replicated Docs", docs);
return await this.services.replication.parseSynchroniseResult(
docs as PouchDB.Core.ExistingDocument<EntryDoc>[]
@@ -107,7 +104,7 @@ export class P2PReplicator extends LiveSyncCommands implements P2PReplicatorBase
}
init() {
this._simpleStore = this.services.database.openSimpleStore("p2p-sync");
this._simpleStore = this.services.keyValueDB.openSimpleStore("p2p-sync");
return Promise.resolve(this);
}

6
src/features/P2PSync/P2PReplicator/P2PReplicatorPaneView.ts
View File

@@ -35,11 +35,11 @@ function removeFromList(item: string, list: string) {
export class P2PReplicatorPaneView extends SvelteItemView {
plugin: ObsidianLiveSyncPlugin;
icon = "waypoints";
override icon = "waypoints";
title: string = "";
navigation = false;
override navigation = false;
getIcon(): string {
override getIcon(): string {
return "waypoints";
}
get replicator() {

2
src/lib

Submodule src/lib updated: 69e7a510f1...1c176da469

482
src/main.ts
View File

@@ -1,76 +1,69 @@
import { Plugin } from "./deps";
import { Plugin, type App, type PluginManifest } from "./deps";
import {
type EntryDoc,
type ObsidianLiveSyncSettings,
type DatabaseConnectingStatus,
type HasSettings,
LOG_LEVEL_INFO,
} from "./lib/src/common/types.ts";
import { type SimpleStore } from "./lib/src/common/utils.ts";
import { LiveSyncLocalDB, type LiveSyncLocalDBEnv } from "./lib/src/pouchdb/LiveSyncLocalDB.ts";
import {
LiveSyncAbstractReplicator,
type LiveSyncReplicatorEnv,
} from "./lib/src/replication/LiveSyncAbstractReplicator.js";
import { type KeyValueDatabase } from "./lib/src/interfaces/KeyValueDatabase.ts";
import { type LiveSyncLocalDBEnv } from "./lib/src/pouchdb/LiveSyncLocalDB.ts";
import { type LiveSyncReplicatorEnv } from "./lib/src/replication/LiveSyncAbstractReplicator.js";
import { LiveSyncCommands } from "./features/LiveSyncCommands.ts";
import { HiddenFileSync } from "./features/HiddenFileSync/CmdHiddenFileSync.ts";
import { ConfigSync } from "./features/ConfigSync/CmdConfigSync.ts";
import { reactiveSource, type ReactiveValue } from "octagonal-wheels/dataobject/reactive";
import { type LiveSyncJournalReplicatorEnv } from "./lib/src/replication/journal/LiveSyncJournalReplicator.js";
import { type LiveSyncCouchDBReplicatorEnv } from "./lib/src/replication/couchdb/LiveSyncReplicator.js";
import type { CheckPointInfo } from "./lib/src/replication/journal/JournalSyncTypes.js";
import type { IObsidianModule } from "./modules/AbstractObsidianModule.ts";
import { ModuleDev } from "./modules/extras/ModuleDev.ts";
import { ModuleFileAccessObsidian } from "./modules/coreObsidian/ModuleFileAccessObsidian.ts";
import { ModuleMigration } from "./modules/essential/ModuleMigration.ts";
import { ModuleCheckRemoteSize } from "./modules/essentialObsidian/ModuleCheckRemoteSize.ts";
import { ModuleConflictResolver } from "./modules/coreFeatures/ModuleConflictResolver.ts";
import { ModuleInteractiveConflictResolver } from "./modules/features/ModuleInteractiveConflictResolver.ts";
import { ModuleLog } from "./modules/features/ModuleLog.ts";
import { ModuleObsidianSettings } from "./modules/features/ModuleObsidianSetting.ts";
import { ModuleRedFlag } from "./modules/coreFeatures/ModuleRedFlag.ts";
import { ModuleObsidianMenu } from "./modules/essentialObsidian/ModuleObsidianMenu.ts";
import { ModuleSetupObsidian } from "./modules/features/ModuleSetupObsidian.ts";
import { SetupManager } from "./modules/features/SetupManager.ts";
import type { StorageAccess } from "./modules/interfaces/StorageAccess.ts";
import type { StorageAccess } from "@lib/interfaces/StorageAccess.ts";
import type { Confirm } from "./lib/src/interfaces/Confirm.ts";
import type { Rebuilder } from "./modules/interfaces/DatabaseRebuilder.ts";
import type { DatabaseFileAccess } from "./modules/interfaces/DatabaseFileAccess.ts";
import { ModuleDatabaseFileAccess } from "./modules/core/ModuleDatabaseFileAccess.ts";
import { ModuleFileHandler } from "./modules/core/ModuleFileHandler.ts";
import type { Rebuilder } from "@lib/interfaces/DatabaseRebuilder.ts";
import type { DatabaseFileAccess } from "@lib/interfaces/DatabaseFileAccess.ts";
import { ModuleObsidianAPI } from "./modules/essentialObsidian/ModuleObsidianAPI.ts";
import { ModuleObsidianEvents } from "./modules/essentialObsidian/ModuleObsidianEvents.ts";
import { type AbstractModule } from "./modules/AbstractModule.ts";
import { AbstractModule } from "./modules/AbstractModule.ts";
import { ModuleObsidianSettingDialogue } from "./modules/features/ModuleObsidianSettingTab.ts";
import { ModuleObsidianDocumentHistory } from "./modules/features/ModuleObsidianDocumentHistory.ts";
import { ModuleObsidianGlobalHistory } from "./modules/features/ModuleGlobalHistory.ts";
import { ModuleObsidianSettingsAsMarkdown } from "./modules/features/ModuleObsidianSettingAsMarkdown.ts";
import { ModuleInitializerFile } from "./modules/essential/ModuleInitializerFile.ts";
import { ModuleKeyValueDB } from "./modules/essential/ModuleKeyValueDB.ts";
import { ModulePouchDB } from "./modules/core/ModulePouchDB.ts";
import { ModuleReplicator } from "./modules/core/ModuleReplicator.ts";
import { ModuleReplicatorCouchDB } from "./modules/core/ModuleReplicatorCouchDB.ts";
import { ModuleReplicatorMinIO } from "./modules/core/ModuleReplicatorMinIO.ts";
import { ModuleTargetFilter } from "./modules/core/ModuleTargetFilter.ts";
import { ModulePeriodicProcess } from "./modules/core/ModulePeriodicProcess.ts";
import { ModuleRemoteGovernor } from "./modules/coreFeatures/ModuleRemoteGovernor.ts";
import { ModuleLocalDatabaseObsidian } from "./modules/core/ModuleLocalDatabaseObsidian.ts";
import { ModuleConflictChecker } from "./modules/coreFeatures/ModuleConflictChecker.ts";
import { ModuleResolvingMismatchedTweaks } from "./modules/coreFeatures/ModuleResolveMismatchedTweaks.ts";
import { ModuleIntegratedTest } from "./modules/extras/ModuleIntegratedTest.ts";
import { ModuleRebuilder } from "./modules/core/ModuleRebuilder.ts";
import { ModuleReplicateTest } from "./modules/extras/ModuleReplicateTest.ts";
import { ModuleLiveSyncMain } from "./modules/main/ModuleLiveSyncMain.ts";
import { ModuleExtraSyncObsidian } from "./modules/extraFeaturesObsidian/ModuleExtraSyncObsidian.ts";
import { LocalDatabaseMaintenance } from "./features/LocalDatabaseMainte/CmdLocalDatabaseMainte.ts";
import { P2PReplicator } from "./features/P2PSync/CmdP2PReplicator.ts";
import type { LiveSyncManagers } from "./lib/src/managers/LiveSyncManagers.ts";
import type { InjectableServiceHub } from "./lib/src/services/implements/injectable/InjectableServiceHub.ts";
import { ObsidianServiceHub } from "./modules/services/ObsidianServiceHub.ts";
import type { ServiceContext } from "./lib/src/services/base/ServiceBase.ts";
// import type { InjectableServiceHub } from "./lib/src/services/InjectableServices.ts";
import { ServiceRebuilder } from "@lib/serviceModules/Rebuilder.ts";
import type { IFileHandler } from "@lib/interfaces/FileHandler.ts";
import { ServiceDatabaseFileAccess } from "@/serviceModules/DatabaseFileAccess.ts";
import { ServiceFileAccessObsidian } from "@/serviceModules/ServiceFileAccessImpl.ts";
import { StorageAccessManager } from "@lib/managers/StorageProcessingManager.ts";
import { __$checkInstanceBinding } from "./lib/src/dev/checks.ts";
import { ServiceFileHandler } from "./serviceModules/FileHandler.ts";
import { FileAccessObsidian } from "./serviceModules/FileAccessObsidian.ts";
import { StorageEventManagerObsidian } from "./managers/StorageEventManagerObsidian.ts";
import { onLayoutReadyFeatures } from "./serviceFeatures/onLayoutReady.ts";
import type { ServiceModules } from "./types.ts";
export default class ObsidianLiveSyncPlugin
extends Plugin
@@ -84,16 +77,58 @@ export default class ObsidianLiveSyncPlugin
/**
* The service hub for managing all services.
*/
_services: InjectableServiceHub<ServiceContext> = new ObsidianServiceHub(this);
_services: InjectableServiceHub<ServiceContext> | undefined = undefined;
get services() {
if (!this._services) {
throw new Error("Services not initialised yet");
}
return this._services;
}
/**
* Bind functions to the service hub (for migration purpose).
*/
// bindFunctions = (this.serviceHub as ObsidianServiceHub).bindFunctions.bind(this.serviceHub);
// --> Module System
/**
* Service Modules
*/
protected _serviceModules: ServiceModules;
get serviceModules() {
return this._serviceModules;
}
/**
* addOns: Non-essential and graphically features
*/
addOns = [] as LiveSyncCommands[];
/**
* The modules of the plug-in. Modules are responsible for specific features or functionalities of the plug-in, such as file handling, conflict resolution, replication, etc.
*/
private modules = [
// Move to registerModules
] as (IObsidianModule | AbstractModule)[];
/**
* register an add-onn to the plug-in.
* Add-ons are features that are not essential to the core functionality of the plugin,
* @param addOn
*/
private _registerAddOn(addOn: LiveSyncCommands) {
this.addOns.push(addOn);
this.services.appLifecycle.onUnload.addHandler(() => Promise.resolve(addOn.onunload()).then(() => true));
}
private registerAddOns() {
this._registerAddOn(new ConfigSync(this));
this._registerAddOn(new HiddenFileSync(this));
this._registerAddOn(new LocalDatabaseMaintenance(this));
this._registerAddOn(new P2PReplicator(this));
}
/**
* Get an add-on by its class name. Returns undefined if not found.
* @param cls
* @returns
*/
getAddOn<T extends LiveSyncCommands>(cls: string) {
for (const addon of this.addOns) {
if (addon.constructor.name == cls) return addon as T;
@@ -101,58 +136,12 @@ export default class ObsidianLiveSyncPlugin
return undefined;
}
// Keep order to display the dialogue in order.
addOns = [
new ConfigSync(this),
new HiddenFileSync(this),
new LocalDatabaseMaintenance(this),
new P2PReplicator(this),
] as LiveSyncCommands[];
modules = [
new ModuleLiveSyncMain(this),
new ModuleExtraSyncObsidian(this, this),
// Only on Obsidian
new ModuleDatabaseFileAccess(this),
// Common
new ModulePouchDB(this),
new ModuleConflictChecker(this),
new ModuleLocalDatabaseObsidian(this),
new ModuleReplicatorMinIO(this),
new ModuleReplicatorCouchDB(this),
new ModuleReplicator(this),
new ModuleFileHandler(this),
new ModuleConflictResolver(this),
new ModuleRemoteGovernor(this),
new ModuleTargetFilter(this),
new ModulePeriodicProcess(this),
// Essential Modules
new ModuleKeyValueDB(this),
new ModuleInitializerFile(this),
new ModuleObsidianAPI(this, this),
new ModuleObsidianEvents(this, this),
new ModuleFileAccessObsidian(this, this),
new ModuleObsidianSettings(this),
new ModuleResolvingMismatchedTweaks(this),
new ModuleObsidianSettingsAsMarkdown(this),
new ModuleObsidianSettingDialogue(this, this),
new ModuleLog(this, this),
new ModuleObsidianMenu(this),
new ModuleRebuilder(this),
new ModuleSetupObsidian(this),
new ModuleObsidianDocumentHistory(this, this),
new ModuleMigration(this),
new ModuleRedFlag(this),
new ModuleInteractiveConflictResolver(this, this),
new ModuleObsidianGlobalHistory(this, this),
new ModuleCheckRemoteSize(this),
// Test and Dev Modules
new ModuleDev(this, this),
new ModuleReplicateTest(this, this),
new ModuleIntegratedTest(this, this),
new SetupManager(this),
] as (IObsidianModule | AbstractModule)[];
/**
* Get a module by its class. Throws an error if not found.
* Mostly used for getting SetupManager.
* @param constructor
* @returns
*/
getModule<T extends IObsidianModule>(constructor: new (...args: any[]) => T): T {
for (const module of this.modules) {
if (module.constructor === constructor) return module as T;
@@ -160,66 +149,301 @@ export default class ObsidianLiveSyncPlugin
throw new Error(`Module ${constructor} not found or not loaded.`);
}
settings!: ObsidianLiveSyncSettings;
localDatabase!: LiveSyncLocalDB;
managers!: LiveSyncManagers;
simpleStore!: SimpleStore<CheckPointInfo>;
replicator!: LiveSyncAbstractReplicator;
/**
* Register a module to the plug-in.
* @param module The module to register.
*/
private _registerModule(module: IObsidianModule) {
this.modules.push(module);
}
private registerModules() {
this._registerModule(new ModuleLiveSyncMain(this));
this._registerModule(new ModuleConflictChecker(this));
this._registerModule(new ModuleReplicatorMinIO(this));
this._registerModule(new ModuleReplicatorCouchDB(this));
this._registerModule(new ModuleReplicator(this));
this._registerModule(new ModuleConflictResolver(this));
this._registerModule(new ModuleTargetFilter(this));
this._registerModule(new ModulePeriodicProcess(this));
this._registerModule(new ModuleInitializerFile(this));
this._registerModule(new ModuleObsidianAPI(this, this));
this._registerModule(new ModuleObsidianEvents(this, this));
this._registerModule(new ModuleResolvingMismatchedTweaks(this));
this._registerModule(new ModuleObsidianSettingsAsMarkdown(this));
this._registerModule(new ModuleObsidianSettingDialogue(this, this));
this._registerModule(new ModuleLog(this, this));
this._registerModule(new ModuleObsidianMenu(this));
this._registerModule(new ModuleSetupObsidian(this));
this._registerModule(new ModuleObsidianDocumentHistory(this, this));
this._registerModule(new ModuleMigration(this));
this._registerModule(new ModuleRedFlag(this));
this._registerModule(new ModuleInteractiveConflictResolver(this, this));
this._registerModule(new ModuleObsidianGlobalHistory(this, this));
this._registerModule(new ModuleCheckRemoteSize(this));
// Test and Dev Modules
this._registerModule(new ModuleDev(this, this));
this._registerModule(new ModuleReplicateTest(this, this));
this._registerModule(new ModuleIntegratedTest(this, this));
this._registerModule(new SetupManager(this));
}
/**
* Bind module functions to services.
*/
private bindModuleFunctions() {
for (const module of this.modules) {
if (module instanceof AbstractModule) {
module.onBindFunction(this, this.services);
__$checkInstanceBinding(module); // Check if all functions are properly bound, and log warnings if not.
} else {
this.services.API.addLog(
`Module ${module.constructor.name} does not have onBindFunction, skipping binding.`,
LOG_LEVEL_INFO
);
}
}
}
/**
* @obsolete Use services.UI.confirm instead. The confirm function to show a confirmation dialog to the user.
*/
get confirm(): Confirm {
return this.services.UI.confirm;
}
storageAccess!: StorageAccess;
databaseFileAccess!: DatabaseFileAccess;
fileHandler!: ModuleFileHandler;
rebuilder!: Rebuilder;
kvDB!: KeyValueDatabase;
getDatabase(): PouchDB.Database<EntryDoc> {
return this.localDatabase.localDatabase;
/**
* @obsolete Use services.setting.currentSettings instead. The current settings of the plug-in.
*/
get settings() {
return this.services.setting.settings;
}
/**
* @obsolete Use services.setting.settings instead. Set the settings of the plug-in.
*/
set settings(value: ObsidianLiveSyncSettings) {
this.services.setting.settings = value;
}
/**
* @obsolete Use services.setting.currentSettings instead. Get the settings of the plug-in.
* @returns The current settings of the plug-in.
*/
getSettings(): ObsidianLiveSyncSettings {
return this.settings;
}
requestCount = reactiveSource(0);
responseCount = reactiveSource(0);
totalQueued = reactiveSource(0);
batched = reactiveSource(0);
processing = reactiveSource(0);
databaseQueueCount = reactiveSource(0);
storageApplyingCount = reactiveSource(0);
replicationResultCount = reactiveSource(0);
conflictProcessQueueCount = reactiveSource(0);
pendingFileEventCount = reactiveSource(0);
processingFileEventCount = reactiveSource(0);
_totalProcessingCount?: ReactiveValue<number>;
replicationStat = reactiveSource({
sent: 0,
arrived: 0,
maxPullSeq: 0,
maxPushSeq: 0,
lastSyncPullSeq: 0,
lastSyncPushSeq: 0,
syncStatus: "CLOSED" as DatabaseConnectingStatus,
});
private async _startUp() {
await this.services.appLifecycle.onLoad();
const onReady = this.services.appLifecycle.onReady.bind(this.services.appLifecycle);
this.app.workspace.onLayoutReady(onReady);
/**
* @obsolete Use services.database.localDatabase instead. The local database instance.
*/
get localDatabase() {
return this.services.database.localDatabase;
}
onload() {
void this._startUp();
/**
* @obsolete Use services.database.managers instead. The database managers, including entry manager, revision manager, etc.
*/
get managers() {
return this.services.database.managers;
}
/**
* @obsolete Use services.database.localDatabase instead. Get the PouchDB database instance. Note that this is not the same as the local database instance, which is a wrapper around the PouchDB database.
* @returns The PouchDB database instance.
*/
getDatabase(): PouchDB.Database<EntryDoc> {
return this.localDatabase.localDatabase;
}
/**
* @obsolete Use services.keyValueDB.simpleStore instead. A simple key-value store for storing non-file data, such as checkpoints, sync status, etc.
*/
get simpleStore() {
return this.services.keyValueDB.simpleStore as SimpleStore<CheckPointInfo>;
}
/**
* @obsolete Use services.replication.getActiveReplicator instead. Get the active replicator instance. Note that there can be multiple replicators, but only one can be active at a time.
*/
get replicator() {
return this.services.replicator.getActiveReplicator()!;
}
/**
* @obsolete Use services.keyValueDB.kvDB instead. Get the key-value database instance. This is used for storing large data that cannot be stored in the simple store, such as file metadata, etc.
*/
get kvDB() {
return this.services.keyValueDB.kvDB;
}
/// Modules which were relied on services
/**
* Storage Accessor for handling file operations.
* @obsolete Use serviceModules.storageAccess instead.
*/
get storageAccess(): StorageAccess {
return this.serviceModules.storageAccess;
}
/**
* Database File Accessor for handling file operations related to the database, such as exporting the database, importing from a file, etc.
* @obsolete Use serviceModules.databaseFileAccess instead.
*/
get databaseFileAccess(): DatabaseFileAccess {
return this.serviceModules.databaseFileAccess;
}
/**
* File Handler for handling file operations related to replication, such as resolving conflicts, applying changes from replication, etc.
* @obsolete Use serviceModules.fileHandler instead.
*/
get fileHandler(): IFileHandler {
return this.serviceModules.fileHandler;
}
/**
* Rebuilder for handling database rebuilding operations.
* @obsolete Use serviceModules.rebuilder instead.
*/
get rebuilder(): Rebuilder {
return this.serviceModules.rebuilder;
}
// requestCount = reactiveSource(0);
// responseCount = reactiveSource(0);
// totalQueued = reactiveSource(0);
// batched = reactiveSource(0);
// processing = reactiveSource(0);
// databaseQueueCount = reactiveSource(0);
// storageApplyingCount = reactiveSource(0);
// replicationResultCount = reactiveSource(0);
// pendingFileEventCount = reactiveSource(0);
// processingFileEventCount = reactiveSource(0);
// _totalProcessingCount?: ReactiveValue<number>;
// replicationStat = reactiveSource({
// sent: 0,
// arrived: 0,
// maxPullSeq: 0,
// maxPushSeq: 0,
// lastSyncPullSeq: 0,
// lastSyncPushSeq: 0,
// syncStatus: "CLOSED" as DatabaseConnectingStatus,
// });
private initialiseServices() {
this._services = new ObsidianServiceHub(this);
}
/**
* Initialise service modules.
*/
private initialiseServiceModules() {
const storageAccessManager = new StorageAccessManager();
// If we want to implement to the other platform, implement ObsidianXXXXXService.
const vaultAccess = new FileAccessObsidian(this.app, {
storageAccessManager: storageAccessManager,
vaultService: this.services.vault,
settingService: this.services.setting,
APIService: this.services.API,
});
const storageEventManager = new StorageEventManagerObsidian(this, this, {
fileProcessing: this.services.fileProcessing,
setting: this.services.setting,
vaultService: this.services.vault,
storageAccessManager: storageAccessManager,
APIService: this.services.API,
});
const storageAccess = new ServiceFileAccessObsidian({
API: this.services.API,
setting: this.services.setting,
fileProcessing: this.services.fileProcessing,
vault: this.services.vault,
appLifecycle: this.services.appLifecycle,
storageEventManager: storageEventManager,
storageAccessManager: storageAccessManager,
vaultAccess: vaultAccess,
});
const databaseFileAccess = new ServiceDatabaseFileAccess({
API: this.services.API,
database: this.services.database,
path: this.services.path,
storageAccess: storageAccess,
vault: this.services.vault,
});
const fileHandler = new ServiceFileHandler({
API: this.services.API,
databaseFileAccess: databaseFileAccess,
conflict: this.services.conflict,
setting: this.services.setting,
fileProcessing: this.services.fileProcessing,
vault: this.services.vault,
path: this.services.path,
replication: this.services.replication,
storageAccess: storageAccess,
});
const rebuilder = new ServiceRebuilder({
API: this.services.API,
database: this.services.database,
appLifecycle: this.services.appLifecycle,
setting: this.services.setting,
remote: this.services.remote,
databaseEvents: this.services.databaseEvents,
replication: this.services.replication,
replicator: this.services.replicator,
UI: this.services.UI,
vault: this.services.vault,
fileHandler: fileHandler,
storageAccess: storageAccess,
control: this.services.control,
});
return {
rebuilder,
fileHandler,
databaseFileAccess,
storageAccess,
};
}
/**
* @obsolete Use services.setting.saveSettingData instead. Save the settings to the disk. This is usually called after changing the settings in the code, to persist the changes.
*/
async saveSettings() {
await this.services.setting.saveSettingData();
}
onunload() {
return void this.services.appLifecycle.onAppUnload();
/**
* Initialise ServiceFeatures.
* (Please refer `serviceFeatures` for more details)
*/
initialiseServiceFeatures() {
for (const feature of onLayoutReadyFeatures) {
const curriedFeature = () => feature(this);
this.services.appLifecycle.onLayoutReady.addHandler(curriedFeature);
}
}
constructor(app: App, manifest: PluginManifest) {
super(app, manifest);
this.initialiseServices();
this.registerModules();
this.registerAddOns();
this._serviceModules = this.initialiseServiceModules();
this.initialiseServiceFeatures();
this.bindModuleFunctions();
}
private async _startUp() {
if (!(await this.services.control.onLoad())) return;
const onReady = this.services.control.onReady.bind(this.services.control);
this.app.workspace.onLayoutReady(onReady);
}
override onload() {
void this._startUp();
}
override onunload() {
return void this.services.control.onUnload();
}
// <-- Plug-in's overrideable functions
}
// For now,

210
src/managers/StorageEventManagerObsidian.ts Normal file
View File

@@ -0,0 +1,210 @@
import type { FileEventItem } from "@/common/types";
import { HiddenFileSync } from "@/features/HiddenFileSync/CmdHiddenFileSync";
import type { FilePath, UXFileInfoStub, UXFolderInfo, UXInternalFileInfoStub } from "@lib/common/types";
import type { FileEvent } from "@lib/interfaces/StorageEventManager";
import { TFile, type TAbstractFile, TFolder } from "@/deps";
import { LOG_LEVEL_DEBUG } from "octagonal-wheels/common/logger";
import type ObsidianLiveSyncPlugin from "@/main";
import type { LiveSyncCore } from "@/main";
import {
StorageEventManagerBase,
type FileEventItemSentinel,
type StorageEventManagerBaseDependencies,
} from "@lib/managers/StorageEventManager";
import { InternalFileToUXFileInfoStub, TFileToUXFileInfoStub } from "@/modules/coreObsidian/storageLib/utilObsidian";
export class StorageEventManagerObsidian extends StorageEventManagerBase {
plugin: ObsidianLiveSyncPlugin;
core: LiveSyncCore;
// Necessary evil.
cmdHiddenFileSync: HiddenFileSync;
override isFile(file: UXFileInfoStub | UXInternalFileInfoStub | UXFolderInfo | TFile): boolean {
if (file instanceof TFile) {
return true;
}
if (super.isFile(file)) {
return true;
}
return !file.isFolder;
}
override isFolder(file: UXFileInfoStub | UXInternalFileInfoStub | UXFolderInfo | TFolder): boolean {
if (file instanceof TFolder) {
return true;
}
if (super.isFolder(file)) {
return true;
}
return !!file.isFolder;
}
constructor(plugin: ObsidianLiveSyncPlugin, core: LiveSyncCore, dependencies: StorageEventManagerBaseDependencies) {
super(dependencies);
this.plugin = plugin;
this.core = core;
this.cmdHiddenFileSync = this.plugin.getAddOn(HiddenFileSync.name) as HiddenFileSync;
}
async beginWatch() {
await this.snapShotRestored;
const plugin = this.plugin;
this.watchVaultChange = this.watchVaultChange.bind(this);
this.watchVaultCreate = this.watchVaultCreate.bind(this);
this.watchVaultDelete = this.watchVaultDelete.bind(this);
this.watchVaultRename = this.watchVaultRename.bind(this);
this.watchVaultRawEvents = this.watchVaultRawEvents.bind(this);
this.watchEditorChange = this.watchEditorChange.bind(this);
plugin.registerEvent(plugin.app.vault.on("modify", this.watchVaultChange));
plugin.registerEvent(plugin.app.vault.on("delete", this.watchVaultDelete));
plugin.registerEvent(plugin.app.vault.on("rename", this.watchVaultRename));
plugin.registerEvent(plugin.app.vault.on("create", this.watchVaultCreate));
//@ts-ignore : Internal API
plugin.registerEvent(plugin.app.vault.on("raw", this.watchVaultRawEvents));
plugin.registerEvent(plugin.app.workspace.on("editor-change", this.watchEditorChange));
}
watchEditorChange(editor: any, info: any) {
if (!("path" in info)) {
return;
}
if (!this.shouldBatchSave) {
return;
}
const file = info?.file as TFile;
if (!file) return;
if (this.storageAccess.isFileProcessing(file.path as FilePath)) {
// this._log(`Editor change skipped because the file is being processed: ${file.path}`, LOG_LEVEL_VERBOSE);
return;
}
if (!this.isWaiting(file.path as FilePath)) {
return;
}
const data = info?.data as string;
const fi: FileEvent = {
type: "CHANGED",
file: TFileToUXFileInfoStub(file),
cachedData: data,
};
void this.appendQueue([fi]);
}
watchVaultCreate(file: TAbstractFile, ctx?: any) {
if (file instanceof TFolder) return;
if (this.storageAccess.isFileProcessing(file.path as FilePath)) {
// this._log(`File create skipped because the file is being processed: ${file.path}`, LOG_LEVEL_VERBOSE);
return;
}
const fileInfo = TFileToUXFileInfoStub(file);
void this.appendQueue([{ type: "CREATE", file: fileInfo }], ctx);
}
watchVaultChange(file: TAbstractFile, ctx?: any) {
if (file instanceof TFolder) return;
if (this.storageAccess.isFileProcessing(file.path as FilePath)) {
// this._log(`File change skipped because the file is being processed: ${file.path}`, LOG_LEVEL_VERBOSE);
return;
}
const fileInfo = TFileToUXFileInfoStub(file);
void this.appendQueue([{ type: "CHANGED", file: fileInfo }], ctx);
}
watchVaultDelete(file: TAbstractFile, ctx?: any) {
if (file instanceof TFolder) return;
if (this.storageAccess.isFileProcessing(file.path as FilePath)) {
// this._log(`File delete skipped because the file is being processed: ${file.path}`, LOG_LEVEL_VERBOSE);
return;
}
const fileInfo = TFileToUXFileInfoStub(file, true);
void this.appendQueue([{ type: "DELETE", file: fileInfo }], ctx);
}
watchVaultRename(file: TAbstractFile, oldFile: string, ctx?: any) {
// vault Rename will not be raised for self-events (Self-hosted LiveSync will not handle 'rename').
if (file instanceof TFile) {
const fileInfo = TFileToUXFileInfoStub(file);
void this.appendQueue(
[
{
type: "DELETE",
file: {
path: oldFile as FilePath,
name: file.name,
stat: {
mtime: file.stat.mtime,
ctime: file.stat.ctime,
size: file.stat.size,
type: "file",
},
deleted: true,
},
skipBatchWait: true,
},
{ type: "CREATE", file: fileInfo, skipBatchWait: true },
],
ctx
);
}
}
// Watch raw events (Internal API)
watchVaultRawEvents(path: FilePath) {
if (this.storageAccess.isFileProcessing(path)) {
// this._log(`Raw file event skipped because the file is being processed: ${path}`, LOG_LEVEL_VERBOSE);
return;
}
// Only for internal files.
if (!this.settings) return;
// if (this.plugin.settings.useIgnoreFiles && this.plugin.ignoreFiles.some(e => path.endsWith(e.trim()))) {
if (this.settings.useIgnoreFiles) {
// If it is one of ignore files, refresh the cached one.
// (Calling$$isTargetFile will refresh the cache)
void this.vaultService.isTargetFile(path).then(() => this._watchVaultRawEvents(path));
} else {
void this._watchVaultRawEvents(path);
}
}
async _watchVaultRawEvents(path: FilePath) {
if (!this.settings.syncInternalFiles && !this.settings.usePluginSync) return;
if (!this.settings.watchInternalFileChanges) return;
if (!path.startsWith(this.plugin.app.vault.configDir)) return;
if (path.endsWith("/")) {
// Folder
return;
}
const isTargetFile = await this.cmdHiddenFileSync.isTargetFile(path);
if (!isTargetFile) return;
void this.appendQueue(
[
{
type: "INTERNAL",
file: InternalFileToUXFileInfoStub(path),
skipBatchWait: true, // Internal files should be processed immediately.
},
],
null
);
}
async _saveSnapshot(snapshot: (FileEventItem | FileEventItemSentinel)[]) {
await this.core.kvDB.set("storage-event-manager-snapshot", snapshot);
this._log(`Storage operation snapshot saved: ${snapshot.length} items`, LOG_LEVEL_DEBUG);
}
async _loadSnapshot() {
const snapShot = await this.core.kvDB.get<(FileEventItem | FileEventItemSentinel)[]>(
"storage-event-manager-snapshot"
);
return snapShot;
}
updateStatus() {
const allFileEventItems = this.bufferedQueuedItems.filter((e): e is FileEventItem => "args" in e);
const allItems = allFileEventItems.filter((e) => !e.cancelled);
const totalItems = allItems.length + this.concurrentProcessing.waiting;
const processing = this.processingCount;
const batchedCount = this._waitingMap.size;
this.fileProcessing.batched.value = batchedCount;
this.fileProcessing.processing.value = processing;
this.fileProcessing.totalQueued.value = totalItems + batchedCount + processing;
}
}

24
src/modules/AbstractModule.ts
View File

@@ -1,17 +1,17 @@
import { LOG_LEVEL_INFO, LOG_LEVEL_NOTICE, LOG_LEVEL_VERBOSE, Logger } from "octagonal-wheels/common/logger";
import type { AnyEntry, FilePathWithPrefix, LOG_LEVEL } from "@lib/common/types";
import { LOG_LEVEL_NOTICE, LOG_LEVEL_VERBOSE, Logger } from "octagonal-wheels/common/logger";
import type { AnyEntry, FilePathWithPrefix } from "@lib/common/types";
import type { LiveSyncCore } from "@/main";
import { __$checkInstanceBinding } from "@lib/dev/checks";
import { stripAllPrefixes } from "@lib/string_and_binary/path";
import { createInstanceLogFunction } from "@lib/services/lib/logUtils";
export abstract class AbstractModule {
_log = (msg: any, level: LOG_LEVEL = LOG_LEVEL_INFO, key?: string) => {
if (typeof msg === "string" && level !== LOG_LEVEL_NOTICE) {
msg = `[${this.constructor.name}]\u{200A} ${msg}`;
_log = createInstanceLogFunction(this.constructor.name, this.services.API);
get services() {
if (!this.core._services) {
throw new Error("Services are not ready yet.");
}
// console.log(msg);
Logger(msg, level, key);
};
return this.core._services;
}
addCommand = this.services.API.addCommand.bind(this.services.API);
registerView = this.services.API.registerWindow.bind(this.services.API);
@@ -40,9 +40,7 @@ export abstract class AbstractModule {
// Override if needed.
}
constructor(public core: LiveSyncCore) {
this.onBindFunction(core, core.services);
Logger(`[${this.constructor.name}] Loaded`, LOG_LEVEL_VERBOSE);
__$checkInstanceBinding(this);
}
saveSettings = this.core.saveSettings.bind(this.core);
@@ -73,10 +71,6 @@ export abstract class AbstractModule {
return this.testDone();
}
get services() {
return this.core._services;
}
isMainReady() {
return this.services.appLifecycle.isReady();
}

2
src/modules/AbstractObsidianModule.ts
View File

@@ -16,7 +16,7 @@ export abstract class AbstractObsidianModule extends AbstractModule {
constructor(
public plugin: ObsidianLiveSyncPlugin,
public core: LiveSyncCore
core: LiveSyncCore
) {
super(core);
}

352
src/modules/core/ModuleDatabaseFileAccess.ts
View File

@@ -1,352 +0,0 @@
import { LOG_LEVEL_VERBOSE } from "octagonal-wheels/common/logger";
import { EVENT_FILE_SAVED, eventHub } from "../../common/events";
import {
getDatabasePathFromUXFileInfo,
getStoragePathFromUXFileInfo,
isInternalMetadata,
markChangesAreSame,
} from "../../common/utils";
import type {
UXFileInfoStub,
FilePathWithPrefix,
UXFileInfo,
MetaEntry,
LoadedEntry,
FilePath,
SavingEntry,
DocumentID,
} from "../../lib/src/common/types";
import type { DatabaseFileAccess } from "../interfaces/DatabaseFileAccess";
import { isPlainText, shouldBeIgnored, stripAllPrefixes } from "../../lib/src/string_and_binary/path";
import {
createBlob,
createTextBlob,
delay,
determineTypeFromBlob,
isDocContentSame,
readContent,
} from "../../lib/src/common/utils";
import { serialized } from "octagonal-wheels/concurrency/lock";
import { AbstractModule } from "../AbstractModule.ts";
import { ICHeader } from "../../common/types.ts";
import type { LiveSyncCore } from "../../main.ts";
export class ModuleDatabaseFileAccess extends AbstractModule implements DatabaseFileAccess {
private _everyOnload(): Promise<boolean> {
this.core.databaseFileAccess = this;
return Promise.resolve(true);
}
private async _everyModuleTest(): Promise<boolean> {
if (!this.settings.enableDebugTools) return Promise.resolve(true);
const testString = "Lorem ipsum dolor sit amet, consectetur adipiscing elit. Nullam nec purus nec nunc";
// Before test, we need to delete completely.
const conflicts = await this.getConflictedRevs("autoTest.md" as FilePathWithPrefix);
for (const rev of conflicts) {
await this.delete("autoTest.md" as FilePathWithPrefix, rev);
}
await this.delete("autoTest.md" as FilePathWithPrefix);
// OK, begin!
await this._test(
"storeContent",
async () => await this.storeContent("autoTest.md" as FilePathWithPrefix, testString)
);
// For test, we need to clear the caches.
this.localDatabase.clearCaches();
await this._test("readContent", async () => {
const content = await this.fetch("autoTest.md" as FilePathWithPrefix);
if (!content) return "File not found";
if (content.deleted) return "File is deleted";
return (await content.body.text()) == testString
? true
: `Content is not same ${await content.body.text()}`;
});
await this._test("delete", async () => await this.delete("autoTest.md" as FilePathWithPrefix));
await this._test("read deleted content", async () => {
const content = await this.fetch("autoTest.md" as FilePathWithPrefix);
if (!content) return true;
if (content.deleted) return true;
return `Still exist !:${await content.body.text()},${JSON.stringify(content, undefined, 2)}`;
});
await delay(100);
return this.testDone();
}
async checkIsTargetFile(file: UXFileInfoStub | FilePathWithPrefix): Promise<boolean> {
const path = getStoragePathFromUXFileInfo(file);
if (!(await this.services.vault.isTargetFile(path))) {
this._log(`File is not target: ${path}`, LOG_LEVEL_VERBOSE);
return false;
}
if (shouldBeIgnored(path)) {
this._log(`File should be ignored: ${path}`, LOG_LEVEL_VERBOSE);
return false;
}
return true;
}
async delete(file: UXFileInfoStub | FilePathWithPrefix, rev?: string): Promise<boolean> {
if (!(await this.checkIsTargetFile(file))) {
return true;
}
const fullPath = getDatabasePathFromUXFileInfo(file);
try {
this._log(`deleteDB By path:${fullPath}`);
return await this.deleteFromDBbyPath(fullPath, rev);
} catch (ex) {
this._log(`Failed to delete ${fullPath}`);
this._log(ex, LOG_LEVEL_VERBOSE);
return false;
}
}
async createChunks(file: UXFileInfo, force: boolean = false, skipCheck?: boolean): Promise<boolean> {
return await this.__store(file, force, skipCheck, true);
}
async store(file: UXFileInfo, force: boolean = false, skipCheck?: boolean): Promise<boolean> {
return await this.__store(file, force, skipCheck, false);
}
async storeContent(path: FilePathWithPrefix, content: string): Promise<boolean> {
const blob = createTextBlob(content);
const bytes = (await blob.arrayBuffer()).byteLength;
const isInternal = path.startsWith(".") ? true : undefined;
const dummyUXFileInfo: UXFileInfo = {
name: path.split("/").pop() as string,
path: path,
stat: {
size: bytes,
ctime: Date.now(),
mtime: Date.now(),
type: "file",
},
body: blob,
isInternal,
};
return await this.__store(dummyUXFileInfo, true, false, false);
}
private async __store(
file: UXFileInfo,
force: boolean = false,
skipCheck?: boolean,
onlyChunks?: boolean
): Promise<boolean> {
if (!skipCheck) {
if (!(await this.checkIsTargetFile(file))) {
return true;
}
}
if (!file) {
this._log("File seems bad", LOG_LEVEL_VERBOSE);
return false;
}
// const path = getPathFromUXFileInfo(file);
const isPlain = isPlainText(file.name);
const possiblyLarge = !isPlain;
const content = file.body;
const datatype = determineTypeFromBlob(content);
const idPrefix = file.isInternal ? ICHeader : "";
const fullPath = getStoragePathFromUXFileInfo(file);
const fullPathOnDB = getDatabasePathFromUXFileInfo(file);
if (possiblyLarge) this._log(`Processing: ${fullPath}`, LOG_LEVEL_VERBOSE);
// if (isInternalMetadata(fullPath)) {
// this._log(`Internal file: ${fullPath}`, LOG_LEVEL_VERBOSE);
// return false;
// }
if (file.isInternal) {
if (file.deleted) {
file.stat = {
size: 0,
ctime: Date.now(),
mtime: Date.now(),
type: "file",
};
} else if (file.stat == undefined) {
const stat = await this.core.storageAccess.statHidden(file.path);
if (!stat) {
// We stored actually deleted or not since here, so this is an unexpected case. we should raise an error.
this._log(`Internal file not found: ${fullPath}`, LOG_LEVEL_VERBOSE);
return false;
}
file.stat = stat;
}
}
const idMain = await this.services.path.path2id(fullPath);
const id = (idPrefix + idMain) as DocumentID;
const d: SavingEntry = {
_id: id,
path: fullPathOnDB,
data: content,
ctime: file.stat.ctime,
mtime: file.stat.mtime,
size: file.stat.size,
children: [],
datatype: datatype,
type: datatype,
eden: {},
};
//upsert should locked
const msg = `STORAGE -> DB (${datatype}) `;
const isNotChanged = await serialized("file-" + fullPath, async () => {
if (force) {
this._log(msg + "Force writing " + fullPath, LOG_LEVEL_VERBOSE);
return false;
}
// Commented out temporarily: this checks that the file was made ourself.
// if (this.core.storageAccess.recentlyTouched(file)) {
// return true;
// }
try {
const old = await this.localDatabase.getDBEntry(d.path, undefined, false, true, false);
if (old !== false) {
const oldData = { data: old.data, deleted: old._deleted || old.deleted };
const newData = { data: d.data, deleted: d._deleted || d.deleted };
if (oldData.deleted != newData.deleted) return false;
if (!(await isDocContentSame(old.data, newData.data))) return false;
this._log(
msg + "Skipped (not changed) " + fullPath + (d._deleted || d.deleted ? " (deleted)" : ""),
LOG_LEVEL_VERBOSE
);
markChangesAreSame(old, d.mtime, old.mtime);
return true;
// d._rev = old._rev;
}
} catch (ex) {
this._log(
msg +
"Error, Could not check the diff for the old one." +
(force ? "force writing." : "") +
fullPath +
(d._deleted || d.deleted ? " (deleted)" : ""),
LOG_LEVEL_VERBOSE
);
this._log(ex, LOG_LEVEL_VERBOSE);
return !force;
}
return false;
});
if (isNotChanged) {
this._log(msg + " Skip " + fullPath, LOG_LEVEL_VERBOSE);
return true;
}
const ret = await this.localDatabase.putDBEntry(d, onlyChunks);
if (ret !== false) {
this._log(msg + fullPath);
eventHub.emitEvent(EVENT_FILE_SAVED);
}
return ret != false;
}
async getConflictedRevs(file: UXFileInfoStub | FilePathWithPrefix): Promise<string[]> {
if (!(await this.checkIsTargetFile(file))) {
return [];
}
const filename = getDatabasePathFromUXFileInfo(file);
const doc = await this.localDatabase.getDBEntryMeta(filename, { conflicts: true }, true);
if (doc === false) {
return [];
}
return doc._conflicts || [];
}
async fetch(
file: UXFileInfoStub | FilePathWithPrefix,
rev?: string,
waitForReady?: boolean,
skipCheck = false
): Promise<UXFileInfo | false> {
if (skipCheck && !(await this.checkIsTargetFile(file))) {
return false;
}
const entry = await this.fetchEntry(file, rev, waitForReady, true);
if (entry === false) {
return false;
}
const data = createBlob(readContent(entry));
const path = stripAllPrefixes(entry.path);
const fileInfo: UXFileInfo = {
name: path.split("/").pop() as string,
path: path,
stat: {
size: entry.size,
ctime: entry.ctime,
mtime: entry.mtime,
type: "file",
},
body: data,
deleted: entry.deleted || entry._deleted,
};
if (isInternalMetadata(entry.path)) {
fileInfo.isInternal = true;
}
return fileInfo;
}
async fetchEntryMeta(
file: UXFileInfoStub | FilePathWithPrefix,
rev?: string,
skipCheck = false
): Promise<MetaEntry | false> {
const dbFileName = getDatabasePathFromUXFileInfo(file);
if (skipCheck && !(await this.checkIsTargetFile(file))) {
return false;
}
const doc = await this.localDatabase.getDBEntryMeta(dbFileName, rev ? { rev: rev } : undefined, true);
if (doc === false) {
return false;
}
return doc as MetaEntry;
}
async fetchEntryFromMeta(
meta: MetaEntry,
waitForReady: boolean = true,
skipCheck = false
): Promise<LoadedEntry | false> {
if (skipCheck && !(await this.checkIsTargetFile(meta.path))) {
return false;
}
const doc = await this.localDatabase.getDBEntryFromMeta(meta as LoadedEntry, false, waitForReady);
if (doc === false) {
return false;
}
return doc;
}
async fetchEntry(
file: UXFileInfoStub | FilePathWithPrefix,
rev?: string,
waitForReady: boolean = true,
skipCheck = false
): Promise<LoadedEntry | false> {
if (skipCheck && !(await this.checkIsTargetFile(file))) {
return false;
}
const entry = await this.fetchEntryMeta(file, rev, true);
if (entry === false) {
return false;
}
const doc = await this.fetchEntryFromMeta(entry, waitForReady, true);
return doc;
}
async deleteFromDBbyPath(fullPath: FilePath | FilePathWithPrefix, rev?: string): Promise<boolean> {
if (!(await this.checkIsTargetFile(fullPath))) {
this._log(`storeFromStorage: File is not target: ${fullPath}`);
return true;
}
const opt = rev ? { rev: rev } : undefined;
const ret = await this.localDatabase.deleteDBEntry(fullPath, opt);
eventHub.emitEvent(EVENT_FILE_SAVED);
return ret;
}
onBindFunction(core: LiveSyncCore, services: typeof core.services): void {
services.appLifecycle.onLoaded.addHandler(this._everyOnload.bind(this));
services.test.test.addHandler(this._everyModuleTest.bind(this));
}
}

436
src/modules/core/ModuleFileHandler.ts
View File

@@ -1,436 +0,0 @@
import { LOG_LEVEL_INFO, LOG_LEVEL_NOTICE, LOG_LEVEL_VERBOSE } from "octagonal-wheels/common/logger";
import { serialized } from "octagonal-wheels/concurrency/lock";
import type { FileEventItem } from "../../common/types";
import type {
FilePath,
FilePathWithPrefix,
MetaEntry,
UXFileInfo,
UXFileInfoStub,
UXInternalFileInfoStub,
} from "../../lib/src/common/types";
import { AbstractModule } from "../AbstractModule.ts";
import { compareFileFreshness, EVEN, getStoragePathFromUXFileInfo, markChangesAreSame } from "../../common/utils";
import { getDocDataAsArray, isDocContentSame, readAsBlob, readContent } from "../../lib/src/common/utils";
import { shouldBeIgnored } from "../../lib/src/string_and_binary/path";
import { Semaphore } from "octagonal-wheels/concurrency/semaphore";
import { eventHub } from "../../common/events.ts";
import type { LiveSyncCore } from "../../main.ts";
export class ModuleFileHandler extends AbstractModule {
get db() {
return this.core.databaseFileAccess;
}
get storage() {
return this.core.storageAccess;
}
_everyOnloadStart(): Promise<boolean> {
this.core.fileHandler = this;
return Promise.resolve(true);
}
async readFileFromStub(file: UXFileInfoStub | UXFileInfo) {
if ("body" in file && file.body) {
return file;
}
const readFile = await this.storage.readStubContent(file);
if (!readFile) {
throw new Error(`File ${file.path} is not exist on the storage`);
}
return readFile;
}
async storeFileToDB(
info: UXFileInfoStub | UXFileInfo | UXInternalFileInfoStub | FilePathWithPrefix,
force: boolean = false,
onlyChunks: boolean = false
): Promise<boolean> {
const file = typeof info === "string" ? this.storage.getFileStub(info) : info;
if (file == null) {
this._log(`File ${info} is not exist on the storage`, LOG_LEVEL_VERBOSE);
return false;
}
// const file = item.args.file;
if (file.isInternal) {
this._log(
`Internal file ${file.path} is not allowed to be processed on processFileEvent`,
LOG_LEVEL_VERBOSE
);
return false;
}
// First, check the file on the database
const entry = await this.db.fetchEntry(file, undefined, true, true);
if (!entry || entry.deleted || entry._deleted) {
// If the file is not exist on the database, then it should be created.
const readFile = await this.readFileFromStub(file);
if (!onlyChunks) {
return await this.db.store(readFile);
} else {
return await this.db.createChunks(readFile, false, true);
}
}
// entry is exist on the database, check the difference between the file and the entry.
let shouldApplied = false;
if (!force && !onlyChunks) {
// 1. if the time stamp is far different, then it should be updated.
// Note: This checks only the mtime with the resolution reduced to 2 seconds.
// 2 seconds it for the ZIP file's mtime. If not, we cannot backup the vault as the ZIP file.
// This is hardcoded on `compareMtime` of `src/common/utils.ts`.
if (compareFileFreshness(file, entry) !== EVEN) {
shouldApplied = true;
}
// 2. if not, the content should be checked.
let readFile: UXFileInfo | undefined = undefined;
if (!shouldApplied) {
readFile = await this.readFileFromStub(file);
if (!readFile) {
this._log(`File ${file.path} is not exist on the storage`, LOG_LEVEL_NOTICE);
return false;
}
if (await isDocContentSame(getDocDataAsArray(entry.data), readFile.body)) {
// Timestamp is different but the content is same. therefore, two timestamps should be handled as same.
// So, mark the changes are same.
markChangesAreSame(readFile, readFile.stat.mtime, entry.mtime);
} else {
shouldApplied = true;
}
}
if (!shouldApplied) {
this._log(`File ${file.path} is not changed`, LOG_LEVEL_VERBOSE);
return true;
}
if (!readFile) readFile = await this.readFileFromStub(file);
// If the file is changed, then the file should be stored.
if (onlyChunks) {
return await this.db.createChunks(readFile, false, true);
} else {
return await this.db.store(readFile, false, true);
}
} else {
// If force is true, then it should be updated.
const readFile = await this.readFileFromStub(file);
if (onlyChunks) {
return await this.db.createChunks(readFile, true, true);
} else {
return await this.db.store(readFile, true, true);
}
}
}
async deleteFileFromDB(info: UXFileInfoStub | UXInternalFileInfoStub | FilePath): Promise<boolean> {
const file = typeof info === "string" ? this.storage.getFileStub(info) : info;
if (file == null) {
this._log(`File ${info} is not exist on the storage`, LOG_LEVEL_VERBOSE);
return false;
}
// const file = item.args.file;
if (file.isInternal) {
this._log(
`Internal file ${file.path} is not allowed to be processed on processFileEvent`,
LOG_LEVEL_VERBOSE
);
return false;
}
// First, check the file on the database
const entry = await this.db.fetchEntry(file, undefined, true, true);
if (!entry || entry.deleted || entry._deleted) {
this._log(`File ${file.path} is not exist or already deleted on the database`, LOG_LEVEL_VERBOSE);
return false;
}
// Check the file is already conflicted. if so, only the conflicted one should be deleted.
const conflictedRevs = await this.db.getConflictedRevs(file);
if (conflictedRevs.length > 0) {
// If conflicted, then it should be deleted. entry._rev should be own file's rev.
// TODO: I BELIEVED SO. BUT I NOTICED THAT I AN NOT SURE. I SHOULD CHECK THIS.
// ANYWAY, I SHOULD DELETE THE FILE. ACTUALLY WE SIMPLY DELETED THE FILE UNTIL PREVIOUS VERSIONS.
return await this.db.delete(file, entry._rev);
}
// Otherwise, the file should be deleted simply. This is the previous behaviour.
return await this.db.delete(file);
}
async deleteRevisionFromDB(
info: UXFileInfoStub | FilePath | FilePathWithPrefix,
rev: string
): Promise<boolean | undefined> {
//TODO: Possibly check the conflicting.
return await this.db.delete(info, rev);
}
async resolveConflictedByDeletingRevision(
info: UXFileInfoStub | FilePath,
rev: string
): Promise<boolean | undefined> {
const path = getStoragePathFromUXFileInfo(info);
if (!(await this.deleteRevisionFromDB(info, rev))) {
this._log(`Failed to delete the conflicted revision ${rev} of ${path}`, LOG_LEVEL_VERBOSE);
return false;
}
if (!(await this.dbToStorageWithSpecificRev(info, rev, true))) {
this._log(`Failed to apply the resolved revision ${rev} of ${path} to the storage`, LOG_LEVEL_VERBOSE);
return false;
}
}
async dbToStorageWithSpecificRev(
info: UXFileInfoStub | UXFileInfo | FilePath | null,
rev: string,
force?: boolean
): Promise<boolean> {
const file = typeof info === "string" ? this.storage.getFileStub(info) : info;
if (file == null) {
this._log(`File ${info} is not exist on the storage`, LOG_LEVEL_VERBOSE);
return false;
}
const docEntry = await this.db.fetchEntryMeta(file, rev, true);
if (!docEntry) {
this._log(`File ${file.path} is not exist on the database`, LOG_LEVEL_VERBOSE);
return false;
}
return await this.dbToStorage(docEntry, file, force);
}
async dbToStorage(
entryInfo: MetaEntry | FilePathWithPrefix,
info: UXFileInfoStub | UXFileInfo | FilePath | null,
force?: boolean
): Promise<boolean> {
const file = typeof info === "string" ? this.storage.getFileStub(info) : info;
const mode = file == null ? "create" : "modify";
const pathFromEntryInfo = typeof entryInfo === "string" ? entryInfo : this.getPath(entryInfo);
const docEntry = await this.db.fetchEntryMeta(pathFromEntryInfo, undefined, true);
if (!docEntry) {
this._log(`File ${pathFromEntryInfo} is not exist on the database`, LOG_LEVEL_VERBOSE);
return false;
}
const path = this.getPath(docEntry);
// 1. Check if it already conflicted.
const revs = await this.db.getConflictedRevs(path);
if (revs.length > 0) {
// Some conflicts are exist.
if (this.settings.writeDocumentsIfConflicted) {
// If configured to write the document even if conflicted, then it should be written.
// NO OP
} else {
// If not, then it should be checked. and will be processed later (i.e., after the conflict is resolved).
await this.services.conflict.queueCheckForIfOpen(path);
return true;
}
}
// 2. Check if the file is already exist on the storage.
const existDoc = this.storage.getStub(path);
if (existDoc && existDoc.isFolder) {
this._log(`Folder ${path} is already exist on the storage as a folder`, LOG_LEVEL_VERBOSE);
// We can do nothing, and other modules should also nothing to do.
return true;
}
// Check existence of both file and docEntry.
const existOnDB = !(docEntry._deleted || docEntry.deleted || false);
const existOnStorage = existDoc != null;
if (!existOnDB && !existOnStorage) {
this._log(`File ${path} seems to be deleted, but already not on storage`, LOG_LEVEL_VERBOSE);
return true;
}
if (!existOnDB && existOnStorage) {
// Deletion has been Transferred. Storage files will be deleted.
// Note: If the folder becomes empty, the folder will be deleted if not configured to keep it.
// This behaviour is implemented on the `ModuleFileAccessObsidian`.
// And it does not care actually deleted.
await this.storage.deleteVaultItem(path);
return true;
}
// Okay, the file is exist on the database. Let's check the file is exist on the storage.
const docRead = await this.db.fetchEntryFromMeta(docEntry);
if (!docRead) {
this._log(`File ${path} is not exist on the database`, LOG_LEVEL_VERBOSE);
return false;
}
// If we want to process size mismatched files -- in case of having files created by some integrations, enable the toggle.
if (!this.settings.processSizeMismatchedFiles) {
// Check the file is not corrupted
// (Zero is a special case, may be created by some APIs and it might be acceptable).
if (docRead.size != 0 && docRead.size !== readAsBlob(docRead).size) {
this._log(
`File ${path} seems to be corrupted! Writing prevented. (${docRead.size} != ${readAsBlob(docRead).size})`,
LOG_LEVEL_NOTICE
);
return false;
}
}
const docData = readContent(docRead);
if (existOnStorage && !force) {
// The file is exist on the storage. Let's check the difference between the file and the entry.
// But, if force is true, then it should be updated.
// Ok, we have to compare.
let shouldApplied = false;
// 1. if the time stamp is far different, then it should be updated.
// Note: This checks only the mtime with the resolution reduced to 2 seconds.
// 2 seconds it for the ZIP file's mtime. If not, we cannot backup the vault as the ZIP file.
// This is hardcoded on `compareMtime` of `src/common/utils.ts`.
if (compareFileFreshness(existDoc, docEntry) !== EVEN) {
shouldApplied = true;
}
// 2. if not, the content should be checked.
if (!shouldApplied) {
const readFile = await this.readFileFromStub(existDoc);
if (await isDocContentSame(docData, readFile.body)) {
// The content is same. So, we do not need to update the file.
shouldApplied = false;
// Timestamp is different but the content is same. therefore, two timestamps should be handled as same.
// So, mark the changes are same.
markChangesAreSame(docRead, docRead.mtime, existDoc.stat.mtime);
} else {
shouldApplied = true;
}
}
if (!shouldApplied) {
this._log(`File ${docRead.path} is not changed`, LOG_LEVEL_VERBOSE);
return true;
}
// Let's apply the changes.
} else {
this._log(
`File ${docRead.path} ${existOnStorage ? "(new) " : ""} ${force ? " (forced)" : ""}`,
LOG_LEVEL_VERBOSE
);
}
await this.storage.ensureDir(path);
const ret = await this.storage.writeFileAuto(path, docData, { ctime: docRead.ctime, mtime: docRead.mtime });
await this.storage.touched(path);
this.storage.triggerFileEvent(mode, path);
return ret;
}
private async _anyHandlerProcessesFileEvent(item: FileEventItem): Promise<boolean> {
const eventItem = item.args;
const type = item.type;
const path = eventItem.file.path;
if (!(await this.services.vault.isTargetFile(path))) {
this._log(`File ${path} is not the target file`, LOG_LEVEL_VERBOSE);
return false;
}
if (shouldBeIgnored(path)) {
this._log(`File ${path} should be ignored`, LOG_LEVEL_VERBOSE);
return false;
}
const lockKey = `processFileEvent-${path}`;
return await serialized(lockKey, async () => {
switch (type) {
case "CREATE":
case "CHANGED":
return await this.storeFileToDB(item.args.file);
case "DELETE":
return await this.deleteFileFromDB(item.args.file);
case "INTERNAL":
// this should be handled on the other module.
return false;
default:
this._log(`Unsupported event type: ${type}`, LOG_LEVEL_VERBOSE);
return false;
}
});
}
async _anyProcessReplicatedDoc(entry: MetaEntry): Promise<boolean> {
return await serialized(entry.path, async () => {
if (!(await this.services.vault.isTargetFile(entry.path))) {
this._log(`File ${entry.path} is not the target file`, LOG_LEVEL_VERBOSE);
return false;
}
if (this.services.vault.isFileSizeTooLarge(entry.size)) {
this._log(`File ${entry.path} is too large (on database) to be processed`, LOG_LEVEL_VERBOSE);
return false;
}
if (shouldBeIgnored(entry.path)) {
this._log(`File ${entry.path} should be ignored`, LOG_LEVEL_VERBOSE);
return false;
}
const path = this.getPath(entry);
const targetFile = this.storage.getStub(this.getPathWithoutPrefix(entry));
if (targetFile && targetFile.isFolder) {
this._log(`${path} is already exist as the folder`);
// Nothing to do and other modules should also nothing to do.
return true;
} else {
if (targetFile && this.services.vault.isFileSizeTooLarge(targetFile.stat.size)) {
this._log(`File ${targetFile.path} is too large (on storage) to be processed`, LOG_LEVEL_VERBOSE);
return false;
}
this._log(
`Processing ${path} (${entry._id.substring(0, 8)} :${entry._rev?.substring(0, 5)}) : Started...`,
LOG_LEVEL_VERBOSE
);
// Before writing (or skipped ), merging dialogue should be cancelled.
eventHub.emitEvent("conflict-cancelled", path);
const ret = await this.dbToStorage(entry, targetFile);
this._log(`Processing ${path} (${entry._id.substring(0, 8)} :${entry._rev?.substring(0, 5)}) : Done`);
return ret;
}
});
}
async createAllChunks(showingNotice?: boolean): Promise<void> {
this._log("Collecting local files on the storage", LOG_LEVEL_VERBOSE);
const semaphore = Semaphore(10);
let processed = 0;
const filesStorageSrc = this.storage.getFiles();
const incProcessed = () => {
processed++;
if (processed % 25 == 0)
this._log(
`Creating missing chunks: ${processed} of ${total} files`,
showingNotice ? LOG_LEVEL_NOTICE : LOG_LEVEL_INFO,
"chunkCreation"
);
};
const total = filesStorageSrc.length;
const procAllChunks = filesStorageSrc.map(async (file) => {
if (!(await this.services.vault.isTargetFile(file))) {
incProcessed();
return true;
}
if (this.services.vault.isFileSizeTooLarge(file.stat.size)) {
incProcessed();
return true;
}
if (shouldBeIgnored(file.path)) {
incProcessed();
return true;
}
const release = await semaphore.acquire();
incProcessed();
try {
await this.storeFileToDB(file, false, true);
} catch (ex) {
this._log(ex, LOG_LEVEL_VERBOSE);
} finally {
release();
}
});
await Promise.all(procAllChunks);
this._log(
`Creating chunks Done: ${processed} of ${total} files`,
showingNotice ? LOG_LEVEL_NOTICE : LOG_LEVEL_INFO,
"chunkCreation"
);
}
onBindFunction(core: LiveSyncCore, services: typeof core.services): void {
services.appLifecycle.onInitialise.addHandler(this._everyOnloadStart.bind(this));
services.fileProcessing.processFileEvent.addHandler(this._anyHandlerProcessesFileEvent.bind(this));
services.replication.processSynchroniseResult.addHandler(this._anyProcessReplicatedDoc.bind(this));
}
}

46
src/modules/core/ModuleLocalDatabaseObsidian.ts
View File

@@ -1,46 +0,0 @@
import { $msg } from "../../lib/src/common/i18n";
import { LiveSyncLocalDB } from "../../lib/src/pouchdb/LiveSyncLocalDB.ts";
import { initializeStores } from "../../common/stores.ts";
import { AbstractModule } from "../AbstractModule.ts";
import { LiveSyncManagers } from "../../lib/src/managers/LiveSyncManagers.ts";
import type { LiveSyncCore } from "../../main.ts";
export class ModuleLocalDatabaseObsidian extends AbstractModule {
_everyOnloadStart(): Promise<boolean> {
return Promise.resolve(true);
}
private async _openDatabase(): Promise<boolean> {
if (this.localDatabase != null) {
await this.localDatabase.close();
}
const vaultName = this.services.vault.getVaultName();
this._log($msg("moduleLocalDatabase.logWaitingForReady"));
const getDB = () => this.core.localDatabase.localDatabase;
const getSettings = () => this.core.settings;
this.core.managers = new LiveSyncManagers({
get database() {
return getDB();
},
getActiveReplicator: () => this.core.replicator,
id2path: this.services.path.id2path.bind(this.services.path),
// path2id: this.core.$$path2id.bind(this.core),
path2id: this.services.path.path2id.bind(this.services.path),
get settings() {
return getSettings();
},
});
this.core.localDatabase = new LiveSyncLocalDB(vaultName, this.core);
initializeStores(vaultName);
return await this.localDatabase.initializeDatabase();
}
_isDatabaseReady(): boolean {
return this.localDatabase != null && this.localDatabase.isReady;
}
onBindFunction(core: LiveSyncCore, services: typeof core.services): void {
services.database.isDatabaseReady.setHandler(this._isDatabaseReady.bind(this));
services.appLifecycle.onInitialise.addHandler(this._everyOnloadStart.bind(this));
services.database.openDatabase.setHandler(this._openDatabase.bind(this));
}
}

2
src/modules/core/ModulePeriodicProcess.ts
View File

@@ -31,7 +31,7 @@ export class ModulePeriodicProcess extends AbstractModule {
return this.resumePeriodic();
}
onBindFunction(core: LiveSyncCore, services: typeof core.services): void {
override onBindFunction(core: LiveSyncCore, services: typeof core.services): void {
services.appLifecycle.onUnload.addHandler(this._allOnUnload.bind(this));
services.setting.onBeforeRealiseSetting.addHandler(this._everyBeforeRealizeSetting.bind(this));
services.setting.onSettingRealised.addHandler(this._everyAfterRealizeSetting.bind(this));

23
src/modules/core/ModulePouchDB.ts
View File

@@ -1,23 +0,0 @@
import { AbstractModule } from "../AbstractModule";
import { PouchDB } from "../../lib/src/pouchdb/pouchdb-browser";
import type { LiveSyncCore } from "../../main";
import { ExtraSuffixIndexedDB } from "../../lib/src/common/types";
export class ModulePouchDB extends AbstractModule {
_createPouchDBInstance<T extends object>(
name?: string,
options?: PouchDB.Configuration.DatabaseConfiguration
): PouchDB.Database<T> {
const optionPass = options ?? {};
if (this.settings.useIndexedDBAdapter) {
optionPass.adapter = "indexeddb";
//@ts-ignore :missing def
optionPass.purged_infos_limit = 1;
return new PouchDB(name + ExtraSuffixIndexedDB, optionPass);
}
return new PouchDB(name, optionPass);
}
onBindFunction(core: LiveSyncCore, services: typeof core.services): void {
services.database.createPouchDBInstance.setHandler(this._createPouchDBInstance.bind(this));
}
}

310
src/modules/core/ModuleRebuilder.ts
View File

@@ -1,310 +0,0 @@
import { delay } from "octagonal-wheels/promises";
import {
DEFAULT_SETTINGS,
FLAGMD_REDFLAG2_HR,
FLAGMD_REDFLAG3_HR,
LOG_LEVEL_NOTICE,
LOG_LEVEL_VERBOSE,
REMOTE_COUCHDB,
REMOTE_MINIO,
} from "../../lib/src/common/types.ts";
import { AbstractModule } from "../AbstractModule.ts";
import type { Rebuilder } from "../interfaces/DatabaseRebuilder.ts";
import type { LiveSyncCouchDBReplicator } from "../../lib/src/replication/couchdb/LiveSyncReplicator.ts";
import { fetchAllUsedChunks } from "@/lib/src/pouchdb/chunks.ts";
import { EVENT_DATABASE_REBUILT, eventHub } from "src/common/events.ts";
import type { LiveSyncCore } from "../../main.ts";
export class ModuleRebuilder extends AbstractModule implements Rebuilder {
private _everyOnload(): Promise<boolean> {
this.core.rebuilder = this;
return Promise.resolve(true);
}
async $performRebuildDB(
method: "localOnly" | "remoteOnly" | "rebuildBothByThisDevice" | "localOnlyWithChunks"
): Promise<void> {
if (method == "localOnly") {
await this.$fetchLocal();
}
if (method == "localOnlyWithChunks") {
await this.$fetchLocal(true);
}
if (method == "remoteOnly") {
await this.$rebuildRemote();
}
if (method == "rebuildBothByThisDevice") {
await this.$rebuildEverything();
}
}
async informOptionalFeatures() {
await this.core.services.UI.showMarkdownDialog(
"All optional features are disabled",
`Customisation Sync and Hidden File Sync will all be disabled.
Please enable them from the settings screen after setup is complete.`,
["OK"]
);
}
async askUsingOptionalFeature(opt: { enableFetch?: boolean; enableOverwrite?: boolean }) {
if (
(await this.core.confirm.askYesNoDialog(
"Do you want to enable extra features? If you are new to Self-hosted LiveSync, try the core feature first!",
{ title: "Enable extra features", defaultOption: "No", timeout: 15 }
)) == "yes"
) {
await this.services.setting.suggestOptionalFeatures(opt);
}
}
async rebuildRemote() {
await this.services.setting.suspendExtraSync();
this.core.settings.isConfigured = true;
this.core.settings.notifyThresholdOfRemoteStorageSize = DEFAULT_SETTINGS.notifyThresholdOfRemoteStorageSize;
await this.services.setting.realiseSetting();
await this.services.remote.markLocked();
await this.services.remote.tryResetDatabase();
await this.services.remote.markLocked();
await delay(500);
// await this.askUsingOptionalFeature({ enableOverwrite: true });
await delay(1000);
await this.services.remote.replicateAllToRemote(true);
await delay(1000);
await this.services.remote.replicateAllToRemote(true, true);
await this.informOptionalFeatures();
}
$rebuildRemote(): Promise<void> {
return this.rebuildRemote();
}
async rebuildEverything() {
await this.services.setting.suspendExtraSync();
// await this.askUseNewAdapter();
this.core.settings.isConfigured = true;
this.core.settings.notifyThresholdOfRemoteStorageSize = DEFAULT_SETTINGS.notifyThresholdOfRemoteStorageSize;
await this.services.setting.realiseSetting();
await this.resetLocalDatabase();
await delay(1000);
await this.services.databaseEvents.initialiseDatabase(true, true, true);
await this.services.remote.markLocked();
await this.services.remote.tryResetDatabase();
await this.services.remote.markLocked();
await delay(500);
// We do not have any other devices' data, so we do not need to ask for overwriting.
// await this.askUsingOptionalFeature({ enableOverwrite: false });
await delay(1000);
await this.services.remote.replicateAllToRemote(true);
await delay(1000);
await this.services.remote.replicateAllToRemote(true, true);
await this.informOptionalFeatures();
}
$rebuildEverything(): Promise<void> {
return this.rebuildEverything();
}
$fetchLocal(makeLocalChunkBeforeSync?: boolean, preventMakeLocalFilesBeforeSync?: boolean): Promise<void> {
return this.fetchLocal(makeLocalChunkBeforeSync, preventMakeLocalFilesBeforeSync);
}
async scheduleRebuild(): Promise<void> {
try {
await this.core.storageAccess.writeFileAuto(FLAGMD_REDFLAG2_HR, "");
} catch (ex) {
this._log("Could not create red_flag_rebuild.md", LOG_LEVEL_NOTICE);
this._log(ex, LOG_LEVEL_VERBOSE);
}
this.services.appLifecycle.performRestart();
}
async scheduleFetch(): Promise<void> {
try {
await this.core.storageAccess.writeFileAuto(FLAGMD_REDFLAG3_HR, "");
} catch (ex) {
this._log("Could not create red_flag_fetch.md", LOG_LEVEL_NOTICE);
this._log(ex, LOG_LEVEL_VERBOSE);
}
this.services.appLifecycle.performRestart();
}
private async _tryResetRemoteDatabase(): Promise<void> {
await this.core.replicator.tryResetRemoteDatabase(this.settings);
}
private async _tryCreateRemoteDatabase(): Promise<void> {
await this.core.replicator.tryCreateRemoteDatabase(this.settings);
}
private async _resetLocalDatabase(): Promise<boolean> {
this.core.storageAccess.clearTouched();
return await this.localDatabase.resetDatabase();
}
async suspendAllSync() {
this.core.settings.liveSync = false;
this.core.settings.periodicReplication = false;
this.core.settings.syncOnSave = false;
this.core.settings.syncOnEditorSave = false;
this.core.settings.syncOnStart = false;
this.core.settings.syncOnFileOpen = false;
this.core.settings.syncAfterMerge = false;
await this.services.setting.suspendExtraSync();
}
async suspendReflectingDatabase() {
if (this.core.settings.doNotSuspendOnFetching) return;
if (this.core.settings.remoteType == REMOTE_MINIO) return;
this._log(
`Suspending reflection: Database and storage changes will not be reflected in each other until completely finished the fetching.`,
LOG_LEVEL_NOTICE
);
this.core.settings.suspendParseReplicationResult = true;
this.core.settings.suspendFileWatching = true;
await this.core.saveSettings();
}
async resumeReflectingDatabase() {
if (this.core.settings.doNotSuspendOnFetching) return;
if (this.core.settings.remoteType == REMOTE_MINIO) return;
this._log(`Database and storage reflection has been resumed!`, LOG_LEVEL_NOTICE);
this.core.settings.suspendParseReplicationResult = false;
this.core.settings.suspendFileWatching = false;
await this.services.vault.scanVault(true);
await this.services.replication.onBeforeReplicate(false); //TODO: Check actual need of this.
await this.core.saveSettings();
}
// No longer needed, both adapters have each advantages and disadvantages.
// async askUseNewAdapter() {
// if (!this.core.settings.useIndexedDBAdapter) {
// const message = `Now this core has been configured to use the old database adapter for keeping compatibility. Do you want to deactivate it?`;
// const CHOICE_YES = "Yes, disable and use latest";
// const CHOICE_NO = "No, keep compatibility";
// const choices = [CHOICE_YES, CHOICE_NO];
//
// const ret = await this.core.confirm.confirmWithMessage(
// "Database adapter",
// message,
// choices,
// CHOICE_YES,
// 10
// );
// if (ret == CHOICE_YES) {
// this.core.settings.useIndexedDBAdapter = true;
// }
// }
// }
async fetchLocal(makeLocalChunkBeforeSync?: boolean, preventMakeLocalFilesBeforeSync?: boolean) {
await this.services.setting.suspendExtraSync();
// await this.askUseNewAdapter();
this.core.settings.isConfigured = true;
this.core.settings.notifyThresholdOfRemoteStorageSize = DEFAULT_SETTINGS.notifyThresholdOfRemoteStorageSize;
if (this.core.settings.maxMTimeForReflectEvents > 0) {
const date = new Date(this.core.settings.maxMTimeForReflectEvents);
const ask = `Your settings restrict file reflection times to no later than ${date}.
**This is a recovery configuration.**
This operation should only be performed on an empty vault.
Are you sure you wish to proceed?`;
const PROCEED = "I understand, proceed";
const CANCEL = "Cancel operation";
const CLEARANDPROCEED = "Clear restriction and proceed";
const choices = [PROCEED, CLEARANDPROCEED, CANCEL] as const;
const ret = await this.core.confirm.askSelectStringDialogue(ask, choices, {
title: "Confirm restricted fetch",
defaultAction: CANCEL,
timeout: 0,
});
if (ret == CLEARANDPROCEED) {
this.core.settings.maxMTimeForReflectEvents = 0;
await this.core.saveSettings();
}
if (ret == CANCEL) {
return;
}
}
await this.suspendReflectingDatabase();
await this.services.setting.realiseSetting();
await this.resetLocalDatabase();
await delay(1000);
await this.services.database.openDatabase();
// this.core.isReady = true;
this.services.appLifecycle.markIsReady();
if (makeLocalChunkBeforeSync) {
await this.core.fileHandler.createAllChunks(true);
} else if (!preventMakeLocalFilesBeforeSync) {
await this.services.databaseEvents.initialiseDatabase(true, true, true);
} else {
// Do not create local file entries before sync (Means use remote information)
}
await this.services.remote.markResolved();
await delay(500);
await this.services.remote.replicateAllFromRemote(true);
await delay(1000);
await this.services.remote.replicateAllFromRemote(true);
await this.resumeReflectingDatabase();
await this.informOptionalFeatures();
// No longer enable
// await this.askUsingOptionalFeature({ enableFetch: true });
}
async fetchLocalWithRebuild() {
return await this.fetchLocal(true);
}
private async _allSuspendAllSync(): Promise<boolean> {
await this.suspendAllSync();
return true;
}
async resetLocalDatabase() {
if (this.core.settings.isConfigured && this.core.settings.additionalSuffixOfDatabaseName == "") {
// Discard the non-suffixed database
await this.services.database.resetDatabase();
}
const suffix = this.services.API.getAppID() || "";
this.core.settings.additionalSuffixOfDatabaseName = suffix;
await this.services.database.resetDatabase();
eventHub.emitEvent(EVENT_DATABASE_REBUILT);
}
async fetchRemoteChunks() {
if (
!this.core.settings.doNotSuspendOnFetching &&
!this.core.settings.useOnlyLocalChunk &&
this.core.settings.remoteType == REMOTE_COUCHDB
) {
this._log(`Fetching chunks`, LOG_LEVEL_NOTICE);
const replicator = this.services.replicator.getActiveReplicator() as LiveSyncCouchDBReplicator;
const remoteDB = await replicator.connectRemoteCouchDBWithSetting(
this.settings,
this.services.API.isMobile(),
true
);
if (typeof remoteDB == "string") {
this._log(remoteDB, LOG_LEVEL_NOTICE);
} else {
await fetchAllUsedChunks(this.localDatabase.localDatabase, remoteDB.db);
}
this._log(`Fetching chunks done`, LOG_LEVEL_NOTICE);
}
}
async resolveAllConflictedFilesByNewerOnes() {
this._log(`Resolving conflicts by newer ones`, LOG_LEVEL_NOTICE);
const files = this.core.storageAccess.getFileNames();
let i = 0;
for (const file of files) {
if (i++ % 10)
this._log(
`Check and Processing ${i} / ${files.length}`,
LOG_LEVEL_NOTICE,
"resolveAllConflictedFilesByNewerOnes"
);
await this.services.conflict.resolveByNewest(file);
}
this._log(`Done!`, LOG_LEVEL_NOTICE, "resolveAllConflictedFilesByNewerOnes");
}
onBindFunction(core: LiveSyncCore, services: typeof core.services): void {
services.appLifecycle.onLoaded.addHandler(this._everyOnload.bind(this));
services.database.resetDatabase.setHandler(this._resetLocalDatabase.bind(this));
services.remote.tryResetDatabase.setHandler(this._tryResetRemoteDatabase.bind(this));
services.remote.tryCreateDatabase.setHandler(this._tryCreateRemoteDatabase.bind(this));
services.setting.suspendAllSync.addHandler(this._allSuspendAllSync.bind(this));
}
}

387
src/modules/core/ModuleReplicator.ts
View File

@@ -1,48 +1,72 @@
import { fireAndForget, yieldMicrotask } from "octagonal-wheels/promises";
import type { LiveSyncLocalDB } from "../../lib/src/pouchdb/LiveSyncLocalDB";
import { fireAndForget } from "octagonal-wheels/promises";
import { AbstractModule } from "../AbstractModule";
import {
Logger,
LOG_LEVEL_NOTICE,
LOG_LEVEL_INFO,
LOG_LEVEL_VERBOSE,
LEVEL_NOTICE,
LEVEL_INFO,
type LOG_LEVEL,
} from "octagonal-wheels/common/logger";
import { isLockAcquired, shareRunningResult, skipIfDuplicated } from "octagonal-wheels/concurrency/lock";
import { balanceChunkPurgedDBs } from "@/lib/src/pouchdb/chunks";
import { purgeUnreferencedChunks } from "@/lib/src/pouchdb/chunks";
import { Logger, LOG_LEVEL_NOTICE, LOG_LEVEL_INFO } from "octagonal-wheels/common/logger";
import { skipIfDuplicated } from "octagonal-wheels/concurrency/lock";
import { balanceChunkPurgedDBs } from "@lib/pouchdb/chunks";
import { purgeUnreferencedChunks } from "@lib/pouchdb/chunks";
import { LiveSyncCouchDBReplicator } from "../../lib/src/replication/couchdb/LiveSyncReplicator";
import { type EntryDoc, type RemoteType } from "../../lib/src/common/types";
import { rateLimitedSharedExecution, scheduleTask, updatePreviousExecutionTime } from "../../common/utils";
import { EVENT_FILE_SAVED, EVENT_ON_UNRESOLVED_ERROR, EVENT_SETTING_SAVED, eventHub } from "../../common/events";
import type { LiveSyncAbstractReplicator } from "../../lib/src/replication/LiveSyncAbstractReplicator";
import { scheduleTask } from "../../common/utils";
import { EVENT_FILE_SAVED, EVENT_SETTING_SAVED, eventHub } from "../../common/events";
import { $msg } from "../../lib/src/common/i18n";
import { clearHandlers } from "../../lib/src/replication/SyncParamsHandler";
import type { LiveSyncCore } from "../../main";
import { ReplicateResultProcessor } from "./ReplicateResultProcessor";
import { UnresolvedErrorManager } from "@lib/services/base/UnresolvedErrorManager";
import { clearHandlers } from "@lib/replication/SyncParamsHandler";
import type { NecessaryServices } from "@/serviceFeatures/types";
import { MARK_LOG_NETWORK_ERROR } from "@lib/services/lib/logUtils";
const KEY_REPLICATION_ON_EVENT = "replicationOnEvent";
const REPLICATION_ON_EVENT_FORECASTED_TIME = 5000;
function isOnlineAndCanReplicate(
errorManager: UnresolvedErrorManager,
host: NecessaryServices<"database", any>,
showMessage: boolean
): Promise<boolean> {
const errorMessage = "Network is offline";
const manager = host.services.database.managers.networkManager;
if (!manager.isOnline) {
errorManager.showError(errorMessage, showMessage ? LOG_LEVEL_NOTICE : LOG_LEVEL_INFO);
return Promise.resolve(false);
}
errorManager.clearError(errorMessage);
return Promise.resolve(true);
}
async function canReplicateWithPBKDF2(
errorManager: UnresolvedErrorManager,
host: NecessaryServices<"replicator" | "setting", any>,
showMessage: boolean
): Promise<boolean> {
const currentSettings = host.services.setting.currentSettings();
// TODO: check using PBKDF2 salt?
const errorMessage = $msg("Replicator.Message.InitialiseFatalError");
const replicator = host.services.replicator.getActiveReplicator();
if (!replicator) {
errorManager.showError(errorMessage, showMessage ? LOG_LEVEL_NOTICE : LOG_LEVEL_INFO);
return false;
}
errorManager.clearError(errorMessage);
// Showing message is false: that because be shown here. (And it is a fatal error, no way to hide it).
// tagged as network error at beginning for error filtering with NetworkWarningStyles
const ensureMessage = `${MARK_LOG_NETWORK_ERROR}Failed to initialise the encryption key, preventing replication.`;
const ensureResult = await replicator.ensurePBKDF2Salt(currentSettings, showMessage, true);
if (!ensureResult) {
errorManager.showError(ensureMessage, showMessage ? LOG_LEVEL_NOTICE : LOG_LEVEL_INFO);
return false;
}
errorManager.clearError(ensureMessage);
return ensureResult; // is true.
}
export class ModuleReplicator extends AbstractModule {
_replicatorType?: RemoteType;
_previousErrors = new Set<string>();
processor: ReplicateResultProcessor = new ReplicateResultProcessor(this);
showError(msg: string, max_log_level: LOG_LEVEL = LEVEL_NOTICE) {
const level = this._previousErrors.has(msg) ? LEVEL_INFO : max_log_level;
this._log(msg, level);
if (!this._previousErrors.has(msg)) {
this._previousErrors.add(msg);
eventHub.emitEvent(EVENT_ON_UNRESOLVED_ERROR);
}
}
processor: ReplicateResultProcessor = new ReplicateResultProcessor(this);
private _unresolvedErrorManager: UnresolvedErrorManager = new UnresolvedErrorManager(
this.core.services.appLifecycle
);
clearErrors() {
this._previousErrors.clear();
eventHub.emitEvent(EVENT_ON_UNRESOLVED_ERROR);
this._unresolvedErrorManager.clearErrors();
}
private _everyOnloadAfterLoadSettings(): Promise<boolean> {
@@ -52,9 +76,6 @@ export class ModuleReplicator extends AbstractModule {
}
});
eventHub.onEvent(EVENT_SETTING_SAVED, (setting) => {
if (this._replicatorType !== setting.remoteType) {
void this.setReplicator();
}
if (this.core.settings.suspendParseReplicationResult) {
this.processor.suspend();
} else {
@@ -65,74 +86,23 @@ export class ModuleReplicator extends AbstractModule {
return Promise.resolve(true);
}
async setReplicator() {
const replicator = await this.services.replicator.getNewReplicator();
if (!replicator) {
this.showError($msg("Replicator.Message.InitialiseFatalError"), LOG_LEVEL_NOTICE);
return false;
}
if (this.core.replicator) {
await this.core.replicator.closeReplication();
this._log("Replicator closed for changing", LOG_LEVEL_VERBOSE);
}
this.core.replicator = replicator;
this._replicatorType = this.settings.remoteType;
await yieldMicrotask();
// Clear any existing sync parameter handlers (means clearing key-deriving salt).
_onReplicatorInitialised(): Promise<boolean> {
// For now, we only need to clear the error related to replicator initialisation, but in the future, if there are more things to do when the replicator is initialised, we can add them here.
clearHandlers();
return true;
return Promise.resolve(true);
}
_getReplicator(): LiveSyncAbstractReplicator {
return this.core.replicator;
}
_everyOnInitializeDatabase(db: LiveSyncLocalDB): Promise<boolean> {
return this.setReplicator();
}
_everyOnDatabaseInitialized(showNotice: boolean): Promise<boolean> {
fireAndForget(() => this.processor.restoreFromSnapshotOnce());
return Promise.resolve(true);
}
_everyOnResetDatabase(db: LiveSyncLocalDB): Promise<boolean> {
return this.setReplicator();
}
async ensureReplicatorPBKDF2Salt(showMessage: boolean = false): Promise<boolean> {
// Checking salt
const replicator = this.services.replicator.getActiveReplicator();
if (!replicator) {
this.showError($msg("Replicator.Message.InitialiseFatalError"), LOG_LEVEL_NOTICE);
return false;
}
return await replicator.ensurePBKDF2Salt(this.settings, showMessage, true);
}
async _everyBeforeReplicate(showMessage: boolean): Promise<boolean> {
// Checking salt
if (!this.core.managers.networkManager.isOnline) {
this.showError("Network is offline", showMessage ? LOG_LEVEL_NOTICE : LOG_LEVEL_INFO);
return false;
}
// Showing message is false: that because be shown here. (And it is a fatal error, no way to hide it).
if (!(await this.ensureReplicatorPBKDF2Salt(false))) {
this.showError("Failed to initialise the encryption key, preventing replication.");
return false;
}
await this.processor.restoreFromSnapshotOnce();
this.clearErrors();
return true;
}
private async _replicate(showMessage: boolean = false): Promise<boolean | void> {
try {
updatePreviousExecutionTime(KEY_REPLICATION_ON_EVENT, REPLICATION_ON_EVENT_FORECASTED_TIME);
return await this.$$_replicate(showMessage);
} finally {
updatePreviousExecutionTime(KEY_REPLICATION_ON_EVENT);
}
}
/**
* obsolete method. No longer maintained and will be removed in the future.
* @deprecated v0.24.17
@@ -192,156 +162,129 @@ Even if you choose to clean up, you will see this option again if you exit Obsid
});
}
async _canReplicate(showMessage: boolean = false): Promise<boolean> {
if (!this.services.appLifecycle.isReady()) {
Logger(`Not ready`);
private async onReplicationFailed(showMessage: boolean = false): Promise<boolean> {
const activeReplicator = this.services.replicator.getActiveReplicator();
if (!activeReplicator) {
Logger(`No active replicator found`, LOG_LEVEL_INFO);
return false;
}
if (isLockAcquired("cleanup")) {
Logger($msg("Replicator.Message.Cleaned"), LOG_LEVEL_NOTICE);
return false;
}
if (this.settings.versionUpFlash != "") {
Logger($msg("Replicator.Message.VersionUpFlash"), LOG_LEVEL_NOTICE);
return false;
}
if (!(await this.services.fileProcessing.commitPendingFileEvents())) {
this.showError($msg("Replicator.Message.Pending"), LOG_LEVEL_NOTICE);
return false;
}
if (!this.core.managers.networkManager.isOnline) {
this.showError("Network is offline", showMessage ? LOG_LEVEL_NOTICE : LOG_LEVEL_INFO);
return false;
}
if (!(await this.services.replication.onBeforeReplicate(showMessage))) {
this.showError($msg("Replicator.Message.SomeModuleFailed"), LOG_LEVEL_NOTICE);
return false;
}
this.clearErrors();
return true;
}
async $$_replicate(showMessage: boolean = false): Promise<boolean | void> {
const checkBeforeReplicate = await this.services.replication.isReplicationReady(showMessage);
if (!checkBeforeReplicate) return false;
//<-- Here could be an module.
const ret = await this.core.replicator.openReplication(this.settings, false, showMessage, false);
if (!ret) {
if (this.core.replicator.tweakSettingsMismatched && this.core.replicator.preferredTweakValue) {
await this.services.tweakValue.askResolvingMismatched(this.core.replicator.preferredTweakValue);
} else {
if (this.core.replicator?.remoteLockedAndDeviceNotAccepted) {
if (this.core.replicator.remoteCleaned && this.settings.useIndexedDBAdapter) {
await this.cleaned(showMessage);
} else {
const message = $msg("Replicator.Dialogue.Locked.Message");
const CHOICE_FETCH = $msg("Replicator.Dialogue.Locked.Action.Fetch");
const CHOICE_DISMISS = $msg("Replicator.Dialogue.Locked.Action.Dismiss");
const CHOICE_UNLOCK = $msg("Replicator.Dialogue.Locked.Action.Unlock");
const ret = await this.core.confirm.askSelectStringDialogue(
message,
[CHOICE_FETCH, CHOICE_UNLOCK, CHOICE_DISMISS],
{
title: $msg("Replicator.Dialogue.Locked.Title"),
defaultAction: CHOICE_DISMISS,
timeout: 60,
}
);
if (ret == CHOICE_FETCH) {
this._log($msg("Replicator.Dialogue.Locked.Message.Fetch"), LOG_LEVEL_NOTICE);
await this.core.rebuilder.scheduleFetch();
this.services.appLifecycle.scheduleRestart();
return;
} else if (ret == CHOICE_UNLOCK) {
await this.core.replicator.markRemoteResolved(this.settings);
this._log($msg("Replicator.Dialogue.Locked.Message.Unlocked"), LOG_LEVEL_NOTICE);
return;
if (activeReplicator.tweakSettingsMismatched && activeReplicator.preferredTweakValue) {
await this.services.tweakValue.askResolvingMismatched(activeReplicator.preferredTweakValue);
} else {
if (activeReplicator.remoteLockedAndDeviceNotAccepted) {
if (activeReplicator.remoteCleaned && this.settings.useIndexedDBAdapter) {
await this.cleaned(showMessage);
} else {
const message = $msg("Replicator.Dialogue.Locked.Message");
const CHOICE_FETCH = $msg("Replicator.Dialogue.Locked.Action.Fetch");
const CHOICE_DISMISS = $msg("Replicator.Dialogue.Locked.Action.Dismiss");
const CHOICE_UNLOCK = $msg("Replicator.Dialogue.Locked.Action.Unlock");
const ret = await this.core.confirm.askSelectStringDialogue(
message,
[CHOICE_FETCH, CHOICE_UNLOCK, CHOICE_DISMISS],
{
title: $msg("Replicator.Dialogue.Locked.Title"),
defaultAction: CHOICE_DISMISS,
timeout: 60,
}
);
if (ret == CHOICE_FETCH) {
this._log($msg("Replicator.Dialogue.Locked.Message.Fetch"), LOG_LEVEL_NOTICE);
await this.core.rebuilder.scheduleFetch();
this.services.appLifecycle.scheduleRestart();
return false;
} else if (ret == CHOICE_UNLOCK) {
await activeReplicator.markRemoteResolved(this.settings);
this._log($msg("Replicator.Dialogue.Locked.Message.Unlocked"), LOG_LEVEL_NOTICE);
return false;
}
}
}
}
return ret;
// TODO: Check again and true/false return. This will be the result for performReplication.
return false;
}
private async _replicateByEvent(): Promise<boolean | void> {
const least = this.settings.syncMinimumInterval;
if (least > 0) {
return rateLimitedSharedExecution(KEY_REPLICATION_ON_EVENT, least, async () => {
return await this.services.replication.replicate();
});
}
return await shareRunningResult(`replication`, () => this.services.replication.replicate());
}
// private async _replicateByEvent(): Promise<boolean | void> {
// const least = this.settings.syncMinimumInterval;
// if (least > 0) {
// return rateLimitedSharedExecution(KEY_REPLICATION_ON_EVENT, least, async () => {
// return await this.services.replication.replicate();
// });
// }
// return await shareRunningResult(`replication`, () => this.services.replication.replicate());
// }
_parseReplicationResult(docs: Array<PouchDB.Core.ExistingDocument<EntryDoc>>): void {
_parseReplicationResult(docs: Array<PouchDB.Core.ExistingDocument<EntryDoc>>): Promise<boolean> {
this.processor.enqueueAll(docs);
}
_everyBeforeSuspendProcess(): Promise<boolean> {
this.core.replicator?.closeReplication();
return Promise.resolve(true);
}
private async _replicateAllToServer(
showingNotice: boolean = false,
sendChunksInBulkDisabled: boolean = false
): Promise<boolean> {
if (!this.services.appLifecycle.isReady()) return false;
if (!(await this.services.replication.onBeforeReplicate(showingNotice))) {
Logger($msg("Replicator.Message.SomeModuleFailed"), LOG_LEVEL_NOTICE);
return false;
}
if (!sendChunksInBulkDisabled) {
if (this.core.replicator instanceof LiveSyncCouchDBReplicator) {
if (
(await this.core.confirm.askYesNoDialog("Do you want to send all chunks before replication?", {
defaultOption: "No",
timeout: 20,
})) == "yes"
) {
await this.core.replicator.sendChunks(this.core.settings, undefined, true, 0);
}
}
}
const ret = await this.core.replicator.replicateAllToServer(this.settings, showingNotice);
if (ret) return true;
const checkResult = await this.services.replication.checkConnectionFailure();
if (checkResult == "CHECKAGAIN") return await this.services.remote.replicateAllToRemote(showingNotice);
return !checkResult;
}
async _replicateAllFromServer(showingNotice: boolean = false): Promise<boolean> {
if (!this.services.appLifecycle.isReady()) return false;
const ret = await this.core.replicator.replicateAllFromServer(this.settings, showingNotice);
if (ret) return true;
const checkResult = await this.services.replication.checkConnectionFailure();
if (checkResult == "CHECKAGAIN") return await this.services.remote.replicateAllFromRemote(showingNotice);
return !checkResult;
}
// _everyBeforeSuspendProcess(): Promise<boolean> {
// this.core.replicator?.closeReplication();
// return Promise.resolve(true);
// }
private _reportUnresolvedMessages(): Promise<string[]> {
return Promise.resolve([...this._previousErrors]);
}
// private async _replicateAllToServer(
// showingNotice: boolean = false,
// sendChunksInBulkDisabled: boolean = false
// ): Promise<boolean> {
// if (!this.services.appLifecycle.isReady()) return false;
// if (!(await this.services.replication.onBeforeReplicate(showingNotice))) {
// Logger($msg("Replicator.Message.SomeModuleFailed"), LOG_LEVEL_NOTICE);
// return false;
// }
// if (!sendChunksInBulkDisabled) {
// if (this.core.replicator instanceof LiveSyncCouchDBReplicator) {
// if (
// (await this.core.confirm.askYesNoDialog("Do you want to send all chunks before replication?", {
// defaultOption: "No",
// timeout: 20,
// })) == "yes"
// ) {
// await this.core.replicator.sendChunks(this.core.settings, undefined, true, 0);
// }
// }
// }
// const ret = await this.core.replicator.replicateAllToServer(this.settings, showingNotice);
// if (ret) return true;
// const checkResult = await this.services.replication.checkConnectionFailure();
// if (checkResult == "CHECKAGAIN") return await this.services.remote.replicateAllToRemote(showingNotice);
// return !checkResult;
// }
// async _replicateAllFromServer(showingNotice: boolean = false): Promise<boolean> {
// if (!this.services.appLifecycle.isReady()) return false;
// const ret = await this.core.replicator.replicateAllFromServer(this.settings, showingNotice);
// if (ret) return true;
// const checkResult = await this.services.replication.checkConnectionFailure();
// if (checkResult == "CHECKAGAIN") return await this.services.remote.replicateAllFromRemote(showingNotice);
// return !checkResult;
// }
onBindFunction(core: LiveSyncCore, services: typeof core.services): void {
services.replicator.getActiveReplicator.setHandler(this._getReplicator.bind(this));
services.databaseEvents.onDatabaseInitialisation.addHandler(this._everyOnInitializeDatabase.bind(this));
override onBindFunction(core: LiveSyncCore, services: typeof core.services): void {
services.replicator.onReplicatorInitialised.addHandler(this._onReplicatorInitialised.bind(this));
services.databaseEvents.onDatabaseInitialised.addHandler(this._everyOnDatabaseInitialized.bind(this));
services.databaseEvents.onResetDatabase.addHandler(this._everyOnResetDatabase.bind(this));
services.appLifecycle.onSettingLoaded.addHandler(this._everyOnloadAfterLoadSettings.bind(this));
services.replication.parseSynchroniseResult.setHandler(this._parseReplicationResult.bind(this));
services.appLifecycle.onSuspending.addHandler(this._everyBeforeSuspendProcess.bind(this));
services.replication.onBeforeReplicate.addHandler(this._everyBeforeReplicate.bind(this));
services.replication.isReplicationReady.setHandler(this._canReplicate.bind(this));
services.replication.replicate.setHandler(this._replicate.bind(this));
services.replication.replicateByEvent.setHandler(this._replicateByEvent.bind(this));
services.remote.replicateAllToRemote.setHandler(this._replicateAllToServer.bind(this));
services.remote.replicateAllFromRemote.setHandler(this._replicateAllFromServer.bind(this));
services.appLifecycle.getUnresolvedMessages.addHandler(this._reportUnresolvedMessages.bind(this));
services.replication.parseSynchroniseResult.addHandler(this._parseReplicationResult.bind(this));
// --> These handlers can be separated.
const isOnlineAndCanReplicateWithHost = isOnlineAndCanReplicate.bind(null, this._unresolvedErrorManager, {
services: {
database: services.database,
},
serviceModules: {},
});
const canReplicateWithPBKDF2WithHost = canReplicateWithPBKDF2.bind(null, this._unresolvedErrorManager, {
services: {
replicator: services.replicator,
setting: services.setting,
},
serviceModules: {},
});
services.replication.onBeforeReplicate.addHandler(isOnlineAndCanReplicateWithHost, 10);
services.replication.onBeforeReplicate.addHandler(canReplicateWithPBKDF2WithHost, 20);
// <-- End of handlers that can be separated.
services.replication.onBeforeReplicate.addHandler(this._everyBeforeReplicate.bind(this), 100);
services.replication.onReplicationFailed.addHandler(this.onReplicationFailed.bind(this));
}
}

2
src/modules/core/ModuleReplicatorCouchDB.ts
View File

@@ -35,7 +35,7 @@ export class ModuleReplicatorCouchDB extends AbstractModule {
return Promise.resolve(true);
}
onBindFunction(core: LiveSyncCore, services: typeof core.services): void {
override onBindFunction(core: LiveSyncCore, services: typeof core.services): void {
services.replicator.getNewReplicator.addHandler(this._anyNewReplicator.bind(this));
services.appLifecycle.onResumed.addHandler(this._everyAfterResumeProcess.bind(this));
}

2
src/modules/core/ModuleReplicatorMinIO.ts
View File

@@ -12,7 +12,7 @@ export class ModuleReplicatorMinIO extends AbstractModule {
}
return Promise.resolve(false);
}
onBindFunction(core: LiveSyncCore, services: typeof core.services): void {
override onBindFunction(core: LiveSyncCore, services: typeof core.services): void {
services.replicator.getNewReplicator.addHandler(this._anyNewReplicator.bind(this));
}
}

2
src/modules/core/ModuleReplicatorP2P.ts
View File

@@ -27,7 +27,7 @@ export class ModuleReplicatorP2P extends AbstractModule {
return Promise.resolve(true);
}
onBindFunction(core: LiveSyncCore, services: typeof core.services): void {
override onBindFunction(core: LiveSyncCore, services: typeof core.services): void {
services.replicator.getNewReplicator.addHandler(this._anyNewReplicator.bind(this));
services.appLifecycle.onResumed.addHandler(this._everyAfterResumeProcess.bind(this));
}

28
src/modules/core/ModuleTargetFilter.ts
View File

@@ -47,7 +47,7 @@ export class ModuleTargetFilter extends AbstractModule {
totalFileEventCount = 0;
private async _isTargetFileByFileNameDuplication(file: string | UXFileInfoStub) {
private async _isTargetAcceptedByFileNameDuplication(file: string | UXFileInfoStub) {
await this.fileCountMap.updateValue(this.totalFileEventCount);
const fileCountMap = this.fileCountMap.value;
if (!fileCountMap) {
@@ -107,7 +107,7 @@ export class ModuleTargetFilter extends AbstractModule {
}
}
private async _isTargetFileByLocalDB(file: string | UXFileInfoStub) {
private async _isTargetAcceptedByLocalDB(file: string | UXFileInfoStub) {
const filepath = getStoragePathFromUXFileInfo(file);
if (!this.localDatabase?.isTargetFile(filepath)) {
this._log("File is not target by local DB: " + filepath);
@@ -117,12 +117,12 @@ export class ModuleTargetFilter extends AbstractModule {
return await Promise.resolve(true);
}
private async _isTargetFileFinal(file: string | UXFileInfoStub) {
private async _isTargetAcceptedFinally(file: string | UXFileInfoStub) {
this._log("File is target finally: " + getStoragePathFromUXFileInfo(file), LOG_LEVEL_DEBUG);
return await Promise.resolve(true);
}
private async _isTargetIgnoredByIgnoreFiles(file: string | UXFileInfoStub): Promise<boolean> {
private async _isTargetAcceptedByIgnoreFiles(file: string | UXFileInfoStub): Promise<boolean> {
if (!this.settings.useIgnoreFiles) {
return true;
}
@@ -137,19 +137,19 @@ export class ModuleTargetFilter extends AbstractModule {
return true;
}
onBindFunction(core: LiveSyncCore, services: typeof core.services): void {
private async _isTargetIgnoredByIgnoreFiles(file: string | UXFileInfoStub) {
const result = await this._isTargetAcceptedByIgnoreFiles(file);
return !result;
}
override onBindFunction(core: LiveSyncCore, services: typeof core.services): void {
services.vault.markFileListPossiblyChanged.setHandler(this._markFileListPossiblyChanged.bind(this));
services.appLifecycle.onLoaded.addHandler(this._everyOnload.bind(this));
services.vault.isIgnoredByIgnoreFile.setHandler(this._isTargetIgnoredByIgnoreFiles.bind(this));
services.vault.isTargetFile.addHandler(this._isTargetFileByFileNameDuplication.bind(this));
services.vault.isTargetFile.addHandler(this._isTargetIgnoredByIgnoreFiles.bind(this));
services.vault.isTargetFile.addHandler(this._isTargetFileByLocalDB.bind(this));
services.vault.isTargetFile.addHandler(this._isTargetFileFinal.bind(this));
services.vault.isTargetFile.addHandler(this._isTargetAcceptedByFileNameDuplication.bind(this), 10);
services.vault.isTargetFile.addHandler(this._isTargetAcceptedByIgnoreFiles.bind(this), 20);
services.vault.isTargetFile.addHandler(this._isTargetAcceptedByLocalDB.bind(this), 30);
services.vault.isTargetFile.addHandler(this._isTargetAcceptedFinally.bind(this), 100);
services.setting.onSettingRealised.addHandler(this.refreshSettings.bind(this));
// services.vault.isTargetFile.use((ctx, next) => {
// const [fileName, keepFileCheckList] = ctx.args;
// const file = getS
// });
}
}

13
src/modules/core/ReplicateResultProcessor.ts
View File

@@ -6,7 +6,7 @@ import {
type EntryLeaf,
type LoadedEntry,
type MetaEntry,
} from "@/lib/src/common/types";
} from "@lib/common/types";
import type { ModuleReplicator } from "./ModuleReplicator";
import { isChunk, isValidPath } from "@/common/utils";
import type { LiveSyncCore } from "@/main";
@@ -17,8 +17,8 @@ import {
LOG_LEVEL_VERBOSE,
Logger,
type LOG_LEVEL,
} from "@/lib/src/common/logger";
import { fireAndForget, isAnyNote, throttle } from "@/lib/src/common/utils";
} from "@lib/common/logger";
import { fireAndForget, isAnyNote, throttle } from "@lib/common/utils";
import { Semaphore } from "octagonal-wheels/concurrency/semaphore_v2";
import { serialized } from "octagonal-wheels/concurrency/lock";
import type { ReactiveSource } from "octagonal-wheels/dataobject/reactive_v2";
@@ -162,7 +162,8 @@ export class ReplicateResultProcessor {
* Report the current status.
*/
protected reportStatus() {
this.core.replicationResultCount.value = this._queuedChanges.length + this._processingChanges.length;
this.services.replication.replicationResultCount.value =
this._queuedChanges.length + this._processingChanges.length;
}
/**
@@ -381,7 +382,7 @@ export class ReplicateResultProcessor {
releaser();
}
}
}, this.replicator.core.databaseQueueCount);
}, this.services.replication.databaseQueueCount);
}
// Phase 2.1: process the document and apply to storage
// This function is serialized per document to avoid race-condition for the same document.
@@ -432,7 +433,7 @@ export class ReplicateResultProcessor {
protected applyToStorage(entry: MetaEntry) {
return this.withCounting(async () => {
await this.services.replication.processSynchroniseResult(entry);
}, this.replicator.core.storageApplyingCount);
}, this.services.replication.storageApplyingCount);
}
/**

4
src/modules/coreFeatures/ModuleConflictChecker.ts
View File

@@ -71,10 +71,10 @@ export class ModuleConflictChecker extends AbstractModule {
delay: 0,
keepResultUntilDownstreamConnected: true,
pipeTo: this.conflictResolveQueue,
totalRemainingReactiveSource: this.core.conflictProcessQueueCount,
totalRemainingReactiveSource: this.services.conflict.conflictProcessQueueCount,
}
);
onBindFunction(core: LiveSyncCore, services: InjectableServiceHub): void {
override onBindFunction(core: LiveSyncCore, services: InjectableServiceHub): void {
services.conflict.queueCheckForIfOpen.setHandler(this._queueConflictCheckIfOpen.bind(this));
services.conflict.queueCheckFor.setHandler(this._queueConflictCheck.bind(this));
services.conflict.ensureAllProcessed.setHandler(this._waitForAllConflictProcessed.bind(this));

22
src/modules/coreFeatures/ModuleConflictResolver.ts
View File

@@ -211,10 +211,30 @@ export class ModuleConflictResolver extends AbstractModule {
}
return true;
}
private async _resolveAllConflictedFilesByNewerOnes() {
this._log(`Resolving conflicts by newer ones`, LOG_LEVEL_NOTICE);
onBindFunction(core: LiveSyncCore, services: InjectableServiceHub): void {
const files = this.core.storageAccess.getFileNames();
let i = 0;
for (const file of files) {
if (i++ % 10)
this._log(
`Check and Processing ${i} / ${files.length}`,
LOG_LEVEL_NOTICE,
"resolveAllConflictedFilesByNewerOnes"
);
await this.services.conflict.resolveByNewest(file);
}
this._log(`Done!`, LOG_LEVEL_NOTICE, "resolveAllConflictedFilesByNewerOnes");
}
override onBindFunction(core: LiveSyncCore, services: InjectableServiceHub): void {
services.conflict.resolveByDeletingRevision.setHandler(this._resolveConflictByDeletingRev.bind(this));
services.conflict.resolve.setHandler(this._resolveConflict.bind(this));
services.conflict.resolveByNewest.setHandler(this._anyResolveConflictByNewest.bind(this));
services.conflict.resolveAllConflictedFilesByNewerOnes.setHandler(
this._resolveAllConflictedFilesByNewerOnes.bind(this)
);
}
}

6
src/modules/coreFeatures/ModuleRedFlag.ts
View File

@@ -12,8 +12,8 @@ import type { LiveSyncCore } from "../../main.ts";
import FetchEverything from "../features/SetupWizard/dialogs/FetchEverything.svelte";
import RebuildEverything from "../features/SetupWizard/dialogs/RebuildEverything.svelte";
import { extractObject } from "octagonal-wheels/object";
import { SvelteDialogManagerBase } from "@/lib/src/UI/svelteDialog.ts";
import type { ServiceContext } from "@/lib/src/services/base/ServiceBase.ts";
import { SvelteDialogManagerBase } from "@lib/UI/svelteDialog.ts";
import type { ServiceContext } from "@lib/services/base/ServiceBase.ts";
export class ModuleRedFlag extends AbstractModule {
async isFlagFileExist(path: string) {
@@ -324,7 +324,7 @@ export class ModuleRedFlag extends AbstractModule {
}
return true;
}
onBindFunction(core: LiveSyncCore, services: typeof core.services): void {
override onBindFunction(core: LiveSyncCore, services: typeof core.services): void {
super.onBindFunction(core, services);
services.appLifecycle.onLayoutReady.addHandler(this._everyOnLayoutReady.bind(this));
}

22
src/modules/coreFeatures/ModuleRemoteGovernor.ts
View File

@@ -1,22 +0,0 @@
import type { InjectableServiceHub } from "../../lib/src/services/InjectableServices.ts";
import type { LiveSyncCore } from "../../main.ts";
import { AbstractModule } from "../AbstractModule.ts";
export class ModuleRemoteGovernor extends AbstractModule {
private async _markRemoteLocked(lockByClean: boolean = false): Promise<void> {
return await this.core.replicator.markRemoteLocked(this.settings, true, lockByClean);
}
private async _markRemoteUnlocked(): Promise<void> {
return await this.core.replicator.markRemoteLocked(this.settings, false, false);
}
private async _markRemoteResolved(): Promise<void> {
return await this.core.replicator.markRemoteResolved(this.settings);
}
onBindFunction(core: LiveSyncCore, services: InjectableServiceHub): void {
services.remote.markLocked.setHandler(this._markRemoteLocked.bind(this));
services.remote.markUnlocked.setHandler(this._markRemoteUnlocked.bind(this));
services.remote.markResolved.setHandler(this._markRemoteResolved.bind(this));
}
}

2
src/modules/coreFeatures/ModuleResolveMismatchedTweaks.ts
View File

@@ -284,7 +284,7 @@ export class ModuleResolvingMismatchedTweaks extends AbstractModule {
return { result: false, requireFetch: false };
}
onBindFunction(core: LiveSyncCore, services: InjectableServiceHub): void {
override onBindFunction(core: LiveSyncCore, services: InjectableServiceHub): void {
services.tweakValue.fetchRemotePreferred.setHandler(this._fetchRemotePreferredTweakValues.bind(this));
services.tweakValue.checkAndAskResolvingMismatched.setHandler(
this._checkAndAskResolvingMismatchedTweaks.bind(this)

381
src/modules/coreObsidian/ModuleFileAccessObsidian.ts
View File

@@ -1,381 +0,0 @@
import { TFile, TFolder, type ListedFiles } from "@/deps.ts";
import { SerializedFileAccess } from "./storageLib/SerializedFileAccess";
import { AbstractObsidianModule } from "../AbstractObsidianModule.ts";
import { LOG_LEVEL_INFO, LOG_LEVEL_VERBOSE } from "octagonal-wheels/common/logger";
import type {
FilePath,
FilePathWithPrefix,
UXDataWriteOptions,
UXFileInfo,
UXFileInfoStub,
UXFolderInfo,
UXStat,
} from "../../lib/src/common/types";
import { TFileToUXFileInfoStub, TFolderToUXFileInfoStub } from "./storageLib/utilObsidian.ts";
import { StorageEventManagerObsidian, type StorageEventManager } from "./storageLib/StorageEventManager";
import type { StorageAccess } from "../interfaces/StorageAccess";
import { createBlob, type CustomRegExp } from "../../lib/src/common/utils";
import { serialized } from "octagonal-wheels/concurrency/lock_v2";
import type { LiveSyncCore } from "../../main.ts";
import type ObsidianLiveSyncPlugin from "../../main.ts";
import type { InjectableServiceHub } from "../../lib/src/services/InjectableServices.ts";
const fileLockPrefix = "file-lock:";
export class ModuleFileAccessObsidian extends AbstractObsidianModule implements StorageAccess {
processingFiles: Set<FilePathWithPrefix> = new Set();
processWriteFile<T>(file: UXFileInfoStub | FilePathWithPrefix, proc: () => Promise<T>): Promise<T> {
const path = typeof file === "string" ? file : file.path;
return serialized(`${fileLockPrefix}${path}`, async () => {
try {
this.processingFiles.add(path);
return await proc();
} finally {
this.processingFiles.delete(path);
}
});
}
processReadFile<T>(file: UXFileInfoStub | FilePathWithPrefix, proc: () => Promise<T>): Promise<T> {
const path = typeof file === "string" ? file : file.path;
return serialized(`${fileLockPrefix}${path}`, async () => {
try {
this.processingFiles.add(path);
return await proc();
} finally {
this.processingFiles.delete(path);
}
});
}
isFileProcessing(file: UXFileInfoStub | FilePathWithPrefix): boolean {
const path = typeof file === "string" ? file : file.path;
return this.processingFiles.has(path);
}
vaultAccess!: SerializedFileAccess;
vaultManager: StorageEventManager = new StorageEventManagerObsidian(this.plugin, this.core, this);
restoreState() {
return this.vaultManager.restoreState();
}
async _everyOnFirstInitialize(): Promise<boolean> {
await this.vaultManager.beginWatch();
return Promise.resolve(true);
}
// $$flushFileEventQueue(): void {
// this.vaultManager.flushQueue();
// }
async _everyCommitPendingFileEvent(): Promise<boolean> {
await this.vaultManager.waitForIdle();
return Promise.resolve(true);
}
_everyOnloadStart(): Promise<boolean> {
this.vaultAccess = new SerializedFileAccess(this.app, this.plugin, this);
this.core.storageAccess = this;
return Promise.resolve(true);
}
async writeFileAuto(path: string, data: string | ArrayBuffer, opt?: UXDataWriteOptions): Promise<boolean> {
const file = this.vaultAccess.getAbstractFileByPath(path);
if (file instanceof TFile) {
return this.vaultAccess.vaultModify(file, data, opt);
} else if (file === null) {
if (!path.endsWith(".md")) {
// Very rare case, we encountered this case with `writing-goals-history.csv` file.
// Indeed, that file not appears in the File Explorer, but it exists in the vault.
// Hence, we cannot retrieve the file from the vault by getAbstractFileByPath, and we cannot write it via vaultModify.
// It makes `File already exists` error.
// Therefore, we need to write it via adapterWrite.
// Maybe there are others like this, so I will write it via adapterWrite.
// This is a workaround for the issue, but I don't know if this is the right solution.
// (So limits to non-md files).
// Has Obsidian been patched?, anyway, writing directly might be a safer approach.
// However, does changes of that file trigger file-change event?
await this.vaultAccess.adapterWrite(path, data, opt);
// For safety, check existence
return await this.vaultAccess.adapterExists(path);
} else {
return (await this.vaultAccess.vaultCreate(path, data, opt)) instanceof TFile;
}
} else {
this._log(`Could not write file (Possibly already exists as a folder): ${path}`, LOG_LEVEL_VERBOSE);
return false;
}
}
readFileAuto(path: string): Promise<string | ArrayBuffer> {
const file = this.vaultAccess.getAbstractFileByPath(path);
if (file instanceof TFile) {
return this.vaultAccess.vaultRead(file);
} else {
throw new Error(`Could not read file (Possibly does not exist): ${path}`);
}
}
readFileText(path: string): Promise<string> {
const file = this.vaultAccess.getAbstractFileByPath(path);
if (file instanceof TFile) {
return this.vaultAccess.vaultRead(file);
} else {
throw new Error(`Could not read file (Possibly does not exist): ${path}`);
}
}
isExists(path: string): Promise<boolean> {
return Promise.resolve(this.vaultAccess.getAbstractFileByPath(path) instanceof TFile);
}
async writeHiddenFileAuto(path: string, data: string | ArrayBuffer, opt?: UXDataWriteOptions): Promise<boolean> {
try {
await this.vaultAccess.adapterWrite(path, data, opt);
return true;
} catch (e) {
this._log(`Could not write hidden file: ${path}`, LOG_LEVEL_VERBOSE);
this._log(e, LOG_LEVEL_VERBOSE);
return false;
}
}
async appendHiddenFile(path: string, data: string, opt?: UXDataWriteOptions): Promise<boolean> {
try {
await this.vaultAccess.adapterAppend(path, data, opt);
return true;
} catch (e) {
this._log(`Could not append hidden file: ${path}`, LOG_LEVEL_VERBOSE);
this._log(e, LOG_LEVEL_VERBOSE);
return false;
}
}
stat(path: string): Promise<UXStat | null> {
const file = this.vaultAccess.getAbstractFileByPath(path);
if (file === null) return Promise.resolve(null);
if (file instanceof TFile) {
return Promise.resolve({
ctime: file.stat.ctime,
mtime: file.stat.mtime,
size: file.stat.size,
type: "file",
});
} else {
throw new Error(`Could not stat file (Possibly does not exist): ${path}`);
}
}
statHidden(path: string): Promise<UXStat | null> {
return this.vaultAccess.tryAdapterStat(path);
}
async removeHidden(path: string): Promise<boolean> {
try {
await this.vaultAccess.adapterRemove(path);
if (this.vaultAccess.tryAdapterStat(path) !== null) {
return false;
}
return true;
} catch (e) {
this._log(`Could not remove hidden file: ${path}`, LOG_LEVEL_VERBOSE);
this._log(e, LOG_LEVEL_VERBOSE);
return false;
}
}
async readHiddenFileAuto(path: string): Promise<string | ArrayBuffer> {
return await this.vaultAccess.adapterReadAuto(path);
}
async readHiddenFileText(path: string): Promise<string> {
return await this.vaultAccess.adapterRead(path);
}
async readHiddenFileBinary(path: string): Promise<ArrayBuffer> {
return await this.vaultAccess.adapterReadBinary(path);
}
async isExistsIncludeHidden(path: string): Promise<boolean> {
return (await this.vaultAccess.tryAdapterStat(path)) !== null;
}
async ensureDir(path: string): Promise<boolean> {
try {
await this.vaultAccess.ensureDirectory(path);
return true;
} catch (e) {
this._log(`Could not ensure directory: ${path}`, LOG_LEVEL_VERBOSE);
this._log(e, LOG_LEVEL_VERBOSE);
return false;
}
}
triggerFileEvent(event: string, path: string): void {
const file = this.vaultAccess.getAbstractFileByPath(path);
if (file === null) return;
this.vaultAccess.trigger(event, file);
}
async triggerHiddenFile(path: string): Promise<void> {
//@ts-ignore internal function
await this.app.vault.adapter.reconcileInternalFile(path);
}
// getFileStub(file: TFile): UXFileInfoStub {
// return TFileToUXFileInfoStub(file);
// }
getFileStub(path: string): UXFileInfoStub | null {
const file = this.vaultAccess.getAbstractFileByPath(path);
if (file instanceof TFile) {
return TFileToUXFileInfoStub(file);
} else {
return null;
}
}
async readStubContent(stub: UXFileInfoStub): Promise<UXFileInfo | false> {
const file = this.vaultAccess.getAbstractFileByPath(stub.path);
if (!(file instanceof TFile)) {
this._log(`Could not read file (Possibly does not exist or a folder): ${stub.path}`, LOG_LEVEL_VERBOSE);
return false;
}
const data = await this.vaultAccess.vaultReadAuto(file);
return {
...stub,
...TFileToUXFileInfoStub(file),
body: createBlob(data),
};
}
getStub(path: string): UXFileInfoStub | UXFolderInfo | null {
const file = this.vaultAccess.getAbstractFileByPath(path);
if (file instanceof TFile) {
return TFileToUXFileInfoStub(file);
} else if (file instanceof TFolder) {
return TFolderToUXFileInfoStub(file);
}
return null;
}
getFiles(): UXFileInfoStub[] {
return this.vaultAccess.getFiles().map((f) => TFileToUXFileInfoStub(f));
}
getFileNames(): FilePath[] {
return this.vaultAccess.getFiles().map((f) => f.path as FilePath);
}
async getFilesIncludeHidden(
basePath: string,
includeFilter?: CustomRegExp[],
excludeFilter?: CustomRegExp[],
skipFolder: string[] = [".git", ".trash", "node_modules"]
): Promise<FilePath[]> {
let w: ListedFiles;
try {
w = await this.app.vault.adapter.list(basePath);
} catch (ex) {
this._log(`Could not traverse(getFilesIncludeHidden):${basePath}`, LOG_LEVEL_INFO);
this._log(ex, LOG_LEVEL_VERBOSE);
return [];
}
skipFolder = skipFolder.map((e) => e.toLowerCase());
let files = [] as string[];
for (const file of w.files) {
if (includeFilter && includeFilter.length > 0) {
if (!includeFilter.some((e) => e.test(file))) continue;
}
if (excludeFilter && excludeFilter.some((ee) => ee.test(file))) {
continue;
}
if (await this.services.vault.isIgnoredByIgnoreFile(file)) continue;
files.push(file);
}
for (const v of w.folders) {
const folderName = (v.split("/").pop() ?? "").toLowerCase();
if (skipFolder.some((e) => folderName === e)) {
continue;
}
if (excludeFilter && excludeFilter.some((e) => e.test(v))) {
continue;
}
if (await this.services.vault.isIgnoredByIgnoreFile(v)) {
continue;
}
// OK, deep dive!
files = files.concat(await this.getFilesIncludeHidden(v, includeFilter, excludeFilter, skipFolder));
}
return files as FilePath[];
}
async touched(file: UXFileInfoStub | FilePathWithPrefix): Promise<void> {
const path = typeof file === "string" ? file : file.path;
await this.vaultAccess.touch(path as FilePath);
}
recentlyTouched(file: UXFileInfoStub | FilePathWithPrefix): boolean {
const xFile = typeof file === "string" ? (this.vaultAccess.getAbstractFileByPath(file) as TFile) : file;
if (xFile === null) return false;
if (xFile instanceof TFolder) return false;
return this.vaultAccess.recentlyTouched(xFile);
}
clearTouched(): void {
this.vaultAccess.clearTouched();
}
delete(file: FilePathWithPrefix | UXFileInfoStub | string, force: boolean): Promise<void> {
const xPath = typeof file === "string" ? file : file.path;
const xFile = this.vaultAccess.getAbstractFileByPath(xPath);
if (xFile === null) return Promise.resolve();
if (!(xFile instanceof TFile) && !(xFile instanceof TFolder)) return Promise.resolve();
return this.vaultAccess.delete(xFile, force);
}
trash(file: FilePathWithPrefix | UXFileInfoStub | string, system: boolean): Promise<void> {
const xPath = typeof file === "string" ? file : file.path;
const xFile = this.vaultAccess.getAbstractFileByPath(xPath);
if (xFile === null) return Promise.resolve();
if (!(xFile instanceof TFile) && !(xFile instanceof TFolder)) return Promise.resolve();
return this.vaultAccess.trash(xFile, system);
}
// $readFileBinary(path: string): Promise<ArrayBuffer> {
// const file = this.vaultAccess.getAbstractFileByPath(path);
// if (file instanceof TFile) {
// return this.vaultAccess.vaultReadBinary(file);
// } else {
// throw new Error(`Could not read file (Possibly does not exist): ${path}`);
// }
// }
// async $appendFileAuto(path: string, data: string | ArrayBuffer, opt?: DataWriteOptions): Promise<boolean> {
// const file = this.vaultAccess.getAbstractFileByPath(path);
// if (file instanceof TFile) {
// return this.vaultAccess.a(file, data, opt);
// } else if (file !== null) {
// return await this.vaultAccess.vaultCreate(path, data, opt) instanceof TFile;
// } else {
// this._log(`Could not append file (Possibly already exists as a folder): ${path}`, LOG_LEVEL_VERBOSE);
// return false;
// }
// }
async __deleteVaultItem(file: TFile | TFolder) {
if (file instanceof TFile) {
if (!(await this.services.vault.isTargetFile(file.path))) return;
}
const dir = file.parent;
if (this.settings.trashInsteadDelete) {
await this.vaultAccess.trash(file, false);
} else {
await this.vaultAccess.delete(file, true);
}
this._log(`xxx <- STORAGE (deleted) ${file.path}`);
if (dir) {
this._log(`files: ${dir.children.length}`);
if (dir.children.length == 0) {
if (!this.settings.doNotDeleteFolder) {
this._log(
`All files under the parent directory (${dir.path}) have been deleted, so delete this one.`
);
await this.__deleteVaultItem(dir);
}
}
}
}
async deleteVaultItem(fileSrc: FilePathWithPrefix | UXFileInfoStub | UXFolderInfo): Promise<void> {
const path = typeof fileSrc === "string" ? fileSrc : fileSrc.path;
const file = this.vaultAccess.getAbstractFileByPath(path);
if (file === null) return;
if (file instanceof TFile || file instanceof TFolder) {
return await this.__deleteVaultItem(file);
}
}
constructor(plugin: ObsidianLiveSyncPlugin, core: LiveSyncCore) {
super(plugin, core);
}
onBindFunction(core: LiveSyncCore, services: InjectableServiceHub): void {
services.appLifecycle.onFirstInitialise.addHandler(this._everyOnFirstInitialize.bind(this));
services.appLifecycle.onInitialise.addHandler(this._everyOnloadStart.bind(this));
services.fileProcessing.commitPendingFileEvents.addHandler(this._everyCommitPendingFileEvent.bind(this));
}
}

16
src/modules/coreObsidian/UILib/dialogs.ts
View File

@@ -13,7 +13,7 @@ class AutoClosableModal extends Modal {
this._closeByUnload = this._closeByUnload.bind(this);
eventHub.once(EVENT_PLUGIN_UNLOADED, this._closeByUnload);
}
onClose() {
override onClose() {
eventHub.off(EVENT_PLUGIN_UNLOADED, this._closeByUnload);
}
}
@@ -43,7 +43,7 @@ export class InputStringDialog extends AutoClosableModal {
this.isPassword = isPassword;
}
onOpen() {
override onOpen() {
const { contentEl } = this;
this.titleEl.setText(this.title);
const formEl = contentEl.createDiv();
@@ -75,7 +75,7 @@ export class InputStringDialog extends AutoClosableModal {
);
}
onClose() {
override onClose() {
super.onClose();
const { contentEl } = this;
contentEl.empty();
@@ -87,7 +87,7 @@ export class InputStringDialog extends AutoClosableModal {
}
}
export class PopoverSelectString extends FuzzySuggestModal<string> {
app: App;
_app: App;
callback: ((e: string) => void) | undefined = () => {};
getItemsFun: () => string[] = () => {
return ["yes", "no"];
@@ -101,7 +101,7 @@ export class PopoverSelectString extends FuzzySuggestModal<string> {
callback: (e: string) => void
) {
super(app);
this.app = app;
this._app = app;
this.setPlaceholder((placeholder ?? "y/n) ") + note);
if (getItemsFun) this.getItemsFun = getItemsFun;
this.callback = callback;
@@ -120,7 +120,7 @@ export class PopoverSelectString extends FuzzySuggestModal<string> {
this.callback?.(item);
this.callback = undefined;
}
onClose(): void {
override onClose(): void {
setTimeout(() => {
if (this.callback) {
this.callback("");
@@ -184,7 +184,7 @@ export class MessageBox<T extends readonly string[]> extends AutoClosableModal {
}
}
onOpen() {
override onOpen() {
const { contentEl } = this;
this.titleEl.setText(this.title);
const div = contentEl.createDiv();
@@ -242,7 +242,7 @@ export class MessageBox<T extends readonly string[]> extends AutoClosableModal {
}
}
onClose() {
override onClose() {
super.onClose();
const { contentEl } = this;
contentEl.empty();

231
src/modules/coreObsidian/storageLib/SerializedFileAccess.ts
View File

@@ -1,231 +0,0 @@
import { type App, TFile, type DataWriteOptions, TFolder, TAbstractFile } from "../../../deps.ts";
import { Logger } from "../../../lib/src/common/logger.ts";
import { isPlainText } from "../../../lib/src/string_and_binary/path.ts";
import type { FilePath, UXFileInfoStub } from "../../../lib/src/common/types.ts";
import { createBinaryBlob, isDocContentSame } from "../../../lib/src/common/utils.ts";
import type { InternalFileInfo } from "../../../common/types.ts";
import { markChangesAreSame } from "../../../common/utils.ts";
import type { StorageAccess } from "../../interfaces/StorageAccess.ts";
import type { LiveSyncCore } from "@/main.ts";
function toArrayBuffer(arr: Uint8Array<ArrayBuffer> | ArrayBuffer | DataView<ArrayBuffer>): ArrayBuffer {
if (arr instanceof Uint8Array) {
return arr.buffer;
}
if (arr instanceof DataView) {
return arr.buffer;
}
return arr;
}
export class SerializedFileAccess {
app: App;
plugin: LiveSyncCore;
storageAccess: StorageAccess;
constructor(app: App, plugin: LiveSyncCore, storageAccess: StorageAccess) {
this.app = app;
this.plugin = plugin;
this.storageAccess = storageAccess;
}
async tryAdapterStat(file: TFile | string) {
const path = file instanceof TFile ? file.path : file;
return await this.storageAccess.processReadFile(path as FilePath, async () => {
if (!(await this.app.vault.adapter.exists(path))) return null;
return this.app.vault.adapter.stat(path);
});
}
async adapterStat(file: TFile | string) {
const path = file instanceof TFile ? file.path : file;
return await this.storageAccess.processReadFile(path as FilePath, () => this.app.vault.adapter.stat(path));
}
async adapterExists(file: TFile | string) {
const path = file instanceof TFile ? file.path : file;
return await this.storageAccess.processReadFile(path as FilePath, () => this.app.vault.adapter.exists(path));
}
async adapterRemove(file: TFile | string) {
const path = file instanceof TFile ? file.path : file;
return await this.storageAccess.processReadFile(path as FilePath, () => this.app.vault.adapter.remove(path));
}
async adapterRead(file: TFile | string) {
const path = file instanceof TFile ? file.path : file;
return await this.storageAccess.processReadFile(path as FilePath, () => this.app.vault.adapter.read(path));
}
async adapterReadBinary(file: TFile | string) {
const path = file instanceof TFile ? file.path : file;
return await this.storageAccess.processReadFile(path as FilePath, () =>
this.app.vault.adapter.readBinary(path)
);
}
async adapterReadAuto(file: TFile | string) {
const path = file instanceof TFile ? file.path : file;
if (isPlainText(path)) {
return await this.storageAccess.processReadFile(path as FilePath, () => this.app.vault.adapter.read(path));
}
return await this.storageAccess.processReadFile(path as FilePath, () =>
this.app.vault.adapter.readBinary(path)
);
}
async adapterWrite(
file: TFile | string,
data: string | ArrayBuffer | Uint8Array<ArrayBuffer>,
options?: DataWriteOptions
) {
const path = file instanceof TFile ? file.path : file;
if (typeof data === "string") {
return await this.storageAccess.processWriteFile(path as FilePath, () =>
this.app.vault.adapter.write(path, data, options)
);
} else {
return await this.storageAccess.processWriteFile(path as FilePath, () =>
this.app.vault.adapter.writeBinary(path, toArrayBuffer(data), options)
);
}
}
async vaultCacheRead(file: TFile) {
return await this.storageAccess.processReadFile(file.path as FilePath, () => this.app.vault.cachedRead(file));
}
async vaultRead(file: TFile) {
return await this.storageAccess.processReadFile(file.path as FilePath, () => this.app.vault.read(file));
}
async vaultReadBinary(file: TFile) {
return await this.storageAccess.processReadFile(file.path as FilePath, () => this.app.vault.readBinary(file));
}
async vaultReadAuto(file: TFile) {
const path = file.path;
if (isPlainText(path)) {
return await this.storageAccess.processReadFile(path as FilePath, () => this.app.vault.read(file));
}
return await this.storageAccess.processReadFile(path as FilePath, () => this.app.vault.readBinary(file));
}
async vaultModify(file: TFile, data: string | ArrayBuffer | Uint8Array<ArrayBuffer>, options?: DataWriteOptions) {
if (typeof data === "string") {
return await this.storageAccess.processWriteFile(file.path as FilePath, async () => {
const oldData = await this.app.vault.read(file);
if (data === oldData) {
if (options && options.mtime) markChangesAreSame(file.path, file.stat.mtime, options.mtime);
return true;
}
await this.app.vault.modify(file, data, options);
return true;
});
} else {
return await this.storageAccess.processWriteFile(file.path as FilePath, async () => {
const oldData = await this.app.vault.readBinary(file);
if (await isDocContentSame(createBinaryBlob(oldData), createBinaryBlob(data))) {
if (options && options.mtime) markChangesAreSame(file.path, file.stat.mtime, options.mtime);
return true;
}
await this.app.vault.modifyBinary(file, toArrayBuffer(data), options);
return true;
});
}
}
async vaultCreate(
path: string,
data: string | ArrayBuffer | Uint8Array<ArrayBuffer>,
options?: DataWriteOptions
): Promise<TFile> {
if (typeof data === "string") {
return await this.storageAccess.processWriteFile(path as FilePath, () =>
this.app.vault.create(path, data, options)
);
} else {
return await this.storageAccess.processWriteFile(path as FilePath, () =>
this.app.vault.createBinary(path, toArrayBuffer(data), options)
);
}
}
trigger(name: string, ...data: any[]) {
return this.app.vault.trigger(name, ...data);
}
async adapterAppend(normalizedPath: string, data: string, options?: DataWriteOptions) {
return await this.app.vault.adapter.append(normalizedPath, data, options);
}
async delete(file: TFile | TFolder, force = false) {
return await this.storageAccess.processWriteFile(file.path as FilePath, () =>
this.app.vault.delete(file, force)
);
}
async trash(file: TFile | TFolder, force = false) {
return await this.storageAccess.processWriteFile(file.path as FilePath, () =>
this.app.vault.trash(file, force)
);
}
isStorageInsensitive(): boolean {
return this.plugin.services.vault.isStorageInsensitive();
}
getAbstractFileByPathInsensitive(path: FilePath | string): TAbstractFile | null {
//@ts-ignore
return this.app.vault.getAbstractFileByPathInsensitive(path);
}
getAbstractFileByPath(path: FilePath | string): TAbstractFile | null {
if (!this.plugin.settings.handleFilenameCaseSensitive || this.isStorageInsensitive()) {
return this.getAbstractFileByPathInsensitive(path);
}
return this.app.vault.getAbstractFileByPath(path);
}
getFiles() {
return this.app.vault.getFiles();
}
async ensureDirectory(fullPath: string) {
const pathElements = fullPath.split("/");
pathElements.pop();
let c = "";
for (const v of pathElements) {
c += v;
try {
await this.app.vault.adapter.mkdir(c);
} catch (ex: any) {
if (ex?.message == "Folder already exists.") {
// Skip if already exists.
} else {
Logger("Folder Create Error");
Logger(ex);
}
}
c += "/";
}
}
touchedFiles: string[] = [];
_statInternal(file: FilePath) {
return this.app.vault.adapter.stat(file);
}
async touch(file: TFile | FilePath) {
const path = file instanceof TFile ? (file.path as FilePath) : file;
const statOrg = file instanceof TFile ? file.stat : await this._statInternal(path);
const stat = statOrg || { mtime: 0, size: 0 };
const key = `${path}-${stat.mtime}-${stat.size}`;
this.touchedFiles.unshift(key);
this.touchedFiles = this.touchedFiles.slice(0, 100);
}
recentlyTouched(file: TFile | InternalFileInfo | UXFileInfoStub) {
const key =
"stat" in file
? `${file.path}-${file.stat.mtime}-${file.stat.size}`
: `${file.path}-${file.mtime}-${file.size}`;
if (this.touchedFiles.indexOf(key) == -1) return false;
return true;
}
clearTouched() {
this.touchedFiles = [];
}
}

650
src/modules/coreObsidian/storageLib/StorageEventManager.ts
View File

@@ -1,650 +0,0 @@
import { TAbstractFile, TFile, TFolder } from "../../../deps.ts";
import { Logger } from "../../../lib/src/common/logger.ts";
import { shouldBeIgnored } from "../../../lib/src/string_and_binary/path.ts";
import {
DEFAULT_SETTINGS,
LOG_LEVEL_DEBUG,
LOG_LEVEL_INFO,
LOG_LEVEL_NOTICE,
LOG_LEVEL_VERBOSE,
type FileEventType,
type FilePath,
type UXFileInfoStub,
type UXInternalFileInfoStub,
} from "../../../lib/src/common/types.ts";
import { delay, fireAndForget, throttle } from "../../../lib/src/common/utils.ts";
import { type FileEventItem } from "../../../common/types.ts";
import { serialized, skipIfDuplicated } from "octagonal-wheels/concurrency/lock";
import { isWaitingForTimeout } from "octagonal-wheels/concurrency/task";
import { Semaphore } from "octagonal-wheels/concurrency/semaphore";
import type { LiveSyncCore } from "../../../main.ts";
import { InternalFileToUXFileInfoStub, TFileToUXFileInfoStub } from "./utilObsidian.ts";
import ObsidianLiveSyncPlugin from "../../../main.ts";
import type { StorageAccess } from "../../interfaces/StorageAccess.ts";
import { HiddenFileSync } from "../../../features/HiddenFileSync/CmdHiddenFileSync.ts";
import { promiseWithResolvers, type PromiseWithResolvers } from "octagonal-wheels/promises";
// import { InternalFileToUXFileInfo } from "../platforms/obsidian.ts";
export type FileEvent = {
type: FileEventType;
file: UXFileInfoStub | UXInternalFileInfoStub;
oldPath?: string;
cachedData?: string;
skipBatchWait?: boolean;
cancelled?: boolean;
};
type WaitInfo = {
since: number;
type: FileEventType;
canProceed: PromiseWithResolvers<boolean>;
timerHandler: ReturnType<typeof setTimeout>;
event: FileEventItem;
};
const TYPE_SENTINEL_FLUSH = "SENTINEL_FLUSH";
type FileEventItemSentinelFlush = {
type: typeof TYPE_SENTINEL_FLUSH;
};
type FileEventItemSentinel = FileEventItemSentinelFlush;
export abstract class StorageEventManager {
abstract beginWatch(): Promise<void>;
abstract appendQueue(items: FileEvent[], ctx?: any): Promise<void>;
abstract isWaiting(filename: FilePath): boolean;
abstract waitForIdle(): Promise<void>;
abstract restoreState(): Promise<void>;
}
export class StorageEventManagerObsidian extends StorageEventManager {
plugin: ObsidianLiveSyncPlugin;
core: LiveSyncCore;
storageAccess: StorageAccess;
get services() {
return this.core.services;
}
get shouldBatchSave() {
return this.core.settings?.batchSave && this.core.settings?.liveSync != true;
}
get batchSaveMinimumDelay(): number {
return this.core.settings?.batchSaveMinimumDelay ?? DEFAULT_SETTINGS.batchSaveMinimumDelay;
}
get batchSaveMaximumDelay(): number {
return this.core.settings?.batchSaveMaximumDelay ?? DEFAULT_SETTINGS.batchSaveMaximumDelay;
}
// Necessary evil.
cmdHiddenFileSync: HiddenFileSync;
/**
* Snapshot restoration promise.
* Snapshot will be restored before starting to watch vault changes.
* In designed time, this has been called from Initialisation process, which has been implemented on `ModuleInitializerFile.ts`.
*/
snapShotRestored: Promise<void> | null = null;
constructor(plugin: ObsidianLiveSyncPlugin, core: LiveSyncCore, storageAccess: StorageAccess) {
super();
this.storageAccess = storageAccess;
this.plugin = plugin;
this.core = core;
this.cmdHiddenFileSync = this.plugin.getAddOn(HiddenFileSync.name) as HiddenFileSync;
}
/**
* Restore the previous snapshot if exists.
* @returns
*/
restoreState(): Promise<void> {
this.snapShotRestored = this._restoreFromSnapshot();
return this.snapShotRestored;
}
async beginWatch() {
await this.snapShotRestored;
const plugin = this.plugin;
this.watchVaultChange = this.watchVaultChange.bind(this);
this.watchVaultCreate = this.watchVaultCreate.bind(this);
this.watchVaultDelete = this.watchVaultDelete.bind(this);
this.watchVaultRename = this.watchVaultRename.bind(this);
this.watchVaultRawEvents = this.watchVaultRawEvents.bind(this);
this.watchEditorChange = this.watchEditorChange.bind(this);
plugin.registerEvent(plugin.app.vault.on("modify", this.watchVaultChange));
plugin.registerEvent(plugin.app.vault.on("delete", this.watchVaultDelete));
plugin.registerEvent(plugin.app.vault.on("rename", this.watchVaultRename));
plugin.registerEvent(plugin.app.vault.on("create", this.watchVaultCreate));
//@ts-ignore : Internal API
plugin.registerEvent(plugin.app.vault.on("raw", this.watchVaultRawEvents));
plugin.registerEvent(plugin.app.workspace.on("editor-change", this.watchEditorChange));
}
watchEditorChange(editor: any, info: any) {
if (!("path" in info)) {
return;
}
if (!this.shouldBatchSave) {
return;
}
const file = info?.file as TFile;
if (!file) return;
if (this.storageAccess.isFileProcessing(file.path as FilePath)) {
// Logger(`Editor change skipped because the file is being processed: ${file.path}`, LOG_LEVEL_VERBOSE);
return;
}
if (!this.isWaiting(file.path as FilePath)) {
return;
}
const data = info?.data as string;
const fi: FileEvent = {
type: "CHANGED",
file: TFileToUXFileInfoStub(file),
cachedData: data,
};
void this.appendQueue([fi]);
}
watchVaultCreate(file: TAbstractFile, ctx?: any) {
if (file instanceof TFolder) return;
if (this.storageAccess.isFileProcessing(file.path as FilePath)) {
// Logger(`File create skipped because the file is being processed: ${file.path}`, LOG_LEVEL_VERBOSE);
return;
}
const fileInfo = TFileToUXFileInfoStub(file);
void this.appendQueue([{ type: "CREATE", file: fileInfo }], ctx);
}
watchVaultChange(file: TAbstractFile, ctx?: any) {
if (file instanceof TFolder) return;
if (this.storageAccess.isFileProcessing(file.path as FilePath)) {
// Logger(`File change skipped because the file is being processed: ${file.path}`, LOG_LEVEL_VERBOSE);
return;
}
const fileInfo = TFileToUXFileInfoStub(file);
void this.appendQueue([{ type: "CHANGED", file: fileInfo }], ctx);
}
watchVaultDelete(file: TAbstractFile, ctx?: any) {
if (file instanceof TFolder) return;
if (this.storageAccess.isFileProcessing(file.path as FilePath)) {
// Logger(`File delete skipped because the file is being processed: ${file.path}`, LOG_LEVEL_VERBOSE);
return;
}
const fileInfo = TFileToUXFileInfoStub(file, true);
void this.appendQueue([{ type: "DELETE", file: fileInfo }], ctx);
}
watchVaultRename(file: TAbstractFile, oldFile: string, ctx?: any) {
// vault Rename will not be raised for self-events (Self-hosted LiveSync will not handle 'rename').
if (file instanceof TFile) {
const fileInfo = TFileToUXFileInfoStub(file);
void this.appendQueue(
[
{
type: "DELETE",
file: {
path: oldFile as FilePath,
name: file.name,
stat: {
mtime: file.stat.mtime,
ctime: file.stat.ctime,
size: file.stat.size,
type: "file",
},
deleted: true,
},
skipBatchWait: true,
},
{ type: "CREATE", file: fileInfo, skipBatchWait: true },
],
ctx
);
}
}
// Watch raw events (Internal API)
watchVaultRawEvents(path: FilePath) {
if (this.storageAccess.isFileProcessing(path)) {
// Logger(`Raw file event skipped because the file is being processed: ${path}`, LOG_LEVEL_VERBOSE);
return;
}
// Only for internal files.
if (!this.plugin.settings) return;
// if (this.plugin.settings.useIgnoreFiles && this.plugin.ignoreFiles.some(e => path.endsWith(e.trim()))) {
if (this.plugin.settings.useIgnoreFiles) {
// If it is one of ignore files, refresh the cached one.
// (Calling$$isTargetFile will refresh the cache)
void this.services.vault.isTargetFile(path).then(() => this._watchVaultRawEvents(path));
} else {
void this._watchVaultRawEvents(path);
}
}
async _watchVaultRawEvents(path: FilePath) {
if (!this.plugin.settings.syncInternalFiles && !this.plugin.settings.usePluginSync) return;
if (!this.plugin.settings.watchInternalFileChanges) return;
if (!path.startsWith(this.plugin.app.vault.configDir)) return;
if (path.endsWith("/")) {
// Folder
return;
}
const isTargetFile = await this.cmdHiddenFileSync.isTargetFile(path);
if (!isTargetFile) return;
void this.appendQueue(
[
{
type: "INTERNAL",
file: InternalFileToUXFileInfoStub(path),
skipBatchWait: true, // Internal files should be processed immediately.
},
],
null
);
}
// Cache file and waiting to can be proceed.
async appendQueue(params: FileEvent[], ctx?: any) {
if (!this.core.settings.isConfigured) return;
if (this.core.settings.suspendFileWatching) return;
if (this.core.settings.maxMTimeForReflectEvents > 0) {
return;
}
this.core.services.vault.markFileListPossiblyChanged();
// Flag up to be reload
for (const param of params) {
if (shouldBeIgnored(param.file.path)) {
continue;
}
const atomicKey = [0, 0, 0, 0, 0, 0].map((e) => `${Math.floor(Math.random() * 100000)}`).join("-");
const type = param.type;
const file = param.file;
const oldPath = param.oldPath;
if (type !== "INTERNAL") {
const size = (file as UXFileInfoStub).stat.size;
if (this.services.vault.isFileSizeTooLarge(size) && (type == "CREATE" || type == "CHANGED")) {
Logger(
`The storage file has been changed but exceeds the maximum size. Skipping: ${param.file.path}`,
LOG_LEVEL_NOTICE
);
continue;
}
}
if (file instanceof TFolder) continue;
// TODO: Confirm why only the TFolder skipping
// Possibly following line is needed...
// if (file?.isFolder) continue;
if (!(await this.services.vault.isTargetFile(file.path))) continue;
// Stop cache using to prevent the corruption;
// let cache: null | string | ArrayBuffer;
// new file or something changed, cache the changes.
// if (file instanceof TFile && (type == "CREATE" || type == "CHANGED")) {
if (file instanceof TFile || !file.isFolder) {
if (type == "CREATE" || type == "CHANGED") {
// Wait for a bit while to let the writer has marked `touched` at the file.
await delay(10);
if (this.core.storageAccess.recentlyTouched(file.path)) {
continue;
}
}
}
let cache: string | undefined = undefined;
if (param.cachedData) {
cache = param.cachedData;
}
void this.enqueue({
type,
args: {
file: file,
oldPath,
cache,
ctx,
},
skipBatchWait: param.skipBatchWait,
key: atomicKey,
});
}
}
private bufferedQueuedItems = [] as (FileEventItem | FileEventItemSentinel)[];
/**
* Immediately take snapshot.
*/
private _triggerTakeSnapshot() {
void this._takeSnapshot();
}
/**
* Trigger taking snapshot after throttled period.
*/
triggerTakeSnapshot = throttle(() => this._triggerTakeSnapshot(), 100);
enqueue(newItem: FileEventItem) {
if (newItem.type == "DELETE") {
// If the sentinel pushed, the runQueuedEvents will wait for idle before processing delete.
this.bufferedQueuedItems.push({
type: TYPE_SENTINEL_FLUSH,
});
}
this.updateStatus();
this.bufferedQueuedItems.push(newItem);
fireAndForget(() => this._takeSnapshot().then(() => this.runQueuedEvents()));
}
// Limit concurrent processing to reduce the IO load. file-processing + scheduler (1), so file events can be processed in 4 slots.
concurrentProcessing = Semaphore(5);
private _waitingMap = new Map<string, WaitInfo>();
private _waitForIdle: Promise<void> | null = null;
/**
* Wait until all queued events are processed.
* Subsequent new events will not be waited, but new events will not be added.
* @returns
*/
waitForIdle(): Promise<void> {
if (this._waitingMap.size === 0) {
return Promise.resolve();
}
if (this._waitForIdle) {
return this._waitForIdle;
}
const promises = [...this._waitingMap.entries()].map(([key, waitInfo]) => {
return new Promise<void>((resolve) => {
waitInfo.canProceed.promise
.then(() => {
Logger(`Processing ${key}: Wait for idle completed`, LOG_LEVEL_DEBUG);
// No op
})
.catch((e) => {
Logger(`Processing ${key}: Wait for idle error`, LOG_LEVEL_INFO);
Logger(e, LOG_LEVEL_VERBOSE);
//no op
})
.finally(() => {
resolve();
});
this._proceedWaiting(key);
});
});
const waitPromise = Promise.all(promises).then(() => {
this._waitForIdle = null;
Logger(`All wait for idle completed`, LOG_LEVEL_VERBOSE);
});
this._waitForIdle = waitPromise;
return waitPromise;
}
/**
* Proceed waiting for the given key immediately.
*/
private _proceedWaiting(key: string) {
const waitInfo = this._waitingMap.get(key);
if (waitInfo) {
waitInfo.canProceed.resolve(true);
clearTimeout(waitInfo.timerHandler);
this._waitingMap.delete(key);
}
this.triggerTakeSnapshot();
}
/**
* Cancel waiting for the given key.
*/
private _cancelWaiting(key: string) {
const waitInfo = this._waitingMap.get(key);
if (waitInfo) {
waitInfo.canProceed.resolve(false);
clearTimeout(waitInfo.timerHandler);
this._waitingMap.delete(key);
}
this.triggerTakeSnapshot();
}
/**
* Add waiting for the given key.
* @param key
* @param event
* @param waitedSince Optional waited since timestamp to calculate the remaining delay.
*/
private _addWaiting(key: string, event: FileEventItem, waitedSince?: number): WaitInfo {
if (this._waitingMap.has(key)) {
// Already waiting
throw new Error(`Already waiting for key: ${key}`);
}
const resolver = promiseWithResolvers<boolean>();
const now = Date.now();
const since = waitedSince ?? now;
const elapsed = now - since;
const maxDelay = this.batchSaveMaximumDelay * 1000;
const remainingDelay = Math.max(0, maxDelay - elapsed);
const nextDelay = Math.min(remainingDelay, this.batchSaveMinimumDelay * 1000);
// x*<------- maxDelay --------->*
// x*<-- minDelay -->*
// x* x<-- nextDelay -->*
// x* x<-- Capped-->*
// x* x.......*
// x: event
// *: save
// When at event (x) At least, save (*) within maxDelay, but maintain minimum delay between saves.
if (elapsed >= maxDelay) {
// Already exceeded maximum delay, do not wait.
Logger(`Processing ${key}: Batch save maximum delay already exceeded: ${event.type}`, LOG_LEVEL_DEBUG);
} else {
Logger(`Processing ${key}: Adding waiting for batch save: ${event.type} (${nextDelay}ms)`, LOG_LEVEL_DEBUG);
}
const waitInfo: WaitInfo = {
since: since,
type: event.type,
event: event,
canProceed: resolver,
timerHandler: setTimeout(() => {
Logger(`Processing ${key}: Batch save timeout reached: ${event.type}`, LOG_LEVEL_DEBUG);
this._proceedWaiting(key);
}, nextDelay),
};
this._waitingMap.set(key, waitInfo);
this.triggerTakeSnapshot();
return waitInfo;
}
/**
* Process the given file event.
*/
async processFileEvent(fei: FileEventItem) {
const releaser = await this.concurrentProcessing.acquire();
try {
this.updateStatus();
const filename = fei.args.file.path;
const waitingKey = `${filename}`;
const previous = this._waitingMap.get(waitingKey);
let isShouldBeCancelled = fei.skipBatchWait || false;
let previousPromise: Promise<boolean> = Promise.resolve(true);
let waitPromise: Promise<boolean> = Promise.resolve(true);
// 1. Check if there is previous waiting for the same file
if (previous) {
previousPromise = previous.canProceed.promise;
if (isShouldBeCancelled) {
Logger(
`Processing ${filename}: Requested to perform immediately, cancelling previous waiting: ${fei.type}`,
LOG_LEVEL_DEBUG
);
}
if (!isShouldBeCancelled && fei.type === "DELETE") {
// For DELETE, cancel any previous waiting and proceed immediately
// That because when deleting, we cannot read the file anymore.
Logger(
`Processing ${filename}: DELETE requested, cancelling previous waiting: ${fei.type}`,
LOG_LEVEL_DEBUG
);
isShouldBeCancelled = true;
}
if (!isShouldBeCancelled && previous.type === fei.type) {
// For the same type, we can cancel the previous waiting and proceed immediately.
Logger(`Processing ${filename}: Cancelling previous waiting: ${fei.type}`, LOG_LEVEL_DEBUG);
isShouldBeCancelled = true;
}
// 2. wait for the previous to complete
if (isShouldBeCancelled) {
this._cancelWaiting(waitingKey);
Logger(`Processing ${filename}: Previous cancelled: ${fei.type}`, LOG_LEVEL_DEBUG);
isShouldBeCancelled = true;
}
if (!isShouldBeCancelled) {
Logger(`Processing ${filename}: Waiting for previous to complete: ${fei.type}`, LOG_LEVEL_DEBUG);
this._proceedWaiting(waitingKey);
Logger(`Processing ${filename}: Previous completed: ${fei.type}`, LOG_LEVEL_DEBUG);
}
}
await previousPromise;
// 3. Check if shouldBatchSave is true
if (this.shouldBatchSave && !fei.skipBatchWait) {
// if type is CREATE or CHANGED, set waiting
if (fei.type == "CREATE" || fei.type == "CHANGED") {
// 3.2. If true, set the queue, and wait for the waiting, or until timeout
// (since is copied from previous waiting if exists to limit the maximum wait time)
// console.warn(`Since:`, previous?.since);
const info = this._addWaiting(waitingKey, fei, previous?.since);
waitPromise = info.canProceed.promise;
} else if (fei.type == "DELETE") {
// For DELETE, cancel any previous waiting and proceed immediately
}
Logger(`Processing ${filename}: Waiting for batch save: ${fei.type}`, LOG_LEVEL_DEBUG);
const canProceed = await waitPromise;
if (!canProceed) {
// 3.2.1. If cancelled by new queue, cancel subsequent process.
Logger(`Processing ${filename}: Cancelled by new queue: ${fei.type}`, LOG_LEVEL_DEBUG);
return;
}
}
// await this.handleFileEvent(fei);
await this.requestProcessQueue(fei);
} finally {
await this._takeSnapshot();
releaser();
}
}
async _takeSnapshot() {
const processingEvents = [...this._waitingMap.values()].map((e) => e.event);
const waitingEvents = this.bufferedQueuedItems;
const snapShot = [...processingEvents, ...waitingEvents];
await this.core.kvDB.set("storage-event-manager-snapshot", snapShot);
Logger(`Storage operation snapshot taken: ${snapShot.length} items`, LOG_LEVEL_DEBUG);
this.updateStatus();
}
async _restoreFromSnapshot() {
const snapShot = await this.core.kvDB.get<(FileEventItem | FileEventItemSentinel)[]>(
"storage-event-manager-snapshot"
);
if (snapShot && Array.isArray(snapShot) && snapShot.length > 0) {
// console.warn(`Restoring snapshot: ${snapShot.length} items`);
Logger(`Restoring storage operation snapshot: ${snapShot.length} items`, LOG_LEVEL_VERBOSE);
// Restore the snapshot
// Note: Mark all items as skipBatchWait to prevent apply the off-line batch saving.
this.bufferedQueuedItems = snapShot.map((e) => ({ ...e, skipBatchWait: true }));
this.updateStatus();
await this.runQueuedEvents();
} else {
Logger(`No snapshot to restore`, LOG_LEVEL_VERBOSE);
// console.warn(`No snapshot to restore`);
}
}
runQueuedEvents() {
return skipIfDuplicated("storage-event-manager-run-queued-events", async () => {
do {
if (this.bufferedQueuedItems.length === 0) {
break;
}
// 1. Get the first queued item
const fei = this.bufferedQueuedItems.shift()!;
await this._takeSnapshot();
this.updateStatus();
// 2. Consume 1 semaphore slot to enqueue processing. Then release immediately.
// (Just to limit the total concurrent processing count, because skipping batch handles at processFileEvent).
const releaser = await this.concurrentProcessing.acquire();
releaser();
this.updateStatus();
// 3. Check if sentinel flush
// If sentinel, wait for idle and continue.
if (fei.type === TYPE_SENTINEL_FLUSH) {
Logger(`Waiting for idle`, LOG_LEVEL_VERBOSE);
// Flush all waiting batch queues
await this.waitForIdle();
this.updateStatus();
continue;
}
// 4. Process the event, this should be fire-and-forget to not block the queue processing in each file.
fireAndForget(() => this.processFileEvent(fei));
} while (this.bufferedQueuedItems.length > 0);
});
}
processingCount = 0;
async requestProcessQueue(fei: FileEventItem) {
try {
this.processingCount++;
// this.bufferedQueuedItems.remove(fei);
this.updateStatus();
// this.waitedSince.delete(fei.args.file.path);
await this.handleFileEvent(fei);
await this._takeSnapshot();
} finally {
this.processingCount--;
this.updateStatus();
}
}
isWaiting(filename: FilePath) {
return isWaitingForTimeout(`storage-event-manager-batchsave-${filename}`);
}
updateStatus() {
const allFileEventItems = this.bufferedQueuedItems.filter((e): e is FileEventItem => "args" in e);
const allItems = allFileEventItems.filter((e) => !e.cancelled);
const totalItems = allItems.length + this.concurrentProcessing.waiting;
const processing = this.processingCount;
const batchedCount = this._waitingMap.size;
this.core.batched.value = batchedCount;
this.core.processing.value = processing;
this.core.totalQueued.value = totalItems + batchedCount + processing;
}
async handleFileEvent(queue: FileEventItem): Promise<any> {
const file = queue.args.file;
const lockKey = `handleFile:${file.path}`;
const ret = await serialized(lockKey, async () => {
if (queue.cancelled) {
Logger(`File event cancelled before processing: ${file.path}`, LOG_LEVEL_INFO);
return;
}
if (queue.type == "INTERNAL" || file.isInternal) {
await this.core.services.fileProcessing.processOptionalFileEvent(file.path as unknown as FilePath);
} else {
const key = `file-last-proc-${queue.type}-${file.path}`;
const last = Number((await this.core.kvDB.get(key)) || 0);
if (queue.type == "DELETE") {
await this.core.services.fileProcessing.processFileEvent(queue);
} else {
if (file.stat.mtime == last) {
Logger(`File has been already scanned on ${queue.type}, skip: ${file.path}`, LOG_LEVEL_VERBOSE);
// Should Cancel the relative operations? (e.g. rename)
// this.cancelRelativeEvent(queue);
return;
}
if (!(await this.core.services.fileProcessing.processFileEvent(queue))) {
Logger(
`STORAGE -> DB: Handler failed, cancel the relative operations: ${file.path}`,
LOG_LEVEL_INFO
);
// cancel running queues and remove one of atomic operation (e.g. rename)
this.cancelRelativeEvent(queue);
return;
}
}
}
});
this.updateStatus();
return ret;
}
cancelRelativeEvent(item: FileEventItem): void {
this._cancelWaiting(item.args.file.path);
}
}

4
src/modules/coreObsidian/storageLib/utilObsidian.ts
View File

@@ -2,7 +2,6 @@
import { TFile, type TAbstractFile, type TFolder } from "../../../deps.ts";
import { ICHeader } from "../../../common/types.ts";
import type { SerializedFileAccess } from "./SerializedFileAccess.ts";
import { addPrefix, isPlainText } from "../../../lib/src/string_and_binary/path.ts";
import { LOG_LEVEL_VERBOSE, Logger } from "octagonal-wheels/common/logger";
import { createBlob } from "../../../lib/src/common/utils.ts";
@@ -15,6 +14,7 @@ import type {
UXInternalFileInfoStub,
} from "../../../lib/src/common/types.ts";
import type { LiveSyncCore } from "../../../main.ts";
import type { FileAccessObsidian } from "@/serviceModules/FileAccessObsidian.ts";
export async function TFileToUXFileInfo(
core: LiveSyncCore,
@@ -51,7 +51,7 @@ export async function TFileToUXFileInfo(
export async function InternalFileToUXFileInfo(
fullPath: string,
vaultAccess: SerializedFileAccess,
vaultAccess: FileAccessObsidian,
prefix: string = ICHeader
): Promise<UXFileInfo> {
const name = fullPath.split("/").pop() as string;

10
src/modules/essential/ModuleInitializerFile.ts
View File

@@ -393,7 +393,13 @@ export class ModuleInitializerFile extends AbstractModule {
ignoreSuspending: boolean = false
): Promise<boolean> {
this.services.appLifecycle.resetIsReady();
if (!reopenDatabase || (await this.services.database.openDatabase())) {
if (
!reopenDatabase ||
(await this.services.database.openDatabase({
databaseEvents: this.services.databaseEvents,
replicator: this.services.replicator,
}))
) {
if (this.localDatabase.isReady) {
await this.services.vault.scanVault(showingNotice, ignoreSuspending);
}
@@ -415,7 +421,7 @@ export class ModuleInitializerFile extends AbstractModule {
private _reportDetectedErrors(): Promise<string[]> {
return Promise.resolve(Array.from(this._detectedErrors));
}
onBindFunction(core: LiveSyncCore, services: InjectableServiceHub): void {
override onBindFunction(core: LiveSyncCore, services: InjectableServiceHub): void {
services.appLifecycle.getUnresolvedMessages.addHandler(this._reportDetectedErrors.bind(this));
services.databaseEvents.initialiseDatabase.setHandler(this._initializeDatabase.bind(this));
services.vault.scanVault.setHandler(this._performFullScan.bind(this));

114
src/modules/essential/ModuleKeyValueDB.ts
View File

@@ -1,114 +0,0 @@
import { delay, yieldMicrotask } from "octagonal-wheels/promises";
import { OpenKeyValueDatabase } from "../../common/KeyValueDB.ts";
import type { LiveSyncLocalDB } from "../../lib/src/pouchdb/LiveSyncLocalDB.ts";
import { LOG_LEVEL_NOTICE, LOG_LEVEL_VERBOSE } from "octagonal-wheels/common/logger";
import { AbstractModule } from "../AbstractModule.ts";
import type { LiveSyncCore } from "../../main.ts";
import type { SimpleStore } from "octagonal-wheels/databases/SimpleStoreBase";
import type { InjectableServiceHub } from "@/lib/src/services/InjectableServices.ts";
import type { ObsidianDatabaseService } from "../services/ObsidianServices.ts";
export class ModuleKeyValueDB extends AbstractModule {
async tryCloseKvDB() {
try {
await this.core.kvDB?.close();
return true;
} catch (e) {
this._log("Failed to close KeyValueDB", LOG_LEVEL_VERBOSE);
this._log(e);
return false;
}
}
async openKeyValueDB(): Promise<boolean> {
await delay(10);
try {
await this.tryCloseKvDB();
await delay(10);
await yieldMicrotask();
this.core.kvDB = await OpenKeyValueDatabase(this.services.vault.getVaultName() + "-livesync-kv");
await yieldMicrotask();
await delay(100);
} catch (e) {
this.core.kvDB = undefined!;
this._log("Failed to open KeyValueDB", LOG_LEVEL_NOTICE);
this._log(e, LOG_LEVEL_VERBOSE);
return false;
}
return true;
}
async _onDBUnload(db: LiveSyncLocalDB) {
if (this.core.kvDB) await this.core.kvDB.close();
return Promise.resolve(true);
}
async _onDBClose(db: LiveSyncLocalDB) {
if (this.core.kvDB) await this.core.kvDB.close();
return Promise.resolve(true);
}
private async _everyOnloadAfterLoadSettings(): Promise<boolean> {
if (!(await this.openKeyValueDB())) {
return false;
}
this.core.simpleStore = this.services.database.openSimpleStore<any>("os");
return Promise.resolve(true);
}
_getSimpleStore<T>(kind: string) {
const getDB = () => this.core.kvDB;
const prefix = `${kind}-`;
return {
get: async (key: string): Promise<T> => {
return await getDB().get(`${prefix}${key}`);
},
set: async (key: string, value: any): Promise<void> => {
await getDB().set(`${prefix}${key}`, value);
},
delete: async (key: string): Promise<void> => {
await getDB().del(`${prefix}${key}`);
},
keys: async (
from: string | undefined,
to: string | undefined,
count?: number | undefined
): Promise<string[]> => {
const ret = await getDB().keys(
IDBKeyRange.bound(`${prefix}${from || ""}`, `${prefix}${to || ""}`),
count
);
return ret
.map((e) => e.toString())
.filter((e) => e.startsWith(prefix))
.map((e) => e.substring(prefix.length));
},
db: Promise.resolve(getDB()),
} satisfies SimpleStore<T>;
}
_everyOnInitializeDatabase(db: LiveSyncLocalDB): Promise<boolean> {
return this.openKeyValueDB();
}
async _everyOnResetDatabase(db: LiveSyncLocalDB): Promise<boolean> {
try {
const kvDBKey = "queued-files";
await this.core.kvDB.del(kvDBKey);
// localStorage.removeItem(lsKey);
await this.core.kvDB.destroy();
await yieldMicrotask();
this.core.kvDB = await OpenKeyValueDatabase(this.services.vault.getVaultName() + "-livesync-kv");
await delay(100);
} catch (e) {
this.core.kvDB = undefined!;
this._log("Failed to reset KeyValueDB", LOG_LEVEL_NOTICE);
this._log(e, LOG_LEVEL_VERBOSE);
return false;
}
return true;
}
onBindFunction(core: LiveSyncCore, services: InjectableServiceHub): void {
services.databaseEvents.onUnloadDatabase.addHandler(this._onDBUnload.bind(this));
services.databaseEvents.onCloseDatabase.addHandler(this._onDBClose.bind(this));
services.databaseEvents.onDatabaseInitialisation.addHandler(this._everyOnInitializeDatabase.bind(this));
services.databaseEvents.onResetDatabase.addHandler(this._everyOnResetDatabase.bind(this));
(services.database as ObsidianDatabaseService).openSimpleStore.setHandler(this._getSimpleStore.bind(this));
services.appLifecycle.onSettingLoaded.addHandler(this._everyOnloadAfterLoadSettings.bind(this));
}
}

2
src/modules/essential/ModuleMigration.ts
View File

@@ -353,7 +353,7 @@ export class ModuleMigration extends AbstractModule {
});
return Promise.resolve(true);
}
onBindFunction(core: LiveSyncCore, services: typeof core.services): void {
override onBindFunction(core: LiveSyncCore, services: typeof core.services): void {
super.onBindFunction(core, services);
services.appLifecycle.onLayoutReady.addHandler(this._everyOnLayoutReady.bind(this));
services.appLifecycle.onFirstInitialise.addHandler(this._everyOnFirstInitialize.bind(this));

5
src/modules/essentialObsidian/APILib/ObsHttpHandler.ts
View File

@@ -28,7 +28,10 @@ export class ObsHttpHandler extends FetchHttpHandler {
this.reverseProxyNoSignUrl = reverseProxyNoSignUrl;
}
// eslint-disable-next-line require-await
async handle(request: HttpRequest, { abortSignal }: HttpHandlerOptions = {}): Promise<{ response: HttpResponse }> {
override async handle(
request: HttpRequest,
{ abortSignal }: HttpHandlerOptions = {}
): Promise<{ response: HttpResponse }> {
if (abortSignal?.aborted) {
const abortError = new Error("Request aborted");
abortError.name = "AbortError";

2
src/modules/essentialObsidian/ModuleCheckRemoteSize.ts
View File

@@ -127,7 +127,7 @@ export class ModuleCheckRemoteSize extends AbstractModule {
eventHub.onEvent(EVENT_REQUEST_CHECK_REMOTE_SIZE, () => this.checkRemoteSize());
return Promise.resolve(true);
}
onBindFunction(core: LiveSyncCore, services: typeof core.services): void {
override onBindFunction(core: LiveSyncCore, services: typeof core.services): void {
services.appLifecycle.onScanningStartupIssues.addHandler(this._allScanStat.bind(this));
services.appLifecycle.onInitialise.addHandler(this._everyOnloadStart.bind(this));
}

19
src/modules/essentialObsidian/ModuleObsidianAPI.ts
View File

@@ -10,14 +10,15 @@ import {
import { Notice, requestUrl, type RequestUrlParam, type RequestUrlResponse } from "../../deps.ts";
import { type CouchDBCredentials, type EntryDoc } from "../../lib/src/common/types.ts";
import { isCloudantURI, isValidRemoteCouchDBURI } from "../../lib/src/pouchdb/utils_couchdb.ts";
import { replicationFilter } from "@/lib/src/pouchdb/compress.ts";
import { disableEncryption } from "@/lib/src/pouchdb/encryption.ts";
import { enableEncryption } from "@/lib/src/pouchdb/encryption.ts";
import { replicationFilter } from "@lib/pouchdb/compress.ts";
import { disableEncryption } from "@lib/pouchdb/encryption.ts";
import { enableEncryption } from "@lib/pouchdb/encryption.ts";
import { setNoticeClass } from "../../lib/src/mock_and_interop/wrapper.ts";
import { PouchDB } from "../../lib/src/pouchdb/pouchdb-browser.ts";
import { AuthorizationHeaderGenerator } from "../../lib/src/replication/httplib.ts";
import type { LiveSyncCore } from "../../main.ts";
import { EVENT_ON_UNRESOLVED_ERROR, eventHub } from "../../common/events.ts";
import { MARK_LOG_NETWORK_ERROR } from "@lib/services/lib/logUtils.ts";
setNoticeClass(Notice);
@@ -96,7 +97,7 @@ export class ModuleObsidianAPI extends AbstractObsidianModule {
const size = body ? ` (${body.length})` : "";
try {
const r = await this.__fetchByAPI(url, authHeader, opts);
this.plugin.requestCount.value = this.plugin.requestCount.value + 1;
this.services.API.requestCount.value = this.services.API.requestCount.value + 1;
if (method == "POST" || method == "PUT") {
this.last_successful_post = r.status - (r.status % 100) == 200;
} else {
@@ -113,7 +114,7 @@ export class ModuleObsidianAPI extends AbstractObsidianModule {
this._log(ex);
throw ex;
} finally {
this.plugin.responseCount.value = this.plugin.responseCount.value + 1;
this.services.API.responseCount.value = this.services.API.responseCount.value + 1;
}
}
@@ -171,7 +172,7 @@ export class ModuleObsidianAPI extends AbstractObsidianModule {
headers.append("authorization", authHeader);
}
try {
this.plugin.requestCount.value = this.plugin.requestCount.value + 1;
this.services.API.requestCount.value = this.services.API.requestCount.value + 1;
const response: Response = await (useRequestAPI
? this.__fetchByAPI(url.toString(), authHeader, { ...opts, headers })
: fetch(url, { ...opts, headers }));
@@ -236,7 +237,7 @@ export class ModuleObsidianAPI extends AbstractObsidianModule {
} catch (ex: any) {
this._log(`HTTP:${method}${size} to:${localURL} -> failed`, LOG_LEVEL_VERBOSE);
const msg = ex instanceof Error ? `${ex?.name}:${ex?.message}` : ex?.toString();
this.showError(`Failed to fetch: ${msg}`); // Do not show notice, due to throwing below
this.showError(`${MARK_LOG_NETWORK_ERROR}Network Error: Failed to fetch: ${msg}`); // Do not show notice, due to throwing below
this._log(ex, LOG_LEVEL_VERBOSE);
// limit only in bulk_docs.
if (url.toString().indexOf("_bulk_docs") !== -1) {
@@ -245,7 +246,7 @@ export class ModuleObsidianAPI extends AbstractObsidianModule {
this._log(ex);
throw ex;
} finally {
this.plugin.responseCount.value = this.plugin.responseCount.value + 1;
this.services.API.responseCount.value = this.services.API.responseCount.value + 1;
}
// return await fetch(url, opts);
@@ -282,7 +283,7 @@ export class ModuleObsidianAPI extends AbstractObsidianModule {
return Promise.resolve([...this._previousErrors]);
}
onBindFunction(core: LiveSyncCore, services: typeof core.services) {
override onBindFunction(core: LiveSyncCore, services: typeof core.services) {
services.API.isLastPostFailedDueToPayloadSize.setHandler(this._getLastPostFailedBySize.bind(this));
services.remote.connect.setHandler(this._connectRemoteCouchDB.bind(this));
services.appLifecycle.getUnresolvedMessages.addHandler(this._reportUnresolvedMessages.bind(this));

35
src/modules/essentialObsidian/ModuleObsidianEvents.ts
View File

@@ -5,7 +5,7 @@ import { scheduleTask } from "octagonal-wheels/concurrency/task";
import { type TFile } from "../../deps.ts";
import { fireAndForget } from "octagonal-wheels/promises";
import { type FilePathWithPrefix } from "../../lib/src/common/types.ts";
import { reactive, reactiveSource } from "octagonal-wheels/dataobject/reactive";
import { reactive, reactiveSource, type ReactiveSource } from "octagonal-wheels/dataobject/reactive";
import {
collectingChunks,
pluginScanningCount,
@@ -45,7 +45,7 @@ export class ModuleObsidianEvents extends AbstractObsidianModule {
this.initialCallback = save;
saveCommandDefinition.callback = () => {
scheduleTask("syncOnEditorSave", 250, () => {
if (this.services.appLifecycle.hasUnloaded()) {
if (this.services.control.hasUnloaded()) {
this._log("Unload and remove the handler.", LOG_LEVEL_VERBOSE);
saveCommandDefinition.callback = this.initialCallback;
this.initialCallback = undefined;
@@ -188,20 +188,25 @@ export class ModuleObsidianEvents extends AbstractObsidianModule {
}
});
}
// TODO: separate
// Process counting for app reload scheduling
_totalProcessingCount?: ReactiveSource<number> = undefined;
private _scheduleAppReload() {
if (!this.core._totalProcessingCount) {
if (!this._totalProcessingCount) {
const __tick = reactiveSource(0);
this.core._totalProcessingCount = reactive(() => {
const dbCount = this.core.databaseQueueCount.value;
const replicationCount = this.core.replicationResultCount.value;
const storageApplyingCount = this.core.storageApplyingCount.value;
this._totalProcessingCount = reactive(() => {
const dbCount = this.services.replication.databaseQueueCount.value;
const replicationCount = this.services.replication.replicationResultCount.value;
const storageApplyingCount = this.services.replication.storageApplyingCount.value;
const chunkCount = collectingChunks.value;
const pluginScanCount = pluginScanningCount.value;
const hiddenFilesCount = hiddenFilesEventCount.value + hiddenFilesProcessingCount.value;
const conflictProcessCount = this.core.conflictProcessQueueCount.value;
const e = this.core.pendingFileEventCount.value;
const proc = this.core.processingFileEventCount.value;
const conflictProcessCount = this.services.conflict.conflictProcessQueueCount.value;
// Now no longer `pendingFileEventCount` and `processingFileEventCount` is used
// const e = this.core.pendingFileEventCount.value;
// const proc = this.core.processingFileEventCount.value;
const e = 0;
const proc = 0;
// eslint-disable-next-line @typescript-eslint/no-unused-vars
const __ = __tick.value;
return (
@@ -223,7 +228,7 @@ export class ModuleObsidianEvents extends AbstractObsidianModule {
);
let stableCheck = 3;
this.core._totalProcessingCount.onChanged((e) => {
this._totalProcessingCount.onChanged((e) => {
if (e.value == 0) {
if (stableCheck-- <= 0) {
this.__performAppReload();
@@ -239,10 +244,14 @@ export class ModuleObsidianEvents extends AbstractObsidianModule {
});
}
}
onBindFunction(core: LiveSyncCore, services: typeof core.services): void {
_isReloadingScheduled(): boolean {
return this._totalProcessingCount !== undefined;
}
override onBindFunction(core: LiveSyncCore, services: typeof core.services): void {
services.appLifecycle.onLayoutReady.addHandler(this._everyOnLayoutReady.bind(this));
services.appLifecycle.onInitialise.addHandler(this._everyOnloadStart.bind(this));
services.appLifecycle.askRestart.setHandler(this._askReload.bind(this));
services.appLifecycle.scheduleRestart.setHandler(this._scheduleAppReload.bind(this));
services.appLifecycle.isReloadingScheduled.setHandler(this._isReloadingScheduled.bind(this));
}
}

6
src/modules/essentialObsidian/ModuleObsidianMenu.ts
View File

@@ -59,7 +59,7 @@ export class ModuleObsidianMenu extends AbstractModule {
this.settings.liveSync = true;
this._log("LiveSync Enabled.", LOG_LEVEL_NOTICE);
}
await this.services.setting.realiseSetting();
await this.services.control.applySettings();
await this.services.setting.saveSettingData();
},
});
@@ -74,7 +74,7 @@ export class ModuleObsidianMenu extends AbstractModule {
this.services.appLifecycle.setSuspended(true);
this._log("Self-hosted LiveSync suspended", LOG_LEVEL_NOTICE);
}
await this.services.setting.realiseSetting();
await this.services.control.applySettings();
await this.services.setting.saveSettingData();
},
});
@@ -106,7 +106,7 @@ export class ModuleObsidianMenu extends AbstractModule {
return Promise.resolve(true);
}
onBindFunction(core: LiveSyncCore, services: typeof core.services): void {
override onBindFunction(core: LiveSyncCore, services: typeof core.services): void {
services.appLifecycle.onInitialise.addHandler(this._everyOnloadStart.bind(this));
}
}

18
src/modules/extraFeaturesObsidian/ModuleExtraSyncObsidian.ts
View File

@@ -1,18 +0,0 @@
import type { LiveSyncCore } from "../../main.ts";
import { AbstractObsidianModule } from "../AbstractObsidianModule.ts";
export class ModuleExtraSyncObsidian extends AbstractObsidianModule {
deviceAndVaultName: string = "";
_getDeviceAndVaultName(): string {
return this.deviceAndVaultName;
}
_setDeviceAndVaultName(name: string): void {
this.deviceAndVaultName = name;
}
onBindFunction(core: LiveSyncCore, services: typeof core.services): void {
services.setting.getDeviceAndVaultName.setHandler(this._getDeviceAndVaultName.bind(this));
services.setting.setDeviceAndVaultName.setHandler(this._setDeviceAndVaultName.bind(this));
}
}

2
src/modules/extras/ModuleDev.ts
View File

@@ -156,7 +156,7 @@ export class ModuleDev extends AbstractObsidianModule {
// this.addTestResult("Test of test3", true);
return this.testDone();
}
onBindFunction(core: LiveSyncCore, services: typeof core.services): void {
override onBindFunction(core: LiveSyncCore, services: typeof core.services): void {
services.appLifecycle.onLayoutReady.addHandler(this._everyOnLayoutReady.bind(this));
services.appLifecycle.onInitialise.addHandler(this._everyOnloadStart.bind(this));
services.appLifecycle.onSettingLoaded.addHandler(this._everyOnloadAfterLoadSettings.bind(this));

2
src/modules/extras/ModuleIntegratedTest.ts
View File

@@ -440,7 +440,7 @@ Line4:D`;
return Promise.resolve(true);
}
onBindFunction(core: typeof this.core, services: typeof core.services): void {
override onBindFunction(core: typeof this.core, services: typeof core.services): void {
services.test.testMultiDevice.addHandler(this._everyModuleTestMultiDevice.bind(this));
}
}

7
src/modules/extras/ModuleReplicateTest.ts
View File

@@ -511,10 +511,11 @@ ABCDEFGHIJKLMNOPQRSTUVWXYZ`;
return this.__assertStorageContent((this.testRootPath + "task.md") as FilePath, mergedDoc, false, true);
}
// No longer tested
async checkConflictResolution() {
this._log("Before testing conflicted files, resolve all once", LOG_LEVEL_NOTICE);
await this.core.rebuilder.resolveAllConflictedFilesByNewerOnes();
await this.core.rebuilder.resolveAllConflictedFilesByNewerOnes();
await this.services.conflict.resolveAllConflictedFilesByNewerOnes();
await this.services.conflict.resolveAllConflictedFilesByNewerOnes();
await this.services.replication.replicate();
await delay(1000);
if (!(await this.testConflictAutomatic())) {
@@ -580,7 +581,7 @@ ABCDEFGHIJKLMNOPQRSTUVWXYZ`;
await this._test("Conflict resolution", async () => await this.checkConflictResolution());
return this.testDone();
}
onBindFunction(core: LiveSyncCore, services: typeof core.services): void {
override onBindFunction(core: LiveSyncCore, services: typeof core.services): void {
services.appLifecycle.onSettingLoaded.addHandler(this._everyOnloadAfterLoadSettings.bind(this));
services.replication.onBeforeReplicate.addHandler(this._everyBeforeReplicate.bind(this));
services.test.testMultiDevice.addHandler(this._everyModuleTestMultiDevice.bind(this));

10
src/modules/extras/devUtil/TestPaneView.ts
View File

@@ -8,11 +8,11 @@ export class TestPaneView extends ItemView {
component?: TestPaneComponent;
plugin: ObsidianLiveSyncPlugin;
moduleDev: ModuleDev;
icon = "view-log";
override icon = "view-log";
title: string = "Self-hosted LiveSync Test and Results";
navigation = true;
override navigation = true;
getIcon(): string {
override getIcon(): string {
return "view-log";
}
@@ -30,7 +30,7 @@ export class TestPaneView extends ItemView {
return "Self-hosted LiveSync Test and Results";
}
async onOpen() {
override async onOpen() {
this.component = new TestPaneComponent({
target: this.contentEl,
props: {
@@ -41,7 +41,7 @@ export class TestPaneView extends ItemView {
await Promise.resolve();
}
async onClose() {
override async onClose() {
this.component?.$destroy();
await Promise.resolve();
}

4
src/modules/features/DocumentHistory/DocumentHistoryModal.ts
View File

@@ -214,7 +214,7 @@ export class DocumentHistoryModal extends Modal {
}
}
onOpen() {
override onOpen() {
const { contentEl } = this;
this.titleEl.setText("Document History");
contentEl.empty();
@@ -299,7 +299,7 @@ export class DocumentHistoryModal extends Modal {
});
});
}
onClose() {
override onClose() {
const { contentEl } = this;
contentEl.empty();
this.BlobURLs.forEach((value) => {

6
src/modules/features/GlobalHistory/GlobalHistoryView.ts
View File

@@ -16,11 +16,11 @@ export class GlobalHistoryView extends SvelteItemView {
}
plugin: ObsidianLiveSyncPlugin;
icon = "clock";
override icon = "clock";
title: string = "";
navigation = true;
override navigation = true;
getIcon(): string {
override getIcon(): string {
return "clock";
}

4
src/modules/features/InteractiveConflictResolving/ConflictResolveModal.ts
View File

@@ -44,7 +44,7 @@ export class ConflictResolveModal extends Modal {
// sendValue("close-resolve-conflict:" + this.filename, false);
}
onOpen() {
override onOpen() {
const { contentEl } = this;
// Send cancel signal for the previous merge dialogue
// if not there, simply be ignored.
@@ -119,7 +119,7 @@ export class ConflictResolveModal extends Modal {
this.close();
}
onClose() {
override onClose() {
const { contentEl } = this;
contentEl.empty();
if (this.offEvent) {

6
src/modules/features/Log/LogPaneView.ts
View File

@@ -19,11 +19,11 @@ export class LogPaneView extends SvelteItemView {
}
plugin: ObsidianLiveSyncPlugin;
icon = "view-log";
override icon = "view-log";
title: string = "";
navigation = false;
override navigation = false;
getIcon(): string {
override getIcon(): string {
return "view-log";
}

2
src/modules/features/ModuleGlobalHistory.ts
View File

@@ -19,7 +19,7 @@ export class ModuleObsidianGlobalHistory extends AbstractObsidianModule {
showGlobalHistory() {
void this.services.API.showWindow(VIEW_TYPE_GLOBAL_HISTORY);
}
onBindFunction(core: typeof this.core, services: typeof core.services): void {
override onBindFunction(core: typeof this.core, services: typeof core.services): void {
services.appLifecycle.onInitialise.addHandler(this._everyOnloadStart.bind(this));
}
}

2
src/modules/features/ModuleInteractiveConflictResolver.ts
View File

@@ -169,7 +169,7 @@ export class ModuleInteractiveConflictResolver extends AbstractObsidianModule {
}
return true;
}
onBindFunction(core: LiveSyncCore, services: typeof core.services): void {
override onBindFunction(core: LiveSyncCore, services: typeof core.services): void {
services.appLifecycle.onScanningStartupIssues.addHandler(this._allScanStat.bind(this));
services.appLifecycle.onInitialise.addHandler(this._everyOnloadStart.bind(this));
services.conflict.resolveByUserInteraction.addHandler(this._anyResolveConflictByUI.bind(this));

52
src/modules/features/ModuleLog.ts
View File

@@ -32,26 +32,30 @@ import { serialized } from "octagonal-wheels/concurrency/lock";
import { $msg } from "src/lib/src/common/i18n.ts";
import { P2PLogCollector } from "../../lib/src/replication/trystero/P2PReplicatorCore.ts";
import type { LiveSyncCore } from "../../main.ts";
import { LiveSyncError } from "@/lib/src/common/LSError.ts";
import { LiveSyncError } from "@lib/common/LSError.ts";
import { isValidPath } from "@/common/utils.ts";
import {
isValidFilenameInAndroid,
isValidFilenameInDarwin,
isValidFilenameInWidows,
} from "@/lib/src/string_and_binary/path.ts";
} from "@lib/string_and_binary/path.ts";
import { MARK_LOG_NETWORK_ERROR, MARK_LOG_SEPARATOR } from "@lib/services/lib/logUtils.ts";
import { NetworkWarningStyles } from "@lib/common/models/setting.const.ts";
// This module cannot be a core module because it depends on the Obsidian UI.
// DI the log again.
const recentLogEntries = reactiveSource<LogEntry[]>([]);
setGlobalLogFunction((message: any, level?: number, key?: string) => {
const globalLogFunction = (message: any, level?: number, key?: string) => {
const messageX =
message instanceof Error
? new LiveSyncError("[Error Logged]: " + message.message, { cause: message })
: message;
const entry = { message: messageX, level, key } as LogEntry;
recentLogEntries.value = [...recentLogEntries.value, entry];
});
};
setGlobalLogFunction(globalLogFunction);
let recentLogs = [] as string[];
function addLog(log: string) {
@@ -99,12 +103,12 @@ export class ModuleLog extends AbstractObsidianModule {
});
return computed(() => formatted.value);
}
const labelReplication = padLeftSpComputed(this.core.replicationResultCount, `📥`);
const labelDBCount = padLeftSpComputed(this.core.databaseQueueCount, `📄`);
const labelStorageCount = padLeftSpComputed(this.core.storageApplyingCount, `💾`);
const labelReplication = padLeftSpComputed(this.services.replication.replicationResultCount, `📥`);
const labelDBCount = padLeftSpComputed(this.services.replication.databaseQueueCount, `📄`);
const labelStorageCount = padLeftSpComputed(this.services.replication.storageApplyingCount, `💾`);
const labelChunkCount = padLeftSpComputed(collectingChunks, `🧩`);
const labelPluginScanCount = padLeftSpComputed(pluginScanningCount, `🔌`);
const labelConflictProcessCount = padLeftSpComputed(this.core.conflictProcessQueueCount, `🔩`);
const labelConflictProcessCount = padLeftSpComputed(this.services.conflict.conflictProcessQueueCount, `🔩`);
const hiddenFilesCount = reactive(() => hiddenFilesEventCount.value - hiddenFilesProcessingCount.value);
const labelHiddenFilesCount = padLeftSpComputed(hiddenFilesCount, `⚙️`);
const queueCountLabelX = reactive(() => {
@@ -113,12 +117,12 @@ export class ModuleLog extends AbstractObsidianModule {
const queueCountLabel = () => queueCountLabelX.value;
const requestingStatLabel = computed(() => {
const diff = this.core.requestCount.value - this.core.responseCount.value;
const diff = this.services.API.requestCount.value - this.services.API.responseCount.value;
return diff != 0 ? "📲 " : "";
});
const replicationStatLabel = computed(() => {
const e = this.core.replicationStat.value;
const e = this.services.replicator.replicationStatics.value;
const sent = e.sent;
const arrived = e.arrived;
const maxPullSeq = e.maxPullSeq;
@@ -170,9 +174,9 @@ export class ModuleLog extends AbstractObsidianModule {
}
return { w, sent, pushLast, arrived, pullLast };
});
const labelProc = padLeftSpComputed(this.core.processing, `⏳`);
const labelPend = padLeftSpComputed(this.core.totalQueued, `🛫`);
const labelInBatchDelay = padLeftSpComputed(this.core.batched, `📬`);
const labelProc = padLeftSpComputed(this.services.fileProcessing.processing, `⏳`);
const labelPend = padLeftSpComputed(this.services.fileProcessing.totalQueued, `🛫`);
const labelInBatchDelay = padLeftSpComputed(this.services.fileProcessing.batched, `📬`);
const waitingLabel = computed(() => {
return `${labelProc()}${labelPend()}${labelInBatchDelay()}`;
});
@@ -279,7 +283,20 @@ export class ModuleLog extends AbstractObsidianModule {
const fileStatus = this.activeFileStatus.value;
if (fileStatus && !this.settings.hideFileWarningNotice) messageLines.push(fileStatus);
const messages = (await this.services.appLifecycle.getUnresolvedMessages()).flat().filter((e) => e);
messageLines.push(...messages);
const stringMessages = messages.filter((m): m is string => typeof m === "string"); // for 'startsWith'
const networkMessages = stringMessages.filter((m) => m.startsWith(MARK_LOG_NETWORK_ERROR));
const otherMessages = stringMessages.filter((m) => !m.startsWith(MARK_LOG_NETWORK_ERROR));
messageLines.push(...otherMessages);
if (
this.settings.networkWarningStyle !== NetworkWarningStyles.ICON &&
this.settings.networkWarningStyle !== NetworkWarningStyles.HIDDEN
) {
messageLines.push(...networkMessages);
} else if (this.settings.networkWarningStyle === NetworkWarningStyles.ICON) {
if (networkMessages.length > 0) messageLines.push("🔗❌");
}
this.messageArea.innerText = messageLines.map((e) => `⚠️ ${e}`).join("\n");
}
}
@@ -304,9 +321,9 @@ export class ModuleLog extends AbstractObsidianModule {
// const recent = logMessages.value;
const newMsg = message;
let newLog = this.settings?.showOnlyIconsOnEditor ? "" : status;
const moduleTagEnd = newLog.indexOf(`]\u{200A}`);
const moduleTagEnd = newLog.indexOf(`]${MARK_LOG_SEPARATOR}`);
if (moduleTagEnd != -1) {
newLog = newLog.substring(moduleTagEnd + 2);
newLog = newLog.substring(moduleTagEnd + MARK_LOG_SEPARATOR.length + 1);
}
this.statusBar?.setText(newMsg.split("\n")[0]);
@@ -492,7 +509,8 @@ export class ModuleLog extends AbstractObsidianModule {
}
}
}
onBindFunction(core: LiveSyncCore, services: typeof core.services): void {
override onBindFunction(core: LiveSyncCore, services: typeof core.services): void {
services.API.addLog.setHandler(globalLogFunction);
services.appLifecycle.onInitialise.addHandler(this._everyOnloadStart.bind(this));
services.appLifecycle.onSettingLoaded.addHandler(this._everyOnloadAfterLoadSettings.bind(this));
services.appLifecycle.onLoaded.addHandler(this._everyOnload.bind(this));

2
src/modules/features/ModuleObsidianDocumentHistory.ts
View File

@@ -50,7 +50,7 @@ export class ModuleObsidianDocumentHistory extends AbstractObsidianModule {
this.showHistory(targetId.path, targetId.id);
}
}
onBindFunction(core: typeof this.core, services: typeof core.services): void {
override onBindFunction(core: typeof this.core, services: typeof core.services): void {
services.appLifecycle.onInitialise.addHandler(this._everyOnloadStart.bind(this));
}
}

335
src/modules/features/ModuleObsidianSetting.ts
View File

@@ -1,335 +0,0 @@
// import { PouchDB } from "../../lib/src/pouchdb/pouchdb-browser";
import { EVENT_REQUEST_RELOAD_SETTING_TAB, EVENT_SETTING_SAVED, eventHub } from "../../common/events.ts";
import {
type BucketSyncSetting,
ChunkAlgorithmNames,
type ConfigPassphraseStore,
type CouchDBConnection,
DEFAULT_SETTINGS,
type ObsidianLiveSyncSettings,
SALT_OF_PASSPHRASE,
SETTING_KEY_P2P_DEVICE_NAME,
} from "../../lib/src/common/types";
import { LOG_LEVEL_NOTICE, LOG_LEVEL_URGENT } from "octagonal-wheels/common/logger";
import { $msg, setLang } from "../../lib/src/common/i18n.ts";
import { isCloudantURI } from "../../lib/src/pouchdb/utils_couchdb.ts";
import { getLanguage } from "@/deps.ts";
import { SUPPORTED_I18N_LANGS, type I18N_LANGS } from "../../lib/src/common/rosetta.ts";
import { decryptString, encryptString } from "@/lib/src/encryption/stringEncryption.ts";
import type { LiveSyncCore } from "../../main.ts";
import { AbstractModule } from "../AbstractModule.ts";
export class ModuleObsidianSettings extends AbstractModule {
async _everyOnLayoutReady(): Promise<boolean> {
let isChanged = false;
if (this.settings.displayLanguage == "") {
const obsidianLanguage = getLanguage();
if (
SUPPORTED_I18N_LANGS.indexOf(obsidianLanguage) !== -1 && // Check if the language is supported
obsidianLanguage != this.settings.displayLanguage // Check if the language is different from the current setting
) {
// Check if the current setting is not empty (Means migrated or installed).
this.settings.displayLanguage = obsidianLanguage as I18N_LANGS;
isChanged = true;
setLang(this.settings.displayLanguage);
} else if (this.settings.displayLanguage == "") {
this.settings.displayLanguage = "def";
setLang(this.settings.displayLanguage);
await this.services.setting.saveSettingData();
}
}
if (isChanged) {
const revert = $msg("dialog.yourLanguageAvailable.btnRevertToDefault");
if (
(await this.core.confirm.askSelectStringDialogue($msg(`dialog.yourLanguageAvailable`), ["OK", revert], {
defaultAction: "OK",
title: $msg(`dialog.yourLanguageAvailable.Title`),
})) == revert
) {
this.settings.displayLanguage = "def";
setLang(this.settings.displayLanguage);
}
await this.services.setting.saveSettingData();
}
return true;
}
getPassphrase(settings: ObsidianLiveSyncSettings) {
const methods: Record<ConfigPassphraseStore, () => Promise<string | false>> = {
"": () => Promise.resolve("*"),
LOCALSTORAGE: () => Promise.resolve(localStorage.getItem("ls-setting-passphrase") ?? false),
ASK_AT_LAUNCH: () => this.core.confirm.askString("Passphrase", "passphrase", ""),
};
const method = settings.configPassphraseStore;
const methodFunc = method in methods ? methods[method] : methods[""];
return methodFunc();
}
_saveDeviceAndVaultName(): void {
const lsKey = "obsidian-live-sync-vaultanddevicename-" + this.services.vault.getVaultName();
localStorage.setItem(lsKey, this.services.setting.getDeviceAndVaultName() || "");
}
usedPassphrase = "";
private _clearUsedPassphrase(): void {
this.usedPassphrase = "";
}
async decryptConfigurationItem(encrypted: string, passphrase: string) {
const dec = await decryptString(encrypted, passphrase + SALT_OF_PASSPHRASE);
if (dec) {
this.usedPassphrase = passphrase;
return dec;
}
return false;
}
async encryptConfigurationItem(src: string, settings: ObsidianLiveSyncSettings) {
if (this.usedPassphrase != "") {
return await encryptString(src, this.usedPassphrase + SALT_OF_PASSPHRASE);
}
const passphrase = await this.getPassphrase(settings);
if (passphrase === false) {
this._log(
"Failed to obtain passphrase when saving data.json! Please verify the configuration.",
LOG_LEVEL_URGENT
);
return "";
}
const dec = await encryptString(src, passphrase + SALT_OF_PASSPHRASE);
if (dec) {
this.usedPassphrase = passphrase;
return dec;
}
return "";
}
get appId() {
return this.services.API.getAppID();
}
async _saveSettingData() {
this.services.setting.saveDeviceAndVaultName();
const settings = { ...this.settings };
settings.deviceAndVaultName = "";
if (settings.P2P_DevicePeerName && settings.P2P_DevicePeerName.trim() !== "") {
console.log("Saving device peer name to small config");
this.services.config.setSmallConfig(SETTING_KEY_P2P_DEVICE_NAME, settings.P2P_DevicePeerName.trim());
settings.P2P_DevicePeerName = "";
}
if (this.usedPassphrase == "" && !(await this.getPassphrase(settings))) {
this._log("Failed to retrieve passphrase. data.json contains unencrypted items!", LOG_LEVEL_NOTICE);
} else {
if (
settings.couchDB_PASSWORD != "" ||
settings.couchDB_URI != "" ||
settings.couchDB_USER != "" ||
settings.couchDB_DBNAME
) {
const connectionSetting: CouchDBConnection & BucketSyncSetting = {
couchDB_DBNAME: settings.couchDB_DBNAME,
couchDB_PASSWORD: settings.couchDB_PASSWORD,
couchDB_URI: settings.couchDB_URI,
couchDB_USER: settings.couchDB_USER,
accessKey: settings.accessKey,
bucket: settings.bucket,
endpoint: settings.endpoint,
region: settings.region,
secretKey: settings.secretKey,
useCustomRequestHandler: settings.useCustomRequestHandler,
bucketCustomHeaders: settings.bucketCustomHeaders,
couchDB_CustomHeaders: settings.couchDB_CustomHeaders,
useJWT: settings.useJWT,
jwtKey: settings.jwtKey,
jwtAlgorithm: settings.jwtAlgorithm,
jwtKid: settings.jwtKid,
jwtExpDuration: settings.jwtExpDuration,
jwtSub: settings.jwtSub,
useRequestAPI: settings.useRequestAPI,
bucketPrefix: settings.bucketPrefix,
forcePathStyle: settings.forcePathStyle,
};
settings.encryptedCouchDBConnection = await this.encryptConfigurationItem(
JSON.stringify(connectionSetting),
settings
);
settings.couchDB_PASSWORD = "";
settings.couchDB_DBNAME = "";
settings.couchDB_URI = "";
settings.couchDB_USER = "";
settings.accessKey = "";
settings.bucket = "";
settings.region = "";
settings.secretKey = "";
settings.endpoint = "";
}
if (settings.encrypt && settings.passphrase != "") {
settings.encryptedPassphrase = await this.encryptConfigurationItem(settings.passphrase, settings);
settings.passphrase = "";
}
}
await this.core.saveData(settings);
eventHub.emitEvent(EVENT_SETTING_SAVED, settings);
}
tryDecodeJson(encoded: string | false): object | false {
try {
if (!encoded) return false;
return JSON.parse(encoded);
} catch {
return false;
}
}
async _decryptSettings(settings: ObsidianLiveSyncSettings): Promise<ObsidianLiveSyncSettings> {
const passphrase = await this.getPassphrase(settings);
if (passphrase === false) {
this._log("No passphrase found for data.json! Verify configuration before syncing.", LOG_LEVEL_URGENT);
} else {
if (settings.encryptedCouchDBConnection) {
const keys = [
"couchDB_URI",
"couchDB_USER",
"couchDB_PASSWORD",
"couchDB_DBNAME",
"accessKey",
"bucket",
"endpoint",
"region",
"secretKey",
] as (keyof CouchDBConnection | keyof BucketSyncSetting)[];
const decrypted = this.tryDecodeJson(
await this.decryptConfigurationItem(settings.encryptedCouchDBConnection, passphrase)
) as CouchDBConnection & BucketSyncSetting;
if (decrypted) {
for (const key of keys) {
if (key in decrypted) {
//@ts-ignore
settings[key] = decrypted[key];
}
}
} else {
this._log(
"Failed to decrypt passphrase from data.json! Ensure configuration is correct before syncing with remote.",
LOG_LEVEL_URGENT
);
for (const key of keys) {
//@ts-ignore
settings[key] = "";
}
}
}
if (settings.encrypt && settings.encryptedPassphrase) {
const encrypted = settings.encryptedPassphrase;
const decrypted = await this.decryptConfigurationItem(encrypted, passphrase);
if (decrypted) {
settings.passphrase = decrypted;
} else {
this._log(
"Failed to decrypt passphrase from data.json! Ensure configuration is correct before syncing with remote.",
LOG_LEVEL_URGENT
);
settings.passphrase = "";
}
}
}
return settings;
}
/**
* This method mutates the settings object.
* @param settings
* @returns
*/
_adjustSettings(settings: ObsidianLiveSyncSettings): Promise<ObsidianLiveSyncSettings> {
// Adjust settings as needed
// Delete this feature to avoid problems on mobile.
settings.disableRequestURI = true;
// GC is disabled.
settings.gcDelay = 0;
// So, use history is always enabled.
settings.useHistory = true;
if ("workingEncrypt" in settings) delete settings.workingEncrypt;
if ("workingPassphrase" in settings) delete settings.workingPassphrase;
// Splitter configurations have been replaced with chunkSplitterVersion.
if (settings.chunkSplitterVersion == "") {
if (settings.enableChunkSplitterV2) {
if (settings.useSegmenter) {
settings.chunkSplitterVersion = "v2-segmenter";
} else {
settings.chunkSplitterVersion = "v2";
}
} else {
settings.chunkSplitterVersion = "";
}
} else if (!(settings.chunkSplitterVersion in ChunkAlgorithmNames)) {
settings.chunkSplitterVersion = "";
}
return Promise.resolve(settings);
}
async _loadSettings(): Promise<void> {
const settings = Object.assign({}, DEFAULT_SETTINGS, await this.core.loadData()) as ObsidianLiveSyncSettings;
if (typeof settings.isConfigured == "undefined") {
// If migrated, mark true
if (JSON.stringify(settings) !== JSON.stringify(DEFAULT_SETTINGS)) {
settings.isConfigured = true;
} else {
settings.additionalSuffixOfDatabaseName = this.appId;
settings.isConfigured = false;
}
}
this.settings = await this.services.setting.decryptSettings(settings);
setLang(this.settings.displayLanguage);
await this.services.setting.adjustSettings(this.settings);
const lsKey = "obsidian-live-sync-vaultanddevicename-" + this.services.vault.getVaultName();
if (this.settings.deviceAndVaultName != "") {
if (!localStorage.getItem(lsKey)) {
this.services.setting.setDeviceAndVaultName(this.settings.deviceAndVaultName);
this.services.setting.saveDeviceAndVaultName();
this.settings.deviceAndVaultName = "";
}
}
if (isCloudantURI(this.settings.couchDB_URI) && this.settings.customChunkSize != 0) {
this._log(
"Configuration issues detected and automatically resolved. However, unsynchronized data may exist. Consider rebuilding if necessary.",
LOG_LEVEL_NOTICE
);
this.settings.customChunkSize = 0;
}
this.services.setting.setDeviceAndVaultName(localStorage.getItem(lsKey) || "");
if (this.services.setting.getDeviceAndVaultName() == "") {
if (this.settings.usePluginSync) {
this._log("Device name missing. Disabling plug-in sync.", LOG_LEVEL_NOTICE);
this.settings.usePluginSync = false;
}
}
// this.core.ignoreFiles = this.settings.ignoreFiles.split(",").map(e => e.trim());
eventHub.emitEvent(EVENT_REQUEST_RELOAD_SETTING_TAB);
}
private _currentSettings(): ObsidianLiveSyncSettings {
return this.settings;
}
onBindFunction(core: LiveSyncCore, services: typeof core.services): void {
super.onBindFunction(core, services);
services.appLifecycle.onLayoutReady.addHandler(this._everyOnLayoutReady.bind(this));
services.setting.clearUsedPassphrase.setHandler(this._clearUsedPassphrase.bind(this));
services.setting.decryptSettings.setHandler(this._decryptSettings.bind(this));
services.setting.adjustSettings.setHandler(this._adjustSettings.bind(this));
services.setting.loadSettings.setHandler(this._loadSettings.bind(this));
services.setting.currentSettings.setHandler(this._currentSettings.bind(this));
services.setting.saveDeviceAndVaultName.setHandler(this._saveDeviceAndVaultName.bind(this));
services.setting.saveSettingData.setHandler(this._saveSettingData.bind(this));
}
}

6
src/modules/features/ModuleObsidianSettingAsMarkdown.ts
View File

@@ -6,8 +6,8 @@ import { DEFAULT_SETTINGS, type FilePathWithPrefix, type ObsidianLiveSyncSetting
import { parseYaml, stringifyYaml } from "../../deps";
import { LOG_LEVEL_DEBUG, LOG_LEVEL_INFO, LOG_LEVEL_NOTICE, LOG_LEVEL_VERBOSE } from "octagonal-wheels/common/logger";
import { AbstractModule } from "../AbstractModule.ts";
import type { ServiceContext } from "@/lib/src/services/base/ServiceBase.ts";
import type { InjectableServiceHub } from "@/lib/src/services/InjectableServices.ts";
import type { ServiceContext } from "@lib/services/base/ServiceBase.ts";
import type { InjectableServiceHub } from "@lib/services/InjectableServices.ts";
import type { LiveSyncCore } from "@/main.ts";
const SETTING_HEADER = "````yaml:livesync-setting\n";
const SETTING_FOOTER = "\n````";
@@ -246,7 +246,7 @@ We can perform a command in this file.
}
}
onBindFunction(core: LiveSyncCore, services: InjectableServiceHub<ServiceContext>): void {
override onBindFunction(core: LiveSyncCore, services: InjectableServiceHub<ServiceContext>): void {
services.appLifecycle.onInitialise.addHandler(this._everyOnloadStart.bind(this));
}
}

2
src/modules/features/ModuleObsidianSettingTab.ts
View File

@@ -29,7 +29,7 @@ export class ModuleObsidianSettingDialogue extends AbstractObsidianModule {
get appId() {
return `${"appId" in this.app ? this.app.appId : ""}`;
}
onBindFunction(core: typeof this.plugin, services: typeof core.services): void {
override onBindFunction(core: typeof this.plugin, services: typeof core.services): void {
services.appLifecycle.onInitialise.addHandler(this._everyOnloadStart.bind(this));
}
}

2
src/modules/features/ModuleSetupObsidian.ts
View File

@@ -194,7 +194,7 @@ export class ModuleSetupObsidian extends AbstractModule {
// }
// }
onBindFunction(core: LiveSyncCore, services: typeof core.services): void {
override onBindFunction(core: LiveSyncCore, services: typeof core.services): void {
services.appLifecycle.onLoaded.addHandler(this._everyOnload.bind(this));
}
}

4
src/modules/features/SettingDialogue/LiveSyncSetting.ts
View File

@@ -58,7 +58,7 @@ export class LiveSyncSetting extends Setting {
}
}
setDesc(desc: string | DocumentFragment): this {
override setDesc(desc: string | DocumentFragment): this {
this.descBuf = desc;
DEV: {
this._createDocStub("desc", desc);
@@ -66,7 +66,7 @@ export class LiveSyncSetting extends Setting {
super.setDesc(desc);
return this;
}
setName(name: string | DocumentFragment): this {
override setName(name: string | DocumentFragment): this {
this.nameBuf = name;
DEV: {
this._createDocStub("name", name);

6
src/modules/features/SettingDialogue/ObsidianLiveSyncSettingTab.ts
View File

@@ -16,7 +16,7 @@ import {
import { delay, isObjectDifferent, sizeToHumanReadable } from "../../../lib/src/common/utils.ts";
import { versionNumberString2Number } from "../../../lib/src/string_and_binary/convert.ts";
import { Logger } from "../../../lib/src/common/logger.ts";
import { checkSyncInfo } from "@/lib/src/pouchdb/negotiation.ts";
import { checkSyncInfo } from "@lib/pouchdb/negotiation.ts";
import { testCrypt } from "octagonal-wheels/encryption/encryption";
import ObsidianLiveSyncPlugin from "../../../main.ts";
import { scheduleTask } from "../../../common/utils.ts";
@@ -374,7 +374,7 @@ export class ObsidianLiveSyncSettingTab extends PluginSettingTab {
this.initialSettings = undefined;
}
hide() {
override hide() {
this.isShown = false;
}
isShown: boolean = false;
@@ -424,8 +424,6 @@ export class ObsidianLiveSyncSettingTab extends PluginSettingTab {
//@ts-ignore
manifestVersion: string = MANIFEST_VERSION || "-";
//@ts-ignore
updateInformation: string = UPDATE_INFO || "";
lastVersion = ~~(versionNumberString2Number(this.manifestVersion) / 1000);

12
src/modules/features/SettingDialogue/PaneGeneral.ts
View File

@@ -4,6 +4,8 @@ import { LiveSyncSetting as Setting } from "./LiveSyncSetting.ts";
import type { ObsidianLiveSyncSettingTab } from "./ObsidianLiveSyncSettingTab.ts";
import type { PageFunctions } from "./SettingPane.ts";
import { visibleOnly } from "./SettingPane.ts";
import { EVENT_ON_UNRESOLVED_ERROR, eventHub } from "@/common/events.ts";
import { NetworkWarningStyles } from "@lib/common/models/setting.const.ts";
export function paneGeneral(
this: ObsidianLiveSyncSettingTab,
paneEl: HTMLElement,
@@ -24,6 +26,16 @@ export function paneGeneral(
});
new Setting(paneEl).autoWireToggle("showStatusOnStatusbar");
new Setting(paneEl).autoWireToggle("hideFileWarningNotice");
new Setting(paneEl).autoWireDropDown("networkWarningStyle", {
options: {
[NetworkWarningStyles.BANNER]: "Show full banner",
[NetworkWarningStyles.ICON]: "Show icon only",
[NetworkWarningStyles.HIDDEN]: "Hide completely",
},
});
this.addOnSaved("networkWarningStyle", () => {
eventHub.emitEvent(EVENT_ON_UNRESOLVED_ERROR);
});
});
void addPanel(paneEl, $msg("obsidianLiveSyncSettingTab.titleLogging")).then((paneEl) => {
paneEl.addClass("wizardHidden");

2
src/modules/features/SettingDialogue/PaneHatch.ts
View File

@@ -361,7 +361,7 @@ ${stringifyYaml({
.setButtonText("Resolve All")
.setCta()
.onClick(async () => {
await this.plugin.rebuilder.resolveAllConflictedFilesByNewerOnes();
await this.services.conflict.resolveAllConflictedFilesByNewerOnes();
})
);

6
src/modules/features/SettingDialogue/PaneMaintenance.ts
View File

@@ -32,7 +32,7 @@ export function paneMaintenance(
(e) => {
e.addEventListener("click", () => {
fireAndForget(async () => {
await this.services.remote.markResolved();
await this.services.replication.markResolved();
this.display();
});
});
@@ -59,7 +59,7 @@ export function paneMaintenance(
(e) => {
e.addEventListener("click", () => {
fireAndForget(async () => {
await this.services.remote.markUnlocked();
await this.services.replication.markUnlocked();
this.display();
});
});
@@ -78,7 +78,7 @@ export function paneMaintenance(
.setDisabled(false)
.setWarning()
.onClick(async () => {
await this.services.remote.markLocked();
await this.services.replication.markLocked();
})
)
.addOnUpdate(this.onlyOnCouchDBOrMinIO);

8
src/modules/features/SettingDialogue/PaneSyncSettings.ts
View File

@@ -105,7 +105,7 @@ export function paneSyncSettings(
if (!this.editingSettings.isConfigured) {
this.editingSettings.isConfigured = true;
await this.saveAllDirtySettings();
await this.services.setting.realiseSetting();
await this.services.control.applySettings();
await this.rebuildDB("localOnly");
// this.resetEditingSettings();
if (
@@ -124,13 +124,13 @@ export function paneSyncSettings(
await this.confirmRebuild();
} else {
await this.saveAllDirtySettings();
await this.services.setting.realiseSetting();
await this.services.control.applySettings();
this.services.appLifecycle.askRestart();
}
}
} else {
await this.saveAllDirtySettings();
await this.services.setting.realiseSetting();
await this.services.control.applySettings();
}
});
});
@@ -169,7 +169,7 @@ export function paneSyncSettings(
}
await this.saveSettings(["liveSync", "periodicReplication"]);
await this.services.setting.realiseSetting();
await this.services.control.applySettings();
});
new Setting(paneEl)

34
src/modules/interfaces/DatabaseFileAccess.ts
View File

@@ -1,34 +0,0 @@
import type {
FilePathWithPrefix,
LoadedEntry,
MetaEntry,
UXFileInfo,
UXFileInfoStub,
} from "../../lib/src/common/types";
export interface DatabaseFileAccess {
delete: (file: UXFileInfoStub | FilePathWithPrefix, rev?: string) => Promise<boolean>;
store: (file: UXFileInfo, force?: boolean, skipCheck?: boolean) => Promise<boolean>;
storeContent(path: FilePathWithPrefix, content: string): Promise<boolean>;
createChunks: (file: UXFileInfo, force?: boolean, skipCheck?: boolean) => Promise<boolean>;
fetch: (
file: UXFileInfoStub | FilePathWithPrefix,
rev?: string,
waitForReady?: boolean,
skipCheck?: boolean
) => Promise<UXFileInfo | false>;
fetchEntryFromMeta: (meta: MetaEntry, waitForReady?: boolean, skipCheck?: boolean) => Promise<LoadedEntry | false>;
fetchEntryMeta: (
file: UXFileInfoStub | FilePathWithPrefix,
rev?: string,
skipCheck?: boolean
) => Promise<MetaEntry | false>;
fetchEntry: (
file: UXFileInfoStub | FilePathWithPrefix,
rev?: string,
waitForReady?: boolean,
skipCheck?: boolean
) => Promise<LoadedEntry | false>;
getConflictedRevs: (file: UXFileInfoStub | FilePathWithPrefix) => Promise<string[]>;
// storeFromStorage: (file: UXFileInfoStub | FilePathWithPrefix, force?: boolean) => Promise<boolean>;
}

12
src/modules/interfaces/DatabaseRebuilder.ts
View File

@@ -1,12 +0,0 @@
export interface Rebuilder {
$performRebuildDB(
method: "localOnly" | "remoteOnly" | "rebuildBothByThisDevice" | "localOnlyWithChunks"
): Promise<void>;
$rebuildRemote(): Promise<void>;
$rebuildEverything(): Promise<void>;
$fetchLocal(makeLocalChunkBeforeSync?: boolean, preventMakeLocalFilesBeforeSync?: boolean): Promise<void>;
scheduleRebuild(): Promise<void>;
scheduleFetch(): Promise<void>;
resolveAllConflictedFilesByNewerOnes(): Promise<void>;
}

61
src/modules/interfaces/StorageAccess.ts
View File

@@ -1,61 +0,0 @@
import type {
FilePath,
FilePathWithPrefix,
UXDataWriteOptions,
UXFileInfo,
UXFileInfoStub,
UXFolderInfo,
UXStat,
} from "../../lib/src/common/types";
import type { CustomRegExp } from "../../lib/src/common/utils";
export interface StorageAccess {
restoreState(): Promise<void>;
processWriteFile<T>(file: UXFileInfoStub | FilePathWithPrefix, proc: () => Promise<T>): Promise<T>;
processReadFile<T>(file: UXFileInfoStub | FilePathWithPrefix, proc: () => Promise<T>): Promise<T>;
isFileProcessing(file: UXFileInfoStub | FilePathWithPrefix): boolean;
deleteVaultItem(file: FilePathWithPrefix | UXFileInfoStub | UXFolderInfo): Promise<void>;
writeFileAuto(path: string, data: string | ArrayBuffer, opt?: UXDataWriteOptions): Promise<boolean>;
readFileAuto(path: string): Promise<string | ArrayBuffer>;
readFileText(path: string): Promise<string>;
isExists(path: string): Promise<boolean>;
writeHiddenFileAuto(path: string, data: string | ArrayBuffer, opt?: UXDataWriteOptions): Promise<boolean>;
appendHiddenFile(path: string, data: string, opt?: UXDataWriteOptions): Promise<boolean>;
stat(path: string): Promise<UXStat | null>;
statHidden(path: string): Promise<UXStat | null>;
removeHidden(path: string): Promise<boolean>;
readHiddenFileAuto(path: string): Promise<string | ArrayBuffer>;
readHiddenFileBinary(path: string): Promise<ArrayBuffer>;
readHiddenFileText(path: string): Promise<string>;
isExistsIncludeHidden(path: string): Promise<boolean>;
// This could be work also for the hidden files.
ensureDir(path: string): Promise<boolean>;
triggerFileEvent(event: string, path: string): void;
triggerHiddenFile(path: string): Promise<void>;
getFileStub(path: string): UXFileInfoStub | null;
readStubContent(stub: UXFileInfoStub): Promise<UXFileInfo | false>;
getStub(path: string): UXFileInfoStub | UXFolderInfo | null;
getFiles(): UXFileInfoStub[];
getFileNames(): FilePathWithPrefix[];
touched(file: UXFileInfoStub | FilePathWithPrefix): Promise<void>;
recentlyTouched(file: UXFileInfoStub | FilePathWithPrefix): boolean;
clearTouched(): void;
// -- Low-Level
delete(file: FilePathWithPrefix | UXFileInfoStub | string, force: boolean): Promise<void>;
trash(file: FilePathWithPrefix | UXFileInfoStub | string, system: boolean): Promise<void>;
getFilesIncludeHidden(
basePath: string,
includeFilter?: CustomRegExp[],
excludeFilter?: CustomRegExp[],
skipFolder?: string[]
): Promise<FilePath[]>;
}

148
src/modules/main/ModuleLiveSyncMain.ts
View File

@@ -3,17 +3,13 @@ import { LOG_LEVEL_NOTICE, LOG_LEVEL_VERBOSE, VER, type ObsidianLiveSyncSettings
import {
EVENT_LAYOUT_READY,
EVENT_PLUGIN_LOADED,
EVENT_PLUGIN_UNLOADED,
EVENT_REQUEST_RELOAD_SETTING_TAB,
EVENT_SETTING_SAVED,
eventHub,
} from "../../common/events.ts";
import { $msg, setLang } from "../../lib/src/common/i18n.ts";
import { versionNumberString2Number } from "../../lib/src/string_and_binary/convert.ts";
import { cancelAllPeriodicTask, cancelAllTasks } from "octagonal-wheels/concurrency/task";
import { stopAllRunningProcessors } from "octagonal-wheels/concurrency/processor";
import { AbstractModule } from "../AbstractModule.ts";
import { EVENT_PLATFORM_UNLOADED } from "@lib/events/coreEvents";
import type { InjectableServiceHub } from "@lib/services/implements/injectable/InjectableServiceHub.ts";
import type { LiveSyncCore } from "../../main.ts";
import { initialiseWorkerModule } from "@lib/worker/bgWorker.ts";
@@ -49,7 +45,7 @@ export class ModuleLiveSyncMain extends AbstractModule {
}
if (!(await this.core.services.appLifecycle.onFirstInitialise())) return false;
// await this.core.$$realizeSettingSyncMode();
await this.services.setting.realiseSetting();
await this.services.control.applySettings();
fireAndForget(async () => {
this._log($msg("moduleLiveSyncMain.logAdditionalSafetyScan"), LOG_LEVEL_VERBOSE);
if (!(await this.services.appLifecycle.onScanningStartupIssues())) {
@@ -65,7 +61,7 @@ export class ModuleLiveSyncMain extends AbstractModule {
eventHub.onEvent(EVENT_SETTING_SAVED, (settings: ObsidianLiveSyncSettings) => {
fireAndForget(async () => {
try {
await this.core.services.setting.realiseSetting();
await this.core.services.control.applySettings();
const lang = this.core.services.setting.currentSettings()?.displayLanguage ?? undefined;
if (lang !== undefined) {
setLang(this.core.services.setting.currentSettings()?.displayLanguage);
@@ -126,7 +122,10 @@ export class ModuleLiveSyncMain extends AbstractModule {
await this.saveSettings();
}
localStorage.setItem(lsKey, `${VER}`);
await this.services.database.openDatabase();
await this.services.database.openDatabase({
databaseEvents: this.services.databaseEvents,
replicator: this.services.replicator,
});
// this.core.$$realizeSettingSyncMode = this.core.$$realizeSettingSyncMode.bind(this);
// this.$$parseReplicationResult = this.$$parseReplicationResult.bind(this);
// this.$$replicate = this.$$replicate.bind(this);
@@ -136,89 +135,82 @@ export class ModuleLiveSyncMain extends AbstractModule {
return true;
}
async _onLiveSyncUnload(): Promise<void> {
eventHub.emitEvent(EVENT_PLUGIN_UNLOADED);
await this.services.appLifecycle.onBeforeUnload();
cancelAllPeriodicTask();
cancelAllTasks();
stopAllRunningProcessors();
await this.services.appLifecycle.onUnload();
this._unloaded = true;
for (const addOn of this.core.addOns) {
addOn.onunload();
}
if (this.localDatabase != null) {
this.localDatabase.onunload();
if (this.core.replicator) {
this.core.replicator?.closeReplication();
}
await this.localDatabase.close();
}
eventHub.emitEvent(EVENT_PLATFORM_UNLOADED);
eventHub.offAll();
this._log($msg("moduleLiveSyncMain.logUnloadingPlugin"));
return;
}
// async _onLiveSyncUnload(): Promise<void> {
// eventHub.emitEvent(EVENT_PLUGIN_UNLOADED);
// await this.services.appLifecycle.onBeforeUnload();
// cancelAllPeriodicTask();
// cancelAllTasks();
// stopAllRunningProcessors();
// await this.services.appLifecycle.onUnload();
// this._unloaded = true;
// for (const addOn of this.core.addOns) {
// addOn.onunload();
// }
// if (this.localDatabase != null) {
// this.localDatabase.onunload();
// if (this.core.replicator) {
// this.core.replicator?.closeReplication();
// }
// await this.localDatabase.close();
// }
// eventHub.emitEvent(EVENT_PLATFORM_UNLOADED);
// eventHub.offAll();
// this._log($msg("moduleLiveSyncMain.logUnloadingPlugin"));
// return;
// }
private async _realizeSettingSyncMode(): Promise<void> {
await this.services.appLifecycle.onSuspending();
await this.services.setting.onBeforeRealiseSetting();
this.localDatabase.refreshSettings();
await this.services.fileProcessing.commitPendingFileEvents();
await this.services.setting.onRealiseSetting();
// disable all sync temporary.
if (this.services.appLifecycle.isSuspended()) return;
await this.services.appLifecycle.onResuming();
await this.services.appLifecycle.onResumed();
await this.services.setting.onSettingRealised();
return;
}
// private async _realizeSettingSyncMode(): Promise<void> {
// await this.services.appLifecycle.onSuspending();
// await this.services.setting.onBeforeRealiseSetting();
// this.localDatabase.refreshSettings();
// await this.services.fileProcessing.commitPendingFileEvents();
// await this.services.setting.onRealiseSetting();
// // disable all sync temporary.
// if (this.services.appLifecycle.isSuspended()) return;
// await this.services.appLifecycle.onResuming();
// await this.services.appLifecycle.onResumed();
// await this.services.setting.onSettingRealised();
// return;
// }
_isReloadingScheduled(): boolean {
return this.core._totalProcessingCount !== undefined;
}
// isReady = false;
isReady = false;
// _isReady(): boolean {
// return this.isReady;
// }
_isReady(): boolean {
return this.isReady;
}
// _markIsReady(): void {
// this.isReady = true;
// }
_markIsReady(): void {
this.isReady = true;
}
// _resetIsReady(): void {
// this.isReady = false;
// }
_resetIsReady(): void {
this.isReady = false;
}
// _suspended = false;
// _isSuspended(): boolean {
// return this._suspended || !this.settings?.isConfigured;
// }
_suspended = false;
_isSuspended(): boolean {
return this._suspended || !this.settings?.isConfigured;
}
// _setSuspended(value: boolean) {
// this._suspended = value;
// }
_setSuspended(value: boolean) {
this._suspended = value;
}
// _unloaded = false;
// _isUnloaded(): boolean {
// return this._unloaded;
// }
_unloaded = false;
_isUnloaded(): boolean {
return this._unloaded;
}
onBindFunction(core: LiveSyncCore, services: InjectableServiceHub): void {
override onBindFunction(core: LiveSyncCore, services: InjectableServiceHub): void {
super.onBindFunction(core, services);
services.appLifecycle.isSuspended.setHandler(this._isSuspended.bind(this));
services.appLifecycle.setSuspended.setHandler(this._setSuspended.bind(this));
services.appLifecycle.isReady.setHandler(this._isReady.bind(this));
services.appLifecycle.markIsReady.setHandler(this._markIsReady.bind(this));
services.appLifecycle.resetIsReady.setHandler(this._resetIsReady.bind(this));
services.appLifecycle.hasUnloaded.setHandler(this._isUnloaded.bind(this));
services.appLifecycle.isReloadingScheduled.setHandler(this._isReloadingScheduled.bind(this));
// services.appLifecycle.isSuspended.setHandler(this._isSuspended.bind(this));
// services.appLifecycle.setSuspended.setHandler(this._setSuspended.bind(this));
// services.appLifecycle.isReady.setHandler(this._isReady.bind(this));
// services.appLifecycle.markIsReady.setHandler(this._markIsReady.bind(this));
// services.appLifecycle.resetIsReady.setHandler(this._resetIsReady.bind(this));
// services.appLifecycle.hasUnloaded.setHandler(this._isUnloaded.bind(this));
services.appLifecycle.onReady.addHandler(this._onLiveSyncReady.bind(this));
services.appLifecycle.onWireUpEvents.addHandler(this._wireUpEvents.bind(this));
services.appLifecycle.onLoad.addHandler(this._onLiveSyncLoad.bind(this));
services.appLifecycle.onAppUnload.addHandler(this._onLiveSyncUnload.bind(this));
services.setting.realiseSetting.setHandler(this._realizeSettingSyncMode.bind(this));
}
}

108
src/modules/services/ObsidianAPIService.ts Normal file
View File

@@ -0,0 +1,108 @@
import { InjectableAPIService } from "@lib/services/implements/injectable/InjectableAPIService";
import type { ObsidianServiceContext } from "@lib/services/implements/obsidian/ObsidianServiceContext";
import { Platform, type Command, type ViewCreator } from "obsidian";
import { ObsHttpHandler } from "../essentialObsidian/APILib/ObsHttpHandler";
import { ObsidianConfirm } from "./ObsidianConfirm";
import type { Confirm } from "@lib/interfaces/Confirm";
// All Services will be migrated to be based on Plain Services, not Injectable Services.
// This is a migration step.
export class ObsidianAPIService extends InjectableAPIService<ObsidianServiceContext> {
_customHandler: ObsHttpHandler | undefined;
_confirmInstance: Confirm;
constructor(context: ObsidianServiceContext) {
super(context);
this._confirmInstance = new ObsidianConfirm(context);
}
getCustomFetchHandler(): ObsHttpHandler {
if (!this._customHandler) this._customHandler = new ObsHttpHandler(undefined, undefined);
return this._customHandler;
}
async showWindow(viewType: string): Promise<void> {
const leaves = this.app.workspace.getLeavesOfType(viewType);
if (leaves.length == 0) {
await this.app.workspace.getLeaf(true).setViewState({
type: viewType,
active: true,
});
} else {
await leaves[0].setViewState({
type: viewType,
active: true,
});
}
if (leaves.length > 0) {
await this.app.workspace.revealLeaf(leaves[0]);
}
}
private get app() {
return this.context.app;
}
override getPlatform(): string {
if (Platform.isAndroidApp) {
return "android-app";
} else if (Platform.isIosApp) {
return "ios";
} else if (Platform.isMacOS) {
return "macos";
} else if (Platform.isMobileApp) {
return "mobile-app";
} else if (Platform.isMobile) {
return "mobile";
} else if (Platform.isSafari) {
return "safari";
} else if (Platform.isDesktop) {
return "desktop";
} else if (Platform.isDesktopApp) {
return "desktop-app";
} else {
return "unknown-obsidian";
}
}
override isMobile(): boolean {
//@ts-ignore : internal API
return this.app.isMobile;
}
override getAppID(): string {
return `${"appId" in this.app ? this.app.appId : ""}`;
}
override getSystemVaultName(): string {
return this.app.vault.getName();
}
override getAppVersion(): string {
const navigatorString = globalThis.navigator?.userAgent ?? "";
const match = navigatorString.match(/obsidian\/([0-9]+\.[0-9]+\.[0-9]+)/);
if (match && match.length >= 2) {
return match[1];
}
return "0.0.0";
}
override getPluginVersion(): string {
return this.context.plugin.manifest.version;
}
get confirm(): Confirm {
return this._confirmInstance;
}
addCommand<TCommand extends Command>(command: TCommand): TCommand {
return this.context.plugin.addCommand(command) as TCommand;
}
registerWindow(type: string, factory: ViewCreator): void {
return this.context.plugin.registerView(type, factory);
}
addRibbonIcon(icon: string, title: string, callback: (evt: MouseEvent) => any): HTMLElement {
return this.context.plugin.addRibbonIcon(icon, title, callback);
}
registerProtocolHandler(action: string, handler: (params: Record<string, string>) => any): void {
return this.context.plugin.registerObsidianProtocolHandler(action, handler);
}
}

16
src/modules/services/ObsidianDatabaseService.ts Normal file
View File

@@ -0,0 +1,16 @@
import { initializeStores } from "@/common/stores";
// import { InjectableDatabaseService } from "@/lib/src/services/implements/injectable/InjectableDatabaseService";
import type { ObsidianServiceContext } from "@/lib/src/services/implements/obsidian/ObsidianServiceContext";
import { DatabaseService, type DatabaseServiceDependencies } from "@lib/services/base/DatabaseService.ts";
export class ObsidianDatabaseService<T extends ObsidianServiceContext> extends DatabaseService<T> {
private __onOpenDatabase(vaultName: string) {
initializeStores(vaultName);
return Promise.resolve(true);
}
constructor(context: T, dependencies: DatabaseServiceDependencies) {
super(context, dependencies);
this.onOpenDatabase.addHandler(this.__onOpenDatabase.bind(this));
}
}

71
src/modules/services/ObsidianServiceHub.ts
View File

@@ -1,21 +1,23 @@
import { InjectableServiceHub } from "@/lib/src/services/implements/injectable/InjectableServiceHub";
import { ObsidianServiceContext } from "@/lib/src/services/implements/obsidian/ObsidianServiceContext";
import type { ServiceInstances } from "@/lib/src/services/ServiceHub";
import { InjectableServiceHub } from "@lib/services/implements/injectable/InjectableServiceHub";
import { ObsidianServiceContext } from "@lib/services/implements/obsidian/ObsidianServiceContext";
import type { ServiceInstances } from "@lib/services/ServiceHub";
import type ObsidianLiveSyncPlugin from "@/main";
import {
ObsidianAPIService,
ObsidianConflictService,
ObsidianDatabaseService,
ObsidianFileProcessingService,
ObsidianReplicationService,
ObsidianReplicatorService,
ObsidianRemoteService,
ObsidianSettingService,
ObsidianTweakValueService,
ObsidianTestService,
ObsidianDatabaseEventService,
ObsidianConfigService,
ObsidianKeyValueDBService,
ObsidianControlService,
} from "./ObsidianServices";
import { ObsidianSettingService } from "./ObsidianSettingService";
import { ObsidianDatabaseService } from "./ObsidianDatabaseService";
import { ObsidianAPIService } from "./ObsidianAPIService";
import { ObsidianAppLifecycleService } from "./ObsidianAppLifecycleService";
import { ObsidianPathService } from "./ObsidianPathService";
import { ObsidianVaultService } from "./ObsidianVaultService";
@@ -28,30 +30,71 @@ export class ObsidianServiceHub extends InjectableServiceHub<ObsidianServiceCont
const context = new ObsidianServiceContext(plugin.app, plugin, plugin);
const API = new ObsidianAPIService(context);
const appLifecycle = new ObsidianAppLifecycleService(context);
const conflict = new ObsidianConflictService(context);
const database = new ObsidianDatabaseService(context);
const fileProcessing = new ObsidianFileProcessingService(context);
const replication = new ObsidianReplicationService(context);
const replicator = new ObsidianReplicatorService(context);
const remote = new ObsidianRemoteService(context);
const setting = new ObsidianSettingService(context);
const tweakValue = new ObsidianTweakValueService(context);
const setting = new ObsidianSettingService(context, {
APIService: API,
});
const appLifecycle = new ObsidianAppLifecycleService(context, {
settingService: setting,
});
const vault = new ObsidianVaultService(context, {
settingService: setting,
APIService: API,
});
const test = new ObsidianTestService(context);
const databaseEvents = new ObsidianDatabaseEventService(context);
const path = new ObsidianPathService(context, {
settingService: setting,
});
const config = new ObsidianConfigService(context, vault);
const database = new ObsidianDatabaseService(context, {
path: path,
vault: vault,
setting: setting,
});
const keyValueDB = new ObsidianKeyValueDBService(context, {
appLifecycle: appLifecycle,
databaseEvents: databaseEvents,
vault: vault,
});
const config = new ObsidianConfigService(context, {
settingService: setting,
APIService: API,
});
const replicator = new ObsidianReplicatorService(context, {
settingService: setting,
appLifecycleService: appLifecycle,
databaseEventService: databaseEvents,
});
const replication = new ObsidianReplicationService(context, {
APIService: API,
appLifecycleService: appLifecycle,
databaseEventService: databaseEvents,
replicatorService: replicator,
settingService: setting,
fileProcessingService: fileProcessing,
databaseService: database,
});
const control = new ObsidianControlService(context, {
appLifecycleService: appLifecycle,
databaseService: database,
fileProcessingService: fileProcessing,
settingService: setting,
APIService: API,
replicatorService: replicator,
});
const ui = new ObsidianUIService(context, {
appLifecycle,
config,
replicator,
APIService: API,
control: control,
});
// Using 'satisfies' to ensure all services are provided
const serviceInstancesToInit = {
appLifecycle: appLifecycle,
@@ -70,6 +113,8 @@ export class ObsidianServiceHub extends InjectableServiceHub<ObsidianServiceCont
path: path,
API: API,
config: config,
keyValueDB: keyValueDB,
control: control,
} satisfies Required<ServiceInstances<ObsidianServiceContext>>;
super(context, serviceInstancesToInit);

109
src/modules/services/ObsidianServices.ts
View File

@@ -1,115 +1,16 @@
import { InjectableAPIService } from "@lib/services/implements/injectable/InjectableAPIService";
import { InjectableConflictService } from "@lib/services/implements/injectable/InjectableConflictService";
import { InjectableDatabaseEventService } from "@lib/services/implements/injectable/InjectableDatabaseEventService";
import { InjectableDatabaseService } from "@lib/services/implements/injectable/InjectableDatabaseService";
import { InjectableFileProcessingService } from "@lib/services/implements/injectable/InjectableFileProcessingService";
import { InjectableRemoteService } from "@lib/services/implements/injectable/InjectableRemoteService";
import { InjectableReplicationService } from "@lib/services/implements/injectable/InjectableReplicationService";
import { InjectableReplicatorService } from "@lib/services/implements/injectable/InjectableReplicatorService";
import { InjectableSettingService } from "@lib/services/implements/injectable/InjectableSettingService";
import { InjectableTestService } from "@lib/services/implements/injectable/InjectableTestService";
import { InjectableTweakValueService } from "@lib/services/implements/injectable/InjectableTweakValueService";
import { ConfigServiceBrowserCompat } from "@lib/services/implements/browser/ConfigServiceBrowserCompat";
import type { ObsidianServiceContext } from "@lib/services/implements/obsidian/ObsidianServiceContext.ts";
import { Platform } from "@/deps";
import type { SimpleStore } from "@/lib/src/common/utils";
import type { IDatabaseService } from "@/lib/src/services/base/IService";
import { handlers } from "@/lib/src/services/lib/HandlerUtils";
import { ObsHttpHandler } from "../essentialObsidian/APILib/ObsHttpHandler";
import type { Command, ViewCreator } from "obsidian";
import { KeyValueDBService } from "@lib/services/base/KeyValueDBService";
import { ControlService } from "@lib/services/base/ControlService";
// All Services will be migrated to be based on Plain Services, not Injectable Services.
// This is a migration step.
export class ObsidianAPIService extends InjectableAPIService<ObsidianServiceContext> {
_customHandler: ObsHttpHandler | undefined;
getCustomFetchHandler(): ObsHttpHandler {
if (!this._customHandler) this._customHandler = new ObsHttpHandler(undefined, undefined);
return this._customHandler;
}
async showWindow(viewType: string): Promise<void> {
const leaves = this.app.workspace.getLeavesOfType(viewType);
if (leaves.length == 0) {
await this.app.workspace.getLeaf(true).setViewState({
type: viewType,
active: true,
});
} else {
await leaves[0].setViewState({
type: viewType,
active: true,
});
}
if (leaves.length > 0) {
await this.app.workspace.revealLeaf(leaves[0]);
}
}
private get app() {
return this.context.app;
}
getPlatform(): string {
if (Platform.isAndroidApp) {
return "android-app";
} else if (Platform.isIosApp) {
return "ios";
} else if (Platform.isMacOS) {
return "macos";
} else if (Platform.isMobileApp) {
return "mobile-app";
} else if (Platform.isMobile) {
return "mobile";
} else if (Platform.isSafari) {
return "safari";
} else if (Platform.isDesktop) {
return "desktop";
} else if (Platform.isDesktopApp) {
return "desktop-app";
} else {
return "unknown-obsidian";
}
}
override isMobile(): boolean {
//@ts-ignore : internal API
return this.app.isMobile;
}
override getAppID(): string {
return `${"appId" in this.app ? this.app.appId : ""}`;
}
override getAppVersion(): string {
const navigatorString = globalThis.navigator?.userAgent ?? "";
const match = navigatorString.match(/obsidian\/([0-9]+\.[0-9]+\.[0-9]+)/);
if (match && match.length >= 2) {
return match[1];
}
return "0.0.0";
}
override getPluginVersion(): string {
return this.context.plugin.manifest.version;
}
addCommand<TCommand extends Command>(command: TCommand): TCommand {
return this.context.plugin.addCommand(command) as TCommand;
}
registerWindow(type: string, factory: ViewCreator): void {
return this.context.plugin.registerView(type, factory);
}
addRibbonIcon(icon: string, title: string, callback: (evt: MouseEvent) => any): HTMLElement {
return this.context.plugin.addRibbonIcon(icon, title, callback);
}
registerProtocolHandler(action: string, handler: (params: Record<string, string>) => any): void {
return this.context.plugin.registerObsidianProtocolHandler(action, handler);
}
}
export class ObsidianDatabaseService extends InjectableDatabaseService<ObsidianServiceContext> {
openSimpleStore = handlers<IDatabaseService>().binder("openSimpleStore") as (<T>(
kind: string
) => SimpleStore<T>) & { setHandler: (handler: IDatabaseService["openSimpleStore"], override?: boolean) => void };
}
export class ObsidianDatabaseEventService extends InjectableDatabaseEventService<ObsidianServiceContext> {}
// InjectableReplicatorService
@@ -122,10 +23,12 @@ export class ObsidianReplicationService extends InjectableReplicationService<Obs
export class ObsidianRemoteService extends InjectableRemoteService<ObsidianServiceContext> {}
// InjectableConflictService
export class ObsidianConflictService extends InjectableConflictService<ObsidianServiceContext> {}
// InjectableSettingService
export class ObsidianSettingService extends InjectableSettingService<ObsidianServiceContext> {}
// InjectableTweakValueService
export class ObsidianTweakValueService extends InjectableTweakValueService<ObsidianServiceContext> {}
// InjectableTestService
export class ObsidianTestService extends InjectableTestService<ObsidianServiceContext> {}
export class ObsidianConfigService extends ConfigServiceBrowserCompat<ObsidianServiceContext> {}
export class ObsidianKeyValueDBService extends KeyValueDBService<ObsidianServiceContext> {}
export class ObsidianControlService extends ControlService<ObsidianServiceContext> {}

35
src/modules/services/ObsidianSettingService.ts Normal file
View File

@@ -0,0 +1,35 @@
import { type ObsidianLiveSyncSettings } from "@lib/common/types";
import { EVENT_REQUEST_RELOAD_SETTING_TAB, EVENT_SETTING_SAVED } from "@lib/events/coreEvents";
import { eventHub } from "@lib/hub/hub";
import { SettingService, type SettingServiceDependencies } from "@lib/services/base/SettingService";
import type { ObsidianServiceContext } from "@lib/services/implements/obsidian/ObsidianServiceContext";
export class ObsidianSettingService<T extends ObsidianServiceContext> extends SettingService<T> {
constructor(context: T, dependencies: SettingServiceDependencies) {
super(context, dependencies);
this.onSettingSaved.addHandler((settings) => {
eventHub.emitEvent(EVENT_SETTING_SAVED, settings);
return Promise.resolve(true);
});
this.onSettingLoaded.addHandler((settings) => {
eventHub.emitEvent(EVENT_REQUEST_RELOAD_SETTING_TAB);
return Promise.resolve(true);
});
}
protected setItem(key: string, value: string) {
return localStorage.setItem(key, value);
}
protected getItem(key: string): string {
return localStorage.getItem(key) ?? "";
}
protected deleteItem(key: string): void {
localStorage.removeItem(key);
}
protected override async saveData(data: ObsidianLiveSyncSettings): Promise<void> {
return await this.context.liveSyncPlugin.saveData(data);
}
protected override async loadData(): Promise<ObsidianLiveSyncSettings | undefined> {
return await this.context.liveSyncPlugin.loadData();
}
}

13
src/modules/services/ObsidianUIService.ts
View File

@@ -2,14 +2,16 @@ import type { ConfigService } from "@lib/services/base/ConfigService";
import type { AppLifecycleService } from "@lib/services/base/AppLifecycleService";
import type { ReplicatorService } from "@lib/services/base/ReplicatorService";
import { UIService } from "@lib/services//implements/base/UIService";
import { ObsidianServiceContext } from "@/lib/src/services/implements/obsidian/ObsidianServiceContext";
import { ObsidianServiceContext } from "@lib/services/implements/obsidian/ObsidianServiceContext";
import { ObsidianSvelteDialogManager } from "./SvelteDialogObsidian";
import { ObsidianConfirm } from "./ObsidianConfirm";
import DialogToCopy from "@/lib/src/UI/dialogues/DialogueToCopy.svelte";
import DialogToCopy from "@lib/UI/dialogues/DialogueToCopy.svelte";
import type { IAPIService, IControlService } from "@lib/services/base/IService";
export type ObsidianUIServiceDependencies<T extends ObsidianServiceContext = ObsidianServiceContext> = {
appLifecycle: AppLifecycleService<T>;
config: ConfigService<T>;
replicator: ReplicatorService<T>;
APIService: IAPIService;
control: IControlService;
};
export class ObsidianUIService extends UIService<ObsidianServiceContext> {
@@ -17,17 +19,18 @@ export class ObsidianUIService extends UIService<ObsidianServiceContext> {
return DialogToCopy;
}
constructor(context: ObsidianServiceContext, dependents: ObsidianUIServiceDependencies<ObsidianServiceContext>) {
const obsidianConfirm = new ObsidianConfirm(context);
const obsidianConfirm = dependents.APIService.confirm;
const obsidianSvelteDialogManager = new ObsidianSvelteDialogManager<ObsidianServiceContext>(context, {
appLifecycle: dependents.appLifecycle,
config: dependents.config,
replicator: dependents.replicator,
confirm: obsidianConfirm,
control: dependents.control,
});
super(context, {
appLifecycle: dependents.appLifecycle,
dialogManager: obsidianSvelteDialogManager,
confirm: obsidianConfirm,
APIService: dependents.APIService,
});
}
}

2
src/modules/services/ObsidianVaultService.ts
View File

@@ -11,7 +11,7 @@ declare module "obsidian" {
// InjectableVaultService
export class ObsidianVaultService extends InjectableVaultService<ObsidianServiceContext> {
vaultName(): string {
override vaultName(): string {
return this.context.app.vault.getName();
}
getActiveFilePath(): FilePath | undefined {

3
src/serviceFeatures/onLayoutReady.ts Normal file
View File

@@ -0,0 +1,3 @@
import { enableI18nFeature } from "./onLayoutReady/enablei18n";
export const onLayoutReadyFeatures = [enableI18nFeature];

41
src/serviceFeatures/onLayoutReady/enablei18n.ts Normal file
View File

@@ -0,0 +1,41 @@
import { getLanguage } from "@/deps";
import { createServiceFeature } from "../types.ts";
import { SUPPORTED_I18N_LANGS, type I18N_LANGS } from "@lib/common/rosetta";
import { $msg, setLang } from "@lib/common/i18n";
export const enableI18nFeature = createServiceFeature(async ({ services: { setting, API } }) => {
let isChanged = false;
const settings = setting.currentSettings();
if (settings.displayLanguage == "") {
const obsidianLanguage = getLanguage();
if (
SUPPORTED_I18N_LANGS.indexOf(obsidianLanguage) !== -1 && // Check if the language is supported
obsidianLanguage != settings.displayLanguage // Check if the language is different from the current setting
) {
// Check if the current setting is not empty (Means migrated or installed).
// settings.displayLanguage = obsidianLanguage as I18N_LANGS;
await setting.applyPartial({ displayLanguage: obsidianLanguage as I18N_LANGS });
isChanged = true;
setLang(settings.displayLanguage);
} else if (settings.displayLanguage == "") {
// settings.displayLanguage = "def";
await setting.applyPartial({ displayLanguage: "def" });
setLang(settings.displayLanguage);
await setting.saveSettingData();
}
}
if (isChanged) {
const revert = $msg("dialog.yourLanguageAvailable.btnRevertToDefault");
if (
(await API.confirm.askSelectStringDialogue($msg(`dialog.yourLanguageAvailable`), ["OK", revert], {
defaultAction: "OK",
title: $msg(`dialog.yourLanguageAvailable.Title`),
})) == revert
) {
await setting.applyPartial({ displayLanguage: "def" });
setLang(settings.displayLanguage);
}
await setting.saveSettingData();
}
return true;
});

78
src/serviceFeatures/types.ts Normal file
View File

@@ -0,0 +1,78 @@
import type { IServiceHub } from "@lib/services/base/IService";
import type { DatabaseFileAccess } from "@lib/interfaces/DatabaseFileAccess";
import type { Rebuilder } from "@lib/interfaces/DatabaseRebuilder";
import type { IFileHandler } from "@lib/interfaces/FileHandler";
import type { StorageAccess } from "@lib/interfaces/StorageAccess";
import type { LogFunction } from "@/lib/src/services/lib/logUtils";
export interface ServiceModules {
storageAccess: StorageAccess;
/**
* Database File Accessor for handling file operations related to the database, such as exporting the database, importing from a file, etc.
*/
databaseFileAccess: DatabaseFileAccess;
/**
* File Handler for handling file operations related to replication, such as resolving conflicts, applying changes from replication, etc.
*/
fileHandler: IFileHandler;
/**
* Rebuilder for handling database rebuilding operations.
*/
rebuilder: Rebuilder;
}
export type RequiredServices<T extends keyof IServiceHub> = Pick<IServiceHub, T>;
export type RequiredServiceModules<T extends keyof ServiceModules> = Pick<ServiceModules, T>;
export type NecessaryServices<T extends keyof IServiceHub, U extends keyof ServiceModules> = {
services: RequiredServices<T>;
serviceModules: RequiredServiceModules<U>;
};
export type ServiceFeatureFunction<T extends keyof IServiceHub, U extends keyof ServiceModules, TR> = (
host: NecessaryServices<T, U>
) => TR;
type ServiceFeatureContext<T> = T & {
_log: LogFunction;
};
export type ServiceFeatureFunctionWithContext<T extends keyof IServiceHub, U extends keyof ServiceModules, C, TR> = (
host: NecessaryServices<T, U>,
context: ServiceFeatureContext<C>
) => TR;
/**
* Helper function to create a service feature with proper typing.
* @param featureFunction The feature function to be wrapped.
* @returns The same feature function with proper typing.
* @example
* const myFeatureDef = createServiceFeature(({ services: { API }, serviceModules: { storageAccess } }) => {
* // ...
* });
* const myFeature = myFeatureDef.bind(null, this); // <- `this` may `ObsidianLiveSyncPlugin` or a custom context object
* appLifecycle.onLayoutReady(myFeature);
*/
export function createServiceFeature<T extends keyof IServiceHub, U extends keyof ServiceModules, TR>(
featureFunction: ServiceFeatureFunction<T, U, TR>
): ServiceFeatureFunction<T, U, TR> {
return featureFunction;
}
type ContextFactory<T extends keyof IServiceHub, U extends keyof ServiceModules, C> = (
host: NecessaryServices<T, U>
) => ServiceFeatureContext<C>;
export function serviceFeature<T extends keyof IServiceHub, U extends keyof ServiceModules>() {
return {
create<TR>(featureFunction: ServiceFeatureFunction<T, U, TR>) {
return featureFunction;
},
withContext<C extends object = object>(ContextFactory: ContextFactory<T, U, C>) {
return {
create:
<TR>(featureFunction: ServiceFeatureFunctionWithContext<T, U, C, TR>) =>
(host: NecessaryServices<T, U>, context: ServiceFeatureContext<C>) =>
featureFunction(host, ContextFactory(host)),
};
},
};
}

15
src/serviceModules/DatabaseFileAccess.ts Normal file
View File

@@ -0,0 +1,15 @@
import { markChangesAreSame } from "@/common/utils";
import type { AnyEntry } from "@lib/common/types";
import type { DatabaseFileAccess } from "@lib/interfaces/DatabaseFileAccess.ts";
import { ServiceDatabaseFileAccessBase } from "@lib/serviceModules/ServiceDatabaseFileAccessBase";
// markChangesAreSame uses persistent data implicitly, we should refactor it too.
// For now, to make the refactoring done once, we just use them directly.
// Hence it is not on /src/lib/src/serviceModules. (markChangesAreSame is using indexedDB).
// TODO: REFACTOR
export class ServiceDatabaseFileAccess extends ServiceDatabaseFileAccessBase implements DatabaseFileAccess {
markChangesAreSame(old: AnyEntry, newMtime: number, oldMtime: number): void {
markChangesAreSame(old, newMtime, oldMtime);
}
}

160
src/serviceModules/FileAccessObsidian.ts Normal file
View File

@@ -0,0 +1,160 @@
import { markChangesAreSame } from "@/common/utils";
import type { FilePath, UXDataWriteOptions, UXFileInfoStub, UXFolderInfo } from "@lib/common/types";
import { TFolder, type TAbstractFile, TFile, type Stat, type App, type DataWriteOptions, normalizePath } from "@/deps";
import { FileAccessBase, toArrayBuffer, type FileAccessBaseDependencies } from "@lib/serviceModules/FileAccessBase.ts";
import { TFileToUXFileInfoStub } from "@/modules/coreObsidian/storageLib/utilObsidian";
declare module "obsidian" {
interface Vault {
getAbstractFileByPathInsensitive(path: string): TAbstractFile | null;
}
interface DataAdapter {
reconcileInternalFile?(path: string): Promise<void>;
}
}
export class FileAccessObsidian extends FileAccessBase<TAbstractFile, TFile, TFolder, Stat> {
app: App;
override getPath(file: string | TAbstractFile): FilePath {
return (typeof file === "string" ? file : file.path) as FilePath;
}
override isFile(file: TAbstractFile | null): file is TFile {
return file instanceof TFile;
}
override isFolder(file: TAbstractFile | null): file is TFolder {
return file instanceof TFolder;
}
override _statFromNative(file: TFile): Promise<TFile["stat"]> {
return Promise.resolve(file.stat);
}
override nativeFileToUXFileInfoStub(file: TFile): UXFileInfoStub {
return TFileToUXFileInfoStub(file);
}
override nativeFolderToUXFolder(folder: TFolder): UXFolderInfo {
if (folder instanceof TFolder) {
return this.nativeFolderToUXFolder(folder);
} else {
throw new Error(`Not a folder: ${(folder as TAbstractFile)?.name}`);
}
}
constructor(app: App, dependencies: FileAccessBaseDependencies) {
super({
storageAccessManager: dependencies.storageAccessManager,
vaultService: dependencies.vaultService,
settingService: dependencies.settingService,
APIService: dependencies.APIService,
});
this.app = app;
}
protected override _normalisePath(path: string): string {
return normalizePath(path);
}
protected async _adapterMkdir(path: string) {
await this.app.vault.adapter.mkdir(path);
}
protected _getAbstractFileByPath(path: FilePath) {
return this.app.vault.getAbstractFileByPath(path);
}
protected _getAbstractFileByPathInsensitive(path: FilePath) {
return this.app.vault.getAbstractFileByPathInsensitive(path);
}
protected async _tryAdapterStat(path: FilePath) {
if (!(await this.app.vault.adapter.exists(path))) return null;
return await this.app.vault.adapter.stat(path);
}
protected async _adapterStat(path: FilePath) {
return await this.app.vault.adapter.stat(path);
}
protected async _adapterExists(path: FilePath) {
return await this.app.vault.adapter.exists(path);
}
protected async _adapterRemove(path: FilePath) {
await this.app.vault.adapter.remove(path);
}
protected async _adapterRead(path: FilePath) {
return await this.app.vault.adapter.read(path);
}
protected async _adapterReadBinary(path: FilePath) {
return await this.app.vault.adapter.readBinary(path);
}
_adapterWrite(file: string, data: string, options?: UXDataWriteOptions): Promise<void> {
return this.app.vault.adapter.write(file, data, options);
}
_adapterWriteBinary(file: string, data: ArrayBuffer, options?: UXDataWriteOptions): Promise<void> {
return this.app.vault.adapter.writeBinary(file, toArrayBuffer(data), options);
}
protected _adapterList(basePath: string): Promise<{ files: string[]; folders: string[] }> {
return Promise.resolve(this.app.vault.adapter.list(basePath));
}
async _vaultCacheRead(file: TFile) {
return await this.app.vault.cachedRead(file);
}
protected async _vaultRead(file: TFile): Promise<string> {
return await this.app.vault.read(file);
}
protected async _vaultReadBinary(file: TFile): Promise<ArrayBuffer> {
return await this.app.vault.readBinary(file);
}
protected override markChangesAreSame(path: string, mtime: number, newMtime: number) {
return markChangesAreSame(path, mtime, newMtime);
}
protected override async _vaultModify(file: TFile, data: string, options?: UXDataWriteOptions): Promise<void> {
return await this.app.vault.modify(file, data, options);
}
protected override async _vaultModifyBinary(
file: TFile,
data: ArrayBuffer,
options?: UXDataWriteOptions
): Promise<void> {
return await this.app.vault.modifyBinary(file, toArrayBuffer(data), options);
}
protected override async _vaultCreate(path: string, data: string, options?: UXDataWriteOptions): Promise<TFile> {
return await this.app.vault.create(path, data, options);
}
protected override async _vaultCreateBinary(
path: string,
data: ArrayBuffer,
options?: UXDataWriteOptions
): Promise<TFile> {
return await this.app.vault.createBinary(path, toArrayBuffer(data), options);
}
protected override _trigger(name: string, ...data: any[]) {
return this.app.vault.trigger(name, ...data);
}
protected override async _reconcileInternalFile(path: string) {
return await Promise.resolve(this.app.vault.adapter.reconcileInternalFile?.(path));
}
protected override async _adapterAppend(normalizedPath: string, data: string, options?: DataWriteOptions) {
return await this.app.vault.adapter.append(normalizedPath, data, options);
}
protected override async _delete(file: TFile | TFolder, force = false) {
return await this.app.vault.delete(file, force);
}
protected override async _trash(file: TFile | TFolder, force = false) {
return await this.app.vault.trash(file, force);
}
protected override _getFiles() {
return this.app.vault.getFiles();
}
}

26
src/serviceModules/FileHandler.ts Normal file
View File

@@ -0,0 +1,26 @@
import {
compareFileFreshness,
markChangesAreSame,
type BASE_IS_NEW,
type EVEN,
type TARGET_IS_NEW,
} from "@/common/utils";
import type { AnyEntry } from "@lib/common/models/db.type";
import type { UXFileInfo, UXFileInfoStub } from "@lib/common/models/fileaccess.type";
import { ServiceFileHandlerBase } from "@lib/serviceModules/ServiceFileHandlerBase";
// markChangesAreSame uses persistent data implicitly, we should refactor it too.
// also, compareFileFreshness depends on marked changes, so we should refactor it as well. For now, to make the refactoring done once, we just use them directly.
// Hence it is not on /src/lib/src/serviceModules. (markChangesAreSame is using indexedDB).
// TODO: REFACTOR
export class ServiceFileHandler extends ServiceFileHandlerBase {
override markChangesAreSame(old: UXFileInfo | AnyEntry, newMtime: number, oldMtime: number) {
return markChangesAreSame(old, newMtime, oldMtime);
}
override compareFileFreshness(
baseFile: UXFileInfoStub | AnyEntry | undefined,
checkTarget: UXFileInfo | AnyEntry | undefined
): typeof TARGET_IS_NEW | typeof BASE_IS_NEW | typeof EVEN {
return compareFileFreshness(baseFile, checkTarget);
}
}

6
src/serviceModules/ServiceFileAccessImpl.ts Normal file
View File

@@ -0,0 +1,6 @@
import type { TAbstractFile, TFile, TFolder, Stat } from "@/deps";
import { ServiceFileAccessBase } from "@lib/serviceModules/ServiceFileAccessBase";
// For typechecking purpose
export class ServiceFileAccessObsidian extends ServiceFileAccessBase<TAbstractFile, TFile, TFolder, Stat> {}

27
src/types.ts Normal file
View File

@@ -0,0 +1,27 @@
import type { DatabaseFileAccess } from "@/lib/src/interfaces/DatabaseFileAccess";
import type { Rebuilder } from "@/lib/src/interfaces/DatabaseRebuilder";
import type { IFileHandler } from "@/lib/src/interfaces/FileHandler";
import type { StorageAccess } from "@/lib/src/interfaces/StorageAccess";
import type { IServiceHub } from "./lib/src/services/base/IService";
export interface ServiceModules {
storageAccess: StorageAccess;
/**
* Database File Accessor for handling file operations related to the database, such as exporting the database, importing from a file, etc.
*/
databaseFileAccess: DatabaseFileAccess;
/**
* File Handler for handling file operations related to replication, such as resolving conflicts, applying changes from replication, etc.
*/
fileHandler: IFileHandler;
/**
* Rebuilder for handling database rebuilding operations.
*/
rebuilder: Rebuilder;
}
export interface LiveSyncHost {
services: IServiceHub;
serviceModules: ServiceModules;
}

49
terser_vite.config.ts Normal file
View File

@@ -0,0 +1,49 @@
import type { TerserOptions } from "vite";
export const terserOption: TerserOptions = {
mangle: {
// properties: {
// regex: /^_p_/,
// },
eval: true,
keep_classnames: true,
keep_fnames: true,
// module: true,
// safari10: true,
// toplevel: true,
},
// mangle: false,
compress: {
defaults: false,
arguments: true,
// drop_console: false,
ecma: 2020,
// keep_classnames: true,
// keep_fnames: false,
// module: true,
passes: 4,
// arrows: true,
// collapse_vars: true,
// comparisons: true,
// computed_props: true,
// conditionals: true,
dead_code: true,
evaluate: true,
// hoist_funs: true,
// hoist_props: true,
// hoist_vars: false,
// if_return: true,
inline: true,
// join_vars: true,
// reduce_funcs: true,
// reduce_vars: true,
// sequences: true,
// side_effects: false,
},
format: {
// beautify: true,
ecma: 2020,
safari10: true,
webkit: true,
},
};

16
test/harness/harness.ts
View File

@@ -122,13 +122,11 @@ export async function waitForIdle(harness: LiveSyncHarness): Promise<void> {
for (let i = 0; i < 20; i++) {
await delay(25);
const processing =
harness.plugin.databaseQueueCount.value +
harness.plugin.processingFileEventCount.value +
harness.plugin.pendingFileEventCount.value +
harness.plugin.totalQueued.value +
harness.plugin.batched.value +
harness.plugin.processing.value +
harness.plugin.storageApplyingCount.value;
harness.plugin.services.replication.databaseQueueCount.value +
harness.plugin.services.fileProcessing.totalQueued.value +
harness.plugin.services.fileProcessing.batched.value +
harness.plugin.services.fileProcessing.processing.value +
harness.plugin.services.replication.storageApplyingCount.value;
if (processing === 0) {
if (i > 0) {
@@ -141,8 +139,8 @@ export async function waitForIdle(harness: LiveSyncHarness): Promise<void> {
export async function waitForClosed(harness: LiveSyncHarness): Promise<void> {
await delay(100);
for (let i = 0; i < 10; i++) {
if (harness.plugin.services.appLifecycle.hasUnloaded()) {
console.log("App Lifecycle has unloaded");
if (harness.plugin.services.control.hasUnloaded()) {
console.log("App has unloaded");
return;
}
await delay(100);

8
test/harness/obsidian-mock.ts
View File

@@ -481,8 +481,14 @@ export class Plugin {
}
export class Notice {
private _key:number;
private static _counter = 0;
constructor(message: string) {
console.log("Notice:", message);
this._key = Notice._counter++;
console.log(`Notice [${this._key}]:`, message);
}
setMessage(message: string) {
console.log(`Notice [${this._key}]:`, message);
}
}

1
tsconfig.json
View File

@@ -14,6 +14,7 @@
"alwaysStrict": true,
"allowImportingTsExtensions": true,
"verbatimModuleSyntax": true,
"noImplicitOverride": true,
"noEmit": true,
"lib": ["es2018", "DOM", "ES5", "ES6", "ES7", "es2019.array", "ES2021.WeakRef", "ES2020.BigInt", "ESNext.Intl"],
"strictBindCallApply": true,

130
updates.md
View File

@@ -3,58 +3,62 @@ Since 19th July, 2025 (beta1 in 0.25.0-beta1, 13th July, 2025)
The head note of 0.25 is now in [updates_old.md](https://github.com/vrtmrz/obsidian-livesync/blob/main/updates_old.md). Because 0.25 got a lot of updates, thankfully, compatibility is kept and we do not need breaking changes! In other words, when get enough stabled. The next version will be v1.0.0. Even though it my hope.
## 0.25.43-patched-2
## 0.25.44
14th February, 2026
24th February, 2026
This release represents a significant architectural overhaul of the plug-in, focusing on modularity, testability, and stability. While many changes are internal, they pave the way for more robust features and easier maintenance.
However, as this update is very substantial, please do feel free to let me know if you encounter any issues.
### Fixed
- Application LifeCycle has now started in Main, not ServiceHub.
- Indeed, ServiceHub cannot be known other things in main have got ready, so it is quite natural to start the lifecycle in main.
## 0.25.43-patched-1
- Ignore files (e.g., `.ignore`) are now handled efficiently.
- Replication & Database:
- Replication statistics are now correctly reset after switching replicators.
- Fixed `File already exists` for .md files has been merged (PR #802) So thanks @waspeer for the contribution!
13th February, 2026
### Improved
**NOTE: Hidden File Sync and Customisation Sync may not work in this version.**
Just a heads-up: this is a patch version, which is essentially a beta release. Do not worry about the following memos, as they are indeed freaking us out. I trust that you have thought this was too large; you're right.
If this cannot be stable, I will revert to 0.24.43 and try again.
- Now we can configure network-error banners as icons, or hide them completely with the new `Network Warning Style` setting in the `General` pane of the settings dialogue. (#770, PR #804)
- Thanks so much to @A-wry!
### Refactored
- Now resolving unexpected and inexplicable dependency order issues...
- The function which is able to implement to the service is now moved to each service.
- AppLifecycleService.performRestart
- VaultService.isTargetFile is now using multiple checkers instead of a single function.
- This change allows better separation of concerns and easier extension in the future.
- Application LifeCycle has now started in ServiceHub, not ObsidianMenuModule.
- It was in a QUITE unexpected place..., isn't it?
- Instead of, we should call `await this.services.appLifecycle.onReady()` in other platforms.
- As in the browser platform, it will be called at `DOMContentLoaded` event.
#### Architectural Overhaul:
- ModuleTargetFilter, which is responsible for parsing ignore files, has been refined.
- This should be separated to a TargetFilter and an IgnoreFileFilter for better maintainability.
- Using `API.addCommand` or some Obsidian API and shimmer APIs, Many modules have been refactored to be derived to AbstractModule from AbstractObsidianModule, to clarify the dependencies. (we should make `app` usage clearer...)
- Fixed initialising `storageAccess` too late in `FileAccessObsidian` module (I am still wondering why it worked before...).
- Remove some redundant overrides in modules.
- A major transition from Class-based Modules to a Service/Middleware architecture has begun.
- Many modules (for example, `ModulePouchDB`, `ModuleLocalDatabaseObsidian`, `ModuleKeyValueDB`) have been removed or integrated into specific Services (`database`, `keyValueDB`, etc.).
- Reduced reliance on dynamic binding and inverted dependencies; dependencies are now explicit.
- `ObsidianLiveSyncPlugin` properties (`replicator`, `localDatabase`, `storageAccess`, etc.) have been moved to their respective services for better separation of concerns.
- In this refactoring, the Service will henceforth, as a rule, cease to use setHandler, that is to say, simple lazy binding.
- They will be implemented directly in the service.
- However, not everything will be middlewarised. Modules that maintain state or make decisions based on the results of multiple handlers are permitted.
- Lifecycle:
- Application LifeCycle now starts in `Main` rather than `ServiceHub` or `ObsidianMenuModule`, ensuring smoother startup coordination.
### Planned
#### New Services & Utilities:
- Some services have an ambiguous name, such as `Injectable`. These will be renamed in the future for better clarity.
- Following properties of `ObsidianLiveSyncPlugin` should be initialised more explicitly:
- property : where it is initialised currently
- `localDatabase` : `ModuleLocalDatabaseObsidian`
- `managers` : `ModuleLocalDatabaseObsidian`
- `replicator` : `ModuleReplicator`
- `simpleStore` : `ModuleKeyValueDB`
- `storageAccess` : `ModuleFileAccessObsidian`
- `databaseFileAccess` : `ModuleDatabaseFileAccess`
- `fileHandler` : `ModuleFileHandler`
- `rebuilder` : `ModuleRebuilder`
- `kvDB`: `ModuleKeyValueDB`
- And I think that having a feature in modules directly is not good for maintainability, these should be separated to some module (loader) and implementation (not only service, but also independent something).
- Plug-in statuses such as requestCount, responseCount... should be moved to a status service or somewhere for better separation of concerns.
- Added a `control` service to orchestrate other services (for example, handling stop/start logic during settings realisation).
- Added `UnresolvedErrorManager` to handle and display unresolved errors in a unified way.
- Added `logUtils` to unify logging injection and formatting.
- `VaultService.isTargetFile` now uses multiple, distinct checkers for better extensibility.
#### Code Separation:
- Separated Obsidian-specific logic from base logic for `StorageEventManager` and `FileAccess` modules.
- Moved reactive state values and statistics from the main plug-in instance to the services responsible for them.
#### Internal Cleanups:
- Many functions have been renamed for clarity (for example, `_isTargetFileByLocalDB` is now `_isTargetAcceptedByLocalDB`).
- Added `override` keywords to overridden items and removed dynamic binding for clearer code inheritance.
- Moved common functions to the common library.
#### Dependencies:
- Bumped dependencies simply to a point where they can be considered problem-free (by human-powered-artefacts-diff).
- Svelte, terser, and more something will be bumped later. They have a significant impact on the diff and paint it totally.
- You may be surprised, but when I bump the library, I am actually checking for any unintended code.
## 0.25.43
@@ -214,49 +218,5 @@ Sorry for a small release! I would like to keep things moving along like this if
- Storage application process has been refactored.
- Please report if you find any unexpected behaviour after this update. A bit of large refactoring.
## 0.25.33
05th December, 2025
### New feature
- We can analyse the local database with the `Analyse database usage` command.
- This command makes a TSV-style report of the database usage, which can be pasted into spreadsheet applications.
- The report contains the number of unique chunks and shared chunks for each document revision.
- Unique chunks indicate the actual consumption.
- Shared chunks indicate the reference counts from other chunks with no consumption.
- We can find which notes or files are using large amounts of storage in the database. Or which notes cannot share chunks effectively.
- This command is useful when optimising the database size or investigating an unexpectedly large database size.
- We can reset the notification threshold and check the remote usage at once with the `Reset notification threshold and check the remote database usage` command.
- Commands are available from the Command Palette, or `Hatch` pane in the settings dialogue.
### Fixed
- Now the plug-in resets the remote size notification threshold after rebuild.
## 0.25.32
02nd December, 2025
Now I am back from a short (?) break! Thank you all for your patience. (It is nothing major, but the first half of the year has finally come to an end).
Anyway, I will release the things a bit by bit. I think that we need a rehabilitation or getting gears in again.
### Improved
- Now the plugin warns when we are in several file-related situations that may cause unexpected behaviour (#300).
- These errors are displayed alongside issues such as file size exceeding limits.
- Such situations include:
- When the document has a name which is not supported by some file systems.
- When the vault has the same file names with different letter cases.
## 0.25.31
18th November, 2025
### Fixed
- Now fetching configuration from the server can handle the empty remote correctly (reported on #756).
- No longer asking to switch adapters during rebuilding.
Older notes are in
Full notes are in
[updates_old.md](https://github.com/vrtmrz/obsidian-livesync/blob/main/updates_old.md).

428
updates_old.md
View File

@@ -1,4 +1,431 @@
# 0.25
Since 19th July, 2025 (beta1 in 0.25.0-beta1, 13th July, 2025)
The head note of 0.25 is now in [updates_old.md](https://github.com/vrtmrz/obsidian-livesync/blob/main/updates_old.md). Because 0.25 got a lot of updates, thankfully, compatibility is kept and we do not need breaking changes! In other words, when get enough stabled. The next version will be v1.0.0. Even though it my hope.
## 0.25.43-patched-9 a.k.a. 0.25.44-rc1
We are finally ready for release. I think I will go ahead and release it after using it for a few days.
### Fixed
- Hidden file synchronisation now works!
- Now Hidden file synchronisation respects `.ignore` files.
- Replicator initialisation during rebuilding now works correctly.
### Refactored
- Some methods naming have been changed for better clarity, i.e., `_isTargetFileByLocalDB` is now `_isTargetAcceptedByLocalDB`.
### Follow-up tasks memo (After 0.25.44)
Going forward, functionality that does not span multiple events is expected to be implemented as middleware-style functions rather than modules based on classes.
Consequently, the existing modules will likely be gradually dismantled.
For reference, `ModuleReplicator.ts` has extracted several functionalities as functions.
However, this does not negate object-oriented design. Where lifecycles and state are present, and the Liskov Substitution Principle can be upheld, we design using classes. After all, a visible state is preferable to a hidden state. In other words, the handler still accepts both functions and member methods, so formally there is no change.
As undertaking this for everything would be a bit longer task, I intend to release it at this stage.
Note: I left using `setHandler`s that as a mark of `need to be refactored`. Basically, they should be implemented in the service itself. That is because it is just a mis-designed, separated implementation.
## 0.25.43-patched-8
I really must thank you all. You know that it seems we have just a little more to do.
Note: This version is not fully tested yet. Be careful to use this. Very dogfood-y one.
### Fixed
- Now the device name is saved correctly.
### Refactored
- Add `override` keyword to all overridden items.
- More dynamic binding has been removed.
- The number of inverted dependencies has decreased much more.
- Some check-logic; i.e., like pre-replication check is now separated into check functions and added to the service as handlers, layered.
- This may help with better testing and better maintainability.
## 0.25.43-patched-7
19th February, 2026
Right then, let us make a decision already.
Last time, since I found a bug, I ended up doing a few other things as well, but next time I intend to release it with just the bug fix. It is quite substantial, after all.
Customisation Sync has mostly been verified. Hidden file synchronisation has not been done yet.
Vite's build system is not in the production. However, I possibly migrate to it in the future.
And, the `daily-progress` will be tidied on releasing 0.25.44. Do not worry!
### Fixed
- Fixed an issue where the StorageEventManager was not correctly loading the settings.
- Replication statistics are now correctly reset after switching replicators.
### Refactored
- Now, many reactive values which keep the state or statistics of the plugin are moved to the services which have the responsibility for these states.
- `serviceFeatures` are now able to be added to the services; this is not a class module, but a function which accepts dependencies and returns an addHandler-able function. This is for better separation of concerns, better maintainability, and testability.
- `control` service; is a meta-service which is responsible for orchestrating services has been added.
- Don't you think stopping replication or something occurs during `settingService.realiseSetting` is quite weird? It may be done by the control service, which can orchestrate the setting service and the replicator service.
-
- Some functions on services have been moved. e.g., `getSystemVaultName` is now on the API service.
- Setting Service is now responsible for the setting, no longer using dynamic binding for the modules.
## 0.25.43-patched-6
18th February, 2026
Let me confess that I have lied about `now all ambiguous properties`... I have found some more implicit calling.
Note: I have not checked hidden file sync and customisation sync yet. Please report if you find any unexpected behaviour in these features.
### Fixed
- Now ReplicatorService responds to database reset and database initialisation events to dispose of the active replicator.
- Fixes some unlocking issues during rebuilding.
### Refactored
- Now `StorageEventManagerBase` is separated from `StorageEventManagerObsidian` following their concerns.
- No longer using `ObsidianFileAccess` indirectly during checking duplicated-file events.
- Last event memorisation is now moved into the StorageAccessManager, just like the file processing interlocking.
- These methods, i.e., `ObsidianFileAccess.touch`. `StorageEventManager.recentlyTouched`, and `StorageEventManager.touch` are still available, but simply call the StorageAccessManager's methods.
- Now `FileAccessBase` is separated from `FileAccessObsidian` following their concerns.
## 0.25.43-patched-5
17th February, 2026
Yes, we mostly have got refactored!
### Refactored
- Following properties of `ObsidianLiveSyncPlugin` are now initialised more explicitly:
- property : what is responsible
- `storageAccess` : `ServiceFileAccessObsidian`
- `databaseFileAccess` : `ServiceDatabaseFileAccess`
- `fileHandler` : `ServiceFileHandler`
- `rebuilder` : `ServiceRebuilder`
- Not so long from now, ServiceFileAccessObsidian might be abstracted to a more general FileAccessService, and make more testable and maintainable.
- These properties are initialised in `initialiseServiceModules` on `ObsidianLiveSyncPlugin`.
- They are `ServiceModule`s.
- Which means they do not use dynamic binding themselves, but they use bound services.
- ServiceModules are in src/lib/src/serviceModules for common implementations, and src/serviceModules for Obsidian-specific implementations.
- Hence, now all ambiguous properties of `ObsidianLiveSyncPlugin` are initialised explicitly. We can proceed to testing.
- Well, I will release v0.25.44 after testing this.
- Conflict service is now responsible for `resolveAllConflictedFilesByNewerOnes` function, which has been in the rebuilder.
- New functions `updateSettings`, and `applyPartial` have been added to the setting service. We should use these functions instead of directly writing the settings on `ObsidianLiveSyncPlugin.setting`.
- Some interfaces for services have been moved to src/lib/src/interfaces.
- `RemoteService.tryResetDatabase` and `tryCreateDatabase` are now moved to the replicator service.
- You know that these functions are surely performed by the replicator.
- Probably, most of the functions in `RemoteService` should be moved to the replicator service, but for now, these two functions are moved as they are the most related ones, to rewrite the rebuilder service.
- Common functions are gradually moved to the common library.
- Now, binding functions on modules have been delayed until the services and service modules are initialised, to avoid fragile behaviour.
## 0.25.43-patched-4
16th February, 2026
I have been working on it little by little in my spare time. Sorry for the delayed response for issues! ! However, thanks for your patience, we seems the `revert to 0.25.43` is not necessary, and I will keep going with this version.
### Refactored
- No longer `DatabaseService` is an injectable service. It is now actually a service which has its own handlers. No dynamic binding for necessary functions.
- Now the following properties of `ObsidianLiveSyncPlugin` belong to each service:
- `replicator` : `services.replicator` (still we can access `ObsidianLiveSyncPlugin.replicator` for the active replicator)
- A Handy class `UnresolvedErrorManager` has been added, which is responsible for managing unresolved errors and their handlers (we will see `unresolved errors` on a red-background-banner in the editor when they occur).
- This manager can be used to handle unresolved errors in a unified way, and it can also be used to display notifications or something when unresolved errors occur.
## 0.25.43-patched-3
16th February, 2026
### Refactored
- Now following properties of `ObsidianLiveSyncPlugin` belong to each service:
- property : service (still we can access these properties from `ObsidianLiveSyncPlugin` for better usability, but probably we should access these from services to clarify the dependencies)
- `localDatabase` : `services.database`
- `managers` : `services.database`
- `simpleStore` : `services.keyValueDB`
- `kvDB`: `services.keyValueDB`
- Initialising modules, addOns, and services are now explicitly separated in the `_startUp` function of the main plug-in class.
- LiveSyncLocalDB now depends more explicitly on specified services, not the whole `ServiceHub`.
- New service `keyValueDB` has been added. This had been separated from the `database` service.
- Non-trivial modules, such as `ModuleExtraSyncObsidian` (which only holds deviceAndVaultName), are simply implemented in the service.
- Add `logUtils` for unifying logging method injection and formatting. This utility is able to accept the API service for log writing.
- `ModuleKeyValueDB` has been removed, and its functionality is now implemented in the `keyValueDB` service.
- `ModulePouchDB` and `ModuleLocalDatabaseObsidian` have been removed, and their functionality is now implemented in the `database` service.
- Please be aware that you have overridden createPouchDBInstance or something by dynamic binding; you should now override the createPouchDBInstance in the database service instead of using the module.
- You can refer to the `DirectFileManipulatorV2` for an example of how to override the createPouchDBInstance function in the database service.
## 0.25.43-patched-2
14th February, 2026
### Fixed
- Application LifeCycle has now started in Main, not ServiceHub.
- Indeed, ServiceHub cannot be known other things in main have got ready, so it is quite natural to start the lifecycle in main.
## 0.25.43-patched-1
13th February, 2026
**NOTE: Hidden File Sync and Customisation Sync may not work in this version.**
Just a heads-up: this is a patch version, which is essentially a beta release. Do not worry about the following memos, as they are indeed freaking us out. I trust that you have thought this was too large; you're right.
If this cannot be stable, I will revert to 0.24.43 and try again.
### Refactored
- Now resolving unexpected and inexplicable dependency order issues...
- The function which is able to implement to the service is now moved to each service.
- AppLifecycleService.performRestart
- VaultService.isTargetFile is now using multiple checkers instead of a single function.
- This change allows better separation of concerns and easier extension in the future.
- Application LifeCycle has now started in ServiceHub, not ObsidianMenuModule.
- It was in a QUITE unexpected place..., isn't it?
- Instead of, we should call `await this.services.appLifecycle.onReady()` in other platforms.
- As in the browser platform, it will be called at `DOMContentLoaded` event.
- ModuleTargetFilter, which is responsible for parsing ignore files, has been refined.
- This should be separated to a TargetFilter and an IgnoreFileFilter for better maintainability.
- Using `API.addCommand` or some Obsidian API and shimmer APIs, Many modules have been refactored to be derived to AbstractModule from AbstractObsidianModule, to clarify the dependencies. (we should make `app` usage clearer...)
- Fixed initialising `storageAccess` too late in `FileAccessObsidian` module (I am still wondering why it worked before...).
- Remove some redundant overrides in modules.
### Planned
- Some services have an ambiguous name, such as `Injectable`. These will be renamed in the future for better clarity.
- Following properties of `ObsidianLiveSyncPlugin` should be initialised more explicitly:
- property : where it is initialised currently
- `localDatabase` : `ModuleLocalDatabaseObsidian`
- `managers` : `ModuleLocalDatabaseObsidian`
- `replicator` : `ModuleReplicator`
- `simpleStore` : `ModuleKeyValueDB`
- `storageAccess` : `ModuleFileAccessObsidian`
- `databaseFileAccess` : `ModuleDatabaseFileAccess`
- `fileHandler` : `ModuleFileHandler`
- `rebuilder` : `ModuleRebuilder`
- `kvDB`: `ModuleKeyValueDB`
- And I think that having a feature in modules directly is not good for maintainability, these should be separated to some module (loader) and implementation (not only service, but also independent something).
- Plug-in statuses such as requestCount, responseCount... should be moved to a status service or somewhere for better separation of concerns.
## 0.25.43
5th, February, 2026
### Fixed
- Encryption/decryption issues when using Object Storage as remote have been fixed.
- Now the plug-in falls back to V1 encryption/decryption when V2 fails (if not configured as ForceV1).
- This may fix the issue reported in #772.
### Notice
Quite a few packages have been updated in this release. Please report if you find any unexpected behaviour after this update.
## 0.25.42
2nd, February, 2026
This release is identical to 0.25.41-patched-3, except for the version number.
### Refactored
- Now the service context is `protected` instead of `private` in `ServiceBase`.
- This change allows derived classes to access the context directly.
- Some dynamically bound services have been moved to services for better dependency management.
- `WebPeer` has been moved to the main repository from the sub repository `livesync-commonlib` for correct dependency management.
- Migrated from the outdated, unstable platform abstraction layer to services.
- A bit more services will be added in the future for better maintainability.
## 0.25.41
24th January, 2026
### Fixed
- No longer `No available splitter for settings!!` errors occur after fetching old remote settings while rebuilding local database. (#748)
### Improved
- Boot sequence warning is now kept in the in-editor notification area.
### New feature
- We can now set the maximum modified time for reflect events in the settings. (for #754)
- This setting can be configured from `Patches` -> `Remediation` in the settings dialogue.
- Enabling this setting will restrict the propagation from the database to storage to only those changes made before the specified date and time.
- This feature is primarily intended for recovery purposes. After placing `redflag.md` in an empty vault and importing the Self-hosted LiveSync configuration, please perform this configuration, and then fetch the local database from the remote.
- This feature is useful when we want to prevent recent unwanted changes from being reflected in the local storage.
### Refactored
- Module to service refactoring has been started for better maintainability:
- UI module has been moved to UI service.
### Behaviour change
- Default chunk splitter version has been changed to `Rabin-Karp` for new installations.
## 0.25.40
23rd January, 2026
### Fixed
- Fixed an issue where some events were not triggered correctly after the refactoring in 0.25.39.
## 0.25.39
23rd January, 2026
Also no behaviour changes or fixes in this release. Just refactoring for better maintainability. Thank you for your patience! I will address some of the reported issues soon.
However, this is not a minor refactoring, so please be careful. Let me know if you find any unexpected behaviour after this update.
### Refactored
- Rewrite the service's binding/handler assignment systems
- Removed loopholes that allowed traversal between services to clarify dependencies.
- Consolidated the hidden state-related state, the handler, and the addition of bindings to the handler into a single object.
- Currently, functions that can have handlers added implement either addHandler or setHandler directly on the function itself.
I understand there are differing opinions on this, but for now, this is how it stands.
- Services now possess a Context. Please ensure each platform has a class that inherits from ServiceContext.
- To permit services to be dynamically bound, the services themselves are now defined by interfaces.
## 0.25.38
17th January, 2026
### Fixed
- Fixed an issue where indexedDB would not close correctly on some environments, causing unexpected errors during database operations.
## 0.25.37
15th January, 2026
Thank you for your patience until my return!
This release contains minor changes discovered and fixed during test implementation.
There are no changes affecting usage.
### Refactored
- Logging system has been slightly refactored to improve maintainability.
- Some import statements have been unified.
## 0.25.36
25th December, 2025
### Improved
- Now the garbage collector (V3) has been implemented. (Beta)
- This garbage collector ensures that all devices are synchronised to the latest progress to prevent inconsistencies.
- In other words, it makes sure that no new conflicts would have arisen.
- This feature requires additional information (via node information), but it should be more reliable.
- This feature requires all devices have v0.25.36 or later.
- After the garbage collector runs, the database size may be reduced (Compaction will be run automatically after GC).
- We should have an administrative privilege on the remote database to run this garbage collector.
- Now the plug-in and device information is stored in the remote database.
- This information is used for the garbage collector (V3).
- Some additional features may be added in the future using this information.
## 0.25.35
24th December, 2025
Sorry for a small release! I would like to keep things moving along like this if possible. After all, the holidays seem to be starting soon. I will be doubled by my business until the 27th though, indeed.
### Fixed
- Now the conflict resolution dialogue shows correctly which device only has older APIs (#764).
## 0.25.34
10th December, 2025
### Behaviour change
- The plug-in automatically fetches the missing chunks even if `Fetch chunks on demand` is disabled.
- This change is to avoid loss of data when receiving a bulk of revisions.
- This can be prevented by enabling `Use Only Local Chunks` in the settings.
- Storage application now saved during each event and restored on startup.
- Synchronisation result application is also now saved during each event and restored on startup.
- These may avoid some unexpected loss of data when the editor crashes.
### Fixed
- Now the plug-in waits for the application of pended batch changes before the synchronisation starts.
- This may avoid some unexpected loss or unexpected conflicts.
Plug-in sends custom headers correctly when RequestAPI is used.
- No longer causing unexpected chunk creation during `Reset synchronisation on This Device` with bucket sync.
### Refactored
- Synchronisation result application process has been refactored.
- Storage application process has been refactored.
- Please report if you find any unexpected behaviour after this update. A bit of large refactoring.
## 0.25.33
05th December, 2025
### New feature
- We can analyse the local database with the `Analyse database usage` command.
- This command makes a TSV-style report of the database usage, which can be pasted into spreadsheet applications.
- The report contains the number of unique chunks and shared chunks for each document revision.
- Unique chunks indicate the actual consumption.
- Shared chunks indicate the reference counts from other chunks with no consumption.
- We can find which notes or files are using large amounts of storage in the database. Or which notes cannot share chunks effectively.
- This command is useful when optimising the database size or investigating an unexpectedly large database size.
- We can reset the notification threshold and check the remote usage at once with the `Reset notification threshold and check the remote database usage` command.
- Commands are available from the Command Palette, or `Hatch` pane in the settings dialogue.
### Fixed
- Now the plug-in resets the remote size notification threshold after rebuild.
## 0.25.32
02nd December, 2025
Now I am back from a short (?) break! Thank you all for your patience. (It is nothing major, but the first half of the year has finally come to an end).
Anyway, I will release the things a bit by bit. I think that we need a rehabilitation or getting gears in again.
### Improved
- Now the plugin warns when we are in several file-related situations that may cause unexpected behaviour (#300).
- These errors are displayed alongside issues such as file size exceeding limits.
- Such situations include:
- When the document has a name which is not supported by some file systems.
- When the vault has the same file names with different letter cases.
## 0.25.31
18th November, 2025
### Fixed
- Now fetching configuration from the server can handle the empty remote correctly (reported on #756).
- No longer asking to switch adapters during rebuilding.
# 0.25
(0.25.0 through 0.25.30)
Since 19th July, 2025 (beta1 in 0.25.0-beta1, 13th July, 2025)
@@ -9,6 +436,7 @@ I have now rewritten the E2EE code to be more robust and easier to understand. I
As a result, this is the first time in a while that forward compatibility has been broken. We have also taken the opportunity to change all metadata to use encryption rather than obfuscation. Furthermore, the `Dynamic Iteration Count` setting is now redundant and has been moved to the `Patches` pane in the settings. Thanks to Rabin-Karp, the eden setting is also no longer necessary and has been relocated accordingly. Therefore, v0.25.0 represents a legitimate and correct evolution.
---
## 0.25.30
17th November, 2025

Some files were not shown because too many files have changed in this diff Show More