public/obsidian-livesync
1
0
Fork 0
mirror of https://github.com/vrtmrz/obsidian-livesync.git synced 2026-03-03 16:38:48 +00:00
Code Issues Packages Projects Releases Wiki Activity

Compare commits

base: public:0.24.0.dev-rc4
Branches Tags
public:main public:snyk-fix-34b8981b68f7111ada8b22377113a98c public:beta public:snyk-upgrade-7fd31e6f474568014502e7205847f257 public:snyk-fix-90f3b71450f7543a1c61724d8afa0e71 public:refactor_services_2 public:refactor_services public:dev public:snyk-upgrade-fe9625385aa4be849c7157033fd2abb0 public:snyk-upgrade-99a31e70c822894595544b0c9813b11a public:svelteui public:disenchant public:snyk-fix-61f76ef0bce2a276df9552c2fc80a14f public:snyk-upgrade-2baa7938ba318464c3bad5c3fd5cbda5 public:snyk-upgrade-7af62a575eab20e8fb4e9565bf1adcfc public:snyk-upgrade-b32f281b31a81483f126b8fb6c54a957 public:snyk-upgrade-bd61b1103442d3faecc9fc726c34e9a4 public:snyk-upgrade-5b2d6584430c061b5187611a5d7fb64a public:snyk-fix-0e9fbbcb374fcb8d5e8cb027d8414fb7 public:snyk-upgrade-5242daf91e706d1ae500c9ea6545636e public:snyk-upgrade-70434a6822fbad83424135dfad9df909 public:snyk-upgrade-8fa01ccad0352125f1866d41e57d35e1 public:snyk-upgrade-39eb97b4655e70091f718a88fccd57c1 public:snyk-upgrade-2b99d54d19e207887e36c3071b796925 public:forprivacy public:snyk-upgrade-dad9088d0a333677e0157e3deef7fc38 public:snyk-upgrade-b9a76d74b3e5723759af5b3319aabb80 public:snyk-upgrade-7da6459d8ff5d293d80de9d80061d912 public:snyk-upgrade-6a3786fc4cbe394542e9d1b6058c46e7 public:snyk-upgrade-8fc9e62771dc41b0f23dc0efefb5c949
public:0.25.50 public:0.25.49 public:0.25.48 public:0.25.47 public:0.25.46 public:0.25.45 public:0.25.44 public:0.25.43-patched-9 public:0.25.43-patched-8 public:0.25.43-patched-7 public:0.25.43-patched-6 public:0.25.43-patched-5 public:0.25.43-patched-4 public:0.25.43-patched-3 public:0.25.43-patched-2 public:0.25.43-patched-1 public:0.25.43 public:0.25.42 public:0.25.41-patched-3 public:0.25.41-patched-2 public:0.25.41-patched-1 public:0.25.41 public:0.25.40 public:0.25.39 public:0.25.38 public:0.25.37 public:0.25.36 public:0.25.35 public:0.25.34 public:0.25.33 public:0.25.32 public:0.25.31 public:0.25.30 public:0.25.30-dev public:0.25.29 public:0.25.28 public:0.25.27 public:0.25.26 public:0.25.25 public:0.25.24 public:0.25.24.beta3 public:0.25.24.beta2 public:0.25.24.beta1 public:0.25.23 public:0.25.23.beta1 public:0.25.22 public:0.25.21 public:0.25.21.beta2 public:0.25.21.beta1 public:0.25.20 public:0.25.19 public:0.25.18 public:0.25.17 public:0.25.16 public:0.25.15 public:0.25.14 public:0.25.13 public:0.25.12 public:0.25.11 public:0.25.10 public:0.25.9 public:0.25.8 public:0.25.7 public:0.25.6 public:0.25.5 public:0.25.4 public:0.25.3 public:0.25.2 public:0.25.1 public:0.25.0 public:0.25.0-beta1 public:0.24.31 public:0.24.30 public:0.24.29 public:0.24.28 public:0.24.27 public:0.24.26 public:0.24.25 public:0.24.24 public:0.24.23 public:0.24.22 public:0.24.21 public:0.24.20 public:0.24.19 public:0.24.18 public:0.24.17 public:0.24.16 public:0.24.15 public:0.24.14 public:0.24.13 public:0.24.12 public:0.24.11 public:0.24.10 public:0.24.9 public:0.24.8 public:0.24.7 public:0.24.6 public:0.24.5 public:0.24.4 public:0.24.3 public:0.24.2 public:0.24.1 public:0.24.0 public:0.24.0.dev-rc8 public:0.24.0.dev-rc7 public:0.24.0.dev-rc6 public:0.24.0.dev-rc5 public:0.24.0.dev-rc4 public:0.24.0.dev-rc3 public:0.24.0.dev-rc2 public:0.23.23 public:0.23.22 public:0.23.21 public:0.23.20 public:0.23.19 public:0.23.18 public:0.23.17 public:0.23.16 public:0.23.15 public:0.23.14 public:0.23.13 public:0.23.12 public:0.23.11 public:0.23.10 public:0.23.9 public:0.23.8 public:0.23.7 public:0.23.6 public:0.23.5 public:0.23.4 public:0.23.3 public:0.23.2 public:0.23.1 public:0.23.0 public:0.22.19 public:0.22.18 public:0.22.17 public:0.22.16 public:0.22.15 public:0.22.14 public:0.22.13 public:0.22.12 public:0.22.11 public:0.22.10 public:0.22.9 public:0.22.8 public:0.22.7 public:0.22.6 public:0.22.5 public:0.22.4 public:0.22.3 public:0.22.2 public:0.22.1 public:0.22.0 public:0.21.5 public:0.21.4 public:0.21.3 public:0.21.2 public:0.21.1 public:0.21.0 public:0.20.7 public:0.20.6 public:0.20.5 public:0.20.4 public:0.20.3 public:0.20.2 public:0.20.1 public:0.20.0 public:0.19.23 public:0.19.22 public:0.19.21 public:0.19.20 public:0.19.19 public:0.19.18 public:0.19.17 public:0.19.16 public:0.19.15 public:0.19.14 public:0.19.13 public:0.19.12 public:0.19.11 public:0.19.10 public:0.19.9 public:0.19.8 public:0.19.7 public:0.19.6 public:0.19.5 public:0.19.4 public:0.19.3 public:0.19.2 public:0.19.1 public:0.19.0 public:0.18.6 public:0.18.5 public:0.18.4 public:0.18.3 public:0.18.2 public:0.18.1 public:0.18.0 public:0.17.34 public:0.17.33 public:0.17.32 public:0.17.31 public:0.17.30 public:0.17.29 public:0.17.28 public:0.17.27 public:0.17.26 public:0.17.25 public:0.17.24 public:0.17.23 public:0.17.22 public:0.17.21 public:0.17.20 public:0.17.19 public:0.17.18 public:0.17.17 public:0.17.16 public:0.17.15 public:0.17.14 public:0.17.13 public:0.17.12 public:0.17.11 public:0.17.10 public:0.17.9 public:0.17.8 public:0.17.7 public:0.17.6 public:0.17.5 public:0.17.4 public:0.17.3 public:0.17.2 public:0.17.1 public:0.17.0 public:0.16.8 public:0.16.7 public:0.16.6 public:0.16.5 public:0.16.4 public:0.16.3 public:0.16.2 public:0.16.1 public:0.16.0 public:0.15.10 public:0.15.9 public:0.15.8 public:0.15.7 public:0.15.6 public:0.15.5 public:0.15.4 public:0.15.3 public:0.15.2 public:0.15.1 public:0.15.0 public:0.14.8 public:0.14.7 public:0.14.6 public:0.14.5 public:0.14.4 public:0.14.3 public:0.14.2 public:0.14.1 public:0.14.0 public:0.13.4 public:0.13.3 public:0.13.2 public:0.13.1 public:0.13.0 public:0.12.5 public:0.12.4 public:0.12.3 public:0.12.2 public:0.12.1 public:0.12.0 public:0.11.10 public:0.11.9 public:0.11.8 public:0.11.7 public:0.11.6 public:0.11.5 public:0.11.4 public:0.11.3 public:0.11.2 public:0.11.1 public:0.11.0 public:0.10.1 public:0.10.0 public:0.9.0 public:0.8.8 public:0.8.7 public:0.8.6 public:0.8.5 public:0.8.3 public:0.8.2 public:0.8.1 public:0.8.0 public:0.7.2 public:0.7.1 public:0.7.0 public:0.6.1 public:0.6.0 public:0.5.0 public:0.4.1 public:0.4.0 public:0.3.1 public:0.3.0 public:0.2.1 public:0.2.0 public:0.1.26 public:0.1.25 public:0.1.24 public:0.1.23 public:0.1.22 public:0.1.21 public:0.1.20 public:0.1.19 public:0.1.18 public:0.1.17 public:0.1.16 public:0.1.15 public:0.1.14 public:0.1.13 public:0.1.12 public:0.1.11 public:0.1.10 public:0.1.9 public:0.1.8 public:0.1.7 public:0.1.6 public:0.1.5 public:0.1.4 public:0.1.3 public:0.1.2 public:0.1.1 public:0.1.0 public:0.0.9 public:0.0.8 public:0.0.7 public:0.0.6
...
compare: public:0.24.0
Branches Tags
public:main public:snyk-fix-34b8981b68f7111ada8b22377113a98c public:beta public:snyk-upgrade-7fd31e6f474568014502e7205847f257 public:snyk-fix-90f3b71450f7543a1c61724d8afa0e71 public:refactor_services_2 public:refactor_services public:dev public:snyk-upgrade-fe9625385aa4be849c7157033fd2abb0 public:snyk-upgrade-99a31e70c822894595544b0c9813b11a public:svelteui public:disenchant public:snyk-fix-61f76ef0bce2a276df9552c2fc80a14f public:snyk-upgrade-2baa7938ba318464c3bad5c3fd5cbda5 public:snyk-upgrade-7af62a575eab20e8fb4e9565bf1adcfc public:snyk-upgrade-b32f281b31a81483f126b8fb6c54a957 public:snyk-upgrade-bd61b1103442d3faecc9fc726c34e9a4 public:snyk-upgrade-5b2d6584430c061b5187611a5d7fb64a public:snyk-fix-0e9fbbcb374fcb8d5e8cb027d8414fb7 public:snyk-upgrade-5242daf91e706d1ae500c9ea6545636e public:snyk-upgrade-70434a6822fbad83424135dfad9df909 public:snyk-upgrade-8fa01ccad0352125f1866d41e57d35e1 public:snyk-upgrade-39eb97b4655e70091f718a88fccd57c1 public:snyk-upgrade-2b99d54d19e207887e36c3071b796925 public:forprivacy public:snyk-upgrade-dad9088d0a333677e0157e3deef7fc38 public:snyk-upgrade-b9a76d74b3e5723759af5b3319aabb80 public:snyk-upgrade-7da6459d8ff5d293d80de9d80061d912 public:snyk-upgrade-6a3786fc4cbe394542e9d1b6058c46e7 public:snyk-upgrade-8fc9e62771dc41b0f23dc0efefb5c949
public:0.25.50 public:0.25.49 public:0.25.48 public:0.25.47 public:0.25.46 public:0.25.45 public:0.25.44 public:0.25.43-patched-9 public:0.25.43-patched-8 public:0.25.43-patched-7 public:0.25.43-patched-6 public:0.25.43-patched-5 public:0.25.43-patched-4 public:0.25.43-patched-3 public:0.25.43-patched-2 public:0.25.43-patched-1 public:0.25.43 public:0.25.42 public:0.25.41-patched-3 public:0.25.41-patched-2 public:0.25.41-patched-1 public:0.25.41 public:0.25.40 public:0.25.39 public:0.25.38 public:0.25.37 public:0.25.36 public:0.25.35 public:0.25.34 public:0.25.33 public:0.25.32 public:0.25.31 public:0.25.30 public:0.25.30-dev public:0.25.29 public:0.25.28 public:0.25.27 public:0.25.26 public:0.25.25 public:0.25.24 public:0.25.24.beta3 public:0.25.24.beta2 public:0.25.24.beta1 public:0.25.23 public:0.25.23.beta1 public:0.25.22 public:0.25.21 public:0.25.21.beta2 public:0.25.21.beta1 public:0.25.20 public:0.25.19 public:0.25.18 public:0.25.17 public:0.25.16 public:0.25.15 public:0.25.14 public:0.25.13 public:0.25.12 public:0.25.11 public:0.25.10 public:0.25.9 public:0.25.8 public:0.25.7 public:0.25.6 public:0.25.5 public:0.25.4 public:0.25.3 public:0.25.2 public:0.25.1 public:0.25.0 public:0.25.0-beta1 public:0.24.31 public:0.24.30 public:0.24.29 public:0.24.28 public:0.24.27 public:0.24.26 public:0.24.25 public:0.24.24 public:0.24.23 public:0.24.22 public:0.24.21 public:0.24.20 public:0.24.19 public:0.24.18 public:0.24.17 public:0.24.16 public:0.24.15 public:0.24.14 public:0.24.13 public:0.24.12 public:0.24.11 public:0.24.10 public:0.24.9 public:0.24.8 public:0.24.7 public:0.24.6 public:0.24.5 public:0.24.4 public:0.24.3 public:0.24.2 public:0.24.1 public:0.24.0 public:0.24.0.dev-rc8 public:0.24.0.dev-rc7 public:0.24.0.dev-rc6 public:0.24.0.dev-rc5 public:0.24.0.dev-rc4 public:0.24.0.dev-rc3 public:0.24.0.dev-rc2 public:0.23.23 public:0.23.22 public:0.23.21 public:0.23.20 public:0.23.19 public:0.23.18 public:0.23.17 public:0.23.16 public:0.23.15 public:0.23.14 public:0.23.13 public:0.23.12 public:0.23.11 public:0.23.10 public:0.23.9 public:0.23.8 public:0.23.7 public:0.23.6 public:0.23.5 public:0.23.4 public:0.23.3 public:0.23.2 public:0.23.1 public:0.23.0 public:0.22.19 public:0.22.18 public:0.22.17 public:0.22.16 public:0.22.15 public:0.22.14 public:0.22.13 public:0.22.12 public:0.22.11 public:0.22.10 public:0.22.9 public:0.22.8 public:0.22.7 public:0.22.6 public:0.22.5 public:0.22.4 public:0.22.3 public:0.22.2 public:0.22.1 public:0.22.0 public:0.21.5 public:0.21.4 public:0.21.3 public:0.21.2 public:0.21.1 public:0.21.0 public:0.20.7 public:0.20.6 public:0.20.5 public:0.20.4 public:0.20.3 public:0.20.2 public:0.20.1 public:0.20.0 public:0.19.23 public:0.19.22 public:0.19.21 public:0.19.20 public:0.19.19 public:0.19.18 public:0.19.17 public:0.19.16 public:0.19.15 public:0.19.14 public:0.19.13 public:0.19.12 public:0.19.11 public:0.19.10 public:0.19.9 public:0.19.8 public:0.19.7 public:0.19.6 public:0.19.5 public:0.19.4 public:0.19.3 public:0.19.2 public:0.19.1 public:0.19.0 public:0.18.6 public:0.18.5 public:0.18.4 public:0.18.3 public:0.18.2 public:0.18.1 public:0.18.0 public:0.17.34 public:0.17.33 public:0.17.32 public:0.17.31 public:0.17.30 public:0.17.29 public:0.17.28 public:0.17.27 public:0.17.26 public:0.17.25 public:0.17.24 public:0.17.23 public:0.17.22 public:0.17.21 public:0.17.20 public:0.17.19 public:0.17.18 public:0.17.17 public:0.17.16 public:0.17.15 public:0.17.14 public:0.17.13 public:0.17.12 public:0.17.11 public:0.17.10 public:0.17.9 public:0.17.8 public:0.17.7 public:0.17.6 public:0.17.5 public:0.17.4 public:0.17.3 public:0.17.2 public:0.17.1 public:0.17.0 public:0.16.8 public:0.16.7 public:0.16.6 public:0.16.5 public:0.16.4 public:0.16.3 public:0.16.2 public:0.16.1 public:0.16.0 public:0.15.10 public:0.15.9 public:0.15.8 public:0.15.7 public:0.15.6 public:0.15.5 public:0.15.4 public:0.15.3 public:0.15.2 public:0.15.1 public:0.15.0 public:0.14.8 public:0.14.7 public:0.14.6 public:0.14.5 public:0.14.4 public:0.14.3 public:0.14.2 public:0.14.1 public:0.14.0 public:0.13.4 public:0.13.3 public:0.13.2 public:0.13.1 public:0.13.0 public:0.12.5 public:0.12.4 public:0.12.3 public:0.12.2 public:0.12.1 public:0.12.0 public:0.11.10 public:0.11.9 public:0.11.8 public:0.11.7 public:0.11.6 public:0.11.5 public:0.11.4 public:0.11.3 public:0.11.2 public:0.11.1 public:0.11.0 public:0.10.1 public:0.10.0 public:0.9.0 public:0.8.8 public:0.8.7 public:0.8.6 public:0.8.5 public:0.8.3 public:0.8.2 public:0.8.1 public:0.8.0 public:0.7.2 public:0.7.1 public:0.7.0 public:0.6.1 public:0.6.0 public:0.5.0 public:0.4.1 public:0.4.0 public:0.3.1 public:0.3.0 public:0.2.1 public:0.2.0 public:0.1.26 public:0.1.25 public:0.1.24 public:0.1.23 public:0.1.22 public:0.1.21 public:0.1.20 public:0.1.19 public:0.1.18 public:0.1.17 public:0.1.16 public:0.1.15 public:0.1.14 public:0.1.13 public:0.1.12 public:0.1.11 public:0.1.10 public:0.1.9 public:0.1.8 public:0.1.7 public:0.1.6 public:0.1.5 public:0.1.4 public:0.1.3 public:0.1.2 public:0.1.1 public:0.1.0 public:0.0.9 public:0.0.8 public:0.0.7 public:0.0.6

17 Commits
0.24.0.dev ... 0.24.0

Author SHA1 Message Date
vorotamoroz
a2b36ccf31 v0.24.0! 2024-10-28 11:18:29 +09:00
vorotamoroz
25e30fa09d Merge tag '0.24.0.dev-rc8' 2024-10-28 10:24:38 +09:00
vorotamoroz
8f5bc387b4 Update manifest-beta.json 2024-10-25 20:44:32 +09:00
vorotamoroz
5afe24c460 0.24.0.dev-rc8 2024-10-25 12:39:32 +01:00
vorotamoroz
658a09f1cc Update manifest-beta.json 2024-10-24 20:46:08 +09:00
vorotamoroz
a9020a3aea 0.24.0.dev-rc7 2024-10-24 12:41:50 +01:00
vorotamoroz
293c731437 Update manifest-beta.json 2024-10-22 18:11:18 +09:00
vorotamoroz
5b4ae37030 0.24.0.dev-rc6 2024-10-22 10:08:20 +01:00
vorotamoroz
9e8d126259 Update manifest-beta.json 2024-10-21 17:52:28 +09:00
vorotamoroz
6d244a6e34 0.24.0.dev-rc5 2024-10-21 09:47:09 +01:00
vorotamoroz
1f0ad4eb1e Update manifest-beta.json 2024-10-18 19:23:48 +09:00
vorotamoroz
5023d6da0b Merge pull request #474 from nyawox/per-file-sync-grammar 
fix: per-file customization sync description grammar
 2024-10-18 11:43:03 +09:00
vorotamoroz
49160c7d57 Merge branch 'main' into per-file-sync-grammar 2024-10-18 11:42:49 +09:00
vorotamoroz
4434224c29 Merge pull request #479 from fuhrysteve/patch-1 
add username and password to setup URI instructions
 2024-10-18 11:40:23 +09:00
vorotamoroz
cf3b9e5522 Add manifest beta 2024-10-17 10:26:19 +01:00
Stephen J. Fuhry
f778107727 add username and password to setup URI instructions 2024-08-07 14:43:38 -04:00
nyawox
12d825ea49 fix: per-file customization sync description grammar 2024-07-26 23:38:01 +09:00
13 changed files with 444 additions and 352 deletions
Expand all files Collapse all files

4
docs/setup_own_server.md
View File

@@ -106,6 +106,8 @@ Now `https://tiles-photograph-routine-groundwater.trycloudflare.com` is our serv
$ export hostname=https://tiles-photograph-routine-groundwater.trycloudflare.com #Point to your vault
$ export database=obsidiannotes #Please change as you like
$ export passphrase=dfsapkdjaskdjasdas #Please change as you like
$ export username=johndoe
$ export password=abc123
$ deno run -A https://raw.githubusercontent.com/vrtmrz/obsidian-livesync/main/utils/flyio/generate_setupuri.ts
obsidian://setuplivesync?settings=%5B%22tm2DpsOE74nJAryprZO2M93wF%2Fvg.......4b26ed33230729%22%5D
@@ -206,4 +208,4 @@ entryPoints:
address: ":443"
...
```
```

2
manifest-beta.json
View File

@@ -1,7 +1,7 @@
{
"id": "obsidian-livesync",
"name": "Self-hosted LiveSync",
"version": "0.24.0.dev-rc4",
"version": "0.24.0",
"minAppVersion": "0.9.12",
"description": "Community implementation of self-hosted livesync. Reflect your vault changes to some other devices immediately. Please make sure to disable other synchronize solutions to avoid content corruption or duplication.",
"author": "vorotamoroz",

2
manifest.json
View File

@@ -1,7 +1,7 @@
{
"id": "obsidian-livesync",
"name": "Self-hosted LiveSync",
"version": "0.23.23",
"version": "0.24.0",
"minAppVersion": "0.9.12",
"description": "Community implementation of self-hosted livesync. Reflect your vault changes to some other devices immediately. Please make sure to disable other synchronize solutions to avoid content corruption or duplication.",
"author": "vorotamoroz",

4
package-lock.json generated
View File

@@ -1,12 +1,12 @@
{
"name": "obsidian-livesync",
"version": "0.24.0.dev-rc4",
"version": "0.24.0",
"lockfileVersion": 2,
"requires": true,
"packages": {
"": {
"name": "obsidian-livesync",
"version": "0.24.0.dev-rc4",
"version": "0.24.0",
"license": "MIT",
"dependencies": {
"@aws-sdk/client-s3": "^3.645.0",

2
package.json
View File

@@ -1,6 +1,6 @@
{
"name": "obsidian-livesync",
"version": "0.24.0.dev-rc4",
"version": "0.24.0",
"description": "Reflect your vault changes to some other devices immediately. Please make sure to disable other synchronize solutions to avoid content corruption or duplication.",
"main": "main.js",
"type": "module",

186
src/features/ConfigSync/CmdConfigSync.ts
View File

@@ -5,7 +5,6 @@ import type { EntryDoc, LoadedEntry, InternalFileEntry, FilePathWithPrefix, File
import { CANCELLED, LEAVE_TO_SUBSEQUENT, LOG_LEVEL_DEBUG, LOG_LEVEL_INFO, LOG_LEVEL_NOTICE, LOG_LEVEL_VERBOSE, MODE_SELECTIVE, MODE_SHINY } from "../../lib/src/common/types.ts";
import { ICXHeader, PERIODIC_PLUGIN_SWEEP, } from "../../common/types.ts";
import { createBlob, createSavingEntryFromLoadedEntry, createTextBlob, delay, fireAndForget, getDocData, getDocDataAsArray, isDocContentSame, isLoadedEntry, isObjectDifferent } from "../../lib/src/common/utils.ts";
import { Logger } from "../../lib/src/common/logger.ts";
import { digestHash } from "../../lib/src/string_and_binary/hash.ts";
import { arrayBufferToBase64, decodeBinary, readString } from '../../lib/src/string_and_binary/convert.ts';
import { serialized, shareRunningResult } from "../../lib/src/concurrency/lock.ts";
@@ -419,12 +418,12 @@ export class ConfigSync extends LiveSyncCommands implements IObsidianModule {
async $everyOnDatabaseInitialized(showNotice: boolean) {
if (!this._isThisModuleEnabled()) return true;
try {
Logger("Scanning customizations...");
this._log("Scanning customizations...");
await this.scanAllConfigFiles(showNotice);
Logger("Scanning customizations : done");
this._log("Scanning customizations : done");
} catch (ex) {
Logger("Scanning customizations : failed");
Logger(ex, LOG_LEVEL_VERBOSE);
this._log("Scanning customizations : failed");
this._log(ex, LOG_LEVEL_VERBOSE);
}
return true;
}
@@ -477,7 +476,7 @@ export class ConfigSync extends LiveSyncCommands implements IObsidianModule {
xFiles.push(work);
}
if (missingHash) {
Logger(`Digest created for ${path} to improve checking`, LOG_LEVEL_VERBOSE);
this._log(`Digest created for ${path} to improve checking`, LOG_LEVEL_VERBOSE);
wx.data = serialize(data);
fireAndForget(() => this.localDatabase.putDBEntry(createSavingEntryFromLoadedEntry(wx)));
}
@@ -512,8 +511,8 @@ export class ConfigSync extends LiveSyncCommands implements IObsidianModule {
return [];
} catch (ex) {
Logger(`Something happened at enumerating customization :${path}`, LOG_LEVEL_NOTICE);
Logger(ex, LOG_LEVEL_VERBOSE);
this._log(`Something happened at enumerating customization :${path}`, LOG_LEVEL_NOTICE);
this._log(ex, LOG_LEVEL_VERBOSE);
}
return [];
}, { suspended: false, batchSize: 1, concurrentLimit: 10, delay: 100, yieldThreshold: 10, maintainDelay: false, totalRemainingReactiveSource: pluginScanningCount }).startPipeline();
@@ -536,8 +535,8 @@ export class ConfigSync extends LiveSyncCommands implements IObsidianModule {
return [];
} catch (ex) {
Logger(`Something happened at enumerating customization :${path}`, LOG_LEVEL_NOTICE);
Logger(ex, LOG_LEVEL_VERBOSE);
this._log(`Something happened at enumerating customization :${path}`, LOG_LEVEL_NOTICE);
this._log(ex, LOG_LEVEL_VERBOSE);
}
return [];
}, { suspended: false, batchSize: 1, concurrentLimit: 10, delay: 100, yieldThreshold: 10, maintainDelay: false, totalRemainingReactiveSource: pluginScanningCount }).startPipeline();
@@ -583,11 +582,11 @@ export class ConfigSync extends LiveSyncCommands implements IObsidianModule {
if (!loaded) {
const d = await this.localDatabase.getDBEntry(unifiedPathV2);
if (!d) {
Logger(`The file ${unifiedPathV2} is not found`, LOG_LEVEL_VERBOSE);
this._log(`The file ${unifiedPathV2} is not found`, LOG_LEVEL_VERBOSE);
return false;
}
if (!isLoadedEntry(d)) {
Logger(`The file ${unifiedPathV2} is not a note`, LOG_LEVEL_VERBOSE);
this._log(`The file ${unifiedPathV2} is not a note`, LOG_LEVEL_VERBOSE);
return false;
}
loaded = d;
@@ -613,8 +612,8 @@ export class ConfigSync extends LiveSyncCommands implements IObsidianModule {
this.pluginList.filter(e => e instanceof PluginDataExDisplayV2 && e.confKey == confKey).forEach(e => (e as PluginDataExDisplayV2).applyLoadedManifest());
pluginList.set(this.pluginList);
} catch (ex) {
Logger(`The file ${loaded.path} seems to manifest, but could not be decoded as JSON`, LOG_LEVEL_VERBOSE);
Logger(ex, LOG_LEVEL_VERBOSE);
this._log(`The file ${loaded.path} seems to manifest, but could not be decoded as JSON`, LOG_LEVEL_VERBOSE);
this._log(ex, LOG_LEVEL_VERBOSE);
}
this.loadedManifest_mTime.set(confKey, file.mtime);
} else {
@@ -687,22 +686,22 @@ export class ConfigSync extends LiveSyncCommands implements IObsidianModule {
async migrateV1ToV2(showMessage: boolean, entry: AnyEntry): Promise<void> {
const v1Path = entry.path;
Logger(`Migrating ${entry.path} to V2`, showMessage ? LOG_LEVEL_NOTICE : LOG_LEVEL_INFO);
this._log(`Migrating ${entry.path} to V2`, showMessage ? LOG_LEVEL_NOTICE : LOG_LEVEL_INFO);
if (entry.deleted) {
Logger(`The entry ${v1Path} is already deleted`, LOG_LEVEL_VERBOSE);
this._log(`The entry ${v1Path} is already deleted`, LOG_LEVEL_VERBOSE);
return;
}
if (!v1Path.endsWith(".md") && !v1Path.startsWith(ICXHeader)) {
Logger(`The entry ${v1Path} is not a customisation sync binder`, LOG_LEVEL_VERBOSE);
this._log(`The entry ${v1Path} is not a customisation sync binder`, LOG_LEVEL_VERBOSE);
return
}
if (v1Path.indexOf("%") !== -1) {
Logger(`The entry ${v1Path} is already migrated`, LOG_LEVEL_VERBOSE);
this._log(`The entry ${v1Path} is already migrated`, LOG_LEVEL_VERBOSE);
return;
}
const loadedEntry = await this.localDatabase.getDBEntry(v1Path);
if (!loadedEntry) {
Logger(`The entry ${v1Path} is not found`, LOG_LEVEL_VERBOSE);
this._log(`The entry ${v1Path} is not found`, LOG_LEVEL_VERBOSE);
return;
}
@@ -723,7 +722,7 @@ export class ConfigSync extends LiveSyncCommands implements IObsidianModule {
const relativeFilename = f.filename.split("/").slice(deletePrefixCount).join("/");
const v2Path = (prefixPath + relativeFilename) as FilePathWithPrefix;
// console.warn(`Migrating ${v1Path} / ${relativeFilename} to ${v2Path}`);
Logger(`Migrating ${v1Path} / ${relativeFilename} to ${v2Path}`, LOG_LEVEL_VERBOSE);
this._log(`Migrating ${v1Path} / ${relativeFilename} to ${v2Path}`, LOG_LEVEL_VERBOSE);
const newId = await this.plugin.$$path2id(v2Path);
// const buf =
@@ -742,12 +741,12 @@ export class ConfigSync extends LiveSyncCommands implements IObsidianModule {
}
const r = await this.plugin.localDatabase.putDBEntry(saving);
if (r && r.ok) {
Logger(`Migrated ${v1Path} / ${f.filename} to ${v2Path}`, LOG_LEVEL_INFO);
this._log(`Migrated ${v1Path} / ${f.filename} to ${v2Path}`, LOG_LEVEL_INFO);
const delR = await this.deleteConfigOnDatabase(v1Path);
if (delR) {
Logger(`Deleted ${v1Path} successfully`, LOG_LEVEL_INFO);
this._log(`Deleted ${v1Path} successfully`, LOG_LEVEL_INFO);
} else {
Logger(`Failed to delete ${v1Path}`, LOG_LEVEL_NOTICE);
this._log(`Failed to delete ${v1Path}`, LOG_LEVEL_NOTICE);
}
}
}
@@ -802,9 +801,9 @@ export class ConfigSync extends LiveSyncCommands implements IObsidianModule {
}
const fileA = await loadFile(dataA);
const fileB = await loadFile(dataB);
Logger(`Comparing: ${dataA.documentPath} <-> ${dataB.documentPath}`, LOG_LEVEL_VERBOSE);
this._log(`Comparing: ${dataA.documentPath} <-> ${dataB.documentPath}`, LOG_LEVEL_VERBOSE);
if (!fileA || !fileB) {
Logger(`Could not load ${dataA.name} for comparison: ${!fileA ? dataA.term : ""}${!fileB ? dataB.term : ""}`, LOG_LEVEL_NOTICE);
this._log(`Could not load ${dataA.name} for comparison: ${!fileA ? dataA.term : ""}${!fileB ? dataB.term : ""}`, LOG_LEVEL_NOTICE);
return false;
}
let path = stripAllPrefixes(fileA.path.split("/").slice(-1).join("/") as FilePath); // TODO:adjust
@@ -813,15 +812,15 @@ export class ConfigSync extends LiveSyncCommands implements IObsidianModule {
}
if (fileA.path.endsWith(".json")) {
return serialized("config:merge-data", () => new Promise<boolean>((res) => {
Logger("Opening data-merging dialog", LOG_LEVEL_VERBOSE);
this._log("Opening data-merging dialog", LOG_LEVEL_VERBOSE);
// const docs = [docA, docB];
const modal = new JsonResolveModal(this.app, path, [fileA, fileB], async (keep, result) => {
if (result == null) return res(false);
try {
res(await this.applyData(dataA, result));
} catch (ex) {
Logger("Could not apply merged file");
Logger(ex, LOG_LEVEL_VERBOSE);
this._log("Could not apply merged file");
this._log(ex, LOG_LEVEL_VERBOSE);
res(false);
}
}, "Local", `${dataB.term}`, "B", true, true, "Difference between local and remote");
@@ -847,7 +846,7 @@ export class ConfigSync extends LiveSyncCommands implements IObsidianModule {
right: { rev: "B", ...fileB, data: docBData },
diff: diff
}
console.dir(diffResult);
// console.dir(diffResult);
const d = new ConflictResolveModal(this.app, path, diffResult, true, dataB.term);
d.open();
const ret = await d.waitForResult();
@@ -866,7 +865,7 @@ export class ConfigSync extends LiveSyncCommands implements IObsidianModule {
if (content) {
// const dt = createBlob(content);
const filename = data.files[0].filename;
Logger(`Applying ${filename} of ${data.displayName || data.name}..`);
this._log(`Applying ${filename} of ${data.displayName || data.name}..`);
const path = `${baseDir}/${filename}` as FilePath;
await this.plugin.storageAccess.ensureDir(path);
// If the content has applied, modified time will be updated to the current time.
@@ -879,7 +878,7 @@ export class ConfigSync extends LiveSyncCommands implements IObsidianModule {
// If files have applied, modified time will be updated to the current time.
const stat = { mtime: f.mtime, ctime: f.ctime };
const path = `${baseDir}/${f.filename}` as FilePath;
Logger(`Applying ${f.filename} of ${data.displayName || data.name}..`);
this._log(`Applying ${f.filename} of ${data.displayName || data.name}..`);
// const contentEach = createBlob(f.data);
await this.plugin.storageAccess.ensureDir(path);
@@ -888,13 +887,13 @@ export class ConfigSync extends LiveSyncCommands implements IObsidianModule {
try {
oldData = await this.plugin.storageAccess.readHiddenFileBinary(path);
} catch (ex) {
Logger(`Could not read the file ${f.filename}`, LOG_LEVEL_VERBOSE);
Logger(ex, LOG_LEVEL_VERBOSE);
this._log(`Could not read the file ${f.filename}`, LOG_LEVEL_VERBOSE);
this._log(ex, LOG_LEVEL_VERBOSE);
oldData = new ArrayBuffer(0);
}
const content = base64ToArrayBuffer(f.data);
if (await isDocContentSame(oldData, content)) {
Logger(`The file ${f.filename} is already up-to-date`, LOG_LEVEL_VERBOSE);
this._log(`The file ${f.filename} is already up-to-date`, LOG_LEVEL_VERBOSE);
continue;
}
await this.plugin.storageAccess.writeHiddenFileAuto(path, content, stat);
@@ -903,30 +902,30 @@ export class ConfigSync extends LiveSyncCommands implements IObsidianModule {
try {
oldData = await this.plugin.storageAccess.readHiddenFileText(path);
} catch (ex) {
Logger(`Could not read the file ${f.filename}`, LOG_LEVEL_VERBOSE);
Logger(ex, LOG_LEVEL_VERBOSE);
this._log(`Could not read the file ${f.filename}`, LOG_LEVEL_VERBOSE);
this._log(ex, LOG_LEVEL_VERBOSE);
oldData = "";
}
const content = getDocData(f.data);
if (await isDocContentSame(oldData, content)) {
Logger(`The file ${f.filename} is already up-to-date`, LOG_LEVEL_VERBOSE);
this._log(`The file ${f.filename} is already up-to-date`, LOG_LEVEL_VERBOSE);
continue;
}
await this.plugin.storageAccess.writeHiddenFileAuto(path, content, stat);
}
Logger(`Applied ${f.filename} of ${data.displayName || data.name}..`);
this._log(`Applied ${f.filename} of ${data.displayName || data.name}..`);
await this.storeCustomisationFileV2(path, this.plugin.deviceAndVaultName);
}
}
} catch (ex) {
Logger(`Applying ${data.displayName || data.name}.. Failed`, LOG_LEVEL_NOTICE);
Logger(ex, LOG_LEVEL_VERBOSE);
this._log(`Applying ${data.displayName || data.name}.. Failed`, LOG_LEVEL_NOTICE);
this._log(ex, LOG_LEVEL_VERBOSE);
return false;
}
return true;
}
async applyData(data: IPluginDataExDisplay, content?: string): Promise<boolean> {
Logger(`Applying ${data.displayName || data.name
this._log(`Applying ${data.displayName || data.name
}..`);
if (data instanceof PluginDataExDisplayV2) {
@@ -941,7 +940,7 @@ export class ConfigSync extends LiveSyncCommands implements IObsidianModule {
}
const loadedData = deserialize(getDocDataAsArray(dx.data), {}) as PluginDataEx;
for (const f of loadedData.files) {
Logger(`Applying ${f.filename} of ${data.displayName || data.name}..`);
this._log(`Applying ${f.filename} of ${data.displayName || data.name}..`);
try {
// console.dir(f);
const path = `${baseDir}/${f.filename}`;
@@ -952,11 +951,11 @@ export class ConfigSync extends LiveSyncCommands implements IObsidianModule {
} else {
await this.plugin.storageAccess.writeHiddenFileAuto(path, content);
}
Logger(`Applying ${f.filename} of ${data.displayName || data.name}.. Done`);
this._log(`Applying ${f.filename} of ${data.displayName || data.name}.. Done`);
} catch (ex) {
Logger(`Applying ${f.filename} of ${data.displayName || data.name}.. Failed`);
Logger(ex, LOG_LEVEL_VERBOSE);
this._log(`Applying ${f.filename} of ${data.displayName || data.name}.. Failed`);
this._log(ex, LOG_LEVEL_VERBOSE);
}
}
@@ -964,7 +963,7 @@ export class ConfigSync extends LiveSyncCommands implements IObsidianModule {
await this.storeCustomizationFiles(uPath);
await this.updatePluginList(true, uPath);
await delay(100);
Logger(`Config ${data.displayName || data.name} has been applied`, LOG_LEVEL_NOTICE);
this._log(`Config ${data.displayName || data.name} has been applied`, LOG_LEVEL_NOTICE);
if (data.category == "PLUGIN_DATA" || data.category == "PLUGIN_MAIN") {
//@ts-ignore
const manifests = Object.values(this.app.plugins.manifests) as any as PluginManifest[];
@@ -972,20 +971,20 @@ export class ConfigSync extends LiveSyncCommands implements IObsidianModule {
const enabledPlugins = this.app.plugins.enabledPlugins as Set<string>;
const pluginManifest = manifests.find((manifest) => enabledPlugins.has(manifest.id) && manifest.dir == `${baseDir}/plugins/${data.name}`);
if (pluginManifest) {
Logger(`Unloading plugin: ${pluginManifest.name}`, LOG_LEVEL_NOTICE, "plugin-reload-" + pluginManifest.id);
this._log(`Unloading plugin: ${pluginManifest.name}`, LOG_LEVEL_NOTICE, "plugin-reload-" + pluginManifest.id);
// @ts-ignore
await this.app.plugins.unloadPlugin(pluginManifest.id);
// @ts-ignore
await this.app.plugins.loadPlugin(pluginManifest.id);
Logger(`Plugin reloaded: ${pluginManifest.name}`, LOG_LEVEL_NOTICE, "plugin-reload-" + pluginManifest.id);
this._log(`Plugin reloaded: ${pluginManifest.name}`, LOG_LEVEL_NOTICE, "plugin-reload-" + pluginManifest.id);
}
} else if (data.category == "CONFIG") {
this.plugin.$$askReload();
}
return true;
} catch (ex) {
Logger(`Applying ${data.displayName || data.name}.. Failed`);
Logger(ex, LOG_LEVEL_VERBOSE);
this._log(`Applying ${data.displayName || data.name}.. Failed`);
this._log(ex, LOG_LEVEL_VERBOSE);
return false;
}
}
@@ -1007,12 +1006,12 @@ export class ConfigSync extends LiveSyncCommands implements IObsidianModule {
await Promise.allSettled(p);
// await this.deleteConfigOnDatabase(data.documentPath);
// await this.updatePluginList(false, data.documentPath);
Logger(`Deleted: ${data.category}/${data.name} of ${data.category} (${delList.length} items)`, LOG_LEVEL_NOTICE);
this._log(`Deleted: ${data.category}/${data.name} of ${data.category} (${delList.length} items)`, LOG_LEVEL_NOTICE);
}
return true;
} catch (ex) {
Logger(`Failed to delete: ${data.documentPath}`, LOG_LEVEL_NOTICE);
Logger(ex, LOG_LEVEL_VERBOSE);
this._log(`Failed to delete: ${data.documentPath}`, LOG_LEVEL_NOTICE);
this._log(ex, LOG_LEVEL_VERBOSE);
return false;
}
@@ -1096,13 +1095,13 @@ export class ConfigSync extends LiveSyncCommands implements IObsidianModule {
displayName = `${json.name}`;
}
} catch (ex) {
Logger(`Configuration sync data: ${path} looks like manifest, but could not read the version`, LOG_LEVEL_INFO);
Logger(ex, LOG_LEVEL_VERBOSE);
this._log(`Configuration sync data: ${path} looks like manifest, but could not read the version`, LOG_LEVEL_INFO);
this._log(ex, LOG_LEVEL_VERBOSE);
}
}
} catch (ex) {
Logger(`The file ${path} could not be encoded`);
Logger(ex, LOG_LEVEL_VERBOSE);
this._log(`The file ${path} could not be encoded`);
this._log(ex, LOG_LEVEL_VERBOSE);
return false;
}
const mtime = stat.mtime;
@@ -1150,7 +1149,7 @@ export class ConfigSync extends LiveSyncCommands implements IObsidianModule {
};
} else {
if (isMarkedAsSameChanges(prefixedFileName, [old.mtime, mtime + 1]) == EVEN) {
Logger(`STORAGE --> DB:${prefixedFileName}: (config) Skipped (Already checked the same)`, LOG_LEVEL_DEBUG);
this._log(`STORAGE --> DB:${prefixedFileName}: (config) Skipped (Already checked the same)`, LOG_LEVEL_DEBUG);
return;
}
const docXDoc = await this.localDatabase.getDBEntryFromMeta(old, {}, false, false);
@@ -1162,7 +1161,7 @@ export class ConfigSync extends LiveSyncCommands implements IObsidianModule {
const oldContent = dataSrc.substring(dataStart + DUMMY_END.length);
const oldContentArray = base64ToArrayBuffer(oldContent);
if (await isDocContentSame(oldContentArray, content)) {
Logger(`STORAGE --> DB:${prefixedFileName}: (config) Skipped (the same content)`, LOG_LEVEL_VERBOSE);
this._log(`STORAGE --> DB:${prefixedFileName}: (config) Skipped (the same content)`, LOG_LEVEL_VERBOSE);
markChangesAreSame(prefixedFileName, old.mtime, mtime + 1);
return true;
}
@@ -1179,12 +1178,12 @@ export class ConfigSync extends LiveSyncCommands implements IObsidianModule {
};
}
const ret = await this.localDatabase.putDBEntry(saveData);
Logger(`STORAGE --> DB:${prefixedFileName}: (config) Done`);
this._log(`STORAGE --> DB:${prefixedFileName}: (config) Done`);
fireAndForget(() => this.updatePluginListV2(false, this.filenameWithUnifiedKey(path)));
return ret;
} catch (ex) {
Logger(`STORAGE --> DB:${prefixedFileName}: (config) Failed`);
Logger(ex, LOG_LEVEL_VERBOSE);
this._log(`STORAGE --> DB:${prefixedFileName}: (config) Failed`);
this._log(ex, LOG_LEVEL_VERBOSE);
return false;
}
})
@@ -1192,7 +1191,7 @@ export class ConfigSync extends LiveSyncCommands implements IObsidianModule {
async storeCustomizationFiles(path: FilePath, termOverRide?: string) {
const term = termOverRide || this.plugin.deviceAndVaultName;
if (term == "") {
Logger("We have to configure the device name", LOG_LEVEL_NOTICE);
this._log("We have to configure the device name", LOG_LEVEL_NOTICE);
return;
}
if (this.useV2) {
@@ -1234,7 +1233,7 @@ export class ConfigSync extends LiveSyncCommands implements IObsidianModule {
for (const target of fileTargets) {
const data = await this.makeEntryFromFile(target);
if (data == false) {
Logger(`Config: skipped (Possibly is not exist): ${target} `, LOG_LEVEL_VERBOSE);
this._log(`Config: skipped (Possibly is not exist): ${target} `, LOG_LEVEL_VERBOSE);
continue;
}
if (data.version) {
@@ -1249,9 +1248,9 @@ export class ConfigSync extends LiveSyncCommands implements IObsidianModule {
}
dt.mtime = mtime;
// Logger(`Configuration saving: ${prefixedFileName}`);
// this._log(`Configuration saving: ${prefixedFileName}`);
if (dt.files.length == 0) {
Logger(`Nothing left: deleting.. ${path}`);
this._log(`Nothing left: deleting.. ${path}`);
await this.deleteConfigOnDatabase(prefixedFileName);
await this.updatePluginList(false, prefixedFileName);
return
@@ -1277,7 +1276,7 @@ export class ConfigSync extends LiveSyncCommands implements IObsidianModule {
};
} else {
if (old.mtime == mtime) {
// Logger(`STORAGE --> DB:${prefixedFileName}: (config) Skipped (Same time)`, LOG_LEVEL_VERBOSE);
// this._log(`STORAGE --> DB:${prefixedFileName}: (config) Skipped (Same time)`, LOG_LEVEL_VERBOSE);
return true;
}
const oldC = await this.localDatabase.getDBEntryFromMeta(old, {}, false, false);
@@ -1289,7 +1288,7 @@ export class ConfigSync extends LiveSyncCommands implements IObsidianModule {
}))
const isSame = (await Promise.all(diffs)).every(e => e == true);
if (isSame) {
Logger(`STORAGE --> DB:${prefixedFileName}: (config) Skipped (Same content)`, LOG_LEVEL_VERBOSE);
this._log(`STORAGE --> DB:${prefixedFileName}: (config) Skipped (Same content)`, LOG_LEVEL_VERBOSE);
return true;
}
}
@@ -1308,25 +1307,24 @@ export class ConfigSync extends LiveSyncCommands implements IObsidianModule {
}
const ret = await this.localDatabase.putDBEntry(saveData);
await this.updatePluginList(false, saveData.path);
Logger(`STORAGE --> DB:${prefixedFileName}: (config) Done`);
this._log(`STORAGE --> DB:${prefixedFileName}: (config) Done`);
return ret;
} catch (ex) {
Logger(`STORAGE --> DB:${prefixedFileName}: (config) Failed`);
Logger(ex, LOG_LEVEL_VERBOSE);
this._log(`STORAGE --> DB:${prefixedFileName}: (config) Failed`);
this._log(ex, LOG_LEVEL_VERBOSE);
return false;
}
})
}
async $anyProcessOptionalFileEvent(path: FilePath): Promise<boolean | undefined> {
return await this.watchVaultRawEventsAsync(path);
}
async watchVaultRawEventsAsync(path: FilePath) {
if (!this._isMainReady) return true;
if (!this._isMainSuspended()) return true;
if (!this._isThisModuleEnabled()) return true;
if (!this.isTargetPath(path)) return false;
if (!this._isMainReady) return false;
if (this._isMainSuspended()) return false;
if (!this._isThisModuleEnabled()) return false;
// if (!this.isTargetPath(path)) return false;
const stat = await this.plugin.storageAccess.statHidden(path);
// Make sure that target is a file.
if (stat && stat.type != "file")
@@ -1337,10 +1335,11 @@ export class ConfigSync extends LiveSyncCommands implements IObsidianModule {
e.mode != MODE_SELECTIVE && e.mode != MODE_SHINY
).map(e => e.files).flat().map(e => `${configDir}/${e}`.toLowerCase());
if (synchronisedInConfigSync.some(e => e.startsWith(path.toLowerCase()))) {
Logger(`Customization file skipped: ${path}`, LOG_LEVEL_VERBOSE);
this._log(`Customization file skipped: ${path}`, LOG_LEVEL_VERBOSE);
// This file could be handled by the other module.
return false;
}
// this._log(`Customization file detected: ${path}`, LOG_LEVEL_VERBOSE);
const storageMTime = ~~((stat && stat.mtime || 0) / 1000);
const key = `${path}-${storageMTime}`;
if (this.recentProcessedInternalFiles.contains(key)) {
@@ -1359,16 +1358,13 @@ export class ConfigSync extends LiveSyncCommands implements IObsidianModule {
return true;
}
async scanAllConfigFiles(showMessage: boolean) {
await shareRunningResult("scanAllConfigFiles", async () => {
const logLevel = showMessage ? LOG_LEVEL_NOTICE : LOG_LEVEL_INFO;
Logger("Scanning customizing files.", logLevel, "scan-all-config");
this._log("Scanning customizing files.", logLevel, "scan-all-config");
const term = this.plugin.deviceAndVaultName;
if (term == "") {
Logger("We have to configure the device name", LOG_LEVEL_NOTICE);
this._log("We have to configure the device name", LOG_LEVEL_NOTICE);
return;
}
const filesAll = await this.scanInternalFiles();
@@ -1396,8 +1392,8 @@ export class ConfigSync extends LiveSyncCommands implements IObsidianModule {
await this.deleteConfigOnDatabase(unifiedFilenameWithKey);
}
} catch (ex) {
Logger(`scanAllConfigFiles - Error: ${item._id}`, LOG_LEVEL_VERBOSE);
Logger(ex, LOG_LEVEL_VERBOSE);
this._log(`scanAllConfigFiles - Error: ${item._id}`, LOG_LEVEL_VERBOSE);
this._log(ex, LOG_LEVEL_VERBOSE);
} finally {
releaser();
}
@@ -1412,8 +1408,8 @@ export class ConfigSync extends LiveSyncCommands implements IObsidianModule {
try {
await this.storeCustomisationFileV2(filePath, term);
} catch (ex) {
Logger(`scanAllConfigFiles - Error: ${filePath}`, LOG_LEVEL_VERBOSE);
Logger(ex, LOG_LEVEL_VERBOSE);
this._log(`scanAllConfigFiles - Error: ${filePath}`, LOG_LEVEL_VERBOSE);
this._log(ex, LOG_LEVEL_VERBOSE);
}
finally {
releaser();
@@ -1430,7 +1426,7 @@ export class ConfigSync extends LiveSyncCommands implements IObsidianModule {
for (const vp of virtualPathsOfLocalFiles) {
const p = files.find(e => e.key == vp)?.file;
if (!p) {
Logger(`scanAllConfigFiles - File not found: ${vp}`, LOG_LEVEL_VERBOSE);
this._log(`scanAllConfigFiles - File not found: ${vp}`, LOG_LEVEL_VERBOSE);
continue;
}
await this.storeCustomizationFiles(p);
@@ -1453,11 +1449,11 @@ export class ConfigSync extends LiveSyncCommands implements IObsidianModule {
const old = await this.localDatabase.getDBEntryMeta(prefixedFileName, undefined, false) as InternalFileEntry | false;
let saveData: InternalFileEntry;
if (old === false) {
Logger(`STORAGE -x> DB:${prefixedFileName}: (config) already deleted (Not found on database)`);
this._log(`STORAGE -x> DB:${prefixedFileName}: (config) already deleted (Not found on database)`);
return true;
} else {
if (old.deleted) {
Logger(`STORAGE -x> DB:${prefixedFileName}: (config) already deleted`);
this._log(`STORAGE -x> DB:${prefixedFileName}: (config) already deleted`);
return true;
}
saveData =
@@ -1472,11 +1468,11 @@ export class ConfigSync extends LiveSyncCommands implements IObsidianModule {
}
await this.localDatabase.putRaw(saveData);
await this.updatePluginList(false, prefixedFileName);
Logger(`STORAGE -x> DB:${prefixedFileName}: (config) Done`);
this._log(`STORAGE -x> DB:${prefixedFileName}: (config) Done`);
return true;
} catch (ex) {
Logger(`STORAGE -x> DB:${prefixedFileName}: (config) Failed`);
Logger(ex, LOG_LEVEL_VERBOSE);
this._log(`STORAGE -x> DB:${prefixedFileName}: (config) Failed`);
this._log(ex, LOG_LEVEL_VERBOSE);
return false;
}
});
@@ -1530,7 +1526,7 @@ export class ConfigSync extends LiveSyncCommands implements IObsidianModule {
$allSuspendExtraSync(): Promise<boolean> {
if (this.plugin.settings.usePluginSync || this.plugin.settings.autoSweepPlugins) {
Logger("Customisation sync have been temporarily disabled. Please enable them after the fetching, if you need them.", LOG_LEVEL_NOTICE)
this._log("Customisation sync have been temporarily disabled. Please enable them after the fetching, if you need them.", LOG_LEVEL_NOTICE)
this.plugin.settings.usePluginSync = false;
this.plugin.settings.autoSweepPlugins = false;
}
@@ -1590,8 +1586,8 @@ export class ConfigSync extends LiveSyncCommands implements IObsidianModule {
try {
w = await this.app.vault.adapter.list(path);
} catch (ex) {
Logger(`Could not traverse(ConfigSync):${path}`, LOG_LEVEL_INFO);
Logger(ex, LOG_LEVEL_VERBOSE);
this._log(`Could not traverse(ConfigSync):${path}`, LOG_LEVEL_INFO);
this._log(ex, LOG_LEVEL_VERBOSE);
return [];
}
let files = [

5
src/features/HiddenFileCommon/JsonResolveModal.ts
View File

@@ -32,9 +32,12 @@ export class JsonResolveModal extends Modal {
this.hideLocal = hideLocal ?? false;
void waitForSignal(`cancel-internal-conflict:${filename}`).then(() => this.close());
}
async UICallback(keepRev?: string, mergedStr?: string) {
if (this.callback) {
await this.callback(keepRev, mergedStr);
}
this.close();
await this.callback?.(keepRev, mergedStr);
this.callback = undefined;
}

390
src/features/HiddenFileSync/CmdHiddenFileSync.ts
View File

@@ -1,9 +1,8 @@
import { normalizePath, type PluginManifest, type ListedFiles } from "../../deps.ts";
import { type EntryDoc, type LoadedEntry, type InternalFileEntry, type FilePathWithPrefix, type FilePath, LOG_LEVEL_INFO, LOG_LEVEL_NOTICE, LOG_LEVEL_VERBOSE, MODE_SELECTIVE, MODE_PAUSED, type SavingEntry, type DocumentID } from "../../lib/src/common/types.ts";
import { type EntryDoc, type LoadedEntry, type InternalFileEntry, type FilePathWithPrefix, type FilePath, LOG_LEVEL_INFO, LOG_LEVEL_NOTICE, LOG_LEVEL_VERBOSE, MODE_SELECTIVE, MODE_PAUSED, type SavingEntry, type DocumentID, type UXStat, MODE_AUTOMATIC } from "../../lib/src/common/types.ts";
import { type InternalFileInfo, ICHeader, ICHeaderEnd } from "../../common/types.ts";
import { readAsBlob, isDocContentSame, sendSignal, readContent, createBlob, fireAndForget } from "../../lib/src/common/utils.ts";
import { Logger } from "../../lib/src/common/logger.ts";
import { getPath, isInternalMetadata, PeriodicProcessor } from "../../common/utils.ts";
import { BASE_IS_NEW, compareMTime, EVEN, getPath, isInternalMetadata, isMarkedAsSameChanges, markChangesAreSame, PeriodicProcessor, TARGET_IS_NEW } from "../../common/utils.ts";
import { serialized } from "../../lib/src/concurrency/lock.ts";
import { JsonResolveModal } from "../HiddenFileCommon/JsonResolveModal.ts";
import { LiveSyncCommands } from "../LiveSyncCommands.ts";
@@ -41,12 +40,12 @@ export class HiddenFileSync extends LiveSyncCommands implements IObsidianModule
async $everyOnDatabaseInitialized(showNotice: boolean) {
if (this._isThisModuleEnabled()) {
try {
Logger("Synchronizing hidden files...");
this._log("Synchronizing hidden files...");
await this.syncInternalFilesAndDatabase("push", showNotice);
Logger("Synchronizing hidden files done");
this._log("Synchronizing hidden files done");
} catch (ex) {
Logger("Synchronizing hidden files failed");
Logger(ex, LOG_LEVEL_VERBOSE);
this._log("Synchronizing hidden files failed");
this._log(ex, LOG_LEVEL_VERBOSE);
}
}
return true;
@@ -58,6 +57,13 @@ export class HiddenFileSync extends LiveSyncCommands implements IObsidianModule
return true;
}
$everyOnloadAfterLoadSettings(): Promise<boolean> {
const ignorePatterns = this.settings.syncInternalFilesIgnorePatterns
.replace(/\n| /g, "")
.split(",").filter(e => e).map(e => new RegExp(e, "i"));
this.ignorePatterns = ignorePatterns;
return Promise.resolve(true);
}
async $everyOnResumeProcess(): Promise<boolean> {
this.periodicInternalFileScanProcessor?.disable();
@@ -77,6 +83,10 @@ export class HiddenFileSync extends LiveSyncCommands implements IObsidianModule
if (!this.plugin.isReady)
return Promise.resolve(true);
this.periodicInternalFileScanProcessor.enable(this._isThisModuleEnabled() && this.settings.syncInternalFilesInterval ? (this.settings.syncInternalFilesInterval * 1000) : 0);
const ignorePatterns = this.settings.syncInternalFilesIgnorePatterns
.replace(/\n| /g, "")
.split(",").filter(e => e).map(e => new RegExp(e, "i"));
this.ignorePatterns = ignorePatterns;
return Promise.resolve(true);
}
@@ -85,56 +95,56 @@ export class HiddenFileSync extends LiveSyncCommands implements IObsidianModule
}
internalFileProcessor = new QueueProcessor<string, any>(
async (filenames) => {
Logger(`START :Applying hidden ${filenames.length} files change`, LOG_LEVEL_VERBOSE);
this._log(`START :Applying hidden ${filenames.length} files change`, LOG_LEVEL_VERBOSE);
await this.syncInternalFilesAndDatabase("pull", false, false, filenames);
Logger(`DONE :Applying hidden ${filenames.length} files change`, LOG_LEVEL_VERBOSE);
this._log(`DONE :Applying hidden ${filenames.length} files change`, LOG_LEVEL_VERBOSE);
return;
}, { batchSize: 100, concurrentLimit: 1, delay: 10, yieldThreshold: 100, suspended: false, totalRemainingReactiveSource: hiddenFilesEventCount }
);
recentProcessedInternalFiles = [] as string[];
async $anyProcessOptionalFileEvent(path: FilePath): Promise<boolean | undefined> {
return await this.watchVaultRawEventsAsync(path);
}
async watchVaultRawEventsAsync(path: FilePath): Promise<boolean | undefined> {
if (!this._isMainReady) return false;
if (this._isMainSuspended()) return false;
if (!this._isThisModuleEnabled()) return false;
if (!isInternalMetadata(path)) return false;
// Exclude files handled by customization sync
const configDir = normalizePath(this.app.vault.configDir);
const synchronisedInConfigSync = !this.settings.usePluginSync ? [] : Object.values(this.settings.pluginSyncExtendedSetting).filter(e => e.mode == MODE_SELECTIVE || e.mode == MODE_PAUSED).map(e => e.files).flat().map(e => `${configDir}/${e}`.toLowerCase());
const synchronisedInConfigSync = !this.settings.usePluginSync ? [] :
Object.values(this.settings.pluginSyncExtendedSetting).
filter(e => e.mode == MODE_SELECTIVE || e.mode == MODE_PAUSED).
map(e => e.files).flat().map(e => `${configDir}/${e}`.toLowerCase());
if (synchronisedInConfigSync.some(e => e.startsWith(path.toLowerCase()))) {
Logger(`Hidden file skipped: ${path} is synchronized in customization sync.`, LOG_LEVEL_VERBOSE);
this._log(`Hidden file skipped: ${path} is synchronized in customization sync.`, LOG_LEVEL_VERBOSE);
return false;
}
const stat = await this.plugin.storageAccess.statHidden(path);
// sometimes folder is coming.
if (stat != null && stat.type != "file") {
return false;
}
const mtime = stat == null ? 0 : stat?.mtime ?? 0;
const storageMTime = ~~((mtime) / 1000);
const key = `${path}-${storageMTime}`;
if (mtime != 0 && this.recentProcessedInternalFiles.contains(key)) {
//If recently processed, it may caused by self.
// Return true to prevent further processing.
if (this.isKnownChange(path, stat?.mtime ?? 0)) {
// This could be caused by self. so return true to prevent further processing.
return true;
}
this.recentProcessedInternalFiles = [key, ...this.recentProcessedInternalFiles].slice(0, 100);
// const id = await this.path2id(path, ICHeader);
const mtime = stat == null ? 0 : stat?.mtime ?? 0;
const storageMTime = ~~((mtime) / 1000);
const prefixedFileName = addPrefix(path, ICHeader);
const filesOnDB = await this.localDatabase.getDBEntryMeta(prefixedFileName);
const dbMTime = ~~((filesOnDB && filesOnDB.mtime || 0) / 1000);
// Skip unchanged file.
if (dbMTime == storageMTime) {
// Logger(`STORAGE --> DB:${path}: (hidden) Nothing changed`);
// this._log(`STORAGE --> DB:${path}: (hidden) Nothing changed`);
// Handled, but nothing changed. also return true to prevent further processing.
return true;
}
// Do not compare timestamp. Always local data should be preferred except this plugin wrote one.
try {
if (storageMTime == 0) {
await this.deleteInternalFileOnDatabase(path);
@@ -144,8 +154,8 @@ export class HiddenFileSync extends LiveSyncCommands implements IObsidianModule
// Surely processed.
return true;
} catch (ex) {
Logger(`Failed to process hidden file:${path}`);
Logger(ex, LOG_LEVEL_VERBOSE);
this._log(`Failed to process hidden file:${path}`);
this._log(ex, LOG_LEVEL_VERBOSE);
}
// Could not be processed. but it was own task. so return true to prevent further processing.
return true;
@@ -164,8 +174,8 @@ export class HiddenFileSync extends LiveSyncCommands implements IObsidianModule
}
}
} catch (ex) {
Logger("something went wrong on resolving all conflicted internal files");
Logger(ex, LOG_LEVEL_VERBOSE);
this._log("something went wrong on resolving all conflicted internal files");
this._log(ex, LOG_LEVEL_VERBOSE);
}
await this.conflictResolutionProcessor.startPipeline().waitForAllProcessed();
}
@@ -176,12 +186,12 @@ export class HiddenFileSync extends LiveSyncCommands implements IObsidianModule
// simply check modified time
const mtimeCurrent = ("mtime" in currentDoc && currentDoc.mtime) || 0;
const mtimeConflicted = ("mtime" in conflictedDoc && conflictedDoc.mtime) || 0;
// Logger(`Revisions:${new Date(mtimeA).toLocaleString} and ${new Date(mtimeB).toLocaleString}`);
// this._log(`Revisions:${new Date(mtimeA).toLocaleString} and ${new Date(mtimeB).toLocaleString}`);
// console.log(`mtime:${mtimeA} - ${mtimeB}`);
const delRev = mtimeCurrent < mtimeConflicted ? currentRev : conflictedRev;
// delete older one.
await this.localDatabase.removeRevision(id, delRev);
Logger(`Older one has been deleted:${path}`);
this._log(`Older one has been deleted:${path}`);
const cc = await this.localDatabase.getRaw(id, { conflicts: true });
if (cc._conflicts?.length === 0) {
await this.extractInternalFileFromDatabase(stripAllPrefixes(path))
@@ -204,7 +214,7 @@ export class HiddenFileSync extends LiveSyncCommands implements IObsidianModule
if (doc._conflicts === undefined) return [];
if (doc._conflicts.length == 0)
return [];
Logger(`Hidden file conflicted:${path}`);
this._log(`Hidden file conflicted:${path}`);
const conflicts = doc._conflicts.sort((a, b) => Number(a.split("-")[0]) - Number(b.split("-")[0]));
const revA = doc._rev;
const revB = conflicts[0];
@@ -217,7 +227,7 @@ export class HiddenFileSync extends LiveSyncCommands implements IObsidianModule
const commonBase = revFrom._revs_info?.filter(e => e.status == "available" && Number(e.rev.split("-")[0]) < conflictedRevNo).first()?.rev ?? "";
const result = await this.plugin.localDatabase.mergeObject(path, commonBase, doc._rev, conflictedRev);
if (result) {
Logger(`Object merge:${path}`, LOG_LEVEL_INFO);
this._log(`Object merge:${path}`, LOG_LEVEL_INFO);
const filename = stripAllPrefixes(path);
const isExists = await this.plugin.storageAccess.isExistsIncludeHidden(filename);
if (!isExists) {
@@ -234,7 +244,7 @@ export class HiddenFileSync extends LiveSyncCommands implements IObsidianModule
this.conflictResolutionProcessor.enqueue(path);
return [];
} else {
Logger(`Object merge is not applicable.`, LOG_LEVEL_VERBOSE);
this._log(`Object merge is not applicable.`, LOG_LEVEL_VERBOSE);
}
return [{ path, revA, revB, id, doc }];
}
@@ -242,8 +252,8 @@ export class HiddenFileSync extends LiveSyncCommands implements IObsidianModule
await this.resolveByNewerEntry(id, path, doc, revA, revB);
return [];
} catch (ex) {
Logger(`Failed to resolve conflict (Hidden): ${path}`);
Logger(ex, LOG_LEVEL_VERBOSE);
this._log(`Failed to resolve conflict (Hidden): ${path}`);
this._log(ex, LOG_LEVEL_VERBOSE);
return [];
}
}, {
@@ -281,7 +291,7 @@ export class HiddenFileSync extends LiveSyncCommands implements IObsidianModule
this.procInternalFile(filename);
return true;
} else {
Logger(`Skipped (Not target:${filename})`, LOG_LEVEL_VERBOSE);
this._log(`Skipped (Not target:${filename})`, LOG_LEVEL_VERBOSE);
return false;
}
}
@@ -291,30 +301,46 @@ export class HiddenFileSync extends LiveSyncCommands implements IObsidianModule
this.conflictResolutionProcessor.enqueue(path);
}
knownChanges: { [key: string]: number; } = {};
markAsKnownChange(path: string, mtime: number) {
this.knownChanges[path] = mtime;
}
isKnownChange(path: string, mtime: number) {
return this.knownChanges[path] == mtime;
}
ignorePatterns: RegExp[] = [];
//TODO: Tidy up. Even though it is experimental feature, So dirty...
async syncInternalFilesAndDatabase(direction: "push" | "pull" | "safe" | "pullForce" | "pushForce", showMessage: boolean, filesAll: InternalFileInfo[] | false = false, targetFiles: string[] | false = false) {
async syncInternalFilesAndDatabase(direction: "push" | "pull" | "safe" | "pullForce" | "pushForce", showMessage: boolean, filesAll: InternalFileInfo[] | false = false, targetFilesSrc: string[] | false = false) {
const targetFiles = targetFilesSrc ? targetFilesSrc.map(e => stripAllPrefixes(e as FilePathWithPrefix)) : false;
// debugger;
await this.resolveConflictOnInternalFiles();
const logLevel = showMessage ? LOG_LEVEL_NOTICE : LOG_LEVEL_INFO;
Logger("Scanning hidden files.", logLevel, "sync_internal");
const ignorePatterns = this.settings.syncInternalFilesIgnorePatterns
.replace(/\n| /g, "")
.split(",").filter(e => e).map(e => new RegExp(e, "i"));
this._log("Scanning hidden files.", logLevel, "sync_internal");
const configDir = normalizePath(this.app.vault.configDir);
let files: InternalFileInfo[] =
filesAll ? filesAll : (await this.scanInternalFiles())
const allowedInHiddenFileSync = this.settings.usePluginSync ? Object.values(this.settings.pluginSyncExtendedSetting).filter(e => e.mode == MODE_AUTOMATIC).map(e => e.files).flat().map(e => `${configDir}/${e}`.toLowerCase()) : undefined;
if (allowedInHiddenFileSync) {
const systemOrNot = files.reduce((acc, cur) => {
if (cur.path.startsWith(configDir)) {
acc.system.push(cur);
} else {
acc.user.push(cur);
}
return acc;
}, { system: [] as InternalFileInfo[], user: [] as InternalFileInfo[] });
const synchronisedInConfigSync = !this.settings.usePluginSync ? [] : Object.values(this.settings.pluginSyncExtendedSetting).filter(e => e.mode == MODE_SELECTIVE || e.mode == MODE_PAUSED).map(e => e.files).flat().map(e => `${configDir}/${e}`.toLowerCase());
files = files.filter(file => synchronisedInConfigSync.every(filterFile => !file.path.toLowerCase().startsWith(filterFile)))
files =
[...systemOrNot.user,
...systemOrNot.system.filter(file => allowedInHiddenFileSync.some(filterFile => file.path.toLowerCase().startsWith(filterFile)))];
}
const filesOnDB = ((await this.localDatabase.allDocsRaw({ startkey: ICHeader, endkey: ICHeaderEnd, include_docs: true })).rows.map(e => e.doc) as InternalFileEntry[]).filter(e => !e.deleted);
const allFileNamesSrc = [...new Set([...files.map(e => normalizePath(e.path)), ...filesOnDB.map(e => stripAllPrefixes(this.getPath(e)))])];
const allFileNames = allFileNamesSrc.filter(filename => !targetFiles || (targetFiles && targetFiles.indexOf(filename) !== -1)).filter(path => synchronisedInConfigSync.every(filterFile => !path.toLowerCase().startsWith(filterFile)))
function compareMTime(a: number, b: number) {
const wa = ~~(a / 1000);
const wb = ~~(b / 1000);
const diff = wa - wb;
return diff;
let allFileNames = allFileNamesSrc.filter(filename => !targetFiles || (targetFiles && targetFiles.indexOf(filename) !== -1));
if (allowedInHiddenFileSync) {
allFileNames = allFileNames.filter(file => allowedInHiddenFileSync.some(filterFile => file.toLowerCase().startsWith(filterFile)));
}
const fileCount = allFileNames.length;
@@ -342,9 +368,7 @@ export class HiddenFileSync extends LiveSyncCommands implements IObsidianModule
c = pieces.shift();
}
};
// Cache update time information for files which have already been processed (mainly for files that were skipped due to the same content)
let caches: { [key: string]: { storageMtime: number; docMtime: number; }; } = {};
caches = await this.kvDB.get<{ [key: string]: { storageMtime: number; docMtime: number; }; }>("diff-caches-internal") || {};
const filesMap = files.reduce((acc, cur) => {
acc[cur.path] = cur;
return acc;
@@ -357,10 +381,10 @@ export class HiddenFileSync extends LiveSyncCommands implements IObsidianModule
const filename = filenames[0];
processed++;
if (processed % 100 == 0) {
Logger(`Hidden file: ${processed}/${fileCount}`, logLevel, "sync_internal");
this._log(`Hidden file: ${processed}/${fileCount}`, logLevel, "sync_internal");
}
if (!filename) return [];
if (ignorePatterns.some(e => filename.match(e)))
if (this.ignorePatterns.some(e => filename.match(e)))
return [];
if (await this.plugin.$$isIgnoredByIgnoreFiles(filename)) {
return [];
@@ -383,27 +407,27 @@ export class HiddenFileSync extends LiveSyncCommands implements IObsidianModule
fileOnStorage: xFileOnStorage,
fileOnDatabase: xFileOnDatabase
} = params[0];
if (xFileOnStorage && xFileOnDatabase) {
const cache = filename in caches ? caches[filename] : { storageMtime: 0, docMtime: 0 };
const xFileOnDatabaseExists = xFileOnDatabase !== undefined && !(xFileOnDatabase.deleted || xFileOnDatabase._deleted);
if (xFileOnStorage && xFileOnDatabaseExists) {
// Both => Synchronize
if ((direction != "pullForce" && direction != "pushForce") && xFileOnDatabase.mtime == cache.docMtime && xFileOnStorage.mtime == cache.storageMtime) {
if ((direction != "pullForce" && direction != "pushForce") && isMarkedAsSameChanges(filename, [xFileOnDatabase.mtime, xFileOnStorage.mtime]) == EVEN) {
this._log(`Hidden file skipped: ${filename} is marked as same`, LOG_LEVEL_VERBOSE);
return;
}
const nw = compareMTime(xFileOnStorage.mtime, xFileOnDatabase.mtime);
if (nw > 0 || direction == "pushForce") {
await this.storeInternalFileToDatabase(xFileOnStorage);
}
if (nw < 0 || direction == "pullForce") {
if (nw == BASE_IS_NEW || direction == "pushForce") {
if (await this.storeInternalFileToDatabase(xFileOnStorage) !== false) {
// countUpdatedFolder(filename);
}
} else if (nw == TARGET_IS_NEW || direction == "pullForce") {
// skip if not extraction performed.
if (!await this.extractInternalFileFromDatabase(filename))
return;
if (await this.extractInternalFileFromDatabase(filename))
countUpdatedFolder(filename);
} else {
// Even, or not forced. skip.
}
// If process successfully updated or file contents are same, update cache.
cache.docMtime = xFileOnDatabase.mtime;
cache.storageMtime = xFileOnStorage.mtime;
caches[filename] = cache;
countUpdatedFolder(filename);
} else if (!xFileOnStorage && xFileOnDatabase) {
} else if (!xFileOnStorage && xFileOnDatabaseExists) {
if (direction == "push" || direction == "pushForce") {
if (xFileOnDatabase.deleted)
return;
@@ -419,11 +443,14 @@ export class HiddenFileSync extends LiveSyncCommands implements IObsidianModule
countUpdatedFolder(filename);
}
}
} else if (xFileOnStorage && !xFileOnDatabase) {
} else if (xFileOnStorage && !xFileOnDatabaseExists) {
if (direction == "push" || direction == "pushForce" || direction == "safe") {
await this.storeInternalFileToDatabase(xFileOnStorage);
} else {
await this.extractInternalFileFromDatabase(xFileOnStorage.path);
// Apply the deletion
if (await this.extractInternalFileFromDatabase(xFileOnStorage.path)) {
countUpdatedFolder(xFileOnStorage.path);
}
}
} else {
throw new Error("Invalid state on hidden file sync");
@@ -435,8 +462,6 @@ export class HiddenFileSync extends LiveSyncCommands implements IObsidianModule
.enqueueAll(allFileNames)
.startPipeline().waitForAllDoneAndTerminate();
await this.kvDB.set("diff-caches-internal", caches);
// When files has been retrieved from the database. they must be reloaded.
if ((direction == "pull" || direction == "pullForce") && filesChanged != 0) {
// Show notification to restart obsidian when something has been changed in configDir.
@@ -459,12 +484,12 @@ export class HiddenFileSync extends LiveSyncCommands implements IObsidianModule
anchor.text = "HERE";
anchor.addEventListener("click", () => {
fireAndForget(async () => {
Logger(`Unloading plugin: ${updatePluginName}`, LOG_LEVEL_NOTICE, "plugin-reload-" + updatePluginId);
this._log(`Unloading plugin: ${updatePluginName}`, LOG_LEVEL_NOTICE, "plugin-reload-" + updatePluginId);
// @ts-ignore
await this.app.plugins.unloadPlugin(updatePluginId);
// @ts-ignore
await this.app.plugins.loadPlugin(updatePluginId);
Logger(`Plugin reloaded: ${updatePluginName}`, LOG_LEVEL_NOTICE, "plugin-reload-" + updatePluginId);
this._log(`Plugin reloaded: ${updatePluginName}`, LOG_LEVEL_NOTICE, "plugin-reload-" + updatePluginId);
});
});
}
@@ -472,8 +497,8 @@ export class HiddenFileSync extends LiveSyncCommands implements IObsidianModule
}
}
} catch (ex) {
Logger("Error on checking plugin status.");
Logger(ex, LOG_LEVEL_VERBOSE);
this._log("Error on checking plugin status.");
this._log(ex, LOG_LEVEL_VERBOSE);
}
@@ -491,17 +516,19 @@ export class HiddenFileSync extends LiveSyncCommands implements IObsidianModule
}
}
Logger(`Hidden files scanned: ${filesChanged} files had been modified`, logLevel, "sync_internal");
this._log(`Hidden files scanned: ${filesChanged} files had been modified`, logLevel, "sync_internal");
}
async storeInternalFileToDatabase(file: InternalFileInfo, forceWrite = false) {
if (await this.plugin.$$isIgnoredByIgnoreFiles(file.path)) {
return
const storeFilePath = file.path;
const storageFilePath = file.path;
if (await this.plugin.$$isIgnoredByIgnoreFiles(storageFilePath)) {
return undefined;
}
const id = await this.path2id(file.path, ICHeader);
const prefixedFileName = addPrefix(file.path, ICHeader);
const content = createBlob(await this.plugin.storageAccess.readHiddenFileAuto(file.path));
const id = await this.path2id(storeFilePath, ICHeader);
const prefixedFileName = addPrefix(storeFilePath, ICHeader);
const content = createBlob(await this.plugin.storageAccess.readHiddenFileAuto(storageFilePath));
const mtime = file.mtime;
return await serialized("file-" + prefixedFileName, async () => {
try {
@@ -523,8 +550,12 @@ export class HiddenFileSync extends LiveSyncCommands implements IObsidianModule
};
} else {
if (await isDocContentSame(readAsBlob(old), content) && !forceWrite) {
// Logger(`STORAGE --> DB:${file.path}: (hidden) Not changed`, LOG_LEVEL_VERBOSE);
return;
// this._log(`STORAGE --> DB:${file.path}: (hidden) Not changed`, LOG_LEVEL_VERBOSE);
const stat = await this.plugin.storageAccess.statHidden(storageFilePath);
if (stat) {
markChangesAreSame(storageFilePath, old.mtime, stat.mtime);
}
return undefined;
}
saveData =
{
@@ -539,24 +570,32 @@ export class HiddenFileSync extends LiveSyncCommands implements IObsidianModule
};
}
const ret = await this.localDatabase.putDBEntry(saveData);
Logger(`STORAGE --> DB:${file.path}: (hidden) Done`);
return ret;
if (ret !== false) {
this._log(`STORAGE --> DB:${storageFilePath}: (hidden) Done`);
return true;
} else {
this._log(`STORAGE --> DB:${storageFilePath}: (hidden) Failed`);
return false;
}
} catch (ex) {
Logger(`STORAGE --> DB:${file.path}: (hidden) Failed`);
Logger(ex, LOG_LEVEL_VERBOSE);
this._log(`STORAGE --> DB:${storageFilePath}: (hidden) Failed`);
this._log(ex, LOG_LEVEL_VERBOSE);
return false;
}
});
}
async deleteInternalFileOnDatabase(filename: FilePath, forceWrite = false) {
const id = await this.path2id(filename, ICHeader);
const prefixedFileName = addPrefix(filename, ICHeader);
async deleteInternalFileOnDatabase(filenameSrc: FilePath, forceWrite = false) {
const storeFilePath = filenameSrc;
const storageFilePath = filenameSrc;
const displayFileName = filenameSrc;
const id = await this.path2id(storeFilePath, ICHeader);
const prefixedFileName = addPrefix(storeFilePath, ICHeader);
const mtime = new Date().getTime();
if (await this.plugin.$$isIgnoredByIgnoreFiles(filename)) {
return
if (await this.plugin.$$isIgnoredByIgnoreFiles(storageFilePath)) {
return undefined
}
await serialized("file-" + prefixedFileName, async () => {
return await serialized("file-" + prefixedFileName, async () => {
try {
const old = await this.localDatabase.getDBEntryMeta(prefixedFileName, undefined, true) as InternalFileEntry | false;
let saveData: InternalFileEntry;
@@ -578,12 +617,12 @@ export class HiddenFileSync extends LiveSyncCommands implements IObsidianModule
if (conflicts._conflicts !== undefined) {
for (const conflictRev of conflicts._conflicts) {
await this.localDatabase.removeRevision(old._id, conflictRev);
Logger(`STORAGE -x> DB:${filename}: (hidden) conflict removed ${old._rev} => ${conflictRev}`, LOG_LEVEL_VERBOSE);
this._log(`STORAGE -x> DB: ${displayFileName}: (hidden) conflict removed ${old._rev} => ${conflictRev}`, LOG_LEVEL_VERBOSE);
}
}
if (old.deleted) {
Logger(`STORAGE -x> DB:${filename}: (hidden) already deleted`);
return;
this._log(`STORAGE -x> DB: ${displayFileName}: (hidden) already deleted`);
return undefined;
}
saveData =
{
@@ -595,85 +634,101 @@ export class HiddenFileSync extends LiveSyncCommands implements IObsidianModule
type: "newnote",
};
}
await this.localDatabase.putRaw(saveData);
Logger(`STORAGE -x> DB:${filename}: (hidden) Done`);
const ret = await this.localDatabase.putRaw(saveData);
if (ret && ret.ok) {
this._log(`STORAGE -x> DB: ${displayFileName}: (hidden) Done`);
return true;
} else {
this._log(`STORAGE -x> DB: ${displayFileName}: (hidden) Failed`);
return false;
}
} catch (ex) {
Logger(`STORAGE -x> DB:${filename}: (hidden) Failed`);
Logger(ex, LOG_LEVEL_VERBOSE);
this._log(`STORAGE -x> DB: ${displayFileName}: (hidden) Failed`);
this._log(ex, LOG_LEVEL_VERBOSE);
return false;
}
});
}
async extractInternalFileFromDatabase(filename: FilePath, force = false) {
const isExists = await this.plugin.storageAccess.isExistsIncludeHidden(filename);
const prefixedFileName = addPrefix(filename, ICHeader);
if (await this.plugin.$$isIgnoredByIgnoreFiles(filename)) {
return;
async extractInternalFileFromDatabase(filenameSrc: FilePath, force = false) {
const storeFilePath = filenameSrc;
const storageFilePath = filenameSrc;
const isExists = await this.plugin.storageAccess.isExistsIncludeHidden(storageFilePath);
const prefixedFileName = addPrefix(storeFilePath, ICHeader);
const displayFileName = `${storeFilePath}`;
if (await this.plugin.$$isIgnoredByIgnoreFiles(storageFilePath)) {
return undefined;
}
return await serialized("file-" + prefixedFileName, async () => {
try {
// Check conflicted status
const fileOnDB = await this.localDatabase.getDBEntry(prefixedFileName, { conflicts: true }, false, true, true);
if (fileOnDB === false)
throw new Error(`File not found on database.:${filename}`);
throw new Error(`File not found on database.:${displayFileName}`);
// Prevent overwrite for Prevent overwriting while some conflicted revision exists.
if (fileOnDB?._conflicts?.length) {
Logger(`Hidden file ${filename} has conflicted revisions, to keep in safe, writing to storage has been prevented`, LOG_LEVEL_INFO);
return;
this._log(`Hidden file ${displayFileName} has conflicted revisions, to keep in safe, writing to storage has been prevented`, LOG_LEVEL_INFO);
return false;
}
const deleted = fileOnDB.deleted || fileOnDB._deleted || false;
if (deleted) {
if (!isExists) {
Logger(`STORAGE <x- DB:${filename}: deleted (hidden) Deleted on DB, but the file is already not found on storage.`);
this._log(`STORAGE <x- DB: ${displayFileName}: deleted (hidden) Deleted on DB, but the file is already not found on storage.`);
} else {
Logger(`STORAGE <x- DB:${filename}: deleted (hidden).`);
await this.plugin.storageAccess.removeHidden(filename);
this._log(`STORAGE <x- DB: ${displayFileName}: deleted (hidden).`);
await this.plugin.storageAccess.removeHidden(storageFilePath);
try {
// -- @ts-ignore internalAPI
// await this.app.vault.adapter.reconcileInternalFile(filename);
await this.plugin.storageAccess.triggerHiddenFile(filename);
await this.plugin.storageAccess.triggerHiddenFile(storageFilePath);
} catch (ex) {
Logger("Failed to call internal API(reconcileInternalFile)", LOG_LEVEL_VERBOSE);
Logger(ex, LOG_LEVEL_VERBOSE);
this._log("Failed to call internal API(reconcileInternalFile)", LOG_LEVEL_VERBOSE);
this._log(ex, LOG_LEVEL_VERBOSE);
}
}
return true;
}
if (!isExists) {
await this.plugin.storageAccess.ensureDir(filename);
await this.plugin.storageAccess.writeHiddenFileAuto(filename, readContent(fileOnDB), { mtime: fileOnDB.mtime, ctime: fileOnDB.ctime });
await this.plugin.storageAccess.ensureDir(storageFilePath);
await this.plugin.storageAccess.writeHiddenFileAuto(storageFilePath, readContent(fileOnDB), { mtime: fileOnDB.mtime, ctime: fileOnDB.ctime });
try {
//@ts-ignore internalAPI
await this.app.vault.adapter.reconcileInternalFile(filename);
await this.plugin.storageAccess.triggerHiddenFile(storageFilePath);
} catch (ex) {
Logger("Failed to call internal API(reconcileInternalFile)", LOG_LEVEL_VERBOSE);
Logger(ex, LOG_LEVEL_VERBOSE);
this._log("Failed to call internal API(reconcileInternalFile)", LOG_LEVEL_VERBOSE);
this._log(ex, LOG_LEVEL_VERBOSE);
}
Logger(`STORAGE <-- DB:${filename}: written (hidden,new${force ? ", force" : ""})`);
this._log(`STORAGE <-- DB: ${displayFileName}: written (hidden,new${force ? ", force" : ""})`);
return true;
} else {
const content = await this.plugin.storageAccess.readHiddenFileAuto(filename);
const content = await this.plugin.storageAccess.readHiddenFileAuto(storageFilePath);
const docContent = readContent(fileOnDB);
if (await isDocContentSame(content, docContent) && !force) {
// Logger(`STORAGE <-- DB:${filename}: skipped (hidden) Not changed`, LOG_LEVEL_VERBOSE);
return true;
// this._log(`STORAGE <-- DB:${filename}: skipped (hidden) Not changed`, LOG_LEVEL_VERBOSE);
const stat = await this.plugin.storageAccess.statHidden(storageFilePath);
if (stat) {
markChangesAreSame(storageFilePath, fileOnDB.mtime, stat.mtime);
}
return undefined;
}
await this.plugin.storageAccess.writeHiddenFileAuto(filename, docContent, { mtime: fileOnDB.mtime, ctime: fileOnDB.ctime });
try {
// await this.app.vault.adapter.reconcileInternalFile(filename);
await this.plugin.storageAccess.triggerHiddenFile(filename);
} catch (ex) {
Logger("Failed to call internal API(reconcileInternalFile)", LOG_LEVEL_VERBOSE);
Logger(ex, LOG_LEVEL_VERBOSE);
}
Logger(`STORAGE <-- DB:${filename}: written (hidden, overwrite${force ? ", force" : ""})`);
return true;
if (await this.plugin.storageAccess.writeHiddenFileAuto(storageFilePath, docContent, { mtime: fileOnDB.mtime, ctime: fileOnDB.ctime })) {
const stat = await this.plugin.storageAccess.statHidden(storageFilePath) as UXStat;
this.markAsKnownChange(storageFilePath, stat.mtime);
try {
// await this.app.vault.adapter.reconcileInternalFile(filename);
await this.plugin.storageAccess.triggerHiddenFile(storageFilePath);
} catch (ex) {
this._log("Failed to call internal API(reconcileInternalFile)", LOG_LEVEL_VERBOSE);
this._log(ex, LOG_LEVEL_VERBOSE);
}
this._log(`STORAGE <-- DB: ${displayFileName}: written (hidden, overwrite${force ? ", force" : ""})`);
return true;
} else {
this._log(`STORAGE <-- DB: ${displayFileName}: written (hidden, overwrite${force ? ", force" : ""}) Failed`);
return false;
}
}
} catch (ex) {
Logger(`STORAGE <-- DB:${filename}: written (hidden, overwrite${force ? ", force" : ""}) Failed`);
Logger(ex, LOG_LEVEL_VERBOSE);
this._log(`STORAGE <-- DB: ${displayFileName}: written (hidden, overwrite${force ? ", force" : ""}) Failed`);
this._log(ex, LOG_LEVEL_VERBOSE);
return false;
}
});
@@ -683,16 +738,20 @@ export class HiddenFileSync extends LiveSyncCommands implements IObsidianModule
showJSONMergeDialogAndMerge(docA: LoadedEntry, docB: LoadedEntry): Promise<boolean> {
return new Promise((res) => {
Logger("Opening data-merging dialog", LOG_LEVEL_VERBOSE);
this._log("Opening data-merging dialog", LOG_LEVEL_VERBOSE);
const docs = [docA, docB];
const path = stripAllPrefixes(docA.path);
const modal = new JsonResolveModal(this.app, path, [docA, docB], async (keep, result) => {
const strippedPath = stripAllPrefixes(docA.path);
const storageFilePath = strippedPath;
const storeFilePath = strippedPath;
const displayFilename = `${storeFilePath}`;
// const path = this.prefixedConfigDir2configDir(stripAllPrefixes(docA.path)) || docA.path;
const modal = new JsonResolveModal(this.app, storageFilePath, [docA, docB], async (keep, result) => {
// modal.close();
try {
const filename = path;
// const filename = storeFilePath;
let needFlush = false;
if (!result && !keep) {
Logger(`Skipped merging: ${filename}`);
this._log(`Skipped merging: ${displayFilename}`);
res(false);
return;
}
@@ -701,41 +760,44 @@ export class HiddenFileSync extends LiveSyncCommands implements IObsidianModule
for (const doc of docs) {
if (doc._rev != keep) {
if (await this.localDatabase.deleteDBEntry(this.getPath(doc), { rev: doc._rev })) {
Logger(`Conflicted revision has been deleted: ${filename}`);
this._log(`Conflicted revision has been deleted: ${displayFilename}`);
needFlush = true;
}
}
}
}
if (!keep && result) {
const isExists = await this.plugin.storageAccess.isExistsIncludeHidden(filename);
const isExists = await this.plugin.storageAccess.isExistsIncludeHidden(storageFilePath);
if (!isExists) {
await this.plugin.storageAccess.ensureDir(filename);
await this.plugin.storageAccess.ensureDir(storageFilePath);
}
await this.plugin.storageAccess.writeHiddenFileAuto(filename, result);
const stat = await this.plugin.storageAccess.statHidden(filename);
await this.plugin.storageAccess.writeHiddenFileAuto(storageFilePath, result);
const stat = await this.plugin.storageAccess.statHidden(storageFilePath);
if (!stat) {
throw new Error("Stat failed");
}
const mtime = stat?.mtime ?? 0;
await this.storeInternalFileToDatabase({ path: filename, mtime, ctime: stat?.ctime ?? mtime, size: stat?.size ?? 0 }, true);
await this.storeInternalFileToDatabase({ path: storageFilePath, mtime, ctime: stat?.ctime ?? mtime, size: stat?.size ?? 0 }, true);
try {
//@ts-ignore internalAPI
await this.app.vault.adapter.reconcileInternalFile(filename);
await this.app.vault.adapter.reconcileInternalFile(storageFilePath);
} catch (ex) {
Logger("Failed to call internal API(reconcileInternalFile)", LOG_LEVEL_VERBOSE);
Logger(ex, LOG_LEVEL_VERBOSE);
this._log("Failed to call internal API(reconcileInternalFile)", LOG_LEVEL_VERBOSE);
this._log(ex, LOG_LEVEL_VERBOSE);
}
Logger(`STORAGE <-- DB:${filename}: written (hidden,merged)`);
this._log(`STORAGE <-- DB:${displayFilename}: written (hidden,merged)`);
}
if (needFlush) {
await this.extractInternalFileFromDatabase(filename, false);
Logger(`STORAGE --> DB:${filename}: extracted (hidden,merged)`);
if (await this.extractInternalFileFromDatabase(storeFilePath, false)) {
this._log(`STORAGE --> DB:${displayFilename}: extracted (hidden,merged)`);
} else {
this._log(`STORAGE --> DB:${displayFilename}: extracted (hidden,merged) Failed`);
}
}
res(true);
} catch (ex) {
Logger("Could not merge conflicted json");
Logger(ex, LOG_LEVEL_VERBOSE);
this._log("Could not merge conflicted json");
this._log(ex, LOG_LEVEL_VERBOSE);
res(false);
}
});
@@ -789,7 +851,7 @@ ${messageFetch}${messageOverwrite}${messageMerge}
$allSuspendExtraSync(): Promise<boolean> {
if (this.plugin.settings.syncInternalFiles) {
Logger("Hidden file synchronization have been temporarily disabled. Please enable them after the fetching, if you need them.", LOG_LEVEL_NOTICE)
this._log("Hidden file synchronization have been temporarily disabled. Please enable them after the fetching, if you need them.", LOG_LEVEL_NOTICE)
this.plugin.settings.syncInternalFiles = false;
}
return Promise.resolve(true);
@@ -810,7 +872,7 @@ ${messageFetch}${messageOverwrite}${messageMerge}
await this.plugin.saveSettings();
return;
}
Logger("Gathering files for enabling Hidden File Sync", LOG_LEVEL_NOTICE);
this._log("Gathering files for enabling Hidden File Sync", LOG_LEVEL_NOTICE);
if (mode == "FETCH") {
await this.syncInternalFilesAndDatabase("pullForce", true);
} else if (mode == "OVERWRITE") {
@@ -821,7 +883,7 @@ ${messageFetch}${messageOverwrite}${messageMerge}
this.plugin.settings.useAdvancedMode = true;
this.plugin.settings.syncInternalFiles = true;
await this.plugin.saveSettings();
Logger(`Done! Restarting the app is strongly recommended!`, LOG_LEVEL_NOTICE);
this._log(`Done! Restarting the app is strongly recommended!`, LOG_LEVEL_NOTICE);
}
async scanInternalFiles(): Promise<InternalFileInfo[]> {
@@ -869,8 +931,8 @@ ${messageFetch}${messageOverwrite}${messageMerge}
try {
w = await this.app.vault.adapter.list(path);
} catch (ex) {
Logger(`Could not traverse(HiddenSync):${path}`, LOG_LEVEL_INFO);
Logger(ex, LOG_LEVEL_VERBOSE);
this._log(`Could not traverse(HiddenSync):${path}`, LOG_LEVEL_INFO);
this._log(ex, LOG_LEVEL_VERBOSE);
return [];
}
const filesSrc = [

11
src/features/LiveSyncCommands.ts
View File

@@ -1,5 +1,6 @@
import { Logger } from "octagonal-wheels/common/logger";
import { getPath } from "../common/utils.ts";
import { type AnyEntry, type DocumentID, type EntryHasPath, type FilePath, type FilePathWithPrefix } from "../lib/src/common/types.ts";
import { LOG_LEVEL_INFO, LOG_LEVEL_NOTICE, type AnyEntry, type DocumentID, type EntryHasPath, type FilePath, type FilePathWithPrefix, type LOG_LEVEL } from "../lib/src/common/types.ts";
import type ObsidianLiveSyncPlugin from "../main.ts";
@@ -40,4 +41,12 @@ export abstract class LiveSyncCommands {
_isDatabaseReady() {
return this.plugin._isDatabaseReady();
}
_log = (msg: any, level: LOG_LEVEL = LOG_LEVEL_INFO, key?: string) => {
if (typeof msg === "string" && level !== LOG_LEVEL_NOTICE) {
msg = `[${this.constructor.name}]\u{200A} ${msg}`;
}
// console.log(msg);
Logger(msg, level, key);
};
}

1
src/modules/features/ModuleObsidianSettingAsMarkdown.ts
View File

@@ -162,6 +162,7 @@ export class ModuleObsidianSettingsAsMarkdown extends AbstractObsidianModule imp
const saveData = { ...(settings ? settings : this.settings) } as Partial<ObsidianLiveSyncSettings>;
delete saveData.encryptedCouchDBConnection;
delete saveData.encryptedPassphrase;
delete saveData.additionalSuffixOfDatabaseName;
if (!saveData.writeCredentialsForSettingSync && !keepCredential) {
delete saveData.couchDB_USER;
delete saveData.couchDB_PASSWORD;

119
src/modules/features/SettingDialogue/ObsidianLiveSyncSettingTab.ts
View File

@@ -1,6 +1,6 @@
import { App, PluginSettingTab, MarkdownRenderer, stringifyYaml } from "../../../deps.ts";
import {
DEFAULT_SETTINGS, type ObsidianLiveSyncSettings, type ConfigPassphraseStore, type RemoteDBSettings, type FilePathWithPrefix, type HashAlgorithm, type DocumentID, LOG_LEVEL_NOTICE, LOG_LEVEL_VERBOSE, LOG_LEVEL_INFO, type LoadedEntry, PREFERRED_SETTING_CLOUDANT, PREFERRED_SETTING_SELF_HOSTED, FLAGMD_REDFLAG2_HR, FLAGMD_REDFLAG3_HR, REMOTE_COUCHDB, REMOTE_MINIO, PREFERRED_JOURNAL_SYNC, FLAGMD_REDFLAG, type ConfigLevel, LEVEL_POWER_USER, LEVEL_ADVANCED, LEVEL_EDGE_CASE, type MetaEntry, type FilePath
DEFAULT_SETTINGS, type ObsidianLiveSyncSettings, type ConfigPassphraseStore, type RemoteDBSettings, type FilePathWithPrefix, type HashAlgorithm, type DocumentID, LOG_LEVEL_NOTICE, LOG_LEVEL_VERBOSE, LOG_LEVEL_INFO, type LoadedEntry, PREFERRED_SETTING_CLOUDANT, PREFERRED_SETTING_SELF_HOSTED, FLAGMD_REDFLAG2_HR, FLAGMD_REDFLAG3_HR, REMOTE_COUCHDB, REMOTE_MINIO, PREFERRED_JOURNAL_SYNC, FLAGMD_REDFLAG, type ConfigLevel, LEVEL_POWER_USER, LEVEL_ADVANCED, LEVEL_EDGE_CASE, type MetaEntry, type FilePath,
} from "../../../lib/src/common/types.ts";
import { createBlob, delay, isDocContentSame, isObjectDifferent, readAsBlob, sizeToHumanReadable } from "../../../lib/src/common/utils.ts";
import { versionNumberString2Number } from "../../../lib/src/string_and_binary/convert.ts";
@@ -9,8 +9,8 @@ import { balanceChunkPurgedDBs, checkSyncInfo, isCloudantURI, purgeUnreferencedC
import { testCrypt } from "../../../lib/src/encryption/e2ee_v2.ts";
import ObsidianLiveSyncPlugin from "../../../main.ts";
import { getPath, requestToCouchDB, scheduleTask } from "../../../common/utils.ts";
import { request, TFile } from "obsidian";
import { shouldBeIgnored } from "../../../lib/src/string_and_binary/path.ts";
import { request } from "obsidian";
import { addPrefix, shouldBeIgnored, stripAllPrefixes } from "../../../lib/src/string_and_binary/path.ts";
import MultipleRegExpControl from './MultipleRegExpControl.svelte';
import { LiveSyncCouchDBReplicator } from "../../../lib/src/replication/couchdb/LiveSyncReplicator.ts";
import { type AllSettingItemKey, type AllStringItemKey, type AllNumericItemKey, type AllBooleanItemKey, type AllSettings, OnDialogSettingsDefault, type OnDialogSettings, getConfName } from "./settingConstants.ts";
@@ -23,6 +23,8 @@ import { confirmWithMessage } from "../../coreObsidian/UILib/dialogs.ts";
import { EVENT_REQUEST_COPY_SETUP_URI, EVENT_REQUEST_OPEN_PLUGIN_SYNC_DIALOG, EVENT_REQUEST_OPEN_SETUP_URI, EVENT_REQUEST_RELOAD_SETTING_TAB, EVENT_REQUEST_SHOW_HISTORY, eventHub } from "../../../common/events.ts";
import { skipIfDuplicated } from "octagonal-wheels/concurrency/lock";
import { JournalSyncMinio } from "../../../lib/src/replication/journal/objectstore/JournalSyncMinio.ts";
import { ICHeader, ICXHeader, PSCHeader } from "../../../common/types.ts";
import { HiddenFileSync } from "../../../features/HiddenFileSync/CmdHiddenFileSync.ts";
export type OnUpdateResult = {
visibility?: boolean,
@@ -1727,6 +1729,17 @@ However, your report is needed to stabilise this. I appreciate you for your grea
const enableOnlyOnPluginSyncIsNotEnabled = enableOnly(() => this.isConfiguredAs("usePluginSync", false));
const visibleOnlyOnPluginSyncEnabled = visibleOnly(() => this.isConfiguredAs("usePluginSync", true));
this.createEl(paneEl, "div", {
text: "Please set device name to identify this device. This name should be unique among your devices. While not configured, we cannot enable this feature.",
cls: "op-warn"
}, c => {
}, visibleOnly(() => this.isConfiguredAs("deviceAndVaultName", "")));
this.createEl(paneEl, "div", {
text: "We cannot change the device name while this feature is enabled. Please disable this feature to change the device name.",
cls: "op-warn-info"
}, c => {
}, visibleOnly(() => this.isConfiguredAs("usePluginSync", true)));
new Setting(paneEl)
.autoWireText("deviceAndVaultName", {
placeHolder: "desktop", onUpdate: enableOnlyOnPluginSyncIsNotEnabled
@@ -1856,11 +1869,12 @@ ${stringifyYaml(pluginConfig)}`;
void addPanel(paneEl, "Recovery and Repair").then((paneEl) => {
const addResult = (path: string, file: TFile | false, fileOnDB: LoadedEntry | false) => {
const addResult = async (path: string, file: FilePathWithPrefix | false, fileOnDB: LoadedEntry | false) => {
const storageFileStat = file ? await this.plugin.storageAccess.statHidden(file) : null;
resultArea.appendChild(this.createEl(resultArea, "div", {}, el => {
el.appendChild(this.createEl(el, "h6", { text: path }));
el.appendChild(this.createEl(el, "div", {}, infoGroupEl => {
infoGroupEl.appendChild(this.createEl(infoGroupEl, "div", { text: `Storage : Modified: ${!file ? `Missing:` : `${new Date(file.stat.mtime).toLocaleString()}, Size:${file.stat.size}`}` }))
infoGroupEl.appendChild(this.createEl(infoGroupEl, "div", { text: `Storage : Modified: ${!storageFileStat ? `Missing:` : `${new Date(storageFileStat.mtime).toLocaleString()}, Size:${storageFileStat.size}`}` }))
infoGroupEl.appendChild(this.createEl(infoGroupEl, "div", { text: `Database: Modified: ${!fileOnDB ? `Missing:` : `${new Date(fileOnDB.mtime).toLocaleString()}, Size:${fileOnDB.size}`}` }))
}));
if (fileOnDB && file) {
@@ -1875,20 +1889,24 @@ ${stringifyYaml(pluginConfig)}`;
if (file) {
el.appendChild(this.createEl(el, "button", { text: "Storage -> Database" }, buttonEl => {
buttonEl.onClickEvent(async () => {
// const file = this.plugin.storageAccess.getFileStub(path);
// if (!file) {
// Logger(`File not found: ${path}`, LOG_LEVEL_NOTICE);
// return;
// }
// const content = await this.plugin.storageAccess.readStubContent(file);
// if (!content) {
// Logger(`Content cannot be read: ${path}`, LOG_LEVEL_NOTICE);
// return;
// }
// this.plugin.databaseFileAccess.store(content, true);
if (!await this.plugin.fileHandler.storeFileToDB(file.path as FilePath, true)) {
Logger(`Failed to store the file to the database: ${file.path}`, LOG_LEVEL_NOTICE);
return;
if (file.startsWith(".")) {
const addOn = this.plugin.getAddOn<HiddenFileSync>(HiddenFileSync.name);
if (addOn) {
const file = (await addOn.scanInternalFiles()).find(e => e.path == path);
if (!file) {
Logger(`Failed to find the file in the internal files: ${path}`, LOG_LEVEL_NOTICE);
return;
}
if (!await addOn.storeInternalFileToDatabase(file, true)) {
Logger(`Failed to store the file to the database (Hidden file): ${file}`, LOG_LEVEL_NOTICE);
return;
}
}
} else {
if (!await this.plugin.fileHandler.storeFileToDB(file as FilePath, true)) {
Logger(`Failed to store the file to the database: ${file}`, LOG_LEVEL_NOTICE);
return;
}
}
el.remove();
})
@@ -1897,10 +1915,20 @@ ${stringifyYaml(pluginConfig)}`;
if (fileOnDB) {
el.appendChild(this.createEl(el, "button", { text: "Database -> Storage" }, buttonEl => {
buttonEl.onClickEvent(async () => {
// this.plugin.pullFile(this.plugin.getPath(fileOnDB), undefined, true, undefined, false);
if (!await this.plugin.fileHandler.dbToStorage(fileOnDB as MetaEntry, null, true)) {
Logger(`Failed to store the file to the storage: ${fileOnDB.path}`, LOG_LEVEL_NOTICE);
return;
if (fileOnDB.path.startsWith(ICHeader)) {
const addOn = this.plugin.getAddOn<HiddenFileSync>(HiddenFileSync.name);
if (addOn) {
if (!await addOn.extractInternalFileFromDatabase(path as FilePath, true)) {
Logger(`Failed to store the file to the database (Hidden file): ${file}`, LOG_LEVEL_NOTICE);
return;
}
}
} else {
if (!await this.plugin.fileHandler.dbToStorage(fileOnDB as MetaEntry, null, true)) {
Logger(`Failed to store the file to the storage: ${fileOnDB.path}`, LOG_LEVEL_NOTICE);
return;
}
}
el.remove();
})
@@ -1910,14 +1938,14 @@ ${stringifyYaml(pluginConfig)}`;
}))
}
const checkBetweenStorageAndDatabase = async (file: TFile, fileOnDB: LoadedEntry) => {
const checkBetweenStorageAndDatabase = async (file: FilePathWithPrefix, fileOnDB: LoadedEntry) => {
const dataContent = readAsBlob(fileOnDB);
const content = createBlob(await this.plugin.storageAccess.readFileAuto(file.path))
const content = createBlob(await this.plugin.storageAccess.readHiddenFileBinary(file))
if (await isDocContentSame(content, dataContent)) {
Logger(`Compare: SAME: ${file.path}`)
Logger(`Compare: SAME: ${file}`)
} else {
Logger(`Compare: CONTENT IS NOT MATCHED! ${file.path}`, LOG_LEVEL_NOTICE);
addResult(file.path, file, fileOnDB)
Logger(`Compare: CONTENT IS NOT MATCHED! ${file}`, LOG_LEVEL_NOTICE);
void addResult(file, file, fileOnDB)
}
}
new Setting(paneEl)
@@ -1945,17 +1973,26 @@ ${stringifyYaml(pluginConfig)}`;
.setDisabled(false)
.setCta()
.onClick(async () => {
Logger("Start verifying all files", LOG_LEVEL_NOTICE, "verify");
const ignorePatterns = this.plugin.settings.syncInternalFilesIgnorePatterns
.replace(/\n| /g, "")
.split(",").filter(e => e).map(e => new RegExp(e, "i"));
this.plugin.localDatabase.hashCaches.clear();
Logger("Start verifying all files", LOG_LEVEL_NOTICE, "verify");
const files = this.app.vault.getFiles();
const documents = [] as FilePathWithPrefix[];
const files = await this.plugin.storageAccess.getFilesIncludeHidden("/", undefined, ignorePatterns)
const documents = [] as FilePath[];
const adn = this.plugin.localDatabase.findAllNormalDocs()
for await (const i of adn) documents.push(getPath(i));
const adn = this.plugin.localDatabase.findAllDocs()
for await (const i of adn) {
const path = getPath(i);
if (path.startsWith(ICXHeader)) continue;
if (path.startsWith(PSCHeader)) continue;
documents.push(stripAllPrefixes(path));
}
const allPaths = [
...new Set([
...documents,
...files.map(e => e.path as FilePathWithPrefix)])];
...files])];
let i = 0;
const incProc = () => {
i++;
@@ -1967,27 +2004,29 @@ ${stringifyYaml(pluginConfig)}`;
if (shouldBeIgnored(path)) {
return incProc();
}
const abstractFile = this.plugin.storageAccess.getFileStub(path);
const fileOnStorage = abstractFile instanceof TFile ? abstractFile : false;
const stat = await this.plugin.storageAccess.isExistsIncludeHidden(path) ? await this.plugin.storageAccess.statHidden(path) : false;
const fileOnStorage = stat != null ? stat : false;
if (!await this.plugin.$$isTargetFile(path)) return incProc();
const releaser = await semaphore.acquire(1)
if (fileOnStorage && this.plugin.$$isFileSizeExceeded(fileOnStorage.stat.size)) return incProc();
if (fileOnStorage && this.plugin.$$isFileSizeExceeded(fileOnStorage.size)) return incProc();
try {
const fileOnDB = await this.plugin.localDatabase.getDBEntry(path);
const isHiddenFile = path.startsWith(".");
const dbPath = isHiddenFile ? addPrefix(path, ICHeader) : path;
const fileOnDB = await this.plugin.localDatabase.getDBEntry(dbPath);
if (fileOnDB && this.plugin.$$isFileSizeExceeded(fileOnDB.size)) return incProc();
if (!fileOnDB && fileOnStorage) {
Logger(`Compare: Not found on the local database: ${path}`, LOG_LEVEL_NOTICE);
addResult(path, fileOnStorage, false)
void addResult(path, path, false)
return incProc();
}
if (fileOnDB && !fileOnStorage) {
Logger(`Compare: Not found on the storage: ${path}`, LOG_LEVEL_NOTICE);
addResult(path, false, fileOnDB)
void addResult(path, false, fileOnDB)
return incProc();
}
if (fileOnStorage && fileOnDB) {
await checkBetweenStorageAndDatabase(fileOnStorage, fileOnDB)
await checkBetweenStorageAndDatabase(path, fileOnDB)
}
} catch (ex) {
Logger(`Error while processing ${path}`, LOG_LEVEL_NOTICE);

4
src/modules/features/SettingDialogue/settingConstants.ts
View File

@@ -327,8 +327,8 @@ export const SettingInformation: Partial<Record<keyof AllSettings, Configuration
desc: "MB (0 to disable)."
},
"usePluginSyncV2": {
name: "Enable per-file-saved customization sync",
desc: "If enabled per-filed efficient customization sync will be used. We need a small migration when enabling this. And all devices should be updated to v0.23.18. Once we enabled this, we lost a compatibility with old versions."
name: "Enable per-file customization sync",
desc: "If enabled, efficient per-file customization sync will be used. A minor migration is required when enabling this feature, and all devices must be updated to v0.23.18. Enabling this feature will result in losing compatibility with older versions."
},
"handleFilenameCaseSensitive": {
name: "Handle files as Case-Sensitive",

66
updates.md
View File

@@ -1,38 +1,29 @@
## 0.24.0 RC Release Note
## 0.24.0
**Note:** This will be rewritten with the stable release. I confess, before you take the time, this is quite long.
I know that we have been waiting for a long time. It is finally released!
Over the past three years since the inception of the plugin, various features have been implemented to address diverse user needs. This is so honourable and I am grateful for your years of support.
However, However, this process has resulted in a codebase that has become increasingly disorganised, with features becoming entangled.
Over the past three years since the inception of the plugin, various features have been implemented to address diverse user needs. This is truly honourable, and I am grateful for your years of support. However, this process has led to an increasingly disorganised codebase, with features becoming entangled. Consequently, this has led to a situation where bugs can go unnoticed and resolving one issue may inadvertently introduce another.
Consequently, this has led to a situation where bugs can go unnoticed or resolving one issue may inadvertently introduce another.
In 0.24.0, I reorganised the previously jumbled main codebase into clearly defined modules. Although I had assumed that the total size of the code would not increase, I discovered that it has in fact increased. While the complexity is still considerable, the refactoring has improved the clarity of the code's structure. Additionally, while testing the release candidates, we still found many bugs to fix, which helped to make this plug-in robust and stable. Therefore, we are now ready to use the updated plug-in, and in addition to that, proceed to the next step.
In 0.24.0, I reorganised the previously disjointed main codebase into clearly defined modules. Although I anticipated that the overall volume of code would not increase, I discovered that it has, in fact, expanded. While the complexity may still be considerable, the refactoring has enhanced clarity regarding the current structure of the code. (The next focus may involve a review of dependencies).
This is also the first step towards a fully-fledged-fancy LiveSync, not just a plug-in from Obsidian. Of course, it will still be a plug-in primarily and foremost, but this development marks a significant step towards the self-hosting concept.
Throughout this process, a significant number of bugs have been resolved. And it may be worth mentioning that these bugs may had given rise to other bugs. I kindly request that you verify whether your issues have been addressed. At least conflict resolution and related issues have improved significantly.
It is also the first step towards a fully-fledged-fancy LiveSync, not just a plug-in from Obsidian. Of course, it will still be a plug-in as a first class and foremost, but this development marks a significant step towards the self-hosting concept.
This dev release is very close to the beta version that I had previously indicated would not be released. As a result, I have faced challenges in maintaining the main branch while working on this dev release. Regrettably, I have not been able to make any commits to the main branch in the last three weeks. Thus, the dev branch will remain reserved for major changes only.
The Release Candidate will be available for a few days and will only be officially released once users, including myself, have confirmed that there are no issues.
Finally, I would like to once again express my respect and gratitude to all of you once again. Thank you for your interest in the development version. Your contributions and dedication are greatly appreciated through testing.
Finally, I would like to once again express my respect and gratitude to all of you. My gratitude extends to all of the dev testers! Your contributions have certainly made the plug-in robust and stable!
Thank you, and I hope your troubles will be resolved!
---
## 0.24.0.dev-rc4
## 0.24.0
### Improved
- The welcome message is now more simple to encourage the use of the Setup-URI.
- And the secondary message is also simpler to guide users to Minimal Setup.
- The secondary message is also simpler to guide users to Minimal Setup.
- But Setup-URI will be recommended again, due to its importance.
- These dialogues contain a link to the documentation which can be clicked.
- The minimal setup is more minimal now. And, the setup is more user-friendly.
- Now the Configuration of the remote database is checked more robust, but we can ignore the warning and proceed with the setup.
- Now the Configuration of the remote database is checked more robustly, but we can ignore the warning and proceed with the setup.
- Before we are asked about each feature, we are asked if we want to use optional features in the first place.
- This is to prevent the user from being overwhelmed by the features.
- And made it clear that it is not recommended for new users.
@@ -43,45 +34,34 @@ Thank you, and I hope your troubles will be resolved!
- Especially auto-closing dialogues are now explicitly labelled: `To stop the countdown, tap anywhere on the dialogue`.
- Now if the is plugin configured to ignore some events, we will get a chance to fix it, in addition to the warning.
- And why that has happened is also explained in the dialogue.
- A note relating to device names has been added to Customisation Sync on the setting dialogue.
- We can verify and resolve also the hidden files now.
### Fixed
- We can resolve the conflict of the JSON file correctly now.
- Verifying files between the local database and storage is now working correctly.
- While restarting the plug-in, the shown dialogues will be automatically closed to avoid unexpected behaviour.
- Replicated documents that the local device has configured to ignore are now correctly ignored.
- The chunks of the document on the local device during the first transfer will be created correctly.
- And why we should create them is now explained in the dialogue.
- If optional features have been enabled in the wizard, `Enable advanced features` will be toggled correctly.
The hidden file sync is now working correctly.
- Now the deletion of hidden files is correctly synchronised.
- Customisation Sync is now working correctly together with hidden file sync.
- No longer database suffix is stored in the setting sharing markdown.
- A fair number of bugs have been fixed.
### Changed
- Some default settings have been changed for easier new user experience.
- Some default settings have been changed for an easier new user experience.
- Preventing the meaningless migration of the settings.
### Tidied
- Commented-out codes have been gradually removed.
## 0.24.0.dev-rc3
### Fixed
- No longer Missing Translation Warning is shown in the console.
- Fixed the issue where some functions were not working properly (`_` started functions).
## 0.24.0.dev-rc2
### Fixed
- Some status icons is now shown correctly.
## 0.24.0-rc1
### Fixed
- A fair numbers of bugs have been fixed.
### Tiding
- The codebase has been reorganised into clearly defined modules.
- Commented-out codes have been gradually removed.
Older notes is in [updates_old.md](https://github.com/vrtmrz/obsidian-livesync/blob/main/updates_old.md).
Older notes are in [updates_old.md](https://github.com/vrtmrz/obsidian-livesync/blob/main/updates_old.md).