public/obsidian-livesync
1
0
Fork 0
mirror of https://github.com/vrtmrz/obsidian-livesync.git synced 2026-05-06 15:51:51 +00:00
Code Issues Packages Projects Releases Wiki Activity

Compare commits

base: public:0.24.0.dev-rc4
Branches Tags
public:main public:auto_conf public:beta public:alpha public:snyk-fix-34b8981b68f7111ada8b22377113a98c public:snyk-upgrade-7fd31e6f474568014502e7205847f257 public:snyk-fix-90f3b71450f7543a1c61724d8afa0e71 public:refactor_services_2 public:refactor_services public:dev public:snyk-upgrade-fe9625385aa4be849c7157033fd2abb0 public:snyk-upgrade-99a31e70c822894595544b0c9813b11a public:svelteui public:disenchant public:snyk-fix-61f76ef0bce2a276df9552c2fc80a14f public:snyk-upgrade-2baa7938ba318464c3bad5c3fd5cbda5 public:snyk-upgrade-7af62a575eab20e8fb4e9565bf1adcfc public:snyk-upgrade-b32f281b31a81483f126b8fb6c54a957 public:snyk-upgrade-bd61b1103442d3faecc9fc726c34e9a4 public:snyk-upgrade-5b2d6584430c061b5187611a5d7fb64a public:snyk-fix-0e9fbbcb374fcb8d5e8cb027d8414fb7 public:snyk-upgrade-5242daf91e706d1ae500c9ea6545636e public:snyk-upgrade-70434a6822fbad83424135dfad9df909 public:snyk-upgrade-8fa01ccad0352125f1866d41e57d35e1 public:snyk-upgrade-39eb97b4655e70091f718a88fccd57c1 public:snyk-upgrade-2b99d54d19e207887e36c3071b796925 public:forprivacy public:snyk-upgrade-dad9088d0a333677e0157e3deef7fc38 public:snyk-upgrade-b9a76d74b3e5723759af5b3319aabb80 public:snyk-upgrade-7da6459d8ff5d293d80de9d80061d912 public:snyk-upgrade-6a3786fc4cbe394542e9d1b6058c46e7 public:snyk-upgrade-8fc9e62771dc41b0f23dc0efefb5c949
public:0.25.60 public:0.25.59 public:0.25.58 public:0.25.57 public:0.25.56+patched5 public:0.25.56+patched4 public:0.25.56+patched3 public:0.25.56+patched2 public:0.25.56+patched1 public:0.25.56 public:0.25.55 public:0.25.54 public:0.25.53 public:0.25.52-patched-3 public:0.25.52-patched-2 public:0.25.52-patched-1 public:0.25.52 public:0.25.51 public:0.25.50 public:0.25.49 public:0.25.48 public:0.25.47 public:0.25.46 public:0.25.45 public:0.25.44 public:0.25.43-patched-9 public:0.25.43-patched-8 public:0.25.43-patched-7 public:0.25.43-patched-6 public:0.25.43-patched-5 public:0.25.43-patched-4 public:0.25.43-patched-3 public:0.25.43-patched-2 public:0.25.43-patched-1 public:0.25.43 public:0.25.42 public:0.25.41-patched-3 public:0.25.41-patched-2 public:0.25.41-patched-1 public:0.25.41 public:0.25.40 public:0.25.39 public:0.25.38 public:0.25.37 public:0.25.36 public:0.25.35 public:0.25.34 public:0.25.33 public:0.25.32 public:0.25.31 public:0.25.30 public:0.25.30-dev public:0.25.29 public:0.25.28 public:0.25.27 public:0.25.26 public:0.25.25 public:0.25.24 public:0.25.24.beta3 public:0.25.24.beta2 public:0.25.24.beta1 public:0.25.23 public:0.25.23.beta1 public:0.25.22 public:0.25.21 public:0.25.21.beta2 public:0.25.21.beta1 public:0.25.20 public:0.25.19 public:0.25.18 public:0.25.17 public:0.25.16 public:0.25.15 public:0.25.14 public:0.25.13 public:0.25.12 public:0.25.11 public:0.25.10 public:0.25.9 public:0.25.8 public:0.25.7 public:0.25.6 public:0.25.5 public:0.25.4 public:0.25.3 public:0.25.2 public:0.25.1 public:0.25.0 public:0.25.0-beta1 public:0.24.31 public:0.24.30 public:0.24.29 public:0.24.28 public:0.24.27 public:0.24.26 public:0.24.25 public:0.24.24 public:0.24.23 public:0.24.22 public:0.24.21 public:0.24.20 public:0.24.19 public:0.24.18 public:0.24.17 public:0.24.16 public:0.24.15 public:0.24.14 public:0.24.13 public:0.24.12 public:0.24.11 public:0.24.10 public:0.24.9 public:0.24.8 public:0.24.7 public:0.24.6 public:0.24.5 public:0.24.4 public:0.24.3 public:0.24.2 public:0.24.1 public:0.24.0 public:0.24.0.dev-rc8 public:0.24.0.dev-rc7 public:0.24.0.dev-rc6 public:0.24.0.dev-rc5 public:0.24.0.dev-rc4 public:0.24.0.dev-rc3 public:0.24.0.dev-rc2 public:0.23.23 public:0.23.22 public:0.23.21 public:0.23.20 public:0.23.19 public:0.23.18 public:0.23.17 public:0.23.16 public:0.23.15 public:0.23.14 public:0.23.13 public:0.23.12 public:0.23.11 public:0.23.10 public:0.23.9 public:0.23.8 public:0.23.7 public:0.23.6 public:0.23.5 public:0.23.4 public:0.23.3 public:0.23.2 public:0.23.1 public:0.23.0 public:0.22.19 public:0.22.18 public:0.22.17 public:0.22.16 public:0.22.15 public:0.22.14 public:0.22.13 public:0.22.12 public:0.22.11 public:0.22.10 public:0.22.9 public:0.22.8 public:0.22.7 public:0.22.6 public:0.22.5 public:0.22.4 public:0.22.3 public:0.22.2 public:0.22.1 public:0.22.0 public:0.21.5 public:0.21.4 public:0.21.3 public:0.21.2 public:0.21.1 public:0.21.0 public:0.20.7 public:0.20.6 public:0.20.5 public:0.20.4 public:0.20.3 public:0.20.2 public:0.20.1 public:0.20.0 public:0.19.23 public:0.19.22 public:0.19.21 public:0.19.20 public:0.19.19 public:0.19.18 public:0.19.17 public:0.19.16 public:0.19.15 public:0.19.14 public:0.19.13 public:0.19.12 public:0.19.11 public:0.19.10 public:0.19.9 public:0.19.8 public:0.19.7 public:0.19.6 public:0.19.5 public:0.19.4 public:0.19.3 public:0.19.2 public:0.19.1 public:0.19.0 public:0.18.6 public:0.18.5 public:0.18.4 public:0.18.3 public:0.18.2 public:0.18.1 public:0.18.0 public:0.17.34 public:0.17.33 public:0.17.32 public:0.17.31 public:0.17.30 public:0.17.29 public:0.17.28 public:0.17.27 public:0.17.26 public:0.17.25 public:0.17.24 public:0.17.23 public:0.17.22 public:0.17.21 public:0.17.20 public:0.17.19 public:0.17.18 public:0.17.17 public:0.17.16 public:0.17.15 public:0.17.14 public:0.17.13 public:0.17.12 public:0.17.11 public:0.17.10 public:0.17.9 public:0.17.8 public:0.17.7 public:0.17.6 public:0.17.5 public:0.17.4 public:0.17.3 public:0.17.2 public:0.17.1 public:0.17.0 public:0.16.8 public:0.16.7 public:0.16.6 public:0.16.5 public:0.16.4 public:0.16.3 public:0.16.2 public:0.16.1 public:0.16.0 public:0.15.10 public:0.15.9 public:0.15.8 public:0.15.7 public:0.15.6 public:0.15.5 public:0.15.4 public:0.15.3 public:0.15.2 public:0.15.1 public:0.15.0 public:0.14.8 public:0.14.7 public:0.14.6 public:0.14.5 public:0.14.4 public:0.14.3 public:0.14.2 public:0.14.1 public:0.14.0 public:0.13.4 public:0.13.3 public:0.13.2 public:0.13.1 public:0.13.0 public:0.12.5 public:0.12.4 public:0.12.3 public:0.12.2 public:0.12.1 public:0.12.0 public:0.11.10 public:0.11.9 public:0.11.8 public:0.11.7 public:0.11.6 public:0.11.5 public:0.11.4 public:0.11.3 public:0.11.2 public:0.11.1 public:0.11.0 public:0.10.1 public:0.10.0 public:0.9.0 public:0.8.8 public:0.8.7 public:0.8.6 public:0.8.5 public:0.8.3 public:0.8.2 public:0.8.1 public:0.8.0 public:0.7.2 public:0.7.1 public:0.7.0 public:0.6.1 public:0.6.0 public:0.5.0 public:0.4.1 public:0.4.0 public:0.3.1 public:0.3.0 public:0.2.1 public:0.2.0 public:0.1.26 public:0.1.25 public:0.1.24 public:0.1.23 public:0.1.22 public:0.1.21 public:0.1.20 public:0.1.19 public:0.1.18 public:0.1.17 public:0.1.16 public:0.1.15 public:0.1.14 public:0.1.13 public:0.1.12 public:0.1.11 public:0.1.10 public:0.1.9 public:0.1.8 public:0.1.7 public:0.1.6 public:0.1.5 public:0.1.4 public:0.1.3 public:0.1.2 public:0.1.1 public:0.1.0 public:0.0.9 public:0.0.8 public:0.0.7 public:0.0.6
...
compare: public:0.24.0
Branches Tags
public:main public:auto_conf public:beta public:alpha public:snyk-fix-34b8981b68f7111ada8b22377113a98c public:snyk-upgrade-7fd31e6f474568014502e7205847f257 public:snyk-fix-90f3b71450f7543a1c61724d8afa0e71 public:refactor_services_2 public:refactor_services public:dev public:snyk-upgrade-fe9625385aa4be849c7157033fd2abb0 public:snyk-upgrade-99a31e70c822894595544b0c9813b11a public:svelteui public:disenchant public:snyk-fix-61f76ef0bce2a276df9552c2fc80a14f public:snyk-upgrade-2baa7938ba318464c3bad5c3fd5cbda5 public:snyk-upgrade-7af62a575eab20e8fb4e9565bf1adcfc public:snyk-upgrade-b32f281b31a81483f126b8fb6c54a957 public:snyk-upgrade-bd61b1103442d3faecc9fc726c34e9a4 public:snyk-upgrade-5b2d6584430c061b5187611a5d7fb64a public:snyk-fix-0e9fbbcb374fcb8d5e8cb027d8414fb7 public:snyk-upgrade-5242daf91e706d1ae500c9ea6545636e public:snyk-upgrade-70434a6822fbad83424135dfad9df909 public:snyk-upgrade-8fa01ccad0352125f1866d41e57d35e1 public:snyk-upgrade-39eb97b4655e70091f718a88fccd57c1 public:snyk-upgrade-2b99d54d19e207887e36c3071b796925 public:forprivacy public:snyk-upgrade-dad9088d0a333677e0157e3deef7fc38 public:snyk-upgrade-b9a76d74b3e5723759af5b3319aabb80 public:snyk-upgrade-7da6459d8ff5d293d80de9d80061d912 public:snyk-upgrade-6a3786fc4cbe394542e9d1b6058c46e7 public:snyk-upgrade-8fc9e62771dc41b0f23dc0efefb5c949
public:0.25.60 public:0.25.59 public:0.25.58 public:0.25.57 public:0.25.56+patched5 public:0.25.56+patched4 public:0.25.56+patched3 public:0.25.56+patched2 public:0.25.56+patched1 public:0.25.56 public:0.25.55 public:0.25.54 public:0.25.53 public:0.25.52-patched-3 public:0.25.52-patched-2 public:0.25.52-patched-1 public:0.25.52 public:0.25.51 public:0.25.50 public:0.25.49 public:0.25.48 public:0.25.47 public:0.25.46 public:0.25.45 public:0.25.44 public:0.25.43-patched-9 public:0.25.43-patched-8 public:0.25.43-patched-7 public:0.25.43-patched-6 public:0.25.43-patched-5 public:0.25.43-patched-4 public:0.25.43-patched-3 public:0.25.43-patched-2 public:0.25.43-patched-1 public:0.25.43 public:0.25.42 public:0.25.41-patched-3 public:0.25.41-patched-2 public:0.25.41-patched-1 public:0.25.41 public:0.25.40 public:0.25.39 public:0.25.38 public:0.25.37 public:0.25.36 public:0.25.35 public:0.25.34 public:0.25.33 public:0.25.32 public:0.25.31 public:0.25.30 public:0.25.30-dev public:0.25.29 public:0.25.28 public:0.25.27 public:0.25.26 public:0.25.25 public:0.25.24 public:0.25.24.beta3 public:0.25.24.beta2 public:0.25.24.beta1 public:0.25.23 public:0.25.23.beta1 public:0.25.22 public:0.25.21 public:0.25.21.beta2 public:0.25.21.beta1 public:0.25.20 public:0.25.19 public:0.25.18 public:0.25.17 public:0.25.16 public:0.25.15 public:0.25.14 public:0.25.13 public:0.25.12 public:0.25.11 public:0.25.10 public:0.25.9 public:0.25.8 public:0.25.7 public:0.25.6 public:0.25.5 public:0.25.4 public:0.25.3 public:0.25.2 public:0.25.1 public:0.25.0 public:0.25.0-beta1 public:0.24.31 public:0.24.30 public:0.24.29 public:0.24.28 public:0.24.27 public:0.24.26 public:0.24.25 public:0.24.24 public:0.24.23 public:0.24.22 public:0.24.21 public:0.24.20 public:0.24.19 public:0.24.18 public:0.24.17 public:0.24.16 public:0.24.15 public:0.24.14 public:0.24.13 public:0.24.12 public:0.24.11 public:0.24.10 public:0.24.9 public:0.24.8 public:0.24.7 public:0.24.6 public:0.24.5 public:0.24.4 public:0.24.3 public:0.24.2 public:0.24.1 public:0.24.0 public:0.24.0.dev-rc8 public:0.24.0.dev-rc7 public:0.24.0.dev-rc6 public:0.24.0.dev-rc5 public:0.24.0.dev-rc4 public:0.24.0.dev-rc3 public:0.24.0.dev-rc2 public:0.23.23 public:0.23.22 public:0.23.21 public:0.23.20 public:0.23.19 public:0.23.18 public:0.23.17 public:0.23.16 public:0.23.15 public:0.23.14 public:0.23.13 public:0.23.12 public:0.23.11 public:0.23.10 public:0.23.9 public:0.23.8 public:0.23.7 public:0.23.6 public:0.23.5 public:0.23.4 public:0.23.3 public:0.23.2 public:0.23.1 public:0.23.0 public:0.22.19 public:0.22.18 public:0.22.17 public:0.22.16 public:0.22.15 public:0.22.14 public:0.22.13 public:0.22.12 public:0.22.11 public:0.22.10 public:0.22.9 public:0.22.8 public:0.22.7 public:0.22.6 public:0.22.5 public:0.22.4 public:0.22.3 public:0.22.2 public:0.22.1 public:0.22.0 public:0.21.5 public:0.21.4 public:0.21.3 public:0.21.2 public:0.21.1 public:0.21.0 public:0.20.7 public:0.20.6 public:0.20.5 public:0.20.4 public:0.20.3 public:0.20.2 public:0.20.1 public:0.20.0 public:0.19.23 public:0.19.22 public:0.19.21 public:0.19.20 public:0.19.19 public:0.19.18 public:0.19.17 public:0.19.16 public:0.19.15 public:0.19.14 public:0.19.13 public:0.19.12 public:0.19.11 public:0.19.10 public:0.19.9 public:0.19.8 public:0.19.7 public:0.19.6 public:0.19.5 public:0.19.4 public:0.19.3 public:0.19.2 public:0.19.1 public:0.19.0 public:0.18.6 public:0.18.5 public:0.18.4 public:0.18.3 public:0.18.2 public:0.18.1 public:0.18.0 public:0.17.34 public:0.17.33 public:0.17.32 public:0.17.31 public:0.17.30 public:0.17.29 public:0.17.28 public:0.17.27 public:0.17.26 public:0.17.25 public:0.17.24 public:0.17.23 public:0.17.22 public:0.17.21 public:0.17.20 public:0.17.19 public:0.17.18 public:0.17.17 public:0.17.16 public:0.17.15 public:0.17.14 public:0.17.13 public:0.17.12 public:0.17.11 public:0.17.10 public:0.17.9 public:0.17.8 public:0.17.7 public:0.17.6 public:0.17.5 public:0.17.4 public:0.17.3 public:0.17.2 public:0.17.1 public:0.17.0 public:0.16.8 public:0.16.7 public:0.16.6 public:0.16.5 public:0.16.4 public:0.16.3 public:0.16.2 public:0.16.1 public:0.16.0 public:0.15.10 public:0.15.9 public:0.15.8 public:0.15.7 public:0.15.6 public:0.15.5 public:0.15.4 public:0.15.3 public:0.15.2 public:0.15.1 public:0.15.0 public:0.14.8 public:0.14.7 public:0.14.6 public:0.14.5 public:0.14.4 public:0.14.3 public:0.14.2 public:0.14.1 public:0.14.0 public:0.13.4 public:0.13.3 public:0.13.2 public:0.13.1 public:0.13.0 public:0.12.5 public:0.12.4 public:0.12.3 public:0.12.2 public:0.12.1 public:0.12.0 public:0.11.10 public:0.11.9 public:0.11.8 public:0.11.7 public:0.11.6 public:0.11.5 public:0.11.4 public:0.11.3 public:0.11.2 public:0.11.1 public:0.11.0 public:0.10.1 public:0.10.0 public:0.9.0 public:0.8.8 public:0.8.7 public:0.8.6 public:0.8.5 public:0.8.3 public:0.8.2 public:0.8.1 public:0.8.0 public:0.7.2 public:0.7.1 public:0.7.0 public:0.6.1 public:0.6.0 public:0.5.0 public:0.4.1 public:0.4.0 public:0.3.1 public:0.3.0 public:0.2.1 public:0.2.0 public:0.1.26 public:0.1.25 public:0.1.24 public:0.1.23 public:0.1.22 public:0.1.21 public:0.1.20 public:0.1.19 public:0.1.18 public:0.1.17 public:0.1.16 public:0.1.15 public:0.1.14 public:0.1.13 public:0.1.12 public:0.1.11 public:0.1.10 public:0.1.9 public:0.1.8 public:0.1.7 public:0.1.6 public:0.1.5 public:0.1.4 public:0.1.3 public:0.1.2 public:0.1.1 public:0.1.0 public:0.0.9 public:0.0.8 public:0.0.7 public:0.0.6

17 Commits
0.24.0.dev ... 0.24.0

Author SHA1 Message Date
vorotamoroz
a2b36ccf31 v0.24.0! 2024-10-28 11:18:29 +09:00
vorotamoroz
25e30fa09d Merge tag '0.24.0.dev-rc8' 2024-10-28 10:24:38 +09:00
vorotamoroz
8f5bc387b4 Update manifest-beta.json 2024-10-25 20:44:32 +09:00
vorotamoroz
5afe24c460 0.24.0.dev-rc8 2024-10-25 12:39:32 +01:00
vorotamoroz
658a09f1cc Update manifest-beta.json 2024-10-24 20:46:08 +09:00
vorotamoroz
a9020a3aea 0.24.0.dev-rc7 2024-10-24 12:41:50 +01:00
vorotamoroz
293c731437 Update manifest-beta.json 2024-10-22 18:11:18 +09:00
vorotamoroz
5b4ae37030 0.24.0.dev-rc6 2024-10-22 10:08:20 +01:00
vorotamoroz
9e8d126259 Update manifest-beta.json 2024-10-21 17:52:28 +09:00
vorotamoroz
6d244a6e34 0.24.0.dev-rc5 2024-10-21 09:47:09 +01:00
vorotamoroz
1f0ad4eb1e Update manifest-beta.json 2024-10-18 19:23:48 +09:00
vorotamoroz
5023d6da0b Merge pull request #474 from nyawox/per-file-sync-grammar 
fix: per-file customization sync description grammar
 2024-10-18 11:43:03 +09:00
vorotamoroz
49160c7d57 Merge branch 'main' into per-file-sync-grammar 2024-10-18 11:42:49 +09:00
vorotamoroz
4434224c29 Merge pull request #479 from fuhrysteve/patch-1 
add username and password to setup URI instructions
 2024-10-18 11:40:23 +09:00
vorotamoroz
cf3b9e5522 Add manifest beta 2024-10-17 10:26:19 +01:00
Stephen J. Fuhry
f778107727 add username and password to setup URI instructions 2024-08-07 14:43:38 -04:00
nyawox
12d825ea49 fix: per-file customization sync description grammar 2024-07-26 23:38:01 +09:00
13 changed files with 444 additions and 352 deletions
Expand all files Collapse all files

2
docs/setup_own_server.md
View File

@@ -106,6 +106,8 @@ Now `https://tiles-photograph-routine-groundwater.trycloudflare.com` is our serv
$ export hostname=https://tiles-photograph-routine-groundwater.trycloudflare.com #Point to your vault $ export hostname=https://tiles-photograph-routine-groundwater.trycloudflare.com #Point to your vault
$ export database=obsidiannotes #Please change as you like $ export database=obsidiannotes #Please change as you like
$ export passphrase=dfsapkdjaskdjasdas #Please change as you like $ export passphrase=dfsapkdjaskdjasdas #Please change as you like
$ export username=johndoe
$ export password=abc123
$ deno run -A https://raw.githubusercontent.com/vrtmrz/obsidian-livesync/main/utils/flyio/generate_setupuri.ts $ deno run -A https://raw.githubusercontent.com/vrtmrz/obsidian-livesync/main/utils/flyio/generate_setupuri.ts
obsidian://setuplivesync?settings=%5B%22tm2DpsOE74nJAryprZO2M93wF%2Fvg.......4b26ed33230729%22%5D obsidian://setuplivesync?settings=%5B%22tm2DpsOE74nJAryprZO2M93wF%2Fvg.......4b26ed33230729%22%5D

2
manifest-beta.json
View File

@@ -1,7 +1,7 @@
{ {
"id": "obsidian-livesync", "id": "obsidian-livesync",
"name": "Self-hosted LiveSync", "name": "Self-hosted LiveSync",
"version": "0.24.0.dev-rc4", "version": "0.24.0",
"minAppVersion": "0.9.12", "minAppVersion": "0.9.12",
"description": "Community implementation of self-hosted livesync. Reflect your vault changes to some other devices immediately. Please make sure to disable other synchronize solutions to avoid content corruption or duplication.", "description": "Community implementation of self-hosted livesync. Reflect your vault changes to some other devices immediately. Please make sure to disable other synchronize solutions to avoid content corruption or duplication.",
"author": "vorotamoroz", "author": "vorotamoroz",

2
manifest.json
View File

@@ -1,7 +1,7 @@
{ {
"id": "obsidian-livesync", "id": "obsidian-livesync",
"name": "Self-hosted LiveSync", "name": "Self-hosted LiveSync",
"version": "0.23.23", "version": "0.24.0",
"minAppVersion": "0.9.12", "minAppVersion": "0.9.12",
"description": "Community implementation of self-hosted livesync. Reflect your vault changes to some other devices immediately. Please make sure to disable other synchronize solutions to avoid content corruption or duplication.", "description": "Community implementation of self-hosted livesync. Reflect your vault changes to some other devices immediately. Please make sure to disable other synchronize solutions to avoid content corruption or duplication.",
"author": "vorotamoroz", "author": "vorotamoroz",

4
package-lock.json generated
View File

@@ -1,12 +1,12 @@
{ {
"name": "obsidian-livesync", "name": "obsidian-livesync",
"version": "0.24.0.dev-rc4", "version": "0.24.0",
"lockfileVersion": 2, "lockfileVersion": 2,
"requires": true, "requires": true,
"packages": { "packages": {
"": { "": {
"name": "obsidian-livesync", "name": "obsidian-livesync",
"version": "0.24.0.dev-rc4", "version": "0.24.0",
"license": "MIT", "license": "MIT",
"dependencies": { "dependencies": {
"@aws-sdk/client-s3": "^3.645.0", "@aws-sdk/client-s3": "^3.645.0",

2
package.json
View File

@@ -1,6 +1,6 @@
{ {
"name": "obsidian-livesync", "name": "obsidian-livesync",
"version": "0.24.0.dev-rc4", "version": "0.24.0",
"description": "Reflect your vault changes to some other devices immediately. Please make sure to disable other synchronize solutions to avoid content corruption or duplication.", "description": "Reflect your vault changes to some other devices immediately. Please make sure to disable other synchronize solutions to avoid content corruption or duplication.",
"main": "main.js", "main": "main.js",
"type": "module", "type": "module",

186
src/features/ConfigSync/CmdConfigSync.ts
View File

@@ -5,7 +5,6 @@ import type { EntryDoc, LoadedEntry, InternalFileEntry, FilePathWithPrefix, File
import { CANCELLED, LEAVE_TO_SUBSEQUENT, LOG_LEVEL_DEBUG, LOG_LEVEL_INFO, LOG_LEVEL_NOTICE, LOG_LEVEL_VERBOSE, MODE_SELECTIVE, MODE_SHINY } from "../../lib/src/common/types.ts"; import { CANCELLED, LEAVE_TO_SUBSEQUENT, LOG_LEVEL_DEBUG, LOG_LEVEL_INFO, LOG_LEVEL_NOTICE, LOG_LEVEL_VERBOSE, MODE_SELECTIVE, MODE_SHINY } from "../../lib/src/common/types.ts";
import { ICXHeader, PERIODIC_PLUGIN_SWEEP, } from "../../common/types.ts"; import { ICXHeader, PERIODIC_PLUGIN_SWEEP, } from "../../common/types.ts";
import { createBlob, createSavingEntryFromLoadedEntry, createTextBlob, delay, fireAndForget, getDocData, getDocDataAsArray, isDocContentSame, isLoadedEntry, isObjectDifferent } from "../../lib/src/common/utils.ts"; import { createBlob, createSavingEntryFromLoadedEntry, createTextBlob, delay, fireAndForget, getDocData, getDocDataAsArray, isDocContentSame, isLoadedEntry, isObjectDifferent } from "../../lib/src/common/utils.ts";
import { Logger } from "../../lib/src/common/logger.ts";
import { digestHash } from "../../lib/src/string_and_binary/hash.ts"; import { digestHash } from "../../lib/src/string_and_binary/hash.ts";
import { arrayBufferToBase64, decodeBinary, readString } from '../../lib/src/string_and_binary/convert.ts'; import { arrayBufferToBase64, decodeBinary, readString } from '../../lib/src/string_and_binary/convert.ts';
import { serialized, shareRunningResult } from "../../lib/src/concurrency/lock.ts"; import { serialized, shareRunningResult } from "../../lib/src/concurrency/lock.ts";
@@ -419,12 +418,12 @@ export class ConfigSync extends LiveSyncCommands implements IObsidianModule {
async $everyOnDatabaseInitialized(showNotice: boolean) { async $everyOnDatabaseInitialized(showNotice: boolean) {
if (!this._isThisModuleEnabled()) return true; if (!this._isThisModuleEnabled()) return true;
try { try {
Logger("Scanning customizations..."); this._log("Scanning customizations...");
await this.scanAllConfigFiles(showNotice); await this.scanAllConfigFiles(showNotice);
Logger("Scanning customizations : done"); this._log("Scanning customizations : done");
} catch (ex) { } catch (ex) {
Logger("Scanning customizations : failed"); this._log("Scanning customizations : failed");
Logger(ex, LOG_LEVEL_VERBOSE); this._log(ex, LOG_LEVEL_VERBOSE);
} }
return true; return true;
} }
@@ -477,7 +476,7 @@ export class ConfigSync extends LiveSyncCommands implements IObsidianModule {
xFiles.push(work); xFiles.push(work);
} }
if (missingHash) { if (missingHash) {
Logger(`Digest created for ${path} to improve checking`, LOG_LEVEL_VERBOSE); this._log(`Digest created for ${path} to improve checking`, LOG_LEVEL_VERBOSE);
wx.data = serialize(data); wx.data = serialize(data);
fireAndForget(() => this.localDatabase.putDBEntry(createSavingEntryFromLoadedEntry(wx))); fireAndForget(() => this.localDatabase.putDBEntry(createSavingEntryFromLoadedEntry(wx)));
} }
@@ -512,8 +511,8 @@ export class ConfigSync extends LiveSyncCommands implements IObsidianModule {
return []; return [];
} catch (ex) { } catch (ex) {
Logger(`Something happened at enumerating customization :${path}`, LOG_LEVEL_NOTICE); this._log(`Something happened at enumerating customization :${path}`, LOG_LEVEL_NOTICE);
Logger(ex, LOG_LEVEL_VERBOSE); this._log(ex, LOG_LEVEL_VERBOSE);
} }
return []; return [];
}, { suspended: false, batchSize: 1, concurrentLimit: 10, delay: 100, yieldThreshold: 10, maintainDelay: false, totalRemainingReactiveSource: pluginScanningCount }).startPipeline(); }, { suspended: false, batchSize: 1, concurrentLimit: 10, delay: 100, yieldThreshold: 10, maintainDelay: false, totalRemainingReactiveSource: pluginScanningCount }).startPipeline();
@@ -536,8 +535,8 @@ export class ConfigSync extends LiveSyncCommands implements IObsidianModule {
return []; return [];
} catch (ex) { } catch (ex) {
Logger(`Something happened at enumerating customization :${path}`, LOG_LEVEL_NOTICE); this._log(`Something happened at enumerating customization :${path}`, LOG_LEVEL_NOTICE);
Logger(ex, LOG_LEVEL_VERBOSE); this._log(ex, LOG_LEVEL_VERBOSE);
} }
return []; return [];
}, { suspended: false, batchSize: 1, concurrentLimit: 10, delay: 100, yieldThreshold: 10, maintainDelay: false, totalRemainingReactiveSource: pluginScanningCount }).startPipeline(); }, { suspended: false, batchSize: 1, concurrentLimit: 10, delay: 100, yieldThreshold: 10, maintainDelay: false, totalRemainingReactiveSource: pluginScanningCount }).startPipeline();
@@ -583,11 +582,11 @@ export class ConfigSync extends LiveSyncCommands implements IObsidianModule {
if (!loaded) { if (!loaded) {
const d = await this.localDatabase.getDBEntry(unifiedPathV2); const d = await this.localDatabase.getDBEntry(unifiedPathV2);
if (!d) { if (!d) {
Logger(`The file ${unifiedPathV2} is not found`, LOG_LEVEL_VERBOSE); this._log(`The file ${unifiedPathV2} is not found`, LOG_LEVEL_VERBOSE);
return false; return false;
} }
if (!isLoadedEntry(d)) { if (!isLoadedEntry(d)) {
Logger(`The file ${unifiedPathV2} is not a note`, LOG_LEVEL_VERBOSE); this._log(`The file ${unifiedPathV2} is not a note`, LOG_LEVEL_VERBOSE);
return false; return false;
} }
loaded = d; loaded = d;
@@ -613,8 +612,8 @@ export class ConfigSync extends LiveSyncCommands implements IObsidianModule {
this.pluginList.filter(e => e instanceof PluginDataExDisplayV2 && e.confKey == confKey).forEach(e => (e as PluginDataExDisplayV2).applyLoadedManifest()); this.pluginList.filter(e => e instanceof PluginDataExDisplayV2 && e.confKey == confKey).forEach(e => (e as PluginDataExDisplayV2).applyLoadedManifest());
pluginList.set(this.pluginList); pluginList.set(this.pluginList);
} catch (ex) { } catch (ex) {
Logger(`The file ${loaded.path} seems to manifest, but could not be decoded as JSON`, LOG_LEVEL_VERBOSE); this._log(`The file ${loaded.path} seems to manifest, but could not be decoded as JSON`, LOG_LEVEL_VERBOSE);
Logger(ex, LOG_LEVEL_VERBOSE); this._log(ex, LOG_LEVEL_VERBOSE);
} }
this.loadedManifest_mTime.set(confKey, file.mtime); this.loadedManifest_mTime.set(confKey, file.mtime);
} else { } else {
@@ -687,22 +686,22 @@ export class ConfigSync extends LiveSyncCommands implements IObsidianModule {
async migrateV1ToV2(showMessage: boolean, entry: AnyEntry): Promise<void> { async migrateV1ToV2(showMessage: boolean, entry: AnyEntry): Promise<void> {
const v1Path = entry.path; const v1Path = entry.path;
Logger(`Migrating ${entry.path} to V2`, showMessage ? LOG_LEVEL_NOTICE : LOG_LEVEL_INFO); this._log(`Migrating ${entry.path} to V2`, showMessage ? LOG_LEVEL_NOTICE : LOG_LEVEL_INFO);
if (entry.deleted) { if (entry.deleted) {
Logger(`The entry ${v1Path} is already deleted`, LOG_LEVEL_VERBOSE); this._log(`The entry ${v1Path} is already deleted`, LOG_LEVEL_VERBOSE);
return; return;
} }
if (!v1Path.endsWith(".md") && !v1Path.startsWith(ICXHeader)) { if (!v1Path.endsWith(".md") && !v1Path.startsWith(ICXHeader)) {
Logger(`The entry ${v1Path} is not a customisation sync binder`, LOG_LEVEL_VERBOSE); this._log(`The entry ${v1Path} is not a customisation sync binder`, LOG_LEVEL_VERBOSE);
return return
} }
if (v1Path.indexOf("%") !== -1) { if (v1Path.indexOf("%") !== -1) {
Logger(`The entry ${v1Path} is already migrated`, LOG_LEVEL_VERBOSE); this._log(`The entry ${v1Path} is already migrated`, LOG_LEVEL_VERBOSE);
return; return;
} }
const loadedEntry = await this.localDatabase.getDBEntry(v1Path); const loadedEntry = await this.localDatabase.getDBEntry(v1Path);
if (!loadedEntry) { if (!loadedEntry) {
Logger(`The entry ${v1Path} is not found`, LOG_LEVEL_VERBOSE); this._log(`The entry ${v1Path} is not found`, LOG_LEVEL_VERBOSE);
return; return;
} }
@@ -723,7 +722,7 @@ export class ConfigSync extends LiveSyncCommands implements IObsidianModule {
const relativeFilename = f.filename.split("/").slice(deletePrefixCount).join("/"); const relativeFilename = f.filename.split("/").slice(deletePrefixCount).join("/");
const v2Path = (prefixPath + relativeFilename) as FilePathWithPrefix; const v2Path = (prefixPath + relativeFilename) as FilePathWithPrefix;
// console.warn(`Migrating ${v1Path} / ${relativeFilename} to ${v2Path}`); // console.warn(`Migrating ${v1Path} / ${relativeFilename} to ${v2Path}`);
Logger(`Migrating ${v1Path} / ${relativeFilename} to ${v2Path}`, LOG_LEVEL_VERBOSE); this._log(`Migrating ${v1Path} / ${relativeFilename} to ${v2Path}`, LOG_LEVEL_VERBOSE);
const newId = await this.plugin.$$path2id(v2Path); const newId = await this.plugin.$$path2id(v2Path);
// const buf = // const buf =
@@ -742,12 +741,12 @@ export class ConfigSync extends LiveSyncCommands implements IObsidianModule {
} }
const r = await this.plugin.localDatabase.putDBEntry(saving); const r = await this.plugin.localDatabase.putDBEntry(saving);
if (r && r.ok) { if (r && r.ok) {
Logger(`Migrated ${v1Path} / ${f.filename} to ${v2Path}`, LOG_LEVEL_INFO); this._log(`Migrated ${v1Path} / ${f.filename} to ${v2Path}`, LOG_LEVEL_INFO);
const delR = await this.deleteConfigOnDatabase(v1Path); const delR = await this.deleteConfigOnDatabase(v1Path);
if (delR) { if (delR) {
Logger(`Deleted ${v1Path} successfully`, LOG_LEVEL_INFO); this._log(`Deleted ${v1Path} successfully`, LOG_LEVEL_INFO);
} else { } else {
Logger(`Failed to delete ${v1Path}`, LOG_LEVEL_NOTICE); this._log(`Failed to delete ${v1Path}`, LOG_LEVEL_NOTICE);
} }
} }
} }
@@ -802,9 +801,9 @@ export class ConfigSync extends LiveSyncCommands implements IObsidianModule {
} }
const fileA = await loadFile(dataA); const fileA = await loadFile(dataA);
const fileB = await loadFile(dataB); const fileB = await loadFile(dataB);
Logger(`Comparing: ${dataA.documentPath} <-> ${dataB.documentPath}`, LOG_LEVEL_VERBOSE); this._log(`Comparing: ${dataA.documentPath} <-> ${dataB.documentPath}`, LOG_LEVEL_VERBOSE);
if (!fileA || !fileB) { if (!fileA || !fileB) {
Logger(`Could not load ${dataA.name} for comparison: ${!fileA ? dataA.term : ""}${!fileB ? dataB.term : ""}`, LOG_LEVEL_NOTICE); this._log(`Could not load ${dataA.name} for comparison: ${!fileA ? dataA.term : ""}${!fileB ? dataB.term : ""}`, LOG_LEVEL_NOTICE);
return false; return false;
} }
let path = stripAllPrefixes(fileA.path.split("/").slice(-1).join("/") as FilePath); // TODO:adjust let path = stripAllPrefixes(fileA.path.split("/").slice(-1).join("/") as FilePath); // TODO:adjust
@@ -813,15 +812,15 @@ export class ConfigSync extends LiveSyncCommands implements IObsidianModule {
} }
if (fileA.path.endsWith(".json")) { if (fileA.path.endsWith(".json")) {
return serialized("config:merge-data", () => new Promise<boolean>((res) => { return serialized("config:merge-data", () => new Promise<boolean>((res) => {
Logger("Opening data-merging dialog", LOG_LEVEL_VERBOSE); this._log("Opening data-merging dialog", LOG_LEVEL_VERBOSE);
// const docs = [docA, docB]; // const docs = [docA, docB];
const modal = new JsonResolveModal(this.app, path, [fileA, fileB], async (keep, result) => { const modal = new JsonResolveModal(this.app, path, [fileA, fileB], async (keep, result) => {
if (result == null) return res(false); if (result == null) return res(false);
try { try {
res(await this.applyData(dataA, result)); res(await this.applyData(dataA, result));
} catch (ex) { } catch (ex) {
Logger("Could not apply merged file"); this._log("Could not apply merged file");
Logger(ex, LOG_LEVEL_VERBOSE); this._log(ex, LOG_LEVEL_VERBOSE);
res(false); res(false);
} }
}, "Local", `${dataB.term}`, "B", true, true, "Difference between local and remote"); }, "Local", `${dataB.term}`, "B", true, true, "Difference between local and remote");
@@ -847,7 +846,7 @@ export class ConfigSync extends LiveSyncCommands implements IObsidianModule {
right: { rev: "B", ...fileB, data: docBData }, right: { rev: "B", ...fileB, data: docBData },
diff: diff diff: diff
} }
console.dir(diffResult); // console.dir(diffResult);
const d = new ConflictResolveModal(this.app, path, diffResult, true, dataB.term); const d = new ConflictResolveModal(this.app, path, diffResult, true, dataB.term);
d.open(); d.open();
const ret = await d.waitForResult(); const ret = await d.waitForResult();
@@ -866,7 +865,7 @@ export class ConfigSync extends LiveSyncCommands implements IObsidianModule {
if (content) { if (content) {
// const dt = createBlob(content); // const dt = createBlob(content);
const filename = data.files[0].filename; const filename = data.files[0].filename;
Logger(`Applying ${filename} of ${data.displayName || data.name}..`); this._log(`Applying ${filename} of ${data.displayName || data.name}..`);
const path = `${baseDir}/${filename}` as FilePath; const path = `${baseDir}/${filename}` as FilePath;
await this.plugin.storageAccess.ensureDir(path); await this.plugin.storageAccess.ensureDir(path);
// If the content has applied, modified time will be updated to the current time. // If the content has applied, modified time will be updated to the current time.
@@ -879,7 +878,7 @@ export class ConfigSync extends LiveSyncCommands implements IObsidianModule {
// If files have applied, modified time will be updated to the current time. // If files have applied, modified time will be updated to the current time.
const stat = { mtime: f.mtime, ctime: f.ctime }; const stat = { mtime: f.mtime, ctime: f.ctime };
const path = `${baseDir}/${f.filename}` as FilePath; const path = `${baseDir}/${f.filename}` as FilePath;
Logger(`Applying ${f.filename} of ${data.displayName || data.name}..`); this._log(`Applying ${f.filename} of ${data.displayName || data.name}..`);
// const contentEach = createBlob(f.data); // const contentEach = createBlob(f.data);
await this.plugin.storageAccess.ensureDir(path); await this.plugin.storageAccess.ensureDir(path);
@@ -888,13 +887,13 @@ export class ConfigSync extends LiveSyncCommands implements IObsidianModule {
try { try {
oldData = await this.plugin.storageAccess.readHiddenFileBinary(path); oldData = await this.plugin.storageAccess.readHiddenFileBinary(path);
} catch (ex) { } catch (ex) {
Logger(`Could not read the file ${f.filename}`, LOG_LEVEL_VERBOSE); this._log(`Could not read the file ${f.filename}`, LOG_LEVEL_VERBOSE);
Logger(ex, LOG_LEVEL_VERBOSE); this._log(ex, LOG_LEVEL_VERBOSE);
oldData = new ArrayBuffer(0); oldData = new ArrayBuffer(0);
} }
const content = base64ToArrayBuffer(f.data); const content = base64ToArrayBuffer(f.data);
if (await isDocContentSame(oldData, content)) { if (await isDocContentSame(oldData, content)) {
Logger(`The file ${f.filename} is already up-to-date`, LOG_LEVEL_VERBOSE); this._log(`The file ${f.filename} is already up-to-date`, LOG_LEVEL_VERBOSE);
continue; continue;
} }
await this.plugin.storageAccess.writeHiddenFileAuto(path, content, stat); await this.plugin.storageAccess.writeHiddenFileAuto(path, content, stat);
@@ -903,30 +902,30 @@ export class ConfigSync extends LiveSyncCommands implements IObsidianModule {
try { try {
oldData = await this.plugin.storageAccess.readHiddenFileText(path); oldData = await this.plugin.storageAccess.readHiddenFileText(path);
} catch (ex) { } catch (ex) {
Logger(`Could not read the file ${f.filename}`, LOG_LEVEL_VERBOSE); this._log(`Could not read the file ${f.filename}`, LOG_LEVEL_VERBOSE);
Logger(ex, LOG_LEVEL_VERBOSE); this._log(ex, LOG_LEVEL_VERBOSE);
oldData = ""; oldData = "";
} }
const content = getDocData(f.data); const content = getDocData(f.data);
if (await isDocContentSame(oldData, content)) { if (await isDocContentSame(oldData, content)) {
Logger(`The file ${f.filename} is already up-to-date`, LOG_LEVEL_VERBOSE); this._log(`The file ${f.filename} is already up-to-date`, LOG_LEVEL_VERBOSE);
continue; continue;
} }
await this.plugin.storageAccess.writeHiddenFileAuto(path, content, stat); await this.plugin.storageAccess.writeHiddenFileAuto(path, content, stat);
} }
Logger(`Applied ${f.filename} of ${data.displayName || data.name}..`); this._log(`Applied ${f.filename} of ${data.displayName || data.name}..`);
await this.storeCustomisationFileV2(path, this.plugin.deviceAndVaultName); await this.storeCustomisationFileV2(path, this.plugin.deviceAndVaultName);
} }
} }
} catch (ex) { } catch (ex) {
Logger(`Applying ${data.displayName || data.name}.. Failed`, LOG_LEVEL_NOTICE); this._log(`Applying ${data.displayName || data.name}.. Failed`, LOG_LEVEL_NOTICE);
Logger(ex, LOG_LEVEL_VERBOSE); this._log(ex, LOG_LEVEL_VERBOSE);
return false; return false;
} }
return true; return true;
} }
async applyData(data: IPluginDataExDisplay, content?: string): Promise<boolean> { async applyData(data: IPluginDataExDisplay, content?: string): Promise<boolean> {
Logger(`Applying ${data.displayName || data.name this._log(`Applying ${data.displayName || data.name
}..`); }..`);
if (data instanceof PluginDataExDisplayV2) { if (data instanceof PluginDataExDisplayV2) {
@@ -941,7 +940,7 @@ export class ConfigSync extends LiveSyncCommands implements IObsidianModule {
} }
const loadedData = deserialize(getDocDataAsArray(dx.data), {}) as PluginDataEx; const loadedData = deserialize(getDocDataAsArray(dx.data), {}) as PluginDataEx;
for (const f of loadedData.files) { for (const f of loadedData.files) {
Logger(`Applying ${f.filename} of ${data.displayName || data.name}..`); this._log(`Applying ${f.filename} of ${data.displayName || data.name}..`);
try { try {
// console.dir(f); // console.dir(f);
const path = `${baseDir}/${f.filename}`; const path = `${baseDir}/${f.filename}`;
@@ -952,11 +951,11 @@ export class ConfigSync extends LiveSyncCommands implements IObsidianModule {
} else { } else {
await this.plugin.storageAccess.writeHiddenFileAuto(path, content); await this.plugin.storageAccess.writeHiddenFileAuto(path, content);
} }
Logger(`Applying ${f.filename} of ${data.displayName || data.name}.. Done`); this._log(`Applying ${f.filename} of ${data.displayName || data.name}.. Done`);
} catch (ex) { } catch (ex) {
Logger(`Applying ${f.filename} of ${data.displayName || data.name}.. Failed`); this._log(`Applying ${f.filename} of ${data.displayName || data.name}.. Failed`);
Logger(ex, LOG_LEVEL_VERBOSE); this._log(ex, LOG_LEVEL_VERBOSE);
} }
} }
@@ -964,7 +963,7 @@ export class ConfigSync extends LiveSyncCommands implements IObsidianModule {
await this.storeCustomizationFiles(uPath); await this.storeCustomizationFiles(uPath);
await this.updatePluginList(true, uPath); await this.updatePluginList(true, uPath);
await delay(100); await delay(100);
Logger(`Config ${data.displayName || data.name} has been applied`, LOG_LEVEL_NOTICE); this._log(`Config ${data.displayName || data.name} has been applied`, LOG_LEVEL_NOTICE);
if (data.category == "PLUGIN_DATA" || data.category == "PLUGIN_MAIN") { if (data.category == "PLUGIN_DATA" || data.category == "PLUGIN_MAIN") {
//@ts-ignore //@ts-ignore
const manifests = Object.values(this.app.plugins.manifests) as any as PluginManifest[]; const manifests = Object.values(this.app.plugins.manifests) as any as PluginManifest[];
@@ -972,20 +971,20 @@ export class ConfigSync extends LiveSyncCommands implements IObsidianModule {
const enabledPlugins = this.app.plugins.enabledPlugins as Set<string>; const enabledPlugins = this.app.plugins.enabledPlugins as Set<string>;
const pluginManifest = manifests.find((manifest) => enabledPlugins.has(manifest.id) && manifest.dir == `${baseDir}/plugins/${data.name}`); const pluginManifest = manifests.find((manifest) => enabledPlugins.has(manifest.id) && manifest.dir == `${baseDir}/plugins/${data.name}`);
if (pluginManifest) { if (pluginManifest) {
Logger(`Unloading plugin: ${pluginManifest.name}`, LOG_LEVEL_NOTICE, "plugin-reload-" + pluginManifest.id); this._log(`Unloading plugin: ${pluginManifest.name}`, LOG_LEVEL_NOTICE, "plugin-reload-" + pluginManifest.id);
// @ts-ignore // @ts-ignore
await this.app.plugins.unloadPlugin(pluginManifest.id); await this.app.plugins.unloadPlugin(pluginManifest.id);
// @ts-ignore // @ts-ignore
await this.app.plugins.loadPlugin(pluginManifest.id); await this.app.plugins.loadPlugin(pluginManifest.id);
Logger(`Plugin reloaded: ${pluginManifest.name}`, LOG_LEVEL_NOTICE, "plugin-reload-" + pluginManifest.id); this._log(`Plugin reloaded: ${pluginManifest.name}`, LOG_LEVEL_NOTICE, "plugin-reload-" + pluginManifest.id);
} }
} else if (data.category == "CONFIG") { } else if (data.category == "CONFIG") {
this.plugin.$$askReload(); this.plugin.$$askReload();
} }
return true; return true;
} catch (ex) { } catch (ex) {
Logger(`Applying ${data.displayName || data.name}.. Failed`); this._log(`Applying ${data.displayName || data.name}.. Failed`);
Logger(ex, LOG_LEVEL_VERBOSE); this._log(ex, LOG_LEVEL_VERBOSE);
return false; return false;
} }
} }
@@ -1007,12 +1006,12 @@ export class ConfigSync extends LiveSyncCommands implements IObsidianModule {
await Promise.allSettled(p); await Promise.allSettled(p);
// await this.deleteConfigOnDatabase(data.documentPath); // await this.deleteConfigOnDatabase(data.documentPath);
// await this.updatePluginList(false, data.documentPath); // await this.updatePluginList(false, data.documentPath);
Logger(`Deleted: ${data.category}/${data.name} of ${data.category} (${delList.length} items)`, LOG_LEVEL_NOTICE); this._log(`Deleted: ${data.category}/${data.name} of ${data.category} (${delList.length} items)`, LOG_LEVEL_NOTICE);
} }
return true; return true;
} catch (ex) { } catch (ex) {
Logger(`Failed to delete: ${data.documentPath}`, LOG_LEVEL_NOTICE); this._log(`Failed to delete: ${data.documentPath}`, LOG_LEVEL_NOTICE);
Logger(ex, LOG_LEVEL_VERBOSE); this._log(ex, LOG_LEVEL_VERBOSE);
return false; return false;
} }
@@ -1096,13 +1095,13 @@ export class ConfigSync extends LiveSyncCommands implements IObsidianModule {
displayName = `${json.name}`; displayName = `${json.name}`;
} }
} catch (ex) { } catch (ex) {
Logger(`Configuration sync data: ${path} looks like manifest, but could not read the version`, LOG_LEVEL_INFO); this._log(`Configuration sync data: ${path} looks like manifest, but could not read the version`, LOG_LEVEL_INFO);
Logger(ex, LOG_LEVEL_VERBOSE); this._log(ex, LOG_LEVEL_VERBOSE);
} }
} }
} catch (ex) { } catch (ex) {
Logger(`The file ${path} could not be encoded`); this._log(`The file ${path} could not be encoded`);
Logger(ex, LOG_LEVEL_VERBOSE); this._log(ex, LOG_LEVEL_VERBOSE);
return false; return false;
} }
const mtime = stat.mtime; const mtime = stat.mtime;
@@ -1150,7 +1149,7 @@ export class ConfigSync extends LiveSyncCommands implements IObsidianModule {
}; };
} else { } else {
if (isMarkedAsSameChanges(prefixedFileName, [old.mtime, mtime + 1]) == EVEN) { if (isMarkedAsSameChanges(prefixedFileName, [old.mtime, mtime + 1]) == EVEN) {
Logger(`STORAGE --> DB:${prefixedFileName}: (config) Skipped (Already checked the same)`, LOG_LEVEL_DEBUG); this._log(`STORAGE --> DB:${prefixedFileName}: (config) Skipped (Already checked the same)`, LOG_LEVEL_DEBUG);
return; return;
} }
const docXDoc = await this.localDatabase.getDBEntryFromMeta(old, {}, false, false); const docXDoc = await this.localDatabase.getDBEntryFromMeta(old, {}, false, false);
@@ -1162,7 +1161,7 @@ export class ConfigSync extends LiveSyncCommands implements IObsidianModule {
const oldContent = dataSrc.substring(dataStart + DUMMY_END.length); const oldContent = dataSrc.substring(dataStart + DUMMY_END.length);
const oldContentArray = base64ToArrayBuffer(oldContent); const oldContentArray = base64ToArrayBuffer(oldContent);
if (await isDocContentSame(oldContentArray, content)) { if (await isDocContentSame(oldContentArray, content)) {
Logger(`STORAGE --> DB:${prefixedFileName}: (config) Skipped (the same content)`, LOG_LEVEL_VERBOSE); this._log(`STORAGE --> DB:${prefixedFileName}: (config) Skipped (the same content)`, LOG_LEVEL_VERBOSE);
markChangesAreSame(prefixedFileName, old.mtime, mtime + 1); markChangesAreSame(prefixedFileName, old.mtime, mtime + 1);
return true; return true;
} }
@@ -1179,12 +1178,12 @@ export class ConfigSync extends LiveSyncCommands implements IObsidianModule {
}; };
} }
const ret = await this.localDatabase.putDBEntry(saveData); const ret = await this.localDatabase.putDBEntry(saveData);
Logger(`STORAGE --> DB:${prefixedFileName}: (config) Done`); this._log(`STORAGE --> DB:${prefixedFileName}: (config) Done`);
fireAndForget(() => this.updatePluginListV2(false, this.filenameWithUnifiedKey(path))); fireAndForget(() => this.updatePluginListV2(false, this.filenameWithUnifiedKey(path)));
return ret; return ret;
} catch (ex) { } catch (ex) {
Logger(`STORAGE --> DB:${prefixedFileName}: (config) Failed`); this._log(`STORAGE --> DB:${prefixedFileName}: (config) Failed`);
Logger(ex, LOG_LEVEL_VERBOSE); this._log(ex, LOG_LEVEL_VERBOSE);
return false; return false;
} }
}) })
@@ -1192,7 +1191,7 @@ export class ConfigSync extends LiveSyncCommands implements IObsidianModule {
async storeCustomizationFiles(path: FilePath, termOverRide?: string) { async storeCustomizationFiles(path: FilePath, termOverRide?: string) {
const term = termOverRide || this.plugin.deviceAndVaultName; const term = termOverRide || this.plugin.deviceAndVaultName;
if (term == "") { if (term == "") {
Logger("We have to configure the device name", LOG_LEVEL_NOTICE); this._log("We have to configure the device name", LOG_LEVEL_NOTICE);
return; return;
} }
if (this.useV2) { if (this.useV2) {
@@ -1234,7 +1233,7 @@ export class ConfigSync extends LiveSyncCommands implements IObsidianModule {
for (const target of fileTargets) { for (const target of fileTargets) {
const data = await this.makeEntryFromFile(target); const data = await this.makeEntryFromFile(target);
if (data == false) { if (data == false) {
Logger(`Config: skipped (Possibly is not exist): ${target} `, LOG_LEVEL_VERBOSE); this._log(`Config: skipped (Possibly is not exist): ${target} `, LOG_LEVEL_VERBOSE);
continue; continue;
} }
if (data.version) { if (data.version) {
@@ -1249,9 +1248,9 @@ export class ConfigSync extends LiveSyncCommands implements IObsidianModule {
} }
dt.mtime = mtime; dt.mtime = mtime;
// Logger(`Configuration saving: ${prefixedFileName}`); // this._log(`Configuration saving: ${prefixedFileName}`);
if (dt.files.length == 0) { if (dt.files.length == 0) {
Logger(`Nothing left: deleting.. ${path}`); this._log(`Nothing left: deleting.. ${path}`);
await this.deleteConfigOnDatabase(prefixedFileName); await this.deleteConfigOnDatabase(prefixedFileName);
await this.updatePluginList(false, prefixedFileName); await this.updatePluginList(false, prefixedFileName);
return return
@@ -1277,7 +1276,7 @@ export class ConfigSync extends LiveSyncCommands implements IObsidianModule {
}; };
} else { } else {
if (old.mtime == mtime) { if (old.mtime == mtime) {
// Logger(`STORAGE --> DB:${prefixedFileName}: (config) Skipped (Same time)`, LOG_LEVEL_VERBOSE); // this._log(`STORAGE --> DB:${prefixedFileName}: (config) Skipped (Same time)`, LOG_LEVEL_VERBOSE);
return true; return true;
} }
const oldC = await this.localDatabase.getDBEntryFromMeta(old, {}, false, false); const oldC = await this.localDatabase.getDBEntryFromMeta(old, {}, false, false);
@@ -1289,7 +1288,7 @@ export class ConfigSync extends LiveSyncCommands implements IObsidianModule {
})) }))
const isSame = (await Promise.all(diffs)).every(e => e == true); const isSame = (await Promise.all(diffs)).every(e => e == true);
if (isSame) { if (isSame) {
Logger(`STORAGE --> DB:${prefixedFileName}: (config) Skipped (Same content)`, LOG_LEVEL_VERBOSE); this._log(`STORAGE --> DB:${prefixedFileName}: (config) Skipped (Same content)`, LOG_LEVEL_VERBOSE);
return true; return true;
} }
} }
@@ -1308,25 +1307,24 @@ export class ConfigSync extends LiveSyncCommands implements IObsidianModule {
} }
const ret = await this.localDatabase.putDBEntry(saveData); const ret = await this.localDatabase.putDBEntry(saveData);
await this.updatePluginList(false, saveData.path); await this.updatePluginList(false, saveData.path);
Logger(`STORAGE --> DB:${prefixedFileName}: (config) Done`); this._log(`STORAGE --> DB:${prefixedFileName}: (config) Done`);
return ret; return ret;
} catch (ex) { } catch (ex) {
Logger(`STORAGE --> DB:${prefixedFileName}: (config) Failed`); this._log(`STORAGE --> DB:${prefixedFileName}: (config) Failed`);
Logger(ex, LOG_LEVEL_VERBOSE); this._log(ex, LOG_LEVEL_VERBOSE);
return false; return false;
} }
}) })
} }
async $anyProcessOptionalFileEvent(path: FilePath): Promise<boolean | undefined> { async $anyProcessOptionalFileEvent(path: FilePath): Promise<boolean | undefined> {
return await this.watchVaultRawEventsAsync(path); return await this.watchVaultRawEventsAsync(path);
} }
async watchVaultRawEventsAsync(path: FilePath) { async watchVaultRawEventsAsync(path: FilePath) {
if (!this._isMainReady) return true; if (!this._isMainReady) return false;
if (!this._isMainSuspended()) return true; if (this._isMainSuspended()) return false;
if (!this._isThisModuleEnabled()) return true; if (!this._isThisModuleEnabled()) return false;
if (!this.isTargetPath(path)) return false; // if (!this.isTargetPath(path)) return false;
const stat = await this.plugin.storageAccess.statHidden(path); const stat = await this.plugin.storageAccess.statHidden(path);
// Make sure that target is a file. // Make sure that target is a file.
if (stat && stat.type != "file") if (stat && stat.type != "file")
@@ -1337,10 +1335,11 @@ export class ConfigSync extends LiveSyncCommands implements IObsidianModule {
e.mode != MODE_SELECTIVE && e.mode != MODE_SHINY e.mode != MODE_SELECTIVE && e.mode != MODE_SHINY
).map(e => e.files).flat().map(e => `${configDir}/${e}`.toLowerCase()); ).map(e => e.files).flat().map(e => `${configDir}/${e}`.toLowerCase());
if (synchronisedInConfigSync.some(e => e.startsWith(path.toLowerCase()))) { if (synchronisedInConfigSync.some(e => e.startsWith(path.toLowerCase()))) {
Logger(`Customization file skipped: ${path}`, LOG_LEVEL_VERBOSE); this._log(`Customization file skipped: ${path}`, LOG_LEVEL_VERBOSE);
// This file could be handled by the other module. // This file could be handled by the other module.
return false; return false;
} }
// this._log(`Customization file detected: ${path}`, LOG_LEVEL_VERBOSE);
const storageMTime = ~~((stat && stat.mtime || 0) / 1000); const storageMTime = ~~((stat && stat.mtime || 0) / 1000);
const key = `${path}-${storageMTime}`; const key = `${path}-${storageMTime}`;
if (this.recentProcessedInternalFiles.contains(key)) { if (this.recentProcessedInternalFiles.contains(key)) {
@@ -1359,16 +1358,13 @@ export class ConfigSync extends LiveSyncCommands implements IObsidianModule {
return true; return true;
} }
async scanAllConfigFiles(showMessage: boolean) { async scanAllConfigFiles(showMessage: boolean) {
await shareRunningResult("scanAllConfigFiles", async () => { await shareRunningResult("scanAllConfigFiles", async () => {
const logLevel = showMessage ? LOG_LEVEL_NOTICE : LOG_LEVEL_INFO; const logLevel = showMessage ? LOG_LEVEL_NOTICE : LOG_LEVEL_INFO;
Logger("Scanning customizing files.", logLevel, "scan-all-config"); this._log("Scanning customizing files.", logLevel, "scan-all-config");
const term = this.plugin.deviceAndVaultName; const term = this.plugin.deviceAndVaultName;
if (term == "") { if (term == "") {
Logger("We have to configure the device name", LOG_LEVEL_NOTICE); this._log("We have to configure the device name", LOG_LEVEL_NOTICE);
return; return;
} }
const filesAll = await this.scanInternalFiles(); const filesAll = await this.scanInternalFiles();
@@ -1396,8 +1392,8 @@ export class ConfigSync extends LiveSyncCommands implements IObsidianModule {
await this.deleteConfigOnDatabase(unifiedFilenameWithKey); await this.deleteConfigOnDatabase(unifiedFilenameWithKey);
} }
} catch (ex) { } catch (ex) {
Logger(`scanAllConfigFiles - Error: ${item._id}`, LOG_LEVEL_VERBOSE); this._log(`scanAllConfigFiles - Error: ${item._id}`, LOG_LEVEL_VERBOSE);
Logger(ex, LOG_LEVEL_VERBOSE); this._log(ex, LOG_LEVEL_VERBOSE);
} finally { } finally {
releaser(); releaser();
} }
@@ -1412,8 +1408,8 @@ export class ConfigSync extends LiveSyncCommands implements IObsidianModule {
try { try {
await this.storeCustomisationFileV2(filePath, term); await this.storeCustomisationFileV2(filePath, term);
} catch (ex) { } catch (ex) {
Logger(`scanAllConfigFiles - Error: ${filePath}`, LOG_LEVEL_VERBOSE); this._log(`scanAllConfigFiles - Error: ${filePath}`, LOG_LEVEL_VERBOSE);
Logger(ex, LOG_LEVEL_VERBOSE); this._log(ex, LOG_LEVEL_VERBOSE);
} }
finally { finally {
releaser(); releaser();
@@ -1430,7 +1426,7 @@ export class ConfigSync extends LiveSyncCommands implements IObsidianModule {
for (const vp of virtualPathsOfLocalFiles) { for (const vp of virtualPathsOfLocalFiles) {
const p = files.find(e => e.key == vp)?.file; const p = files.find(e => e.key == vp)?.file;
if (!p) { if (!p) {
Logger(`scanAllConfigFiles - File not found: ${vp}`, LOG_LEVEL_VERBOSE); this._log(`scanAllConfigFiles - File not found: ${vp}`, LOG_LEVEL_VERBOSE);
continue; continue;
} }
await this.storeCustomizationFiles(p); await this.storeCustomizationFiles(p);
@@ -1453,11 +1449,11 @@ export class ConfigSync extends LiveSyncCommands implements IObsidianModule {
const old = await this.localDatabase.getDBEntryMeta(prefixedFileName, undefined, false) as InternalFileEntry | false; const old = await this.localDatabase.getDBEntryMeta(prefixedFileName, undefined, false) as InternalFileEntry | false;
let saveData: InternalFileEntry; let saveData: InternalFileEntry;
if (old === false) { if (old === false) {
Logger(`STORAGE -x> DB:${prefixedFileName}: (config) already deleted (Not found on database)`); this._log(`STORAGE -x> DB:${prefixedFileName}: (config) already deleted (Not found on database)`);
return true; return true;
} else { } else {
if (old.deleted) { if (old.deleted) {
Logger(`STORAGE -x> DB:${prefixedFileName}: (config) already deleted`); this._log(`STORAGE -x> DB:${prefixedFileName}: (config) already deleted`);
return true; return true;
} }
saveData = saveData =
@@ -1472,11 +1468,11 @@ export class ConfigSync extends LiveSyncCommands implements IObsidianModule {
} }
await this.localDatabase.putRaw(saveData); await this.localDatabase.putRaw(saveData);
await this.updatePluginList(false, prefixedFileName); await this.updatePluginList(false, prefixedFileName);
Logger(`STORAGE -x> DB:${prefixedFileName}: (config) Done`); this._log(`STORAGE -x> DB:${prefixedFileName}: (config) Done`);
return true; return true;
} catch (ex) { } catch (ex) {
Logger(`STORAGE -x> DB:${prefixedFileName}: (config) Failed`); this._log(`STORAGE -x> DB:${prefixedFileName}: (config) Failed`);
Logger(ex, LOG_LEVEL_VERBOSE); this._log(ex, LOG_LEVEL_VERBOSE);
return false; return false;
} }
}); });
@@ -1530,7 +1526,7 @@ export class ConfigSync extends LiveSyncCommands implements IObsidianModule {
$allSuspendExtraSync(): Promise<boolean> { $allSuspendExtraSync(): Promise<boolean> {
if (this.plugin.settings.usePluginSync || this.plugin.settings.autoSweepPlugins) { if (this.plugin.settings.usePluginSync || this.plugin.settings.autoSweepPlugins) {
Logger("Customisation sync have been temporarily disabled. Please enable them after the fetching, if you need them.", LOG_LEVEL_NOTICE) this._log("Customisation sync have been temporarily disabled. Please enable them after the fetching, if you need them.", LOG_LEVEL_NOTICE)
this.plugin.settings.usePluginSync = false; this.plugin.settings.usePluginSync = false;
this.plugin.settings.autoSweepPlugins = false; this.plugin.settings.autoSweepPlugins = false;
} }
@@ -1590,8 +1586,8 @@ export class ConfigSync extends LiveSyncCommands implements IObsidianModule {
try { try {
w = await this.app.vault.adapter.list(path); w = await this.app.vault.adapter.list(path);
} catch (ex) { } catch (ex) {
Logger(`Could not traverse(ConfigSync):${path}`, LOG_LEVEL_INFO); this._log(`Could not traverse(ConfigSync):${path}`, LOG_LEVEL_INFO);
Logger(ex, LOG_LEVEL_VERBOSE); this._log(ex, LOG_LEVEL_VERBOSE);
return []; return [];
} }
let files = [ let files = [

5
src/features/HiddenFileCommon/JsonResolveModal.ts
View File

@@ -32,9 +32,12 @@ export class JsonResolveModal extends Modal {
this.hideLocal = hideLocal ?? false; this.hideLocal = hideLocal ?? false;
void waitForSignal(`cancel-internal-conflict:${filename}`).then(() => this.close()); void waitForSignal(`cancel-internal-conflict:${filename}`).then(() => this.close());
} }
async UICallback(keepRev?: string, mergedStr?: string) { async UICallback(keepRev?: string, mergedStr?: string) {
if (this.callback) {
await this.callback(keepRev, mergedStr);
}
this.close(); this.close();
await this.callback?.(keepRev, mergedStr);
this.callback = undefined; this.callback = undefined;
} }

390
src/features/HiddenFileSync/CmdHiddenFileSync.ts
View File

@@ -1,9 +1,8 @@
import { normalizePath, type PluginManifest, type ListedFiles } from "../../deps.ts"; import { normalizePath, type PluginManifest, type ListedFiles } from "../../deps.ts";
import { type EntryDoc, type LoadedEntry, type InternalFileEntry, type FilePathWithPrefix, type FilePath, LOG_LEVEL_INFO, LOG_LEVEL_NOTICE, LOG_LEVEL_VERBOSE, MODE_SELECTIVE, MODE_PAUSED, type SavingEntry, type DocumentID } from "../../lib/src/common/types.ts"; import { type EntryDoc, type LoadedEntry, type InternalFileEntry, type FilePathWithPrefix, type FilePath, LOG_LEVEL_INFO, LOG_LEVEL_NOTICE, LOG_LEVEL_VERBOSE, MODE_SELECTIVE, MODE_PAUSED, type SavingEntry, type DocumentID, type UXStat, MODE_AUTOMATIC } from "../../lib/src/common/types.ts";
import { type InternalFileInfo, ICHeader, ICHeaderEnd } from "../../common/types.ts"; import { type InternalFileInfo, ICHeader, ICHeaderEnd } from "../../common/types.ts";
import { readAsBlob, isDocContentSame, sendSignal, readContent, createBlob, fireAndForget } from "../../lib/src/common/utils.ts"; import { readAsBlob, isDocContentSame, sendSignal, readContent, createBlob, fireAndForget } from "../../lib/src/common/utils.ts";
import { Logger } from "../../lib/src/common/logger.ts"; import { BASE_IS_NEW, compareMTime, EVEN, getPath, isInternalMetadata, isMarkedAsSameChanges, markChangesAreSame, PeriodicProcessor, TARGET_IS_NEW } from "../../common/utils.ts";
import { getPath, isInternalMetadata, PeriodicProcessor } from "../../common/utils.ts";
import { serialized } from "../../lib/src/concurrency/lock.ts"; import { serialized } from "../../lib/src/concurrency/lock.ts";
import { JsonResolveModal } from "../HiddenFileCommon/JsonResolveModal.ts"; import { JsonResolveModal } from "../HiddenFileCommon/JsonResolveModal.ts";
import { LiveSyncCommands } from "../LiveSyncCommands.ts"; import { LiveSyncCommands } from "../LiveSyncCommands.ts";
@@ -41,12 +40,12 @@ export class HiddenFileSync extends LiveSyncCommands implements IObsidianModule
async $everyOnDatabaseInitialized(showNotice: boolean) { async $everyOnDatabaseInitialized(showNotice: boolean) {
if (this._isThisModuleEnabled()) { if (this._isThisModuleEnabled()) {
try { try {
Logger("Synchronizing hidden files..."); this._log("Synchronizing hidden files...");
await this.syncInternalFilesAndDatabase("push", showNotice); await this.syncInternalFilesAndDatabase("push", showNotice);
Logger("Synchronizing hidden files done"); this._log("Synchronizing hidden files done");
} catch (ex) { } catch (ex) {
Logger("Synchronizing hidden files failed"); this._log("Synchronizing hidden files failed");
Logger(ex, LOG_LEVEL_VERBOSE); this._log(ex, LOG_LEVEL_VERBOSE);
} }
} }
return true; return true;
@@ -58,6 +57,13 @@ export class HiddenFileSync extends LiveSyncCommands implements IObsidianModule
return true; return true;
} }
$everyOnloadAfterLoadSettings(): Promise<boolean> {
const ignorePatterns = this.settings.syncInternalFilesIgnorePatterns
.replace(/\n| /g, "")
.split(",").filter(e => e).map(e => new RegExp(e, "i"));
this.ignorePatterns = ignorePatterns;
return Promise.resolve(true);
}
async $everyOnResumeProcess(): Promise<boolean> { async $everyOnResumeProcess(): Promise<boolean> {
this.periodicInternalFileScanProcessor?.disable(); this.periodicInternalFileScanProcessor?.disable();
@@ -77,6 +83,10 @@ export class HiddenFileSync extends LiveSyncCommands implements IObsidianModule
if (!this.plugin.isReady) if (!this.plugin.isReady)
return Promise.resolve(true); return Promise.resolve(true);
this.periodicInternalFileScanProcessor.enable(this._isThisModuleEnabled() && this.settings.syncInternalFilesInterval ? (this.settings.syncInternalFilesInterval * 1000) : 0); this.periodicInternalFileScanProcessor.enable(this._isThisModuleEnabled() && this.settings.syncInternalFilesInterval ? (this.settings.syncInternalFilesInterval * 1000) : 0);
const ignorePatterns = this.settings.syncInternalFilesIgnorePatterns
.replace(/\n| /g, "")
.split(",").filter(e => e).map(e => new RegExp(e, "i"));
this.ignorePatterns = ignorePatterns;
return Promise.resolve(true); return Promise.resolve(true);
} }
@@ -85,56 +95,56 @@ export class HiddenFileSync extends LiveSyncCommands implements IObsidianModule
} }
internalFileProcessor = new QueueProcessor<string, any>( internalFileProcessor = new QueueProcessor<string, any>(
async (filenames) => { async (filenames) => {
Logger(`START :Applying hidden ${filenames.length} files change`, LOG_LEVEL_VERBOSE); this._log(`START :Applying hidden ${filenames.length} files change`, LOG_LEVEL_VERBOSE);
await this.syncInternalFilesAndDatabase("pull", false, false, filenames); await this.syncInternalFilesAndDatabase("pull", false, false, filenames);
Logger(`DONE :Applying hidden ${filenames.length} files change`, LOG_LEVEL_VERBOSE); this._log(`DONE :Applying hidden ${filenames.length} files change`, LOG_LEVEL_VERBOSE);
return; return;
}, { batchSize: 100, concurrentLimit: 1, delay: 10, yieldThreshold: 100, suspended: false, totalRemainingReactiveSource: hiddenFilesEventCount } }, { batchSize: 100, concurrentLimit: 1, delay: 10, yieldThreshold: 100, suspended: false, totalRemainingReactiveSource: hiddenFilesEventCount }
); );
recentProcessedInternalFiles = [] as string[];
async $anyProcessOptionalFileEvent(path: FilePath): Promise<boolean | undefined> { async $anyProcessOptionalFileEvent(path: FilePath): Promise<boolean | undefined> {
return await this.watchVaultRawEventsAsync(path); return await this.watchVaultRawEventsAsync(path);
} }
async watchVaultRawEventsAsync(path: FilePath): Promise<boolean | undefined> { async watchVaultRawEventsAsync(path: FilePath): Promise<boolean | undefined> {
if (!this._isMainReady) return false;
if (this._isMainSuspended()) return false;
if (!this._isThisModuleEnabled()) return false; if (!this._isThisModuleEnabled()) return false;
if (!isInternalMetadata(path)) return false;
// Exclude files handled by customization sync // Exclude files handled by customization sync
const configDir = normalizePath(this.app.vault.configDir); const configDir = normalizePath(this.app.vault.configDir);
const synchronisedInConfigSync = !this.settings.usePluginSync ? [] : Object.values(this.settings.pluginSyncExtendedSetting).filter(e => e.mode == MODE_SELECTIVE || e.mode == MODE_PAUSED).map(e => e.files).flat().map(e => `${configDir}/${e}`.toLowerCase()); const synchronisedInConfigSync = !this.settings.usePluginSync ? [] :
Object.values(this.settings.pluginSyncExtendedSetting).
filter(e => e.mode == MODE_SELECTIVE || e.mode == MODE_PAUSED).
map(e => e.files).flat().map(e => `${configDir}/${e}`.toLowerCase());
if (synchronisedInConfigSync.some(e => e.startsWith(path.toLowerCase()))) { if (synchronisedInConfigSync.some(e => e.startsWith(path.toLowerCase()))) {
Logger(`Hidden file skipped: ${path} is synchronized in customization sync.`, LOG_LEVEL_VERBOSE); this._log(`Hidden file skipped: ${path} is synchronized in customization sync.`, LOG_LEVEL_VERBOSE);
return false; return false;
} }
const stat = await this.plugin.storageAccess.statHidden(path); const stat = await this.plugin.storageAccess.statHidden(path);
// sometimes folder is coming. // sometimes folder is coming.
if (stat != null && stat.type != "file") { if (stat != null && stat.type != "file") {
return false; return false;
} }
const mtime = stat == null ? 0 : stat?.mtime ?? 0;
const storageMTime = ~~((mtime) / 1000); if (this.isKnownChange(path, stat?.mtime ?? 0)) {
const key = `${path}-${storageMTime}`; // This could be caused by self. so return true to prevent further processing.
if (mtime != 0 && this.recentProcessedInternalFiles.contains(key)) {
//If recently processed, it may caused by self.
// Return true to prevent further processing.
return true; return true;
} }
this.recentProcessedInternalFiles = [key, ...this.recentProcessedInternalFiles].slice(0, 100); const mtime = stat == null ? 0 : stat?.mtime ?? 0;
// const id = await this.path2id(path, ICHeader); const storageMTime = ~~((mtime) / 1000);
const prefixedFileName = addPrefix(path, ICHeader); const prefixedFileName = addPrefix(path, ICHeader);
const filesOnDB = await this.localDatabase.getDBEntryMeta(prefixedFileName); const filesOnDB = await this.localDatabase.getDBEntryMeta(prefixedFileName);
const dbMTime = ~~((filesOnDB && filesOnDB.mtime || 0) / 1000); const dbMTime = ~~((filesOnDB && filesOnDB.mtime || 0) / 1000);
// Skip unchanged file. // Skip unchanged file.
if (dbMTime == storageMTime) { if (dbMTime == storageMTime) {
// Logger(`STORAGE --> DB:${path}: (hidden) Nothing changed`); // this._log(`STORAGE --> DB:${path}: (hidden) Nothing changed`);
// Handled, but nothing changed. also return true to prevent further processing. // Handled, but nothing changed. also return true to prevent further processing.
return true; return true;
} }
// Do not compare timestamp. Always local data should be preferred except this plugin wrote one.
try { try {
if (storageMTime == 0) { if (storageMTime == 0) {
await this.deleteInternalFileOnDatabase(path); await this.deleteInternalFileOnDatabase(path);
@@ -144,8 +154,8 @@ export class HiddenFileSync extends LiveSyncCommands implements IObsidianModule
// Surely processed. // Surely processed.
return true; return true;
} catch (ex) { } catch (ex) {
Logger(`Failed to process hidden file:${path}`); this._log(`Failed to process hidden file:${path}`);
Logger(ex, LOG_LEVEL_VERBOSE); this._log(ex, LOG_LEVEL_VERBOSE);
} }
// Could not be processed. but it was own task. so return true to prevent further processing. // Could not be processed. but it was own task. so return true to prevent further processing.
return true; return true;
@@ -164,8 +174,8 @@ export class HiddenFileSync extends LiveSyncCommands implements IObsidianModule
} }
} }
} catch (ex) { } catch (ex) {
Logger("something went wrong on resolving all conflicted internal files"); this._log("something went wrong on resolving all conflicted internal files");
Logger(ex, LOG_LEVEL_VERBOSE); this._log(ex, LOG_LEVEL_VERBOSE);
} }
await this.conflictResolutionProcessor.startPipeline().waitForAllProcessed(); await this.conflictResolutionProcessor.startPipeline().waitForAllProcessed();
} }
@@ -176,12 +186,12 @@ export class HiddenFileSync extends LiveSyncCommands implements IObsidianModule
// simply check modified time // simply check modified time
const mtimeCurrent = ("mtime" in currentDoc && currentDoc.mtime) || 0; const mtimeCurrent = ("mtime" in currentDoc && currentDoc.mtime) || 0;
const mtimeConflicted = ("mtime" in conflictedDoc && conflictedDoc.mtime) || 0; const mtimeConflicted = ("mtime" in conflictedDoc && conflictedDoc.mtime) || 0;
// Logger(`Revisions:${new Date(mtimeA).toLocaleString} and ${new Date(mtimeB).toLocaleString}`); // this._log(`Revisions:${new Date(mtimeA).toLocaleString} and ${new Date(mtimeB).toLocaleString}`);
// console.log(`mtime:${mtimeA} - ${mtimeB}`); // console.log(`mtime:${mtimeA} - ${mtimeB}`);
const delRev = mtimeCurrent < mtimeConflicted ? currentRev : conflictedRev; const delRev = mtimeCurrent < mtimeConflicted ? currentRev : conflictedRev;
// delete older one. // delete older one.
await this.localDatabase.removeRevision(id, delRev); await this.localDatabase.removeRevision(id, delRev);
Logger(`Older one has been deleted:${path}`); this._log(`Older one has been deleted:${path}`);
const cc = await this.localDatabase.getRaw(id, { conflicts: true }); const cc = await this.localDatabase.getRaw(id, { conflicts: true });
if (cc._conflicts?.length === 0) { if (cc._conflicts?.length === 0) {
await this.extractInternalFileFromDatabase(stripAllPrefixes(path)) await this.extractInternalFileFromDatabase(stripAllPrefixes(path))
@@ -204,7 +214,7 @@ export class HiddenFileSync extends LiveSyncCommands implements IObsidianModule
if (doc._conflicts === undefined) return []; if (doc._conflicts === undefined) return [];
if (doc._conflicts.length == 0) if (doc._conflicts.length == 0)
return []; return [];
Logger(`Hidden file conflicted:${path}`); this._log(`Hidden file conflicted:${path}`);
const conflicts = doc._conflicts.sort((a, b) => Number(a.split("-")[0]) - Number(b.split("-")[0])); const conflicts = doc._conflicts.sort((a, b) => Number(a.split("-")[0]) - Number(b.split("-")[0]));
const revA = doc._rev; const revA = doc._rev;
const revB = conflicts[0]; const revB = conflicts[0];
@@ -217,7 +227,7 @@ export class HiddenFileSync extends LiveSyncCommands implements IObsidianModule
const commonBase = revFrom._revs_info?.filter(e => e.status == "available" && Number(e.rev.split("-")[0]) < conflictedRevNo).first()?.rev ?? ""; const commonBase = revFrom._revs_info?.filter(e => e.status == "available" && Number(e.rev.split("-")[0]) < conflictedRevNo).first()?.rev ?? "";
const result = await this.plugin.localDatabase.mergeObject(path, commonBase, doc._rev, conflictedRev); const result = await this.plugin.localDatabase.mergeObject(path, commonBase, doc._rev, conflictedRev);
if (result) { if (result) {
Logger(`Object merge:${path}`, LOG_LEVEL_INFO); this._log(`Object merge:${path}`, LOG_LEVEL_INFO);
const filename = stripAllPrefixes(path); const filename = stripAllPrefixes(path);
const isExists = await this.plugin.storageAccess.isExistsIncludeHidden(filename); const isExists = await this.plugin.storageAccess.isExistsIncludeHidden(filename);
if (!isExists) { if (!isExists) {
@@ -234,7 +244,7 @@ export class HiddenFileSync extends LiveSyncCommands implements IObsidianModule
this.conflictResolutionProcessor.enqueue(path); this.conflictResolutionProcessor.enqueue(path);
return []; return [];
} else { } else {
Logger(`Object merge is not applicable.`, LOG_LEVEL_VERBOSE); this._log(`Object merge is not applicable.`, LOG_LEVEL_VERBOSE);
} }
return [{ path, revA, revB, id, doc }]; return [{ path, revA, revB, id, doc }];
} }
@@ -242,8 +252,8 @@ export class HiddenFileSync extends LiveSyncCommands implements IObsidianModule
await this.resolveByNewerEntry(id, path, doc, revA, revB); await this.resolveByNewerEntry(id, path, doc, revA, revB);
return []; return [];
} catch (ex) { } catch (ex) {
Logger(`Failed to resolve conflict (Hidden): ${path}`); this._log(`Failed to resolve conflict (Hidden): ${path}`);
Logger(ex, LOG_LEVEL_VERBOSE); this._log(ex, LOG_LEVEL_VERBOSE);
return []; return [];
} }
}, { }, {
@@ -281,7 +291,7 @@ export class HiddenFileSync extends LiveSyncCommands implements IObsidianModule
this.procInternalFile(filename); this.procInternalFile(filename);
return true; return true;
} else { } else {
Logger(`Skipped (Not target:${filename})`, LOG_LEVEL_VERBOSE); this._log(`Skipped (Not target:${filename})`, LOG_LEVEL_VERBOSE);
return false; return false;
} }
} }
@@ -291,30 +301,46 @@ export class HiddenFileSync extends LiveSyncCommands implements IObsidianModule
this.conflictResolutionProcessor.enqueue(path); this.conflictResolutionProcessor.enqueue(path);
} }
knownChanges: { [key: string]: number; } = {};
markAsKnownChange(path: string, mtime: number) {
this.knownChanges[path] = mtime;
}
isKnownChange(path: string, mtime: number) {
return this.knownChanges[path] == mtime;
}
ignorePatterns: RegExp[] = [];
//TODO: Tidy up. Even though it is experimental feature, So dirty... //TODO: Tidy up. Even though it is experimental feature, So dirty...
async syncInternalFilesAndDatabase(direction: "push" | "pull" | "safe" | "pullForce" | "pushForce", showMessage: boolean, filesAll: InternalFileInfo[] | false = false, targetFiles: string[] | false = false) { async syncInternalFilesAndDatabase(direction: "push" | "pull" | "safe" | "pullForce" | "pushForce", showMessage: boolean, filesAll: InternalFileInfo[] | false = false, targetFilesSrc: string[] | false = false) {
const targetFiles = targetFilesSrc ? targetFilesSrc.map(e => stripAllPrefixes(e as FilePathWithPrefix)) : false;
// debugger;
await this.resolveConflictOnInternalFiles(); await this.resolveConflictOnInternalFiles();
const logLevel = showMessage ? LOG_LEVEL_NOTICE : LOG_LEVEL_INFO; const logLevel = showMessage ? LOG_LEVEL_NOTICE : LOG_LEVEL_INFO;
Logger("Scanning hidden files.", logLevel, "sync_internal"); this._log("Scanning hidden files.", logLevel, "sync_internal");
const ignorePatterns = this.settings.syncInternalFilesIgnorePatterns
.replace(/\n| /g, "")
.split(",").filter(e => e).map(e => new RegExp(e, "i"));
const configDir = normalizePath(this.app.vault.configDir); const configDir = normalizePath(this.app.vault.configDir);
let files: InternalFileInfo[] = let files: InternalFileInfo[] =
filesAll ? filesAll : (await this.scanInternalFiles()) filesAll ? filesAll : (await this.scanInternalFiles())
const allowedInHiddenFileSync = this.settings.usePluginSync ? Object.values(this.settings.pluginSyncExtendedSetting).filter(e => e.mode == MODE_AUTOMATIC).map(e => e.files).flat().map(e => `${configDir}/${e}`.toLowerCase()) : undefined;
if (allowedInHiddenFileSync) {
const systemOrNot = files.reduce((acc, cur) => {
if (cur.path.startsWith(configDir)) {
acc.system.push(cur);
} else {
acc.user.push(cur);
}
return acc;
}, { system: [] as InternalFileInfo[], user: [] as InternalFileInfo[] });
const synchronisedInConfigSync = !this.settings.usePluginSync ? [] : Object.values(this.settings.pluginSyncExtendedSetting).filter(e => e.mode == MODE_SELECTIVE || e.mode == MODE_PAUSED).map(e => e.files).flat().map(e => `${configDir}/${e}`.toLowerCase()); files =
files = files.filter(file => synchronisedInConfigSync.every(filterFile => !file.path.toLowerCase().startsWith(filterFile))) [...systemOrNot.user,
...systemOrNot.system.filter(file => allowedInHiddenFileSync.some(filterFile => file.path.toLowerCase().startsWith(filterFile)))];
}
const filesOnDB = ((await this.localDatabase.allDocsRaw({ startkey: ICHeader, endkey: ICHeaderEnd, include_docs: true })).rows.map(e => e.doc) as InternalFileEntry[]).filter(e => !e.deleted); const filesOnDB = ((await this.localDatabase.allDocsRaw({ startkey: ICHeader, endkey: ICHeaderEnd, include_docs: true })).rows.map(e => e.doc) as InternalFileEntry[]).filter(e => !e.deleted);
const allFileNamesSrc = [...new Set([...files.map(e => normalizePath(e.path)), ...filesOnDB.map(e => stripAllPrefixes(this.getPath(e)))])]; const allFileNamesSrc = [...new Set([...files.map(e => normalizePath(e.path)), ...filesOnDB.map(e => stripAllPrefixes(this.getPath(e)))])];
const allFileNames = allFileNamesSrc.filter(filename => !targetFiles || (targetFiles && targetFiles.indexOf(filename) !== -1)).filter(path => synchronisedInConfigSync.every(filterFile => !path.toLowerCase().startsWith(filterFile))) let allFileNames = allFileNamesSrc.filter(filename => !targetFiles || (targetFiles && targetFiles.indexOf(filename) !== -1));
function compareMTime(a: number, b: number) { if (allowedInHiddenFileSync) {
const wa = ~~(a / 1000); allFileNames = allFileNames.filter(file => allowedInHiddenFileSync.some(filterFile => file.toLowerCase().startsWith(filterFile)));
const wb = ~~(b / 1000);
const diff = wa - wb;
return diff;
} }
const fileCount = allFileNames.length; const fileCount = allFileNames.length;
@@ -342,9 +368,7 @@ export class HiddenFileSync extends LiveSyncCommands implements IObsidianModule
c = pieces.shift(); c = pieces.shift();
} }
}; };
// Cache update time information for files which have already been processed (mainly for files that were skipped due to the same content)
let caches: { [key: string]: { storageMtime: number; docMtime: number; }; } = {};
caches = await this.kvDB.get<{ [key: string]: { storageMtime: number; docMtime: number; }; }>("diff-caches-internal") || {};
const filesMap = files.reduce((acc, cur) => { const filesMap = files.reduce((acc, cur) => {
acc[cur.path] = cur; acc[cur.path] = cur;
return acc; return acc;
@@ -357,10 +381,10 @@ export class HiddenFileSync extends LiveSyncCommands implements IObsidianModule
const filename = filenames[0]; const filename = filenames[0];
processed++; processed++;
if (processed % 100 == 0) { if (processed % 100 == 0) {
Logger(`Hidden file: ${processed}/${fileCount}`, logLevel, "sync_internal"); this._log(`Hidden file: ${processed}/${fileCount}`, logLevel, "sync_internal");
} }
if (!filename) return []; if (!filename) return [];
if (ignorePatterns.some(e => filename.match(e))) if (this.ignorePatterns.some(e => filename.match(e)))
return []; return [];
if (await this.plugin.$$isIgnoredByIgnoreFiles(filename)) { if (await this.plugin.$$isIgnoredByIgnoreFiles(filename)) {
return []; return [];
@@ -383,27 +407,27 @@ export class HiddenFileSync extends LiveSyncCommands implements IObsidianModule
fileOnStorage: xFileOnStorage, fileOnStorage: xFileOnStorage,
fileOnDatabase: xFileOnDatabase fileOnDatabase: xFileOnDatabase
} = params[0]; } = params[0];
if (xFileOnStorage && xFileOnDatabase) { const xFileOnDatabaseExists = xFileOnDatabase !== undefined && !(xFileOnDatabase.deleted || xFileOnDatabase._deleted);
const cache = filename in caches ? caches[filename] : { storageMtime: 0, docMtime: 0 }; if (xFileOnStorage && xFileOnDatabaseExists) {
// Both => Synchronize // Both => Synchronize
if ((direction != "pullForce" && direction != "pushForce") && xFileOnDatabase.mtime == cache.docMtime && xFileOnStorage.mtime == cache.storageMtime) { if ((direction != "pullForce" && direction != "pushForce") && isMarkedAsSameChanges(filename, [xFileOnDatabase.mtime, xFileOnStorage.mtime]) == EVEN) {
this._log(`Hidden file skipped: ${filename} is marked as same`, LOG_LEVEL_VERBOSE);
return; return;
} }
const nw = compareMTime(xFileOnStorage.mtime, xFileOnDatabase.mtime); const nw = compareMTime(xFileOnStorage.mtime, xFileOnDatabase.mtime);
if (nw > 0 || direction == "pushForce") { if (nw == BASE_IS_NEW || direction == "pushForce") {
await this.storeInternalFileToDatabase(xFileOnStorage); if (await this.storeInternalFileToDatabase(xFileOnStorage) !== false) {
} // countUpdatedFolder(filename);
if (nw < 0 || direction == "pullForce") { }
} else if (nw == TARGET_IS_NEW || direction == "pullForce") {
// skip if not extraction performed. // skip if not extraction performed.
if (!await this.extractInternalFileFromDatabase(filename)) if (await this.extractInternalFileFromDatabase(filename))
return; countUpdatedFolder(filename);
} else {
// Even, or not forced. skip.
} }
// If process successfully updated or file contents are same, update cache. } else if (!xFileOnStorage && xFileOnDatabaseExists) {
cache.docMtime = xFileOnDatabase.mtime;
cache.storageMtime = xFileOnStorage.mtime;
caches[filename] = cache;
countUpdatedFolder(filename);
} else if (!xFileOnStorage && xFileOnDatabase) {
if (direction == "push" || direction == "pushForce") { if (direction == "push" || direction == "pushForce") {
if (xFileOnDatabase.deleted) if (xFileOnDatabase.deleted)
return; return;
@@ -419,11 +443,14 @@ export class HiddenFileSync extends LiveSyncCommands implements IObsidianModule
countUpdatedFolder(filename); countUpdatedFolder(filename);
} }
} }
} else if (xFileOnStorage && !xFileOnDatabase) { } else if (xFileOnStorage && !xFileOnDatabaseExists) {
if (direction == "push" || direction == "pushForce" || direction == "safe") { if (direction == "push" || direction == "pushForce" || direction == "safe") {
await this.storeInternalFileToDatabase(xFileOnStorage); await this.storeInternalFileToDatabase(xFileOnStorage);
} else { } else {
await this.extractInternalFileFromDatabase(xFileOnStorage.path); // Apply the deletion
if (await this.extractInternalFileFromDatabase(xFileOnStorage.path)) {
countUpdatedFolder(xFileOnStorage.path);
}
} }
} else { } else {
throw new Error("Invalid state on hidden file sync"); throw new Error("Invalid state on hidden file sync");
@@ -435,8 +462,6 @@ export class HiddenFileSync extends LiveSyncCommands implements IObsidianModule
.enqueueAll(allFileNames) .enqueueAll(allFileNames)
.startPipeline().waitForAllDoneAndTerminate(); .startPipeline().waitForAllDoneAndTerminate();
await this.kvDB.set("diff-caches-internal", caches);
// When files has been retrieved from the database. they must be reloaded. // When files has been retrieved from the database. they must be reloaded.
if ((direction == "pull" || direction == "pullForce") && filesChanged != 0) { if ((direction == "pull" || direction == "pullForce") && filesChanged != 0) {
// Show notification to restart obsidian when something has been changed in configDir. // Show notification to restart obsidian when something has been changed in configDir.
@@ -459,12 +484,12 @@ export class HiddenFileSync extends LiveSyncCommands implements IObsidianModule
anchor.text = "HERE"; anchor.text = "HERE";
anchor.addEventListener("click", () => { anchor.addEventListener("click", () => {
fireAndForget(async () => { fireAndForget(async () => {
Logger(`Unloading plugin: ${updatePluginName}`, LOG_LEVEL_NOTICE, "plugin-reload-" + updatePluginId); this._log(`Unloading plugin: ${updatePluginName}`, LOG_LEVEL_NOTICE, "plugin-reload-" + updatePluginId);
// @ts-ignore // @ts-ignore
await this.app.plugins.unloadPlugin(updatePluginId); await this.app.plugins.unloadPlugin(updatePluginId);
// @ts-ignore // @ts-ignore
await this.app.plugins.loadPlugin(updatePluginId); await this.app.plugins.loadPlugin(updatePluginId);
Logger(`Plugin reloaded: ${updatePluginName}`, LOG_LEVEL_NOTICE, "plugin-reload-" + updatePluginId); this._log(`Plugin reloaded: ${updatePluginName}`, LOG_LEVEL_NOTICE, "plugin-reload-" + updatePluginId);
}); });
}); });
} }
@@ -472,8 +497,8 @@ export class HiddenFileSync extends LiveSyncCommands implements IObsidianModule
} }
} }
} catch (ex) { } catch (ex) {
Logger("Error on checking plugin status."); this._log("Error on checking plugin status.");
Logger(ex, LOG_LEVEL_VERBOSE); this._log(ex, LOG_LEVEL_VERBOSE);
} }
@@ -491,17 +516,19 @@ export class HiddenFileSync extends LiveSyncCommands implements IObsidianModule
} }
} }
Logger(`Hidden files scanned: ${filesChanged} files had been modified`, logLevel, "sync_internal"); this._log(`Hidden files scanned: ${filesChanged} files had been modified`, logLevel, "sync_internal");
} }
async storeInternalFileToDatabase(file: InternalFileInfo, forceWrite = false) { async storeInternalFileToDatabase(file: InternalFileInfo, forceWrite = false) {
if (await this.plugin.$$isIgnoredByIgnoreFiles(file.path)) { const storeFilePath = file.path;
return const storageFilePath = file.path;
if (await this.plugin.$$isIgnoredByIgnoreFiles(storageFilePath)) {
return undefined;
} }
const id = await this.path2id(file.path, ICHeader); const id = await this.path2id(storeFilePath, ICHeader);
const prefixedFileName = addPrefix(file.path, ICHeader); const prefixedFileName = addPrefix(storeFilePath, ICHeader);
const content = createBlob(await this.plugin.storageAccess.readHiddenFileAuto(file.path)); const content = createBlob(await this.plugin.storageAccess.readHiddenFileAuto(storageFilePath));
const mtime = file.mtime; const mtime = file.mtime;
return await serialized("file-" + prefixedFileName, async () => { return await serialized("file-" + prefixedFileName, async () => {
try { try {
@@ -523,8 +550,12 @@ export class HiddenFileSync extends LiveSyncCommands implements IObsidianModule
}; };
} else { } else {
if (await isDocContentSame(readAsBlob(old), content) && !forceWrite) { if (await isDocContentSame(readAsBlob(old), content) && !forceWrite) {
// Logger(`STORAGE --> DB:${file.path}: (hidden) Not changed`, LOG_LEVEL_VERBOSE); // this._log(`STORAGE --> DB:${file.path}: (hidden) Not changed`, LOG_LEVEL_VERBOSE);
return; const stat = await this.plugin.storageAccess.statHidden(storageFilePath);
if (stat) {
markChangesAreSame(storageFilePath, old.mtime, stat.mtime);
}
return undefined;
} }
saveData = saveData =
{ {
@@ -539,24 +570,32 @@ export class HiddenFileSync extends LiveSyncCommands implements IObsidianModule
}; };
} }
const ret = await this.localDatabase.putDBEntry(saveData); const ret = await this.localDatabase.putDBEntry(saveData);
Logger(`STORAGE --> DB:${file.path}: (hidden) Done`); if (ret !== false) {
return ret; this._log(`STORAGE --> DB:${storageFilePath}: (hidden) Done`);
return true;
} else {
this._log(`STORAGE --> DB:${storageFilePath}: (hidden) Failed`);
return false;
}
} catch (ex) { } catch (ex) {
Logger(`STORAGE --> DB:${file.path}: (hidden) Failed`); this._log(`STORAGE --> DB:${storageFilePath}: (hidden) Failed`);
Logger(ex, LOG_LEVEL_VERBOSE); this._log(ex, LOG_LEVEL_VERBOSE);
return false; return false;
} }
}); });
} }
async deleteInternalFileOnDatabase(filename: FilePath, forceWrite = false) { async deleteInternalFileOnDatabase(filenameSrc: FilePath, forceWrite = false) {
const id = await this.path2id(filename, ICHeader); const storeFilePath = filenameSrc;
const prefixedFileName = addPrefix(filename, ICHeader); const storageFilePath = filenameSrc;
const displayFileName = filenameSrc;
const id = await this.path2id(storeFilePath, ICHeader);
const prefixedFileName = addPrefix(storeFilePath, ICHeader);
const mtime = new Date().getTime(); const mtime = new Date().getTime();
if (await this.plugin.$$isIgnoredByIgnoreFiles(filename)) { if (await this.plugin.$$isIgnoredByIgnoreFiles(storageFilePath)) {
return return undefined
} }
await serialized("file-" + prefixedFileName, async () => { return await serialized("file-" + prefixedFileName, async () => {
try { try {
const old = await this.localDatabase.getDBEntryMeta(prefixedFileName, undefined, true) as InternalFileEntry | false; const old = await this.localDatabase.getDBEntryMeta(prefixedFileName, undefined, true) as InternalFileEntry | false;
let saveData: InternalFileEntry; let saveData: InternalFileEntry;
@@ -578,12 +617,12 @@ export class HiddenFileSync extends LiveSyncCommands implements IObsidianModule
if (conflicts._conflicts !== undefined) { if (conflicts._conflicts !== undefined) {
for (const conflictRev of conflicts._conflicts) { for (const conflictRev of conflicts._conflicts) {
await this.localDatabase.removeRevision(old._id, conflictRev); await this.localDatabase.removeRevision(old._id, conflictRev);
Logger(`STORAGE -x> DB:${filename}: (hidden) conflict removed ${old._rev} => ${conflictRev}`, LOG_LEVEL_VERBOSE); this._log(`STORAGE -x> DB: ${displayFileName}: (hidden) conflict removed ${old._rev} => ${conflictRev}`, LOG_LEVEL_VERBOSE);
} }
} }
if (old.deleted) { if (old.deleted) {
Logger(`STORAGE -x> DB:${filename}: (hidden) already deleted`); this._log(`STORAGE -x> DB: ${displayFileName}: (hidden) already deleted`);
return; return undefined;
} }
saveData = saveData =
{ {
@@ -595,85 +634,101 @@ export class HiddenFileSync extends LiveSyncCommands implements IObsidianModule
type: "newnote", type: "newnote",
}; };
} }
await this.localDatabase.putRaw(saveData); const ret = await this.localDatabase.putRaw(saveData);
Logger(`STORAGE -x> DB:${filename}: (hidden) Done`); if (ret && ret.ok) {
this._log(`STORAGE -x> DB: ${displayFileName}: (hidden) Done`);
return true;
} else {
this._log(`STORAGE -x> DB: ${displayFileName}: (hidden) Failed`);
return false;
}
} catch (ex) { } catch (ex) {
Logger(`STORAGE -x> DB:${filename}: (hidden) Failed`); this._log(`STORAGE -x> DB: ${displayFileName}: (hidden) Failed`);
Logger(ex, LOG_LEVEL_VERBOSE); this._log(ex, LOG_LEVEL_VERBOSE);
return false; return false;
} }
}); });
} }
async extractInternalFileFromDatabase(filename: FilePath, force = false) { async extractInternalFileFromDatabase(filenameSrc: FilePath, force = false) {
const isExists = await this.plugin.storageAccess.isExistsIncludeHidden(filename); const storeFilePath = filenameSrc;
const prefixedFileName = addPrefix(filename, ICHeader); const storageFilePath = filenameSrc;
if (await this.plugin.$$isIgnoredByIgnoreFiles(filename)) { const isExists = await this.plugin.storageAccess.isExistsIncludeHidden(storageFilePath);
return; const prefixedFileName = addPrefix(storeFilePath, ICHeader);
const displayFileName = `${storeFilePath}`;
if (await this.plugin.$$isIgnoredByIgnoreFiles(storageFilePath)) {
return undefined;
} }
return await serialized("file-" + prefixedFileName, async () => { return await serialized("file-" + prefixedFileName, async () => {
try { try {
// Check conflicted status // Check conflicted status
const fileOnDB = await this.localDatabase.getDBEntry(prefixedFileName, { conflicts: true }, false, true, true); const fileOnDB = await this.localDatabase.getDBEntry(prefixedFileName, { conflicts: true }, false, true, true);
if (fileOnDB === false) if (fileOnDB === false)
throw new Error(`File not found on database.:${filename}`); throw new Error(`File not found on database.:${displayFileName}`);
// Prevent overwrite for Prevent overwriting while some conflicted revision exists. // Prevent overwrite for Prevent overwriting while some conflicted revision exists.
if (fileOnDB?._conflicts?.length) { if (fileOnDB?._conflicts?.length) {
Logger(`Hidden file ${filename} has conflicted revisions, to keep in safe, writing to storage has been prevented`, LOG_LEVEL_INFO); this._log(`Hidden file ${displayFileName} has conflicted revisions, to keep in safe, writing to storage has been prevented`, LOG_LEVEL_INFO);
return; return false;
} }
const deleted = fileOnDB.deleted || fileOnDB._deleted || false; const deleted = fileOnDB.deleted || fileOnDB._deleted || false;
if (deleted) { if (deleted) {
if (!isExists) { if (!isExists) {
Logger(`STORAGE <x- DB:${filename}: deleted (hidden) Deleted on DB, but the file is already not found on storage.`); this._log(`STORAGE <x- DB: ${displayFileName}: deleted (hidden) Deleted on DB, but the file is already not found on storage.`);
} else { } else {
Logger(`STORAGE <x- DB:${filename}: deleted (hidden).`); this._log(`STORAGE <x- DB: ${displayFileName}: deleted (hidden).`);
await this.plugin.storageAccess.removeHidden(filename); await this.plugin.storageAccess.removeHidden(storageFilePath);
try { try {
// -- @ts-ignore internalAPI await this.plugin.storageAccess.triggerHiddenFile(storageFilePath);
// await this.app.vault.adapter.reconcileInternalFile(filename);
await this.plugin.storageAccess.triggerHiddenFile(filename);
} catch (ex) { } catch (ex) {
Logger("Failed to call internal API(reconcileInternalFile)", LOG_LEVEL_VERBOSE); this._log("Failed to call internal API(reconcileInternalFile)", LOG_LEVEL_VERBOSE);
Logger(ex, LOG_LEVEL_VERBOSE); this._log(ex, LOG_LEVEL_VERBOSE);
} }
} }
return true; return true;
} }
if (!isExists) { if (!isExists) {
await this.plugin.storageAccess.ensureDir(filename); await this.plugin.storageAccess.ensureDir(storageFilePath);
await this.plugin.storageAccess.writeHiddenFileAuto(filename, readContent(fileOnDB), { mtime: fileOnDB.mtime, ctime: fileOnDB.ctime }); await this.plugin.storageAccess.writeHiddenFileAuto(storageFilePath, readContent(fileOnDB), { mtime: fileOnDB.mtime, ctime: fileOnDB.ctime });
try { try {
//@ts-ignore internalAPI await this.plugin.storageAccess.triggerHiddenFile(storageFilePath);
await this.app.vault.adapter.reconcileInternalFile(filename);
} catch (ex) { } catch (ex) {
Logger("Failed to call internal API(reconcileInternalFile)", LOG_LEVEL_VERBOSE); this._log("Failed to call internal API(reconcileInternalFile)", LOG_LEVEL_VERBOSE);
Logger(ex, LOG_LEVEL_VERBOSE); this._log(ex, LOG_LEVEL_VERBOSE);
} }
Logger(`STORAGE <-- DB:${filename}: written (hidden,new${force ? ", force" : ""})`); this._log(`STORAGE <-- DB: ${displayFileName}: written (hidden,new${force ? ", force" : ""})`);
return true; return true;
} else { } else {
const content = await this.plugin.storageAccess.readHiddenFileAuto(filename); const content = await this.plugin.storageAccess.readHiddenFileAuto(storageFilePath);
const docContent = readContent(fileOnDB); const docContent = readContent(fileOnDB);
if (await isDocContentSame(content, docContent) && !force) { if (await isDocContentSame(content, docContent) && !force) {
// Logger(`STORAGE <-- DB:${filename}: skipped (hidden) Not changed`, LOG_LEVEL_VERBOSE); // this._log(`STORAGE <-- DB:${filename}: skipped (hidden) Not changed`, LOG_LEVEL_VERBOSE);
return true; const stat = await this.plugin.storageAccess.statHidden(storageFilePath);
if (stat) {
markChangesAreSame(storageFilePath, fileOnDB.mtime, stat.mtime);
}
return undefined;
} }
await this.plugin.storageAccess.writeHiddenFileAuto(filename, docContent, { mtime: fileOnDB.mtime, ctime: fileOnDB.ctime }); if (await this.plugin.storageAccess.writeHiddenFileAuto(storageFilePath, docContent, { mtime: fileOnDB.mtime, ctime: fileOnDB.ctime })) {
try { const stat = await this.plugin.storageAccess.statHidden(storageFilePath) as UXStat;
// await this.app.vault.adapter.reconcileInternalFile(filename); this.markAsKnownChange(storageFilePath, stat.mtime);
await this.plugin.storageAccess.triggerHiddenFile(filename); try {
} catch (ex) { // await this.app.vault.adapter.reconcileInternalFile(filename);
Logger("Failed to call internal API(reconcileInternalFile)", LOG_LEVEL_VERBOSE); await this.plugin.storageAccess.triggerHiddenFile(storageFilePath);
Logger(ex, LOG_LEVEL_VERBOSE); } catch (ex) {
} this._log("Failed to call internal API(reconcileInternalFile)", LOG_LEVEL_VERBOSE);
Logger(`STORAGE <-- DB:${filename}: written (hidden, overwrite${force ? ", force" : ""})`); this._log(ex, LOG_LEVEL_VERBOSE);
return true; }
this._log(`STORAGE <-- DB: ${displayFileName}: written (hidden, overwrite${force ? ", force" : ""})`);
return true;
} else {
this._log(`STORAGE <-- DB: ${displayFileName}: written (hidden, overwrite${force ? ", force" : ""}) Failed`);
return false;
}
} }
} catch (ex) { } catch (ex) {
Logger(`STORAGE <-- DB:${filename}: written (hidden, overwrite${force ? ", force" : ""}) Failed`); this._log(`STORAGE <-- DB: ${displayFileName}: written (hidden, overwrite${force ? ", force" : ""}) Failed`);
Logger(ex, LOG_LEVEL_VERBOSE); this._log(ex, LOG_LEVEL_VERBOSE);
return false; return false;
} }
}); });
@@ -683,16 +738,20 @@ export class HiddenFileSync extends LiveSyncCommands implements IObsidianModule
showJSONMergeDialogAndMerge(docA: LoadedEntry, docB: LoadedEntry): Promise<boolean> { showJSONMergeDialogAndMerge(docA: LoadedEntry, docB: LoadedEntry): Promise<boolean> {
return new Promise((res) => { return new Promise((res) => {
Logger("Opening data-merging dialog", LOG_LEVEL_VERBOSE); this._log("Opening data-merging dialog", LOG_LEVEL_VERBOSE);
const docs = [docA, docB]; const docs = [docA, docB];
const path = stripAllPrefixes(docA.path); const strippedPath = stripAllPrefixes(docA.path);
const modal = new JsonResolveModal(this.app, path, [docA, docB], async (keep, result) => { const storageFilePath = strippedPath;
const storeFilePath = strippedPath;
const displayFilename = `${storeFilePath}`;
// const path = this.prefixedConfigDir2configDir(stripAllPrefixes(docA.path)) || docA.path;
const modal = new JsonResolveModal(this.app, storageFilePath, [docA, docB], async (keep, result) => {
// modal.close(); // modal.close();
try { try {
const filename = path; // const filename = storeFilePath;
let needFlush = false; let needFlush = false;
if (!result && !keep) { if (!result && !keep) {
Logger(`Skipped merging: ${filename}`); this._log(`Skipped merging: ${displayFilename}`);
res(false); res(false);
return; return;
} }
@@ -701,41 +760,44 @@ export class HiddenFileSync extends LiveSyncCommands implements IObsidianModule
for (const doc of docs) { for (const doc of docs) {
if (doc._rev != keep) { if (doc._rev != keep) {
if (await this.localDatabase.deleteDBEntry(this.getPath(doc), { rev: doc._rev })) { if (await this.localDatabase.deleteDBEntry(this.getPath(doc), { rev: doc._rev })) {
Logger(`Conflicted revision has been deleted: ${filename}`); this._log(`Conflicted revision has been deleted: ${displayFilename}`);
needFlush = true; needFlush = true;
} }
} }
} }
} }
if (!keep && result) { if (!keep && result) {
const isExists = await this.plugin.storageAccess.isExistsIncludeHidden(filename); const isExists = await this.plugin.storageAccess.isExistsIncludeHidden(storageFilePath);
if (!isExists) { if (!isExists) {
await this.plugin.storageAccess.ensureDir(filename); await this.plugin.storageAccess.ensureDir(storageFilePath);
} }
await this.plugin.storageAccess.writeHiddenFileAuto(filename, result); await this.plugin.storageAccess.writeHiddenFileAuto(storageFilePath, result);
const stat = await this.plugin.storageAccess.statHidden(filename); const stat = await this.plugin.storageAccess.statHidden(storageFilePath);
if (!stat) { if (!stat) {
throw new Error("Stat failed"); throw new Error("Stat failed");
} }
const mtime = stat?.mtime ?? 0; const mtime = stat?.mtime ?? 0;
await this.storeInternalFileToDatabase({ path: filename, mtime, ctime: stat?.ctime ?? mtime, size: stat?.size ?? 0 }, true); await this.storeInternalFileToDatabase({ path: storageFilePath, mtime, ctime: stat?.ctime ?? mtime, size: stat?.size ?? 0 }, true);
try { try {
//@ts-ignore internalAPI //@ts-ignore internalAPI
await this.app.vault.adapter.reconcileInternalFile(filename); await this.app.vault.adapter.reconcileInternalFile(storageFilePath);
} catch (ex) { } catch (ex) {
Logger("Failed to call internal API(reconcileInternalFile)", LOG_LEVEL_VERBOSE); this._log("Failed to call internal API(reconcileInternalFile)", LOG_LEVEL_VERBOSE);
Logger(ex, LOG_LEVEL_VERBOSE); this._log(ex, LOG_LEVEL_VERBOSE);
} }
Logger(`STORAGE <-- DB:${filename}: written (hidden,merged)`); this._log(`STORAGE <-- DB:${displayFilename}: written (hidden,merged)`);
} }
if (needFlush) { if (needFlush) {
await this.extractInternalFileFromDatabase(filename, false); if (await this.extractInternalFileFromDatabase(storeFilePath, false)) {
Logger(`STORAGE --> DB:${filename}: extracted (hidden,merged)`); this._log(`STORAGE --> DB:${displayFilename}: extracted (hidden,merged)`);
} else {
this._log(`STORAGE --> DB:${displayFilename}: extracted (hidden,merged) Failed`);
}
} }
res(true); res(true);
} catch (ex) { } catch (ex) {
Logger("Could not merge conflicted json"); this._log("Could not merge conflicted json");
Logger(ex, LOG_LEVEL_VERBOSE); this._log(ex, LOG_LEVEL_VERBOSE);
res(false); res(false);
} }
}); });
@@ -789,7 +851,7 @@ ${messageFetch}${messageOverwrite}${messageMerge}
$allSuspendExtraSync(): Promise<boolean> { $allSuspendExtraSync(): Promise<boolean> {
if (this.plugin.settings.syncInternalFiles) { if (this.plugin.settings.syncInternalFiles) {
Logger("Hidden file synchronization have been temporarily disabled. Please enable them after the fetching, if you need them.", LOG_LEVEL_NOTICE) this._log("Hidden file synchronization have been temporarily disabled. Please enable them after the fetching, if you need them.", LOG_LEVEL_NOTICE)
this.plugin.settings.syncInternalFiles = false; this.plugin.settings.syncInternalFiles = false;
} }
return Promise.resolve(true); return Promise.resolve(true);
@@ -810,7 +872,7 @@ ${messageFetch}${messageOverwrite}${messageMerge}
await this.plugin.saveSettings(); await this.plugin.saveSettings();
return; return;
} }
Logger("Gathering files for enabling Hidden File Sync", LOG_LEVEL_NOTICE); this._log("Gathering files for enabling Hidden File Sync", LOG_LEVEL_NOTICE);
if (mode == "FETCH") { if (mode == "FETCH") {
await this.syncInternalFilesAndDatabase("pullForce", true); await this.syncInternalFilesAndDatabase("pullForce", true);
} else if (mode == "OVERWRITE") { } else if (mode == "OVERWRITE") {
@@ -821,7 +883,7 @@ ${messageFetch}${messageOverwrite}${messageMerge}
this.plugin.settings.useAdvancedMode = true; this.plugin.settings.useAdvancedMode = true;
this.plugin.settings.syncInternalFiles = true; this.plugin.settings.syncInternalFiles = true;
await this.plugin.saveSettings(); await this.plugin.saveSettings();
Logger(`Done! Restarting the app is strongly recommended!`, LOG_LEVEL_NOTICE); this._log(`Done! Restarting the app is strongly recommended!`, LOG_LEVEL_NOTICE);
} }
async scanInternalFiles(): Promise<InternalFileInfo[]> { async scanInternalFiles(): Promise<InternalFileInfo[]> {
@@ -869,8 +931,8 @@ ${messageFetch}${messageOverwrite}${messageMerge}
try { try {
w = await this.app.vault.adapter.list(path); w = await this.app.vault.adapter.list(path);
} catch (ex) { } catch (ex) {
Logger(`Could not traverse(HiddenSync):${path}`, LOG_LEVEL_INFO); this._log(`Could not traverse(HiddenSync):${path}`, LOG_LEVEL_INFO);
Logger(ex, LOG_LEVEL_VERBOSE); this._log(ex, LOG_LEVEL_VERBOSE);
return []; return [];
} }
const filesSrc = [ const filesSrc = [

11
src/features/LiveSyncCommands.ts
View File

@@ -1,5 +1,6 @@
import { Logger } from "octagonal-wheels/common/logger";
import { getPath } from "../common/utils.ts"; import { getPath } from "../common/utils.ts";
import { type AnyEntry, type DocumentID, type EntryHasPath, type FilePath, type FilePathWithPrefix } from "../lib/src/common/types.ts"; import { LOG_LEVEL_INFO, LOG_LEVEL_NOTICE, type AnyEntry, type DocumentID, type EntryHasPath, type FilePath, type FilePathWithPrefix, type LOG_LEVEL } from "../lib/src/common/types.ts";
import type ObsidianLiveSyncPlugin from "../main.ts"; import type ObsidianLiveSyncPlugin from "../main.ts";
@@ -40,4 +41,12 @@ export abstract class LiveSyncCommands {
_isDatabaseReady() { _isDatabaseReady() {
return this.plugin._isDatabaseReady(); return this.plugin._isDatabaseReady();
} }
_log = (msg: any, level: LOG_LEVEL = LOG_LEVEL_INFO, key?: string) => {
if (typeof msg === "string" && level !== LOG_LEVEL_NOTICE) {
msg = `[${this.constructor.name}]\u{200A} ${msg}`;
}
// console.log(msg);
Logger(msg, level, key);
};
} }

1
src/modules/features/ModuleObsidianSettingAsMarkdown.ts
View File

@@ -162,6 +162,7 @@ export class ModuleObsidianSettingsAsMarkdown extends AbstractObsidianModule imp
const saveData = { ...(settings ? settings : this.settings) } as Partial<ObsidianLiveSyncSettings>; const saveData = { ...(settings ? settings : this.settings) } as Partial<ObsidianLiveSyncSettings>;
delete saveData.encryptedCouchDBConnection; delete saveData.encryptedCouchDBConnection;
delete saveData.encryptedPassphrase; delete saveData.encryptedPassphrase;
delete saveData.additionalSuffixOfDatabaseName;
if (!saveData.writeCredentialsForSettingSync && !keepCredential) { if (!saveData.writeCredentialsForSettingSync && !keepCredential) {
delete saveData.couchDB_USER; delete saveData.couchDB_USER;
delete saveData.couchDB_PASSWORD; delete saveData.couchDB_PASSWORD;

119
src/modules/features/SettingDialogue/ObsidianLiveSyncSettingTab.ts
View File

@@ -1,6 +1,6 @@
import { App, PluginSettingTab, MarkdownRenderer, stringifyYaml } from "../../../deps.ts"; import { App, PluginSettingTab, MarkdownRenderer, stringifyYaml } from "../../../deps.ts";
import { import {
DEFAULT_SETTINGS, type ObsidianLiveSyncSettings, type ConfigPassphraseStore, type RemoteDBSettings, type FilePathWithPrefix, type HashAlgorithm, type DocumentID, LOG_LEVEL_NOTICE, LOG_LEVEL_VERBOSE, LOG_LEVEL_INFO, type LoadedEntry, PREFERRED_SETTING_CLOUDANT, PREFERRED_SETTING_SELF_HOSTED, FLAGMD_REDFLAG2_HR, FLAGMD_REDFLAG3_HR, REMOTE_COUCHDB, REMOTE_MINIO, PREFERRED_JOURNAL_SYNC, FLAGMD_REDFLAG, type ConfigLevel, LEVEL_POWER_USER, LEVEL_ADVANCED, LEVEL_EDGE_CASE, type MetaEntry, type FilePath DEFAULT_SETTINGS, type ObsidianLiveSyncSettings, type ConfigPassphraseStore, type RemoteDBSettings, type FilePathWithPrefix, type HashAlgorithm, type DocumentID, LOG_LEVEL_NOTICE, LOG_LEVEL_VERBOSE, LOG_LEVEL_INFO, type LoadedEntry, PREFERRED_SETTING_CLOUDANT, PREFERRED_SETTING_SELF_HOSTED, FLAGMD_REDFLAG2_HR, FLAGMD_REDFLAG3_HR, REMOTE_COUCHDB, REMOTE_MINIO, PREFERRED_JOURNAL_SYNC, FLAGMD_REDFLAG, type ConfigLevel, LEVEL_POWER_USER, LEVEL_ADVANCED, LEVEL_EDGE_CASE, type MetaEntry, type FilePath,
} from "../../../lib/src/common/types.ts"; } from "../../../lib/src/common/types.ts";
import { createBlob, delay, isDocContentSame, isObjectDifferent, readAsBlob, sizeToHumanReadable } from "../../../lib/src/common/utils.ts"; import { createBlob, delay, isDocContentSame, isObjectDifferent, readAsBlob, sizeToHumanReadable } from "../../../lib/src/common/utils.ts";
import { versionNumberString2Number } from "../../../lib/src/string_and_binary/convert.ts"; import { versionNumberString2Number } from "../../../lib/src/string_and_binary/convert.ts";
@@ -9,8 +9,8 @@ import { balanceChunkPurgedDBs, checkSyncInfo, isCloudantURI, purgeUnreferencedC
import { testCrypt } from "../../../lib/src/encryption/e2ee_v2.ts"; import { testCrypt } from "../../../lib/src/encryption/e2ee_v2.ts";
import ObsidianLiveSyncPlugin from "../../../main.ts"; import ObsidianLiveSyncPlugin from "../../../main.ts";
import { getPath, requestToCouchDB, scheduleTask } from "../../../common/utils.ts"; import { getPath, requestToCouchDB, scheduleTask } from "../../../common/utils.ts";
import { request, TFile } from "obsidian"; import { request } from "obsidian";
import { shouldBeIgnored } from "../../../lib/src/string_and_binary/path.ts"; import { addPrefix, shouldBeIgnored, stripAllPrefixes } from "../../../lib/src/string_and_binary/path.ts";
import MultipleRegExpControl from './MultipleRegExpControl.svelte'; import MultipleRegExpControl from './MultipleRegExpControl.svelte';
import { LiveSyncCouchDBReplicator } from "../../../lib/src/replication/couchdb/LiveSyncReplicator.ts"; import { LiveSyncCouchDBReplicator } from "../../../lib/src/replication/couchdb/LiveSyncReplicator.ts";
import { type AllSettingItemKey, type AllStringItemKey, type AllNumericItemKey, type AllBooleanItemKey, type AllSettings, OnDialogSettingsDefault, type OnDialogSettings, getConfName } from "./settingConstants.ts"; import { type AllSettingItemKey, type AllStringItemKey, type AllNumericItemKey, type AllBooleanItemKey, type AllSettings, OnDialogSettingsDefault, type OnDialogSettings, getConfName } from "./settingConstants.ts";
@@ -23,6 +23,8 @@ import { confirmWithMessage } from "../../coreObsidian/UILib/dialogs.ts";
import { EVENT_REQUEST_COPY_SETUP_URI, EVENT_REQUEST_OPEN_PLUGIN_SYNC_DIALOG, EVENT_REQUEST_OPEN_SETUP_URI, EVENT_REQUEST_RELOAD_SETTING_TAB, EVENT_REQUEST_SHOW_HISTORY, eventHub } from "../../../common/events.ts"; import { EVENT_REQUEST_COPY_SETUP_URI, EVENT_REQUEST_OPEN_PLUGIN_SYNC_DIALOG, EVENT_REQUEST_OPEN_SETUP_URI, EVENT_REQUEST_RELOAD_SETTING_TAB, EVENT_REQUEST_SHOW_HISTORY, eventHub } from "../../../common/events.ts";
import { skipIfDuplicated } from "octagonal-wheels/concurrency/lock"; import { skipIfDuplicated } from "octagonal-wheels/concurrency/lock";
import { JournalSyncMinio } from "../../../lib/src/replication/journal/objectstore/JournalSyncMinio.ts"; import { JournalSyncMinio } from "../../../lib/src/replication/journal/objectstore/JournalSyncMinio.ts";
import { ICHeader, ICXHeader, PSCHeader } from "../../../common/types.ts";
import { HiddenFileSync } from "../../../features/HiddenFileSync/CmdHiddenFileSync.ts";
export type OnUpdateResult = { export type OnUpdateResult = {
visibility?: boolean, visibility?: boolean,
@@ -1727,6 +1729,17 @@ However, your report is needed to stabilise this. I appreciate you for your grea
const enableOnlyOnPluginSyncIsNotEnabled = enableOnly(() => this.isConfiguredAs("usePluginSync", false)); const enableOnlyOnPluginSyncIsNotEnabled = enableOnly(() => this.isConfiguredAs("usePluginSync", false));
const visibleOnlyOnPluginSyncEnabled = visibleOnly(() => this.isConfiguredAs("usePluginSync", true)); const visibleOnlyOnPluginSyncEnabled = visibleOnly(() => this.isConfiguredAs("usePluginSync", true));
this.createEl(paneEl, "div", {
text: "Please set device name to identify this device. This name should be unique among your devices. While not configured, we cannot enable this feature.",
cls: "op-warn"
}, c => {
}, visibleOnly(() => this.isConfiguredAs("deviceAndVaultName", "")));
this.createEl(paneEl, "div", {
text: "We cannot change the device name while this feature is enabled. Please disable this feature to change the device name.",
cls: "op-warn-info"
}, c => {
}, visibleOnly(() => this.isConfiguredAs("usePluginSync", true)));
new Setting(paneEl) new Setting(paneEl)
.autoWireText("deviceAndVaultName", { .autoWireText("deviceAndVaultName", {
placeHolder: "desktop", onUpdate: enableOnlyOnPluginSyncIsNotEnabled placeHolder: "desktop", onUpdate: enableOnlyOnPluginSyncIsNotEnabled
@@ -1856,11 +1869,12 @@ ${stringifyYaml(pluginConfig)}`;
void addPanel(paneEl, "Recovery and Repair").then((paneEl) => { void addPanel(paneEl, "Recovery and Repair").then((paneEl) => {
const addResult = (path: string, file: TFile | false, fileOnDB: LoadedEntry | false) => { const addResult = async (path: string, file: FilePathWithPrefix | false, fileOnDB: LoadedEntry | false) => {
const storageFileStat = file ? await this.plugin.storageAccess.statHidden(file) : null;
resultArea.appendChild(this.createEl(resultArea, "div", {}, el => { resultArea.appendChild(this.createEl(resultArea, "div", {}, el => {
el.appendChild(this.createEl(el, "h6", { text: path })); el.appendChild(this.createEl(el, "h6", { text: path }));
el.appendChild(this.createEl(el, "div", {}, infoGroupEl => { el.appendChild(this.createEl(el, "div", {}, infoGroupEl => {
infoGroupEl.appendChild(this.createEl(infoGroupEl, "div", { text: `Storage : Modified: ${!file ? `Missing:` : `${new Date(file.stat.mtime).toLocaleString()}, Size:${file.stat.size}`}` })) infoGroupEl.appendChild(this.createEl(infoGroupEl, "div", { text: `Storage : Modified: ${!storageFileStat ? `Missing:` : `${new Date(storageFileStat.mtime).toLocaleString()}, Size:${storageFileStat.size}`}` }))
infoGroupEl.appendChild(this.createEl(infoGroupEl, "div", { text: `Database: Modified: ${!fileOnDB ? `Missing:` : `${new Date(fileOnDB.mtime).toLocaleString()}, Size:${fileOnDB.size}`}` })) infoGroupEl.appendChild(this.createEl(infoGroupEl, "div", { text: `Database: Modified: ${!fileOnDB ? `Missing:` : `${new Date(fileOnDB.mtime).toLocaleString()}, Size:${fileOnDB.size}`}` }))
})); }));
if (fileOnDB && file) { if (fileOnDB && file) {
@@ -1875,20 +1889,24 @@ ${stringifyYaml(pluginConfig)}`;
if (file) { if (file) {
el.appendChild(this.createEl(el, "button", { text: "Storage -> Database" }, buttonEl => { el.appendChild(this.createEl(el, "button", { text: "Storage -> Database" }, buttonEl => {
buttonEl.onClickEvent(async () => { buttonEl.onClickEvent(async () => {
// const file = this.plugin.storageAccess.getFileStub(path); if (file.startsWith(".")) {
// if (!file) { const addOn = this.plugin.getAddOn<HiddenFileSync>(HiddenFileSync.name);
// Logger(`File not found: ${path}`, LOG_LEVEL_NOTICE); if (addOn) {
// return; const file = (await addOn.scanInternalFiles()).find(e => e.path == path);
// } if (!file) {
// const content = await this.plugin.storageAccess.readStubContent(file); Logger(`Failed to find the file in the internal files: ${path}`, LOG_LEVEL_NOTICE);
// if (!content) { return;
// Logger(`Content cannot be read: ${path}`, LOG_LEVEL_NOTICE); }
// return; if (!await addOn.storeInternalFileToDatabase(file, true)) {
// } Logger(`Failed to store the file to the database (Hidden file): ${file}`, LOG_LEVEL_NOTICE);
// this.plugin.databaseFileAccess.store(content, true); return;
if (!await this.plugin.fileHandler.storeFileToDB(file.path as FilePath, true)) { }
Logger(`Failed to store the file to the database: ${file.path}`, LOG_LEVEL_NOTICE); }
return; } else {
if (!await this.plugin.fileHandler.storeFileToDB(file as FilePath, true)) {
Logger(`Failed to store the file to the database: ${file}`, LOG_LEVEL_NOTICE);
return;
}
} }
el.remove(); el.remove();
}) })
@@ -1897,10 +1915,20 @@ ${stringifyYaml(pluginConfig)}`;
if (fileOnDB) { if (fileOnDB) {
el.appendChild(this.createEl(el, "button", { text: "Database -> Storage" }, buttonEl => { el.appendChild(this.createEl(el, "button", { text: "Database -> Storage" }, buttonEl => {
buttonEl.onClickEvent(async () => { buttonEl.onClickEvent(async () => {
// this.plugin.pullFile(this.plugin.getPath(fileOnDB), undefined, true, undefined, false); if (fileOnDB.path.startsWith(ICHeader)) {
if (!await this.plugin.fileHandler.dbToStorage(fileOnDB as MetaEntry, null, true)) { const addOn = this.plugin.getAddOn<HiddenFileSync>(HiddenFileSync.name);
Logger(`Failed to store the file to the storage: ${fileOnDB.path}`, LOG_LEVEL_NOTICE); if (addOn) {
return; if (!await addOn.extractInternalFileFromDatabase(path as FilePath, true)) {
Logger(`Failed to store the file to the database (Hidden file): ${file}`, LOG_LEVEL_NOTICE);
return;
}
}
} else {
if (!await this.plugin.fileHandler.dbToStorage(fileOnDB as MetaEntry, null, true)) {
Logger(`Failed to store the file to the storage: ${fileOnDB.path}`, LOG_LEVEL_NOTICE);
return;
}
} }
el.remove(); el.remove();
}) })
@@ -1910,14 +1938,14 @@ ${stringifyYaml(pluginConfig)}`;
})) }))
} }
const checkBetweenStorageAndDatabase = async (file: TFile, fileOnDB: LoadedEntry) => { const checkBetweenStorageAndDatabase = async (file: FilePathWithPrefix, fileOnDB: LoadedEntry) => {
const dataContent = readAsBlob(fileOnDB); const dataContent = readAsBlob(fileOnDB);
const content = createBlob(await this.plugin.storageAccess.readFileAuto(file.path)) const content = createBlob(await this.plugin.storageAccess.readHiddenFileBinary(file))
if (await isDocContentSame(content, dataContent)) { if (await isDocContentSame(content, dataContent)) {
Logger(`Compare: SAME: ${file.path}`) Logger(`Compare: SAME: ${file}`)
} else { } else {
Logger(`Compare: CONTENT IS NOT MATCHED! ${file.path}`, LOG_LEVEL_NOTICE); Logger(`Compare: CONTENT IS NOT MATCHED! ${file}`, LOG_LEVEL_NOTICE);
addResult(file.path, file, fileOnDB) void addResult(file, file, fileOnDB)
} }
} }
new Setting(paneEl) new Setting(paneEl)
@@ -1945,17 +1973,26 @@ ${stringifyYaml(pluginConfig)}`;
.setDisabled(false) .setDisabled(false)
.setCta() .setCta()
.onClick(async () => { .onClick(async () => {
Logger("Start verifying all files", LOG_LEVEL_NOTICE, "verify");
const ignorePatterns = this.plugin.settings.syncInternalFilesIgnorePatterns
.replace(/\n| /g, "")
.split(",").filter(e => e).map(e => new RegExp(e, "i"));
this.plugin.localDatabase.hashCaches.clear(); this.plugin.localDatabase.hashCaches.clear();
Logger("Start verifying all files", LOG_LEVEL_NOTICE, "verify"); Logger("Start verifying all files", LOG_LEVEL_NOTICE, "verify");
const files = this.app.vault.getFiles(); const files = await this.plugin.storageAccess.getFilesIncludeHidden("/", undefined, ignorePatterns)
const documents = [] as FilePathWithPrefix[]; const documents = [] as FilePath[];
const adn = this.plugin.localDatabase.findAllNormalDocs() const adn = this.plugin.localDatabase.findAllDocs()
for await (const i of adn) documents.push(getPath(i)); for await (const i of adn) {
const path = getPath(i);
if (path.startsWith(ICXHeader)) continue;
if (path.startsWith(PSCHeader)) continue;
documents.push(stripAllPrefixes(path));
}
const allPaths = [ const allPaths = [
...new Set([ ...new Set([
...documents, ...documents,
...files.map(e => e.path as FilePathWithPrefix)])]; ...files])];
let i = 0; let i = 0;
const incProc = () => { const incProc = () => {
i++; i++;
@@ -1967,27 +2004,29 @@ ${stringifyYaml(pluginConfig)}`;
if (shouldBeIgnored(path)) { if (shouldBeIgnored(path)) {
return incProc(); return incProc();
} }
const abstractFile = this.plugin.storageAccess.getFileStub(path); const stat = await this.plugin.storageAccess.isExistsIncludeHidden(path) ? await this.plugin.storageAccess.statHidden(path) : false;
const fileOnStorage = abstractFile instanceof TFile ? abstractFile : false; const fileOnStorage = stat != null ? stat : false;
if (!await this.plugin.$$isTargetFile(path)) return incProc(); if (!await this.plugin.$$isTargetFile(path)) return incProc();
const releaser = await semaphore.acquire(1) const releaser = await semaphore.acquire(1)
if (fileOnStorage && this.plugin.$$isFileSizeExceeded(fileOnStorage.stat.size)) return incProc(); if (fileOnStorage && this.plugin.$$isFileSizeExceeded(fileOnStorage.size)) return incProc();
try { try {
const fileOnDB = await this.plugin.localDatabase.getDBEntry(path); const isHiddenFile = path.startsWith(".");
const dbPath = isHiddenFile ? addPrefix(path, ICHeader) : path;
const fileOnDB = await this.plugin.localDatabase.getDBEntry(dbPath);
if (fileOnDB && this.plugin.$$isFileSizeExceeded(fileOnDB.size)) return incProc(); if (fileOnDB && this.plugin.$$isFileSizeExceeded(fileOnDB.size)) return incProc();
if (!fileOnDB && fileOnStorage) { if (!fileOnDB && fileOnStorage) {
Logger(`Compare: Not found on the local database: ${path}`, LOG_LEVEL_NOTICE); Logger(`Compare: Not found on the local database: ${path}`, LOG_LEVEL_NOTICE);
addResult(path, fileOnStorage, false) void addResult(path, path, false)
return incProc(); return incProc();
} }
if (fileOnDB && !fileOnStorage) { if (fileOnDB && !fileOnStorage) {
Logger(`Compare: Not found on the storage: ${path}`, LOG_LEVEL_NOTICE); Logger(`Compare: Not found on the storage: ${path}`, LOG_LEVEL_NOTICE);
addResult(path, false, fileOnDB) void addResult(path, false, fileOnDB)
return incProc(); return incProc();
} }
if (fileOnStorage && fileOnDB) { if (fileOnStorage && fileOnDB) {
await checkBetweenStorageAndDatabase(fileOnStorage, fileOnDB) await checkBetweenStorageAndDatabase(path, fileOnDB)
} }
} catch (ex) { } catch (ex) {
Logger(`Error while processing ${path}`, LOG_LEVEL_NOTICE); Logger(`Error while processing ${path}`, LOG_LEVEL_NOTICE);

4
src/modules/features/SettingDialogue/settingConstants.ts
View File

@@ -327,8 +327,8 @@ export const SettingInformation: Partial<Record<keyof AllSettings, Configuration
desc: "MB (0 to disable)." desc: "MB (0 to disable)."
}, },
"usePluginSyncV2": { "usePluginSyncV2": {
name: "Enable per-file-saved customization sync", name: "Enable per-file customization sync",
desc: "If enabled per-filed efficient customization sync will be used. We need a small migration when enabling this. And all devices should be updated to v0.23.18. Once we enabled this, we lost a compatibility with old versions." desc: "If enabled, efficient per-file customization sync will be used. A minor migration is required when enabling this feature, and all devices must be updated to v0.23.18. Enabling this feature will result in losing compatibility with older versions."
}, },
"handleFilenameCaseSensitive": { "handleFilenameCaseSensitive": {
name: "Handle files as Case-Sensitive", name: "Handle files as Case-Sensitive",

66
updates.md
View File

@@ -1,38 +1,29 @@
## 0.24.0 RC Release Note ## 0.24.0
**Note:** This will be rewritten with the stable release. I confess, before you take the time, this is quite long. I know that we have been waiting for a long time. It is finally released!
Over the past three years since the inception of the plugin, various features have been implemented to address diverse user needs. This is so honourable and I am grateful for your years of support. Over the past three years since the inception of the plugin, various features have been implemented to address diverse user needs. This is truly honourable, and I am grateful for your years of support. However, this process has led to an increasingly disorganised codebase, with features becoming entangled. Consequently, this has led to a situation where bugs can go unnoticed and resolving one issue may inadvertently introduce another.
However, However, this process has resulted in a codebase that has become increasingly disorganised, with features becoming entangled.
Consequently, this has led to a situation where bugs can go unnoticed or resolving one issue may inadvertently introduce another. In 0.24.0, I reorganised the previously jumbled main codebase into clearly defined modules. Although I had assumed that the total size of the code would not increase, I discovered that it has in fact increased. While the complexity is still considerable, the refactoring has improved the clarity of the code's structure. Additionally, while testing the release candidates, we still found many bugs to fix, which helped to make this plug-in robust and stable. Therefore, we are now ready to use the updated plug-in, and in addition to that, proceed to the next step.
In 0.24.0, I reorganised the previously disjointed main codebase into clearly defined modules. Although I anticipated that the overall volume of code would not increase, I discovered that it has, in fact, expanded. While the complexity may still be considerable, the refactoring has enhanced clarity regarding the current structure of the code. (The next focus may involve a review of dependencies). This is also the first step towards a fully-fledged-fancy LiveSync, not just a plug-in from Obsidian. Of course, it will still be a plug-in primarily and foremost, but this development marks a significant step towards the self-hosting concept.
Throughout this process, a significant number of bugs have been resolved. And it may be worth mentioning that these bugs may had given rise to other bugs. I kindly request that you verify whether your issues have been addressed. At least conflict resolution and related issues have improved significantly. Finally, I would like to once again express my respect and gratitude to all of you. My gratitude extends to all of the dev testers! Your contributions have certainly made the plug-in robust and stable!
It is also the first step towards a fully-fledged-fancy LiveSync, not just a plug-in from Obsidian. Of course, it will still be a plug-in as a first class and foremost, but this development marks a significant step towards the self-hosting concept.
This dev release is very close to the beta version that I had previously indicated would not be released. As a result, I have faced challenges in maintaining the main branch while working on this dev release. Regrettably, I have not been able to make any commits to the main branch in the last three weeks. Thus, the dev branch will remain reserved for major changes only.
The Release Candidate will be available for a few days and will only be officially released once users, including myself, have confirmed that there are no issues.
Finally, I would like to once again express my respect and gratitude to all of you once again. Thank you for your interest in the development version. Your contributions and dedication are greatly appreciated through testing.
Thank you, and I hope your troubles will be resolved! Thank you, and I hope your troubles will be resolved!
--- ---
## 0.24.0.dev-rc4 ## 0.24.0
### Improved ### Improved
- The welcome message is now more simple to encourage the use of the Setup-URI. - The welcome message is now more simple to encourage the use of the Setup-URI.
- And the secondary message is also simpler to guide users to Minimal Setup. - The secondary message is also simpler to guide users to Minimal Setup.
- But Setup-URI will be recommended again, due to its importance. - But Setup-URI will be recommended again, due to its importance.
- These dialogues contain a link to the documentation which can be clicked. - These dialogues contain a link to the documentation which can be clicked.
- The minimal setup is more minimal now. And, the setup is more user-friendly. - The minimal setup is more minimal now. And, the setup is more user-friendly.
- Now the Configuration of the remote database is checked more robust, but we can ignore the warning and proceed with the setup. - Now the Configuration of the remote database is checked more robustly, but we can ignore the warning and proceed with the setup.
- Before we are asked about each feature, we are asked if we want to use optional features in the first place. - Before we are asked about each feature, we are asked if we want to use optional features in the first place.
- This is to prevent the user from being overwhelmed by the features. - This is to prevent the user from being overwhelmed by the features.
- And made it clear that it is not recommended for new users. - And made it clear that it is not recommended for new users.
@@ -43,45 +34,34 @@ Thank you, and I hope your troubles will be resolved!
- Especially auto-closing dialogues are now explicitly labelled: `To stop the countdown, tap anywhere on the dialogue`. - Especially auto-closing dialogues are now explicitly labelled: `To stop the countdown, tap anywhere on the dialogue`.
- Now if the is plugin configured to ignore some events, we will get a chance to fix it, in addition to the warning. - Now if the is plugin configured to ignore some events, we will get a chance to fix it, in addition to the warning.
- And why that has happened is also explained in the dialogue. - And why that has happened is also explained in the dialogue.
- A note relating to device names has been added to Customisation Sync on the setting dialogue.
- We can verify and resolve also the hidden files now.
### Fixed ### Fixed
- We can resolve the conflict of the JSON file correctly now.
- Verifying files between the local database and storage is now working correctly.
- While restarting the plug-in, the shown dialogues will be automatically closed to avoid unexpected behaviour. - While restarting the plug-in, the shown dialogues will be automatically closed to avoid unexpected behaviour.
- Replicated documents that the local device has configured to ignore are now correctly ignored. - Replicated documents that the local device has configured to ignore are now correctly ignored.
- The chunks of the document on the local device during the first transfer will be created correctly. - The chunks of the document on the local device during the first transfer will be created correctly.
- And why we should create them is now explained in the dialogue. - And why we should create them is now explained in the dialogue.
- If optional features have been enabled in the wizard, `Enable advanced features` will be toggled correctly. - If optional features have been enabled in the wizard, `Enable advanced features` will be toggled correctly.
The hidden file sync is now working correctly.
- Now the deletion of hidden files is correctly synchronised.
- Customisation Sync is now working correctly together with hidden file sync.
- No longer database suffix is stored in the setting sharing markdown.
- A fair number of bugs have been fixed.
### Changed ### Changed
- Some default settings have been changed for easier new user experience. - Some default settings have been changed for an easier new user experience.
- Preventing the meaningless migration of the settings. - Preventing the meaningless migration of the settings.
### Tidied
- Commented-out codes have been gradually removed.
## 0.24.0.dev-rc3
### Fixed
- No longer Missing Translation Warning is shown in the console.
- Fixed the issue where some functions were not working properly (`_` started functions).
## 0.24.0.dev-rc2
### Fixed
- Some status icons is now shown correctly.
## 0.24.0-rc1
### Fixed
- A fair numbers of bugs have been fixed.
### Tiding ### Tiding
- The codebase has been reorganised into clearly defined modules. - The codebase has been reorganised into clearly defined modules.
- Commented-out codes have been gradually removed.
Older notes is in [updates_old.md](https://github.com/vrtmrz/obsidian-livesync/blob/main/updates_old.md).
Older notes are in [updates_old.md](https://github.com/vrtmrz/obsidian-livesync/blob/main/updates_old.md).