public/obsidian-livesync
1
0
Fork 0
mirror of https://github.com/vrtmrz/obsidian-livesync.git synced 2026-02-22 20:18:48 +00:00
Code Issues Packages Projects Releases Wiki Activity

Compare commits

base: public:0.20.3
Branches Tags
public:main public:beta public:snyk-upgrade-7fd31e6f474568014502e7205847f257 public:snyk-fix-90f3b71450f7543a1c61724d8afa0e71 public:refactor_services_2 public:refactor_services public:dev public:snyk-upgrade-fe9625385aa4be849c7157033fd2abb0 public:snyk-upgrade-99a31e70c822894595544b0c9813b11a public:svelteui public:disenchant public:snyk-fix-61f76ef0bce2a276df9552c2fc80a14f public:snyk-upgrade-2baa7938ba318464c3bad5c3fd5cbda5 public:snyk-upgrade-7af62a575eab20e8fb4e9565bf1adcfc public:snyk-upgrade-b32f281b31a81483f126b8fb6c54a957 public:snyk-upgrade-bd61b1103442d3faecc9fc726c34e9a4 public:snyk-upgrade-5b2d6584430c061b5187611a5d7fb64a public:snyk-fix-0e9fbbcb374fcb8d5e8cb027d8414fb7 public:snyk-upgrade-5242daf91e706d1ae500c9ea6545636e public:snyk-upgrade-70434a6822fbad83424135dfad9df909 public:snyk-upgrade-8fa01ccad0352125f1866d41e57d35e1 public:snyk-upgrade-39eb97b4655e70091f718a88fccd57c1 public:snyk-upgrade-2b99d54d19e207887e36c3071b796925 public:forprivacy public:snyk-upgrade-dad9088d0a333677e0157e3deef7fc38 public:snyk-upgrade-b9a76d74b3e5723759af5b3319aabb80 public:snyk-upgrade-7da6459d8ff5d293d80de9d80061d912 public:snyk-upgrade-6a3786fc4cbe394542e9d1b6058c46e7 public:snyk-upgrade-8fc9e62771dc41b0f23dc0efefb5c949
public:0.25.43-patched-9 public:0.25.43-patched-8 public:0.25.43-patched-7 public:0.25.43-patched-6 public:0.25.43-patched-5 public:0.25.43-patched-4 public:0.25.43-patched-3 public:0.25.43-patched-2 public:0.25.43-patched-1 public:0.25.43 public:0.25.42 public:0.25.41-patched-3 public:0.25.41-patched-2 public:0.25.41-patched-1 public:0.25.41 public:0.25.40 public:0.25.39 public:0.25.38 public:0.25.37 public:0.25.36 public:0.25.35 public:0.25.34 public:0.25.33 public:0.25.32 public:0.25.31 public:0.25.30 public:0.25.30-dev public:0.25.29 public:0.25.28 public:0.25.27 public:0.25.26 public:0.25.25 public:0.25.24 public:0.25.24.beta3 public:0.25.24.beta2 public:0.25.24.beta1 public:0.25.23 public:0.25.23.beta1 public:0.25.22 public:0.25.21 public:0.25.21.beta2 public:0.25.21.beta1 public:0.25.20 public:0.25.19 public:0.25.18 public:0.25.17 public:0.25.16 public:0.25.15 public:0.25.14 public:0.25.13 public:0.25.12 public:0.25.11 public:0.25.10 public:0.25.9 public:0.25.8 public:0.25.7 public:0.25.6 public:0.25.5 public:0.25.4 public:0.25.3 public:0.25.2 public:0.25.1 public:0.25.0 public:0.25.0-beta1 public:0.24.31 public:0.24.30 public:0.24.29 public:0.24.28 public:0.24.27 public:0.24.26 public:0.24.25 public:0.24.24 public:0.24.23 public:0.24.22 public:0.24.21 public:0.24.20 public:0.24.19 public:0.24.18 public:0.24.17 public:0.24.16 public:0.24.15 public:0.24.14 public:0.24.13 public:0.24.12 public:0.24.11 public:0.24.10 public:0.24.9 public:0.24.8 public:0.24.7 public:0.24.6 public:0.24.5 public:0.24.4 public:0.24.3 public:0.24.2 public:0.24.1 public:0.24.0 public:0.24.0.dev-rc8 public:0.24.0.dev-rc7 public:0.24.0.dev-rc6 public:0.24.0.dev-rc5 public:0.24.0.dev-rc4 public:0.24.0.dev-rc3 public:0.24.0.dev-rc2 public:0.23.23 public:0.23.22 public:0.23.21 public:0.23.20 public:0.23.19 public:0.23.18 public:0.23.17 public:0.23.16 public:0.23.15 public:0.23.14 public:0.23.13 public:0.23.12 public:0.23.11 public:0.23.10 public:0.23.9 public:0.23.8 public:0.23.7 public:0.23.6 public:0.23.5 public:0.23.4 public:0.23.3 public:0.23.2 public:0.23.1 public:0.23.0 public:0.22.19 public:0.22.18 public:0.22.17 public:0.22.16 public:0.22.15 public:0.22.14 public:0.22.13 public:0.22.12 public:0.22.11 public:0.22.10 public:0.22.9 public:0.22.8 public:0.22.7 public:0.22.6 public:0.22.5 public:0.22.4 public:0.22.3 public:0.22.2 public:0.22.1 public:0.22.0 public:0.21.5 public:0.21.4 public:0.21.3 public:0.21.2 public:0.21.1 public:0.21.0 public:0.20.7 public:0.20.6 public:0.20.5 public:0.20.4 public:0.20.3 public:0.20.2 public:0.20.1 public:0.20.0 public:0.19.23 public:0.19.22 public:0.19.21 public:0.19.20 public:0.19.19 public:0.19.18 public:0.19.17 public:0.19.16 public:0.19.15 public:0.19.14 public:0.19.13 public:0.19.12 public:0.19.11 public:0.19.10 public:0.19.9 public:0.19.8 public:0.19.7 public:0.19.6 public:0.19.5 public:0.19.4 public:0.19.3 public:0.19.2 public:0.19.1 public:0.19.0 public:0.18.6 public:0.18.5 public:0.18.4 public:0.18.3 public:0.18.2 public:0.18.1 public:0.18.0 public:0.17.34 public:0.17.33 public:0.17.32 public:0.17.31 public:0.17.30 public:0.17.29 public:0.17.28 public:0.17.27 public:0.17.26 public:0.17.25 public:0.17.24 public:0.17.23 public:0.17.22 public:0.17.21 public:0.17.20 public:0.17.19 public:0.17.18 public:0.17.17 public:0.17.16 public:0.17.15 public:0.17.14 public:0.17.13 public:0.17.12 public:0.17.11 public:0.17.10 public:0.17.9 public:0.17.8 public:0.17.7 public:0.17.6 public:0.17.5 public:0.17.4 public:0.17.3 public:0.17.2 public:0.17.1 public:0.17.0 public:0.16.8 public:0.16.7 public:0.16.6 public:0.16.5 public:0.16.4 public:0.16.3 public:0.16.2 public:0.16.1 public:0.16.0 public:0.15.10 public:0.15.9 public:0.15.8 public:0.15.7 public:0.15.6 public:0.15.5 public:0.15.4 public:0.15.3 public:0.15.2 public:0.15.1 public:0.15.0 public:0.14.8 public:0.14.7 public:0.14.6 public:0.14.5 public:0.14.4 public:0.14.3 public:0.14.2 public:0.14.1 public:0.14.0 public:0.13.4 public:0.13.3 public:0.13.2 public:0.13.1 public:0.13.0 public:0.12.5 public:0.12.4 public:0.12.3 public:0.12.2 public:0.12.1 public:0.12.0 public:0.11.10 public:0.11.9 public:0.11.8 public:0.11.7 public:0.11.6 public:0.11.5 public:0.11.4 public:0.11.3 public:0.11.2 public:0.11.1 public:0.11.0 public:0.10.1 public:0.10.0 public:0.9.0 public:0.8.8 public:0.8.7 public:0.8.6 public:0.8.5 public:0.8.3 public:0.8.2 public:0.8.1 public:0.8.0 public:0.7.2 public:0.7.1 public:0.7.0 public:0.6.1 public:0.6.0 public:0.5.0 public:0.4.1 public:0.4.0 public:0.3.1 public:0.3.0 public:0.2.1 public:0.2.0 public:0.1.26 public:0.1.25 public:0.1.24 public:0.1.23 public:0.1.22 public:0.1.21 public:0.1.20 public:0.1.19 public:0.1.18 public:0.1.17 public:0.1.16 public:0.1.15 public:0.1.14 public:0.1.13 public:0.1.12 public:0.1.11 public:0.1.10 public:0.1.9 public:0.1.8 public:0.1.7 public:0.1.6 public:0.1.5 public:0.1.4 public:0.1.3 public:0.1.2 public:0.1.1 public:0.1.0 public:0.0.9 public:0.0.8 public:0.0.7 public:0.0.6
...
compare: public:0.22.3
Branches Tags
public:beta public:main public:snyk-upgrade-7fd31e6f474568014502e7205847f257 public:snyk-fix-90f3b71450f7543a1c61724d8afa0e71 public:refactor_services_2 public:refactor_services public:dev public:snyk-upgrade-fe9625385aa4be849c7157033fd2abb0 public:snyk-upgrade-99a31e70c822894595544b0c9813b11a public:svelteui public:disenchant public:snyk-fix-61f76ef0bce2a276df9552c2fc80a14f public:snyk-upgrade-2baa7938ba318464c3bad5c3fd5cbda5 public:snyk-upgrade-7af62a575eab20e8fb4e9565bf1adcfc public:snyk-upgrade-b32f281b31a81483f126b8fb6c54a957 public:snyk-upgrade-bd61b1103442d3faecc9fc726c34e9a4 public:snyk-upgrade-5b2d6584430c061b5187611a5d7fb64a public:snyk-fix-0e9fbbcb374fcb8d5e8cb027d8414fb7 public:snyk-upgrade-5242daf91e706d1ae500c9ea6545636e public:snyk-upgrade-70434a6822fbad83424135dfad9df909 public:snyk-upgrade-8fa01ccad0352125f1866d41e57d35e1 public:snyk-upgrade-39eb97b4655e70091f718a88fccd57c1 public:snyk-upgrade-2b99d54d19e207887e36c3071b796925 public:forprivacy public:snyk-upgrade-dad9088d0a333677e0157e3deef7fc38 public:snyk-upgrade-b9a76d74b3e5723759af5b3319aabb80 public:snyk-upgrade-7da6459d8ff5d293d80de9d80061d912 public:snyk-upgrade-6a3786fc4cbe394542e9d1b6058c46e7 public:snyk-upgrade-8fc9e62771dc41b0f23dc0efefb5c949
public:0.25.43-patched-9 public:0.25.43-patched-8 public:0.25.43-patched-7 public:0.25.43-patched-6 public:0.25.43-patched-5 public:0.25.43-patched-4 public:0.25.43-patched-3 public:0.25.43-patched-2 public:0.25.43-patched-1 public:0.25.43 public:0.25.42 public:0.25.41-patched-3 public:0.25.41-patched-2 public:0.25.41-patched-1 public:0.25.41 public:0.25.40 public:0.25.39 public:0.25.38 public:0.25.37 public:0.25.36 public:0.25.35 public:0.25.34 public:0.25.33 public:0.25.32 public:0.25.31 public:0.25.30 public:0.25.30-dev public:0.25.29 public:0.25.28 public:0.25.27 public:0.25.26 public:0.25.25 public:0.25.24 public:0.25.24.beta3 public:0.25.24.beta2 public:0.25.24.beta1 public:0.25.23 public:0.25.23.beta1 public:0.25.22 public:0.25.21 public:0.25.21.beta2 public:0.25.21.beta1 public:0.25.20 public:0.25.19 public:0.25.18 public:0.25.17 public:0.25.16 public:0.25.15 public:0.25.14 public:0.25.13 public:0.25.12 public:0.25.11 public:0.25.10 public:0.25.9 public:0.25.8 public:0.25.7 public:0.25.6 public:0.25.5 public:0.25.4 public:0.25.3 public:0.25.2 public:0.25.1 public:0.25.0 public:0.25.0-beta1 public:0.24.31 public:0.24.30 public:0.24.29 public:0.24.28 public:0.24.27 public:0.24.26 public:0.24.25 public:0.24.24 public:0.24.23 public:0.24.22 public:0.24.21 public:0.24.20 public:0.24.19 public:0.24.18 public:0.24.17 public:0.24.16 public:0.24.15 public:0.24.14 public:0.24.13 public:0.24.12 public:0.24.11 public:0.24.10 public:0.24.9 public:0.24.8 public:0.24.7 public:0.24.6 public:0.24.5 public:0.24.4 public:0.24.3 public:0.24.2 public:0.24.1 public:0.24.0 public:0.24.0.dev-rc8 public:0.24.0.dev-rc7 public:0.24.0.dev-rc6 public:0.24.0.dev-rc5 public:0.24.0.dev-rc4 public:0.24.0.dev-rc3 public:0.24.0.dev-rc2 public:0.23.23 public:0.23.22 public:0.23.21 public:0.23.20 public:0.23.19 public:0.23.18 public:0.23.17 public:0.23.16 public:0.23.15 public:0.23.14 public:0.23.13 public:0.23.12 public:0.23.11 public:0.23.10 public:0.23.9 public:0.23.8 public:0.23.7 public:0.23.6 public:0.23.5 public:0.23.4 public:0.23.3 public:0.23.2 public:0.23.1 public:0.23.0 public:0.22.19 public:0.22.18 public:0.22.17 public:0.22.16 public:0.22.15 public:0.22.14 public:0.22.13 public:0.22.12 public:0.22.11 public:0.22.10 public:0.22.9 public:0.22.8 public:0.22.7 public:0.22.6 public:0.22.5 public:0.22.4 public:0.22.3 public:0.22.2 public:0.22.1 public:0.22.0 public:0.21.5 public:0.21.4 public:0.21.3 public:0.21.2 public:0.21.1 public:0.21.0 public:0.20.7 public:0.20.6 public:0.20.5 public:0.20.4 public:0.20.3 public:0.20.2 public:0.20.1 public:0.20.0 public:0.19.23 public:0.19.22 public:0.19.21 public:0.19.20 public:0.19.19 public:0.19.18 public:0.19.17 public:0.19.16 public:0.19.15 public:0.19.14 public:0.19.13 public:0.19.12 public:0.19.11 public:0.19.10 public:0.19.9 public:0.19.8 public:0.19.7 public:0.19.6 public:0.19.5 public:0.19.4 public:0.19.3 public:0.19.2 public:0.19.1 public:0.19.0 public:0.18.6 public:0.18.5 public:0.18.4 public:0.18.3 public:0.18.2 public:0.18.1 public:0.18.0 public:0.17.34 public:0.17.33 public:0.17.32 public:0.17.31 public:0.17.30 public:0.17.29 public:0.17.28 public:0.17.27 public:0.17.26 public:0.17.25 public:0.17.24 public:0.17.23 public:0.17.22 public:0.17.21 public:0.17.20 public:0.17.19 public:0.17.18 public:0.17.17 public:0.17.16 public:0.17.15 public:0.17.14 public:0.17.13 public:0.17.12 public:0.17.11 public:0.17.10 public:0.17.9 public:0.17.8 public:0.17.7 public:0.17.6 public:0.17.5 public:0.17.4 public:0.17.3 public:0.17.2 public:0.17.1 public:0.17.0 public:0.16.8 public:0.16.7 public:0.16.6 public:0.16.5 public:0.16.4 public:0.16.3 public:0.16.2 public:0.16.1 public:0.16.0 public:0.15.10 public:0.15.9 public:0.15.8 public:0.15.7 public:0.15.6 public:0.15.5 public:0.15.4 public:0.15.3 public:0.15.2 public:0.15.1 public:0.15.0 public:0.14.8 public:0.14.7 public:0.14.6 public:0.14.5 public:0.14.4 public:0.14.3 public:0.14.2 public:0.14.1 public:0.14.0 public:0.13.4 public:0.13.3 public:0.13.2 public:0.13.1 public:0.13.0 public:0.12.5 public:0.12.4 public:0.12.3 public:0.12.2 public:0.12.1 public:0.12.0 public:0.11.10 public:0.11.9 public:0.11.8 public:0.11.7 public:0.11.6 public:0.11.5 public:0.11.4 public:0.11.3 public:0.11.2 public:0.11.1 public:0.11.0 public:0.10.1 public:0.10.0 public:0.9.0 public:0.8.8 public:0.8.7 public:0.8.6 public:0.8.5 public:0.8.3 public:0.8.2 public:0.8.1 public:0.8.0 public:0.7.2 public:0.7.1 public:0.7.0 public:0.6.1 public:0.6.0 public:0.5.0 public:0.4.1 public:0.4.0 public:0.3.1 public:0.3.0 public:0.2.1 public:0.2.0 public:0.1.26 public:0.1.25 public:0.1.24 public:0.1.23 public:0.1.22 public:0.1.21 public:0.1.20 public:0.1.19 public:0.1.18 public:0.1.17 public:0.1.16 public:0.1.15 public:0.1.14 public:0.1.13 public:0.1.12 public:0.1.11 public:0.1.10 public:0.1.9 public:0.1.8 public:0.1.7 public:0.1.6 public:0.1.5 public:0.1.4 public:0.1.3 public:0.1.2 public:0.1.1 public:0.1.0 public:0.0.9 public:0.0.8 public:0.0.7 public:0.0.6

37 Commits
0.20.3 ... 0.22.3

Author SHA1 Message Date
vorotamoroz
93e7cbb133 bump. 2024-01-29 08:41:03 +00:00
vorotamoroz
716ae32e02 Fixed: 
- Deletion of files is now reliably synchronised.
 2024-01-29 08:40:41 +00:00
vorotamoroz
d6d8cbcf5a bump 2024-01-29 07:57:02 +00:00
vorotamoroz
efd348b266 Fixed: 
- No longer detects storage changes which have been caused by Self-hosted LiveSync itself.
- Setting sync file will be detected only if it has been configured now.
  - And its log will be shown only while the verbose log is enabled.
- Customisation file enumeration has got less blingy.
Fixed and improved:
- In-editor-status is now shown in the following areas:
  - Note editing pane (Source mode and live-preview mode).
  - New tab pane.
  - Canvas pane.
 2024-01-29 07:56:02 +00:00
vorotamoroz
8969b1800a bump 2024-01-24 08:53:00 +00:00
vorotamoroz
2c8e026e29 Fixed: 
- Now the results of resolving conflicts are surely synchronised.
Modified:
- Some setting items got new clear names.
New feature:
- We can limit the synchronising files by their size.
- Now the settings could be stored in a specific markdown file to synchronise or switch it
- Customisation of the obsoleted device is now able to be deleted at once.
 2024-01-24 08:52:47 +00:00
vorotamoroz
a6c27eab3d Merge pull request #367 from calvinbui/patch-1 
Skip workspace-mobile.json for cross-platform sync
 2024-01-24 15:55:50 +09:00
vorotamoroz
9b5c57d540 Merge pull request #336 from toon159/patch-1 
Lower payload size limit and batch limit from 10 to 2
 2024-01-22 13:03:29 +09:00
Calvin Bui
c251c596e8 Skip workspace-mobile.json for cross-platform sync 2024-01-18 13:23:48 +11:00
vorotamoroz
61188cfaef bump 2024-01-16 08:36:37 +00:00
vorotamoroz
97d944fd75 New feature: 
- We can perform automatic conflict resolution for inactive files, and postpone only manual ones by `Postpone manual resolution of inactive files`.
- Now we can see the image in the document history dialogue.
  - We can see the difference of the image, in the document history dialogue.
	- And also we can highlight differences.

Improved:
- Hidden file sync has been stabilised.
- Now automatically reloads the conflict-resolution dialogue when new conflicted revisions have arrived.

Fixed:
- No longer periodic process runs after unloading the plug-in.
- Now the modification of binary files is surely stored in the storage.
 2024-01-16 08:32:43 +00:00
vorotamoroz
d3dc1e7328 Minor fix and refine the readme 2024-01-12 10:29:18 +00:00
vorotamoroz
45304af369 bump 2024-01-12 09:38:57 +00:00
vorotamoroz
7f422d58f2 - Refined: 
- Task scheduling logics has been rewritten.
  - Possibly many bugs and fragile behaviour has been fixed
- Fixed:
  - Remote-chunk-fetching now works with keeping request intervals
- New feature:
  - We can show only the icons in the editor.
 2024-01-12 09:36:49 +00:00
vorotamoroz
c2491fdfad bump 2023-12-11 12:55:01 +09:00
vorotamoroz
06a6e391e8 Fixed for change detection bug. 2023-12-11 12:53:50 +09:00
vorotamoroz
f99475f6b7 bump 2023-12-11 12:46:23 +09:00
vorotamoroz
109fc00b9d Fixed 
- Now ID of the documents is shown in the log with the first 8 letters.
 2023-12-11 12:45:40 +09:00
vorotamoroz
c071d822e1 - Improved: 
- Now all revisions will be shown only its first a few letters.
- Fixed:
  - Check before modifying files has been implemented.
  - Content change detection has been improved.
 2023-12-11 12:22:17 +09:00
vorotamoroz
d2de5b4710 bump 2023-12-04 19:39:47 +09:00
vorotamoroz
cf5ecd8922 Implemented: 
- Now we can use SHA1 for hash function as fallback.
 2023-12-04 19:39:04 +09:00
vorotamoroz
b337a05b5a bump 2023-11-27 07:13:15 +00:00
vorotamoroz
9ea6bee9d1 - Fixed: 
- No longer files are broken while rebuilding.
    - Now, Large binary files can be written correctly on a mobile platform.
    - Any decoding errors now make zero-byte files.
  - Modified:
    - All files are processed sequentially for each.
 2023-11-27 06:55:55 +00:00
vorotamoroz
9747c26d50 bump 2023-11-25 02:22:26 +09:00
vorotamoroz
bb4b764586 - Fixed: 
- No more infinity loops on larger files.
    - Show message on decode error.
  - Refactored:
    - Fixed to avoid obsolete global variables.
 2023-11-25 02:21:44 +09:00
vorotamoroz
279b4b41e5 bump 2023-11-24 10:32:46 +00:00
vorotamoroz
b644fb791d - Changes and performance improvements: 
- Now the saving files are processed by Blob.
    - The V2-Format has been reverted.
    - New encoding format has been enabled in default.
    - WARNING: Since this version, the compatibilities with older Filesystem LiveSync have been lost.
 2023-11-24 10:31:58 +00:00
Vichaya Raksakunpanich
5802ed31be Update ObsidianLiveSyncSettingTab.ts 
Lower payload size limit and batch limit to 2 due to IBM Cloudant read/write limitation. Hope it will fix the "Replication error".
 2023-11-24 16:25:16 +07:00
vorotamoroz
ac9428e96b Fixed 
- To better replication, path obfuscation is now deterministic even if with E2EE.
 2023-11-15 08:44:03 +00:00
vorotamoroz
280d9e1dd9 Fixed: Fixed the issue of TOML editing. 2023-11-07 01:07:58 +00:00
vorotamoroz
f7209e566c bump 2023-10-24 10:07:29 +01:00
vorotamoroz
4a9ab2d1de Fixed: 
- No longer enumerating file names is broken.
 2023-10-24 10:07:17 +01:00
vorotamoroz
cb74b5ee93 - Fixed 
- Now empty file could be decoded.
    - Local files are no longer pre-saved before fetching from a remote database.
    - No longer deadlock while applying customisation sync.
    - Configuration with multiple files is now able to be applied correctly.
    - Deleting folder propagation now works without enabling the use of a trash bin.
 2023-10-24 09:54:56 +01:00
vorotamoroz
60eecd7001 bump 2023-10-17 12:00:59 +09:00
vorotamoroz
4bd7b54bcd Fixed: 
- Now the files which having digit or character prefixes in the path will not be ignored.
 2023-10-17 12:00:19 +09:00
vorotamoroz
8923c73d1b bump 2023-10-14 23:08:34 +09:00
vorotamoroz
11e64b13e2 The text-input-dialogue is no longer broken. 2023-10-14 23:07:51 +09:00
29 changed files with 1918 additions and 1323 deletions
Expand all files Collapse all files

25
README.md
View File

@@ -59,14 +59,23 @@ Synchronization status is shown in statusbar.
- Status
- ⏹️ Stopped
- 💤 LiveSync enabled. Waiting for changes.
- ⚡️ Synchronization in progress.
- ⚠ An error occurred.
- ↑ Uploaded chunks and metadata
- ↓ Downloaded chunks and metadata
- ⏳ Number of pending processes
- 🧩 Number of files waiting for their chunks.
If you have deleted or renamed files, please wait until ⏳ icon disappears.
- 💤 LiveSync enabled. Waiting for changes
- ⚡️ Synchronization in progress
- ⚠ An error occurred
- Statistical indicator
- ↑ Uploaded chunks and metadata
- ↓ Downloaded chunks and metadata
- Progress indicator
- 📥 Unprocessed transferred items
- 📄 Working database operation
- 💾 Working write storage processes
- ⏳ Working read storage processes
- 🛫 Pending read storage processes
- ⚙️ Working or pending storage processes of hidden files
- 🧩 Waiting chunks
- 🔌 Working Customisation items (Configuration, snippets and plug-ins)
To prevent file and database corruption, please wait until all progress indicators have disappeared. Especially in case of if you have deleted or renamed files.
## Hints

204
deploy_couchdb_to_flyio_v2_with_swap.ipynb
View File

@@ -3,8 +3,8 @@
{
"cell_type": "markdown",
"metadata": {
"id": "view-in-github",
"colab_type": "text"
"colab_type": "text",
"id": "view-in-github"
},
"source": [
"<a href=\"https://colab.research.google.com/gist/vrtmrz/37c3efd7842e49947aaaa7f665e5020a/deploy_couchdb_to_flyio_v2_with_swap.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>"
@@ -12,15 +12,16 @@
},
{
"cell_type": "markdown",
"metadata": {
"id": "HiRV7G8Gk1Rs"
},
"source": [
"History:\n",
"- 18, May, 2023: Initial.\n",
"- 19, Jun., 2023: Patched for enabling swap.\n",
"- 22, Aug., 2023: Generating Setup-URI implemented."
],
"metadata": {
"id": "HiRV7G8Gk1Rs"
}
"- 22, Aug., 2023: Generating Setup-URI implemented.\n",
"- 7, Nov., 2023: Fixed the issue of TOML editing."
]
},
{
"cell_type": "code",
@@ -45,7 +46,7 @@
},
"outputs": [],
"source": [
"# Delete once\n",
"# Delete once (Do not care about `cannot remove './fly.toml': No such file or directory`)\n",
"!rm ./fly.toml"
]
},
@@ -78,15 +79,15 @@
},
{
"cell_type": "code",
"source": [
"# Check the toml once.\n",
"!cat fly.toml"
],
"execution_count": null,
"metadata": {
"id": "2RSoO9o-i2TT"
},
"execution_count": null,
"outputs": []
"outputs": [],
"source": [
"# Check the toml once.\n",
"!cat fly.toml"
]
},
{
"cell_type": "code",
@@ -96,52 +97,45 @@
},
"outputs": [],
"source": [
"# Modify fly.toml\n",
"## Port modification\n",
"!sed -i 's/8080/5984/g' fly.toml\n",
"## Add user into.\n",
"!echo -e \"\\n[env]\\n COUCHDB_USER = \\\"${couchUser}\\\"\" >> ./fly.toml\n",
"## Set the location of an ini file which to save configurations persistently via erlang flags.\n",
"!echo -e \"\\nERL_FLAGS=\\\"-couch_ini /opt/couchdb/etc/default.ini /opt/couchdb/etc/default.d/ /opt/couchdb/etc/local.d /opt/couchdb/etc/local.ini /opt/couchdb/data/persistence.ini\\\"\" >> ./fly.toml\n",
"## Mounting volumes to store data and ini file.\n",
"!echo -e \"\\n[mounts]\\n source=\\\"couchdata\\\"\\n destination=\\\"/opt/couchdb/data\\\"\" >> ./fly.toml\n",
"!cat fly.toml"
]
},
{
"cell_type": "code",
"source": [
"# Modify the TOML and generate Dockerfile\n",
"!pip install mergedeep\n",
"from mergedeep import merge\n",
"import toml\n",
"fly = toml.load('fly.toml')\n",
"override = {\n",
" \"http_service\":{\n",
" \"internal_port\":5984\n",
" },\n",
" \"build\":{\n",
" \"dockerfile\":\"./Dockerfile\"\n",
" },\n",
" \"mounts\":{\n",
" \"source\":\"couchdata\",\n",
" \"destination\":\"/opt/couchdb/data\"\n",
" },\n",
" \"env\":{\n",
" \"COUCHDB_USER\":os.environ['couchUser'],\n",
" \"ERL_FLAGS\":\"-couch_ini /opt/couchdb/etc/default.ini /opt/couchdb/etc/default.d/ /opt/couchdb/etc/local.d /opt/couchdb/etc/local.ini /opt/couchdb/data/persistence.ini\",\n",
" }\n",
"}\n",
"out = merge(fly,override)\n",
"with open('fly.toml', 'wt') as fp:\n",
" toml.dump(out, fp)\n",
" fp.close()\n",
"\n",
"# Make the Dockerfile to modify the permission of the ini file. If you want to use a specific version, you should change `latest` here.\n",
"!echo -e \"\\n[build]\\n dockerfile = \\\"./Dockerfile\\\"\" >> ./fly.toml"
],
"metadata": {
"id": "LQPsZ_dYxkTu"
},
"execution_count": null,
"outputs": []
},
{
"cell_type": "code",
"source": [
"!echo -e \"FROM couchdb:latest\\nRUN sed -i '2itouch /opt/couchdb/data/persistence.ini && chmod +w /opt/couchdb/data/persistence.ini && fallocate -l 512M /swapfile && chmod 0600 /swapfile && mkswap /swapfile && echo 10 > /proc/sys/vm/swappiness && swapon /swapfile && echo 1 > /proc/sys/vm/overcommit_memory' /docker-entrypoint.sh\" > ./Dockerfile"
],
"metadata": {
"id": "44cBeGJ9on5i"
},
"execution_count": null,
"outputs": []
},
{
"cell_type": "code",
"source": [
"# Check dockerfile\n",
"!cat ./Dockerfile"
],
"metadata": {
"id": "ai2R3BbpxRSe"
},
"execution_count": null,
"outputs": []
"dockerfile = '''FROM couchdb:latest\n",
"RUN sed -i '2itouch /opt/couchdb/data/persistence.ini && chmod +w /opt/couchdb/data/persistence.ini && fallocate -l 512M /swapfile && chmod 0600 /swapfile && mkswap /swapfile && echo 10 > /proc/sys/vm/swappiness && swapon /swapfile && echo 1 > /proc/sys/vm/overcommit_memory' /docker-entrypoint.sh\n",
"'''\n",
"with open(\"./Dockerfile\",\"wt\") as fp:\n",
" fp.write(dockerfile)\n",
" fp.close()\n",
"\n",
"!echo ------\n",
"!cat fly.toml\n",
"!echo ------\n",
"!cat Dockerfile"
]
},
{
"cell_type": "code",
@@ -189,20 +183,27 @@
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "cGlSzVqlQG_z"
},
"outputs": [],
"source": [
"# Finish setting up the CouchDB\n",
"# Please repeat until the request is completed without error messages\n",
"# i.e., You have to redo this block while \"curl: (35) OpenSSL SSL_connect: Connection reset by peer in connection to xxxx\" is showing.\n",
"#\n",
"# Note: A few minutes might be required to be booted.\n",
"!curl -X POST \"${couchHost}/_cluster_setup\" -H \"Content-Type: application/json\" -d \"{\\\"action\\\":\\\"enable_single_node\\\",\\\"username\\\":\\\"${couchUser}\\\",\\\"password\\\":\\\"${couchPwd}\\\",\\\"bind_address\\\":\\\"0.0.0.0\\\",\\\"port\\\":5984,\\\"singlenode\\\":true}\" --user \"${couchUser}:${couchPwd}\""
],
"metadata": {
"id": "cGlSzVqlQG_z"
},
"execution_count": null,
"outputs": []
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "JePzrsHypY18"
},
"outputs": [],
"source": [
"# Please repeat until all lines are completed without error messages\n",
"!curl -X PUT \"${couchHost}/_node/nonode@nohost/_config/chttpd/require_valid_user\" -H \"Content-Type: application/json\" -d '\"true\"' --user \"${couchUser}:${couchPwd}\"\n",
@@ -214,28 +215,28 @@
"!curl -X PUT \"${couchHost}/_node/nonode@nohost/_config/couchdb/max_document_size\" -H \"Content-Type: application/json\" -d '\"50000000\"' --user \"${couchUser}:${couchPwd}\"\n",
"!curl -X PUT \"${couchHost}/_node/nonode@nohost/_config/cors/credentials\" -H \"Content-Type: application/json\" -d '\"true\"' --user \"${couchUser}:${couchPwd}\"\n",
"!curl -X PUT \"${couchHost}/_node/nonode@nohost/_config/cors/origins\" -H \"Content-Type: application/json\" -d '\"app://obsidian.md,capacitor://localhost,http://localhost\"' --user \"${couchUser}:${couchPwd}\""
],
"metadata": {
"id": "JePzrsHypY18"
},
"execution_count": null,
"outputs": []
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "YfSOomsoXbGS"
},
"source": [
"Now, our CouchDB has been surely installed and configured. Cheers!\n",
"\n",
"In the steps that follow, create a setup-URI.\n",
"\n",
"This URI could be imported directly into Self-hosted LiveSync, to configure the use of the CouchDB which we configured now."
],
"metadata": {
"id": "YfSOomsoXbGS"
}
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "416YncOqXdNn"
},
"outputs": [],
"source": [
"# Database config\n",
"import random, string\n",
@@ -250,39 +251,39 @@
"\n",
"print(\"Your database:\"+os.environ['database'])\n",
"print(\"Your passphrase:\"+os.environ['passphrase'])"
],
"metadata": {
"id": "416YncOqXdNn"
},
"execution_count": null,
"outputs": []
]
},
{
"cell_type": "code",
"source": [
"# Install deno for make setup uri\n",
"!curl -fsSL https://deno.land/x/install/install.sh | sh"
],
"execution_count": null,
"metadata": {
"id": "C4d7C0HAXgsr"
},
"execution_count": null,
"outputs": []
"outputs": [],
"source": [
"# Install deno for make setup uri\n",
"!curl -fsSL https://deno.land/x/install/install.sh | sh"
]
},
{
"cell_type": "code",
"source": [
"# Fetch module for encrypting a Setup URI\n",
"!curl -o encrypt.ts https://gist.githubusercontent.com/vrtmrz/f9d1d95ee2ca3afa1a924a2c6759b854/raw/d7a070d864a6f61403d8dc74208238d5741aeb5a/encrypt.ts"
],
"execution_count": null,
"metadata": {
"id": "hQL_Dx-PXise"
},
"execution_count": null,
"outputs": []
"outputs": [],
"source": [
"# Fetch module for encrypting a Setup URI\n",
"!curl -o encrypt.ts https://gist.githubusercontent.com/vrtmrz/f9d1d95ee2ca3afa1a924a2c6759b854/raw/d7a070d864a6f61403d8dc74208238d5741aeb5a/encrypt.ts"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "o0gX_thFXlIZ"
},
"outputs": [],
"source": [
"# Make buttons!\n",
"from IPython.display import HTML\n",
@@ -294,29 +295,24 @@
"else:\n",
" result = \"Failed to encrypt the setup URI\"\n",
"result"
],
"metadata": {
"id": "o0gX_thFXlIZ"
},
"execution_count": null,
"outputs": []
]
}
],
"metadata": {
"colab": {
"provenance": [],
"include_colab_link": true,
"private_outputs": true,
"include_colab_link": true
"provenance": []
},
"gpuClass": "standard",
"kernelspec": {
"display_name": "Python 3",
"name": "python3"
},
"language_info": {
"name": "python"
},
"gpuClass": "standard"
}
},
"nbformat": 4,
"nbformat_minor": 0
}
}

2
manifest.json
View File

@@ -1,7 +1,7 @@
{
"id": "obsidian-livesync",
"name": "Self-hosted LiveSync",
"version": "0.20.3",
"version": "0.22.3",
"minAppVersion": "0.9.12",
"description": "Community implementation of self-hosted livesync. Reflect your vault changes to some other devices immediately. Please make sure to disable other synchronize solutions to avoid content corruption or duplication.",
"author": "vorotamoroz",

18
package-lock.json generated
View File

@@ -1,12 +1,12 @@
{
"name": "obsidian-livesync",
"version": "0.20.3",
"version": "0.22.3",
"lockfileVersion": 2,
"requires": true,
"packages": {
"": {
"name": "obsidian-livesync",
"version": "0.20.3",
"version": "0.22.3",
"license": "MIT",
"dependencies": {
"diff-match-patch": "^1.0.5",
@@ -36,7 +36,7 @@
"eslint-config-airbnb-base": "^15.0.0",
"eslint-plugin-import": "^2.28.0",
"events": "^3.3.0",
"obsidian": "^1.3.5",
"obsidian": "^1.4.11",
"postcss": "^8.4.27",
"postcss-load-config": "^4.0.1",
"pouchdb-adapter-http": "^8.0.1",
@@ -3223,9 +3223,9 @@
}
},
"node_modules/obsidian": {
"version": "1.4.0",
"resolved": "https://registry.npmjs.org/obsidian/-/obsidian-1.4.0.tgz",
"integrity": "sha512-fsZMPlxgflGSBSP6P4BjQi5+0MqZl3h6FEDEZ3CNnweNdDw0doyqN3FMO/PGWfuxPT77WicVwUxekuI3e6eCGg==",
"version": "1.4.11",
"resolved": "https://registry.npmjs.org/obsidian/-/obsidian-1.4.11.tgz",
"integrity": "sha512-BCVYTvaXxElJMl6MMbDdY/CGK+aq18SdtDY/7vH8v6BxCBQ6KF4kKxL0vG9UZ0o5qh139KpUoJHNm+6O5dllKA==",
"dev": true,
"dependencies": {
"@types/codemirror": "5.60.8",
@@ -6913,9 +6913,9 @@
}
},
"obsidian": {
"version": "1.4.0",
"resolved": "https://registry.npmjs.org/obsidian/-/obsidian-1.4.0.tgz",
"integrity": "sha512-fsZMPlxgflGSBSP6P4BjQi5+0MqZl3h6FEDEZ3CNnweNdDw0doyqN3FMO/PGWfuxPT77WicVwUxekuI3e6eCGg==",
"version": "1.4.11",
"resolved": "https://registry.npmjs.org/obsidian/-/obsidian-1.4.11.tgz",
"integrity": "sha512-BCVYTvaXxElJMl6MMbDdY/CGK+aq18SdtDY/7vH8v6BxCBQ6KF4kKxL0vG9UZ0o5qh139KpUoJHNm+6O5dllKA==",
"dev": true,
"requires": {
"@types/codemirror": "5.60.8",

8
package.json
View File

@@ -1,6 +1,6 @@
{
"name": "obsidian-livesync",
"version": "0.20.3",
"version": "0.22.3",
"description": "Reflect your vault changes to some other devices immediately. Please make sure to disable other synchronize solutions to avoid content corruption or duplication.",
"main": "main.js",
"type": "module",
@@ -17,9 +17,9 @@
"@types/diff-match-patch": "^1.0.32",
"@types/node": "^20.2.5",
"@types/pouchdb": "^6.4.0",
"@types/pouchdb-browser": "^6.1.3",
"@types/pouchdb-adapter-http": "^6.1.3",
"@types/pouchdb-adapter-idb": "^6.1.4",
"@types/pouchdb-browser": "^6.1.3",
"@types/pouchdb-core": "^7.0.11",
"@types/pouchdb-mapreduce": "^6.1.7",
"@types/pouchdb-replication": "^6.4.4",
@@ -33,7 +33,7 @@
"eslint-config-airbnb-base": "^15.0.0",
"eslint-plugin-import": "^2.28.0",
"events": "^3.3.0",
"obsidian": "^1.3.5",
"obsidian": "^1.4.11",
"postcss": "^8.4.27",
"postcss-load-config": "^4.0.1",
"pouchdb-adapter-http": "^8.0.1",
@@ -60,4 +60,4 @@
"xxhash-wasm": "0.4.2",
"xxhash-wasm-102": "npm:xxhash-wasm@^1.0.2"
}
}
}

151
src/CmdConfigSync.ts
View File

@@ -1,20 +1,22 @@
import { writable } from 'svelte/store';
import { Notice, type PluginManifest, parseYaml, normalizePath } from "./deps";
import type { EntryDoc, LoadedEntry, InternalFileEntry, FilePathWithPrefix, FilePath, DocumentID, AnyEntry } from "./lib/src/types";
import type { EntryDoc, LoadedEntry, InternalFileEntry, FilePathWithPrefix, FilePath, DocumentID, AnyEntry, SavingEntry } from "./lib/src/types";
import { LOG_LEVEL_INFO, LOG_LEVEL_NOTICE, LOG_LEVEL_VERBOSE, MODE_SELECTIVE } from "./lib/src/types";
import { ICXHeader, PERIODIC_PLUGIN_SWEEP, } from "./types";
import { delay, getDocData } from "./lib/src/utils";
import { createTextBlob, delay, getDocData, sendSignal, waitForSignal } from "./lib/src/utils";
import { Logger } from "./lib/src/logger";
import { WrappedNotice } from "./lib/src/wrapper";
import { readString, crc32CKHash, decodeBinary, encodeBinary } from "./lib/src/strbin";
import { readString, decodeBinary, arrayBufferToBase64, sha1 } from "./lib/src/strbin";
import { serialized } from "./lib/src/lock";
import { LiveSyncCommands } from "./LiveSyncCommands";
import { stripAllPrefixes } from "./lib/src/path";
import { PeriodicProcessor, askYesNo, disposeMemoObject, memoIfNotExist, memoObject, retrieveMemoObject, scheduleTask } from "./utils";
import { PluginDialogModal } from "./dialogs";
import { JsonResolveModal } from "./JsonResolveModal";
import { pipeGeneratorToGenerator, processAllGeneratorTasksWithConcurrencyLimit } from './lib/src/task';
import { QueueProcessor } from './lib/src/processor';
import { pluginScanningCount } from './lib/src/stores';
import type ObsidianLiveSyncPlugin from './main';
const d = "\u200b";
const d2 = "\n";
@@ -111,6 +113,7 @@ function deserialize2(str: string): PluginDataEx {
data
}
)
tokens.nextLine();
} while (filename);
return result;
}
@@ -161,6 +164,16 @@ export type PluginDataEx = {
mtime: number,
};
export class ConfigSync extends LiveSyncCommands {
constructor(plugin: ObsidianLiveSyncPlugin) {
super(plugin);
pluginScanningCount.onChanged((e) => {
const total = e.value;
pluginIsEnumerating.set(total != 0);
if (total == 0) {
Logger(`Processing configurations done`, LOG_LEVEL_INFO, "get-plugins");
}
})
}
confirmPopup: WrappedNotice = null;
get kvDB() {
return this.plugin.kvDB;
@@ -269,7 +282,7 @@ export class ConfigSync extends LiveSyncCommands {
for (const file of data.files) {
const work = { ...file };
const tempStr = getDocData(work.data);
work.data = [crc32CKHash(tempStr)];
work.data = [await sha1(tempStr)];
xFiles.push(work);
}
return ({
@@ -301,63 +314,71 @@ export class ConfigSync extends LiveSyncCommands {
this.plugin.saveSettingData();
}
}
pluginScanProcessor = new QueueProcessor(async (v: AnyEntry[]) => {
const plugin = v[0];
const path = plugin.path || this.getPath(plugin);
const oldEntry = (this.pluginList.find(e => e.documentPath == path));
if (oldEntry && oldEntry.mtime == plugin.mtime) return;
try {
const pluginData = await this.loadPluginData(path);
if (pluginData) {
return [pluginData];
}
// Failed to load
return;
} catch (ex) {
Logger(`Something happened at enumerating customization :${path}`, LOG_LEVEL_NOTICE);
Logger(ex, LOG_LEVEL_VERBOSE);
}
return;
}, { suspended: true, batchSize: 1, concurrentLimit: 5, delay: 300, yieldThreshold: 10 }).pipeTo(
new QueueProcessor(
async (pluginDataList) => {
// Concurrency is two, therefore, we can unlock the previous awaiting.
sendSignal("plugin-next-load");
let newList = [...this.pluginList];
for (const item of pluginDataList) {
newList = newList.filter(x => x.documentPath != item.documentPath);
newList.push(item)
}
this.pluginList = newList;
pluginList.set(newList);
if (pluginDataList.length != 10) {
// If the queue is going to be empty, await subsequent for a while.
await waitForSignal("plugin-next-load", 1000);
}
return;
}
, { suspended: true, batchSize: 10, concurrentLimit: 2, delay: 250, yieldThreshold: 25, totalRemainingReactiveSource: pluginScanningCount })).startPipeline().root.onIdle(() => {
Logger(`All files enumerated`, LOG_LEVEL_INFO, "get-plugins");
this.createMissingConfigurationEntry();
});
async updatePluginList(showMessage: boolean, updatedDocumentPath?: FilePathWithPrefix): Promise<void> {
const logLevel = showMessage ? LOG_LEVEL_NOTICE : LOG_LEVEL_INFO;
// pluginList.set([]);
if (!this.settings.usePluginSync) {
this.pluginScanProcessor.clearQueue();
this.pluginList = [];
pluginList.set(this.pluginList)
return;
}
await Promise.resolve(); // Just to prevent warning.
scheduleTask("update-plugin-list-task", 200, async () => {
await serialized("update-plugin-list", async () => {
try {
const updatedDocumentId = updatedDocumentPath ? await this.path2id(updatedDocumentPath) : "";
const plugins = updatedDocumentPath ?
this.localDatabase.findEntries(updatedDocumentId, updatedDocumentId + "\u{10ffff}", { include_docs: true, key: updatedDocumentId, limit: 1 }) :
this.localDatabase.findEntries(ICXHeader + "", `${ICXHeader}\u{10ffff}`, { include_docs: true });
let count = 0;
pluginIsEnumerating.set(true);
for await (const v of processAllGeneratorTasksWithConcurrencyLimit(20, pipeGeneratorToGenerator(plugins, async plugin => {
const path = plugin.path || this.getPath(plugin);
if (updatedDocumentPath && updatedDocumentPath != path) {
return false;
}
const oldEntry = (this.pluginList.find(e => e.documentPath == path));
if (oldEntry && oldEntry.mtime == plugin.mtime) return false;
try {
count++;
if (count % 10 == 0) Logger(`Enumerating files... ${count}`, logLevel, "get-plugins");
Logger(`plugin-${path}`, LOG_LEVEL_VERBOSE);
return this.loadPluginData(path);
// return entries;
} catch (ex) {
//TODO
Logger(`Something happened at enumerating customization :${path}`, LOG_LEVEL_NOTICE);
console.warn(ex);
}
return false;
}))) {
if ("ok" in v) {
if (v.ok !== false) {
let newList = [...this.pluginList];
const item = v.ok;
newList = newList.filter(x => x.documentPath != item.documentPath);
newList.push(item)
if (updatedDocumentPath != "") newList = newList.filter(e => e.documentPath != updatedDocumentPath);
this.pluginList = newList;
pluginList.set(newList);
}
}
}
Logger(`All files enumerated`, logLevel, "get-plugins");
this.createMissingConfigurationEntry();
} finally {
pluginIsEnumerating.set(false);
}
});
});
try {
const updatedDocumentId = updatedDocumentPath ? await this.path2id(updatedDocumentPath) : "";
const plugins = updatedDocumentPath ?
this.localDatabase.findEntries(updatedDocumentId, updatedDocumentId + "\u{10ffff}", { include_docs: true, key: updatedDocumentId, limit: 1 }) :
this.localDatabase.findEntries(ICXHeader + "", `${ICXHeader}\u{10ffff}`, { include_docs: true });
for await (const v of plugins) {
const path = v.path || this.getPath(v);
if (updatedDocumentPath && updatedDocumentPath != path) continue;
this.pluginScanProcessor.enqueue(v);
}
} finally {
pluginIsEnumerating.set(false);
}
pluginIsEnumerating.set(false);
// return entries;
}
async compareUsingDisplayData(dataA: PluginDataExDisplay, dataB: PluginDataExDisplay) {
@@ -415,9 +436,9 @@ export class ConfigSync extends LiveSyncCommands {
await this.ensureDirectoryEx(path);
if (!content) {
const dt = decodeBinary(f.data);
await this.app.vault.adapter.writeBinary(path, dt);
await this.vaultAccess.adapterWrite(path, dt);
} else {
await this.app.vault.adapter.write(path, content);
await this.vaultAccess.adapterWrite(path, content);
}
Logger(`Applying ${f.filename} of ${data.displayName || data.name}.. Done`);
@@ -537,16 +558,16 @@ export class ConfigSync extends LiveSyncCommands {
recentProcessedInternalFiles = [] as string[];
async makeEntryFromFile(path: FilePath): Promise<false | PluginDataExFile> {
const stat = await this.app.vault.adapter.stat(path);
const stat = await this.vaultAccess.adapterStat(path);
let version: string | undefined;
let displayName: string | undefined;
if (!stat) {
return false;
}
const contentBin = await this.app.vault.adapter.readBinary(path);
const contentBin = await this.vaultAccess.adapterReadBinary(path);
let content: string[];
try {
content = await encodeBinary(contentBin, this.settings.useV1);
content = await arrayBufferToBase64(contentBin);
if (path.toLowerCase().endsWith("/manifest.json")) {
const v = readString(new Uint8Array(contentBin));
try {
@@ -649,10 +670,10 @@ export class ConfigSync extends LiveSyncCommands {
return
}
const content = serialize(dt);
const content = createTextBlob(serialize(dt));
try {
const old = await this.localDatabase.getDBEntryMeta(prefixedFileName, null, false);
let saveData: LoadedEntry;
let saveData: SavingEntry;
if (old === false) {
saveData = {
_id: id,
@@ -661,7 +682,7 @@ export class ConfigSync extends LiveSyncCommands {
mtime,
ctime: mtime,
datatype: "newnote",
size: content.length,
size: content.size,
children: [],
deleted: false,
type: "newnote",
@@ -676,7 +697,7 @@ export class ConfigSync extends LiveSyncCommands {
...old,
data: content,
mtime,
size: content.length,
size: content.size,
datatype: "newnote",
children: [],
deleted: false,
@@ -698,7 +719,7 @@ export class ConfigSync extends LiveSyncCommands {
async watchVaultRawEventsAsync(path: FilePath) {
if (!this.settings.usePluginSync) return false;
if (!this.isTargetPath(path)) return false;
const stat = await this.app.vault.adapter.stat(path);
const stat = await this.vaultAccess.adapterStat(path);
// Make sure that target is a file.
if (stat && stat.type != "file")
return false;

175
src/CmdHiddenFileSync.ts
View File

@@ -1,16 +1,18 @@
import { normalizePath, type PluginManifest } from "./deps";
import { type EntryDoc, type LoadedEntry, type InternalFileEntry, type FilePathWithPrefix, type FilePath, LOG_LEVEL_INFO, LOG_LEVEL_NOTICE, LOG_LEVEL_VERBOSE, MODE_SELECTIVE, MODE_PAUSED } from "./lib/src/types";
import { type EntryDoc, type LoadedEntry, type InternalFileEntry, type FilePathWithPrefix, type FilePath, LOG_LEVEL_INFO, LOG_LEVEL_NOTICE, LOG_LEVEL_VERBOSE, MODE_SELECTIVE, MODE_PAUSED, type SavingEntry } from "./lib/src/types";
import { type InternalFileInfo, ICHeader, ICHeaderEnd } from "./types";
import { Parallels, delay, isDocContentSame } from "./lib/src/utils";
import { createBinaryBlob, isDocContentSame, sendSignal } from "./lib/src/utils";
import { Logger } from "./lib/src/logger";
import { PouchDB } from "./lib/src/pouchdb-browser.js";
import { scheduleTask, isInternalMetadata, PeriodicProcessor } from "./utils";
import { isInternalMetadata, PeriodicProcessor } from "./utils";
import { WrappedNotice } from "./lib/src/wrapper";
import { decodeBinary, encodeBinary } from "./lib/src/strbin";
import { serialized } from "./lib/src/lock";
import { JsonResolveModal } from "./JsonResolveModal";
import { LiveSyncCommands } from "./LiveSyncCommands";
import { addPrefix, stripAllPrefixes } from "./lib/src/path";
import { KeyedQueueProcessor, QueueProcessor } from "./lib/src/processor";
import { hiddenFilesEventCount, hiddenFilesProcessingCount } from "./lib/src/stores";
export class HiddenFileSync extends LiveSyncCommands {
periodicInternalFileScanProcessor: PeriodicProcessor = new PeriodicProcessor(this.plugin, async () => this.settings.syncInternalFiles && this.localDatabase.isReady && await this.syncInternalFilesAndDatabase("push", false));
@@ -75,22 +77,17 @@ export class HiddenFileSync extends LiveSyncCommands {
return;
}
procInternalFiles: string[] = [];
async execInternalFile() {
await serialized("execInternal", async () => {
const w = [...this.procInternalFiles];
this.procInternalFiles = [];
Logger(`Applying hidden ${w.length} files change...`);
await this.syncInternalFilesAndDatabase("pull", false, false, w);
Logger(`Applying hidden ${w.length} files changed`);
});
}
procInternalFile(filename: string) {
this.procInternalFiles.push(filename);
scheduleTask("procInternal", 500, async () => {
await this.execInternalFile();
});
this.internalFileProcessor.enqueueWithKey(filename, filename);
}
internalFileProcessor = new KeyedQueueProcessor<string, any>(
async (filenames) => {
Logger(`START :Applying hidden ${filenames.length} files change`, LOG_LEVEL_VERBOSE);
await this.syncInternalFilesAndDatabase("pull", false, false, filenames);
Logger(`DONE :Applying hidden ${filenames.length} files change`, LOG_LEVEL_VERBOSE);
return;
}, { batchSize: 100, concurrentLimit: 1, delay: 10, yieldThreshold: 10, suspended: false, totalRemainingReactiveSource: hiddenFilesEventCount }
);
recentProcessedInternalFiles = [] as string[];
async watchVaultRawEventsAsync(path: FilePath) {
@@ -103,7 +100,7 @@ export class HiddenFileSync extends LiveSyncCommands {
Logger(`Hidden file skipped: ${path} is synchronized in customization sync.`, LOG_LEVEL_VERBOSE);
return;
}
const stat = await this.app.vault.adapter.stat(path);
const stat = await this.vaultAccess.adapterStat(path);
// sometimes folder is coming.
if (stat && stat.type != "file")
return;
@@ -137,25 +134,34 @@ export class HiddenFileSync extends LiveSyncCommands {
async resolveConflictOnInternalFiles() {
// Scan all conflicted internal files
const conflicted = this.localDatabase.findEntries(ICHeader, ICHeaderEnd, { conflicts: true });
for await (const doc of conflicted) {
if (!("_conflicts" in doc))
continue;
if (isInternalMetadata(doc._id)) {
await this.resolveConflictOnInternalFile(doc.path);
this.conflictResolutionProcessor.suspend();
try {
for await (const doc of conflicted) {
if (!("_conflicts" in doc))
continue;
if (isInternalMetadata(doc._id)) {
this.conflictResolutionProcessor.enqueue(doc.path);
}
}
} catch (ex) {
Logger("something went wrong on resolving all conflicted internal files");
Logger(ex, LOG_LEVEL_VERBOSE);
}
await this.conflictResolutionProcessor.startPipeline().waitForPipeline();
}
async resolveConflictOnInternalFile(path: FilePathWithPrefix): Promise<boolean> {
conflictResolutionProcessor = new QueueProcessor(async (paths: FilePathWithPrefix[]) => {
const path = paths[0];
sendSignal(`cancel-internal-conflict:${path}`);
try {
// Retrieve data
const id = await this.path2id(path, ICHeader);
const doc = await this.localDatabase.getRaw(id, { conflicts: true });
// If there is no conflict, return with false.
if (!("_conflicts" in doc))
return false;
return;
if (doc._conflicts.length == 0)
return false;
return;
Logger(`Hidden file conflicted:${path}`);
const conflicts = doc._conflicts.sort((a, b) => Number(a.split("-")[0]) - Number(b.split("-")[0]));
const revA = doc._rev;
@@ -171,30 +177,21 @@ export class HiddenFileSync extends LiveSyncCommands {
if (result) {
Logger(`Object merge:${path}`, LOG_LEVEL_INFO);
const filename = stripAllPrefixes(path);
const isExists = await this.app.vault.adapter.exists(filename);
const isExists = await this.plugin.vaultAccess.adapterExists(filename);
if (!isExists) {
await this.ensureDirectoryEx(filename);
}
await this.app.vault.adapter.write(filename, result);
const stat = await this.app.vault.adapter.stat(filename);
await this.plugin.vaultAccess.adapterWrite(filename, result);
const stat = await this.vaultAccess.adapterStat(filename);
await this.storeInternalFileToDatabase({ path: filename, ...stat });
await this.extractInternalFileFromDatabase(filename);
await this.localDatabase.removeRaw(id, revB);
return this.resolveConflictOnInternalFile(path);
this.conflictResolutionProcessor.enqueue(path);
return;
} else {
Logger(`Object merge is not applicable.`, LOG_LEVEL_VERBOSE);
}
const docAMerge = await this.localDatabase.getDBEntry(path, { rev: revA });
const docBMerge = await this.localDatabase.getDBEntry(path, { rev: revB });
if (docAMerge != false && docBMerge != false) {
if (await this.showJSONMergeDialogAndMerge(docAMerge, docBMerge)) {
await delay(200);
// Again for other conflicted revisions.
return this.resolveConflictOnInternalFile(path);
}
return false;
}
return [{ path, revA, revB }];
}
const revBDoc = await this.localDatabase.getRaw(id, { rev: revB });
// determine which revision should been deleted.
@@ -208,12 +205,31 @@ export class HiddenFileSync extends LiveSyncCommands {
await this.localDatabase.removeRaw(id, delRev);
Logger(`Older one has been deleted:${path}`);
// check the file again
return this.resolveConflictOnInternalFile(path);
this.conflictResolutionProcessor.enqueue(path);
return;
} catch (ex) {
Logger(`Failed to resolve conflict (Hidden): ${path}`);
Logger(ex, LOG_LEVEL_VERBOSE);
return false;
return;
}
}, {
suspended: false, batchSize: 1, concurrentLimit: 5, delay: 10, keepResultUntilDownstreamConnected: true, yieldThreshold: 10,
pipeTo: new QueueProcessor(async (results) => {
const { path, revA, revB } = results[0]
const docAMerge = await this.localDatabase.getDBEntry(path, { rev: revA });
const docBMerge = await this.localDatabase.getDBEntry(path, { rev: revB });
if (docAMerge != false && docBMerge != false) {
if (await this.showJSONMergeDialogAndMerge(docAMerge, docBMerge)) {
// Again for other conflicted revisions.
this.conflictResolutionProcessor.enqueue(path);
}
return;
}
}, { suspended: false, batchSize: 1, concurrentLimit: 1, delay: 10, keepResultUntilDownstreamConnected: false, yieldThreshold: 10 })
})
queueConflictCheck(path: FilePathWithPrefix) {
this.conflictResolutionProcessor.enqueue(path);
}
//TODO: Tidy up. Even though it is experimental feature, So dirty...
@@ -278,28 +294,38 @@ export class HiddenFileSync extends LiveSyncCommands {
acc[stripAllPrefixes(this.getPath(cur))] = cur;
return acc;
}, {} as { [key: string]: InternalFileEntry; });
const para = Parallels();
for (const filename of allFileNames) {
await new QueueProcessor(async (filenames: FilePath[]) => {
const filename = filenames[0];
processed++;
if (processed % 100 == 0) {
Logger(`Hidden file: ${processed}/${fileCount}`, logLevel, "sync_internal");
}
if (!filename) continue;
if (!filename) return;
if (ignorePatterns.some(e => filename.match(e)))
continue;
return;
if (await this.plugin.isIgnoredByIgnoreFiles(filename)) {
continue;
return;
}
const fileOnStorage = filename in filesMap ? filesMap[filename] : undefined;
const fileOnDatabase = filename in filesOnDBMap ? filesOnDBMap[filename] : undefined;
const cache = filename in caches ? caches[filename] : { storageMtime: 0, docMtime: 0 };
await para.wait(5);
const proc = (async (xFileOnStorage: InternalFileInfo, xFileOnDatabase: InternalFileEntry) => {
return [{
filename,
fileOnStorage,
fileOnDatabase,
}]
}, { suspended: true, batchSize: 1, concurrentLimit: 10, delay: 0, totalRemainingReactiveSource: hiddenFilesProcessingCount })
.pipeTo(new QueueProcessor(async (params) => {
const
{
filename,
fileOnStorage: xFileOnStorage,
fileOnDatabase: xFileOnDatabase
} = params[0];
if (xFileOnStorage && xFileOnDatabase) {
const cache = filename in caches ? caches[filename] : { storageMtime: 0, docMtime: 0 };
// Both => Synchronize
if ((direction != "pullForce" && direction != "pushForce") && xFileOnDatabase.mtime == cache.docMtime && xFileOnStorage.mtime == cache.storageMtime) {
return;
@@ -340,11 +366,12 @@ export class HiddenFileSync extends LiveSyncCommands {
throw new Error("Invalid state on hidden file sync");
// Something corrupted?
}
return;
}, { suspended: true, batchSize: 1, concurrentLimit: 5, delay: 0 }))
.root
.enqueueAll(allFileNames)
.startPipeline().waitForPipeline();
});
para.add(proc(fileOnStorage, fileOnDatabase))
}
await para.all();
await this.kvDB.set("diff-caches-internal", caches);
// When files has been retrieved from the database. they must be reloaded.
@@ -408,10 +435,10 @@ export class HiddenFileSync extends LiveSyncCommands {
const id = await this.path2id(file.path, ICHeader);
const prefixedFileName = addPrefix(file.path, ICHeader);
const contentBin = await this.app.vault.adapter.readBinary(file.path);
let content: string[];
const contentBin = await this.plugin.vaultAccess.adapterReadBinary(file.path);
let content: Blob;
try {
content = await encodeBinary(contentBin, this.settings.useV1);
content = createBinaryBlob(contentBin);
} catch (ex) {
Logger(`The file ${file.path} could not be encoded`);
Logger(ex, LOG_LEVEL_VERBOSE);
@@ -421,7 +448,7 @@ export class HiddenFileSync extends LiveSyncCommands {
return await serialized("file-" + prefixedFileName, async () => {
try {
const old = await this.localDatabase.getDBEntry(prefixedFileName, null, false, false);
let saveData: LoadedEntry;
let saveData: SavingEntry;
if (old === false) {
saveData = {
_id: id,
@@ -436,7 +463,7 @@ export class HiddenFileSync extends LiveSyncCommands {
type: "newnote",
};
} else {
if (isDocContentSame(old.data, content) && !forceWrite) {
if (await isDocContentSame(old.data, content) && !forceWrite) {
// Logger(`STORAGE --> DB:${file.path}: (hidden) Not changed`, LOG_LEVEL_VERBOSE);
return;
}
@@ -511,7 +538,7 @@ export class HiddenFileSync extends LiveSyncCommands {
}
async extractInternalFileFromDatabase(filename: FilePath, force = false) {
const isExists = await this.app.vault.adapter.exists(filename);
const isExists = await this.plugin.vaultAccess.adapterExists(filename);
const prefixedFileName = addPrefix(filename, ICHeader);
if (await this.plugin.isIgnoredByIgnoreFiles(filename)) {
return;
@@ -534,7 +561,7 @@ export class HiddenFileSync extends LiveSyncCommands {
Logger(`STORAGE <x- DB:${filename}: deleted (hidden) Deleted on DB, but the file is already not found on storage.`);
} else {
Logger(`STORAGE <x- DB:${filename}: deleted (hidden).`);
await this.app.vault.adapter.remove(filename);
await this.plugin.vaultAccess.adapterRemove(filename);
try {
//@ts-ignore internalAPI
await this.app.vault.adapter.reconcileInternalFile(filename);
@@ -547,7 +574,7 @@ export class HiddenFileSync extends LiveSyncCommands {
}
if (!isExists) {
await this.ensureDirectoryEx(filename);
await this.app.vault.adapter.writeBinary(filename, decodeBinary(fileOnDB.data), { mtime: fileOnDB.mtime, ctime: fileOnDB.ctime });
await this.plugin.vaultAccess.adapterWrite(filename, decodeBinary(fileOnDB.data), { mtime: fileOnDB.mtime, ctime: fileOnDB.ctime });
try {
//@ts-ignore internalAPI
await this.app.vault.adapter.reconcileInternalFile(filename);
@@ -558,13 +585,13 @@ export class HiddenFileSync extends LiveSyncCommands {
Logger(`STORAGE <-- DB:${filename}: written (hidden,new${force ? ", force" : ""})`);
return true;
} else {
const contentBin = await this.app.vault.adapter.readBinary(filename);
const content = await encodeBinary(contentBin, this.settings.useV1);
if (isDocContentSame(content, fileOnDB.data) && !force) {
const contentBin = await this.plugin.vaultAccess.adapterReadBinary(filename);
const content = await encodeBinary(contentBin);
if (await isDocContentSame(content, fileOnDB.data) && !force) {
// Logger(`STORAGE <-- DB:${filename}: skipped (hidden) Not changed`, LOG_LEVEL_VERBOSE);
return true;
}
await this.app.vault.adapter.writeBinary(filename, decodeBinary(fileOnDB.data), { mtime: fileOnDB.mtime, ctime: fileOnDB.ctime });
await this.plugin.vaultAccess.adapterWrite(filename, decodeBinary(fileOnDB.data), { mtime: fileOnDB.mtime, ctime: fileOnDB.ctime });
try {
//@ts-ignore internalAPI
await this.app.vault.adapter.reconcileInternalFile(filename);
@@ -587,7 +614,7 @@ export class HiddenFileSync extends LiveSyncCommands {
showJSONMergeDialogAndMerge(docA: LoadedEntry, docB: LoadedEntry): Promise<boolean> {
return serialized("conflict:merge-data", () => new Promise((res) => {
return new Promise((res) => {
Logger("Opening data-merging dialog", LOG_LEVEL_VERBOSE);
const docs = [docA, docB];
const path = stripAllPrefixes(docA.path);
@@ -613,12 +640,12 @@ export class HiddenFileSync extends LiveSyncCommands {
}
}
if (!keep && result) {
const isExists = await this.app.vault.adapter.exists(filename);
const isExists = await this.plugin.vaultAccess.adapterExists(filename);
if (!isExists) {
await this.ensureDirectoryEx(filename);
}
await this.app.vault.adapter.write(filename, result);
const stat = await this.app.vault.adapter.stat(filename);
await this.plugin.vaultAccess.adapterWrite(filename, result);
const stat = await this.plugin.vaultAccess.adapterStat(filename);
await this.storeInternalFileToDatabase({ path: filename, ...stat }, true);
try {
//@ts-ignore internalAPI
@@ -641,7 +668,7 @@ export class HiddenFileSync extends LiveSyncCommands {
}
});
modal.open();
}));
});
}
async scanInternalFiles(): Promise<InternalFileInfo[]> {
@@ -657,7 +684,7 @@ export class HiddenFileSync extends LiveSyncCommands {
const files = filenames.filter(path => synchronisedInConfigSync.every(filterFile => !path.toLowerCase().startsWith(filterFile))).map(async (e) => {
return {
path: e as FilePath,
stat: await this.app.vault.adapter.stat(e)
stat: await this.plugin.vaultAccess.adapterStat(e)
};
});
const result: InternalFileInfo[] = [];

36
src/CmdPluginAndTheirSettings.ts
View File

@@ -1,8 +1,8 @@
import { normalizePath, type PluginManifest } from "./deps";
import type { DocumentID, EntryDoc, FilePathWithPrefix, LoadedEntry } from "./lib/src/types";
import type { DocumentID, EntryDoc, FilePathWithPrefix, LoadedEntry, SavingEntry } from "./lib/src/types";
import { LOG_LEVEL_INFO, LOG_LEVEL_NOTICE, LOG_LEVEL_VERBOSE } from "./lib/src/types";
import { type PluginDataEntry, PERIODIC_PLUGIN_SWEEP, type PluginList, type DevicePluginList, PSCHeader, PSCHeaderEnd } from "./types";
import { getDocData, isDocContentSame } from "./lib/src/utils";
import { createTextBlob, getDocData, isDocContentSame } from "./lib/src/utils";
import { Logger } from "./lib/src/logger";
import { PouchDB } from "./lib/src/pouchdb-browser.js";
import { isPluginMetadata, PeriodicProcessor } from "./utils";
@@ -186,18 +186,17 @@ export class PluginAndTheirSettings extends LiveSyncCommands {
}
Logger(`Reading plugin:${m.name}(${m.id})`, LOG_LEVEL_VERBOSE);
const path = normalizePath(m.dir) + "/";
const adapter = this.app.vault.adapter;
const files = ["manifest.json", "main.js", "styles.css", "data.json"];
const pluginData: { [key: string]: string; } = {};
for (const file of files) {
const thePath = path + file;
if (await adapter.exists(thePath)) {
pluginData[file] = await adapter.read(thePath);
if (await this.plugin.vaultAccess.adapterExists(thePath)) {
pluginData[file] = await this.plugin.vaultAccess.adapterRead(thePath);
}
}
let mtime = 0;
if (await adapter.exists(path + "/data.json")) {
mtime = (await adapter.stat(path + "/data.json")).mtime;
if (await this.plugin.vaultAccess.adapterExists(path + "/data.json")) {
mtime = (await this.plugin.vaultAccess.adapterStat(path + "/data.json")).mtime;
}
const p: PluginDataEntry = {
@@ -211,13 +210,14 @@ export class PluginAndTheirSettings extends LiveSyncCommands {
mtime: mtime,
type: "plugin",
};
const d: LoadedEntry = {
const blob = createTextBlob(JSON.stringify(p));
const d: SavingEntry = {
_id: p._id,
path: p._id as string as FilePathWithPrefix,
data: JSON.stringify(p),
data: blob,
ctime: mtime,
mtime: mtime,
size: 0,
size: blob.size,
children: [],
datatype: "plain",
type: "plain"
@@ -228,7 +228,7 @@ export class PluginAndTheirSettings extends LiveSyncCommands {
if (old !== false) {
const oldData = { data: old.data, deleted: old._deleted };
const newData = { data: d.data, deleted: d._deleted };
if (isDocContentSame(oldData.data, newData.data) && oldData.deleted == newData.deleted) {
if (await isDocContentSame(oldData.data, newData.data) && oldData.deleted == newData.deleted) {
Logger(`Nothing changed:${m.name}`);
return;
}
@@ -268,7 +268,6 @@ export class PluginAndTheirSettings extends LiveSyncCommands {
async applyPluginData(plugin: PluginDataEntry) {
await serialized("plugin-" + plugin.manifest.id, async () => {
const pluginTargetFolderPath = normalizePath(plugin.manifest.dir) + "/";
const adapter = this.app.vault.adapter;
// @ts-ignore
const stat = this.app.plugins.enabledPlugins.has(plugin.manifest.id) == true;
if (stat) {
@@ -277,7 +276,7 @@ export class PluginAndTheirSettings extends LiveSyncCommands {
Logger(`Unload plugin:${plugin.manifest.id}`, LOG_LEVEL_NOTICE);
}
if (plugin.dataJson)
await adapter.write(pluginTargetFolderPath + "data.json", plugin.dataJson);
await this.plugin.vaultAccess.adapterWrite(pluginTargetFolderPath + "data.json", plugin.dataJson);
Logger("wrote:" + pluginTargetFolderPath + "data.json", LOG_LEVEL_NOTICE);
if (stat) {
// @ts-ignore
@@ -298,14 +297,13 @@ export class PluginAndTheirSettings extends LiveSyncCommands {
}
const pluginTargetFolderPath = normalizePath(plugin.manifest.dir) + "/";
const adapter = this.app.vault.adapter;
if ((await adapter.exists(pluginTargetFolderPath)) === false) {
await adapter.mkdir(pluginTargetFolderPath);
if ((await this.plugin.vaultAccess.adapterExists(pluginTargetFolderPath)) === false) {
await this.app.vault.adapter.mkdir(pluginTargetFolderPath);
}
await adapter.write(pluginTargetFolderPath + "main.js", plugin.mainJs);
await adapter.write(pluginTargetFolderPath + "manifest.json", plugin.manifestJson);
await this.plugin.vaultAccess.adapterWrite(pluginTargetFolderPath + "main.js", plugin.mainJs);
await this.plugin.vaultAccess.adapterWrite(pluginTargetFolderPath + "manifest.json", plugin.manifestJson);
if (plugin.styleCss)
await adapter.write(pluginTargetFolderPath + "styles.css", plugin.styleCss);
await this.plugin.vaultAccess.adapterWrite(pluginTargetFolderPath + "styles.css", plugin.styleCss);
if (stat) {
// @ts-ignore
await this.app.plugins.loadPlugin(plugin.manifest.id);

5
src/CmdSetupLiveSync.ts
View File

@@ -60,7 +60,7 @@ export class SetupLiveSync extends LiveSyncCommands {
delete setting[k];
}
}
const encryptedSetting = encodeURIComponent(await encrypt(JSON.stringify(setting), encryptingPassphrase, false, true));
const encryptedSetting = encodeURIComponent(await encrypt(JSON.stringify(setting), encryptingPassphrase, false));
const uri = `${configURIBase}${encryptedSetting}`;
await navigator.clipboard.writeText(uri);
Logger("Setup URI copied to clipboard", LOG_LEVEL_NOTICE);
@@ -70,7 +70,7 @@ export class SetupLiveSync extends LiveSyncCommands {
if (encryptingPassphrase === false)
return;
const setting = { ...this.settings, configPassphraseStore: "", encryptedCouchDBConnection: "", encryptedPassphrase: "" };
const encryptedSetting = encodeURIComponent(await encrypt(JSON.stringify(setting), encryptingPassphrase, false, true));
const encryptedSetting = encodeURIComponent(await encrypt(JSON.stringify(setting), encryptingPassphrase, false));
const uri = `${configURIBase}${encryptedSetting}`;
await navigator.clipboard.writeText(uri);
Logger("Setup URI copied to clipboard", LOG_LEVEL_NOTICE);
@@ -321,7 +321,6 @@ Of course, we are able to disable these features.`
this.plugin.settings.suspendFileWatching = false;
await this.plugin.syncAllFiles(true);
await this.plugin.loadQueuedFiles();
this.plugin.procQueuedFiles();
await this.plugin.saveSettings();
}

82
src/ConflictResolveModal.ts
View File

@@ -1,23 +1,41 @@
import { App, Modal } from "./deps";
import { DIFF_DELETE, DIFF_EQUAL, DIFF_INSERT } from "diff-match-patch";
import { type diff_result } from "./lib/src/types";
import { CANCELLED, LEAVE_TO_SUBSEQUENT, RESULT_TIMED_OUT, type diff_result } from "./lib/src/types";
import { escapeStringToHTML } from "./lib/src/strbin";
import { delay, sendValue, waitForValue } from "./lib/src/utils";
export type MergeDialogResult = typeof LEAVE_TO_SUBSEQUENT | typeof CANCELLED | string;
export class ConflictResolveModal extends Modal {
// result: Array<[number, string]>;
result: diff_result;
filename: string;
callback: (remove_rev: string) => Promise<void>;
constructor(app: App, filename: string, diff: diff_result, callback: (remove_rev: string) => Promise<void>) {
response: MergeDialogResult = CANCELLED;
isClosed = false;
consumed = false;
constructor(app: App, filename: string, diff: diff_result) {
super(app);
this.result = diff;
this.callback = callback;
this.filename = filename;
// Send cancel signal for the previous merge dialogue
// if not there, simply be ignored.
// sendValue("close-resolve-conflict:" + this.filename, false);
sendValue("cancel-resolve-conflict:" + this.filename, true);
}
onOpen() {
const { contentEl } = this;
// Send cancel signal for the previous merge dialogue
// if not there, simply be ignored.
sendValue("cancel-resolve-conflict:" + this.filename, true);
setTimeout(async () => {
const forceClose = await waitForValue("cancel-resolve-conflict:" + this.filename);
// debugger;
if (forceClose) {
this.sendResponse(CANCELLED);
}
}, 10)
// sendValue("close-resolve-conflict:" + this.filename, false);
this.titleEl.setText("Conflicting changes");
contentEl.empty();
contentEl.createEl("span", { text: this.filename });
@@ -44,42 +62,32 @@ export class ConflictResolveModal extends Modal {
div2.innerHTML = `
<span class='deleted'>A:${date1}</span><br /><span class='added'>B:${date2}</span><br>
`;
contentEl.createEl("button", { text: "Keep A" }, (e) => {
e.addEventListener("click", async () => {
const callback = this.callback;
this.callback = null;
this.close();
await callback(this.result.right.rev);
});
});
contentEl.createEl("button", { text: "Keep B" }, (e) => {
e.addEventListener("click", async () => {
const callback = this.callback;
this.callback = null;
this.close();
await callback(this.result.left.rev);
});
});
contentEl.createEl("button", { text: "Concat both" }, (e) => {
e.addEventListener("click", async () => {
const callback = this.callback;
this.callback = null;
this.close();
await callback("");
});
});
contentEl.createEl("button", { text: "Not now" }, (e) => {
e.addEventListener("click", () => {
this.close();
});
});
contentEl.createEl("button", { text: "Keep A" }, (e) => e.addEventListener("click", () => this.sendResponse(this.result.right.rev)));
contentEl.createEl("button", { text: "Keep B" }, (e) => e.addEventListener("click", () => this.sendResponse(this.result.left.rev)));
contentEl.createEl("button", { text: "Concat both" }, (e) => e.addEventListener("click", () => this.sendResponse(LEAVE_TO_SUBSEQUENT)));
contentEl.createEl("button", { text: "Not now" }, (e) => e.addEventListener("click", () => this.sendResponse(CANCELLED)));
}
sendResponse(result: MergeDialogResult) {
this.response = result;
this.close();
}
onClose() {
const { contentEl } = this;
contentEl.empty();
if (this.callback != null) {
this.callback(null);
if (this.consumed) {
return;
}
this.consumed = true;
sendValue("close-resolve-conflict:" + this.filename, this.response);
sendValue("cancel-resolve-conflict:" + this.filename, false);
}
}
async waitForResult(): Promise<MergeDialogResult> {
await delay(100);
const r = await waitForValue<MergeDialogResult>("close-resolve-conflict:" + this.filename);
if (r === RESULT_TIMED_OUT) return CANCELLED;
return r;
}
}

120
src/DocumentHistoryModal.ts
View File

@@ -6,8 +6,34 @@ import { type DocumentID, type FilePathWithPrefix, type LoadedEntry, LOG_LEVEL_I
import { Logger } from "./lib/src/logger";
import { isErrorOfMissingDoc } from "./lib/src/utils_couchdb";
import { getDocData } from "./lib/src/utils";
import { stripPrefix } from "./lib/src/path";
import { isPlainText, stripPrefix } from "./lib/src/path";
function isImage(path: string) {
const ext = path.split(".").splice(-1)[0].toLowerCase();
return ["png", "jpg", "jpeg", "gif", "bmp", "webp"].includes(ext);
}
function isComparableText(path: string) {
const ext = path.split(".").splice(-1)[0].toLowerCase();
return isPlainText(path) || ["md", "mdx", "txt", "json"].includes(ext);
}
function isComparableTextDecode(path: string) {
const ext = path.split(".").splice(-1)[0].toLowerCase();
return ["json"].includes(ext)
}
function readDocument(w: LoadedEntry) {
if (isImage(w.path)) {
return new Uint8Array(decodeBinary(w.data));
}
if (w.data == "plain") return getDocData(w.data);
if (isComparableTextDecode(w.path)) return readString(new Uint8Array(decodeBinary(w.data)));
if (isComparableText(w.path)) return getDocData(w.data);
try {
return readString(new Uint8Array(decodeBinary(w.data)));
} catch (ex) {
// NO OP.
}
return getDocData(w.data);
}
export class DocumentHistoryModal extends Modal {
plugin: ObsidianLiveSyncPlugin;
range!: HTMLInputElement;
@@ -56,7 +82,6 @@ export class DocumentHistoryModal extends Modal {
this.range.max = "0";
this.range.value = "";
this.range.disabled = true;
this.showDiff
this.contentView.setText(`History of this file was not recorded.`);
} else {
this.contentView.setText(`Error occurred.`);
@@ -76,6 +101,22 @@ export class DocumentHistoryModal extends Modal {
const rev = this.revs_info[index];
await this.showExactRev(rev.rev);
}
BlobURLs = new Map<string, string>();
revokeURL(key: string) {
const v = this.BlobURLs.get(key);
if (v) {
URL.revokeObjectURL(v);
}
this.BlobURLs.set(key, undefined);
}
generateBlobURL(key: string, data: Uint8Array) {
this.revokeURL(key);
const v = URL.createObjectURL(new Blob([data], { endings: "transparent", type: "application/octet-stream" }));
this.BlobURLs.set(key, v);
return v;
}
async showExactRev(rev: string) {
const db = this.plugin.localDatabase;
const w = await db.getDBEntry(this.file, { rev: rev }, false, false, true);
@@ -88,42 +129,67 @@ export class DocumentHistoryModal extends Modal {
} else {
this.currentDoc = w;
this.info.innerHTML = `Modified:${new Date(w.mtime).toLocaleString()}`;
let result = "";
const w1data = w.datatype == "plain" ? getDocData(w.data) : readString(new Uint8Array(decodeBinary(w.data)));
let result = undefined;
const w1data = readDocument(w);
this.currentDeleted = !!w.deleted;
this.currentText = w1data;
// this.currentText = w1data;
if (this.showDiff) {
const prevRevIdx = this.revs_info.length - 1 - ((this.range.value as any) / 1 - 1);
if (prevRevIdx >= 0 && prevRevIdx < this.revs_info.length) {
const oldRev = this.revs_info[prevRevIdx].rev;
const w2 = await db.getDBEntry(this.file, { rev: oldRev }, false, false, true);
if (w2 != false) {
const dmp = new diff_match_patch();
const w2data = w2.datatype == "plain" ? getDocData(w2.data) : readString(new Uint8Array(decodeBinary(w2.data)));
const diff = dmp.diff_main(w2data, w1data);
dmp.diff_cleanupSemantic(diff);
for (const v of diff) {
const x1 = v[0];
const x2 = v[1];
if (x1 == DIFF_DELETE) {
result += "<span class='history-deleted'>" + escapeStringToHTML(x2) + "</span>";
} else if (x1 == DIFF_EQUAL) {
result += "<span class='history-normal'>" + escapeStringToHTML(x2) + "</span>";
} else if (x1 == DIFF_INSERT) {
result += "<span class='history-added'>" + escapeStringToHTML(x2) + "</span>";
if (typeof w1data == "string") {
result = "";
const dmp = new diff_match_patch();
const w2data = readDocument(w2) as string;
const diff = dmp.diff_main(w2data, w1data);
dmp.diff_cleanupSemantic(diff);
for (const v of diff) {
const x1 = v[0];
const x2 = v[1];
if (x1 == DIFF_DELETE) {
result += "<span class='history-deleted'>" + escapeStringToHTML(x2) + "</span>";
} else if (x1 == DIFF_EQUAL) {
result += "<span class='history-normal'>" + escapeStringToHTML(x2) + "</span>";
} else if (x1 == DIFF_INSERT) {
result += "<span class='history-added'>" + escapeStringToHTML(x2) + "</span>";
}
}
result = result.replace(/\n/g, "<br>");
} else if (isImage(this.file)) {
const src = this.generateBlobURL("base", w1data);
const overlay = this.generateBlobURL("overlay", readDocument(w2) as Uint8Array);
result =
`<div class='ls-imgdiff-wrap'>
<div class='overlay'>
<img class='img-base' src="${src}">
<img class='img-overlay' src='${overlay}'>
</div>
</div>`;
this.contentView.removeClass("op-pre");
}
}
}
result = result.replace(/\n/g, "<br>");
} else {
result = escapeStringToHTML(w1data);
}
if (result == undefined) {
if (typeof w1data != "string") {
if (isImage(this.file)) {
const src = this.generateBlobURL("base", w1data);
result =
`<div class='ls-imgdiff-wrap'>
<div class='overlay'>
<img class='img-base' src="${src}">
</div>
</div>`;
this.contentView.removeClass("op-pre");
}
} else {
result = escapeStringToHTML(w1data);
}
} else {
result = escapeStringToHTML(w1data);
}
if (result == undefined) result = typeof w1data == "string" ? escapeStringToHTML(w1data) : "Binary file";
this.contentView.innerHTML = (this.currentDeleted ? "(At this revision, the file has been deleted)\n" : "") + result;
}
}
@@ -200,11 +266,11 @@ export class DocumentHistoryModal extends Modal {
Logger("Path is not valid to write content.", LOG_LEVEL_INFO);
}
if (this.currentDoc?.datatype == "plain") {
await this.app.vault.adapter.write(pathToWrite, getDocData(this.currentDoc.data));
await this.plugin.vaultAccess.adapterWrite(pathToWrite, getDocData(this.currentDoc.data));
await focusFile(pathToWrite);
this.close();
} else if (this.currentDoc?.datatype == "newnote") {
await this.app.vault.adapter.writeBinary(pathToWrite, decodeBinary(this.currentDoc.data));
await this.plugin.vaultAccess.adapterWrite(pathToWrite, decodeBinary(this.currentDoc.data));
await focusFile(pathToWrite);
this.close();
} else {
@@ -217,5 +283,9 @@ export class DocumentHistoryModal extends Modal {
onClose() {
const { contentEl } = this;
contentEl.empty();
this.BlobURLs.forEach(value => {
console.log(value);
if (value) URL.revokeObjectURL(value);
})
}
}

20
src/GlobalHistory.svelte
View File

@@ -2,12 +2,11 @@
import ObsidianLiveSyncPlugin from "./main";
import { onDestroy, onMount } from "svelte";
import type { AnyEntry, FilePathWithPrefix } from "./lib/src/types";
import { getDocData, isDocContentSame } from "./lib/src/utils";
import { createBinaryBlob, getDocData, isDocContentSame } from "./lib/src/utils";
import { diff_match_patch } from "./deps";
import { DocumentHistoryModal } from "./DocumentHistoryModal";
import { isPlainText, stripAllPrefixes } from "./lib/src/path";
import { TFile } from "./deps";
import { encodeBinary } from "./lib/src/strbin";
export let plugin: ObsidianLiveSyncPlugin;
let showDiffInfo = false;
@@ -67,10 +66,7 @@
for (const revInfo of reversedRevs) {
if (revInfo.status == "available") {
const doc =
(!isPlain && showDiffInfo) || (checkStorageDiff && revInfo.rev == docA._rev)
? await db.getDBEntry(path, { rev: revInfo.rev }, false, false, true)
: await db.getDBEntryMeta(path, { rev: revInfo.rev }, true);
const doc = (!isPlain && showDiffInfo) || (checkStorageDiff && revInfo.rev == docA._rev) ? await db.getDBEntry(path, { rev: revInfo.rev }, false, false, true) : await db.getDBEntryMeta(path, { rev: revInfo.rev }, true);
if (doc === false) continue;
const rev = revInfo.rev;
@@ -108,16 +104,16 @@
}
if (rev == docA._rev) {
if (checkStorageDiff) {
const abs = plugin.app.vault.getAbstractFileByPath(stripAllPrefixes(plugin.getPath(docA)));
const abs = plugin.vaultAccess.getAbstractFileByPath(stripAllPrefixes(plugin.getPath(docA)));
if (abs instanceof TFile) {
let result = false;
if (isPlainText(docA.path)) {
const data = await plugin.app.vault.read(abs);
result = isDocContentSame(data, doc.data);
const data = await plugin.vaultAccess.adapterRead(abs);
result = await isDocContentSame(data, doc.data);
} else {
const data = await plugin.app.vault.readBinary(abs);
const dataEEncoded = await encodeBinary(data, plugin.settings.useV1);
result = isDocContentSame(dataEEncoded, doc.data);
const data = await plugin.vaultAccess.adapterReadBinary(abs);
const dataEEncoded = createBinaryBlob(data);
result = await isDocContentSame(dataEEncoded, doc.data);
}
if (result) {
diffDetail += " ⚖️";

2
src/JsonResolveModal.ts
View File

@@ -1,6 +1,7 @@
import { App, Modal } from "./deps";
import { type FilePath, type LoadedEntry } from "./lib/src/types";
import JsonResolvePane from "./JsonResolvePane.svelte";
import { waitForSignal } from "./lib/src/utils";
export class JsonResolveModal extends Modal {
// result: Array<[number, string]>;
@@ -20,6 +21,7 @@ export class JsonResolveModal extends Modal {
this.nameA = nameA;
this.nameB = nameB;
this.defaultSelect = defaultSelect;
waitForSignal(`cancel-internal-conflict:${filename}`).then(() => this.close());
}
async UICallback(keepRev: string, mergedStr?: string) {
this.close();

3
src/LiveSyncCommands.ts
View File

@@ -14,6 +14,9 @@ export abstract class LiveSyncCommands {
get localDatabase() {
return this.plugin.localDatabase;
}
get vaultAccess() {
return this.plugin.vaultAccess;
}
id2path(id: DocumentID, entry?: EntryHasPath, stripPrefix?: boolean): FilePathWithPrefix {
return this.plugin.id2path(id, entry, stripPrefix);
}

11
src/LogDisplayModal.ts
View File

@@ -1,5 +1,6 @@
import { App, Modal } from "./deps";
import { logMessageStore } from "./lib/src/stores";
import type { ReactiveInstance, } from "./lib/src/reactive";
import { logMessages } from "./lib/src/stores";
import { escapeStringToHTML } from "./lib/src/strbin";
import ObsidianLiveSyncPlugin from "./main";
@@ -21,14 +22,16 @@ export class LogDisplayModal extends Modal {
div.addClass("op-scrollable");
div.addClass("op-pre");
this.logEl = div;
this.unsubscribe = logMessageStore.observe((e) => {
function updateLog(logs: ReactiveInstance<string[]>) {
const e = logs.value;
let msg = "";
for (const v of e) {
msg += escapeStringToHTML(v) + "<br>";
}
this.logEl.innerHTML = msg;
})
logMessageStore.invalidate();
}
logMessages.onChanged(updateLog);
this.unsubscribe = () => logMessages.offChanged(updateLog);
}
onClose() {
const { contentEl } = this;

29
src/LogPane.svelte
View File

@@ -1,26 +1,27 @@
<script lang="ts">
import { onDestroy, onMount } from "svelte";
import { logMessageStore } from "./lib/src/stores";
import { logMessages } from "./lib/src/stores";
import type { ReactiveInstance } from "./lib/src/reactive";
import { Logger } from "./lib/src/logger";
let unsubscribe: () => void;
let messages = [] as string[];
let wrapRight = false;
let autoScroll = true;
let suspended = false;
function updateLog(logs: ReactiveInstance<string[]>) {
const e = logs.value;
if (!suspended) {
messages = [...e];
setTimeout(() => {
if (scroll) scroll.scrollTop = scroll.scrollHeight;
}, 10);
}
}
onMount(async () => {
unsubscribe = logMessageStore.observe((e) => {
if (!suspended) {
messages = [...e];
if (autoScroll) {
if (scroll) scroll.scrollTop = scroll.scrollHeight;
}
}
});
logMessageStore.invalidate();
setTimeout(() => {
if (scroll) scroll.scrollTop = scroll.scrollHeight;
}, 100);
logMessages.onChanged(updateLog);
Logger("Log window opened");
unsubscribe = () => logMessages.offChanged(updateLog);
});
onDestroy(() => {
if (unsubscribe) unsubscribe();

173
src/ObsidianLiveSyncSettingTab.ts
View File

@@ -4,10 +4,11 @@ import { delay } from "./lib/src/utils";
import { Semaphore } from "./lib/src/semaphore";
import { versionNumberString2Number } from "./lib/src/strbin";
import { Logger } from "./lib/src/logger";
import { checkSyncInfo, isCloudantURI } from "./lib/src/utils_couchdb.js";
import { checkSyncInfo, isCloudantURI } from "./lib/src/utils_couchdb";
import { testCrypt } from "./lib/src/e2ee_v2";
import ObsidianLiveSyncPlugin from "./main";
import { askYesNo, performRebuildDB, requestToCouchDB, scheduleTask } from "./utils";
import type { ButtonComponent } from "obsidian";
export class ObsidianLiveSyncSettingTab extends PluginSettingTab {
@@ -34,6 +35,10 @@ export class ObsidianLiveSyncSettingTab extends PluginSettingTab {
containerEl.empty();
// const preferred_setting = isCloudantURI(this.plugin.settings.couchDB_URI) ? PREFERRED_SETTING_CLOUDANT : PREFERRED_SETTING_SELF_HOSTED;
// const default_setting = { ...DEFAULT_SETTINGS };
containerEl.createEl("h2", { text: "Settings for Self-hosted LiveSync." });
containerEl.addClass("sls-setting");
containerEl.removeClass("isWizard");
@@ -746,8 +751,20 @@ export class ObsidianLiveSyncSettingTab extends PluginSettingTab {
toggle.setValue(this.plugin.settings.showStatusOnEditor).onChange(async (value) => {
this.plugin.settings.showStatusOnEditor = value;
await this.plugin.saveSettings();
this.display();
})
);
if (this.plugin.settings.showStatusOnEditor) {
new Setting(containerGeneralSettingsEl)
.setName("Show status as icons only")
.setDesc("")
.addToggle((toggle) =>
toggle.setValue(this.plugin.settings.showOnlyIconsOnEditor).onChange(async (value) => {
this.plugin.settings.showOnlyIconsOnEditor = value;
await this.plugin.saveSettings();
})
);
}
containerGeneralSettingsEl.createEl("h4", { text: "Logging" });
new Setting(containerGeneralSettingsEl)
@@ -807,6 +824,53 @@ export class ObsidianLiveSyncSettingTab extends PluginSettingTab {
text.inputEl.setAttribute("type", "number");
});
containerGeneralSettingsEl.createEl("h4", { text: "Share settings via markdown" });
let settingSyncFile = this.plugin.settings.settingSyncFile;
let buttonApplyFilename: ButtonComponent;
new Setting(containerGeneralSettingsEl)
.setName("Filename")
.setDesc("If you set this, all settings are saved in a markdown file. You will also be notified when new settings were arrived. You can set different files by the platform.")
.addText((text) => {
text.setPlaceholder("livesync/setting.md")
.setValue(settingSyncFile)
.onChange((value) => {
settingSyncFile = value;
if (settingSyncFile == this.plugin.settings.settingSyncFile) {
buttonApplyFilename.removeCta()
buttonApplyFilename.setDisabled(true);
} else {
buttonApplyFilename.setCta()
buttonApplyFilename.setDisabled(false);
}
})
}).addButton(button => {
button.setButtonText("Apply")
.onClick(async () => {
this.plugin.settings.settingSyncFile = settingSyncFile;
await this.plugin.saveSettings();
this.display();
})
buttonApplyFilename = button;
})
new Setting(containerGeneralSettingsEl)
.setName("Write credentials in the file")
.setDesc("(Not recommended) If set, credentials will be stored in the file.")
.addToggle(toggle => {
toggle.setValue(this.plugin.settings.writeCredentialsForSettingSync)
.onChange(async (value) => {
this.plugin.settings.writeCredentialsForSettingSync = value;
await this.plugin.saveSettings();
})
});
new Setting(containerGeneralSettingsEl)
.setName("Notify all setting files")
.addToggle(toggle => {
toggle.setValue(this.plugin.settings.notifyAllSettingSyncFile)
.onChange(async (value) => {
this.plugin.settings.notifyAllSettingSyncFile = value;
await this.plugin.saveSettings();
})
});
containerGeneralSettingsEl.createEl("h4", { text: "Advanced Confidentiality" });
@@ -1104,7 +1168,7 @@ export class ObsidianLiveSyncSettingTab extends PluginSettingTab {
);
new Setting(containerSyncSettingEl)
.setName("Postpone resolution of unopened files")
.setName("Postpone resolution of inactive files")
.setClass("wizardHidden")
.addToggle((toggle) =>
toggle.setValue(this.plugin.settings.checkConflictOnlyOnOpen).onChange(async (value) => {
@@ -1112,6 +1176,15 @@ export class ObsidianLiveSyncSettingTab extends PluginSettingTab {
await this.plugin.saveSettings();
})
);
new Setting(containerSyncSettingEl)
.setName("Postpone manual resolution of inactive files")
.setClass("wizardHidden")
.addToggle((toggle) =>
toggle.setValue(this.plugin.settings.showMergeDialogOnlyOnActive).onChange(async (value) => {
this.plugin.settings.showMergeDialogOnlyOnActive = value;
await this.plugin.saveSettings();
})
);
containerSyncSettingEl.createEl("h4", { text: "Compatibility" }).addClass("wizardHidden");
new Setting(containerSyncSettingEl)
.setName("Always resolve conflict manually")
@@ -1217,7 +1290,7 @@ export class ObsidianLiveSyncSettingTab extends PluginSettingTab {
});
let skipPatternTextArea: TextAreaComponent = null;
const defaultSkipPattern = "\\/node_modules\\/, \\/\\.git\\/, \\/obsidian-livesync\\/";
const defaultSkipPatternXPlat = defaultSkipPattern + ",\\/workspace$ ,\\/workspace.json$";
const defaultSkipPatternXPlat = defaultSkipPattern + ",\\/workspace$ ,\\/workspace.json$,\\/workspace-mobile.json$";
new Setting(containerSyncSettingEl)
.setName("Folders and files to ignore")
.setDesc(
@@ -1301,11 +1374,35 @@ export class ObsidianLiveSyncSettingTab extends PluginSettingTab {
});
containerSyncSettingEl.createEl("h4", {
text: sanitizeHTMLToDom(`Synchronization target filters`),
text: sanitizeHTMLToDom(`Targets`),
}).addClass("wizardHidden");
new Setting(containerSyncSettingEl)
.setName("Regular expression to ignore files")
.setDesc("If this is set, any changes to local and remote files that match this will be skipped.")
.setName("Synchronising files")
.setDesc("(RegExp) Empty to sync all files. set filter as a regular expression to limit synchronising files.")
.setClass("wizardHidden")
.addTextArea((text) => {
text
.setValue(this.plugin.settings.syncOnlyRegEx)
.setPlaceholder("\\.md$|\\.txt")
.onChange(async (value) => {
let isValidRegExp = false;
try {
new RegExp(value);
isValidRegExp = true;
} catch (_) {
// NO OP.
}
if (isValidRegExp || value.trim() == "") {
this.plugin.settings.syncOnlyRegEx = value;
await this.plugin.saveSettings();
}
})
return text;
}
);
new Setting(containerSyncSettingEl)
.setName("Non-Synchronising files")
.setDesc("(RegExp) If this is set, any changes to local and remote files that match this will be skipped.")
.setClass("wizardHidden")
.addTextArea((text) => {
text
@@ -1328,29 +1425,22 @@ export class ObsidianLiveSyncSettingTab extends PluginSettingTab {
}
);
new Setting(containerSyncSettingEl)
.setName("Regular expression for restricting synchronization targets")
.setDesc("If this is set, changes to local and remote files that only match this will be processed.")
.setName("Maximum file size")
.setDesc("(MB) If this is set, changes to local and remote files that are larger than this will be skipped. If the file becomes smaller again, a newer one will be used.")
.setClass("wizardHidden")
.addTextArea((text) => {
text
.setValue(this.plugin.settings.syncOnlyRegEx)
.setPlaceholder("\\.md$|\\.txt")
.addText((text) => {
text.setPlaceholder("")
.setValue(this.plugin.settings.syncMaxSizeInMB + "")
.onChange(async (value) => {
let isValidRegExp = false;
try {
new RegExp(value);
isValidRegExp = true;
} catch (_) {
// NO OP.
let v = Number(value);
if (isNaN(v) || v < 1) {
v = 0;
}
if (isValidRegExp || value.trim() == "") {
this.plugin.settings.syncOnlyRegEx = value;
await this.plugin.saveSettings();
}
})
return text;
}
);
this.plugin.settings.syncMaxSizeInMB = v;
await this.plugin.saveSettings();
});
text.inputEl.setAttribute("type", "number");
});
new Setting(containerSyncSettingEl)
.setName("(Beta) Use ignore files")
.setDesc("If this is set, changes to local files which are matched by the ignore files will be skipped. Remote changes are determined using local ignore files.")
@@ -1391,15 +1481,15 @@ export class ObsidianLiveSyncSettingTab extends PluginSettingTab {
}).addClass("wizardHidden");
new Setting(containerSyncSettingEl)
.setName("Batch size")
.setDesc("Number of change feed items to process at a time. Defaults to 50.")
.setDesc("Number of change feed items to process at a time. Defaults to 50. Minimum is 2.")
.setClass("wizardHidden")
.addText((text) => {
text.setPlaceholder("")
.setValue(this.plugin.settings.batch_size + "")
.onChange(async (value) => {
let v = Number(value);
if (isNaN(v) || v < 10) {
v = 10;
if (isNaN(v) || v < 2) {
v = 2;
}
this.plugin.settings.batch_size = v;
await this.plugin.saveSettings();
@@ -1409,15 +1499,15 @@ export class ObsidianLiveSyncSettingTab extends PluginSettingTab {
new Setting(containerSyncSettingEl)
.setName("Batch limit")
.setDesc("Number of batches to process at a time. Defaults to 40. This along with batch size controls how many docs are kept in memory at a time.")
.setDesc("Number of batches to process at a time. Defaults to 40. Minimum is 2. This along with batch size controls how many docs are kept in memory at a time.")
.setClass("wizardHidden")
.addText((text) => {
text.setPlaceholder("")
.setValue(this.plugin.settings.batches_limit + "")
.onChange(async (value) => {
let v = Number(value);
if (isNaN(v) || v < 10) {
v = 10;
if (isNaN(v) || v < 2) {
v = 2;
}
this.plugin.settings.batches_limit = v;
await this.plugin.saveSettings();
@@ -1514,8 +1604,10 @@ export class ObsidianLiveSyncSettingTab extends PluginSettingTab {
pluginConfig.encryptedPassphrase = REDACTED;
pluginConfig.encryptedCouchDBConnection = REDACTED;
pluginConfig.pluginSyncExtendedSetting = {};
const msgConfig = `----remote config----
const obsidianInfo = navigator.userAgent;
const msgConfig = `---- Obsidian info ----
${obsidianInfo}
---- remote config ----
${stringifyYaml(responseConfig)}
---- Plug-in config ---
version:${manifestVersion}
@@ -1632,7 +1724,7 @@ ${stringifyYaml(pluginConfig)}`;
if ((await this.plugin.localDatabase.putRaw(doc)).ok) {
Logger(`Old ${docName} has been deleted`, LOG_LEVEL_NOTICE);
}
await this.plugin.showIfConflicted(docName as FilePathWithPrefix);
await this.plugin.queueConflictCheck(docName as FilePathWithPrefix);
} else {
Logger(`Converting ${docName} Failed!`, LOG_LEVEL_NOTICE);
Logger(ret, LOG_LEVEL_VERBOSE);
@@ -1813,7 +1905,7 @@ ${stringifyYaml(pluginConfig)}`;
.setClass("wizardHidden")
.addDropdown((dropdown) =>
dropdown
.addOptions({ "": "Old Algorithm", "xxhash32": "xxhash32 (Fast)", "xxhash64": "xxhash64 (Fastest)" } as Record<HashAlgorithm, string>)
.addOptions({ "": "Old Algorithm", "xxhash32": "xxhash32 (Fast)", "xxhash64": "xxhash64 (Fastest)", "sha1": "Fallback (Without WebAssembly)" } as Record<HashAlgorithm, string>)
.setValue(this.plugin.settings.hashAlg)
.onChange(async (value) => {
this.plugin.settings.hashAlg = value as HashAlgorithm;
@@ -1832,15 +1924,6 @@ ${stringifyYaml(pluginConfig)}`;
await this.plugin.saveSettings();
})
);
new Setting(containerHatchEl)
.setName("Use binary and encryption version 1")
.setDesc("Use the previous data format for other products which shares the remote database.")
.addToggle((toggle) =>
toggle.setValue(this.plugin.settings.useV1).onChange(async (value) => {
this.plugin.settings.useV1 = value;
await this.plugin.saveSettings();
})
);
addScreenElement("50", containerHatchEl);

6
src/PluginCombo.svelte
View File

@@ -207,21 +207,21 @@
const local = list.find((e) => e.term == thisTerm);
const selectedItem = list.find((e) => e.term == selected);
if (selectedItem && (await applyData(selectedItem))) {
scheduleTask("update-plugin-list", 250, () => addOn.updatePluginList(true, local.documentPath));
addOn.updatePluginList(true, local?.documentPath);
}
}
async function compareSelected() {
const local = list.find((e) => e.term == thisTerm);
const selectedItem = list.find((e) => e.term == selected);
if (local && selectedItem && (await compareData(local, selectedItem))) {
scheduleTask("update-plugin-list", 250, () => addOn.updatePluginList(true, local.documentPath));
addOn.updatePluginList(true, local.documentPath);
}
}
async function deleteSelected() {
const selectedItem = list.find((e) => e.term == selected);
// const deletedPath = selectedItem.documentPath;
if (selectedItem && (await deleteData(selectedItem))) {
scheduleTask("update-plugin-list", 250, () => addOn.reloadPluginList(true));
addOn.reloadPluginList(true);
}
}
async function duplicateItem() {

46
src/PluginPane.svelte
View File

@@ -27,8 +27,10 @@
async function requestReload() {
await addOn.reloadPluginList(true);
}
let allTerms = [] as string[];
pluginList.subscribe((e) => {
list = e;
allTerms = unique(list.map((e) => e.term));
});
pluginIsEnumerating.subscribe((e) => {
loading = e;
@@ -172,7 +174,7 @@
.filter((e) => `${e.category}/${e.name}` == key)
.map((e) => e.files)
.flat()
.map((e) => e.filename)
.map((e) => e.filename),
);
automaticList.set(key, mode);
automaticListDisp = automaticList;
@@ -218,6 +220,16 @@
.sort((a, b) => (a.displayName ?? a.name).localeCompare(b.displayName ?? b.name))
.reduce((p, c) => ({ ...p, [c.category]: unique(c.category in p ? [...p[c.category], c.displayName ?? c.name] : [c.displayName ?? c.name]) }), {} as Record<string, string[]>);
}
let deleteTerm = "";
async function deleteAllItems(term: string) {
const deleteItems = list.filter((e) => e.term == term);
for (const item of deleteItems) {
await deleteData(item);
}
addOn.reloadPluginList(true);
}
</script>
<div>
@@ -322,6 +334,29 @@
</div>
{/if}
</div>
{#if isMaintenanceMode}
<div class="list">
<div>
<h3>Maintenance Commands</h3>
<div class="maintenancerow">
<label for="">Delete All of </label>
<select bind:value={deleteTerm}>
{#each allTerms as term}
<option value={term}>{term}</option>
{/each}
</select>
<button
class="status"
on:click={(evt) => {
deleteAllItems(deleteTerm);
}}
>
🗑️
</button>
</div>
</div>
</div>
{/if}
<div class="buttons">
<label><span>Hide not applicable items</span><input type="checkbox" bind:checked={hideEven} /></label>
</div>
@@ -423,4 +458,13 @@
align-items: center;
min-height: 3em;
}
.maintenancerow {
display: flex;
justify-content: flex-end;
align-items: center;
}
.maintenancerow label {
margin-right: 0.5em;
margin-left: 0.5em;
}
</style>

133
src/SerializedFileAccess.ts Normal file
View File

@@ -0,0 +1,133 @@
import { type App, TFile, type DataWriteOptions, TFolder, TAbstractFile } from "./deps";
import { serialized } from "./lib/src/lock";
import type { FilePath } from "./lib/src/types";
import { createBinaryBlob, isDocContentSame } from "./lib/src/utils";
import type { InternalFileInfo } from "./types";
function getFileLockKey(file: TFile | TFolder | string) {
return `fl:${typeof (file) == "string" ? file : file.path}`;
}
function toArrayBuffer(arr: Uint8Array | ArrayBuffer | DataView): ArrayBufferLike {
if (arr instanceof Uint8Array) {
return arr.buffer;
}
if (arr instanceof DataView) {
return arr.buffer;
}
return arr;
}
export class SerializedFileAccess {
app: App
constructor(app: App) {
this.app = app;
}
async adapterStat(file: TFile | string) {
const path = file instanceof TFile ? file.path : file;
return await serialized(getFileLockKey(path), () => this.app.vault.adapter.stat(path));
}
async adapterExists(file: TFile | string) {
const path = file instanceof TFile ? file.path : file;
return await serialized(getFileLockKey(path), () => this.app.vault.adapter.exists(path));
}
async adapterRemove(file: TFile | string) {
const path = file instanceof TFile ? file.path : file;
return await serialized(getFileLockKey(path), () => this.app.vault.adapter.remove(path));
}
async adapterRead(file: TFile | string) {
const path = file instanceof TFile ? file.path : file;
return await serialized(getFileLockKey(path), () => this.app.vault.adapter.read(path));
}
async adapterReadBinary(file: TFile | string) {
const path = file instanceof TFile ? file.path : file;
return await serialized(getFileLockKey(path), () => this.app.vault.adapter.readBinary(path));
}
async adapterWrite(file: TFile | string, data: string | ArrayBuffer | Uint8Array, options?: DataWriteOptions) {
const path = file instanceof TFile ? file.path : file;
if (typeof (data) === "string") {
return await serialized(getFileLockKey(path), () => this.app.vault.adapter.write(path, data, options));
} else {
return await serialized(getFileLockKey(path), () => this.app.vault.adapter.writeBinary(path, toArrayBuffer(data), options));
}
}
async vaultCacheRead(file: TFile) {
return await serialized(getFileLockKey(file), () => this.app.vault.cachedRead(file));
}
async vaultRead(file: TFile) {
return await serialized(getFileLockKey(file), () => this.app.vault.read(file));
}
async vaultReadBinary(file: TFile) {
return await serialized(getFileLockKey(file), () => this.app.vault.readBinary(file));
}
async vaultModify(file: TFile, data: string | ArrayBuffer | Uint8Array, options?: DataWriteOptions) {
if (typeof (data) === "string") {
return await serialized(getFileLockKey(file), async () => {
const oldData = await this.app.vault.read(file);
if (data === oldData) return false
await this.app.vault.modify(file, data, options)
return true;
}
);
} else {
return await serialized(getFileLockKey(file), async () => {
const oldData = await this.app.vault.readBinary(file);
if (await isDocContentSame(createBinaryBlob(oldData), createBinaryBlob(data))) {
return false;
}
await this.app.vault.modifyBinary(file, toArrayBuffer(data), options)
return true;
});
}
}
async vaultCreate(path: string, data: string | ArrayBuffer | Uint8Array, options?: DataWriteOptions): Promise<TFile> {
if (typeof (data) === "string") {
return await serialized(getFileLockKey(path), () => this.app.vault.create(path, data, options));
} else {
return await serialized(getFileLockKey(path), () => this.app.vault.createBinary(path, toArrayBuffer(data), options));
}
}
async delete(file: TFile | TFolder, force = false) {
return await serialized(getFileLockKey(file), () => this.app.vault.delete(file, force));
}
async trash(file: TFile | TFolder, force = false) {
return await serialized(getFileLockKey(file), () => this.app.vault.trash(file, force));
}
getAbstractFileByPath(path: FilePath | string): TAbstractFile | null {
// Disabled temporary.
return this.app.vault.getAbstractFileByPath(path);
// // Hidden API but so useful.
// // @ts-ignore
// if ("getAbstractFileByPathInsensitive" in app.vault && (app.vault.adapter?.insensitive ?? false)) {
// // @ts-ignore
// return app.vault.getAbstractFileByPathInsensitive(path);
// } else {
// return app.vault.getAbstractFileByPath(path);
// }
}
touchedFiles: string[] = [];
touch(file: TFile | FilePath) {
const f = file instanceof TFile ? file : this.getAbstractFileByPath(file) as TFile;
const key = `${f.path}-${f.stat.mtime}-${f.stat.size}`;
this.touchedFiles.unshift(key);
this.touchedFiles = this.touchedFiles.slice(0, 100);
}
recentlyTouched(file: TFile | InternalFileInfo) {
const key = file instanceof TFile ? `${file.path}-${file.stat.mtime}-${file.stat.size}` : `${file.path}-${file.mtime}-${file.size}`;
if (this.touchedFiles.indexOf(key) == -1) return false;
return true;
}
clearTouched() {
this.touchedFiles = [];
}
}

89
src/StorageEventManager.ts
View File

@@ -1,36 +1,37 @@
import type { SerializedFileAccess } from "./SerializedFileAccess";
import { Plugin, TAbstractFile, TFile, TFolder } from "./deps";
import { Logger } from "./lib/src/logger";
import { isPlainText, shouldBeIgnored } from "./lib/src/path";
import { getGlobalStore } from "./lib/src/store";
import { type FilePath, type ObsidianLiveSyncSettings } from "./lib/src/types";
import { type FileEventItem, type FileEventType, type FileInfo, type InternalFileInfo, type queueItem } from "./types";
import { recentlyTouched } from "./utils";
import type { KeyedQueueProcessor } from "./lib/src/processor";
import { LOG_LEVEL_NOTICE, type FilePath, type ObsidianLiveSyncSettings } from "./lib/src/types";
import { delay } from "./lib/src/utils";
import { type FileEventItem, type FileEventType, type FileInfo, type InternalFileInfo } from "./types";
export abstract class StorageEventManager {
abstract fetchEvent(): FileEventItem | false;
abstract cancelRelativeEvent(item: FileEventItem): void;
abstract getQueueLength(): number;
abstract beginWatch(): void;
}
type LiveSyncForStorageEventManager = Plugin &
{
settings: ObsidianLiveSyncSettings
ignoreFiles: string[],
vaultAccess: SerializedFileAccess
} & {
isTargetFile: (file: string | TAbstractFile) => Promise<boolean>,
procFileEvent: (applyBatch?: boolean) => Promise<any>,
fileEventQueue: KeyedQueueProcessor<FileEventItem, any>,
isFileSizeExceeded: (size: number) => boolean;
};
export class StorageEventManagerObsidian extends StorageEventManager {
plugin: LiveSyncForStorageEventManager;
queuedFilesStore = getGlobalStore("queuedFiles", { queuedItems: [] as queueItem[], fileEventItems: [] as FileEventItem[] });
watchedFileEventQueue = [] as FileEventItem[];
constructor(plugin: LiveSyncForStorageEventManager) {
super();
this.plugin = plugin;
}
beginWatch() {
const plugin = this.plugin;
this.watchVaultChange = this.watchVaultChange.bind(this);
this.watchVaultCreate = this.watchVaultCreate.bind(this);
this.watchVaultDelete = this.watchVaultDelete.bind(this);
@@ -42,6 +43,7 @@ export class StorageEventManagerObsidian extends StorageEventManager {
plugin.registerEvent(plugin.app.vault.on("create", this.watchVaultCreate));
//@ts-ignore : Internal API
plugin.registerEvent(plugin.app.vault.on("raw", this.watchVaultRawEvents));
plugin.fileEventQueue.startPipeline();
}
watchVaultCreate(file: TAbstractFile, ctx?: any) {
@@ -89,7 +91,6 @@ export class StorageEventManagerObsidian extends StorageEventManager {
}
// Cache file and waiting to can be proceed.
async appendWatchEvent(params: { type: FileEventType, file: TAbstractFile | InternalFileInfo, oldPath?: string }[], ctx?: any) {
let forcePerform = false;
for (const param of params) {
if (shouldBeIgnored(param.file.path)) {
continue;
@@ -98,6 +99,11 @@ export class StorageEventManagerObsidian extends StorageEventManager {
const type = param.type;
const file = param.file;
const oldPath = param.oldPath;
const size = file instanceof TFile ? file.stat.size : (file as InternalFileInfo)?.size ?? 0;
if (this.plugin.isFileSizeExceeded(size) && (type == "CREATE" || type == "CHANGED")) {
Logger(`The storage file has been changed but exceeds the maximum size. Skipping: ${param.file.path}`, LOG_LEVEL_NOTICE);
continue;
}
if (file instanceof TFolder) continue;
if (!await this.plugin.isTargetFile(file.path)) continue;
if (this.plugin.settings.suspendFileWatching) continue;
@@ -105,44 +111,18 @@ export class StorageEventManagerObsidian extends StorageEventManager {
let cache: null | string | ArrayBuffer;
// new file or something changed, cache the changes.
if (file instanceof TFile && (type == "CREATE" || type == "CHANGED")) {
if (recentlyTouched(file)) {
// Wait for a bit while to let the writer has marked `touched` at the file.
await delay(10);
if (this.plugin.vaultAccess.recentlyTouched(file)) {
continue;
}
if (!isPlainText(file.name)) {
cache = await this.plugin.app.vault.readBinary(file);
cache = await this.plugin.vaultAccess.vaultReadBinary(file);
} else {
// cache = await this.app.vault.read(file);
cache = await this.plugin.app.vault.cachedRead(file);
if (!cache) cache = await this.plugin.app.vault.read(file);
cache = await this.plugin.vaultAccess.vaultCacheRead(file);
if (!cache) cache = await this.plugin.vaultAccess.vaultRead(file);
}
}
if (type == "DELETE" || type == "RENAME") {
forcePerform = true;
}
if (this.plugin.settings.batchSave && !this.plugin.settings.liveSync) {
// if the latest event is the same type, omit that
// a.md MODIFY <- this should be cancelled when a.md MODIFIED
// b.md MODIFY <- this should be cancelled when b.md MODIFIED
// a.md MODIFY
// a.md CREATE
// :
let i = this.watchedFileEventQueue.length;
L1:
while (i >= 0) {
i--;
if (i < 0) break L1;
if (this.watchedFileEventQueue[i].args.file.path != file.path) {
continue L1;
}
if (this.watchedFileEventQueue[i].type != type) break L1;
this.watchedFileEventQueue.remove(this.watchedFileEventQueue[i]);
//this.queuedFilesStore.set({ queuedItems: this.queuedFiles, fileEventItems: this.watchedFileEventQueue });
this.queuedFilesStore.apply((value) => ({ ...value, fileEventItems: this.watchedFileEventQueue }));
}
}
const fileInfo = file instanceof TFile ? {
ctime: file.stat.ctime,
mtime: file.stat.mtime,
@@ -150,7 +130,8 @@ export class StorageEventManagerObsidian extends StorageEventManager {
path: file.path,
size: file.stat.size
} as FileInfo : file as InternalFileInfo;
this.watchedFileEventQueue.push({
this.plugin.fileEventQueue.enqueueWithKey(`file-${fileInfo.path}`, {
type,
args: {
file: fileInfo,
@@ -161,21 +142,5 @@ export class StorageEventManagerObsidian extends StorageEventManager {
key: atomicKey
})
}
// this.queuedFilesStore.set({ queuedItems: this.queuedFiles, fileEventItems: this.watchedFileEventQueue });
this.queuedFilesStore.apply((value) => ({ ...value, fileEventItems: this.watchedFileEventQueue }));
this.plugin.procFileEvent(forcePerform);
}
fetchEvent(): FileEventItem | false {
if (this.watchedFileEventQueue.length == 0) return false;
const item = this.watchedFileEventQueue.shift();
this.queuedFilesStore.apply((value) => ({ ...value, fileEventItems: this.watchedFileEventQueue }));
return item;
}
cancelRelativeEvent(item: FileEventItem) {
this.watchedFileEventQueue = [...this.watchedFileEventQueue].filter(e => e.key != item.key);
this.queuedFilesStore.apply((value) => ({ ...value, fileEventItems: this.watchedFileEventQueue }));
}
getQueueLength() {
return this.watchedFileEventQueue.length;
}
}

3
src/dialogs.ts
View File

@@ -58,8 +58,7 @@ export class InputStringDialog extends Modal {
onOpen() {
const { contentEl } = this;
this.titleEl.setText(this.title);
// For enter to submit
const formEl = contentEl.createEl("form");
const formEl = contentEl.createDiv();
new Setting(formEl).setName(this.key).setClass(this.isPassword ? "password-input" : "normal-input").addText((text) =>
text.onChange((value) => {
this.result = value;

2
src/lib

Submodule src/lib updated: b2788a8d98...dc9cbfe007

1506
src/main.ts
View File

File diff suppressed because it is too large Load Diff

111
src/utils.ts
View File

@@ -1,12 +1,15 @@
import { type DataWriteOptions, normalizePath, TFile, Platform, TAbstractFile, App, Plugin, type RequestUrlParam, requestUrl } from "./deps";
import { normalizePath, Platform, TAbstractFile, App, type RequestUrlParam, requestUrl } from "./deps";
import { path2id_base, id2path_base, isValidFilenameInLinux, isValidFilenameInDarwin, isValidFilenameInWidows, isValidFilenameInAndroid, stripAllPrefixes } from "./lib/src/path";
import { Logger } from "./lib/src/logger";
import { LOG_LEVEL_VERBOSE, type AnyEntry, type DocumentID, type EntryHasPath, type FilePath, type FilePathWithPrefix } from "./lib/src/types";
import { CHeader, ICHeader, ICHeaderLength, ICXHeader, PSCHeader } from "./types";
import { InputStringDialog, PopoverSelectString } from "./dialogs";
import ObsidianLiveSyncPlugin from "./main";
import type ObsidianLiveSyncPlugin from "./main";
import { writeString } from "./lib/src/strbin";
import { fireAndForget } from "./lib/src/utils";
export { scheduleTask, setPeriodicTask, cancelTask, cancelAllTasks, cancelPeriodicTask, cancelAllPeriodicTask, } from "./lib/src/task";
// For backward compatibility, using the path for determining id.
// Only CouchDB unacceptable ID (that starts with an underscore) has been prefixed with "/".
@@ -43,49 +46,6 @@ export function getPathFromTFile(file: TAbstractFile) {
return file.path as FilePath;
}
const tasks: { [key: string]: ReturnType<typeof setTimeout> } = {};
export function scheduleTask(key: string, timeout: number, proc: (() => Promise<any> | void), skipIfTaskExist?: boolean) {
if (skipIfTaskExist && key in tasks) {
return;
}
cancelTask(key);
tasks[key] = setTimeout(async () => {
delete tasks[key];
await proc();
}, timeout);
}
export function cancelTask(key: string) {
if (key in tasks) {
clearTimeout(tasks[key]);
delete tasks[key];
}
}
export function cancelAllTasks() {
for (const v in tasks) {
clearTimeout(tasks[v]);
delete tasks[v];
}
}
const intervals: { [key: string]: ReturnType<typeof setInterval> } = {};
export function setPeriodicTask(key: string, timeout: number, proc: (() => Promise<any> | void)) {
cancelPeriodicTask(key);
intervals[key] = setInterval(async () => {
delete intervals[key];
await proc();
}, timeout);
}
export function cancelPeriodicTask(key: string) {
if (key in intervals) {
clearInterval(intervals[key]);
delete intervals[key];
}
}
export function cancelAllPeriodicTask() {
for (const v in intervals) {
clearInterval(intervals[v]);
delete intervals[v];
}
}
const memos: { [key: string]: any } = {};
export function memoObject<T>(key: string, obj: T): T {
@@ -303,20 +263,6 @@ export function flattenObject(obj: Record<string | number | symbol, any>, path:
return ret;
}
export function modifyFile(file: TFile, data: string | ArrayBuffer, options?: DataWriteOptions) {
if (typeof (data) === "string") {
return app.vault.modify(file, data, options);
} else {
return app.vault.modifyBinary(file, data, options);
}
}
export function createFile(path: string, data: string | ArrayBuffer, options?: DataWriteOptions): Promise<TFile> {
if (typeof (data) === "string") {
return app.vault.create(path, data, options);
} else {
return app.vault.createBinary(path, data, options);
}
}
export function isValidPath(filename: string) {
if (Platform.isDesktop) {
@@ -332,38 +278,10 @@ export function isValidPath(filename: string) {
return isValidFilenameInWidows(filename);
}
let touchedFiles: string[] = [];
export function getAbstractFileByPath(path: FilePath): TAbstractFile | null {
// Disabled temporary.
return app.vault.getAbstractFileByPath(path);
// // Hidden API but so useful.
// // @ts-ignore
// if ("getAbstractFileByPathInsensitive" in app.vault && (app.vault.adapter?.insensitive ?? false)) {
// // @ts-ignore
// return app.vault.getAbstractFileByPathInsensitive(path);
// } else {
// return app.vault.getAbstractFileByPath(path);
// }
}
export function trimPrefix(target: string, prefix: string) {
return target.startsWith(prefix) ? target.substring(prefix.length) : target;
}
export function touch(file: TFile | FilePath) {
const f = file instanceof TFile ? file : getAbstractFileByPath(file) as TFile;
const key = `${f.path}-${f.stat.mtime}-${f.stat.size}`;
touchedFiles.unshift(key);
touchedFiles = touchedFiles.slice(0, 100);
}
export function recentlyTouched(file: TFile) {
const key = `${file.path}-${file.stat.mtime}-${file.stat.size}`;
if (touchedFiles.indexOf(key) == -1) return false;
return true;
}
export function clearTouched() {
touchedFiles = [];
}
/**
* returns is internal chunk of file
@@ -419,8 +337,8 @@ export const askString = (app: App, title: string, key: string, placeholder: str
export class PeriodicProcessor {
_process: () => Promise<any>;
_timer?: number;
_plugin: Plugin;
constructor(plugin: Plugin, process: () => Promise<any>) {
_plugin: ObsidianLiveSyncPlugin;
constructor(plugin: ObsidianLiveSyncPlugin, process: () => Promise<any>) {
this._plugin = plugin;
this._process = process;
}
@@ -434,12 +352,19 @@ export class PeriodicProcessor {
enable(interval: number) {
this.disable();
if (interval == 0) return;
this._timer = window.setInterval(() => this.process().then(() => { }), interval);
this._timer = window.setInterval(() => fireAndForget(async () => {
await this.process();
if (this._plugin._unloaded) {
this.disable();
}
}), interval);
this._plugin.registerInterval(this._timer);
}
disable() {
if (this._timer !== undefined) window.clearInterval(this._timer);
this._timer = undefined;
if (this._timer !== undefined) {
window.clearInterval(this._timer);
this._timer = undefined;
}
}
}
@@ -481,7 +406,7 @@ export const requestToCouchDB = async (baseUri: string, username: string, passwo
export async function performRebuildDB(plugin: ObsidianLiveSyncPlugin, method: "localOnly" | "remoteOnly" | "rebuildBothByThisDevice") {
if (method == "localOnly") {
await plugin.addOnSetup.fetchLocalWithKeepLocal();
await plugin.addOnSetup.fetchLocal();
}
if (method == "remoteOnly") {
await plugin.addOnSetup.rebuildRemote();

67
styles.css
View File

@@ -85,7 +85,6 @@
} */
.sls-header-button {
margin-left: 2em;
}
@@ -99,8 +98,10 @@
}
.CodeMirror-wrap::before,
.cm-s-obsidian>.cm-editor::before,
.canvas-wrapper::before {
.markdown-preview-view.cm-s-obsidian::before,
.markdown-source-view.cm-s-obsidian::before,
.canvas-wrapper::before,
.empty-state::before {
content: attr(data-log);
text-align: right;
white-space: pre-wrap;
@@ -116,6 +117,19 @@
filter: grayscale(100%);
}
.empty-state::before,
.markdown-preview-view.cm-s-obsidian::before,
.markdown-source-view.cm-s-obsidian::before {
top: var(--header-height);
right: 1em;
}
.is-mobile .empty-state::before,
.is-mobile .markdown-preview-view.cm-s-obsidian::before,
.is-mobile .markdown-source-view.cm-s-obsidian::before {
top: var(--view-header-height);
right: 1em;
}
.canvas-wrapper::before {
right: 48px;
}
@@ -124,7 +138,7 @@
right: 0px;
}
.cm-s-obsidian>.cm-editor::before {
.cm-s-obsidian > .cm-editor::before {
right: 16px;
}
@@ -153,8 +167,8 @@ div.sls-setting-menu-btn {
/* width: 100%; */
}
.sls-setting-tab:hover~div.sls-setting-menu-btn,
.sls-setting-label.selected .sls-setting-tab:checked~div.sls-setting-menu-btn {
.sls-setting-tab:hover ~ div.sls-setting-menu-btn,
.sls-setting-label.selected .sls-setting-tab:checked ~ div.sls-setting-menu-btn {
background-color: var(--interactive-accent);
color: var(--text-on-accent);
}
@@ -257,8 +271,8 @@ div.sls-setting-menu-btn {
display: none;
}
.password-input > .setting-item-control >input {
-webkit-text-security: disc;
.password-input > .setting-item-control > input {
-webkit-text-security: disc;
}
span.ls-mark-cr::after {
@@ -270,4 +284,39 @@ span.ls-mark-cr::after {
.deleted span.ls-mark-cr::after {
color: var(--text-on-accent);
}
}
.ls-imgdiff-wrap {
display: flex;
justify-content: center;
align-items: center;
}
.ls-imgdiff-wrap .overlay {
position: relative;
}
.ls-imgdiff-wrap .overlay .img-base {
position: relative;
top: 0;
left: 0;
}
.ls-imgdiff-wrap .overlay .img-overlay {
-webkit-filter: invert(100%) opacity(50%);
filter: invert(100%) opacity(50%);
position: absolute;
top: 0;
left: 0;
animation: ls-blink-diff 0.5s cubic-bezier(0.4, 0, 1, 1) infinite alternate;
}
@keyframes ls-blink-diff {
0% {
opacity: 0;
}
50% {
opacity: 0;
}
100% {
opacity: 1;
}
}

1
tsconfig.json
View File

@@ -16,6 +16,7 @@
"importHelpers": false,
"alwaysStrict": true,
"allowImportingTsExtensions": true,
"noEmit": true,
"lib": [
"es2018",
"DOM",

104
updates.md
View File

@@ -1,53 +1,71 @@
### 0.20.0
### 0.22.0
A few years passed since Self-hosted LiveSync was born, and our codebase had been very complicated. This could be patient now, but it should be a tremendous hurt.
Therefore at v0.22.0, for future maintainability, I refined task scheduling logic totally.
At 0.20.0, Self-hosted LiveSync has changed the binary file format and encrypting format, for efficient synchronisation.
The dialogue will be shown and asks us to decide whether to keep v1 or use v2. Once we have enabled v2, all subsequent edits will be saved in v2. Therefore, devices running 0.19 or below cannot understand this and they might say that decryption error. Please update all devices.
Then we will have an impressive performance.
Of course, I think this would be our suffering in some cases. However, I would love to ask you for your cooperation and contribution.
Of course, these are very impactful changes. If you have any questions or troubled things, please feel free to open an issue and mention me.
Sorry for being absent so much long. And thank you for your patience!
Note: if you want to roll it back to v1, please enable `Use binary and encryption version 1` on the `Hatch` pane and perform the `rebuild everything` once.
Extra but notable information:
This format change gives us the ability to detect some `marks` in the binary files as same as text files. Therefore, we can split binary files and some specific sort of them (i.e., PDF files) at the specific character. It means that editing the middle of files could be detected with marks.
Now only a few chunks are transferred, even if we add a comment to the PDF or put new files into the ZIP archives.
Note: we got a very performance improvement.
Note at 0.22.2: **Now, to rescue mobile devices, Maximum file size is set to 50 by default**. Please configure the limit as you need. If you do not want to limit the sizes, set zero manually, please.
#### Version history
- 0.20.3
- 0.22.3
- Fixed:
- No longer detects storage changes which have been caused by Self-hosted LiveSync itself.
- Setting sync file will be detected only if it has been configured now.
- And its log will be shown only while the verbose log is enabled.
- Customisation file enumeration has got less blingy.
- Deletion of files is now reliably synchronised.
- Fixed and improved:
- In-editor-status is now shown in the following areas:
- Note editing pane (Source mode and live-preview mode).
- New tab pane.
- Canvas pane.
- 0.22.2
- Fixed:
- Now the results of resolving conflicts are surely synchronised.
- Modified:
- Some setting items got new clear names. (`Sync Settings` -> `Targets`).
- New feature:
- We can launch Customization sync from the Ribbon if we enabled it.
- Fixed:
- Setup URI is now back to the previous spec; be encrypted by V1.
- It may avoid the trouble with iOS 17.
- The Settings dialogue is now registered at the beginning of the start-up process.
- We can change the configuration even though LiveSync could not be launched in normal.
- Improved:
- Enumerating documents has been faster.
- 0.20.2
- We can limit the synchronising files by their size. (`Sync Settings` -> `Targets` -> `Maximum file size`).
- It depends on the size of the newer one.
- At Obsidian 1.5.3 on mobile, we should set this to around 50MB to avoid restarting Obsidian.
- Now the settings could be stored in a specific markdown file to synchronise or switch it (`General Setting` -> `Share settings via markdown`).
- [Screwdriver](https://github.com/vrtmrz/obsidian-screwdriver) is quite good, but mostly we only need this.
- Customisation of the obsoleted device is now able to be deleted at once.
- We have to put the maintenance mode in at the Customisation sync dialogue.
- 0.22.1
- New feature:
- We can delete all data of customization sync from the `Delete all customization sync data` on the `Hatch` pane.
- Fixed:
- Prevent keep restarting on iOS by yielding microtasks.
- 0.20.1
- Fixed:
- No more UI freezing and keep restarting on iOS.
- Diff of Non-markdown documents are now shown correctly.
- We can perform automatic conflict resolution for inactive files, and postpone only manual ones by `Postpone manual resolution of inactive files`.
- Now we can see the image in the document history dialogue.
- We can see the difference of the image, in the document history dialogue.
- And also we can highlight differences.
- Improved:
- Performance has been a bit improved.
- Customization sync has gotten faster.
- However, We lost forward compatibility again (only for this feature). Please update all devices.
- Misc
- Terser configuration has been more aggressive.
- 0.20.0
- Improved:
- A New binary file handling implemented
- A new encrypted format has been implemented
- Now the chunk sizes will be adjusted for efficient sync
- Hidden file sync has been stabilised.
- Now automatically reloads the conflict-resolution dialogue when new conflicted revisions have arrived.
- Fixed:
- levels of exception in some logs have been fixed
- Tidied:
- Some Lint warnings have been suppressed.
- No longer periodic process runs after unloading the plug-in.
- Now the modification of binary files is surely stored in the storage.
- 0.22.0
- Refined:
- Task scheduling logics has been rewritten.
- Screen updates are also now efficient.
- Possibly many bugs and fragile behaviour has been fixed.
- Status updates and logging have been thinned out to display.
- Fixed:
- Remote-chunk-fetching now works with keeping request intervals
- New feature:
- We can show only the icons in the editor.
- Progress indicators have been more meaningful:
- 📥 Unprocessed transferred items
- 📄 Working database operation
- 💾 Working write storage processes
- ⏳ Working read storage processes
- 🛫 Pending read storage processes
- ⚙️ Working or pending storage processes of hidden files
- 🧩 Waiting chunks
- 🔌 Working Customisation items (Configuration, snippets and plug-ins)
... To continue on to `updates_old.md`.
... To continue on to `updates_old.md`.

109
updates_old.md
View File

@@ -1,3 +1,112 @@
### 0.21.0
The E2EE encryption V2 format has been reverted. That was probably the cause of the glitch.
Instead, to maintain efficiency, files are treated with Blob until just before saving. Along with this, the old-fashioned encryption format has also been discontinued.
There are both forward and backwards compatibilities, with recent versions. However, unfortunately, we lost compatibility with filesystem-livesync or some.
It will be addressed soon. Please be patient if you are using filesystem-livesync with E2EE.
- 0.21.5
- Improved:
- Now all revisions will be shown only its first a few letters.
- Now ID of the documents is shown in the log with the first 8 letters.
- Fixed:
- Check before modifying files has been implemented.
- Content change detection has been improved.
- 0.21.4
- This release had been skipped.
- 0.21.3
- Implemented:
- Now we can use SHA1 for hash function as fallback.
- 0.21.2
- IMPORTANT NOTICE: **0.21.1 CONTAINS A BUG WHILE REBUILDING THE DATABASE. IF YOU HAVE BEEN REBUILT, PLEASE MAKE SURE THAT ALL FILES ARE SANE.**
- This has been fixed in this version.
- Fixed:
- No longer files are broken while rebuilding.
- Now, Large binary files can be written correctly on a mobile platform.
- Any decoding errors now make zero-byte files.
- Modified:
- All files are processed sequentially for each.
- 0.21.1
- Fixed:
- No more infinity loops on larger files.
- Show message on decode error.
- Refactored:
- Fixed to avoid obsolete global variables.
- 0.21.0
- Changes and performance improvements:
- Now the saving files are processed by Blob.
- The V2-Format has been reverted.
- New encoding format has been enabled in default.
- WARNING: Since this version, the compatibilities with older Filesystem LiveSync have been lost.
## 0.20.0
At 0.20.0, Self-hosted LiveSync has changed the binary file format and encrypting format, for efficient synchronisation.
The dialogue will be shown and asks us to decide whether to keep v1 or use v2. Once we have enabled v2, all subsequent edits will be saved in v2. Therefore, devices running 0.19 or below cannot understand this and they might say that decryption error. Please update all devices.
Then we will have an impressive performance.
Of course, these are very impactful changes. If you have any questions or troubled things, please feel free to open an issue and mention me.
Note: if you want to roll it back to v1, please enable `Use binary and encryption version 1` on the `Hatch` pane and perform the `rebuild everything` once.
Extra but notable information:
This format change gives us the ability to detect some `marks` in the binary files as same as text files. Therefore, we can split binary files and some specific sort of them (i.e., PDF files) at the specific character. It means that editing the middle of files could be detected with marks.
Now only a few chunks are transferred, even if we add a comment to the PDF or put new files into the ZIP archives.
- 0.20.7
- Fixed
- To better replication, path obfuscation is now deterministic even if with E2EE.
Note: Compatible with previous database without any conversion. Only new files will be obfuscated in deterministic.
- 0.20.6
- Fixed
- Now empty file could be decoded.
- Local files are no longer pre-saved before fetching from a remote database.
- No longer deadlock while applying customisation sync.
- Configuration with multiple files is now able to be applied correctly.
- Deleting folder propagation now works without enabling the use of a trash bin.
- 0.20.5
- Fixed
- Now the files which having digit or character prefixes in the path will not be ignored.
- 0.20.4
- Fixed
- The text-input-dialogue is no longer broken.
- Finally, we can use the Setup URI again on mobile.
- 0.20.3
- New feature:
- We can launch Customization sync from the Ribbon if we enabled it.
- Fixed:
- Setup URI is now back to the previous spec; be encrypted by V1.
- It may avoid the trouble with iOS 17.
- The Settings dialogue is now registered at the beginning of the start-up process.
- We can change the configuration even though LiveSync could not be launched in normal.
- Improved:
- Enumerating documents has been faster.
- 0.20.2
- New feature:
- We can delete all data of customization sync from the `Delete all customization sync data` on the `Hatch` pane.
- Fixed:
- Prevent keep restarting on iOS by yielding microtasks.
- 0.20.1
- Fixed:
- No more UI freezing and keep restarting on iOS.
- Diff of Non-markdown documents are now shown correctly.
- Improved:
- Performance has been a bit improved.
- Customization sync has gotten faster.
- However, We lost forward compatibility again (only for this feature). Please update all devices.
- Misc
- Terser configuration has been more aggressive.
- 0.20.0
- Improved:
- A New binary file handling implemented
- A new encrypted format has been implemented
- Now the chunk sizes will be adjusted for efficient sync
- Fixed:
- levels of exception in some logs have been fixed
- Tidied:
- Some Lint warnings have been suppressed.
### 0.19.0
#### Customization sync