public/obsidian-livesync
1
0
Fork 0
mirror of https://github.com/vrtmrz/obsidian-livesync.git synced 2026-05-12 18:41:52 +00:00
Code Issues Packages Projects Releases Wiki Activity

Compare commits

base: public:0.22.10
Branches Tags
public:main public:test_real_obsidian public:feat-userhashsalt public:p2p-rpc public:auto_conf_2 public:auto_conf public:beta public:alpha public:refactor_services_2 public:refactor_services public:dev public:svelteui public:forprivacy
public:0.25.60 public:0.25.59 public:0.25.58 public:0.25.57 public:0.25.56+patched5 public:0.25.56+patched4 public:0.25.56+patched3 public:0.25.56+patched2 public:0.25.56+patched1 public:0.25.56 public:0.25.55 public:0.25.54 public:0.25.53 public:0.25.52-patched-3 public:0.25.52-patched-2 public:0.25.52-patched-1 public:0.25.52 public:0.25.51 public:0.25.50 public:0.25.49 public:0.25.48 public:0.25.47 public:0.25.46 public:0.25.45 public:0.25.44 public:0.25.43-patched-9 public:0.25.43-patched-8 public:0.25.43-patched-7 public:0.25.43-patched-6 public:0.25.43-patched-5 public:0.25.43-patched-4 public:0.25.43-patched-3 public:0.25.43-patched-2 public:0.25.43-patched-1 public:0.25.43 public:0.25.42 public:0.25.41-patched-3 public:0.25.41-patched-2 public:0.25.41-patched-1 public:0.25.41 public:0.25.40 public:0.25.39 public:0.25.38 public:0.25.37 public:0.25.36 public:0.25.35 public:0.25.34 public:0.25.33 public:0.25.32 public:0.25.31 public:0.25.30 public:0.25.30-dev public:0.25.29 public:0.25.28 public:0.25.27 public:0.25.26 public:0.25.25 public:0.25.24 public:0.25.24.beta3 public:0.25.24.beta2 public:0.25.24.beta1 public:0.25.23 public:0.25.23.beta1 public:0.25.22 public:0.25.21 public:0.25.21.beta2 public:0.25.21.beta1 public:0.25.20 public:0.25.19 public:0.25.18 public:0.25.17 public:0.25.16 public:0.25.15 public:0.25.14 public:0.25.13 public:0.25.12 public:0.25.11 public:0.25.10 public:0.25.9 public:0.25.8 public:0.25.7 public:0.25.6 public:0.25.5 public:0.25.4 public:0.25.3 public:0.25.2 public:0.25.1 public:0.25.0 public:0.25.0-beta1 public:0.24.31 public:0.24.30 public:0.24.29 public:0.24.28 public:0.24.27 public:0.24.26 public:0.24.25 public:0.24.24 public:0.24.23 public:0.24.22 public:0.24.21 public:0.24.20 public:0.24.19 public:0.24.18 public:0.24.17 public:0.24.16 public:0.24.15 public:0.24.14 public:0.24.13 public:0.24.12 public:0.24.11 public:0.24.10 public:0.24.9 public:0.24.8 public:0.24.7 public:0.24.6 public:0.24.5 public:0.24.4 public:0.24.3 public:0.24.2 public:0.24.1 public:0.24.0 public:0.24.0.dev-rc8 public:0.24.0.dev-rc7 public:0.24.0.dev-rc6 public:0.24.0.dev-rc5 public:0.24.0.dev-rc4 public:0.24.0.dev-rc3 public:0.24.0.dev-rc2 public:0.23.23 public:0.23.22 public:0.23.21 public:0.23.20 public:0.23.19 public:0.23.18 public:0.23.17 public:0.23.16 public:0.23.15 public:0.23.14 public:0.23.13 public:0.23.12 public:0.23.11 public:0.23.10 public:0.23.9 public:0.23.8 public:0.23.7 public:0.23.6 public:0.23.5 public:0.23.4 public:0.23.3 public:0.23.2 public:0.23.1 public:0.23.0 public:0.22.19 public:0.22.18 public:0.22.17 public:0.22.16 public:0.22.15 public:0.22.14 public:0.22.13 public:0.22.12 public:0.22.11 public:0.22.10 public:0.22.9 public:0.22.8 public:0.22.7 public:0.22.6 public:0.22.5 public:0.22.4 public:0.22.3 public:0.22.2 public:0.22.1 public:0.22.0 public:0.21.5 public:0.21.4 public:0.21.3 public:0.21.2 public:0.21.1 public:0.21.0 public:0.20.7 public:0.20.6 public:0.20.5 public:0.20.4 public:0.20.3 public:0.20.2 public:0.20.1 public:0.20.0 public:0.19.23 public:0.19.22 public:0.19.21 public:0.19.20 public:0.19.19 public:0.19.18 public:0.19.17 public:0.19.16 public:0.19.15 public:0.19.14 public:0.19.13 public:0.19.12 public:0.19.11 public:0.19.10 public:0.19.9 public:0.19.8 public:0.19.7 public:0.19.6 public:0.19.5 public:0.19.4 public:0.19.3 public:0.19.2 public:0.19.1 public:0.19.0 public:0.18.6 public:0.18.5 public:0.18.4 public:0.18.3 public:0.18.2 public:0.18.1 public:0.18.0 public:0.17.34 public:0.17.33 public:0.17.32 public:0.17.31 public:0.17.30 public:0.17.29 public:0.17.28 public:0.17.27 public:0.17.26 public:0.17.25 public:0.17.24 public:0.17.23 public:0.17.22 public:0.17.21 public:0.17.20 public:0.17.19 public:0.17.18 public:0.17.17 public:0.17.16 public:0.17.15 public:0.17.14 public:0.17.13 public:0.17.12 public:0.17.11 public:0.17.10 public:0.17.9 public:0.17.8 public:0.17.7 public:0.17.6 public:0.17.5 public:0.17.4 public:0.17.3 public:0.17.2 public:0.17.1 public:0.17.0 public:0.16.8 public:0.16.7 public:0.16.6 public:0.16.5 public:0.16.4 public:0.16.3 public:0.16.2 public:0.16.1 public:0.16.0 public:0.15.10 public:0.15.9 public:0.15.8 public:0.15.7 public:0.15.6 public:0.15.5 public:0.15.4 public:0.15.3 public:0.15.2 public:0.15.1 public:0.15.0 public:0.14.8 public:0.14.7 public:0.14.6 public:0.14.5 public:0.14.4 public:0.14.3 public:0.14.2 public:0.14.1 public:0.14.0 public:0.13.4 public:0.13.3 public:0.13.2 public:0.13.1 public:0.13.0 public:0.12.5 public:0.12.4 public:0.12.3 public:0.12.2 public:0.12.1 public:0.12.0 public:0.11.10 public:0.11.9 public:0.11.8 public:0.11.7 public:0.11.6 public:0.11.5 public:0.11.4 public:0.11.3 public:0.11.2 public:0.11.1 public:0.11.0 public:0.10.1 public:0.10.0 public:0.9.0 public:0.8.8 public:0.8.7 public:0.8.6 public:0.8.5 public:0.8.3 public:0.8.2 public:0.8.1 public:0.8.0 public:0.7.2 public:0.7.1 public:0.7.0 public:0.6.1 public:0.6.0 public:0.5.0 public:0.4.1 public:0.4.0 public:0.3.1 public:0.3.0 public:0.2.1 public:0.2.0 public:0.1.26 public:0.1.25 public:0.1.24 public:0.1.23 public:0.1.22 public:0.1.21 public:0.1.20 public:0.1.19 public:0.1.18 public:0.1.17 public:0.1.16 public:0.1.15 public:0.1.14 public:0.1.13 public:0.1.12 public:0.1.11 public:0.1.10 public:0.1.9 public:0.1.8 public:0.1.7 public:0.1.6 public:0.1.5 public:0.1.4 public:0.1.3 public:0.1.2 public:0.1.1 public:0.1.0 public:0.0.9 public:0.0.8 public:0.0.7 public:0.0.6
...
compare: public:0.22.19
Branches Tags
public:test_real_obsidian public:main public:feat-userhashsalt public:p2p-rpc public:auto_conf_2 public:auto_conf public:beta public:alpha public:refactor_services_2 public:refactor_services public:dev public:svelteui public:forprivacy
public:0.25.60 public:0.25.59 public:0.25.58 public:0.25.57 public:0.25.56+patched5 public:0.25.56+patched4 public:0.25.56+patched3 public:0.25.56+patched2 public:0.25.56+patched1 public:0.25.56 public:0.25.55 public:0.25.54 public:0.25.53 public:0.25.52-patched-3 public:0.25.52-patched-2 public:0.25.52-patched-1 public:0.25.52 public:0.25.51 public:0.25.50 public:0.25.49 public:0.25.48 public:0.25.47 public:0.25.46 public:0.25.45 public:0.25.44 public:0.25.43-patched-9 public:0.25.43-patched-8 public:0.25.43-patched-7 public:0.25.43-patched-6 public:0.25.43-patched-5 public:0.25.43-patched-4 public:0.25.43-patched-3 public:0.25.43-patched-2 public:0.25.43-patched-1 public:0.25.43 public:0.25.42 public:0.25.41-patched-3 public:0.25.41-patched-2 public:0.25.41-patched-1 public:0.25.41 public:0.25.40 public:0.25.39 public:0.25.38 public:0.25.37 public:0.25.36 public:0.25.35 public:0.25.34 public:0.25.33 public:0.25.32 public:0.25.31 public:0.25.30 public:0.25.30-dev public:0.25.29 public:0.25.28 public:0.25.27 public:0.25.26 public:0.25.25 public:0.25.24 public:0.25.24.beta3 public:0.25.24.beta2 public:0.25.24.beta1 public:0.25.23 public:0.25.23.beta1 public:0.25.22 public:0.25.21 public:0.25.21.beta2 public:0.25.21.beta1 public:0.25.20 public:0.25.19 public:0.25.18 public:0.25.17 public:0.25.16 public:0.25.15 public:0.25.14 public:0.25.13 public:0.25.12 public:0.25.11 public:0.25.10 public:0.25.9 public:0.25.8 public:0.25.7 public:0.25.6 public:0.25.5 public:0.25.4 public:0.25.3 public:0.25.2 public:0.25.1 public:0.25.0 public:0.25.0-beta1 public:0.24.31 public:0.24.30 public:0.24.29 public:0.24.28 public:0.24.27 public:0.24.26 public:0.24.25 public:0.24.24 public:0.24.23 public:0.24.22 public:0.24.21 public:0.24.20 public:0.24.19 public:0.24.18 public:0.24.17 public:0.24.16 public:0.24.15 public:0.24.14 public:0.24.13 public:0.24.12 public:0.24.11 public:0.24.10 public:0.24.9 public:0.24.8 public:0.24.7 public:0.24.6 public:0.24.5 public:0.24.4 public:0.24.3 public:0.24.2 public:0.24.1 public:0.24.0 public:0.24.0.dev-rc8 public:0.24.0.dev-rc7 public:0.24.0.dev-rc6 public:0.24.0.dev-rc5 public:0.24.0.dev-rc4 public:0.24.0.dev-rc3 public:0.24.0.dev-rc2 public:0.23.23 public:0.23.22 public:0.23.21 public:0.23.20 public:0.23.19 public:0.23.18 public:0.23.17 public:0.23.16 public:0.23.15 public:0.23.14 public:0.23.13 public:0.23.12 public:0.23.11 public:0.23.10 public:0.23.9 public:0.23.8 public:0.23.7 public:0.23.6 public:0.23.5 public:0.23.4 public:0.23.3 public:0.23.2 public:0.23.1 public:0.23.0 public:0.22.19 public:0.22.18 public:0.22.17 public:0.22.16 public:0.22.15 public:0.22.14 public:0.22.13 public:0.22.12 public:0.22.11 public:0.22.10 public:0.22.9 public:0.22.8 public:0.22.7 public:0.22.6 public:0.22.5 public:0.22.4 public:0.22.3 public:0.22.2 public:0.22.1 public:0.22.0 public:0.21.5 public:0.21.4 public:0.21.3 public:0.21.2 public:0.21.1 public:0.21.0 public:0.20.7 public:0.20.6 public:0.20.5 public:0.20.4 public:0.20.3 public:0.20.2 public:0.20.1 public:0.20.0 public:0.19.23 public:0.19.22 public:0.19.21 public:0.19.20 public:0.19.19 public:0.19.18 public:0.19.17 public:0.19.16 public:0.19.15 public:0.19.14 public:0.19.13 public:0.19.12 public:0.19.11 public:0.19.10 public:0.19.9 public:0.19.8 public:0.19.7 public:0.19.6 public:0.19.5 public:0.19.4 public:0.19.3 public:0.19.2 public:0.19.1 public:0.19.0 public:0.18.6 public:0.18.5 public:0.18.4 public:0.18.3 public:0.18.2 public:0.18.1 public:0.18.0 public:0.17.34 public:0.17.33 public:0.17.32 public:0.17.31 public:0.17.30 public:0.17.29 public:0.17.28 public:0.17.27 public:0.17.26 public:0.17.25 public:0.17.24 public:0.17.23 public:0.17.22 public:0.17.21 public:0.17.20 public:0.17.19 public:0.17.18 public:0.17.17 public:0.17.16 public:0.17.15 public:0.17.14 public:0.17.13 public:0.17.12 public:0.17.11 public:0.17.10 public:0.17.9 public:0.17.8 public:0.17.7 public:0.17.6 public:0.17.5 public:0.17.4 public:0.17.3 public:0.17.2 public:0.17.1 public:0.17.0 public:0.16.8 public:0.16.7 public:0.16.6 public:0.16.5 public:0.16.4 public:0.16.3 public:0.16.2 public:0.16.1 public:0.16.0 public:0.15.10 public:0.15.9 public:0.15.8 public:0.15.7 public:0.15.6 public:0.15.5 public:0.15.4 public:0.15.3 public:0.15.2 public:0.15.1 public:0.15.0 public:0.14.8 public:0.14.7 public:0.14.6 public:0.14.5 public:0.14.4 public:0.14.3 public:0.14.2 public:0.14.1 public:0.14.0 public:0.13.4 public:0.13.3 public:0.13.2 public:0.13.1 public:0.13.0 public:0.12.5 public:0.12.4 public:0.12.3 public:0.12.2 public:0.12.1 public:0.12.0 public:0.11.10 public:0.11.9 public:0.11.8 public:0.11.7 public:0.11.6 public:0.11.5 public:0.11.4 public:0.11.3 public:0.11.2 public:0.11.1 public:0.11.0 public:0.10.1 public:0.10.0 public:0.9.0 public:0.8.8 public:0.8.7 public:0.8.6 public:0.8.5 public:0.8.3 public:0.8.2 public:0.8.1 public:0.8.0 public:0.7.2 public:0.7.1 public:0.7.0 public:0.6.1 public:0.6.0 public:0.5.0 public:0.4.1 public:0.4.0 public:0.3.1 public:0.3.0 public:0.2.1 public:0.2.0 public:0.1.26 public:0.1.25 public:0.1.24 public:0.1.23 public:0.1.22 public:0.1.21 public:0.1.20 public:0.1.19 public:0.1.18 public:0.1.17 public:0.1.16 public:0.1.15 public:0.1.14 public:0.1.13 public:0.1.12 public:0.1.11 public:0.1.10 public:0.1.9 public:0.1.8 public:0.1.7 public:0.1.6 public:0.1.5 public:0.1.4 public:0.1.3 public:0.1.2 public:0.1.1 public:0.1.0 public:0.0.9 public:0.0.8 public:0.0.7 public:0.0.6

24 Commits
0.22.10 ... 0.22.19

Author SHA1 Message Date
vorotamoroz
c8fcf2d0d5 Bump 2024-04-19 12:06:09 +01:00
vorotamoroz
c384e2f7fb Fixed: 
- No longer data corrupting due to false BASE64 detections.
 2024-04-19 12:04:14 +01:00
vorotamoroz
99c1c7dc1a bump 2024-04-18 12:37:49 +01:00
vorotamoroz
84adec4b1a New feature: Automatic data compression to reduce amount of traffic and the usage of remote database. 2024-04-18 12:30:29 +01:00
vorotamoroz
f0b202bd91 bump 2024-04-12 01:32:03 +09:00
vorotamoroz
d54b7e2d93 - Fixed: 
- Error handling on booting now works fine.
  - Replication is now started automatically in LiveSync mode.
  - Batch database update is now disabled in LiveSync mode.
  - No longer automatically reconnection while off-focused.
  - Status saves are thinned out.
  - Now Self-hosted LiveSync waits for all files between the local database and storage to be surely checked.
- Improved:
  - The job scheduler is now more robust and stable.
  - The status indicator no longer flickers and keeps zero for a while.
  - No longer meaningless frequent updates of status indicators.
  - Now we can configure regular expression filters in handy UI. Thank you so much, @eth-p!
  - `Fetch` or `Rebuild everything` is now more safely performed.
- Minor things
  - Some utility function has been added.
  - Customisation sync now less wrong messages.
  - Digging the weeds for eradication of type errors.
 2024-04-12 01:30:35 +09:00
vorotamoroz
6952ef37f5 Update quick_setup.md 2024-04-09 13:10:31 +09:00
vorotamoroz
9630bcbae8 bump 2024-03-22 10:50:03 +01:00
vorotamoroz
c3f925ab9a Merge branch 'main' of https://github.com/vrtmrz/obsidian-livesync 2024-03-22 10:48:25 +01:00
vorotamoroz
034dc0538f - Fixed: 
- Fixed the issue that binary files were sometimes corrupted.
  - Fixed customisation sync data could be corrupted.
- Improved:
  - Now the remote database costs lower memory.
    - This release requires a brief wait on the first synchronisation, to track the latest changeset again.
  - Description added for the `Device name`.
- Refactored:
  - Many type-errors have been resolved.
  - Obsolete file has been deleted.
 2024-03-22 10:48:16 +01:00
vorotamoroz
b6136df836 Update quick_setup.md 2024-03-22 14:27:34 +09:00
vorotamoroz
24aacdc2a1 bump 2024-03-22 04:07:17 +01:00
vorotamoroz
f91109b1ad - Improved: 
- Faster start-up by removing too many logs which indicates normality
  - By streamlined scanning of customised synchronisation extra phases have been deleted.
 2024-03-22 04:07:07 +01:00
vorotamoroz
e76e7ae8ea bump 2024-03-19 17:59:38 +01:00
vorotamoroz
f7fbe85d65 - New feature: 
- We can disable the status bar in the setting dialogue.
- Improved:
  - Now some files are handled as correct data type.
  - Customisation sync now uses the digest of each file for better performance.
  - The status in the Editor now works performant.
- Refactored:
  - Common functions have been ready and the codebase has been organised.
  - Stricter type checking following TypeScript updates.
  - Remove old iOS workaround for simplicity and performance.
 2024-03-19 17:58:55 +01:00
vorotamoroz
0313443b29 Merge pull request #389 from Seeker0472/fix-command 
Fixed docker-compose command in docs
 2024-03-19 14:06:23 +09:00
seeker0472
755c30f468 fix docker-compose command 2024-03-17 14:30:35 +08:00
vorotamoroz
b00b0cc5e5 bump 2024-03-15 10:37:15 +01:00
vorotamoroz
d7985a6b41 Improved: 
- Now using HTTP for the remote database URI warns of an error (on mobile) or notice (on desktop).
 2024-03-15 10:36:00 +01:00
vorotamoroz
486e816902 Update dependencies 2024-03-15 10:35:41 +01:00
vorotamoroz
ef9b19c24b bump 2024-03-04 04:07:51 +00:00
vorotamoroz
4ed9494176 Changed: 
- The default settings has been changed.
Improved:
- Default and preferred settings are applied on completion of the wizard.
Fixed:
- Now Initialisation `Fetch` will be performed smoothly and there will be fewer conflicts.
- No longer stuck while Handling transferred or initialised documents.
 2024-03-04 04:07:11 +00:00
vorotamoroz
fcd56d59d5 bump 2024-03-01 08:33:37 +00:00
vorotamoroz
1cabfcfd19 Fixed: 
- `Verify and repair all files` is no longer broken.
New feature::
- Now `Verify and repair all files` can restore or show history
Improved:
- Performance improved
 2024-03-01 08:32:48 +00:00
24 changed files with 2506 additions and 2223 deletions
Expand all files Collapse all files

6
docs/quick_setup.md
View File

@@ -16,7 +16,8 @@ There are three methods to set up Self-hosted LiveSync.
### 1. Using setup URIs ### 1. Using setup URIs
> [!TIP] What is the setup URI? Why is it required? > [!TIP]
> What is the setup URI? Why is it required?
> The setup URI is the encrypted representation of Self-hosted LiveSync configuration as a URI. This starts `obsidian://setuplivesync?settings=`. This is encrypted with a passphrase, so that it can be shared relatively securely between devices. It is a bit long, but it is one line. This allows a series of settings to be set at once without any inconsistencies. > The setup URI is the encrypted representation of Self-hosted LiveSync configuration as a URI. This starts `obsidian://setuplivesync?settings=`. This is encrypted with a passphrase, so that it can be shared relatively securely between devices. It is a bit long, but it is one line. This allows a series of settings to be set at once without any inconsistencies.
> >
> If you have configured the remote database by [Automated setup on Fly.io](./setup_flyio.md#a-very-automated-setup) or [set up your server with the tool](./setup_own_server.md#1-generate-the-setup-uri-on-a-desktop-device-or-server), **you should have one of them** > If you have configured the remote database by [Automated setup on Fly.io](./setup_flyio.md#a-very-automated-setup) or [set up your server with the tool](./setup_own_server.md#1-generate-the-setup-uri-on-a-desktop-device-or-server), **you should have one of them**
@@ -54,6 +55,7 @@ If you do not have any setup URI, Press the `start` button. The setting dialogue
#### Test database connection and Check database configuration #### Test database connection and Check database configuration
We can check the connectivity to the database, and the database settings. We can check the connectivity to the database, and the database settings.
![](../images/quick_setup_5.png) ![](../images/quick_setup_5.png)
#### Check and Fix database configuration #### Check and Fix database configuration
@@ -104,4 +106,4 @@ And, please copy the setup URI by `Copy current settings as a new setup URI` and
## At the subsequent device ## At the subsequent device
After installing Self-hosted LiveSync on the first device, we should have a setup URI. **The first choice is to use it**. Please share it with the device you want to setup. After installing Self-hosted LiveSync on the first device, we should have a setup URI. **The first choice is to use it**. Please share it with the device you want to setup.
It is completely same as [Using setup URIs on the first device](#1-using-setup-uris). Please refer it. It is completely same as [Using setup URIs on the first device](#1-using-setup-uris). Please refer it.

2
docs/setup_own_server_cn.md
View File

@@ -91,7 +91,7 @@ services:
最后, 创建并启动容器: 最后, 创建并启动容器:
``` ```
# -d will launch detached so the container runs in background # -d will launch detached so the container runs in background
docker compose up -d docker-compose up -d
``` ```
## 创建数据库 ## 创建数据库

2
manifest.json
View File

@@ -1,7 +1,7 @@
{ {
"id": "obsidian-livesync", "id": "obsidian-livesync",
"name": "Self-hosted LiveSync", "name": "Self-hosted LiveSync",
"version": "0.22.10", "version": "0.22.19",
"minAppVersion": "0.9.12", "minAppVersion": "0.9.12",
"description": "Community implementation of self-hosted livesync. Reflect your vault changes to some other devices immediately. Please make sure to disable other synchronize solutions to avoid content corruption or duplication.", "description": "Community implementation of self-hosted livesync. Reflect your vault changes to some other devices immediately. Please make sure to disable other synchronize solutions to avoid content corruption or duplication.",
"author": "vorotamoroz", "author": "vorotamoroz",

2657
package-lock.json generated
View File

File diff suppressed because it is too large Load Diff

55
package.json
View File

@@ -1,6 +1,6 @@
{ {
"name": "obsidian-livesync", "name": "obsidian-livesync",
"version": "0.22.10", "version": "0.22.19",
"description": "Reflect your vault changes to some other devices immediately. Please make sure to disable other synchronize solutions to avoid content corruption or duplication.", "description": "Reflect your vault changes to some other devices immediately. Please make sure to disable other synchronize solutions to avoid content corruption or duplication.",
"main": "main.js", "main": "main.js",
"type": "module", "type": "module",
@@ -13,29 +13,29 @@
"author": "vorotamoroz", "author": "vorotamoroz",
"license": "MIT", "license": "MIT",
"devDependencies": { "devDependencies": {
"@tsconfig/svelte": "^5.0.0", "@tsconfig/svelte": "^5.0.2",
"@types/diff-match-patch": "^1.0.32", "@types/diff-match-patch": "^1.0.36",
"@types/node": "^20.2.5", "@types/node": "^20.11.28",
"@types/pouchdb": "^6.4.0", "@types/pouchdb": "^6.4.2",
"@types/pouchdb-adapter-http": "^6.1.3", "@types/pouchdb-adapter-http": "^6.1.6",
"@types/pouchdb-adapter-idb": "^6.1.4", "@types/pouchdb-adapter-idb": "^6.1.7",
"@types/pouchdb-browser": "^6.1.3", "@types/pouchdb-browser": "^6.1.5",
"@types/pouchdb-core": "^7.0.11", "@types/pouchdb-core": "^7.0.14",
"@types/pouchdb-mapreduce": "^6.1.7", "@types/pouchdb-mapreduce": "^6.1.10",
"@types/pouchdb-replication": "^6.4.4", "@types/pouchdb-replication": "^6.4.7",
"@types/transform-pouch": "^1.0.2", "@types/transform-pouch": "^1.0.6",
"@typescript-eslint/eslint-plugin": "^6.2.1", "@typescript-eslint/eslint-plugin": "^7.2.0",
"@typescript-eslint/parser": "^6.2.1", "@typescript-eslint/parser": "^7.2.0",
"builtin-modules": "^3.3.0", "builtin-modules": "^3.3.0",
"esbuild": "0.18.17", "esbuild": "0.20.2",
"esbuild-svelte": "^0.7.4", "esbuild-svelte": "^0.8.0",
"eslint": "^8.46.0", "eslint": "^8.57.0",
"eslint-config-airbnb-base": "^15.0.0", "eslint-config-airbnb-base": "^15.0.0",
"eslint-plugin-import": "^2.28.0", "eslint-plugin-import": "^2.29.1",
"events": "^3.3.0", "events": "^3.3.0",
"obsidian": "^1.4.11", "obsidian": "^1.5.7",
"postcss": "^8.4.27", "postcss": "^8.4.35",
"postcss-load-config": "^4.0.1", "postcss-load-config": "^5.0.3",
"pouchdb-adapter-http": "^8.0.1", "pouchdb-adapter-http": "^8.0.1",
"pouchdb-adapter-idb": "^8.0.1", "pouchdb-adapter-idb": "^8.0.1",
"pouchdb-adapter-indexeddb": "^8.0.1", "pouchdb-adapter-indexeddb": "^8.0.1",
@@ -46,16 +46,17 @@
"pouchdb-merge": "^8.0.1", "pouchdb-merge": "^8.0.1",
"pouchdb-replication": "^8.0.1", "pouchdb-replication": "^8.0.1",
"pouchdb-utils": "^8.0.1", "pouchdb-utils": "^8.0.1",
"svelte": "^4.1.2", "svelte": "^4.2.12",
"svelte-preprocess": "^5.0.4", "svelte-preprocess": "^5.1.3",
"terser": "^5.19.2", "terser": "^5.29.2",
"transform-pouch": "^2.0.0", "transform-pouch": "^2.0.0",
"tslib": "^2.6.1", "tslib": "^2.6.2",
"typescript": "^5.1.6" "typescript": "^5.4.2"
}, },
"dependencies": { "dependencies": {
"diff-match-patch": "^1.0.5", "diff-match-patch": "^1.0.5",
"idb": "^7.1.1", "fflate": "^0.8.2",
"idb": "^8.0.0",
"minimatch": "^9.0.3", "minimatch": "^9.0.3",
"xxhash-wasm": "0.4.2", "xxhash-wasm": "0.4.2",
"xxhash-wasm-102": "npm:xxhash-wasm@^1.0.2" "xxhash-wasm-102": "npm:xxhash-wasm@^1.0.2"

107
src/CmdConfigSync.ts
View File

@@ -4,10 +4,9 @@ import { Notice, type PluginManifest, parseYaml, normalizePath, type ListedFiles
import type { EntryDoc, LoadedEntry, InternalFileEntry, FilePathWithPrefix, FilePath, DocumentID, AnyEntry, SavingEntry } from "./lib/src/types"; import type { EntryDoc, LoadedEntry, InternalFileEntry, FilePathWithPrefix, FilePath, DocumentID, AnyEntry, SavingEntry } from "./lib/src/types";
import { LOG_LEVEL_INFO, LOG_LEVEL_NOTICE, LOG_LEVEL_VERBOSE, MODE_SELECTIVE } from "./lib/src/types"; import { LOG_LEVEL_INFO, LOG_LEVEL_NOTICE, LOG_LEVEL_VERBOSE, MODE_SELECTIVE } from "./lib/src/types";
import { ICXHeader, PERIODIC_PLUGIN_SWEEP, } from "./types"; import { ICXHeader, PERIODIC_PLUGIN_SWEEP, } from "./types";
import { createTextBlob, delay, getDocData, isDocContentSame, sendSignal, waitForSignal } from "./lib/src/utils"; import { createSavingEntryFromLoadedEntry, createTextBlob, delay, fireAndForget, getDocData, isDocContentSame, throttle } from "./lib/src/utils";
import { Logger } from "./lib/src/logger"; import { Logger } from "./lib/src/logger";
import { WrappedNotice } from "./lib/src/wrapper"; import { readString, decodeBinary, arrayBufferToBase64, digestHash } from "./lib/src/strbin";
import { readString, decodeBinary, arrayBufferToBase64, sha1 } from "./lib/src/strbin";
import { serialized } from "./lib/src/lock"; import { serialized } from "./lib/src/lock";
import { LiveSyncCommands } from "./LiveSyncCommands"; import { LiveSyncCommands } from "./LiveSyncCommands";
import { stripAllPrefixes } from "./lib/src/path"; import { stripAllPrefixes } from "./lib/src/path";
@@ -31,7 +30,8 @@ function serialize(data: PluginDataEx): string {
ret += data.mtime + d2; ret += data.mtime + d2;
for (const file of data.files) { for (const file of data.files) {
ret += file.filename + d + (file.displayName ?? "") + d + (file.version ?? "") + d2; ret += file.filename + d + (file.displayName ?? "") + d + (file.version ?? "") + d2;
ret += file.mtime + d + file.size + d2; const hash = digestHash((file.data ?? []).join());
ret += file.mtime + d + file.size + d + hash + d2;
for (const data of file.data ?? []) { for (const data of file.data ?? []) {
ret += data + d ret += data + d
} }
@@ -95,6 +95,7 @@ function deserialize2(str: string): PluginDataEx {
tokens.nextLine(); tokens.nextLine();
const mtime = Number(tokens.next()); const mtime = Number(tokens.next());
const size = Number(tokens.next()); const size = Number(tokens.next());
const hash = tokens.next();
tokens.nextLine(); tokens.nextLine();
const data = [] as string[]; const data = [] as string[];
let piece = ""; let piece = "";
@@ -110,7 +111,8 @@ function deserialize2(str: string): PluginDataEx {
version, version,
mtime, mtime,
size, size,
data data,
hash
} }
) )
tokens.nextLine(); tokens.nextLine();
@@ -137,10 +139,11 @@ export const pluginIsEnumerating = writable(false);
export type PluginDataExFile = { export type PluginDataExFile = {
filename: string, filename: string,
data?: string[], data: string[],
mtime: number, mtime: number,
size: number, size: number,
version?: string, version?: string,
hash?: string,
displayName?: string, displayName?: string,
} }
export type PluginDataExDisplay = { export type PluginDataExDisplay = {
@@ -169,17 +172,16 @@ export class ConfigSync extends LiveSyncCommands {
pluginScanningCount.onChanged((e) => { pluginScanningCount.onChanged((e) => {
const total = e.value; const total = e.value;
pluginIsEnumerating.set(total != 0); pluginIsEnumerating.set(total != 0);
if (total == 0) { // if (total == 0) {
Logger(`Processing configurations done`, LOG_LEVEL_INFO, "get-plugins"); // Logger(`Processing configurations done`, LOG_LEVEL_INFO, "get-plugins");
} // }
}) })
} }
confirmPopup: WrappedNotice = null;
get kvDB() { get kvDB() {
return this.plugin.kvDB; return this.plugin.kvDB;
} }
pluginDialog: PluginDialogModal = null; pluginDialog?: PluginDialogModal = undefined;
periodicPluginSweepProcessor = new PeriodicProcessor(this.plugin, async () => await this.scanAllConfigFiles(false)); periodicPluginSweepProcessor = new PeriodicProcessor(this.plugin, async () => await this.scanAllConfigFiles(false));
pluginList: PluginDataExDisplay[] = []; pluginList: PluginDataExDisplay[] = [];
@@ -187,7 +189,7 @@ export class ConfigSync extends LiveSyncCommands {
if (!this.settings.usePluginSync) { if (!this.settings.usePluginSync) {
return; return;
} }
if (this.pluginDialog != null) { if (this.pluginDialog) {
this.pluginDialog.open(); this.pluginDialog.open();
} else { } else {
this.pluginDialog = new PluginDialogModal(this.app, this.plugin); this.pluginDialog = new PluginDialogModal(this.app, this.plugin);
@@ -198,7 +200,7 @@ export class ConfigSync extends LiveSyncCommands {
hidePluginSyncModal() { hidePluginSyncModal() {
if (this.pluginDialog != null) { if (this.pluginDialog != null) {
this.pluginDialog.close(); this.pluginDialog.close();
this.pluginDialog = null; this.pluginDialog = undefined;
} }
} }
onunload() { onunload() {
@@ -273,16 +275,28 @@ export class ConfigSync extends LiveSyncCommands {
await this.updatePluginList(showMessage); await this.updatePluginList(showMessage);
} }
async loadPluginData(path: FilePathWithPrefix): Promise<PluginDataExDisplay | false> { async loadPluginData(path: FilePathWithPrefix): Promise<PluginDataExDisplay | false> {
const wx = await this.localDatabase.getDBEntry(path, null, false, false); const wx = await this.localDatabase.getDBEntry(path, undefined, false, false);
if (wx) { if (wx) {
const data = deserialize(getDocData(wx.data), {}) as PluginDataEx; const data = deserialize(getDocData(wx.data), {}) as PluginDataEx;
const xFiles = [] as PluginDataExFile[]; const xFiles = [] as PluginDataExFile[];
let missingHash = false;
for (const file of data.files) { for (const file of data.files) {
const work = { ...file }; const work = { ...file, data: [] as string[] };
const tempStr = getDocData(work.data); if (!file.hash) {
work.data = [await sha1(tempStr)]; // debugger;
const tempStr = getDocData(work.data);
const hash = digestHash(tempStr);
file.hash = hash;
missingHash = true;
}
work.data = [file.hash];
xFiles.push(work); xFiles.push(work);
} }
if (missingHash) {
Logger(`Digest created for ${path} to improve checking`, LOG_LEVEL_VERBOSE);
wx.data = serialize(data);
fireAndForget(() => this.localDatabase.putDBEntry(createSavingEntryFromLoadedEntry(wx)));
}
return ({ return ({
...data, ...data,
documentPath: this.getPath(wx), documentPath: this.getPath(wx),
@@ -291,7 +305,8 @@ export class ConfigSync extends LiveSyncCommands {
} }
return false; return false;
} }
createMissingConfigurationEntry() { createMissingConfigurationEntry = throttle(() => this._createMissingConfigurationEntry(), 1000);
_createMissingConfigurationEntry() {
let saveRequired = false; let saveRequired = false;
for (const v of this.pluginList) { for (const v of this.pluginList) {
const key = `${v.category}/${v.name}`; const key = `${v.category}/${v.name}`;
@@ -317,42 +332,27 @@ export class ConfigSync extends LiveSyncCommands {
const plugin = v[0]; const plugin = v[0];
const path = plugin.path || this.getPath(plugin); const path = plugin.path || this.getPath(plugin);
const oldEntry = (this.pluginList.find(e => e.documentPath == path)); const oldEntry = (this.pluginList.find(e => e.documentPath == path));
if (oldEntry && oldEntry.mtime == plugin.mtime) return; if (oldEntry && oldEntry.mtime == plugin.mtime) return [];
try { try {
const pluginData = await this.loadPluginData(path); const pluginData = await this.loadPluginData(path);
if (pluginData) { if (pluginData) {
return [pluginData]; let newList = [...this.pluginList];
newList = newList.filter(x => x.documentPath != pluginData.documentPath);
newList.push(pluginData);
this.pluginList = newList;
pluginList.set(newList);
} }
// Failed to load // Failed to load
return; return [];
} catch (ex) { } catch (ex) {
Logger(`Something happened at enumerating customization :${path}`, LOG_LEVEL_NOTICE); Logger(`Something happened at enumerating customization :${path}`, LOG_LEVEL_NOTICE);
Logger(ex, LOG_LEVEL_VERBOSE); Logger(ex, LOG_LEVEL_VERBOSE);
} }
return; return [];
}, { suspended: true, batchSize: 1, concurrentLimit: 5, delay: 300, yieldThreshold: 10 }).pipeTo( }, { suspended: false, batchSize: 1, concurrentLimit: 10, delay: 100, yieldThreshold: 10, maintainDelay: false, totalRemainingReactiveSource: pluginScanningCount }).startPipeline().root.onUpdateProgress(() => {
new QueueProcessor( this.createMissingConfigurationEntry();
async (pluginDataList) => { });
// Concurrency is two, therefore, we can unlock the previous awaiting.
sendSignal("plugin-next-load");
let newList = [...this.pluginList];
for (const item of pluginDataList) {
newList = newList.filter(x => x.documentPath != item.documentPath);
newList.push(item)
}
this.pluginList = newList;
pluginList.set(newList);
if (pluginDataList.length != 10) {
// If the queue is going to be empty, await subsequent for a while.
await waitForSignal("plugin-next-load", 1000);
}
return;
}
, { suspended: true, batchSize: 10, concurrentLimit: 2, delay: 250, yieldThreshold: 25, totalRemainingReactiveSource: pluginScanningCount })).startPipeline().root.onIdle(() => {
Logger(`All files enumerated`, LOG_LEVEL_INFO, "get-plugins");
this.createMissingConfigurationEntry();
});
async updatePluginList(showMessage: boolean, updatedDocumentPath?: FilePathWithPrefix): Promise<void> { async updatePluginList(showMessage: boolean, updatedDocumentPath?: FilePathWithPrefix): Promise<void> {
@@ -398,7 +398,7 @@ export class ConfigSync extends LiveSyncCommands {
showJSONMergeDialogAndMerge(docA: LoadedEntry, docB: LoadedEntry, pluginDataA: PluginDataEx, pluginDataB: PluginDataEx): Promise<boolean> { showJSONMergeDialogAndMerge(docA: LoadedEntry, docB: LoadedEntry, pluginDataA: PluginDataEx, pluginDataB: PluginDataEx): Promise<boolean> {
const fileA = { ...pluginDataA.files[0], ctime: pluginDataA.files[0].mtime, _id: `${pluginDataA.documentPath}` as DocumentID }; const fileA = { ...pluginDataA.files[0], ctime: pluginDataA.files[0].mtime, _id: `${pluginDataA.documentPath}` as DocumentID };
const fileB = pluginDataB.files[0]; const fileB = pluginDataB.files[0];
const docAx = { ...docA, ...fileA } as LoadedEntry, docBx = { ...docB, ...fileB } as LoadedEntry const docAx = { ...docA, ...fileA, datatype: "newnote" } as LoadedEntry, docBx = { ...docB, ...fileB, datatype: "newnote" } as LoadedEntry
return serialized("config:merge-data", () => new Promise((res) => { return serialized("config:merge-data", () => new Promise((res) => {
Logger("Opening data-merging dialog", LOG_LEVEL_VERBOSE); Logger("Opening data-merging dialog", LOG_LEVEL_VERBOSE);
// const docs = [docA, docB]; // const docs = [docA, docB];
@@ -502,9 +502,9 @@ export class ConfigSync extends LiveSyncCommands {
if (this.plugin.settings.usePluginSync && this.plugin.settings.notifyPluginOrSettingUpdated) { if (this.plugin.settings.usePluginSync && this.plugin.settings.notifyPluginOrSettingUpdated) {
if (!this.pluginDialog || (this.pluginDialog && !this.pluginDialog.isOpened())) { if (!this.pluginDialog || (this.pluginDialog && !this.pluginDialog.isOpened())) {
const fragment = createFragment((doc) => { const fragment = createFragment((doc) => {
doc.createEl("span", null, (a) => { doc.createEl("span", undefined, (a) => {
a.appendText(`Some configuration has been arrived, Press `); a.appendText(`Some configuration has been arrived, Press `);
a.appendChild(a.createEl("a", null, (anchor) => { a.appendChild(a.createEl("a", undefined, (anchor) => {
anchor.text = "HERE"; anchor.text = "HERE";
anchor.addEventListener("click", () => { anchor.addEventListener("click", () => {
this.showPluginSyncModal(); this.showPluginSyncModal();
@@ -670,7 +670,7 @@ export class ConfigSync extends LiveSyncCommands {
const content = createTextBlob(serialize(dt)); const content = createTextBlob(serialize(dt));
try { try {
const old = await this.localDatabase.getDBEntryMeta(prefixedFileName, null, false); const old = await this.localDatabase.getDBEntryMeta(prefixedFileName, undefined, false);
let saveData: SavingEntry; let saveData: SavingEntry;
if (old === false) { if (old === false) {
saveData = { saveData = {
@@ -694,7 +694,7 @@ export class ConfigSync extends LiveSyncCommands {
if (oldC) { if (oldC) {
const d = await deserialize(getDocData(oldC.data), {}) as PluginDataEx; const d = await deserialize(getDocData(oldC.data), {}) as PluginDataEx;
const diffs = (d.files.map(previous => ({ prev: previous, curr: dt.files.find(e => e.filename == previous.filename) })).map(async e => { const diffs = (d.files.map(previous => ({ prev: previous, curr: dt.files.find(e => e.filename == previous.filename) })).map(async e => {
try { return await isDocContentSame(e.curr.data, e.prev.data) } catch (_) { return false } try { return await isDocContentSame(e.curr?.data ?? [], e.prev.data) } catch (_) { return false }
})) }))
const isSame = (await Promise.all(diffs)).every(e => e == true); const isSame = (await Promise.all(diffs)).every(e => e == true);
if (isSame) { if (isSame) {
@@ -767,7 +767,11 @@ export class ConfigSync extends LiveSyncCommands {
const filesOnDB = ((await this.localDatabase.allDocsRaw({ startkey: ICXHeader + "", endkey: `${ICXHeader}\u{10ffff}`, include_docs: true })).rows.map(e => e.doc) as InternalFileEntry[]).filter(e => !e.deleted); const filesOnDB = ((await this.localDatabase.allDocsRaw({ startkey: ICXHeader + "", endkey: `${ICXHeader}\u{10ffff}`, include_docs: true })).rows.map(e => e.doc) as InternalFileEntry[]).filter(e => !e.deleted);
let deleteCandidate = filesOnDB.map(e => this.getPath(e)).filter(e => e.startsWith(`${ICXHeader}${term}/`)); let deleteCandidate = filesOnDB.map(e => this.getPath(e)).filter(e => e.startsWith(`${ICXHeader}${term}/`));
for (const vp of virtualPathsOfLocalFiles) { for (const vp of virtualPathsOfLocalFiles) {
const p = files.find(e => e.key == vp).file; const p = files.find(e => e.key == vp)?.file;
if (!p) {
Logger(`scanAllConfigFiles - File not found: ${vp}`, LOG_LEVEL_VERBOSE);
continue;
}
await this.storeCustomizationFiles(p); await this.storeCustomizationFiles(p);
deleteCandidate = deleteCandidate.filter(e => e != vp); deleteCandidate = deleteCandidate.filter(e => e != vp);
} }
@@ -782,10 +786,11 @@ export class ConfigSync extends LiveSyncCommands {
const mtime = new Date().getTime(); const mtime = new Date().getTime();
await serialized("file-x-" + prefixedFileName, async () => { await serialized("file-x-" + prefixedFileName, async () => {
try { try {
const old = await this.localDatabase.getDBEntryMeta(prefixedFileName, null, false) as InternalFileEntry | false; const old = await this.localDatabase.getDBEntryMeta(prefixedFileName, undefined, false) as InternalFileEntry | false;
let saveData: InternalFileEntry; let saveData: InternalFileEntry;
if (old === false) { if (old === false) {
Logger(`STORAGE -x> DB:${prefixedFileName}: (config) already deleted (Not found on database)`); Logger(`STORAGE -x> DB:${prefixedFileName}: (config) already deleted (Not found on database)`);
return;
} else { } else {
if (old.deleted) { if (old.deleted) {
Logger(`STORAGE -x> DB:${prefixedFileName}: (config) already deleted`); Logger(`STORAGE -x> DB:${prefixedFileName}: (config) already deleted`);

103
src/CmdHiddenFileSync.ts
View File

@@ -1,22 +1,20 @@
import { normalizePath, type PluginManifest, type ListedFiles } from "./deps"; import { normalizePath, type PluginManifest, type ListedFiles } from "./deps";
import { type EntryDoc, type LoadedEntry, type InternalFileEntry, type FilePathWithPrefix, type FilePath, LOG_LEVEL_INFO, LOG_LEVEL_NOTICE, LOG_LEVEL_VERBOSE, MODE_SELECTIVE, MODE_PAUSED, type SavingEntry, type DocumentID } from "./lib/src/types"; import { type EntryDoc, type LoadedEntry, type InternalFileEntry, type FilePathWithPrefix, type FilePath, LOG_LEVEL_INFO, LOG_LEVEL_NOTICE, LOG_LEVEL_VERBOSE, MODE_SELECTIVE, MODE_PAUSED, type SavingEntry, type DocumentID } from "./lib/src/types";
import { type InternalFileInfo, ICHeader, ICHeaderEnd } from "./types"; import { type InternalFileInfo, ICHeader, ICHeaderEnd } from "./types";
import { createBinaryBlob, isDocContentSame, sendSignal } from "./lib/src/utils"; import { readAsBlob, isDocContentSame, sendSignal, readContent, createBlob } from "./lib/src/utils";
import { Logger } from "./lib/src/logger"; import { Logger } from "./lib/src/logger";
import { PouchDB } from "./lib/src/pouchdb-browser.js"; import { PouchDB } from "./lib/src/pouchdb-browser.js";
import { isInternalMetadata, PeriodicProcessor } from "./utils"; import { isInternalMetadata, PeriodicProcessor } from "./utils";
import { WrappedNotice } from "./lib/src/wrapper";
import { decodeBinary, encodeBinary } from "./lib/src/strbin";
import { serialized } from "./lib/src/lock"; import { serialized } from "./lib/src/lock";
import { JsonResolveModal } from "./JsonResolveModal"; import { JsonResolveModal } from "./JsonResolveModal";
import { LiveSyncCommands } from "./LiveSyncCommands"; import { LiveSyncCommands } from "./LiveSyncCommands";
import { addPrefix, stripAllPrefixes } from "./lib/src/path"; import { addPrefix, stripAllPrefixes } from "./lib/src/path";
import { KeyedQueueProcessor, QueueProcessor } from "./lib/src/processor"; import { QueueProcessor } from "./lib/src/processor";
import { hiddenFilesEventCount, hiddenFilesProcessingCount } from "./lib/src/stores"; import { hiddenFilesEventCount, hiddenFilesProcessingCount } from "./lib/src/stores";
export class HiddenFileSync extends LiveSyncCommands { export class HiddenFileSync extends LiveSyncCommands {
periodicInternalFileScanProcessor: PeriodicProcessor = new PeriodicProcessor(this.plugin, async () => this.settings.syncInternalFiles && this.localDatabase.isReady && await this.syncInternalFilesAndDatabase("push", false)); periodicInternalFileScanProcessor: PeriodicProcessor = new PeriodicProcessor(this.plugin, async () => this.settings.syncInternalFiles && this.localDatabase.isReady && await this.syncInternalFilesAndDatabase("push", false));
confirmPopup: WrappedNotice = null;
get kvDB() { get kvDB() {
return this.plugin.kvDB; return this.plugin.kvDB;
} }
@@ -67,23 +65,23 @@ export class HiddenFileSync extends LiveSyncCommands {
realizeSettingSyncMode(): Promise<void> { realizeSettingSyncMode(): Promise<void> {
this.periodicInternalFileScanProcessor?.disable(); this.periodicInternalFileScanProcessor?.disable();
if (this.plugin.suspended) if (this.plugin.suspended)
return; return Promise.resolve();
if (!this.plugin.isReady) if (!this.plugin.isReady)
return; return Promise.resolve();
this.periodicInternalFileScanProcessor.enable(this.settings.syncInternalFiles && this.settings.syncInternalFilesInterval ? (this.settings.syncInternalFilesInterval * 1000) : 0); this.periodicInternalFileScanProcessor.enable(this.settings.syncInternalFiles && this.settings.syncInternalFilesInterval ? (this.settings.syncInternalFilesInterval * 1000) : 0);
return; return Promise.resolve();
} }
procInternalFile(filename: string) { procInternalFile(filename: string) {
this.internalFileProcessor.enqueueWithKey(filename, filename); this.internalFileProcessor.enqueue(filename);
} }
internalFileProcessor = new KeyedQueueProcessor<string, any>( internalFileProcessor = new QueueProcessor<string, any>(
async (filenames) => { async (filenames) => {
Logger(`START :Applying hidden ${filenames.length} files change`, LOG_LEVEL_VERBOSE); Logger(`START :Applying hidden ${filenames.length} files change`, LOG_LEVEL_VERBOSE);
await this.syncInternalFilesAndDatabase("pull", false, false, filenames); await this.syncInternalFilesAndDatabase("pull", false, false, filenames);
Logger(`DONE :Applying hidden ${filenames.length} files change`, LOG_LEVEL_VERBOSE); Logger(`DONE :Applying hidden ${filenames.length} files change`, LOG_LEVEL_VERBOSE);
return; return;
}, { batchSize: 100, concurrentLimit: 1, delay: 10, yieldThreshold: 10, suspended: false, totalRemainingReactiveSource: hiddenFilesEventCount } }, { batchSize: 100, concurrentLimit: 1, delay: 10, yieldThreshold: 100, suspended: false, totalRemainingReactiveSource: hiddenFilesEventCount }
); );
recentProcessedInternalFiles = [] as string[]; recentProcessedInternalFiles = [] as string[];
@@ -125,7 +123,7 @@ export class HiddenFileSync extends LiveSyncCommands {
if (storageMTime == 0) { if (storageMTime == 0) {
await this.deleteInternalFileOnDatabase(path); await this.deleteInternalFileOnDatabase(path);
} else { } else {
await this.storeInternalFileToDatabase({ path: path, ...stat }); await this.storeInternalFileToDatabase({ path: path, mtime, ctime: stat?.ctime ?? mtime, size: stat?.size ?? 0 });
} }
} }
@@ -162,7 +160,7 @@ export class HiddenFileSync extends LiveSyncCommands {
await this.localDatabase.removeRevision(id, delRev); await this.localDatabase.removeRevision(id, delRev);
Logger(`Older one has been deleted:${path}`); Logger(`Older one has been deleted:${path}`);
const cc = await this.localDatabase.getRaw(id, { conflicts: true }); const cc = await this.localDatabase.getRaw(id, { conflicts: true });
if (cc._conflicts.length == 0) { if (cc._conflicts?.length === 0) {
await this.extractInternalFileFromDatabase(stripAllPrefixes(path)) await this.extractInternalFileFromDatabase(stripAllPrefixes(path))
} else { } else {
this.conflictResolutionProcessor.enqueue(path); this.conflictResolutionProcessor.enqueue(path);
@@ -177,11 +175,12 @@ export class HiddenFileSync extends LiveSyncCommands {
// Retrieve data // Retrieve data
const id = await this.path2id(path, ICHeader); const id = await this.path2id(path, ICHeader);
const doc = await this.localDatabase.getRaw(id, { conflicts: true }); const doc = await this.localDatabase.getRaw(id, { conflicts: true });
// If there is no conflict, return with false. // if (!("_conflicts" in doc)){
if (!("_conflicts" in doc)) // return [];
return; // }
if (doc._conflicts === undefined) return [];
if (doc._conflicts.length == 0) if (doc._conflicts.length == 0)
return; return [];
Logger(`Hidden file conflicted:${path}`); Logger(`Hidden file conflicted:${path}`);
const conflicts = doc._conflicts.sort((a, b) => Number(a.split("-")[0]) - Number(b.split("-")[0])); const conflicts = doc._conflicts.sort((a, b) => Number(a.split("-")[0]) - Number(b.split("-")[0]));
const revA = doc._rev; const revA = doc._rev;
@@ -192,7 +191,7 @@ export class HiddenFileSync extends LiveSyncCommands {
const conflictedRevNo = Number(conflictedRev.split("-")[0]); const conflictedRevNo = Number(conflictedRev.split("-")[0]);
//Search //Search
const revFrom = (await this.localDatabase.getRaw<EntryDoc>(id, { revs_info: true })); const revFrom = (await this.localDatabase.getRaw<EntryDoc>(id, { revs_info: true }));
const commonBase = revFrom._revs_info.filter(e => e.status == "available" && Number(e.rev.split("-")[0]) < conflictedRevNo).first()?.rev ?? ""; const commonBase = revFrom._revs_info?.filter(e => e.status == "available" && Number(e.rev.split("-")[0]) < conflictedRevNo).first()?.rev ?? "";
const result = await this.plugin.mergeObject(path, commonBase, doc._rev, conflictedRev); const result = await this.plugin.mergeObject(path, commonBase, doc._rev, conflictedRev);
if (result) { if (result) {
Logger(`Object merge:${path}`, LOG_LEVEL_INFO); Logger(`Object merge:${path}`, LOG_LEVEL_INFO);
@@ -203,11 +202,14 @@ export class HiddenFileSync extends LiveSyncCommands {
} }
await this.plugin.vaultAccess.adapterWrite(filename, result); await this.plugin.vaultAccess.adapterWrite(filename, result);
const stat = await this.vaultAccess.adapterStat(filename); const stat = await this.vaultAccess.adapterStat(filename);
if (!stat) {
throw new Error(`conflictResolutionProcessor: Failed to stat file ${filename}`);
}
await this.storeInternalFileToDatabase({ path: filename, ...stat }); await this.storeInternalFileToDatabase({ path: filename, ...stat });
await this.extractInternalFileFromDatabase(filename); await this.extractInternalFileFromDatabase(filename);
await this.localDatabase.removeRevision(id, revB); await this.localDatabase.removeRevision(id, revB);
this.conflictResolutionProcessor.enqueue(path); this.conflictResolutionProcessor.enqueue(path);
return; return [];
} else { } else {
Logger(`Object merge is not applicable.`, LOG_LEVEL_VERBOSE); Logger(`Object merge is not applicable.`, LOG_LEVEL_VERBOSE);
} }
@@ -215,11 +217,11 @@ export class HiddenFileSync extends LiveSyncCommands {
} }
// When not JSON file, resolve conflicts by choosing a newer one. // When not JSON file, resolve conflicts by choosing a newer one.
await this.resolveByNewerEntry(id, path, doc, revA, revB); await this.resolveByNewerEntry(id, path, doc, revA, revB);
return; return [];
} catch (ex) { } catch (ex) {
Logger(`Failed to resolve conflict (Hidden): ${path}`); Logger(`Failed to resolve conflict (Hidden): ${path}`);
Logger(ex, LOG_LEVEL_VERBOSE); Logger(ex, LOG_LEVEL_VERBOSE);
return; return [];
} }
}, { }, {
suspended: false, batchSize: 1, concurrentLimit: 5, delay: 10, keepResultUntilDownstreamConnected: true, yieldThreshold: 10, suspended: false, batchSize: 1, concurrentLimit: 5, delay: 10, keepResultUntilDownstreamConnected: true, yieldThreshold: 10,
@@ -312,11 +314,11 @@ export class HiddenFileSync extends LiveSyncCommands {
if (processed % 100 == 0) { if (processed % 100 == 0) {
Logger(`Hidden file: ${processed}/${fileCount}`, logLevel, "sync_internal"); Logger(`Hidden file: ${processed}/${fileCount}`, logLevel, "sync_internal");
} }
if (!filename) return; if (!filename) return [];
if (ignorePatterns.some(e => filename.match(e))) if (ignorePatterns.some(e => filename.match(e)))
return; return [];
if (await this.plugin.isIgnoredByIgnoreFiles(filename)) { if (await this.plugin.isIgnoredByIgnoreFiles(filename)) {
return; return [];
} }
const fileOnStorage = filename in filesMap ? filesMap[filename] : undefined; const fileOnStorage = filename in filesMap ? filesMap[filename] : undefined;
@@ -403,7 +405,7 @@ export class HiddenFileSync extends LiveSyncCommands {
const enabledPlugins = this.app.plugins.enabledPlugins as Set<string>; const enabledPlugins = this.app.plugins.enabledPlugins as Set<string>;
const enabledPluginManifests = manifests.filter(e => enabledPlugins.has(e.id)); const enabledPluginManifests = manifests.filter(e => enabledPlugins.has(e.id));
for (const manifest of enabledPluginManifests) { for (const manifest of enabledPluginManifests) {
if (manifest.dir in updatedFolders) { if (manifest.dir && manifest.dir in updatedFolders) {
// If notified about plug-ins, reloading Obsidian may not be necessary. // If notified about plug-ins, reloading Obsidian may not be necessary.
updatedCount -= updatedFolders[manifest.dir]; updatedCount -= updatedFolders[manifest.dir];
const updatePluginId = manifest.id; const updatePluginId = manifest.id;
@@ -451,19 +453,11 @@ export class HiddenFileSync extends LiveSyncCommands {
const id = await this.path2id(file.path, ICHeader); const id = await this.path2id(file.path, ICHeader);
const prefixedFileName = addPrefix(file.path, ICHeader); const prefixedFileName = addPrefix(file.path, ICHeader);
const contentBin = await this.plugin.vaultAccess.adapterReadBinary(file.path); const content = createBlob(await this.plugin.vaultAccess.adapterReadAuto(file.path));
let content: Blob;
try {
content = createBinaryBlob(contentBin);
} catch (ex) {
Logger(`The file ${file.path} could not be encoded`);
Logger(ex, LOG_LEVEL_VERBOSE);
return false;
}
const mtime = file.mtime; const mtime = file.mtime;
return await serialized("file-" + prefixedFileName, async () => { return await serialized("file-" + prefixedFileName, async () => {
try { try {
const old = await this.localDatabase.getDBEntry(prefixedFileName, null, false, false); const old = await this.localDatabase.getDBEntry(prefixedFileName, undefined, false, false);
let saveData: SavingEntry; let saveData: SavingEntry;
if (old === false) { if (old === false) {
saveData = { saveData = {
@@ -479,7 +473,7 @@ export class HiddenFileSync extends LiveSyncCommands {
type: "newnote", type: "newnote",
}; };
} else { } else {
if (await isDocContentSame(createBinaryBlob(decodeBinary(old.data)), content) && !forceWrite) { if (await isDocContentSame(readAsBlob(old), content) && !forceWrite) {
// Logger(`STORAGE --> DB:${file.path}: (hidden) Not changed`, LOG_LEVEL_VERBOSE); // Logger(`STORAGE --> DB:${file.path}: (hidden) Not changed`, LOG_LEVEL_VERBOSE);
return; return;
} }
@@ -489,10 +483,10 @@ export class HiddenFileSync extends LiveSyncCommands {
data: content, data: content,
mtime, mtime,
size: file.size, size: file.size,
datatype: "newnote", datatype: old.datatype,
children: [], children: [],
deleted: false, deleted: false,
type: "newnote", type: old.datatype,
}; };
} }
const ret = await this.localDatabase.putDBEntry(saveData); const ret = await this.localDatabase.putDBEntry(saveData);
@@ -515,7 +509,7 @@ export class HiddenFileSync extends LiveSyncCommands {
} }
await serialized("file-" + prefixedFileName, async () => { await serialized("file-" + prefixedFileName, async () => {
try { try {
const old = await this.localDatabase.getDBEntryMeta(prefixedFileName, null, true) as InternalFileEntry | false; const old = await this.localDatabase.getDBEntryMeta(prefixedFileName, undefined, true) as InternalFileEntry | false;
let saveData: InternalFileEntry; let saveData: InternalFileEntry;
if (old === false) { if (old === false) {
saveData = { saveData = {
@@ -531,7 +525,7 @@ export class HiddenFileSync extends LiveSyncCommands {
} else { } else {
// Remove all conflicted before deleting. // Remove all conflicted before deleting.
const conflicts = await this.localDatabase.getRaw(old._id, { conflicts: true }); const conflicts = await this.localDatabase.getRaw(old._id, { conflicts: true });
if ("_conflicts" in conflicts) { if (conflicts._conflicts !== undefined) {
for (const conflictRev of conflicts._conflicts) { for (const conflictRev of conflicts._conflicts) {
await this.localDatabase.removeRevision(old._id, conflictRev); await this.localDatabase.removeRevision(old._id, conflictRev);
Logger(`STORAGE -x> DB:${filename}: (hidden) conflict removed ${old._rev} => ${conflictRev}`, LOG_LEVEL_VERBOSE); Logger(`STORAGE -x> DB:${filename}: (hidden) conflict removed ${old._rev} => ${conflictRev}`, LOG_LEVEL_VERBOSE);
@@ -581,7 +575,7 @@ export class HiddenFileSync extends LiveSyncCommands {
const deleted = fileOnDB.deleted || fileOnDB._deleted || false; const deleted = fileOnDB.deleted || fileOnDB._deleted || false;
if (deleted) { if (deleted) {
if (!isExists) { if (!isExists) {
Logger(`STORAGE <x- DB:${filename}: deleted (hidden) Deleted on DB, but the file is already not found on storage.`); Logger(`STORAGE <x- DB:${filename}: deleted (hidden) Deleted on DB, but the file is already not found on storage.`);
} else { } else {
Logger(`STORAGE <x- DB:${filename}: deleted (hidden).`); Logger(`STORAGE <x- DB:${filename}: deleted (hidden).`);
await this.plugin.vaultAccess.adapterRemove(filename); await this.plugin.vaultAccess.adapterRemove(filename);
@@ -597,7 +591,7 @@ export class HiddenFileSync extends LiveSyncCommands {
} }
if (!isExists) { if (!isExists) {
await this.vaultAccess.ensureDirectory(filename); await this.vaultAccess.ensureDirectory(filename);
await this.plugin.vaultAccess.adapterWrite(filename, decodeBinary(fileOnDB.data), { mtime: fileOnDB.mtime, ctime: fileOnDB.ctime }); await this.plugin.vaultAccess.adapterWrite(filename, readContent(fileOnDB), { mtime: fileOnDB.mtime, ctime: fileOnDB.ctime });
try { try {
//@ts-ignore internalAPI //@ts-ignore internalAPI
await this.app.vault.adapter.reconcileInternalFile(filename); await this.app.vault.adapter.reconcileInternalFile(filename);
@@ -608,13 +602,13 @@ export class HiddenFileSync extends LiveSyncCommands {
Logger(`STORAGE <-- DB:${filename}: written (hidden,new${force ? ", force" : ""})`); Logger(`STORAGE <-- DB:${filename}: written (hidden,new${force ? ", force" : ""})`);
return true; return true;
} else { } else {
const contentBin = await this.plugin.vaultAccess.adapterReadBinary(filename); const content = await this.plugin.vaultAccess.adapterReadAuto(filename);
const content = await encodeBinary(contentBin); const docContent = readContent(fileOnDB);
if (await isDocContentSame(content, fileOnDB.data) && !force) { if (await isDocContentSame(content, docContent) && !force) {
// Logger(`STORAGE <-- DB:${filename}: skipped (hidden) Not changed`, LOG_LEVEL_VERBOSE); // Logger(`STORAGE <-- DB:${filename}: skipped (hidden) Not changed`, LOG_LEVEL_VERBOSE);
return true; return true;
} }
await this.plugin.vaultAccess.adapterWrite(filename, decodeBinary(fileOnDB.data), { mtime: fileOnDB.mtime, ctime: fileOnDB.ctime }); await this.plugin.vaultAccess.adapterWrite(filename, docContent, { mtime: fileOnDB.mtime, ctime: fileOnDB.ctime });
try { try {
//@ts-ignore internalAPI //@ts-ignore internalAPI
await this.app.vault.adapter.reconcileInternalFile(filename); await this.app.vault.adapter.reconcileInternalFile(filename);
@@ -669,7 +663,11 @@ export class HiddenFileSync extends LiveSyncCommands {
} }
await this.plugin.vaultAccess.adapterWrite(filename, result); await this.plugin.vaultAccess.adapterWrite(filename, result);
const stat = await this.plugin.vaultAccess.adapterStat(filename); const stat = await this.plugin.vaultAccess.adapterStat(filename);
await this.storeInternalFileToDatabase({ path: filename, ...stat }, true); if (!stat) {
throw new Error("Stat failed");
}
const mtime = stat?.mtime ?? 0;
await this.storeInternalFileToDatabase({ path: filename, mtime, ctime: stat?.ctime ?? mtime, size: stat?.size ?? 0 }, true);
try { try {
//@ts-ignore internalAPI //@ts-ignore internalAPI
await this.app.vault.adapter.reconcileInternalFile(filename); await this.app.vault.adapter.reconcileInternalFile(filename);
@@ -703,7 +701,7 @@ export class HiddenFileSync extends LiveSyncCommands {
const root = this.app.vault.getRoot(); const root = this.app.vault.getRoot();
const findRoot = root.path; const findRoot = root.path;
const filenames = (await this.getFiles(findRoot, [], null, ignoreFilter)).filter(e => e.startsWith(".")).filter(e => !e.startsWith(".trash")); const filenames = (await this.getFiles(findRoot, [], undefined, ignoreFilter)).filter(e => e.startsWith(".")).filter(e => !e.startsWith(".trash"));
const files = filenames.filter(path => synchronisedInConfigSync.every(filterFile => !path.toLowerCase().startsWith(filterFile))).map(async (e) => { const files = filenames.filter(path => synchronisedInConfigSync.every(filterFile => !path.toLowerCase().startsWith(filterFile))).map(async (e) => {
return { return {
path: e as FilePath, path: e as FilePath,
@@ -716,9 +714,12 @@ export class HiddenFileSync extends LiveSyncCommands {
if (await this.plugin.isIgnoredByIgnoreFiles(w.path)) { if (await this.plugin.isIgnoredByIgnoreFiles(w.path)) {
continue continue
} }
const mtime = w.stat?.mtime ?? 0
const ctime = w.stat?.ctime ?? mtime;
const size = w.stat?.size ?? 0;
result.push({ result.push({
...w, ...w,
...w.stat mtime, ctime, size
}); });
} }
return result; return result;
@@ -729,8 +730,8 @@ export class HiddenFileSync extends LiveSyncCommands {
async getFiles( async getFiles(
path: string, path: string,
ignoreList: string[], ignoreList: string[],
filter: RegExp[], filter?: RegExp[],
ignoreFilter: RegExp[] ignoreFilter?: RegExp[]
) { ) {
let w: ListedFiles; let w: ListedFiles;
try { try {

314
src/CmdPluginAndTheirSettings.ts
View File

@@ -1,314 +0,0 @@
import { normalizePath, type PluginManifest } from "./deps";
import type { DocumentID, EntryDoc, FilePathWithPrefix, LoadedEntry, SavingEntry } from "./lib/src/types";
import { LOG_LEVEL_INFO, LOG_LEVEL_NOTICE, LOG_LEVEL_VERBOSE } from "./lib/src/types";
import { type PluginDataEntry, PERIODIC_PLUGIN_SWEEP, type PluginList, type DevicePluginList, PSCHeader, PSCHeaderEnd } from "./types";
import { createTextBlob, getDocData, isDocContentSame } from "./lib/src/utils";
import { Logger } from "./lib/src/logger";
import { PouchDB } from "./lib/src/pouchdb-browser.js";
import { isPluginMetadata, PeriodicProcessor } from "./utils";
import { PluginDialogModal } from "./dialogs";
import { NewNotice } from "./lib/src/wrapper";
import { versionNumberString2Number } from "./lib/src/strbin";
import { serialized, skipIfDuplicated } from "./lib/src/lock";
import { LiveSyncCommands } from "./LiveSyncCommands";
export class PluginAndTheirSettings extends LiveSyncCommands {
get deviceAndVaultName() {
return this.plugin.deviceAndVaultName;
}
pluginDialog: PluginDialogModal = null;
periodicPluginSweepProcessor = new PeriodicProcessor(this.plugin, async () => await this.sweepPlugin(false));
showPluginSyncModal() {
if (this.pluginDialog != null) {
this.pluginDialog.open();
} else {
this.pluginDialog = new PluginDialogModal(this.app, this.plugin);
this.pluginDialog.open();
}
}
hidePluginSyncModal() {
if (this.pluginDialog != null) {
this.pluginDialog.close();
this.pluginDialog = null;
}
}
onload(): void | Promise<void> {
this.plugin.addCommand({
id: "livesync-plugin-dialog",
name: "Show Plugins and their settings",
callback: () => {
this.showPluginSyncModal();
},
});
this.showPluginSyncModal();
}
onunload() {
this.hidePluginSyncModal();
this.periodicPluginSweepProcessor?.disable();
}
parseReplicationResultItem(doc: PouchDB.Core.ExistingDocument<EntryDoc>) {
if (isPluginMetadata(doc._id)) {
if (this.settings.notifyPluginOrSettingUpdated) {
this.triggerCheckPluginUpdate();
return true;
}
}
return false;
}
async beforeReplicate(showMessage: boolean) {
if (this.settings.autoSweepPlugins) {
await this.sweepPlugin(showMessage);
}
}
async onResume() {
if (this.plugin.suspended)
return;
if (this.settings.autoSweepPlugins) {
await this.sweepPlugin(false);
}
this.periodicPluginSweepProcessor.enable(this.settings.autoSweepPluginsPeriodic && !this.settings.watchInternalFileChanges ? (PERIODIC_PLUGIN_SWEEP * 1000) : 0);
}
async onInitializeDatabase(showNotice: boolean) {
if (this.settings.usePluginSync) {
try {
Logger("Scanning plugins...");
await this.sweepPlugin(showNotice);
Logger("Scanning plugins done");
} catch (ex) {
Logger("Scanning plugins failed");
Logger(ex, LOG_LEVEL_VERBOSE);
}
}
}
async realizeSettingSyncMode() {
this.periodicPluginSweepProcessor?.disable();
if (this.plugin.suspended)
return;
if (this.settings.autoSweepPlugins) {
await this.sweepPlugin(false);
}
this.periodicPluginSweepProcessor.enable(this.settings.autoSweepPluginsPeriodic && !this.settings.watchInternalFileChanges ? (PERIODIC_PLUGIN_SWEEP * 1000) : 0);
}
triggerCheckPluginUpdate() {
(async () => await this.checkPluginUpdate())();
}
async getPluginList(): Promise<{ plugins: PluginList; allPlugins: DevicePluginList; thisDevicePlugins: DevicePluginList; }> {
const docList = await this.localDatabase.allDocsRaw<PluginDataEntry>({ startkey: PSCHeader, endkey: PSCHeaderEnd, include_docs: false });
const oldDocs: PluginDataEntry[] = ((await Promise.all(docList.rows.map(async (e) => await this.localDatabase.getDBEntry(e.id as FilePathWithPrefix /* WARN!! THIS SHOULD BE WRAPPED */)))).filter((e) => e !== false) as LoadedEntry[]).map((e) => JSON.parse(getDocData(e.data)));
const plugins: { [key: string]: PluginDataEntry[]; } = {};
const allPlugins: { [key: string]: PluginDataEntry; } = {};
const thisDevicePlugins: { [key: string]: PluginDataEntry; } = {};
for (const v of oldDocs) {
if (typeof plugins[v.deviceVaultName] === "undefined") {
plugins[v.deviceVaultName] = [];
}
plugins[v.deviceVaultName].push(v);
allPlugins[v._id] = v;
if (v.deviceVaultName == this.deviceAndVaultName) {
thisDevicePlugins[v.manifest.id] = v;
}
}
return { plugins, allPlugins, thisDevicePlugins };
}
async checkPluginUpdate() {
if (!this.plugin.settings.usePluginSync)
return;
await this.sweepPlugin(false);
const { allPlugins, thisDevicePlugins } = await this.getPluginList();
const arrPlugins = Object.values(allPlugins);
let updateFound = false;
for (const plugin of arrPlugins) {
const ownPlugin = thisDevicePlugins[plugin.manifest.id];
if (ownPlugin) {
const remoteVersion = versionNumberString2Number(plugin.manifest.version);
const ownVersion = versionNumberString2Number(ownPlugin.manifest.version);
if (remoteVersion > ownVersion) {
updateFound = true;
}
if (((plugin.mtime / 1000) | 0) > ((ownPlugin.mtime / 1000) | 0) && (plugin.dataJson ?? "") != (ownPlugin.dataJson ?? "")) {
updateFound = true;
}
}
}
if (updateFound) {
const fragment = createFragment((doc) => {
doc.createEl("a", null, (a) => {
a.text = "There're some new plugins or their settings";
a.addEventListener("click", () => this.showPluginSyncModal());
});
});
NewNotice(fragment, 10000);
} else {
Logger("Everything is up to date.", LOG_LEVEL_NOTICE);
}
}
async sweepPlugin(showMessage = false, specificPluginPath = "") {
if (!this.settings.usePluginSync)
return;
if (!this.localDatabase.isReady)
return;
// @ts-ignore
const pl = this.app.plugins;
const manifests: PluginManifest[] = Object.values(pl.manifests);
let specificPlugin = "";
if (specificPluginPath != "") {
specificPlugin = manifests.find(e => e.dir.endsWith("/" + specificPluginPath))?.id ?? "";
}
await skipIfDuplicated("sweepplugin", async () => {
const logLevel = showMessage ? LOG_LEVEL_NOTICE : LOG_LEVEL_INFO;
if (!this.deviceAndVaultName) {
Logger("You have to set your device name.", LOG_LEVEL_NOTICE);
return;
}
Logger("Scanning plugins", logLevel);
const oldDocs = await this.localDatabase.allDocsRaw<EntryDoc>({
startkey: `ps:${this.deviceAndVaultName}-${specificPlugin}`,
endkey: `ps:${this.deviceAndVaultName}-${specificPlugin}\u{10ffff}`,
include_docs: true,
});
// Logger("OLD DOCS.", LOG_LEVEL_VERBOSE);
// sweep current plugin.
const procs = manifests.map(async (m) => {
const pluginDataEntryID = `ps:${this.deviceAndVaultName}-${m.id}` as DocumentID;
try {
if (specificPlugin && m.id != specificPlugin) {
return;
}
Logger(`Reading plugin:${m.name}(${m.id})`, LOG_LEVEL_VERBOSE);
const path = normalizePath(m.dir) + "/";
const files = ["manifest.json", "main.js", "styles.css", "data.json"];
const pluginData: { [key: string]: string; } = {};
for (const file of files) {
const thePath = path + file;
if (await this.plugin.vaultAccess.adapterExists(thePath)) {
pluginData[file] = await this.plugin.vaultAccess.adapterRead(thePath);
}
}
let mtime = 0;
if (await this.plugin.vaultAccess.adapterExists(path + "/data.json")) {
mtime = (await this.plugin.vaultAccess.adapterStat(path + "/data.json")).mtime;
}
const p: PluginDataEntry = {
_id: pluginDataEntryID,
dataJson: pluginData["data.json"],
deviceVaultName: this.deviceAndVaultName,
mainJs: pluginData["main.js"],
styleCss: pluginData["styles.css"],
manifest: m,
manifestJson: pluginData["manifest.json"],
mtime: mtime,
type: "plugin",
};
const blob = createTextBlob(JSON.stringify(p));
const d: SavingEntry = {
_id: p._id,
path: p._id as string as FilePathWithPrefix,
data: blob,
ctime: mtime,
mtime: mtime,
size: blob.size,
children: [],
datatype: "plain",
type: "plain"
};
Logger(`check diff:${m.name}(${m.id})`, LOG_LEVEL_VERBOSE);
await serialized("plugin-" + m.id, async () => {
const old = await this.localDatabase.getDBEntry(p._id as string as FilePathWithPrefix /* This also should be explained */, null, false, false);
if (old !== false) {
const oldData = { data: old.data, deleted: old._deleted };
const newData = { data: d.data, deleted: d._deleted };
if (await isDocContentSame(oldData.data, newData.data) && oldData.deleted == newData.deleted) {
Logger(`Nothing changed:${m.name}`);
return;
}
}
await this.localDatabase.putDBEntry(d);
Logger(`Plugin saved:${m.name}`, logLevel);
});
} catch (ex) {
Logger(`Plugin save failed:${m.name}`, LOG_LEVEL_NOTICE);
} finally {
oldDocs.rows = oldDocs.rows.filter((e) => e.id != pluginDataEntryID);
}
//remove saved plugin data.
}
);
await Promise.all(procs);
const delDocs = oldDocs.rows.map((e) => {
// e.doc._deleted = true;
if (e.doc.type == "newnote" || e.doc.type == "plain") {
e.doc.deleted = true;
if (this.settings.deleteMetadataOfDeletedFiles) {
e.doc._deleted = true;
}
} else {
e.doc._deleted = true;
}
return e.doc;
});
Logger(`Deleting old plugin:(${delDocs.length})`, LOG_LEVEL_VERBOSE);
await this.localDatabase.bulkDocsRaw(delDocs);
Logger(`Scan plugin done.`, logLevel);
});
}
async applyPluginData(plugin: PluginDataEntry) {
await serialized("plugin-" + plugin.manifest.id, async () => {
const pluginTargetFolderPath = normalizePath(plugin.manifest.dir) + "/";
// @ts-ignore
const stat = this.app.plugins.enabledPlugins.has(plugin.manifest.id) == true;
if (stat) {
// @ts-ignore
await this.app.plugins.unloadPlugin(plugin.manifest.id);
Logger(`Unload plugin:${plugin.manifest.id}`, LOG_LEVEL_NOTICE);
}
if (plugin.dataJson)
await this.plugin.vaultAccess.adapterWrite(pluginTargetFolderPath + "data.json", plugin.dataJson);
Logger("wrote:" + pluginTargetFolderPath + "data.json", LOG_LEVEL_NOTICE);
if (stat) {
// @ts-ignore
await this.app.plugins.loadPlugin(plugin.manifest.id);
Logger(`Load plugin:${plugin.manifest.id}`, LOG_LEVEL_NOTICE);
}
});
}
async applyPlugin(plugin: PluginDataEntry) {
await serialized("plugin-" + plugin.manifest.id, async () => {
// @ts-ignore
const stat = this.app.plugins.enabledPlugins.has(plugin.manifest.id) == true;
if (stat) {
// @ts-ignore
await this.app.plugins.unloadPlugin(plugin.manifest.id);
Logger(`Unload plugin:${plugin.manifest.id}`, LOG_LEVEL_NOTICE);
}
const pluginTargetFolderPath = normalizePath(plugin.manifest.dir) + "/";
if ((await this.plugin.vaultAccess.adapterExists(pluginTargetFolderPath)) === false) {
await this.app.vault.adapter.mkdir(pluginTargetFolderPath);
}
await this.plugin.vaultAccess.adapterWrite(pluginTargetFolderPath + "main.js", plugin.mainJs);
await this.plugin.vaultAccess.adapterWrite(pluginTargetFolderPath + "manifest.json", plugin.manifestJson);
if (plugin.styleCss)
await this.plugin.vaultAccess.adapterWrite(pluginTargetFolderPath + "styles.css", plugin.styleCss);
if (stat) {
// @ts-ignore
await this.app.plugins.loadPlugin(plugin.manifest.id);
Logger(`Load plugin:${plugin.manifest.id}`, LOG_LEVEL_NOTICE);
}
});
}
}

7
src/CmdSetupLiveSync.ts
View File

@@ -50,7 +50,7 @@ export class SetupLiveSync extends LiveSyncCommands {
const encryptingPassphrase = await askString(this.app, "Encrypt your settings", "The passphrase to encrypt the setup URI", "", true); const encryptingPassphrase = await askString(this.app, "Encrypt your settings", "The passphrase to encrypt the setup URI", "", true);
if (encryptingPassphrase === false) if (encryptingPassphrase === false)
return; return;
const setting = { ...this.settings, configPassphraseStore: "", encryptedCouchDBConnection: "", encryptedPassphrase: "" }; const setting = { ...this.settings, configPassphraseStore: "", encryptedCouchDBConnection: "", encryptedPassphrase: "" } as Partial<ObsidianLiveSyncSettings>;
if (stripExtra) { if (stripExtra) {
delete setting.pluginSyncExtendedSetting; delete setting.pluginSyncExtendedSetting;
} }
@@ -134,7 +134,7 @@ export class SetupLiveSync extends LiveSyncCommands {
} else if (setupType == setupAsMerge) { } else if (setupType == setupAsMerge) {
this.plugin.settings = newSettingW; this.plugin.settings = newSettingW;
this.plugin.usedPassphrase = ""; this.plugin.usedPassphrase = "";
await this.fetchLocalWithKeepLocal(); await this.fetchLocalWithRebuild();
} else if (setupType == setupAgain) { } else if (setupType == setupAgain) {
const confirm = "I know this operation will rebuild all my databases with files on this device, and files that are on the remote database and I didn't synchronize to any other devices will be lost and want to proceed indeed."; const confirm = "I know this operation will rebuild all my databases with files on this device, and files that are on the remote database and I didn't synchronize to any other devices will be lost and want to proceed indeed.";
if (await askSelectString(this.app, "Do you really want to do this?", ["Cancel", confirm]) != confirm) { if (await askSelectString(this.app, "Do you really want to do this?", ["Cancel", confirm]) != confirm) {
@@ -377,11 +377,10 @@ Of course, we are able to disable these features.`
await this.plugin.replicateAllFromServer(true); await this.plugin.replicateAllFromServer(true);
await delay(1000); await delay(1000);
await this.plugin.replicateAllFromServer(true); await this.plugin.replicateAllFromServer(true);
await this.fetchRemoteChunks();
await this.resumeReflectingDatabase(); await this.resumeReflectingDatabase();
await this.askHiddenFileConfiguration({ enableFetch: true }); await this.askHiddenFileConfiguration({ enableFetch: true });
} }
async fetchLocalWithKeepLocal() { async fetchLocalWithRebuild() {
return await this.fetchLocal(true); return await this.fetchLocal(true);
} }
async rebuildRemote() { async rebuildRemote() {

35
src/DocumentHistoryModal.ts
View File

@@ -5,7 +5,7 @@ import ObsidianLiveSyncPlugin from "./main";
import { type DocumentID, type FilePathWithPrefix, type LoadedEntry, LOG_LEVEL_INFO, LOG_LEVEL_NOTICE, LOG_LEVEL_VERBOSE } from "./lib/src/types"; import { type DocumentID, type FilePathWithPrefix, type LoadedEntry, LOG_LEVEL_INFO, LOG_LEVEL_NOTICE, LOG_LEVEL_VERBOSE } from "./lib/src/types";
import { Logger } from "./lib/src/logger"; import { Logger } from "./lib/src/logger";
import { isErrorOfMissingDoc } from "./lib/src/utils_couchdb"; import { isErrorOfMissingDoc } from "./lib/src/utils_couchdb";
import { getDocData } from "./lib/src/utils"; import { getDocData, readContent } from "./lib/src/utils";
import { isPlainText, stripPrefix } from "./lib/src/path"; import { isPlainText, stripPrefix } from "./lib/src/path";
function isImage(path: string) { function isImage(path: string) {
@@ -25,7 +25,7 @@ function readDocument(w: LoadedEntry) {
if (isImage(w.path)) { if (isImage(w.path)) {
return new Uint8Array(decodeBinary(w.data)); return new Uint8Array(decodeBinary(w.data));
} }
if (w.data == "plain") return getDocData(w.data); if (w.type == "plain" || w.datatype == "plain") return getDocData(w.data);
if (isComparableTextDecode(w.path)) return readString(new Uint8Array(decodeBinary(w.data))); if (isComparableTextDecode(w.path)) return readString(new Uint8Array(decodeBinary(w.data)));
if (isComparableText(w.path)) return getDocData(w.data); if (isComparableText(w.path)) return getDocData(w.data);
try { try {
@@ -66,7 +66,7 @@ export class DocumentHistoryModal extends Modal {
} }
} }
async loadFile(initialRev: string) { async loadFile(initialRev?: string) {
if (!this.id) { if (!this.id) {
this.id = await this.plugin.path2id(this.file); this.id = await this.plugin.path2id(this.file);
} }
@@ -109,7 +109,7 @@ export class DocumentHistoryModal extends Modal {
if (v) { if (v) {
URL.revokeObjectURL(v); URL.revokeObjectURL(v);
} }
this.BlobURLs.set(key, undefined); this.BlobURLs.delete(key);
} }
generateBlobURL(key: string, data: Uint8Array) { generateBlobURL(key: string, data: Uint8Array) {
this.revokeURL(key); this.revokeURL(key);
@@ -247,12 +247,10 @@ export class DocumentHistoryModal extends Modal {
Logger(`Old content copied to clipboard`, LOG_LEVEL_NOTICE); Logger(`Old content copied to clipboard`, LOG_LEVEL_NOTICE);
}); });
}); });
async function focusFile(path: string) { const focusFile = async (path: string) => {
const targetFile = app.vault const targetFile = this.plugin.app.vault.getFileByPath(path);
.getFiles()
.find((f) => f.path === path);
if (targetFile) { if (targetFile) {
const leaf = app.workspace.getLeaf(false); const leaf = this.plugin.app.workspace.getLeaf(false);
await leaf.openFile(targetFile); await leaf.openFile(targetFile);
} else { } else {
Logger("The file could not view on the editor", LOG_LEVEL_NOTICE) Logger("The file could not view on the editor", LOG_LEVEL_NOTICE)
@@ -265,19 +263,16 @@ export class DocumentHistoryModal extends Modal {
const pathToWrite = stripPrefix(this.file); const pathToWrite = stripPrefix(this.file);
if (!isValidPath(pathToWrite)) { if (!isValidPath(pathToWrite)) {
Logger("Path is not valid to write content.", LOG_LEVEL_INFO); Logger("Path is not valid to write content.", LOG_LEVEL_INFO);
return;
} }
if (this.currentDoc?.datatype == "plain") { if (!this.currentDoc) {
await this.plugin.vaultAccess.adapterWrite(pathToWrite, getDocData(this.currentDoc.data)); Logger("No active file loaded.", LOG_LEVEL_INFO);
await focusFile(pathToWrite); return;
this.close();
} else if (this.currentDoc?.datatype == "newnote") {
await this.plugin.vaultAccess.adapterWrite(pathToWrite, decodeBinary(this.currentDoc.data));
await focusFile(pathToWrite);
this.close();
} else {
Logger(`Could not parse entry`, LOG_LEVEL_NOTICE);
} }
const d = readContent(this.currentDoc);
await this.plugin.vaultAccess.adapterWrite(pathToWrite, d);
await focusFile(pathToWrite);
this.close();
}); });
}); });
} }

27
src/GlobalHistory.svelte
View File

@@ -2,12 +2,11 @@
import ObsidianLiveSyncPlugin from "./main"; import ObsidianLiveSyncPlugin from "./main";
import { onDestroy, onMount } from "svelte"; import { onDestroy, onMount } from "svelte";
import type { AnyEntry, FilePathWithPrefix } from "./lib/src/types"; import type { AnyEntry, FilePathWithPrefix } from "./lib/src/types";
import { createBinaryBlob, getDocData, isDocContentSame } from "./lib/src/utils"; import { getDocData, isAnyNote, isDocContentSame, readAsBlob } from "./lib/src/utils";
import { diff_match_patch } from "./deps"; import { diff_match_patch } from "./deps";
import { DocumentHistoryModal } from "./DocumentHistoryModal"; import { DocumentHistoryModal } from "./DocumentHistoryModal";
import { isPlainText, stripAllPrefixes } from "./lib/src/path"; import { isPlainText, stripAllPrefixes } from "./lib/src/path";
import { TFile } from "./deps"; import { TFile } from "./deps";
import { decodeBinary } from "./lib/src/strbin";
export let plugin: ObsidianLiveSyncPlugin; export let plugin: ObsidianLiveSyncPlugin;
let showDiffInfo = false; let showDiffInfo = false;
@@ -31,7 +30,7 @@
type HistoryData = { type HistoryData = {
id: string; id: string;
rev: string; rev?: string;
path: string; path: string;
dirname: string; dirname: string;
filename: string; filename: string;
@@ -54,12 +53,12 @@
if (docA.mtime < range_from_epoch) { if (docA.mtime < range_from_epoch) {
continue; continue;
} }
if (docA.type != "newnote" && docA.type != "plain") continue; if (!isAnyNote(docA)) continue;
const path = plugin.getPath(docA as AnyEntry); const path = plugin.getPath(docA as AnyEntry);
const isPlain = isPlainText(docA.path); const isPlain = isPlainText(docA.path);
const revs = await db.getRaw(docA._id, { revs_info: true }); const revs = await db.getRaw(docA._id, { revs_info: true });
let p: string = undefined; let p: string | undefined = undefined;
const reversedRevs = revs._revs_info.reverse(); const reversedRevs = (revs._revs_info ?? []).reverse();
const DIFF_DELETE = -1; const DIFF_DELETE = -1;
const DIFF_EQUAL = 0; const DIFF_EQUAL = 0;
@@ -107,15 +106,9 @@
if (checkStorageDiff) { if (checkStorageDiff) {
const abs = plugin.vaultAccess.getAbstractFileByPath(stripAllPrefixes(plugin.getPath(docA))); const abs = plugin.vaultAccess.getAbstractFileByPath(stripAllPrefixes(plugin.getPath(docA)));
if (abs instanceof TFile) { if (abs instanceof TFile) {
let result = false; const data = await plugin.vaultAccess.adapterReadAuto(abs);
if (isPlainText(docA.path)) { const d = readAsBlob(doc);
const data = await plugin.vaultAccess.adapterRead(abs); const result = await isDocContentSame(data, d);
result = await isDocContentSame(data, doc.data);
} else {
const data = await plugin.vaultAccess.adapterReadBinary(abs);
const dataEEncoded = createBinaryBlob(data);
result = await isDocContentSame(dataEEncoded, createBinaryBlob(decodeBinary(doc.data)));
}
if (result) { if (result) {
diffDetail += " ⚖️"; diffDetail += " ⚖️";
} else { } else {
@@ -184,7 +177,7 @@
onDestroy(() => {}); onDestroy(() => {});
function showHistory(file: string, rev: string) { function showHistory(file: string, rev: string) {
new DocumentHistoryModal(plugin.app, plugin, file as unknown as FilePathWithPrefix, null, rev).open(); new DocumentHistoryModal(plugin.app, plugin, file as unknown as FilePathWithPrefix, undefined, rev).open();
} }
function openFile(file: string) { function openFile(file: string) {
plugin.app.workspace.openLinkText(file, file); plugin.app.workspace.openLinkText(file, file);
@@ -239,7 +232,7 @@
<td> <td>
<span class="rev"> <span class="rev">
{#if entry.isPlain} {#if entry.isPlain}
<a on:click={() => showHistory(entry.path, entry.rev)}>{entry.rev}</a> <a on:click={() => showHistory(entry.path, entry?.rev || "")}>{entry.rev}</a>
{:else} {:else}
{entry.rev} {entry.rev}
{/if} {/if}

31
src/JsonResolvePane.svelte
View File

@@ -6,15 +6,15 @@
import { mergeObject } from "./utils"; import { mergeObject } from "./utils";
export let docs: LoadedEntry[] = []; export let docs: LoadedEntry[] = [];
export let callback: (keepRev: string, mergedStr?: string) => Promise<void> = async (_, __) => { export let callback: (keepRev?: string, mergedStr?: string) => Promise<void> = async (_, __) => {
Promise.resolve(); Promise.resolve();
}; };
export let filename: FilePath = "" as FilePath; export let filename: FilePath = "" as FilePath;
export let nameA: string = "A"; export let nameA: string = "A";
export let nameB: string = "B"; export let nameB: string = "B";
export let defaultSelect: string = ""; export let defaultSelect: string = "";
let docA: LoadedEntry = undefined; let docA: LoadedEntry;
let docB: LoadedEntry = undefined; let docB: LoadedEntry;
let docAContent = ""; let docAContent = "";
let docBContent = ""; let docBContent = "";
let objA: any = {}; let objA: any = {};
@@ -28,7 +28,8 @@
function docToString(doc: LoadedEntry) { function docToString(doc: LoadedEntry) {
return doc.datatype == "plain" ? getDocData(doc.data) : readString(new Uint8Array(decodeBinary(doc.data))); return doc.datatype == "plain" ? getDocData(doc.data) : readString(new Uint8Array(decodeBinary(doc.data)));
} }
function revStringToRevNumber(rev: string) { function revStringToRevNumber(rev?: string) {
if (!rev) return "";
return rev.split("-")[0]; return rev.split("-")[0];
} }
@@ -44,15 +45,15 @@
} }
function apply() { function apply() {
if (docA._id == docB._id) { if (docA._id == docB._id) {
if (mode == "A") return callback(docA._rev, null); if (mode == "A") return callback(docA._rev!, undefined);
if (mode == "B") return callback(docB._rev, null); if (mode == "B") return callback(docB._rev!, undefined);
} else { } else {
if (mode == "A") return callback(null, docToString(docA)); if (mode == "A") return callback(undefined, docToString(docA));
if (mode == "B") return callback(null, docToString(docB)); if (mode == "B") return callback(undefined, docToString(docB));
} }
if (mode == "BA") return callback(null, JSON.stringify(objBA, null, 2)); if (mode == "BA") return callback(undefined, JSON.stringify(objBA, null, 2));
if (mode == "AB") return callback(null, JSON.stringify(objAB, null, 2)); if (mode == "AB") return callback(undefined, JSON.stringify(objAB, null, 2));
callback(null, null); callback(undefined, undefined);
} }
$: { $: {
if (docs && docs.length >= 1) { if (docs && docs.length >= 1) {
@@ -133,13 +134,17 @@
{/if} {/if}
<div> <div>
{nameA} {nameA}
{#if docA._id == docB._id} Rev:{revStringToRevNumber(docA._rev)} {/if} ,{new Date(docA.mtime).toLocaleString()} {#if docA._id == docB._id}
Rev:{revStringToRevNumber(docA._rev)}
{/if} ,{new Date(docA.mtime).toLocaleString()}
{docAContent.length} letters {docAContent.length} letters
</div> </div>
<div> <div>
{nameB} {nameB}
{#if docA._id == docB._id} Rev:{revStringToRevNumber(docB._rev)} {/if} ,{new Date(docB.mtime).toLocaleString()} {#if docA._id == docB._id}
Rev:{revStringToRevNumber(docB._rev)}
{/if} ,{new Date(docB.mtime).toLocaleString()}
{docBContent.length} letters {docBContent.length} letters
</div> </div>

5
src/KeyValueDB.ts
View File

@@ -6,7 +6,7 @@ export interface KeyValueDatabase {
clear(): Promise<void>; clear(): Promise<void>;
keys(query?: IDBValidKey | IDBKeyRange, count?: number): Promise<IDBValidKey[]>; keys(query?: IDBValidKey | IDBKeyRange, count?: number): Promise<IDBValidKey[]>;
close(): void; close(): void;
destroy(): void; destroy(): Promise<void>;
} }
const databaseCache: { [key: string]: IDBPDatabase<any> } = {}; const databaseCache: { [key: string]: IDBPDatabase<any> } = {};
export const OpenKeyValueDatabase = async (dbKey: string): Promise<KeyValueDatabase> => { export const OpenKeyValueDatabase = async (dbKey: string): Promise<KeyValueDatabase> => {
@@ -20,8 +20,7 @@ export const OpenKeyValueDatabase = async (dbKey: string): Promise<KeyValueDatab
db.createObjectStore(storeKey); db.createObjectStore(storeKey);
}, },
}); });
let db: IDBPDatabase<any> = null; const db = await dbPromise;
db = await dbPromise;
databaseCache[dbKey] = db; databaseCache[dbKey] = db;
return { return {
get<T>(key: string): Promise<T> { get<T>(key: string): Promise<T> {

83
src/MultipleRegExpControl.svelte Normal file
View File

@@ -0,0 +1,83 @@
<script lang="ts">
export let patterns = [] as string[];
export let originals = [] as string[];
export let apply: (args: string[]) => Promise<void> = (_: string[]) => Promise.resolve();
function revert() {
patterns = [...originals];
}
const CHECK_OK = "✔";
const CHECK_NG = "⚠";
const MARK_MODIFIED = "✏ ";
function checkRegExp(pattern: string) {
if (pattern.trim() == "") return "";
try {
const _ = new RegExp(pattern);
return CHECK_OK;
} catch (ex) {
return CHECK_NG;
}
}
$: status = patterns.map((e) => checkRegExp(e));
$: modified = patterns.map((e, i) => (e != originals?.[i] ?? "" ? MARK_MODIFIED : ""));
function remove(idx: number) {
patterns[idx] = "";
}
function add() {
patterns = [...patterns, ""];
}
</script>
<ul>
{#each patterns as pattern, idx}
<li><label>{modified[idx]}{status[idx]}</label><input type="text" bind:value={pattern} class={modified[idx]} /><button class="iconbutton" on:click={() => remove(idx)}>🗑</button></li>
{/each}
<li>
<label><button on:click={() => add()}>Add</button></label>
</li>
<li class="buttons">
<button on:click={() => apply(patterns)} disabled={status.some((e) => e == CHECK_NG) || modified.every((e) => e == "")}>Apply</button>
<button on:click={() => revert()} disabled={status.some((e) => e == CHECK_NG) || modified.every((e) => e == "")}>Revert</button>
</li>
</ul>
<style>
label {
min-width: 4em;
width: 4em;
display: inline-flex;
flex-direction: row;
justify-content: flex-end;
}
ul {
flex-grow: 1;
display: inline-flex;
flex-direction: column;
list-style-type: none;
margin-block-start: 0;
margin-block-end: 0;
margin-inline-start: 0px;
margin-inline-end: 0px;
padding-inline-start: 0;
}
li {
padding: var(--size-2-1) var(--size-4-1);
display: inline-flex;
flex-grow: 1;
align-items: center;
justify-content: flex-end;
gap: var(--size-4-2);
}
li input {
min-width: 10em;
}
li.buttons {
}
button.iconbutton {
max-width: 4em;
}
span.spacer {
flex-grow: 1;
}
</style>

362
src/ObsidianLiveSyncSettingTab.ts
View File

@@ -1,13 +1,15 @@
import { App, PluginSettingTab, Setting, sanitizeHTMLToDom, TextAreaComponent, MarkdownRenderer, stringifyYaml } from "./deps"; import { App, PluginSettingTab, Setting, sanitizeHTMLToDom, MarkdownRenderer, stringifyYaml } from "./deps";
import { DEFAULT_SETTINGS, type ObsidianLiveSyncSettings, type ConfigPassphraseStore, type RemoteDBSettings, type FilePathWithPrefix, type HashAlgorithm, type DocumentID, LOG_LEVEL_NOTICE, LOG_LEVEL_VERBOSE, LOG_LEVEL_INFO } from "./lib/src/types"; import { DEFAULT_SETTINGS, type ObsidianLiveSyncSettings, type ConfigPassphraseStore, type RemoteDBSettings, type FilePathWithPrefix, type HashAlgorithm, type DocumentID, LOG_LEVEL_NOTICE, LOG_LEVEL_VERBOSE, LOG_LEVEL_INFO, type LoadedEntry, PREFERRED_SETTING_CLOUDANT, PREFERRED_SETTING_SELF_HOSTED, FLAGMD_REDFLAG2_HR, FLAGMD_REDFLAG3_HR } from "./lib/src/types";
import { createBinaryBlob, createTextBlob, delay, isDocContentSame } from "./lib/src/utils"; import { createBlob, delay, isDocContentSame, readAsBlob } from "./lib/src/utils";
import { versionNumberString2Number } from "./lib/src/strbin"; import { versionNumberString2Number } from "./lib/src/strbin";
import { Logger } from "./lib/src/logger"; import { Logger } from "./lib/src/logger";
import { checkSyncInfo, isCloudantURI } from "./lib/src/utils_couchdb"; import { checkSyncInfo, isCloudantURI } from "./lib/src/utils_couchdb";
import { testCrypt } from "./lib/src/e2ee_v2"; import { testCrypt } from "./lib/src/e2ee_v2";
import ObsidianLiveSyncPlugin from "./main"; import ObsidianLiveSyncPlugin from "./main";
import { askYesNo, performRebuildDB, requestToCouchDB, scheduleTask } from "./utils"; import { askYesNo, performRebuildDB, requestToCouchDB, scheduleTask } from "./utils";
import { request, type ButtonComponent } from "obsidian"; import { request, type ButtonComponent, TFile } from "obsidian";
import { shouldBeIgnored } from "./lib/src/path";
import MultipleRegExpControl from './MultipleRegExpControl.svelte';
export class ObsidianLiveSyncSettingTab extends PluginSettingTab { export class ObsidianLiveSyncSettingTab extends PluginSettingTab {
@@ -45,11 +47,6 @@ export class ObsidianLiveSyncSettingTab extends PluginSettingTab {
let useDynamicIterationCount = this.plugin.settings.useDynamicIterationCount; let useDynamicIterationCount = this.plugin.settings.useDynamicIterationCount;
containerEl.empty(); containerEl.empty();
// const preferred_setting = isCloudantURI(this.plugin.settings.couchDB_URI) ? PREFERRED_SETTING_CLOUDANT : PREFERRED_SETTING_SELF_HOSTED;
// const default_setting = { ...DEFAULT_SETTINGS };
containerEl.createEl("h2", { text: "Settings for Self-hosted LiveSync." }); containerEl.createEl("h2", { text: "Settings for Self-hosted LiveSync." });
containerEl.addClass("sls-setting"); containerEl.addClass("sls-setting");
containerEl.removeClass("isWizard"); containerEl.removeClass("isWizard");
@@ -85,11 +82,11 @@ export class ObsidianLiveSyncSettingTab extends PluginSettingTab {
} }
w.querySelectorAll(`.sls-setting-label`).forEach((element) => { w.querySelectorAll(`.sls-setting-label`).forEach((element) => {
element.removeClass("selected"); element.removeClass("selected");
(element.querySelector<HTMLInputElement>("input[type=radio]")).checked = false; (element.querySelector<HTMLInputElement>("input[type=radio]"))!.checked = false;
}); });
w.querySelectorAll(`.sls-setting-label.c-${screen}`).forEach((element) => { w.querySelectorAll(`.sls-setting-label.c-${screen}`).forEach((element) => {
element.addClass("selected"); element.addClass("selected");
(element.querySelector<HTMLInputElement>("input[type=radio]")).checked = true; (element.querySelector<HTMLInputElement>("input[type=radio]"))!.checked = true;
}); });
this.selectedScreen = screen; this.selectedScreen = screen;
}; };
@@ -119,7 +116,7 @@ export class ObsidianLiveSyncSettingTab extends PluginSettingTab {
tmpDiv.innerHTML = `<button> OK, I read all. </button>`; tmpDiv.innerHTML = `<button> OK, I read all. </button>`;
if (lastVersion > this.plugin.settings.lastReadUpdates) { if (lastVersion > this.plugin.settings.lastReadUpdates) {
const informationButtonDiv = h3El.appendChild(tmpDiv); const informationButtonDiv = h3El.appendChild(tmpDiv);
informationButtonDiv.querySelector("button").addEventListener("click", async () => { informationButtonDiv.querySelector("button")?.addEventListener("click", async () => {
this.plugin.settings.lastReadUpdates = lastVersion; this.plugin.settings.lastReadUpdates = lastVersion;
await this.plugin.saveSettings(); await this.plugin.saveSettings();
informationButtonDiv.remove(); informationButtonDiv.remove();
@@ -229,23 +226,23 @@ export class ObsidianLiveSyncSettingTab extends PluginSettingTab {
let remoteTroubleShootMDSrc = ""; let remoteTroubleShootMDSrc = "";
try { try {
remoteTroubleShootMDSrc = await request(`${rawRepoURI}${basePath}/${filename}`); remoteTroubleShootMDSrc = await request(`${rawRepoURI}${basePath}/${filename}`);
} catch (ex) { } catch (ex: any) {
remoteTroubleShootMDSrc = "Error Occurred!!\n" + ex.toString(); remoteTroubleShootMDSrc = "Error Occurred!!\n" + ex.toString();
} }
const remoteTroubleShootMD = remoteTroubleShootMDSrc.replace(/\((.*?(.png)|(.jpg))\)/g, `(${rawRepoURI}${basePath}/$1)`) const remoteTroubleShootMD = remoteTroubleShootMDSrc.replace(/\((.*?(.png)|(.jpg))\)/g, `(${rawRepoURI}${basePath}/$1)`)
// Render markdown // Render markdown
await MarkdownRenderer.render(this.plugin.app, `<a class='sls-troubleshoot-anchor'></a> [Tips and Troubleshooting](${topPath}) [PageTop](${filename})\n\n${remoteTroubleShootMD}`, troubleShootEl, `${rawRepoURI}`, this.plugin); await MarkdownRenderer.render(this.plugin.app, `<a class='sls-troubleshoot-anchor'></a> [Tips and Troubleshooting](${topPath}) [PageTop](${filename})\n\n${remoteTroubleShootMD}`, troubleShootEl, `${rawRepoURI}`, this.plugin);
// Menu // Menu
troubleShootEl.querySelector<HTMLAnchorElement>(".sls-troubleshoot-anchor") troubleShootEl.querySelector<HTMLAnchorElement>(".sls-troubleshoot-anchor")?.parentElement?.setCssStyles({ position: "sticky", top: "-1em", backgroundColor: "var(--modal-background)" });
.parentElement.setCssStyles({ position: "sticky", top: "-1em", backgroundColor: "var(--modal-background)" });
// Trap internal links. // Trap internal links.
troubleShootEl.querySelectorAll<HTMLAnchorElement>("a.internal-link").forEach((anchorEl) => { troubleShootEl.querySelectorAll<HTMLAnchorElement>("a.internal-link").forEach((anchorEl) => {
anchorEl.addEventListener("click", async (evt) => { anchorEl.addEventListener("click", async (evt) => {
const uri = anchorEl.getAttr("data-href"); const uri = anchorEl.getAttr("data-href");
if (!uri) return;
if (uri.startsWith("#")) { if (uri.startsWith("#")) {
evt.preventDefault(); evt.preventDefault();
const elements = Array.from(troubleShootEl.querySelectorAll<HTMLHeadingElement>("[data-heading]")) const elements = Array.from(troubleShootEl.querySelectorAll<HTMLHeadingElement>("[data-heading]"))
const p = elements.find(e => e.getAttr("data-heading").toLowerCase().split(" ").join("-") == uri.substring(1).toLowerCase()); const p = elements.find(e => e.getAttr("data-heading")?.toLowerCase().split(" ").join("-") == uri.substring(1).toLowerCase());
if (p) { if (p) {
p.setCssStyles({ scrollMargin: "3em" }); p.setCssStyles({ scrollMargin: "3em" });
p.scrollIntoView({ behavior: "instant", block: "start" }); p.scrollIntoView({ behavior: "instant", block: "start" });
@@ -266,6 +263,16 @@ export class ObsidianLiveSyncSettingTab extends PluginSettingTab {
const containerRemoteDatabaseEl = containerEl.createDiv(); const containerRemoteDatabaseEl = containerEl.createDiv();
containerRemoteDatabaseEl.createEl("h3", { text: "Remote Database configuration" }); containerRemoteDatabaseEl.createEl("h3", { text: "Remote Database configuration" });
const syncWarn = containerRemoteDatabaseEl.createEl("div", { text: `These settings are kept locked while any synchronization options are enabled. Disable these options in the "Sync Settings" tab to unlock.` }); const syncWarn = containerRemoteDatabaseEl.createEl("div", { text: `These settings are kept locked while any synchronization options are enabled. Disable these options in the "Sync Settings" tab to unlock.` });
if (this.plugin.settings.couchDB_URI.startsWith("http://")) {
if (this.plugin.isMobile) {
containerRemoteDatabaseEl.createEl("div", { text: `Configured as using plain HTTP. We cannot connect to the remote. Please set up the credentials and use HTTPS for the remote URI.` })
.addClass("op-warn");
} else {
containerRemoteDatabaseEl.createEl("div", { text: `Configured as using plain HTTP. We might fail on mobile devices.` })
.addClass("op-warn-info");
}
}
syncWarn.addClass("op-warn-info"); syncWarn.addClass("op-warn-info");
syncWarn.addClass("sls-hidden"); syncWarn.addClass("sls-hidden");
@@ -403,7 +410,7 @@ export class ObsidianLiveSyncSettingTab extends PluginSettingTab {
tmpDiv.addClass("ob-btn-config-fix"); tmpDiv.addClass("ob-btn-config-fix");
tmpDiv.innerHTML = `<label>${title}</label><button>Fix</button>`; tmpDiv.innerHTML = `<label>${title}</label><button>Fix</button>`;
const x = checkResultDiv.appendChild(tmpDiv); const x = checkResultDiv.appendChild(tmpDiv);
x.querySelector("button").addEventListener("click", async () => { x.querySelector("button")?.addEventListener("click", async () => {
Logger(`CouchDB Configuration: ${title} -> Set ${key} to ${value}`) Logger(`CouchDB Configuration: ${title} -> Set ${key} to ${value}`)
const res = await requestToCouchDB(this.plugin.settings.couchDB_URI, this.plugin.settings.couchDB_USER, this.plugin.settings.couchDB_PASSWORD, undefined, key, value); const res = await requestToCouchDB(this.plugin.settings.couchDB_URI, this.plugin.settings.couchDB_USER, this.plugin.settings.couchDB_PASSWORD, undefined, key, value);
if (res.status == 200) { if (res.status == 200) {
@@ -499,15 +506,11 @@ export class ObsidianLiveSyncSettingTab extends PluginSettingTab {
const origins = ["app://obsidian.md", "capacitor://localhost", "http://localhost"]; const origins = ["app://obsidian.md", "capacitor://localhost", "http://localhost"];
for (const org of origins) { for (const org of origins) {
const rr = await requestToCouchDB(this.plugin.settings.couchDB_URI, this.plugin.settings.couchDB_USER, this.plugin.settings.couchDB_PASSWORD, org); const rr = await requestToCouchDB(this.plugin.settings.couchDB_URI, this.plugin.settings.couchDB_USER, this.plugin.settings.couchDB_PASSWORD, org);
const responseHeaders = Object.entries(rr.headers) const responseHeaders = Object.fromEntries(Object.entries(rr.headers)
.map((e) => { .map((e) => {
e[0] = (e[0] + "").toLowerCase(); e[0] = `${e[0]}`.toLowerCase();
return e; return e;
}) }));
.reduce((obj, [key, val]) => {
obj[key] = val;
return obj;
}, {} as { [key: string]: string });
addResult(`Origin check:${org}`); addResult(`Origin check:${org}`);
if (responseHeaders["access-control-allow-credentials"] != "true") { if (responseHeaders["access-control-allow-credentials"] != "true") {
addResult("❗ CORS is not allowing credential"); addResult("❗ CORS is not allowing credential");
@@ -523,7 +526,7 @@ export class ObsidianLiveSyncSettingTab extends PluginSettingTab {
addResult("--Done--", ["ob-btn-config-head"]); addResult("--Done--", ["ob-btn-config-head"]);
addResult("If you have some trouble with Connection-check even though all Config-check has been passed, Please check your reverse proxy's configuration.", ["ob-btn-config-info"]); addResult("If you have some trouble with Connection-check even though all Config-check has been passed, Please check your reverse proxy's configuration.", ["ob-btn-config-info"]);
Logger(`Checking configuration done`, LOG_LEVEL_INFO); Logger(`Checking configuration done`, LOG_LEVEL_INFO);
} catch (ex) { } catch (ex: any) {
if (ex?.status == 401) { if (ex?.status == 401) {
addResult(`❗ Access forbidden.`); addResult(`❗ Access forbidden.`);
addResult(`We could not continue the test.`); addResult(`We could not continue the test.`);
@@ -541,6 +544,19 @@ export class ObsidianLiveSyncSettingTab extends PluginSettingTab {
text: "", text: "",
}); });
containerRemoteDatabaseEl.createEl("h4", { text: "Effective Storage Using" });
new Setting(containerRemoteDatabaseEl)
.setName("Data Compression (Experimental)")
.setDesc("Compresses data during transfer, saving space in the remote database. Note: Please ensure that all devices have v0.22.18 and connected tools are also supported compression.")
.addToggle((toggle) =>
toggle.setValue(this.plugin.settings.enableCompression).onChange(async (value) => {
this.plugin.settings.enableCompression = value;
await this.plugin.saveSettings();
this.display();
})
);
containerRemoteDatabaseEl.createEl("h4", { text: "Confidentiality" }); containerRemoteDatabaseEl.createEl("h4", { text: "Confidentiality" });
const e2e = new Setting(containerRemoteDatabaseEl) const e2e = new Setting(containerRemoteDatabaseEl)
@@ -663,7 +679,7 @@ export class ObsidianLiveSyncSettingTab extends PluginSettingTab {
.setWarning() .setWarning()
.setDisabled(false) .setDisabled(false)
.onClick(async () => { .onClick(async () => {
await rebuildDB("localOnlyWithChunks"); await rebuildDB("localOnly");
}) })
) )
.addButton((button) => .addButton((button) =>
@@ -773,9 +789,10 @@ export class ObsidianLiveSyncSettingTab extends PluginSettingTab {
this.plugin.settings.passphrase = ""; this.plugin.settings.passphrase = "";
} }
if (isCloudantURI(this.plugin.settings.couchDB_URI)) { if (isCloudantURI(this.plugin.settings.couchDB_URI)) {
this.plugin.settings.customChunkSize = 0; // this.plugin.settings.customChunkSize = 0;
this.plugin.settings = { ...this.plugin.settings, ...PREFERRED_SETTING_CLOUDANT };
} else { } else {
this.plugin.settings.customChunkSize = 50; this.plugin.settings = { ...this.plugin.settings, ...PREFERRED_SETTING_SELF_HOSTED };
} }
changeDisplay("30") changeDisplay("30")
}) })
@@ -791,7 +808,7 @@ export class ObsidianLiveSyncSettingTab extends PluginSettingTab {
new Setting(containerGeneralSettingsEl) new Setting(containerGeneralSettingsEl)
.setName("Show status inside the editor") .setName("Show status inside the editor")
.setDesc("") .setDesc("Reflected after reboot")
.addToggle((toggle) => .addToggle((toggle) =>
toggle.setValue(this.plugin.settings.showStatusOnEditor).onChange(async (value) => { toggle.setValue(this.plugin.settings.showStatusOnEditor).onChange(async (value) => {
this.plugin.settings.showStatusOnEditor = value; this.plugin.settings.showStatusOnEditor = value;
@@ -810,7 +827,16 @@ export class ObsidianLiveSyncSettingTab extends PluginSettingTab {
}) })
); );
} }
new Setting(containerGeneralSettingsEl)
.setName("Show status on the status bar")
.setDesc("Reflected after reboot.")
.addToggle((toggle) =>
toggle.setValue(this.plugin.settings.showStatusOnStatusbar).onChange(async (value) => {
this.plugin.settings.showStatusOnStatusbar = value;
await this.plugin.saveSettings();
this.display();
})
);
containerGeneralSettingsEl.createEl("h4", { text: "Logging" }); containerGeneralSettingsEl.createEl("h4", { text: "Logging" });
new Setting(containerGeneralSettingsEl) new Setting(containerGeneralSettingsEl)
.setName("Show only notifications") .setName("Show only notifications")
@@ -1053,7 +1079,7 @@ export class ObsidianLiveSyncSettingTab extends PluginSettingTab {
if (!this.plugin.settings.isConfigured) { if (!this.plugin.settings.isConfigured) {
this.plugin.settings.isConfigured = true; this.plugin.settings.isConfigured = true;
await this.plugin.saveSettings(); await this.plugin.saveSettings();
await rebuildDB("localOnlyWithChunks"); await rebuildDB("localOnly");
Logger("All done! Please set up subsequent devices with 'Copy current settings as a new setup URI' and 'Use the copied setup URI'.", LOG_LEVEL_NOTICE); Logger("All done! Please set up subsequent devices with 'Copy current settings as a new setup URI' and 'Use the copied setup URI'.", LOG_LEVEL_NOTICE);
await this.plugin.addOnSetup.command_copySetupURI(); await this.plugin.addOnSetup.command_copySetupURI();
} else { } else {
@@ -1325,43 +1351,48 @@ export class ObsidianLiveSyncSettingTab extends PluginSettingTab {
}); });
text.inputEl.setAttribute("type", "number"); text.inputEl.setAttribute("type", "number");
}); });
let skipPatternTextArea: TextAreaComponent = null; const defaultSkipPattern = "\\/node_modules\\/, \\/\\.git\\/, ^\\.git\\/, \\/obsidian-livesync\\/";
const defaultSkipPattern = "\\/node_modules\\/, \\/\\.git\\/, \\/obsidian-livesync\\/";
const defaultSkipPatternXPlat = defaultSkipPattern + ",\\/workspace$ ,\\/workspace.json$,\\/workspace-mobile.json$"; const defaultSkipPatternXPlat = defaultSkipPattern + ",\\/workspace$ ,\\/workspace.json$,\\/workspace-mobile.json$";
new Setting(containerSyncSettingEl)
.setName("Folders and files to ignore") const pat = this.plugin.settings.syncInternalFilesIgnorePatterns.split(",").map(x => x.trim()).filter(x => x != "");
.setDesc( const patSetting = new Setting(containerSyncSettingEl)
"Regular expression, If you use hidden file sync between desktop and mobile, adding `workspace$` is recommended." .setName("Hidden files ignore patterns")
) .setDesc("");
.setClass("wizardHidden")
.addTextArea((text) => { new MultipleRegExpControl(
text {
.setValue(this.plugin.settings.syncInternalFilesIgnorePatterns) target: patSetting.controlEl,
.setPlaceholder("\\/node_modules\\/, \\/\\.git\\/") props: {
.onChange(async (value) => { patterns: pat, originals: [...pat], apply: async (newPatterns) => {
this.plugin.settings.syncInternalFilesIgnorePatterns = value; this.plugin.settings.syncInternalFilesIgnorePatterns = newPatterns.map(e => e.trim()).filter(e => e != "").join(", ");
await this.plugin.saveSettings(); await this.plugin.saveSettings();
}) this.display();
skipPatternTextArea = text; }
return text; }
} }
); )
const addDefaultPatterns = async (patterns: string) => {
const oldList = this.plugin.settings.syncInternalFilesIgnorePatterns.split(",").map(x => x.trim()).filter(x => x != "");
const newList = patterns.split(",").map(x => x.trim()).filter(x => x != "");
const allSet = new Set([...oldList, ...newList]);
this.plugin.settings.syncInternalFilesIgnorePatterns = [...allSet].join(", ");
await this.plugin.saveSettings();
this.display();
}
new Setting(containerSyncSettingEl) new Setting(containerSyncSettingEl)
.setName("Restore the skip pattern to default") .setName("Add default patterns")
.setClass("wizardHidden") .setClass("wizardHidden")
.addButton((button) => { .addButton((button) => {
button.setButtonText("Default") button.setButtonText("Default")
.onClick(async () => { .onClick(async () => {
skipPatternTextArea.setValue(defaultSkipPattern); await addDefaultPatterns(defaultSkipPattern);
this.plugin.settings.syncInternalFilesIgnorePatterns = defaultSkipPattern;
await this.plugin.saveSettings();
}) })
}).addButton((button) => { }).addButton((button) => {
button.setButtonText("Cross-platform") button.setButtonText("Cross-platform")
.onClick(async () => { .onClick(async () => {
skipPatternTextArea.setValue(defaultSkipPatternXPlat); await addDefaultPatterns(defaultSkipPatternXPlat);
this.plugin.settings.syncInternalFilesIgnorePatterns = defaultSkipPatternXPlat;
await this.plugin.saveSettings();
}) })
}) })
@@ -1413,54 +1444,41 @@ export class ObsidianLiveSyncSettingTab extends PluginSettingTab {
containerSyncSettingEl.createEl("h4", { containerSyncSettingEl.createEl("h4", {
text: sanitizeHTMLToDom(`Targets`), text: sanitizeHTMLToDom(`Targets`),
}).addClass("wizardHidden"); }).addClass("wizardHidden");
new Setting(containerSyncSettingEl)
const syncFilesSetting = new Setting(containerSyncSettingEl)
.setName("Synchronising files") .setName("Synchronising files")
.setDesc("(RegExp) Empty to sync all files. set filter as a regular expression to limit synchronising files.") .setDesc("(RegExp) Empty to sync all files. set filter as a regular expression to limit synchronising files.")
.setClass("wizardHidden") .setClass("wizardHidden")
.addTextArea((text) => { new MultipleRegExpControl(
text {
.setValue(this.plugin.settings.syncOnlyRegEx) target: syncFilesSetting.controlEl,
.setPlaceholder("\\.md$|\\.txt") props: {
.onChange(async (value) => { patterns: this.plugin.settings.syncOnlyRegEx.split("|[]|"), originals: [...this.plugin.settings.syncOnlyRegEx.split("|[]|")], apply: async (newPatterns) => {
let isValidRegExp = false; this.plugin.settings.syncOnlyRegEx = newPatterns.map(e => e.trim()).filter(e => e != "").join("|[]|");
try { await this.plugin.saveSettings();
new RegExp(value); this.display();
isValidRegExp = true; }
} catch (_) { }
// NO OP.
}
if (isValidRegExp || value.trim() == "") {
this.plugin.settings.syncOnlyRegEx = value;
await this.plugin.saveSettings();
}
})
return text;
} }
); )
new Setting(containerSyncSettingEl)
const nonSyncFilesSetting = new Setting(containerSyncSettingEl)
.setName("Non-Synchronising files") .setName("Non-Synchronising files")
.setDesc("(RegExp) If this is set, any changes to local and remote files that match this will be skipped.") .setDesc("(RegExp) If this is set, any changes to local and remote files that match this will be skipped.")
.setClass("wizardHidden") .setClass("wizardHidden");
.addTextArea((text) => {
text new MultipleRegExpControl(
.setValue(this.plugin.settings.syncIgnoreRegEx) {
.setPlaceholder("\\.pdf$") target: nonSyncFilesSetting.controlEl,
.onChange(async (value) => { props: {
let isValidRegExp = false; patterns: this.plugin.settings.syncIgnoreRegEx.split("|[]|"), originals: [...this.plugin.settings.syncIgnoreRegEx.split("|[]|")], apply: async (newPatterns) => {
try { this.plugin.settings.syncIgnoreRegEx = newPatterns.map(e => e.trim()).filter(e => e != "").join("|[]|");
new RegExp(value); await this.plugin.saveSettings();
isValidRegExp = true; this.display();
} catch (_) { }
// NO OP. }
}
if (isValidRegExp || value.trim() == "") {
this.plugin.settings.syncIgnoreRegEx = value;
await this.plugin.saveSettings();
}
})
return text;
} }
); )
new Setting(containerSyncSettingEl) new Setting(containerSyncSettingEl)
.setName("Maximum file size") .setName("Maximum file size")
.setDesc("(MB) If this is set, changes to local and remote files that are larger than this will be skipped. If the file becomes smaller again, a newer one will be used.") .setDesc("(MB) If this is set, changes to local and remote files that are larger than this will be skipped. If the file becomes smaller again, a newer one will be used.")
@@ -1698,6 +1716,52 @@ ${stringifyYaml(pluginConfig)}`;
const hatchWarn = containerHatchEl.createEl("div", { text: `To stop the boot up sequence for fixing problems on databases, you can put redflag.md on top of your vault (Rebooting obsidian is required).` }); const hatchWarn = containerHatchEl.createEl("div", { text: `To stop the boot up sequence for fixing problems on databases, you can put redflag.md on top of your vault (Rebooting obsidian is required).` });
hatchWarn.addClass("op-warn-info"); hatchWarn.addClass("op-warn-info");
const addResult = (path: string, file: TFile | false, fileOnDB: LoadedEntry | false) => {
resultArea.appendChild(resultArea.createEl("div", {}, el => {
el.appendChild(el.createEl("h6", { text: path }));
el.appendChild(el.createEl("div", {}, infoGroupEl => {
infoGroupEl.appendChild(infoGroupEl.createEl("div", { text: `Storage : Modified: ${!file ? `Missing:` : `${new Date(file.stat.mtime).toLocaleString()}, Size:${file.stat.size}`}` }))
infoGroupEl.appendChild(infoGroupEl.createEl("div", { text: `Database: Modified: ${!fileOnDB ? `Missing:` : `${new Date(fileOnDB.mtime).toLocaleString()}, Size:${fileOnDB.size}`}` }))
}));
if (fileOnDB && file) {
el.appendChild(el.createEl("button", { text: "Show history" }, buttonEl => {
buttonEl.onClickEvent(() => {
this.plugin.showHistory(file, fileOnDB._id);
})
}))
}
if (file) {
el.appendChild(el.createEl("button", { text: "Storage -> Database" }, buttonEl => {
buttonEl.onClickEvent(() => {
this.plugin.updateIntoDB(file, undefined, true);
el.remove();
})
}))
}
if (fileOnDB) {
el.appendChild(el.createEl("button", { text: "Database -> Storage" }, buttonEl => {
buttonEl.onClickEvent(() => {
this.plugin.pullFile(this.plugin.getPath(fileOnDB), [], true, undefined, false);
el.remove();
})
}))
}
return el;
}))
}
const checkBetweenStorageAndDatabase = async (file: TFile, fileOnDB: LoadedEntry) => {
const dataContent = readAsBlob(fileOnDB);
const content = createBlob(await this.plugin.vaultAccess.vaultReadAuto(file))
if (await isDocContentSame(content, dataContent)) {
Logger(`Compare: SAME: ${file.path}`)
} else {
Logger(`Compare: CONTENT IS NOT MATCHED! ${file.path}`, LOG_LEVEL_NOTICE);
addResult(file.path, file, fileOnDB)
}
}
new Setting(containerHatchEl) new Setting(containerHatchEl)
.setName("Verify and repair all files") .setName("Verify and repair all files")
.setDesc("Compare the content of files between on local database and storage. If not matched, you will asked which one want to keep.") .setDesc("Compare the content of files between on local database and storage. If not matched, you will asked which one want to keep.")
@@ -1708,47 +1772,36 @@ ${stringifyYaml(pluginConfig)}`;
.setWarning() .setWarning()
.onClick(async () => { .onClick(async () => {
const files = this.app.vault.getFiles(); const files = this.app.vault.getFiles();
const documents = [] as FilePathWithPrefix[];
const adn = this.plugin.localDatabase.findAllNormalDocs()
for await (const i of adn) documents.push(this.plugin.getPath(i));
const allPaths = [...new Set([...documents, ...files.map(e => e.path as FilePathWithPrefix)])];
let i = 0; let i = 0;
for (const file of files) { for (const path of allPaths) {
i++; i++;
Logger(`${i}/${files.length}\n${file.path}`, LOG_LEVEL_NOTICE, "verify"); Logger(`${i}/${files.length}\n${path}`, LOG_LEVEL_NOTICE, "verify");
if (!await this.plugin.isTargetFile(file)) continue; if (shouldBeIgnored(path)) continue;
const fileOnDB = await this.plugin.localDatabase.getDBEntry(file.path as FilePathWithPrefix); const abstractFile = this.plugin.vaultAccess.getAbstractFileByPath(path);
if (!fileOnDB) { const fileOnStorage = abstractFile instanceof TFile ? abstractFile : false;
Logger(`Compare: Not found on local database: ${file.path}`, LOG_LEVEL_NOTICE); if (!await this.plugin.isTargetFile(path)) continue;
if (fileOnStorage && this.plugin.isFileSizeExceeded(fileOnStorage.stat.size)) continue;
const fileOnDB = await this.plugin.localDatabase.getDBEntry(path);
if (fileOnDB && this.plugin.isFileSizeExceeded(fileOnDB.size)) continue;
if (!fileOnDB && fileOnStorage) {
Logger(`Compare: Not found on the local database: ${path}`, LOG_LEVEL_NOTICE);
addResult(path, fileOnStorage, false)
continue; continue;
} }
let content: Blob; if (fileOnDB && !fileOnStorage) {
if (fileOnDB.type == "newnote") { Logger(`Compare: Not found on the storage: ${path}`, LOG_LEVEL_NOTICE);
content = createBinaryBlob(await this.plugin.vaultAccess.vaultReadBinary(file)); addResult(path, false, fileOnDB)
} else { continue;
content = createTextBlob(await this.plugin.vaultAccess.vaultRead(file));
} }
if (isDocContentSame(content, fileOnDB.data)) { if (fileOnStorage && fileOnDB) {
Logger(`Compare: SAME: ${file.path}`) await checkBetweenStorageAndDatabase(fileOnStorage, fileOnDB)
} else {
Logger(`Compare: CONTENT IS NOT MATCHED! ${file.path}`, LOG_LEVEL_NOTICE);
resultArea.appendChild(resultArea.createEl("div", {}, el => {
el.appendChild(el.createEl("h6", { text: file.path }));
el.appendChild(el.createEl("div", {}, infoGroupEl => {
infoGroupEl.appendChild(infoGroupEl.createEl("div", { text: `Storage : Modified: ${new Date(file.stat.mtime).toLocaleString()}, Size:${file.stat.size}` }))
infoGroupEl.appendChild(infoGroupEl.createEl("div", { text: `Database: Modified: ${new Date(fileOnDB.mtime).toLocaleString()}, Size:${content.size}` }))
}));
el.appendChild(el.createEl("button", { text: "Storage -> Database" }, buttonEl => {
buttonEl.onClickEvent(() => {
this.plugin.updateIntoDB(file, undefined, true);
el.remove();
})
}))
el.appendChild(el.createEl("button", { text: "Database -> Storage" }, buttonEl => {
buttonEl.onClickEvent(() => {
this.plugin.pullFile(file.path as FilePathWithPrefix, [], true, undefined, false);
el.remove();
})
}))
return el;
}))
} }
} }
Logger("done", LOG_LEVEL_NOTICE, "verify"); Logger("done", LOG_LEVEL_NOTICE, "verify");
@@ -1777,6 +1830,7 @@ ${stringifyYaml(pluginConfig)}`;
//Prepare converted data //Prepare converted data
newDoc._id = idEncoded; newDoc._id = idEncoded;
newDoc.path = docName as FilePathWithPrefix; newDoc.path = docName as FilePathWithPrefix;
// @ts-ignore
delete newDoc._rev; delete newDoc._rev;
try { try {
const obfuscatedDoc = await this.plugin.localDatabase.getRaw(idEncoded, { revs_info: true }); const obfuscatedDoc = await this.plugin.localDatabase.getRaw(idEncoded, { revs_info: true });
@@ -1802,7 +1856,7 @@ ${stringifyYaml(pluginConfig)}`;
Logger(`Converting ${docName} Failed!`, LOG_LEVEL_NOTICE); Logger(`Converting ${docName} Failed!`, LOG_LEVEL_NOTICE);
Logger(ret, LOG_LEVEL_VERBOSE); Logger(ret, LOG_LEVEL_VERBOSE);
} }
} catch (ex) { } catch (ex: any) {
if (ex?.status == 404) { if (ex?.status == 404) {
// We can perform this safely // We can perform this safely
if ((await this.plugin.localDatabase.putRaw(newDoc)).ok) { if ((await this.plugin.localDatabase.putRaw(newDoc)).ok) {
@@ -1923,7 +1977,7 @@ ${stringifyYaml(pluginConfig)}`;
toggle.setValue(!this.plugin.settings.useIndexedDBAdapter).onChange(async (value) => { toggle.setValue(!this.plugin.settings.useIndexedDBAdapter).onChange(async (value) => {
this.plugin.settings.useIndexedDBAdapter = !value; this.plugin.settings.useIndexedDBAdapter = !value;
await this.plugin.saveSettings(); await this.plugin.saveSettings();
await rebuildDB("localOnlyWithChunks"); await rebuildDB("localOnly");
}) })
); );
@@ -1997,7 +2051,7 @@ ${stringifyYaml(pluginConfig)}`;
const vaultName = new Setting(containerPluginSettings) const vaultName = new Setting(containerPluginSettings)
.setName("Device name") .setName("Device name")
.setDesc("Unique name between all synchronized devices") .setDesc("Unique name between all synchronized devices. To edit this setting, please disable customization sync once.")
.addText((text) => { .addText((text) => {
text.setPlaceholder("desktop") text.setPlaceholder("desktop")
.setValue(this.plugin.deviceAndVaultName) .setValue(this.plugin.deviceAndVaultName)
@@ -2121,16 +2175,12 @@ ${stringifyYaml(pluginConfig)}`;
.setWarning() .setWarning()
.setDisabled(false) .setDisabled(false)
.onClick(async () => { .onClick(async () => {
await rebuildDB("localOnlyWithChunks"); await this.plugin.vaultAccess.vaultCreate(FLAGMD_REDFLAG3_HR, "");
this.plugin.performAppReload();
}) })
) ).addButton((button) =>
new Setting(containerMaintenanceEl)
.setName("Fetch rebuilt DB with all remote chunks")
.setDesc("Restore or reconstruct local database from remote database but use remote chunk .")
.addButton((button) =>
button button
.setButtonText("Fetch all") .setButtonText("Fetch w/o restarting")
.setWarning() .setWarning()
.setDisabled(false) .setDisabled(false)
.onClick(async () => { .onClick(async () => {
@@ -2138,6 +2188,19 @@ ${stringifyYaml(pluginConfig)}`;
}) })
) )
new Setting(containerMaintenanceEl)
.setName("Fetch rebuilt DB (Save local documents before)")
.setDesc("Restore or reconstruct local database from remote database but use local chunks.")
.addButton((button) =>
button
.setButtonText("Save and Fetch")
.setWarning()
.setDisabled(false)
.onClick(async () => {
await rebuildDB("localOnlyWithChunks");
})
)
new Setting(containerMaintenanceEl) new Setting(containerMaintenanceEl)
.setName("Discard local database to reset or uninstall Self-hosted LiveSync") .setName("Discard local database to reset or uninstall Self-hosted LiveSync")
.addButton((button) => .addButton((button) =>
@@ -2179,10 +2242,21 @@ ${stringifyYaml(pluginConfig)}`;
.setButtonText("Rebuild") .setButtonText("Rebuild")
.setWarning() .setWarning()
.setDisabled(false) .setDisabled(false)
.onClick(async () => {
await this.plugin.vaultAccess.vaultCreate(FLAGMD_REDFLAG2_HR, "");
this.plugin.performAppReload();
})
)
.addButton((button) =>
button
.setButtonText("Rebuild w/o restarting")
.setWarning()
.setDisabled(false)
.onClick(async () => { .onClick(async () => {
await rebuildDB("rebuildBothByThisDevice"); await rebuildDB("rebuildBothByThisDevice");
}) })
) )
applyDisplayEnabled(); applyDisplayEnabled();
addScreenElement("70", containerMaintenanceEl); addScreenElement("70", containerMaintenanceEl);

25
src/SerializedFileAccess.ts
View File

@@ -1,6 +1,7 @@
import { type App, TFile, type DataWriteOptions, TFolder, TAbstractFile } from "./deps"; import { type App, TFile, type DataWriteOptions, TFolder, TAbstractFile } from "./deps";
import { serialized } from "./lib/src/lock"; import { serialized } from "./lib/src/lock";
import { Logger } from "./lib/src/logger"; import { Logger } from "./lib/src/logger";
import { isPlainText } from "./lib/src/path";
import type { FilePath } from "./lib/src/types"; import type { FilePath } from "./lib/src/types";
import { createBinaryBlob, isDocContentSame } from "./lib/src/utils"; import { createBinaryBlob, isDocContentSame } from "./lib/src/utils";
import type { InternalFileInfo } from "./types"; import type { InternalFileInfo } from "./types";
@@ -56,6 +57,12 @@ export class SerializedFileAccess {
return await processReadFile(file, () => this.app.vault.adapter.readBinary(path)); return await processReadFile(file, () => this.app.vault.adapter.readBinary(path));
} }
async adapterReadAuto(file: TFile | string) {
const path = file instanceof TFile ? file.path : file;
if (isPlainText(path)) return await processReadFile(file, () => this.app.vault.adapter.read(path));
return await processReadFile(file, () => this.app.vault.adapter.readBinary(path));
}
async adapterWrite(file: TFile | string, data: string | ArrayBuffer | Uint8Array, options?: DataWriteOptions) { async adapterWrite(file: TFile | string, data: string | ArrayBuffer | Uint8Array, options?: DataWriteOptions) {
const path = file instanceof TFile ? file.path : file; const path = file instanceof TFile ? file.path : file;
if (typeof (data) === "string") { if (typeof (data) === "string") {
@@ -77,12 +84,19 @@ export class SerializedFileAccess {
return await processReadFile(file, () => this.app.vault.readBinary(file)); return await processReadFile(file, () => this.app.vault.readBinary(file));
} }
async vaultReadAuto(file: TFile) {
const path = file.path;
if (isPlainText(path)) return await processReadFile(file, () => this.app.vault.read(file));
return await processReadFile(file, () => this.app.vault.readBinary(file));
}
async vaultModify(file: TFile, data: string | ArrayBuffer | Uint8Array, options?: DataWriteOptions) { async vaultModify(file: TFile, data: string | ArrayBuffer | Uint8Array, options?: DataWriteOptions) {
if (typeof (data) === "string") { if (typeof (data) === "string") {
return await processWriteFile(file, async () => { return await processWriteFile(file, async () => {
const oldData = await this.app.vault.read(file); const oldData = await this.app.vault.read(file);
if (data === oldData) { if (data === oldData) {
markChangesAreSame(file, file.stat.mtime, options.mtime); if (options && options.mtime) markChangesAreSame(file, file.stat.mtime, options.mtime);
return false return false
} }
await this.app.vault.modify(file, data, options) await this.app.vault.modify(file, data, options)
@@ -93,7 +107,7 @@ export class SerializedFileAccess {
return await processWriteFile(file, async () => { return await processWriteFile(file, async () => {
const oldData = await this.app.vault.readBinary(file); const oldData = await this.app.vault.readBinary(file);
if (await isDocContentSame(createBinaryBlob(oldData), createBinaryBlob(data))) { if (await isDocContentSame(createBinaryBlob(oldData), createBinaryBlob(data))) {
markChangesAreSame(file, file.stat.mtime, options.mtime); if (options && options.mtime) markChangesAreSame(file, file.stat.mtime, options.mtime);
return false; return false;
} }
await this.app.vault.modifyBinary(file, toArrayBuffer(data), options) await this.app.vault.modifyBinary(file, toArrayBuffer(data), options)
@@ -149,10 +163,9 @@ export class SerializedFileAccess {
c += v; c += v;
try { try {
await this.app.vault.adapter.mkdir(c); await this.app.vault.adapter.mkdir(c);
} catch (ex) { } catch (ex: any) {
// basically skip exceptions. if (ex?.message == "Folder already exists.") {
if (ex.message && ex.message == "Folder already exists.") { // Skip if already exists.
// especially this message is.
} else { } else {
Logger("Folder Create Error"); Logger("Folder Create Error");
Logger(ex); Logger(ex);

27
src/StorageEventManager.ts
View File

@@ -1,8 +1,8 @@
import type { SerializedFileAccess } from "./SerializedFileAccess"; import type { SerializedFileAccess } from "./SerializedFileAccess";
import { Plugin, TAbstractFile, TFile, TFolder } from "./deps"; import { Plugin, TAbstractFile, TFile, TFolder } from "./deps";
import { Logger } from "./lib/src/logger"; import { Logger } from "./lib/src/logger";
import { isPlainText, shouldBeIgnored } from "./lib/src/path"; import { shouldBeIgnored } from "./lib/src/path";
import type { KeyedQueueProcessor } from "./lib/src/processor"; import type { QueueProcessor } from "./lib/src/processor";
import { LOG_LEVEL_NOTICE, type FilePath, type ObsidianLiveSyncSettings } from "./lib/src/types"; import { LOG_LEVEL_NOTICE, type FilePath, type ObsidianLiveSyncSettings } from "./lib/src/types";
import { delay } from "./lib/src/utils"; import { delay } from "./lib/src/utils";
import { type FileEventItem, type FileEventType, type FileInfo, type InternalFileInfo } from "./types"; import { type FileEventItem, type FileEventType, type FileInfo, type InternalFileInfo } from "./types";
@@ -19,7 +19,7 @@ type LiveSyncForStorageEventManager = Plugin &
vaultAccess: SerializedFileAccess vaultAccess: SerializedFileAccess
} & { } & {
isTargetFile: (file: string | TAbstractFile) => Promise<boolean>, isTargetFile: (file: string | TAbstractFile) => Promise<boolean>,
fileEventQueue: KeyedQueueProcessor<FileEventItem, any>, fileEventQueue: QueueProcessor<FileEventItem, any>,
isFileSizeExceeded: (size: number) => boolean; isFileSizeExceeded: (size: number) => boolean;
}; };
@@ -109,7 +109,8 @@ export class StorageEventManagerObsidian extends StorageEventManager {
if (file instanceof TFolder) continue; if (file instanceof TFolder) continue;
if (!await this.plugin.isTargetFile(file.path)) continue; if (!await this.plugin.isTargetFile(file.path)) continue;
let cache: null | string | ArrayBuffer; // Stop cache using to prevent the corruption;
// let cache: null | string | ArrayBuffer;
// new file or something changed, cache the changes. // new file or something changed, cache the changes.
if (file instanceof TFile && (type == "CREATE" || type == "CHANGED")) { if (file instanceof TFile && (type == "CREATE" || type == "CHANGED")) {
// Wait for a bit while to let the writer has marked `touched` at the file. // Wait for a bit while to let the writer has marked `touched` at the file.
@@ -117,12 +118,13 @@ export class StorageEventManagerObsidian extends StorageEventManager {
if (this.plugin.vaultAccess.recentlyTouched(file)) { if (this.plugin.vaultAccess.recentlyTouched(file)) {
continue; continue;
} }
if (!isPlainText(file.name)) { // cache = await this.plugin.vaultAccess.vaultReadAuto(file);
cache = await this.plugin.vaultAccess.vaultReadBinary(file); // if (!isPlainText(file.name)) {
} else { // cache = await this.plugin.vaultAccess.vaultReadBinary(file);
cache = await this.plugin.vaultAccess.vaultCacheRead(file); // } else {
if (!cache) cache = await this.plugin.vaultAccess.vaultRead(file); // cache = await this.plugin.vaultAccess.vaultCacheRead(file);
} // if (!cache) cache = await this.plugin.vaultAccess.vaultRead(file);
// }
} }
const fileInfo = file instanceof TFile ? { const fileInfo = file instanceof TFile ? {
ctime: file.stat.ctime, ctime: file.stat.ctime,
@@ -131,13 +133,12 @@ export class StorageEventManagerObsidian extends StorageEventManager {
path: file.path, path: file.path,
size: file.stat.size size: file.stat.size
} as FileInfo : file as InternalFileInfo; } as FileInfo : file as InternalFileInfo;
this.plugin.fileEventQueue.enqueue({
this.plugin.fileEventQueue.enqueueWithKey(`file-${fileInfo.path}`, {
type, type,
args: { args: {
file: fileInfo, file: fileInfo,
oldPath, oldPath,
cache, // cache,
ctx ctx
}, },
key: atomicKey key: atomicKey

33
src/dialogs.ts
View File

@@ -7,10 +7,9 @@ import PluginPane from "./PluginPane.svelte";
export class PluginDialogModal extends Modal { export class PluginDialogModal extends Modal {
plugin: ObsidianLiveSyncPlugin; plugin: ObsidianLiveSyncPlugin;
logEl: HTMLDivElement; component: PluginPane | undefined;
component: PluginPane = null;
isOpened() { isOpened() {
return this.component != null; return this.component != undefined;
} }
constructor(app: App, plugin: ObsidianLiveSyncPlugin) { constructor(app: App, plugin: ObsidianLiveSyncPlugin) {
@@ -21,7 +20,7 @@ export class PluginDialogModal extends Modal {
onOpen() { onOpen() {
const { contentEl } = this; const { contentEl } = this;
this.titleEl.setText("Customization Sync (Beta2)") this.titleEl.setText("Customization Sync (Beta2)")
if (this.component == null) { if (!this.component) {
this.component = new PluginPane({ this.component = new PluginPane({
target: contentEl, target: contentEl,
props: { plugin: this.plugin }, props: { plugin: this.plugin },
@@ -30,9 +29,9 @@ export class PluginDialogModal extends Modal {
} }
onClose() { onClose() {
if (this.component != null) { if (this.component) {
this.component.$destroy(); this.component.$destroy();
this.component = null; this.component = undefined;
} }
} }
} }
@@ -94,13 +93,13 @@ export class InputStringDialog extends Modal {
} }
export class PopoverSelectString extends FuzzySuggestModal<string> { export class PopoverSelectString extends FuzzySuggestModal<string> {
app: App; app: App;
callback: (e: string) => void = () => { }; callback: ((e: string) => void) | undefined = () => { };
getItemsFun: () => string[] = () => { getItemsFun: () => string[] = () => {
return ["yes", "no"]; return ["yes", "no"];
} }
constructor(app: App, note: string, placeholder: string | null, getItemsFun: () => string[], callback: (e: string) => void) { constructor(app: App, note: string, placeholder: string | undefined, getItemsFun: (() => string[]) | undefined, callback: (e: string) => void) {
super(app); super(app);
this.app = app; this.app = app;
this.setPlaceholder((placeholder ?? "y/n) ") + note); this.setPlaceholder((placeholder ?? "y/n) ") + note);
@@ -118,13 +117,14 @@ export class PopoverSelectString extends FuzzySuggestModal<string> {
onChooseItem(item: string, evt: MouseEvent | KeyboardEvent): void { onChooseItem(item: string, evt: MouseEvent | KeyboardEvent): void {
// debugger; // debugger;
this.callback(item); this.callback?.(item);
this.callback = null; this.callback = undefined;
} }
onClose(): void { onClose(): void {
setTimeout(() => { setTimeout(() => {
if (this.callback != null) { if (this.callback) {
this.callback(""); this.callback("");
this.callback = undefined;
} }
}, 100); }, 100);
} }
@@ -136,16 +136,16 @@ export class MessageBox extends Modal {
title: string; title: string;
contentMd: string; contentMd: string;
buttons: string[]; buttons: string[];
result: string; result: string | false = false;
isManuallyClosed = false; isManuallyClosed = false;
defaultAction: string | undefined; defaultAction: string | undefined;
timeout: number | undefined; timeout: number | undefined;
timer: ReturnType<typeof setInterval> = undefined; timer: ReturnType<typeof setInterval> | undefined = undefined;
defaultButtonComponent: ButtonComponent | undefined; defaultButtonComponent: ButtonComponent | undefined;
onSubmit: (result: string | false) => void; onSubmit: (result: string | false) => void;
constructor(plugin: Plugin, title: string, contentMd: string, buttons: string[], defaultAction: (typeof buttons)[number], timeout: number, onSubmit: (result: (typeof buttons)[number] | false) => void) { constructor(plugin: Plugin, title: string, contentMd: string, buttons: string[], defaultAction: (typeof buttons)[number], timeout: number | undefined, onSubmit: (result: (typeof buttons)[number] | false) => void) {
super(plugin.app); super(plugin.app);
this.plugin = plugin; this.plugin = plugin;
this.title = title; this.title = title;
@@ -156,6 +156,7 @@ export class MessageBox extends Modal {
this.timeout = timeout; this.timeout = timeout;
if (this.timeout) { if (this.timeout) {
this.timer = setInterval(() => { this.timer = setInterval(() => {
if (this.timeout === undefined) return;
this.timeout--; this.timeout--;
if (this.timeout < 0) { if (this.timeout < 0) {
if (this.timer) { if (this.timer) {
@@ -166,7 +167,7 @@ export class MessageBox extends Modal {
this.isManuallyClosed = true; this.isManuallyClosed = true;
this.close(); this.close();
} else { } else {
this.defaultButtonComponent.setButtonText(`( ${this.timeout} ) ${defaultAction}`); this.defaultButtonComponent?.setButtonText(`( ${this.timeout} ) ${defaultAction}`);
} }
}, 1000); }, 1000);
} }
@@ -223,7 +224,7 @@ export class MessageBox extends Modal {
} }
export function confirmWithMessage(plugin: Plugin, title: string, contentMd: string, buttons: string[], defaultAction?: (typeof buttons)[number], timeout?: number): Promise<(typeof buttons)[number] | false> { export function confirmWithMessage(plugin: Plugin, title: string, contentMd: string, buttons: string[], defaultAction: (typeof buttons)[number], timeout?: number): Promise<(typeof buttons)[number] | false> {
return new Promise((res) => { return new Promise((res) => {
const dialog = new MessageBox(plugin, title, contentMd, buttons, defaultAction, timeout, (result) => res(result)); const dialog = new MessageBox(plugin, title, contentMd, buttons, defaultAction, timeout, (result) => res(result));
dialog.open(); dialog.open();

2
src/lib

Submodule src/lib updated: 8a8177c1f0...b05e493258

471
src/main.ts
View File

@@ -4,7 +4,7 @@ import { type Diff, DIFF_DELETE, DIFF_EQUAL, DIFF_INSERT, diff_match_patch, stri
import { Notice, Plugin, TFile, addIcon, TFolder, normalizePath, TAbstractFile, Editor, MarkdownView, type RequestUrlParam, type RequestUrlResponse, requestUrl, type MarkdownFileInfo } from "./deps"; import { Notice, Plugin, TFile, addIcon, TFolder, normalizePath, TAbstractFile, Editor, MarkdownView, type RequestUrlParam, type RequestUrlResponse, requestUrl, type MarkdownFileInfo } from "./deps";
import { type EntryDoc, type LoadedEntry, type ObsidianLiveSyncSettings, type diff_check_result, type diff_result_leaf, type EntryBody, LOG_LEVEL, VER, DEFAULT_SETTINGS, type diff_result, FLAGMD_REDFLAG, SYNCINFO_ID, SALT_OF_PASSPHRASE, type ConfigPassphraseStore, type CouchDBConnection, FLAGMD_REDFLAG2, FLAGMD_REDFLAG3, PREFIXMD_LOGFILE, type DatabaseConnectingStatus, type EntryHasPath, type DocumentID, type FilePathWithPrefix, type FilePath, type AnyEntry, LOG_LEVEL_DEBUG, LOG_LEVEL_INFO, LOG_LEVEL_NOTICE, LOG_LEVEL_URGENT, LOG_LEVEL_VERBOSE, type SavingEntry, MISSING_OR_ERROR, NOT_CONFLICTED, AUTO_MERGED, CANCELLED, LEAVE_TO_SUBSEQUENT, FLAGMD_REDFLAG2_HR, FLAGMD_REDFLAG3_HR, } from "./lib/src/types"; import { type EntryDoc, type LoadedEntry, type ObsidianLiveSyncSettings, type diff_check_result, type diff_result_leaf, type EntryBody, LOG_LEVEL, VER, DEFAULT_SETTINGS, type diff_result, FLAGMD_REDFLAG, SYNCINFO_ID, SALT_OF_PASSPHRASE, type ConfigPassphraseStore, type CouchDBConnection, FLAGMD_REDFLAG2, FLAGMD_REDFLAG3, PREFIXMD_LOGFILE, type DatabaseConnectingStatus, type EntryHasPath, type DocumentID, type FilePathWithPrefix, type FilePath, type AnyEntry, LOG_LEVEL_DEBUG, LOG_LEVEL_INFO, LOG_LEVEL_NOTICE, LOG_LEVEL_URGENT, LOG_LEVEL_VERBOSE, type SavingEntry, MISSING_OR_ERROR, NOT_CONFLICTED, AUTO_MERGED, CANCELLED, LEAVE_TO_SUBSEQUENT, FLAGMD_REDFLAG2_HR, FLAGMD_REDFLAG3_HR, } from "./lib/src/types";
import { type InternalFileInfo, type CacheData, type FileEventItem, FileWatchEventQueueMax } from "./types"; import { type InternalFileInfo, type CacheData, type FileEventItem, FileWatchEventQueueMax } from "./types";
import { arrayToChunkedArray, createBinaryBlob, createTextBlob, fireAndForget, getDocData, isDocContentSame, isObjectDifferent, sendValue } from "./lib/src/utils"; import { arrayToChunkedArray, createBlob, delay, determineTypeFromBlob, fireAndForget, getDocData, isAnyNote, isDocContentSame, isObjectDifferent, readContent, sendValue, throttle } from "./lib/src/utils";
import { Logger, setGlobalLogFunction } from "./lib/src/logger"; import { Logger, setGlobalLogFunction } from "./lib/src/logger";
import { PouchDB } from "./lib/src/pouchdb-browser.js"; import { PouchDB } from "./lib/src/pouchdb-browser.js";
import { ConflictResolveModal } from "./ConflictResolveModal"; import { ConflictResolveModal } from "./ConflictResolveModal";
@@ -12,7 +12,7 @@ import { ObsidianLiveSyncSettingTab } from "./ObsidianLiveSyncSettingTab";
import { DocumentHistoryModal } from "./DocumentHistoryModal"; import { DocumentHistoryModal } from "./DocumentHistoryModal";
import { applyPatch, cancelAllPeriodicTask, cancelAllTasks, cancelTask, generatePatchObj, id2path, isObjectMargeApplicable, isSensibleMargeApplicable, flattenObject, path2id, scheduleTask, tryParseJSON, isValidPath, isInternalMetadata, isPluginMetadata, stripInternalMetadataPrefix, isChunk, askSelectString, askYesNo, askString, PeriodicProcessor, getPath, getPathWithoutPrefix, getPathFromTFile, performRebuildDB, memoIfNotExist, memoObject, retrieveMemoObject, disposeMemoObject, isCustomisationSyncMetadata, compareFileFreshness, BASE_IS_NEW, TARGET_IS_NEW, EVEN, compareMTime, markChangesAreSame } from "./utils"; import { applyPatch, cancelAllPeriodicTask, cancelAllTasks, cancelTask, generatePatchObj, id2path, isObjectMargeApplicable, isSensibleMargeApplicable, flattenObject, path2id, scheduleTask, tryParseJSON, isValidPath, isInternalMetadata, isPluginMetadata, stripInternalMetadataPrefix, isChunk, askSelectString, askYesNo, askString, PeriodicProcessor, getPath, getPathWithoutPrefix, getPathFromTFile, performRebuildDB, memoIfNotExist, memoObject, retrieveMemoObject, disposeMemoObject, isCustomisationSyncMetadata, compareFileFreshness, BASE_IS_NEW, TARGET_IS_NEW, EVEN, compareMTime, markChangesAreSame } from "./utils";
import { encrypt, tryDecrypt } from "./lib/src/e2ee_v2"; import { encrypt, tryDecrypt } from "./lib/src/e2ee_v2";
import { balanceChunkPurgedDBs, enableEncryption, isCloudantURI, isErrorOfMissingDoc, isValidRemoteCouchDBURI, purgeUnreferencedChunks } from "./lib/src/utils_couchdb"; import { balanceChunkPurgedDBs, enableCompression, enableEncryption, isCloudantURI, isErrorOfMissingDoc, isValidRemoteCouchDBURI, purgeUnreferencedChunks } from "./lib/src/utils_couchdb";
import { logStore, type LogEntry, collectingChunks, pluginScanningCount, hiddenFilesProcessingCount, hiddenFilesEventCount, logMessages } from "./lib/src/stores"; import { logStore, type LogEntry, collectingChunks, pluginScanningCount, hiddenFilesProcessingCount, hiddenFilesEventCount, logMessages } from "./lib/src/stores";
import { setNoticeClass } from "./lib/src/wrapper"; import { setNoticeClass } from "./lib/src/wrapper";
import { versionNumberString2Number, writeString, decodeBinary, readString } from "./lib/src/strbin"; import { versionNumberString2Number, writeString, decodeBinary, readString } from "./lib/src/strbin";
@@ -31,7 +31,7 @@ import { GlobalHistoryView, VIEW_TYPE_GLOBAL_HISTORY } from "./GlobalHistoryView
import { LogPaneView, VIEW_TYPE_LOG } from "./LogPaneView"; import { LogPaneView, VIEW_TYPE_LOG } from "./LogPaneView";
import { LRUCache } from "./lib/src/LRUCache"; import { LRUCache } from "./lib/src/LRUCache";
import { SerializedFileAccess } from "./SerializedFileAccess.js"; import { SerializedFileAccess } from "./SerializedFileAccess.js";
import { KeyedQueueProcessor, QueueProcessor, type QueueItemWithKey } from "./lib/src/processor.js"; import { QueueProcessor } from "./lib/src/processor.js";
import { reactive, reactiveSource } from "./lib/src/reactive.js"; import { reactive, reactiveSource } from "./lib/src/reactive.js";
import { initializeStores } from "./stores.js"; import { initializeStores } from "./stores.js";
@@ -119,7 +119,7 @@ export default class ObsidianLiveSyncPlugin extends Plugin
requestCount = reactiveSource(0); requestCount = reactiveSource(0);
responseCount = reactiveSource(0); responseCount = reactiveSource(0);
processReplication = (e: PouchDB.Core.ExistingDocument<EntryDoc>[]) => this.parseReplicationResult(e); processReplication = (e: PouchDB.Core.ExistingDocument<EntryDoc>[]) => this.parseReplicationResult(e);
async connectRemoteCouchDB(uri: string, auth: { username: string; password: string }, disableRequestURI: boolean, passphrase: string | false, useDynamicIterationCount: boolean, performSetup: boolean, skipInfo: boolean): Promise<string | { db: PouchDB.Database<EntryDoc>; info: PouchDB.Core.DatabaseInfo }> { async connectRemoteCouchDB(uri: string, auth: { username: string; password: string }, disableRequestURI: boolean, passphrase: string | false, useDynamicIterationCount: boolean, performSetup: boolean, skipInfo: boolean, compression: boolean): Promise<string | { db: PouchDB.Database<EntryDoc>; info: PouchDB.Core.DatabaseInfo }> {
if (!isValidRemoteCouchDBURI(uri)) return "Remote URI is not valid"; if (!isValidRemoteCouchDBURI(uri)) return "Remote URI is not valid";
if (uri.toLowerCase() != uri) return "Remote URI and database name could not contain capital letters."; if (uri.toLowerCase() != uri) return "Remote URI and database name could not contain capital letters.";
if (uri.indexOf(" ") !== -1) return "Remote URI and database name could not contain spaces."; if (uri.indexOf(" ") !== -1) return "Remote URI and database name could not contain spaces.";
@@ -211,6 +211,15 @@ export default class ObsidianLiveSyncPlugin extends Plugin
this.last_successful_post = true; this.last_successful_post = true;
} }
Logger(`HTTP:${method}${size} to:${localURL} -> ${response.status}`, LOG_LEVEL_DEBUG); Logger(`HTTP:${method}${size} to:${localURL} -> ${response.status}`, LOG_LEVEL_DEBUG);
if (Math.floor(response.status / 100) !== 2) {
const r = response.clone();
Logger(`The request may have failed. The reason sent by the server: ${r.status}: ${r.statusText}`);
try {
Logger(await (await r.blob()).text(), LOG_LEVEL_VERBOSE);
} catch (_) {
Logger("Cloud not parse response", LOG_LEVEL_VERBOSE);
}
}
return response; return response;
} catch (ex) { } catch (ex) {
Logger(`HTTP:${method}${size} to:${localURL} -> failed`, LOG_LEVEL_VERBOSE); Logger(`HTTP:${method}${size} to:${localURL} -> failed`, LOG_LEVEL_VERBOSE);
@@ -228,6 +237,7 @@ export default class ObsidianLiveSyncPlugin extends Plugin
}; };
const db: PouchDB.Database<EntryDoc> = new PouchDB<EntryDoc>(uri, conf); const db: PouchDB.Database<EntryDoc> = new PouchDB<EntryDoc>(uri, conf);
enableCompression(db, compression);
if (passphrase !== "false" && typeof passphrase === "string") { if (passphrase !== "false" && typeof passphrase === "string") {
enableEncryption(db, passphrase, useDynamicIterationCount, false); enableEncryption(db, passphrase, useDynamicIterationCount, false);
} }
@@ -303,8 +313,9 @@ export default class ObsidianLiveSyncPlugin extends Plugin
this.replicator = new LiveSyncDBReplicator(this); this.replicator = new LiveSyncDBReplicator(this);
} }
async onResetDatabase(db: LiveSyncLocalDB): Promise<void> { async onResetDatabase(db: LiveSyncLocalDB): Promise<void> {
const lsKey = "obsidian-livesync-queuefiles-" + this.getVaultName(); const kvDBKey = "queued-files"
localStorage.removeItem(lsKey); this.kvDB.del(kvDBKey);
// localStorage.removeItem(lsKey);
await this.kvDB.destroy(); await this.kvDB.destroy();
this.kvDB = await OpenKeyValueDatabase(db.dbname + "-livesync-kv"); this.kvDB = await OpenKeyValueDatabase(db.dbname + "-livesync-kv");
this.replicator = new LiveSyncDBReplicator(this); this.replicator = new LiveSyncDBReplicator(this);
@@ -372,7 +383,7 @@ export default class ObsidianLiveSyncPlugin extends Plugin
const notesList = notes.map(e => e.dispPath); const notesList = notes.map(e => e.dispPath);
const target = await this.askSelectString("File to view History", notesList); const target = await this.askSelectString("File to view History", notesList);
if (target) { if (target) {
const targetId = notes.find(e => e.dispPath == target); const targetId = notes.find(e => e.dispPath == target)!;
this.showHistory(targetId.path, targetId.id); this.showHistory(targetId.path, targetId.id);
} }
} }
@@ -390,7 +401,7 @@ export default class ObsidianLiveSyncPlugin extends Plugin
} }
const target = await this.askSelectString("File to resolve conflict", notesList); const target = await this.askSelectString("File to resolve conflict", notesList);
if (target) { if (target) {
const targetItem = notes.find(e => e.dispPath == target); const targetItem = notes.find(e => e.dispPath == target)!;
this.resolveConflicted(targetItem.path); this.resolveConflicted(targetItem.path);
await this.conflictCheckQueue.waitForPipeline(); await this.conflictCheckQueue.waitForPipeline();
return true; return true;
@@ -417,7 +428,7 @@ export default class ObsidianLiveSyncPlugin extends Plugin
const limit = Date.now() - (86400 * 1000 * limitDays); const limit = Date.now() - (86400 * 1000 * limitDays);
const notes: { path: string, mtime: number, ttl: number, doc: PouchDB.Core.ExistingDocument<EntryDoc & PouchDB.Core.AllDocsMeta> }[] = []; const notes: { path: string, mtime: number, ttl: number, doc: PouchDB.Core.ExistingDocument<EntryDoc & PouchDB.Core.AllDocsMeta> }[] = [];
for await (const doc of this.localDatabase.findAllDocs({ conflicts: true })) { for await (const doc of this.localDatabase.findAllDocs({ conflicts: true })) {
if (doc.type == "newnote" || doc.type == "plain") { if (isAnyNote(doc)) {
if (doc.deleted && (doc.mtime - limit) < 0) { if (doc.deleted && (doc.mtime - limit) < 0) {
notes.push({ path: this.getPath(doc), mtime: doc.mtime, ttl: (doc.mtime - limit) / 1000 / 86400, doc: doc }); notes.push({ path: this.getPath(doc), mtime: doc.mtime, ttl: (doc.mtime - limit) / 1000 / 86400, doc: doc });
} }
@@ -526,7 +537,7 @@ Click anywhere to stop counting down.
this.registerWatchEvents(); this.registerWatchEvents();
await this.realizeSettingSyncMode(); await this.realizeSettingSyncMode();
this.swapSaveCommand(); this.swapSaveCommand();
if (this.settings.syncOnStart) { if (!this.settings.liveSync && this.settings.syncOnStart) {
this.replicator.openReplication(this.settings, false, false); this.replicator.openReplication(this.settings, false, false);
} }
this.scanStat(); this.scanStat();
@@ -682,7 +693,7 @@ Note: We can always able to read V1 format. It will be progressively converted.
name: "Show history", name: "Show history",
callback: () => { callback: () => {
const file = this.getActiveFile(); const file = this.getActiveFile();
if (file) this.showHistory(file, null); if (file) this.showHistory(file, undefined);
} }
}); });
this.addCommand({ this.addCommand({
@@ -754,7 +765,7 @@ Note: We can always able to read V1 format. It will be progressively converted.
const ret = this.extractSettingFromWholeText(doc); const ret = this.extractSettingFromWholeText(doc);
return ret.body != ""; return ret.body != "";
} }
this.checkAndApplySettingFromMarkdown(ctx.file.path, false); if (ctx.file) this.checkAndApplySettingFromMarkdown(ctx.file.path, false);
}, },
}) })
@@ -786,8 +797,10 @@ Note: We can always able to read V1 format. It will be progressively converted.
const lsKey = "obsidian-live-sync-ver" + this.getVaultName(); const lsKey = "obsidian-live-sync-ver" + this.getVaultName();
const last_version = localStorage.getItem(lsKey); const last_version = localStorage.getItem(lsKey);
this.observeForLogs(); this.observeForLogs();
this.statusBar = this.addStatusBarItem(); if (this.settings.showStatusOnStatusbar) {
this.statusBar.addClass("syncstatusbar"); this.statusBar = this.addStatusBarItem();
this.statusBar.addClass("syncstatusbar");
}
const lastVersion = ~~(versionNumberString2Number(manifestVersion) / 1000); const lastVersion = ~~(versionNumberString2Number(manifestVersion) / 1000);
if (lastVersion > this.settings.lastReadUpdates && this.settings.isConfigured) { if (lastVersion > this.settings.lastReadUpdates && this.settings.isConfigured) {
Logger("Self-hosted LiveSync has undergone a major upgrade. Please open the setting dialog, and check the information pane.", LOG_LEVEL_NOTICE); Logger("Self-hosted LiveSync has undergone a major upgrade. Please open the setting dialog, and check the information pane.", LOG_LEVEL_NOTICE);
@@ -996,7 +1009,7 @@ Note: We can always able to read V1 format. It will be progressively converted.
} }
this.deviceAndVaultName = localStorage.getItem(lsKey) || ""; this.deviceAndVaultName = localStorage.getItem(lsKey) || "";
this.ignoreFiles = this.settings.ignoreFiles.split(",").map(e => e.trim()); this.ignoreFiles = this.settings.ignoreFiles.split(",").map(e => e.trim());
this.fileEventQueue.delay = this.settings.batchSave ? 5000 : 100; this.fileEventQueue.delay = (!this.settings.liveSync && this.settings.batchSave) ? 5000 : 100;
} }
async saveSettingData() { async saveSettingData() {
@@ -1028,7 +1041,7 @@ Note: We can always able to read V1 format. It will be progressively converted.
} }
await this.saveData(settings); await this.saveData(settings);
this.localDatabase.settings = this.settings; this.localDatabase.settings = this.settings;
this.fileEventQueue.delay = this.settings.batchSave ? 5000 : 100; this.fileEventQueue.delay = (!this.settings.liveSync && this.settings.batchSave) ? 5000 : 100;
this.ignoreFiles = this.settings.ignoreFiles.split(",").map(e => e.trim()); this.ignoreFiles = this.settings.ignoreFiles.split(",").map(e => e.trim());
if (this.settings.settingSyncFile != "") { if (this.settings.settingSyncFile != "") {
fireAndForget(() => this.saveSettingToMarkdown(this.settings.settingSyncFile)); fireAndForget(() => this.saveSettingToMarkdown(this.settings.settingSyncFile));
@@ -1073,7 +1086,7 @@ Note: We can always able to read V1 format. It will be progressively converted.
async checkAndApplySettingFromMarkdown(filename: string, automated?: boolean) { async checkAndApplySettingFromMarkdown(filename: string, automated?: boolean) {
if (automated && !this.settings.notifyAllSettingSyncFile) { if (automated && !this.settings.notifyAllSettingSyncFile) {
if (!this.settings.settingSyncFile || this.settings.settingSyncFile != filename) { if (!this.settings.settingSyncFile || this.settings.settingSyncFile != filename) {
Logger(`Setting file (${filename}) is not matched to the current configuration. skipped.`, LOG_LEVEL_VERBOSE); Logger(`Setting file (${filename}) is not matched to the current configuration. skipped.`, LOG_LEVEL_DEBUG);
return; return;
} }
} }
@@ -1136,7 +1149,7 @@ Note: We can always able to read V1 format. It will be progressively converted.
}) })
} }
generateSettingForMarkdown(settings?: ObsidianLiveSyncSettings, keepCredential?: boolean): Partial<ObsidianLiveSyncSettings> { generateSettingForMarkdown(settings?: ObsidianLiveSyncSettings, keepCredential?: boolean): Partial<ObsidianLiveSyncSettings> {
const saveData = { ...(settings ? settings : this.settings) }; const saveData = { ...(settings ? settings : this.settings) } as Partial<ObsidianLiveSyncSettings>;
delete saveData.encryptedCouchDBConnection; delete saveData.encryptedCouchDBConnection;
delete saveData.encryptedPassphrase; delete saveData.encryptedPassphrase;
if (!saveData.writeCredentialsForSettingSync && !keepCredential) { if (!saveData.writeCredentialsForSettingSync && !keepCredential) {
@@ -1226,9 +1239,13 @@ We can perform a command in this file.
_this.performCommand('editor:save-file'); _this.performCommand('editor:save-file');
}; };
} }
hasFocus = true;
isLastHidden = false;
registerWatchEvents() { registerWatchEvents() {
this.registerEvent(this.app.workspace.on("file-open", this.watchWorkspaceOpen)); this.registerEvent(this.app.workspace.on("file-open", this.watchWorkspaceOpen));
this.registerDomEvent(document, "visibilitychange", this.watchWindowVisibility); this.registerDomEvent(document, "visibilitychange", this.watchWindowVisibility);
this.registerDomEvent(window, "focus", () => this.setHasFocus(true));
this.registerDomEvent(window, "blur", () => this.setHasFocus(false));
this.registerDomEvent(window, "online", this.watchOnline); this.registerDomEvent(window, "online", this.watchOnline);
this.registerDomEvent(window, "offline", this.watchOnline); this.registerDomEvent(window, "offline", this.watchOnline);
} }
@@ -1244,15 +1261,30 @@ We can perform a command in this file.
await this.syncAllFiles(); await this.syncAllFiles();
} }
} }
setHasFocus(hasFocus: boolean) {
this.hasFocus = hasFocus;
this.watchWindowVisibility();
}
watchWindowVisibility() { watchWindowVisibility() {
scheduleTask("watch-window-visibility", 500, () => fireAndForget(() => this.watchWindowVisibilityAsync())); scheduleTask("watch-window-visibility", 100, () => fireAndForget(() => this.watchWindowVisibilityAsync()));
} }
async watchWindowVisibilityAsync() { async watchWindowVisibilityAsync() {
if (this.settings.suspendFileWatching) return; if (this.settings.suspendFileWatching) return;
if (!this.settings.isConfigured) return; if (!this.settings.isConfigured) return;
if (!this.isReady) return; if (!this.isReady) return;
if (this.isLastHidden && !this.hasFocus) {
// NO OP while non-focused after made hidden;
return;
}
const isHidden = document.hidden; const isHidden = document.hidden;
if (this.isLastHidden === isHidden) {
return;
}
this.isLastHidden = isHidden;
await this.applyBatchChange(); await this.applyBatchChange();
if (isHidden) { if (isHidden) {
this.replicator.closeReplication(); this.replicator.closeReplication();
@@ -1272,12 +1304,12 @@ We can perform a command in this file.
} }
cancelRelativeEvent(item: FileEventItem) { cancelRelativeEvent(item: FileEventItem) {
this.fileEventQueue.modifyQueue((items) => [...items.filter(e => e.entity.key != item.key)]) this.fileEventQueue.modifyQueue((items) => [...items.filter(e => e.key != item.key)])
} }
queueNextFileEvent(items: QueueItemWithKey<FileEventItem>[], newItem: QueueItemWithKey<FileEventItem>): QueueItemWithKey<FileEventItem>[] { queueNextFileEvent(items: FileEventItem[], newItem: FileEventItem): FileEventItem[] {
if (this.settings.batchSave && !this.settings.liveSync) { if (this.settings.batchSave && !this.settings.liveSync) {
const file = newItem.entity.args.file; const file = newItem.args.file;
// if the latest event is the same type, omit that // if the latest event is the same type, omit that
// a.md MODIFY <- this should be cancelled when a.md MODIFIED // a.md MODIFY <- this should be cancelled when a.md MODIFIED
// b.md MODIFY <- this should be cancelled when b.md MODIFIED // b.md MODIFY <- this should be cancelled when b.md MODIFIED
@@ -1289,16 +1321,16 @@ We can perform a command in this file.
while (i >= 0) { while (i >= 0) {
i--; i--;
if (i < 0) break L1; if (i < 0) break L1;
if (items[i].entity.args.file.path != file.path) { if (items[i].args.file.path != file.path) {
continue L1; continue L1;
} }
if (items[i].entity.type != newItem.entity.type) break L1; if (items[i].type != newItem.type) break L1;
items.remove(items[i]); items.remove(items[i]);
} }
} }
items.push(newItem); items.push(newItem);
// When deleting or renaming, the queue must be flushed once before processing subsequent processes to prevent unexpected race condition. // When deleting or renaming, the queue must be flushed once before processing subsequent processes to prevent unexpected race condition.
if (newItem.entity.type == "DELETE" || newItem.entity.type == "RENAME") { if (newItem.type == "DELETE" || newItem.type == "RENAME") {
this.fileEventQueue.requestNextFlush(); this.fileEventQueue.requestNextFlush();
} }
return items; return items;
@@ -1329,12 +1361,12 @@ We can perform a command in this file.
return; return;
} }
const cache = queue.args.cache; // const cache = queue.args.cache;
if (queue.type == "CREATE" || queue.type == "CHANGED") { if (queue.type == "CREATE" || queue.type == "CHANGED") {
fireAndForget(() => this.checkAndApplySettingFromMarkdown(queue.args.file.path, true)); fireAndForget(() => this.checkAndApplySettingFromMarkdown(queue.args.file.path, true));
const keyD1 = `file-last-proc-DELETED-${file.path}`; const keyD1 = `file-last-proc-DELETED-${file.path}`;
await this.kvDB.set(keyD1, mtime); await this.kvDB.set(keyD1, mtime);
if (!await this.updateIntoDB(targetFile, cache)) { if (!await this.updateIntoDB(targetFile, undefined)) {
Logger(`STORAGE -> DB: failed, cancel the relative operations: ${targetFile.path}`, LOG_LEVEL_INFO); Logger(`STORAGE -> DB: failed, cancel the relative operations: ${targetFile.path}`, LOG_LEVEL_INFO);
// cancel running queues and remove one of atomic operation // cancel running queues and remove one of atomic operation
this.cancelRelativeEvent(queue); this.cancelRelativeEvent(queue);
@@ -1352,7 +1384,7 @@ We can perform a command in this file.
pendingFileEventCount = reactiveSource(0); pendingFileEventCount = reactiveSource(0);
processingFileEventCount = reactiveSource(0); processingFileEventCount = reactiveSource(0);
fileEventQueue = fileEventQueue =
new KeyedQueueProcessor( new QueueProcessor(
(items: FileEventItem[]) => this.handleFileEvent(items[0]), (items: FileEventItem[]) => this.handleFileEvent(items[0]),
{ suspended: true, batchSize: 1, concurrentLimit: 5, delay: 100, yieldThreshold: FileWatchEventQueueMax, totalRemainingReactiveSource: this.pendingFileEventCount, processingEntitiesReactiveSource: this.processingFileEventCount } { suspended: true, batchSize: 1, concurrentLimit: 5, delay: 100, yieldThreshold: FileWatchEventQueueMax, totalRemainingReactiveSource: this.pendingFileEventCount, processingEntitiesReactiveSource: this.processingFileEventCount }
).replaceEnqueueProcessor((items, newItem) => this.queueNextFileEvent(items, newItem)); ).replaceEnqueueProcessor((items, newItem) => this.queueNextFileEvent(items, newItem));
@@ -1393,13 +1425,12 @@ We can perform a command in this file.
getFilePath(file: TAbstractFile): string { getFilePath(file: TAbstractFile): string {
if (file instanceof TFolder) { if (file instanceof TFolder) {
if (file.isRoot()) return ""; if (file.isRoot()) return "";
return this.getFilePath(file.parent) + "/" + file.name; return this.getFilePath(file.parent!) + "/" + file.name;
} }
if (file instanceof TFile) { if (file instanceof TFile) {
return this.getFilePath(file.parent) + "/" + file.name; return this.getFilePath(file.parent!) + "/" + file.name;
} }
return this.getFilePath(file.parent!) + "/" + file.name;
return this.getFilePath(file.parent) + "/" + file.name;
} }
async watchVaultRenameAsync(file: TFile, oldFile: any, cache?: CacheData) { async watchVaultRenameAsync(file: TFile, oldFile: any, cache?: CacheData) {
@@ -1532,7 +1563,7 @@ We can perform a command in this file.
await this.deleteVaultItem(file); await this.deleteVaultItem(file);
} else { } else {
// Conflict has been resolved at this time, // Conflict has been resolved at this time,
await this.pullFile(path, null, true); await this.pullFile(path, undefined, true);
} }
return; return;
} }
@@ -1541,8 +1572,8 @@ We can perform a command in this file.
const doc = existDoc; const doc = existDoc;
if (doc.datatype != "newnote" && doc.datatype != "plain") { if (!isAnyNote(doc)) {
Logger(msg + "ERROR, Invalid datatype: " + path + "(" + doc.datatype + ")", LOG_LEVEL_NOTICE); Logger(msg + "ERROR, Invalid type: " + path + "(" + (doc as any)?.type || "type missing" + ")", LOG_LEVEL_NOTICE);
return; return;
} }
// if (!force && localMtime >= docMtime) return; // if (!force && localMtime >= docMtime) return;
@@ -1551,7 +1582,7 @@ We can perform a command in this file.
Logger(msg + "ERROR, invalid path: " + path, LOG_LEVEL_NOTICE); Logger(msg + "ERROR, invalid path: " + path, LOG_LEVEL_NOTICE);
return; return;
} }
const writeData = doc.datatype == "newnote" ? decodeBinary(doc.data) : getDocData(doc.data); const writeData = readContent(doc);
await this.vaultAccess.ensureDirectory(path); await this.vaultAccess.ensureDirectory(path);
try { try {
let outFile; let outFile;
@@ -1589,11 +1620,13 @@ We can perform a command in this file.
await this.vaultAccess.delete(file, true); await this.vaultAccess.delete(file, true);
} }
Logger(`xxx <- STORAGE (deleted) ${file.path}`); Logger(`xxx <- STORAGE (deleted) ${file.path}`);
Logger(`files: ${dir.children.length}`); if (dir) {
if (dir.children.length == 0) { Logger(`files: ${dir.children.length}`);
if (!this.settings.doNotDeleteFolder) { if (dir.children.length == 0) {
Logger(`All files under the parent directory (${dir.path}) have been deleted, so delete this one.`); if (!this.settings.doNotDeleteFolder) {
await this.deleteVaultItem(dir); Logger(`All files under the parent directory (${dir.path}) have been deleted, so delete this one.`);
await this.deleteVaultItem(dir);
}
} }
} }
} }
@@ -1610,34 +1643,44 @@ We can perform a command in this file.
this.conflictCheckQueue.enqueue(path); this.conflictCheckQueue.enqueue(path);
} }
_saveQueuedFiles = throttle(() => {
const saveData = this.replicationResultProcessor._queue.filter(e => e !== undefined && e !== null).map((e) => e?._id ?? "" as string) as string[];
const kvDBKey = "queued-files"
// localStorage.setItem(lsKey, saveData);
fireAndForget(() => this.kvDB.set(kvDBKey, saveData));
}, 100);
saveQueuedFiles() { saveQueuedFiles() {
const saveData = JSON.stringify(this.replicationResultProcessor._queue.map((e) => e._id)); this._saveQueuedFiles();
const lsKey = "obsidian-livesync-queuefiles-" + this.getVaultName();
localStorage.setItem(lsKey, saveData);
} }
async loadQueuedFiles() { async loadQueuedFiles() {
if (this.settings.suspendParseReplicationResult) return; if (this.settings.suspendParseReplicationResult) return;
if (!this.settings.isConfigured) return; if (!this.settings.isConfigured) return;
const lsKey = "obsidian-livesync-queuefiles-" + this.getVaultName(); const kvDBKey = "queued-files"
const ids = [...new Set(JSON.parse(localStorage.getItem(lsKey) || "[]"))] as string[]; // const ids = [...new Set(JSON.parse(localStorage.getItem(lsKey) || "[]"))] as string[];
const ids = [...new Set(await this.kvDB.get<string[]>(kvDBKey) ?? [])];
const batchSize = 100; const batchSize = 100;
const chunkedIds = arrayToChunkedArray(ids, batchSize); const chunkedIds = arrayToChunkedArray(ids, batchSize);
for await (const idsBatch of chunkedIds) { for await (const idsBatch of chunkedIds) {
const ret = await this.localDatabase.allDocsRaw<EntryDoc>({ keys: idsBatch, include_docs: true, limit: 100 }); const ret = await this.localDatabase.allDocsRaw<EntryDoc>({ keys: idsBatch, include_docs: true, limit: 100 });
this.replicationResultProcessor.enqueueAll(ret.rows.map(doc => doc.doc)); const docs = ret.rows.filter(e => e.doc).map(e => e.doc) as PouchDB.Core.ExistingDocument<EntryDoc>[];
const errors = ret.rows.filter(e => !e.doc && !e.value.deleted);
if (errors.length > 0) {
Logger("Some queued processes were not resurrected");
Logger(JSON.stringify(errors), LOG_LEVEL_VERBOSE);
}
this.replicationResultProcessor.enqueueAll(docs);
await this.replicationResultProcessor.waitForPipeline(); await this.replicationResultProcessor.waitForPipeline();
} }
} }
databaseQueueCount = reactiveSource(0); databaseQueueCount = reactiveSource(0);
databaseQueuedProcessor = new KeyedQueueProcessor(async (docs: EntryBody[]) => { databaseQueuedProcessor = new QueueProcessor(async (docs: EntryBody[]) => {
const dbDoc = docs[0]; const dbDoc = docs[0] as LoadedEntry; // It has no `data`
const path = this.getPath(dbDoc); const path = this.getPath(dbDoc);
// If `Read chunks online` is disabled, chunks should be transferred before here. // If `Read chunks online` is disabled, chunks should be transferred before here.
// However, in some cases, chunks are after that. So, if missing chunks exist, we have to wait for them. // However, in some cases, chunks are after that. So, if missing chunks exist, we have to wait for them.
const datatype = (!("type" in dbDoc) || dbDoc.type == "notes") ? "newnote" : dbDoc.type; const doc = await this.localDatabase.getDBEntryFromMeta({ ...dbDoc }, {}, false, true, true);
const doc = await this.localDatabase.getDBEntryFromMeta({ ...dbDoc, datatype, data: [] }, {}, false, true, true);
if (!doc) { if (!doc) {
Logger(`Something went wrong while gathering content of ${path} (${dbDoc._id.substring(0, 8)}, ${dbDoc._rev?.substring(0, 10)}) `, LOG_LEVEL_NOTICE) Logger(`Something went wrong while gathering content of ${path} (${dbDoc._id.substring(0, 8)}, ${dbDoc._rev?.substring(0, 10)}) `, LOG_LEVEL_NOTICE)
return; return;
@@ -1647,34 +1690,43 @@ We can perform a command in this file.
const filename = this.getPathWithoutPrefix(doc); const filename = this.getPathWithoutPrefix(doc);
this.isTargetFile(filename).then((ret) => ret ? this.addOnHiddenFileSync.procInternalFile(filename) : Logger(`Skipped (Not target:${filename})`, LOG_LEVEL_VERBOSE)); this.isTargetFile(filename).then((ret) => ret ? this.addOnHiddenFileSync.procInternalFile(filename) : Logger(`Skipped (Not target:${filename})`, LOG_LEVEL_VERBOSE));
} else if (isValidPath(this.getPath(doc))) { } else if (isValidPath(this.getPath(doc))) {
this.storageApplyingProcessor.enqueueWithKey(doc.path, doc); this.storageApplyingProcessor.enqueue(doc);
} else { } else {
Logger(`Skipped: ${doc._id.substring(0, 8)}`, LOG_LEVEL_VERBOSE); Logger(`Skipped: ${doc._id.substring(0, 8)}`, LOG_LEVEL_VERBOSE);
} }
return; return;
}, { suspended: true, batchSize: 1, concurrentLimit: 10, yieldThreshold: 1, delay: 0, totalRemainingReactiveSource: this.databaseQueueCount }).startPipeline(); }, { suspended: true, batchSize: 1, concurrentLimit: 10, yieldThreshold: 1, delay: 0, totalRemainingReactiveSource: this.databaseQueueCount }).replaceEnqueueProcessor((queue, newItem) => {
const q = queue.filter(e => e._id != newItem._id);
return [...q, newItem];
}).startPipeline();
storageApplyingCount = reactiveSource(0); storageApplyingCount = reactiveSource(0);
storageApplyingProcessor = new KeyedQueueProcessor(async (docs: LoadedEntry[]) => { storageApplyingProcessor = new QueueProcessor(async (docs: LoadedEntry[]) => {
const entry = docs[0]; const entry = docs[0];
const path = this.getPath(entry); await serialized(entry.path, async () => {
Logger(`Processing ${path} (${entry._id.substring(0, 8)}: ${entry._rev?.substring(0, 5)}) change...`, LOG_LEVEL_VERBOSE); const path = this.getPath(entry);
const targetFile = this.vaultAccess.getAbstractFileByPath(this.getPathWithoutPrefix(entry)); Logger(`Processing ${path} (${entry._id.substring(0, 8)}: ${entry._rev?.substring(0, 5)}) :Started...`, LOG_LEVEL_VERBOSE);
if (targetFile instanceof TFolder) { const targetFile = this.vaultAccess.getAbstractFileByPath(this.getPathWithoutPrefix(entry));
Logger(`${this.getPath(entry)} is already exist as the folder`); if (targetFile instanceof TFolder) {
} else { Logger(`${this.getPath(entry)} is already exist as the folder`);
await this.processEntryDoc(entry, targetFile instanceof TFile ? targetFile : undefined); } else {
Logger(`Processing ${path} (${entry._id.substring(0, 8)}:${entry._rev?.substring(0, 5)}) `); await this.processEntryDoc(entry, targetFile instanceof TFile ? targetFile : undefined);
} Logger(`Processing ${path} (${entry._id.substring(0, 8)} :${entry._rev?.substring(0, 5)}) : Done`);
}
});
return; return;
}, { suspended: true, batchSize: 1, concurrentLimit: 2, yieldThreshold: 1, delay: 0, totalRemainingReactiveSource: this.storageApplyingCount }).startPipeline() }, { suspended: true, batchSize: 1, concurrentLimit: 6, yieldThreshold: 1, delay: 0, totalRemainingReactiveSource: this.storageApplyingCount }).replaceEnqueueProcessor((queue, newItem) => {
const q = queue.filter(e => e._id != newItem._id);
return [...q, newItem];
}).startPipeline()
replicationResultCount = reactiveSource(0); replicationResultCount = reactiveSource(0);
replicationResultProcessor = new QueueProcessor(async (docs: PouchDB.Core.ExistingDocument<EntryDoc>[]) => { replicationResultProcessor = new QueueProcessor(async (docs: PouchDB.Core.ExistingDocument<EntryDoc>[]) => {
if (this.settings.suspendParseReplicationResult) return; if (this.settings.suspendParseReplicationResult) return;
const change = docs[0]; const change = docs[0];
if (!change) return;
if (isChunk(change._id)) { if (isChunk(change._id)) {
// SendSignal? // SendSignal?
// this.parseIncomingChunk(change); // this.parseIncomingChunk(change);
@@ -1699,7 +1751,7 @@ We can perform a command in this file.
) { ) {
return; return;
} }
if (change.type == "plain" || change.type == "newnote") { if (isAnyNote(change)) {
if (this.databaseQueuedProcessor._isSuspended) { if (this.databaseQueuedProcessor._isSuspended) {
Logger(`Processing scheduled: ${change.path}`, LOG_LEVEL_INFO); Logger(`Processing scheduled: ${change.path}`, LOG_LEVEL_INFO);
} }
@@ -1708,19 +1760,22 @@ We can perform a command in this file.
Logger(`Processing ${change.path} has been skipped due to file size exceeding the limit`, LOG_LEVEL_NOTICE); Logger(`Processing ${change.path} has been skipped due to file size exceeding the limit`, LOG_LEVEL_NOTICE);
return; return;
} }
this.databaseQueuedProcessor.enqueueWithKey(change.path, change); this.databaseQueuedProcessor.enqueue(change);
} }
return; return;
}, { batchSize: 1, suspended: true, concurrentLimit: 100, delay: 0, totalRemainingReactiveSource: this.replicationResultCount }).startPipeline().onUpdateProgress(() => { }, { batchSize: 1, suspended: true, concurrentLimit: 100, delay: 0, totalRemainingReactiveSource: this.replicationResultCount }).replaceEnqueueProcessor((queue, newItem) => {
const q = queue.filter(e => e._id != newItem._id);
return [...q, newItem];
}).startPipeline().onUpdateProgress(() => {
this.saveQueuedFiles(); this.saveQueuedFiles();
}); });
//---> Sync //---> Sync
parseReplicationResult(docs: Array<PouchDB.Core.ExistingDocument<EntryDoc>>) { parseReplicationResult(docs: Array<PouchDB.Core.ExistingDocument<EntryDoc>>) {
if (this.settings.suspendParseReplicationResult) { if (this.settings.suspendParseReplicationResult && !this.replicationResultProcessor.isSuspended) {
this.replicationResultProcessor.suspend() this.replicationResultProcessor.suspend()
} }
this.replicationResultProcessor.enqueueAll(docs); this.replicationResultProcessor.enqueueAll(docs);
if (!this.settings.suspendParseReplicationResult) { if (!this.settings.suspendParseReplicationResult && this.replicationResultProcessor.isSuspended) {
this.replicationResultProcessor.resume() this.replicationResultProcessor.resume()
} }
} }
@@ -1750,8 +1805,33 @@ We can perform a command in this file.
lastMessage = ""; lastMessage = "";
observeForLogs() { observeForLogs() {
const padSpaces = `\u{2007}`.repeat(10);
// const emptyMark = `\u{2003}`;
const rerenderTimer = new Map<string, [ReturnType<typeof setTimeout>, number]>;
const tick = reactiveSource(0);
function padLeftSp(num: number, mark: string) {
const numLen = `${num}`.length + 1;
const [timer, len] = rerenderTimer.get(mark) ?? [undefined, numLen];
if (num || timer) {
if (num) {
if (timer) clearTimeout(timer);
rerenderTimer.set(mark, [setTimeout(async () => {
rerenderTimer.delete(mark);
await delay(100);
tick.value = tick.value + 1;
}, 3000), Math.max(len, numLen)]);
}
return ` ${mark}${`${padSpaces}${num}`.slice(-(len))}`;
} else {
return "";
}
}
// const logStore // const logStore
const queueCountLabel = reactive(() => { const queueCountLabel = reactive(() => {
// For invalidating
// @ts-ignore
// eslint-disable-next-line @typescript-eslint/no-unused-vars
const _ = tick.value;
const dbCount = this.databaseQueueCount.value; const dbCount = this.databaseQueueCount.value;
const replicationCount = this.replicationResultCount.value; const replicationCount = this.replicationResultCount.value;
const storageApplyingCount = this.storageApplyingCount.value; const storageApplyingCount = this.storageApplyingCount.value;
@@ -1759,13 +1839,13 @@ We can perform a command in this file.
const pluginScanCount = pluginScanningCount.value; const pluginScanCount = pluginScanningCount.value;
const hiddenFilesCount = hiddenFilesEventCount.value + hiddenFilesProcessingCount.value; const hiddenFilesCount = hiddenFilesEventCount.value + hiddenFilesProcessingCount.value;
const conflictProcessCount = this.conflictProcessQueueCount.value; const conflictProcessCount = this.conflictProcessQueueCount.value;
const labelReplication = replicationCount ? `📥 ${replicationCount} ` : ""; const labelReplication = padLeftSp(replicationCount, `📥`);
const labelDBCount = dbCount ? `📄 ${dbCount} ` : ""; const labelDBCount = padLeftSp(dbCount, `📄`);
const labelStorageCount = storageApplyingCount ? `💾 ${storageApplyingCount}` : ""; const labelStorageCount = padLeftSp(storageApplyingCount, `💾`);
const labelChunkCount = chunkCount ? `🧩${chunkCount} ` : ""; const labelChunkCount = padLeftSp(chunkCount, `🧩`);
const labelPluginScanCount = pluginScanCount ? `🔌${pluginScanCount} ` : ""; const labelPluginScanCount = padLeftSp(pluginScanCount, `🔌`);
const labelHiddenFilesCount = hiddenFilesCount ? `⚙️${hiddenFilesCount} ` : ""; const labelHiddenFilesCount = padLeftSp(hiddenFilesCount, `⚙️`)
const labelConflictProcessCount = conflictProcessCount ? `🔩${conflictProcessCount} ` : ""; const labelConflictProcessCount = padLeftSp(conflictProcessCount, `🔩`);
return `${labelReplication}${labelDBCount}${labelStorageCount}${labelChunkCount}${labelPluginScanCount}${labelHiddenFilesCount}${labelConflictProcessCount}`; return `${labelReplication}${labelDBCount}${labelStorageCount}${labelChunkCount}${labelPluginScanCount}${labelHiddenFilesCount}${labelConflictProcessCount}`;
}) })
const requestingStatLabel = reactive(() => { const requestingStatLabel = reactive(() => {
@@ -1810,11 +1890,15 @@ We can perform a command in this file.
return { w, sent, pushLast, arrived, pullLast }; return { w, sent, pushLast, arrived, pullLast };
}) })
const waitingLabel = reactive(() => { const waitingLabel = reactive(() => {
// For invalidating
// @ts-ignore
// eslint-disable-next-line @typescript-eslint/no-unused-vars
const _ = tick.value;
const e = this.pendingFileEventCount.value; const e = this.pendingFileEventCount.value;
const proc = this.processingFileEventCount.value; const proc = this.processingFileEventCount.value;
const pend = e - proc; const pend = e - proc;
const labelProc = proc != 0 ? `⏳${proc} ` : ""; const labelProc = padLeftSp(proc, `⏳`);
const labelPend = pend != 0 ? ` 🛫${pend}` : ""; const labelPend = padLeftSp(pend, `🛫`);
return `${labelProc}${labelPend}`; return `${labelProc}${labelPend}`;
}) })
const statusLineLabel = reactive(() => { const statusLineLabel = reactive(() => {
@@ -1823,7 +1907,7 @@ We can perform a command in this file.
const waiting = waitingLabel.value; const waiting = waitingLabel.value;
const networkActivity = requestingStatLabel.value; const networkActivity = requestingStatLabel.value;
return { return {
message: `${networkActivity}Sync: ${w} ↑${sent}${pushLast} ↓${arrived}${pullLast}${waiting} ${queued}`, message: `${networkActivity}Sync: ${w} ↑ ${sent}${pushLast} ↓ ${arrived}${pullLast}${waiting}${queued}`,
}; };
}) })
const statusBarLabels = reactive(() => { const statusBarLabels = reactive(() => {
@@ -1834,45 +1918,33 @@ We can perform a command in this file.
message, status message, status
} }
}) })
let last = 0;
const applyToDisplay = () => { const applyToDisplay = throttle(() => {
const v = statusBarLabels.value; const v = statusBarLabels.value;
const now = Date.now();
if (now - last < 10) {
scheduleTask("applyToDisplay", 20, () => applyToDisplay());
return;
}
this.applyStatusBarText(v.message, v.status); this.applyStatusBarText(v.message, v.status);
last = now;
} }, 20);
statusBarLabels.onChanged(applyToDisplay); statusBarLabels.onChanged(applyToDisplay);
} }
applyStatusBarText(message: string, log: string) { applyStatusBarText(message: string, log: string) {
const newMsg = message; const newMsg = message.replace(/\n/g, "\\A ");
const newLog = log; const newLog = log.replace(/\n/g, "\\A ");
// scheduleTask("update-display", 50, () => {
this.statusBar?.setText(newMsg.split("\n")[0]); this.statusBar?.setText(newMsg.split("\n")[0]);
const selector = `.CodeMirror-wrap,` +
`.markdown-preview-view.cm-s-obsidian,` +
`.markdown-source-view.cm-s-obsidian,` +
`.canvas-wrapper,` +
`.empty-state`
;
if (this.settings.showStatusOnEditor) { if (this.settings.showStatusOnEditor) {
const root = activeDocument.documentElement; const root = activeDocument.documentElement;
const q = root.querySelectorAll(selector); root.style.setProperty("--sls-log-text", "'" + (newMsg + "\\A " + newLog) + "'");
q.forEach(e => e.setAttr("data-log", '' + (newMsg + "\n" + newLog) + ''))
} else { } else {
const root = activeDocument.documentElement; // const root = activeDocument.documentElement;
const q = root.querySelectorAll(selector); // root.style.setProperty("--log-text", "'" + (newMsg + "\\A " + newLog) + "'");
q.forEach(e => e.setAttr("data-log", ''))
} }
// }, true);
scheduleTask("log-hide", 3000, () => { this.statusLog.value = "" }); scheduleTask("log-hide", 3000, () => { this.statusLog.value = "" });
} }
async replicate(showMessage?: boolean) { async replicate(showMessage: boolean = false) {
if (!this.isReady) return; if (!this.isReady) return;
if (isLockAcquired("cleanup")) { if (isLockAcquired("cleanup")) {
Logger("Database cleaning up is in process. replication has been cancelled", LOG_LEVEL_NOTICE); Logger("Database cleaning up is in process. replication has been cancelled", LOG_LEVEL_NOTICE);
@@ -1902,7 +1974,7 @@ Even if you choose to clean up, you will see this option again if you exit Obsid
const CHOICE_DISMISS = "Dismiss"; const CHOICE_DISMISS = "Dismiss";
const ret = await confirmWithMessage(this, "Cleaned", message, [CHOICE_FETCH, CHOICE_CLEAN, CHOICE_DISMISS], CHOICE_DISMISS, 30); const ret = await confirmWithMessage(this, "Cleaned", message, [CHOICE_FETCH, CHOICE_CLEAN, CHOICE_DISMISS], CHOICE_DISMISS, 30);
if (ret == CHOICE_FETCH) { if (ret == CHOICE_FETCH) {
await performRebuildDB(this, "localOnlyWithChunks"); await performRebuildDB(this, "localOnly");
} }
if (ret == CHOICE_CLEAN) { if (ret == CHOICE_CLEAN) {
const remoteDB = await this.getReplicator().connectRemoteCouchDBWithSetting(this.settings, this.getIsMobile(), true); const remoteDB = await this.getReplicator().connectRemoteCouchDBWithSetting(this.settings, this.getIsMobile(), true);
@@ -1936,7 +2008,7 @@ Or if you are sure know what had been happened, we can unlock the database from
const CHOICE_DISMISS = "Dismiss"; const CHOICE_DISMISS = "Dismiss";
const ret = await confirmWithMessage(this, "Locked", message, [CHOICE_FETCH, CHOICE_DISMISS], CHOICE_DISMISS, 10); const ret = await confirmWithMessage(this, "Locked", message, [CHOICE_FETCH, CHOICE_DISMISS], CHOICE_DISMISS, 10);
if (ret == CHOICE_FETCH) { if (ret == CHOICE_FETCH) {
await performRebuildDB(this, "localOnlyWithChunks"); await performRebuildDB(this, "localOnly");
} }
} }
} }
@@ -1945,7 +2017,7 @@ Or if you are sure know what had been happened, we can unlock the database from
return ret; return ret;
} }
async initializeDatabase(showingNotice?: boolean, reopenDatabase = true) { async initializeDatabase(showingNotice: boolean = false, reopenDatabase = true) {
this.isReady = false; this.isReady = false;
if ((!reopenDatabase) || await this.openDatabase()) { if ((!reopenDatabase) || await this.openDatabase()) {
if (this.localDatabase.isReady) { if (this.localDatabase.isReady) {
@@ -1963,17 +2035,17 @@ Or if you are sure know what had been happened, we can unlock the database from
} }
} }
async replicateAllToServer(showingNotice?: boolean) { async replicateAllToServer(showingNotice: boolean = false) {
if (!this.isReady) return false; if (!this.isReady) return false;
await Promise.all(this.addOns.map(e => e.beforeReplicate(showingNotice))); await Promise.all(this.addOns.map(e => e.beforeReplicate(showingNotice)));
return await this.replicator.replicateAllToServer(this.settings, showingNotice); return await this.replicator.replicateAllToServer(this.settings, showingNotice);
} }
async replicateAllFromServer(showingNotice?: boolean) { async replicateAllFromServer(showingNotice: boolean = false) {
if (!this.isReady) return false; if (!this.isReady) return false;
return await this.replicator.replicateAllFromServer(this.settings, showingNotice); return await this.replicator.replicateAllFromServer(this.settings, showingNotice);
} }
async markRemoteLocked(lockByClean?: boolean) { async markRemoteLocked(lockByClean: boolean = false) {
return await this.replicator.markRemoteLocked(this.settings, true, lockByClean); return await this.replicator.markRemoteLocked(this.settings, true, lockByClean);
} }
@@ -2042,9 +2114,15 @@ Or if you are sure know what had been happened, we can unlock the database from
const syncFiles = filesStorage.filter((e) => onlyInStorageNames.indexOf(e.path) == -1); const syncFiles = filesStorage.filter((e) => onlyInStorageNames.indexOf(e.path) == -1);
Logger("Updating database by new files"); Logger("Updating database by new files");
const processStatus = {} as Record<string, string>;
const logLevel = showingNotice ? LOG_LEVEL_NOTICE : LOG_LEVEL_INFO;
const updateLog = throttle((key: string, msg: string) => {
processStatus[key] = msg;
const log = Object.values(processStatus).join("\n");
Logger(log, logLevel, "syncAll");
}, 25);
const initProcess = []; const initProcess = [];
const logLevel = showingNotice ? LOG_LEVEL_NOTICE : LOG_LEVEL_INFO;
const runAll = async<T>(procedureName: string, objects: T[], callback: (arg: T) => Promise<void>) => { const runAll = async<T>(procedureName: string, objects: T[], callback: (arg: T) => Promise<void>) => {
if (objects.length == 0) { if (objects.length == 0) {
Logger(`${procedureName}: Nothing to do`); Logger(`${procedureName}: Nothing to do`);
@@ -2066,12 +2144,14 @@ Or if you are sure know what had been happened, we can unlock the database from
failed++; failed++;
} }
if ((success + failed) % step == 0) { if ((success + failed) % step == 0) {
Logger(`${procedureName}: DONE:${success}, FAILED:${failed}, LAST:${processor._queue.length}`, logLevel, `log-${procedureName}`); const msg = `${procedureName}: DONE:${success}, FAILED:${failed}, LAST:${processor._queue.length}`;
updateLog(procedureName, msg);
} }
return; return;
}, { batchSize: 1, concurrentLimit: 10, delay: 0, suspended: true }, objects) }, { batchSize: 1, concurrentLimit: 10, delay: 0, suspended: true }, objects)
await processor.waitForPipeline(); await processor.waitForPipeline();
Logger(`${procedureName} All done: DONE:${success}, FAILED:${failed}`, logLevel, `log-${procedureName}`); const msg = `${procedureName} All done: DONE:${success}, FAILED:${failed}`;
updateLog(procedureName, msg)
} }
initProcess.push(runAll("UPDATE DATABASE", onlyInStorage, async (e) => { initProcess.push(runAll("UPDATE DATABASE", onlyInStorage, async (e) => {
if (!this.isFileSizeExceeded(e.stat.size)) { if (!this.isFileSizeExceeded(e.stat.size)) {
@@ -2085,7 +2165,7 @@ Or if you are sure know what had been happened, we can unlock the database from
const w = await this.localDatabase.getDBEntryMeta(e, {}, true); const w = await this.localDatabase.getDBEntryMeta(e, {}, true);
if (w && !(w.deleted || w._deleted)) { if (w && !(w.deleted || w._deleted)) {
if (!this.isFileSizeExceeded(w.size)) { if (!this.isFileSizeExceeded(w.size)) {
await this.pullFile(e, filesStorage, false, null, false); await this.pullFile(e, filesStorage, false, undefined, false);
fireAndForget(() => this.checkAndApplySettingFromMarkdown(e, true)); fireAndForget(() => this.checkAndApplySettingFromMarkdown(e, true));
Logger(`Check or pull from db:${e} OK`); Logger(`Check or pull from db:${e} OK`);
} else { } else {
@@ -2105,7 +2185,6 @@ Or if you are sure know what had been happened, we can unlock the database from
const id = await this.path2id(getPathFromTFile(file)); const id = await this.path2id(getPathFromTFile(file));
const pair: FileDocPair = { file, id }; const pair: FileDocPair = { file, id };
return [pair]; return [pair];
// processSyncFile.enqueue(pair);
} }
, { batchSize: 1, concurrentLimit: 10, delay: 0, suspended: true }, syncFiles); , { batchSize: 1, concurrentLimit: 10, delay: 0, suspended: true }, syncFiles);
processPrepareSyncFile processPrepareSyncFile
@@ -2113,7 +2192,7 @@ Or if you are sure know what had been happened, we can unlock the database from
new QueueProcessor( new QueueProcessor(
async (pairs) => { async (pairs) => {
const docs = await this.localDatabase.allDocsRaw<EntryDoc>({ keys: pairs.map(e => e.id), include_docs: true }); const docs = await this.localDatabase.allDocsRaw<EntryDoc>({ keys: pairs.map(e => e.id), include_docs: true });
const docsMap = docs.rows.reduce((p, c) => ({ ...p, [c.id]: c.doc }), {} as Record<DocumentID, EntryDoc>); const docsMap = Object.fromEntries(docs.rows.map(e => [e.id, e.doc]));
const syncFilesToSync = pairs.map((e) => ({ file: e.file, doc: docsMap[e.id] as LoadedEntry })); const syncFilesToSync = pairs.map((e) => ({ file: e.file, doc: docsMap[e.id] as LoadedEntry }));
return syncFilesToSync; return syncFilesToSync;
} }
@@ -2127,10 +2206,18 @@ Or if you are sure know what had been happened, we can unlock the database from
}, { batchSize: 1, concurrentLimit: 5, delay: 10, suspended: false } }, { batchSize: 1, concurrentLimit: 5, delay: 10, suspended: false }
)) ))
processPrepareSyncFile.startPipeline(); const allSyncFiles = syncFiles.length;
initProcess.push(async () => { let lastRemain = allSyncFiles;
await processPrepareSyncFile.waitForPipeline(); const step = 25;
}) const remainLog = (remain: number) => {
if (lastRemain - remain > step) {
const msg = ` CHECK AND SYNC: ${remain} / ${allSyncFiles}`;
updateLog("sync", msg);
lastRemain = remain;
}
}
processPrepareSyncFile.startPipeline().onUpdateProgress(() => remainLog(processPrepareSyncFile.totalRemaining + processPrepareSyncFile.nowProcessing))
initProcess.push(processPrepareSyncFile.waitForPipeline());
await Promise.all(initProcess); await Promise.all(initProcess);
// this.setStatusBarText(`NOW TRACKING!`); // this.setStatusBarText(`NOW TRACKING!`);
@@ -2403,11 +2490,11 @@ Or if you are sure know what had been happened, we can unlock the database from
const conflictedRevNo = Number(conflictedRev.split("-")[0]); const conflictedRevNo = Number(conflictedRev.split("-")[0]);
//Search //Search
const revFrom = (await this.localDatabase.getRaw<EntryDoc>(await this.path2id(path), { revs_info: true })); const revFrom = (await this.localDatabase.getRaw<EntryDoc>(await this.path2id(path), { revs_info: true }));
const commonBase = revFrom._revs_info.filter(e => e.status == "available" && Number(e.rev.split("-")[0]) < conflictedRevNo).first()?.rev ?? ""; const commonBase = (revFrom._revs_info || []).filter(e => e.status == "available" && Number(e.rev.split("-")[0]) < conflictedRevNo).first()?.rev ?? "";
let p = undefined; let p = undefined;
if (commonBase) { if (commonBase) {
if (isSensibleMargeApplicable(path)) { if (isSensibleMargeApplicable(path)) {
const result = await this.mergeSensibly(path, commonBase, test._rev, conflictedRev); const result = await this.mergeSensibly(path, commonBase, test._rev!, conflictedRev);
if (result) { if (result) {
p = result.filter(e => e[0] != DIFF_DELETE).map((e) => e[1]).join(""); p = result.filter(e => e[0] != DIFF_DELETE).map((e) => e[1]).join("");
// can be merged. // can be merged.
@@ -2417,7 +2504,7 @@ Or if you are sure know what had been happened, we can unlock the database from
} }
} else if (isObjectMargeApplicable(path)) { } else if (isObjectMargeApplicable(path)) {
// can be merged. // can be merged.
const result = await this.mergeObject(path, commonBase, test._rev, conflictedRev); const result = await this.mergeObject(path, commonBase, test._rev!, conflictedRev);
if (result) { if (result) {
Logger(`Object merge:${path}`, LOG_LEVEL_INFO); Logger(`Object merge:${path}`, LOG_LEVEL_INFO);
p = result; p = result;
@@ -2446,7 +2533,7 @@ Or if you are sure know what had been happened, we can unlock the database from
} }
} }
// should be one or more conflicts; // should be one or more conflicts;
const leftLeaf = await this.getConflictedDoc(path, test._rev); const leftLeaf = await this.getConflictedDoc(path, test._rev!);
const rightLeaf = await this.getConflictedDoc(path, conflicts[0]); const rightLeaf = await this.getConflictedDoc(path, conflicts[0]);
if (leftLeaf == false) { if (leftLeaf == false) {
// what's going on.. // what's going on..
@@ -2456,7 +2543,7 @@ Or if you are sure know what had been happened, we can unlock the database from
if (rightLeaf == false) { if (rightLeaf == false) {
// Conflicted item could not load, delete this. // Conflicted item could not load, delete this.
await this.localDatabase.deleteDBEntry(path, { rev: conflicts[0] }); await this.localDatabase.deleteDBEntry(path, { rev: conflicts[0] });
await this.pullFile(path, null, true); await this.pullFile(path, undefined, true);
Logger(`could not get old revisions, automatically used newer one:${path}`, LOG_LEVEL_NOTICE); Logger(`could not get old revisions, automatically used newer one:${path}`, LOG_LEVEL_NOTICE);
return AUTO_MERGED; return AUTO_MERGED;
} }
@@ -2472,7 +2559,7 @@ Or if you are sure know what had been happened, we can unlock the database from
loser = rightLeaf; loser = rightLeaf;
} }
await this.localDatabase.deleteDBEntry(path, { rev: loser.rev }); await this.localDatabase.deleteDBEntry(path, { rev: loser.rev });
await this.pullFile(path, null, true); await this.pullFile(path, undefined, true);
Logger(`Automatically merged (${isSame ? "same," : ""}${isBinary ? "binary," : ""}${alwaysNewer ? "alwaysNewer" : ""}) :${path}`, LOG_LEVEL_NOTICE); Logger(`Automatically merged (${isSame ? "same," : ""}${isBinary ? "binary," : ""}${alwaysNewer ? "alwaysNewer" : ""}) :${path}`, LOG_LEVEL_NOTICE);
return AUTO_MERGED; return AUTO_MERGED;
} }
@@ -2490,38 +2577,39 @@ Or if you are sure know what had been happened, we can unlock the database from
conflictProcessQueueCount = reactiveSource(0); conflictProcessQueueCount = reactiveSource(0);
conflictResolveQueue = conflictResolveQueue =
new KeyedQueueProcessor(async (entries: { filename: FilePathWithPrefix }[]) => { new QueueProcessor(async (filenames: FilePathWithPrefix[]) => {
const entry = entries[0]; const filename = filenames[0];
const filename = entry.filename; await serialized(`conflict-resolve:${filename}`, async () => {
const conflictCheckResult = await this.checkConflictAndPerformAutoMerge(filename); const conflictCheckResult = await this.checkConflictAndPerformAutoMerge(filename);
if (conflictCheckResult === MISSING_OR_ERROR || conflictCheckResult === NOT_CONFLICTED || conflictCheckResult === CANCELLED) { if (conflictCheckResult === MISSING_OR_ERROR || conflictCheckResult === NOT_CONFLICTED || conflictCheckResult === CANCELLED) {
// nothing to do. // nothing to do.
return;
}
if (conflictCheckResult === AUTO_MERGED) {
//auto resolved, but need check again;
if (this.settings.syncAfterMerge && !this.suspended) {
//Wait for the running replication, if not running replication, run it once.
await shareRunningResult(`replication`, () => this.replicate());
}
Logger("conflict:Automatically merged, but we have to check it again");
this.conflictCheckQueue.enqueue(filename);
return;
}
if (this.settings.showMergeDialogOnlyOnActive) {
const af = this.getActiveFile();
if (af && af.path != filename) {
Logger(`${filename} is conflicted. Merging process has been postponed to the file have got opened.`, LOG_LEVEL_NOTICE);
return; return;
} }
} if (conflictCheckResult === AUTO_MERGED) {
Logger("conflict:Manual merge required!"); //auto resolved, but need check again;
await this.resolveConflictByUI(filename, conflictCheckResult); if (this.settings.syncAfterMerge && !this.suspended) {
//Wait for the running replication, if not running replication, run it once.
await shareRunningResult(`replication`, () => this.replicate());
}
Logger("conflict:Automatically merged, but we have to check it again");
this.conflictCheckQueue.enqueue(filename);
return;
}
if (this.settings.showMergeDialogOnlyOnActive) {
const af = this.getActiveFile();
if (af && af.path != filename) {
Logger(`${filename} is conflicted. Merging process has been postponed to the file have got opened.`, LOG_LEVEL_NOTICE);
return;
}
}
Logger("conflict:Manual merge required!");
await this.resolveConflictByUI(filename, conflictCheckResult);
});
}, { suspended: false, batchSize: 1, concurrentLimit: 1, delay: 10, keepResultUntilDownstreamConnected: false }).replaceEnqueueProcessor( }, { suspended: false, batchSize: 1, concurrentLimit: 1, delay: 10, keepResultUntilDownstreamConnected: false }).replaceEnqueueProcessor(
(queue, newEntity) => { (queue, newEntity) => {
const filename = newEntity.entity.filename; const filename = newEntity;
sendValue("cancel-resolve-conflict:" + filename, true); sendValue("cancel-resolve-conflict:" + filename, true);
const newQueue = [...queue].filter(e => e.key != newEntity.key); const newQueue = [...queue].filter(e => e != newEntity);
return [...newQueue, newEntity]; return [...newQueue, newEntity];
}); });
@@ -2533,10 +2621,9 @@ Or if you are sure know what had been happened, we can unlock the database from
const file = this.vaultAccess.getAbstractFileByPath(filename); const file = this.vaultAccess.getAbstractFileByPath(filename);
// if (!file) return; // if (!file) return;
// if (!(file instanceof TFile)) return; // if (!(file instanceof TFile)) return;
if ((file instanceof TFolder)) return; if ((file instanceof TFolder)) return [];
// Check again? // Check again?
return [filename];
return [{ key: filename, entity: { filename } }];
// this.conflictResolveQueue.enqueueWithKey(filename, { filename, file }); // this.conflictResolveQueue.enqueueWithKey(filename, { filename, file });
}, { }, {
suspended: false, batchSize: 1, concurrentLimit: 5, delay: 10, keepResultUntilDownstreamConnected: true, pipeTo: this.conflictResolveQueue, totalRemainingReactiveSource: this.conflictProcessQueueCount suspended: false, batchSize: 1, concurrentLimit: 5, delay: 10, keepResultUntilDownstreamConnected: true, pipeTo: this.conflictResolveQueue, totalRemainingReactiveSource: this.conflictProcessQueueCount
@@ -2550,16 +2637,16 @@ Or if you are sure know what had been happened, we can unlock the database from
if (selected === CANCELLED) { if (selected === CANCELLED) {
// Cancelled by UI, or another conflict. // Cancelled by UI, or another conflict.
Logger(`Merge: Cancelled ${filename}`, LOG_LEVEL_INFO); Logger(`Merge: Cancelled ${filename}`, LOG_LEVEL_INFO);
return; return false;
} }
const testDoc = await this.localDatabase.getDBEntry(filename, { conflicts: true }, false, false, true); const testDoc = await this.localDatabase.getDBEntry(filename, { conflicts: true }, false, false, true);
if (testDoc === false) { if (testDoc === false) {
Logger(`Merge: Could not read ${filename} from the local database`, LOG_LEVEL_VERBOSE); Logger(`Merge: Could not read ${filename} from the local database`, LOG_LEVEL_VERBOSE);
return; return false;
} }
if (!testDoc._conflicts) { if (!testDoc._conflicts) {
Logger(`Merge: Nothing to do ${filename}`, LOG_LEVEL_VERBOSE); Logger(`Merge: Nothing to do ${filename}`, LOG_LEVEL_VERBOSE);
return; return false;
} }
const toDelete = selected; const toDelete = selected;
const toKeep = conflictCheckResult.left.rev != toDelete ? conflictCheckResult.left.rev : conflictCheckResult.right.rev; const toKeep = conflictCheckResult.left.rev != toDelete ? conflictCheckResult.left.rev : conflictCheckResult.right.rev;
@@ -2581,11 +2668,11 @@ Or if you are sure know what had been happened, we can unlock the database from
Logger(`Merge: Changes has been concatenated: ${filename}`); Logger(`Merge: Changes has been concatenated: ${filename}`);
} else if (typeof toDelete === "string") { } else if (typeof toDelete === "string") {
await this.localDatabase.deleteDBEntry(filename, { rev: toDelete }); await this.localDatabase.deleteDBEntry(filename, { rev: toDelete });
await this.pullFile(filename, null, true, toKeep); await this.pullFile(filename, undefined, true, toKeep);
Logger(`Conflict resolved:${filename}`); Logger(`Conflict resolved:${filename}`);
} else { } else {
Logger(`Merge: Something went wrong: ${filename}, (${toDelete})`, LOG_LEVEL_NOTICE); Logger(`Merge: Something went wrong: ${filename}, (${toDelete})`, LOG_LEVEL_NOTICE);
return; return false;
} }
// In here, some merge has been processed. // In here, some merge has been processed.
// So we have to run replication if configured. // So we have to run replication if configured.
@@ -2594,6 +2681,7 @@ Or if you are sure know what had been happened, we can unlock the database from
} }
// And, check it again. // And, check it again.
this.conflictCheckQueue.enqueue(filename); this.conflictCheckQueue.enqueue(filename);
return false;
} }
async pullFile(filename: FilePathWithPrefix, fileList?: TFile[], force?: boolean, rev?: string, waitForReady = true) { async pullFile(filename: FilePathWithPrefix, fileList?: TFile[], force?: boolean, rev?: string, waitForReady = true) {
@@ -2601,7 +2689,7 @@ Or if you are sure know what had been happened, we can unlock the database from
if (!await this.isTargetFile(filename)) return; if (!await this.isTargetFile(filename)) return;
if (targetFile == null) { if (targetFile == null) {
//have to create; //have to create;
const doc = await this.localDatabase.getDBEntry(filename, rev ? { rev: rev } : null, false, waitForReady); const doc = await this.localDatabase.getDBEntry(filename, rev ? { rev: rev } : undefined, false, waitForReady);
if (doc === false) { if (doc === false) {
Logger(`${filename} Skipped`); Logger(`${filename} Skipped`);
return; return;
@@ -2610,7 +2698,7 @@ Or if you are sure know what had been happened, we can unlock the database from
} else if (targetFile instanceof TFile) { } else if (targetFile instanceof TFile) {
//normal case //normal case
const file = targetFile; const file = targetFile;
const doc = await this.localDatabase.getDBEntry(filename, rev ? { rev: rev } : null, false, waitForReady); const doc = await this.localDatabase.getDBEntry(filename, rev ? { rev: rev } : undefined, false, waitForReady);
if (doc === false) { if (doc === false) {
Logger(`${filename} Skipped`); Logger(`${filename} Skipped`);
return; return;
@@ -2650,7 +2738,7 @@ Or if you are sure know what had been happened, we can unlock the database from
case TARGET_IS_NEW: case TARGET_IS_NEW:
if (!this.isFileSizeExceeded(doc.size)) { if (!this.isFileSizeExceeded(doc.size)) {
Logger("STORAGE <- DB :" + file.path); Logger("STORAGE <- DB :" + file.path);
const docx = await this.localDatabase.getDBEntry(getPathFromTFile(file), null, false, false, true); const docx = await this.localDatabase.getDBEntry(getPathFromTFile(file), undefined, false, false, true);
if (docx != false) { if (docx != false) {
await this.processEntryDoc(docx, file); await this.processEntryDoc(docx, file);
} else { } else {
@@ -2662,7 +2750,7 @@ Or if you are sure know what had been happened, we can unlock the database from
} }
break; break;
case EVEN: case EVEN:
Logger("STORAGE == DB :" + file.path + "", LOG_LEVEL_VERBOSE); Logger("STORAGE == DB :" + file.path + "", LOG_LEVEL_DEBUG);
break; break;
default: default:
Logger("STORAGE ?? DB :" + file.path + " Something got weird"); Logger("STORAGE ?? DB :" + file.path + " Something got weird");
@@ -2675,41 +2763,14 @@ Or if you are sure know what had been happened, we can unlock the database from
if (shouldBeIgnored(file.path)) { if (shouldBeIgnored(file.path)) {
return true; return true;
} }
let content: Blob; // let content: Blob;
let datatype: "plain" | "newnote" = "newnote"; const isPlain = isPlainText(file.name);
if (!cache) { const possiblyLarge = !isPlain;
if (!isPlainText(file.name)) { // if (!cache) {
Logger(`Reading : ${file.path}`, LOG_LEVEL_VERBOSE); if (possiblyLarge) Logger(`Reading : ${file.path}`, LOG_LEVEL_VERBOSE);
const contentBin = await this.vaultAccess.vaultReadBinary(file); const content = createBlob(await this.vaultAccess.vaultReadAuto(file));
Logger(`Processing: ${file.path}`, LOG_LEVEL_VERBOSE); const datatype = determineTypeFromBlob(content);
try { if (possiblyLarge) Logger(`Processing: ${file.path}`, LOG_LEVEL_VERBOSE);
content = createBinaryBlob(contentBin);
} catch (ex) {
Logger(`The file ${file.path} could not be encoded`);
Logger(ex, LOG_LEVEL_VERBOSE);
return false;
}
datatype = "newnote";
} else {
content = createTextBlob(await this.vaultAccess.vaultRead(file));
datatype = "plain";
}
} else {
if (cache instanceof ArrayBuffer) {
Logger(`Cache Processing: ${file.path}`, LOG_LEVEL_VERBOSE);
try {
content = createBinaryBlob(cache);
} catch (ex) {
Logger(`The file ${file.path} could not be encoded`);
Logger(ex, LOG_LEVEL_VERBOSE);
return false;
}
datatype = "newnote"
} else {
content = createTextBlob(cache);
datatype = "plain";
}
}
const fullPath = getPathFromTFile(file); const fullPath = getPathFromTFile(file);
const id = await this.path2id(fullPath); const id = await this.path2id(fullPath);
const d: SavingEntry = { const d: SavingEntry = {
@@ -2730,7 +2791,7 @@ Or if you are sure know what had been happened, we can unlock the database from
return true; return true;
} }
try { try {
const old = await this.localDatabase.getDBEntry(fullPath, null, false, false); const old = await this.localDatabase.getDBEntry(fullPath, undefined, false, false);
if (old !== false) { if (old !== false) {
const oldData = { data: old.data, deleted: old._deleted || old.deleted }; const oldData = { data: old.data, deleted: old._deleted || old.deleted };
const newData = { data: d.data, deleted: d._deleted || d.deleted }; const newData = { data: d.data, deleted: d._deleted || d.deleted };
@@ -2806,7 +2867,7 @@ Or if you are sure know what had been happened, we can unlock the database from
const id = await this.path2id(path); const id = await this.path2id(path);
const doc = await this.localDatabase.getRaw<AnyEntry>(id, { conflicts: true }); const doc = await this.localDatabase.getRaw<AnyEntry>(id, { conflicts: true });
// If there is no conflict, return with false. // If there is no conflict, return with false.
if (!("_conflicts" in doc)) return false; if (!("_conflicts" in doc) || doc._conflicts === undefined) return false;
if (doc._conflicts.length == 0) return false; if (doc._conflicts.length == 0) return false;
Logger(`Hidden file conflicted:${this.getPath(doc)}`); Logger(`Hidden file conflicted:${this.getPath(doc)}`);
const conflicts = doc._conflicts.sort((a, b) => Number(a.split("-")[0]) - Number(b.split("-")[0])); const conflicts = doc._conflicts.sort((a, b) => Number(a.split("-")[0]) - Number(b.split("-")[0]));
@@ -2839,7 +2900,7 @@ Or if you are sure know what had been happened, we can unlock the database from
} }
async getIgnoreFile(path: string) { async getIgnoreFile(path: string) {
if (this.ignoreFileCache.has(path)) { if (this.ignoreFileCache.has(path)) {
return this.ignoreFileCache.get(path); return this.ignoreFileCache.get(path) ?? false;
} else { } else {
return await this.readIgnoreFile(path); return await this.readIgnoreFile(path);
} }
@@ -2915,9 +2976,9 @@ Or if you are sure know what had been happened, we can unlock the database from
const fragment = createFragment((doc) => { const fragment = createFragment((doc) => {
const [beforeText, afterText] = dialogText.split("{HERE}", 2); const [beforeText, afterText] = dialogText.split("{HERE}", 2);
doc.createEl("span", null, (a) => { doc.createEl("span", undefined, (a) => {
a.appendText(beforeText); a.appendText(beforeText);
a.appendChild(a.createEl("a", null, (anchor) => { a.appendChild(a.createEl("a", undefined, (anchor) => {
anchorCallback(anchor); anchorCallback(anchor);
})); }));

25
src/utils.ts
View File

@@ -242,14 +242,11 @@ export function mergeObject(
ret[key] = v; ret[key] = v;
} }
} }
const retSorted = Object.fromEntries(Object.entries(ret).sort((a, b) => a[0] < b[0] ? -1 : a[0] > b[0] ? 1 : 0));
if (Array.isArray(objA) && Array.isArray(objB)) { if (Array.isArray(objA) && Array.isArray(objB)) {
return Object.values(Object.entries(ret) return Object.values(retSorted);
.sort()
.reduce((p, [key, value]) => ({ ...p, [key]: value }), {}));
} }
return Object.entries(ret) return retSorted;
.sort()
.reduce((p, [key, value]) => ({ ...p, [key]: value }), {});
} }
export function flattenObject(obj: Record<string | number | symbol, any>, path: string[] = []): [string, any][] { export function flattenObject(obj: Record<string | number | symbol, any>, path: string[] = []): [string, any][] {
@@ -313,7 +310,7 @@ export function isCustomisationSyncMetadata(str: string): boolean {
export const askYesNo = (app: App, message: string): Promise<"yes" | "no"> => { export const askYesNo = (app: App, message: string): Promise<"yes" | "no"> => {
return new Promise((res) => { return new Promise((res) => {
const popover = new PopoverSelectString(app, message, null, null, (result) => res(result as "yes" | "no")); const popover = new PopoverSelectString(app, message, undefined, undefined, (result) => res(result as "yes" | "no"));
popover.open(); popover.open();
}); });
}; };
@@ -327,7 +324,7 @@ export const askSelectString = (app: App, message: string, items: string[]): Pro
}; };
export const askString = (app: App, title: string, key: string, placeholder: string, isPassword?: boolean): Promise<string | false> => { export const askString = (app: App, title: string, key: string, placeholder: string, isPassword: boolean = false): Promise<string | false> => {
return new Promise((res) => { return new Promise((res) => {
const dialog = new InputStringDialog(app, title, key, placeholder, isPassword, (result) => res(result)); const dialog = new InputStringDialog(app, title, key, placeholder, isPassword, (result) => res(result));
dialog.open(); dialog.open();
@@ -400,7 +397,7 @@ export const _requestToCouchDB = async (baseUri: string, username: string, passw
}; };
return await requestUrl(requestParam); return await requestUrl(requestParam);
} }
export const requestToCouchDB = async (baseUri: string, username: string, password: string, origin: string, key?: string, body?: string, method?: string) => { export const requestToCouchDB = async (baseUri: string, username: string, password: string, origin: string = "", key?: string, body?: string, method?: string) => {
const uri = `_node/_local/_config${key ? "/" + key : ""}`; const uri = `_node/_local/_config${key ? "/" + key : ""}`;
return await _requestToCouchDB(baseUri, username, password, origin, uri, body, method); return await _requestToCouchDB(baseUri, username, password, origin, uri, body, method);
}; };
@@ -440,7 +437,7 @@ export function compareMTime(baseMTime: number, targetMTime: number): typeof BAS
export function markChangesAreSame(file: TFile | AnyEntry | string, mtime1: number, mtime2: number) { export function markChangesAreSame(file: TFile | AnyEntry | string, mtime1: number, mtime2: number) {
if (mtime1 === mtime2) return true; if (mtime1 === mtime2) return true;
const key = typeof file == "string" ? file : file instanceof TFile ? file.path : file.path ?? file._id; const key = typeof file == "string" ? file : file instanceof TFile ? file.path : file.path ?? file._id;
const pairs = sameChangePairs.get(key, []); const pairs = sameChangePairs.get(key, []) || [];
if (pairs.some(e => e == mtime1 || e == mtime2)) { if (pairs.some(e => e == mtime1 || e == mtime2)) {
sameChangePairs.set(key, [...new Set([...pairs, mtime1, mtime2])]); sameChangePairs.set(key, [...new Set([...pairs, mtime1, mtime2])]);
} else { } else {
@@ -449,12 +446,16 @@ export function markChangesAreSame(file: TFile | AnyEntry | string, mtime1: numb
} }
export function isMarkedAsSameChanges(file: TFile | AnyEntry | string, mtimes: number[]) { export function isMarkedAsSameChanges(file: TFile | AnyEntry | string, mtimes: number[]) {
const key = typeof file == "string" ? file : file instanceof TFile ? file.path : file.path ?? file._id; const key = typeof file == "string" ? file : file instanceof TFile ? file.path : file.path ?? file._id;
const pairs = sameChangePairs.get(key, []); const pairs = sameChangePairs.get(key, []) || [];
if (mtimes.every(e => pairs.indexOf(e) !== -1)) { if (mtimes.every(e => pairs.indexOf(e) !== -1)) {
return EVEN; return EVEN;
} }
} }
export function compareFileFreshness(baseFile: TFile | AnyEntry, checkTarget: TFile | AnyEntry): typeof BASE_IS_NEW | typeof TARGET_IS_NEW | typeof EVEN { export function compareFileFreshness(baseFile: TFile | AnyEntry | undefined, checkTarget: TFile | AnyEntry | undefined): typeof BASE_IS_NEW | typeof TARGET_IS_NEW | typeof EVEN {
if (baseFile === undefined && checkTarget == undefined) return EVEN;
if (baseFile == undefined) return TARGET_IS_NEW;
if (checkTarget == undefined) return BASE_IS_NEW;
const modifiedBase = baseFile instanceof TFile ? baseFile?.stat?.mtime ?? 0 : baseFile?.mtime ?? 0; const modifiedBase = baseFile instanceof TFile ? baseFile?.stat?.mtime ?? 0 : baseFile?.mtime ?? 0;
const modifiedTarget = checkTarget instanceof TFile ? checkTarget?.stat?.mtime ?? 0 : checkTarget?.mtime ?? 0; const modifiedTarget = checkTarget instanceof TFile ? checkTarget?.stat?.mtime ?? 0 : checkTarget?.mtime ?? 0;

15
styles.css
View File

@@ -77,14 +77,6 @@
border-top: 1px solid var(--background-modifier-border); border-top: 1px solid var(--background-modifier-border);
} }
/* .sls-table-head{
width:50%;
}
.sls-table-tail{
width:50%;
} */
.sls-header-button { .sls-header-button {
margin-left: 2em; margin-left: 2em;
} }
@@ -94,7 +86,7 @@
} }
:root { :root {
--slsmessage: ""; --sls-log-text: "";
} }
.sls-troubleshoot-preview { .sls-troubleshoot-preview {
@@ -110,7 +102,10 @@
.markdown-source-view.cm-s-obsidian::before, .markdown-source-view.cm-s-obsidian::before,
.canvas-wrapper::before, .canvas-wrapper::before,
.empty-state::before { .empty-state::before {
content: attr(data-log); content: var(--sls-log-text, "");
font-variant-numeric: tabular-nums;
font-variant-emoji: emoji;
tab-size: 4;
text-align: right; text-align: right;
white-space: pre-wrap; white-space: pre-wrap;
position: absolute; position: absolute;

158
updates.md
View File

@@ -10,134 +10,48 @@ Note: we got a very performance improvement.
Note at 0.22.2: **Now, to rescue mobile devices, Maximum file size is set to 50 by default**. Please configure the limit as you need. If you do not want to limit the sizes, set zero manually, please. Note at 0.22.2: **Now, to rescue mobile devices, Maximum file size is set to 50 by default**. Please configure the limit as you need. If you do not want to limit the sizes, set zero manually, please.
#### Version history #### Version history
- 0.22.10 - 0.22.19
- Fixed: - Fixed:
- No longer unchanged hidden files and customisations are saved and transferred now. - No longer data corrupting due to false BASE64 detections.
- File integrity of vault history indicates the integrity correctly.
- Improved: - Improved:
- In the report, the schema of the remote database URI is now printed. - A bit more efficient in Automatic data compression.
- 0.22.9 - 0.22.18
- New feature (Very Experimental):
- Now we can use `Automatic data compression` to reduce amount of traffic and the usage of remote database.
- Please make sure all devices are updated to v0.22.18 before trying this feature.
- If you are using some other utilities which connected to your vault, please make sure that they have compatibilities.
- Note: Setting `File Compression` on the remote database works for shrink the size of remote database. Please refer the [Doc](https://docs.couchdb.org/en/stable/config/couchdb.html#couchdb/file_compression).
- 0.22.17:
- Fixed: - Fixed:
- Fixed a bug on `fetch chunks on demand` that could not fetch the chunks on demand. - Error handling on booting now works fine.
- Replication is now started automatically in LiveSync mode.
- Batch database update is now disabled in LiveSync mode.
- No longer automatically reconnection while off-focused.
- Status saves are thinned out.
- Now Self-hosted LiveSync waits for all files between the local database and storage to be surely checked.
- Improved: - Improved:
- `fetch chunks on demand` works more smoothly. - The job scheduler is now more robust and stable.
- Initialisation `Fetch` is now more efficient. - The status indicator no longer flickers and keeps zero for a while.
- Tidied: - No longer meaningless frequent updates of status indicators.
- Removed some meaningless codes. - Now we can configure regular expression filters in handy UI. Thank you so much, @eth-p!
- 0.22.8 - `Fetch` or `Rebuild everything` is now more safely performed.
- Minor things
- Some utility function has been added.
- Customisation sync now less wrong messages.
- Digging the weeds for eradication of type errors.
- 0.22.16:
- Fixed: - Fixed:
- Now fetch and unlock the locked remote database works well again. - Fixed the issue that binary files were sometimes corrupted.
- No longer crash on symbolic links inside hidden folders. - Fixed customisation sync data could be corrupted.
- Improved: - Improved:
- Chunks are now created more efficiently. - Now the remote database costs lower memory.
- Splitting old notes into a larger chunk. - This release requires a brief wait on the first synchronisation, to track the latest changeset again.
- Better performance in saving notes. - Description added for the `Device name`.
- Network activities are indicated as an icon.
- Less memory used for binary processing.
- Tidied:
- Cleaned unused functions up.
- Sorting out the codes that have become nonsense.
- Changed:
- Now no longer `fetch chunks on demand` needs `Pacing replication`
- The setting `Do not pace synchronization` has been deleted.
- 0.22.7
- Fixed:
- No longer deleted hidden files were ignored.
- The document history dialogue is now able to process the deleted revisions.
- Deletion of a hidden file is now surely performed even if the file is already conflicted.
- 0.22.6
- Fixed:
- Fixed a problem with synchronisation taking a long time to start in some cases.
- The first synchronisation after update might take a bit longer.
- Now we can disable E2EE encryption.
- Improved:
- `Setup Wizard` is now more clear.
- `Minimal Setup` is now more simple.
- Self-hosted LiveSync now be able to use even if there are vaults with the same name.
- Database suffix will automatically added.
- Now Self-hosted LiveSync waits until set-up is complete.
- Show reload prompts when possibly recommended while settings.
- New feature:
- A guidance dialogue prompting for settings will be shown after the installation.
- Changed
- `Open setup URI` is now `Use the copied setup URI`
- `Copy setup URI` is now `Copy current settings as a new setup URI`
- `Setup Wizard` is now `Minimal Setup`
- `Check database configuration` is now `Check and Fix database configuration`
- 0.22.5
- Fixed:
- Some description of settings have been refined
- New feature:
- TroubleShooting is now shown in the setting dialogue.
- 0.22.4
- Fixed:
- Now the result of conflict resolution could be surely written into the storage.
- Deleted files can be handled correctly again in the history dialogue and conflict dialogue.
- Some wrong log messages were fixed.
- Change handling now has become more stable.
- Some event handling became to be safer.
- Improved:
- Dumping document information shows conflicts and revisions.
- The timestamp-only differences can be surely cached.
- Timestamp difference detection can be rounded by two seconds.
- Refactored: - Refactored:
- A bit of organisation to write the test. - Many type-errors have been resolved.
- 0.22.3 - Obsolete file has been deleted.
- Fixed: - 0.22.15:
- No longer detects storage changes which have been caused by Self-hosted LiveSync itself.
- Setting sync file will be detected only if it has been configured now.
- And its log will be shown only while the verbose log is enabled.
- Customisation file enumeration has got less blingy.
- Deletion of files is now reliably synchronised.
- Fixed and improved:
- In-editor-status is now shown in the following areas:
- Note editing pane (Source mode and live-preview mode).
- New tab pane.
- Canvas pane.
- 0.22.2
- Fixed:
- Now the results of resolving conflicts are surely synchronised.
- Modified:
- Some setting items got new clear names. (`Sync Settings` -> `Targets`).
- New feature:
- We can limit the synchronising files by their size. (`Sync Settings` -> `Targets` -> `Maximum file size`).
- It depends on the size of the newer one.
- At Obsidian 1.5.3 on mobile, we should set this to around 50MB to avoid restarting Obsidian.
- Now the settings could be stored in a specific markdown file to synchronise or switch it (`General Setting` -> `Share settings via markdown`).
- [Screwdriver](https://github.com/vrtmrz/obsidian-screwdriver) is quite good, but mostly we only need this.
- Customisation of the obsoleted device is now able to be deleted at once.
- We have to put the maintenance mode in at the Customisation sync dialogue.
- 0.22.1
- New feature:
- We can perform automatic conflict resolution for inactive files, and postpone only manual ones by `Postpone manual resolution of inactive files`.
- Now we can see the image in the document history dialogue.
- We can see the difference of the image, in the document history dialogue.
- And also we can highlight differences.
- Improved: - Improved:
- Hidden file sync has been stabilised. - Faster start-up by removing too many logs which indicates normality
- Now automatically reloads the conflict-resolution dialogue when new conflicted revisions have arrived. - By streamlined scanning of customised synchronisation extra phases have been deleted.
- Fixed:
- No longer periodic process runs after unloading the plug-in.
- Now the modification of binary files is surely stored in the storage.
- 0.22.0
- Refined:
- Task scheduling logics has been rewritten.
- Screen updates are also now efficient.
- Possibly many bugs and fragile behaviour has been fixed.
- Status updates and logging have been thinned out to display.
- Fixed:
- Remote-chunk-fetching now works with keeping request intervals
- New feature:
- We can show only the icons in the editor.
- Progress indicators have been more meaningful:
- 📥 Unprocessed transferred items
- 📄 Working database operation
- 💾 Working write storage processes
- ⏳ Working read storage processes
- 🛫 Pending read storage processes
- ⚙️ Working or pending storage processes of hidden files
- 🧩 Waiting chunks
- 🔌 Working Customisation items (Configuration, snippets and plug-ins)
... To continue on to `updates_old.md`. ... To continue on to `updates_old.md`.

177
updates_old.md
View File

@@ -1,3 +1,180 @@
### 0.22.0
A few years passed since Self-hosted LiveSync was born, and our codebase had been very complicated. This could be patient now, but it should be a tremendous hurt.
Therefore at v0.22.0, for future maintainability, I refined task scheduling logic totally.
Of course, I think this would be our suffering in some cases. However, I would love to ask you for your cooperation and contribution.
Sorry for being absent so much long. And thank you for your patience!
Note: we got a very performance improvement.
Note at 0.22.2: **Now, to rescue mobile devices, Maximum file size is set to 50 by default**. Please configure the limit as you need. If you do not want to limit the sizes, set zero manually, please.
#### Version history
- 0.22.14:
- New feature:
- We can disable the status bar in the setting dialogue.
- Improved:
- Now some files are handled as correct data type.
- Customisation sync now uses the digest of each file for better performance.
- The status in the Editor now works performant.
- Refactored:
- Common functions have been ready and the codebase has been organised.
- Stricter type checking following TypeScript updates.
- Remove old iOS workaround for simplicity and performance.
- 0.22.13:
- Improved:
- Now using HTTP for the remote database URI warns of an error (on mobile) or notice (on desktop).
- Refactored:
- Dependencies have been polished.
- 0.22.12:
- Changed:
- The default settings has been changed.
- Improved:
- Default and preferred settings are applied on completion of the wizard.
- Fixed:
- Now Initialisation `Fetch` will be performed smoothly and there will be fewer conflicts.
- No longer stuck while Handling transferred or initialised documents.
- 0.22.11:
- Fixed:
- `Verify and repair all files` is no longer broken.
- New feature:
- Now `Verify and repair all files` is able to...
- Restore if the file only in the local database.
- Show the history.
- Improved:
- Performance improved.
- 0.22.10
- Fixed:
- No longer unchanged hidden files and customisations are saved and transferred now.
- File integrity of vault history indicates the integrity correctly.
- Improved:
- In the report, the schema of the remote database URI is now printed.
- 0.22.9
- Fixed:
- Fixed a bug on `fetch chunks on demand` that could not fetch the chunks on demand.
- Improved:
- `fetch chunks on demand` works more smoothly.
- Initialisation `Fetch` is now more efficient.
- Tidied:
- Removed some meaningless codes.
- 0.22.8
- Fixed:
- Now fetch and unlock the locked remote database works well again.
- No longer crash on symbolic links inside hidden folders.
- Improved:
- Chunks are now created more efficiently.
- Splitting old notes into a larger chunk.
- Better performance in saving notes.
- Network activities are indicated as an icon.
- Less memory used for binary processing.
- Tidied:
- Cleaned unused functions up.
- Sorting out the codes that have become nonsense.
- Changed:
- Now no longer `fetch chunks on demand` needs `Pacing replication`
- The setting `Do not pace synchronization` has been deleted.
- 0.22.7
- Fixed:
- No longer deleted hidden files were ignored.
- The document history dialogue is now able to process the deleted revisions.
- Deletion of a hidden file is now surely performed even if the file is already conflicted.
- 0.22.6
- Fixed:
- Fixed a problem with synchronisation taking a long time to start in some cases.
- The first synchronisation after update might take a bit longer.
- Now we can disable E2EE encryption.
- Improved:
- `Setup Wizard` is now more clear.
- `Minimal Setup` is now more simple.
- Self-hosted LiveSync now be able to use even if there are vaults with the same name.
- Database suffix will automatically added.
- Now Self-hosted LiveSync waits until set-up is complete.
- Show reload prompts when possibly recommended while settings.
- New feature:
- A guidance dialogue prompting for settings will be shown after the installation.
- Changed
- `Open setup URI` is now `Use the copied setup URI`
- `Copy setup URI` is now `Copy current settings as a new setup URI`
- `Setup Wizard` is now `Minimal Setup`
- `Check database configuration` is now `Check and Fix database configuration`
- 0.22.5
- Fixed:
- Some description of settings have been refined
- New feature:
- TroubleShooting is now shown in the setting dialogue.
- 0.22.4
- Fixed:
- Now the result of conflict resolution could be surely written into the storage.
- Deleted files can be handled correctly again in the history dialogue and conflict dialogue.
- Some wrong log messages were fixed.
- Change handling now has become more stable.
- Some event handling became to be safer.
- Improved:
- Dumping document information shows conflicts and revisions.
- The timestamp-only differences can be surely cached.
- Timestamp difference detection can be rounded by two seconds.
- Refactored:
- A bit of organisation to write the test.
- 0.22.3
- Fixed:
- No longer detects storage changes which have been caused by Self-hosted LiveSync itself.
- Setting sync file will be detected only if it has been configured now.
- And its log will be shown only while the verbose log is enabled.
- Customisation file enumeration has got less blingy.
- Deletion of files is now reliably synchronised.
- Fixed and improved:
- In-editor-status is now shown in the following areas:
- Note editing pane (Source mode and live-preview mode).
- New tab pane.
- Canvas pane.
- 0.22.2
- Fixed:
- Now the results of resolving conflicts are surely synchronised.
- Modified:
- Some setting items got new clear names. (`Sync Settings` -> `Targets`).
- New feature:
- We can limit the synchronising files by their size. (`Sync Settings` -> `Targets` -> `Maximum file size`).
- It depends on the size of the newer one.
- At Obsidian 1.5.3 on mobile, we should set this to around 50MB to avoid restarting Obsidian.
- Now the settings could be stored in a specific markdown file to synchronise or switch it (`General Setting` -> `Share settings via markdown`).
- [Screwdriver](https://github.com/vrtmrz/obsidian-screwdriver) is quite good, but mostly we only need this.
- Customisation of the obsoleted device is now able to be deleted at once.
- We have to put the maintenance mode in at the Customisation sync dialogue.
- 0.22.1
- New feature:
- We can perform automatic conflict resolution for inactive files, and postpone only manual ones by `Postpone manual resolution of inactive files`.
- Now we can see the image in the document history dialogue.
- We can see the difference of the image, in the document history dialogue.
- And also we can highlight differences.
- Improved:
- Hidden file sync has been stabilised.
- Now automatically reloads the conflict-resolution dialogue when new conflicted revisions have arrived.
- Fixed:
- No longer periodic process runs after unloading the plug-in.
- Now the modification of binary files is surely stored in the storage.
- 0.22.0
- Refined:
- Task scheduling logics has been rewritten.
- Screen updates are also now efficient.
- Possibly many bugs and fragile behaviour has been fixed.
- Status updates and logging have been thinned out to display.
- Fixed:
- Remote-chunk-fetching now works with keeping request intervals
- New feature:
- We can show only the icons in the editor.
- Progress indicators have been more meaningful:
- 📥 Unprocessed transferred items
- 📄 Working database operation
- 💾 Working write storage processes
- ⏳ Working read storage processes
- 🛫 Pending read storage processes
- ⚙️ Working or pending storage processes of hidden files
- 🧩 Waiting chunks
- 🔌 Working Customisation items (Configuration, snippets and plug-ins)
... To continue on to `updates_old.md`.
### 0.21.0 ### 0.21.0
The E2EE encryption V2 format has been reverted. That was probably the cause of the glitch. The E2EE encryption V2 format has been reverted. That was probably the cause of the glitch.
Instead, to maintain efficiency, files are treated with Blob until just before saving. Along with this, the old-fashioned encryption format has also been discontinued. Instead, to maintain efficiency, files are treated with Blob until just before saving. Along with this, the old-fashioned encryption format has also been discontinued.