public/obsidian-livesync
1
0
Fork 0
mirror of https://github.com/vrtmrz/obsidian-livesync.git synced 2026-05-21 06:41:33 +00:00
Code Issues Packages Projects Releases Wiki Activity

Compare commits

base: public:0.23.14
Branches Tags
public:main public:feat_tweak_auto_adjust public:test_real_obsidian public:beta public:enhance_fix_p2p public:v0_25_62 public:update_lib public:feat-userhashsalt public:auto_conf_2 public:auto_conf public:alpha public:refactor_services_2 public:refactor_services public:dev public:svelteui public:forprivacy
public:0.25.67 public:0.25.66 public:0.25.65 public:0.25.64-patch1 public:0.25.64 public:0.25.63 public:0.25.62 public:0.25.61 public:0.25.60 public:0.25.59 public:0.25.58 public:0.25.57 public:0.25.56+patched5 public:0.25.56+patched4 public:0.25.56+patched3 public:0.25.56+patched2 public:0.25.56+patched1 public:0.25.56 public:0.25.55 public:0.25.54 public:0.25.53 public:0.25.52-patched-3 public:0.25.52-patched-2 public:0.25.52-patched-1 public:0.25.52 public:0.25.51 public:0.25.50 public:0.25.49 public:0.25.48 public:0.25.47 public:0.25.46 public:0.25.45 public:0.25.44 public:0.25.43-patched-9 public:0.25.43-patched-8 public:0.25.43-patched-7 public:0.25.43-patched-6 public:0.25.43-patched-5 public:0.25.43-patched-4 public:0.25.43-patched-3 public:0.25.43-patched-2 public:0.25.43-patched-1 public:0.25.43 public:0.25.42 public:0.25.41-patched-3 public:0.25.41-patched-2 public:0.25.41-patched-1 public:0.25.41 public:0.25.40 public:0.25.39 public:0.25.38 public:0.25.37 public:0.25.36 public:0.25.35 public:0.25.34 public:0.25.33 public:0.25.32 public:0.25.31 public:0.25.30 public:0.25.30-dev public:0.25.29 public:0.25.28 public:0.25.27 public:0.25.26 public:0.25.25 public:0.25.24 public:0.25.24.beta3 public:0.25.24.beta2 public:0.25.24.beta1 public:0.25.23 public:0.25.23.beta1 public:0.25.22 public:0.25.21 public:0.25.21.beta2 public:0.25.21.beta1 public:0.25.20 public:0.25.19 public:0.25.18 public:0.25.17 public:0.25.16 public:0.25.15 public:0.25.14 public:0.25.13 public:0.25.12 public:0.25.11 public:0.25.10 public:0.25.9 public:0.25.8 public:0.25.7 public:0.25.6 public:0.25.5 public:0.25.4 public:0.25.3 public:0.25.2 public:0.25.1 public:0.25.0 public:0.25.0-beta1 public:0.24.31 public:0.24.30 public:0.24.29 public:0.24.28 public:0.24.27 public:0.24.26 public:0.24.25 public:0.24.24 public:0.24.23 public:0.24.22 public:0.24.21 public:0.24.20 public:0.24.19 public:0.24.18 public:0.24.17 public:0.24.16 public:0.24.15 public:0.24.14 public:0.24.13 public:0.24.12 public:0.24.11 public:0.24.10 public:0.24.9 public:0.24.8 public:0.24.7 public:0.24.6 public:0.24.5 public:0.24.4 public:0.24.3 public:0.24.2 public:0.24.1 public:0.24.0 public:0.24.0.dev-rc8 public:0.24.0.dev-rc7 public:0.24.0.dev-rc6 public:0.24.0.dev-rc5 public:0.24.0.dev-rc4 public:0.24.0.dev-rc3 public:0.24.0.dev-rc2 public:0.23.23 public:0.23.22 public:0.23.21 public:0.23.20 public:0.23.19 public:0.23.18 public:0.23.17 public:0.23.16 public:0.23.15 public:0.23.14 public:0.23.13 public:0.23.12 public:0.23.11 public:0.23.10 public:0.23.9 public:0.23.8 public:0.23.7 public:0.23.6 public:0.23.5 public:0.23.4 public:0.23.3 public:0.23.2 public:0.23.1 public:0.23.0 public:0.22.19 public:0.22.18 public:0.22.17 public:0.22.16 public:0.22.15 public:0.22.14 public:0.22.13 public:0.22.12 public:0.22.11 public:0.22.10 public:0.22.9 public:0.22.8 public:0.22.7 public:0.22.6 public:0.22.5 public:0.22.4 public:0.22.3 public:0.22.2 public:0.22.1 public:0.22.0 public:0.21.5 public:0.21.4 public:0.21.3 public:0.21.2 public:0.21.1 public:0.21.0 public:0.20.7 public:0.20.6 public:0.20.5 public:0.20.4 public:0.20.3 public:0.20.2 public:0.20.1 public:0.20.0 public:0.19.23 public:0.19.22 public:0.19.21 public:0.19.20 public:0.19.19 public:0.19.18 public:0.19.17 public:0.19.16 public:0.19.15 public:0.19.14 public:0.19.13 public:0.19.12 public:0.19.11 public:0.19.10 public:0.19.9 public:0.19.8 public:0.19.7 public:0.19.6 public:0.19.5 public:0.19.4 public:0.19.3 public:0.19.2 public:0.19.1 public:0.19.0 public:0.18.6 public:0.18.5 public:0.18.4 public:0.18.3 public:0.18.2 public:0.18.1 public:0.18.0 public:0.17.34 public:0.17.33 public:0.17.32 public:0.17.31 public:0.17.30 public:0.17.29 public:0.17.28 public:0.17.27 public:0.17.26 public:0.17.25 public:0.17.24 public:0.17.23 public:0.17.22 public:0.17.21 public:0.17.20 public:0.17.19 public:0.17.18 public:0.17.17 public:0.17.16 public:0.17.15 public:0.17.14 public:0.17.13 public:0.17.12 public:0.17.11 public:0.17.10 public:0.17.9 public:0.17.8 public:0.17.7 public:0.17.6 public:0.17.5 public:0.17.4 public:0.17.3 public:0.17.2 public:0.17.1 public:0.17.0 public:0.16.8 public:0.16.7 public:0.16.6 public:0.16.5 public:0.16.4 public:0.16.3 public:0.16.2 public:0.16.1 public:0.16.0 public:0.15.10 public:0.15.9 public:0.15.8 public:0.15.7 public:0.15.6 public:0.15.5 public:0.15.4 public:0.15.3 public:0.15.2 public:0.15.1 public:0.15.0 public:0.14.8 public:0.14.7 public:0.14.6 public:0.14.5 public:0.14.4 public:0.14.3 public:0.14.2 public:0.14.1 public:0.14.0 public:0.13.4 public:0.13.3 public:0.13.2 public:0.13.1 public:0.13.0 public:0.12.5 public:0.12.4 public:0.12.3 public:0.12.2 public:0.12.1 public:0.12.0 public:0.11.10 public:0.11.9 public:0.11.8 public:0.11.7 public:0.11.6 public:0.11.5 public:0.11.4 public:0.11.3 public:0.11.2 public:0.11.1 public:0.11.0 public:0.10.1 public:0.10.0 public:0.9.0 public:0.8.8 public:0.8.7 public:0.8.6 public:0.8.5 public:0.8.3 public:0.8.2 public:0.8.1 public:0.8.0 public:0.7.2 public:0.7.1 public:0.7.0 public:0.6.1 public:0.6.0 public:0.5.0 public:0.4.1 public:0.4.0 public:0.3.1 public:0.3.0 public:0.2.1 public:0.2.0 public:0.1.26 public:0.1.25 public:0.1.24 public:0.1.23 public:0.1.22 public:0.1.21 public:0.1.20 public:0.1.19 public:0.1.18 public:0.1.17 public:0.1.16 public:0.1.15 public:0.1.14 public:0.1.13 public:0.1.12 public:0.1.11 public:0.1.10 public:0.1.9 public:0.1.8 public:0.1.7 public:0.1.6 public:0.1.5 public:0.1.4 public:0.1.3 public:0.1.2 public:0.1.1 public:0.1.0 public:0.0.9 public:0.0.8 public:0.0.7 public:0.0.6
...
compare: public:0.23.22
Branches Tags
public:main public:feat_tweak_auto_adjust public:test_real_obsidian public:beta public:enhance_fix_p2p public:v0_25_62 public:update_lib public:feat-userhashsalt public:auto_conf_2 public:auto_conf public:alpha public:refactor_services_2 public:refactor_services public:dev public:svelteui public:forprivacy
public:0.25.67 public:0.25.66 public:0.25.65 public:0.25.64-patch1 public:0.25.64 public:0.25.63 public:0.25.62 public:0.25.61 public:0.25.60 public:0.25.59 public:0.25.58 public:0.25.57 public:0.25.56+patched5 public:0.25.56+patched4 public:0.25.56+patched3 public:0.25.56+patched2 public:0.25.56+patched1 public:0.25.56 public:0.25.55 public:0.25.54 public:0.25.53 public:0.25.52-patched-3 public:0.25.52-patched-2 public:0.25.52-patched-1 public:0.25.52 public:0.25.51 public:0.25.50 public:0.25.49 public:0.25.48 public:0.25.47 public:0.25.46 public:0.25.45 public:0.25.44 public:0.25.43-patched-9 public:0.25.43-patched-8 public:0.25.43-patched-7 public:0.25.43-patched-6 public:0.25.43-patched-5 public:0.25.43-patched-4 public:0.25.43-patched-3 public:0.25.43-patched-2 public:0.25.43-patched-1 public:0.25.43 public:0.25.42 public:0.25.41-patched-3 public:0.25.41-patched-2 public:0.25.41-patched-1 public:0.25.41 public:0.25.40 public:0.25.39 public:0.25.38 public:0.25.37 public:0.25.36 public:0.25.35 public:0.25.34 public:0.25.33 public:0.25.32 public:0.25.31 public:0.25.30 public:0.25.30-dev public:0.25.29 public:0.25.28 public:0.25.27 public:0.25.26 public:0.25.25 public:0.25.24 public:0.25.24.beta3 public:0.25.24.beta2 public:0.25.24.beta1 public:0.25.23 public:0.25.23.beta1 public:0.25.22 public:0.25.21 public:0.25.21.beta2 public:0.25.21.beta1 public:0.25.20 public:0.25.19 public:0.25.18 public:0.25.17 public:0.25.16 public:0.25.15 public:0.25.14 public:0.25.13 public:0.25.12 public:0.25.11 public:0.25.10 public:0.25.9 public:0.25.8 public:0.25.7 public:0.25.6 public:0.25.5 public:0.25.4 public:0.25.3 public:0.25.2 public:0.25.1 public:0.25.0 public:0.25.0-beta1 public:0.24.31 public:0.24.30 public:0.24.29 public:0.24.28 public:0.24.27 public:0.24.26 public:0.24.25 public:0.24.24 public:0.24.23 public:0.24.22 public:0.24.21 public:0.24.20 public:0.24.19 public:0.24.18 public:0.24.17 public:0.24.16 public:0.24.15 public:0.24.14 public:0.24.13 public:0.24.12 public:0.24.11 public:0.24.10 public:0.24.9 public:0.24.8 public:0.24.7 public:0.24.6 public:0.24.5 public:0.24.4 public:0.24.3 public:0.24.2 public:0.24.1 public:0.24.0 public:0.24.0.dev-rc8 public:0.24.0.dev-rc7 public:0.24.0.dev-rc6 public:0.24.0.dev-rc5 public:0.24.0.dev-rc4 public:0.24.0.dev-rc3 public:0.24.0.dev-rc2 public:0.23.23 public:0.23.22 public:0.23.21 public:0.23.20 public:0.23.19 public:0.23.18 public:0.23.17 public:0.23.16 public:0.23.15 public:0.23.14 public:0.23.13 public:0.23.12 public:0.23.11 public:0.23.10 public:0.23.9 public:0.23.8 public:0.23.7 public:0.23.6 public:0.23.5 public:0.23.4 public:0.23.3 public:0.23.2 public:0.23.1 public:0.23.0 public:0.22.19 public:0.22.18 public:0.22.17 public:0.22.16 public:0.22.15 public:0.22.14 public:0.22.13 public:0.22.12 public:0.22.11 public:0.22.10 public:0.22.9 public:0.22.8 public:0.22.7 public:0.22.6 public:0.22.5 public:0.22.4 public:0.22.3 public:0.22.2 public:0.22.1 public:0.22.0 public:0.21.5 public:0.21.4 public:0.21.3 public:0.21.2 public:0.21.1 public:0.21.0 public:0.20.7 public:0.20.6 public:0.20.5 public:0.20.4 public:0.20.3 public:0.20.2 public:0.20.1 public:0.20.0 public:0.19.23 public:0.19.22 public:0.19.21 public:0.19.20 public:0.19.19 public:0.19.18 public:0.19.17 public:0.19.16 public:0.19.15 public:0.19.14 public:0.19.13 public:0.19.12 public:0.19.11 public:0.19.10 public:0.19.9 public:0.19.8 public:0.19.7 public:0.19.6 public:0.19.5 public:0.19.4 public:0.19.3 public:0.19.2 public:0.19.1 public:0.19.0 public:0.18.6 public:0.18.5 public:0.18.4 public:0.18.3 public:0.18.2 public:0.18.1 public:0.18.0 public:0.17.34 public:0.17.33 public:0.17.32 public:0.17.31 public:0.17.30 public:0.17.29 public:0.17.28 public:0.17.27 public:0.17.26 public:0.17.25 public:0.17.24 public:0.17.23 public:0.17.22 public:0.17.21 public:0.17.20 public:0.17.19 public:0.17.18 public:0.17.17 public:0.17.16 public:0.17.15 public:0.17.14 public:0.17.13 public:0.17.12 public:0.17.11 public:0.17.10 public:0.17.9 public:0.17.8 public:0.17.7 public:0.17.6 public:0.17.5 public:0.17.4 public:0.17.3 public:0.17.2 public:0.17.1 public:0.17.0 public:0.16.8 public:0.16.7 public:0.16.6 public:0.16.5 public:0.16.4 public:0.16.3 public:0.16.2 public:0.16.1 public:0.16.0 public:0.15.10 public:0.15.9 public:0.15.8 public:0.15.7 public:0.15.6 public:0.15.5 public:0.15.4 public:0.15.3 public:0.15.2 public:0.15.1 public:0.15.0 public:0.14.8 public:0.14.7 public:0.14.6 public:0.14.5 public:0.14.4 public:0.14.3 public:0.14.2 public:0.14.1 public:0.14.0 public:0.13.4 public:0.13.3 public:0.13.2 public:0.13.1 public:0.13.0 public:0.12.5 public:0.12.4 public:0.12.3 public:0.12.2 public:0.12.1 public:0.12.0 public:0.11.10 public:0.11.9 public:0.11.8 public:0.11.7 public:0.11.6 public:0.11.5 public:0.11.4 public:0.11.3 public:0.11.2 public:0.11.1 public:0.11.0 public:0.10.1 public:0.10.0 public:0.9.0 public:0.8.8 public:0.8.7 public:0.8.6 public:0.8.5 public:0.8.3 public:0.8.2 public:0.8.1 public:0.8.0 public:0.7.2 public:0.7.1 public:0.7.0 public:0.6.1 public:0.6.0 public:0.5.0 public:0.4.1 public:0.4.0 public:0.3.1 public:0.3.0 public:0.2.1 public:0.2.0 public:0.1.26 public:0.1.25 public:0.1.24 public:0.1.23 public:0.1.22 public:0.1.21 public:0.1.20 public:0.1.19 public:0.1.18 public:0.1.17 public:0.1.16 public:0.1.15 public:0.1.14 public:0.1.13 public:0.1.12 public:0.1.11 public:0.1.10 public:0.1.9 public:0.1.8 public:0.1.7 public:0.1.6 public:0.1.5 public:0.1.4 public:0.1.3 public:0.1.2 public:0.1.1 public:0.1.0 public:0.0.9 public:0.0.8 public:0.0.7 public:0.0.6

18 Commits
0.23.14 ... 0.23.22

Author SHA1 Message Date
vorotamoroz
48e4d57278 bump 2024-09-08 17:58:10 +09:00
vorotamoroz
7eae25edd0 - Fixed: 
- Case-insensitive file handling
    - Full-lower-case files are no longer created during database checking.
  - Bulk chunk transfer
    - The default value will automatically adjust to an acceptable size when using IBM Cloudant.
 2024-09-08 17:55:04 +09:00
vorotamoroz
3285c1694b bump 2024-09-07 01:45:12 +09:00
vorotamoroz
ede126d7d4 - 0.23.21: 
- New Features:
    - Case-insensitive file handling
      - Files can now be handled case-insensitively.
      - This behaviour can be modified in the settings under `Handle files as Case-Sensitive` (Default: Prompt, Enabled for previous behaviour).
    - Improved chunk revision fixing
        - Revisions for chunks can now be fixed for faster chunk creation.
        - This can be adjusted in the settings under `Compute revisions for chunks` (Default: Prompt, Enabled for previous behaviour).
    - Bulk chunk transfer
      - Chunks can now be transferred in bulk during uploads.
      - This feature is enabled by default through `Send chunks in bulk`.
    - Creation of missing chunks without
      - Missing chunks can be created without storing notes, enhancing efficiency for first synchronisation or after prolonged periods without synchronisation.
  - Improvements:
    - File status scanning on the startup
      - Quite significant performance improvements.
      - No more missing scans of some files.
    - Status in editor enhancements
      - Significant performance improvements in the status display within the editor.
      - Notifications for files that will not be synchronised will now be properly communicated.
    - Encryption and Decryption
      - These processes are now performed in background threads to ensure fast and stable transfers.
    - Verify and repair all files
      - Got faster through parallel checking.
    - Migration on update
      - Migration messages and wizards have become more helpful.
  - Behavioural changes:
    - Chunk size adjustments
      - Large chunks will no longer be created for older, stable files, addressing storage consumption issues.
    - Flag file automation
      - Confirmation will be shown and we can cancel it.
  - Fixed:
    - Database File Scanning
      - All files in the database will now be enumerated correctly.
  - Miscellaneous
    - Dependency updated.
    - Now, tree shaking is left to terser, from esbuild.
 2024-09-07 01:43:21 +09:00
vorotamoroz
630889680e bump 2024-07-31 02:32:02 +01:00
vorotamoroz
e46714e0f9 Fixed: 
- Remote Storage Limit Notification dialogue has been fixed, now the chosen value is saved.
Improved:
- The Enlarging button on the enlarging threshold dialogue now displays the new value.
 2024-07-31 02:31:13 +01:00
vorotamoroz
86d5582f37 bump 2024-07-31 02:14:11 +01:00
vorotamoroz
697ee1855b Fixed: 
- Customisation Sync now checks the difference while storing or applying the configuration.
- Time difference in the dialogue has been fixed.
 2024-07-31 02:13:25 +01:00
vorotamoroz
b8edc85528 bump 2024-07-25 13:37:34 +01:00
vorotamoroz
e2740cbefe New feature: 
- Per-file-saved customization sync has been shipped.
- Customisation sync has got beta3.
Improved:
- Start-up speed has been improved.
Fixed:
- On the customisation sync dialogue, buttons are kept within the screen.
- No more unnecessary entries on `data.json` for customisation sync.
- Selections are no longer lost while updating customisation items.
Tidied on source codes:
- Many typos have been fixed.
- Some unnecessary type casting removed.
 2024-07-25 13:36:26 +01:00
vorotamoroz
a96e4e4472 bump 2024-07-12 10:13:04 +01:00
vorotamoroz
dd26bbfe64 Improved: 
- Overall performance has been improved by using PouchDB 9.0.0.
- Configuration mismatch detection is refined. We can resolve mismatches more smoothly and naturally.
Fixed:
- Customisation Sync will be disabled when a corrupted configuration is detected.
New feature:
- We can get a notification about the storage usage of the remote database.
 2024-07-12 10:11:16 +01:00
vorotamoroz
6b9bd473cf bump 2024-07-10 05:24:26 +01:00
vorotamoroz
4be4fa6cc7 Maintenance: 
- Library refining (Phase 1 - step 2). There are no significant changes on the user side.
 2024-07-10 05:23:34 +01:00
vorotamoroz
a9745e850e Improved: 
- The passphrase of the Setup URI is now automatically generated. (#426)
 2024-07-01 11:05:33 +01:00
vorotamoroz
7b9515a47e bump 2024-07-01 06:18:52 +01:00
vorotamoroz
220dce51f2 Dependency Update 2024-07-01 06:16:04 +01:00
vorotamoroz
a23fc866c0 Tidied: 
- Thinning of this repository through the creation of a library of universal functions
 2024-07-01 06:12:23 +01:00
30 changed files with 5964 additions and 4114 deletions
Expand all files Collapse all files

7
docs/setup_own_server.md
View File

@@ -108,13 +108,18 @@ $ export database=obsidiannotes #Please change as you like
$ export passphrase=dfsapkdjaskdjasdas #Please change as you like $ export passphrase=dfsapkdjaskdjasdas #Please change as you like
$ deno run -A https://raw.githubusercontent.com/vrtmrz/obsidian-livesync/main/utils/flyio/generate_setupuri.ts $ deno run -A https://raw.githubusercontent.com/vrtmrz/obsidian-livesync/main/utils/flyio/generate_setupuri.ts
obsidian://setuplivesync?settings=%5B%22tm2DpsOE74nJAryprZO2M93wF%2Fvg.......4b26ed33230729%22%5D obsidian://setuplivesync?settings=%5B%22tm2DpsOE74nJAryprZO2M93wF%2Fvg.......4b26ed33230729%22%5D
Your passphrase of Setup-URI is: patient-haze
This passphrase is never shown again, so please note it in a safe place.
``` ```
Please keep your passphrase of Setup-URI.
### 2. Setup Self-hosted LiveSync to Obsidian ### 2. Setup Self-hosted LiveSync to Obsidian
[This video](https://youtu.be/7sa_I1832Xc?t=146) may help us. [This video](https://youtu.be/7sa_I1832Xc?t=146) may help us.
1. Install Self-hosted LiveSync 1. Install Self-hosted LiveSync
2. Choose `Use the copied setup URI` from the command palette and paste the setup URI. (obsidian://setuplivesync?settings=.....). 2. Choose `Use the copied setup URI` from the command palette and paste the setup URI. (obsidian://setuplivesync?settings=.....).
3. Type `welcome` for setup-uri passphrase. 3. Type the previously displayed passphrase (`patient-haze`) for setup-uri passphrase.
4. Answer `yes` and `Set it up...`, and finish the first dialogue with `Keep them disabled`. 4. Answer `yes` and `Set it up...`, and finish the first dialogue with `Keep them disabled`.
5. `Reload app without save` once. 5. `Reload app without save` once.

21
docs/troubleshooting.md
View File

@@ -20,6 +20,7 @@
- [On the mobile device, cannot synchronise on the local network!](#on-the-mobile-device-cannot-synchronise-on-the-local-network) - [On the mobile device, cannot synchronise on the local network!](#on-the-mobile-device-cannot-synchronise-on-the-local-network)
- [I think that something bad happening on the vault...](#i-think-that-something-bad-happening-on-the-vault) - [I think that something bad happening on the vault...](#i-think-that-something-bad-happening-on-the-vault)
- [Tips](#tips) - [Tips](#tips)
- [How to resolve `Tweaks Mismatched of Changed`](#how-to-resolve-tweaks-mismatched-of-changed)
- [Old tips](#old-tips) - [Old tips](#old-tips)
<!-- - --> <!-- - -->
@@ -111,8 +112,28 @@ Place `redflag.md` on top of the vault, and restart Obsidian. The most simple wa
If there is `redflag.md`, Self-hosted LiveSync suspends all database and storage processes. If there is `redflag.md`, Self-hosted LiveSync suspends all database and storage processes.
## Tips ## Tips
### How to resolve `Tweaks Mismatched of Changed`
(Since v0.23.17)
If you have changed some configurations or tweaks which should be unified between the devices, you will be asked how to reflect (or not) other devices at the next synchronisation. It also occurs on the device itself, where changes are made, to prevent unexpected configuration changes from unwanted propagation.
(We may thank this behaviour if we have synchronised or backed up and restored Self-hosted LiveSync. At least, for me so).
Following dialogue will be shown:
![Dialogue](tweak_mismatch_dialogue.png)
- If we want to propagate the setting of the device, we should choose `Update with mine`.
- On other devices, we should choose `Use configured` to accept and use the configured configuration.
- `Dismiss` can postpone a decision. However, we cannot synchronise until we have decided.
Rest assured that in most cases we can choose `Use configured`. (Unless you are certain that you have not changed the configuration).
If we see it for the first time, it reflects the settings of the device that has been synchronised with the remote for the first time since the upgrade. Probably, we can accept that.
<!-- Add here --> <!-- Add here -->
### Old tips ### Old tips
- Rarely, a file in the database could be corrupted. The plugin will not write to local storage when a file looks corrupted. If a local version of the file is on your device, the corruption could be fixed by editing the local file and synchronizing it. But if the file does not exist on any of your devices, then it can not be rescued. In this case, you can delete these items from the settings dialog. - Rarely, a file in the database could be corrupted. The plugin will not write to local storage when a file looks corrupted. If a local version of the file is on your device, the corruption could be fixed by editing the local file and synchronizing it. But if the file does not exist on any of your devices, then it can not be rescued. In this case, you can delete these items from the settings dialog.
- To stop the boot-up sequence (eg. for fixing problems on databases), you can put a `redflag.md` file (or directory) at the root of your vault. - To stop the boot-up sequence (eg. for fixing problems on databases), you can put a `redflag.md` file (or directory) at the root of your vault.

BIN
docs/tweak_mismatch_dialogue.png Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 46 KiB

8
esbuild.config.mjs
View File

@@ -23,8 +23,8 @@ const keepTest = !prod || dev;
const terserOpt = { const terserOpt = {
sourceMap: !prod sourceMap: !prod
? { ? {
url: "inline", url: "inline",
} }
: {}, : {},
format: { format: {
indent_level: 2, indent_level: 2,
@@ -41,6 +41,7 @@ const terserOpt = {
// compress options // compress options
defaults: false, defaults: false,
evaluate: true, evaluate: true,
dead_code: true,
inline: 3, inline: 3,
join_vars: true, join_vars: true,
loops: true, loops: true,
@@ -57,6 +58,7 @@ const terserOpt = {
ecma: 2018, ecma: 2018,
unused: true, unused: true,
}, },
// mangle: false,
ecma: 2018, // specify one of: 5, 2015, 2016, etc. ecma: 2018, // specify one of: 5, 2015, 2016, etc.
enclose: false, // or specify true, or "args:values" enclose: false, // or specify true, or "args:values"
@@ -122,7 +124,7 @@ const context = await esbuild.context({
logLevel: "info", logLevel: "info",
platform: "browser", platform: "browser",
sourcemap: prod ? false : "inline", sourcemap: prod ? false : "inline",
treeShaking: true, treeShaking: false,
outfile: "main_org.js", outfile: "main_org.js",
mainFields: ["browser", "module", "main"], mainFields: ["browser", "module", "main"],
minifyWhitespace: false, minifyWhitespace: false,

2
manifest.json
View File

@@ -1,7 +1,7 @@
{ {
"id": "obsidian-livesync", "id": "obsidian-livesync",
"name": "Self-hosted LiveSync", "name": "Self-hosted LiveSync",
"version": "0.23.14", "version": "0.23.22",
"minAppVersion": "0.9.12", "minAppVersion": "0.9.12",
"description": "Community implementation of self-hosted livesync. Reflect your vault changes to some other devices immediately. Please make sure to disable other synchronize solutions to avoid content corruption or duplication.", "description": "Community implementation of self-hosted livesync. Reflect your vault changes to some other devices immediately. Please make sure to disable other synchronize solutions to avoid content corruption or duplication.",
"author": "vorotamoroz", "author": "vorotamoroz",

6533
package-lock.json generated
View File

File diff suppressed because it is too large Load Diff

68
package.json
View File

@@ -1,6 +1,6 @@
{ {
"name": "obsidian-livesync", "name": "obsidian-livesync",
"version": "0.23.14", "version": "0.23.22",
"description": "Reflect your vault changes to some other devices immediately. Please make sure to disable other synchronize solutions to avoid content corruption or duplication.", "description": "Reflect your vault changes to some other devices immediately. Please make sure to disable other synchronize solutions to avoid content corruption or duplication.",
"main": "main.js", "main": "main.js",
"type": "module", "type": "module",
@@ -14,56 +14,58 @@
"author": "vorotamoroz", "author": "vorotamoroz",
"license": "MIT", "license": "MIT",
"devDependencies": { "devDependencies": {
"@tsconfig/svelte": "^5.0.2", "@chialab/esbuild-plugin-worker": "^0.18.1",
"@tsconfig/svelte": "^5.0.4",
"@types/diff-match-patch": "^1.0.36", "@types/diff-match-patch": "^1.0.36",
"@types/node": "^20.11.28", "@types/node": "^22.5.4",
"@types/pouchdb": "^6.4.2", "@types/pouchdb": "^6.4.2",
"@types/pouchdb-adapter-http": "^6.1.6", "@types/pouchdb-adapter-http": "^6.1.6",
"@types/pouchdb-adapter-idb": "^6.1.7", "@types/pouchdb-adapter-idb": "^6.1.7",
"@types/pouchdb-browser": "^6.1.5", "@types/pouchdb-browser": "^6.1.5",
"@types/pouchdb-core": "^7.0.14", "@types/pouchdb-core": "^7.0.15",
"@types/pouchdb-mapreduce": "^6.1.10", "@types/pouchdb-mapreduce": "^6.1.10",
"@types/pouchdb-replication": "^6.4.7", "@types/pouchdb-replication": "^6.4.7",
"@types/transform-pouch": "^1.0.6", "@types/transform-pouch": "^1.0.6",
"@typescript-eslint/eslint-plugin": "^7.2.0", "@typescript-eslint/eslint-plugin": "^8.4.0",
"@typescript-eslint/parser": "^7.2.0", "@typescript-eslint/parser": "^8.4.0",
"builtin-modules": "^3.3.0", "builtin-modules": "^4.0.0",
"esbuild": "0.20.2", "esbuild": "0.23.1",
"esbuild-svelte": "^0.8.0", "esbuild-svelte": "^0.8.1",
"eslint": "^8.57.0", "eslint": "^8.57.0",
"eslint-config-airbnb-base": "^15.0.0", "eslint-config-airbnb-base": "^15.0.0",
"eslint-plugin-import": "^2.29.1", "eslint-plugin-import": "^2.30.0",
"events": "^3.3.0", "events": "^3.3.0",
"obsidian": "^1.5.7", "obsidian": "^1.6.6",
"postcss": "^8.4.35", "postcss": "^8.4.45",
"postcss-load-config": "^5.0.3", "postcss-load-config": "^6.0.1",
"pouchdb-adapter-http": "^8.0.1", "pouchdb-adapter-http": "^9.0.0",
"pouchdb-adapter-idb": "^8.0.1", "pouchdb-adapter-idb": "^9.0.0",
"pouchdb-adapter-indexeddb": "^8.0.1", "pouchdb-adapter-indexeddb": "^9.0.0",
"pouchdb-core": "^8.0.1", "pouchdb-core": "^9.0.0",
"pouchdb-errors": "^8.0.1", "pouchdb-errors": "^9.0.0",
"pouchdb-find": "^8.0.1", "pouchdb-find": "^9.0.0",
"pouchdb-mapreduce": "^8.0.1", "pouchdb-mapreduce": "^9.0.0",
"pouchdb-merge": "^8.0.1", "pouchdb-merge": "^9.0.0",
"pouchdb-replication": "^8.0.1", "pouchdb-replication": "^9.0.0",
"pouchdb-utils": "^8.0.1", "pouchdb-utils": "^9.0.0",
"svelte": "^4.2.16", "svelte": "^4.2.19",
"svelte-preprocess": "^5.1.3", "svelte-preprocess": "^6.0.2",
"terser": "^5.29.2", "terser": "^5.31.6",
"transform-pouch": "^2.0.0", "transform-pouch": "^2.0.0",
"tslib": "^2.6.2", "tslib": "^2.7.0",
"typescript": "^5.4.2" "typescript": "^5.5.4"
}, },
"dependencies": { "dependencies": {
"@aws-sdk/client-s3": "^3.556.0", "@aws-sdk/client-s3": "^3.645.0",
"@smithy/fetch-http-handler": "^2.5.0", "@smithy/fetch-http-handler": "^3.2.4",
"@smithy/protocol-http": "^3.3.0", "@smithy/protocol-http": "^4.1.0",
"@smithy/querystring-builder": "^2.2.0", "@smithy/querystring-builder": "^3.0.3",
"diff-match-patch": "^1.0.5", "diff-match-patch": "^1.0.5",
"esbuild-plugin-inline-worker": "^0.1.1", "esbuild-plugin-inline-worker": "^0.1.1",
"fflate": "^0.8.2", "fflate": "^0.8.2",
"idb": "^8.0.0", "idb": "^8.0.0",
"minimatch": "^9.0.3", "minimatch": "^10.0.1",
"octagonal-wheels": "^0.1.14",
"xxhash-wasm": "0.4.2", "xxhash-wasm": "0.4.2",
"xxhash-wasm-102": "npm:xxhash-wasm@^1.0.2" "xxhash-wasm-102": "npm:xxhash-wasm@^1.0.2"
} }

108
setup-flyio-on-the-fly-v2.ipynb
View File

@@ -1,27 +1,10 @@
{ {
"nbformat": 4,
"nbformat_minor": 0,
"metadata": {
"colab": {
"provenance": [],
"private_outputs": true,
"authorship_tag": "ABX9TyMexQ5pErH5LBG2tENtEVWf",
"include_colab_link": true
},
"kernelspec": {
"name": "python3",
"display_name": "Python 3"
},
"language_info": {
"name": "python"
}
},
"cells": [ "cells": [
{ {
"cell_type": "markdown", "cell_type": "markdown",
"metadata": { "metadata": {
"id": "view-in-github", "colab_type": "text",
"colab_type": "text" "id": "view-in-github"
}, },
"source": [ "source": [
"<a href=\"https://colab.research.google.com/gist/vrtmrz/9402b101746e08e969b1a4f5f0deb465/setup-flyio-on-the-fly-v2.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>" "<a href=\"https://colab.research.google.com/gist/vrtmrz/9402b101746e08e969b1a4f5f0deb465/setup-flyio-on-the-fly-v2.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>"
@@ -29,12 +12,12 @@
}, },
{ {
"cell_type": "markdown", "cell_type": "markdown",
"source": [
"- Initial version 7th Feb. 2024"
],
"metadata": { "metadata": {
"id": "AzLlAcLFRO5A" "id": "AzLlAcLFRO5A"
} },
"source": [
"- Initial version 7th Feb. 2024"
]
}, },
{ {
"cell_type": "code", "cell_type": "code",
@@ -55,27 +38,32 @@
}, },
{ {
"cell_type": "code", "cell_type": "code",
"source": [ "execution_count": null,
"# Login up sign up\n",
"!flyctl auth signup"
],
"metadata": { "metadata": {
"id": "mGN08BaFDviy" "id": "mGN08BaFDviy"
}, },
"execution_count": null, "outputs": [],
"outputs": [] "source": [
"# Login up sign up\n",
"!flyctl auth signup"
]
}, },
{ {
"cell_type": "markdown", "cell_type": "markdown",
"source": [
"Select a region and execute the block."
],
"metadata": { "metadata": {
"id": "BBFTFOP6vA8m" "id": "BBFTFOP6vA8m"
} },
"source": [
"Select a region and execute the block."
]
}, },
{ {
"cell_type": "code", "cell_type": "code",
"execution_count": null,
"metadata": {
"id": "TNl0A603EF9E"
},
"outputs": [],
"source": [ "source": [
"# see https://fly.io/docs/reference/regions/\n", "# see https://fly.io/docs/reference/regions/\n",
"region = \"nrt/Tokyo, Japan\" #@param [\"ams/Amsterdam, Netherlands\",\"arn/Stockholm, Sweden\",\"atl/Atlanta, Georgia (US)\",\"bog/Bogotá, Colombia\",\"bos/Boston, Massachusetts (US)\",\"cdg/Paris, France\",\"den/Denver, Colorado (US)\",\"dfw/Dallas, Texas (US)\",\"ewr/Secaucus, NJ (US)\",\"eze/Ezeiza, Argentina\",\"gdl/Guadalajara, Mexico\",\"gig/Rio de Janeiro, Brazil\",\"gru/Sao Paulo, Brazil\",\"hkg/Hong Kong, Hong Kong\",\"iad/Ashburn, Virginia (US)\",\"jnb/Johannesburg, South Africa\",\"lax/Los Angeles, California (US)\",\"lhr/London, United Kingdom\",\"mad/Madrid, Spain\",\"mia/Miami, Florida (US)\",\"nrt/Tokyo, Japan\",\"ord/Chicago, Illinois (US)\",\"otp/Bucharest, Romania\",\"phx/Phoenix, Arizona (US)\",\"qro/Querétaro, Mexico\",\"scl/Santiago, Chile\",\"sea/Seattle, Washington (US)\",\"sin/Singapore, Singapore\",\"sjc/San Jose, California (US)\",\"syd/Sydney, Australia\",\"waw/Warsaw, Poland\",\"yul/Montreal, Canada\",\"yyz/Toronto, Canada\" ] {allow-input: true}\n", "region = \"nrt/Tokyo, Japan\" #@param [\"ams/Amsterdam, Netherlands\",\"arn/Stockholm, Sweden\",\"atl/Atlanta, Georgia (US)\",\"bog/Bogotá, Colombia\",\"bos/Boston, Massachusetts (US)\",\"cdg/Paris, France\",\"den/Denver, Colorado (US)\",\"dfw/Dallas, Texas (US)\",\"ewr/Secaucus, NJ (US)\",\"eze/Ezeiza, Argentina\",\"gdl/Guadalajara, Mexico\",\"gig/Rio de Janeiro, Brazil\",\"gru/Sao Paulo, Brazil\",\"hkg/Hong Kong, Hong Kong\",\"iad/Ashburn, Virginia (US)\",\"jnb/Johannesburg, South Africa\",\"lax/Los Angeles, California (US)\",\"lhr/London, United Kingdom\",\"mad/Madrid, Spain\",\"mia/Miami, Florida (US)\",\"nrt/Tokyo, Japan\",\"ord/Chicago, Illinois (US)\",\"otp/Bucharest, Romania\",\"phx/Phoenix, Arizona (US)\",\"qro/Querétaro, Mexico\",\"scl/Santiago, Chile\",\"sea/Seattle, Washington (US)\",\"sin/Singapore, Singapore\",\"sjc/San Jose, California (US)\",\"syd/Sydney, Australia\",\"waw/Warsaw, Poland\",\"yul/Montreal, Canada\",\"yyz/Toronto, Canada\" ] {allow-input: true}\n",
@@ -98,31 +86,29 @@
" last_line = str.strip(last_line)\n", " last_line = str.strip(last_line)\n",
"\n", "\n",
"if last_line.startswith(\"obsidian://\"):\n", "if last_line.startswith(\"obsidian://\"):\n",
" result = HTML(f\"Copy your setup-URI with this button! -&gt; <button onclick=\\\"navigator.clipboard.writeText('{last_line}')\\\">Copy setup uri</button><br>Importing passphrase is `welcome`. <br>If you want to synchronise in live mode, please apply a preset after ensuring the imported configuration works.\")\n", " result = HTML(f\"Copy your setup-URI with this button! -&gt; <button onclick=\\\"navigator.clipboard.writeText('{last_line}')\\\">Copy setup uri</button><br>Importing passphrase is displayed one. <br>If you want to synchronise in live mode, please apply a preset after ensuring the imported configuration works.\")\n",
"else:\n", "else:\n",
" result = \"Failed to encrypt the setup URI\"\n", " result = \"Failed to encrypt the setup URI\"\n",
"result" "result"
], ]
"metadata": {
"id": "TNl0A603EF9E"
},
"execution_count": null,
"outputs": []
}, },
{ {
"cell_type": "markdown", "cell_type": "markdown",
"metadata": {
"id": "oeIzExnEKhFp"
},
"source": [ "source": [
"If you see the `Copy setup URI` button, Congratulations! Your CouchDB is ready to use! Please click the button. And open this on Obsidian.\n", "If you see the `Copy setup URI` button, Congratulations! Your CouchDB is ready to use! Please click the button. And open this on Obsidian.\n",
"\n", "\n",
"And, you should keep the output to your secret memo.\n", "And, you should keep the output to your secret memo.\n",
"\n" "\n"
], ]
"metadata": {
"id": "oeIzExnEKhFp"
}
}, },
{ {
"cell_type": "markdown", "cell_type": "markdown",
"metadata": {
"id": "sdQrqOjERN3K"
},
"source": [ "source": [
"\n", "\n",
"\n", "\n",
@@ -131,21 +117,35 @@
"\n", "\n",
"If you want to delete this CouchDB instance, you can do it by executing next cell. \n", "If you want to delete this CouchDB instance, you can do it by executing next cell. \n",
"If your fly.toml has been gone, access https://fly.io/dashboard and check the existing app." "If your fly.toml has been gone, access https://fly.io/dashboard and check the existing app."
], ]
"metadata": {
"id": "sdQrqOjERN3K"
}
}, },
{ {
"cell_type": "code", "cell_type": "code",
"source": [ "execution_count": null,
"!./delete-server.sh"
],
"metadata": { "metadata": {
"id": "7JMSkNvVIIfg" "id": "7JMSkNvVIIfg"
}, },
"execution_count": null, "outputs": [],
"outputs": [] "source": [
"!./delete-server.sh"
]
} }
] ],
"metadata": {
"colab": {
"authorship_tag": "ABX9TyMexQ5pErH5LBG2tENtEVWf",
"include_colab_link": true,
"private_outputs": true,
"provenance": []
},
"kernelspec": {
"display_name": "Python 3",
"name": "python3"
},
"language_info": {
"name": "python"
}
},
"nbformat": 4,
"nbformat_minor": 0
} }

5
src/common/dialogs.ts
View File

@@ -19,7 +19,10 @@ export class PluginDialogModal extends Modal {
onOpen() { onOpen() {
const { contentEl } = this; const { contentEl } = this;
this.titleEl.setText("Customization Sync (Beta2)") this.contentEl.style.overflow = "auto";
this.contentEl.style.display = "flex";
this.contentEl.style.flexDirection = "column";
this.titleEl.setText("Customization Sync (Beta3)")
if (!this.component) { if (!this.component) {
this.component = new PluginPane({ this.component = new PluginPane({
target: contentEl, target: contentEl,

16
src/common/events.ts Normal file
View File

@@ -0,0 +1,16 @@
export const EVENT_LAYOUT_READY = "layout-ready";
export const EVENT_PLUGIN_LOADED = "plugin-loaded";
export const EVENT_PLUGIN_UNLOADED = "plugin-unloaded";
export const EVENT_SETTING_SAVED = "setting-saved";
export const EVENT_FILE_RENAMED = "file-renamed";
export const EVENT_LEAF_ACTIVE_CHANGED = "leaf-active-changed";
// export const EVENT_FILE_CHANGED = "file-changed";
import { eventHub } from "../lib/src/hub/hub";
// TODO: Add overloads for the emit method to allow for type checking
export { eventHub };

35
src/common/utils.ts
View File

@@ -15,14 +15,14 @@ export { scheduleTask, setPeriodicTask, cancelTask, cancelAllTasks, cancelPeriod
// For backward compatibility, using the path for determining id. // For backward compatibility, using the path for determining id.
// Only CouchDB unacceptable ID (that starts with an underscore) has been prefixed with "/". // Only CouchDB unacceptable ID (that starts with an underscore) has been prefixed with "/".
// The first slash will be deleted when the path is normalized. // The first slash will be deleted when the path is normalized.
export async function path2id(filename: FilePathWithPrefix | FilePath, obfuscatePassphrase: string | false): Promise<DocumentID> { export async function path2id(filename: FilePathWithPrefix | FilePath, obfuscatePassphrase: string | false, caseInsensitive: boolean): Promise<DocumentID> {
const temp = filename.split(":"); const temp = filename.split(":");
const path = temp.pop(); const path = temp.pop();
const normalizedPath = normalizePath(path as FilePath); const normalizedPath = normalizePath(path as FilePath);
temp.push(normalizedPath); temp.push(normalizedPath);
const fixedPath = temp.join(":") as FilePathWithPrefix; const fixedPath = temp.join(":") as FilePathWithPrefix;
const out = await path2id_base(fixedPath, obfuscatePassphrase); const out = await path2id_base(fixedPath, obfuscatePassphrase, caseInsensitive);
return out; return out;
} }
export function id2path(id: DocumentID, entry?: EntryHasPath): FilePathWithPrefix { export function id2path(id: DocumentID, entry?: EntryHasPath): FilePathWithPrefix {
@@ -465,3 +465,34 @@ export function compareFileFreshness(baseFile: TFile | AnyEntry | undefined, che
return compareMTime(modifiedBase, modifiedTarget); return compareMTime(modifiedBase, modifiedTarget);
} }
const _cached = new Map<string, {
value: any;
context: Map<string, any>;
}>();
export type MemoOption = {
key: string;
forceUpdate?: boolean;
validator?: () => boolean;
}
export function useMemo<T>({ key, forceUpdate, validator }: MemoOption, updateFunc: (context: Map<string, any>, prev: T) => T): T {
const cached = _cached.get(key);
if (cached && !forceUpdate && (!validator || validator && !validator())) {
return cached.value;
}
const context = cached?.context || new Map<string, any>();
const value = updateFunc(context, cached?.value);
if (value !== cached?.value) {
_cached.set(key, { value, context });
}
return value;
}
export function disposeMemo(key: string) {
_cached.delete(key);
}
export function disposeAllMemo() {
_cached.clear();
}

794
src/features/CmdConfigSync.ts
View File

@@ -1,27 +1,29 @@
import { writable } from 'svelte/store'; import { writable } from 'svelte/store';
import { Notice, type PluginManifest, parseYaml, normalizePath, type ListedFiles } from "../deps.ts"; import { Notice, type PluginManifest, parseYaml, normalizePath, type ListedFiles, diff_match_patch } from "../deps.ts";
import type { EntryDoc, LoadedEntry, InternalFileEntry, FilePathWithPrefix, FilePath, DocumentID, AnyEntry, SavingEntry } from "../lib/src/common/types.ts"; import type { EntryDoc, LoadedEntry, InternalFileEntry, FilePathWithPrefix, FilePath, AnyEntry, SavingEntry, diff_result } from "../lib/src/common/types.ts";
import { LOG_LEVEL_INFO, LOG_LEVEL_NOTICE, LOG_LEVEL_VERBOSE, MODE_SELECTIVE } from "../lib/src/common/types.ts"; import { CANCELLED, LEAVE_TO_SUBSEQUENT, LOG_LEVEL_DEBUG, LOG_LEVEL_INFO, LOG_LEVEL_NOTICE, LOG_LEVEL_VERBOSE, MODE_SELECTIVE, MODE_SHINY } from "../lib/src/common/types.ts";
import { ICXHeader, PERIODIC_PLUGIN_SWEEP, } from "../common/types.ts"; import { ICXHeader, PERIODIC_PLUGIN_SWEEP, } from "../common/types.ts";
import { createSavingEntryFromLoadedEntry, createTextBlob, delay, fireAndForget, getDocDataAsArray, isDocContentSame } from "../lib/src/common/utils.ts"; import { createBlob, createSavingEntryFromLoadedEntry, createTextBlob, delay, fireAndForget, getDocData, getDocDataAsArray, isDocContentSame, isLoadedEntry, isObjectDifferent } from "../lib/src/common/utils.ts";
import { Logger } from "../lib/src/common/logger.ts"; import { Logger } from "../lib/src/common/logger.ts";
import { digestHash } from "../lib/src/string_and_binary/hash.ts"; import { digestHash } from "../lib/src/string_and_binary/hash.ts";
import { arrayBufferToBase64, decodeBinary, readString } from 'src/lib/src/string_and_binary/convert.ts'; import { arrayBufferToBase64, decodeBinary, readString } from 'src/lib/src/string_and_binary/convert.ts';
import { serialized, shareRunningResult } from "../lib/src/concurrency/lock.ts"; import { serialized, shareRunningResult } from "../lib/src/concurrency/lock.ts";
import { LiveSyncCommands } from "./LiveSyncCommands.ts"; import { LiveSyncCommands } from "./LiveSyncCommands.ts";
import { stripAllPrefixes } from "../lib/src/string_and_binary/path.ts"; import { stripAllPrefixes } from "../lib/src/string_and_binary/path.ts";
import { PeriodicProcessor, disposeMemoObject, memoIfNotExist, memoObject, retrieveMemoObject, scheduleTask } from "../common/utils.ts"; import { EVEN, PeriodicProcessor, disposeMemoObject, isMarkedAsSameChanges, markChangesAreSame, memoIfNotExist, memoObject, retrieveMemoObject, scheduleTask } from "../common/utils.ts";
import { PluginDialogModal } from "../common/dialogs.ts"; import { PluginDialogModal } from "../common/dialogs.ts";
import { JsonResolveModal } from "../ui/JsonResolveModal.ts"; import { JsonResolveModal } from "../ui/JsonResolveModal.ts";
import { QueueProcessor } from '../lib/src/concurrency/processor.ts'; import { QueueProcessor } from '../lib/src/concurrency/processor.ts';
import { pluginScanningCount } from '../lib/src/mock_and_interop/stores.ts'; import { pluginScanningCount } from '../lib/src/mock_and_interop/stores.ts';
import type ObsidianLiveSyncPlugin from '../main.ts'; import type ObsidianLiveSyncPlugin from '../main.ts';
import { base64ToArrayBuffer, base64ToString } from 'octagonal-wheels/binary/base64';
import { ConflictResolveModal } from '../ui/ConflictResolveModal.ts';
import { Semaphore } from 'octagonal-wheels/concurrency/semaphore';
const d = "\u200b"; const d = "\u200b";
const d2 = "\n"; const d2 = "\n";
function serialize(data: PluginDataEx): string { function serialize(data: PluginDataEx): string {
// For higher performance, create custom plug-in data strings. // For higher performance, create custom plug-in data strings.
// Self-hosted LiveSync uses `\n` to split chunks. Therefore, grouping together those with similar entropy would work nicely. // Self-hosted LiveSync uses `\n` to split chunks. Therefore, grouping together those with similar entropy would work nicely.
@@ -41,7 +43,15 @@ function serialize(data: PluginDataEx): string {
} }
return ret; return ret;
} }
const DUMMY_HEAD = serialize({
category: "CONFIG",
name: "migrated",
files: [],
mtime: 0,
term: "-",
displayName: `MIRAGED`
});
const DUMMY_END = d + d2 + "\u200c";
function splitWithDelimiters(sources: string[]): string[] { function splitWithDelimiters(sources: string[]): string[] {
const result: string[] = []; const result: string[] = [];
for (const str of sources) { for (const str of sources) {
@@ -186,6 +196,7 @@ function deserialize<T>(str: string[], def: T) {
export const pluginList = writable([] as PluginDataExDisplay[]); export const pluginList = writable([] as PluginDataExDisplay[]);
export const pluginIsEnumerating = writable(false); export const pluginIsEnumerating = writable(false);
export const pluginV2Progress = writable(0);
export type PluginDataExFile = { export type PluginDataExFile = {
filename: string, filename: string,
@@ -196,6 +207,16 @@ export type PluginDataExFile = {
hash?: string, hash?: string,
displayName?: string, displayName?: string,
} }
export interface IPluginDataExDisplay {
documentPath: FilePathWithPrefix;
category: string;
name: string;
term: string;
displayName?: string;
files: (LoadedEntryPluginDataExFile | PluginDataExFile)[];
version?: string;
mtime: number;
}
export type PluginDataExDisplay = { export type PluginDataExDisplay = {
documentPath: FilePathWithPrefix, documentPath: FilePathWithPrefix,
category: string, category: string,
@@ -206,6 +227,90 @@ export type PluginDataExDisplay = {
version?: string, version?: string,
mtime: number, mtime: number,
} }
type LoadedEntryPluginDataExFile = LoadedEntry & PluginDataExFile;
function categoryToFolder(category: string, configDir: string = ""): string {
switch (category) {
case "CONFIG": return `${configDir}/`;
case "THEME": return `${configDir}/themes/`;
case "SNIPPET": return `${configDir}/snippets/`;
case "PLUGIN_MAIN": return `${configDir}/plugins/`;
case "PLUGIN_DATA": return `${configDir}/plugins/`;
case "PLUGIN_ETC": return `${configDir}/plugins/`;
default: return "";
}
}
export const pluginManifests = new Map<string, PluginManifest>();
export const pluginManifestStore = writable(pluginManifests);
function setManifest(key: string, manifest: PluginManifest) {
const old = pluginManifests.get(key);
if (old && !isObjectDifferent(manifest, old)) {
return;
}
pluginManifests.set(key, manifest);
pluginManifestStore.set(pluginManifests);
}
export class PluginDataExDisplayV2 {
documentPath: FilePathWithPrefix;
category: string;
term: string;
files = [] as LoadedEntryPluginDataExFile[];
name: string;
confKey: string;
constructor(data: IPluginDataExDisplay) {
this.documentPath = `${data.documentPath}` as FilePathWithPrefix;
this.category = `${data.category}`;
this.name = `${data.name}`;
this.term = `${data.term}`;
this.files = [...data.files as LoadedEntryPluginDataExFile[]];
this.confKey = `${categoryToFolder(this.category, this.term)}${this.name}`;
this.applyLoadedManifest();
}
async setFile(file: LoadedEntryPluginDataExFile) {
const old = this.files.find(e => e.filename == file.filename);
if (old) {
if (old.mtime == file.mtime && await isDocContentSame(old.data, file.data)) return;
this.files = this.files.filter(e => e.filename != file.filename);
}
this.files.push(file);
if (file.filename == "manifest.json") {
this.applyLoadedManifest();
}
}
deleteFile(filename: string) {
this.files = this.files.filter(e => e.filename != filename);
}
_displayName: string | undefined;
_version: string | undefined;
applyLoadedManifest() {
const manifest = pluginManifests.get(this.confKey);
if (manifest) {
this._displayName = manifest.name;
if (this.category == "PLUGIN_MAIN" || this.category == "THEME") {
this._version = manifest?.version;
}
}
}
get displayName(): string {
// if (this._displayNameBuffer !== symbolUnInitialised) return this._displayNameBuffer;
// return this._bufferManifest().displayName;
return this._displayName || this.name;
}
get version(): string | undefined {
return this._version;
}
get mtime(): number {
return ~~this.files.reduce((a, b) => a + b.mtime, 0) / this.files.length;
}
}
export type PluginDataEx = { export type PluginDataEx = {
documentPath?: FilePathWithPrefix, documentPath?: FilePathWithPrefix,
category: string, category: string,
@@ -216,25 +321,30 @@ export type PluginDataEx = {
version?: string, version?: string,
mtime: number, mtime: number,
}; };
export class ConfigSync extends LiveSyncCommands { export class ConfigSync extends LiveSyncCommands {
constructor(plugin: ObsidianLiveSyncPlugin) { constructor(plugin: ObsidianLiveSyncPlugin) {
super(plugin); super(plugin);
pluginScanningCount.onChanged((e) => { pluginScanningCount.onChanged((e) => {
const total = e.value; const total = e.value;
pluginIsEnumerating.set(total != 0); pluginIsEnumerating.set(total != 0);
// if (total == 0) {
// Logger(`Processing configurations done`, LOG_LEVEL_INFO, "get-plugins");
// }
}) })
} }
get kvDB() { get kvDB() {
return this.plugin.kvDB; return this.plugin.kvDB;
} }
get useV2() {
return this.plugin.settings.usePluginSyncV2;
}
get useSyncPluginEtc() {
return this.plugin.settings.usePluginEtc;
}
pluginDialog?: PluginDialogModal = undefined; pluginDialog?: PluginDialogModal = undefined;
periodicPluginSweepProcessor = new PeriodicProcessor(this.plugin, async () => await this.scanAllConfigFiles(false)); periodicPluginSweepProcessor = new PeriodicProcessor(this.plugin, async () => await this.scanAllConfigFiles(false));
pluginList: PluginDataExDisplay[] = []; pluginList: IPluginDataExDisplay[] = [];
showPluginSyncModal() { showPluginSyncModal() {
if (!this.settings.usePluginSync) { if (!this.settings.usePluginSync) {
return; return;
@@ -277,10 +387,8 @@ export class ConfigSync extends LiveSyncCommands {
} else if (filePath.endsWith("/data.json")) { } else if (filePath.endsWith("/data.json")) {
return "PLUGIN_DATA"; return "PLUGIN_DATA";
} else { } else {
//TODO: to be configurable. // Planned at v0.19.0, realised v0.23.18!
// With algorithm which implemented at v0.19.0, is too heavy. return (this.useV2 && this.useSyncPluginEtc) ? "PLUGIN_ETC" : "";
return "";
// return "PLUGIN_ETC";
} }
// return "PLUGIN"; // return "PLUGIN";
} }
@@ -321,6 +429,7 @@ export class ConfigSync extends LiveSyncCommands {
} }
async reloadPluginList(showMessage: boolean) { async reloadPluginList(showMessage: boolean) {
this.pluginList = []; this.pluginList = [];
this.loadedManifest_mTime.clear();
pluginList.set(this.pluginList) pluginList.set(this.pluginList)
await this.updatePluginList(showMessage); await this.updatePluginList(showMessage);
} }
@@ -355,30 +464,36 @@ export class ConfigSync extends LiveSyncCommands {
} }
return false; return false;
} }
async createMissingConfigurationEntry() {
let saveRequired = false;
for (const v of this.pluginList) {
const key = `${v.category}/${v.name}`;
if (!(key in this.plugin.settings.pluginSyncExtendedSetting)) {
this.plugin.settings.pluginSyncExtendedSetting[key] = {
key,
mode: MODE_SELECTIVE,
files: []
}
}
if (this.plugin.settings.pluginSyncExtendedSetting[key].files.sort().join(",").toLowerCase() !=
v.files.map(e => e.filename).sort().join(",").toLowerCase()) {
this.plugin.settings.pluginSyncExtendedSetting[key].files = v.files.map(e => e.filename).sort();
saveRequired = true;
}
}
if (saveRequired) {
await this.plugin.saveSettingData();
}
}
pluginScanProcessor = new QueueProcessor(async (v: AnyEntry[]) => { pluginScanProcessor = new QueueProcessor(async (v: AnyEntry[]) => {
const plugin = v[0];
if (this.useV2) {
await this.migrateV1ToV2(false, plugin);
return [];
}
const path = plugin.path || this.getPath(plugin);
const oldEntry = (this.pluginList.find(e => e.documentPath == path));
if (oldEntry && oldEntry.mtime == plugin.mtime) return [];
try {
const pluginData = await this.loadPluginData(path);
if (pluginData) {
let newList = [...this.pluginList];
newList = newList.filter(x => x.documentPath != pluginData.documentPath);
newList.push(pluginData);
this.pluginList = newList;
pluginList.set(newList);
}
// Failed to load
return [];
} catch (ex) {
Logger(`Something happened at enumerating customization :${path}`, LOG_LEVEL_NOTICE);
Logger(ex, LOG_LEVEL_VERBOSE);
}
return [];
}, { suspended: false, batchSize: 1, concurrentLimit: 10, delay: 100, yieldThreshold: 10, maintainDelay: false, totalRemainingReactiveSource: pluginScanningCount }).startPipeline();
pluginScanProcessorV2 = new QueueProcessor(async (v: AnyEntry[]) => {
const plugin = v[0]; const plugin = v[0];
const path = plugin.path || this.getPath(plugin); const path = plugin.path || this.getPath(plugin);
const oldEntry = (this.pluginList.find(e => e.documentPath == path)); const oldEntry = (this.pluginList.find(e => e.documentPath == path));
@@ -400,17 +515,220 @@ export class ConfigSync extends LiveSyncCommands {
Logger(ex, LOG_LEVEL_VERBOSE); Logger(ex, LOG_LEVEL_VERBOSE);
} }
return []; return [];
}, { suspended: false, batchSize: 1, concurrentLimit: 10, delay: 100, yieldThreshold: 10, maintainDelay: false, totalRemainingReactiveSource: pluginScanningCount }).startPipeline().root.onUpdateProgress(() => { }, { suspended: false, batchSize: 1, concurrentLimit: 10, delay: 100, yieldThreshold: 10, maintainDelay: false, totalRemainingReactiveSource: pluginScanningCount }).startPipeline();
scheduleTask("checkMissingConfigurations", 250, async () => {
if (this.pluginScanProcessor.isIdle()) {
await this.createMissingConfigurationEntry();
}
});
});
filenameToUnifiedKey(path: string, termOverRide?: string) {
const term = termOverRide || this.plugin.deviceAndVaultName;
const category = this.getFileCategory(path);
const name = (category == "CONFIG" || category == "SNIPPET") ?
(path.split("/").slice(-1)[0]) :
(category == "PLUGIN_ETC" ?
path.split("/").slice(-2).join("/") :
path.split("/").slice(-2)[0]);
return `${ICXHeader}${term}/${category}/${name}.md` as FilePathWithPrefix
}
filenameWithUnifiedKey(path: string, termOverRide?: string) {
const term = termOverRide || this.plugin.deviceAndVaultName;
const category = this.getFileCategory(path);
const name = (category == "CONFIG" || category == "SNIPPET") ?
(path.split("/").slice(-1)[0]) : path.split("/").slice(-2)[0];
const baseName = category == "CONFIG" || category == "SNIPPET" ? name : path.split("/").slice(3).join("/");
return `${ICXHeader}${term}/${category}/${name}%${baseName}` as FilePathWithPrefix;
}
unifiedKeyPrefixOfTerminal(termOverRide?: string) {
const term = termOverRide || this.plugin.deviceAndVaultName;
return `${ICXHeader}${term}/` as FilePathWithPrefix;
}
parseUnifiedPath(unifiedPath: FilePathWithPrefix): { category: string, device: string, key: string, filename: string, pathV1: FilePathWithPrefix } {
const [device, category, ...rest] = stripAllPrefixes(unifiedPath).split("/");
const relativePath = rest.join("/");
const [key, filename] = relativePath.split("%");
const pathV1 = (unifiedPath.split("%")[0] + ".md") as FilePathWithPrefix;
return { device, category, key, filename, pathV1 };
}
loadedManifest_mTime = new Map<string, number>();
async createPluginDataExFileV2(unifiedPathV2: FilePathWithPrefix, loaded?: LoadedEntry): Promise<false | LoadedEntryPluginDataExFile> {
const { category, key, filename, device } = this.parseUnifiedPath(unifiedPathV2);
if (!loaded) {
const d = await this.localDatabase.getDBEntry(unifiedPathV2);
if (!d) {
Logger(`The file ${unifiedPathV2} is not found`, LOG_LEVEL_VERBOSE);
return false;
}
if (!isLoadedEntry(d)) {
Logger(`The file ${unifiedPathV2} is not a note`, LOG_LEVEL_VERBOSE);
return false;
}
loaded = d;
}
const confKey = `${categoryToFolder(category, device)}${key}`;
const relativeFilename = `${categoryToFolder(category, "")}${(category == "CONFIG" || category == "SNIPPET") ? "" : (key + "/")}${filename}`.substring(1);
const dataSrc = getDocData(loaded.data);
const dataStart = dataSrc.indexOf(DUMMY_END);
const data = dataSrc.substring(dataStart + DUMMY_END.length);
const file: LoadedEntryPluginDataExFile = {
...loaded,
hash: "",
data: [base64ToString(data)],
filename: relativeFilename,
displayName: filename,
};
if (filename == "manifest.json") {
// Same as previously loaded
if (this.loadedManifest_mTime.get(confKey) != file.mtime && pluginManifests.get(confKey) == undefined) {
try {
const parsedManifest = JSON.parse(base64ToString(data)) as PluginManifest;
setManifest(confKey, parsedManifest);
this.pluginList.filter(e => e instanceof PluginDataExDisplayV2 && e.confKey == confKey).forEach(e => (e as PluginDataExDisplayV2).applyLoadedManifest());
pluginList.set(this.pluginList);
} catch (ex) {
Logger(`The file ${loaded.path} seems to manifest, but could not be decoded as JSON`, LOG_LEVEL_VERBOSE);
Logger(ex, LOG_LEVEL_VERBOSE);
}
this.loadedManifest_mTime.set(confKey, file.mtime);
} else {
this.pluginList.filter(e => e instanceof PluginDataExDisplayV2 && e.confKey == confKey).forEach(e => (e as PluginDataExDisplayV2).applyLoadedManifest());
pluginList.set(this.pluginList);
}
// }
}
return file;
}
createPluginDataFromV2(unifiedPathV2: FilePathWithPrefix) {
const { category, device, key, pathV1 } = this.parseUnifiedPath(unifiedPathV2);
if (category == "") return;
const ret: PluginDataExDisplayV2 = new PluginDataExDisplayV2({
documentPath: pathV1,
category: category,
name: key,
term: `${device}`,
files: [],
mtime: 0,
});
return ret;
}
updatingV2Count = 0;
async updatePluginListV2(showMessage: boolean, unifiedFilenameWithKey: FilePathWithPrefix): Promise<void> {
try {
this.updatingV2Count++;
pluginV2Progress.set(this.updatingV2Count);
// const unifiedFilenameWithKey = this.filenameWithUnifiedKey(updatedDocumentPath);
const { pathV1 } = this.parseUnifiedPath(unifiedFilenameWithKey);
const oldEntry = this.pluginList.find(e => e.documentPath == pathV1);
let entry: PluginDataExDisplayV2 | undefined = undefined;
if (!oldEntry || !(oldEntry instanceof PluginDataExDisplayV2)) {
const newEntry = this.createPluginDataFromV2(unifiedFilenameWithKey);
if (newEntry) {
entry = newEntry;
}
} else if (oldEntry instanceof PluginDataExDisplayV2) {
entry = oldEntry;
}
if (!entry) return;
const file = await this.createPluginDataExFileV2(unifiedFilenameWithKey);
if (file) {
await entry.setFile(file);
} else {
entry.deleteFile(unifiedFilenameWithKey);
if (entry.files.length == 0) {
this.pluginList = this.pluginList.filter(e => e.documentPath != pathV1);
}
}
const newList = this.pluginList.filter(e => e.documentPath != entry.documentPath);
newList.push(entry);
this.pluginList = newList;
scheduleTask("updatePluginListV2", 100, () => {
pluginList.set(this.pluginList);
});
} finally {
this.updatingV2Count--;
pluginV2Progress.set(this.updatingV2Count);
}
}
async migrateV1ToV2(showMessage: boolean, entry: AnyEntry): Promise<void> {
const v1Path = entry.path;
Logger(`Migrating ${entry.path} to V2`, showMessage ? LOG_LEVEL_NOTICE : LOG_LEVEL_INFO);
if (entry.deleted) {
Logger(`The entry ${v1Path} is already deleted`, LOG_LEVEL_VERBOSE);
return;
}
if (!v1Path.endsWith(".md") && !v1Path.startsWith(ICXHeader)) {
Logger(`The entry ${v1Path} is not a customisation sync binder`, LOG_LEVEL_VERBOSE);
return
}
if (v1Path.indexOf("%") !== -1) {
Logger(`The entry ${v1Path} is already migrated`, LOG_LEVEL_VERBOSE);
return;
}
const loadedEntry = await this.localDatabase.getDBEntry(v1Path);
if (!loadedEntry) {
Logger(`The entry ${v1Path} is not found`, LOG_LEVEL_VERBOSE);
return;
}
const pluginData = deserialize(getDocDataAsArray(loadedEntry.data), {}) as PluginDataEx;
const prefixPath = v1Path.slice(0, -(".md".length)) + "%";
const category = pluginData.category;
for (const f of pluginData.files) {
const stripTable: Record<string, number> = {
"CONFIG": 0,
"THEME": 2,
"SNIPPET": 1,
"PLUGIN_MAIN": 2,
"PLUGIN_DATA": 2,
"PLUGIN_ETC": 2,
}
const deletePrefixCount = stripTable?.[category] ?? 1;
const relativeFilename = f.filename.split("/").slice(deletePrefixCount).join("/");
const v2Path = (prefixPath + relativeFilename) as FilePathWithPrefix;
// console.warn(`Migrating ${v1Path} / ${relativeFilename} to ${v2Path}`);
Logger(`Migrating ${v1Path} / ${relativeFilename} to ${v2Path}`, LOG_LEVEL_VERBOSE);
const newId = await this.plugin.path2id(v2Path);
// const buf =
const data = createBlob([DUMMY_HEAD, DUMMY_END, ...getDocDataAsArray(f.data)]);
const saving: SavingEntry = {
...loadedEntry,
_rev: undefined,
_id: newId,
path: v2Path,
data: data,
datatype: "plain",
type: "plain",
children: [],
eden: {}
}
const r = await this.plugin.localDatabase.putDBEntry(saving);
if (r && r.ok) {
Logger(`Migrated ${v1Path} / ${f.filename} to ${v2Path}`, LOG_LEVEL_INFO);
const delR = await this.deleteConfigOnDatabase(v1Path);
if (delR) {
Logger(`Deleted ${v1Path} successfully`, LOG_LEVEL_INFO);
} else {
Logger(`Failed to delete ${v1Path}`, LOG_LEVEL_NOTICE);
}
}
}
}
async updatePluginList(showMessage: boolean, updatedDocumentPath?: FilePathWithPrefix): Promise<void> { async updatePluginList(showMessage: boolean, updatedDocumentPath?: FilePathWithPrefix): Promise<void> {
// pluginList.set([]);
if (!this.settings.usePluginSync) { if (!this.settings.usePluginSync) {
this.pluginScanProcessor.clearQueue(); this.pluginScanProcessor.clearQueue();
this.pluginList = []; this.pluginList = [];
@@ -418,60 +736,173 @@ export class ConfigSync extends LiveSyncCommands {
return; return;
} }
try { try {
this.updatingV2Count++;
pluginV2Progress.set(this.updatingV2Count);
const updatedDocumentId = updatedDocumentPath ? await this.path2id(updatedDocumentPath) : ""; const updatedDocumentId = updatedDocumentPath ? await this.path2id(updatedDocumentPath) : "";
const plugins = updatedDocumentPath ? const plugins = updatedDocumentPath ?
this.localDatabase.findEntries(updatedDocumentId, updatedDocumentId + "\u{10ffff}", { include_docs: true, key: updatedDocumentId, limit: 1 }) : this.localDatabase.findEntries(updatedDocumentId, updatedDocumentId + "\u{10ffff}", { include_docs: true, key: updatedDocumentId, limit: 1 }) :
this.localDatabase.findEntries(ICXHeader + "", `${ICXHeader}\u{10ffff}`, { include_docs: true }); this.localDatabase.findEntries(ICXHeader + "", `${ICXHeader}\u{10ffff}`, { include_docs: true });
for await (const v of plugins) { for await (const v of plugins) {
if (v.deleted || v._deleted) continue;
if (v.path.indexOf("%") !== -1) {
fireAndForget(() => this.updatePluginListV2(showMessage, v.path));
continue;
}
const path = v.path || this.getPath(v); const path = v.path || this.getPath(v);
if (updatedDocumentPath && updatedDocumentPath != path) continue; if (updatedDocumentPath && updatedDocumentPath != path) continue;
this.pluginScanProcessor.enqueue(v); this.pluginScanProcessor.enqueue(v);
} }
} finally { } finally {
pluginIsEnumerating.set(false); pluginIsEnumerating.set(false);
this.updatingV2Count--;
pluginV2Progress.set(this.updatingV2Count);
} }
pluginIsEnumerating.set(false); pluginIsEnumerating.set(false);
// return entries; // return entries;
} }
async compareUsingDisplayData(dataA: PluginDataExDisplay, dataB: PluginDataExDisplay) { async compareUsingDisplayData(dataA: IPluginDataExDisplay, dataB: IPluginDataExDisplay, compareEach = false) {
const docA = await this.localDatabase.getDBEntry(dataA.documentPath); const loadFile = async (data: IPluginDataExDisplay) => {
const docB = await this.localDatabase.getDBEntry(dataB.documentPath); if (data instanceof PluginDataExDisplayV2 || compareEach) {
return data.files[0] as LoadedEntryPluginDataExFile;
if (docA && docB) { }
const pluginDataA = deserialize(getDocDataAsArray(docA.data), {}) as PluginDataEx; const loadDoc = await this.localDatabase.getDBEntry(data.documentPath);
pluginDataA.documentPath = dataA.documentPath; if (!loadDoc) return false;
const pluginDataB = deserialize(getDocDataAsArray(docB.data), {}) as PluginDataEx; const pluginData = deserialize(getDocDataAsArray(loadDoc.data), {}) as PluginDataEx;
pluginDataB.documentPath = dataB.documentPath; pluginData.documentPath = data.documentPath;
const file = pluginData.files[0];
// Use outer structure to wrap each data. const doc = { ...loadDoc, ...file, datatype: "newnote" } as LoadedEntryPluginDataExFile;
return await this.showJSONMergeDialogAndMerge(docA, docB, pluginDataA, pluginDataB); return doc;
}
const fileA = await loadFile(dataA);
const fileB = await loadFile(dataB);
Logger(`Comparing: ${dataA.documentPath} <-> ${dataB.documentPath}`, LOG_LEVEL_VERBOSE);
if (!fileA || !fileB) {
Logger(`Could not load ${dataA.name} for comparison: ${!fileA ? dataA.term : ""}${!fileB ? dataB.term : ""}`, LOG_LEVEL_NOTICE);
return false;
}
let path = stripAllPrefixes(fileA.path.split("/").slice(-1).join("/") as FilePath); // TODO:adjust
if (path.indexOf("%") !== -1) {
path = path.split("%")[1] as FilePath;
}
if (fileA.path.endsWith(".json")) {
return serialized("config:merge-data", () => new Promise<boolean>((res) => {
Logger("Opening data-merging dialog", LOG_LEVEL_VERBOSE);
// const docs = [docA, docB];
const modal = new JsonResolveModal(this.app, path, [fileA, fileB], async (keep, result) => {
if (result == null) return res(false);
try {
res(await this.applyData(dataA, result));
} catch (ex) {
Logger("Could not apply merged file");
Logger(ex, LOG_LEVEL_VERBOSE);
res(false);
}
}, "Local", `${dataB.term}`, "B", true, true, "Difference between local and remote");
modal.open();
}));
} else {
const dmp = new diff_match_patch();
let docAData = getDocData(fileA.data);
let docBData = getDocData(fileB.data);
if (fileA?.datatype != "plain") {
docAData = base64ToString(docAData);
}
if (fileB?.datatype != "plain") {
docBData = base64ToString(docBData);
}
const diffMap = dmp.diff_linesToChars_(docAData, docBData);
const diff = dmp.diff_main(diffMap.chars1, diffMap.chars2, false);
dmp.diff_charsToLines_(diff, diffMap.lineArray);
dmp.diff_cleanupSemantic(diff);
const diffResult: diff_result = {
left: { rev: "A", ...fileA, data: docAData },
right: { rev: "B", ...fileB, data: docBData },
diff: diff
}
console.dir(diffResult);
const d = new ConflictResolveModal(this.app, path, diffResult, true, dataB.term);
d.open();
const ret = await d.waitForResult();
if (ret === CANCELLED) return false;
if (ret === LEAVE_TO_SUBSEQUENT) return false;
const resultContent = ret == "A" ? docAData : ret == "B" ? docBData : undefined;
if (resultContent) {
return await this.applyData(dataA, resultContent);
}
return false;
} }
return false;
} }
showJSONMergeDialogAndMerge(docA: LoadedEntry, docB: LoadedEntry, pluginDataA: PluginDataEx, pluginDataB: PluginDataEx): Promise<boolean> { async applyDataV2(data: PluginDataExDisplayV2, content?: string): Promise<boolean> {
const fileA = { ...pluginDataA.files[0], ctime: pluginDataA.files[0].mtime, _id: `${pluginDataA.documentPath}` as DocumentID }; const baseDir = this.app.vault.configDir;
const fileB = pluginDataB.files[0]; try {
const docAx = { ...docA, ...fileA, datatype: "newnote" } as LoadedEntry, docBx = { ...docB, ...fileB, datatype: "newnote" } as LoadedEntry if (content) {
return serialized("config:merge-data", () => new Promise((res) => { // const dt = createBlob(content);
Logger("Opening data-merging dialog", LOG_LEVEL_VERBOSE); const filename = data.files[0].filename;
// const docs = [docA, docB]; Logger(`Applying ${filename} of ${data.displayName || data.name}..`);
const path = stripAllPrefixes(docAx.path.split("/").slice(-1).join("/") as FilePath); const path = `${baseDir}/${filename}` as FilePath;
const modal = new JsonResolveModal(this.app, path, [docAx, docBx], async (keep, result) => { await this.vaultAccess.ensureDirectory(path);
if (result == null) return res(false); // If the content has applied, modified time will be updated to the current time.
try { await this.vaultAccess.adapterWrite(path, content);
res(await this.applyData(pluginDataA, result)); await this.storeCustomisationFileV2(path, this.plugin.deviceAndVaultName);
} catch (ex) {
Logger("Could not apply merged file"); } else {
Logger(ex, LOG_LEVEL_VERBOSE); const files = data.files;
res(false); for (const f of files) {
// If files have applied, modified time will be updated to the current time.
const stat = { mtime: f.mtime, ctime: f.ctime };
const path = `${baseDir}/${f.filename}` as FilePath;
Logger(`Applying ${f.filename} of ${data.displayName || data.name}..`);
// const contentEach = createBlob(f.data);
this.vaultAccess.ensureDirectory(path);
if (f.datatype == "newnote") {
let oldData;
try {
oldData = await this.vaultAccess.adapterReadBinary(path);
} catch (ex) {
oldData = new ArrayBuffer(0);
}
const content = base64ToArrayBuffer(f.data);
if (await isDocContentSame(oldData, content)) {
Logger(`The file ${f.filename} is already up-to-date`, LOG_LEVEL_VERBOSE);
continue;
}
await this.vaultAccess.adapterWrite(path, content, stat);
} else {
let oldData;
try {
oldData = await this.vaultAccess.adapterRead(path);
} catch (ex) {
oldData = "";
}
const content = getDocData(f.data);
if (await isDocContentSame(oldData, content)) {
Logger(`The file ${f.filename} is already up-to-date`, LOG_LEVEL_VERBOSE);
continue;
}
await this.vaultAccess.adapterWrite(path, content, stat);
}
Logger(`Applied ${f.filename} of ${data.displayName || data.name}..`);
await this.storeCustomisationFileV2(path, this.plugin.deviceAndVaultName);
} }
}, "📡", "🛰️", "B"); }
modal.open(); } catch (ex) {
})); Logger(`Applying ${data.displayName || data.name}.. Failed`, LOG_LEVEL_NOTICE);
Logger(ex, LOG_LEVEL_VERBOSE);
return false;
}
return true;
} }
async applyData(data: PluginDataEx, content?: string): Promise<boolean> { async applyData(data: IPluginDataExDisplay, content?: string): Promise<boolean> {
Logger(`Applying ${data.displayName || data.name}..`); Logger(`Applying ${data.displayName || data.name
}..`);
if (data instanceof PluginDataExDisplayV2) {
return this.applyDataV2(data, content);
}
const baseDir = this.app.vault.configDir; const baseDir = this.app.vault.configDir;
try { try {
if (!data.documentPath) throw "InternalError: Document path not exist"; if (!data.documentPath) throw "InternalError: Document path not exist";
@@ -532,9 +963,22 @@ export class ConfigSync extends LiveSyncCommands {
async deleteData(data: PluginDataEx): Promise<boolean> { async deleteData(data: PluginDataEx): Promise<boolean> {
try { try {
if (data.documentPath) { if (data.documentPath) {
await this.deleteConfigOnDatabase(data.documentPath); const delList = [];
await this.updatePluginList(false, data.documentPath); if (this.useV2) {
Logger(`Delete: ${data.documentPath}`, LOG_LEVEL_NOTICE); const deleteList = this.pluginList.filter(e => e.documentPath == data.documentPath).filter(e => e instanceof PluginDataExDisplayV2).map(e => e.files).flat();
for (const e of deleteList) {
delList.push(e.path);
}
}
delList.push(data.documentPath);
const p = delList.map(async e => {
await this.deleteConfigOnDatabase(e);
await this.updatePluginList(false, e)
});
await Promise.allSettled(p);
// await this.deleteConfigOnDatabase(data.documentPath);
// await this.updatePluginList(false, data.documentPath);
Logger(`Deleted: ${data.category}/${data.name} of ${data.category} (${delList.length} items)`, LOG_LEVEL_NOTICE);
} }
return true; return true;
} catch (ex) { } catch (ex) {
@@ -645,15 +1089,78 @@ export class ConfigSync extends LiveSyncCommands {
} }
} }
filenameToUnifiedKey(path: string, termOverRide?: string) {
const term = termOverRide || this.plugin.deviceAndVaultName; async storeCustomisationFileV2(path: FilePath, term: string, force = false) {
const category = this.getFileCategory(path); const vf = this.filenameWithUnifiedKey(path, term);
const name = (category == "CONFIG" || category == "SNIPPET") ? return await serialized(`plugin-${vf}`, async () => {
(path.split("/").slice(-1)[0]) : const prefixedFileName = vf;
(category == "PLUGIN_ETC" ?
path.split("/").slice(-2).join("/") : const id = await this.path2id(prefixedFileName);
path.split("/").slice(-2)[0]); const stat = await this.vaultAccess.adapterStat(path);
return `${ICXHeader}${term}/${category}/${name}.md` as FilePathWithPrefix if (!stat) {
return false;
}
const mtime = stat.mtime;
const content = await this.vaultAccess.adapterReadBinary(path);
const contentBlob = createBlob([DUMMY_HEAD, DUMMY_END, ...await arrayBufferToBase64(content)]);
// const contentBlob = createBlob(content);
try {
const old = await this.localDatabase.getDBEntryMeta(prefixedFileName, undefined, false);
let saveData: SavingEntry;
if (old === false) {
saveData = {
_id: id,
path: prefixedFileName,
data: contentBlob,
mtime,
ctime: mtime,
datatype: "plain",
size: contentBlob.size,
children: [],
deleted: false,
type: "plain",
eden: {}
};
} else {
if (isMarkedAsSameChanges(prefixedFileName, [old.mtime, mtime + 1]) == EVEN) {
Logger(`STORAGE --> DB:${prefixedFileName}: (config) Skipped (Already checked the same)`, LOG_LEVEL_DEBUG);
return;
}
const docXDoc = await this.localDatabase.getDBEntryFromMeta(old, {}, false, false);
if (docXDoc == false) {
throw "Could not load the document";
}
const dataSrc = getDocData(docXDoc.data);
const dataStart = dataSrc.indexOf(DUMMY_END);
const oldContent = dataSrc.substring(dataStart + DUMMY_END.length);
const oldContentArray = base64ToArrayBuffer(oldContent);
if (await isDocContentSame(oldContentArray, content)) {
Logger(`STORAGE --> DB:${prefixedFileName}: (config) Skipped (the same content)`, LOG_LEVEL_VERBOSE);
markChangesAreSame(prefixedFileName, old.mtime, mtime + 1);
return true;
}
saveData =
{
...old,
data: contentBlob,
mtime,
size: contentBlob.size,
datatype: "plain",
children: [],
deleted: false,
type: "plain",
};
}
const ret = await this.localDatabase.putDBEntry(saveData);
Logger(`STORAGE --> DB:${prefixedFileName}: (config) Done`);
fireAndForget(() => this.updatePluginListV2(false, this.filenameWithUnifiedKey(path)));
return ret;
} catch (ex) {
Logger(`STORAGE --> DB:${prefixedFileName}: (config) Failed`);
Logger(ex, LOG_LEVEL_VERBOSE);
return false;
}
})
} }
async storeCustomizationFiles(path: FilePath, termOverRide?: string) { async storeCustomizationFiles(path: FilePath, termOverRide?: string) {
const term = termOverRide || this.plugin.deviceAndVaultName; const term = termOverRide || this.plugin.deviceAndVaultName;
@@ -661,7 +1168,13 @@ export class ConfigSync extends LiveSyncCommands {
Logger("We have to configure the device name", LOG_LEVEL_NOTICE); Logger("We have to configure the device name", LOG_LEVEL_NOTICE);
return; return;
} }
if (this.useV2) {
return await this.storeCustomisationFileV2(path, term);
}
const vf = this.filenameToUnifiedKey(path, term); const vf = this.filenameToUnifiedKey(path, term);
// console.warn(`Storing ${path} to ${bareVF} :--> ${keyedVF}`);
return await serialized(`plugin-${vf}`, async () => { return await serialized(`plugin-${vf}`, async () => {
const category = this.getFileCategory(path); const category = this.getFileCategory(path);
let mtime = 0; let mtime = 0;
@@ -787,7 +1300,9 @@ export class ConfigSync extends LiveSyncCommands {
return false; return false;
const configDir = normalizePath(this.app.vault.configDir); const configDir = normalizePath(this.app.vault.configDir);
const synchronisedInConfigSync = Object.values(this.settings.pluginSyncExtendedSetting).filter(e => e.mode != MODE_SELECTIVE).map(e => e.files).flat().map(e => `${configDir}/${e}`.toLowerCase()); const synchronisedInConfigSync = Object.values(this.settings.pluginSyncExtendedSetting).filter(e =>
e.mode != MODE_SELECTIVE && e.mode != MODE_SHINY
).map(e => e.files).flat().map(e => `${configDir}/${e}`.toLowerCase());
if (synchronisedInConfigSync.some(e => e.startsWith(path.toLowerCase()))) { if (synchronisedInConfigSync.some(e => e.startsWith(path.toLowerCase()))) {
Logger(`Customization file skipped: ${path}`, LOG_LEVEL_VERBOSE); Logger(`Customization file skipped: ${path}`, LOG_LEVEL_VERBOSE);
return; return;
@@ -807,6 +1322,8 @@ export class ConfigSync extends LiveSyncCommands {
} }
async scanAllConfigFiles(showMessage: boolean) { async scanAllConfigFiles(showMessage: boolean) {
await shareRunningResult("scanAllConfigFiles", async () => { await shareRunningResult("scanAllConfigFiles", async () => {
const logLevel = showMessage ? LOG_LEVEL_NOTICE : LOG_LEVEL_INFO; const logLevel = showMessage ? LOG_LEVEL_NOTICE : LOG_LEVEL_INFO;
@@ -817,40 +1334,94 @@ export class ConfigSync extends LiveSyncCommands {
return; return;
} }
const filesAll = await this.scanInternalFiles(); const filesAll = await this.scanInternalFiles();
const files = filesAll.filter(e => this.isTargetPath(e)).map(e => ({ key: this.filenameToUnifiedKey(e), file: e })); if (this.useV2) {
const virtualPathsOfLocalFiles = [...new Set(files.map(e => e.key))]; const filesAllUnified = filesAll.filter(e => this.isTargetPath(e)).map(e => [this.filenameWithUnifiedKey(e, term), e] as [FilePathWithPrefix, FilePath]);
const filesOnDB = ((await this.localDatabase.allDocsRaw({ startkey: ICXHeader + "", endkey: `${ICXHeader}\u{10ffff}`, include_docs: true })).rows.map(e => e.doc) as InternalFileEntry[]).filter(e => !e.deleted); const localFileMap = new Map(filesAllUnified.map(e => [e[0], e[1]]));
let deleteCandidate = filesOnDB.map(e => this.getPath(e)).filter(e => e.startsWith(`${ICXHeader}${term}/`)); const prefix = this.unifiedKeyPrefixOfTerminal(term);
for (const vp of virtualPathsOfLocalFiles) { const entries = this.localDatabase.findEntries(prefix + "", `${prefix}\u{10ffff}`, { include_docs: true });
const p = files.find(e => e.key == vp)?.file; const tasks = [] as (() => Promise<void>)[];
if (!p) { const concurrency = 10;
Logger(`scanAllConfigFiles - File not found: ${vp}`, LOG_LEVEL_VERBOSE); const semaphore = Semaphore(concurrency);
continue; for await (const item of entries) {
if (item.path.indexOf("%") !== -1) {
continue;
}
tasks.push(async () => {
const releaser = await semaphore.acquire();
try {
const unifiedFilenameWithKey = `${item._id}` as FilePathWithPrefix;
const localPath = localFileMap.get(unifiedFilenameWithKey);
if (localPath) {
await this.storeCustomisationFileV2(localPath, term);
localFileMap.delete(unifiedFilenameWithKey);
} else {
await this.deleteConfigOnDatabase(unifiedFilenameWithKey);
}
} catch (ex) {
Logger(`scanAllConfigFiles - Error: ${item._id}`, LOG_LEVEL_VERBOSE);
Logger(ex, LOG_LEVEL_VERBOSE);
} finally {
releaser();
}
})
} }
await this.storeCustomizationFiles(p); await Promise.all(tasks.map(e => e()));
deleteCandidate = deleteCandidate.filter(e => e != vp); // Extra files
const taskExtra = [] as (() => Promise<void>)[];
for (const [, filePath] of localFileMap) {
taskExtra.push(async () => {
const releaser = await semaphore.acquire();
try {
await this.storeCustomisationFileV2(filePath, term);
} catch (ex) {
Logger(`scanAllConfigFiles - Error: ${filePath}`, LOG_LEVEL_VERBOSE);
Logger(ex, LOG_LEVEL_VERBOSE);
}
finally {
releaser();
}
})
}
await Promise.all(taskExtra.map(e => e()));
this.updatePluginList(false).then(/* fire and forget */);
} else {
const files = filesAll.filter(e => this.isTargetPath(e)).map(e => ({ key: this.filenameToUnifiedKey(e), file: e }));
const virtualPathsOfLocalFiles = [...new Set(files.map(e => e.key))];
const filesOnDB = ((await this.localDatabase.allDocsRaw({ startkey: ICXHeader + "", endkey: `${ICXHeader}\u{10ffff}`, include_docs: true })).rows.map(e => e.doc) as InternalFileEntry[]).filter(e => !e.deleted);
let deleteCandidate = filesOnDB.map(e => this.getPath(e)).filter(e => e.startsWith(`${ICXHeader}${term}/`));
for (const vp of virtualPathsOfLocalFiles) {
const p = files.find(e => e.key == vp)?.file;
if (!p) {
Logger(`scanAllConfigFiles - File not found: ${vp}`, LOG_LEVEL_VERBOSE);
continue;
}
await this.storeCustomizationFiles(p);
deleteCandidate = deleteCandidate.filter(e => e != vp);
}
for (const vp of deleteCandidate) {
await this.deleteConfigOnDatabase(vp);
}
this.updatePluginList(false).then(/* fire and forget */);
} }
for (const vp of deleteCandidate) {
await this.deleteConfigOnDatabase(vp);
}
this.updatePluginList(false).then(/* fire and forget */);
}); });
} }
async deleteConfigOnDatabase(prefixedFileName: FilePathWithPrefix, forceWrite = false) { async deleteConfigOnDatabase(prefixedFileName: FilePathWithPrefix, forceWrite = false) {
// const id = await this.path2id(prefixedFileName); // const id = await this.path2id(prefixedFileName);
const mtime = new Date().getTime(); const mtime = new Date().getTime();
await serialized("file-x-" + prefixedFileName, async () => { return await serialized("file-x-" + prefixedFileName, async () => {
try { try {
const old = await this.localDatabase.getDBEntryMeta(prefixedFileName, undefined, false) as InternalFileEntry | false; const old = await this.localDatabase.getDBEntryMeta(prefixedFileName, undefined, false) as InternalFileEntry | false;
let saveData: InternalFileEntry; let saveData: InternalFileEntry;
if (old === false) { if (old === false) {
Logger(`STORAGE -x> DB:${prefixedFileName}: (config) already deleted (Not found on database)`); Logger(`STORAGE -x> DB:${prefixedFileName}: (config) already deleted (Not found on database)`);
return; return true;
} else { } else {
if (old.deleted) { if (old.deleted) {
Logger(`STORAGE -x> DB:${prefixedFileName}: (config) already deleted`); Logger(`STORAGE -x> DB:${prefixedFileName}: (config) already deleted`);
return; return true;
} }
saveData = saveData =
{ {
@@ -865,6 +1436,7 @@ export class ConfigSync extends LiveSyncCommands {
await this.localDatabase.putRaw(saveData); await this.localDatabase.putRaw(saveData);
await this.updatePluginList(false, prefixedFileName); await this.updatePluginList(false, prefixedFileName);
Logger(`STORAGE -x> DB:${prefixedFileName}: (config) Done`); Logger(`STORAGE -x> DB:${prefixedFileName}: (config) Done`);
return true;
} catch (ex) { } catch (ex) {
Logger(`STORAGE -x> DB:${prefixedFileName}: (config) Failed`); Logger(`STORAGE -x> DB:${prefixedFileName}: (config) Failed`);
Logger(ex, LOG_LEVEL_VERBOSE); Logger(ex, LOG_LEVEL_VERBOSE);

3
src/features/CmdSetupLiveSync.ts
View File

@@ -374,7 +374,7 @@ Of course, we are able to disable these features.`
await this.plugin.openDatabase(); await this.plugin.openDatabase();
this.plugin.isReady = true; this.plugin.isReady = true;
if (makeLocalChunkBeforeSync) { if (makeLocalChunkBeforeSync) {
await this.plugin.initializeDatabase(true); await this.plugin.createAllChunks(true);
} }
await this.plugin.markRemoteResolved(); await this.plugin.markRemoteResolved();
await delay(500); await delay(500);
@@ -390,6 +390,7 @@ Of course, we are able to disable these features.`
async rebuildRemote() { async rebuildRemote() {
this.suspendExtraSync(); this.suspendExtraSync();
this.plugin.settings.isConfigured = true; this.plugin.settings.isConfigured = true;
await this.plugin.realizeSettingSyncMode(); await this.plugin.realizeSettingSyncMode();
await this.plugin.markRemoteLocked(); await this.plugin.markRemoteLocked();
await this.plugin.tryResetRemoteDatabase(); await this.plugin.tryResetRemoteDatabase();

234
src/features/CmdStatusInsideEditor.ts Normal file
View File

@@ -0,0 +1,234 @@
import { computed, reactive, reactiveSource, type ReactiveValue } from "octagonal-wheels/dataobject/reactive";
import type { DatabaseConnectingStatus, EntryDoc } from "../lib/src/common/types";
import { LiveSyncCommands } from "./LiveSyncCommands";
import { scheduleTask } from "octagonal-wheels/concurrency/task";
import { isDirty, throttle } from "../lib/src/common/utils";
import { collectingChunks, pluginScanningCount, hiddenFilesEventCount, hiddenFilesProcessingCount } from "../lib/src/mock_and_interop/stores";
import { eventHub } from "../lib/src/hub/hub";
import { EVENT_FILE_RENAMED, EVENT_LAYOUT_READY, EVENT_LEAF_ACTIVE_CHANGED } from "../common/events";
export class LogAddOn extends LiveSyncCommands {
statusBar?: HTMLElement;
statusDiv?: HTMLElement;
statusLine?: HTMLDivElement;
logMessage?: HTMLDivElement;
logHistory?: HTMLDivElement;
messageArea?: HTMLDivElement;
statusBarLabels!: ReactiveValue<{ message: string, status: string }>;
observeForLogs() {
const padSpaces = `\u{2007}`.repeat(10);
// const emptyMark = `\u{2003}`;
function padLeftSpComputed(numI: ReactiveValue<number>, mark: string) {
const formatted = reactiveSource("");
let timer: ReturnType<typeof setTimeout> | undefined = undefined;
let maxLen = 1;
numI.onChanged(numX => {
const num = numX.value;
const numLen = `${Math.abs(num)}`.length + 1;
maxLen = maxLen < numLen ? numLen : maxLen;
if (timer) clearTimeout(timer);
if (num == 0) {
timer = setTimeout(() => {
formatted.value = "";
maxLen = 1;
}, 3000);
}
formatted.value = ` ${mark}${`${padSpaces}${num}`.slice(-(maxLen))}`;
})
return computed(() => formatted.value);
}
const labelReplication = padLeftSpComputed(this.plugin.replicationResultCount, `📥`);
const labelDBCount = padLeftSpComputed(this.plugin.databaseQueueCount, `📄`);
const labelStorageCount = padLeftSpComputed(this.plugin.storageApplyingCount, `💾`);
const labelChunkCount = padLeftSpComputed(collectingChunks, `🧩`);
const labelPluginScanCount = padLeftSpComputed(pluginScanningCount, `🔌`);
const labelConflictProcessCount = padLeftSpComputed(this.plugin.conflictProcessQueueCount, `🔩`);
const hiddenFilesCount = reactive(() => hiddenFilesEventCount.value + hiddenFilesProcessingCount.value);
const labelHiddenFilesCount = padLeftSpComputed(hiddenFilesCount, `⚙️`)
const queueCountLabelX = reactive(() => {
return `${labelReplication()}${labelDBCount()}${labelStorageCount()}${labelChunkCount()}${labelPluginScanCount()}${labelHiddenFilesCount()}${labelConflictProcessCount()}`;
})
const queueCountLabel = () => queueCountLabelX.value;
const requestingStatLabel = computed(() => {
const diff = this.plugin.requestCount.value - this.plugin.responseCount.value;
return diff != 0 ? "📲 " : "";
})
const replicationStatLabel = computed(() => {
const e = this.plugin.replicationStat.value;
const sent = e.sent;
const arrived = e.arrived;
const maxPullSeq = e.maxPullSeq;
const maxPushSeq = e.maxPushSeq;
const lastSyncPullSeq = e.lastSyncPullSeq;
const lastSyncPushSeq = e.lastSyncPushSeq;
let pushLast = "";
let pullLast = "";
let w = "";
const labels: Partial<Record<DatabaseConnectingStatus, string>> = {
"CONNECTED": "⚡",
"JOURNAL_SEND": "📦↑",
"JOURNAL_RECEIVE": "📦↓",
}
switch (e.syncStatus) {
case "CLOSED":
case "COMPLETED":
case "NOT_CONNECTED":
w = "⏹";
break;
case "STARTED":
w = "🌀";
break;
case "PAUSED":
w = "💤";
break;
case "CONNECTED":
case "JOURNAL_SEND":
case "JOURNAL_RECEIVE":
w = labels[e.syncStatus] || "⚡";
pushLast = ((lastSyncPushSeq == 0) ? "" : (lastSyncPushSeq >= maxPushSeq ? " (LIVE)" : ` (${maxPushSeq - lastSyncPushSeq})`));
pullLast = ((lastSyncPullSeq == 0) ? "" : (lastSyncPullSeq >= maxPullSeq ? " (LIVE)" : ` (${maxPullSeq - lastSyncPullSeq})`));
break;
case "ERRORED":
w = "⚠";
break;
default:
w = "?";
}
return { w, sent, pushLast, arrived, pullLast };
})
const labelProc = padLeftSpComputed(this.plugin.vaultManager.processing, `⏳`);
const labelPend = padLeftSpComputed(this.plugin.vaultManager.totalQueued, `🛫`);
const labelInBatchDelay = padLeftSpComputed(this.plugin.vaultManager.batched, `📬`);
const waitingLabel = computed(() => {
return `${labelProc()}${labelPend()}${labelInBatchDelay()}`;
})
const statusLineLabel = computed(() => {
const { w, sent, pushLast, arrived, pullLast } = replicationStatLabel();
const queued = queueCountLabel();
const waiting = waitingLabel();
const networkActivity = requestingStatLabel();
return {
message: `${networkActivity}Sync: ${w} ↑ ${sent}${pushLast} ↓ ${arrived}${pullLast}${waiting}${queued}`,
};
})
const statusBarLabels = reactive(() => {
const scheduleMessage = this.plugin.isReloadingScheduled ? `WARNING! RESTARTING OBSIDIAN IS SCHEDULED!\n` : "";
const { message } = statusLineLabel();
const status = scheduleMessage + this.plugin.statusLog.value;
return {
message, status
}
})
this.statusBarLabels = statusBarLabels;
const applyToDisplay = throttle((label: typeof statusBarLabels.value) => {
const v = label;
this.applyStatusBarText();
}, 20);
statusBarLabels.onChanged(label => applyToDisplay(label.value))
}
adjustStatusDivPosition() {
const mdv = this.app.workspace.getMostRecentLeaf();
if (mdv && this.statusDiv) {
this.statusDiv.remove();
// this.statusDiv.pa();
const container = mdv.view.containerEl;
container.insertBefore(this.statusDiv, container.lastChild);
}
}
onunload() {
if (this.statusDiv) {
this.statusDiv.remove();
}
document.querySelectorAll(`.livesync-status`)?.forEach(e => e.remove());
}
async setFileStatus() {
this.messageArea!.innerText = await this.plugin.getActiveFileStatus();
}
onActiveLeafChange() {
this.adjustStatusDivPosition();
this.setFileStatus();
}
onload(): void | Promise<void> {
eventHub.on(EVENT_FILE_RENAMED, (evt: CustomEvent<{ oldPath: string, newPath: string }>) => {
this.setFileStatus();
});
eventHub.on(EVENT_LEAF_ACTIVE_CHANGED, () => this.onActiveLeafChange());
const w = document.querySelectorAll(`.livesync-status`);
w.forEach(e => e.remove());
this.observeForLogs();
this.adjustStatusDivPosition();
this.statusDiv = this.app.workspace.containerEl.createDiv({ cls: "livesync-status" });
this.statusLine = this.statusDiv.createDiv({ cls: "livesync-status-statusline" });
this.messageArea = this.statusDiv.createDiv({ cls: "livesync-status-messagearea" });
this.logMessage = this.statusDiv.createDiv({ cls: "livesync-status-logmessage" });
this.logHistory = this.statusDiv.createDiv({ cls: "livesync-status-loghistory" });
eventHub.on(EVENT_LAYOUT_READY, () => this.adjustStatusDivPosition());
if (this.settings.showStatusOnStatusbar) {
this.statusBar = this.plugin.addStatusBarItem();
this.statusBar.addClass("syncstatusbar");
}
}
nextFrameQueue: ReturnType<typeof requestAnimationFrame> | undefined = undefined;
logLines: { ttl: number, message: string }[] = [];
applyStatusBarText() {
if (this.nextFrameQueue) {
return;
}
this.nextFrameQueue = requestAnimationFrame(() => {
this.nextFrameQueue = undefined;
const { message, status } = this.statusBarLabels.value;
// const recent = logMessages.value;
const newMsg = message;
const newLog = this.settings.showOnlyIconsOnEditor ? "" : status;
this.statusBar?.setText(newMsg.split("\n")[0]);
if (this.settings.showStatusOnEditor && this.statusDiv) {
// const root = activeDocument.documentElement;
// root.style.setProperty("--sls-log-text", "'" + (newMsg + "\\A " + newLog) + "'");
// this.statusDiv.innerText = newMsg + "\\A " + newLog;
if (this.settings.showLongerLogInsideEditor) {
const now = new Date().getTime();
this.logLines = this.logLines.filter(e => e.ttl > now);
const minimumNext = this.logLines.reduce((a, b) => a < b.ttl ? a : b.ttl, Number.MAX_SAFE_INTEGER);
if (this.logLines.length > 0) setTimeout(() => this.applyStatusBarText(), minimumNext - now);
const recent = this.logLines.map(e => e.message);
const recentLogs = recent.reverse().join("\n");
if (isDirty("recentLogs", recentLogs)) this.logHistory!.innerText = recentLogs;
}
if (isDirty("newMsg", newMsg)) this.statusLine!.innerText = newMsg;
if (isDirty("newLog", newLog)) this.logMessage!.innerText = newLog;
} else {
// const root = activeDocument.documentElement;
// root.style.setProperty("--log-text", "'" + (newMsg + "\\A " + newLog) + "'");
}
});
scheduleTask("log-hide", 3000, () => { this.plugin.statusLog.value = "" });
}
onInitializeDatabase(showNotice: boolean) { }
beforeReplicate(showNotice: boolean) { }
onResume() { }
parseReplicationResultItem(docs: PouchDB.Core.ExistingDocument<EntryDoc>): boolean | Promise<boolean> {
return false;
}
async realizeSettingSyncMode() { }
}

2
src/lib

Submodule src/lib updated: acad314d1a...633af447d2

870
src/main.ts
View File

File diff suppressed because it is too large Load Diff

30
src/storages/SerializedFileAccess.ts
View File

@@ -2,7 +2,7 @@ import { type App, TFile, type DataWriteOptions, TFolder, TAbstractFile } from "
import { serialized } from "../lib/src/concurrency/lock.ts"; import { serialized } from "../lib/src/concurrency/lock.ts";
import { Logger } from "../lib/src/common/logger.ts"; import { Logger } from "../lib/src/common/logger.ts";
import { isPlainText } from "../lib/src/string_and_binary/path.ts"; import { isPlainText } from "../lib/src/string_and_binary/path.ts";
import type { FilePath } from "../lib/src/common/types.ts"; import type { FilePath, HasSettings } from "../lib/src/common/types.ts";
import { createBinaryBlob, isDocContentSame } from "../lib/src/common/utils.ts"; import { createBinaryBlob, isDocContentSame } from "../lib/src/common/utils.ts";
import type { InternalFileInfo } from "../common/types.ts"; import type { InternalFileInfo } from "../common/types.ts";
import { markChangesAreSame } from "../common/utils.ts"; import { markChangesAreSame } from "../common/utils.ts";
@@ -31,8 +31,10 @@ async function processWriteFile<T>(file: TFile | TFolder | string, proc: () => P
} }
export class SerializedFileAccess { export class SerializedFileAccess {
app: App app: App
constructor(app: App) { plugin: HasSettings<{ handleFilenameCaseSensitive: boolean }>
constructor(app: App, plugin: typeof this["plugin"]) {
this.app = app; this.app = app;
this.plugin = plugin;
} }
async adapterStat(file: TFile | string) { async adapterStat(file: TFile | string) {
@@ -138,17 +140,23 @@ export class SerializedFileAccess {
return await processWriteFile(file, () => this.app.vault.trash(file, force)); return await processWriteFile(file, () => this.app.vault.trash(file, force));
} }
isStorageInsensitive(): boolean {
//@ts-ignore
return this.app.vault.adapter.insensitive ?? true;
}
getAbstractFileByPathInsensitive(path: FilePath | string): TAbstractFile | null {
//@ts-ignore
return app.vault.getAbstractFileByPathInsensitive(path);
}
getAbstractFileByPath(path: FilePath | string): TAbstractFile | null { getAbstractFileByPath(path: FilePath | string): TAbstractFile | null {
// Disabled temporary. if (!this.plugin.settings.handleFilenameCaseSensitive || this.isStorageInsensitive()) {
return this.getAbstractFileByPathInsensitive(path);
}
return this.app.vault.getAbstractFileByPath(path); return this.app.vault.getAbstractFileByPath(path);
// // Hidden API but so useful.
// // @ts-ignore
// if ("getAbstractFileByPathInsensitive" in app.vault && (app.vault.adapter?.insensitive ?? false)) {
// // @ts-ignore
// return app.vault.getAbstractFileByPathInsensitive(path);
// } else {
// return app.vault.getAbstractFileByPath(path);
// }
} }
getFiles() { getFiles() {

26
src/ui/ConflictResolveModal.ts
View File

@@ -13,10 +13,22 @@ export class ConflictResolveModal extends Modal {
isClosed = false; isClosed = false;
consumed = false; consumed = false;
constructor(app: App, filename: string, diff: diff_result) { title: string = "Conflicting changes";
pluginPickMode: boolean = false;
localName: string = "Keep A";
remoteName: string = "Keep B";
constructor(app: App, filename: string, diff: diff_result, pluginPickMode?: boolean, remoteName?: string) {
super(app); super(app);
this.result = diff; this.result = diff;
this.filename = filename; this.filename = filename;
this.pluginPickMode = pluginPickMode || false;
if (this.pluginPickMode) {
this.title = "Pick a version";
this.remoteName = `Use ${remoteName || "Remote"}`;
this.localName = "Use Local"
}
// Send cancel signal for the previous merge dialogue // Send cancel signal for the previous merge dialogue
// if not there, simply be ignored. // if not there, simply be ignored.
// sendValue("close-resolve-conflict:" + this.filename, false); // sendValue("close-resolve-conflict:" + this.filename, false);
@@ -36,7 +48,7 @@ export class ConflictResolveModal extends Modal {
} }
}, 10) }, 10)
// sendValue("close-resolve-conflict:" + this.filename, false); // sendValue("close-resolve-conflict:" + this.filename, false);
this.titleEl.setText("Conflicting changes"); this.titleEl.setText(this.title);
contentEl.empty(); contentEl.empty();
contentEl.createEl("span", { text: this.filename }); contentEl.createEl("span", { text: this.filename });
const div = contentEl.createDiv(""); const div = contentEl.createDiv("");
@@ -62,10 +74,12 @@ export class ConflictResolveModal extends Modal {
div2.innerHTML = ` div2.innerHTML = `
<span class='deleted'>A:${date1}</span><br /><span class='added'>B:${date2}</span><br> <span class='deleted'>A:${date1}</span><br /><span class='added'>B:${date2}</span><br>
`; `;
contentEl.createEl("button", { text: "Keep A" }, (e) => e.addEventListener("click", () => this.sendResponse(this.result.right.rev))); contentEl.createEl("button", { text: this.localName }, (e) => e.addEventListener("click", () => this.sendResponse(this.result.right.rev))).style.marginRight = "4px";
contentEl.createEl("button", { text: "Keep B" }, (e) => e.addEventListener("click", () => this.sendResponse(this.result.left.rev))); contentEl.createEl("button", { text: this.remoteName }, (e) => e.addEventListener("click", () => this.sendResponse(this.result.left.rev))).style.marginRight = "4px";
contentEl.createEl("button", { text: "Concat both" }, (e) => e.addEventListener("click", () => this.sendResponse(LEAVE_TO_SUBSEQUENT))); if (!this.pluginPickMode) {
contentEl.createEl("button", { text: "Not now" }, (e) => e.addEventListener("click", () => this.sendResponse(CANCELLED))); contentEl.createEl("button", { text: "Concat both" }, (e) => e.addEventListener("click", () => this.sendResponse(LEAVE_TO_SUBSEQUENT))).style.marginRight = "4px";
}
contentEl.createEl("button", { text: !this.pluginPickMode ? "Not now" : "Cancel" }, (e) => e.addEventListener("click", () => this.sendResponse(CANCELLED))).style.marginRight = "4px";
} }
sendResponse(result: MergeDialogResult) { sendResponse(result: MergeDialogResult) {

17
src/ui/JsonResolveModal.ts
View File

@@ -12,15 +12,24 @@ export class JsonResolveModal extends Modal {
nameA: string; nameA: string;
nameB: string; nameB: string;
defaultSelect: string; defaultSelect: string;
keepOrder: boolean;
hideLocal: boolean;
title: string = "Conflicted Setting";
constructor(app: App, filename: FilePath, docs: LoadedEntry[], callback: (keepRev?: string, mergedStr?: string) => Promise<void>, nameA?: string, nameB?: string, defaultSelect?: string) { constructor(app: App, filename: FilePath,
docs: LoadedEntry[], callback: (keepRev?: string, mergedStr?: string) => Promise<void>,
nameA?: string, nameB?: string, defaultSelect?: string,
keepOrder?: boolean, hideLocal?: boolean, title: string = "Conflicted Setting") {
super(app); super(app);
this.callback = callback; this.callback = callback;
this.filename = filename; this.filename = filename;
this.docs = docs; this.docs = docs;
this.nameA = nameA || ""; this.nameA = nameA || "";
this.nameB = nameB || ""; this.nameB = nameB || "";
this.keepOrder = keepOrder || false;
this.defaultSelect = defaultSelect || ""; this.defaultSelect = defaultSelect || "";
this.title = title;
this.hideLocal = hideLocal ?? false;
waitForSignal(`cancel-internal-conflict:${filename}`).then(() => this.close()); waitForSignal(`cancel-internal-conflict:${filename}`).then(() => this.close());
} }
async UICallback(keepRev?: string, mergedStr?: string) { async UICallback(keepRev?: string, mergedStr?: string) {
@@ -31,7 +40,7 @@ export class JsonResolveModal extends Modal {
onOpen() { onOpen() {
const { contentEl } = this; const { contentEl } = this;
this.titleEl.setText("Conflicted Setting"); this.titleEl.setText(this.title);
contentEl.empty(); contentEl.empty();
if (this.component == undefined) { if (this.component == undefined) {
@@ -43,7 +52,9 @@ export class JsonResolveModal extends Modal {
nameA: this.nameA, nameA: this.nameA,
nameB: this.nameB, nameB: this.nameB,
defaultSelect: this.defaultSelect, defaultSelect: this.defaultSelect,
callback: (keepRev, mergedStr) => this.UICallback(keepRev, mergedStr), keepOrder: this.keepOrder,
hideLocal: this.hideLocal,
callback: (keepRev: string | undefined, mergedStr: string | undefined) => this.UICallback(keepRev, mergedStr),
}, },
}); });
} }

79
src/ui/JsonResolvePane.svelte
View File

@@ -13,6 +13,8 @@
export let nameA: string = "A"; export let nameA: string = "A";
export let nameB: string = "B"; export let nameB: string = "B";
export let defaultSelect: string = ""; export let defaultSelect: string = "";
export let keepOrder = false;
export let hideLocal: boolean = false;
let docA: LoadedEntry; let docA: LoadedEntry;
let docB: LoadedEntry; let docB: LoadedEntry;
let docAContent = ""; let docAContent = "";
@@ -55,9 +57,12 @@
if (mode == "AB") return callback(undefined, JSON.stringify(objAB, null, 2)); if (mode == "AB") return callback(undefined, JSON.stringify(objAB, null, 2));
callback(undefined, undefined); callback(undefined, undefined);
} }
function cancel() {
callback(undefined, undefined);
}
$: { $: {
if (docs && docs.length >= 1) { if (docs && docs.length >= 1) {
if (docs[0].mtime < docs[1].mtime) { if (keepOrder || docs[0].mtime < docs[1].mtime) {
docA = docs[0]; docA = docs[0];
docB = docs[1]; docB = docs[1];
} else { } else {
@@ -96,13 +101,19 @@
diffs = getJsonDiff(objA, selectedObj); diffs = getJsonDiff(objA, selectedObj);
} }
$: modes = [ let modes = [] as ["" | "A" | "B" | "AB" | "BA", string][];
["", "Not now"], $: {
["A", nameA || "A"], let newModes = [] as typeof modes;
["B", nameB || "B"],
["AB", `${nameA || "A"} + ${nameB || "B"}`], if (!hideLocal) {
["BA", `${nameB || "B"} + ${nameA || "A"}`], newModes.push(["", "Not now"]);
] as ["" | "A" | "B" | "AB" | "BA", string][]; newModes.push(["A", nameA || "A"]);
}
newModes.push(["B", nameB || "B"]);
newModes.push(["AB", `${nameA || "A"} + ${nameB || "B"}`]);
newModes.push(["BA", `${nameB || "B"} + ${nameA || "A"}`]);
modes = newModes;
}
</script> </script>
<h2>{filename}</h2> <h2>{filename}</h2>
@@ -132,28 +143,54 @@
{:else} {:else}
NO PREVIEW NO PREVIEW
{/if} {/if}
<div>
{nameA}
{#if docA._id == docB._id}
Rev:{revStringToRevNumber(docA._rev)}
{/if} ,{new Date(docA.mtime).toLocaleString()}
{docAContent.length} letters
</div>
<div> <div class="infos">
{nameB} <table>
{#if docA._id == docB._id} <tr>
Rev:{revStringToRevNumber(docB._rev)} <th>{nameA}</th>
{/if} ,{new Date(docB.mtime).toLocaleString()} <td
{docBContent.length} letters >{#if docA._id == docB._id}
Rev:{revStringToRevNumber(docA._rev)}
{/if}
{new Date(docA.mtime).toLocaleString()}</td
>
<td>
{docAContent.length} letters
</td>
</tr>
<tr>
<th>{nameB}</th>
<td
>{#if docA._id == docB._id}
Rev:{revStringToRevNumber(docB._rev)}
{/if}
{new Date(docB.mtime).toLocaleString()}</td
>
<td>
{docBContent.length} letters
</td>
</tr>
</table>
</div> </div>
<div class="buttons"> <div class="buttons">
{#if hideLocal}
<button on:click={cancel}>Cancel</button>
{/if}
<button on:click={apply}>Apply</button> <button on:click={apply}>Apply</button>
</div> </div>
{/if} {/if}
<style> <style>
.spacer {
flex-grow: 1;
}
.infos {
display: flex;
justify-content: space-between;
margin: 4px 0.5em;
}
.deleted { .deleted {
text-decoration: line-through; text-decoration: line-through;
} }

166
src/ui/ObsidianLiveSyncSettingTab.ts
View File

@@ -21,7 +21,7 @@ import {
statusDisplay, statusDisplay,
type ConfigurationItem type ConfigurationItem
} from "../lib/src/common/types.ts"; } from "../lib/src/common/types.ts";
import { createBlob, delay, isDocContentSame, isObjectDifferent, readAsBlob, unique } from "../lib/src/common/utils.ts"; import { createBlob, delay, isDocContentSame, isObjectDifferent, readAsBlob, sizeToHumanReadable, unique } from "../lib/src/common/utils.ts";
import { versionNumberString2Number } from "../lib/src/string_and_binary/convert.ts"; import { versionNumberString2Number } from "../lib/src/string_and_binary/convert.ts";
import { Logger } from "../lib/src/common/logger.ts"; import { Logger } from "../lib/src/common/logger.ts";
import { checkSyncInfo, isCloudantURI } from "../lib/src/pouchdb/utils_couchdb.ts"; import { checkSyncInfo, isCloudantURI } from "../lib/src/pouchdb/utils_couchdb.ts";
@@ -35,6 +35,7 @@ import { LiveSyncCouchDBReplicator } from "../lib/src/replication/couchdb/LiveSy
import { type AllSettingItemKey, type AllStringItemKey, type AllNumericItemKey, type AllBooleanItemKey, type AllSettings, OnDialogSettingsDefault, getConfig, type OnDialogSettings, getConfName } from "./settingConstants.ts"; import { type AllSettingItemKey, type AllStringItemKey, type AllNumericItemKey, type AllBooleanItemKey, type AllSettings, OnDialogSettingsDefault, getConfig, type OnDialogSettings, getConfName } from "./settingConstants.ts";
import { SUPPORTED_I18N_LANGS, type I18N_LANGS } from "src/lib/src/common/rosetta.ts"; import { SUPPORTED_I18N_LANGS, type I18N_LANGS } from "src/lib/src/common/rosetta.ts";
import { $t } from "src/lib/src/common/i18n.ts"; import { $t } from "src/lib/src/common/i18n.ts";
import { Semaphore } from "octagonal-wheels/concurrency/semaphore";
type OnUpdateResult = { type OnUpdateResult = {
visibility?: boolean, visibility?: boolean,
@@ -272,11 +273,11 @@ class Setting extends SettingOrg {
}) })
return this; return this;
} }
addApplyButton(keys: AllSettingItemKey[]) { addApplyButton(keys: AllSettingItemKey[], text?: string) {
this.addButton((button) => { this.addButton((button) => {
this.applyButtonComponent = button; this.applyButtonComponent = button;
this.watchDirtyKeys = unique([...keys, ...this.watchDirtyKeys]); this.watchDirtyKeys = unique([...keys, ...this.watchDirtyKeys]);
button.setButtonText("Apply") button.setButtonText(text ?? "Apply")
button.onClick(async () => { button.onClick(async () => {
await Setting.env.saveSettings(keys); await Setting.env.saveSettings(keys);
Setting.env.reloadAllSettings(); Setting.env.reloadAllSettings();
@@ -586,6 +587,12 @@ export class ObsidianLiveSyncSettingTab extends PluginSettingTab {
const trialSetting = { ...this.editingSettings, ...settingOverride }; const trialSetting = { ...this.editingSettings, ...settingOverride };
const replicator = this.plugin.getNewReplicator(trialSetting); const replicator = this.plugin.getNewReplicator(trialSetting);
await replicator.tryConnectRemote(trialSetting); await replicator.tryConnectRemote(trialSetting);
const status = await replicator.getRemoteStatus(trialSetting);
if (status) {
if (status.estimatedSize) {
Logger(`Estimated size: ${sizeToHumanReadable(status.estimatedSize)}`, LOG_LEVEL_NOTICE);
}
}
} }
closeSetting() { closeSetting() {
@@ -1188,6 +1195,10 @@ However, your report is needed to stabilise this. I appreciate you for your grea
.addOnUpdate(onlyOnCouchDB) .addOnUpdate(onlyOnCouchDB)
}, onlyOnCouchDB); }, onlyOnCouchDB);
this.createEl(containerRemoteDatabaseEl, "h4", { text: "Notification" }).addClass("wizardHidden")
new Setting(containerRemoteDatabaseEl).autoWireNumeric("notifyThresholdOfRemoteStorageSize", {}).setClass("wizardHidden");
this.createEl(containerRemoteDatabaseEl, "h4", { text: "Effective Storage Using" }).addClass("wizardHidden") this.createEl(containerRemoteDatabaseEl, "h4", { text: "Effective Storage Using" }).addClass("wizardHidden")
new Setting(containerRemoteDatabaseEl).autoWireToggle("useEden").setClass("wizardHidden"); new Setting(containerRemoteDatabaseEl).autoWireToggle("useEden").setClass("wizardHidden");
@@ -1428,6 +1439,7 @@ However, your report is needed to stabilise this. I appreciate you for your grea
.addApplyButton(["configPassphrase", "configPassphraseStore"]) .addApplyButton(["configPassphrase", "configPassphraseStore"])
.setClass("wizardHidden") .setClass("wizardHidden")
addScreenElement("20", containerGeneralSettingsEl); addScreenElement("20", containerGeneralSettingsEl);
const containerSyncSettingEl = containerEl.createDiv(); const containerSyncSettingEl = containerEl.createDiv();
this.createEl(containerSyncSettingEl, "h3", { text: "Sync Settings" }); this.createEl(containerSyncSettingEl, "h3", { text: "Sync Settings" });
@@ -1702,7 +1714,7 @@ However, your report is needed to stabilise this. I appreciate you for your grea
{ {
target: patSetting.controlEl, target: patSetting.controlEl,
props: { props: {
patterns: pat, originals: [...pat], apply: async (newPatterns) => { patterns: pat, originals: [...pat], apply: async (newPatterns: string[]) => {
this.editingSettings.syncInternalFilesIgnorePatterns = newPatterns.map(e => e.trim()).filter(e => e != "").join(", "); this.editingSettings.syncInternalFilesIgnorePatterns = newPatterns.map(e => e.trim()).filter(e => e != "").join(", ");
await this.saveAllDirtySettings(); await this.saveAllDirtySettings();
this.display(); this.display();
@@ -1766,6 +1778,13 @@ However, your report is needed to stabilise this. I appreciate you for your grea
new Setting(containerSyncSettingEl) new Setting(containerSyncSettingEl)
.setClass("wizardHidden") .setClass("wizardHidden")
.autoWireToggle("readChunksOnline", { onUpdate: onlyOnCouchDB }) .autoWireToggle("readChunksOnline", { onUpdate: onlyOnCouchDB })
new Setting(containerSyncSettingEl)
.setClass("wizardHidden")
.autoWireToggle("sendChunksBulk", { onUpdate: onlyOnCouchDB })
new Setting(containerSyncSettingEl)
.setClass("wizardHidden")
.autoWireNumeric("sendChunksBulkMaxSize", { clampMax: 100, clampMin: 1, onUpdate: onlyOnCouchDB })
new Setting(containerSyncSettingEl) new Setting(containerSyncSettingEl)
.setClass("wizardHidden") .setClass("wizardHidden")
@@ -1785,8 +1804,8 @@ However, your report is needed to stabilise this. I appreciate you for your grea
props: { props: {
patterns: this.editingSettings.syncOnlyRegEx.split("|[]|"), patterns: this.editingSettings.syncOnlyRegEx.split("|[]|"),
originals: [...this.editingSettings.syncOnlyRegEx.split("|[]|")], originals: [...this.editingSettings.syncOnlyRegEx.split("|[]|")],
apply: async (newPatterns) => { apply: async (newPatterns: string[]) => {
this.editingSettings.syncOnlyRegEx = newPatterns.map(e => e.trim()).filter(e => e != "").join("|[]|"); this.editingSettings.syncOnlyRegEx = newPatterns.map((e: string) => e.trim()).filter(e => e != "").join("|[]|");
await this.saveAllDirtySettings(); await this.saveAllDirtySettings();
this.display(); this.display();
} }
@@ -1805,7 +1824,7 @@ However, your report is needed to stabilise this. I appreciate you for your grea
props: { props: {
patterns: this.editingSettings.syncIgnoreRegEx.split("|[]|"), patterns: this.editingSettings.syncIgnoreRegEx.split("|[]|"),
originals: [...this.editingSettings.syncIgnoreRegEx.split("|[]|")], originals: [...this.editingSettings.syncIgnoreRegEx.split("|[]|")],
apply: async (newPatterns) => { apply: async (newPatterns: string[]) => {
this.editingSettings.syncIgnoreRegEx = newPatterns.map(e => e.trim()).filter(e => e != "").join("|[]|"); this.editingSettings.syncIgnoreRegEx = newPatterns.map(e => e.trim()).filter(e => e != "").join("|[]|");
await this.saveAllDirtySettings(); await this.saveAllDirtySettings();
this.display(); this.display();
@@ -1906,7 +1925,8 @@ However, your report is needed to stabilise this. I appreciate you for your grea
const endpointScheme = pluginConfig.endpoint.startsWith("http:") ? "(HTTP)" : (pluginConfig.endpoint.startsWith("https:")) ? "(HTTPS)" : ""; const endpointScheme = pluginConfig.endpoint.startsWith("http:") ? "(HTTP)" : (pluginConfig.endpoint.startsWith("https:")) ? "(HTTPS)" : "";
pluginConfig.endpoint = `${endpoint.indexOf(".r2.cloudflarestorage.") !== -1 ? "R2" : "self-hosted?"}(${endpointScheme})`; pluginConfig.endpoint = `${endpoint.indexOf(".r2.cloudflarestorage.") !== -1 ? "R2" : "self-hosted?"}(${endpointScheme})`;
} }
const obsidianInfo = navigator.userAgent; const obsidianInfo = `Navigator: ${navigator.userAgent}
FileSystem: ${this.plugin.vaultAccess.isStorageInsensitive() ? "insensitive" : "sensitive"}`;
const msgConfig = `---- Obsidian info ---- const msgConfig = `---- Obsidian info ----
${obsidianInfo} ${obsidianInfo}
---- remote config ---- ---- remote config ----
@@ -1988,7 +2008,7 @@ ${stringifyYaml(pluginConfig)}`;
if (fileOnDB) { if (fileOnDB) {
el.appendChild(this.createEl(el, "button", { text: "Database -> Storage" }, buttonEl => { el.appendChild(this.createEl(el, "button", { text: "Database -> Storage" }, buttonEl => {
buttonEl.onClickEvent(() => { buttonEl.onClickEvent(() => {
this.plugin.pullFile(this.plugin.getPath(fileOnDB), [], true, undefined, false); this.plugin.pullFile(this.plugin.getPath(fileOnDB), undefined, true, undefined, false);
el.remove(); el.remove();
}) })
})) }))
@@ -2007,6 +2027,18 @@ ${stringifyYaml(pluginConfig)}`;
addResult(file.path, file, fileOnDB) addResult(file.path, file, fileOnDB)
} }
} }
new Setting(containerHatchEl)
.setName("Recreate missing chunks for all files")
.setDesc("This will recreate chunks for all files. If there were missing chunks, this may fix the errors.")
.addButton((button) =>
button.
setButtonText("Recreate all")
.setCta()
.onClick(async () => {
await this.plugin.createAllChunks(true);
})
)
new Setting(containerHatchEl) new Setting(containerHatchEl)
.setName("Verify and repair all files") .setName("Verify and repair all files")
.setDesc("Compare the content of files between on local database and storage. If not matched, you will be asked which one you want to keep.") .setDesc("Compare the content of files between on local database and storage. If not matched, you will be asked which one you want to keep.")
@@ -2014,8 +2046,9 @@ ${stringifyYaml(pluginConfig)}`;
button button
.setButtonText("Verify all") .setButtonText("Verify all")
.setDisabled(false) .setDisabled(false)
.setWarning() .setCta()
.onClick(async () => { .onClick(async () => {
Logger("Start verifying all files", LOG_LEVEL_NOTICE, "verify");
const files = this.app.vault.getFiles(); const files = this.app.vault.getFiles();
const documents = [] as FilePathWithPrefix[]; const documents = [] as FilePathWithPrefix[];
@@ -2023,33 +2056,53 @@ ${stringifyYaml(pluginConfig)}`;
for await (const i of adn) documents.push(this.plugin.getPath(i)); for await (const i of adn) documents.push(this.plugin.getPath(i));
const allPaths = [...new Set([...documents, ...files.map(e => e.path as FilePathWithPrefix)])]; const allPaths = [...new Set([...documents, ...files.map(e => e.path as FilePathWithPrefix)])];
let i = 0; let i = 0;
for (const path of allPaths) { const incProc = () => {
i++; i++;
Logger(`${i}/${files.length}\n${path}`, LOG_LEVEL_NOTICE, "verify"); if (i % 25 == 0) Logger(`Checking ${i}/${files.length} files \n`, LOG_LEVEL_NOTICE, "verify-processed");
if (shouldBeIgnored(path)) continue;
const abstractFile = this.plugin.vaultAccess.getAbstractFileByPath(path);
const fileOnStorage = abstractFile instanceof TFile ? abstractFile : false;
if (!await this.plugin.isTargetFile(path)) continue;
if (fileOnStorage && this.plugin.isFileSizeExceeded(fileOnStorage.stat.size)) continue;
const fileOnDB = await this.plugin.localDatabase.getDBEntry(path);
if (fileOnDB && this.plugin.isFileSizeExceeded(fileOnDB.size)) continue;
if (!fileOnDB && fileOnStorage) {
Logger(`Compare: Not found on the local database: ${path}`, LOG_LEVEL_NOTICE);
addResult(path, fileOnStorage, false)
continue;
}
if (fileOnDB && !fileOnStorage) {
Logger(`Compare: Not found on the storage: ${path}`, LOG_LEVEL_NOTICE);
addResult(path, false, fileOnDB)
continue;
}
if (fileOnStorage && fileOnDB) {
await checkBetweenStorageAndDatabase(fileOnStorage, fileOnDB)
}
} }
const semaphore = Semaphore(10);
const processes = allPaths.map(async path => {
try {
if (shouldBeIgnored(path)) {
return incProc();
}
const abstractFile = this.plugin.vaultAccess.getAbstractFileByPath(path);
const fileOnStorage = abstractFile instanceof TFile ? abstractFile : false;
if (!await this.plugin.isTargetFile(path)) return incProc();
const releaser = await semaphore.acquire(1)
if (fileOnStorage && this.plugin.isFileSizeExceeded(fileOnStorage.stat.size)) return incProc();
try {
const fileOnDB = await this.plugin.localDatabase.getDBEntry(path);
if (fileOnDB && this.plugin.isFileSizeExceeded(fileOnDB.size)) return incProc();
if (!fileOnDB && fileOnStorage) {
Logger(`Compare: Not found on the local database: ${path}`, LOG_LEVEL_NOTICE);
addResult(path, fileOnStorage, false)
return incProc();
}
if (fileOnDB && !fileOnStorage) {
Logger(`Compare: Not found on the storage: ${path}`, LOG_LEVEL_NOTICE);
addResult(path, false, fileOnDB)
return incProc();
}
if (fileOnStorage && fileOnDB) {
await checkBetweenStorageAndDatabase(fileOnStorage, fileOnDB)
}
} catch (ex) {
Logger(`Error while processing ${path}`, LOG_LEVEL_NOTICE);
Logger(ex, LOG_LEVEL_VERBOSE);
} finally {
releaser();
incProc();
}
} catch (ex) {
Logger(`Error while processing without semaphore ${path}`, LOG_LEVEL_NOTICE);
Logger(ex, LOG_LEVEL_VERBOSE);
}
});
await Promise.all(processes);
Logger("done", LOG_LEVEL_NOTICE, "verify"); Logger("done", LOG_LEVEL_NOTICE, "verify");
// Logger(`${i}/${files.length}\n`, LOG_LEVEL_NOTICE, "verify-processed");
}) })
); );
const resultArea = containerHatchEl.createDiv({ text: "" }); const resultArea = containerHatchEl.createDiv({ text: "" });
@@ -2171,6 +2224,34 @@ ${stringifyYaml(pluginConfig)}`;
new Setting(containerHatchEl) new Setting(containerHatchEl)
.autoWireToggle("useIndexedDBAdapter", { invert: true }) .autoWireToggle("useIndexedDBAdapter", { invert: true })
new Setting(containerHatchEl)
.autoWireToggle("doNotUseFixedRevisionForChunks", { holdValue: true })
.setClass("wizardHidden")
new Setting(containerHatchEl)
.autoWireToggle("handleFilenameCaseSensitive", { holdValue: true })
.setClass("wizardHidden")
new Setting(containerHatchEl)
.setName("Apply")
.setDesc("These configurations require a database rebuild.")
.setClass("wizardHidden")
.addButton((button) =>
button
.setButtonText("Apply and rebuild")
.setWarning()
.setDisabled(false)
.onClick(async () => {
await this.saveAllDirtySettings();
// await this.applySetting(["doNotUseFixedRevisionForChunks", "handleFilenameCaseSensitive"]);
// await this.saveSettings(["doNotUseFixedRevisionForChunks", "handleFilenameCaseSensitive"]);
// debugger;
await rebuildDB("rebuildBothByThisDevice");
})
)
.addOnUpdate(() => ({
isCta: this.isSomeDirty(["doNotUseFixedRevisionForChunks", "handleFilenameCaseSensitive"]),
disabled: !this.isSomeDirty(["doNotUseFixedRevisionForChunks", "handleFilenameCaseSensitive"]),
}))
this.addOnSaved("useIndexedDBAdapter", async () => { this.addOnSaved("useIndexedDBAdapter", async () => {
await this.saveAllDirtySettings(); await this.saveAllDirtySettings();
await rebuildDB("localOnly"); await rebuildDB("localOnly");
@@ -2221,7 +2302,7 @@ ${stringifyYaml(pluginConfig)}`;
// With great respect, thank you TfTHacker! // With great respect, thank you TfTHacker!
// Refer: https://github.com/TfTHacker/obsidian42-brat/blob/main/src/features/BetaPlugins.ts // Refer: https://github.com/TfTHacker/obsidian42-brat/blob/main/src/features/BetaPlugins.ts
const containerPluginSettings = containerEl.createDiv(); const containerPluginSettings = containerEl.createDiv();
this.createEl(containerPluginSettings, "h3", { text: "Customization sync (beta)" }); this.createEl(containerPluginSettings, "h3", { text: "Customization sync (beta 3)" });
const enableOnlyOnPluginSyncIsNotEnabled = enableOnly(() => this.isConfiguredAs("usePluginSync", false)); const enableOnlyOnPluginSyncIsNotEnabled = enableOnly(() => this.isConfiguredAs("usePluginSync", false));
const visibleOnlyOnPluginSyncEnabled = visibleOnly(() => this.isConfiguredAs("usePluginSync", true)); const visibleOnlyOnPluginSyncEnabled = visibleOnly(() => this.isConfiguredAs("usePluginSync", true));
@@ -2232,6 +2313,9 @@ ${stringifyYaml(pluginConfig)}`;
onUpdate: enableOnlyOnPluginSyncIsNotEnabled onUpdate: enableOnlyOnPluginSyncIsNotEnabled
}); });
new Setting(containerPluginSettings)
.autoWireToggle("usePluginSyncV2")
new Setting(containerPluginSettings) new Setting(containerPluginSettings)
.autoWireToggle("usePluginSync", { .autoWireToggle("usePluginSync", {
onUpdate: enableOnly(() => !this.isConfiguredAs("deviceAndVaultName", "")) onUpdate: enableOnly(() => !this.isConfiguredAs("deviceAndVaultName", ""))
@@ -2315,7 +2399,17 @@ ${stringifyYaml(pluginConfig)}`;
await rebuildDB("remoteOnly"); await rebuildDB("remoteOnly");
}) })
) )
.addButton((button) =>
button
.setButtonText("Send chunks")
.setWarning()
.setDisabled(false)
.onClick(async () => {
if (this.plugin.replicator instanceof LiveSyncCouchDBReplicator) {
await this.plugin.replicator.sendChunks(this.plugin.settings, undefined, true, 0);
}
})
)
new Setting(containerMaintenanceEl) new Setting(containerMaintenanceEl)
.setName("Reset journal received history") .setName("Reset journal received history")
.setDesc("Initialise journal received history. On the next sync, every item except this device sent will be downloaded again.") .setDesc("Initialise journal received history. On the next sync, every item except this device sent will be downloaded again.")

386
src/ui/PluginPane.svelte
View File

@@ -1,11 +1,11 @@
<script lang="ts"> <script lang="ts">
import { onMount } from "svelte"; import { onMount } from "svelte";
import ObsidianLiveSyncPlugin from "../main"; import ObsidianLiveSyncPlugin from "../main";
import { type PluginDataExDisplay, pluginIsEnumerating, pluginList } from "../features/CmdConfigSync"; import { type IPluginDataExDisplay, pluginIsEnumerating, pluginList, pluginManifestStore, pluginV2Progress } from "../features/CmdConfigSync";
import PluginCombo from "./components/PluginCombo.svelte"; import PluginCombo from "./components/PluginCombo.svelte";
import { Menu } from "obsidian"; import { Menu, type PluginManifest } from "obsidian";
import { unique } from "../lib/src/common/utils"; import { unique } from "../lib/src/common/utils";
import { MODE_SELECTIVE, MODE_AUTOMATIC, MODE_PAUSED, type SYNC_MODE, type PluginSyncSettingEntry } from "../lib/src/common/types"; import { MODE_SELECTIVE, MODE_AUTOMATIC, MODE_PAUSED, type SYNC_MODE, type PluginSyncSettingEntry, MODE_SHINY } from "../lib/src/common/types";
import { normalizePath } from "../deps"; import { normalizePath } from "../deps";
export let plugin: ObsidianLiveSyncPlugin; export let plugin: ObsidianLiveSyncPlugin;
@@ -14,9 +14,10 @@
const addOn = plugin.addOnConfigSync; const addOn = plugin.addOnConfigSync;
let list: PluginDataExDisplay[] = []; let list: IPluginDataExDisplay[] = [];
let selectNewestPulse = 0; let selectNewestPulse = 0;
let selectNewestStyle = 0;
let hideEven = false; let hideEven = false;
let loading = false; let loading = false;
let applyAllPluse = 0; let applyAllPluse = 0;
@@ -39,13 +40,13 @@
requestUpdate(); requestUpdate();
}); });
function filterList(list: PluginDataExDisplay[], categories: string[]) { function filterList(list: IPluginDataExDisplay[], categories: string[]) {
const w = list.filter((e) => categories.indexOf(e.category) !== -1); const w = list.filter((e) => categories.indexOf(e.category) !== -1);
return w.sort((a, b) => `${a.category}-${a.name}`.localeCompare(`${b.category}-${b.name}`)); return w.sort((a, b) => `${a.category}-${a.name}`.localeCompare(`${b.category}-${b.name}`));
} }
function groupBy(items: PluginDataExDisplay[], key: string) { function groupBy(items: IPluginDataExDisplay[], key: string) {
let ret = {} as Record<string, PluginDataExDisplay[]>; let ret = {} as Record<string, IPluginDataExDisplay[]>;
for (const v of items) { for (const v of items) {
//@ts-ignore //@ts-ignore
const k = (key in v ? v[key] : "") as string; const k = (key in v ? v[key] : "") as string;
@@ -71,19 +72,24 @@
async function replicate() { async function replicate() {
await plugin.replicate(true); await plugin.replicate(true);
} }
function selectAllNewest() { function selectAllNewest(selectMode: boolean) {
selectNewestPulse++; selectNewestPulse++;
selectNewestStyle = selectMode ? 1 : 2;
}
function resetSelectNewest() {
selectNewestPulse++;
selectNewestStyle = 3;
} }
function applyAll() { function applyAll() {
applyAllPluse++; applyAllPluse++;
} }
async function applyData(data: PluginDataExDisplay): Promise<boolean> { async function applyData(data: IPluginDataExDisplay): Promise<boolean> {
return await addOn.applyData(data); return await addOn.applyData(data);
} }
async function compareData(docA: PluginDataExDisplay, docB: PluginDataExDisplay): Promise<boolean> { async function compareData(docA: IPluginDataExDisplay, docB: IPluginDataExDisplay, compareEach = false): Promise<boolean> {
return await addOn.compareUsingDisplayData(docA, docB); return await addOn.compareUsingDisplayData(docA, docB, compareEach);
} }
async function deleteData(data: PluginDataExDisplay): Promise<boolean> { async function deleteData(data: IPluginDataExDisplay): Promise<boolean> {
return await addOn.deleteData(data); return await addOn.deleteData(data);
} }
function askMode(evt: MouseEvent, title: string, key: string) { function askMode(evt: MouseEvent, title: string, key: string) {
@@ -91,7 +97,7 @@
menu.addItem((item) => item.setTitle(title).setIsLabel(true)); menu.addItem((item) => item.setTitle(title).setIsLabel(true));
menu.addSeparator(); menu.addSeparator();
const prevMode = automaticList.get(key) ?? MODE_SELECTIVE; const prevMode = automaticList.get(key) ?? MODE_SELECTIVE;
for (const mode of [MODE_SELECTIVE, MODE_AUTOMATIC, MODE_PAUSED]) { for (const mode of [MODE_SELECTIVE, MODE_AUTOMATIC, MODE_PAUSED, MODE_SHINY]) {
menu.addItem((item) => { menu.addItem((item) => {
item.setTitle(`${getIcon(mode as SYNC_MODE)}:${TITLES[mode]}`) item.setTitle(`${getIcon(mode as SYNC_MODE)}:${TITLES[mode]}`)
.onClick((e) => { .onClick((e) => {
@@ -139,6 +145,7 @@
thisTerm, thisTerm,
hideNotApplicable, hideNotApplicable,
selectNewest: selectNewestPulse, selectNewest: selectNewestPulse,
selectNewestStyle,
applyAllPluse, applyAllPluse,
applyData, applyData,
compareData, compareData,
@@ -150,24 +157,29 @@
const ICON_EMOJI_PAUSED = `⛔`; const ICON_EMOJI_PAUSED = `⛔`;
const ICON_EMOJI_AUTOMATIC = `✨`; const ICON_EMOJI_AUTOMATIC = `✨`;
const ICON_EMOJI_SELECTIVE = `🔀`; const ICON_EMOJI_SELECTIVE = `🔀`;
const ICON_EMOJI_FLAGGED = `🚩`;
const ICONS: { [key: number]: string } = { const ICONS: { [key: number]: string } = {
[MODE_SELECTIVE]: ICON_EMOJI_SELECTIVE, [MODE_SELECTIVE]: ICON_EMOJI_SELECTIVE,
[MODE_PAUSED]: ICON_EMOJI_PAUSED, [MODE_PAUSED]: ICON_EMOJI_PAUSED,
[MODE_AUTOMATIC]: ICON_EMOJI_AUTOMATIC, [MODE_AUTOMATIC]: ICON_EMOJI_AUTOMATIC,
[MODE_SHINY]: ICON_EMOJI_FLAGGED,
}; };
const TITLES: { [key: number]: string } = { const TITLES: { [key: number]: string } = {
[MODE_SELECTIVE]: "Selective", [MODE_SELECTIVE]: "Selective",
[MODE_PAUSED]: "Ignore", [MODE_PAUSED]: "Ignore",
[MODE_AUTOMATIC]: "Automatic", [MODE_AUTOMATIC]: "Automatic",
[MODE_SHINY]: "Flagged Selective",
}; };
const PREFIX_PLUGIN_ALL = "PLUGIN_ALL"; const PREFIX_PLUGIN_ALL = "PLUGIN_ALL";
const PREFIX_PLUGIN_DATA = "PLUGIN_DATA"; const PREFIX_PLUGIN_DATA = "PLUGIN_DATA";
const PREFIX_PLUGIN_MAIN = "PLUGIN_MAIN"; const PREFIX_PLUGIN_MAIN = "PLUGIN_MAIN";
const PREFIX_PLUGIN_ETC = "PLUGIN_ETC";
function setMode(key: string, mode: SYNC_MODE) { function setMode(key: string, mode: SYNC_MODE) {
if (key.startsWith(PREFIX_PLUGIN_ALL + "/")) { if (key.startsWith(PREFIX_PLUGIN_ALL + "/")) {
setMode(PREFIX_PLUGIN_DATA + key.substring(PREFIX_PLUGIN_ALL.length), mode); setMode(PREFIX_PLUGIN_DATA + key.substring(PREFIX_PLUGIN_ALL.length), mode);
setMode(PREFIX_PLUGIN_MAIN + key.substring(PREFIX_PLUGIN_ALL.length), mode); setMode(PREFIX_PLUGIN_MAIN + key.substring(PREFIX_PLUGIN_ALL.length), mode);
return;
} }
const files = unique( const files = unique(
list list
@@ -176,17 +188,23 @@
.flat() .flat()
.map((e) => e.filename), .map((e) => e.filename),
); );
automaticList.set(key, mode); if (mode == MODE_SELECTIVE) {
automaticListDisp = automaticList; automaticList.delete(key);
if (!(key in plugin.settings.pluginSyncExtendedSetting)) { delete plugin.settings.pluginSyncExtendedSetting[key];
plugin.settings.pluginSyncExtendedSetting[key] = { automaticListDisp = automaticList;
key, } else {
mode, automaticList.set(key, mode);
files: [], automaticListDisp = automaticList;
}; if (!(key in plugin.settings.pluginSyncExtendedSetting)) {
plugin.settings.pluginSyncExtendedSetting[key] = {
key,
mode,
files: [],
};
}
plugin.settings.pluginSyncExtendedSetting[key].files = files;
plugin.settings.pluginSyncExtendedSetting[key].mode = mode;
} }
plugin.settings.pluginSyncExtendedSetting[key].files = files;
plugin.settings.pluginSyncExtendedSetting[key].mode = mode;
plugin.saveSettingData(); plugin.saveSettingData();
} }
function getIcon(mode: SYNC_MODE) { function getIcon(mode: SYNC_MODE) {
@@ -208,9 +226,9 @@
let displayKeys: Record<string, string[]> = {}; let displayKeys: Record<string, string[]> = {};
$: { function computeDisplayKeys(list: IPluginDataExDisplay[]) {
const extraKeys = Object.keys(plugin.settings.pluginSyncExtendedSetting); const extraKeys = Object.keys(plugin.settings.pluginSyncExtendedSetting);
displayKeys = [ return [
...list, ...list,
...extraKeys ...extraKeys
.map((e) => `${e}///`.split("/")) .map((e) => `${e}///`.split("/"))
@@ -220,6 +238,9 @@
.sort((a, b) => (a.displayName ?? a.name).localeCompare(b.displayName ?? b.name)) .sort((a, b) => (a.displayName ?? a.name).localeCompare(b.displayName ?? b.name))
.reduce((p, c) => ({ ...p, [c.category]: unique(c.category in p ? [...p[c.category], c.displayName ?? c.name] : [c.displayName ?? c.name]) }), {} as Record<string, string[]>); .reduce((p, c) => ({ ...p, [c.category]: unique(c.category in p ? [...p[c.category], c.displayName ?? c.name] : [c.displayName ?? c.name]) }), {} as Record<string, string[]>);
} }
$: {
displayKeys = computeDisplayKeys(list);
}
let deleteTerm = ""; let deleteTerm = "";
@@ -230,146 +251,203 @@
} }
addOn.reloadPluginList(true); addOn.reloadPluginList(true);
} }
let nameMap = new Map<string, string>();
function updateNameMap(e: Map<string, PluginManifest>) {
const items = [...e.entries()].map(([k, v]) => [k.split("/").slice(-2).join("/"), v.name] as [string, string]);
const newMap = new Map(items);
if (newMap.size == nameMap.size) {
let diff = false;
for (const [k, v] of newMap) {
if (nameMap.get(k) != v) {
diff = true;
break;
}
}
if (!diff) {
return;
}
}
nameMap = newMap;
}
$: updateNameMap($pluginManifestStore);
let displayEntries = [] as [string, string][];
$: {
displayEntries = Object.entries(displays).filter(([key, _]) => key in displayKeys);
}
let pluginEntries = [] as [string, IPluginDataExDisplay[]][];
$: {
pluginEntries = groupBy(filterList(list, ["PLUGIN_MAIN", "PLUGIN_DATA", "PLUGIN_ETC"]), "name");
}
let useSyncPluginEtc = plugin.settings.usePluginEtc;
</script> </script>
<div> <div class="buttonsWrap">
<div> <div class="buttons">
<div class="buttons"> <button on:click={() => scanAgain()}>Scan changes</button>
<button on:click={() => scanAgain()}>Scan changes</button> <button on:click={() => replicate()}>Sync once</button>
<button on:click={() => replicate()}>Sync once</button> <button on:click={() => requestUpdate()}>Refresh</button>
<button on:click={() => requestUpdate()}>Refresh</button> {#if isMaintenanceMode}
{#if isMaintenanceMode} <button on:click={() => requestReload()}>Reload</button>
<button on:click={() => requestReload()}>Reload</button> {/if}
{/if}
<button on:click={() => selectAllNewest()}>Select All Shiny</button>
</div>
<div class="buttons">
<button on:click={() => applyAll()}>Apply All</button>
</div>
</div> </div>
{#if loading} <div class="buttons">
<div> <button on:click={() => selectAllNewest(true)}>Select All Shiny</button>
<span>Updating list...</span> <button on:click={() => selectAllNewest(false)}>{ICON_EMOJI_FLAGGED} Select Flagged Shiny</button>
</div> <button on:click={() => resetSelectNewest()}>Deselect all</button>
<button on:click={() => applyAll()} class="mod-cta">Apply All Selected</button>
</div>
</div>
<div class="loading">
{#if loading || $pluginV2Progress !== 0}
<span>Updating list...{$pluginV2Progress == 0 ? "" : ` (${$pluginV2Progress})`}</span>
{/if} {/if}
<div class="list"> </div>
{#if list.length == 0} <div class="list">
<div class="center">No Items.</div> {#if list.length == 0}
{:else} <div class="center">No Items.</div>
{#each Object.entries(displays).filter(([key, _]) => key in displayKeys) as [key, label]} {:else}
<div> {#each displayEntries as [key, label]}
<h3>{label}</h3> <div>
{#each displayKeys[key] as name} <h3>{label}</h3>
{@const bindKey = `${key}/${name}`} {#each displayKeys[key] as name}
{@const mode = automaticListDisp.get(bindKey) ?? MODE_SELECTIVE} {@const bindKey = `${key}/${name}`}
<div class="labelrow {hideEven ? 'hideeven' : ''}"> {@const mode = automaticListDisp.get(bindKey) ?? MODE_SELECTIVE}
<div class="title"> <div class="labelrow {hideEven ? 'hideeven' : ''}">
<button class="status" on:click={(evt) => askMode(evt, `${key}/${name}`, bindKey)}> <div class="title">
{getIcon(mode)} <button class="status" on:click={(evt) => askMode(evt, `${key}/${name}`, bindKey)}>
</button> {getIcon(mode)}
<span class="name">{name}</span> </button>
</div> <span class="name">{(key == "THEME" && nameMap.get(`themes/${name}`)) || name}</span>
{#if mode == MODE_SELECTIVE} </div>
<PluginCombo {...options} list={list.filter((e) => e.category == key && e.name == name)} hidden={false} /> <div class="body">
{#if mode == MODE_SELECTIVE || mode == MODE_SHINY}
<PluginCombo {...options} isFlagged={mode == MODE_SHINY} list={list.filter((e) => e.category == key && e.name == name)} hidden={false} />
{:else} {:else}
<div class="statusnote">{TITLES[mode]}</div> <div class="statusnote">{TITLES[mode]}</div>
{/if} {/if}
</div> </div>
{/each} </div>
</div> {/each}
{/each} </div>
<div> {/each}
<h3>Plugins</h3> <div>
{#each groupBy(filterList(list, ["PLUGIN_MAIN", "PLUGIN_DATA", "PLUGIN_ETC"]), "name") as [name, listX]} <h3>Plugins</h3>
{@const bindKeyAll = `${PREFIX_PLUGIN_ALL}/${name}`} {#each pluginEntries as [name, listX]}
{@const modeAll = automaticListDisp.get(bindKeyAll) ?? MODE_SELECTIVE} {@const bindKeyAll = `${PREFIX_PLUGIN_ALL}/${name}`}
{@const bindKeyMain = `${PREFIX_PLUGIN_MAIN}/${name}`} {@const modeAll = automaticListDisp.get(bindKeyAll) ?? MODE_SELECTIVE}
{@const modeMain = automaticListDisp.get(bindKeyMain) ?? MODE_SELECTIVE} {@const bindKeyMain = `${PREFIX_PLUGIN_MAIN}/${name}`}
{@const bindKeyData = `${PREFIX_PLUGIN_DATA}/${name}`} {@const modeMain = automaticListDisp.get(bindKeyMain) ?? MODE_SELECTIVE}
{@const modeData = automaticListDisp.get(bindKeyData) ?? MODE_SELECTIVE} {@const bindKeyData = `${PREFIX_PLUGIN_DATA}/${name}`}
<div class="labelrow {hideEven ? 'hideeven' : ''}"> {@const modeData = automaticListDisp.get(bindKeyData) ?? MODE_SELECTIVE}
<div class="title"> {@const bindKeyETC = `${PREFIX_PLUGIN_ETC}/${name}`}
<button class="status" on:click={(evt) => askMode(evt, `${PREFIX_PLUGIN_ALL}/${name}`, bindKeyAll)}> {@const modeEtc = automaticListDisp.get(bindKeyETC) ?? MODE_SELECTIVE}
{getIcon(modeAll)} <div class="labelrow {hideEven ? 'hideeven' : ''}">
</button> <div class="title">
<span class="name">{name}</span> <button class="status" on:click={(evt) => askMode(evt, `${PREFIX_PLUGIN_ALL}/${name}`, bindKeyAll)}>
</div> {getIcon(modeAll)}
{#if modeAll == MODE_SELECTIVE} </button>
<PluginCombo {...options} list={listX} hidden={true} /> <span class="name">{nameMap.get(`plugins/${name}`) || name}</span>
</div>
<div class="body">
{#if modeAll == MODE_SELECTIVE || modeAll == MODE_SHINY}
<PluginCombo {...options} isFlagged={modeAll == MODE_SHINY} list={listX} hidden={true} />
{/if} {/if}
</div> </div>
{#if modeAll == MODE_SELECTIVE} </div>
<div class="filerow {hideEven ? 'hideeven' : ''}"> {#if modeAll == MODE_SELECTIVE || modeAll == MODE_SHINY}
<div class="filetitle"> <div class="filerow {hideEven ? 'hideeven' : ''}">
<button class="status" on:click={(evt) => askMode(evt, `${PREFIX_PLUGIN_MAIN}/${name}/MAIN`, bindKeyMain)}> <div class="filetitle">
{getIcon(modeMain)} <button class="status" on:click={(evt) => askMode(evt, `${PREFIX_PLUGIN_MAIN}/${name}/MAIN`, bindKeyMain)}>
</button> {getIcon(modeMain)}
<span class="name">MAIN</span> </button>
</div> <span class="name">MAIN</span>
{#if modeMain == MODE_SELECTIVE} </div>
<PluginCombo {...options} list={filterList(listX, ["PLUGIN_MAIN"])} hidden={false} /> <div class="body">
{#if modeMain == MODE_SELECTIVE || modeMain == MODE_SHINY}
<PluginCombo {...options} isFlagged={modeMain == MODE_SHINY} list={filterList(listX, ["PLUGIN_MAIN"])} hidden={false} />
{:else} {:else}
<div class="statusnote">{TITLES[modeMain]}</div> <div class="statusnote">{TITLES[modeMain]}</div>
{/if} {/if}
</div> </div>
<div class="filerow {hideEven ? 'hideeven' : ''}"> </div>
<div class="filetitle"> <div class="filerow {hideEven ? 'hideeven' : ''}">
<button class="status" on:click={(evt) => askMode(evt, `${PREFIX_PLUGIN_DATA}/${name}`, bindKeyData)}> <div class="filetitle">
{getIcon(modeData)} <button class="status" on:click={(evt) => askMode(evt, `${PREFIX_PLUGIN_DATA}/${name}`, bindKeyData)}>
</button> {getIcon(modeData)}
<span class="name">DATA</span> </button>
</div> <span class="name">DATA</span>
{#if modeData == MODE_SELECTIVE} </div>
<PluginCombo {...options} list={filterList(listX, ["PLUGIN_DATA"])} hidden={false} /> <div class="body">
{#if modeData == MODE_SELECTIVE || modeData == MODE_SHINY}
<PluginCombo {...options} isFlagged={modeData == MODE_SHINY} list={filterList(listX, ["PLUGIN_DATA"])} hidden={false} />
{:else} {:else}
<div class="statusnote">{TITLES[modeData]}</div> <div class="statusnote">{TITLES[modeData]}</div>
{/if} {/if}
</div> </div>
{:else} </div>
<div class="noterow"> {#if useSyncPluginEtc}
<div class="statusnote">{TITLES[modeAll]}</div> <div class="filerow {hideEven ? 'hideeven' : ''}">
<div class="filetitle">
<button class="status" on:click={(evt) => askMode(evt, `${PREFIX_PLUGIN_ETC}/${name}`, bindKeyETC)}>
{getIcon(modeEtc)}
</button>
<span class="name">Other files</span>
</div>
<div class="body">
{#if modeEtc == MODE_SELECTIVE || modeEtc == MODE_SHINY}
<PluginCombo {...options} isFlagged={modeEtc == MODE_SHINY} list={filterList(listX, ["PLUGIN_ETC"])} hidden={false} />
{:else}
<div class="statusnote">{TITLES[modeEtc]}</div>
{/if}
</div>
</div> </div>
{/if} {/if}
{/each} {:else}
</div> <div class="noterow">
{/if} <div class="statusnote">{TITLES[modeAll]}</div>
</div> </div>
{#if isMaintenanceMode} {/if}
<div class="list"> {/each}
<div>
<h3>Maintenance Commands</h3>
<div class="maintenancerow">
<label for="">Delete All of </label>
<select bind:value={deleteTerm}>
{#each allTerms as term}
<option value={term}>{term}</option>
{/each}
</select>
<button
class="status"
on:click={(evt) => {
deleteAllItems(deleteTerm);
}}
>
🗑️
</button>
</div>
</div>
</div> </div>
{/if} {/if}
</div>
{#if isMaintenanceMode}
<div class="buttons"> <div class="buttons">
<label><span>Hide not applicable items</span><input type="checkbox" bind:checked={hideEven} /></label> <div>
</div> <h3>Maintenance Commands</h3>
<div class="buttons"> <div class="maintenancerow">
<label><span>Maintenance mode</span><input type="checkbox" bind:checked={isMaintenanceMode} /></label> <label for="">Delete All of </label>
<select bind:value={deleteTerm}>
{#each allTerms as term}
<option value={term}>{term}</option>
{/each}
</select>
<button
class="status"
on:click={(evt) => {
deleteAllItems(deleteTerm);
}}
>
🗑️
</button>
</div>
</div>
</div> </div>
{/if}
<div class="buttons">
<label><span>Hide not applicable items</span><input type="checkbox" bind:checked={hideEven} /></label>
</div>
<div class="buttons">
<label><span>Maintenance mode</span><input type="checkbox" bind:checked={isMaintenanceMode} /></label>
</div> </div>
<style> <style>
/* span.spacer { .buttonsWrap {
min-width: 1px; padding-bottom: 4px;
flex-grow: 1; }
} */
h3 { h3 {
position: sticky; position: sticky;
top: 0; top: 0;
@@ -414,13 +492,23 @@
min-width: 10em; min-width: 10em;
flex-grow: 1; flex-grow: 1;
} }
.list {
overflow-y: auto;
}
.title { .title {
color: var(--text-normal); color: var(--text-normal);
font-size: var(--font-ui-medium); font-size: var(--font-ui-medium);
line-height: var(--line-height-tight); line-height: var(--line-height-tight);
margin-right: auto; margin-right: auto;
} }
.body {
/* margin-left: 0.4em; */
margin-left: auto;
display: flex;
justify-content: flex-start;
align-items: center;
/* flex-wrap: wrap; */
}
.filetitle { .filetitle {
color: var(--text-normal); color: var(--text-normal);
font-size: var(--font-ui-medium); font-size: var(--font-ui-medium);
@@ -467,4 +555,24 @@
margin-right: 0.5em; margin-right: 0.5em;
margin-left: 0.5em; margin-left: 0.5em;
} }
.loading {
transition: height 0.25s ease-in-out;
transition-delay: 4ms;
overflow-y: hidden;
flex-shrink: 0;
display: flex;
justify-content: flex-start;
align-items: center;
}
.loading:empty {
height: 0px;
transition: height 0.25s ease-in-out;
transition-delay: 1s;
}
.loading:not(:empty) {
height: 2em;
transition: height 0.25s ease-in-out;
transition-delay: 0;
}
</style> </style>

216
src/ui/components/PluginCombo.svelte
View File

@@ -1,39 +1,42 @@
<script lang="ts"> <script lang="ts">
import type { PluginDataExDisplay } from "../../features/CmdConfigSync"; import { PluginDataExDisplayV2, type IPluginDataExDisplay } from "../../features/CmdConfigSync";
import { Logger } from "../../lib/src/common/logger"; import { Logger } from "../../lib/src/common/logger";
import { versionNumberString2Number } from "../../lib/src/string_and_binary/convert"; import { type FilePath, LOG_LEVEL_INFO, LOG_LEVEL_NOTICE, LOG_LEVEL_VERBOSE } from "../../lib/src/common/types";
import { type FilePath, LOG_LEVEL_NOTICE } from "../../lib/src/common/types"; import { getDocData, timeDeltaToHumanReadable, unique } from "../../lib/src/common/utils";
import { getDocData } from "../../lib/src/common/utils";
import type ObsidianLiveSyncPlugin from "../../main"; import type ObsidianLiveSyncPlugin from "../../main";
import { askString, scheduleTask } from "../../common/utils"; import { askString } from "../../common/utils";
import { Menu } from "obsidian";
export let list: PluginDataExDisplay[] = []; export let list: IPluginDataExDisplay[] = [];
export let thisTerm = ""; export let thisTerm = "";
export let hideNotApplicable = false; export let hideNotApplicable = false;
export let selectNewest = 0; export let selectNewest = 0;
export let selectNewestStyle = 0;
export let applyAllPluse = 0; export let applyAllPluse = 0;
export let applyData: (data: PluginDataExDisplay) => Promise<boolean>; export let applyData: (data: IPluginDataExDisplay) => Promise<boolean>;
export let compareData: (dataA: PluginDataExDisplay, dataB: PluginDataExDisplay) => Promise<boolean>; export let compareData: (dataA: IPluginDataExDisplay, dataB: IPluginDataExDisplay, compareEach?: boolean) => Promise<boolean>;
export let deleteData: (data: PluginDataExDisplay) => Promise<boolean>; export let deleteData: (data: IPluginDataExDisplay) => Promise<boolean>;
export let hidden: boolean; export let hidden: boolean;
export let plugin: ObsidianLiveSyncPlugin; export let plugin: ObsidianLiveSyncPlugin;
export let isMaintenanceMode: boolean = false; export let isMaintenanceMode: boolean = false;
export let isFlagged: boolean = false;
const addOn = plugin.addOnConfigSync; const addOn = plugin.addOnConfigSync;
let selected = ""; export let selected = "";
let freshness = ""; let freshness = "";
let equivalency = ""; let equivalency = "";
let version = ""; let version = "";
let canApply: boolean = false; let canApply: boolean = false;
let canCompare: boolean = false; let canCompare: boolean = false;
let pickToCompare: boolean = false;
let currentSelectNewest = 0; let currentSelectNewest = 0;
let currentApplyAll = 0; let currentApplyAll = 0;
// Selectable terminals // Selectable terminals
let terms = [] as string[]; let terms = [] as string[];
async function comparePlugin(local: PluginDataExDisplay, remote: PluginDataExDisplay) { async function comparePlugin(local: IPluginDataExDisplay | undefined, remote: IPluginDataExDisplay | undefined) {
let freshness = ""; let freshness = "";
let equivalency = ""; let equivalency = "";
let version = ""; let version = "";
@@ -41,25 +44,28 @@
let canApply: boolean = false; let canApply: boolean = false;
let canCompare = false; let canCompare = false;
if (!local && !remote) { if (!local && !remote) {
// NO OP. whats happened? // NO OP. what's happened?
freshness = ""; freshness = "";
} else if (local && !remote) { } else if (local && !remote) {
freshness = "⚠ Local only"; freshness = "Local only";
} else if (remote && !local) { } else if (remote && !local) {
freshness = "✓ Remote only"; freshness = "Remote only";
canApply = true; canApply = true;
} else { } else {
const dtDiff = (local?.mtime ?? 0) - (remote?.mtime ?? 0); const dtDiff = (local?.mtime ?? 0) - (remote?.mtime ?? 0);
const diff = timeDeltaToHumanReadable(Math.abs(dtDiff));
if (dtDiff / 1000 < -10) { if (dtDiff / 1000 < -10) {
freshness = "✓ Newer"; // freshness = "✓ Newer";
freshness = `Newer (${diff})`;
canApply = true; canApply = true;
contentCheck = true; contentCheck = true;
} else if (dtDiff / 1000 > 10) { } else if (dtDiff / 1000 > 10) {
freshness = "⚠ Older"; // freshness = "⚠ Older";
freshness = `Older (${diff})`;
canApply = true; canApply = true;
contentCheck = true; contentCheck = true;
} else { } else {
freshness = "⚖️ Same old"; freshness = "Same";
canApply = false; canApply = false;
contentCheck = true; contentCheck = true;
} }
@@ -67,25 +73,26 @@
const localVersionStr = local?.version || "0.0.0"; const localVersionStr = local?.version || "0.0.0";
const remoteVersionStr = remote?.version || "0.0.0"; const remoteVersionStr = remote?.version || "0.0.0";
if (local?.version || remote?.version) { if (local?.version || remote?.version) {
const localVersion = versionNumberString2Number(localVersionStr); const compare = `${localVersionStr}`.localeCompare(remoteVersionStr, undefined, { numeric: true });
const remoteVersion = versionNumberString2Number(remoteVersionStr); if (compare == 0) {
if (localVersion == remoteVersion) { version = "Same";
version = "⚖️ Same ver."; } else if (compare < 0) {
} else if (localVersion > remoteVersion) { version = `Lower (${localVersionStr} < ${remoteVersionStr})`;
version = `⚠ Lower ${localVersionStr} > ${remoteVersionStr}`; } else if (compare > 0) {
} else if (localVersion < remoteVersion) { version = `Higher (${localVersionStr} > ${remoteVersionStr})`;
version = `✓ Higher ${localVersionStr} < ${remoteVersionStr}`;
} }
} }
if (contentCheck) { if (contentCheck) {
const { canApply, equivalency, canCompare } = await checkEquivalency(local, remote); if (local && remote) {
return { canApply, freshness, equivalency, version, canCompare }; const { canApply, equivalency, canCompare } = await checkEquivalency(local, remote);
return { canApply, freshness, equivalency, version, canCompare };
}
} }
return { canApply, freshness, equivalency, version, canCompare }; return { canApply, freshness, equivalency, version, canCompare };
} }
async function checkEquivalency(local: PluginDataExDisplay, remote: PluginDataExDisplay) { async function checkEquivalency(local: IPluginDataExDisplay, remote: IPluginDataExDisplay) {
let equivalency = ""; let equivalency = "";
let canApply = false; let canApply = false;
let canCompare = false; let canCompare = false;
@@ -100,17 +107,21 @@
return 0b0000010; //"LOCAL_ONLY"; return 0b0000010; //"LOCAL_ONLY";
} else if (!localFile && remoteFile) { } else if (!localFile && remoteFile) {
return 0b0001000; //"REMOTE ONLY" return 0b0001000; //"REMOTE ONLY"
} else { } else if (localFile && remoteFile) {
if (getDocData(localFile.data) == getDocData(remoteFile.data)) { const localDoc = getDocData(localFile.data);
const remoteDoc = getDocData(remoteFile.data);
if (localDoc == remoteDoc) {
return 0b0000100; //"EVEN" return 0b0000100; //"EVEN"
} else { } else {
return 0b0010000; //"DIFFERENT"; return 0b0010000; //"DIFFERENT";
} }
} else {
return 0b0010000; //"DIFFERENT";
} }
}) })
.reduce((p, c) => p | (c as number), 0 as number); .reduce((p, c) => p | (c as number), 0 as number);
if (matchingStatus == 0b0000100) { if (matchingStatus == 0b0000100) {
equivalency = "⚖️ Same"; equivalency = "Same";
canApply = false; canApply = false;
} else if (matchingStatus <= 0b0000100) { } else if (matchingStatus <= 0b0000100) {
equivalency = "Same or local only"; equivalency = "Same or local only";
@@ -118,30 +129,37 @@
} else if (matchingStatus == 0b0010000) { } else if (matchingStatus == 0b0010000) {
canApply = true; canApply = true;
canCompare = true; canCompare = true;
equivalency = "≠ Different"; equivalency = "Different";
} else { } else {
canApply = true; canApply = true;
canCompare = true; canCompare = true;
equivalency = "≠ Different"; equivalency = "Mixed";
} }
return { equivalency, canApply, canCompare }; return { equivalency, canApply, canCompare };
} }
async function performCompare(local: PluginDataExDisplay, remote: PluginDataExDisplay) { async function performCompare(local: IPluginDataExDisplay | undefined, remote: IPluginDataExDisplay | undefined) {
const result = await comparePlugin(local, remote); const result = await comparePlugin(local, remote);
canApply = result.canApply; canApply = result.canApply;
freshness = result.freshness; freshness = result.freshness;
equivalency = result.equivalency; equivalency = result.equivalency;
version = result.version; version = result.version;
canCompare = result.canCompare; canCompare = result.canCompare;
if (local?.files.length != 1 || !local?.files?.first()?.filename?.endsWith(".json")) { pickToCompare = false;
canCompare = false; if (canCompare) {
if (local?.files.length == remote?.files.length && local?.files.length == 1 && local?.files[0].filename == remote?.files[0].filename) {
pickToCompare = false;
} else {
pickToCompare = true;
// pickToCompare = false;
// canCompare = false;
}
} }
} }
async function updateTerms(list: PluginDataExDisplay[], selectNewest: boolean, isMaintenanceMode: boolean) { async function updateTerms(list: IPluginDataExDisplay[], selectNewest: boolean, isMaintenanceMode: boolean) {
const local = list.find((e) => e.term == thisTerm); const local = list.find((e) => e.term == thisTerm);
selected = ""; // selected = "";
if (isMaintenanceMode) { if (isMaintenanceMode) {
terms = [...new Set(list.map((e) => e.term))]; terms = [...new Set(list.map((e) => e.term))];
} else if (hideNotApplicable) { } else if (hideNotApplicable) {
@@ -157,7 +175,7 @@
} else { } else {
terms = [...new Set(list.map((e) => e.term))].filter((e) => e != thisTerm); terms = [...new Set(list.map((e) => e.term))].filter((e) => e != thisTerm);
} }
let newest: PluginDataExDisplay = local; let newest: IPluginDataExDisplay | undefined = local;
if (selectNewest) { if (selectNewest) {
for (const term of terms) { for (const term of terms) {
const remote = list.find((e) => e.term == term); const remote = list.find((e) => e.term == term);
@@ -170,12 +188,25 @@
} }
// selectNewest = false; // selectNewest = false;
} }
if (terms.indexOf(selected) < 0) {
selected = "";
}
} }
$: { $: {
// React pulse and select // React pulse and select
const doSelectNewest = selectNewest != currentSelectNewest; let doSelectNewest = false;
currentSelectNewest = selectNewest; if (selectNewest != currentSelectNewest) {
if (selectNewestStyle == 1) {
doSelectNewest = true;
} else if (selectNewestStyle == 2) {
doSelectNewest = isFlagged;
} else if (selectNewestStyle == 3) {
selected = "";
}
// currentSelectNewest = selectNewest;
}
updateTerms(list, doSelectNewest, isMaintenanceMode); updateTerms(list, doSelectNewest, isMaintenanceMode);
currentSelectNewest = selectNewest;
} }
$: { $: {
// React pulse and apply // React pulse and apply
@@ -213,10 +244,52 @@
async function compareSelected() { async function compareSelected() {
const local = list.find((e) => e.term == thisTerm); const local = list.find((e) => e.term == thisTerm);
const selectedItem = list.find((e) => e.term == selected); const selectedItem = list.find((e) => e.term == selected);
if (local && selectedItem && (await compareData(local, selectedItem))) { await compareItems(local, selectedItem);
addOn.updatePluginList(true, local.documentPath); }
async function compareItems(local: IPluginDataExDisplay | undefined, remote: IPluginDataExDisplay | undefined, filename?: string) {
if (local && remote) {
if (!filename) {
if (await compareData(local, remote)) {
addOn.updatePluginList(true, local.documentPath);
}
return;
} else {
const localCopy = local instanceof PluginDataExDisplayV2 ? new PluginDataExDisplayV2(local) : { ...local };
const remoteCopy = remote instanceof PluginDataExDisplayV2 ? new PluginDataExDisplayV2(remote) : { ...remote };
localCopy.files = localCopy.files.filter((e) => e.filename == filename);
remoteCopy.files = remoteCopy.files.filter((e) => e.filename == filename);
if (await compareData(localCopy, remoteCopy, true)) {
addOn.updatePluginList(true, local.documentPath);
}
}
return;
} else {
if (!remote && !local) {
Logger(`Could not find both remote and local item`, LOG_LEVEL_INFO);
} else if (!remote) {
Logger(`Could not find remote item`, LOG_LEVEL_INFO);
} else if (!local) {
Logger(`Could not locally item`, LOG_LEVEL_INFO);
}
} }
} }
async function pickCompareItem(evt: MouseEvent) {
const local = list.find((e) => e.term == thisTerm);
const selectedItem = list.find((e) => e.term == selected);
if (!local) return;
if (!selectedItem) return;
const menu = new Menu();
menu.addItem((item) => item.setTitle("Compare file").setIsLabel(true));
menu.addSeparator();
const files = unique(local.files.map((e) => e.filename).concat(selectedItem.files.map((e) => e.filename)));
for (const filename of files) {
menu.addItem((item) => {
item.setTitle(filename).onClick((e) => compareItems(local, selectedItem, filename));
});
}
menu.showAtMouseEvent(evt);
}
async function deleteSelected() { async function deleteSelected() {
const selectedItem = list.find((e) => e.term == selected); const selectedItem = list.find((e) => e.term == selected);
// const deletedPath = selectedItem.documentPath; // const deletedPath = selectedItem.documentPath;
@@ -226,6 +299,10 @@
} }
async function duplicateItem() { async function duplicateItem() {
const local = list.find((e) => e.term == thisTerm); const local = list.find((e) => e.term == thisTerm);
if (!local) {
Logger(`Could not find local item`, LOG_LEVEL_VERBOSE);
return;
}
const duplicateTermName = await askString(plugin.app, "Duplicate", "device name", ""); const duplicateTermName = await askString(plugin.app, "Duplicate", "device name", "");
if (duplicateTermName) { if (duplicateTermName) {
if (duplicateTermName.contains("/")) { if (duplicateTermName.contains("/")) {
@@ -242,10 +319,10 @@
{#if terms.length > 0} {#if terms.length > 0}
<span class="spacer" /> <span class="spacer" />
{#if !hidden} {#if !hidden}
<span class="messages"> <span class="chip-wrap">
<span class="message">{freshness}</span> <span class="chip modified">{freshness}</span>
<span class="message">{equivalency}</span> <span class="chip content">{equivalency}</span>
<span class="message">{version}</span> <span class="chip version">{version}</span>
</span> </span>
<select bind:value={selected}> <select bind:value={selected}>
<option value={""}>-</option> <option value={""}>-</option>
@@ -255,7 +332,12 @@
</select> </select>
{#if canApply || (isMaintenanceMode && selected != "")} {#if canApply || (isMaintenanceMode && selected != "")}
{#if canCompare} {#if canCompare}
<button on:click={compareSelected}>🔍</button> {#if pickToCompare}
<button on:click={pickCompareItem}>🗃️</button>
{:else}
<!--🔍 -->
<button on:click={compareSelected}>⮂</button>
{/if}
{:else} {:else}
<button disabled /> <button disabled />
{/if} {/if}
@@ -307,12 +389,46 @@
padding: 0 1em; padding: 0 1em;
line-height: var(--line-height-tight); line-height: var(--line-height-tight);
} }
span.messages { /* span.messages {
display: flex; display: flex;
flex-direction: column; flex-direction: column;
align-items: center; align-items: center;
} } */
:global(.is-mobile) .spacer { :global(.is-mobile) .spacer {
margin-left: auto; margin-left: auto;
} }
.chip-wrap {
display: flex;
gap: 2px;
flex-direction: column;
justify-content: center;
align-items: flex-start;
}
.chip {
display: inline-block;
border-radius: 2px;
font-size: 0.8em;
padding: 0 4px;
margin: 0 2px;
border-color: var(--tag-border-color);
background-color: var(--tag-background);
color: var(--tag-color);
}
.chip:empty {
display: none;
}
.chip:not(:empty)::before {
min-width: 1.8em;
display: inline-block;
}
.chip.content:not(:empty)::before {
content: "📄: ";
}
.chip.version:not(:empty)::before {
content: "🏷️: ";
}
.chip.modified:not(:empty)::before {
content: "📅: ";
}
</style> </style>

22
src/ui/settingConstants.ts
View File

@@ -320,7 +320,27 @@ export const SettingInformation: Partial<Record<keyof AllSettings, Configuration
batchSaveMaximumDelay: { batchSaveMaximumDelay: {
name: "Maximum delay for batch database updating", name: "Maximum delay for batch database updating",
desc: "Saving will be performed forcefully after this number of seconds." desc: "Saving will be performed forcefully after this number of seconds."
} },
"notifyThresholdOfRemoteStorageSize": {
name: "Notify when the estimated remote storage size exceeds on start up",
desc: "MB (0 to disable)."
},
"usePluginSyncV2": {
name: "Enable per-file-saved customization sync",
desc: "If enabled per-filed efficient customization sync will be used. We need a small migration when enabling this. And all devices should be updated to v0.23.18. Once we enabled this, we lost a compatibility with old versions."
},
"handleFilenameCaseSensitive": {
name: "Handle files as Case-Sensitive",
desc: "If this enabled, All files are handled as case-Sensitive (Previous behaviour)."
},
"doNotUseFixedRevisionForChunks": {
name: "Compute revisions for chunks (Previous behaviour)",
desc: "If this enabled, all chunks will be stored with the revision made from its content. (Previous behaviour)"
},
"sendChunksBulkMaxSize": {
name: "Maximum size of chunks to send in one request",
desc: "MB"
},
} }
function translateInfo(infoSrc: ConfigurationItem | undefined | false) { function translateInfo(infoSrc: ConfigurationItem | undefined | false) {
if (!infoSrc) return false; if (!infoSrc) return false;

103
styles.css
View File

@@ -97,55 +97,6 @@
max-width: 100%; max-width: 100%;
} }
.CodeMirror-wrap::before,
.markdown-preview-view.cm-s-obsidian::before,
.markdown-source-view.cm-s-obsidian::before,
.canvas-wrapper::before,
.empty-state::before {
content: var(--sls-log-text, "");
font-variant-numeric: tabular-nums;
font-variant-emoji: emoji;
tab-size: 4;
text-align: right;
white-space: pre-wrap;
position: absolute;
border-radius: 4px;
/* border:1px solid --background-modifier-border; */
display: inline-block;
top: 8px;
color: --text-normal;
opacity: 0.5;
font-size: 80%;
-webkit-filter: grayscale(100%);
filter: grayscale(100%);
}
.empty-state::before,
.markdown-preview-view.cm-s-obsidian::before,
.markdown-source-view.cm-s-obsidian::before {
top: var(--header-height);
right: 1em;
}
.is-mobile .empty-state::before,
.is-mobile .markdown-preview-view.cm-s-obsidian::before,
.is-mobile .markdown-source-view.cm-s-obsidian::before {
top: var(--view-header-height);
right: 1em;
}
.canvas-wrapper::before {
right: 48px;
}
.CodeMirror-wrap::before {
right: 0px;
}
.cm-s-obsidian > .cm-editor::before {
right: 16px;
}
.sls-setting-tab { .sls-setting-tab {
display: none; display: none;
} }
@@ -171,8 +122,8 @@ div.sls-setting-menu-btn {
/* width: 100%; */ /* width: 100%; */
} }
.sls-setting-tab:hover ~ div.sls-setting-menu-btn, .sls-setting-tab:hover~div.sls-setting-menu-btn,
.sls-setting-label.selected .sls-setting-tab:checked ~ div.sls-setting-menu-btn { .sls-setting-label.selected .sls-setting-tab:checked~div.sls-setting-menu-btn {
background-color: var(--interactive-accent); background-color: var(--interactive-accent);
color: var(--text-on-accent); color: var(--text-on-accent);
} }
@@ -291,7 +242,7 @@ div.sls-setting-menu-btn {
display: none; display: none;
} }
.password-input > .setting-item-control > input { .password-input>.setting-item-control>input {
-webkit-text-security: disc; -webkit-text-security: disc;
} }
@@ -321,6 +272,7 @@ span.ls-mark-cr::after {
top: 0; top: 0;
left: 0; left: 0;
} }
.ls-imgdiff-wrap .overlay .img-overlay { .ls-imgdiff-wrap .overlay .img-overlay {
-webkit-filter: invert(100%) opacity(50%); -webkit-filter: invert(100%) opacity(50%);
filter: invert(100%) opacity(50%); filter: invert(100%) opacity(50%);
@@ -329,14 +281,61 @@ span.ls-mark-cr::after {
left: 0; left: 0;
animation: ls-blink-diff 0.5s cubic-bezier(0.4, 0, 1, 1) infinite alternate; animation: ls-blink-diff 0.5s cubic-bezier(0.4, 0, 1, 1) infinite alternate;
} }
@keyframes ls-blink-diff { @keyframes ls-blink-diff {
0% { 0% {
opacity: 0; opacity: 0;
} }
50% { 50% {
opacity: 0; opacity: 0;
} }
100% { 100% {
opacity: 1; opacity: 1;
} }
} }
.livesync-status {
user-select: none;
pointer-events: none;
height: auto;
min-height: 1em;
position: absolute;
background-color: transparent;
width: 100%;
padding: 10px;
padding-right: 16px;
top: var(--header-height);
z-index: calc(var(--layer-cover) + 1);
font-variant-numeric: tabular-nums;
font-variant-emoji: emoji;
tab-size: 4;
text-align: right;
white-space: pre-wrap;
display: inline-block;
color: var(--text-normal);
font-size: 80%;
}
.livesync-status div {
opacity: 0.6;
-webkit-filter: grayscale(100%);
filter: grayscale(100%);
}
.livesync-status .livesync-status-loghistory {
text-align: left;
opacity: 0.4;
}
.livesync-status div.livesync-status-messagearea {
opacity: 0.6;
color: var(--text-on-accent);
background: var(--background-modifier-error);
-webkit-filter: unset;
filter: unset;
}

111
updates.md
View File

@@ -18,46 +18,87 @@ I have a lot of respect for that plugin, even though it is sometimes treated as
Hooray for open source, and generous licences, and the sharing of knowledge by experts. Hooray for open source, and generous licences, and the sharing of knowledge by experts.
#### Version history #### Version history
- 0.23.14: - 0.23.22:
- Fixed: - Fixed:
- No longer batch-saving ignores editor inputs. - Case-insensitive file handling
- The file-watching and serialisation processes have been changed to the one which is similar to previous implementations. - Full-lower-case files are no longer created during database checking.
- We can configure the settings (Especially about text-boxes) even if we have configured the device name. - Bulk chunk transfer
- The default value will automatically adjust to an acceptable size when using IBM Cloudant.
- 0.23.21:
- New Features:
- Case-insensitive file handling
- Files can now be handled case-insensitively.
- This behaviour can be modified in the settings under `Handle files as Case-Sensitive` (Default: Prompt, Enabled for previous behaviour).
- Improved chunk revision fixing
- Revisions for chunks can now be fixed for faster chunk creation.
- This can be adjusted in the settings under `Compute revisions for chunks` (Default: Prompt, Enabled for previous behaviour).
- Bulk chunk transfer
- Chunks can now be transferred in bulk during uploads.
- This feature is enabled by default through `Send chunks in bulk`.
- Creation of missing chunks without
- Missing chunks can be created without storing notes, enhancing efficiency for first synchronisation or after prolonged periods without synchronisation.
- Improvements:
- File status scanning on the startup
- Quite significant performance improvements.
- No more missing scans of some files.
- Status in editor enhancements
- Significant performance improvements in the status display within the editor.
- Notifications for files that will not be synchronised will now be properly communicated.
- Encryption and Decryption
- These processes are now performed in background threads to ensure fast and stable transfers.
- Verify and repair all files
- Got faster through parallel checking.
- Migration on update
- Migration messages and wizards have become more helpful.
- Behavioural changes:
- Chunk size adjustments
- Large chunks will no longer be created for older, stable files, addressing storage consumption issues.
- Flag file automation
- Confirmation will be shown and we can cancel it.
- Fixed:
- Database File Scanning
- All files in the database will now be enumerated correctly.
- Miscellaneous
- Dependency updated.
- Now, tree shaking is left to terser, from esbuild.
- 0.23.20:
- Fixed:
- Customisation Sync now checks the difference while storing or applying the configuration.
- No longer storing the same configuration multiple times.
- Time difference in the dialogue has been fixed.
- Remote Storage Limit Notification dialogue has been fixed, now the chosen value is saved.
- Improved: - Improved:
- We can configure the delay of batch-saving. - The Enlarging button on the enlarging threshold dialogue now displays the new value.
- Default: 5 seconds, the same as the previous hard-coded value. (Note: also, the previous behaviour was not correct). - 0.23.19:
- Also, we can configure the limit of delaying batch-saving. - Not released.
- The performance of showing status indicators has been improved. - 0.23.18:
- 0.23.13: - New feature:
- Fixed: - Per-file-saved customization sync has been shipped.
- No longer files have been trimmed even delimiters have been continuous. - We can synchronise plug-igs etc., more smoothly.
- Fixed the toggle title to `Do not split chunks in the background` from `Do not split chunks in the foreground`. - Default: disabled. We need a small migration when enabling this. And all devices should be updated to v0.23.18. Once we enabled this, we lost compatibility with old versions.
- Non-configured item mismatches are no longer detected. - Customisation sync has got beta3.
- 0.23.12: - We can set `Flag` to each item to select the newest, automatically.
- This configuration is per device.
- Improved: - Improved:
- Now notes will be split into chunks in the background thread to improve smoothness. - Start-up speed has been improved.
- Default enabled, to disable, toggle `Do not split chunks in the foreground` on `Hatch` -> `Compatibility`.
- If you want to process very small notes in the foreground, please enable `Process small files in the foreground` on `Hatch` -> `Compatibility`.
- We can use a `splitting-limit-capped chunk splitter`; which performs more simple and make less amount of chunks.
- Default disabled, to enable, toggle `Use splitting-limit-capped chunk splitter` on `Sync settings` -> `Performance tweaks`
- Tidied
- Some files have been separated into multiple files to make them more explicit in what they are responsible for.
- 0.23.11:
- Fixed: - Fixed:
- Now we *surely* can set the device name and enable customised synchronisation. - On the customisation sync dialogue, buttons are kept within the screen.
- Unnecessary dialogue update processes have been eliminated. - No more unnecessary entries on `data.json` for customisation sync.
- Customisation sync no longer stores half-collected files. - Selections are no longer lost while updating customisation items.
- No longer hangs up when removing or renaming files with the `Sync on Save` toggle enabled. - Tidied on source codes:
- Many typos have been fixed.
- Some unnecessary type casting removed.
- 0.23.17:
- Improved: - Improved:
- Customisation sync now performs data deserialization more smoothly. - Overall performance has been improved by using PouchDB 9.0.0.
- New translations have been merged. - Configuration mismatch detection is refined. We can resolve mismatches more smoothly and naturally.
- 0.23.10 More detail is on `troubleshooting.md` on the repository.
- Fixed: - Fixed:
- No longer configurations have been locked in the minimal setup. - Customisation Sync will be disabled when a corrupted configuration is detected.
- 0.23.9 Therefore, the Device Name can be changed even in the event of a configuration mismatch.
- Fixed: - New feature:
- No longer unexpected parallel replication is performed. - We can get a notification about the storage usage of the remote database.
- Now we can set the device name and enable customised synchronisation again. - Default: We will be asked.
- If the remote storage usage approaches the configured value, we will be asked whether we want to Rebuild or increase the limit.
Older notes is in [updates_old.md](https://github.com/vrtmrz/obsidian-livesync/blob/main/updates_old.md). Older notes is in [updates_old.md](https://github.com/vrtmrz/obsidian-livesync/blob/main/updates_old.md).

49
updates_old.md
View File

@@ -18,6 +18,55 @@ I have a lot of respect for that plugin, even though it is sometimes treated as
Hooray for open source, and generous licences, and the sharing of knowledge by experts. Hooray for open source, and generous licences, and the sharing of knowledge by experts.
#### Version history #### Version history
- 0.23.16:
- Maintenance Update:
- Library refining (Phase 1 - step 2). There are no significant changes on the user side.
- Including the following fixes of potentially problems:
- the problem which the path had been obfuscating twice has been resolved.
- Note: Potential problems of the library; which has not happened in Self-hosted LiveSync for some reasons.
- 0.23.15:
- Maintenance Update:
- Library refining (Phase 1). There are no significant changes on the user side.
- 0.23.14:
- Fixed:
- No longer batch-saving ignores editor inputs.
- The file-watching and serialisation processes have been changed to the one which is similar to previous implementations.
- We can configure the settings (Especially about text-boxes) even if we have configured the device name.
- Improved:
- We can configure the delay of batch-saving.
- Default: 5 seconds, the same as the previous hard-coded value. (Note: also, the previous behaviour was not correct).
- Also, we can configure the limit of delaying batch-saving.
- The performance of showing status indicators has been improved.
- 0.23.13:
- Fixed:
- No longer files have been trimmed even delimiters have been continuous.
- Fixed the toggle title to `Do not split chunks in the background` from `Do not split chunks in the foreground`.
- Non-configured item mismatches are no longer detected.
- 0.23.12:
- Improved:
- Now notes will be split into chunks in the background thread to improve smoothness.
- Default enabled, to disable, toggle `Do not split chunks in the foreground` on `Hatch` -> `Compatibility`.
- If you want to process very small notes in the foreground, please enable `Process small files in the foreground` on `Hatch` -> `Compatibility`.
- We can use a `splitting-limit-capped chunk splitter`; which performs more simple and make less amount of chunks.
- Default disabled, to enable, toggle `Use splitting-limit-capped chunk splitter` on `Sync settings` -> `Performance tweaks`
- Tidied
- Some files have been separated into multiple files to make them more explicit in what they are responsible for.
- 0.23.11:
- Fixed:
- Now we *surely* can set the device name and enable customised synchronisation.
- Unnecessary dialogue update processes have been eliminated.
- Customisation sync no longer stores half-collected files.
- No longer hangs up when removing or renaming files with the `Sync on Save` toggle enabled.
- Improved:
- Customisation sync now performs data deserialization more smoothly.
- New translations have been merged.
- 0.23.10
- Fixed:
- No longer configurations have been locked in the minimal setup.
- 0.23.9
- Fixed:
- No longer unexpected parallel replication is performed.
- Now we can set the device name and enable customised synchronisation again.
- 0.23.8 - 0.23.8
- New feature: - New feature:
- Now we are ready for i18n. - Now we are ready for i18n.

2
utils/flyio/deploy-server.sh
View File

@@ -26,7 +26,7 @@ echo "OK!"
if command -v deno >/dev/null 2>&1; then if command -v deno >/dev/null 2>&1; then
echo "Setup finished! Also, we can set up Self-hosted LiveSync instantly, by the following setup uri." echo "Setup finished! Also, we can set up Self-hosted LiveSync instantly, by the following setup uri."
echo "Passphrase of setup-uri is \`welcome\`". echo "Passphrase of setup-uri will be printed only one time. Keep it safe!"
echo "--- configured ---" echo "--- configured ---"
echo "database : ${database}" echo "database : ${database}"
echo "E2EE passphrase: ${passphrase}" echo "E2EE passphrase: ${passphrase}"

156
utils/flyio/generate_setupuri.ts
View File

@@ -1,153 +1,13 @@
import { webcrypto } from "node:crypto"; import { encrypt } from "npm:octagonal-wheels@0.1.11/encryption/encryption.js";
const KEY_RECYCLE_COUNT = 100; const noun = ["waterfall", "river", "breeze", "moon", "rain", "wind", "sea", "morning", "snow", "lake", "sunset", "pine", "shadow", "leaf", "dawn", "glitter", "forest", "hill", "cloud", "meadow", "sun", "glade", "bird", "brook", "butterfly", "bush", "dew", "dust", "field", "fire", "flower", "firefly", "feather", "grass", "haze", "mountain", "night", "pond", "darkness", "snowflake", "silence", "sound", "sky", "shape", "surf", "thunder", "violet", "water", "wildflower", "wave", "water", "resonance", "sun", "log", "dream", "cherry", "tree", "fog", "frost", "voice", "paper", "frog", "smoke", "star"];
type KeyBuffer = { const adjectives = ["autumn", "hidden", "bitter", "misty", "silent", "empty", "dry", "dark", "summer", "icy", "delicate", "quiet", "white", "cool", "spring", "winter", "patient", "twilight", "dawn", "crimson", "wispy", "weathered", "blue", "billowing", "broken", "cold", "damp", "falling", "frosty", "green", "long", "late", "lingering", "bold", "little", "morning", "muddy", "old", "red", "rough", "still", "small", "sparkling", "thrumming", "shy", "wandering", "withered", "wild", "black", "young", "holy", "solitary", "fragrant", "aged", "snowy", "proud", "floral", "restless", "divine", "polished", "ancient", "purple", "lively", "nameless"];
key: CryptoKey; function friendlyString() {
salt: Uint8Array; return `${adjectives[Math.floor(Math.random() * adjectives.length)]}-${noun[Math.floor(Math.random() * noun.length)]}`;
count: number;
};
let semiStaticFieldBuffer: Uint8Array;
const nonceBuffer: Uint32Array = new Uint32Array(1);
const writeString = (string: string) => {
// Prepare enough buffer.
const buffer = new Uint8Array(string.length * 4);
const length = string.length;
let index = 0;
let chr = 0;
let idx = 0;
while (idx < length) {
chr = string.charCodeAt(idx++);
if (chr < 128) {
buffer[index++] = chr;
} else if (chr < 0x800) {
// 2 bytes
buffer[index++] = 0xC0 | (chr >>> 6);
buffer[index++] = 0x80 | (chr & 0x3F);
} else if (chr < 0xD800 || chr > 0xDFFF) {
// 3 bytes
buffer[index++] = 0xE0 | (chr >>> 12);
buffer[index++] = 0x80 | ((chr >>> 6) & 0x3F);
buffer[index++] = 0x80 | (chr & 0x3F);
} else {
// 4 bytes - surrogate pair
chr = (((chr - 0xD800) << 10) | (string.charCodeAt(idx++) - 0xDC00)) + 0x10000;
buffer[index++] = 0xF0 | (chr >>> 18);
buffer[index++] = 0x80 | ((chr >>> 12) & 0x3F);
buffer[index++] = 0x80 | ((chr >>> 6) & 0x3F);
buffer[index++] = 0x80 | (chr & 0x3F);
}
}
return buffer.slice(0, index);
};
const KeyBuffs = new Map<string, KeyBuffer>();
async function getKeyForEncrypt(passphrase: string, autoCalculateIterations: boolean): Promise<[CryptoKey, Uint8Array]> {
// For performance, the plugin reuses the key KEY_RECYCLE_COUNT times.
const buffKey = `${passphrase}-${autoCalculateIterations}`;
const f = KeyBuffs.get(buffKey);
if (f) {
f.count--;
if (f.count > 0) {
return [f.key, f.salt];
}
f.count--;
}
const passphraseLen = 15 - passphrase.length;
const iteration = autoCalculateIterations ? ((passphraseLen > 0 ? passphraseLen : 0) * 1000) + 121 - passphraseLen : 100000;
const passphraseBin = new TextEncoder().encode(passphrase);
const digest = await webcrypto.subtle.digest({ name: "SHA-256" }, passphraseBin);
const keyMaterial = await webcrypto.subtle.importKey("raw", digest, { name: "PBKDF2" }, false, ["deriveKey"]);
const salt = webcrypto.getRandomValues(new Uint8Array(16));
const key = await webcrypto.subtle.deriveKey(
{
name: "PBKDF2",
salt,
iterations: iteration,
hash: "SHA-256",
},
keyMaterial,
{ name: "AES-GCM", length: 256 },
false,
["encrypt"]
);
KeyBuffs.set(buffKey, {
key,
salt,
count: KEY_RECYCLE_COUNT,
});
return [key, salt];
} }
function getSemiStaticField(reset?: boolean) { const uri_passphrase = `${Deno.env.get("uri_passphrase") ?? friendlyString()}`;
// return fixed field of iv.
if (semiStaticFieldBuffer != null && !reset) {
return semiStaticFieldBuffer;
}
semiStaticFieldBuffer = webcrypto.getRandomValues(new Uint8Array(12));
return semiStaticFieldBuffer;
}
function getNonce() {
// This is nonce, so do not send same thing.
nonceBuffer[0]++;
if (nonceBuffer[0] > 10000) {
// reset semi-static field.
getSemiStaticField(true);
}
return nonceBuffer;
}
function arrayBufferToBase64internalBrowser(buffer: DataView | Uint8Array): Promise<string> {
return new Promise((res, rej) => {
const blob = new Blob([buffer], { type: "application/octet-binary" });
const reader = new FileReader();
reader.onload = function (evt) {
const dataURI = evt.target?.result?.toString() || "";
if (buffer.byteLength != 0 && (dataURI == "" || dataURI == "data:")) return rej(new TypeError("Could not parse the encoded string"));
const result = dataURI.substring(dataURI.indexOf(",") + 1);
res(result);
};
reader.readAsDataURL(blob);
});
}
// Map for converting hexString
const revMap: { [key: string]: number } = {};
const numMap: { [key: number]: string } = {};
for (let i = 0; i < 256; i++) {
revMap[(`00${i.toString(16)}`.slice(-2))] = i;
numMap[i] = (`00${i.toString(16)}`.slice(-2));
}
function uint8ArrayToHexString(src: Uint8Array): string {
return [...src].map(e => numMap[e]).join("");
}
const QUANTUM = 32768;
async function arrayBufferToBase64Single(buffer: ArrayBuffer): Promise<string> {
const buf = buffer instanceof Uint8Array ? buffer : new Uint8Array(buffer);
if (buf.byteLength < QUANTUM) return btoa(String.fromCharCode.apply(null, [...buf]));
return await arrayBufferToBase64internalBrowser(buf);
}
export async function encrypt(input: string, passphrase: string, autoCalculateIterations: boolean) {
const [key, salt] = await getKeyForEncrypt(passphrase, autoCalculateIterations);
// Create initial vector with semi-fixed part and incremental part
// I think it's not good against related-key attacks.
const fixedPart = getSemiStaticField();
const invocationPart = getNonce();
const iv = new Uint8Array([...fixedPart, ...new Uint8Array(invocationPart.buffer)]);
const plainStringified = JSON.stringify(input);
// const plainStringBuffer: Uint8Array = tex.encode(plainStringified)
const plainStringBuffer: Uint8Array = writeString(plainStringified);
const encryptedDataArrayBuffer = await webcrypto.subtle.encrypt({ name: "AES-GCM", iv }, key, plainStringBuffer);
const encryptedData2 = (await arrayBufferToBase64Single(encryptedDataArrayBuffer));
//return data with iv and salt.
const ret = `["${encryptedData2}","${uint8ArrayToHexString(iv)}","${uint8ArrayToHexString(salt)}"]`;
return ret;
}
const URIBASE = "obsidian://setuplivesync?settings="; const URIBASE = "obsidian://setuplivesync?settings=";
async function main() { async function main() {
@@ -173,8 +33,10 @@ async function main() {
"concurrencyOfReadChunksOnline": 100, "concurrencyOfReadChunksOnline": 100,
"minimumIntervalOfReadChunksOnline": 100, "minimumIntervalOfReadChunksOnline": 100,
} }
const encryptedConf = encodeURIComponent(await encrypt(JSON.stringify(conf), "welcome", false)); const encryptedConf = encodeURIComponent(await encrypt(JSON.stringify(conf), uri_passphrase, false));
const theURI = `${URIBASE}${encryptedConf}`; const theURI = `${URIBASE}${encryptedConf}`;
console.log(theURI); console.log(theURI);
console.log("\nYour passphrase of Setup-URI is: ", uri_passphrase);
console.log("This passphrase is never shown again, so please note it in a safe place.")
} }
await main(); await main();

5
utils/readme.md
View File

@@ -129,12 +129,15 @@ curl: (35) OpenSSL SSL_connect: Connection reset by peer in connection to young-
<-- Configuring CouchDB by REST APIs Done! <-- Configuring CouchDB by REST APIs Done!
OK! OK!
Setup finished! Also, we can set up Self-hosted LiveSync instantly, by the following setup uri. Setup finished! Also, we can set up Self-hosted LiveSync instantly, by the following setup uri.
Passphrase of setup-uri is `welcome`. Passphrase of setup-uri will be printed only one time. Keep it safe!
--- configured --- --- configured ---
database : obsidiannotes database : obsidiannotes
E2EE passphrase: dark-wildflower-26467 E2EE passphrase: dark-wildflower-26467
--- setup uri --- --- setup uri ---
obsidian://setuplivesync?settings=%5B%22gZkBwjFbLqxbdSIbJymU%2FmTPBPAKUiHVGDRKYiNnKhW0auQeBgJOfvnxexZtMCn8sNiIUTAlxNaMGF2t%2BCEhpJoeCP%2FO%2BrwfN5LaNDQyky1Uf7E%2B64A5UWyjOYvZDOgq4iCKSdBAXp9oO%2BwKh4MQjUZ78vIVvJp8Mo6NWHfm5fkiWoAoddki1xBMvi%2BmmN%2FhZatQGcslVb9oyYWpZocduTl0a5Dv%2FQviGwlYQ%2F4NY0dVDIoOdvaYS%2FX4GhNAnLzyJKMXhPEJHo9FvR%2FEOBuwyfMdftV1SQUZ8YDCuiR3T7fh7Kn1c6OFgaFMpFm%2BWgIJ%2FZpmAyhZFpEcjpd7ty%2BN9kfd9gQsZM4%2BYyU9OwDd2DahVMBWkqoV12QIJ8OlJScHHdcUfMW5ex%2F4UZTWKNEHJsigITXBrtq11qGk3rBfHys8O0vY6sz%2FaYNM3iAOsR1aoZGyvwZm4O6VwtzK8edg0T15TL4O%2B7UajQgtCGxgKNYxb8EMOGeskv7NifYhjCWcveeTYOJzBhnIDyRbYaWbkAXQgHPBxzJRkkG%2FpBPfBBoJarj7wgjMvhLJ9xtL4FbP6sBNlr8jtAUCoq4L7LJcRNF4hlgvjJpL2BpFZMzkRNtUBcsRYR5J%2BM1X2buWi2BHncbSiRRDKEwNOQkc%2FmhMJjbAn%2F8eNKRuIICOLD5OvxD7FZNCJ0R%2BWzgrzcNV%22%2C%22ec7edc900516b4fcedb4c7cc01000000%22%2C%22fceb5fe54f6619ee266ed9a887634e07%22%5D obsidian://setuplivesync?settings=%5B%22gZkBwjFbLqxbdSIbJymU%2FmTPBPAKUiHVGDRKYiNnKhW0auQeBgJOfvnxexZtMCn8sNiIUTAlxNaMGF2t%2BCEhpJoeCP%2FO%2BrwfN5LaNDQyky1Uf7E%2B64A5UWyjOYvZDOgq4iCKSdBAXp9oO%2BwKh4MQjUZ78vIVvJp8Mo6NWHfm5fkiWoAoddki1xBMvi%2BmmN%2FhZatQGcslVb9oyYWpZocduTl0a5Dv%2FQviGwlYQ%2F4NY0dVDIoOdvaYS%2FX4GhNAnLzyJKMXhPEJHo9FvR%2FEOBuwyfMdftV1SQUZ8YDCuiR3T7fh7Kn1c6OFgaFMpFm%2BWgIJ%2FZpmAyhZFpEcjpd7ty%2BN9kfd9gQsZM4%2BYyU9OwDd2DahVMBWkqoV12QIJ8OlJScHHdcUfMW5ex%2F4UZTWKNEHJsigITXBrtq11qGk3rBfHys8O0vY6sz%2FaYNM3iAOsR1aoZGyvwZm4O6VwtzK8edg0T15TL4O%2B7UajQgtCGxgKNYxb8EMOGeskv7NifYhjCWcveeTYOJzBhnIDyRbYaWbkAXQgHPBxzJRkkG%2FpBPfBBoJarj7wgjMvhLJ9xtL4FbP6sBNlr8jtAUCoq4L7LJcRNF4hlgvjJpL2BpFZMzkRNtUBcsRYR5J%2BM1X2buWi2BHncbSiRRDKEwNOQkc%2FmhMJjbAn%2F8eNKRuIICOLD5OvxD7FZNCJ0R%2BWzgrzcNV%22%2C%22ec7edc900516b4fcedb4c7cc01000000%22%2C%22fceb5fe54f6619ee266ed9a887634e07%22%5D
Your passphrase of Setup-URI is: patient-haze
This passphrase is never shown again, so please note it in a safe place.
``` ```
All we have to do is copy the setup-URI (`obsidian`://...`) and open it from Self-hosted LiveSync on Obsidian. All we have to do is copy the setup-URI (`obsidian`://...`) and open it from Self-hosted LiveSync on Obsidian.