public/obsidian-livesync
1
0
Fork 0
mirror of https://github.com/vrtmrz/obsidian-livesync.git synced 2026-05-15 12:01:16 +00:00
Code Issues Packages Projects Releases Wiki Activity

Compare commits

base: public:0.22.8
Branches Tags
public:main public:enhance_fix_p2p public:test_real_obsidian public:v0_25_62 public:update_lib public:feat-userhashsalt public:auto_conf_2 public:auto_conf public:beta public:alpha public:refactor_services_2 public:refactor_services public:dev public:svelteui public:forprivacy
public:0.25.62 public:0.25.61 public:0.25.60 public:0.25.59 public:0.25.58 public:0.25.57 public:0.25.56+patched5 public:0.25.56+patched4 public:0.25.56+patched3 public:0.25.56+patched2 public:0.25.56+patched1 public:0.25.56 public:0.25.55 public:0.25.54 public:0.25.53 public:0.25.52-patched-3 public:0.25.52-patched-2 public:0.25.52-patched-1 public:0.25.52 public:0.25.51 public:0.25.50 public:0.25.49 public:0.25.48 public:0.25.47 public:0.25.46 public:0.25.45 public:0.25.44 public:0.25.43-patched-9 public:0.25.43-patched-8 public:0.25.43-patched-7 public:0.25.43-patched-6 public:0.25.43-patched-5 public:0.25.43-patched-4 public:0.25.43-patched-3 public:0.25.43-patched-2 public:0.25.43-patched-1 public:0.25.43 public:0.25.42 public:0.25.41-patched-3 public:0.25.41-patched-2 public:0.25.41-patched-1 public:0.25.41 public:0.25.40 public:0.25.39 public:0.25.38 public:0.25.37 public:0.25.36 public:0.25.35 public:0.25.34 public:0.25.33 public:0.25.32 public:0.25.31 public:0.25.30 public:0.25.30-dev public:0.25.29 public:0.25.28 public:0.25.27 public:0.25.26 public:0.25.25 public:0.25.24 public:0.25.24.beta3 public:0.25.24.beta2 public:0.25.24.beta1 public:0.25.23 public:0.25.23.beta1 public:0.25.22 public:0.25.21 public:0.25.21.beta2 public:0.25.21.beta1 public:0.25.20 public:0.25.19 public:0.25.18 public:0.25.17 public:0.25.16 public:0.25.15 public:0.25.14 public:0.25.13 public:0.25.12 public:0.25.11 public:0.25.10 public:0.25.9 public:0.25.8 public:0.25.7 public:0.25.6 public:0.25.5 public:0.25.4 public:0.25.3 public:0.25.2 public:0.25.1 public:0.25.0 public:0.25.0-beta1 public:0.24.31 public:0.24.30 public:0.24.29 public:0.24.28 public:0.24.27 public:0.24.26 public:0.24.25 public:0.24.24 public:0.24.23 public:0.24.22 public:0.24.21 public:0.24.20 public:0.24.19 public:0.24.18 public:0.24.17 public:0.24.16 public:0.24.15 public:0.24.14 public:0.24.13 public:0.24.12 public:0.24.11 public:0.24.10 public:0.24.9 public:0.24.8 public:0.24.7 public:0.24.6 public:0.24.5 public:0.24.4 public:0.24.3 public:0.24.2 public:0.24.1 public:0.24.0 public:0.24.0.dev-rc8 public:0.24.0.dev-rc7 public:0.24.0.dev-rc6 public:0.24.0.dev-rc5 public:0.24.0.dev-rc4 public:0.24.0.dev-rc3 public:0.24.0.dev-rc2 public:0.23.23 public:0.23.22 public:0.23.21 public:0.23.20 public:0.23.19 public:0.23.18 public:0.23.17 public:0.23.16 public:0.23.15 public:0.23.14 public:0.23.13 public:0.23.12 public:0.23.11 public:0.23.10 public:0.23.9 public:0.23.8 public:0.23.7 public:0.23.6 public:0.23.5 public:0.23.4 public:0.23.3 public:0.23.2 public:0.23.1 public:0.23.0 public:0.22.19 public:0.22.18 public:0.22.17 public:0.22.16 public:0.22.15 public:0.22.14 public:0.22.13 public:0.22.12 public:0.22.11 public:0.22.10 public:0.22.9 public:0.22.8 public:0.22.7 public:0.22.6 public:0.22.5 public:0.22.4 public:0.22.3 public:0.22.2 public:0.22.1 public:0.22.0 public:0.21.5 public:0.21.4 public:0.21.3 public:0.21.2 public:0.21.1 public:0.21.0 public:0.20.7 public:0.20.6 public:0.20.5 public:0.20.4 public:0.20.3 public:0.20.2 public:0.20.1 public:0.20.0 public:0.19.23 public:0.19.22 public:0.19.21 public:0.19.20 public:0.19.19 public:0.19.18 public:0.19.17 public:0.19.16 public:0.19.15 public:0.19.14 public:0.19.13 public:0.19.12 public:0.19.11 public:0.19.10 public:0.19.9 public:0.19.8 public:0.19.7 public:0.19.6 public:0.19.5 public:0.19.4 public:0.19.3 public:0.19.2 public:0.19.1 public:0.19.0 public:0.18.6 public:0.18.5 public:0.18.4 public:0.18.3 public:0.18.2 public:0.18.1 public:0.18.0 public:0.17.34 public:0.17.33 public:0.17.32 public:0.17.31 public:0.17.30 public:0.17.29 public:0.17.28 public:0.17.27 public:0.17.26 public:0.17.25 public:0.17.24 public:0.17.23 public:0.17.22 public:0.17.21 public:0.17.20 public:0.17.19 public:0.17.18 public:0.17.17 public:0.17.16 public:0.17.15 public:0.17.14 public:0.17.13 public:0.17.12 public:0.17.11 public:0.17.10 public:0.17.9 public:0.17.8 public:0.17.7 public:0.17.6 public:0.17.5 public:0.17.4 public:0.17.3 public:0.17.2 public:0.17.1 public:0.17.0 public:0.16.8 public:0.16.7 public:0.16.6 public:0.16.5 public:0.16.4 public:0.16.3 public:0.16.2 public:0.16.1 public:0.16.0 public:0.15.10 public:0.15.9 public:0.15.8 public:0.15.7 public:0.15.6 public:0.15.5 public:0.15.4 public:0.15.3 public:0.15.2 public:0.15.1 public:0.15.0 public:0.14.8 public:0.14.7 public:0.14.6 public:0.14.5 public:0.14.4 public:0.14.3 public:0.14.2 public:0.14.1 public:0.14.0 public:0.13.4 public:0.13.3 public:0.13.2 public:0.13.1 public:0.13.0 public:0.12.5 public:0.12.4 public:0.12.3 public:0.12.2 public:0.12.1 public:0.12.0 public:0.11.10 public:0.11.9 public:0.11.8 public:0.11.7 public:0.11.6 public:0.11.5 public:0.11.4 public:0.11.3 public:0.11.2 public:0.11.1 public:0.11.0 public:0.10.1 public:0.10.0 public:0.9.0 public:0.8.8 public:0.8.7 public:0.8.6 public:0.8.5 public:0.8.3 public:0.8.2 public:0.8.1 public:0.8.0 public:0.7.2 public:0.7.1 public:0.7.0 public:0.6.1 public:0.6.0 public:0.5.0 public:0.4.1 public:0.4.0 public:0.3.1 public:0.3.0 public:0.2.1 public:0.2.0 public:0.1.26 public:0.1.25 public:0.1.24 public:0.1.23 public:0.1.22 public:0.1.21 public:0.1.20 public:0.1.19 public:0.1.18 public:0.1.17 public:0.1.16 public:0.1.15 public:0.1.14 public:0.1.13 public:0.1.12 public:0.1.11 public:0.1.10 public:0.1.9 public:0.1.8 public:0.1.7 public:0.1.6 public:0.1.5 public:0.1.4 public:0.1.3 public:0.1.2 public:0.1.1 public:0.1.0 public:0.0.9 public:0.0.8 public:0.0.7 public:0.0.6
...
compare: public:0.22.17
Branches Tags
public:enhance_fix_p2p public:test_real_obsidian public:main public:v0_25_62 public:update_lib public:feat-userhashsalt public:auto_conf_2 public:auto_conf public:beta public:alpha public:refactor_services_2 public:refactor_services public:dev public:svelteui public:forprivacy
public:0.25.62 public:0.25.61 public:0.25.60 public:0.25.59 public:0.25.58 public:0.25.57 public:0.25.56+patched5 public:0.25.56+patched4 public:0.25.56+patched3 public:0.25.56+patched2 public:0.25.56+patched1 public:0.25.56 public:0.25.55 public:0.25.54 public:0.25.53 public:0.25.52-patched-3 public:0.25.52-patched-2 public:0.25.52-patched-1 public:0.25.52 public:0.25.51 public:0.25.50 public:0.25.49 public:0.25.48 public:0.25.47 public:0.25.46 public:0.25.45 public:0.25.44 public:0.25.43-patched-9 public:0.25.43-patched-8 public:0.25.43-patched-7 public:0.25.43-patched-6 public:0.25.43-patched-5 public:0.25.43-patched-4 public:0.25.43-patched-3 public:0.25.43-patched-2 public:0.25.43-patched-1 public:0.25.43 public:0.25.42 public:0.25.41-patched-3 public:0.25.41-patched-2 public:0.25.41-patched-1 public:0.25.41 public:0.25.40 public:0.25.39 public:0.25.38 public:0.25.37 public:0.25.36 public:0.25.35 public:0.25.34 public:0.25.33 public:0.25.32 public:0.25.31 public:0.25.30 public:0.25.30-dev public:0.25.29 public:0.25.28 public:0.25.27 public:0.25.26 public:0.25.25 public:0.25.24 public:0.25.24.beta3 public:0.25.24.beta2 public:0.25.24.beta1 public:0.25.23 public:0.25.23.beta1 public:0.25.22 public:0.25.21 public:0.25.21.beta2 public:0.25.21.beta1 public:0.25.20 public:0.25.19 public:0.25.18 public:0.25.17 public:0.25.16 public:0.25.15 public:0.25.14 public:0.25.13 public:0.25.12 public:0.25.11 public:0.25.10 public:0.25.9 public:0.25.8 public:0.25.7 public:0.25.6 public:0.25.5 public:0.25.4 public:0.25.3 public:0.25.2 public:0.25.1 public:0.25.0 public:0.25.0-beta1 public:0.24.31 public:0.24.30 public:0.24.29 public:0.24.28 public:0.24.27 public:0.24.26 public:0.24.25 public:0.24.24 public:0.24.23 public:0.24.22 public:0.24.21 public:0.24.20 public:0.24.19 public:0.24.18 public:0.24.17 public:0.24.16 public:0.24.15 public:0.24.14 public:0.24.13 public:0.24.12 public:0.24.11 public:0.24.10 public:0.24.9 public:0.24.8 public:0.24.7 public:0.24.6 public:0.24.5 public:0.24.4 public:0.24.3 public:0.24.2 public:0.24.1 public:0.24.0 public:0.24.0.dev-rc8 public:0.24.0.dev-rc7 public:0.24.0.dev-rc6 public:0.24.0.dev-rc5 public:0.24.0.dev-rc4 public:0.24.0.dev-rc3 public:0.24.0.dev-rc2 public:0.23.23 public:0.23.22 public:0.23.21 public:0.23.20 public:0.23.19 public:0.23.18 public:0.23.17 public:0.23.16 public:0.23.15 public:0.23.14 public:0.23.13 public:0.23.12 public:0.23.11 public:0.23.10 public:0.23.9 public:0.23.8 public:0.23.7 public:0.23.6 public:0.23.5 public:0.23.4 public:0.23.3 public:0.23.2 public:0.23.1 public:0.23.0 public:0.22.19 public:0.22.18 public:0.22.17 public:0.22.16 public:0.22.15 public:0.22.14 public:0.22.13 public:0.22.12 public:0.22.11 public:0.22.10 public:0.22.9 public:0.22.8 public:0.22.7 public:0.22.6 public:0.22.5 public:0.22.4 public:0.22.3 public:0.22.2 public:0.22.1 public:0.22.0 public:0.21.5 public:0.21.4 public:0.21.3 public:0.21.2 public:0.21.1 public:0.21.0 public:0.20.7 public:0.20.6 public:0.20.5 public:0.20.4 public:0.20.3 public:0.20.2 public:0.20.1 public:0.20.0 public:0.19.23 public:0.19.22 public:0.19.21 public:0.19.20 public:0.19.19 public:0.19.18 public:0.19.17 public:0.19.16 public:0.19.15 public:0.19.14 public:0.19.13 public:0.19.12 public:0.19.11 public:0.19.10 public:0.19.9 public:0.19.8 public:0.19.7 public:0.19.6 public:0.19.5 public:0.19.4 public:0.19.3 public:0.19.2 public:0.19.1 public:0.19.0 public:0.18.6 public:0.18.5 public:0.18.4 public:0.18.3 public:0.18.2 public:0.18.1 public:0.18.0 public:0.17.34 public:0.17.33 public:0.17.32 public:0.17.31 public:0.17.30 public:0.17.29 public:0.17.28 public:0.17.27 public:0.17.26 public:0.17.25 public:0.17.24 public:0.17.23 public:0.17.22 public:0.17.21 public:0.17.20 public:0.17.19 public:0.17.18 public:0.17.17 public:0.17.16 public:0.17.15 public:0.17.14 public:0.17.13 public:0.17.12 public:0.17.11 public:0.17.10 public:0.17.9 public:0.17.8 public:0.17.7 public:0.17.6 public:0.17.5 public:0.17.4 public:0.17.3 public:0.17.2 public:0.17.1 public:0.17.0 public:0.16.8 public:0.16.7 public:0.16.6 public:0.16.5 public:0.16.4 public:0.16.3 public:0.16.2 public:0.16.1 public:0.16.0 public:0.15.10 public:0.15.9 public:0.15.8 public:0.15.7 public:0.15.6 public:0.15.5 public:0.15.4 public:0.15.3 public:0.15.2 public:0.15.1 public:0.15.0 public:0.14.8 public:0.14.7 public:0.14.6 public:0.14.5 public:0.14.4 public:0.14.3 public:0.14.2 public:0.14.1 public:0.14.0 public:0.13.4 public:0.13.3 public:0.13.2 public:0.13.1 public:0.13.0 public:0.12.5 public:0.12.4 public:0.12.3 public:0.12.2 public:0.12.1 public:0.12.0 public:0.11.10 public:0.11.9 public:0.11.8 public:0.11.7 public:0.11.6 public:0.11.5 public:0.11.4 public:0.11.3 public:0.11.2 public:0.11.1 public:0.11.0 public:0.10.1 public:0.10.0 public:0.9.0 public:0.8.8 public:0.8.7 public:0.8.6 public:0.8.5 public:0.8.3 public:0.8.2 public:0.8.1 public:0.8.0 public:0.7.2 public:0.7.1 public:0.7.0 public:0.6.1 public:0.6.0 public:0.5.0 public:0.4.1 public:0.4.0 public:0.3.1 public:0.3.0 public:0.2.1 public:0.2.0 public:0.1.26 public:0.1.25 public:0.1.24 public:0.1.23 public:0.1.22 public:0.1.21 public:0.1.20 public:0.1.19 public:0.1.18 public:0.1.17 public:0.1.16 public:0.1.15 public:0.1.14 public:0.1.13 public:0.1.12 public:0.1.11 public:0.1.10 public:0.1.9 public:0.1.8 public:0.1.7 public:0.1.6 public:0.1.5 public:0.1.4 public:0.1.3 public:0.1.2 public:0.1.1 public:0.1.0 public:0.0.9 public:0.0.8 public:0.0.7 public:0.0.6

27 Commits
0.22.8 ... 0.22.17

Author SHA1 Message Date
vorotamoroz
f0b202bd91 bump 2024-04-12 01:32:03 +09:00
vorotamoroz
d54b7e2d93 - Fixed: 
- Error handling on booting now works fine.
  - Replication is now started automatically in LiveSync mode.
  - Batch database update is now disabled in LiveSync mode.
  - No longer automatically reconnection while off-focused.
  - Status saves are thinned out.
  - Now Self-hosted LiveSync waits for all files between the local database and storage to be surely checked.
- Improved:
  - The job scheduler is now more robust and stable.
  - The status indicator no longer flickers and keeps zero for a while.
  - No longer meaningless frequent updates of status indicators.
  - Now we can configure regular expression filters in handy UI. Thank you so much, @eth-p!
  - `Fetch` or `Rebuild everything` is now more safely performed.
- Minor things
  - Some utility function has been added.
  - Customisation sync now less wrong messages.
  - Digging the weeds for eradication of type errors.
 2024-04-12 01:30:35 +09:00
vorotamoroz
6952ef37f5 Update quick_setup.md 2024-04-09 13:10:31 +09:00
vorotamoroz
9630bcbae8 bump 2024-03-22 10:50:03 +01:00
vorotamoroz
c3f925ab9a Merge branch 'main' of https://github.com/vrtmrz/obsidian-livesync 2024-03-22 10:48:25 +01:00
vorotamoroz
034dc0538f - Fixed: 
- Fixed the issue that binary files were sometimes corrupted.
  - Fixed customisation sync data could be corrupted.
- Improved:
  - Now the remote database costs lower memory.
    - This release requires a brief wait on the first synchronisation, to track the latest changeset again.
  - Description added for the `Device name`.
- Refactored:
  - Many type-errors have been resolved.
  - Obsolete file has been deleted.
 2024-03-22 10:48:16 +01:00
vorotamoroz
b6136df836 Update quick_setup.md 2024-03-22 14:27:34 +09:00
vorotamoroz
24aacdc2a1 bump 2024-03-22 04:07:17 +01:00
vorotamoroz
f91109b1ad - Improved: 
- Faster start-up by removing too many logs which indicates normality
  - By streamlined scanning of customised synchronisation extra phases have been deleted.
 2024-03-22 04:07:07 +01:00
vorotamoroz
e76e7ae8ea bump 2024-03-19 17:59:38 +01:00
vorotamoroz
f7fbe85d65 - New feature: 
- We can disable the status bar in the setting dialogue.
- Improved:
  - Now some files are handled as correct data type.
  - Customisation sync now uses the digest of each file for better performance.
  - The status in the Editor now works performant.
- Refactored:
  - Common functions have been ready and the codebase has been organised.
  - Stricter type checking following TypeScript updates.
  - Remove old iOS workaround for simplicity and performance.
 2024-03-19 17:58:55 +01:00
vorotamoroz
0313443b29 Merge pull request #389 from Seeker0472/fix-command 
Fixed docker-compose command in docs
 2024-03-19 14:06:23 +09:00
seeker0472
755c30f468 fix docker-compose command 2024-03-17 14:30:35 +08:00
vorotamoroz
b00b0cc5e5 bump 2024-03-15 10:37:15 +01:00
vorotamoroz
d7985a6b41 Improved: 
- Now using HTTP for the remote database URI warns of an error (on mobile) or notice (on desktop).
 2024-03-15 10:36:00 +01:00
vorotamoroz
486e816902 Update dependencies 2024-03-15 10:35:41 +01:00
vorotamoroz
ef9b19c24b bump 2024-03-04 04:07:51 +00:00
vorotamoroz
4ed9494176 Changed: 
- The default settings has been changed.
Improved:
- Default and preferred settings are applied on completion of the wizard.
Fixed:
- Now Initialisation `Fetch` will be performed smoothly and there will be fewer conflicts.
- No longer stuck while Handling transferred or initialised documents.
 2024-03-04 04:07:11 +00:00
vorotamoroz
fcd56d59d5 bump 2024-03-01 08:33:37 +00:00
vorotamoroz
1cabfcfd19 Fixed: 
- `Verify and repair all files` is no longer broken.
New feature::
- Now `Verify and repair all files` can restore or show history
Improved:
- Performance improved
 2024-03-01 08:32:48 +00:00
vorotamoroz
37a18dbfef bump 2024-03-01 03:28:46 +00:00
vorotamoroz
e7edf88713 Fixed 
- No longer unchanged hidden files and customisations are saved and transferred now.
- File integrity of vault history indicates the integrity correctly.
Improved
- In the report, the schema of the remote database URI is now printed.
 2024-03-01 03:28:06 +00:00
vorotamoroz
90ff75ab35 add notes. 2024-02-29 00:30:07 +00:00
vorotamoroz
bff1d661f5 Update troubleshooting.md 
Fix grammar
 2024-02-29 00:42:28 +09:00
vorotamoroz
6b59c14774 Update doc 2024-02-28 08:29:06 +00:00
vorotamoroz
8249274eac bump 2024-02-28 08:28:07 +00:00
vorotamoroz
3c6dae7814 - Fixed: 
- Fixed a bug on `fetch chunks on demand` that could not fetch the chunks on demand.
- Improved:
  - `fetch chunks on demand` works more smoothly.
  - Initialisation `Fetch` is now more efficient.
- Tidied:
  - Removed some meaningless codes.
 2024-02-28 08:27:17 +00:00
27 changed files with 2659 additions and 2330 deletions
Expand all files Collapse all files

4
docs/quick_setup.md
View File

@@ -16,7 +16,8 @@ There are three methods to set up Self-hosted LiveSync.
### 1. Using setup URIs ### 1. Using setup URIs
> [!TIP] What is the setup URI? Why is it required? > [!TIP]
> What is the setup URI? Why is it required?
> The setup URI is the encrypted representation of Self-hosted LiveSync configuration as a URI. This starts `obsidian://setuplivesync?settings=`. This is encrypted with a passphrase, so that it can be shared relatively securely between devices. It is a bit long, but it is one line. This allows a series of settings to be set at once without any inconsistencies. > The setup URI is the encrypted representation of Self-hosted LiveSync configuration as a URI. This starts `obsidian://setuplivesync?settings=`. This is encrypted with a passphrase, so that it can be shared relatively securely between devices. It is a bit long, but it is one line. This allows a series of settings to be set at once without any inconsistencies.
> >
> If you have configured the remote database by [Automated setup on Fly.io](./setup_flyio.md#a-very-automated-setup) or [set up your server with the tool](./setup_own_server.md#1-generate-the-setup-uri-on-a-desktop-device-or-server), **you should have one of them** > If you have configured the remote database by [Automated setup on Fly.io](./setup_flyio.md#a-very-automated-setup) or [set up your server with the tool](./setup_own_server.md#1-generate-the-setup-uri-on-a-desktop-device-or-server), **you should have one of them**
@@ -54,6 +55,7 @@ If you do not have any setup URI, Press the `start` button. The setting dialogue
#### Test database connection and Check database configuration #### Test database connection and Check database configuration
We can check the connectivity to the database, and the database settings. We can check the connectivity to the database, and the database settings.
![](../images/quick_setup_5.png) ![](../images/quick_setup_5.png)
#### Check and Fix database configuration #### Check and Fix database configuration

2
docs/setup_own_server_cn.md
View File

@@ -91,7 +91,7 @@ services:
最后, 创建并启动容器: 最后, 创建并启动容器:
``` ```
# -d will launch detached so the container runs in background # -d will launch detached so the container runs in background
docker compose up -d docker-compose up -d
``` ```
## 创建数据库 ## 创建数据库

69
docs/troubleshooting.md
View File

@@ -1,10 +1,25 @@
<!-- 2024-02-15 --> <!-- 2024-02-15 -->
# Tips and Troubleshooting # Tips and Troubleshooting
- [Notable bugs and fixes](#notable-bugs-and-fixes)
- [FAQ](#faq) - [Tips and Troubleshooting](#tips-and-troubleshooting)
- [Troubleshooting](#troubleshooting) - [Notable bugs and fixes](#notable-bugs-and-fixes)
- [Tips](#tips) - [Binary files get bigger on iOS](#binary-files-get-bigger-on-ios)
- [Some setting name has been changed](#some-setting-name-has-been-changed)
- [FAQ](#faq)
- [Why `Use an old adapter for compatibility` is somehow enabled in my vault?](#why-use-an-old-adapter-for-compatibility-is-somehow-enabled-in-my-vault)
- [ZIP (or any extensions) files were not synchronised. Why?](#zip-or-any-extensions-files-were-not-synchronised-why)
- [I hope to report the issue, but you said you needs `Report`. How to make it?](#i-hope-to-report-the-issue-but-you-said-you-needs-report-how-to-make-it)
- [Where can I check the log?](#where-can-i-check-the-log)
- [Why are the logs volatile and ephemeral?](#why-are-the-logs-volatile-and-ephemeral)
- [Some network logs are not written into the file.](#some-network-logs-are-not-written-into-the-file)
- [If a file were deleted or trimmed, the capacity of the database should be reduced, right?](#if-a-file-were-deleted-or-trimmed-the-capacity-of-the-database-should-be-reduced-right)
- [Troubleshooting](#troubleshooting)
- [On the mobile device, cannot synchronise on the local network!](#on-the-mobile-device-cannot-synchronise-on-the-local-network)
- [I think that something bad happening on the vault...](#i-think-that-something-bad-happening-on-the-vault)
- [Tips](#tips)
- [Old tips](#old-tips)
<!-- - --> <!-- - -->
@@ -39,20 +54,56 @@ When you rebuild everything or fetch from the remote again, you will be asked to
Therefore, experienced users (especially those stable enough not to have to rebuild the database) may have this toggle enabled in their Vault. Therefore, experienced users (especially those stable enough not to have to rebuild the database) may have this toggle enabled in their Vault.
Please disable it when you have enough time. Please disable it when you have enough time.
### ZIP (or any extensions) files were not synchronised. Why?
It depends on Obsidian detects. May toggling `Detect all extensions` of `File and links` (setting of Obsidian) will help us.
### I hope to report the issue, but you said you needs `Report`. How to make it?
We can copy the report to the clipboard, by pressing the `Make report` button on the `Hatch` pane.
![Screenshot](../images/hatch.png)
### Where can I check the log?
We can launch the log pane by `Show log` on the command palette.
And if you have troubled something, please enable the `Verbose Log` on the `General Setting` pane.
However, the logs would not be kept so long and cleared when restarted. If you want to check the logs, please enable `Write logs into the file` temporarily.
![ScreenShot](../images/write_logs_into_the_file.png)
> [!IMPORTANT]
> - Writing logs into the file will impact the performance.
> - Please make sure that you have erased all your confidential information before reporting issue.
### Why are the logs volatile and ephemeral?
To avoid unexpected exposure to our confidential things.
### Some network logs are not written into the file.
Especially the CORS error will be reported as a general error to the plug-in for security reasons. So we cannot detect and log it.
### If a file were deleted or trimmed, the capacity of the database should be reduced, right?
No, even though if files were deleted, chunks were not deleted.
Self-hosted LiveSync splits the files into multiple chunks and transfers only newly created. This behaviour enables us to less traffic. And, the chunks will be shared between the files to reduce the total usage of the database.
And one more thing, we can handle the conflicts on any device even though it has happened on other devices. This means that conflicts will happen in the past, after the time we have synchronised. Hence we cannot collect and delete the unused chunks even though if we are not currently referenced.
To shrink the database size, `Rebuild everything` only reliably and effectively. But do not worry, if we have synchronised well. We have the actual and real files. Only it takes a bit of time and traffics.
<!-- Add here --> <!-- Add here -->
## Troubleshooting ## Troubleshooting
<!-- Add here --> <!-- Add here -->
### On the mobile device, cannot synchronise on the local network!
Obsidian mobile is not able to connect to the non-secure end-point, such as starting with `http://`. Make sure your URI of CouchDB. Also not able to use a self-signed certificate.
### I think that something bad happening on the vault...
Place `redflag.md` on top of the vault, and restart Obsidian. The most simple way is to create a new note and rename it to `redflag`. Of course, we can put it without Obsidian.
If there is `redflag.md`, Self-hosted LiveSync suspends all database and storage processes.
## Tips ## Tips
<!-- Add here --> <!-- Add here -->
### Old tips ### Old tips
- If a folder becomes empty after a replication, it will be deleted by default. But you can toggle this behaviour. Check the [Settings](settings.md).
- LiveSync mode drains more batteries in mobile devices. Periodic sync with some automatic sync is recommended.
- Mobile Obsidian can not connect to non-secure (HTTP) or locally-signed servers, even if the root certificate is installed on the device.
- There are no 'exclude_folders' like configurations.
- While synchronizing, files are compared by their modification time and the older ones will be overwritten by the newer ones. Then plugin checks for conflicts and if a merge is needed, a dialog will open.
- Rarely, a file in the database could be corrupted. The plugin will not write to local storage when a file looks corrupted. If a local version of the file is on your device, the corruption could be fixed by editing the local file and synchronizing it. But if the file does not exist on any of your devices, then it can not be rescued. In this case, you can delete these items from the settings dialog. - Rarely, a file in the database could be corrupted. The plugin will not write to local storage when a file looks corrupted. If a local version of the file is on your device, the corruption could be fixed by editing the local file and synchronizing it. But if the file does not exist on any of your devices, then it can not be rescued. In this case, you can delete these items from the settings dialog.
- To stop the boot-up sequence (eg. for fixing problems on databases), you can put a `redflag.md` file (or directory) at the root of your vault. - To stop the boot-up sequence (eg. for fixing problems on databases), you can put a `redflag.md` file (or directory) at the root of your vault.
Tip for iOS: a redflag directory can be created at the root of the vault using the File application. Tip for iOS: a redflag directory can be created at the root of the vault using the File application.

BIN
images/hatch.png Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 13 KiB

BIN
images/write_logs_into_the_file.png Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 17 KiB

2
manifest.json
View File

@@ -1,7 +1,7 @@
{ {
"id": "obsidian-livesync", "id": "obsidian-livesync",
"name": "Self-hosted LiveSync", "name": "Self-hosted LiveSync",
"version": "0.22.8", "version": "0.22.17",
"minAppVersion": "0.9.12", "minAppVersion": "0.9.12",
"description": "Community implementation of self-hosted livesync. Reflect your vault changes to some other devices immediately. Please make sure to disable other synchronize solutions to avoid content corruption or duplication.", "description": "Community implementation of self-hosted livesync. Reflect your vault changes to some other devices immediately. Please make sure to disable other synchronize solutions to avoid content corruption or duplication.",
"author": "vorotamoroz", "author": "vorotamoroz",

2646
package-lock.json generated
View File

File diff suppressed because it is too large Load Diff

54
package.json
View File

@@ -1,6 +1,6 @@
{ {
"name": "obsidian-livesync", "name": "obsidian-livesync",
"version": "0.22.8", "version": "0.22.17",
"description": "Reflect your vault changes to some other devices immediately. Please make sure to disable other synchronize solutions to avoid content corruption or duplication.", "description": "Reflect your vault changes to some other devices immediately. Please make sure to disable other synchronize solutions to avoid content corruption or duplication.",
"main": "main.js", "main": "main.js",
"type": "module", "type": "module",
@@ -13,29 +13,29 @@
"author": "vorotamoroz", "author": "vorotamoroz",
"license": "MIT", "license": "MIT",
"devDependencies": { "devDependencies": {
"@tsconfig/svelte": "^5.0.0", "@tsconfig/svelte": "^5.0.2",
"@types/diff-match-patch": "^1.0.32", "@types/diff-match-patch": "^1.0.36",
"@types/node": "^20.2.5", "@types/node": "^20.11.28",
"@types/pouchdb": "^6.4.0", "@types/pouchdb": "^6.4.2",
"@types/pouchdb-adapter-http": "^6.1.3", "@types/pouchdb-adapter-http": "^6.1.6",
"@types/pouchdb-adapter-idb": "^6.1.4", "@types/pouchdb-adapter-idb": "^6.1.7",
"@types/pouchdb-browser": "^6.1.3", "@types/pouchdb-browser": "^6.1.5",
"@types/pouchdb-core": "^7.0.11", "@types/pouchdb-core": "^7.0.14",
"@types/pouchdb-mapreduce": "^6.1.7", "@types/pouchdb-mapreduce": "^6.1.10",
"@types/pouchdb-replication": "^6.4.4", "@types/pouchdb-replication": "^6.4.7",
"@types/transform-pouch": "^1.0.2", "@types/transform-pouch": "^1.0.6",
"@typescript-eslint/eslint-plugin": "^6.2.1", "@typescript-eslint/eslint-plugin": "^7.2.0",
"@typescript-eslint/parser": "^6.2.1", "@typescript-eslint/parser": "^7.2.0",
"builtin-modules": "^3.3.0", "builtin-modules": "^3.3.0",
"esbuild": "0.18.17", "esbuild": "0.20.2",
"esbuild-svelte": "^0.7.4", "esbuild-svelte": "^0.8.0",
"eslint": "^8.46.0", "eslint": "^8.57.0",
"eslint-config-airbnb-base": "^15.0.0", "eslint-config-airbnb-base": "^15.0.0",
"eslint-plugin-import": "^2.28.0", "eslint-plugin-import": "^2.29.1",
"events": "^3.3.0", "events": "^3.3.0",
"obsidian": "^1.4.11", "obsidian": "^1.5.7",
"postcss": "^8.4.27", "postcss": "^8.4.35",
"postcss-load-config": "^4.0.1", "postcss-load-config": "^5.0.3",
"pouchdb-adapter-http": "^8.0.1", "pouchdb-adapter-http": "^8.0.1",
"pouchdb-adapter-idb": "^8.0.1", "pouchdb-adapter-idb": "^8.0.1",
"pouchdb-adapter-indexeddb": "^8.0.1", "pouchdb-adapter-indexeddb": "^8.0.1",
@@ -46,16 +46,16 @@
"pouchdb-merge": "^8.0.1", "pouchdb-merge": "^8.0.1",
"pouchdb-replication": "^8.0.1", "pouchdb-replication": "^8.0.1",
"pouchdb-utils": "^8.0.1", "pouchdb-utils": "^8.0.1",
"svelte": "^4.1.2", "svelte": "^4.2.12",
"svelte-preprocess": "^5.0.4", "svelte-preprocess": "^5.1.3",
"terser": "^5.19.2", "terser": "^5.29.2",
"transform-pouch": "^2.0.0", "transform-pouch": "^2.0.0",
"tslib": "^2.6.1", "tslib": "^2.6.2",
"typescript": "^5.1.6" "typescript": "^5.4.2"
}, },
"dependencies": { "dependencies": {
"diff-match-patch": "^1.0.5", "diff-match-patch": "^1.0.5",
"idb": "^7.1.1", "idb": "^8.0.0",
"minimatch": "^9.0.3", "minimatch": "^9.0.3",
"xxhash-wasm": "0.4.2", "xxhash-wasm": "0.4.2",
"xxhash-wasm-102": "npm:xxhash-wasm@^1.0.2" "xxhash-wasm-102": "npm:xxhash-wasm@^1.0.2"

125
src/CmdConfigSync.ts
View File

@@ -4,10 +4,9 @@ import { Notice, type PluginManifest, parseYaml, normalizePath, type ListedFiles
import type { EntryDoc, LoadedEntry, InternalFileEntry, FilePathWithPrefix, FilePath, DocumentID, AnyEntry, SavingEntry } from "./lib/src/types"; import type { EntryDoc, LoadedEntry, InternalFileEntry, FilePathWithPrefix, FilePath, DocumentID, AnyEntry, SavingEntry } from "./lib/src/types";
import { LOG_LEVEL_INFO, LOG_LEVEL_NOTICE, LOG_LEVEL_VERBOSE, MODE_SELECTIVE } from "./lib/src/types"; import { LOG_LEVEL_INFO, LOG_LEVEL_NOTICE, LOG_LEVEL_VERBOSE, MODE_SELECTIVE } from "./lib/src/types";
import { ICXHeader, PERIODIC_PLUGIN_SWEEP, } from "./types"; import { ICXHeader, PERIODIC_PLUGIN_SWEEP, } from "./types";
import { createTextBlob, delay, getDocData, sendSignal, waitForSignal } from "./lib/src/utils"; import { createSavingEntryFromLoadedEntry, createTextBlob, delay, fireAndForget, getDocData, isDocContentSame, throttle } from "./lib/src/utils";
import { Logger } from "./lib/src/logger"; import { Logger } from "./lib/src/logger";
import { WrappedNotice } from "./lib/src/wrapper"; import { readString, decodeBinary, arrayBufferToBase64, digestHash } from "./lib/src/strbin";
import { readString, decodeBinary, arrayBufferToBase64, sha1 } from "./lib/src/strbin";
import { serialized } from "./lib/src/lock"; import { serialized } from "./lib/src/lock";
import { LiveSyncCommands } from "./LiveSyncCommands"; import { LiveSyncCommands } from "./LiveSyncCommands";
import { stripAllPrefixes } from "./lib/src/path"; import { stripAllPrefixes } from "./lib/src/path";
@@ -31,7 +30,8 @@ function serialize(data: PluginDataEx): string {
ret += data.mtime + d2; ret += data.mtime + d2;
for (const file of data.files) { for (const file of data.files) {
ret += file.filename + d + (file.displayName ?? "") + d + (file.version ?? "") + d2; ret += file.filename + d + (file.displayName ?? "") + d + (file.version ?? "") + d2;
ret += file.mtime + d + file.size + d2; const hash = digestHash((file.data ?? []).join());
ret += file.mtime + d + file.size + d + hash + d2;
for (const data of file.data ?? []) { for (const data of file.data ?? []) {
ret += data + d ret += data + d
} }
@@ -95,6 +95,7 @@ function deserialize2(str: string): PluginDataEx {
tokens.nextLine(); tokens.nextLine();
const mtime = Number(tokens.next()); const mtime = Number(tokens.next());
const size = Number(tokens.next()); const size = Number(tokens.next());
const hash = tokens.next();
tokens.nextLine(); tokens.nextLine();
const data = [] as string[]; const data = [] as string[];
let piece = ""; let piece = "";
@@ -110,7 +111,8 @@ function deserialize2(str: string): PluginDataEx {
version, version,
mtime, mtime,
size, size,
data data,
hash
} }
) )
tokens.nextLine(); tokens.nextLine();
@@ -137,10 +139,11 @@ export const pluginIsEnumerating = writable(false);
export type PluginDataExFile = { export type PluginDataExFile = {
filename: string, filename: string,
data?: string[], data: string[],
mtime: number, mtime: number,
size: number, size: number,
version?: string, version?: string,
hash?: string,
displayName?: string, displayName?: string,
} }
export type PluginDataExDisplay = { export type PluginDataExDisplay = {
@@ -169,19 +172,16 @@ export class ConfigSync extends LiveSyncCommands {
pluginScanningCount.onChanged((e) => { pluginScanningCount.onChanged((e) => {
const total = e.value; const total = e.value;
pluginIsEnumerating.set(total != 0); pluginIsEnumerating.set(total != 0);
if (total == 0) { // if (total == 0) {
Logger(`Processing configurations done`, LOG_LEVEL_INFO, "get-plugins"); // Logger(`Processing configurations done`, LOG_LEVEL_INFO, "get-plugins");
} // }
}) })
} }
confirmPopup: WrappedNotice = null;
get kvDB() { get kvDB() {
return this.plugin.kvDB; return this.plugin.kvDB;
} }
ensureDirectoryEx(fullPath: string) {
return this.plugin.ensureDirectoryEx(fullPath); pluginDialog?: PluginDialogModal = undefined;
}
pluginDialog: PluginDialogModal = null;
periodicPluginSweepProcessor = new PeriodicProcessor(this.plugin, async () => await this.scanAllConfigFiles(false)); periodicPluginSweepProcessor = new PeriodicProcessor(this.plugin, async () => await this.scanAllConfigFiles(false));
pluginList: PluginDataExDisplay[] = []; pluginList: PluginDataExDisplay[] = [];
@@ -189,7 +189,7 @@ export class ConfigSync extends LiveSyncCommands {
if (!this.settings.usePluginSync) { if (!this.settings.usePluginSync) {
return; return;
} }
if (this.pluginDialog != null) { if (this.pluginDialog) {
this.pluginDialog.open(); this.pluginDialog.open();
} else { } else {
this.pluginDialog = new PluginDialogModal(this.app, this.plugin); this.pluginDialog = new PluginDialogModal(this.app, this.plugin);
@@ -200,7 +200,7 @@ export class ConfigSync extends LiveSyncCommands {
hidePluginSyncModal() { hidePluginSyncModal() {
if (this.pluginDialog != null) { if (this.pluginDialog != null) {
this.pluginDialog.close(); this.pluginDialog.close();
this.pluginDialog = null; this.pluginDialog = undefined;
} }
} }
onunload() { onunload() {
@@ -275,16 +275,28 @@ export class ConfigSync extends LiveSyncCommands {
await this.updatePluginList(showMessage); await this.updatePluginList(showMessage);
} }
async loadPluginData(path: FilePathWithPrefix): Promise<PluginDataExDisplay | false> { async loadPluginData(path: FilePathWithPrefix): Promise<PluginDataExDisplay | false> {
const wx = await this.localDatabase.getDBEntry(path, null, false, false); const wx = await this.localDatabase.getDBEntry(path, undefined, false, false);
if (wx) { if (wx) {
const data = deserialize(getDocData(wx.data), {}) as PluginDataEx; const data = deserialize(getDocData(wx.data), {}) as PluginDataEx;
const xFiles = [] as PluginDataExFile[]; const xFiles = [] as PluginDataExFile[];
let missingHash = false;
for (const file of data.files) { for (const file of data.files) {
const work = { ...file }; const work = { ...file, data: [] as string[] };
const tempStr = getDocData(work.data); if (!file.hash) {
work.data = [await sha1(tempStr)]; // debugger;
const tempStr = getDocData(work.data);
const hash = digestHash(tempStr);
file.hash = hash;
missingHash = true;
}
work.data = [file.hash];
xFiles.push(work); xFiles.push(work);
} }
if (missingHash) {
Logger(`Digest created for ${path} to improve checking`, LOG_LEVEL_VERBOSE);
wx.data = serialize(data);
fireAndForget(() => this.localDatabase.putDBEntry(createSavingEntryFromLoadedEntry(wx)));
}
return ({ return ({
...data, ...data,
documentPath: this.getPath(wx), documentPath: this.getPath(wx),
@@ -293,7 +305,8 @@ export class ConfigSync extends LiveSyncCommands {
} }
return false; return false;
} }
createMissingConfigurationEntry() { createMissingConfigurationEntry = throttle(() => this._createMissingConfigurationEntry(), 1000);
_createMissingConfigurationEntry() {
let saveRequired = false; let saveRequired = false;
for (const v of this.pluginList) { for (const v of this.pluginList) {
const key = `${v.category}/${v.name}`; const key = `${v.category}/${v.name}`;
@@ -319,42 +332,27 @@ export class ConfigSync extends LiveSyncCommands {
const plugin = v[0]; const plugin = v[0];
const path = plugin.path || this.getPath(plugin); const path = plugin.path || this.getPath(plugin);
const oldEntry = (this.pluginList.find(e => e.documentPath == path)); const oldEntry = (this.pluginList.find(e => e.documentPath == path));
if (oldEntry && oldEntry.mtime == plugin.mtime) return; if (oldEntry && oldEntry.mtime == plugin.mtime) return [];
try { try {
const pluginData = await this.loadPluginData(path); const pluginData = await this.loadPluginData(path);
if (pluginData) { if (pluginData) {
return [pluginData]; let newList = [...this.pluginList];
newList = newList.filter(x => x.documentPath != pluginData.documentPath);
newList.push(pluginData);
this.pluginList = newList;
pluginList.set(newList);
} }
// Failed to load // Failed to load
return; return [];
} catch (ex) { } catch (ex) {
Logger(`Something happened at enumerating customization :${path}`, LOG_LEVEL_NOTICE); Logger(`Something happened at enumerating customization :${path}`, LOG_LEVEL_NOTICE);
Logger(ex, LOG_LEVEL_VERBOSE); Logger(ex, LOG_LEVEL_VERBOSE);
} }
return; return [];
}, { suspended: true, batchSize: 1, concurrentLimit: 5, delay: 300, yieldThreshold: 10 }).pipeTo( }, { suspended: false, batchSize: 1, concurrentLimit: 10, delay: 100, yieldThreshold: 10, maintainDelay: false, totalRemainingReactiveSource: pluginScanningCount }).startPipeline().root.onUpdateProgress(() => {
new QueueProcessor( this.createMissingConfigurationEntry();
async (pluginDataList) => { });
// Concurrency is two, therefore, we can unlock the previous awaiting.
sendSignal("plugin-next-load");
let newList = [...this.pluginList];
for (const item of pluginDataList) {
newList = newList.filter(x => x.documentPath != item.documentPath);
newList.push(item)
}
this.pluginList = newList;
pluginList.set(newList);
if (pluginDataList.length != 10) {
// If the queue is going to be empty, await subsequent for a while.
await waitForSignal("plugin-next-load", 1000);
}
return;
}
, { suspended: true, batchSize: 10, concurrentLimit: 2, delay: 250, yieldThreshold: 25, totalRemainingReactiveSource: pluginScanningCount })).startPipeline().root.onIdle(() => {
Logger(`All files enumerated`, LOG_LEVEL_INFO, "get-plugins");
this.createMissingConfigurationEntry();
});
async updatePluginList(showMessage: boolean, updatedDocumentPath?: FilePathWithPrefix): Promise<void> { async updatePluginList(showMessage: boolean, updatedDocumentPath?: FilePathWithPrefix): Promise<void> {
@@ -400,7 +398,7 @@ export class ConfigSync extends LiveSyncCommands {
showJSONMergeDialogAndMerge(docA: LoadedEntry, docB: LoadedEntry, pluginDataA: PluginDataEx, pluginDataB: PluginDataEx): Promise<boolean> { showJSONMergeDialogAndMerge(docA: LoadedEntry, docB: LoadedEntry, pluginDataA: PluginDataEx, pluginDataB: PluginDataEx): Promise<boolean> {
const fileA = { ...pluginDataA.files[0], ctime: pluginDataA.files[0].mtime, _id: `${pluginDataA.documentPath}` as DocumentID }; const fileA = { ...pluginDataA.files[0], ctime: pluginDataA.files[0].mtime, _id: `${pluginDataA.documentPath}` as DocumentID };
const fileB = pluginDataB.files[0]; const fileB = pluginDataB.files[0];
const docAx = { ...docA, ...fileA } as LoadedEntry, docBx = { ...docB, ...fileB } as LoadedEntry const docAx = { ...docA, ...fileA, datatype: "newnote" } as LoadedEntry, docBx = { ...docB, ...fileB, datatype: "newnote" } as LoadedEntry
return serialized("config:merge-data", () => new Promise((res) => { return serialized("config:merge-data", () => new Promise((res) => {
Logger("Opening data-merging dialog", LOG_LEVEL_VERBOSE); Logger("Opening data-merging dialog", LOG_LEVEL_VERBOSE);
// const docs = [docA, docB]; // const docs = [docA, docB];
@@ -433,7 +431,7 @@ export class ConfigSync extends LiveSyncCommands {
try { try {
// console.dir(f); // console.dir(f);
const path = `${baseDir}/${f.filename}`; const path = `${baseDir}/${f.filename}`;
await this.ensureDirectoryEx(path); await this.vaultAccess.ensureDirectory(path);
if (!content) { if (!content) {
const dt = decodeBinary(f.data); const dt = decodeBinary(f.data);
await this.vaultAccess.adapterWrite(path, dt); await this.vaultAccess.adapterWrite(path, dt);
@@ -504,9 +502,9 @@ export class ConfigSync extends LiveSyncCommands {
if (this.plugin.settings.usePluginSync && this.plugin.settings.notifyPluginOrSettingUpdated) { if (this.plugin.settings.usePluginSync && this.plugin.settings.notifyPluginOrSettingUpdated) {
if (!this.pluginDialog || (this.pluginDialog && !this.pluginDialog.isOpened())) { if (!this.pluginDialog || (this.pluginDialog && !this.pluginDialog.isOpened())) {
const fragment = createFragment((doc) => { const fragment = createFragment((doc) => {
doc.createEl("span", null, (a) => { doc.createEl("span", undefined, (a) => {
a.appendText(`Some configuration has been arrived, Press `); a.appendText(`Some configuration has been arrived, Press `);
a.appendChild(a.createEl("a", null, (anchor) => { a.appendChild(a.createEl("a", undefined, (anchor) => {
anchor.text = "HERE"; anchor.text = "HERE";
anchor.addEventListener("click", () => { anchor.addEventListener("click", () => {
this.showPluginSyncModal(); this.showPluginSyncModal();
@@ -672,7 +670,7 @@ export class ConfigSync extends LiveSyncCommands {
const content = createTextBlob(serialize(dt)); const content = createTextBlob(serialize(dt));
try { try {
const old = await this.localDatabase.getDBEntryMeta(prefixedFileName, null, false); const old = await this.localDatabase.getDBEntryMeta(prefixedFileName, undefined, false);
let saveData: SavingEntry; let saveData: SavingEntry;
if (old === false) { if (old === false) {
saveData = { saveData = {
@@ -689,9 +687,21 @@ export class ConfigSync extends LiveSyncCommands {
}; };
} else { } else {
if (old.mtime == mtime) { if (old.mtime == mtime) {
// Logger(`STORAGE --> DB:${file.path}: (hidden) Not changed`, LOG_LEVEL_VERBOSE); // Logger(`STORAGE --> DB:${prefixedFileName}: (config) Skipped (Same time)`, LOG_LEVEL_VERBOSE);
return true; return true;
} }
const oldC = await this.localDatabase.getDBEntryFromMeta(old, {}, false, false);
if (oldC) {
const d = await deserialize(getDocData(oldC.data), {}) as PluginDataEx;
const diffs = (d.files.map(previous => ({ prev: previous, curr: dt.files.find(e => e.filename == previous.filename) })).map(async e => {
try { return await isDocContentSame(e.curr?.data ?? [], e.prev.data) } catch (_) { return false }
}))
const isSame = (await Promise.all(diffs)).every(e => e == true);
if (isSame) {
Logger(`STORAGE --> DB:${prefixedFileName}: (config) Skipped (Same content)`, LOG_LEVEL_VERBOSE);
return true;
}
}
saveData = saveData =
{ {
...old, ...old,
@@ -757,7 +767,11 @@ export class ConfigSync extends LiveSyncCommands {
const filesOnDB = ((await this.localDatabase.allDocsRaw({ startkey: ICXHeader + "", endkey: `${ICXHeader}\u{10ffff}`, include_docs: true })).rows.map(e => e.doc) as InternalFileEntry[]).filter(e => !e.deleted); const filesOnDB = ((await this.localDatabase.allDocsRaw({ startkey: ICXHeader + "", endkey: `${ICXHeader}\u{10ffff}`, include_docs: true })).rows.map(e => e.doc) as InternalFileEntry[]).filter(e => !e.deleted);
let deleteCandidate = filesOnDB.map(e => this.getPath(e)).filter(e => e.startsWith(`${ICXHeader}${term}/`)); let deleteCandidate = filesOnDB.map(e => this.getPath(e)).filter(e => e.startsWith(`${ICXHeader}${term}/`));
for (const vp of virtualPathsOfLocalFiles) { for (const vp of virtualPathsOfLocalFiles) {
const p = files.find(e => e.key == vp).file; const p = files.find(e => e.key == vp)?.file;
if (!p) {
Logger(`scanAllConfigFiles - File not found: ${vp}`, LOG_LEVEL_VERBOSE);
continue;
}
await this.storeCustomizationFiles(p); await this.storeCustomizationFiles(p);
deleteCandidate = deleteCandidate.filter(e => e != vp); deleteCandidate = deleteCandidate.filter(e => e != vp);
} }
@@ -772,10 +786,11 @@ export class ConfigSync extends LiveSyncCommands {
const mtime = new Date().getTime(); const mtime = new Date().getTime();
await serialized("file-x-" + prefixedFileName, async () => { await serialized("file-x-" + prefixedFileName, async () => {
try { try {
const old = await this.localDatabase.getDBEntryMeta(prefixedFileName, null, false) as InternalFileEntry | false; const old = await this.localDatabase.getDBEntryMeta(prefixedFileName, undefined, false) as InternalFileEntry | false;
let saveData: InternalFileEntry; let saveData: InternalFileEntry;
if (old === false) { if (old === false) {
Logger(`STORAGE -x> DB:${prefixedFileName}: (config) already deleted (Not found on database)`); Logger(`STORAGE -x> DB:${prefixedFileName}: (config) already deleted (Not found on database)`);
return;
} else { } else {
if (old.deleted) { if (old.deleted) {
Logger(`STORAGE -x> DB:${prefixedFileName}: (config) already deleted`); Logger(`STORAGE -x> DB:${prefixedFileName}: (config) already deleted`);

114
src/CmdHiddenFileSync.ts
View File

@@ -1,28 +1,23 @@
import { normalizePath, type PluginManifest, type ListedFiles } from "./deps"; import { normalizePath, type PluginManifest, type ListedFiles } from "./deps";
import { type EntryDoc, type LoadedEntry, type InternalFileEntry, type FilePathWithPrefix, type FilePath, LOG_LEVEL_INFO, LOG_LEVEL_NOTICE, LOG_LEVEL_VERBOSE, MODE_SELECTIVE, MODE_PAUSED, type SavingEntry, type DocumentID } from "./lib/src/types"; import { type EntryDoc, type LoadedEntry, type InternalFileEntry, type FilePathWithPrefix, type FilePath, LOG_LEVEL_INFO, LOG_LEVEL_NOTICE, LOG_LEVEL_VERBOSE, MODE_SELECTIVE, MODE_PAUSED, type SavingEntry, type DocumentID } from "./lib/src/types";
import { type InternalFileInfo, ICHeader, ICHeaderEnd } from "./types"; import { type InternalFileInfo, ICHeader, ICHeaderEnd } from "./types";
import { createBinaryBlob, isDocContentSame, sendSignal } from "./lib/src/utils"; import { readAsBlob, isDocContentSame, sendSignal, readContent, createBlob } from "./lib/src/utils";
import { Logger } from "./lib/src/logger"; import { Logger } from "./lib/src/logger";
import { PouchDB } from "./lib/src/pouchdb-browser.js"; import { PouchDB } from "./lib/src/pouchdb-browser.js";
import { isInternalMetadata, PeriodicProcessor } from "./utils"; import { isInternalMetadata, PeriodicProcessor } from "./utils";
import { WrappedNotice } from "./lib/src/wrapper";
import { decodeBinary, encodeBinary } from "./lib/src/strbin";
import { serialized } from "./lib/src/lock"; import { serialized } from "./lib/src/lock";
import { JsonResolveModal } from "./JsonResolveModal"; import { JsonResolveModal } from "./JsonResolveModal";
import { LiveSyncCommands } from "./LiveSyncCommands"; import { LiveSyncCommands } from "./LiveSyncCommands";
import { addPrefix, stripAllPrefixes } from "./lib/src/path"; import { addPrefix, stripAllPrefixes } from "./lib/src/path";
import { KeyedQueueProcessor, QueueProcessor } from "./lib/src/processor"; import { QueueProcessor } from "./lib/src/processor";
import { hiddenFilesEventCount, hiddenFilesProcessingCount } from "./lib/src/stores"; import { hiddenFilesEventCount, hiddenFilesProcessingCount } from "./lib/src/stores";
export class HiddenFileSync extends LiveSyncCommands { export class HiddenFileSync extends LiveSyncCommands {
periodicInternalFileScanProcessor: PeriodicProcessor = new PeriodicProcessor(this.plugin, async () => this.settings.syncInternalFiles && this.localDatabase.isReady && await this.syncInternalFilesAndDatabase("push", false)); periodicInternalFileScanProcessor: PeriodicProcessor = new PeriodicProcessor(this.plugin, async () => this.settings.syncInternalFiles && this.localDatabase.isReady && await this.syncInternalFilesAndDatabase("push", false));
confirmPopup: WrappedNotice = null;
get kvDB() { get kvDB() {
return this.plugin.kvDB; return this.plugin.kvDB;
} }
ensureDirectoryEx(fullPath: string) {
return this.plugin.ensureDirectoryEx(fullPath);
}
getConflictedDoc(path: FilePathWithPrefix, rev: string) { getConflictedDoc(path: FilePathWithPrefix, rev: string) {
return this.plugin.getConflictedDoc(path, rev); return this.plugin.getConflictedDoc(path, rev);
} }
@@ -70,23 +65,23 @@ export class HiddenFileSync extends LiveSyncCommands {
realizeSettingSyncMode(): Promise<void> { realizeSettingSyncMode(): Promise<void> {
this.periodicInternalFileScanProcessor?.disable(); this.periodicInternalFileScanProcessor?.disable();
if (this.plugin.suspended) if (this.plugin.suspended)
return; return Promise.resolve();
if (!this.plugin.isReady) if (!this.plugin.isReady)
return; return Promise.resolve();
this.periodicInternalFileScanProcessor.enable(this.settings.syncInternalFiles && this.settings.syncInternalFilesInterval ? (this.settings.syncInternalFilesInterval * 1000) : 0); this.periodicInternalFileScanProcessor.enable(this.settings.syncInternalFiles && this.settings.syncInternalFilesInterval ? (this.settings.syncInternalFilesInterval * 1000) : 0);
return; return Promise.resolve();
} }
procInternalFile(filename: string) { procInternalFile(filename: string) {
this.internalFileProcessor.enqueueWithKey(filename, filename); this.internalFileProcessor.enqueue(filename);
} }
internalFileProcessor = new KeyedQueueProcessor<string, any>( internalFileProcessor = new QueueProcessor<string, any>(
async (filenames) => { async (filenames) => {
Logger(`START :Applying hidden ${filenames.length} files change`, LOG_LEVEL_VERBOSE); Logger(`START :Applying hidden ${filenames.length} files change`, LOG_LEVEL_VERBOSE);
await this.syncInternalFilesAndDatabase("pull", false, false, filenames); await this.syncInternalFilesAndDatabase("pull", false, false, filenames);
Logger(`DONE :Applying hidden ${filenames.length} files change`, LOG_LEVEL_VERBOSE); Logger(`DONE :Applying hidden ${filenames.length} files change`, LOG_LEVEL_VERBOSE);
return; return;
}, { batchSize: 100, concurrentLimit: 1, delay: 10, yieldThreshold: 10, suspended: false, totalRemainingReactiveSource: hiddenFilesEventCount } }, { batchSize: 100, concurrentLimit: 1, delay: 10, yieldThreshold: 100, suspended: false, totalRemainingReactiveSource: hiddenFilesEventCount }
); );
recentProcessedInternalFiles = [] as string[]; recentProcessedInternalFiles = [] as string[];
@@ -128,7 +123,7 @@ export class HiddenFileSync extends LiveSyncCommands {
if (storageMTime == 0) { if (storageMTime == 0) {
await this.deleteInternalFileOnDatabase(path); await this.deleteInternalFileOnDatabase(path);
} else { } else {
await this.storeInternalFileToDatabase({ path: path, ...stat }); await this.storeInternalFileToDatabase({ path: path, mtime, ctime: stat?.ctime ?? mtime, size: stat?.size ?? 0 });
} }
} }
@@ -165,7 +160,7 @@ export class HiddenFileSync extends LiveSyncCommands {
await this.localDatabase.removeRevision(id, delRev); await this.localDatabase.removeRevision(id, delRev);
Logger(`Older one has been deleted:${path}`); Logger(`Older one has been deleted:${path}`);
const cc = await this.localDatabase.getRaw(id, { conflicts: true }); const cc = await this.localDatabase.getRaw(id, { conflicts: true });
if (cc._conflicts.length == 0) { if (cc._conflicts?.length === 0) {
await this.extractInternalFileFromDatabase(stripAllPrefixes(path)) await this.extractInternalFileFromDatabase(stripAllPrefixes(path))
} else { } else {
this.conflictResolutionProcessor.enqueue(path); this.conflictResolutionProcessor.enqueue(path);
@@ -180,11 +175,12 @@ export class HiddenFileSync extends LiveSyncCommands {
// Retrieve data // Retrieve data
const id = await this.path2id(path, ICHeader); const id = await this.path2id(path, ICHeader);
const doc = await this.localDatabase.getRaw(id, { conflicts: true }); const doc = await this.localDatabase.getRaw(id, { conflicts: true });
// If there is no conflict, return with false. // if (!("_conflicts" in doc)){
if (!("_conflicts" in doc)) // return [];
return; // }
if (doc._conflicts === undefined) return [];
if (doc._conflicts.length == 0) if (doc._conflicts.length == 0)
return; return [];
Logger(`Hidden file conflicted:${path}`); Logger(`Hidden file conflicted:${path}`);
const conflicts = doc._conflicts.sort((a, b) => Number(a.split("-")[0]) - Number(b.split("-")[0])); const conflicts = doc._conflicts.sort((a, b) => Number(a.split("-")[0]) - Number(b.split("-")[0]));
const revA = doc._rev; const revA = doc._rev;
@@ -195,22 +191,25 @@ export class HiddenFileSync extends LiveSyncCommands {
const conflictedRevNo = Number(conflictedRev.split("-")[0]); const conflictedRevNo = Number(conflictedRev.split("-")[0]);
//Search //Search
const revFrom = (await this.localDatabase.getRaw<EntryDoc>(id, { revs_info: true })); const revFrom = (await this.localDatabase.getRaw<EntryDoc>(id, { revs_info: true }));
const commonBase = revFrom._revs_info.filter(e => e.status == "available" && Number(e.rev.split("-")[0]) < conflictedRevNo).first()?.rev ?? ""; const commonBase = revFrom._revs_info?.filter(e => e.status == "available" && Number(e.rev.split("-")[0]) < conflictedRevNo).first()?.rev ?? "";
const result = await this.plugin.mergeObject(path, commonBase, doc._rev, conflictedRev); const result = await this.plugin.mergeObject(path, commonBase, doc._rev, conflictedRev);
if (result) { if (result) {
Logger(`Object merge:${path}`, LOG_LEVEL_INFO); Logger(`Object merge:${path}`, LOG_LEVEL_INFO);
const filename = stripAllPrefixes(path); const filename = stripAllPrefixes(path);
const isExists = await this.plugin.vaultAccess.adapterExists(filename); const isExists = await this.plugin.vaultAccess.adapterExists(filename);
if (!isExists) { if (!isExists) {
await this.ensureDirectoryEx(filename); await this.vaultAccess.ensureDirectory(filename);
} }
await this.plugin.vaultAccess.adapterWrite(filename, result); await this.plugin.vaultAccess.adapterWrite(filename, result);
const stat = await this.vaultAccess.adapterStat(filename); const stat = await this.vaultAccess.adapterStat(filename);
if (!stat) {
throw new Error(`conflictResolutionProcessor: Failed to stat file ${filename}`);
}
await this.storeInternalFileToDatabase({ path: filename, ...stat }); await this.storeInternalFileToDatabase({ path: filename, ...stat });
await this.extractInternalFileFromDatabase(filename); await this.extractInternalFileFromDatabase(filename);
await this.localDatabase.removeRevision(id, revB); await this.localDatabase.removeRevision(id, revB);
this.conflictResolutionProcessor.enqueue(path); this.conflictResolutionProcessor.enqueue(path);
return; return [];
} else { } else {
Logger(`Object merge is not applicable.`, LOG_LEVEL_VERBOSE); Logger(`Object merge is not applicable.`, LOG_LEVEL_VERBOSE);
} }
@@ -218,11 +217,11 @@ export class HiddenFileSync extends LiveSyncCommands {
} }
// When not JSON file, resolve conflicts by choosing a newer one. // When not JSON file, resolve conflicts by choosing a newer one.
await this.resolveByNewerEntry(id, path, doc, revA, revB); await this.resolveByNewerEntry(id, path, doc, revA, revB);
return; return [];
} catch (ex) { } catch (ex) {
Logger(`Failed to resolve conflict (Hidden): ${path}`); Logger(`Failed to resolve conflict (Hidden): ${path}`);
Logger(ex, LOG_LEVEL_VERBOSE); Logger(ex, LOG_LEVEL_VERBOSE);
return; return [];
} }
}, { }, {
suspended: false, batchSize: 1, concurrentLimit: 5, delay: 10, keepResultUntilDownstreamConnected: true, yieldThreshold: 10, suspended: false, batchSize: 1, concurrentLimit: 5, delay: 10, keepResultUntilDownstreamConnected: true, yieldThreshold: 10,
@@ -315,11 +314,11 @@ export class HiddenFileSync extends LiveSyncCommands {
if (processed % 100 == 0) { if (processed % 100 == 0) {
Logger(`Hidden file: ${processed}/${fileCount}`, logLevel, "sync_internal"); Logger(`Hidden file: ${processed}/${fileCount}`, logLevel, "sync_internal");
} }
if (!filename) return; if (!filename) return [];
if (ignorePatterns.some(e => filename.match(e))) if (ignorePatterns.some(e => filename.match(e)))
return; return [];
if (await this.plugin.isIgnoredByIgnoreFiles(filename)) { if (await this.plugin.isIgnoredByIgnoreFiles(filename)) {
return; return [];
} }
const fileOnStorage = filename in filesMap ? filesMap[filename] : undefined; const fileOnStorage = filename in filesMap ? filesMap[filename] : undefined;
@@ -406,7 +405,7 @@ export class HiddenFileSync extends LiveSyncCommands {
const enabledPlugins = this.app.plugins.enabledPlugins as Set<string>; const enabledPlugins = this.app.plugins.enabledPlugins as Set<string>;
const enabledPluginManifests = manifests.filter(e => enabledPlugins.has(e.id)); const enabledPluginManifests = manifests.filter(e => enabledPlugins.has(e.id));
for (const manifest of enabledPluginManifests) { for (const manifest of enabledPluginManifests) {
if (manifest.dir in updatedFolders) { if (manifest.dir && manifest.dir in updatedFolders) {
// If notified about plug-ins, reloading Obsidian may not be necessary. // If notified about plug-ins, reloading Obsidian may not be necessary.
updatedCount -= updatedFolders[manifest.dir]; updatedCount -= updatedFolders[manifest.dir];
const updatePluginId = manifest.id; const updatePluginId = manifest.id;
@@ -454,19 +453,11 @@ export class HiddenFileSync extends LiveSyncCommands {
const id = await this.path2id(file.path, ICHeader); const id = await this.path2id(file.path, ICHeader);
const prefixedFileName = addPrefix(file.path, ICHeader); const prefixedFileName = addPrefix(file.path, ICHeader);
const contentBin = await this.plugin.vaultAccess.adapterReadBinary(file.path); const content = createBlob(await this.plugin.vaultAccess.adapterReadAuto(file.path));
let content: Blob;
try {
content = createBinaryBlob(contentBin);
} catch (ex) {
Logger(`The file ${file.path} could not be encoded`);
Logger(ex, LOG_LEVEL_VERBOSE);
return false;
}
const mtime = file.mtime; const mtime = file.mtime;
return await serialized("file-" + prefixedFileName, async () => { return await serialized("file-" + prefixedFileName, async () => {
try { try {
const old = await this.localDatabase.getDBEntry(prefixedFileName, null, false, false); const old = await this.localDatabase.getDBEntry(prefixedFileName, undefined, false, false);
let saveData: SavingEntry; let saveData: SavingEntry;
if (old === false) { if (old === false) {
saveData = { saveData = {
@@ -482,7 +473,7 @@ export class HiddenFileSync extends LiveSyncCommands {
type: "newnote", type: "newnote",
}; };
} else { } else {
if (await isDocContentSame(old.data, content) && !forceWrite) { if (await isDocContentSame(readAsBlob(old), content) && !forceWrite) {
// Logger(`STORAGE --> DB:${file.path}: (hidden) Not changed`, LOG_LEVEL_VERBOSE); // Logger(`STORAGE --> DB:${file.path}: (hidden) Not changed`, LOG_LEVEL_VERBOSE);
return; return;
} }
@@ -492,13 +483,13 @@ export class HiddenFileSync extends LiveSyncCommands {
data: content, data: content,
mtime, mtime,
size: file.size, size: file.size,
datatype: "newnote", datatype: old.datatype,
children: [], children: [],
deleted: false, deleted: false,
type: "newnote", type: old.datatype,
}; };
} }
const ret = await this.localDatabase.putDBEntry(saveData, true); const ret = await this.localDatabase.putDBEntry(saveData);
Logger(`STORAGE --> DB:${file.path}: (hidden) Done`); Logger(`STORAGE --> DB:${file.path}: (hidden) Done`);
return ret; return ret;
} catch (ex) { } catch (ex) {
@@ -518,7 +509,7 @@ export class HiddenFileSync extends LiveSyncCommands {
} }
await serialized("file-" + prefixedFileName, async () => { await serialized("file-" + prefixedFileName, async () => {
try { try {
const old = await this.localDatabase.getDBEntryMeta(prefixedFileName, null, true) as InternalFileEntry | false; const old = await this.localDatabase.getDBEntryMeta(prefixedFileName, undefined, true) as InternalFileEntry | false;
let saveData: InternalFileEntry; let saveData: InternalFileEntry;
if (old === false) { if (old === false) {
saveData = { saveData = {
@@ -534,7 +525,7 @@ export class HiddenFileSync extends LiveSyncCommands {
} else { } else {
// Remove all conflicted before deleting. // Remove all conflicted before deleting.
const conflicts = await this.localDatabase.getRaw(old._id, { conflicts: true }); const conflicts = await this.localDatabase.getRaw(old._id, { conflicts: true });
if ("_conflicts" in conflicts) { if (conflicts._conflicts !== undefined) {
for (const conflictRev of conflicts._conflicts) { for (const conflictRev of conflicts._conflicts) {
await this.localDatabase.removeRevision(old._id, conflictRev); await this.localDatabase.removeRevision(old._id, conflictRev);
Logger(`STORAGE -x> DB:${filename}: (hidden) conflict removed ${old._rev} => ${conflictRev}`, LOG_LEVEL_VERBOSE); Logger(`STORAGE -x> DB:${filename}: (hidden) conflict removed ${old._rev} => ${conflictRev}`, LOG_LEVEL_VERBOSE);
@@ -584,7 +575,7 @@ export class HiddenFileSync extends LiveSyncCommands {
const deleted = fileOnDB.deleted || fileOnDB._deleted || false; const deleted = fileOnDB.deleted || fileOnDB._deleted || false;
if (deleted) { if (deleted) {
if (!isExists) { if (!isExists) {
Logger(`STORAGE <x- DB:${filename}: deleted (hidden) Deleted on DB, but the file is already not found on storage.`); Logger(`STORAGE <x- DB:${filename}: deleted (hidden) Deleted on DB, but the file is already not found on storage.`);
} else { } else {
Logger(`STORAGE <x- DB:${filename}: deleted (hidden).`); Logger(`STORAGE <x- DB:${filename}: deleted (hidden).`);
await this.plugin.vaultAccess.adapterRemove(filename); await this.plugin.vaultAccess.adapterRemove(filename);
@@ -599,8 +590,8 @@ export class HiddenFileSync extends LiveSyncCommands {
return true; return true;
} }
if (!isExists) { if (!isExists) {
await this.ensureDirectoryEx(filename); await this.vaultAccess.ensureDirectory(filename);
await this.plugin.vaultAccess.adapterWrite(filename, decodeBinary(fileOnDB.data), { mtime: fileOnDB.mtime, ctime: fileOnDB.ctime }); await this.plugin.vaultAccess.adapterWrite(filename, readContent(fileOnDB), { mtime: fileOnDB.mtime, ctime: fileOnDB.ctime });
try { try {
//@ts-ignore internalAPI //@ts-ignore internalAPI
await this.app.vault.adapter.reconcileInternalFile(filename); await this.app.vault.adapter.reconcileInternalFile(filename);
@@ -611,13 +602,13 @@ export class HiddenFileSync extends LiveSyncCommands {
Logger(`STORAGE <-- DB:${filename}: written (hidden,new${force ? ", force" : ""})`); Logger(`STORAGE <-- DB:${filename}: written (hidden,new${force ? ", force" : ""})`);
return true; return true;
} else { } else {
const contentBin = await this.plugin.vaultAccess.adapterReadBinary(filename); const content = await this.plugin.vaultAccess.adapterReadAuto(filename);
const content = await encodeBinary(contentBin); const docContent = readContent(fileOnDB);
if (await isDocContentSame(content, fileOnDB.data) && !force) { if (await isDocContentSame(content, docContent) && !force) {
// Logger(`STORAGE <-- DB:${filename}: skipped (hidden) Not changed`, LOG_LEVEL_VERBOSE); // Logger(`STORAGE <-- DB:${filename}: skipped (hidden) Not changed`, LOG_LEVEL_VERBOSE);
return true; return true;
} }
await this.plugin.vaultAccess.adapterWrite(filename, decodeBinary(fileOnDB.data), { mtime: fileOnDB.mtime, ctime: fileOnDB.ctime }); await this.plugin.vaultAccess.adapterWrite(filename, docContent, { mtime: fileOnDB.mtime, ctime: fileOnDB.ctime });
try { try {
//@ts-ignore internalAPI //@ts-ignore internalAPI
await this.app.vault.adapter.reconcileInternalFile(filename); await this.app.vault.adapter.reconcileInternalFile(filename);
@@ -668,11 +659,15 @@ export class HiddenFileSync extends LiveSyncCommands {
if (!keep && result) { if (!keep && result) {
const isExists = await this.plugin.vaultAccess.adapterExists(filename); const isExists = await this.plugin.vaultAccess.adapterExists(filename);
if (!isExists) { if (!isExists) {
await this.ensureDirectoryEx(filename); await this.vaultAccess.ensureDirectory(filename);
} }
await this.plugin.vaultAccess.adapterWrite(filename, result); await this.plugin.vaultAccess.adapterWrite(filename, result);
const stat = await this.plugin.vaultAccess.adapterStat(filename); const stat = await this.plugin.vaultAccess.adapterStat(filename);
await this.storeInternalFileToDatabase({ path: filename, ...stat }, true); if (!stat) {
throw new Error("Stat failed");
}
const mtime = stat?.mtime ?? 0;
await this.storeInternalFileToDatabase({ path: filename, mtime, ctime: stat?.ctime ?? mtime, size: stat?.size ?? 0 }, true);
try { try {
//@ts-ignore internalAPI //@ts-ignore internalAPI
await this.app.vault.adapter.reconcileInternalFile(filename); await this.app.vault.adapter.reconcileInternalFile(filename);
@@ -706,7 +701,7 @@ export class HiddenFileSync extends LiveSyncCommands {
const root = this.app.vault.getRoot(); const root = this.app.vault.getRoot();
const findRoot = root.path; const findRoot = root.path;
const filenames = (await this.getFiles(findRoot, [], null, ignoreFilter)).filter(e => e.startsWith(".")).filter(e => !e.startsWith(".trash")); const filenames = (await this.getFiles(findRoot, [], undefined, ignoreFilter)).filter(e => e.startsWith(".")).filter(e => !e.startsWith(".trash"));
const files = filenames.filter(path => synchronisedInConfigSync.every(filterFile => !path.toLowerCase().startsWith(filterFile))).map(async (e) => { const files = filenames.filter(path => synchronisedInConfigSync.every(filterFile => !path.toLowerCase().startsWith(filterFile))).map(async (e) => {
return { return {
path: e as FilePath, path: e as FilePath,
@@ -719,9 +714,12 @@ export class HiddenFileSync extends LiveSyncCommands {
if (await this.plugin.isIgnoredByIgnoreFiles(w.path)) { if (await this.plugin.isIgnoredByIgnoreFiles(w.path)) {
continue continue
} }
const mtime = w.stat?.mtime ?? 0
const ctime = w.stat?.ctime ?? mtime;
const size = w.stat?.size ?? 0;
result.push({ result.push({
...w, ...w,
...w.stat mtime, ctime, size
}); });
} }
return result; return result;
@@ -732,8 +730,8 @@ export class HiddenFileSync extends LiveSyncCommands {
async getFiles( async getFiles(
path: string, path: string,
ignoreList: string[], ignoreList: string[],
filter: RegExp[], filter?: RegExp[],
ignoreFilter: RegExp[] ignoreFilter?: RegExp[]
) { ) {
let w: ListedFiles; let w: ListedFiles;
try { try {

314
src/CmdPluginAndTheirSettings.ts
View File

@@ -1,314 +0,0 @@
import { normalizePath, type PluginManifest } from "./deps";
import type { DocumentID, EntryDoc, FilePathWithPrefix, LoadedEntry, SavingEntry } from "./lib/src/types";
import { LOG_LEVEL_INFO, LOG_LEVEL_NOTICE, LOG_LEVEL_VERBOSE } from "./lib/src/types";
import { type PluginDataEntry, PERIODIC_PLUGIN_SWEEP, type PluginList, type DevicePluginList, PSCHeader, PSCHeaderEnd } from "./types";
import { createTextBlob, getDocData, isDocContentSame } from "./lib/src/utils";
import { Logger } from "./lib/src/logger";
import { PouchDB } from "./lib/src/pouchdb-browser.js";
import { isPluginMetadata, PeriodicProcessor } from "./utils";
import { PluginDialogModal } from "./dialogs";
import { NewNotice } from "./lib/src/wrapper";
import { versionNumberString2Number } from "./lib/src/strbin";
import { serialized, skipIfDuplicated } from "./lib/src/lock";
import { LiveSyncCommands } from "./LiveSyncCommands";
export class PluginAndTheirSettings extends LiveSyncCommands {
get deviceAndVaultName() {
return this.plugin.deviceAndVaultName;
}
pluginDialog: PluginDialogModal = null;
periodicPluginSweepProcessor = new PeriodicProcessor(this.plugin, async () => await this.sweepPlugin(false));
showPluginSyncModal() {
if (this.pluginDialog != null) {
this.pluginDialog.open();
} else {
this.pluginDialog = new PluginDialogModal(this.app, this.plugin);
this.pluginDialog.open();
}
}
hidePluginSyncModal() {
if (this.pluginDialog != null) {
this.pluginDialog.close();
this.pluginDialog = null;
}
}
onload(): void | Promise<void> {
this.plugin.addCommand({
id: "livesync-plugin-dialog",
name: "Show Plugins and their settings",
callback: () => {
this.showPluginSyncModal();
},
});
this.showPluginSyncModal();
}
onunload() {
this.hidePluginSyncModal();
this.periodicPluginSweepProcessor?.disable();
}
parseReplicationResultItem(doc: PouchDB.Core.ExistingDocument<EntryDoc>) {
if (isPluginMetadata(doc._id)) {
if (this.settings.notifyPluginOrSettingUpdated) {
this.triggerCheckPluginUpdate();
return true;
}
}
return false;
}
async beforeReplicate(showMessage: boolean) {
if (this.settings.autoSweepPlugins) {
await this.sweepPlugin(showMessage);
}
}
async onResume() {
if (this.plugin.suspended)
return;
if (this.settings.autoSweepPlugins) {
await this.sweepPlugin(false);
}
this.periodicPluginSweepProcessor.enable(this.settings.autoSweepPluginsPeriodic && !this.settings.watchInternalFileChanges ? (PERIODIC_PLUGIN_SWEEP * 1000) : 0);
}
async onInitializeDatabase(showNotice: boolean) {
if (this.settings.usePluginSync) {
try {
Logger("Scanning plugins...");
await this.sweepPlugin(showNotice);
Logger("Scanning plugins done");
} catch (ex) {
Logger("Scanning plugins failed");
Logger(ex, LOG_LEVEL_VERBOSE);
}
}
}
async realizeSettingSyncMode() {
this.periodicPluginSweepProcessor?.disable();
if (this.plugin.suspended)
return;
if (this.settings.autoSweepPlugins) {
await this.sweepPlugin(false);
}
this.periodicPluginSweepProcessor.enable(this.settings.autoSweepPluginsPeriodic && !this.settings.watchInternalFileChanges ? (PERIODIC_PLUGIN_SWEEP * 1000) : 0);
}
triggerCheckPluginUpdate() {
(async () => await this.checkPluginUpdate())();
}
async getPluginList(): Promise<{ plugins: PluginList; allPlugins: DevicePluginList; thisDevicePlugins: DevicePluginList; }> {
const docList = await this.localDatabase.allDocsRaw<PluginDataEntry>({ startkey: PSCHeader, endkey: PSCHeaderEnd, include_docs: false });
const oldDocs: PluginDataEntry[] = ((await Promise.all(docList.rows.map(async (e) => await this.localDatabase.getDBEntry(e.id as FilePathWithPrefix /* WARN!! THIS SHOULD BE WRAPPED */)))).filter((e) => e !== false) as LoadedEntry[]).map((e) => JSON.parse(getDocData(e.data)));
const plugins: { [key: string]: PluginDataEntry[]; } = {};
const allPlugins: { [key: string]: PluginDataEntry; } = {};
const thisDevicePlugins: { [key: string]: PluginDataEntry; } = {};
for (const v of oldDocs) {
if (typeof plugins[v.deviceVaultName] === "undefined") {
plugins[v.deviceVaultName] = [];
}
plugins[v.deviceVaultName].push(v);
allPlugins[v._id] = v;
if (v.deviceVaultName == this.deviceAndVaultName) {
thisDevicePlugins[v.manifest.id] = v;
}
}
return { plugins, allPlugins, thisDevicePlugins };
}
async checkPluginUpdate() {
if (!this.plugin.settings.usePluginSync)
return;
await this.sweepPlugin(false);
const { allPlugins, thisDevicePlugins } = await this.getPluginList();
const arrPlugins = Object.values(allPlugins);
let updateFound = false;
for (const plugin of arrPlugins) {
const ownPlugin = thisDevicePlugins[plugin.manifest.id];
if (ownPlugin) {
const remoteVersion = versionNumberString2Number(plugin.manifest.version);
const ownVersion = versionNumberString2Number(ownPlugin.manifest.version);
if (remoteVersion > ownVersion) {
updateFound = true;
}
if (((plugin.mtime / 1000) | 0) > ((ownPlugin.mtime / 1000) | 0) && (plugin.dataJson ?? "") != (ownPlugin.dataJson ?? "")) {
updateFound = true;
}
}
}
if (updateFound) {
const fragment = createFragment((doc) => {
doc.createEl("a", null, (a) => {
a.text = "There're some new plugins or their settings";
a.addEventListener("click", () => this.showPluginSyncModal());
});
});
NewNotice(fragment, 10000);
} else {
Logger("Everything is up to date.", LOG_LEVEL_NOTICE);
}
}
async sweepPlugin(showMessage = false, specificPluginPath = "") {
if (!this.settings.usePluginSync)
return;
if (!this.localDatabase.isReady)
return;
// @ts-ignore
const pl = this.app.plugins;
const manifests: PluginManifest[] = Object.values(pl.manifests);
let specificPlugin = "";
if (specificPluginPath != "") {
specificPlugin = manifests.find(e => e.dir.endsWith("/" + specificPluginPath))?.id ?? "";
}
await skipIfDuplicated("sweepplugin", async () => {
const logLevel = showMessage ? LOG_LEVEL_NOTICE : LOG_LEVEL_INFO;
if (!this.deviceAndVaultName) {
Logger("You have to set your device name.", LOG_LEVEL_NOTICE);
return;
}
Logger("Scanning plugins", logLevel);
const oldDocs = await this.localDatabase.allDocsRaw<EntryDoc>({
startkey: `ps:${this.deviceAndVaultName}-${specificPlugin}`,
endkey: `ps:${this.deviceAndVaultName}-${specificPlugin}\u{10ffff}`,
include_docs: true,
});
// Logger("OLD DOCS.", LOG_LEVEL_VERBOSE);
// sweep current plugin.
const procs = manifests.map(async (m) => {
const pluginDataEntryID = `ps:${this.deviceAndVaultName}-${m.id}` as DocumentID;
try {
if (specificPlugin && m.id != specificPlugin) {
return;
}
Logger(`Reading plugin:${m.name}(${m.id})`, LOG_LEVEL_VERBOSE);
const path = normalizePath(m.dir) + "/";
const files = ["manifest.json", "main.js", "styles.css", "data.json"];
const pluginData: { [key: string]: string; } = {};
for (const file of files) {
const thePath = path + file;
if (await this.plugin.vaultAccess.adapterExists(thePath)) {
pluginData[file] = await this.plugin.vaultAccess.adapterRead(thePath);
}
}
let mtime = 0;
if (await this.plugin.vaultAccess.adapterExists(path + "/data.json")) {
mtime = (await this.plugin.vaultAccess.adapterStat(path + "/data.json")).mtime;
}
const p: PluginDataEntry = {
_id: pluginDataEntryID,
dataJson: pluginData["data.json"],
deviceVaultName: this.deviceAndVaultName,
mainJs: pluginData["main.js"],
styleCss: pluginData["styles.css"],
manifest: m,
manifestJson: pluginData["manifest.json"],
mtime: mtime,
type: "plugin",
};
const blob = createTextBlob(JSON.stringify(p));
const d: SavingEntry = {
_id: p._id,
path: p._id as string as FilePathWithPrefix,
data: blob,
ctime: mtime,
mtime: mtime,
size: blob.size,
children: [],
datatype: "plain",
type: "plain"
};
Logger(`check diff:${m.name}(${m.id})`, LOG_LEVEL_VERBOSE);
await serialized("plugin-" + m.id, async () => {
const old = await this.localDatabase.getDBEntry(p._id as string as FilePathWithPrefix /* This also should be explained */, null, false, false);
if (old !== false) {
const oldData = { data: old.data, deleted: old._deleted };
const newData = { data: d.data, deleted: d._deleted };
if (await isDocContentSame(oldData.data, newData.data) && oldData.deleted == newData.deleted) {
Logger(`Nothing changed:${m.name}`);
return;
}
}
await this.localDatabase.putDBEntry(d);
Logger(`Plugin saved:${m.name}`, logLevel);
});
} catch (ex) {
Logger(`Plugin save failed:${m.name}`, LOG_LEVEL_NOTICE);
} finally {
oldDocs.rows = oldDocs.rows.filter((e) => e.id != pluginDataEntryID);
}
//remove saved plugin data.
}
);
await Promise.all(procs);
const delDocs = oldDocs.rows.map((e) => {
// e.doc._deleted = true;
if (e.doc.type == "newnote" || e.doc.type == "plain") {
e.doc.deleted = true;
if (this.settings.deleteMetadataOfDeletedFiles) {
e.doc._deleted = true;
}
} else {
e.doc._deleted = true;
}
return e.doc;
});
Logger(`Deleting old plugin:(${delDocs.length})`, LOG_LEVEL_VERBOSE);
await this.localDatabase.bulkDocsRaw(delDocs);
Logger(`Scan plugin done.`, logLevel);
});
}
async applyPluginData(plugin: PluginDataEntry) {
await serialized("plugin-" + plugin.manifest.id, async () => {
const pluginTargetFolderPath = normalizePath(plugin.manifest.dir) + "/";
// @ts-ignore
const stat = this.app.plugins.enabledPlugins.has(plugin.manifest.id) == true;
if (stat) {
// @ts-ignore
await this.app.plugins.unloadPlugin(plugin.manifest.id);
Logger(`Unload plugin:${plugin.manifest.id}`, LOG_LEVEL_NOTICE);
}
if (plugin.dataJson)
await this.plugin.vaultAccess.adapterWrite(pluginTargetFolderPath + "data.json", plugin.dataJson);
Logger("wrote:" + pluginTargetFolderPath + "data.json", LOG_LEVEL_NOTICE);
if (stat) {
// @ts-ignore
await this.app.plugins.loadPlugin(plugin.manifest.id);
Logger(`Load plugin:${plugin.manifest.id}`, LOG_LEVEL_NOTICE);
}
});
}
async applyPlugin(plugin: PluginDataEntry) {
await serialized("plugin-" + plugin.manifest.id, async () => {
// @ts-ignore
const stat = this.app.plugins.enabledPlugins.has(plugin.manifest.id) == true;
if (stat) {
// @ts-ignore
await this.app.plugins.unloadPlugin(plugin.manifest.id);
Logger(`Unload plugin:${plugin.manifest.id}`, LOG_LEVEL_NOTICE);
}
const pluginTargetFolderPath = normalizePath(plugin.manifest.dir) + "/";
if ((await this.plugin.vaultAccess.adapterExists(pluginTargetFolderPath)) === false) {
await this.app.vault.adapter.mkdir(pluginTargetFolderPath);
}
await this.plugin.vaultAccess.adapterWrite(pluginTargetFolderPath + "main.js", plugin.mainJs);
await this.plugin.vaultAccess.adapterWrite(pluginTargetFolderPath + "manifest.json", plugin.manifestJson);
if (plugin.styleCss)
await this.plugin.vaultAccess.adapterWrite(pluginTargetFolderPath + "styles.css", plugin.styleCss);
if (stat) {
// @ts-ignore
await this.app.plugins.loadPlugin(plugin.manifest.id);
Logger(`Load plugin:${plugin.manifest.id}`, LOG_LEVEL_NOTICE);
}
});
}
}

33
src/CmdSetupLiveSync.ts
View File

@@ -50,7 +50,7 @@ export class SetupLiveSync extends LiveSyncCommands {
const encryptingPassphrase = await askString(this.app, "Encrypt your settings", "The passphrase to encrypt the setup URI", "", true); const encryptingPassphrase = await askString(this.app, "Encrypt your settings", "The passphrase to encrypt the setup URI", "", true);
if (encryptingPassphrase === false) if (encryptingPassphrase === false)
return; return;
const setting = { ...this.settings, configPassphraseStore: "", encryptedCouchDBConnection: "", encryptedPassphrase: "" }; const setting = { ...this.settings, configPassphraseStore: "", encryptedCouchDBConnection: "", encryptedPassphrase: "" } as Partial<ObsidianLiveSyncSettings>;
if (stripExtra) { if (stripExtra) {
delete setting.pluginSyncExtendedSetting; delete setting.pluginSyncExtendedSetting;
} }
@@ -134,7 +134,7 @@ export class SetupLiveSync extends LiveSyncCommands {
} else if (setupType == setupAsMerge) { } else if (setupType == setupAsMerge) {
this.plugin.settings = newSettingW; this.plugin.settings = newSettingW;
this.plugin.usedPassphrase = ""; this.plugin.usedPassphrase = "";
await this.fetchLocalWithKeepLocal(); await this.fetchLocalWithRebuild();
} else if (setupType == setupAgain) { } else if (setupType == setupAgain) {
const confirm = "I know this operation will rebuild all my databases with files on this device, and files that are on the remote database and I didn't synchronize to any other devices will be lost and want to proceed indeed."; const confirm = "I know this operation will rebuild all my databases with files on this device, and files that are on the remote database and I didn't synchronize to any other devices will be lost and want to proceed indeed.";
if (await askSelectString(this.app, "Do you really want to do this?", ["Cancel", confirm]) != confirm) { if (await askSelectString(this.app, "Do you really want to do this?", ["Cancel", confirm]) != confirm) {
@@ -359,7 +359,7 @@ Of course, we are able to disable these features.`
Logger(`Fetching chunks done`, LOG_LEVEL_NOTICE); Logger(`Fetching chunks done`, LOG_LEVEL_NOTICE);
} }
} }
async fetchLocal() { async fetchLocal(makeLocalChunkBeforeSync?: boolean) {
this.suspendExtraSync(); this.suspendExtraSync();
await this.askUseNewAdapter(); await this.askUseNewAdapter();
this.plugin.settings.isConfigured = true; this.plugin.settings.isConfigured = true;
@@ -368,35 +368,20 @@ Of course, we are able to disable these features.`
await this.resetLocalDatabase(); await this.resetLocalDatabase();
await delay(1000); await delay(1000);
await this.plugin.openDatabase(); await this.plugin.openDatabase();
await this.plugin.markRemoteResolved();
this.plugin.isReady = true; this.plugin.isReady = true;
if (makeLocalChunkBeforeSync) {
await this.plugin.initializeDatabase(true);
}
await this.plugin.markRemoteResolved();
await delay(500); await delay(500);
await this.plugin.replicateAllFromServer(true); await this.plugin.replicateAllFromServer(true);
await delay(1000); await delay(1000);
await this.plugin.replicateAllFromServer(true); await this.plugin.replicateAllFromServer(true);
await this.fetchRemoteChunks();
await this.resumeReflectingDatabase(); await this.resumeReflectingDatabase();
await this.askHiddenFileConfiguration({ enableFetch: true }); await this.askHiddenFileConfiguration({ enableFetch: true });
} }
async fetchLocalWithKeepLocal() { async fetchLocalWithRebuild() {
this.suspendExtraSync(); return await this.fetchLocal(true);
await this.askUseNewAdapter();
this.plugin.settings.isConfigured = true;
await this.suspendReflectingDatabase();
await this.plugin.realizeSettingSyncMode();
await this.resetLocalDatabase();
await delay(1000);
await this.plugin.initializeDatabase(true);
await this.plugin.openDatabase();
await this.plugin.markRemoteResolved();
this.plugin.isReady = true;
await delay(500);
await this.plugin.replicateAllFromServer(true);
await delay(1000);
await this.plugin.replicateAllFromServer(true);
await this.fetchRemoteChunks();
await this.resumeReflectingDatabase();
await this.askHiddenFileConfiguration({ enableFetch: true });
} }
async rebuildRemote() { async rebuildRemote() {
this.suspendExtraSync(); this.suspendExtraSync();

35
src/DocumentHistoryModal.ts
View File

@@ -5,7 +5,7 @@ import ObsidianLiveSyncPlugin from "./main";
import { type DocumentID, type FilePathWithPrefix, type LoadedEntry, LOG_LEVEL_INFO, LOG_LEVEL_NOTICE, LOG_LEVEL_VERBOSE } from "./lib/src/types"; import { type DocumentID, type FilePathWithPrefix, type LoadedEntry, LOG_LEVEL_INFO, LOG_LEVEL_NOTICE, LOG_LEVEL_VERBOSE } from "./lib/src/types";
import { Logger } from "./lib/src/logger"; import { Logger } from "./lib/src/logger";
import { isErrorOfMissingDoc } from "./lib/src/utils_couchdb"; import { isErrorOfMissingDoc } from "./lib/src/utils_couchdb";
import { getDocData } from "./lib/src/utils"; import { getDocData, readContent } from "./lib/src/utils";
import { isPlainText, stripPrefix } from "./lib/src/path"; import { isPlainText, stripPrefix } from "./lib/src/path";
function isImage(path: string) { function isImage(path: string) {
@@ -25,7 +25,7 @@ function readDocument(w: LoadedEntry) {
if (isImage(w.path)) { if (isImage(w.path)) {
return new Uint8Array(decodeBinary(w.data)); return new Uint8Array(decodeBinary(w.data));
} }
if (w.data == "plain") return getDocData(w.data); if (w.type == "plain" || w.datatype == "plain") return getDocData(w.data);
if (isComparableTextDecode(w.path)) return readString(new Uint8Array(decodeBinary(w.data))); if (isComparableTextDecode(w.path)) return readString(new Uint8Array(decodeBinary(w.data)));
if (isComparableText(w.path)) return getDocData(w.data); if (isComparableText(w.path)) return getDocData(w.data);
try { try {
@@ -66,7 +66,7 @@ export class DocumentHistoryModal extends Modal {
} }
} }
async loadFile(initialRev: string) { async loadFile(initialRev?: string) {
if (!this.id) { if (!this.id) {
this.id = await this.plugin.path2id(this.file); this.id = await this.plugin.path2id(this.file);
} }
@@ -109,7 +109,7 @@ export class DocumentHistoryModal extends Modal {
if (v) { if (v) {
URL.revokeObjectURL(v); URL.revokeObjectURL(v);
} }
this.BlobURLs.set(key, undefined); this.BlobURLs.delete(key);
} }
generateBlobURL(key: string, data: Uint8Array) { generateBlobURL(key: string, data: Uint8Array) {
this.revokeURL(key); this.revokeURL(key);
@@ -247,12 +247,10 @@ export class DocumentHistoryModal extends Modal {
Logger(`Old content copied to clipboard`, LOG_LEVEL_NOTICE); Logger(`Old content copied to clipboard`, LOG_LEVEL_NOTICE);
}); });
}); });
async function focusFile(path: string) { const focusFile = async (path: string) => {
const targetFile = app.vault const targetFile = this.plugin.app.vault.getFileByPath(path);
.getFiles()
.find((f) => f.path === path);
if (targetFile) { if (targetFile) {
const leaf = app.workspace.getLeaf(false); const leaf = this.plugin.app.workspace.getLeaf(false);
await leaf.openFile(targetFile); await leaf.openFile(targetFile);
} else { } else {
Logger("The file could not view on the editor", LOG_LEVEL_NOTICE) Logger("The file could not view on the editor", LOG_LEVEL_NOTICE)
@@ -265,19 +263,16 @@ export class DocumentHistoryModal extends Modal {
const pathToWrite = stripPrefix(this.file); const pathToWrite = stripPrefix(this.file);
if (!isValidPath(pathToWrite)) { if (!isValidPath(pathToWrite)) {
Logger("Path is not valid to write content.", LOG_LEVEL_INFO); Logger("Path is not valid to write content.", LOG_LEVEL_INFO);
return;
} }
if (this.currentDoc?.datatype == "plain") { if (!this.currentDoc) {
await this.plugin.vaultAccess.adapterWrite(pathToWrite, getDocData(this.currentDoc.data)); Logger("No active file loaded.", LOG_LEVEL_INFO);
await focusFile(pathToWrite); return;
this.close();
} else if (this.currentDoc?.datatype == "newnote") {
await this.plugin.vaultAccess.adapterWrite(pathToWrite, decodeBinary(this.currentDoc.data));
await focusFile(pathToWrite);
this.close();
} else {
Logger(`Could not parse entry`, LOG_LEVEL_NOTICE);
} }
const d = readContent(this.currentDoc);
await this.plugin.vaultAccess.adapterWrite(pathToWrite, d);
await focusFile(pathToWrite);
this.close();
}); });
}); });
} }

26
src/GlobalHistory.svelte
View File

@@ -2,7 +2,7 @@
import ObsidianLiveSyncPlugin from "./main"; import ObsidianLiveSyncPlugin from "./main";
import { onDestroy, onMount } from "svelte"; import { onDestroy, onMount } from "svelte";
import type { AnyEntry, FilePathWithPrefix } from "./lib/src/types"; import type { AnyEntry, FilePathWithPrefix } from "./lib/src/types";
import { createBinaryBlob, getDocData, isDocContentSame } from "./lib/src/utils"; import { getDocData, isAnyNote, isDocContentSame, readAsBlob } from "./lib/src/utils";
import { diff_match_patch } from "./deps"; import { diff_match_patch } from "./deps";
import { DocumentHistoryModal } from "./DocumentHistoryModal"; import { DocumentHistoryModal } from "./DocumentHistoryModal";
import { isPlainText, stripAllPrefixes } from "./lib/src/path"; import { isPlainText, stripAllPrefixes } from "./lib/src/path";
@@ -30,7 +30,7 @@
type HistoryData = { type HistoryData = {
id: string; id: string;
rev: string; rev?: string;
path: string; path: string;
dirname: string; dirname: string;
filename: string; filename: string;
@@ -53,12 +53,12 @@
if (docA.mtime < range_from_epoch) { if (docA.mtime < range_from_epoch) {
continue; continue;
} }
if (docA.type != "newnote" && docA.type != "plain") continue; if (!isAnyNote(docA)) continue;
const path = plugin.getPath(docA as AnyEntry); const path = plugin.getPath(docA as AnyEntry);
const isPlain = isPlainText(docA.path); const isPlain = isPlainText(docA.path);
const revs = await db.getRaw(docA._id, { revs_info: true }); const revs = await db.getRaw(docA._id, { revs_info: true });
let p: string = undefined; let p: string | undefined = undefined;
const reversedRevs = revs._revs_info.reverse(); const reversedRevs = (revs._revs_info ?? []).reverse();
const DIFF_DELETE = -1; const DIFF_DELETE = -1;
const DIFF_EQUAL = 0; const DIFF_EQUAL = 0;
@@ -106,15 +106,9 @@
if (checkStorageDiff) { if (checkStorageDiff) {
const abs = plugin.vaultAccess.getAbstractFileByPath(stripAllPrefixes(plugin.getPath(docA))); const abs = plugin.vaultAccess.getAbstractFileByPath(stripAllPrefixes(plugin.getPath(docA)));
if (abs instanceof TFile) { if (abs instanceof TFile) {
let result = false; const data = await plugin.vaultAccess.adapterReadAuto(abs);
if (isPlainText(docA.path)) { const d = readAsBlob(doc);
const data = await plugin.vaultAccess.adapterRead(abs); const result = await isDocContentSame(data, d);
result = await isDocContentSame(data, doc.data);
} else {
const data = await plugin.vaultAccess.adapterReadBinary(abs);
const dataEEncoded = createBinaryBlob(data);
result = await isDocContentSame(dataEEncoded, doc.data);
}
if (result) { if (result) {
diffDetail += " ⚖️"; diffDetail += " ⚖️";
} else { } else {
@@ -183,7 +177,7 @@
onDestroy(() => {}); onDestroy(() => {});
function showHistory(file: string, rev: string) { function showHistory(file: string, rev: string) {
new DocumentHistoryModal(plugin.app, plugin, file as unknown as FilePathWithPrefix, null, rev).open(); new DocumentHistoryModal(plugin.app, plugin, file as unknown as FilePathWithPrefix, undefined, rev).open();
} }
function openFile(file: string) { function openFile(file: string) {
plugin.app.workspace.openLinkText(file, file); plugin.app.workspace.openLinkText(file, file);
@@ -238,7 +232,7 @@
<td> <td>
<span class="rev"> <span class="rev">
{#if entry.isPlain} {#if entry.isPlain}
<a on:click={() => showHistory(entry.path, entry.rev)}>{entry.rev}</a> <a on:click={() => showHistory(entry.path, entry?.rev || "")}>{entry.rev}</a>
{:else} {:else}
{entry.rev} {entry.rev}
{/if} {/if}

31
src/JsonResolvePane.svelte
View File

@@ -6,15 +6,15 @@
import { mergeObject } from "./utils"; import { mergeObject } from "./utils";
export let docs: LoadedEntry[] = []; export let docs: LoadedEntry[] = [];
export let callback: (keepRev: string, mergedStr?: string) => Promise<void> = async (_, __) => { export let callback: (keepRev?: string, mergedStr?: string) => Promise<void> = async (_, __) => {
Promise.resolve(); Promise.resolve();
}; };
export let filename: FilePath = "" as FilePath; export let filename: FilePath = "" as FilePath;
export let nameA: string = "A"; export let nameA: string = "A";
export let nameB: string = "B"; export let nameB: string = "B";
export let defaultSelect: string = ""; export let defaultSelect: string = "";
let docA: LoadedEntry = undefined; let docA: LoadedEntry;
let docB: LoadedEntry = undefined; let docB: LoadedEntry;
let docAContent = ""; let docAContent = "";
let docBContent = ""; let docBContent = "";
let objA: any = {}; let objA: any = {};
@@ -28,7 +28,8 @@
function docToString(doc: LoadedEntry) { function docToString(doc: LoadedEntry) {
return doc.datatype == "plain" ? getDocData(doc.data) : readString(new Uint8Array(decodeBinary(doc.data))); return doc.datatype == "plain" ? getDocData(doc.data) : readString(new Uint8Array(decodeBinary(doc.data)));
} }
function revStringToRevNumber(rev: string) { function revStringToRevNumber(rev?: string) {
if (!rev) return "";
return rev.split("-")[0]; return rev.split("-")[0];
} }
@@ -44,15 +45,15 @@
} }
function apply() { function apply() {
if (docA._id == docB._id) { if (docA._id == docB._id) {
if (mode == "A") return callback(docA._rev, null); if (mode == "A") return callback(docA._rev!, undefined);
if (mode == "B") return callback(docB._rev, null); if (mode == "B") return callback(docB._rev!, undefined);
} else { } else {
if (mode == "A") return callback(null, docToString(docA)); if (mode == "A") return callback(undefined, docToString(docA));
if (mode == "B") return callback(null, docToString(docB)); if (mode == "B") return callback(undefined, docToString(docB));
} }
if (mode == "BA") return callback(null, JSON.stringify(objBA, null, 2)); if (mode == "BA") return callback(undefined, JSON.stringify(objBA, null, 2));
if (mode == "AB") return callback(null, JSON.stringify(objAB, null, 2)); if (mode == "AB") return callback(undefined, JSON.stringify(objAB, null, 2));
callback(null, null); callback(undefined, undefined);
} }
$: { $: {
if (docs && docs.length >= 1) { if (docs && docs.length >= 1) {
@@ -133,13 +134,17 @@
{/if} {/if}
<div> <div>
{nameA} {nameA}
{#if docA._id == docB._id} Rev:{revStringToRevNumber(docA._rev)} {/if} ,{new Date(docA.mtime).toLocaleString()} {#if docA._id == docB._id}
Rev:{revStringToRevNumber(docA._rev)}
{/if} ,{new Date(docA.mtime).toLocaleString()}
{docAContent.length} letters {docAContent.length} letters
</div> </div>
<div> <div>
{nameB} {nameB}
{#if docA._id == docB._id} Rev:{revStringToRevNumber(docB._rev)} {/if} ,{new Date(docB.mtime).toLocaleString()} {#if docA._id == docB._id}
Rev:{revStringToRevNumber(docB._rev)}
{/if} ,{new Date(docB.mtime).toLocaleString()}
{docBContent.length} letters {docBContent.length} letters
</div> </div>

5
src/KeyValueDB.ts
View File

@@ -6,7 +6,7 @@ export interface KeyValueDatabase {
clear(): Promise<void>; clear(): Promise<void>;
keys(query?: IDBValidKey | IDBKeyRange, count?: number): Promise<IDBValidKey[]>; keys(query?: IDBValidKey | IDBKeyRange, count?: number): Promise<IDBValidKey[]>;
close(): void; close(): void;
destroy(): void; destroy(): Promise<void>;
} }
const databaseCache: { [key: string]: IDBPDatabase<any> } = {}; const databaseCache: { [key: string]: IDBPDatabase<any> } = {};
export const OpenKeyValueDatabase = async (dbKey: string): Promise<KeyValueDatabase> => { export const OpenKeyValueDatabase = async (dbKey: string): Promise<KeyValueDatabase> => {
@@ -20,8 +20,7 @@ export const OpenKeyValueDatabase = async (dbKey: string): Promise<KeyValueDatab
db.createObjectStore(storeKey); db.createObjectStore(storeKey);
}, },
}); });
let db: IDBPDatabase<any> = null; const db = await dbPromise;
db = await dbPromise;
databaseCache[dbKey] = db; databaseCache[dbKey] = db;
return { return {
get<T>(key: string): Promise<T> { get<T>(key: string): Promise<T> {

83
src/MultipleRegExpControl.svelte Normal file
View File

@@ -0,0 +1,83 @@
<script lang="ts">
export let patterns = [] as string[];
export let originals = [] as string[];
export let apply: (args: string[]) => Promise<void> = (_: string[]) => Promise.resolve();
function revert() {
patterns = [...originals];
}
const CHECK_OK = "✔";
const CHECK_NG = "⚠";
const MARK_MODIFIED = "✏ ";
function checkRegExp(pattern: string) {
if (pattern.trim() == "") return "";
try {
const _ = new RegExp(pattern);
return CHECK_OK;
} catch (ex) {
return CHECK_NG;
}
}
$: status = patterns.map((e) => checkRegExp(e));
$: modified = patterns.map((e, i) => (e != originals?.[i] ?? "" ? MARK_MODIFIED : ""));
function remove(idx: number) {
patterns[idx] = "";
}
function add() {
patterns = [...patterns, ""];
}
</script>
<ul>
{#each patterns as pattern, idx}
<li><label>{modified[idx]}{status[idx]}</label><input type="text" bind:value={pattern} class={modified[idx]} /><button class="iconbutton" on:click={() => remove(idx)}>🗑</button></li>
{/each}
<li>
<label><button on:click={() => add()}>Add</button></label>
</li>
<li class="buttons">
<button on:click={() => apply(patterns)} disabled={status.some((e) => e == CHECK_NG) || modified.every((e) => e == "")}>Apply</button>
<button on:click={() => revert()} disabled={status.some((e) => e == CHECK_NG) || modified.every((e) => e == "")}>Revert</button>
</li>
</ul>
<style>
label {
min-width: 4em;
width: 4em;
display: inline-flex;
flex-direction: row;
justify-content: flex-end;
}
ul {
flex-grow: 1;
display: inline-flex;
flex-direction: column;
list-style-type: none;
margin-block-start: 0;
margin-block-end: 0;
margin-inline-start: 0px;
margin-inline-end: 0px;
padding-inline-start: 0;
}
li {
padding: var(--size-2-1) var(--size-4-1);
display: inline-flex;
flex-grow: 1;
align-items: center;
justify-content: flex-end;
gap: var(--size-4-2);
}
li input {
min-width: 10em;
}
li.buttons {
}
button.iconbutton {
max-width: 4em;
}
span.spacer {
flex-grow: 1;
}
</style>

347
src/ObsidianLiveSyncSettingTab.ts
View File

@@ -1,13 +1,15 @@
import { App, PluginSettingTab, Setting, sanitizeHTMLToDom, TextAreaComponent, MarkdownRenderer, stringifyYaml } from "./deps"; import { App, PluginSettingTab, Setting, sanitizeHTMLToDom, MarkdownRenderer, stringifyYaml } from "./deps";
import { DEFAULT_SETTINGS, type ObsidianLiveSyncSettings, type ConfigPassphraseStore, type RemoteDBSettings, type FilePathWithPrefix, type HashAlgorithm, type DocumentID, LOG_LEVEL_NOTICE, LOG_LEVEL_VERBOSE, LOG_LEVEL_INFO } from "./lib/src/types"; import { DEFAULT_SETTINGS, type ObsidianLiveSyncSettings, type ConfigPassphraseStore, type RemoteDBSettings, type FilePathWithPrefix, type HashAlgorithm, type DocumentID, LOG_LEVEL_NOTICE, LOG_LEVEL_VERBOSE, LOG_LEVEL_INFO, type LoadedEntry, PREFERRED_SETTING_CLOUDANT, PREFERRED_SETTING_SELF_HOSTED, FLAGMD_REDFLAG2_HR, FLAGMD_REDFLAG3_HR } from "./lib/src/types";
import { createBinaryBlob, createTextBlob, delay, isDocContentSame } from "./lib/src/utils"; import { createBlob, delay, isDocContentSame, readAsBlob } from "./lib/src/utils";
import { versionNumberString2Number } from "./lib/src/strbin"; import { versionNumberString2Number } from "./lib/src/strbin";
import { Logger } from "./lib/src/logger"; import { Logger } from "./lib/src/logger";
import { checkSyncInfo, isCloudantURI } from "./lib/src/utils_couchdb"; import { checkSyncInfo, isCloudantURI } from "./lib/src/utils_couchdb";
import { testCrypt } from "./lib/src/e2ee_v2"; import { testCrypt } from "./lib/src/e2ee_v2";
import ObsidianLiveSyncPlugin from "./main"; import ObsidianLiveSyncPlugin from "./main";
import { askYesNo, performRebuildDB, requestToCouchDB, scheduleTask } from "./utils"; import { askYesNo, performRebuildDB, requestToCouchDB, scheduleTask } from "./utils";
import { request, type ButtonComponent } from "obsidian"; import { request, type ButtonComponent, TFile } from "obsidian";
import { shouldBeIgnored } from "./lib/src/path";
import MultipleRegExpControl from './MultipleRegExpControl.svelte';
export class ObsidianLiveSyncSettingTab extends PluginSettingTab { export class ObsidianLiveSyncSettingTab extends PluginSettingTab {
@@ -45,11 +47,6 @@ export class ObsidianLiveSyncSettingTab extends PluginSettingTab {
let useDynamicIterationCount = this.plugin.settings.useDynamicIterationCount; let useDynamicIterationCount = this.plugin.settings.useDynamicIterationCount;
containerEl.empty(); containerEl.empty();
// const preferred_setting = isCloudantURI(this.plugin.settings.couchDB_URI) ? PREFERRED_SETTING_CLOUDANT : PREFERRED_SETTING_SELF_HOSTED;
// const default_setting = { ...DEFAULT_SETTINGS };
containerEl.createEl("h2", { text: "Settings for Self-hosted LiveSync." }); containerEl.createEl("h2", { text: "Settings for Self-hosted LiveSync." });
containerEl.addClass("sls-setting"); containerEl.addClass("sls-setting");
containerEl.removeClass("isWizard"); containerEl.removeClass("isWizard");
@@ -85,11 +82,11 @@ export class ObsidianLiveSyncSettingTab extends PluginSettingTab {
} }
w.querySelectorAll(`.sls-setting-label`).forEach((element) => { w.querySelectorAll(`.sls-setting-label`).forEach((element) => {
element.removeClass("selected"); element.removeClass("selected");
(element.querySelector<HTMLInputElement>("input[type=radio]")).checked = false; (element.querySelector<HTMLInputElement>("input[type=radio]"))!.checked = false;
}); });
w.querySelectorAll(`.sls-setting-label.c-${screen}`).forEach((element) => { w.querySelectorAll(`.sls-setting-label.c-${screen}`).forEach((element) => {
element.addClass("selected"); element.addClass("selected");
(element.querySelector<HTMLInputElement>("input[type=radio]")).checked = true; (element.querySelector<HTMLInputElement>("input[type=radio]"))!.checked = true;
}); });
this.selectedScreen = screen; this.selectedScreen = screen;
}; };
@@ -119,7 +116,7 @@ export class ObsidianLiveSyncSettingTab extends PluginSettingTab {
tmpDiv.innerHTML = `<button> OK, I read all. </button>`; tmpDiv.innerHTML = `<button> OK, I read all. </button>`;
if (lastVersion > this.plugin.settings.lastReadUpdates) { if (lastVersion > this.plugin.settings.lastReadUpdates) {
const informationButtonDiv = h3El.appendChild(tmpDiv); const informationButtonDiv = h3El.appendChild(tmpDiv);
informationButtonDiv.querySelector("button").addEventListener("click", async () => { informationButtonDiv.querySelector("button")?.addEventListener("click", async () => {
this.plugin.settings.lastReadUpdates = lastVersion; this.plugin.settings.lastReadUpdates = lastVersion;
await this.plugin.saveSettings(); await this.plugin.saveSettings();
informationButtonDiv.remove(); informationButtonDiv.remove();
@@ -229,23 +226,23 @@ export class ObsidianLiveSyncSettingTab extends PluginSettingTab {
let remoteTroubleShootMDSrc = ""; let remoteTroubleShootMDSrc = "";
try { try {
remoteTroubleShootMDSrc = await request(`${rawRepoURI}${basePath}/${filename}`); remoteTroubleShootMDSrc = await request(`${rawRepoURI}${basePath}/${filename}`);
} catch (ex) { } catch (ex: any) {
remoteTroubleShootMDSrc = "Error Occurred!!\n" + ex.toString(); remoteTroubleShootMDSrc = "Error Occurred!!\n" + ex.toString();
} }
const remoteTroubleShootMD = remoteTroubleShootMDSrc.replace(/\((.*?(.png)|(.jpg))\)/g, `(${rawRepoURI}${basePath}/$1)`) const remoteTroubleShootMD = remoteTroubleShootMDSrc.replace(/\((.*?(.png)|(.jpg))\)/g, `(${rawRepoURI}${basePath}/$1)`)
// Render markdown // Render markdown
await MarkdownRenderer.render(this.plugin.app, `<a class='sls-troubleshoot-anchor'></a> [Tips and Troubleshooting](${topPath}) [PageTop](${filename})\n\n${remoteTroubleShootMD}`, troubleShootEl, `${rawRepoURI}`, this.plugin); await MarkdownRenderer.render(this.plugin.app, `<a class='sls-troubleshoot-anchor'></a> [Tips and Troubleshooting](${topPath}) [PageTop](${filename})\n\n${remoteTroubleShootMD}`, troubleShootEl, `${rawRepoURI}`, this.plugin);
// Menu // Menu
troubleShootEl.querySelector<HTMLAnchorElement>(".sls-troubleshoot-anchor") troubleShootEl.querySelector<HTMLAnchorElement>(".sls-troubleshoot-anchor")?.parentElement?.setCssStyles({ position: "sticky", top: "-1em", backgroundColor: "var(--modal-background)" });
.parentElement.setCssStyles({ position: "sticky", top: "-1em", backgroundColor: "var(--modal-background)" });
// Trap internal links. // Trap internal links.
troubleShootEl.querySelectorAll<HTMLAnchorElement>("a.internal-link").forEach((anchorEl) => { troubleShootEl.querySelectorAll<HTMLAnchorElement>("a.internal-link").forEach((anchorEl) => {
anchorEl.addEventListener("click", async (evt) => { anchorEl.addEventListener("click", async (evt) => {
const uri = anchorEl.getAttr("data-href"); const uri = anchorEl.getAttr("data-href");
if (!uri) return;
if (uri.startsWith("#")) { if (uri.startsWith("#")) {
evt.preventDefault(); evt.preventDefault();
const elements = Array.from(troubleShootEl.querySelectorAll<HTMLHeadingElement>("[data-heading]")) const elements = Array.from(troubleShootEl.querySelectorAll<HTMLHeadingElement>("[data-heading]"))
const p = elements.find(e => e.getAttr("data-heading").toLowerCase().split(" ").join("-") == uri.substring(1).toLowerCase()); const p = elements.find(e => e.getAttr("data-heading")?.toLowerCase().split(" ").join("-") == uri.substring(1).toLowerCase());
if (p) { if (p) {
p.setCssStyles({ scrollMargin: "3em" }); p.setCssStyles({ scrollMargin: "3em" });
p.scrollIntoView({ behavior: "instant", block: "start" }); p.scrollIntoView({ behavior: "instant", block: "start" });
@@ -266,6 +263,16 @@ export class ObsidianLiveSyncSettingTab extends PluginSettingTab {
const containerRemoteDatabaseEl = containerEl.createDiv(); const containerRemoteDatabaseEl = containerEl.createDiv();
containerRemoteDatabaseEl.createEl("h3", { text: "Remote Database configuration" }); containerRemoteDatabaseEl.createEl("h3", { text: "Remote Database configuration" });
const syncWarn = containerRemoteDatabaseEl.createEl("div", { text: `These settings are kept locked while any synchronization options are enabled. Disable these options in the "Sync Settings" tab to unlock.` }); const syncWarn = containerRemoteDatabaseEl.createEl("div", { text: `These settings are kept locked while any synchronization options are enabled. Disable these options in the "Sync Settings" tab to unlock.` });
if (this.plugin.settings.couchDB_URI.startsWith("http://")) {
if (this.plugin.isMobile) {
containerRemoteDatabaseEl.createEl("div", { text: `Configured as using plain HTTP. We cannot connect to the remote. Please set up the credentials and use HTTPS for the remote URI.` })
.addClass("op-warn");
} else {
containerRemoteDatabaseEl.createEl("div", { text: `Configured as using plain HTTP. We might fail on mobile devices.` })
.addClass("op-warn-info");
}
}
syncWarn.addClass("op-warn-info"); syncWarn.addClass("op-warn-info");
syncWarn.addClass("sls-hidden"); syncWarn.addClass("sls-hidden");
@@ -403,7 +410,7 @@ export class ObsidianLiveSyncSettingTab extends PluginSettingTab {
tmpDiv.addClass("ob-btn-config-fix"); tmpDiv.addClass("ob-btn-config-fix");
tmpDiv.innerHTML = `<label>${title}</label><button>Fix</button>`; tmpDiv.innerHTML = `<label>${title}</label><button>Fix</button>`;
const x = checkResultDiv.appendChild(tmpDiv); const x = checkResultDiv.appendChild(tmpDiv);
x.querySelector("button").addEventListener("click", async () => { x.querySelector("button")?.addEventListener("click", async () => {
Logger(`CouchDB Configuration: ${title} -> Set ${key} to ${value}`) Logger(`CouchDB Configuration: ${title} -> Set ${key} to ${value}`)
const res = await requestToCouchDB(this.plugin.settings.couchDB_URI, this.plugin.settings.couchDB_USER, this.plugin.settings.couchDB_PASSWORD, undefined, key, value); const res = await requestToCouchDB(this.plugin.settings.couchDB_URI, this.plugin.settings.couchDB_USER, this.plugin.settings.couchDB_PASSWORD, undefined, key, value);
if (res.status == 200) { if (res.status == 200) {
@@ -499,15 +506,11 @@ export class ObsidianLiveSyncSettingTab extends PluginSettingTab {
const origins = ["app://obsidian.md", "capacitor://localhost", "http://localhost"]; const origins = ["app://obsidian.md", "capacitor://localhost", "http://localhost"];
for (const org of origins) { for (const org of origins) {
const rr = await requestToCouchDB(this.plugin.settings.couchDB_URI, this.plugin.settings.couchDB_USER, this.plugin.settings.couchDB_PASSWORD, org); const rr = await requestToCouchDB(this.plugin.settings.couchDB_URI, this.plugin.settings.couchDB_USER, this.plugin.settings.couchDB_PASSWORD, org);
const responseHeaders = Object.entries(rr.headers) const responseHeaders = Object.fromEntries(Object.entries(rr.headers)
.map((e) => { .map((e) => {
e[0] = (e[0] + "").toLowerCase(); e[0] = `${e[0]}`.toLowerCase();
return e; return e;
}) }));
.reduce((obj, [key, val]) => {
obj[key] = val;
return obj;
}, {} as { [key: string]: string });
addResult(`Origin check:${org}`); addResult(`Origin check:${org}`);
if (responseHeaders["access-control-allow-credentials"] != "true") { if (responseHeaders["access-control-allow-credentials"] != "true") {
addResult("❗ CORS is not allowing credential"); addResult("❗ CORS is not allowing credential");
@@ -523,7 +526,7 @@ export class ObsidianLiveSyncSettingTab extends PluginSettingTab {
addResult("--Done--", ["ob-btn-config-head"]); addResult("--Done--", ["ob-btn-config-head"]);
addResult("If you have some trouble with Connection-check even though all Config-check has been passed, Please check your reverse proxy's configuration.", ["ob-btn-config-info"]); addResult("If you have some trouble with Connection-check even though all Config-check has been passed, Please check your reverse proxy's configuration.", ["ob-btn-config-info"]);
Logger(`Checking configuration done`, LOG_LEVEL_INFO); Logger(`Checking configuration done`, LOG_LEVEL_INFO);
} catch (ex) { } catch (ex: any) {
if (ex?.status == 401) { if (ex?.status == 401) {
addResult(`❗ Access forbidden.`); addResult(`❗ Access forbidden.`);
addResult(`We could not continue the test.`); addResult(`We could not continue the test.`);
@@ -732,7 +735,7 @@ export class ObsidianLiveSyncSettingTab extends PluginSettingTab {
} }
}; };
const rebuildDB = async (method: "localOnly" | "remoteOnly" | "rebuildBothByThisDevice") => { const rebuildDB = async (method: "localOnly" | "remoteOnly" | "rebuildBothByThisDevice" | "localOnlyWithChunks") => {
if (encrypt && passphrase == "") { if (encrypt && passphrase == "") {
Logger("If you enable encryption, you have to set the passphrase", LOG_LEVEL_NOTICE); Logger("If you enable encryption, you have to set the passphrase", LOG_LEVEL_NOTICE);
return; return;
@@ -773,9 +776,10 @@ export class ObsidianLiveSyncSettingTab extends PluginSettingTab {
this.plugin.settings.passphrase = ""; this.plugin.settings.passphrase = "";
} }
if (isCloudantURI(this.plugin.settings.couchDB_URI)) { if (isCloudantURI(this.plugin.settings.couchDB_URI)) {
this.plugin.settings.customChunkSize = 0; // this.plugin.settings.customChunkSize = 0;
this.plugin.settings = { ...this.plugin.settings, ...PREFERRED_SETTING_CLOUDANT };
} else { } else {
this.plugin.settings.customChunkSize = 50; this.plugin.settings = { ...this.plugin.settings, ...PREFERRED_SETTING_SELF_HOSTED };
} }
changeDisplay("30") changeDisplay("30")
}) })
@@ -791,7 +795,7 @@ export class ObsidianLiveSyncSettingTab extends PluginSettingTab {
new Setting(containerGeneralSettingsEl) new Setting(containerGeneralSettingsEl)
.setName("Show status inside the editor") .setName("Show status inside the editor")
.setDesc("") .setDesc("Reflected after reboot")
.addToggle((toggle) => .addToggle((toggle) =>
toggle.setValue(this.plugin.settings.showStatusOnEditor).onChange(async (value) => { toggle.setValue(this.plugin.settings.showStatusOnEditor).onChange(async (value) => {
this.plugin.settings.showStatusOnEditor = value; this.plugin.settings.showStatusOnEditor = value;
@@ -810,7 +814,16 @@ export class ObsidianLiveSyncSettingTab extends PluginSettingTab {
}) })
); );
} }
new Setting(containerGeneralSettingsEl)
.setName("Show status on the status bar")
.setDesc("Reflected after reboot.")
.addToggle((toggle) =>
toggle.setValue(this.plugin.settings.showStatusOnStatusbar).onChange(async (value) => {
this.plugin.settings.showStatusOnStatusbar = value;
await this.plugin.saveSettings();
this.display();
})
);
containerGeneralSettingsEl.createEl("h4", { text: "Logging" }); containerGeneralSettingsEl.createEl("h4", { text: "Logging" });
new Setting(containerGeneralSettingsEl) new Setting(containerGeneralSettingsEl)
.setName("Show only notifications") .setName("Show only notifications")
@@ -1055,7 +1068,7 @@ export class ObsidianLiveSyncSettingTab extends PluginSettingTab {
await this.plugin.saveSettings(); await this.plugin.saveSettings();
await rebuildDB("localOnly"); await rebuildDB("localOnly");
Logger("All done! Please set up subsequent devices with 'Copy current settings as a new setup URI' and 'Use the copied setup URI'.", LOG_LEVEL_NOTICE); Logger("All done! Please set up subsequent devices with 'Copy current settings as a new setup URI' and 'Use the copied setup URI'.", LOG_LEVEL_NOTICE);
await this.plugin.addOnSetup.command_openSetupURI(); await this.plugin.addOnSetup.command_copySetupURI();
} else { } else {
this.askReload(); this.askReload();
} }
@@ -1325,43 +1338,48 @@ export class ObsidianLiveSyncSettingTab extends PluginSettingTab {
}); });
text.inputEl.setAttribute("type", "number"); text.inputEl.setAttribute("type", "number");
}); });
let skipPatternTextArea: TextAreaComponent = null; const defaultSkipPattern = "\\/node_modules\\/, \\/\\.git\\/, ^\\.git\\/, \\/obsidian-livesync\\/";
const defaultSkipPattern = "\\/node_modules\\/, \\/\\.git\\/, \\/obsidian-livesync\\/";
const defaultSkipPatternXPlat = defaultSkipPattern + ",\\/workspace$ ,\\/workspace.json$,\\/workspace-mobile.json$"; const defaultSkipPatternXPlat = defaultSkipPattern + ",\\/workspace$ ,\\/workspace.json$,\\/workspace-mobile.json$";
new Setting(containerSyncSettingEl)
.setName("Folders and files to ignore") const pat = this.plugin.settings.syncInternalFilesIgnorePatterns.split(",").map(x => x.trim()).filter(x => x != "");
.setDesc( const patSetting = new Setting(containerSyncSettingEl)
"Regular expression, If you use hidden file sync between desktop and mobile, adding `workspace$` is recommended." .setName("Hidden files ignore patterns")
) .setDesc("");
.setClass("wizardHidden")
.addTextArea((text) => { new MultipleRegExpControl(
text {
.setValue(this.plugin.settings.syncInternalFilesIgnorePatterns) target: patSetting.controlEl,
.setPlaceholder("\\/node_modules\\/, \\/\\.git\\/") props: {
.onChange(async (value) => { patterns: pat, originals: [...pat], apply: async (newPatterns) => {
this.plugin.settings.syncInternalFilesIgnorePatterns = value; this.plugin.settings.syncInternalFilesIgnorePatterns = newPatterns.map(e => e.trim()).filter(e => e != "").join(", ");
await this.plugin.saveSettings(); await this.plugin.saveSettings();
}) this.display();
skipPatternTextArea = text; }
return text; }
} }
); )
const addDefaultPatterns = async (patterns: string) => {
const oldList = this.plugin.settings.syncInternalFilesIgnorePatterns.split(",").map(x => x.trim()).filter(x => x != "");
const newList = patterns.split(",").map(x => x.trim()).filter(x => x != "");
const allSet = new Set([...oldList, ...newList]);
this.plugin.settings.syncInternalFilesIgnorePatterns = [...allSet].join(", ");
await this.plugin.saveSettings();
this.display();
}
new Setting(containerSyncSettingEl) new Setting(containerSyncSettingEl)
.setName("Restore the skip pattern to default") .setName("Add default patterns")
.setClass("wizardHidden") .setClass("wizardHidden")
.addButton((button) => { .addButton((button) => {
button.setButtonText("Default") button.setButtonText("Default")
.onClick(async () => { .onClick(async () => {
skipPatternTextArea.setValue(defaultSkipPattern); await addDefaultPatterns(defaultSkipPattern);
this.plugin.settings.syncInternalFilesIgnorePatterns = defaultSkipPattern;
await this.plugin.saveSettings();
}) })
}).addButton((button) => { }).addButton((button) => {
button.setButtonText("Cross-platform") button.setButtonText("Cross-platform")
.onClick(async () => { .onClick(async () => {
skipPatternTextArea.setValue(defaultSkipPatternXPlat); await addDefaultPatterns(defaultSkipPatternXPlat);
this.plugin.settings.syncInternalFilesIgnorePatterns = defaultSkipPatternXPlat;
await this.plugin.saveSettings();
}) })
}) })
@@ -1413,54 +1431,41 @@ export class ObsidianLiveSyncSettingTab extends PluginSettingTab {
containerSyncSettingEl.createEl("h4", { containerSyncSettingEl.createEl("h4", {
text: sanitizeHTMLToDom(`Targets`), text: sanitizeHTMLToDom(`Targets`),
}).addClass("wizardHidden"); }).addClass("wizardHidden");
new Setting(containerSyncSettingEl)
const syncFilesSetting = new Setting(containerSyncSettingEl)
.setName("Synchronising files") .setName("Synchronising files")
.setDesc("(RegExp) Empty to sync all files. set filter as a regular expression to limit synchronising files.") .setDesc("(RegExp) Empty to sync all files. set filter as a regular expression to limit synchronising files.")
.setClass("wizardHidden") .setClass("wizardHidden")
.addTextArea((text) => { new MultipleRegExpControl(
text {
.setValue(this.plugin.settings.syncOnlyRegEx) target: syncFilesSetting.controlEl,
.setPlaceholder("\\.md$|\\.txt") props: {
.onChange(async (value) => { patterns: this.plugin.settings.syncOnlyRegEx.split("|[]|"), originals: [...this.plugin.settings.syncOnlyRegEx.split("|[]|")], apply: async (newPatterns) => {
let isValidRegExp = false; this.plugin.settings.syncOnlyRegEx = newPatterns.map(e => e.trim()).filter(e => e != "").join("|[]|");
try { await this.plugin.saveSettings();
new RegExp(value); this.display();
isValidRegExp = true; }
} catch (_) { }
// NO OP.
}
if (isValidRegExp || value.trim() == "") {
this.plugin.settings.syncOnlyRegEx = value;
await this.plugin.saveSettings();
}
})
return text;
} }
); )
new Setting(containerSyncSettingEl)
const nonSyncFilesSetting = new Setting(containerSyncSettingEl)
.setName("Non-Synchronising files") .setName("Non-Synchronising files")
.setDesc("(RegExp) If this is set, any changes to local and remote files that match this will be skipped.") .setDesc("(RegExp) If this is set, any changes to local and remote files that match this will be skipped.")
.setClass("wizardHidden") .setClass("wizardHidden");
.addTextArea((text) => {
text new MultipleRegExpControl(
.setValue(this.plugin.settings.syncIgnoreRegEx) {
.setPlaceholder("\\.pdf$") target: nonSyncFilesSetting.controlEl,
.onChange(async (value) => { props: {
let isValidRegExp = false; patterns: this.plugin.settings.syncIgnoreRegEx.split("|[]|"), originals: [...this.plugin.settings.syncIgnoreRegEx.split("|[]|")], apply: async (newPatterns) => {
try { this.plugin.settings.syncIgnoreRegEx = newPatterns.map(e => e.trim()).filter(e => e != "").join("|[]|");
new RegExp(value); await this.plugin.saveSettings();
isValidRegExp = true; this.display();
} catch (_) { }
// NO OP. }
}
if (isValidRegExp || value.trim() == "") {
this.plugin.settings.syncIgnoreRegEx = value;
await this.plugin.saveSettings();
}
})
return text;
} }
); )
new Setting(containerSyncSettingEl) new Setting(containerSyncSettingEl)
.setName("Maximum file size") .setName("Maximum file size")
.setDesc("(MB) If this is set, changes to local and remote files that are larger than this will be skipped. If the file becomes smaller again, a newer one will be used.") .setDesc("(MB) If this is set, changes to local and remote files that are larger than this will be skipped. If the file becomes smaller again, a newer one will be used.")
@@ -1635,7 +1640,8 @@ export class ObsidianLiveSyncSettingTab extends PluginSettingTab {
const pluginConfig = JSON.parse(JSON.stringify(this.plugin.settings)) as ObsidianLiveSyncSettings; const pluginConfig = JSON.parse(JSON.stringify(this.plugin.settings)) as ObsidianLiveSyncSettings;
pluginConfig.couchDB_DBNAME = REDACTED; pluginConfig.couchDB_DBNAME = REDACTED;
pluginConfig.couchDB_PASSWORD = REDACTED; pluginConfig.couchDB_PASSWORD = REDACTED;
pluginConfig.couchDB_URI = isCloudantURI(pluginConfig.couchDB_URI) ? "cloudant" : "self-hosted"; const scheme = pluginConfig.couchDB_URI.startsWith("http:") ? "(HTTP)" : (pluginConfig.couchDB_URI.startsWith("https:")) ? "(HTTPS)" : ""
pluginConfig.couchDB_URI = isCloudantURI(pluginConfig.couchDB_URI) ? "cloudant" : `self-hosted${scheme}`;
pluginConfig.couchDB_USER = REDACTED; pluginConfig.couchDB_USER = REDACTED;
pluginConfig.passphrase = REDACTED; pluginConfig.passphrase = REDACTED;
pluginConfig.encryptedPassphrase = REDACTED; pluginConfig.encryptedPassphrase = REDACTED;
@@ -1697,6 +1703,52 @@ ${stringifyYaml(pluginConfig)}`;
const hatchWarn = containerHatchEl.createEl("div", { text: `To stop the boot up sequence for fixing problems on databases, you can put redflag.md on top of your vault (Rebooting obsidian is required).` }); const hatchWarn = containerHatchEl.createEl("div", { text: `To stop the boot up sequence for fixing problems on databases, you can put redflag.md on top of your vault (Rebooting obsidian is required).` });
hatchWarn.addClass("op-warn-info"); hatchWarn.addClass("op-warn-info");
const addResult = (path: string, file: TFile | false, fileOnDB: LoadedEntry | false) => {
resultArea.appendChild(resultArea.createEl("div", {}, el => {
el.appendChild(el.createEl("h6", { text: path }));
el.appendChild(el.createEl("div", {}, infoGroupEl => {
infoGroupEl.appendChild(infoGroupEl.createEl("div", { text: `Storage : Modified: ${!file ? `Missing:` : `${new Date(file.stat.mtime).toLocaleString()}, Size:${file.stat.size}`}` }))
infoGroupEl.appendChild(infoGroupEl.createEl("div", { text: `Database: Modified: ${!fileOnDB ? `Missing:` : `${new Date(fileOnDB.mtime).toLocaleString()}, Size:${fileOnDB.size}`}` }))
}));
if (fileOnDB && file) {
el.appendChild(el.createEl("button", { text: "Show history" }, buttonEl => {
buttonEl.onClickEvent(() => {
this.plugin.showHistory(file, fileOnDB._id);
})
}))
}
if (file) {
el.appendChild(el.createEl("button", { text: "Storage -> Database" }, buttonEl => {
buttonEl.onClickEvent(() => {
this.plugin.updateIntoDB(file, undefined, true);
el.remove();
})
}))
}
if (fileOnDB) {
el.appendChild(el.createEl("button", { text: "Database -> Storage" }, buttonEl => {
buttonEl.onClickEvent(() => {
this.plugin.pullFile(this.plugin.getPath(fileOnDB), [], true, undefined, false);
el.remove();
})
}))
}
return el;
}))
}
const checkBetweenStorageAndDatabase = async (file: TFile, fileOnDB: LoadedEntry) => {
const dataContent = readAsBlob(fileOnDB);
const content = createBlob(await this.plugin.vaultAccess.vaultReadAuto(file))
if (await isDocContentSame(content, dataContent)) {
Logger(`Compare: SAME: ${file.path}`)
} else {
Logger(`Compare: CONTENT IS NOT MATCHED! ${file.path}`, LOG_LEVEL_NOTICE);
addResult(file.path, file, fileOnDB)
}
}
new Setting(containerHatchEl) new Setting(containerHatchEl)
.setName("Verify and repair all files") .setName("Verify and repair all files")
.setDesc("Compare the content of files between on local database and storage. If not matched, you will asked which one want to keep.") .setDesc("Compare the content of files between on local database and storage. If not matched, you will asked which one want to keep.")
@@ -1707,47 +1759,36 @@ ${stringifyYaml(pluginConfig)}`;
.setWarning() .setWarning()
.onClick(async () => { .onClick(async () => {
const files = this.app.vault.getFiles(); const files = this.app.vault.getFiles();
const documents = [] as FilePathWithPrefix[];
const adn = this.plugin.localDatabase.findAllNormalDocs()
for await (const i of adn) documents.push(this.plugin.getPath(i));
const allPaths = [...new Set([...documents, ...files.map(e => e.path as FilePathWithPrefix)])];
let i = 0; let i = 0;
for (const file of files) { for (const path of allPaths) {
i++; i++;
Logger(`${i}/${files.length}\n${file.path}`, LOG_LEVEL_NOTICE, "verify"); Logger(`${i}/${files.length}\n${path}`, LOG_LEVEL_NOTICE, "verify");
if (!await this.plugin.isTargetFile(file)) continue; if (shouldBeIgnored(path)) continue;
const fileOnDB = await this.plugin.localDatabase.getDBEntry(file.path as FilePathWithPrefix); const abstractFile = this.plugin.vaultAccess.getAbstractFileByPath(path);
if (!fileOnDB) { const fileOnStorage = abstractFile instanceof TFile ? abstractFile : false;
Logger(`Compare: Not found on local database: ${file.path}`, LOG_LEVEL_NOTICE); if (!await this.plugin.isTargetFile(path)) continue;
if (fileOnStorage && this.plugin.isFileSizeExceeded(fileOnStorage.stat.size)) continue;
const fileOnDB = await this.plugin.localDatabase.getDBEntry(path);
if (fileOnDB && this.plugin.isFileSizeExceeded(fileOnDB.size)) continue;
if (!fileOnDB && fileOnStorage) {
Logger(`Compare: Not found on the local database: ${path}`, LOG_LEVEL_NOTICE);
addResult(path, fileOnStorage, false)
continue; continue;
} }
let content: Blob; if (fileOnDB && !fileOnStorage) {
if (fileOnDB.type == "newnote") { Logger(`Compare: Not found on the storage: ${path}`, LOG_LEVEL_NOTICE);
content = createBinaryBlob(await this.plugin.vaultAccess.vaultReadBinary(file)); addResult(path, false, fileOnDB)
} else { continue;
content = createTextBlob(await this.plugin.vaultAccess.vaultRead(file));
} }
if (isDocContentSame(content, fileOnDB.data)) { if (fileOnStorage && fileOnDB) {
Logger(`Compare: SAME: ${file.path}`) await checkBetweenStorageAndDatabase(fileOnStorage, fileOnDB)
} else {
Logger(`Compare: CONTENT IS NOT MATCHED! ${file.path}`, LOG_LEVEL_NOTICE);
resultArea.appendChild(resultArea.createEl("div", {}, el => {
el.appendChild(el.createEl("h6", { text: file.path }));
el.appendChild(el.createEl("div", {}, infoGroupEl => {
infoGroupEl.appendChild(infoGroupEl.createEl("div", { text: `Storage : Modified: ${new Date(file.stat.mtime).toLocaleString()}, Size:${file.stat.size}` }))
infoGroupEl.appendChild(infoGroupEl.createEl("div", { text: `Database: Modified: ${new Date(fileOnDB.mtime).toLocaleString()}, Size:${content.size}` }))
}));
el.appendChild(el.createEl("button", { text: "Storage -> Database" }, buttonEl => {
buttonEl.onClickEvent(() => {
this.plugin.updateIntoDB(file, undefined, true);
el.remove();
})
}))
el.appendChild(el.createEl("button", { text: "Database -> Storage" }, buttonEl => {
buttonEl.onClickEvent(() => {
this.plugin.pullFile(file.path as FilePathWithPrefix, [], true, undefined, false);
el.remove();
})
}))
return el;
}))
} }
} }
Logger("done", LOG_LEVEL_NOTICE, "verify"); Logger("done", LOG_LEVEL_NOTICE, "verify");
@@ -1776,6 +1817,7 @@ ${stringifyYaml(pluginConfig)}`;
//Prepare converted data //Prepare converted data
newDoc._id = idEncoded; newDoc._id = idEncoded;
newDoc.path = docName as FilePathWithPrefix; newDoc.path = docName as FilePathWithPrefix;
// @ts-ignore
delete newDoc._rev; delete newDoc._rev;
try { try {
const obfuscatedDoc = await this.plugin.localDatabase.getRaw(idEncoded, { revs_info: true }); const obfuscatedDoc = await this.plugin.localDatabase.getRaw(idEncoded, { revs_info: true });
@@ -1801,7 +1843,7 @@ ${stringifyYaml(pluginConfig)}`;
Logger(`Converting ${docName} Failed!`, LOG_LEVEL_NOTICE); Logger(`Converting ${docName} Failed!`, LOG_LEVEL_NOTICE);
Logger(ret, LOG_LEVEL_VERBOSE); Logger(ret, LOG_LEVEL_VERBOSE);
} }
} catch (ex) { } catch (ex: any) {
if (ex?.status == 404) { if (ex?.status == 404) {
// We can perform this safely // We can perform this safely
if ((await this.plugin.localDatabase.putRaw(newDoc)).ok) { if ((await this.plugin.localDatabase.putRaw(newDoc)).ok) {
@@ -1996,7 +2038,7 @@ ${stringifyYaml(pluginConfig)}`;
const vaultName = new Setting(containerPluginSettings) const vaultName = new Setting(containerPluginSettings)
.setName("Device name") .setName("Device name")
.setDesc("Unique name between all synchronized devices") .setDesc("Unique name between all synchronized devices. To edit this setting, please disable customization sync once.")
.addText((text) => { .addText((text) => {
text.setPlaceholder("desktop") text.setPlaceholder("desktop")
.setValue(this.plugin.deviceAndVaultName) .setValue(this.plugin.deviceAndVaultName)
@@ -2119,11 +2161,33 @@ ${stringifyYaml(pluginConfig)}`;
.setButtonText("Fetch") .setButtonText("Fetch")
.setWarning() .setWarning()
.setDisabled(false) .setDisabled(false)
.onClick(async () => {
await this.plugin.vaultAccess.vaultCreate(FLAGMD_REDFLAG3_HR, "");
this.plugin.performAppReload();
})
).addButton((button) =>
button
.setButtonText("Fetch w/o restarting")
.setWarning()
.setDisabled(false)
.onClick(async () => { .onClick(async () => {
await rebuildDB("localOnly"); await rebuildDB("localOnly");
}) })
) )
new Setting(containerMaintenanceEl)
.setName("Fetch rebuilt DB (Save local documents before)")
.setDesc("Restore or reconstruct local database from remote database but use local chunks.")
.addButton((button) =>
button
.setButtonText("Save and Fetch")
.setWarning()
.setDisabled(false)
.onClick(async () => {
await rebuildDB("localOnlyWithChunks");
})
)
new Setting(containerMaintenanceEl) new Setting(containerMaintenanceEl)
.setName("Discard local database to reset or uninstall Self-hosted LiveSync") .setName("Discard local database to reset or uninstall Self-hosted LiveSync")
.addButton((button) => .addButton((button) =>
@@ -2165,10 +2229,21 @@ ${stringifyYaml(pluginConfig)}`;
.setButtonText("Rebuild") .setButtonText("Rebuild")
.setWarning() .setWarning()
.setDisabled(false) .setDisabled(false)
.onClick(async () => {
await this.plugin.vaultAccess.vaultCreate(FLAGMD_REDFLAG2_HR, "");
this.plugin.performAppReload();
})
)
.addButton((button) =>
button
.setButtonText("Rebuild w/o restarting")
.setWarning()
.setDisabled(false)
.onClick(async () => { .onClick(async () => {
await rebuildDB("rebuildBothByThisDevice"); await rebuildDB("rebuildBothByThisDevice");
}) })
) )
applyDisplayEnabled(); applyDisplayEnabled();
addScreenElement("70", containerMaintenanceEl); addScreenElement("70", containerMaintenanceEl);

52
src/SerializedFileAccess.ts
View File

@@ -1,5 +1,7 @@
import { type App, TFile, type DataWriteOptions, TFolder, TAbstractFile } from "./deps"; import { type App, TFile, type DataWriteOptions, TFolder, TAbstractFile } from "./deps";
import { serialized } from "./lib/src/lock"; import { serialized } from "./lib/src/lock";
import { Logger } from "./lib/src/logger";
import { isPlainText } from "./lib/src/path";
import type { FilePath } from "./lib/src/types"; import type { FilePath } from "./lib/src/types";
import { createBinaryBlob, isDocContentSame } from "./lib/src/utils"; import { createBinaryBlob, isDocContentSame } from "./lib/src/utils";
import type { InternalFileInfo } from "./types"; import type { InternalFileInfo } from "./types";
@@ -55,6 +57,12 @@ export class SerializedFileAccess {
return await processReadFile(file, () => this.app.vault.adapter.readBinary(path)); return await processReadFile(file, () => this.app.vault.adapter.readBinary(path));
} }
async adapterReadAuto(file: TFile | string) {
const path = file instanceof TFile ? file.path : file;
if (isPlainText(path)) return await processReadFile(file, () => this.app.vault.adapter.read(path));
return await processReadFile(file, () => this.app.vault.adapter.readBinary(path));
}
async adapterWrite(file: TFile | string, data: string | ArrayBuffer | Uint8Array, options?: DataWriteOptions) { async adapterWrite(file: TFile | string, data: string | ArrayBuffer | Uint8Array, options?: DataWriteOptions) {
const path = file instanceof TFile ? file.path : file; const path = file instanceof TFile ? file.path : file;
if (typeof (data) === "string") { if (typeof (data) === "string") {
@@ -76,12 +84,19 @@ export class SerializedFileAccess {
return await processReadFile(file, () => this.app.vault.readBinary(file)); return await processReadFile(file, () => this.app.vault.readBinary(file));
} }
async vaultReadAuto(file: TFile) {
const path = file.path;
if (isPlainText(path)) return await processReadFile(file, () => this.app.vault.read(file));
return await processReadFile(file, () => this.app.vault.readBinary(file));
}
async vaultModify(file: TFile, data: string | ArrayBuffer | Uint8Array, options?: DataWriteOptions) { async vaultModify(file: TFile, data: string | ArrayBuffer | Uint8Array, options?: DataWriteOptions) {
if (typeof (data) === "string") { if (typeof (data) === "string") {
return await processWriteFile(file, async () => { return await processWriteFile(file, async () => {
const oldData = await this.app.vault.read(file); const oldData = await this.app.vault.read(file);
if (data === oldData) { if (data === oldData) {
markChangesAreSame(file, file.stat.mtime, options.mtime); if (options && options.mtime) markChangesAreSame(file, file.stat.mtime, options.mtime);
return false return false
} }
await this.app.vault.modify(file, data, options) await this.app.vault.modify(file, data, options)
@@ -92,7 +107,7 @@ export class SerializedFileAccess {
return await processWriteFile(file, async () => { return await processWriteFile(file, async () => {
const oldData = await this.app.vault.readBinary(file); const oldData = await this.app.vault.readBinary(file);
if (await isDocContentSame(createBinaryBlob(oldData), createBinaryBlob(data))) { if (await isDocContentSame(createBinaryBlob(oldData), createBinaryBlob(data))) {
markChangesAreSame(file, file.stat.mtime, options.mtime); if (options && options.mtime) markChangesAreSame(file, file.stat.mtime, options.mtime);
return false; return false;
} }
await this.app.vault.modifyBinary(file, toArrayBuffer(data), options) await this.app.vault.modifyBinary(file, toArrayBuffer(data), options)
@@ -107,6 +122,15 @@ export class SerializedFileAccess {
return await processWriteFile(path, () => this.app.vault.createBinary(path, toArrayBuffer(data), options)); return await processWriteFile(path, () => this.app.vault.createBinary(path, toArrayBuffer(data), options));
} }
} }
trigger(name: string, ...data: any[]) {
return this.app.vault.trigger(name, ...data);
}
async adapterAppend(normalizedPath: string, data: string, options?: DataWriteOptions) {
return await this.app.vault.adapter.append(normalizedPath, data, options)
}
async delete(file: TFile | TFolder, force = false) { async delete(file: TFile | TFolder, force = false) {
return await processWriteFile(file, () => this.app.vault.delete(file, force)); return await processWriteFile(file, () => this.app.vault.delete(file, force));
} }
@@ -127,6 +151,30 @@ export class SerializedFileAccess {
// } // }
} }
getFiles() {
return this.app.vault.getFiles();
}
async ensureDirectory(fullPath: string) {
const pathElements = fullPath.split("/");
pathElements.pop();
let c = "";
for (const v of pathElements) {
c += v;
try {
await this.app.vault.adapter.mkdir(c);
} catch (ex: any) {
if (ex?.message == "Folder already exists.") {
// Skip if already exists.
} else {
Logger("Folder Create Error");
Logger(ex);
}
}
c += "/";
}
}
touchedFiles: string[] = []; touchedFiles: string[] = [];

27
src/StorageEventManager.ts
View File

@@ -1,8 +1,8 @@
import type { SerializedFileAccess } from "./SerializedFileAccess"; import type { SerializedFileAccess } from "./SerializedFileAccess";
import { Plugin, TAbstractFile, TFile, TFolder } from "./deps"; import { Plugin, TAbstractFile, TFile, TFolder } from "./deps";
import { Logger } from "./lib/src/logger"; import { Logger } from "./lib/src/logger";
import { isPlainText, shouldBeIgnored } from "./lib/src/path"; import { shouldBeIgnored } from "./lib/src/path";
import type { KeyedQueueProcessor } from "./lib/src/processor"; import type { QueueProcessor } from "./lib/src/processor";
import { LOG_LEVEL_NOTICE, type FilePath, type ObsidianLiveSyncSettings } from "./lib/src/types"; import { LOG_LEVEL_NOTICE, type FilePath, type ObsidianLiveSyncSettings } from "./lib/src/types";
import { delay } from "./lib/src/utils"; import { delay } from "./lib/src/utils";
import { type FileEventItem, type FileEventType, type FileInfo, type InternalFileInfo } from "./types"; import { type FileEventItem, type FileEventType, type FileInfo, type InternalFileInfo } from "./types";
@@ -19,7 +19,7 @@ type LiveSyncForStorageEventManager = Plugin &
vaultAccess: SerializedFileAccess vaultAccess: SerializedFileAccess
} & { } & {
isTargetFile: (file: string | TAbstractFile) => Promise<boolean>, isTargetFile: (file: string | TAbstractFile) => Promise<boolean>,
fileEventQueue: KeyedQueueProcessor<FileEventItem, any>, fileEventQueue: QueueProcessor<FileEventItem, any>,
isFileSizeExceeded: (size: number) => boolean; isFileSizeExceeded: (size: number) => boolean;
}; };
@@ -109,7 +109,8 @@ export class StorageEventManagerObsidian extends StorageEventManager {
if (file instanceof TFolder) continue; if (file instanceof TFolder) continue;
if (!await this.plugin.isTargetFile(file.path)) continue; if (!await this.plugin.isTargetFile(file.path)) continue;
let cache: null | string | ArrayBuffer; // Stop cache using to prevent the corruption;
// let cache: null | string | ArrayBuffer;
// new file or something changed, cache the changes. // new file or something changed, cache the changes.
if (file instanceof TFile && (type == "CREATE" || type == "CHANGED")) { if (file instanceof TFile && (type == "CREATE" || type == "CHANGED")) {
// Wait for a bit while to let the writer has marked `touched` at the file. // Wait for a bit while to let the writer has marked `touched` at the file.
@@ -117,12 +118,13 @@ export class StorageEventManagerObsidian extends StorageEventManager {
if (this.plugin.vaultAccess.recentlyTouched(file)) { if (this.plugin.vaultAccess.recentlyTouched(file)) {
continue; continue;
} }
if (!isPlainText(file.name)) { // cache = await this.plugin.vaultAccess.vaultReadAuto(file);
cache = await this.plugin.vaultAccess.vaultReadBinary(file); // if (!isPlainText(file.name)) {
} else { // cache = await this.plugin.vaultAccess.vaultReadBinary(file);
cache = await this.plugin.vaultAccess.vaultCacheRead(file); // } else {
if (!cache) cache = await this.plugin.vaultAccess.vaultRead(file); // cache = await this.plugin.vaultAccess.vaultCacheRead(file);
} // if (!cache) cache = await this.plugin.vaultAccess.vaultRead(file);
// }
} }
const fileInfo = file instanceof TFile ? { const fileInfo = file instanceof TFile ? {
ctime: file.stat.ctime, ctime: file.stat.ctime,
@@ -131,13 +133,12 @@ export class StorageEventManagerObsidian extends StorageEventManager {
path: file.path, path: file.path,
size: file.stat.size size: file.stat.size
} as FileInfo : file as InternalFileInfo; } as FileInfo : file as InternalFileInfo;
this.plugin.fileEventQueue.enqueue({
this.plugin.fileEventQueue.enqueueWithKey(`file-${fileInfo.path}`, {
type, type,
args: { args: {
file: fileInfo, file: fileInfo,
oldPath, oldPath,
cache, // cache,
ctx ctx
}, },
key: atomicKey key: atomicKey

33
src/dialogs.ts
View File

@@ -7,10 +7,9 @@ import PluginPane from "./PluginPane.svelte";
export class PluginDialogModal extends Modal { export class PluginDialogModal extends Modal {
plugin: ObsidianLiveSyncPlugin; plugin: ObsidianLiveSyncPlugin;
logEl: HTMLDivElement; component: PluginPane | undefined;
component: PluginPane = null;
isOpened() { isOpened() {
return this.component != null; return this.component != undefined;
} }
constructor(app: App, plugin: ObsidianLiveSyncPlugin) { constructor(app: App, plugin: ObsidianLiveSyncPlugin) {
@@ -21,7 +20,7 @@ export class PluginDialogModal extends Modal {
onOpen() { onOpen() {
const { contentEl } = this; const { contentEl } = this;
this.titleEl.setText("Customization Sync (Beta2)") this.titleEl.setText("Customization Sync (Beta2)")
if (this.component == null) { if (!this.component) {
this.component = new PluginPane({ this.component = new PluginPane({
target: contentEl, target: contentEl,
props: { plugin: this.plugin }, props: { plugin: this.plugin },
@@ -30,9 +29,9 @@ export class PluginDialogModal extends Modal {
} }
onClose() { onClose() {
if (this.component != null) { if (this.component) {
this.component.$destroy(); this.component.$destroy();
this.component = null; this.component = undefined;
} }
} }
} }
@@ -94,13 +93,13 @@ export class InputStringDialog extends Modal {
} }
export class PopoverSelectString extends FuzzySuggestModal<string> { export class PopoverSelectString extends FuzzySuggestModal<string> {
app: App; app: App;
callback: (e: string) => void = () => { }; callback: ((e: string) => void) | undefined = () => { };
getItemsFun: () => string[] = () => { getItemsFun: () => string[] = () => {
return ["yes", "no"]; return ["yes", "no"];
} }
constructor(app: App, note: string, placeholder: string | null, getItemsFun: () => string[], callback: (e: string) => void) { constructor(app: App, note: string, placeholder: string | undefined, getItemsFun: (() => string[]) | undefined, callback: (e: string) => void) {
super(app); super(app);
this.app = app; this.app = app;
this.setPlaceholder((placeholder ?? "y/n) ") + note); this.setPlaceholder((placeholder ?? "y/n) ") + note);
@@ -118,13 +117,14 @@ export class PopoverSelectString extends FuzzySuggestModal<string> {
onChooseItem(item: string, evt: MouseEvent | KeyboardEvent): void { onChooseItem(item: string, evt: MouseEvent | KeyboardEvent): void {
// debugger; // debugger;
this.callback(item); this.callback?.(item);
this.callback = null; this.callback = undefined;
} }
onClose(): void { onClose(): void {
setTimeout(() => { setTimeout(() => {
if (this.callback != null) { if (this.callback) {
this.callback(""); this.callback("");
this.callback = undefined;
} }
}, 100); }, 100);
} }
@@ -136,16 +136,16 @@ export class MessageBox extends Modal {
title: string; title: string;
contentMd: string; contentMd: string;
buttons: string[]; buttons: string[];
result: string; result: string | false = false;
isManuallyClosed = false; isManuallyClosed = false;
defaultAction: string | undefined; defaultAction: string | undefined;
timeout: number | undefined; timeout: number | undefined;
timer: ReturnType<typeof setInterval> = undefined; timer: ReturnType<typeof setInterval> | undefined = undefined;
defaultButtonComponent: ButtonComponent | undefined; defaultButtonComponent: ButtonComponent | undefined;
onSubmit: (result: string | false) => void; onSubmit: (result: string | false) => void;
constructor(plugin: Plugin, title: string, contentMd: string, buttons: string[], defaultAction: (typeof buttons)[number], timeout: number, onSubmit: (result: (typeof buttons)[number] | false) => void) { constructor(plugin: Plugin, title: string, contentMd: string, buttons: string[], defaultAction: (typeof buttons)[number], timeout: number | undefined, onSubmit: (result: (typeof buttons)[number] | false) => void) {
super(plugin.app); super(plugin.app);
this.plugin = plugin; this.plugin = plugin;
this.title = title; this.title = title;
@@ -156,6 +156,7 @@ export class MessageBox extends Modal {
this.timeout = timeout; this.timeout = timeout;
if (this.timeout) { if (this.timeout) {
this.timer = setInterval(() => { this.timer = setInterval(() => {
if (this.timeout === undefined) return;
this.timeout--; this.timeout--;
if (this.timeout < 0) { if (this.timeout < 0) {
if (this.timer) { if (this.timer) {
@@ -166,7 +167,7 @@ export class MessageBox extends Modal {
this.isManuallyClosed = true; this.isManuallyClosed = true;
this.close(); this.close();
} else { } else {
this.defaultButtonComponent.setButtonText(`( ${this.timeout} ) ${defaultAction}`); this.defaultButtonComponent?.setButtonText(`( ${this.timeout} ) ${defaultAction}`);
} }
}, 1000); }, 1000);
} }
@@ -223,7 +224,7 @@ export class MessageBox extends Modal {
} }
export function confirmWithMessage(plugin: Plugin, title: string, contentMd: string, buttons: string[], defaultAction?: (typeof buttons)[number], timeout?: number): Promise<(typeof buttons)[number] | false> { export function confirmWithMessage(plugin: Plugin, title: string, contentMd: string, buttons: string[], defaultAction: (typeof buttons)[number], timeout?: number): Promise<(typeof buttons)[number] | false> {
return new Promise((res) => { return new Promise((res) => {
const dialog = new MessageBox(plugin, title, contentMd, buttons, defaultAction, timeout, (result) => res(result)); const dialog = new MessageBox(plugin, title, contentMd, buttons, defaultAction, timeout, (result) => res(result));
dialog.open(); dialog.open();

2
src/lib

Submodule src/lib updated: 1a3488339e...0d217242a8

624
src/main.ts
View File

File diff suppressed because it is too large Load Diff

30
src/utils.ts
View File

@@ -242,14 +242,11 @@ export function mergeObject(
ret[key] = v; ret[key] = v;
} }
} }
const retSorted = Object.fromEntries(Object.entries(ret).sort((a, b) => a[0] < b[0] ? -1 : a[0] > b[0] ? 1 : 0));
if (Array.isArray(objA) && Array.isArray(objB)) { if (Array.isArray(objA) && Array.isArray(objB)) {
return Object.values(Object.entries(ret) return Object.values(retSorted);
.sort()
.reduce((p, [key, value]) => ({ ...p, [key]: value }), {}));
} }
return Object.entries(ret) return retSorted;
.sort()
.reduce((p, [key, value]) => ({ ...p, [key]: value }), {});
} }
export function flattenObject(obj: Record<string | number | symbol, any>, path: string[] = []): [string, any][] { export function flattenObject(obj: Record<string | number | symbol, any>, path: string[] = []): [string, any][] {
@@ -313,7 +310,7 @@ export function isCustomisationSyncMetadata(str: string): boolean {
export const askYesNo = (app: App, message: string): Promise<"yes" | "no"> => { export const askYesNo = (app: App, message: string): Promise<"yes" | "no"> => {
return new Promise((res) => { return new Promise((res) => {
const popover = new PopoverSelectString(app, message, null, null, (result) => res(result as "yes" | "no")); const popover = new PopoverSelectString(app, message, undefined, undefined, (result) => res(result as "yes" | "no"));
popover.open(); popover.open();
}); });
}; };
@@ -327,7 +324,7 @@ export const askSelectString = (app: App, message: string, items: string[]): Pro
}; };
export const askString = (app: App, title: string, key: string, placeholder: string, isPassword?: boolean): Promise<string | false> => { export const askString = (app: App, title: string, key: string, placeholder: string, isPassword: boolean = false): Promise<string | false> => {
return new Promise((res) => { return new Promise((res) => {
const dialog = new InputStringDialog(app, title, key, placeholder, isPassword, (result) => res(result)); const dialog = new InputStringDialog(app, title, key, placeholder, isPassword, (result) => res(result));
dialog.open(); dialog.open();
@@ -400,15 +397,18 @@ export const _requestToCouchDB = async (baseUri: string, username: string, passw
}; };
return await requestUrl(requestParam); return await requestUrl(requestParam);
} }
export const requestToCouchDB = async (baseUri: string, username: string, password: string, origin: string, key?: string, body?: string, method?: string) => { export const requestToCouchDB = async (baseUri: string, username: string, password: string, origin: string = "", key?: string, body?: string, method?: string) => {
const uri = `_node/_local/_config${key ? "/" + key : ""}`; const uri = `_node/_local/_config${key ? "/" + key : ""}`;
return await _requestToCouchDB(baseUri, username, password, origin, uri, body, method); return await _requestToCouchDB(baseUri, username, password, origin, uri, body, method);
}; };
export async function performRebuildDB(plugin: ObsidianLiveSyncPlugin, method: "localOnly" | "remoteOnly" | "rebuildBothByThisDevice") { export async function performRebuildDB(plugin: ObsidianLiveSyncPlugin, method: "localOnly" | "remoteOnly" | "rebuildBothByThisDevice" | "localOnlyWithChunks") {
if (method == "localOnly") { if (method == "localOnly") {
await plugin.addOnSetup.fetchLocal(); await plugin.addOnSetup.fetchLocal();
} }
if (method == "localOnlyWithChunks") {
await plugin.addOnSetup.fetchLocal(true);
}
if (method == "remoteOnly") { if (method == "remoteOnly") {
await plugin.addOnSetup.rebuildRemote(); await plugin.addOnSetup.rebuildRemote();
} }
@@ -437,7 +437,7 @@ export function compareMTime(baseMTime: number, targetMTime: number): typeof BAS
export function markChangesAreSame(file: TFile | AnyEntry | string, mtime1: number, mtime2: number) { export function markChangesAreSame(file: TFile | AnyEntry | string, mtime1: number, mtime2: number) {
if (mtime1 === mtime2) return true; if (mtime1 === mtime2) return true;
const key = typeof file == "string" ? file : file instanceof TFile ? file.path : file.path ?? file._id; const key = typeof file == "string" ? file : file instanceof TFile ? file.path : file.path ?? file._id;
const pairs = sameChangePairs.get(key, []); const pairs = sameChangePairs.get(key, []) || [];
if (pairs.some(e => e == mtime1 || e == mtime2)) { if (pairs.some(e => e == mtime1 || e == mtime2)) {
sameChangePairs.set(key, [...new Set([...pairs, mtime1, mtime2])]); sameChangePairs.set(key, [...new Set([...pairs, mtime1, mtime2])]);
} else { } else {
@@ -446,12 +446,16 @@ export function markChangesAreSame(file: TFile | AnyEntry | string, mtime1: numb
} }
export function isMarkedAsSameChanges(file: TFile | AnyEntry | string, mtimes: number[]) { export function isMarkedAsSameChanges(file: TFile | AnyEntry | string, mtimes: number[]) {
const key = typeof file == "string" ? file : file instanceof TFile ? file.path : file.path ?? file._id; const key = typeof file == "string" ? file : file instanceof TFile ? file.path : file.path ?? file._id;
const pairs = sameChangePairs.get(key, []); const pairs = sameChangePairs.get(key, []) || [];
if (mtimes.every(e => pairs.indexOf(e) !== -1)) { if (mtimes.every(e => pairs.indexOf(e) !== -1)) {
return EVEN; return EVEN;
} }
} }
export function compareFileFreshness(baseFile: TFile | AnyEntry, checkTarget: TFile | AnyEntry): typeof BASE_IS_NEW | typeof TARGET_IS_NEW | typeof EVEN { export function compareFileFreshness(baseFile: TFile | AnyEntry | undefined, checkTarget: TFile | AnyEntry | undefined): typeof BASE_IS_NEW | typeof TARGET_IS_NEW | typeof EVEN {
if (baseFile === undefined && checkTarget == undefined) return EVEN;
if (baseFile == undefined) return TARGET_IS_NEW;
if (checkTarget == undefined) return BASE_IS_NEW;
const modifiedBase = baseFile instanceof TFile ? baseFile?.stat?.mtime ?? 0 : baseFile?.mtime ?? 0; const modifiedBase = baseFile instanceof TFile ? baseFile?.stat?.mtime ?? 0 : baseFile?.mtime ?? 0;
const modifiedTarget = checkTarget instanceof TFile ? checkTarget?.stat?.mtime ?? 0 : checkTarget?.mtime ?? 0; const modifiedTarget = checkTarget instanceof TFile ? checkTarget?.stat?.mtime ?? 0 : checkTarget?.mtime ?? 0;

15
styles.css
View File

@@ -77,14 +77,6 @@
border-top: 1px solid var(--background-modifier-border); border-top: 1px solid var(--background-modifier-border);
} }
/* .sls-table-head{
width:50%;
}
.sls-table-tail{
width:50%;
} */
.sls-header-button { .sls-header-button {
margin-left: 2em; margin-left: 2em;
} }
@@ -94,7 +86,7 @@
} }
:root { :root {
--slsmessage: ""; --sls-log-text: "";
} }
.sls-troubleshoot-preview { .sls-troubleshoot-preview {
@@ -110,7 +102,10 @@
.markdown-source-view.cm-s-obsidian::before, .markdown-source-view.cm-s-obsidian::before,
.canvas-wrapper::before, .canvas-wrapper::before,
.empty-state::before { .empty-state::before {
content: attr(data-log); content: var(--sls-log-text, "");
font-variant-numeric: tabular-nums;
font-variant-emoji: emoji;
tab-size: 4;
text-align: right; text-align: right;
white-space: pre-wrap; white-space: pre-wrap;
position: absolute; position: absolute;

137
updates.md
View File

@@ -10,120 +10,37 @@ Note: we got a very performance improvement.
Note at 0.22.2: **Now, to rescue mobile devices, Maximum file size is set to 50 by default**. Please configure the limit as you need. If you do not want to limit the sizes, set zero manually, please. Note at 0.22.2: **Now, to rescue mobile devices, Maximum file size is set to 50 by default**. Please configure the limit as you need. If you do not want to limit the sizes, set zero manually, please.
#### Version history #### Version history
- 0.22.8 - 0.22.17:
- Fixed: - Fixed:
- Now fetch and unlock the locked remote database works well again. - Error handling on booting now works fine.
- No longer crash on symbolic links inside hidden folders. - Replication is now started automatically in LiveSync mode.
- Batch database update is now disabled in LiveSync mode.
- No longer automatically reconnection while off-focused.
- Status saves are thinned out.
- Now Self-hosted LiveSync waits for all files between the local database and storage to be surely checked.
- Improved: - Improved:
- Chunks are now created more efficiently. - The job scheduler is now more robust and stable.
- Splitting old notes into a larger chunk. - The status indicator no longer flickers and keeps zero for a while.
- Better performance in saving notes. - No longer meaningless frequent updates of status indicators.
- Network activities are indicated as an icon. - Now we can configure regular expression filters in handy UI. Thank you so much, @eth-p!
- Less memory used for binary processing. - `Fetch` or `Rebuild everything` is now more safely performed.
- Tidied: - Minor things
- Cleaned unused functions up. - Some utility function has been added.
- Sorting out the codes that have become nonsense. - Customisation sync now less wrong messages.
- Changed: - Digging the weeds for eradication of type errors.
- Now no longer `fetch chunks on demand` needs `Pacing replication` - 0.22.16:
- The setting `Do not pace synchronization` has been deleted.
- 0.22.7
- Fixed: - Fixed:
- No longer deleted hidden files were ignored. - Fixed the issue that binary files were sometimes corrupted.
- The document history dialogue is now able to process the deleted revisions. - Fixed customisation sync data could be corrupted.
- Deletion of a hidden file is now surely performed even if the file is already conflicted.
- 0.22.6
- Fixed:
- Fixed a problem with synchronisation taking a long time to start in some cases.
- The first synchronisation after update might take a bit longer.
- Now we can disable E2EE encryption.
- Improved: - Improved:
- `Setup Wizard` is now more clear. - Now the remote database costs lower memory.
- `Minimal Setup` is now more simple. - This release requires a brief wait on the first synchronisation, to track the latest changeset again.
- Self-hosted LiveSync now be able to use even if there are vaults with the same name. - Description added for the `Device name`.
- Database suffix will automatically added.
- Now Self-hosted LiveSync waits until set-up is complete.
- Show reload prompts when possibly recommended while settings.
- New feature:
- A guidance dialogue prompting for settings will be shown after the installation.
- Changed
- `Open setup URI` is now `Use the copied setup URI`
- `Copy setup URI` is now `Copy current settings as a new setup URI`
- `Setup Wizard` is now `Minimal Setup`
- `Check database configuration` is now `Check and Fix database configuration`
- 0.22.5
- Fixed:
- Some description of settings have been refined
- New feature:
- TroubleShooting is now shown in the setting dialogue.
- 0.22.4
- Fixed:
- Now the result of conflict resolution could be surely written into the storage.
- Deleted files can be handled correctly again in the history dialogue and conflict dialogue.
- Some wrong log messages were fixed.
- Change handling now has become more stable.
- Some event handling became to be safer.
- Improved:
- Dumping document information shows conflicts and revisions.
- The timestamp-only differences can be surely cached.
- Timestamp difference detection can be rounded by two seconds.
- Refactored: - Refactored:
- A bit of organisation to write the test. - Many type-errors have been resolved.
- 0.22.3 - Obsolete file has been deleted.
- Fixed: - 0.22.15:
- No longer detects storage changes which have been caused by Self-hosted LiveSync itself.
- Setting sync file will be detected only if it has been configured now.
- And its log will be shown only while the verbose log is enabled.
- Customisation file enumeration has got less blingy.
- Deletion of files is now reliably synchronised.
- Fixed and improved:
- In-editor-status is now shown in the following areas:
- Note editing pane (Source mode and live-preview mode).
- New tab pane.
- Canvas pane.
- 0.22.2
- Fixed:
- Now the results of resolving conflicts are surely synchronised.
- Modified:
- Some setting items got new clear names. (`Sync Settings` -> `Targets`).
- New feature:
- We can limit the synchronising files by their size. (`Sync Settings` -> `Targets` -> `Maximum file size`).
- It depends on the size of the newer one.
- At Obsidian 1.5.3 on mobile, we should set this to around 50MB to avoid restarting Obsidian.
- Now the settings could be stored in a specific markdown file to synchronise or switch it (`General Setting` -> `Share settings via markdown`).
- [Screwdriver](https://github.com/vrtmrz/obsidian-screwdriver) is quite good, but mostly we only need this.
- Customisation of the obsoleted device is now able to be deleted at once.
- We have to put the maintenance mode in at the Customisation sync dialogue.
- 0.22.1
- New feature:
- We can perform automatic conflict resolution for inactive files, and postpone only manual ones by `Postpone manual resolution of inactive files`.
- Now we can see the image in the document history dialogue.
- We can see the difference of the image, in the document history dialogue.
- And also we can highlight differences.
- Improved: - Improved:
- Hidden file sync has been stabilised. - Faster start-up by removing too many logs which indicates normality
- Now automatically reloads the conflict-resolution dialogue when new conflicted revisions have arrived. - By streamlined scanning of customised synchronisation extra phases have been deleted.
- Fixed:
- No longer periodic process runs after unloading the plug-in.
- Now the modification of binary files is surely stored in the storage.
- 0.22.0
- Refined:
- Task scheduling logics has been rewritten.
- Screen updates are also now efficient.
- Possibly many bugs and fragile behaviour has been fixed.
- Status updates and logging have been thinned out to display.
- Fixed:
- Remote-chunk-fetching now works with keeping request intervals
- New feature:
- We can show only the icons in the editor.
- Progress indicators have been more meaningful:
- 📥 Unprocessed transferred items
- 📄 Working database operation
- 💾 Working write storage processes
- ⏳ Working read storage processes
- 🛫 Pending read storage processes
- ⚙️ Working or pending storage processes of hidden files
- 🧩 Waiting chunks
- 🔌 Working Customisation items (Configuration, snippets and plug-ins)
... To continue on to `updates_old.md`. ... To continue on to `updates_old.md`.

177
updates_old.md
View File

@@ -1,3 +1,180 @@
### 0.22.0
A few years passed since Self-hosted LiveSync was born, and our codebase had been very complicated. This could be patient now, but it should be a tremendous hurt.
Therefore at v0.22.0, for future maintainability, I refined task scheduling logic totally.
Of course, I think this would be our suffering in some cases. However, I would love to ask you for your cooperation and contribution.
Sorry for being absent so much long. And thank you for your patience!
Note: we got a very performance improvement.
Note at 0.22.2: **Now, to rescue mobile devices, Maximum file size is set to 50 by default**. Please configure the limit as you need. If you do not want to limit the sizes, set zero manually, please.
#### Version history
- 0.22.14:
- New feature:
- We can disable the status bar in the setting dialogue.
- Improved:
- Now some files are handled as correct data type.
- Customisation sync now uses the digest of each file for better performance.
- The status in the Editor now works performant.
- Refactored:
- Common functions have been ready and the codebase has been organised.
- Stricter type checking following TypeScript updates.
- Remove old iOS workaround for simplicity and performance.
- 0.22.13:
- Improved:
- Now using HTTP for the remote database URI warns of an error (on mobile) or notice (on desktop).
- Refactored:
- Dependencies have been polished.
- 0.22.12:
- Changed:
- The default settings has been changed.
- Improved:
- Default and preferred settings are applied on completion of the wizard.
- Fixed:
- Now Initialisation `Fetch` will be performed smoothly and there will be fewer conflicts.
- No longer stuck while Handling transferred or initialised documents.
- 0.22.11:
- Fixed:
- `Verify and repair all files` is no longer broken.
- New feature:
- Now `Verify and repair all files` is able to...
- Restore if the file only in the local database.
- Show the history.
- Improved:
- Performance improved.
- 0.22.10
- Fixed:
- No longer unchanged hidden files and customisations are saved and transferred now.
- File integrity of vault history indicates the integrity correctly.
- Improved:
- In the report, the schema of the remote database URI is now printed.
- 0.22.9
- Fixed:
- Fixed a bug on `fetch chunks on demand` that could not fetch the chunks on demand.
- Improved:
- `fetch chunks on demand` works more smoothly.
- Initialisation `Fetch` is now more efficient.
- Tidied:
- Removed some meaningless codes.
- 0.22.8
- Fixed:
- Now fetch and unlock the locked remote database works well again.
- No longer crash on symbolic links inside hidden folders.
- Improved:
- Chunks are now created more efficiently.
- Splitting old notes into a larger chunk.
- Better performance in saving notes.
- Network activities are indicated as an icon.
- Less memory used for binary processing.
- Tidied:
- Cleaned unused functions up.
- Sorting out the codes that have become nonsense.
- Changed:
- Now no longer `fetch chunks on demand` needs `Pacing replication`
- The setting `Do not pace synchronization` has been deleted.
- 0.22.7
- Fixed:
- No longer deleted hidden files were ignored.
- The document history dialogue is now able to process the deleted revisions.
- Deletion of a hidden file is now surely performed even if the file is already conflicted.
- 0.22.6
- Fixed:
- Fixed a problem with synchronisation taking a long time to start in some cases.
- The first synchronisation after update might take a bit longer.
- Now we can disable E2EE encryption.
- Improved:
- `Setup Wizard` is now more clear.
- `Minimal Setup` is now more simple.
- Self-hosted LiveSync now be able to use even if there are vaults with the same name.
- Database suffix will automatically added.
- Now Self-hosted LiveSync waits until set-up is complete.
- Show reload prompts when possibly recommended while settings.
- New feature:
- A guidance dialogue prompting for settings will be shown after the installation.
- Changed
- `Open setup URI` is now `Use the copied setup URI`
- `Copy setup URI` is now `Copy current settings as a new setup URI`
- `Setup Wizard` is now `Minimal Setup`
- `Check database configuration` is now `Check and Fix database configuration`
- 0.22.5
- Fixed:
- Some description of settings have been refined
- New feature:
- TroubleShooting is now shown in the setting dialogue.
- 0.22.4
- Fixed:
- Now the result of conflict resolution could be surely written into the storage.
- Deleted files can be handled correctly again in the history dialogue and conflict dialogue.
- Some wrong log messages were fixed.
- Change handling now has become more stable.
- Some event handling became to be safer.
- Improved:
- Dumping document information shows conflicts and revisions.
- The timestamp-only differences can be surely cached.
- Timestamp difference detection can be rounded by two seconds.
- Refactored:
- A bit of organisation to write the test.
- 0.22.3
- Fixed:
- No longer detects storage changes which have been caused by Self-hosted LiveSync itself.
- Setting sync file will be detected only if it has been configured now.
- And its log will be shown only while the verbose log is enabled.
- Customisation file enumeration has got less blingy.
- Deletion of files is now reliably synchronised.
- Fixed and improved:
- In-editor-status is now shown in the following areas:
- Note editing pane (Source mode and live-preview mode).
- New tab pane.
- Canvas pane.
- 0.22.2
- Fixed:
- Now the results of resolving conflicts are surely synchronised.
- Modified:
- Some setting items got new clear names. (`Sync Settings` -> `Targets`).
- New feature:
- We can limit the synchronising files by their size. (`Sync Settings` -> `Targets` -> `Maximum file size`).
- It depends on the size of the newer one.
- At Obsidian 1.5.3 on mobile, we should set this to around 50MB to avoid restarting Obsidian.
- Now the settings could be stored in a specific markdown file to synchronise or switch it (`General Setting` -> `Share settings via markdown`).
- [Screwdriver](https://github.com/vrtmrz/obsidian-screwdriver) is quite good, but mostly we only need this.
- Customisation of the obsoleted device is now able to be deleted at once.
- We have to put the maintenance mode in at the Customisation sync dialogue.
- 0.22.1
- New feature:
- We can perform automatic conflict resolution for inactive files, and postpone only manual ones by `Postpone manual resolution of inactive files`.
- Now we can see the image in the document history dialogue.
- We can see the difference of the image, in the document history dialogue.
- And also we can highlight differences.
- Improved:
- Hidden file sync has been stabilised.
- Now automatically reloads the conflict-resolution dialogue when new conflicted revisions have arrived.
- Fixed:
- No longer periodic process runs after unloading the plug-in.
- Now the modification of binary files is surely stored in the storage.
- 0.22.0
- Refined:
- Task scheduling logics has been rewritten.
- Screen updates are also now efficient.
- Possibly many bugs and fragile behaviour has been fixed.
- Status updates and logging have been thinned out to display.
- Fixed:
- Remote-chunk-fetching now works with keeping request intervals
- New feature:
- We can show only the icons in the editor.
- Progress indicators have been more meaningful:
- 📥 Unprocessed transferred items
- 📄 Working database operation
- 💾 Working write storage processes
- ⏳ Working read storage processes
- 🛫 Pending read storage processes
- ⚙️ Working or pending storage processes of hidden files
- 🧩 Waiting chunks
- 🔌 Working Customisation items (Configuration, snippets and plug-ins)
... To continue on to `updates_old.md`.
### 0.21.0 ### 0.21.0
The E2EE encryption V2 format has been reverted. That was probably the cause of the glitch. The E2EE encryption V2 format has been reverted. That was probably the cause of the glitch.
Instead, to maintain efficiency, files are treated with Blob until just before saving. Along with this, the old-fashioned encryption format has also been discontinued. Instead, to maintain efficiency, files are treated with Blob until just before saving. Along with this, the old-fashioned encryption format has also been discontinued.