public/obsidian-livesync
1
0
Fork 0
mirror of https://github.com/vrtmrz/obsidian-livesync.git synced 2026-05-11 10:11:54 +00:00
Code Issues Packages Projects Releases Wiki Activity

Compare commits

base: public:0.19.16
Branches Tags
public:main public:feat-userhashsalt public:p2p-rpc public:test_real_obsidian public:auto_conf_2 public:auto_conf public:beta public:alpha public:refactor_services_2 public:refactor_services public:dev public:svelteui public:forprivacy
public:0.25.60 public:0.25.59 public:0.25.58 public:0.25.57 public:0.25.56+patched5 public:0.25.56+patched4 public:0.25.56+patched3 public:0.25.56+patched2 public:0.25.56+patched1 public:0.25.56 public:0.25.55 public:0.25.54 public:0.25.53 public:0.25.52-patched-3 public:0.25.52-patched-2 public:0.25.52-patched-1 public:0.25.52 public:0.25.51 public:0.25.50 public:0.25.49 public:0.25.48 public:0.25.47 public:0.25.46 public:0.25.45 public:0.25.44 public:0.25.43-patched-9 public:0.25.43-patched-8 public:0.25.43-patched-7 public:0.25.43-patched-6 public:0.25.43-patched-5 public:0.25.43-patched-4 public:0.25.43-patched-3 public:0.25.43-patched-2 public:0.25.43-patched-1 public:0.25.43 public:0.25.42 public:0.25.41-patched-3 public:0.25.41-patched-2 public:0.25.41-patched-1 public:0.25.41 public:0.25.40 public:0.25.39 public:0.25.38 public:0.25.37 public:0.25.36 public:0.25.35 public:0.25.34 public:0.25.33 public:0.25.32 public:0.25.31 public:0.25.30 public:0.25.30-dev public:0.25.29 public:0.25.28 public:0.25.27 public:0.25.26 public:0.25.25 public:0.25.24 public:0.25.24.beta3 public:0.25.24.beta2 public:0.25.24.beta1 public:0.25.23 public:0.25.23.beta1 public:0.25.22 public:0.25.21 public:0.25.21.beta2 public:0.25.21.beta1 public:0.25.20 public:0.25.19 public:0.25.18 public:0.25.17 public:0.25.16 public:0.25.15 public:0.25.14 public:0.25.13 public:0.25.12 public:0.25.11 public:0.25.10 public:0.25.9 public:0.25.8 public:0.25.7 public:0.25.6 public:0.25.5 public:0.25.4 public:0.25.3 public:0.25.2 public:0.25.1 public:0.25.0 public:0.25.0-beta1 public:0.24.31 public:0.24.30 public:0.24.29 public:0.24.28 public:0.24.27 public:0.24.26 public:0.24.25 public:0.24.24 public:0.24.23 public:0.24.22 public:0.24.21 public:0.24.20 public:0.24.19 public:0.24.18 public:0.24.17 public:0.24.16 public:0.24.15 public:0.24.14 public:0.24.13 public:0.24.12 public:0.24.11 public:0.24.10 public:0.24.9 public:0.24.8 public:0.24.7 public:0.24.6 public:0.24.5 public:0.24.4 public:0.24.3 public:0.24.2 public:0.24.1 public:0.24.0 public:0.24.0.dev-rc8 public:0.24.0.dev-rc7 public:0.24.0.dev-rc6 public:0.24.0.dev-rc5 public:0.24.0.dev-rc4 public:0.24.0.dev-rc3 public:0.24.0.dev-rc2 public:0.23.23 public:0.23.22 public:0.23.21 public:0.23.20 public:0.23.19 public:0.23.18 public:0.23.17 public:0.23.16 public:0.23.15 public:0.23.14 public:0.23.13 public:0.23.12 public:0.23.11 public:0.23.10 public:0.23.9 public:0.23.8 public:0.23.7 public:0.23.6 public:0.23.5 public:0.23.4 public:0.23.3 public:0.23.2 public:0.23.1 public:0.23.0 public:0.22.19 public:0.22.18 public:0.22.17 public:0.22.16 public:0.22.15 public:0.22.14 public:0.22.13 public:0.22.12 public:0.22.11 public:0.22.10 public:0.22.9 public:0.22.8 public:0.22.7 public:0.22.6 public:0.22.5 public:0.22.4 public:0.22.3 public:0.22.2 public:0.22.1 public:0.22.0 public:0.21.5 public:0.21.4 public:0.21.3 public:0.21.2 public:0.21.1 public:0.21.0 public:0.20.7 public:0.20.6 public:0.20.5 public:0.20.4 public:0.20.3 public:0.20.2 public:0.20.1 public:0.20.0 public:0.19.23 public:0.19.22 public:0.19.21 public:0.19.20 public:0.19.19 public:0.19.18 public:0.19.17 public:0.19.16 public:0.19.15 public:0.19.14 public:0.19.13 public:0.19.12 public:0.19.11 public:0.19.10 public:0.19.9 public:0.19.8 public:0.19.7 public:0.19.6 public:0.19.5 public:0.19.4 public:0.19.3 public:0.19.2 public:0.19.1 public:0.19.0 public:0.18.6 public:0.18.5 public:0.18.4 public:0.18.3 public:0.18.2 public:0.18.1 public:0.18.0 public:0.17.34 public:0.17.33 public:0.17.32 public:0.17.31 public:0.17.30 public:0.17.29 public:0.17.28 public:0.17.27 public:0.17.26 public:0.17.25 public:0.17.24 public:0.17.23 public:0.17.22 public:0.17.21 public:0.17.20 public:0.17.19 public:0.17.18 public:0.17.17 public:0.17.16 public:0.17.15 public:0.17.14 public:0.17.13 public:0.17.12 public:0.17.11 public:0.17.10 public:0.17.9 public:0.17.8 public:0.17.7 public:0.17.6 public:0.17.5 public:0.17.4 public:0.17.3 public:0.17.2 public:0.17.1 public:0.17.0 public:0.16.8 public:0.16.7 public:0.16.6 public:0.16.5 public:0.16.4 public:0.16.3 public:0.16.2 public:0.16.1 public:0.16.0 public:0.15.10 public:0.15.9 public:0.15.8 public:0.15.7 public:0.15.6 public:0.15.5 public:0.15.4 public:0.15.3 public:0.15.2 public:0.15.1 public:0.15.0 public:0.14.8 public:0.14.7 public:0.14.6 public:0.14.5 public:0.14.4 public:0.14.3 public:0.14.2 public:0.14.1 public:0.14.0 public:0.13.4 public:0.13.3 public:0.13.2 public:0.13.1 public:0.13.0 public:0.12.5 public:0.12.4 public:0.12.3 public:0.12.2 public:0.12.1 public:0.12.0 public:0.11.10 public:0.11.9 public:0.11.8 public:0.11.7 public:0.11.6 public:0.11.5 public:0.11.4 public:0.11.3 public:0.11.2 public:0.11.1 public:0.11.0 public:0.10.1 public:0.10.0 public:0.9.0 public:0.8.8 public:0.8.7 public:0.8.6 public:0.8.5 public:0.8.3 public:0.8.2 public:0.8.1 public:0.8.0 public:0.7.2 public:0.7.1 public:0.7.0 public:0.6.1 public:0.6.0 public:0.5.0 public:0.4.1 public:0.4.0 public:0.3.1 public:0.3.0 public:0.2.1 public:0.2.0 public:0.1.26 public:0.1.25 public:0.1.24 public:0.1.23 public:0.1.22 public:0.1.21 public:0.1.20 public:0.1.19 public:0.1.18 public:0.1.17 public:0.1.16 public:0.1.15 public:0.1.14 public:0.1.13 public:0.1.12 public:0.1.11 public:0.1.10 public:0.1.9 public:0.1.8 public:0.1.7 public:0.1.6 public:0.1.5 public:0.1.4 public:0.1.3 public:0.1.2 public:0.1.1 public:0.1.0 public:0.0.9 public:0.0.8 public:0.0.7 public:0.0.6
...
compare: public:0.20.2
Branches Tags
public:main public:feat-userhashsalt public:p2p-rpc public:test_real_obsidian public:auto_conf_2 public:auto_conf public:beta public:alpha public:refactor_services_2 public:refactor_services public:dev public:svelteui public:forprivacy
public:0.25.60 public:0.25.59 public:0.25.58 public:0.25.57 public:0.25.56+patched5 public:0.25.56+patched4 public:0.25.56+patched3 public:0.25.56+patched2 public:0.25.56+patched1 public:0.25.56 public:0.25.55 public:0.25.54 public:0.25.53 public:0.25.52-patched-3 public:0.25.52-patched-2 public:0.25.52-patched-1 public:0.25.52 public:0.25.51 public:0.25.50 public:0.25.49 public:0.25.48 public:0.25.47 public:0.25.46 public:0.25.45 public:0.25.44 public:0.25.43-patched-9 public:0.25.43-patched-8 public:0.25.43-patched-7 public:0.25.43-patched-6 public:0.25.43-patched-5 public:0.25.43-patched-4 public:0.25.43-patched-3 public:0.25.43-patched-2 public:0.25.43-patched-1 public:0.25.43 public:0.25.42 public:0.25.41-patched-3 public:0.25.41-patched-2 public:0.25.41-patched-1 public:0.25.41 public:0.25.40 public:0.25.39 public:0.25.38 public:0.25.37 public:0.25.36 public:0.25.35 public:0.25.34 public:0.25.33 public:0.25.32 public:0.25.31 public:0.25.30 public:0.25.30-dev public:0.25.29 public:0.25.28 public:0.25.27 public:0.25.26 public:0.25.25 public:0.25.24 public:0.25.24.beta3 public:0.25.24.beta2 public:0.25.24.beta1 public:0.25.23 public:0.25.23.beta1 public:0.25.22 public:0.25.21 public:0.25.21.beta2 public:0.25.21.beta1 public:0.25.20 public:0.25.19 public:0.25.18 public:0.25.17 public:0.25.16 public:0.25.15 public:0.25.14 public:0.25.13 public:0.25.12 public:0.25.11 public:0.25.10 public:0.25.9 public:0.25.8 public:0.25.7 public:0.25.6 public:0.25.5 public:0.25.4 public:0.25.3 public:0.25.2 public:0.25.1 public:0.25.0 public:0.25.0-beta1 public:0.24.31 public:0.24.30 public:0.24.29 public:0.24.28 public:0.24.27 public:0.24.26 public:0.24.25 public:0.24.24 public:0.24.23 public:0.24.22 public:0.24.21 public:0.24.20 public:0.24.19 public:0.24.18 public:0.24.17 public:0.24.16 public:0.24.15 public:0.24.14 public:0.24.13 public:0.24.12 public:0.24.11 public:0.24.10 public:0.24.9 public:0.24.8 public:0.24.7 public:0.24.6 public:0.24.5 public:0.24.4 public:0.24.3 public:0.24.2 public:0.24.1 public:0.24.0 public:0.24.0.dev-rc8 public:0.24.0.dev-rc7 public:0.24.0.dev-rc6 public:0.24.0.dev-rc5 public:0.24.0.dev-rc4 public:0.24.0.dev-rc3 public:0.24.0.dev-rc2 public:0.23.23 public:0.23.22 public:0.23.21 public:0.23.20 public:0.23.19 public:0.23.18 public:0.23.17 public:0.23.16 public:0.23.15 public:0.23.14 public:0.23.13 public:0.23.12 public:0.23.11 public:0.23.10 public:0.23.9 public:0.23.8 public:0.23.7 public:0.23.6 public:0.23.5 public:0.23.4 public:0.23.3 public:0.23.2 public:0.23.1 public:0.23.0 public:0.22.19 public:0.22.18 public:0.22.17 public:0.22.16 public:0.22.15 public:0.22.14 public:0.22.13 public:0.22.12 public:0.22.11 public:0.22.10 public:0.22.9 public:0.22.8 public:0.22.7 public:0.22.6 public:0.22.5 public:0.22.4 public:0.22.3 public:0.22.2 public:0.22.1 public:0.22.0 public:0.21.5 public:0.21.4 public:0.21.3 public:0.21.2 public:0.21.1 public:0.21.0 public:0.20.7 public:0.20.6 public:0.20.5 public:0.20.4 public:0.20.3 public:0.20.2 public:0.20.1 public:0.20.0 public:0.19.23 public:0.19.22 public:0.19.21 public:0.19.20 public:0.19.19 public:0.19.18 public:0.19.17 public:0.19.16 public:0.19.15 public:0.19.14 public:0.19.13 public:0.19.12 public:0.19.11 public:0.19.10 public:0.19.9 public:0.19.8 public:0.19.7 public:0.19.6 public:0.19.5 public:0.19.4 public:0.19.3 public:0.19.2 public:0.19.1 public:0.19.0 public:0.18.6 public:0.18.5 public:0.18.4 public:0.18.3 public:0.18.2 public:0.18.1 public:0.18.0 public:0.17.34 public:0.17.33 public:0.17.32 public:0.17.31 public:0.17.30 public:0.17.29 public:0.17.28 public:0.17.27 public:0.17.26 public:0.17.25 public:0.17.24 public:0.17.23 public:0.17.22 public:0.17.21 public:0.17.20 public:0.17.19 public:0.17.18 public:0.17.17 public:0.17.16 public:0.17.15 public:0.17.14 public:0.17.13 public:0.17.12 public:0.17.11 public:0.17.10 public:0.17.9 public:0.17.8 public:0.17.7 public:0.17.6 public:0.17.5 public:0.17.4 public:0.17.3 public:0.17.2 public:0.17.1 public:0.17.0 public:0.16.8 public:0.16.7 public:0.16.6 public:0.16.5 public:0.16.4 public:0.16.3 public:0.16.2 public:0.16.1 public:0.16.0 public:0.15.10 public:0.15.9 public:0.15.8 public:0.15.7 public:0.15.6 public:0.15.5 public:0.15.4 public:0.15.3 public:0.15.2 public:0.15.1 public:0.15.0 public:0.14.8 public:0.14.7 public:0.14.6 public:0.14.5 public:0.14.4 public:0.14.3 public:0.14.2 public:0.14.1 public:0.14.0 public:0.13.4 public:0.13.3 public:0.13.2 public:0.13.1 public:0.13.0 public:0.12.5 public:0.12.4 public:0.12.3 public:0.12.2 public:0.12.1 public:0.12.0 public:0.11.10 public:0.11.9 public:0.11.8 public:0.11.7 public:0.11.6 public:0.11.5 public:0.11.4 public:0.11.3 public:0.11.2 public:0.11.1 public:0.11.0 public:0.10.1 public:0.10.0 public:0.9.0 public:0.8.8 public:0.8.7 public:0.8.6 public:0.8.5 public:0.8.3 public:0.8.2 public:0.8.1 public:0.8.0 public:0.7.2 public:0.7.1 public:0.7.0 public:0.6.1 public:0.6.0 public:0.5.0 public:0.4.1 public:0.4.0 public:0.3.1 public:0.3.0 public:0.2.1 public:0.2.0 public:0.1.26 public:0.1.25 public:0.1.24 public:0.1.23 public:0.1.22 public:0.1.21 public:0.1.20 public:0.1.19 public:0.1.18 public:0.1.17 public:0.1.16 public:0.1.15 public:0.1.14 public:0.1.13 public:0.1.12 public:0.1.11 public:0.1.10 public:0.1.9 public:0.1.8 public:0.1.7 public:0.1.6 public:0.1.5 public:0.1.4 public:0.1.3 public:0.1.2 public:0.1.1 public:0.1.0 public:0.0.9 public:0.0.8 public:0.0.7 public:0.0.6

28 Commits
0.19.16 ... 0.20.2

Author SHA1 Message Date
vorotamoroz
c000a02f4a bump 2023-10-02 10:39:54 +01:00
vorotamoroz
79754f48d6 New feature: 
- We can delete all data of customization sync from the `Delete all customization sync data` on the `Hatch` pane.
Fixed:
- Prevent keep restarting on iOS by yielding microtasks.
 2023-10-02 10:38:54 +01:00
vorotamoroz
dd7a40630b bump 2023-10-02 09:54:56 +01:00
vorotamoroz
14406f8213 - Fixed: 
- No more UI freezing and keep restarting on iOS.
  - Diff of Non-markdown documents are now shown correctly.
- Improved:
  - Performance has been a bit improved.
  - Customization sync has gotten faster.
    - However, We lost forward compatibility again (only for this feature). Please update all devices.
- Misc
  - Terser configuration has been more aggressive.
 2023-10-02 09:53:41 +01:00
vorotamoroz
3bbd9c048d bump 2023-09-29 18:57:54 +09:00
vorotamoroz
d91c4f50b4 - Improved: 
- A New binary file handling implemented
  - A new encrypted format has been implemented
  - Now the chunk sizes will be adjusted for efficient sync
- Fixed:
  - levels of exception in some logs have been fixed
- Tidied:
  - Some Lint warnings have been suppressed.
 2023-09-29 18:55:46 +09:00
vorotamoroz
395b7fbc42 bump 2023-09-22 18:03:09 +09:00
vorotamoroz
3773e57429 Improved: 
- We can open the log pane also from the command palette now.
- Now, the hidden file scanning interval could be configured to 0.
- `Check database configuration` now points out that we do not have administrator permission.
 2023-09-22 17:59:32 +09:00
vorotamoroz
4835fce62a bump 2023-09-21 09:44:50 +01:00
vorotamoroz
ff814be4a0 Fixed: 
- Now the synchronisation will begin without our interaction.
- No longer puts the configuration of the remote database into the log while checking configuration.
- Some outdated description notes have been removed.
- Options that are meaningless depending on other settings configured are now hidden.
 2023-09-21 09:44:07 +01:00
vorotamoroz
b271b63efa bump 2023-09-20 07:05:18 +01:00
vorotamoroz
23419e476a - Fixed: 
- Hidden files are no longer handled in the initial replication.
  - Report from `Making report` fixed
 2023-09-20 07:04:31 +01:00
vorotamoroz
b9bd1f17b8 bump 2023-09-19 09:58:04 +01:00
vorotamoroz
bcce277c36 New feature: 
- `Sync on Editor save` has been implemented
- Now we can use the `Hidden file sync` and the `Customization sync` cooperatively.
- We can ignore specific plugins in Customization sync.
- Now the message of leftover conflicted files accepts our click.

Refactored:
- Parallelism functions made more explicit.
- Type errors have been reduced.

Fixed:
- Now documents would not be overwritten if they are conflicted.
- Some error messages have been fixed.
- Missing dialogue titles have been shown now.
 2023-09-19 09:53:48 +01:00
vorotamoroz
5acbbe479e Merge pull request #276 from cgcel/main 
Add Quick setup Chinese translation.
 2023-09-13 17:13:55 +09:00
vorotamoroz
c9f9d511e0 bump 2023-09-05 09:16:49 +01:00
vorotamoroz
b8cb94c498 Fixed: 
- Resolving conflicted revision has become more robust.
- LiveSync now try to keep local changes when fetching from the rebuilt remote database.
- Now, all files will be restored after performing `fetch` immediately.
 2023-09-05 09:16:11 +01:00
cgcel
52c736f6b9 update: format markdown titles. 2023-09-01 11:01:06 +08:00
cgcel
ebd1cb7777 add: Quick setup Chinese translation. 2023-09-01 10:58:07 +08:00
vorotamoroz
10decb7909 Merge pull request #259 from jt-wang/main 
Upload colab notebook into the repo, and update setup_flyio.md to reference it
 2023-08-24 13:04:02 +09:00
vorotamoroz
e0aab8d69d bump 2023-08-24 04:03:03 +01:00
vorotamoroz
618600c753 Fixed: 
- Now the empty (or deleted) file could be conflict-resolved.
 2023-08-24 04:00:56 +01:00
Jingtao Wang
d1aba87e37 Updated with latest changes 2023-08-23 19:22:07 -07:00
Jingtao Wang
db889f635e Merge pull request #1 from jt-wang/move-colab-script-into-repo 
Upload deploy_couchdb_to_flyio_v2_with_swap.ipynb into the repo, and reference it from fly.io setup doc.
 2023-08-15 14:55:37 -07:00
Jingtao Wang
dd80e634f5 Update setup_flyio.md to reference colab notebook within the repo 2023-08-15 14:54:43 -07:00
Jingtao Wang
bec6fc1a74 Upload deploy_couchdb_to_flyio_v2_with_swap.ipynb 
Upload https://gist.github.com/vrtmrz/37c3efd7842e49947aaaa7f665e5020a into the repo for unified management.
 2023-08-15 14:49:24 -07:00
vorotamoroz
5c96c7f99b bump 2023-08-08 10:45:50 +01:00
vorotamoroz
7b9724f713 Fixed: 
- Now nested ignore files could be parsed correctly.
- The unexpected deletion of hidden files in some cases has been corrected.
- Hidden file change is no longer reflected on the device which has made the change itself.
 2023-08-08 10:42:07 +01:00
30 changed files with 1457 additions and 431 deletions
Expand all files Collapse all files

322
deploy_couchdb_to_flyio_v2_with_swap.ipynb Normal file
View File

@@ -0,0 +1,322 @@
{
"cells": [
{
"cell_type": "markdown",
"metadata": {
"id": "view-in-github",
"colab_type": "text"
},
"source": [
"<a href=\"https://colab.research.google.com/gist/vrtmrz/37c3efd7842e49947aaaa7f665e5020a/deploy_couchdb_to_flyio_v2_with_swap.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>"
]
},
{
"cell_type": "markdown",
"source": [
"History:\n",
"- 18, May, 2023: Initial.\n",
"- 19, Jun., 2023: Patched for enabling swap.\n",
"- 22, Aug., 2023: Generating Setup-URI implemented."
],
"metadata": {
"id": "HiRV7G8Gk1Rs"
}
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "2Vh0mEQEZuAK"
},
"outputs": [],
"source": [
"# Configurations\n",
"import os\n",
"os.environ['region']=\"nrt\"\n",
"os.environ['couchUser']=\"alkcsa93\"\n",
"os.environ['couchPwd']=\"c349usdfnv48fsasd\""
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "SPmbB0jZauQ1"
},
"outputs": [],
"source": [
"# Delete once\n",
"!rm ./fly.toml"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "Nze7QoxLZ7Yx"
},
"outputs": [],
"source": [
"# Installation\n",
"# You have to set up your account in here.\n",
"!curl -L https://fly.io/install.sh | sh\n",
"!/root/.fly/bin/flyctl auth signup"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "MVJwsIYrbgtx"
},
"outputs": [],
"source": [
"# Generate server\n",
"!/root/.fly/bin/flyctl launch --auto-confirm --generate-name --detach --no-deploy --region ${region}\n",
"!/root/.fly/bin/fly volumes create --region ${region} couchdata --size 2 --yes"
]
},
{
"cell_type": "code",
"source": [
"# Check the toml once.\n",
"!cat fly.toml"
],
"metadata": {
"id": "2RSoO9o-i2TT"
},
"execution_count": null,
"outputs": []
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "zUtPZLVnbvdQ"
},
"outputs": [],
"source": [
"# Modify fly.toml\n",
"## Port modification\n",
"!sed -i 's/8080/5984/g' fly.toml\n",
"## Add user into.\n",
"!echo -e \"\\n[env]\\n COUCHDB_USER = \\\"${couchUser}\\\"\" >> ./fly.toml\n",
"## Set the location of an ini file which to save configurations persistently via erlang flags.\n",
"!echo -e \"\\nERL_FLAGS=\\\"-couch_ini /opt/couchdb/etc/default.ini /opt/couchdb/etc/default.d/ /opt/couchdb/etc/local.d /opt/couchdb/etc/local.ini /opt/couchdb/data/persistence.ini\\\"\" >> ./fly.toml\n",
"## Mounting volumes to store data and ini file.\n",
"!echo -e \"\\n[mounts]\\n source=\\\"couchdata\\\"\\n destination=\\\"/opt/couchdb/data\\\"\" >> ./fly.toml\n",
"!cat fly.toml"
]
},
{
"cell_type": "code",
"source": [
"# Make the Dockerfile to modify the permission of the ini file. If you want to use a specific version, you should change `latest` here.\n",
"!echo -e \"\\n[build]\\n dockerfile = \\\"./Dockerfile\\\"\" >> ./fly.toml"
],
"metadata": {
"id": "LQPsZ_dYxkTu"
},
"execution_count": null,
"outputs": []
},
{
"cell_type": "code",
"source": [
"!echo -e \"FROM couchdb:latest\\nRUN sed -i '2itouch /opt/couchdb/data/persistence.ini && chmod +w /opt/couchdb/data/persistence.ini && fallocate -l 512M /swapfile && chmod 0600 /swapfile && mkswap /swapfile && echo 10 > /proc/sys/vm/swappiness && swapon /swapfile && echo 1 > /proc/sys/vm/overcommit_memory' /docker-entrypoint.sh\" > ./Dockerfile"
],
"metadata": {
"id": "44cBeGJ9on5i"
},
"execution_count": null,
"outputs": []
},
{
"cell_type": "code",
"source": [
"# Check dockerfile\n",
"!cat ./Dockerfile"
],
"metadata": {
"id": "ai2R3BbpxRSe"
},
"execution_count": null,
"outputs": []
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "xWdsTCI6bzk2"
},
"outputs": [],
"source": [
"# Configure password\n",
"!/root/.fly/bin/flyctl secrets set COUCHDB_PASSWORD=${couchPwd}"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "k0WIQlShcXGa"
},
"outputs": [],
"source": [
"# Deploy server\n",
"# Be sure to shutdown after the test.\n",
"!/root/.fly/bin/flyctl deploy --detach --remote-only\n",
"!/root/.fly/bin/flyctl status"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "0ySggkdlfq7M"
},
"outputs": [],
"source": [
"import subprocess, json\n",
"result = subprocess.run([\"/root/.fly/bin/flyctl\",\"status\",\"-j\"], capture_output=True, text=True)\n",
"if result.returncode==0:\n",
" hostname = json.loads(result.stdout)[\"Hostname\"]\n",
" os.environ['couchHost']=\"https://%s\" % (hostname)\n",
" print(\"Your couchDB server is https://%s/\" % (hostname))\n",
"else:\n",
" print(\"Something occured.\")\n"
]
},
{
"cell_type": "code",
"source": [
"# Finish setting up the CouchDB\n",
"# Please repeat until the request is completed without error messages\n",
"# i.e., You have to redo this block while \"curl: (35) OpenSSL SSL_connect: Connection reset by peer in connection to xxxx\" is showing.\n",
"!curl -X POST \"${couchHost}/_cluster_setup\" -H \"Content-Type: application/json\" -d \"{\\\"action\\\":\\\"enable_single_node\\\",\\\"username\\\":\\\"${couchUser}\\\",\\\"password\\\":\\\"${couchPwd}\\\",\\\"bind_address\\\":\\\"0.0.0.0\\\",\\\"port\\\":5984,\\\"singlenode\\\":true}\" --user \"${couchUser}:${couchPwd}\""
],
"metadata": {
"id": "cGlSzVqlQG_z"
},
"execution_count": null,
"outputs": []
},
{
"cell_type": "code",
"source": [
"# Please repeat until all lines are completed without error messages\n",
"!curl -X PUT \"${couchHost}/_node/nonode@nohost/_config/chttpd/require_valid_user\" -H \"Content-Type: application/json\" -d '\"true\"' --user \"${couchUser}:${couchPwd}\"\n",
"!curl -X PUT \"${couchHost}/_node/nonode@nohost/_config/chttpd_auth/require_valid_user\" -H \"Content-Type: application/json\" -d '\"true\"' --user \"${couchUser}:${couchPwd}\"\n",
"!curl -X PUT \"${couchHost}/_node/nonode@nohost/_config/httpd/WWW-Authenticate\" -H \"Content-Type: application/json\" -d '\"Basic realm=\\\"couchdb\\\"\"' --user \"${couchUser}:${couchPwd}\"\n",
"!curl -X PUT \"${couchHost}/_node/nonode@nohost/_config/httpd/enable_cors\" -H \"Content-Type: application/json\" -d '\"true\"' --user \"${couchUser}:${couchPwd}\"\n",
"!curl -X PUT \"${couchHost}/_node/nonode@nohost/_config/chttpd/enable_cors\" -H \"Content-Type: application/json\" -d '\"true\"' --user \"${couchUser}:${couchPwd}\"\n",
"!curl -X PUT \"${couchHost}/_node/nonode@nohost/_config/chttpd/max_http_request_size\" -H \"Content-Type: application/json\" -d '\"4294967296\"' --user \"${couchUser}:${couchPwd}\"\n",
"!curl -X PUT \"${couchHost}/_node/nonode@nohost/_config/couchdb/max_document_size\" -H \"Content-Type: application/json\" -d '\"50000000\"' --user \"${couchUser}:${couchPwd}\"\n",
"!curl -X PUT \"${couchHost}/_node/nonode@nohost/_config/cors/credentials\" -H \"Content-Type: application/json\" -d '\"true\"' --user \"${couchUser}:${couchPwd}\"\n",
"!curl -X PUT \"${couchHost}/_node/nonode@nohost/_config/cors/origins\" -H \"Content-Type: application/json\" -d '\"app://obsidian.md,capacitor://localhost,http://localhost\"' --user \"${couchUser}:${couchPwd}\""
],
"metadata": {
"id": "JePzrsHypY18"
},
"execution_count": null,
"outputs": []
},
{
"cell_type": "markdown",
"source": [
"Now, our CouchDB has been surely installed and configured. Cheers!\n",
"\n",
"In the steps that follow, create a setup-URI.\n",
"\n",
"This URI could be imported directly into Self-hosted LiveSync, to configure the use of the CouchDB which we configured now."
],
"metadata": {
"id": "YfSOomsoXbGS"
}
},
{
"cell_type": "code",
"source": [
"# Database config\n",
"import random, string\n",
"\n",
"def randomname(n):\n",
" return ''.join(random.choices(string.ascii_letters + string.digits, k=n))\n",
"\n",
"# The database name\n",
"os.environ['database']=\"obsidiannote\"\n",
"# The passphrase to E2EE\n",
"os.environ['passphrase']=randomname(20)\n",
"\n",
"print(\"Your database:\"+os.environ['database'])\n",
"print(\"Your passphrase:\"+os.environ['passphrase'])"
],
"metadata": {
"id": "416YncOqXdNn"
},
"execution_count": null,
"outputs": []
},
{
"cell_type": "code",
"source": [
"# Install deno for make setup uri\n",
"!curl -fsSL https://deno.land/x/install/install.sh | sh"
],
"metadata": {
"id": "C4d7C0HAXgsr"
},
"execution_count": null,
"outputs": []
},
{
"cell_type": "code",
"source": [
"# Fetch module for encrypting a Setup URI\n",
"!curl -o encrypt.ts https://gist.githubusercontent.com/vrtmrz/f9d1d95ee2ca3afa1a924a2c6759b854/raw/d7a070d864a6f61403d8dc74208238d5741aeb5a/encrypt.ts"
],
"metadata": {
"id": "hQL_Dx-PXise"
},
"execution_count": null,
"outputs": []
},
{
"cell_type": "code",
"source": [
"# Make buttons!\n",
"from IPython.display import HTML\n",
"result = subprocess.run([\"/root/.deno/bin/deno\",\"run\",\"-A\",\"encrypt.ts\"], capture_output=True, text=True)\n",
"text=\"\"\n",
"if result.returncode==0:\n",
" text = result.stdout.strip()\n",
" result = HTML(f\"<button onclick=navigator.clipboard.writeText('{text}')>Copy setup uri</button><br>Importing passphrase is `welcome`. <br>If you want to synchronise in live mode, please apply a preset after setup.)\")\n",
"else:\n",
" result = \"Failed to encrypt the setup URI\"\n",
"result"
],
"metadata": {
"id": "o0gX_thFXlIZ"
},
"execution_count": null,
"outputs": []
}
],
"metadata": {
"colab": {
"provenance": [],
"private_outputs": true,
"include_colab_link": true
},
"kernelspec": {
"display_name": "Python 3",
"name": "python3"
},
"language_info": {
"name": "python"
},
"gpuClass": "standard"
},
"nbformat": 4,
"nbformat_minor": 0
}

3
docs/quick_setup.md
View File

@@ -1,4 +1,7 @@
# Quick setup # Quick setup
[Japanese docs](./quick_setup_ja.md) - [Chinese docs](./quick_setup_cn.md).
The plugin has so many configuration options to deal with different circumstances. However, there are not so many settings that are actually used. Therefore, `The Setup wizard` has been implemented to simplify the initial setup. The plugin has so many configuration options to deal with different circumstances. However, there are not so many settings that are actually used. Therefore, `The Setup wizard` has been implemented to simplify the initial setup.
Note: Subsequent devices are recommended to be set up using the `Copy setup URI` and `Open setup URI`. Note: Subsequent devices are recommended to be set up using the `Copy setup URI` and `Open setup URI`.

93
docs/quick_setup_cn.md Normal file
View File

@@ -0,0 +1,93 @@
# 快速配置 (Quick setup)
该插件有较多配置项, 可以应对不同的情况. 不过, 实际使用的设置并不多. 因此, 我们采用了 "设置向导 (The Setup wizard)" 来简化初始设置.
Note: 建议使用 `Copy setup URI` and `Open setup URI` 来设置后续设备.
## 设置向导 (The Setup wizard)
在设置对话框中打开 `🧙‍♂️ Setup wizard`. 如果之前未配置插件, 则会自动打开该页面.
![quick_setup_1](../images/quick_setup_1.png)
- 放弃现有配置并进行设置
如果您先前有过任何设置, 此按钮允许您在设置前放弃所有更改.
- 保留现有配置和设置
快速重新配置. 请注意, 在向导模式下, 您无法看到所有已经配置过的配置项.
在上述选项中按下 `Next`, 配置对话框将进入向导模式 (wizard mode).
### 向导模式 (Wizard mode)
![quick_setup_2](../images/quick_setup_2.png)
接下来将介绍如何逐步使用向导模式.
## 配置远程数据库
### 开始配置远程数据库
输入已部署好的数据库的信息.
![quick_setup_3](../images/quick_setup_3.png)
#### 测试数据库连接并检查数据库配置
我们可以检查数据库的连接性和数据库设置.
![quick_setup_5](../images/quick_setup_5.png)
#### 测试数据库连接
检查是否能成功连接数据库. 如果连接失败, 可能是多种原因导致的, 但请先点击 `Check database configuration` 来检查数据库配置是否有问题.
#### 检查数据库配置
检查数据库设置并修复问题.
![quick_setup_6](../images/quick_setup_6.png)
Config check 的显示内容可能因不同连接而异. 在上图情况下, 按下所有三个修复按钮.
如果修复按钮消失, 全部变为复选标记, 则表示修复完成.
### 加密配置
![quick_setup_4](../images/quick_setup_4.png)
为您的数据库加密, 以防数据库意外曝光; 启用端到端加密后, 笔记内容在离开设备时就会被加密. 我们强烈建议启用该功能. `路径混淆 (Path Obfuscation)` 还能混淆文件名. 现已稳定并推荐使用.
加密基于 256 位 AES-GCM.
如果你在一个封闭的网络中, 而且很明显第三方不会访问你的文件, 则可以禁用这些设置.
![quick_setup_7](../images/quick_setup_7.png)
#### Next
转到同步设置.
#### 放弃现有数据库并继续
清除远程数据库的内容, 然后转到同步设置.
### 同步设置
最后, 选择一个同步预设完成向导.
![quick_setup_9_1](../images/quick_setup_9_1.png)
选择我们要使用的任何同步方法, 然后 `Apply` 初始化并按要求建立本地和远程数据库. 如果显示 `All done!`, 我们就完成了. `Copy setup URI` 将自动打开，并要求我们输入密码以加密 `Setup URI`.
![quick_setup_10](../images/quick_setup_10.png)
根据需要设置密码。.
设置 URI (Setup URI) 将被复制到剪贴板, 然后您可以通过某种方式将其传输到第二个及后续设备.
## 如何设置第二单元和后续单元 (the second and subsequent units)
在第一台设备上安装 Self-hosted LiveSync 后, 从命令面板上选择 `Open setup URI`, 然后输入您传输的设置 URI (Setup URI). 然后输入密码，安装向导就会打开.
在弹窗中选择以下内容.
- `Importing LiveSync's conf, OK?` 选择 `Yes`
- `How would you like to set it up?`. 选择 `Set it up as secondary or subsequent device`
然后, 配置将生效并开始复制. 您的文件很快就会同步! 您可能需要关闭设置对话框并重新打开, 才能看到设置字段正确填充, 但它们都将设置好.

2
docs/setup_flyio.md
View File

@@ -280,7 +280,7 @@ Now the CouchDB is ready to use from Self-hosted LiveSync. We can use `https://b
## Automatic setup using Colaboratory ## Automatic setup using Colaboratory
We can perform all these steps by using [this Colaboratory notebook](https://gist.github.com/vrtmrz/b437a539af25ef191bd452aae369242f) without installing anything. We can perform all these steps by using [this Colaboratory notebook](/deploy_couchdb_to_flyio_v2_with_swap.ipynb) without installing anything.
## After testing / before creating a new instance ## After testing / before creating a new instance

4
esbuild.config.mjs
View File

@@ -48,7 +48,9 @@ const terserOpt = {
lhs_constants: true, lhs_constants: true,
hoist_props: true, hoist_props: true,
side_effects: true, side_effects: true,
// if_return: true, if_return: true,
ecma: 2018,
unused: true,
}, },
// mangle: { // mangle: {
// // mangle options // // mangle options

2
manifest.json
View File

@@ -1,7 +1,7 @@
{ {
"id": "obsidian-livesync", "id": "obsidian-livesync",
"name": "Self-hosted LiveSync", "name": "Self-hosted LiveSync",
"version": "0.19.16", "version": "0.20.2",
"minAppVersion": "0.9.12", "minAppVersion": "0.9.12",
"description": "Community implementation of self-hosted livesync. Reflect your vault changes to some other devices immediately. Please make sure to disable other synchronize solutions to avoid content corruption or duplication.", "description": "Community implementation of self-hosted livesync. Reflect your vault changes to some other devices immediately. Please make sure to disable other synchronize solutions to avoid content corruption or duplication.",
"author": "vorotamoroz", "author": "vorotamoroz",

4
package-lock.json generated
View File

@@ -1,12 +1,12 @@
{ {
"name": "obsidian-livesync", "name": "obsidian-livesync",
"version": "0.19.16", "version": "0.20.2",
"lockfileVersion": 2, "lockfileVersion": 2,
"requires": true, "requires": true,
"packages": { "packages": {
"": { "": {
"name": "obsidian-livesync", "name": "obsidian-livesync",
"version": "0.19.16", "version": "0.20.2",
"license": "MIT", "license": "MIT",
"dependencies": { "dependencies": {
"diff-match-patch": "^1.0.5", "diff-match-patch": "^1.0.5",

2
package.json
View File

@@ -1,6 +1,6 @@
{ {
"name": "obsidian-livesync", "name": "obsidian-livesync",
"version": "0.19.16", "version": "0.20.2",
"description": "Reflect your vault changes to some other devices immediately. Please make sure to disable other synchronize solutions to avoid content corruption or duplication.", "description": "Reflect your vault changes to some other devices immediately. Please make sure to disable other synchronize solutions to avoid content corruption or duplication.",
"main": "main.js", "main": "main.js",
"type": "module", "type": "module",

189
src/CmdConfigSync.ts
View File

@@ -1,14 +1,14 @@
import { writable } from 'svelte/store'; import { writable } from 'svelte/store';
import { Notice, type PluginManifest, parseYaml } from "./deps"; import { Notice, type PluginManifest, parseYaml, normalizePath } from "./deps";
import type { EntryDoc, LoadedEntry, InternalFileEntry, FilePathWithPrefix, FilePath, DocumentID, AnyEntry } from "./lib/src/types"; import type { EntryDoc, LoadedEntry, InternalFileEntry, FilePathWithPrefix, FilePath, DocumentID, AnyEntry } from "./lib/src/types";
import { LOG_LEVEL_INFO, LOG_LEVEL_NOTICE, LOG_LEVEL_VERBOSE } from "./lib/src/types"; import { LOG_LEVEL_INFO, LOG_LEVEL_NOTICE, LOG_LEVEL_VERBOSE, MODE_SELECTIVE } from "./lib/src/types";
import { ICXHeader, PERIODIC_PLUGIN_SWEEP, } from "./types"; import { ICXHeader, PERIODIC_PLUGIN_SWEEP, } from "./types";
import { delay, getDocData } from "./lib/src/utils"; import { delay, getDocData } from "./lib/src/utils";
import { Logger } from "./lib/src/logger"; import { Logger } from "./lib/src/logger";
import { WrappedNotice } from "./lib/src/wrapper"; import { WrappedNotice } from "./lib/src/wrapper";
import { base64ToArrayBuffer, arrayBufferToBase64, readString, crc32CKHash } from "./lib/src/strbin"; import { readString, crc32CKHash, decodeBinary, encodeBinary } from "./lib/src/strbin";
import { runWithLock } from "./lib/src/lock"; import { serialized } from "./lib/src/lock";
import { LiveSyncCommands } from "./LiveSyncCommands"; import { LiveSyncCommands } from "./LiveSyncCommands";
import { stripAllPrefixes } from "./lib/src/path"; import { stripAllPrefixes } from "./lib/src/path";
import { PeriodicProcessor, askYesNo, disposeMemoObject, memoIfNotExist, memoObject, retrieveMemoObject, scheduleTask } from "./utils"; import { PeriodicProcessor, askYesNo, disposeMemoObject, memoIfNotExist, memoObject, retrieveMemoObject, scheduleTask } from "./utils";
@@ -16,12 +16,108 @@ import { PluginDialogModal } from "./dialogs";
import { JsonResolveModal } from "./JsonResolveModal"; import { JsonResolveModal } from "./JsonResolveModal";
import { pipeGeneratorToGenerator, processAllGeneratorTasksWithConcurrencyLimit } from './lib/src/task'; import { pipeGeneratorToGenerator, processAllGeneratorTasksWithConcurrencyLimit } from './lib/src/task';
const d = "\u200b";
const d2 = "\n";
function serialize<T>(obj: T): string { function serialize(data: PluginDataEx): string {
return JSON.stringify(obj, null, 1); // For higher performance, create custom plug-in data strings.
// Self-hosted LiveSync uses `\n` to split chunks. Therefore, grouping together those with similar entropy would work nicely.
let ret = "";
ret += ":";
ret += data.category + d + data.name + d + data.term + d2;
ret += (data.version ?? "") + d2;
ret += data.mtime + d2;
for (const file of data.files) {
ret += file.filename + d + (file.displayName ?? "") + d + (file.version ?? "") + d2;
ret += file.mtime + d + file.size + d2;
for (const data of file.data ?? []) {
ret += data + d
}
ret += d2;
}
return ret;
} }
function fetchToken(source: string, from: number): [next: number, token: string] {
const limitIdx = source.indexOf(d2, from);
const limit = limitIdx == -1 ? source.length : limitIdx;
const delimiterIdx = source.indexOf(d, from);
const delimiter = delimiterIdx == -1 ? source.length : delimiterIdx;
const tokenEnd = Math.min(limit, delimiter);
let next = tokenEnd;
if (limit < delimiter) {
next = tokenEnd;
} else {
next = tokenEnd + 1
}
return [next, source.substring(from, tokenEnd)];
}
function getTokenizer(source: string) {
const t = {
pos: 1,
next() {
const [next, token] = fetchToken(source, this.pos);
this.pos = next;
return token;
},
nextLine() {
const nextPos = source.indexOf(d2, this.pos);
if (nextPos == -1) {
this.pos = source.length;
} else {
this.pos = nextPos + 1;
}
}
}
return t;
}
function deserialize2(str: string): PluginDataEx {
const tokens = getTokenizer(str);
const ret = {} as PluginDataEx;
const category = tokens.next();
const name = tokens.next();
const term = tokens.next();
tokens.nextLine();
const version = tokens.next();
tokens.nextLine();
const mtime = Number(tokens.next());
tokens.nextLine();
const result: PluginDataEx = Object.assign(ret,
{ category, name, term, version, mtime, files: [] as PluginDataExFile[] })
let filename = "";
do {
filename = tokens.next();
if (!filename) break;
const displayName = tokens.next();
const version = tokens.next();
tokens.nextLine();
const mtime = Number(tokens.next());
const size = Number(tokens.next());
tokens.nextLine();
const data = [] as string[];
let piece = "";
do {
piece = tokens.next();
if (piece == "") break;
data.push(piece);
} while (piece != "");
result.files.push(
{
filename,
displayName,
version,
mtime,
size,
data
}
)
} while (filename);
return result;
}
function deserialize<T>(str: string, def: T) { function deserialize<T>(str: string, def: T) {
try { try {
if (str[0] == ":") return deserialize2(str);
return JSON.parse(str) as T; return JSON.parse(str) as T;
} catch (ex) { } catch (ex) {
try { try {
@@ -107,6 +203,7 @@ export class ConfigSync extends LiveSyncCommands {
}, },
}); });
} }
getFileCategory(filePath: string): "CONFIG" | "THEME" | "SNIPPET" | "PLUGIN_MAIN" | "PLUGIN_ETC" | "PLUGIN_DATA" | "" { getFileCategory(filePath: string): "CONFIG" | "THEME" | "SNIPPET" | "PLUGIN_MAIN" | "PLUGIN_ETC" | "PLUGIN_DATA" | "" {
if (filePath.split("/").length == 2 && filePath.endsWith(".json")) return "CONFIG"; if (filePath.split("/").length == 2 && filePath.endsWith(".json")) return "CONFIG";
if (filePath.split("/").length == 4 && filePath.startsWith(`${this.app.vault.configDir}/themes/`)) return "THEME"; if (filePath.split("/").length == 4 && filePath.startsWith(`${this.app.vault.configDir}/themes/`)) return "THEME";
@@ -164,6 +261,46 @@ export class ConfigSync extends LiveSyncCommands {
pluginList.set(this.pluginList) pluginList.set(this.pluginList)
await this.updatePluginList(showMessage); await this.updatePluginList(showMessage);
} }
async loadPluginData(path: FilePathWithPrefix): Promise<PluginDataExDisplay | false> {
const wx = await this.localDatabase.getDBEntry(path, null, false, false);
if (wx) {
const data = deserialize(getDocData(wx.data), {}) as PluginDataEx;
const xFiles = [] as PluginDataExFile[];
for (const file of data.files) {
const work = { ...file };
const tempStr = getDocData(work.data);
work.data = [crc32CKHash(tempStr)];
xFiles.push(work);
}
return ({
...data,
documentPath: this.getPath(wx),
files: xFiles
}) as PluginDataExDisplay;
}
return false;
}
createMissingConfigurationEntry() {
let saveRequired = false;
for (const v of this.pluginList) {
const key = `${v.category}/${v.name}`;
if (!(key in this.plugin.settings.pluginSyncExtendedSetting)) {
this.plugin.settings.pluginSyncExtendedSetting[key] = {
key,
mode: MODE_SELECTIVE,
files: []
}
}
if (this.plugin.settings.pluginSyncExtendedSetting[key].files.sort().join(",").toLowerCase() !=
v.files.map(e => e.filename).sort().join(",").toLowerCase()) {
this.plugin.settings.pluginSyncExtendedSetting[key].files = v.files.map(e => e.filename).sort();
saveRequired = true;
}
}
if (saveRequired) {
this.plugin.saveSettingData();
}
}
async updatePluginList(showMessage: boolean, updatedDocumentPath?: FilePathWithPrefix): Promise<void> { async updatePluginList(showMessage: boolean, updatedDocumentPath?: FilePathWithPrefix): Promise<void> {
const logLevel = showMessage ? LOG_LEVEL_NOTICE : LOG_LEVEL_INFO; const logLevel = showMessage ? LOG_LEVEL_NOTICE : LOG_LEVEL_INFO;
// pluginList.set([]); // pluginList.set([]);
@@ -174,7 +311,7 @@ export class ConfigSync extends LiveSyncCommands {
} }
await Promise.resolve(); // Just to prevent warning. await Promise.resolve(); // Just to prevent warning.
scheduleTask("update-plugin-list-task", 200, async () => { scheduleTask("update-plugin-list-task", 200, async () => {
await runWithLock("update-plugin-list", false, async () => { await serialized("update-plugin-list", async () => {
try { try {
const updatedDocumentId = updatedDocumentPath ? await this.path2id(updatedDocumentPath) : ""; const updatedDocumentId = updatedDocumentPath ? await this.path2id(updatedDocumentPath) : "";
const plugins = updatedDocumentPath ? const plugins = updatedDocumentPath ?
@@ -193,22 +330,7 @@ export class ConfigSync extends LiveSyncCommands {
count++; count++;
if (count % 10 == 0) Logger(`Enumerating files... ${count}`, logLevel, "get-plugins"); if (count % 10 == 0) Logger(`Enumerating files... ${count}`, logLevel, "get-plugins");
Logger(`plugin-${path}`, LOG_LEVEL_VERBOSE); Logger(`plugin-${path}`, LOG_LEVEL_VERBOSE);
const wx = await this.localDatabase.getDBEntry(path, null, false, false); return this.loadPluginData(path);
if (wx) {
const data = deserialize(getDocData(wx.data), {}) as PluginDataEx;
const xFiles = [] as PluginDataExFile[];
for (const file of data.files) {
const work = { ...file };
const tempStr = getDocData(work.data);
work.data = [crc32CKHash(tempStr)];
xFiles.push(work);
}
return ({
...data,
documentPath: this.getPath(wx),
files: xFiles
});
}
// return entries; // return entries;
} catch (ex) { } catch (ex) {
//TODO //TODO
@@ -218,7 +340,7 @@ export class ConfigSync extends LiveSyncCommands {
return false; return false;
}))) { }))) {
if ("ok" in v) { if ("ok" in v) {
if (v.ok != false) { if (v.ok !== false) {
let newList = [...this.pluginList]; let newList = [...this.pluginList];
const item = v.ok; const item = v.ok;
newList = newList.filter(x => x.documentPath != item.documentPath); newList = newList.filter(x => x.documentPath != item.documentPath);
@@ -230,6 +352,7 @@ export class ConfigSync extends LiveSyncCommands {
} }
} }
Logger(`All files enumerated`, logLevel, "get-plugins"); Logger(`All files enumerated`, logLevel, "get-plugins");
this.createMissingConfigurationEntry();
} finally { } finally {
pluginIsEnumerating.set(false); pluginIsEnumerating.set(false);
} }
@@ -257,7 +380,7 @@ export class ConfigSync extends LiveSyncCommands {
const fileA = { ...pluginDataA.files[0], ctime: pluginDataA.files[0].mtime, _id: `${pluginDataA.documentPath}` as DocumentID }; const fileA = { ...pluginDataA.files[0], ctime: pluginDataA.files[0].mtime, _id: `${pluginDataA.documentPath}` as DocumentID };
const fileB = pluginDataB.files[0]; const fileB = pluginDataB.files[0];
const docAx = { ...docA, ...fileA } as LoadedEntry, docBx = { ...docB, ...fileB } as LoadedEntry const docAx = { ...docA, ...fileA } as LoadedEntry, docBx = { ...docB, ...fileB } as LoadedEntry
return runWithLock("config:merge-data", false, () => new Promise((res) => { return serialized("config:merge-data", () => new Promise((res) => {
Logger("Opening data-merging dialog", LOG_LEVEL_VERBOSE); Logger("Opening data-merging dialog", LOG_LEVEL_VERBOSE);
// const docs = [docA, docB]; // const docs = [docA, docB];
const path = stripAllPrefixes(docAx.path.split("/").slice(-1).join("/") as FilePath); const path = stripAllPrefixes(docAx.path.split("/").slice(-1).join("/") as FilePath);
@@ -291,7 +414,7 @@ export class ConfigSync extends LiveSyncCommands {
const path = `${baseDir}/${f.filename}`; const path = `${baseDir}/${f.filename}`;
await this.ensureDirectoryEx(path); await this.ensureDirectoryEx(path);
if (!content) { if (!content) {
const dt = base64ToArrayBuffer(f.data); const dt = decodeBinary(f.data);
await this.app.vault.adapter.writeBinary(path, dt); await this.app.vault.adapter.writeBinary(path, dt);
} else { } else {
await this.app.vault.adapter.write(path, content); await this.app.vault.adapter.write(path, content);
@@ -411,6 +534,7 @@ export class ConfigSync extends LiveSyncCommands {
this.periodicPluginSweepProcessor.enable(this.settings.autoSweepPluginsPeriodic && !this.settings.watchInternalFileChanges ? (PERIODIC_PLUGIN_SWEEP * 1000) : 0); this.periodicPluginSweepProcessor.enable(this.settings.autoSweepPluginsPeriodic && !this.settings.watchInternalFileChanges ? (PERIODIC_PLUGIN_SWEEP * 1000) : 0);
return; return;
} }
recentProcessedInternalFiles = [] as string[]; recentProcessedInternalFiles = [] as string[];
async makeEntryFromFile(path: FilePath): Promise<false | PluginDataExFile> { async makeEntryFromFile(path: FilePath): Promise<false | PluginDataExFile> {
const stat = await this.app.vault.adapter.stat(path); const stat = await this.app.vault.adapter.stat(path);
@@ -422,7 +546,7 @@ export class ConfigSync extends LiveSyncCommands {
const contentBin = await this.app.vault.adapter.readBinary(path); const contentBin = await this.app.vault.adapter.readBinary(path);
let content: string[]; let content: string[];
try { try {
content = await arrayBufferToBase64(contentBin); content = await encodeBinary(contentBin, this.settings.useV1);
if (path.toLowerCase().endsWith("/manifest.json")) { if (path.toLowerCase().endsWith("/manifest.json")) {
const v = readString(new Uint8Array(contentBin)); const v = readString(new Uint8Array(contentBin));
try { try {
@@ -470,7 +594,7 @@ export class ConfigSync extends LiveSyncCommands {
return; return;
} }
const vf = this.filenameToUnifiedKey(path, term); const vf = this.filenameToUnifiedKey(path, term);
return await runWithLock(`plugin-${vf}`, false, async () => { return await serialized(`plugin-${vf}`, async () => {
const category = this.getFileCategory(path); const category = this.getFileCategory(path);
let mtime = 0; let mtime = 0;
let fileTargets = [] as FilePath[]; let fileTargets = [] as FilePath[];
@@ -578,6 +702,13 @@ export class ConfigSync extends LiveSyncCommands {
// Make sure that target is a file. // Make sure that target is a file.
if (stat && stat.type != "file") if (stat && stat.type != "file")
return false; return false;
const configDir = normalizePath(this.app.vault.configDir);
const synchronisedInConfigSync = Object.values(this.settings.pluginSyncExtendedSetting).filter(e => e.mode != MODE_SELECTIVE).map(e => e.files).flat().map(e => `${configDir}/${e}`.toLowerCase());
if (synchronisedInConfigSync.some(e => e.startsWith(path.toLowerCase()))) {
Logger(`Customization file skipped: ${path}`, LOG_LEVEL_VERBOSE);
return;
}
const storageMTime = ~~((stat && stat.mtime || 0) / 1000); const storageMTime = ~~((stat && stat.mtime || 0) / 1000);
const key = `${path}-${storageMTime}`; const key = `${path}-${storageMTime}`;
if (this.recentProcessedInternalFiles.contains(key)) { if (this.recentProcessedInternalFiles.contains(key)) {
@@ -618,7 +749,7 @@ export class ConfigSync extends LiveSyncCommands {
// const id = await this.path2id(prefixedFileName); // const id = await this.path2id(prefixedFileName);
const mtime = new Date().getTime(); const mtime = new Date().getTime();
await runWithLock("file-x-" + prefixedFileName, false, async () => { await serialized("file-x-" + prefixedFileName, async () => {
try { try {
const old = await this.localDatabase.getDBEntryMeta(prefixedFileName, null, false) as InternalFileEntry | false; const old = await this.localDatabase.getDBEntryMeta(prefixedFileName, null, false) as InternalFileEntry | false;
let saveData: InternalFileEntry; let saveData: InternalFileEntry;

160
src/CmdHiddenFileSync.ts
View File

@@ -1,13 +1,13 @@
import { Notice, normalizePath, type PluginManifest } from "./deps"; import { normalizePath, type PluginManifest } from "./deps";
import { type EntryDoc, type LoadedEntry, type InternalFileEntry, type FilePathWithPrefix, type FilePath, LOG_LEVEL_INFO, LOG_LEVEL_NOTICE, LOG_LEVEL_VERBOSE } from "./lib/src/types"; import { type EntryDoc, type LoadedEntry, type InternalFileEntry, type FilePathWithPrefix, type FilePath, LOG_LEVEL_INFO, LOG_LEVEL_NOTICE, LOG_LEVEL_VERBOSE, MODE_SELECTIVE, MODE_PAUSED } from "./lib/src/types";
import { type InternalFileInfo, ICHeader, ICHeaderEnd } from "./types"; import { type InternalFileInfo, ICHeader, ICHeaderEnd } from "./types";
import { Parallels, delay, isDocContentSame } from "./lib/src/utils"; import { Parallels, delay, isDocContentSame } from "./lib/src/utils";
import { Logger } from "./lib/src/logger"; import { Logger } from "./lib/src/logger";
import { PouchDB } from "./lib/src/pouchdb-browser.js"; import { PouchDB } from "./lib/src/pouchdb-browser.js";
import { disposeMemoObject, memoIfNotExist, memoObject, retrieveMemoObject, scheduleTask, isInternalMetadata, PeriodicProcessor } from "./utils"; import { scheduleTask, isInternalMetadata, PeriodicProcessor } from "./utils";
import { WrappedNotice } from "./lib/src/wrapper"; import { WrappedNotice } from "./lib/src/wrapper";
import { base64ToArrayBuffer, arrayBufferToBase64 } from "./lib/src/strbin"; import { decodeBinary, encodeBinary } from "./lib/src/strbin";
import { runWithLock } from "./lib/src/lock"; import { serialized } from "./lib/src/lock";
import { JsonResolveModal } from "./JsonResolveModal"; import { JsonResolveModal } from "./JsonResolveModal";
import { LiveSyncCommands } from "./LiveSyncCommands"; import { LiveSyncCommands } from "./LiveSyncCommands";
import { addPrefix, stripAllPrefixes } from "./lib/src/path"; import { addPrefix, stripAllPrefixes } from "./lib/src/path";
@@ -77,7 +77,7 @@ export class HiddenFileSync extends LiveSyncCommands {
procInternalFiles: string[] = []; procInternalFiles: string[] = [];
async execInternalFile() { async execInternalFile() {
await runWithLock("execInternal", false, async () => { await serialized("execInternal", async () => {
const w = [...this.procInternalFiles]; const w = [...this.procInternalFiles];
this.procInternalFiles = []; this.procInternalFiles = [];
Logger(`Applying hidden ${w.length} files change...`); Logger(`Applying hidden ${w.length} files change...`);
@@ -95,6 +95,14 @@ export class HiddenFileSync extends LiveSyncCommands {
recentProcessedInternalFiles = [] as string[]; recentProcessedInternalFiles = [] as string[];
async watchVaultRawEventsAsync(path: FilePath) { async watchVaultRawEventsAsync(path: FilePath) {
if (!this.settings.syncInternalFiles) return; if (!this.settings.syncInternalFiles) return;
// Exclude files handled by customization sync
const configDir = normalizePath(this.app.vault.configDir);
const synchronisedInConfigSync = !this.settings.usePluginSync ? [] : Object.values(this.settings.pluginSyncExtendedSetting).filter(e => e.mode == MODE_SELECTIVE || e.mode == MODE_PAUSED).map(e => e.files).flat().map(e => `${configDir}/${e}`.toLowerCase());
if (synchronisedInConfigSync.some(e => e.startsWith(path.toLowerCase()))) {
Logger(`Hidden file skipped: ${path} is synchronized in customization sync.`, LOG_LEVEL_VERBOSE);
return;
}
const stat = await this.app.vault.adapter.stat(path); const stat = await this.app.vault.adapter.stat(path);
// sometimes folder is coming. // sometimes folder is coming.
if (stat && stat.type != "file") if (stat && stat.type != "file")
@@ -209,18 +217,24 @@ export class HiddenFileSync extends LiveSyncCommands {
} }
//TODO: Tidy up. Even though it is experimental feature, So dirty... //TODO: Tidy up. Even though it is experimental feature, So dirty...
async syncInternalFilesAndDatabase(direction: "push" | "pull" | "safe" | "pullForce" | "pushForce", showMessage: boolean, files: InternalFileInfo[] | false = false, targetFiles: string[] | false = false) { async syncInternalFilesAndDatabase(direction: "push" | "pull" | "safe" | "pullForce" | "pushForce", showMessage: boolean, filesAll: InternalFileInfo[] | false = false, targetFiles: string[] | false = false) {
await this.resolveConflictOnInternalFiles(); await this.resolveConflictOnInternalFiles();
const logLevel = showMessage ? LOG_LEVEL_NOTICE : LOG_LEVEL_INFO; const logLevel = showMessage ? LOG_LEVEL_NOTICE : LOG_LEVEL_INFO;
Logger("Scanning hidden files.", logLevel, "sync_internal"); Logger("Scanning hidden files.", logLevel, "sync_internal");
const ignorePatterns = this.settings.syncInternalFilesIgnorePatterns const ignorePatterns = this.settings.syncInternalFilesIgnorePatterns
.replace(/\n| /g, "") .replace(/\n| /g, "")
.split(",").filter(e => e).map(e => new RegExp(e, "i")); .split(",").filter(e => e).map(e => new RegExp(e, "i"));
if (!files)
files = await this.scanInternalFiles(); const configDir = normalizePath(this.app.vault.configDir);
let files: InternalFileInfo[] =
filesAll ? filesAll : (await this.scanInternalFiles())
const synchronisedInConfigSync = !this.settings.usePluginSync ? [] : Object.values(this.settings.pluginSyncExtendedSetting).filter(e => e.mode == MODE_SELECTIVE || e.mode == MODE_PAUSED).map(e => e.files).flat().map(e => `${configDir}/${e}`.toLowerCase());
files = files.filter(file => synchronisedInConfigSync.every(filterFile => !file.path.toLowerCase().startsWith(filterFile)))
const filesOnDB = ((await this.localDatabase.allDocsRaw({ startkey: ICHeader, endkey: ICHeaderEnd, include_docs: true })).rows.map(e => e.doc) as InternalFileEntry[]).filter(e => !e.deleted); const filesOnDB = ((await this.localDatabase.allDocsRaw({ startkey: ICHeader, endkey: ICHeaderEnd, include_docs: true })).rows.map(e => e.doc) as InternalFileEntry[]).filter(e => !e.deleted);
const allFileNamesSrc = [...new Set([...files.map(e => normalizePath(e.path)), ...filesOnDB.map(e => stripAllPrefixes(this.getPath(e)))])]; const allFileNamesSrc = [...new Set([...files.map(e => normalizePath(e.path)), ...filesOnDB.map(e => stripAllPrefixes(this.getPath(e)))])];
const allFileNames = allFileNamesSrc.filter(filename => !targetFiles || (targetFiles && targetFiles.indexOf(filename) !== -1)); const allFileNames = allFileNamesSrc.filter(filename => !targetFiles || (targetFiles && targetFiles.indexOf(filename) !== -1)).filter(path => synchronisedInConfigSync.every(filterFile => !path.toLowerCase().startsWith(filterFile)))
function compareMTime(a: number, b: number) { function compareMTime(a: number, b: number) {
const wa = ~~(a / 1000); const wa = ~~(a / 1000);
const wb = ~~(b / 1000); const wb = ~~(b / 1000);
@@ -273,8 +287,8 @@ export class HiddenFileSync extends LiveSyncCommands {
if (!filename) continue; if (!filename) continue;
if (ignorePatterns.some(e => filename.match(e))) if (ignorePatterns.some(e => filename.match(e)))
continue; continue;
if (!await this.plugin.isIgnoredByIgnoreFiles(filename)) { if (await this.plugin.isIgnoredByIgnoreFiles(filename)) {
continue continue;
} }
const fileOnStorage = filename in filesMap ? filesMap[filename] : undefined; const fileOnStorage = filename in filesMap ? filesMap[filename] : undefined;
@@ -335,7 +349,6 @@ export class HiddenFileSync extends LiveSyncCommands {
// When files has been retrieved from the database. they must be reloaded. // When files has been retrieved from the database. they must be reloaded.
if ((direction == "pull" || direction == "pullForce") && filesChanged != 0) { if ((direction == "pull" || direction == "pullForce") && filesChanged != 0) {
const configDir = normalizePath(this.app.vault.configDir);
// Show notification to restart obsidian when something has been changed in configDir. // Show notification to restart obsidian when something has been changed in configDir.
if (configDir in updatedFolders) { if (configDir in updatedFolders) {
// Numbers of updated files that is below of configDir. // Numbers of updated files that is below of configDir.
@@ -352,44 +365,18 @@ export class HiddenFileSync extends LiveSyncCommands {
updatedCount -= updatedFolders[manifest.dir]; updatedCount -= updatedFolders[manifest.dir];
const updatePluginId = manifest.id; const updatePluginId = manifest.id;
const updatePluginName = manifest.name; const updatePluginName = manifest.name;
const fragment = createFragment((doc) => { this.plugin.askInPopup(`updated-${updatePluginId}`, `Files in ${updatePluginName} has been updated, Press {HERE} to reload ${updatePluginName}, or press elsewhere to dismiss this message.`, (anchor) => {
doc.createEl("span", null, (a) => { anchor.text = "HERE";
a.appendText(`Files in ${updatePluginName} has been updated, Press `); anchor.addEventListener("click", async () => {
a.appendChild(a.createEl("a", null, (anchor) => { Logger(`Unloading plugin: ${updatePluginName}`, LOG_LEVEL_NOTICE, "plugin-reload-" + updatePluginId);
anchor.text = "HERE"; // @ts-ignore
anchor.addEventListener("click", async () => { await this.app.plugins.unloadPlugin(updatePluginId);
Logger(`Unloading plugin: ${updatePluginName}`, LOG_LEVEL_NOTICE, "plugin-reload-" + updatePluginId); // @ts-ignore
// @ts-ignore await this.app.plugins.loadPlugin(updatePluginId);
await this.app.plugins.unloadPlugin(updatePluginId); Logger(`Plugin reloaded: ${updatePluginName}`, LOG_LEVEL_NOTICE, "plugin-reload-" + updatePluginId);
// @ts-ignore
await this.app.plugins.loadPlugin(updatePluginId);
Logger(`Plugin reloaded: ${updatePluginName}`, LOG_LEVEL_NOTICE, "plugin-reload-" + updatePluginId);
});
}));
a.appendText(` to reload ${updatePluginName}, or press elsewhere to dismiss this message.`);
}); });
}); }
);
const updatedPluginKey = "popupUpdated-" + updatePluginId;
scheduleTask(updatedPluginKey, 1000, async () => {
const popup = await memoIfNotExist(updatedPluginKey, () => new Notice(fragment, 0));
//@ts-ignore
const isShown = popup?.noticeEl?.isShown();
if (!isShown) {
memoObject(updatedPluginKey, new Notice(fragment, 0));
}
scheduleTask(updatedPluginKey + "-close", 20000, () => {
const popup = retrieveMemoObject<Notice>(updatedPluginKey);
if (!popup)
return;
//@ts-ignore
if (popup?.noticeEl?.isShown()) {
popup.hide();
}
disposeMemoObject(updatedPluginKey);
});
});
} }
} }
} catch (ex) { } catch (ex) {
@@ -400,30 +387,11 @@ export class HiddenFileSync extends LiveSyncCommands {
// If something changes left, notify for reloading Obsidian. // If something changes left, notify for reloading Obsidian.
if (updatedCount != 0) { if (updatedCount != 0) {
const fragment = createFragment((doc) => { this.plugin.askInPopup(`updated-any-hidden`, `Hidden files have been synchronized, Press {HERE} to reload Obsidian, or press elsewhere to dismiss this message.`, (anchor) => {
doc.createEl("span", null, (a) => { anchor.text = "HERE";
a.appendText(`Hidden files have been synchronized, Press `); anchor.addEventListener("click", () => {
a.appendChild(a.createEl("a", null, (anchor) => { // @ts-ignore
anchor.text = "HERE"; this.app.commands.executeCommandById("app:reload");
anchor.addEventListener("click", () => {
// @ts-ignore
this.app.commands.executeCommandById("app:reload");
});
}));
a.appendText(` to reload obsidian, or press elsewhere to dismiss this message.`);
});
});
scheduleTask("popupUpdated-" + configDir, 1000, () => {
//@ts-ignore
const isShown = this.confirmPopup?.noticeEl?.isShown();
if (!isShown) {
this.confirmPopup = new Notice(fragment, 0);
}
scheduleTask("popupClose" + configDir, 20000, () => {
this.confirmPopup?.hide();
this.confirmPopup = null;
}); });
}); });
} }
@@ -434,22 +402,23 @@ export class HiddenFileSync extends LiveSyncCommands {
} }
async storeInternalFileToDatabase(file: InternalFileInfo, forceWrite = false) { async storeInternalFileToDatabase(file: InternalFileInfo, forceWrite = false) {
if (!await this.plugin.isIgnoredByIgnoreFiles(file.path)) { if (await this.plugin.isIgnoredByIgnoreFiles(file.path)) {
return return
} }
const id = await this.path2id(file.path, ICHeader); const id = await this.path2id(file.path, ICHeader);
const prefixedFileName = addPrefix(file.path, ICHeader); const prefixedFileName = addPrefix(file.path, ICHeader);
const contentBin = await this.app.vault.adapter.readBinary(file.path); const contentBin = await this.app.vault.adapter.readBinary(file.path);
let content: string[]; let content: string[];
try { try {
content = await arrayBufferToBase64(contentBin); content = await encodeBinary(contentBin, this.settings.useV1);
} catch (ex) { } catch (ex) {
Logger(`The file ${file.path} could not be encoded`); Logger(`The file ${file.path} could not be encoded`);
Logger(ex, LOG_LEVEL_VERBOSE); Logger(ex, LOG_LEVEL_VERBOSE);
return false; return false;
} }
const mtime = file.mtime; const mtime = file.mtime;
return await runWithLock("file-" + prefixedFileName, false, async () => { return await serialized("file-" + prefixedFileName, async () => {
try { try {
const old = await this.localDatabase.getDBEntry(prefixedFileName, null, false, false); const old = await this.localDatabase.getDBEntry(prefixedFileName, null, false, false);
let saveData: LoadedEntry; let saveData: LoadedEntry;
@@ -498,10 +467,10 @@ export class HiddenFileSync extends LiveSyncCommands {
const id = await this.path2id(filename, ICHeader); const id = await this.path2id(filename, ICHeader);
const prefixedFileName = addPrefix(filename, ICHeader); const prefixedFileName = addPrefix(filename, ICHeader);
const mtime = new Date().getTime(); const mtime = new Date().getTime();
if (!await this.plugin.isIgnoredByIgnoreFiles(filename)) { if (await this.plugin.isIgnoredByIgnoreFiles(filename)) {
return return
} }
await runWithLock("file-" + prefixedFileName, false, async () => { await serialized("file-" + prefixedFileName, async () => {
try { try {
const old = await this.localDatabase.getDBEntryMeta(prefixedFileName, null, true) as InternalFileEntry | false; const old = await this.localDatabase.getDBEntryMeta(prefixedFileName, null, true) as InternalFileEntry | false;
let saveData: InternalFileEntry; let saveData: InternalFileEntry;
@@ -544,10 +513,10 @@ export class HiddenFileSync extends LiveSyncCommands {
async extractInternalFileFromDatabase(filename: FilePath, force = false) { async extractInternalFileFromDatabase(filename: FilePath, force = false) {
const isExists = await this.app.vault.adapter.exists(filename); const isExists = await this.app.vault.adapter.exists(filename);
const prefixedFileName = addPrefix(filename, ICHeader); const prefixedFileName = addPrefix(filename, ICHeader);
if (!await this.plugin.isIgnoredByIgnoreFiles(filename)) { if (await this.plugin.isIgnoredByIgnoreFiles(filename)) {
return; return;
} }
return await runWithLock("file-" + prefixedFileName, false, async () => { return await serialized("file-" + prefixedFileName, async () => {
try { try {
// Check conflicted status // Check conflicted status
//TODO option //TODO option
@@ -568,7 +537,7 @@ export class HiddenFileSync extends LiveSyncCommands {
await this.app.vault.adapter.remove(filename); await this.app.vault.adapter.remove(filename);
try { try {
//@ts-ignore internalAPI //@ts-ignore internalAPI
await app.vault.adapter.reconcileInternalFile(filename); await this.app.vault.adapter.reconcileInternalFile(filename);
} catch (ex) { } catch (ex) {
Logger("Failed to call internal API(reconcileInternalFile)", LOG_LEVEL_VERBOSE); Logger("Failed to call internal API(reconcileInternalFile)", LOG_LEVEL_VERBOSE);
Logger(ex, LOG_LEVEL_VERBOSE); Logger(ex, LOG_LEVEL_VERBOSE);
@@ -578,10 +547,10 @@ export class HiddenFileSync extends LiveSyncCommands {
} }
if (!isExists) { if (!isExists) {
await this.ensureDirectoryEx(filename); await this.ensureDirectoryEx(filename);
await this.app.vault.adapter.writeBinary(filename, base64ToArrayBuffer(fileOnDB.data), { mtime: fileOnDB.mtime, ctime: fileOnDB.ctime }); await this.app.vault.adapter.writeBinary(filename, decodeBinary(fileOnDB.data), { mtime: fileOnDB.mtime, ctime: fileOnDB.ctime });
try { try {
//@ts-ignore internalAPI //@ts-ignore internalAPI
await app.vault.adapter.reconcileInternalFile(filename); await this.app.vault.adapter.reconcileInternalFile(filename);
} catch (ex) { } catch (ex) {
Logger("Failed to call internal API(reconcileInternalFile)", LOG_LEVEL_VERBOSE); Logger("Failed to call internal API(reconcileInternalFile)", LOG_LEVEL_VERBOSE);
Logger(ex, LOG_LEVEL_VERBOSE); Logger(ex, LOG_LEVEL_VERBOSE);
@@ -590,15 +559,15 @@ export class HiddenFileSync extends LiveSyncCommands {
return true; return true;
} else { } else {
const contentBin = await this.app.vault.adapter.readBinary(filename); const contentBin = await this.app.vault.adapter.readBinary(filename);
const content = await arrayBufferToBase64(contentBin); const content = await encodeBinary(contentBin, this.settings.useV1);
if (content == fileOnDB.data && !force) { if (isDocContentSame(content, fileOnDB.data) && !force) {
// Logger(`STORAGE <-- DB:${filename}: skipped (hidden) Not changed`, LOG_LEVEL_VERBOSE); // Logger(`STORAGE <-- DB:${filename}: skipped (hidden) Not changed`, LOG_LEVEL_VERBOSE);
return true; return true;
} }
await this.app.vault.adapter.writeBinary(filename, base64ToArrayBuffer(fileOnDB.data), { mtime: fileOnDB.mtime, ctime: fileOnDB.ctime }); await this.app.vault.adapter.writeBinary(filename, decodeBinary(fileOnDB.data), { mtime: fileOnDB.mtime, ctime: fileOnDB.ctime });
try { try {
//@ts-ignore internalAPI //@ts-ignore internalAPI
await app.vault.adapter.reconcileInternalFile(filename); await this.app.vault.adapter.reconcileInternalFile(filename);
} catch (ex) { } catch (ex) {
Logger("Failed to call internal API(reconcileInternalFile)", LOG_LEVEL_VERBOSE); Logger("Failed to call internal API(reconcileInternalFile)", LOG_LEVEL_VERBOSE);
Logger(ex, LOG_LEVEL_VERBOSE); Logger(ex, LOG_LEVEL_VERBOSE);
@@ -618,7 +587,7 @@ export class HiddenFileSync extends LiveSyncCommands {
showJSONMergeDialogAndMerge(docA: LoadedEntry, docB: LoadedEntry): Promise<boolean> { showJSONMergeDialogAndMerge(docA: LoadedEntry, docB: LoadedEntry): Promise<boolean> {
return runWithLock("conflict:merge-data", false, () => new Promise((res) => { return serialized("conflict:merge-data", () => new Promise((res) => {
Logger("Opening data-merging dialog", LOG_LEVEL_VERBOSE); Logger("Opening data-merging dialog", LOG_LEVEL_VERBOSE);
const docs = [docA, docB]; const docs = [docA, docB];
const path = stripAllPrefixes(docA.path); const path = stripAllPrefixes(docA.path);
@@ -653,7 +622,7 @@ export class HiddenFileSync extends LiveSyncCommands {
await this.storeInternalFileToDatabase({ path: filename, ...stat }, true); await this.storeInternalFileToDatabase({ path: filename, ...stat }, true);
try { try {
//@ts-ignore internalAPI //@ts-ignore internalAPI
await app.vault.adapter.reconcileInternalFile(filename); await this.app.vault.adapter.reconcileInternalFile(filename);
} catch (ex) { } catch (ex) {
Logger("Failed to call internal API(reconcileInternalFile)", LOG_LEVEL_VERBOSE); Logger("Failed to call internal API(reconcileInternalFile)", LOG_LEVEL_VERBOSE);
Logger(ex, LOG_LEVEL_VERBOSE); Logger(ex, LOG_LEVEL_VERBOSE);
@@ -676,13 +645,16 @@ export class HiddenFileSync extends LiveSyncCommands {
} }
async scanInternalFiles(): Promise<InternalFileInfo[]> { async scanInternalFiles(): Promise<InternalFileInfo[]> {
const configDir = normalizePath(this.app.vault.configDir);
const ignoreFilter = this.settings.syncInternalFilesIgnorePatterns const ignoreFilter = this.settings.syncInternalFilesIgnorePatterns
.replace(/\n| /g, "") .replace(/\n| /g, "")
.split(",").filter(e => e).map(e => new RegExp(e, "i")); .split(",").filter(e => e).map(e => new RegExp(e, "i"));
const synchronisedInConfigSync = !this.settings.usePluginSync ? [] : Object.values(this.settings.pluginSyncExtendedSetting).filter(e => e.mode == MODE_SELECTIVE || e.mode == MODE_PAUSED).map(e => e.files).flat().map(e => `${configDir}/${e}`.toLowerCase());
const root = this.app.vault.getRoot(); const root = this.app.vault.getRoot();
const findRoot = root.path; const findRoot = root.path;
const filenames = (await this.getFiles(findRoot, [], null, ignoreFilter)).filter(e => e.startsWith(".")).filter(e => !e.startsWith(".trash")); const filenames = (await this.getFiles(findRoot, [], null, ignoreFilter)).filter(e => e.startsWith(".")).filter(e => !e.startsWith(".trash"));
const files = filenames.map(async (e) => { const files = filenames.filter(path => synchronisedInConfigSync.every(filterFile => !path.toLowerCase().startsWith(filterFile))).map(async (e) => {
return { return {
path: e as FilePath, path: e as FilePath,
stat: await this.app.vault.adapter.stat(e) stat: await this.app.vault.adapter.stat(e)
@@ -691,7 +663,7 @@ export class HiddenFileSync extends LiveSyncCommands {
const result: InternalFileInfo[] = []; const result: InternalFileInfo[] = [];
for (const f of files) { for (const f of files) {
const w = await f; const w = await f;
if (!await this.plugin.isIgnoredByIgnoreFiles(w.path)) { if (await this.plugin.isIgnoredByIgnoreFiles(w.path)) {
continue continue
} }
result.push({ result.push({
@@ -716,7 +688,7 @@ export class HiddenFileSync extends LiveSyncCommands {
...w.files ...w.files
.filter((e) => !ignoreList.some((ee) => e.endsWith(ee))) .filter((e) => !ignoreList.some((ee) => e.endsWith(ee)))
.filter((e) => !filter || filter.some((ee) => e.match(ee))) .filter((e) => !filter || filter.some((ee) => e.match(ee)))
.filter((e) => !ignoreFilter || ignoreFilter.every((ee) => !e.match(ee))), .filter((e) => !ignoreFilter || ignoreFilter.every((ee) => !e.match(ee)))
]; ];
let files = [] as string[]; let files = [] as string[];
for (const file of filesSrc) { for (const file of filesSrc) {
@@ -734,7 +706,7 @@ export class HiddenFileSync extends LiveSyncCommands {
if (ignoreFilter && ignoreFilter.some(e => v.match(e))) { if (ignoreFilter && ignoreFilter.some(e => v.match(e))) {
continue L1; continue L1;
} }
if (!await this.plugin.isIgnoredByIgnoreFiles(v)) { if (await this.plugin.isIgnoredByIgnoreFiles(v)) {
continue L1; continue L1;
} }
files = files.concat(await this.getFiles(v, ignoreList, filter, ignoreFilter)); files = files.concat(await this.getFiles(v, ignoreList, filter, ignoreFilter));

10
src/CmdPluginAndTheirSettings.ts
View File

@@ -9,7 +9,7 @@ import { isPluginMetadata, PeriodicProcessor } from "./utils";
import { PluginDialogModal } from "./dialogs"; import { PluginDialogModal } from "./dialogs";
import { NewNotice } from "./lib/src/wrapper"; import { NewNotice } from "./lib/src/wrapper";
import { versionNumberString2Number } from "./lib/src/strbin"; import { versionNumberString2Number } from "./lib/src/strbin";
import { runWithLock } from "./lib/src/lock"; import { serialized, skipIfDuplicated } from "./lib/src/lock";
import { LiveSyncCommands } from "./LiveSyncCommands"; import { LiveSyncCommands } from "./LiveSyncCommands";
export class PluginAndTheirSettings extends LiveSyncCommands { export class PluginAndTheirSettings extends LiveSyncCommands {
@@ -164,7 +164,7 @@ export class PluginAndTheirSettings extends LiveSyncCommands {
if (specificPluginPath != "") { if (specificPluginPath != "") {
specificPlugin = manifests.find(e => e.dir.endsWith("/" + specificPluginPath))?.id ?? ""; specificPlugin = manifests.find(e => e.dir.endsWith("/" + specificPluginPath))?.id ?? "";
} }
await runWithLock("sweepplugin", true, async () => { await skipIfDuplicated("sweepplugin", async () => {
const logLevel = showMessage ? LOG_LEVEL_NOTICE : LOG_LEVEL_INFO; const logLevel = showMessage ? LOG_LEVEL_NOTICE : LOG_LEVEL_INFO;
if (!this.deviceAndVaultName) { if (!this.deviceAndVaultName) {
Logger("You have to set your device name.", LOG_LEVEL_NOTICE); Logger("You have to set your device name.", LOG_LEVEL_NOTICE);
@@ -223,7 +223,7 @@ export class PluginAndTheirSettings extends LiveSyncCommands {
type: "plain" type: "plain"
}; };
Logger(`check diff:${m.name}(${m.id})`, LOG_LEVEL_VERBOSE); Logger(`check diff:${m.name}(${m.id})`, LOG_LEVEL_VERBOSE);
await runWithLock("plugin-" + m.id, false, async () => { await serialized("plugin-" + m.id, async () => {
const old = await this.localDatabase.getDBEntry(p._id as string as FilePathWithPrefix /* This also should be explained */, null, false, false); const old = await this.localDatabase.getDBEntry(p._id as string as FilePathWithPrefix /* This also should be explained */, null, false, false);
if (old !== false) { if (old !== false) {
const oldData = { data: old.data, deleted: old._deleted }; const oldData = { data: old.data, deleted: old._deleted };
@@ -266,7 +266,7 @@ export class PluginAndTheirSettings extends LiveSyncCommands {
} }
async applyPluginData(plugin: PluginDataEntry) { async applyPluginData(plugin: PluginDataEntry) {
await runWithLock("plugin-" + plugin.manifest.id, false, async () => { await serialized("plugin-" + plugin.manifest.id, async () => {
const pluginTargetFolderPath = normalizePath(plugin.manifest.dir) + "/"; const pluginTargetFolderPath = normalizePath(plugin.manifest.dir) + "/";
const adapter = this.app.vault.adapter; const adapter = this.app.vault.adapter;
// @ts-ignore // @ts-ignore
@@ -288,7 +288,7 @@ export class PluginAndTheirSettings extends LiveSyncCommands {
} }
async applyPlugin(plugin: PluginDataEntry) { async applyPlugin(plugin: PluginDataEntry) {
await runWithLock("plugin-" + plugin.manifest.id, false, async () => { await serialized("plugin-" + plugin.manifest.id, async () => {
// @ts-ignore // @ts-ignore
const stat = this.app.plugins.enabledPlugins.has(plugin.manifest.id) == true; const stat = this.app.plugins.enabledPlugins.has(plugin.manifest.id) == true;
if (stat) { if (stat) {

54
src/CmdSetupLiveSync.ts
View File

@@ -20,6 +20,11 @@ export class SetupLiveSync extends LiveSyncCommands {
name: "Copy the setup URI", name: "Copy the setup URI",
callback: this.command_copySetupURI.bind(this), callback: this.command_copySetupURI.bind(this),
}); });
this.plugin.addCommand({
id: "livesync-copysetupuri-short",
name: "Copy the setup URI (With customization sync)",
callback: this.command_copySetupURIWithSync.bind(this),
});
this.plugin.addCommand({ this.plugin.addCommand({
id: "livesync-copysetupurifull", id: "livesync-copysetupurifull",
@@ -41,18 +46,21 @@ export class SetupLiveSync extends LiveSyncCommands {
} }
async realizeSettingSyncMode() { } async realizeSettingSyncMode() { }
async command_copySetupURI() { async command_copySetupURI(stripExtra = true) {
const encryptingPassphrase = await askString(this.app, "Encrypt your settings", "The passphrase to encrypt the setup URI", "", true); const encryptingPassphrase = await askString(this.app, "Encrypt your settings", "The passphrase to encrypt the setup URI", "", true);
if (encryptingPassphrase === false) if (encryptingPassphrase === false)
return; return;
const setting = { ...this.settings, configPassphraseStore: "", encryptedCouchDBConnection: "", encryptedPassphrase: "" }; const setting = { ...this.settings, configPassphraseStore: "", encryptedCouchDBConnection: "", encryptedPassphrase: "" };
if (stripExtra) {
delete setting.pluginSyncExtendedSetting;
}
const keys = Object.keys(setting) as (keyof ObsidianLiveSyncSettings)[]; const keys = Object.keys(setting) as (keyof ObsidianLiveSyncSettings)[];
for (const k of keys) { for (const k of keys) {
if (JSON.stringify(k in setting ? setting[k] : "") == JSON.stringify(k in DEFAULT_SETTINGS ? DEFAULT_SETTINGS[k] : "*")) { if (JSON.stringify(k in setting ? setting[k] : "") == JSON.stringify(k in DEFAULT_SETTINGS ? DEFAULT_SETTINGS[k] : "*")) {
delete setting[k]; delete setting[k];
} }
} }
const encryptedSetting = encodeURIComponent(await encrypt(JSON.stringify(setting), encryptingPassphrase, false)); const encryptedSetting = encodeURIComponent(await encrypt(JSON.stringify(setting), encryptingPassphrase, false, false));
const uri = `${configURIBase}${encryptedSetting}`; const uri = `${configURIBase}${encryptedSetting}`;
await navigator.clipboard.writeText(uri); await navigator.clipboard.writeText(uri);
Logger("Setup URI copied to clipboard", LOG_LEVEL_NOTICE); Logger("Setup URI copied to clipboard", LOG_LEVEL_NOTICE);
@@ -62,11 +70,14 @@ export class SetupLiveSync extends LiveSyncCommands {
if (encryptingPassphrase === false) if (encryptingPassphrase === false)
return; return;
const setting = { ...this.settings, configPassphraseStore: "", encryptedCouchDBConnection: "", encryptedPassphrase: "" }; const setting = { ...this.settings, configPassphraseStore: "", encryptedCouchDBConnection: "", encryptedPassphrase: "" };
const encryptedSetting = encodeURIComponent(await encrypt(JSON.stringify(setting), encryptingPassphrase, false)); const encryptedSetting = encodeURIComponent(await encrypt(JSON.stringify(setting), encryptingPassphrase, false, false));
const uri = `${configURIBase}${encryptedSetting}`; const uri = `${configURIBase}${encryptedSetting}`;
await navigator.clipboard.writeText(uri); await navigator.clipboard.writeText(uri);
Logger("Setup URI copied to clipboard", LOG_LEVEL_NOTICE); Logger("Setup URI copied to clipboard", LOG_LEVEL_NOTICE);
} }
async command_copySetupURIWithSync() {
this.command_copySetupURI(false);
}
async command_openSetupURI() { async command_openSetupURI() {
const setupURI = await askString(this.app, "Easy setup", "Set up URI", `${configURIBase}aaaaa`); const setupURI = await askString(this.app, "Easy setup", "Set up URI", `${configURIBase}aaaaa`);
if (setupURI === false) if (setupURI === false)
@@ -99,6 +110,7 @@ export class SetupLiveSync extends LiveSyncCommands {
newSettingW.encryptedCouchDBConnection = ""; newSettingW.encryptedCouchDBConnection = "";
const setupJustImport = "Just import setting"; const setupJustImport = "Just import setting";
const setupAsNew = "Set it up as secondary or subsequent device"; const setupAsNew = "Set it up as secondary or subsequent device";
const setupAsMerge = "Secondary device but try keeping local changes";
const setupAgain = "Reconfigure and reconstitute the data"; const setupAgain = "Reconfigure and reconstitute the data";
const setupManually = "Leave everything to me"; const setupManually = "Leave everything to me";
newSettingW.syncInternalFiles = false; newSettingW.syncInternalFiles = false;
@@ -108,7 +120,7 @@ export class SetupLiveSync extends LiveSyncCommands {
newSettingW.useIndexedDBAdapter = true; newSettingW.useIndexedDBAdapter = true;
} }
const setupType = await askSelectString(this.app, "How would you like to set it up?", [setupAsNew, setupAgain, setupJustImport, setupManually]); const setupType = await askSelectString(this.app, "How would you like to set it up?", [setupAsNew, setupAgain, setupAsMerge, setupJustImport, setupManually]);
if (setupType == setupJustImport) { if (setupType == setupJustImport) {
this.plugin.settings = newSettingW; this.plugin.settings = newSettingW;
this.plugin.usedPassphrase = ""; this.plugin.usedPassphrase = "";
@@ -117,6 +129,10 @@ export class SetupLiveSync extends LiveSyncCommands {
this.plugin.settings = newSettingW; this.plugin.settings = newSettingW;
this.plugin.usedPassphrase = ""; this.plugin.usedPassphrase = "";
await this.fetchLocal(); await this.fetchLocal();
} else if (setupType == setupAsMerge) {
this.plugin.settings = newSettingW;
this.plugin.usedPassphrase = "";
await this.fetchLocalWithKeepLocal();
} else if (setupType == setupAgain) { } else if (setupType == setupAgain) {
const confirm = "I know this operation will rebuild all my databases with files on this device, and files that are on the remote database and I didn't synchronize to any other devices will be lost and want to proceed indeed."; const confirm = "I know this operation will rebuild all my databases with files on this device, and files that are on the remote database and I didn't synchronize to any other devices will be lost and want to proceed indeed.";
if (await askSelectString(this.app, "Do you really want to do this?", ["Cancel", confirm]) != confirm) { if (await askSelectString(this.app, "Do you really want to do this?", ["Cancel", confirm]) != confirm) {
@@ -285,6 +301,7 @@ Of course, we are able to disable these features.`
this.plugin.settings.liveSync = false; this.plugin.settings.liveSync = false;
this.plugin.settings.periodicReplication = false; this.plugin.settings.periodicReplication = false;
this.plugin.settings.syncOnSave = false; this.plugin.settings.syncOnSave = false;
this.plugin.settings.syncOnEditorSave = false;
this.plugin.settings.syncOnStart = false; this.plugin.settings.syncOnStart = false;
this.plugin.settings.syncOnFileOpen = false; this.plugin.settings.syncOnFileOpen = false;
this.plugin.settings.syncAfterMerge = false; this.plugin.settings.syncAfterMerge = false;
@@ -302,13 +319,11 @@ Of course, we are able to disable these features.`
Logger(`Database and storage reflection has been resumed!`, LOG_LEVEL_NOTICE); Logger(`Database and storage reflection has been resumed!`, LOG_LEVEL_NOTICE);
this.plugin.settings.suspendParseReplicationResult = false; this.plugin.settings.suspendParseReplicationResult = false;
this.plugin.settings.suspendFileWatching = false; this.plugin.settings.suspendFileWatching = false;
await this.plugin.syncAllFiles(true);
await this.plugin.loadQueuedFiles();
this.plugin.procQueuedFiles();
await this.plugin.saveSettings(); await this.plugin.saveSettings();
if (this.plugin.settings.readChunksOnline) {
await this.plugin.syncAllFiles(true);
await this.plugin.loadQueuedFiles();
// Start processing
this.plugin.procQueuedFiles();
}
} }
async askUseNewAdapter() { async askUseNewAdapter() {
if (!this.plugin.settings.useIndexedDBAdapter) { if (!this.plugin.settings.useIndexedDBAdapter) {
@@ -353,6 +368,25 @@ Of course, we are able to disable these features.`
await this.resumeReflectingDatabase(); await this.resumeReflectingDatabase();
await this.askHiddenFileConfiguration({ enableFetch: true }); await this.askHiddenFileConfiguration({ enableFetch: true });
} }
async fetchLocalWithKeepLocal() {
this.suspendExtraSync();
this.askUseNewAdapter();
await this.suspendReflectingDatabase();
await this.plugin.realizeSettingSyncMode();
await this.plugin.resetLocalDatabase();
await delay(1000);
await this.plugin.initializeDatabase(true);
await this.plugin.markRemoteResolved();
await this.plugin.openDatabase();
this.plugin.isReady = true;
await delay(500);
await this.plugin.replicateAllFromServer(true);
await delay(1000);
await this.plugin.replicateAllFromServer(true);
await this.fetchRemoteChunks();
await this.resumeReflectingDatabase();
await this.askHiddenFileConfiguration({ enableFetch: true });
}
async rebuildRemote() { async rebuildRemote() {
this.suspendExtraSync(); this.suspendExtraSync();
await this.plugin.realizeSettingSyncMode(); await this.plugin.realizeSettingSyncMode();

4
src/ConflictResolveModal.ts
View File

@@ -18,10 +18,8 @@ export class ConflictResolveModal extends Modal {
onOpen() { onOpen() {
const { contentEl } = this; const { contentEl } = this;
this.titleEl.setText("Conflicting changes");
contentEl.empty(); contentEl.empty();
contentEl.createEl("h2", { text: "This document has conflicted changes." });
contentEl.createEl("span", { text: this.filename }); contentEl.createEl("span", { text: this.filename });
const div = contentEl.createDiv(""); const div = contentEl.createDiv("");
div.addClass("op-scrollable"); div.addClass("op-scrollable");

37
src/DocumentHistoryModal.ts
View File

@@ -1,6 +1,6 @@
import { TFile, Modal, App, DIFF_DELETE, DIFF_EQUAL, DIFF_INSERT, diff_match_patch } from "./deps"; import { TFile, Modal, App, DIFF_DELETE, DIFF_EQUAL, DIFF_INSERT, diff_match_patch } from "./deps";
import { getPathFromTFile, isValidPath } from "./utils"; import { getPathFromTFile, isValidPath } from "./utils";
import { base64ToArrayBuffer, base64ToString, escapeStringToHTML } from "./lib/src/strbin"; import { decodeBinary, escapeStringToHTML, readString } from "./lib/src/strbin";
import ObsidianLiveSyncPlugin from "./main"; import ObsidianLiveSyncPlugin from "./main";
import { type DocumentID, type FilePathWithPrefix, type LoadedEntry, LOG_LEVEL_INFO, LOG_LEVEL_NOTICE, LOG_LEVEL_VERBOSE } from "./lib/src/types"; import { type DocumentID, type FilePathWithPrefix, type LoadedEntry, LOG_LEVEL_INFO, LOG_LEVEL_NOTICE, LOG_LEVEL_VERBOSE } from "./lib/src/types";
import { Logger } from "./lib/src/logger"; import { Logger } from "./lib/src/logger";
@@ -10,29 +10,29 @@ import { stripPrefix } from "./lib/src/path";
export class DocumentHistoryModal extends Modal { export class DocumentHistoryModal extends Modal {
plugin: ObsidianLiveSyncPlugin; plugin: ObsidianLiveSyncPlugin;
range: HTMLInputElement; range!: HTMLInputElement;
contentView: HTMLDivElement; contentView!: HTMLDivElement;
info: HTMLDivElement; info!: HTMLDivElement;
fileInfo: HTMLDivElement; fileInfo!: HTMLDivElement;
showDiff = false; showDiff = false;
id: DocumentID; id?: DocumentID;
file: FilePathWithPrefix; file: FilePathWithPrefix;
revs_info: PouchDB.Core.RevisionInfo[] = []; revs_info: PouchDB.Core.RevisionInfo[] = [];
currentDoc: LoadedEntry; currentDoc?: LoadedEntry;
currentText = ""; currentText = "";
currentDeleted = false; currentDeleted = false;
initialRev: string; initialRev?: string;
constructor(app: App, plugin: ObsidianLiveSyncPlugin, file: TFile | FilePathWithPrefix, id: DocumentID, revision?: string) { constructor(app: App, plugin: ObsidianLiveSyncPlugin, file: TFile | FilePathWithPrefix, id?: DocumentID, revision?: string) {
super(app); super(app);
this.plugin = plugin; this.plugin = plugin;
this.file = (file instanceof TFile) ? getPathFromTFile(file) : file; this.file = (file instanceof TFile) ? getPathFromTFile(file) : file;
this.id = id; this.id = id;
this.initialRev = revision; this.initialRev = revision;
if (!file) { if (!file && id) {
this.file = this.plugin.id2path(id, null); this.file = this.plugin.id2path(id);
} }
if (localStorage.getItem("ols-history-highlightdiff") == "1") { if (localStorage.getItem("ols-history-highlightdiff") == "1") {
this.showDiff = true; this.showDiff = true;
@@ -46,8 +46,8 @@ export class DocumentHistoryModal extends Modal {
const db = this.plugin.localDatabase; const db = this.plugin.localDatabase;
try { try {
const w = await db.localDatabase.get(this.id, { revs_info: true }); const w = await db.localDatabase.get(this.id, { revs_info: true });
this.revs_info = w._revs_info.filter((e) => e?.status == "available"); this.revs_info = w._revs_info?.filter((e) => e?.status == "available") ?? [];
this.range.max = `${this.revs_info.length - 1}`; this.range.max = `${Math.max(this.revs_info.length - 1, 0)}`;
this.range.value = this.range.max; this.range.value = this.range.max;
this.fileInfo.setText(`${this.file} / ${this.revs_info.length} revisions`); this.fileInfo.setText(`${this.file} / ${this.revs_info.length} revisions`);
await this.loadRevs(initialRev); await this.loadRevs(initialRev);
@@ -89,8 +89,8 @@ export class DocumentHistoryModal extends Modal {
this.currentDoc = w; this.currentDoc = w;
this.info.innerHTML = `Modified:${new Date(w.mtime).toLocaleString()}`; this.info.innerHTML = `Modified:${new Date(w.mtime).toLocaleString()}`;
let result = ""; let result = "";
const w1data = w.datatype == "plain" ? getDocData(w.data) : base64ToString(w.data); const w1data = w.datatype == "plain" ? getDocData(w.data) : readString(new Uint8Array(decodeBinary(w.data)));
this.currentDeleted = w.deleted; this.currentDeleted = !!w.deleted;
this.currentText = w1data; this.currentText = w1data;
if (this.showDiff) { if (this.showDiff) {
const prevRevIdx = this.revs_info.length - 1 - ((this.range.value as any) / 1 - 1); const prevRevIdx = this.revs_info.length - 1 - ((this.range.value as any) / 1 - 1);
@@ -99,7 +99,7 @@ export class DocumentHistoryModal extends Modal {
const w2 = await db.getDBEntry(this.file, { rev: oldRev }, false, false, true); const w2 = await db.getDBEntry(this.file, { rev: oldRev }, false, false, true);
if (w2 != false) { if (w2 != false) {
const dmp = new diff_match_patch(); const dmp = new diff_match_patch();
const w2data = w2.datatype == "plain" ? getDocData(w2.data) : base64ToString(w2.data); const w2data = w2.datatype == "plain" ? getDocData(w2.data) : readString(new Uint8Array(decodeBinary(w2.data)));
const diff = dmp.diff_main(w2data, w1data); const diff = dmp.diff_main(w2data, w1data);
dmp.diff_cleanupSemantic(diff); dmp.diff_cleanupSemantic(diff);
for (const v of diff) { for (const v of diff) {
@@ -130,9 +130,8 @@ export class DocumentHistoryModal extends Modal {
onOpen() { onOpen() {
const { contentEl } = this; const { contentEl } = this;
this.titleEl.setText("Document History");
contentEl.empty(); contentEl.empty();
contentEl.createEl("h2", { text: "Document History" });
this.fileInfo = contentEl.createDiv(""); this.fileInfo = contentEl.createDiv("");
this.fileInfo.addClass("op-info"); this.fileInfo.addClass("op-info");
const divView = contentEl.createDiv(""); const divView = contentEl.createDiv("");
@@ -205,7 +204,7 @@ export class DocumentHistoryModal extends Modal {
await focusFile(pathToWrite); await focusFile(pathToWrite);
this.close(); this.close();
} else if (this.currentDoc?.datatype == "newnote") { } else if (this.currentDoc?.datatype == "newnote") {
await this.app.vault.adapter.writeBinary(pathToWrite, base64ToArrayBuffer(this.currentDoc.data)); await this.app.vault.adapter.writeBinary(pathToWrite, decodeBinary(this.currentDoc.data));
await focusFile(pathToWrite); await focusFile(pathToWrite);
this.close(); this.close();
} else { } else {

4
src/GlobalHistory.svelte
View File

@@ -7,7 +7,7 @@
import { DocumentHistoryModal } from "./DocumentHistoryModal"; import { DocumentHistoryModal } from "./DocumentHistoryModal";
import { isPlainText, stripAllPrefixes } from "./lib/src/path"; import { isPlainText, stripAllPrefixes } from "./lib/src/path";
import { TFile } from "./deps"; import { TFile } from "./deps";
import { arrayBufferToBase64 } from "./lib/src/strbin"; import { encodeBinary } from "./lib/src/strbin";
export let plugin: ObsidianLiveSyncPlugin; export let plugin: ObsidianLiveSyncPlugin;
let showDiffInfo = false; let showDiffInfo = false;
@@ -116,7 +116,7 @@
result = isDocContentSame(data, doc.data); result = isDocContentSame(data, doc.data);
} else { } else {
const data = await plugin.app.vault.readBinary(abs); const data = await plugin.app.vault.readBinary(abs);
const dataEEncoded = await arrayBufferToBase64(data); const dataEEncoded = await encodeBinary(data, plugin.settings.useV1);
result = isDocContentSame(dataEEncoded, doc.data); result = isDocContentSame(dataEEncoded, doc.data);
} }
if (result) { if (result) {

2
src/JsonResolveModal.ts
View File

@@ -29,7 +29,7 @@ export class JsonResolveModal extends Modal {
onOpen() { onOpen() {
const { contentEl } = this; const { contentEl } = this;
this.titleEl.setText("Conflicted Setting");
contentEl.empty(); contentEl.empty();
if (this.component == null) { if (this.component == null) {

5
src/JsonResolvePane.svelte
View File

@@ -1,7 +1,7 @@
<script lang="ts"> <script lang="ts">
import { type Diff, DIFF_DELETE, DIFF_INSERT, diff_match_patch } from "./deps"; import { type Diff, DIFF_DELETE, DIFF_INSERT, diff_match_patch } from "./deps";
import type { FilePath, LoadedEntry } from "./lib/src/types"; import type { FilePath, LoadedEntry } from "./lib/src/types";
import { base64ToString } from "./lib/src/strbin"; import { decodeBinary, readString } from "./lib/src/strbin";
import { getDocData } from "./lib/src/utils"; import { getDocData } from "./lib/src/utils";
import { mergeObject } from "./utils"; import { mergeObject } from "./utils";
@@ -26,7 +26,7 @@
let mode: SelectModes = defaultSelect as SelectModes; let mode: SelectModes = defaultSelect as SelectModes;
function docToString(doc: LoadedEntry) { function docToString(doc: LoadedEntry) {
return doc.datatype == "plain" ? getDocData(doc.data) : base64ToString(doc.data); return doc.datatype == "plain" ? getDocData(doc.data) : readString(new Uint8Array(decodeBinary(doc.data)));
} }
function revStringToRevNumber(rev: string) { function revStringToRevNumber(rev: string) {
return rev.split("-")[0]; return rev.split("-")[0];
@@ -104,7 +104,6 @@
] as ["" | "A" | "B" | "AB" | "BA", string][]; ] as ["" | "A" | "B" | "AB" | "BA", string][];
</script> </script>
<h1>Conflicted settings</h1>
<h2>{filename}</h2> <h2>{filename}</h2>
{#if !docA || !docB} {#if !docA || !docB}
<div class="message">Just for a minute, please!</div> <div class="message">Just for a minute, please!</div>

2
src/LogDisplayModal.ts
View File

@@ -14,9 +14,9 @@ export class LogDisplayModal extends Modal {
onOpen() { onOpen() {
const { contentEl } = this; const { contentEl } = this;
this.titleEl.setText("Sync status");
contentEl.empty(); contentEl.empty();
contentEl.createEl("h2", { text: "Sync Status" });
const div = contentEl.createDiv(""); const div = contentEl.createDiv("");
div.addClass("op-scrollable"); div.addClass("op-scrollable");
div.addClass("op-pre"); div.addClass("op-pre");

155
src/ObsidianLiveSyncSettingTab.ts
View File

@@ -1,5 +1,5 @@
import { App, PluginSettingTab, Setting, sanitizeHTMLToDom, TextAreaComponent, MarkdownRenderer, stringifyYaml } from "./deps"; import { App, PluginSettingTab, Setting, sanitizeHTMLToDom, TextAreaComponent, MarkdownRenderer, stringifyYaml } from "./deps";
import { DEFAULT_SETTINGS, type ObsidianLiveSyncSettings, type ConfigPassphraseStore, type RemoteDBSettings, type FilePathWithPrefix, type HashAlgorithm, type DocumentID, LOG_LEVEL_NOTICE, LOG_LEVEL_VERBOSE } from "./lib/src/types"; import { DEFAULT_SETTINGS, type ObsidianLiveSyncSettings, type ConfigPassphraseStore, type RemoteDBSettings, type FilePathWithPrefix, type HashAlgorithm, type DocumentID, LOG_LEVEL_NOTICE, LOG_LEVEL_VERBOSE, LOG_LEVEL_INFO } from "./lib/src/types";
import { delay } from "./lib/src/utils"; import { delay } from "./lib/src/utils";
import { Semaphore } from "./lib/src/semaphore"; import { Semaphore } from "./lib/src/semaphore";
import { versionNumberString2Number } from "./lib/src/strbin"; import { versionNumberString2Number } from "./lib/src/strbin";
@@ -120,6 +120,7 @@ export class ObsidianLiveSyncSettingTab extends PluginSettingTab {
if (this.plugin.settings.periodicReplication) return true; if (this.plugin.settings.periodicReplication) return true;
if (this.plugin.settings.syncOnFileOpen) return true; if (this.plugin.settings.syncOnFileOpen) return true;
if (this.plugin.settings.syncOnSave) return true; if (this.plugin.settings.syncOnSave) return true;
if (this.plugin.settings.syncOnEditorSave) return true;
if (this.plugin.settings.syncOnStart) return true; if (this.plugin.settings.syncOnStart) return true;
if (this.plugin.settings.syncAfterMerge) return true; if (this.plugin.settings.syncAfterMerge) return true;
if (this.plugin.replicator.syncStatus == "CONNECTED") return true; if (this.plugin.replicator.syncStatus == "CONNECTED") return true;
@@ -157,6 +158,7 @@ export class ObsidianLiveSyncSettingTab extends PluginSettingTab {
this.plugin.settings.liveSync = false; this.plugin.settings.liveSync = false;
this.plugin.settings.periodicReplication = false; this.plugin.settings.periodicReplication = false;
this.plugin.settings.syncOnSave = false; this.plugin.settings.syncOnSave = false;
this.plugin.settings.syncOnEditorSave = false;
this.plugin.settings.syncOnStart = false; this.plugin.settings.syncOnStart = false;
this.plugin.settings.syncOnFileOpen = false; this.plugin.settings.syncOnFileOpen = false;
this.plugin.settings.syncAfterMerge = false; this.plugin.settings.syncAfterMerge = false;
@@ -216,7 +218,7 @@ export class ObsidianLiveSyncSettingTab extends PluginSettingTab {
syncLive.forEach((e) => { syncLive.forEach((e) => {
e.setDisabled(false).setTooltip(""); e.setDisabled(false).setTooltip("");
}); });
} else if (this.plugin.settings.syncOnFileOpen || this.plugin.settings.syncOnSave || this.plugin.settings.syncOnStart || this.plugin.settings.periodicReplication || this.plugin.settings.syncAfterMerge) { } else if (this.plugin.settings.syncOnFileOpen || this.plugin.settings.syncOnSave || this.plugin.settings.syncOnEditorSave || this.plugin.settings.syncOnStart || this.plugin.settings.periodicReplication || this.plugin.settings.syncAfterMerge) {
syncNonLive.forEach((e) => { syncNonLive.forEach((e) => {
e.setDisabled(false).setTooltip(""); e.setDisabled(false).setTooltip("");
}); });
@@ -301,46 +303,44 @@ export class ObsidianLiveSyncSettingTab extends PluginSettingTab {
.setDisabled(false) .setDisabled(false)
.onClick(async () => { .onClick(async () => {
const checkConfig = async () => { const checkConfig = async () => {
Logger(`Checking database configuration`, LOG_LEVEL_INFO);
const emptyDiv = createDiv();
emptyDiv.innerHTML = "<span></span>";
checkResultDiv.replaceChildren(...[emptyDiv]);
const addResult = (msg: string, classes?: string[]) => {
const tmpDiv = createDiv();
tmpDiv.addClass("ob-btn-config-fix");
if (classes) {
tmpDiv.addClasses(classes);
}
tmpDiv.innerHTML = `${msg}`;
checkResultDiv.appendChild(tmpDiv);
};
try { try {
if (isCloudantURI(this.plugin.settings.couchDB_URI)) { if (isCloudantURI(this.plugin.settings.couchDB_URI)) {
Logger("This feature cannot be used with IBM Cloudant.", LOG_LEVEL_NOTICE); Logger("This feature cannot be used with IBM Cloudant.", LOG_LEVEL_NOTICE);
return; return;
} }
const r = await requestToCouchDB(this.plugin.settings.couchDB_URI, this.plugin.settings.couchDB_USER, this.plugin.settings.couchDB_PASSWORD, window.origin); const r = await requestToCouchDB(this.plugin.settings.couchDB_URI, this.plugin.settings.couchDB_USER, this.plugin.settings.couchDB_PASSWORD, window.origin);
Logger(JSON.stringify(r.json, null, 2));
const responseConfig = r.json; const responseConfig = r.json;
const emptyDiv = createDiv();
emptyDiv.innerHTML = "<span></span>";
checkResultDiv.replaceChildren(...[emptyDiv]);
const addResult = (msg: string, classes?: string[]) => {
const tmpDiv = createDiv();
tmpDiv.addClass("ob-btn-config-fix");
if (classes) {
tmpDiv.addClasses(classes);
}
tmpDiv.innerHTML = `${msg}`;
checkResultDiv.appendChild(tmpDiv);
};
const addConfigFixButton = (title: string, key: string, value: string) => { const addConfigFixButton = (title: string, key: string, value: string) => {
const tmpDiv = createDiv(); const tmpDiv = createDiv();
tmpDiv.addClass("ob-btn-config-fix"); tmpDiv.addClass("ob-btn-config-fix");
tmpDiv.innerHTML = `<label>${title}</label><button>Fix</button>`; tmpDiv.innerHTML = `<label>${title}</label><button>Fix</button>`;
const x = checkResultDiv.appendChild(tmpDiv); const x = checkResultDiv.appendChild(tmpDiv);
x.querySelector("button").addEventListener("click", async () => { x.querySelector("button").addEventListener("click", async () => {
console.dir({ key, value }); Logger(`CouchDB Configuration: ${title} -> Set ${key} to ${value}`)
const res = await requestToCouchDB(this.plugin.settings.couchDB_URI, this.plugin.settings.couchDB_USER, this.plugin.settings.couchDB_PASSWORD, undefined, key, value); const res = await requestToCouchDB(this.plugin.settings.couchDB_URI, this.plugin.settings.couchDB_USER, this.plugin.settings.couchDB_PASSWORD, undefined, key, value);
console.dir(res);
if (res.status == 200) { if (res.status == 200) {
Logger(`${title} successfully updated`, LOG_LEVEL_NOTICE); Logger(`CouchDB Configuration: ${title} successfully updated`, LOG_LEVEL_NOTICE);
checkResultDiv.removeChild(x); checkResultDiv.removeChild(x);
checkConfig(); checkConfig();
} else { } else {
Logger(`${title} failed`, LOG_LEVEL_NOTICE); Logger(`CouchDB Configuration: ${title} failed`, LOG_LEVEL_NOTICE);
Logger(res.text); Logger(res.text, LOG_LEVEL_VERBOSE);
} }
}); });
}; };
@@ -350,7 +350,6 @@ export class ObsidianLiveSyncSettingTab extends PluginSettingTab {
["ob-btn-config-info"] ["ob-btn-config-info"]
); );
addResult("Your configuration is dumped to Log", ["ob-btn-config-info"]);
addResult("--Config check--", ["ob-btn-config-head"]); addResult("--Config check--", ["ob-btn-config-head"]);
// Admin check // Admin check
@@ -451,9 +450,16 @@ export class ObsidianLiveSyncSettingTab extends PluginSettingTab {
} }
addResult("--Done--", ["ob-btn-config-head"]); addResult("--Done--", ["ob-btn-config-head"]);
addResult("If you have some trouble with Connection-check even though all Config-check has been passed, Please check your reverse proxy's configuration.", ["ob-btn-config-info"]); addResult("If you have some trouble with Connection-check even though all Config-check has been passed, Please check your reverse proxy's configuration.", ["ob-btn-config-info"]);
Logger(`Checking configuration done`, LOG_LEVEL_INFO);
} catch (ex) { } catch (ex) {
Logger(`Checking configuration failed`, LOG_LEVEL_NOTICE); if (ex?.status == 401) {
Logger(ex); addResult(`❗ Access forbidden.`);
addResult(`We could not continue the test.`);
Logger(`Checking configuration done`, LOG_LEVEL_INFO);
} else {
Logger(`Checking configuration failed`, LOG_LEVEL_NOTICE);
Logger(ex);
}
} }
}; };
await checkConfig(); await checkConfig();
@@ -891,6 +897,7 @@ export class ObsidianLiveSyncSettingTab extends PluginSettingTab {
liveSync: false, liveSync: false,
periodicReplication: false, periodicReplication: false,
syncOnSave: false, syncOnSave: false,
syncOnEditorSave: false,
syncOnStart: false, syncOnStart: false,
syncOnFileOpen: false, syncOnFileOpen: false,
syncAfterMerge: false, syncAfterMerge: false,
@@ -904,6 +911,7 @@ export class ObsidianLiveSyncSettingTab extends PluginSettingTab {
batchSave: true, batchSave: true,
periodicReplication: true, periodicReplication: true,
syncOnSave: false, syncOnSave: false,
syncOnEditorSave: false,
syncOnStart: true, syncOnStart: true,
syncOnFileOpen: true, syncOnFileOpen: true,
syncAfterMerge: true, syncAfterMerge: true,
@@ -1014,6 +1022,17 @@ export class ObsidianLiveSyncSettingTab extends PluginSettingTab {
applyDisplayEnabled(); applyDisplayEnabled();
}) })
) )
new Setting(containerSyncSettingEl)
.setName("Sync on Editor Save")
.setDesc("When you save file on the editor, sync automatically")
.setClass("wizardHidden")
.addToggle((toggle) =>
toggle.setValue(this.plugin.settings.syncOnEditorSave).onChange(async (value) => {
this.plugin.settings.syncOnEditorSave = value;
await this.plugin.saveSettings();
applyDisplayEnabled();
})
)
new Setting(containerSyncSettingEl) new Setting(containerSyncSettingEl)
.setName("Sync on File Open") .setName("Sync on File Open")
.setDesc("When you open file, sync automatically") .setDesc("When you open file, sync automatically")
@@ -1168,26 +1187,27 @@ export class ObsidianLiveSyncSettingTab extends PluginSettingTab {
}); });
} }
new Setting(containerSyncSettingEl) if (!this.plugin.settings.watchInternalFileChanges) {
.setName("Scan for hidden files before replication") new Setting(containerSyncSettingEl)
.setDesc("This configuration will be ignored if monitoring changes is enabled.") .setName("Scan for hidden files before replication")
.setClass("wizardHidden") .setClass("wizardHidden")
.addToggle((toggle) => .addToggle((toggle) =>
toggle.setValue(this.plugin.settings.syncInternalFilesBeforeReplication).onChange(async (value) => { toggle.setValue(this.plugin.settings.syncInternalFilesBeforeReplication).onChange(async (value) => {
this.plugin.settings.syncInternalFilesBeforeReplication = value; this.plugin.settings.syncInternalFilesBeforeReplication = value;
await this.plugin.saveSettings(); await this.plugin.saveSettings();
}) })
); );
}
new Setting(containerSyncSettingEl) new Setting(containerSyncSettingEl)
.setName("Scan hidden files periodically") .setName("Scan hidden files periodically")
.setDesc("Seconds, 0 to disable. This configuration will be ignored if monitoring changes is enabled.") .setDesc("Seconds, 0 to disable")
.setClass("wizardHidden") .setClass("wizardHidden")
.addText((text) => { .addText((text) => {
text.setPlaceholder("") text.setPlaceholder("")
.setValue(this.plugin.settings.syncInternalFilesInterval + "") .setValue(this.plugin.settings.syncInternalFilesInterval + "")
.onChange(async (value) => { .onChange(async (value) => {
let v = Number(value); let v = Number(value);
if (isNaN(v) || v < 10) { if (v !== 0 && (isNaN(v) || v < 10)) {
v = 10; v = 10;
} }
this.plugin.settings.syncInternalFilesInterval = v; this.plugin.settings.syncInternalFilesInterval = v;
@@ -1199,7 +1219,7 @@ export class ObsidianLiveSyncSettingTab extends PluginSettingTab {
const defaultSkipPattern = "\\/node_modules\\/, \\/\\.git\\/, \\/obsidian-livesync\\/"; const defaultSkipPattern = "\\/node_modules\\/, \\/\\.git\\/, \\/obsidian-livesync\\/";
const defaultSkipPatternXPlat = defaultSkipPattern + ",\\/workspace$ ,\\/workspace.json$"; const defaultSkipPatternXPlat = defaultSkipPattern + ",\\/workspace$ ,\\/workspace.json$";
new Setting(containerSyncSettingEl) new Setting(containerSyncSettingEl)
.setName("Skip patterns") .setName("Folders and files to ignore")
.setDesc( .setDesc(
"Regular expression, If you use hidden file sync between desktop and mobile, adding `workspace$` is recommended." "Regular expression, If you use hidden file sync between desktop and mobile, adding `workspace$` is recommended."
) )
@@ -1250,7 +1270,7 @@ export class ObsidianLiveSyncSettingTab extends PluginSettingTab {
new Setting(containerSyncSettingEl) new Setting(containerSyncSettingEl)
.setName("Enhance chunk size") .setName("Enhance chunk size")
.setDesc("Enhance chunk size for binary files (0.1MBytes). This cannot be increased when using IBM Cloudant.") .setDesc("Enhance chunk size for binary files (Ratio). This cannot be increased when using IBM Cloudant.")
.setClass("wizardHidden") .setClass("wizardHidden")
.addText((text) => { .addText((text) => {
text.setPlaceholder("") text.setPlaceholder("")
@@ -1258,7 +1278,7 @@ export class ObsidianLiveSyncSettingTab extends PluginSettingTab {
.onChange(async (value) => { .onChange(async (value) => {
let v = Number(value); let v = Number(value);
if (isNaN(v) || v < 1) { if (isNaN(v) || v < 1) {
v = 1; v = 0;
} }
this.plugin.settings.customChunkSize = v; this.plugin.settings.customChunkSize = v;
await this.plugin.saveSettings(); await this.plugin.saveSettings();
@@ -1493,10 +1513,12 @@ export class ObsidianLiveSyncSettingTab extends PluginSettingTab {
pluginConfig.passphrase = REDACTED; pluginConfig.passphrase = REDACTED;
pluginConfig.encryptedPassphrase = REDACTED; pluginConfig.encryptedPassphrase = REDACTED;
pluginConfig.encryptedCouchDBConnection = REDACTED; pluginConfig.encryptedCouchDBConnection = REDACTED;
pluginConfig.pluginSyncExtendedSetting = {};
const msgConfig = `----remote config---- const msgConfig = `----remote config----
${stringifyYaml(responseConfig)} ${stringifyYaml(responseConfig)}
---- Plug-in config --- ---- Plug-in config ---
version:${manifestVersion}
${stringifyYaml(pluginConfig)}`; ${stringifyYaml(pluginConfig)}`;
console.log(msgConfig); console.log(msgConfig);
await navigator.clipboard.writeText(msgConfig); await navigator.clipboard.writeText(msgConfig);
@@ -1635,6 +1657,22 @@ ${stringifyYaml(pluginConfig)}`;
} }
Logger(`Converting finished`, LOG_LEVEL_NOTICE); Logger(`Converting finished`, LOG_LEVEL_NOTICE);
})); }));
new Setting(containerHatchEl)
.setName("Delete all customization sync data")
.addButton((button) =>
button
.setButtonText("Delete")
.setDisabled(false)
.setWarning()
.onClick(async () => {
Logger(`Deleting customization sync data`, LOG_LEVEL_NOTICE);
const entriesToDelete = (await this.plugin.localDatabase.allDocsRaw({ startkey: "ix:", endkey: "ix:\u{10ffff}", include_docs: true }));
const newData = entriesToDelete.rows.map(e => ({ ...e.doc, _deleted: true }));
const r = await this.plugin.localDatabase.bulkDocsRaw(newData as any[]);
// Do not care about the result.
Logger(`${r.length} items have been removed, to confirm how many items are left, please perform it again.`, LOG_LEVEL_NOTICE);
}))
new Setting(containerHatchEl) new Setting(containerHatchEl)
.setName("Suspend file watching") .setName("Suspend file watching")
.setDesc("Stop watching for file change.") .setDesc("Stop watching for file change.")
@@ -1777,8 +1815,8 @@ ${stringifyYaml(pluginConfig)}`;
dropdown dropdown
.addOptions({ "": "Old Algorithm", "xxhash32": "xxhash32 (Fast)", "xxhash64": "xxhash64 (Fastest)" } as Record<HashAlgorithm, string>) .addOptions({ "": "Old Algorithm", "xxhash32": "xxhash32 (Fast)", "xxhash64": "xxhash64 (Fastest)" } as Record<HashAlgorithm, string>)
.setValue(this.plugin.settings.hashAlg) .setValue(this.plugin.settings.hashAlg)
.onChange(async (value: HashAlgorithm) => { .onChange(async (value) => {
this.plugin.settings.hashAlg = value; this.plugin.settings.hashAlg = value as HashAlgorithm;
await this.plugin.saveSettings(); await this.plugin.saveSettings();
}) })
) )
@@ -1794,6 +1832,15 @@ ${stringifyYaml(pluginConfig)}`;
await this.plugin.saveSettings(); await this.plugin.saveSettings();
}) })
); );
new Setting(containerHatchEl)
.setName("Use binary and encryption version 1")
.setDesc("Use the previous data format for other products which shares the remote database.")
.addToggle((toggle) =>
toggle.setValue(this.plugin.settings.useV1).onChange(async (value) => {
this.plugin.settings.useV1 = value;
await this.plugin.saveSettings();
})
);
addScreenElement("50", containerHatchEl); addScreenElement("50", containerHatchEl);
@@ -1844,16 +1891,18 @@ ${stringifyYaml(pluginConfig)}`;
}) })
); );
new Setting(containerPluginSettings) if (!this.plugin.settings.watchInternalFileChanges) {
.setName("Scan customization periodically") new Setting(containerPluginSettings)
.setDesc("Scan customization every 1 minute. This configuration will be ignored if monitoring changes of hidden files has been enabled.") .setName("Scan customization periodically")
.addToggle((toggle) => .setDesc("Scan customization every 1 minute.")
toggle.setValue(this.plugin.settings.autoSweepPluginsPeriodic).onChange(async (value) => { .addToggle((toggle) =>
this.plugin.settings.autoSweepPluginsPeriodic = value; toggle.setValue(this.plugin.settings.autoSweepPluginsPeriodic).onChange(async (value) => {
updateDisabledOfDeviceAndVaultName(); this.plugin.settings.autoSweepPluginsPeriodic = value;
await this.plugin.saveSettings(); updateDisabledOfDeviceAndVaultName();
}) await this.plugin.saveSettings();
); })
);
}
new Setting(containerPluginSettings) new Setting(containerPluginSettings)
.setName("Notify customized") .setName("Notify customized")

4
src/PluginCombo.svelte
View File

@@ -108,7 +108,7 @@
} }
} }
}) })
.reduce((p, c) => p | c, 0); .reduce((p, c) => p | (c as number), 0 as number);
if (matchingStatus == 0b0000100) { if (matchingStatus == 0b0000100) {
equivalency = "⚖️ Same"; equivalency = "⚖️ Same";
canApply = false; canApply = false;
@@ -274,7 +274,7 @@
{/if} {/if}
{:else} {:else}
<span class="spacer" /> <span class="spacer" />
<span class="message even">All devices are even</span> <span class="message even">All the same or non-existent</span>
<button disabled /> <button disabled />
<button disabled /> <button disabled />
{/if} {/if}

230
src/PluginPane.svelte
View File

@@ -3,9 +3,13 @@
import ObsidianLiveSyncPlugin from "./main"; import ObsidianLiveSyncPlugin from "./main";
import { type PluginDataExDisplay, pluginIsEnumerating, pluginList } from "./CmdConfigSync"; import { type PluginDataExDisplay, pluginIsEnumerating, pluginList } from "./CmdConfigSync";
import PluginCombo from "./PluginCombo.svelte"; import PluginCombo from "./PluginCombo.svelte";
import { Menu } from "obsidian";
import { unique } from "./lib/src/utils";
import { MODE_SELECTIVE, MODE_AUTOMATIC, MODE_PAUSED, type SYNC_MODE, type PluginSyncSettingEntry } from "./lib/src/types";
import { normalizePath } from "./deps";
export let plugin: ObsidianLiveSyncPlugin; export let plugin: ObsidianLiveSyncPlugin;
$: hideNotApplicable = true; $: hideNotApplicable = false;
$: thisTerm = plugin.deviceAndVaultName; $: thisTerm = plugin.deviceAndVaultName;
const addOn = plugin.addOnConfigSync; const addOn = plugin.addOnConfigSync;
@@ -13,7 +17,7 @@
let list: PluginDataExDisplay[] = []; let list: PluginDataExDisplay[] = [];
let selectNewestPulse = 0; let selectNewestPulse = 0;
let hideEven = true; let hideEven = false;
let loading = false; let loading = false;
let applyAllPluse = 0; let applyAllPluse = 0;
let isMaintenanceMode = false; let isMaintenanceMode = false;
@@ -80,6 +84,54 @@
async function deleteData(data: PluginDataExDisplay): Promise<boolean> { async function deleteData(data: PluginDataExDisplay): Promise<boolean> {
return await addOn.deleteData(data); return await addOn.deleteData(data);
} }
function askMode(evt: MouseEvent, title: string, key: string) {
const menu = new Menu();
menu.addItem((item) => item.setTitle(title).setIsLabel(true));
menu.addSeparator();
const prevMode = automaticList.get(key) ?? MODE_SELECTIVE;
for (const mode of [MODE_SELECTIVE, MODE_AUTOMATIC, MODE_PAUSED]) {
menu.addItem((item) => {
item.setTitle(`${getIcon(mode as SYNC_MODE)}:${TITLES[mode]}`)
.onClick((e) => {
if (mode === MODE_AUTOMATIC) {
askOverwriteModeForAutomatic(evt, key);
} else {
setMode(key, mode as SYNC_MODE);
}
})
.setChecked(prevMode == mode)
.setDisabled(prevMode == mode);
});
}
menu.showAtMouseEvent(evt);
}
function applyAutomaticSync(key: string, direction: "pushForce" | "pullForce" | "safe") {
setMode(key, MODE_AUTOMATIC);
const configDir = normalizePath(plugin.app.vault.configDir);
const files = (plugin.settings.pluginSyncExtendedSetting[key]?.files ?? []).map((e) => `${configDir}/${e}`);
plugin.addOnHiddenFileSync.syncInternalFilesAndDatabase(direction, true, false, files);
}
function askOverwriteModeForAutomatic(evt: MouseEvent, key: string) {
const menu = new Menu();
menu.addItem((item) => item.setTitle("Initial Action").setIsLabel(true));
menu.addSeparator();
menu.addItem((item) => {
item.setTitle(`↑: Overwrite Remote`).onClick((e) => {
applyAutomaticSync(key, "pushForce");
});
})
.addItem((item) => {
item.setTitle(`↓: Overwrite Local`).onClick((e) => {
applyAutomaticSync(key, "pullForce");
});
})
.addItem((item) => {
item.setTitle(`⇅: Use newer`).onClick((e) => {
applyAutomaticSync(key, "safe");
});
});
menu.showAtMouseEvent(evt);
}
$: options = { $: options = {
thisTerm, thisTerm,
@@ -92,11 +144,84 @@
plugin, plugin,
isMaintenanceMode, isMaintenanceMode,
}; };
const ICON_EMOJI_PAUSED = `⛔`;
const ICON_EMOJI_AUTOMATIC = `✨`;
const ICON_EMOJI_SELECTIVE = `🔀`;
const ICONS: { [key: number]: string } = {
[MODE_SELECTIVE]: ICON_EMOJI_SELECTIVE,
[MODE_PAUSED]: ICON_EMOJI_PAUSED,
[MODE_AUTOMATIC]: ICON_EMOJI_AUTOMATIC,
};
const TITLES: { [key: number]: string } = {
[MODE_SELECTIVE]: "Selective",
[MODE_PAUSED]: "Ignore",
[MODE_AUTOMATIC]: "Automatic",
};
const PREFIX_PLUGIN_ALL = "PLUGIN_ALL";
const PREFIX_PLUGIN_DATA = "PLUGIN_DATA";
const PREFIX_PLUGIN_MAIN = "PLUGIN_MAIN";
function setMode(key: string, mode: SYNC_MODE) {
if (key.startsWith(PREFIX_PLUGIN_ALL + "/")) {
setMode(PREFIX_PLUGIN_DATA + key.substring(PREFIX_PLUGIN_ALL.length), mode);
setMode(PREFIX_PLUGIN_MAIN + key.substring(PREFIX_PLUGIN_ALL.length), mode);
}
const files = unique(
list
.filter((e) => `${e.category}/${e.name}` == key)
.map((e) => e.files)
.flat()
.map((e) => e.filename)
);
automaticList.set(key, mode);
automaticListDisp = automaticList;
if (!(key in plugin.settings.pluginSyncExtendedSetting)) {
plugin.settings.pluginSyncExtendedSetting[key] = {
key,
mode,
files: [],
};
}
plugin.settings.pluginSyncExtendedSetting[key].files = files;
plugin.settings.pluginSyncExtendedSetting[key].mode = mode;
plugin.saveSettingData();
}
function getIcon(mode: SYNC_MODE) {
if (mode in ICONS) {
return ICONS[mode];
} else {
("");
}
}
let automaticList = new Map<string, SYNC_MODE>();
let automaticListDisp = new Map<string, SYNC_MODE>();
// apply current configuration to the dialogue
for (const { key, mode } of Object.values(plugin.settings.pluginSyncExtendedSetting)) {
automaticList.set(key, mode);
}
automaticListDisp = automaticList;
let displayKeys: Record<string, string[]> = {};
$: {
const extraKeys = Object.keys(plugin.settings.pluginSyncExtendedSetting);
displayKeys = [
...list,
...extraKeys
.map((e) => `${e}///`.split("/"))
.filter((e) => e[0] && e[1])
.map((e) => ({ category: e[0], name: e[1], displayName: e[1] })),
]
.sort((a, b) => (a.displayName ?? a.name).localeCompare(b.displayName ?? b.name))
.reduce((p, c) => ({ ...p, [c.category]: unique(c.category in p ? [...p[c.category], c.displayName ?? c.name] : [c.displayName ?? c.name]) }), {} as Record<string, string[]>);
}
</script> </script>
<div> <div>
<div> <div>
<h1>Customization sync</h1>
<div class="buttons"> <div class="buttons">
<button on:click={() => scanAgain()}>Scan changes</button> <button on:click={() => scanAgain()}>Scan changes</button>
<button on:click={() => replicate()}>Sync once</button> <button on:click={() => replicate()}>Sync once</button>
@@ -119,15 +244,24 @@
{#if list.length == 0} {#if list.length == 0}
<div class="center">No Items.</div> <div class="center">No Items.</div>
{:else} {:else}
{#each Object.entries(displays) as [key, label]} {#each Object.entries(displays).filter(([key, _]) => key in displayKeys) as [key, label]}
<div> <div>
<h3>{label}</h3> <h3>{label}</h3>
{#each groupBy(filterList(list, [key]), "name") as [name, listX]} {#each displayKeys[key] as name}
{@const bindKey = `${key}/${name}`}
{@const mode = automaticListDisp.get(bindKey) ?? MODE_SELECTIVE}
<div class="labelrow {hideEven ? 'hideeven' : ''}"> <div class="labelrow {hideEven ? 'hideeven' : ''}">
<div class="title"> <div class="title">
{name} <button class="status" on:click={(evt) => askMode(evt, `${key}/${name}`, bindKey)}>
{getIcon(mode)}
</button>
<span class="name">{name}</span>
</div> </div>
<PluginCombo {...options} list={listX} hidden={false} /> {#if mode == MODE_SELECTIVE}
<PluginCombo {...options} list={list.filter((e) => e.category == key && e.name == name)} hidden={false} />
{:else}
<div class="statusnote">{TITLES[mode]}</div>
{/if}
</div> </div>
{/each} {/each}
</div> </div>
@@ -135,20 +269,55 @@
<div> <div>
<h3>Plugins</h3> <h3>Plugins</h3>
{#each groupBy(filterList(list, ["PLUGIN_MAIN", "PLUGIN_DATA", "PLUGIN_ETC"]), "name") as [name, listX]} {#each groupBy(filterList(list, ["PLUGIN_MAIN", "PLUGIN_DATA", "PLUGIN_ETC"]), "name") as [name, listX]}
{@const bindKeyAll = `${PREFIX_PLUGIN_ALL}/${name}`}
{@const modeAll = automaticListDisp.get(bindKeyAll) ?? MODE_SELECTIVE}
{@const bindKeyMain = `${PREFIX_PLUGIN_MAIN}/${name}`}
{@const modeMain = automaticListDisp.get(bindKeyMain) ?? MODE_SELECTIVE}
{@const bindKeyData = `${PREFIX_PLUGIN_DATA}/${name}`}
{@const modeData = automaticListDisp.get(bindKeyData) ?? MODE_SELECTIVE}
<div class="labelrow {hideEven ? 'hideeven' : ''}"> <div class="labelrow {hideEven ? 'hideeven' : ''}">
<div class="title"> <div class="title">
{name} <button class="status" on:click={(evt) => askMode(evt, `${PREFIX_PLUGIN_ALL}/${name}`, bindKeyAll)}>
{getIcon(modeAll)}
</button>
<span class="name">{name}</span>
</div> </div>
<PluginCombo {...options} list={listX} hidden={true} /> {#if modeAll == MODE_SELECTIVE}
</div> <PluginCombo {...options} list={listX} hidden={true} />
<div class="filerow {hideEven ? 'hideeven' : ''}"> {/if}
<div class="filetitle">Main</div>
<PluginCombo {...options} list={filterList(listX, ["PLUGIN_MAIN"])} hidden={false} />
</div>
<div class="filerow {hideEven ? 'hideeven' : ''}">
<div class="filetitle">Data</div>
<PluginCombo {...options} list={filterList(listX, ["PLUGIN_DATA"])} hidden={false} />
</div> </div>
{#if modeAll == MODE_SELECTIVE}
<div class="filerow {hideEven ? 'hideeven' : ''}">
<div class="filetitle">
<button class="status" on:click={(evt) => askMode(evt, `${PREFIX_PLUGIN_MAIN}/${name}/MAIN`, bindKeyMain)}>
{getIcon(modeMain)}
</button>
<span class="name">MAIN</span>
</div>
{#if modeMain == MODE_SELECTIVE}
<PluginCombo {...options} list={filterList(listX, ["PLUGIN_MAIN"])} hidden={false} />
{:else}
<div class="statusnote">{TITLES[modeMain]}</div>
{/if}
</div>
<div class="filerow {hideEven ? 'hideeven' : ''}">
<div class="filetitle">
<button class="status" on:click={(evt) => askMode(evt, `${PREFIX_PLUGIN_DATA}/${name}`, bindKeyData)}>
{getIcon(modeData)}
</button>
<span class="name">DATA</span>
</div>
{#if modeData == MODE_SELECTIVE}
<PluginCombo {...options} list={filterList(listX, ["PLUGIN_DATA"])} hidden={false} />
{:else}
<div class="statusnote">{TITLES[modeData]}</div>
{/if}
</div>
{:else}
<div class="noterow">
<div class="statusnote">{TITLES[modeAll]}</div>
</div>
{/if}
{/each} {/each}
</div> </div>
{/if} {/if}
@@ -162,6 +331,15 @@
</div> </div>
<style> <style>
span.spacer {
min-width: 1px;
flex-grow: 1;
}
h3 {
position: sticky;
top: 0;
background-color: var(--modal-background);
}
.labelrow { .labelrow {
margin-left: 0.4em; margin-left: 0.4em;
display: flex; display: flex;
@@ -183,6 +361,24 @@
.labelrow.hideeven:has(.even) { .labelrow.hideeven:has(.even) {
display: none; display: none;
} }
.noterow {
min-height: 2em;
display: flex;
}
button.status {
flex-grow: 0;
margin: 2px 4px;
min-width: 3em;
max-width: 4em;
}
.statusnote {
display: flex;
justify-content: flex-end;
padding-right: var(--size-4-12);
align-items: center;
min-width: 10em;
flex-grow: 1;
}
.title { .title {
color: var(--text-normal); color: var(--text-normal);

2
src/StorageEventManager.ts
View File

@@ -18,7 +18,7 @@ type LiveSyncForStorageEventManager = Plugin &
ignoreFiles: string[], ignoreFiles: string[],
} & { } & {
isTargetFile: (file: string | TAbstractFile) => Promise<boolean>, isTargetFile: (file: string | TAbstractFile) => Promise<boolean>,
procFileEvent: (applyBatch?: boolean) => Promise<boolean>, procFileEvent: (applyBatch?: boolean) => Promise<any>,
}; };

2
src/deps.ts
View File

@@ -4,7 +4,7 @@ export {
addIcon, App, debounce, Editor, FuzzySuggestModal, MarkdownRenderer, MarkdownView, Modal, Notice, Platform, Plugin, PluginSettingTab, requestUrl, sanitizeHTMLToDom, Setting, stringifyYaml, TAbstractFile, TextAreaComponent, TFile, TFolder, addIcon, App, debounce, Editor, FuzzySuggestModal, MarkdownRenderer, MarkdownView, Modal, Notice, Platform, Plugin, PluginSettingTab, requestUrl, sanitizeHTMLToDom, Setting, stringifyYaml, TAbstractFile, TextAreaComponent, TFile, TFolder,
parseYaml, ItemView, WorkspaceLeaf parseYaml, ItemView, WorkspaceLeaf
} from "obsidian"; } from "obsidian";
export type { DataWriteOptions, PluginManifest, RequestUrlParam, RequestUrlResponse } from "obsidian"; export type { DataWriteOptions, PluginManifest, RequestUrlParam, RequestUrlResponse, MarkdownFileInfo } from "obsidian";
import { import {
normalizePath as normalizePath_ normalizePath as normalizePath_
} from "obsidian"; } from "obsidian";

10
src/dialogs.ts
View File

@@ -20,6 +20,7 @@ export class PluginDialogModal extends Modal {
onOpen() { onOpen() {
const { contentEl } = this; const { contentEl } = this;
this.titleEl.setText("Customization Sync (Beta2)")
if (this.component == null) { if (this.component == null) {
this.component = new PluginPane({ this.component = new PluginPane({
target: contentEl, target: contentEl,
@@ -38,7 +39,7 @@ export class PluginDialogModal extends Modal {
export class InputStringDialog extends Modal { export class InputStringDialog extends Modal {
result: string | false = false; result: string | false = false;
onSubmit: (result: string | boolean) => void; onSubmit: (result: string | false) => void;
title: string; title: string;
key: string; key: string;
placeholder: string; placeholder: string;
@@ -56,8 +57,7 @@ export class InputStringDialog extends Modal {
onOpen() { onOpen() {
const { contentEl } = this; const { contentEl } = this;
this.titleEl.setText(this.title);
contentEl.createEl("h1", { text: this.title });
// For enter to submit // For enter to submit
const formEl = contentEl.createEl("form"); const formEl = contentEl.createEl("form");
new Setting(formEl).setName(this.key).setClass(this.isPassword ? "password-input" : "normal-input").addText((text) => new Setting(formEl).setName(this.key).setClass(this.isPassword ? "password-input" : "normal-input").addText((text) =>
@@ -144,7 +144,7 @@ export class MessageBox extends Modal {
timer: ReturnType<typeof setInterval> = undefined; timer: ReturnType<typeof setInterval> = undefined;
defaultButtonComponent: ButtonComponent | undefined; defaultButtonComponent: ButtonComponent | undefined;
onSubmit: (result: string | boolean) => void; onSubmit: (result: string | false) => void;
constructor(plugin: Plugin, title: string, contentMd: string, buttons: string[], defaultAction: (typeof buttons)[number], timeout: number, onSubmit: (result: (typeof buttons)[number] | false) => void) { constructor(plugin: Plugin, title: string, contentMd: string, buttons: string[], defaultAction: (typeof buttons)[number], timeout: number, onSubmit: (result: (typeof buttons)[number] | false) => void) {
super(plugin.app); super(plugin.app);
@@ -175,13 +175,13 @@ export class MessageBox extends Modal {
onOpen() { onOpen() {
const { contentEl } = this; const { contentEl } = this;
this.titleEl.setText(this.title);
contentEl.addEventListener("click", () => { contentEl.addEventListener("click", () => {
if (this.timer) { if (this.timer) {
clearInterval(this.timer); clearInterval(this.timer);
this.timer = undefined; this.timer = undefined;
} }
}) })
contentEl.createEl("h1", { text: this.title });
const div = contentEl.createDiv(); const div = contentEl.createDiv();
MarkdownRenderer.render(this.plugin.app, this.contentMd, div, "/", this.plugin); MarkdownRenderer.render(this.plugin.app, this.contentMd, div, "/", this.plugin);
const buttonSetting = new Setting(contentEl); const buttonSetting = new Setting(contentEl);

2
src/lib

Submodule src/lib updated: 6efd115e0e...609c7aecf3

375
src/main.ts
View File

@@ -1,8 +1,8 @@
const isDebug = false; const isDebug = false;
import { type Diff, DIFF_DELETE, DIFF_EQUAL, DIFF_INSERT, diff_match_patch } from "./deps"; import { type Diff, DIFF_DELETE, DIFF_EQUAL, DIFF_INSERT, diff_match_patch } from "./deps";
import { debounce, Notice, Plugin, TFile, addIcon, TFolder, normalizePath, TAbstractFile, Editor, MarkdownView, type RequestUrlParam, type RequestUrlResponse, requestUrl } from "./deps"; import { debounce, Notice, Plugin, TFile, addIcon, TFolder, normalizePath, TAbstractFile, Editor, MarkdownView, type RequestUrlParam, type RequestUrlResponse, requestUrl, type MarkdownFileInfo } from "./deps";
import { type EntryDoc, type LoadedEntry, type ObsidianLiveSyncSettings, type diff_check_result, type diff_result_leaf, type EntryBody, LOG_LEVEL, VER, DEFAULT_SETTINGS, type diff_result, FLAGMD_REDFLAG, SYNCINFO_ID, SALT_OF_PASSPHRASE, type ConfigPassphraseStore, type CouchDBConnection, FLAGMD_REDFLAG2, FLAGMD_REDFLAG3, PREFIXMD_LOGFILE, type DatabaseConnectingStatus, type EntryHasPath, type DocumentID, type FilePathWithPrefix, type FilePath, type AnyEntry, LOG_LEVEL_DEBUG, LOG_LEVEL_INFO, LOG_LEVEL_NOTICE, LOG_LEVEL_URGENT, LOG_LEVEL_VERBOSE } from "./lib/src/types"; import { type EntryDoc, type LoadedEntry, type ObsidianLiveSyncSettings, type diff_check_result, type diff_result_leaf, type EntryBody, LOG_LEVEL, VER, DEFAULT_SETTINGS, type diff_result, FLAGMD_REDFLAG, SYNCINFO_ID, SALT_OF_PASSPHRASE, type ConfigPassphraseStore, type CouchDBConnection, FLAGMD_REDFLAG2, FLAGMD_REDFLAG3, PREFIXMD_LOGFILE, type DatabaseConnectingStatus, type EntryHasPath, type DocumentID, type FilePathWithPrefix, type FilePath, type AnyEntry, LOG_LEVEL_DEBUG, LOG_LEVEL_INFO, LOG_LEVEL_NOTICE, LOG_LEVEL_URGENT, LOG_LEVEL_VERBOSE, } from "./lib/src/types";
import { type InternalFileInfo, type queueItem, type CacheData, type FileEventItem, FileWatchEventQueueMax } from "./types"; import { type InternalFileInfo, type queueItem, type CacheData, type FileEventItem, FileWatchEventQueueMax } from "./types";
import { arrayToChunkedArray, getDocData, isDocContentSame } from "./lib/src/utils"; import { arrayToChunkedArray, getDocData, isDocContentSame } from "./lib/src/utils";
import { Logger, setGlobalLogFunction } from "./lib/src/logger"; import { Logger, setGlobalLogFunction } from "./lib/src/logger";
@@ -10,15 +10,15 @@ import { PouchDB } from "./lib/src/pouchdb-browser.js";
import { ConflictResolveModal } from "./ConflictResolveModal"; import { ConflictResolveModal } from "./ConflictResolveModal";
import { ObsidianLiveSyncSettingTab } from "./ObsidianLiveSyncSettingTab"; import { ObsidianLiveSyncSettingTab } from "./ObsidianLiveSyncSettingTab";
import { DocumentHistoryModal } from "./DocumentHistoryModal"; import { DocumentHistoryModal } from "./DocumentHistoryModal";
import { applyPatch, cancelAllPeriodicTask, cancelAllTasks, cancelTask, generatePatchObj, id2path, isObjectMargeApplicable, isSensibleMargeApplicable, flattenObject, path2id, scheduleTask, tryParseJSON, createFile, modifyFile, isValidPath, getAbstractFileByPath, touch, recentlyTouched, isInternalMetadata, isPluginMetadata, stripInternalMetadataPrefix, isChunk, askSelectString, askYesNo, askString, PeriodicProcessor, clearTouched, getPath, getPathWithoutPrefix, getPathFromTFile, performRebuildDB } from "./utils"; import { applyPatch, cancelAllPeriodicTask, cancelAllTasks, cancelTask, generatePatchObj, id2path, isObjectMargeApplicable, isSensibleMargeApplicable, flattenObject, path2id, scheduleTask, tryParseJSON, createFile, modifyFile, isValidPath, getAbstractFileByPath, touch, recentlyTouched, isInternalMetadata, isPluginMetadata, stripInternalMetadataPrefix, isChunk, askSelectString, askYesNo, askString, PeriodicProcessor, clearTouched, getPath, getPathWithoutPrefix, getPathFromTFile, performRebuildDB, memoIfNotExist, memoObject, retrieveMemoObject, disposeMemoObject, isCustomisationSyncMetadata } from "./utils";
import { encrypt, tryDecrypt } from "./lib/src/e2ee_v2"; import { encrypt, tryDecrypt } from "./lib/src/e2ee_v2";
import { balanceChunkPurgedDBs, enableEncryption, isCloudantURI, isErrorOfMissingDoc, isValidRemoteCouchDBURI, purgeUnreferencedChunks } from "./lib/src/utils_couchdb"; import { balanceChunkPurgedDBs, enableEncryption, isCloudantURI, isErrorOfMissingDoc, isValidRemoteCouchDBURI, purgeUnreferencedChunks } from "./lib/src/utils_couchdb";
import { getGlobalStore, ObservableStore, observeStores } from "./lib/src/store"; import { getGlobalStore, ObservableStore, observeStores } from "./lib/src/store";
import { lockStore, logMessageStore, logStore, type LogEntry } from "./lib/src/stores"; import { lockStore, logMessageStore, logStore, type LogEntry } from "./lib/src/stores";
import { setNoticeClass } from "./lib/src/wrapper"; import { setNoticeClass } from "./lib/src/wrapper";
import { base64ToString, versionNumberString2Number, base64ToArrayBuffer, arrayBufferToBase64 } from "./lib/src/strbin"; import { versionNumberString2Number, writeString, decodeBinary, encodeBinary, readString } from "./lib/src/strbin";
import { addPrefix, isAcceptedAll, isPlainText, shouldBeIgnored, stripAllPrefixes } from "./lib/src/path"; import { addPrefix, isAcceptedAll, isPlainText, shouldBeIgnored, stripAllPrefixes } from "./lib/src/path";
import { isLockAcquired, runWithLock } from "./lib/src/lock"; import { isLockAcquired, serialized, skipIfDuplicated } from "./lib/src/lock";
import { Semaphore } from "./lib/src/semaphore"; import { Semaphore } from "./lib/src/semaphore";
import { StorageEventManager, StorageEventManagerObsidian } from "./StorageEventManager"; import { StorageEventManager, StorageEventManagerObsidian } from "./StorageEventManager";
import { LiveSyncLocalDB, type LiveSyncLocalDBEnv } from "./lib/src/LiveSyncLocalDB"; import { LiveSyncLocalDB, type LiveSyncLocalDBEnv } from "./lib/src/LiveSyncLocalDB";
@@ -43,16 +43,29 @@ setGlobalLogFunction((message: any, level?: LOG_LEVEL, key?: string) => {
}); });
logStore.intercept(e => e.slice(Math.min(e.length - 200, 0))); logStore.intercept(e => e.slice(Math.min(e.length - 200, 0)));
async function fetchByAPI(request: RequestUrlParam): Promise<RequestUrlResponse> {
const ret = await requestUrl(request);
if (ret.status - (ret.status % 100) !== 200) {
const er: Error & { status?: number } = new Error(`Request Error:${ret.status}`);
if (ret.json) {
er.message = ret.json.reason;
er.name = `${ret.json.error ?? ""}:${ret.json.message ?? ""}`;
}
er.status = ret.status;
throw er;
}
return ret;
}
export default class ObsidianLiveSyncPlugin extends Plugin export default class ObsidianLiveSyncPlugin extends Plugin
implements LiveSyncLocalDBEnv, LiveSyncReplicatorEnv { implements LiveSyncLocalDBEnv, LiveSyncReplicatorEnv {
settings: ObsidianLiveSyncSettings; settings!: ObsidianLiveSyncSettings;
localDatabase: LiveSyncLocalDB; localDatabase!: LiveSyncLocalDB;
replicator: LiveSyncDBReplicator; replicator!: LiveSyncDBReplicator;
statusBar: HTMLElement; statusBar?: HTMLElement;
suspended: boolean; suspended = false;
deviceAndVaultName: string; deviceAndVaultName = "";
isMobile = false; isMobile = false;
isReady = false; isReady = false;
packageVersion = ""; packageVersion = "";
@@ -67,25 +80,15 @@ export default class ObsidianLiveSyncPlugin extends Plugin
periodicSyncProcessor = new PeriodicProcessor(this, async () => await this.replicate()); periodicSyncProcessor = new PeriodicProcessor(this, async () => await this.replicate());
// implementing interfaces // implementing interfaces
kvDB: KeyValueDatabase; kvDB!: KeyValueDatabase;
last_successful_post = false; last_successful_post = false;
getLastPostFailedBySize() { getLastPostFailedBySize() {
return !this.last_successful_post; return !this.last_successful_post;
} }
async fetchByAPI(request: RequestUrlParam): Promise<RequestUrlResponse> {
const ret = await requestUrl(request); _unloaded = false;
if (ret.status - (ret.status % 100) !== 200) {
const er: Error & { status?: number } = new Error(`Request Error:${ret.status}`);
if (ret.json) {
er.message = ret.json.reason;
er.name = `${ret.json.error ?? ""}:${ret.json.message ?? ""}`;
}
er.status = ret.status;
throw er;
}
return ret;
}
getDatabase(): PouchDB.Database<EntryDoc> { getDatabase(): PouchDB.Database<EntryDoc> {
return this.localDatabase.localDatabase; return this.localDatabase.localDatabase;
} }
@@ -103,7 +106,7 @@ export default class ObsidianLiveSyncPlugin extends Plugin
if (uri.indexOf(" ") !== -1) return "Remote URI and database name could not contain spaces."; if (uri.indexOf(" ") !== -1) return "Remote URI and database name could not contain spaces.";
let authHeader = ""; let authHeader = "";
if (auth.username && auth.password) { if (auth.username && auth.password) {
const utf8str = String.fromCharCode.apply(null, new TextEncoder().encode(`${auth.username}:${auth.password}`)); const utf8str = String.fromCharCode.apply(null, [...writeString(`${auth.username}:${auth.password}`)]);
const encoded = window.btoa(utf8str); const encoded = window.btoa(utf8str);
authHeader = "Basic " + encoded; authHeader = "Basic " + encoded;
} else { } else {
@@ -115,11 +118,11 @@ export default class ObsidianLiveSyncPlugin extends Plugin
adapter: "http", adapter: "http",
auth, auth,
skip_setup: !performSetup, skip_setup: !performSetup,
fetch: async (url: string | Request, opts: RequestInit) => { fetch: async (url: string | Request, opts?: RequestInit) => {
let size = ""; let size = "";
const localURL = url.toString().substring(uri.length); const localURL = url.toString().substring(uri.length);
const method = opts.method ?? "GET"; const method = opts?.method ?? "GET";
if (opts.body) { if (opts?.body) {
const opts_length = opts.body.toString().length; const opts_length = opts.body.toString().length;
if (opts_length > 1000 * 1000 * 10) { if (opts_length > 1000 * 1000 * 10) {
// over 10MB // over 10MB
@@ -132,10 +135,10 @@ export default class ObsidianLiveSyncPlugin extends Plugin
size = ` (${opts_length})`; size = ` (${opts_length})`;
} }
if (!disableRequestURI && typeof url == "string" && typeof (opts.body ?? "") == "string") { if (!disableRequestURI && typeof url == "string" && typeof (opts?.body ?? "") == "string") {
const body = opts.body as string; const body = opts?.body as string;
const transformedHeaders = { ...(opts.headers as Record<string, string>) }; const transformedHeaders = { ...(opts?.headers as Record<string, string>) };
if (authHeader != "") transformedHeaders["authorization"] = authHeader; if (authHeader != "") transformedHeaders["authorization"] = authHeader;
delete transformedHeaders["host"]; delete transformedHeaders["host"];
delete transformedHeaders["Host"]; delete transformedHeaders["Host"];
@@ -143,7 +146,7 @@ export default class ObsidianLiveSyncPlugin extends Plugin
delete transformedHeaders["Content-Length"]; delete transformedHeaders["Content-Length"];
const requestParam: RequestUrlParam = { const requestParam: RequestUrlParam = {
url, url,
method: opts.method, method: opts?.method,
body: body, body: body,
headers: transformedHeaders, headers: transformedHeaders,
contentType: "application/json", contentType: "application/json",
@@ -151,7 +154,7 @@ export default class ObsidianLiveSyncPlugin extends Plugin
}; };
try { try {
const r = await this.fetchByAPI(requestParam); const r = await fetchByAPI(requestParam);
if (method == "POST" || method == "PUT") { if (method == "POST" || method == "PUT") {
this.last_successful_post = r.status - (r.status % 100) == 200; this.last_successful_post = r.status - (r.status % 100) == 200;
} else { } else {
@@ -201,7 +204,7 @@ export default class ObsidianLiveSyncPlugin extends Plugin
const db: PouchDB.Database<EntryDoc> = new PouchDB<EntryDoc>(uri, conf); const db: PouchDB.Database<EntryDoc> = new PouchDB<EntryDoc>(uri, conf);
if (passphrase !== "false" && typeof passphrase === "string") { if (passphrase !== "false" && typeof passphrase === "string") {
enableEncryption(db, passphrase, useDynamicIterationCount); enableEncryption(db, passphrase, useDynamicIterationCount, false, this.settings.useV1);
} }
if (skipInfo) { if (skipInfo) {
return { db: db, info: { db_name: "", doc_count: 0, update_seq: "" } }; return { db: db, info: { db_name: "", doc_count: 0, update_seq: "" } };
@@ -209,9 +212,9 @@ export default class ObsidianLiveSyncPlugin extends Plugin
try { try {
const info = await db.info(); const info = await db.info();
return { db: db, info: info }; return { db: db, info: info };
} catch (ex) { } catch (ex: any) {
let msg = `${ex.name}:${ex.message}`; let msg = `${ex?.name}:${ex?.message}`;
if (ex.name == "TypeError" && ex.message == "Failed to fetch") { if (ex?.name == "TypeError" && ex?.message == "Failed to fetch") {
msg += "\n**Note** This error caused by many reasons. The only sure thing is you didn't touch the server.\nTo check details, open inspector."; msg += "\n**Note** This error caused by many reasons. The only sure thing is you didn't touch the server.\nTo check details, open inspector.";
} }
Logger(ex, LOG_LEVEL_VERBOSE); Logger(ex, LOG_LEVEL_VERBOSE);
@@ -219,7 +222,7 @@ export default class ObsidianLiveSyncPlugin extends Plugin
} }
} }
id2path(id: DocumentID, entry: EntryHasPath, stripPrefix?: boolean): FilePathWithPrefix { id2path(id: DocumentID, entry?: EntryHasPath, stripPrefix?: boolean): FilePathWithPrefix {
const tempId = id2path(id, entry); const tempId = id2path(id, entry);
if (stripPrefix && isInternalMetadata(tempId)) { if (stripPrefix && isInternalMetadata(tempId)) {
const out = stripInternalMetadataPrefix(tempId); const out = stripInternalMetadataPrefix(tempId);
@@ -234,18 +237,19 @@ export default class ObsidianLiveSyncPlugin extends Plugin
return getPathWithoutPrefix(entry); return getPathWithoutPrefix(entry);
} }
async path2id(filename: FilePathWithPrefix | FilePath, prefix?: string): Promise<DocumentID> { async path2id(filename: FilePathWithPrefix | FilePath, prefix?: string): Promise<DocumentID> {
const destPath = addPrefix(filename, prefix); const destPath = addPrefix(filename, prefix ?? "");
return await path2id(destPath, this.settings.usePathObfuscation ? this.settings.passphrase : ""); return await path2id(destPath, this.settings.usePathObfuscation ? this.settings.passphrase : "");
} }
createPouchDBInstance<T>(name?: string, options?: PouchDB.Configuration.DatabaseConfiguration): PouchDB.Database<T> { createPouchDBInstance<T extends object>(name?: string, options?: PouchDB.Configuration.DatabaseConfiguration): PouchDB.Database<T> {
const optionPass = options ?? {};
if (this.settings.useIndexedDBAdapter) { if (this.settings.useIndexedDBAdapter) {
options.adapter = "indexeddb"; optionPass.adapter = "indexeddb";
//@ts-ignore :missing def //@ts-ignore :missing def
options.purged_infos_limit = 1; optionPass.purged_infos_limit = 1;
return new PouchDB(name + "-indexeddb", options); return new PouchDB(name + "-indexeddb", optionPass);
} }
return new PouchDB(name, options); return new PouchDB(name, optionPass);
} }
beforeOnUnload(db: LiveSyncLocalDB): void { beforeOnUnload(db: LiveSyncLocalDB): void {
this.kvDB.close(); this.kvDB.close();
@@ -258,6 +262,8 @@ export default class ObsidianLiveSyncPlugin extends Plugin
this.replicator = new LiveSyncDBReplicator(this); this.replicator = new LiveSyncDBReplicator(this);
} }
async onResetDatabase(db: LiveSyncLocalDB): Promise<void> { async onResetDatabase(db: LiveSyncLocalDB): Promise<void> {
const lsKey = "obsidian-livesync-queuefiles-" + this.getVaultName();
localStorage.removeItem(lsKey);
await this.kvDB.destroy(); await this.kvDB.destroy();
this.kvDB = await OpenKeyValueDatabase(db.dbname + "-livesync-kv"); this.kvDB = await OpenKeyValueDatabase(db.dbname + "-livesync-kv");
this.replicator = new LiveSyncDBReplicator(this); this.replicator = new LiveSyncDBReplicator(this);
@@ -320,7 +326,7 @@ export default class ObsidianLiveSyncPlugin extends Plugin
} }
} }
showHistory(file: TFile | FilePathWithPrefix, id: DocumentID) { showHistory(file: TFile | FilePathWithPrefix, id?: DocumentID) {
new DocumentHistoryModal(this.app, this, file, id).open(); new DocumentHistoryModal(this.app, this, file, id).open();
} }
@@ -334,7 +340,7 @@ export default class ObsidianLiveSyncPlugin extends Plugin
const target = await askSelectString(this.app, "File to view History", notesList); const target = await askSelectString(this.app, "File to view History", notesList);
if (target) { if (target) {
const targetId = notes.find(e => e.dispPath == target); const targetId = notes.find(e => e.dispPath == target);
this.showHistory(targetId.path, undefined); this.showHistory(targetId.path, targetId.id);
} }
} }
async pickFileForResolve() { async pickFileForResolve() {
@@ -349,7 +355,7 @@ export default class ObsidianLiveSyncPlugin extends Plugin
Logger("There are no conflicted documents", LOG_LEVEL_NOTICE); Logger("There are no conflicted documents", LOG_LEVEL_NOTICE);
return false; return false;
} }
const target = await askSelectString(this.app, "File to view History", notesList); const target = await askSelectString(this.app, "File to resolve conflict", notesList);
if (target) { if (target) {
const targetItem = notes.find(e => e.dispPath == target); const targetItem = notes.find(e => e.dispPath == target);
await this.resolveConflicted(targetItem.path); await this.resolveConflicted(targetItem.path);
@@ -363,6 +369,8 @@ export default class ObsidianLiveSyncPlugin extends Plugin
await this.addOnHiddenFileSync.resolveConflictOnInternalFile(target); await this.addOnHiddenFileSync.resolveConflictOnInternalFile(target);
} else if (isPluginMetadata(target)) { } else if (isPluginMetadata(target)) {
await this.resolveConflictByNewerEntry(target); await this.resolveConflictByNewerEntry(target);
} else if (isCustomisationSyncMetadata(target)) {
await this.resolveConflictByNewerEntry(target);
} else { } else {
await this.showIfConflicted(target); await this.showIfConflicted(target);
} }
@@ -396,6 +404,10 @@ export default class ObsidianLiveSyncPlugin extends Plugin
Logger(`Checking expired file history done`); Logger(`Checking expired file history done`);
} }
async onLayoutReady() { async onLayoutReady() {
if (this.settings.useV1 === undefined) {
this.settings.useV1 = await this.askEnableV2();
await this.saveSettingData();
}
this.registerFileWatchEvents(); this.registerFileWatchEvents();
if (!this.localDatabase.isReady) { if (!this.localDatabase.isReady) {
Logger(`Something went wrong! The local database is not ready`, LOG_LEVEL_NOTICE); Logger(`Something went wrong! The local database is not ready`, LOG_LEVEL_NOTICE);
@@ -451,8 +463,9 @@ export default class ObsidianLiveSyncPlugin extends Plugin
return false; return false;
} }
} }
await this.realizeSettingSyncMode();
this.registerWatchEvents(); this.registerWatchEvents();
await this.realizeSettingSyncMode();
this.swapSaveCommand();
if (this.settings.syncOnStart) { if (this.settings.syncOnStart) {
this.replicator.openReplication(this.settings, false, false); this.replicator.openReplication(this.settings, false, false);
} }
@@ -474,7 +487,15 @@ export default class ObsidianLiveSyncPlugin extends Plugin
notes.push({ path: this.getPath(doc), mtime: doc.mtime }); notes.push({ path: this.getPath(doc), mtime: doc.mtime });
} }
if (notes.length > 0) { if (notes.length > 0) {
Logger(`Some files have been left conflicted! Please resolve them by "Pick a file to resolve conflict". The list is written in the log.`, LOG_LEVEL_NOTICE); this.askInPopup(`conflicting-detected-on-safety`, `Some files have been left conflicted! Press {HERE} to resolve them, or you can do it later by "Pick a file to resolve conflict`, (anchor) => {
anchor.text = "HERE";
anchor.addEventListener("click", () => {
// @ts-ignore
this.app.commands.executeCommandById("obsidian-livesync:livesync-all-conflictcheck");
});
}
);
Logger(`Some files have been left conflicted! Please resolve them by "Pick a file to resolve conflict". The list is written in the log.`, LOG_LEVEL_VERBOSE);
for (const note of notes) { for (const note of notes) {
Logger(`Conflicted: ${note.path}`); Logger(`Conflicted: ${note.path}`);
} }
@@ -483,6 +504,16 @@ export default class ObsidianLiveSyncPlugin extends Plugin
} }
Logger(`Additional safety scan done`, LOG_LEVEL_VERBOSE); Logger(`Additional safety scan done`, LOG_LEVEL_VERBOSE);
} }
async askEnableV2() {
const message = `Since v0.20.0, Self-hosted LiveSync uses a new format for binary files and encrypted things. In the new format, files are split at meaningful delimitations, increasing the effectiveness of deduplication.
However, the new format lacks compatibility with LiveSync before v0.20.0 and related projects. Basically enabling V2 is recommended. but If you are using some related products, stay in a while, please!
Note: We can always able to read V1 format. It will be progressively converted. And, we can change this toggle later.`
const CHOICE_V2 = "Enable v2";
const CHOICE_V1 = "Keep v1";
const ret = await confirmWithMessage(this, "binary and encryption", message, [CHOICE_V2, CHOICE_V1], CHOICE_V1, 40);
return ret == CHOICE_V1;
}
async onload() { async onload() {
logStore.subscribe(e => this.addLog(e.message, e.level, e.key)); logStore.subscribe(e => this.addLog(e.message, e.level, e.key));
@@ -504,6 +535,7 @@ export default class ObsidianLiveSyncPlugin extends Plugin
if (lastVersion > this.settings.lastReadUpdates) { if (lastVersion > this.settings.lastReadUpdates) {
Logger("Self-hosted LiveSync has undergone a major upgrade. Please open the setting dialog, and check the information pane.", LOG_LEVEL_NOTICE); Logger("Self-hosted LiveSync has undergone a major upgrade. Please open the setting dialog, and check the information pane.", LOG_LEVEL_NOTICE);
} }
//@ts-ignore //@ts-ignore
if (this.app.isMobile) { if (this.app.isMobile) {
this.isMobile = true; this.isMobile = true;
@@ -512,6 +544,7 @@ export default class ObsidianLiveSyncPlugin extends Plugin
if (last_version && Number(last_version) < VER) { if (last_version && Number(last_version) < VER) {
this.settings.liveSync = false; this.settings.liveSync = false;
this.settings.syncOnSave = false; this.settings.syncOnSave = false;
this.settings.syncOnEditorSave = false;
this.settings.syncOnStart = false; this.settings.syncOnStart = false;
this.settings.syncOnFileOpen = false; this.settings.syncOnFileOpen = false;
this.settings.syncAfterMerge = false; this.settings.syncAfterMerge = false;
@@ -558,7 +591,13 @@ export default class ObsidianLiveSyncPlugin extends Plugin
this.addRibbonIcon("view-log", "Show log", () => { this.addRibbonIcon("view-log", "Show log", () => {
this.showView(VIEW_TYPE_LOG); this.showView(VIEW_TYPE_LOG);
}); });
this.addCommand({
id: "view-log",
name: "Show log",
callback: () => {
this.showView(VIEW_TYPE_LOG);
}
});
this.addSettingTab(new ObsidianLiveSyncSettingTab(this.app, this)); this.addSettingTab(new ObsidianLiveSyncSettingTab(this.app, this));
this.app.workspace.onLayoutReady(this.onLayoutReady.bind(this)); this.app.workspace.onLayoutReady(this.onLayoutReady.bind(this));
@@ -572,15 +611,19 @@ export default class ObsidianLiveSyncPlugin extends Plugin
this.addCommand({ this.addCommand({
id: "livesync-dump", id: "livesync-dump",
name: "Dump information of this doc ", name: "Dump information of this doc ",
editorCallback: (editor: Editor, view: MarkdownView) => { editorCallback: (editor: Editor, view: MarkdownView | MarkdownFileInfo) => {
this.localDatabase.getDBEntry(getPathFromTFile(view.file), {}, true, false); const file = view.file;
if (!file) return;
this.localDatabase.getDBEntry(getPathFromTFile(file), {}, true, false);
}, },
}); });
this.addCommand({ this.addCommand({
id: "livesync-checkdoc-conflicted", id: "livesync-checkdoc-conflicted",
name: "Resolve if conflicted.", name: "Resolve if conflicted.",
editorCallback: async (editor: Editor, view: MarkdownView) => { editorCallback: async (editor: Editor, view: MarkdownView | MarkdownFileInfo) => {
await this.showIfConflicted(getPathFromTFile(view.file)); const file = view.file;
if (!file) return;
await this.showIfConflicted(getPathFromTFile(file));
}, },
}); });
@@ -617,8 +660,8 @@ export default class ObsidianLiveSyncPlugin extends Plugin
this.addCommand({ this.addCommand({
id: "livesync-history", id: "livesync-history",
name: "Show history", name: "Show history",
editorCallback: (editor: Editor, view: MarkdownView) => { editorCallback: (editor: Editor, view: MarkdownView | MarkdownFileInfo) => {
this.showHistory(view.file, null); if (view.file) this.showHistory(view.file, null);
}, },
}); });
this.addCommand({ this.addCommand({
@@ -720,6 +763,7 @@ export default class ObsidianLiveSyncPlugin extends Plugin
} }
cancelAllPeriodicTask(); cancelAllPeriodicTask();
cancelAllTasks(); cancelAllTasks();
this._unloaded = true;
Logger("unloading plugin"); Logger("unloading plugin");
} }
@@ -768,7 +812,7 @@ export default class ObsidianLiveSyncPlugin extends Plugin
async encryptConfigurationItem(src: string, settings: ObsidianLiveSyncSettings) { async encryptConfigurationItem(src: string, settings: ObsidianLiveSyncSettings) {
if (this.usedPassphrase != "") { if (this.usedPassphrase != "") {
return await encrypt(src, this.usedPassphrase + SALT_OF_PASSPHRASE, false); return await encrypt(src, this.usedPassphrase + SALT_OF_PASSPHRASE, false, true);
} }
const passphrase = await this.getPassphrase(settings); const passphrase = await this.getPassphrase(settings);
@@ -776,7 +820,7 @@ export default class ObsidianLiveSyncPlugin extends Plugin
Logger("Could not determine passphrase to save data.json! You probably make the configuration sure again!", LOG_LEVEL_URGENT); Logger("Could not determine passphrase to save data.json! You probably make the configuration sure again!", LOG_LEVEL_URGENT);
return ""; return "";
} }
const dec = await encrypt(src, passphrase + SALT_OF_PASSPHRASE, false); const dec = await encrypt(src, passphrase + SALT_OF_PASSPHRASE, false, true);
if (dec) { if (dec) {
this.usedPassphrase = passphrase; this.usedPassphrase = passphrase;
return dec; return dec;
@@ -852,7 +896,8 @@ export default class ObsidianLiveSyncPlugin extends Plugin
(async () => await this.realizeSettingSyncMode())(); (async () => await this.realizeSettingSyncMode())();
} }
async saveSettings() {
async saveSettingData() {
const lsKey = "obsidian-live-sync-vaultanddevicename-" + this.getVaultName(); const lsKey = "obsidian-live-sync-vaultanddevicename-" + this.getVaultName();
localStorage.setItem(lsKey, this.deviceAndVaultName || ""); localStorage.setItem(lsKey, this.deviceAndVaultName || "");
@@ -882,6 +927,10 @@ export default class ObsidianLiveSyncPlugin extends Plugin
await this.saveData(settings); await this.saveData(settings);
this.localDatabase.settings = this.settings; this.localDatabase.settings = this.settings;
this.ignoreFiles = this.settings.ignoreFiles.split(",").map(e => e.trim()); this.ignoreFiles = this.settings.ignoreFiles.split(",").map(e => e.trim());
}
async saveSettings() {
await this.saveSettingData();
this.triggerRealizeSettingSyncMode(); this.triggerRealizeSettingSyncMode();
} }
@@ -889,7 +938,38 @@ export default class ObsidianLiveSyncPlugin extends Plugin
registerFileWatchEvents() { registerFileWatchEvents() {
this.vaultManager = new StorageEventManagerObsidian(this) this.vaultManager = new StorageEventManagerObsidian(this)
} }
_initialCallback: any;
swapSaveCommand() {
Logger("Modifying callback of the save command", LOG_LEVEL_VERBOSE);
const saveCommandDefinition = (this.app as any).commands?.commands?.[
"editor:save-file"
];
const save = saveCommandDefinition?.callback;
if (typeof save === "function") {
this._initialCallback = save;
saveCommandDefinition.callback = () => {
scheduleTask("syncOnEditorSave", 250, () => {
if (this._unloaded) {
Logger("Unload and remove the handler.", LOG_LEVEL_VERBOSE);
saveCommandDefinition.callback = this._initialCallback;
} else {
Logger("Sync on Editor Save.", LOG_LEVEL_VERBOSE);
if (this.settings.syncOnEditorSave) {
this.replicate();
}
}
});
save();
};
}
// eslint-disable-next-line @typescript-eslint/no-this-alias
const _this = this;
//@ts-ignore
window.CodeMirrorAdapter.commands.save = () => {
//@ts-ignore
_this.app.commands.executeCommandById('editor:save-file');
};
}
registerWatchEvents() { registerWatchEvents() {
this.registerEvent(this.app.workspace.on("file-open", this.watchWorkspaceOpen)); this.registerEvent(this.app.workspace.on("file-open", this.watchWorkspaceOpen));
this.registerDomEvent(document, "visibilitychange", this.watchWindowVisibility); this.registerDomEvent(document, "visibilitychange", this.watchWindowVisibility);
@@ -949,7 +1029,7 @@ export default class ObsidianLiveSyncPlugin extends Plugin
} }
} }
cancelTask("applyBatchAuto"); cancelTask("applyBatchAuto");
const ret = await runWithLock("procFiles", true, async () => { const ret = await skipIfDuplicated("procFiles", async () => {
do { do {
const queue = this.vaultManager.fetchEvent(); const queue = this.vaultManager.fetchEvent();
if (queue === false) break; if (queue === false) break;
@@ -1004,9 +1084,10 @@ export default class ObsidianLiveSyncPlugin extends Plugin
} }
watchWorkspaceOpen(file: TFile) { watchWorkspaceOpen(file: TFile | null) {
if (this.settings.suspendFileWatching) return; if (this.settings.suspendFileWatching) return;
if (!this.isReady) return; if (!this.isReady) return;
if (!file) return;
this.watchWorkspaceOpenAsync(file); this.watchWorkspaceOpenAsync(file);
} }
@@ -1167,17 +1248,25 @@ export default class ObsidianLiveSyncPlugin extends Plugin
if (docEntry._deleted || docEntry.deleted) { if (docEntry._deleted || docEntry.deleted) {
// This occurs not only when files are deleted, but also when conflicts are resolved. // This occurs not only when files are deleted, but also when conflicts are resolved.
// We have to check no other revisions are left. // We have to check no other revisions are left.
const lastDocs = await this.localDatabase.getDBEntry(path); const existDoc = await this.localDatabase.getDBEntry(path, { conflicts: true });
if (path != file.path) { if (path != file.path) {
Logger(`delete skipped: ${file.path} :Not exactly matched`, LOG_LEVEL_VERBOSE); Logger(`delete skipped: ${file.path} :Not exactly matched`, LOG_LEVEL_VERBOSE);
} }
if (lastDocs === false) { if (existDoc === false) {
await this.deleteVaultItem(file); await this.deleteVaultItem(file);
} else { } else {
// it perhaps delete some revisions. if (existDoc._conflicts) {
// may be we have to reload this if (this.settings.writeDocumentsIfConflicted) {
await this.pullFile(path, null, true); Logger(`Delete: ${file.path}: Conflicted revision has been deleted, but there were more conflicts. `, LOG_LEVEL_INFO);
Logger(`delete skipped:${file.path}`, LOG_LEVEL_VERBOSE); await this.pullFile(path, null, true);
} else {
Logger(`Delete: ${file.path}: Conflicted revision has been deleted, but there were more conflicts...`);
this.queueConflictedOnlyActiveFile(file);
}
} else {
Logger(`Delete: ${file.path}: Conflict revision has been deleted and resolved`);
await this.pullFile(path, null, true);
}
} }
return; return;
} }
@@ -1196,7 +1285,7 @@ export default class ObsidianLiveSyncPlugin extends Plugin
Logger(msg + "ERROR, invalid path: " + path, LOG_LEVEL_NOTICE); Logger(msg + "ERROR, invalid path: " + path, LOG_LEVEL_NOTICE);
return; return;
} }
const writeData = doc.datatype == "newnote" ? base64ToArrayBuffer(doc.data) : getDocData(doc.data); const writeData = doc.datatype == "newnote" ? decodeBinary(doc.data) : getDocData(doc.data);
await this.ensureDirectoryEx(path); await this.ensureDirectoryEx(path);
try { try {
let outFile; let outFile;
@@ -1261,7 +1350,7 @@ export default class ObsidianLiveSyncPlugin extends Plugin
const path = getPath(entry); const path = getPath(entry);
try { try {
const releaser = await semaphore.acquire(1); const releaser = await semaphore.acquire(1);
runWithLock(`dbchanged-${path}`, false, async () => { serialized(`dbchanged-${path}`, async () => {
Logger(`Applying ${path} (${entry._id}: ${entry._rev}) change...`, LOG_LEVEL_VERBOSE); Logger(`Applying ${path} (${entry._id}: ${entry._rev}) change...`, LOG_LEVEL_VERBOSE);
await this.handleDBChangedAsync(entry); await this.handleDBChangedAsync(entry);
Logger(`Applied ${path} (${entry._id}:${entry._rev}) change...`); Logger(`Applied ${path} (${entry._id}:${entry._rev}) change...`);
@@ -1274,6 +1363,19 @@ export default class ObsidianLiveSyncPlugin extends Plugin
this.dbChangeProcRunning = false; this.dbChangeProcRunning = false;
} }
} }
queueConflictedOnlyActiveFile(file: TFile) {
if (!this.settings.checkConflictOnlyOnOpen) {
this.queueConflictedCheck(file);
return true;
} else {
const af = this.app.workspace.getActiveFile();
if (af && af.path == file.path) {
this.queueConflictedCheck(file);
return true;
}
}
return false;
}
async handleDBChangedAsync(change: EntryBody) { async handleDBChangedAsync(change: EntryBody) {
const targetFile = getAbstractFileByPath(this.getPathWithoutPrefix(change)); const targetFile = getAbstractFileByPath(this.getPathWithoutPrefix(change));
@@ -1286,29 +1388,16 @@ export default class ObsidianLiveSyncPlugin extends Plugin
} else if (targetFile instanceof TFile) { } else if (targetFile instanceof TFile) {
const doc = change; const doc = change;
const file = targetFile; const file = targetFile;
const queueConflictCheck = () => {
if (!this.settings.checkConflictOnlyOnOpen) {
this.queueConflictedCheck(file);
return true;
} else {
const af = app.workspace.getActiveFile();
if (af && af.path == file.path) {
this.queueConflictedCheck(file);
return true;
}
}
return false;
}
if (this.settings.writeDocumentsIfConflicted) { if (this.settings.writeDocumentsIfConflicted) {
await this.doc2storage(doc, file); await this.doc2storage(doc, file);
queueConflictCheck(); this.queueConflictedOnlyActiveFile(file);
} else { } else {
const d = await this.localDatabase.getDBEntryMeta(this.getPath(change), { conflicts: true }, true); const d = await this.localDatabase.getDBEntryMeta(this.getPath(change), { conflicts: true }, true);
if (d && !d._conflicts) { if (d && !d._conflicts) {
await this.doc2storage(doc, file); await this.doc2storage(doc, file);
} else { } else {
if (!queueConflictCheck()) { if (!this.queueConflictedOnlyActiveFile(file)) {
Logger(`${this.getPath(change)} is conflicted, write to the storage has been pended.`, LOG_LEVEL_NOTICE); Logger(`${this.getPath(change)} is conflicted, write to the storage has been postponed.`, LOG_LEVEL_NOTICE);
} }
} }
} }
@@ -1347,10 +1436,10 @@ export default class ObsidianLiveSyncPlugin extends Plugin
const now = new Date().getTime(); const now = new Date().getTime();
if (queue.missingChildren.length == 0) { if (queue.missingChildren.length == 0) {
queue.done = true; queue.done = true;
if (isInternalMetadata(queue.entry._id)) { if (isInternalMetadata(queue.entry._id) && this.settings.syncInternalFiles) {
//system file //system file
const filename = this.getPathWithoutPrefix(queue.entry); const filename = this.getPathWithoutPrefix(queue.entry);
this.addOnHiddenFileSync.procInternalFile(filename); this.isTargetFile(filename).then((ret) => ret ? this.addOnHiddenFileSync.procInternalFile(filename) : Logger(`Skipped (Not target:${filename})`, LOG_LEVEL_VERBOSE));
} else if (isValidPath(this.getPath(queue.entry))) { } else if (isValidPath(this.getPath(queue.entry))) {
this.handleDBChanged(queue.entry); this.handleDBChanged(queue.entry);
} else { } else {
@@ -1392,7 +1481,14 @@ export default class ObsidianLiveSyncPlugin extends Plugin
if (!await this.isTargetFile(path)) return; if (!await this.isTargetFile(path)) return;
const skipOldFile = this.settings.skipOlderFilesOnSync && false; //patched temporary. const skipOldFile = this.settings.skipOlderFilesOnSync && false; //patched temporary.
// Do not handle internal files if the feature has not been enabled. // Do not handle internal files if the feature has not been enabled.
if (isInternalMetadata(doc._id) && !this.settings.syncInternalFiles) return; if (isInternalMetadata(doc._id) && !this.settings.syncInternalFiles) {
Logger(`Skipped: ${path} (${doc._id}, ${doc._rev}) Hidden file sync is disabled.`, LOG_LEVEL_VERBOSE);
return;
}
if (isCustomisationSyncMetadata(doc._id) && !this.settings.usePluginSync) {
Logger(`Skipped: ${path} (${doc._id}, ${doc._rev}) Customization sync is disabled.`, LOG_LEVEL_VERBOSE);
return;
}
// It is better for your own safety, not to handle the following files // It is better for your own safety, not to handle the following files
const ignoreFiles = [ const ignoreFiles = [
"_design/replicate", "_design/replicate",
@@ -1466,12 +1562,22 @@ export default class ObsidianLiveSyncPlugin extends Plugin
if (change.type != "leaf" && change.type != "versioninfo" && change.type != "milestoneinfo" && change.type != "nodeinfo") { if (change.type != "leaf" && change.type != "versioninfo" && change.type != "milestoneinfo" && change.type != "nodeinfo") {
if (this.settings.suspendParseReplicationResult) { if (this.settings.suspendParseReplicationResult) {
if (isInternalMetadata(change._id) && !this.settings.syncInternalFiles) {
continue;
}
if (isCustomisationSyncMetadata(change._id) && !this.settings.usePluginSync) {
continue;
}
if (!change.path) {
continue;
}
const newQueue = { const newQueue = {
entry: change, entry: change,
missingChildren: [] as string[], missingChildren: [] as string[],
timeout: 0, timeout: 0,
}; };
Logger(`Processing scheduled: ${change.path}`, LOG_LEVEL_INFO); Logger(`Processing scheduled: ${change.path}`, LOG_LEVEL_INFO);
this.queuedFiles = this.queuedFiles.filter(e => e.entry.path != change.path);
this.queuedFiles.push(newQueue); this.queuedFiles.push(newQueue);
this.saveQueuedFiles(); this.saveQueuedFiles();
continue; continue;
@@ -1638,7 +1744,7 @@ export default class ObsidianLiveSyncPlugin extends Plugin
if (this.replicator.remoteLockedAndDeviceNotAccepted) { if (this.replicator.remoteLockedAndDeviceNotAccepted) {
if (this.replicator.remoteCleaned && this.settings.useIndexedDBAdapter) { if (this.replicator.remoteCleaned && this.settings.useIndexedDBAdapter) {
Logger(`The remote database has been cleaned.`, showMessage ? LOG_LEVEL_NOTICE : LOG_LEVEL_INFO); Logger(`The remote database has been cleaned.`, showMessage ? LOG_LEVEL_NOTICE : LOG_LEVEL_INFO);
await runWithLock("cleanup", true, async () => { await skipIfDuplicated("cleanup", async () => {
const count = await purgeUnreferencedChunks(this.localDatabase.localDatabase, true); const count = await purgeUnreferencedChunks(this.localDatabase.localDatabase, true);
const message = `The remote database has been cleaned up. const message = `The remote database has been cleaned up.
To synchronize, this device must be also cleaned up. ${count} chunk(s) will be erased from this device. To synchronize, this device must be also cleaned up. ${count} chunk(s) will be erased from this device.
@@ -1862,7 +1968,7 @@ Or if you are sure know what had been happened, we can unlock the database from
if (doc === false) return false; if (doc === false) return false;
let data = getDocData(doc.data) let data = getDocData(doc.data)
if (doc.datatype == "newnote") { if (doc.datatype == "newnote") {
data = base64ToString(data); data = readString(new Uint8Array(decodeBinary(doc.data)));
} else if (doc.datatype == "plain") { } else if (doc.datatype == "plain") {
// NO OP. // NO OP.
} }
@@ -1901,6 +2007,9 @@ Or if you are sure know what had been happened, we can unlock the database from
const ret = [] as Diff[]; const ret = [] as Diff[];
do { do {
const d = src.shift(); const d = src.shift();
if (d === undefined) {
return ret;
}
const pieces = d[1].split(/([^\n]*\n)/).filter(f => f != ""); const pieces = d[1].split(/([^\n]*\n)/).filter(f => f != "");
if (typeof (d) == "undefined") { if (typeof (d) == "undefined") {
break; break;
@@ -2161,18 +2270,11 @@ Or if you are sure know what had been happened, we can unlock the database from
Logger(`could not get old revisions, automatically used newer one:${path}`, LOG_LEVEL_NOTICE); Logger(`could not get old revisions, automatically used newer one:${path}`, LOG_LEVEL_NOTICE);
return true; return true;
} }
// first, check for same contents and deletion status.
if (leftLeaf.data == rightLeaf.data && leftLeaf.deleted == rightLeaf.deleted) { const isSame = leftLeaf.data == rightLeaf.data && leftLeaf.deleted == rightLeaf.deleted;
let leaf = leftLeaf; const isBinary = !isPlainText(path);
if (leftLeaf.mtime > rightLeaf.mtime) { const alwaysNewer = this.settings.resolveConflictsByNewerFile;
leaf = rightLeaf; if (isSame || isBinary || alwaysNewer) {
}
await this.localDatabase.deleteDBEntry(path, { rev: leaf.rev });
await this.pullFile(path, null, true);
Logger(`automatically merged:${path}`);
return true;
}
if (this.settings.resolveConflictsByNewerFile) {
const lMtime = ~~(leftLeaf.mtime / 1000); const lMtime = ~~(leftLeaf.mtime / 1000);
const rMtime = ~~(rightLeaf.mtime / 1000); const rMtime = ~~(rightLeaf.mtime / 1000);
let loser = leftLeaf; let loser = leftLeaf;
@@ -2181,7 +2283,7 @@ Or if you are sure know what had been happened, we can unlock the database from
} }
await this.localDatabase.deleteDBEntry(path, { rev: loser.rev }); await this.localDatabase.deleteDBEntry(path, { rev: loser.rev });
await this.pullFile(path, null, true); await this.pullFile(path, null, true);
Logger(`Automatically merged (newerFileResolve) :${path}`, LOG_LEVEL_NOTICE); Logger(`Automatically merged (${isSame ? "same," : ""}${isBinary ? "binary," : ""}${alwaysNewer ? "alwaysNewer" : ""}) :${path}`, LOG_LEVEL_NOTICE);
return true; return true;
} }
// make diff. // make diff.
@@ -2197,7 +2299,7 @@ Or if you are sure know what had been happened, we can unlock the database from
} }
showMergeDialog(filename: FilePathWithPrefix, conflictCheckResult: diff_result): Promise<boolean> { showMergeDialog(filename: FilePathWithPrefix, conflictCheckResult: diff_result): Promise<boolean> {
return runWithLock("resolve-conflict:" + filename, false, () => return serialized("resolve-conflict:" + filename, () =>
new Promise((res, rej) => { new Promise((res, rej) => {
Logger("open conflict dialog", LOG_LEVEL_VERBOSE); Logger("open conflict dialog", LOG_LEVEL_VERBOSE);
new ConflictResolveModal(this.app, filename, conflictCheckResult, async (selected) => { new ConflictResolveModal(this.app, filename, conflictCheckResult, async (selected) => {
@@ -2276,7 +2378,7 @@ Or if you are sure know what had been happened, we can unlock the database from
} }
async showIfConflicted(filename: FilePathWithPrefix) { async showIfConflicted(filename: FilePathWithPrefix) {
await runWithLock("conflicted", false, async () => { await serialized("conflicted", async () => {
const conflictCheckResult = await this.getConflictedStatus(filename); const conflictCheckResult = await this.getConflictedStatus(filename);
if (conflictCheckResult === false) { if (conflictCheckResult === false) {
//nothing to do. //nothing to do.
@@ -2386,7 +2488,7 @@ Or if you are sure know what had been happened, we can unlock the database from
const contentBin = await this.app.vault.readBinary(file); const contentBin = await this.app.vault.readBinary(file);
Logger(`Processing: ${file.path}`, LOG_LEVEL_VERBOSE); Logger(`Processing: ${file.path}`, LOG_LEVEL_VERBOSE);
try { try {
content = await arrayBufferToBase64(contentBin); content = await encodeBinary(contentBin, this.settings.useV1);
} catch (ex) { } catch (ex) {
Logger(`The file ${file.path} could not be encoded`); Logger(`The file ${file.path} could not be encoded`);
Logger(ex, LOG_LEVEL_VERBOSE); Logger(ex, LOG_LEVEL_VERBOSE);
@@ -2401,7 +2503,7 @@ Or if you are sure know what had been happened, we can unlock the database from
if (cache instanceof ArrayBuffer) { if (cache instanceof ArrayBuffer) {
Logger(`Processing: ${file.path}`, LOG_LEVEL_VERBOSE); Logger(`Processing: ${file.path}`, LOG_LEVEL_VERBOSE);
try { try {
content = await arrayBufferToBase64(cache); content = await encodeBinary(cache, this.settings.useV1);
} catch (ex) { } catch (ex) {
Logger(`The file ${file.path} could not be encoded`); Logger(`The file ${file.path} could not be encoded`);
Logger(ex, LOG_LEVEL_VERBOSE); Logger(ex, LOG_LEVEL_VERBOSE);
@@ -2428,7 +2530,7 @@ Or if you are sure know what had been happened, we can unlock the database from
}; };
//upsert should locked //upsert should locked
const msg = `DB <- STORAGE (${datatype}) `; const msg = `DB <- STORAGE (${datatype}) `;
const isNotChanged = await runWithLock("file-" + fullPath, false, async () => { const isNotChanged = await serialized("file-" + fullPath, async () => {
if (recentlyTouched(file)) { if (recentlyTouched(file)) {
return true; return true;
} }
@@ -2569,9 +2671,14 @@ Or if you are sure know what had been happened, we can unlock the database from
} }
} }
/**
* Check the file is ignored by the ignore files.
* @param file
* @returns true if the file should be ignored, false if the file should be processed.
*/
async isIgnoredByIgnoreFiles(file: string | TAbstractFile) { async isIgnoredByIgnoreFiles(file: string | TAbstractFile) {
if (!this.settings.useIgnoreFiles) { if (!this.settings.useIgnoreFiles) {
return true; return false;
} }
const filepath = file instanceof TFile ? file.path : file as string; const filepath = file instanceof TFile ? file.path : file as string;
if (this.ignoreFileCache.has(filepath)) { if (this.ignoreFileCache.has(filepath)) {
@@ -2579,20 +2686,20 @@ Or if you are sure know what had been happened, we can unlock the database from
await this.readIgnoreFile(filepath); await this.readIgnoreFile(filepath);
} }
if (!await isAcceptedAll(stripAllPrefixes(filepath as FilePathWithPrefix), this.ignoreFiles, (filename) => this.getIgnoreFile(filename))) { if (!await isAcceptedAll(stripAllPrefixes(filepath as FilePathWithPrefix), this.ignoreFiles, (filename) => this.getIgnoreFile(filename))) {
return false; return true;
} }
return true; return false;
} }
async isTargetFile(file: string | TAbstractFile) { async isTargetFile(file: string | TAbstractFile) {
const filepath = file instanceof TFile ? file.path : file as string; const filepath = file instanceof TFile ? file.path : file as string;
if (this.settings.useIgnoreFiles && !await this.isIgnoredByIgnoreFiles(file)) { if (this.settings.useIgnoreFiles && await this.isIgnoredByIgnoreFiles(file)) {
return false; return false;
} }
return this.localDatabase.isTargetFile(filepath); return this.localDatabase.isTargetFile(filepath);
} }
async dryRunGC() { async dryRunGC() {
await runWithLock("cleanup", true, async () => { await skipIfDuplicated("cleanup", async () => {
const remoteDBConn = await this.getReplicator().connectRemoteCouchDBWithSetting(this.settings, this.isMobile) const remoteDBConn = await this.getReplicator().connectRemoteCouchDBWithSetting(this.settings, this.isMobile)
if (typeof (remoteDBConn) == "string") { if (typeof (remoteDBConn) == "string") {
Logger(remoteDBConn); Logger(remoteDBConn);
@@ -2606,7 +2713,7 @@ Or if you are sure know what had been happened, we can unlock the database from
async dbGC() { async dbGC() {
// Lock the remote completely once. // Lock the remote completely once.
await runWithLock("cleanup", true, async () => { await skipIfDuplicated("cleanup", async () => {
this.getReplicator().markRemoteLocked(this.settings, true, true); this.getReplicator().markRemoteLocked(this.settings, true, true);
const remoteDBConn = await this.getReplicator().connectRemoteCouchDBWithSetting(this.settings, this.isMobile) const remoteDBConn = await this.getReplicator().connectRemoteCouchDBWithSetting(this.settings, this.isMobile)
if (typeof (remoteDBConn) == "string") { if (typeof (remoteDBConn) == "string") {
@@ -2621,5 +2728,41 @@ Or if you are sure know what had been happened, we can unlock the database from
Logger("The remote database has been cleaned up! Other devices will be cleaned up on the next synchronisation.") Logger("The remote database has been cleaned up! Other devices will be cleaned up on the next synchronisation.")
}); });
} }
askInPopup(key: string, dialogText: string, anchorCallback: (anchor: HTMLAnchorElement) => void) {
const fragment = createFragment((doc) => {
const [beforeText, afterText] = dialogText.split("{HERE}", 2);
doc.createEl("span", null, (a) => {
a.appendText(beforeText);
a.appendChild(a.createEl("a", null, (anchor) => {
anchorCallback(anchor);
}));
a.appendText(afterText);
});
});
const popupKey = "popup-" + key;
scheduleTask(popupKey, 1000, async () => {
const popup = await memoIfNotExist(popupKey, () => new Notice(fragment, 0));
//@ts-ignore
const isShown = popup?.noticeEl?.isShown();
if (!isShown) {
memoObject(popupKey, new Notice(fragment, 0));
}
scheduleTask(popupKey + "-close", 20000, () => {
const popup = retrieveMemoObject<Notice>(popupKey);
if (!popup)
return;
//@ts-ignore
if (popup?.noticeEl?.isShown()) {
popup.hide();
}
disposeMemoObject(popupKey);
});
});
}
} }

12
src/utils.ts
View File

@@ -3,9 +3,10 @@ import { path2id_base, id2path_base, isValidFilenameInLinux, isValidFilenameInDa
import { Logger } from "./lib/src/logger"; import { Logger } from "./lib/src/logger";
import { LOG_LEVEL_VERBOSE, type AnyEntry, type DocumentID, type EntryHasPath, type FilePath, type FilePathWithPrefix } from "./lib/src/types"; import { LOG_LEVEL_VERBOSE, type AnyEntry, type DocumentID, type EntryHasPath, type FilePath, type FilePathWithPrefix } from "./lib/src/types";
import { CHeader, ICHeader, ICHeaderLength, PSCHeader } from "./types"; import { CHeader, ICHeader, ICHeaderLength, ICXHeader, PSCHeader } from "./types";
import { InputStringDialog, PopoverSelectString } from "./dialogs"; import { InputStringDialog, PopoverSelectString } from "./dialogs";
import ObsidianLiveSyncPlugin from "./main"; import ObsidianLiveSyncPlugin from "./main";
import { writeString } from "./lib/src/strbin";
// For backward compatibility, using the path for determining id. // For backward compatibility, using the path for determining id.
// Only CouchDB unacceptable ID (that starts with an underscore) has been prefixed with "/". // Only CouchDB unacceptable ID (that starts with an underscore) has been prefixed with "/".
@@ -387,6 +388,9 @@ export function isChunk(str: string): boolean {
export function isPluginMetadata(str: string): boolean { export function isPluginMetadata(str: string): boolean {
return str.startsWith(PSCHeader); return str.startsWith(PSCHeader);
} }
export function isCustomisationSyncMetadata(str: string): boolean {
return str.startsWith(ICXHeader);
}
export const askYesNo = (app: App, message: string): Promise<"yes" | "no"> => { export const askYesNo = (app: App, message: string): Promise<"yes" | "no"> => {
return new Promise((res) => { return new Promise((res) => {
@@ -440,7 +444,7 @@ export class PeriodicProcessor {
} }
export const _requestToCouchDBFetch = async (baseUri: string, username: string, password: string, path?: string, body?: string | any, method?: string) => { export const _requestToCouchDBFetch = async (baseUri: string, username: string, password: string, path?: string, body?: string | any, method?: string) => {
const utf8str = String.fromCharCode.apply(null, new TextEncoder().encode(`${username}:${password}`)); const utf8str = String.fromCharCode.apply(null, [...writeString(`${username}:${password}`)]);
const encoded = window.btoa(utf8str); const encoded = window.btoa(utf8str);
const authHeader = "Basic " + encoded; const authHeader = "Basic " + encoded;
const transformedHeaders: Record<string, string> = { authorization: authHeader, "content-type": "application/json" }; const transformedHeaders: Record<string, string> = { authorization: authHeader, "content-type": "application/json" };
@@ -456,7 +460,7 @@ export const _requestToCouchDBFetch = async (baseUri: string, username: string,
} }
export const _requestToCouchDB = async (baseUri: string, username: string, password: string, origin: string, path?: string, body?: any, method?: string) => { export const _requestToCouchDB = async (baseUri: string, username: string, password: string, origin: string, path?: string, body?: any, method?: string) => {
const utf8str = String.fromCharCode.apply(null, new TextEncoder().encode(`${username}:${password}`)); const utf8str = String.fromCharCode.apply(null, [...writeString(`${username}:${password}`)]);
const encoded = window.btoa(utf8str); const encoded = window.btoa(utf8str);
const authHeader = "Basic " + encoded; const authHeader = "Basic " + encoded;
const transformedHeaders: Record<string, string> = { authorization: authHeader, origin: origin }; const transformedHeaders: Record<string, string> = { authorization: authHeader, origin: origin };
@@ -477,7 +481,7 @@ export const requestToCouchDB = async (baseUri: string, username: string, passwo
export async function performRebuildDB(plugin: ObsidianLiveSyncPlugin, method: "localOnly" | "remoteOnly" | "rebuildBothByThisDevice") { export async function performRebuildDB(plugin: ObsidianLiveSyncPlugin, method: "localOnly" | "remoteOnly" | "rebuildBothByThisDevice") {
if (method == "localOnly") { if (method == "localOnly") {
await plugin.addOnSetup.fetchLocal(); await plugin.addOnSetup.fetchLocalWithKeepLocal();
} }
if (method == "remoteOnly") { if (method == "remoteOnly") {
await plugin.addOnSetup.rebuildRemote(); await plugin.addOnSetup.rebuildRemote();

1
tsconfig.json
View File

@@ -15,6 +15,7 @@
// "importsNotUsedAsValues": "error", // "importsNotUsedAsValues": "error",
"importHelpers": false, "importHelpers": false,
"alwaysStrict": true, "alwaysStrict": true,
"allowImportingTsExtensions": true,
"lib": [ "lib": [
"es2018", "es2018",
"DOM", "DOM",

88
updates.md
View File

@@ -1,71 +1,43 @@
### 0.19.0 ### 0.20.0
#### Customization sync At 0.20.0, Self-hosted LiveSync has changed the binary file format and encrypting format, for efficient synchronisation.
The dialogue will be shown and asks us to decide whether to keep v1 or use v2. Once we have enabled v2, all subsequent edits will be saved in v2. Therefore, devices running 0.19 or below cannot understand this and they might say that decryption error. Please update all devices.
Then we will have an impressive performance.
Since `Plugin and their settings` have been broken, so I tried to fix it, not just fix it, but fix it the way it should be. Of course, these are very impactful changes. If you have any questions or troubled things, please feel free to open an issue and mention me.
Now, we have `Customization sync`. Note: if you want to roll it back to v1, please enable `Use binary and encryption version 1` on the `Hatch` pane and perform the `rebuild everything` once.
It is a real shame that the compatibility between these features has been broken. However, this new feature is surely useful and I believe that worth getting over the pain. Extra but notable information:
We can use the new feature with the same configuration. Only the menu on the command palette has been changed. The dialog can be opened by `Show customization sync dialog`.
I hope you will give it a try. This format change gives us the ability to detect some `marks` in the binary files as same as text files. Therefore, we can split binary files and some specific sort of them (i.e., PDF files) at the specific character. It means that editing the middle of files could be detected with marks.
Now only a few chunks are transferred, even if we add a comment to the PDF or put new files into the ZIP archives.
#### Minors #### Version history
- 0.20.2
- 0.19.1 to 0.19.11 has been moved into the updates_old.md
- 0.19.12
- Improved:
- Boot-up performance has been improved.
- Customisation sync performance has been improved.
- Synchronising performance has been improved.
- 0.19.13
- Implemented:
- Database clean-up is now in beta 2!
We can shrink the remote database by deleting unused chunks, with keeping history.
Note: Local database is not cleaned up totally. We have to `Fetch` again to let it done.
**Note2**: Still in beta. Please back your vault up anything before.
- Fixed:
- The log updates are not thinned out now.
- 0.19.14
- Fixed:
- Internal documents are now ignored.
- Merge dialogue now respond immediately to button pressing.
- Periodic processing now works fine.
- The checking interval of detecting conflicted has got shorter.
- Replication is now cancelled while cleaning up.
- The database locking by the cleaning up is now carefully unlocked.
- Missing chunks message is correctly reported.
- New feature: - New feature:
- Suspend database reflecting has been implemented. - We can delete all data of customization sync from the `Delete all customization sync data` on the `Hatch` pane.
- This can be disabled by `Fetch database with previous behaviour`.
- Now fetch suspends the reflecting database and storage changes temporarily to improve the performance.
- We can choose the action when the remote database has been cleaned
- Merge dialogue now show `↲` before the new line.
- Improved:
- Now progress is reported while the cleaning up and fetch process.
- Cancelled replication is now detected.
- 0.19.15
- Fixed: - Fixed:
- Now storing files after cleaning up is correct works. - Prevent keep restarting on iOS by yielding microtasks.
- Improved: - 0.20.1
- Cleaning the local database up got incredibly fastened.
Now we can clean instead of fetching again when synchronising with the remote which has been cleaned up.
- 0.19.16
- Many upgrades on this release. I have tried not to let that happen, if something got corrupted, please feel free to notify me.
- New feature:
- (Beta) ignore files handling
We can use `.gitignore`, `.dockerignore`, and anything you like to filter the synchronising files.
- Fixed: - Fixed:
- Buttons on lock-detected-dialogue now can be shown in narrow-width devices. - No more UI freezing and keep restarting on iOS.
- Diff of Non-markdown documents are now shown correctly.
- Improved: - Improved:
- Some constant has been flattened to be evaluated. - Performance has been a bit improved.
- The usage of the deprecated API of obsidian has been reduced. - Customization sync has gotten faster.
- Now the indexedDB adapter will be enabled while the importing configuration. - However, We lost forward compatibility again (only for this feature). Please update all devices.
- Misc: - Misc
- Compiler, framework, and dependencies have been upgraded. - Terser configuration has been more aggressive.
- Due to standing for these impacts (especially in esbuild and svelte,) terser has been introduced. - 0.20.0
Feel free to notify your opinion to me! I do not like to obfuscate the code too. - Improved:
- A New binary file handling implemented
- A new encrypted format has been implemented
- Now the chunk sizes will be adjusted for efficient sync
- Fixed:
- levels of exception in some logs have been fixed
- Tidied:
- Some Lint warnings have been suppressed.
... To continue on to `updates_old.md`. ... To continue on to `updates_old.md`.

108
updates_old.md
View File

@@ -82,6 +82,114 @@ I hope you will give it a try.
- Logging keeps 400 lines now. - Logging keeps 400 lines now.
- Refactored: - Refactored:
- Import statement has been fixed about types. - Import statement has been fixed about types.
- 0.19.12
- Improved:
- Boot-up performance has been improved.
- Customisation sync performance has been improved.
- Synchronising performance has been improved.
- 0.19.13
- Implemented:
- Database clean-up is now in beta 2!
We can shrink the remote database by deleting unused chunks, with keeping history.
Note: Local database is not cleaned up totally. We have to `Fetch` again to let it done.
**Note2**: Still in beta. Please back your vault up anything before.
- Fixed:
- The log updates are not thinned out now.
- 0.19.14
- Fixed:
- Internal documents are now ignored.
- Merge dialogue now respond immediately to button pressing.
- Periodic processing now works fine.
- The checking interval of detecting conflicted has got shorter.
- Replication is now cancelled while cleaning up.
- The database locking by the cleaning up is now carefully unlocked.
- Missing chunks message is correctly reported.
- New feature:
- Suspend database reflecting has been implemented.
- This can be disabled by `Fetch database with previous behaviour`.
- Now fetch suspends the reflecting database and storage changes temporarily to improve the performance.
- We can choose the action when the remote database has been cleaned
- Merge dialogue now show `↲` before the new line.
- Improved:
- Now progress is reported while the cleaning up and fetch process.
- Cancelled replication is now detected.
- 0.19.15
- Fixed:
- Now storing files after cleaning up is correct works.
- Improved:
- Cleaning the local database up got incredibly fastened.
Now we can clean instead of fetching again when synchronising with the remote which has been cleaned up.
- 0.19.16
- Many upgrades on this release. I have tried not to let that happen, if something got corrupted, please feel free to notify me.
- New feature:
- (Beta) ignore files handling
We can use `.gitignore`, `.dockerignore`, and anything you like to filter the synchronising files.
- Fixed:
- Buttons on lock-detected-dialogue now can be shown in narrow-width devices.
- Improved:
- Some constant has been flattened to be evaluated.
- The usage of the deprecated API of obsidian has been reduced.
- Now the indexedDB adapter will be enabled while the importing configuration.
- Misc:
- Compiler, framework, and dependencies have been upgraded.
- Due to standing for these impacts (especially in esbuild and svelte,) terser has been introduced.
Feel free to notify your opinion to me! I do not like to obfuscate the code too.
- 0.19.17
- Fixed:
- Now nested ignore files could be parsed correctly.
- The unexpected deletion of hidden files in some cases has been corrected.
- Hidden file change is no longer reflected on the device which has made the change itself.
- Behaviour changed:
- From this version, the file which has `:` in its name should be ignored even if on Linux devices.
- 0.19.18
- Fixed:
- Now the empty (or deleted) file could be conflict-resolved.
- 0.19.19
- Fixed:
- Resolving conflicted revision has become more robust.
- LiveSync now try to keep local changes when fetching from the rebuilt remote database.
Local changes now have been kept as a revision and fetched things will be new revisions.
- Now, all files will be restored after performing `fetch` immediately.
- 0.19.20
- New feature:
- `Sync on Editor save` has been implemented
- We can start synchronisation when we save from the Obsidian explicitly.
- Now we can use the `Hidden file sync` and the `Customization sync` cooperatively.
- We can exclude files from `Hidden file sync` which is already handled in Customization sync.
- We can ignore specific plugins in Customization sync.
- Now the message of leftover conflicted files accepts our click.
- We can open `Resolve all conflicted files` in an instant.
- Refactored:
- Parallelism functions made more explicit.
- Type errors have been reduced.
- Fixed:
- Now documents would not be overwritten if they are conflicted.
It will be saved as a new conflicted revision.
- Some error messages have been fixed.
- Missing dialogue titles have been shown now.
- We can click close buttons on mobile now.
- Conflicted Customisation sync files will be resolved automatically by their modified time.
- 0.19.21
- Fixed:
- Hidden files are no longer handled in the initial replication.
- Report from `Making report` fixed
- No longer contains customisation sync information.
- Version of LiveSync has been added.
- 0.19.22
- Fixed:
- Now the synchronisation will begin without our interaction.
- No longer puts the configuration of the remote database into the log while checking configuration.
- Some outdated description notes have been removed.
- Options that are meaningless depending on other settings configured are now hidden.
- Scan for hidden files before replication
- Scan customization periodically
- 0.19.23
-Improved:
- We can open the log pane also from the command palette now.
- Now, the hidden file scanning interval could be configured to 0.
- `Check database configuration` now points out that we do not have administrator permission.
### 0.18.0 ### 0.18.0
#### Now, paths of files in the database can now be obfuscated. (Experimental Feature) #### Now, paths of files in the database can now be obfuscated. (Experimental Feature)