public/obsidian-livesync
1
0
Fork 0
mirror of https://github.com/vrtmrz/obsidian-livesync.git synced 2026-03-02 07:58:48 +00:00
Code Issues Packages Projects Releases Wiki Activity

Compare commits

base: public:0.19.7
Branches Tags
public:main public:snyk-fix-34b8981b68f7111ada8b22377113a98c public:beta public:snyk-upgrade-7fd31e6f474568014502e7205847f257 public:snyk-fix-90f3b71450f7543a1c61724d8afa0e71 public:refactor_services_2 public:refactor_services public:dev public:snyk-upgrade-fe9625385aa4be849c7157033fd2abb0 public:snyk-upgrade-99a31e70c822894595544b0c9813b11a public:svelteui public:disenchant public:snyk-fix-61f76ef0bce2a276df9552c2fc80a14f public:snyk-upgrade-2baa7938ba318464c3bad5c3fd5cbda5 public:snyk-upgrade-7af62a575eab20e8fb4e9565bf1adcfc public:snyk-upgrade-b32f281b31a81483f126b8fb6c54a957 public:snyk-upgrade-bd61b1103442d3faecc9fc726c34e9a4 public:snyk-upgrade-5b2d6584430c061b5187611a5d7fb64a public:snyk-fix-0e9fbbcb374fcb8d5e8cb027d8414fb7 public:snyk-upgrade-5242daf91e706d1ae500c9ea6545636e public:snyk-upgrade-70434a6822fbad83424135dfad9df909 public:snyk-upgrade-8fa01ccad0352125f1866d41e57d35e1 public:snyk-upgrade-39eb97b4655e70091f718a88fccd57c1 public:snyk-upgrade-2b99d54d19e207887e36c3071b796925 public:forprivacy public:snyk-upgrade-dad9088d0a333677e0157e3deef7fc38 public:snyk-upgrade-b9a76d74b3e5723759af5b3319aabb80 public:snyk-upgrade-7da6459d8ff5d293d80de9d80061d912 public:snyk-upgrade-6a3786fc4cbe394542e9d1b6058c46e7 public:snyk-upgrade-8fc9e62771dc41b0f23dc0efefb5c949
public:0.25.47 public:0.25.46 public:0.25.45 public:0.25.44 public:0.25.43-patched-9 public:0.25.43-patched-8 public:0.25.43-patched-7 public:0.25.43-patched-6 public:0.25.43-patched-5 public:0.25.43-patched-4 public:0.25.43-patched-3 public:0.25.43-patched-2 public:0.25.43-patched-1 public:0.25.43 public:0.25.42 public:0.25.41-patched-3 public:0.25.41-patched-2 public:0.25.41-patched-1 public:0.25.41 public:0.25.40 public:0.25.39 public:0.25.38 public:0.25.37 public:0.25.36 public:0.25.35 public:0.25.34 public:0.25.33 public:0.25.32 public:0.25.31 public:0.25.30 public:0.25.30-dev public:0.25.29 public:0.25.28 public:0.25.27 public:0.25.26 public:0.25.25 public:0.25.24 public:0.25.24.beta3 public:0.25.24.beta2 public:0.25.24.beta1 public:0.25.23 public:0.25.23.beta1 public:0.25.22 public:0.25.21 public:0.25.21.beta2 public:0.25.21.beta1 public:0.25.20 public:0.25.19 public:0.25.18 public:0.25.17 public:0.25.16 public:0.25.15 public:0.25.14 public:0.25.13 public:0.25.12 public:0.25.11 public:0.25.10 public:0.25.9 public:0.25.8 public:0.25.7 public:0.25.6 public:0.25.5 public:0.25.4 public:0.25.3 public:0.25.2 public:0.25.1 public:0.25.0 public:0.25.0-beta1 public:0.24.31 public:0.24.30 public:0.24.29 public:0.24.28 public:0.24.27 public:0.24.26 public:0.24.25 public:0.24.24 public:0.24.23 public:0.24.22 public:0.24.21 public:0.24.20 public:0.24.19 public:0.24.18 public:0.24.17 public:0.24.16 public:0.24.15 public:0.24.14 public:0.24.13 public:0.24.12 public:0.24.11 public:0.24.10 public:0.24.9 public:0.24.8 public:0.24.7 public:0.24.6 public:0.24.5 public:0.24.4 public:0.24.3 public:0.24.2 public:0.24.1 public:0.24.0 public:0.24.0.dev-rc8 public:0.24.0.dev-rc7 public:0.24.0.dev-rc6 public:0.24.0.dev-rc5 public:0.24.0.dev-rc4 public:0.24.0.dev-rc3 public:0.24.0.dev-rc2 public:0.23.23 public:0.23.22 public:0.23.21 public:0.23.20 public:0.23.19 public:0.23.18 public:0.23.17 public:0.23.16 public:0.23.15 public:0.23.14 public:0.23.13 public:0.23.12 public:0.23.11 public:0.23.10 public:0.23.9 public:0.23.8 public:0.23.7 public:0.23.6 public:0.23.5 public:0.23.4 public:0.23.3 public:0.23.2 public:0.23.1 public:0.23.0 public:0.22.19 public:0.22.18 public:0.22.17 public:0.22.16 public:0.22.15 public:0.22.14 public:0.22.13 public:0.22.12 public:0.22.11 public:0.22.10 public:0.22.9 public:0.22.8 public:0.22.7 public:0.22.6 public:0.22.5 public:0.22.4 public:0.22.3 public:0.22.2 public:0.22.1 public:0.22.0 public:0.21.5 public:0.21.4 public:0.21.3 public:0.21.2 public:0.21.1 public:0.21.0 public:0.20.7 public:0.20.6 public:0.20.5 public:0.20.4 public:0.20.3 public:0.20.2 public:0.20.1 public:0.20.0 public:0.19.23 public:0.19.22 public:0.19.21 public:0.19.20 public:0.19.19 public:0.19.18 public:0.19.17 public:0.19.16 public:0.19.15 public:0.19.14 public:0.19.13 public:0.19.12 public:0.19.11 public:0.19.10 public:0.19.9 public:0.19.8 public:0.19.7 public:0.19.6 public:0.19.5 public:0.19.4 public:0.19.3 public:0.19.2 public:0.19.1 public:0.19.0 public:0.18.6 public:0.18.5 public:0.18.4 public:0.18.3 public:0.18.2 public:0.18.1 public:0.18.0 public:0.17.34 public:0.17.33 public:0.17.32 public:0.17.31 public:0.17.30 public:0.17.29 public:0.17.28 public:0.17.27 public:0.17.26 public:0.17.25 public:0.17.24 public:0.17.23 public:0.17.22 public:0.17.21 public:0.17.20 public:0.17.19 public:0.17.18 public:0.17.17 public:0.17.16 public:0.17.15 public:0.17.14 public:0.17.13 public:0.17.12 public:0.17.11 public:0.17.10 public:0.17.9 public:0.17.8 public:0.17.7 public:0.17.6 public:0.17.5 public:0.17.4 public:0.17.3 public:0.17.2 public:0.17.1 public:0.17.0 public:0.16.8 public:0.16.7 public:0.16.6 public:0.16.5 public:0.16.4 public:0.16.3 public:0.16.2 public:0.16.1 public:0.16.0 public:0.15.10 public:0.15.9 public:0.15.8 public:0.15.7 public:0.15.6 public:0.15.5 public:0.15.4 public:0.15.3 public:0.15.2 public:0.15.1 public:0.15.0 public:0.14.8 public:0.14.7 public:0.14.6 public:0.14.5 public:0.14.4 public:0.14.3 public:0.14.2 public:0.14.1 public:0.14.0 public:0.13.4 public:0.13.3 public:0.13.2 public:0.13.1 public:0.13.0 public:0.12.5 public:0.12.4 public:0.12.3 public:0.12.2 public:0.12.1 public:0.12.0 public:0.11.10 public:0.11.9 public:0.11.8 public:0.11.7 public:0.11.6 public:0.11.5 public:0.11.4 public:0.11.3 public:0.11.2 public:0.11.1 public:0.11.0 public:0.10.1 public:0.10.0 public:0.9.0 public:0.8.8 public:0.8.7 public:0.8.6 public:0.8.5 public:0.8.3 public:0.8.2 public:0.8.1 public:0.8.0 public:0.7.2 public:0.7.1 public:0.7.0 public:0.6.1 public:0.6.0 public:0.5.0 public:0.4.1 public:0.4.0 public:0.3.1 public:0.3.0 public:0.2.1 public:0.2.0 public:0.1.26 public:0.1.25 public:0.1.24 public:0.1.23 public:0.1.22 public:0.1.21 public:0.1.20 public:0.1.19 public:0.1.18 public:0.1.17 public:0.1.16 public:0.1.15 public:0.1.14 public:0.1.13 public:0.1.12 public:0.1.11 public:0.1.10 public:0.1.9 public:0.1.8 public:0.1.7 public:0.1.6 public:0.1.5 public:0.1.4 public:0.1.3 public:0.1.2 public:0.1.1 public:0.1.0 public:0.0.9 public:0.0.8 public:0.0.7 public:0.0.6
...
compare: public:0.19.12
Branches Tags
public:main public:snyk-fix-34b8981b68f7111ada8b22377113a98c public:beta public:snyk-upgrade-7fd31e6f474568014502e7205847f257 public:snyk-fix-90f3b71450f7543a1c61724d8afa0e71 public:refactor_services_2 public:refactor_services public:dev public:snyk-upgrade-fe9625385aa4be849c7157033fd2abb0 public:snyk-upgrade-99a31e70c822894595544b0c9813b11a public:svelteui public:disenchant public:snyk-fix-61f76ef0bce2a276df9552c2fc80a14f public:snyk-upgrade-2baa7938ba318464c3bad5c3fd5cbda5 public:snyk-upgrade-7af62a575eab20e8fb4e9565bf1adcfc public:snyk-upgrade-b32f281b31a81483f126b8fb6c54a957 public:snyk-upgrade-bd61b1103442d3faecc9fc726c34e9a4 public:snyk-upgrade-5b2d6584430c061b5187611a5d7fb64a public:snyk-fix-0e9fbbcb374fcb8d5e8cb027d8414fb7 public:snyk-upgrade-5242daf91e706d1ae500c9ea6545636e public:snyk-upgrade-70434a6822fbad83424135dfad9df909 public:snyk-upgrade-8fa01ccad0352125f1866d41e57d35e1 public:snyk-upgrade-39eb97b4655e70091f718a88fccd57c1 public:snyk-upgrade-2b99d54d19e207887e36c3071b796925 public:forprivacy public:snyk-upgrade-dad9088d0a333677e0157e3deef7fc38 public:snyk-upgrade-b9a76d74b3e5723759af5b3319aabb80 public:snyk-upgrade-7da6459d8ff5d293d80de9d80061d912 public:snyk-upgrade-6a3786fc4cbe394542e9d1b6058c46e7 public:snyk-upgrade-8fc9e62771dc41b0f23dc0efefb5c949
public:0.25.47 public:0.25.46 public:0.25.45 public:0.25.44 public:0.25.43-patched-9 public:0.25.43-patched-8 public:0.25.43-patched-7 public:0.25.43-patched-6 public:0.25.43-patched-5 public:0.25.43-patched-4 public:0.25.43-patched-3 public:0.25.43-patched-2 public:0.25.43-patched-1 public:0.25.43 public:0.25.42 public:0.25.41-patched-3 public:0.25.41-patched-2 public:0.25.41-patched-1 public:0.25.41 public:0.25.40 public:0.25.39 public:0.25.38 public:0.25.37 public:0.25.36 public:0.25.35 public:0.25.34 public:0.25.33 public:0.25.32 public:0.25.31 public:0.25.30 public:0.25.30-dev public:0.25.29 public:0.25.28 public:0.25.27 public:0.25.26 public:0.25.25 public:0.25.24 public:0.25.24.beta3 public:0.25.24.beta2 public:0.25.24.beta1 public:0.25.23 public:0.25.23.beta1 public:0.25.22 public:0.25.21 public:0.25.21.beta2 public:0.25.21.beta1 public:0.25.20 public:0.25.19 public:0.25.18 public:0.25.17 public:0.25.16 public:0.25.15 public:0.25.14 public:0.25.13 public:0.25.12 public:0.25.11 public:0.25.10 public:0.25.9 public:0.25.8 public:0.25.7 public:0.25.6 public:0.25.5 public:0.25.4 public:0.25.3 public:0.25.2 public:0.25.1 public:0.25.0 public:0.25.0-beta1 public:0.24.31 public:0.24.30 public:0.24.29 public:0.24.28 public:0.24.27 public:0.24.26 public:0.24.25 public:0.24.24 public:0.24.23 public:0.24.22 public:0.24.21 public:0.24.20 public:0.24.19 public:0.24.18 public:0.24.17 public:0.24.16 public:0.24.15 public:0.24.14 public:0.24.13 public:0.24.12 public:0.24.11 public:0.24.10 public:0.24.9 public:0.24.8 public:0.24.7 public:0.24.6 public:0.24.5 public:0.24.4 public:0.24.3 public:0.24.2 public:0.24.1 public:0.24.0 public:0.24.0.dev-rc8 public:0.24.0.dev-rc7 public:0.24.0.dev-rc6 public:0.24.0.dev-rc5 public:0.24.0.dev-rc4 public:0.24.0.dev-rc3 public:0.24.0.dev-rc2 public:0.23.23 public:0.23.22 public:0.23.21 public:0.23.20 public:0.23.19 public:0.23.18 public:0.23.17 public:0.23.16 public:0.23.15 public:0.23.14 public:0.23.13 public:0.23.12 public:0.23.11 public:0.23.10 public:0.23.9 public:0.23.8 public:0.23.7 public:0.23.6 public:0.23.5 public:0.23.4 public:0.23.3 public:0.23.2 public:0.23.1 public:0.23.0 public:0.22.19 public:0.22.18 public:0.22.17 public:0.22.16 public:0.22.15 public:0.22.14 public:0.22.13 public:0.22.12 public:0.22.11 public:0.22.10 public:0.22.9 public:0.22.8 public:0.22.7 public:0.22.6 public:0.22.5 public:0.22.4 public:0.22.3 public:0.22.2 public:0.22.1 public:0.22.0 public:0.21.5 public:0.21.4 public:0.21.3 public:0.21.2 public:0.21.1 public:0.21.0 public:0.20.7 public:0.20.6 public:0.20.5 public:0.20.4 public:0.20.3 public:0.20.2 public:0.20.1 public:0.20.0 public:0.19.23 public:0.19.22 public:0.19.21 public:0.19.20 public:0.19.19 public:0.19.18 public:0.19.17 public:0.19.16 public:0.19.15 public:0.19.14 public:0.19.13 public:0.19.12 public:0.19.11 public:0.19.10 public:0.19.9 public:0.19.8 public:0.19.7 public:0.19.6 public:0.19.5 public:0.19.4 public:0.19.3 public:0.19.2 public:0.19.1 public:0.19.0 public:0.18.6 public:0.18.5 public:0.18.4 public:0.18.3 public:0.18.2 public:0.18.1 public:0.18.0 public:0.17.34 public:0.17.33 public:0.17.32 public:0.17.31 public:0.17.30 public:0.17.29 public:0.17.28 public:0.17.27 public:0.17.26 public:0.17.25 public:0.17.24 public:0.17.23 public:0.17.22 public:0.17.21 public:0.17.20 public:0.17.19 public:0.17.18 public:0.17.17 public:0.17.16 public:0.17.15 public:0.17.14 public:0.17.13 public:0.17.12 public:0.17.11 public:0.17.10 public:0.17.9 public:0.17.8 public:0.17.7 public:0.17.6 public:0.17.5 public:0.17.4 public:0.17.3 public:0.17.2 public:0.17.1 public:0.17.0 public:0.16.8 public:0.16.7 public:0.16.6 public:0.16.5 public:0.16.4 public:0.16.3 public:0.16.2 public:0.16.1 public:0.16.0 public:0.15.10 public:0.15.9 public:0.15.8 public:0.15.7 public:0.15.6 public:0.15.5 public:0.15.4 public:0.15.3 public:0.15.2 public:0.15.1 public:0.15.0 public:0.14.8 public:0.14.7 public:0.14.6 public:0.14.5 public:0.14.4 public:0.14.3 public:0.14.2 public:0.14.1 public:0.14.0 public:0.13.4 public:0.13.3 public:0.13.2 public:0.13.1 public:0.13.0 public:0.12.5 public:0.12.4 public:0.12.3 public:0.12.2 public:0.12.1 public:0.12.0 public:0.11.10 public:0.11.9 public:0.11.8 public:0.11.7 public:0.11.6 public:0.11.5 public:0.11.4 public:0.11.3 public:0.11.2 public:0.11.1 public:0.11.0 public:0.10.1 public:0.10.0 public:0.9.0 public:0.8.8 public:0.8.7 public:0.8.6 public:0.8.5 public:0.8.3 public:0.8.2 public:0.8.1 public:0.8.0 public:0.7.2 public:0.7.1 public:0.7.0 public:0.6.1 public:0.6.0 public:0.5.0 public:0.4.1 public:0.4.0 public:0.3.1 public:0.3.0 public:0.2.1 public:0.2.0 public:0.1.26 public:0.1.25 public:0.1.24 public:0.1.23 public:0.1.22 public:0.1.21 public:0.1.20 public:0.1.19 public:0.1.18 public:0.1.17 public:0.1.16 public:0.1.15 public:0.1.14 public:0.1.13 public:0.1.12 public:0.1.11 public:0.1.10 public:0.1.9 public:0.1.8 public:0.1.7 public:0.1.6 public:0.1.5 public:0.1.4 public:0.1.3 public:0.1.2 public:0.1.1 public:0.1.0 public:0.0.9 public:0.0.8 public:0.0.7 public:0.0.6

17 Commits
0.19.7 ... 0.19.12

Author SHA1 Message Date
vorotamoroz
8a75d41cbb bump 2023-07-03 18:45:05 +09:00
vorotamoroz
5252cc0372 Improved: 
- Boot-up performance has been improved.
- Customisation sync performance has been improved.
- Synchronising performance has been improved.
 2023-07-03 18:44:02 +09:00
vorotamoroz
5022155317 Fix documentation 2023-06-19 16:31:52 +09:00
vorotamoroz
d36f925c65 bump 2023-06-15 18:13:43 +09:00
vorotamoroz
3ae33e0500 Refactored: External dependency merged. 2023-06-15 18:07:11 +09:00
vorotamoroz
13e442a0c7 Improvements: 
- Hashing ChunkID has been improved.
- Logging keeps 400 lines now.

Refactored:
- Import statement has been fixed about types.
 2023-06-15 17:55:58 +09:00
vorotamoroz
6288716966 - Fixed 
- Fixed the issue about fixing the database.
 2023-06-09 19:28:29 +09:00
vorotamoroz
47d2cf9733 bump 2023-06-09 18:48:59 +09:00
vorotamoroz
ae6a9ecee4 - New feature (For fixing a problem): 
- We can fix the database obfuscated and plain paths that have been mixed up.
- Improvements
  - Customisation Sync performance has been improved.
 2023-06-09 18:48:10 +09:00
vorotamoroz
2289bea8d9 bump 2023-06-07 17:32:28 +09:00
vorotamoroz
cda90259c5 - New feature: 
- Vault history: A tab has been implemented to give a birds-eye view of the changes that have occurred in the vault.
- Improved:
  - Log dialogue is now shown as one of tabs.
- Fixed:
  - Some minor issues has been fixed.
 2023-06-07 17:29:53 +09:00
vorotamoroz
432a211f80 Merge pull request #224 from antoKeinanen/main 
[Feature] Add password protection to askString function
 2023-06-07 17:04:31 +09:00
antoKeinanen
eaf8c4998e feat: add password protection for required inputs 2023-06-05 13:27:06 +03:00
antoKeinanen
55601f7910 feat: add option for password protection in askString function 2023-06-05 13:24:50 +03:00
vorotamoroz
13e70475d9 Add new documentation 
Thanks for your discussion!!
 2023-06-02 14:36:00 +09:00
vorotamoroz
2572177879 Merge pull request #222 from Hugo-Persson/add-troubleshooting-guide 
Added troubleshooting guide
 2023-06-02 09:25:18 +09:00
Hugo Persson
e82a2560e4 Added troubleshooting guide 2023-06-01 19:54:42 +02:00
26 changed files with 1169 additions and 188 deletions
Expand all files Collapse all files

18
README.md
View File

@@ -34,11 +34,10 @@ Useful for researchers, engineers and developers with a need to keep their notes
### Get your database ready.
First, get your database ready. IBM Cloudant is preferred for testing. Or you can use your own server with CouchDB. For more information, refer below:
1. [Setup IBM Cloudant](docs/setup_cloudant.md)
2. [Setup your CouchDB](docs/setup_own_server.md)
Note: More information about alternative hosting methods is needed! Currently, [using fly.io](https://github.com/vrtmrz/obsidian-livesync/discussions/85) is being discussed.
First, get your database ready. fly.io is preferred for testing. Or you can use your own server with CouchDB. For more information, refer below:
1. [Setup fly.io](docs/setup_flyio.md)
2. [Setup IBM Cloudant](docs/setup_cloudant.md)
3. [Setup your CouchDB](docs/setup_own_server.md)
### Configure the plugin
@@ -54,12 +53,6 @@ Please open the configuration link again and Answer below:
If you answered `No` to both, your databases will be rebuilt by the content on your device. And the remote database will lock out other devices. You have to synchronize all your devices again. (When this time, almost all your files should be synchronized with a timestamp. So you can use an existing vault).
## Test Server
~~Setting up an instance of Cloudant or local CouchDB is a little complicated, so I set up a [Tasting server for self-hosted-livesync](https://olstaste.vrtmrz.net/). Try it out for free!~~
Now (30 May 2023) suspending while the server transfer.
Note: Please read "Limitations" carefully. Do not send your private vault.
## Information in StatusBar
Synchronization status is shown in statusbar.
@@ -94,6 +87,9 @@ If you have deleted or renamed files, please wait until ⏳ icon disappeared.
Repo is here: [obsidian-livesync-webclip](https://github.com/vrtmrz/obsidian-livesync-webclip). (Docs are a work in progress.)
## Troubleshooting
If you are having problems getting the plugin working see: [Troubleshooting](docs/troubleshooting.md)
## License
The source code is licensed under the MIT License.

287
docs/setup_flyio.md Normal file
View File

@@ -0,0 +1,287 @@
# Setup CouchDB on fly.io
In some cases, the use of IBM Cloudant was found to be hard. We looked for alternatives, but there were no services available. Therefore, we have to build our own servers, which is quite a challenge. In this situation, with fly.io, most of the processes are simplified and only CouchDB needs to be configured.
This is how to configure fly.io and CouchDB on it for Self-hosted LiveSync.
It generally falls within the `Free Allowances` range in most cases.
**[Automatic setup using Colaboratory](#automatic-setup-using-colaboratory) is recommended, after reading this document through. It is reproducible and hard to fail.**
## Required materials
- A valid credit or debit card.
## Warning
- It will be `your` instance. Check the log regularly.
## Prerequisites
For simplicity, the following description assumes that the settings are as shown in the table below. Please read it in accordance with the actual settings you wish to make.
| Used in the text | Meaning and where to use | Memo |
| ---------------- | --------------------------- | ------------------------------------------------------------------------ |
| campanella | Username | It is less likely to fail if it consists only of letters and numbers. |
| dfusiuada9suy | Password | |
| nrt | Region to make the instance | We can use any [region](https://fly.io/docs/reference/regions/) near us. |
## Steps with your computer
If you want to avoid installing anything, please skip to [Automatic setup using Colaboratory](#automatic-setup-using-colaboratory).
### 1. Install flyctl
- Mac or Linux
```sh
$ curl -L https://fly.io/install.sh | sh
```
- Windows
```powershell
$ iwr https://fly.io/install.ps1 -useb | iex
```
### 2. Sign up or Sign in to fly.io
- Sign up
```bash
$ fly auth signup
```
- Sign in
```bash
$ fly auth login
```
For more information, please refer [Sign up](https://fly.io/docs/hands-on/sign-up/) and [Sign in](https://fly.io/docs/hands-on/sign-in/).
### 3. Make configuration files
Please be careful, `nrt` is the region where near to Japan. Please use your preferred region.
1. Make fly.toml
```
$ flyctl launch --generate-name --detach --no-deploy --region nrt
Creating app in /home/vrtmrz/dev/fly/demo
Scanning source code
Could not find a Dockerfile, nor detect a runtime or framework from source code. Continuing with a blank app.
automatically selected personal organization: vorotamoroz
App will use 'nrt' region as primary
Created app 'billowing-dawn-6619' in organization 'personal'
Admin URL: https://fly.io/apps/billowing-dawn-6619
Hostname: billowing-dawn-6619.fly.dev
Wrote config file fly.toml
```
`billowing-dawn-6619` is an automatically generated name. It is used as the hostname. Please note it in something.
Note: we can specify this without `--generate-name`, but does not recommend in the trial phases.
1. Make volume
```
$ flyctl volumes create --region nrt couchdata --size 2 --yes
ID: vol_g67340kxgmmvydxw
Name: couchdata
App: billowing-dawn-6619
Region: nrt
Zone: 35b7
Size GB: 2
Encrypted: true
Created at: 02 Jun 23 01:19 UTC
```
3. Edit fly.toml
Changes:
- Change exposing port from `8080` to `5984`
- Mounting the volume `couchdata` created in step 2 under `/opt/couchdb/data`
- Set `campanella` for the administrator of CouchDB
- Customise CouchDB to use persistent ini-file; which is located under the data directory.
- To use Dockerfile
```diff
# fly.toml app configuration file generated for billowing-dawn-6619 on 2023-06-02T10:18:59+09:00
#
# See https://fly.io/docs/reference/configuration/ for information about how to use this file.
#
app = "billowing-dawn-6619"
primary_region = "nrt"
[http_service]
- internal_port = 8080
+ internal_port = 5984
force_https = true
auto_stop_machines = true
auto_start_machines = true
min_machines_running = 0
+[mounts]
+ source="couchdata"
+ destination="/opt/couchdb/data"
+
+[env]
+ COUCHDB_USER = "campanella"
+ ERL_FLAGS="-couch_ini /opt/couchdb/etc/default.ini /opt/couchdb/etc/default.d/ /opt/couchdb/etc/local.d /opt/couchdb/etc/local.ini /opt/couchdb/data/persistence.ini"
+
+[build]
+ dockerfile = "./Dockerfile"
```
4. Make `Dockerfile`
Create a Dockerfile that patches the start-up script to fix ini file permissions.
```dockerfile
FROM couchdb:latest
RUN sed -i '2itouch /opt/couchdb/data/persistence.ini && chmod +w /opt/couchdb/data/persistence.ini' /docker-entrypoint.sh
```
5. Set credential
```
flyctl secrets set COUCHDB_PASSWORD=dfusiuada9suy
```
### 4. Deploy
```
$ flyctl deploy --detach --remote-only
==> Verifying app config
Validating /home/vrtmrz/dev/fly/demo/fly.toml
Platform: machines
✓ Configuration is valid
--> Verified app config
==> Building image
Remote builder fly-builder-bold-sky-4515 ready
==> Creating build context
--> Creating build context done
==> Building image with Docker
--> docker host: 20.10.12 linux x86_64
-------------:SNIPPED:-------------
Watch your app at https://fly.io/apps/billowing-dawn-6619/monitoring
Provisioning ips for billowing-dawn-6619
Dedicated ipv6: 2a09:8280:1::2d:240f
Shared ipv4: 66.241.125.213
Add a dedicated ipv4 with: fly ips allocate-v4
This deployment will:
* create 1 "app" machine
No machines in group app, launching a new machine
Machine e7845d1f297183 [app] has state: started
Finished launching new machines
NOTE: The machines for [app] have services with 'auto_stop_machines = true' that will be stopped when idling
```
Now your CouchDB has been launched. (Do not forget to delete it if no longer need).
If failed, please check by `flyctl doctor`. Failure of remote build may be resolved by `flyctl` wireguard reset` or something.
```
$ flyctl status
App
Name = billowing-dawn-6619
Owner = personal
Hostname = billowing-dawn-6619.fly.dev
Image = billowing-dawn-6619:deployment-01H1WWB3CK5Z9ZX71KHBSDGHF1
Platform = machines
Machines
PROCESS ID VERSION REGION STATE CHECKS LAST UPDATED
app e7845d1f297183 1 nrt started 2023-06-02T01:43:34Z
```
### 5. Apply CouchDB configuration
After the initial setup, CouchDB needs some more customisations to be used from Self-hosted LiveSync. It can be configured in browsers or by HTTP-REST APIs.
This section is set up using the REST API.
1. Prepare environment variables.
- Mac or Linux:
```bash
export couchHost=https://billowing-dawn-6619.fly.dev
export couchUser=campanella
export couchPwd=dfusiuada9suy
```
- Windows
```powershell
set couchHost https://billowing-dawn-6619.fly.dev
set couchUser campanella
set couchPwd dfusiuada9suy
$creds = [System.Convert]::ToBase64String([System.Text.Encoding]::UTF8.GetBytes("${couchUser}:${couchPwd}"))
```
2. Perform cluster setup
- Mac or Linux
```bash
curl -X POST "${couchHost}/_cluster_setup" -H "Content-Type: application/json" -d "{\"action\":\"enable_single_node\",\"username\":\"${couchUser}\",\"password\":\"${couchPwd}\",\"bind_address\":\"0.0.0.0\",\"port\":5984,\"singlenode\":true}" --user "${couchUser}:${couchPwd}"
```
- Windows
```powershell
iwr -UseBasicParsing -Method 'POST' -ContentType 'application/json; charset=utf-8' -Headers @{ 'Authorization' = 'Basic ' + $creds } "${couchHost}/_cluster_setup" -Body "{""action"":""enable_single_node"",""username"":""${couchUser}"",""password"":""${couchPwd}"",""bind_address"":""0.0.0.0"",""port"":5984,""singlenode"":true}"
```
Note: if the response code is not 200. We have to retry the request once again.
If you run the request several times and it does not result in 200, something is wrong. Please report it.
3. Configure parameters
- Mac or Linux
```bash
curl -X PUT "${couchHost}/_node/nonode@nohost/_config/chttpd/require_valid_user" -H "Content-Type: application/json" -d '"true"' --user "${couchUser}:${couchPwd}"
curl -X PUT "${couchHost}/_node/nonode@nohost/_config/chttpd_auth/require_valid_user" -H "Content-Type: application/json" -d '"true"' --user "${couchUser}:${couchPwd}"
curl -X PUT "${couchHost}/_node/nonode@nohost/_config/httpd/WWW-Authenticate" -H "Content-Type: application/json" -d '"Basic realm=\"couchdb\""' --user "${couchUser}:${couchPwd}"
curl -X PUT "${couchHost}/_node/nonode@nohost/_config/httpd/enable_cors" -H "Content-Type: application/json" -d '"true"' --user "${couchUser}:${couchPwd}"
curl -X PUT "${couchHost}/_node/nonode@nohost/_config/chttpd/enable_cors" -H "Content-Type: application/json" -d '"true"' --user "${couchUser}:${couchPwd}"
curl -X PUT "${couchHost}/_node/nonode@nohost/_config/chttpd/max_http_request_size" -H "Content-Type: application/json" -d '"4294967296"' --user "${couchUser}:${couchPwd}"
curl -X PUT "${couchHost}/_node/nonode@nohost/_config/couchdb/max_document_size" -H "Content-Type: application/json" -d '"50000000"' --user "${couchUser}:${couchPwd}"
curl -X PUT "${couchHost}/_node/nonode@nohost/_config/cors/credentials" -H "Content-Type: application/json" -d '"true"' --user "${couchUser}:${couchPwd}"
curl -X PUT "${couchHost}/_node/nonode@nohost/_config/cors/origins" -H "Content-Type: application/json" -d '"app://obsidian.md,capacitor://localhost,http://localhost"' --user "${couchUser}:${couchPwd}"
```
- Windows
```powershell
iwr -UseBasicParsing -Method 'PUT' -ContentType 'application/json; charset=utf-8' -Headers @{ 'Authorization' = 'Basic ' + $creds } "${couchHost}/_node/nonode@nohost/_config/chttpd/require_valid_user" -Body '"true"'
iwr -UseBasicParsing -Method 'PUT' -ContentType 'application/json; charset=utf-8' -Headers @{ 'Authorization' = 'Basic ' + $creds } "${couchHost}/_node/nonode@nohost/_config/chttpd_auth/require_valid_user" -Body '"true"'
iwr -UseBasicParsing -Method 'PUT' -ContentType 'application/json; charset=utf-8' -Headers @{ 'Authorization' = 'Basic ' + $creds } "${couchHost}/_node/nonode@nohost/_config/httpd/WWW-Authenticate" -Body '"Basic realm=\"couchdb\""'
iwr -UseBasicParsing -Method 'PUT' -ContentType 'application/json; charset=utf-8' -Headers @{ 'Authorization' = 'Basic ' + $creds } "${couchHost}/_node/nonode@nohost/_config/httpd/enable_cors" -Body '"true"'
iwr -UseBasicParsing -Method 'PUT' -ContentType 'application/json; charset=utf-8' -Headers @{ 'Authorization' = 'Basic ' + $creds } "${couchHost}/_node/nonode@nohost/_config/chttpd/enable_cors" -Body '"true"'
iwr -UseBasicParsing -Method 'PUT' -ContentType 'application/json; charset=utf-8' -Headers @{ 'Authorization' = 'Basic ' + $creds } "${couchHost}/_node/nonode@nohost/_config/chttpd/max_http_request_size" -Body '"4294967296"'
iwr -UseBasicParsing -Method 'PUT' -ContentType 'application/json; charset=utf-8' -Headers @{ 'Authorization' = 'Basic ' + $creds } "${couchHost}/_node/nonode@nohost/_config/couchdb/max_document_size" -Body '"50000000"'
iwr -UseBasicParsing -Method 'PUT' -ContentType 'application/json; charset=utf-8' -Headers @{ 'Authorization' = 'Basic ' + $creds } "${couchHost}/_node/nonode@nohost/_config/cors/credentials" -Body '"true"'
iwr -UseBasicParsing -Method 'PUT' -ContentType 'application/json; charset=utf-8' -Headers @{ 'Authorization' = 'Basic ' + $creds } "${couchHost}/_node/nonode@nohost/_config/cors/origins" -Body '"app://obsidian.md,capacitor://localhost,http://localhost"'
```
Note: Each of these should also be repeated until finished in 200.
### 6. Use it from Self-hosted LiveSync
Now the CouchDB is ready to use from Self-hosted LiveSync. We can use `https://billowing-dawn-6619.fly.dev` in URI, `campanella` in `Username` and `dfusiuada9suy` in `Password` on Self-hosted LiveSync. `Database name` could be anything you want.
`Enhance chunk size` could be up to around `100`.
## Automatic setup using Colaboratory
We can perform all these steps by using [this Colaboratory notebook](https://gist.github.com/vrtmrz/b437a539af25ef191bd452aae369242f) without installing anything.
## After testing / before creating a new instance
**Be sure to delete the instance. We can check instances on the [Dashboard](https://fly.io/dashboard/personal)**

2
docs/setup_own_server_ja.md
View File

@@ -12,10 +12,10 @@ max_document_size = 50000000
[chttpd]
require_valid_user = true
max_http_request_size = 4294967296
[chttpd_auth]
require_valid_user = true
max_http_request_size = 4294967296
authentication_redirect = /_utils/session.html
[httpd]

12
docs/troubleshooting.md Normal file
View File

@@ -0,0 +1,12 @@
# Info
In this document some tips for solving issues will be given.
# Issue with syncing with mobile device
- If you are getting problem like could not sync files when doing initial sync. Try lower batch size
- Open setting
- Open Live Sync settings
- Go to sync settings
- Go down to Advanced settings.
- Lower "Batch size" and "Batch limit" and try to sync again.
- If you keep getting error keep lowering until you find the sweet spot.

2
manifest.json
View File

@@ -1,7 +1,7 @@
{
"id": "obsidian-livesync",
"name": "Self-hosted LiveSync",
"version": "0.19.7",
"version": "0.19.12",
"minAppVersion": "0.9.12",
"description": "Community implementation of self-hosted livesync. Reflect your vault changes to some other devices immediately. Please make sure to disable other synchronize solutions to avoid content corruption or duplication.",
"author": "vorotamoroz",

18
package-lock.json generated
View File

@@ -1,17 +1,18 @@
{
"name": "obsidian-livesync",
"version": "0.19.7",
"version": "0.19.12",
"lockfileVersion": 2,
"requires": true,
"packages": {
"": {
"name": "obsidian-livesync",
"version": "0.19.7",
"version": "0.19.12",
"license": "MIT",
"dependencies": {
"diff-match-patch": "^1.0.5",
"idb": "^7.1.1",
"xxhash-wasm": "^0.4.2"
"xxhash-wasm": "^0.4.2",
"xxhash-wasm-102": "npm:xxhash-wasm@^1.0.2"
},
"devDependencies": {
"@tsconfig/svelte": "^4.0.1",
@@ -4076,6 +4077,12 @@
"resolved": "https://registry.npmjs.org/xxhash-wasm/-/xxhash-wasm-0.4.2.tgz",
"integrity": "sha512-/eyHVRJQCirEkSZ1agRSCwriMhwlyUcFkXD5TPVSLP+IPzjsqMVzZwdoczLp1SoQU0R3dxz1RpIK+4YNQbCVOA=="
},
"node_modules/xxhash-wasm-102": {
"name": "xxhash-wasm",
"version": "1.0.2",
"resolved": "https://registry.npmjs.org/xxhash-wasm/-/xxhash-wasm-1.0.2.tgz",
"integrity": "sha512-ibF0Or+FivM9lNrg+HGJfVX8WJqgo+kCLDc4vx6xMeTce7Aj+DLttKbxxRR/gNLSAelRc1omAPlJ77N/Jem07A=="
},
"node_modules/yallist": {
"version": "4.0.0",
"resolved": "https://registry.npmjs.org/yallist/-/yallist-4.0.0.tgz",
@@ -7032,6 +7039,11 @@
"resolved": "https://registry.npmjs.org/xxhash-wasm/-/xxhash-wasm-0.4.2.tgz",
"integrity": "sha512-/eyHVRJQCirEkSZ1agRSCwriMhwlyUcFkXD5TPVSLP+IPzjsqMVzZwdoczLp1SoQU0R3dxz1RpIK+4YNQbCVOA=="
},
"xxhash-wasm-102": {
"version": "npm:xxhash-wasm@1.0.2",
"resolved": "https://registry.npmjs.org/xxhash-wasm/-/xxhash-wasm-1.0.2.tgz",
"integrity": "sha512-ibF0Or+FivM9lNrg+HGJfVX8WJqgo+kCLDc4vx6xMeTce7Aj+DLttKbxxRR/gNLSAelRc1omAPlJ77N/Jem07A=="
},
"yallist": {
"version": "4.0.0",
"resolved": "https://registry.npmjs.org/yallist/-/yallist-4.0.0.tgz",

5
package.json
View File

@@ -1,6 +1,6 @@
{
"name": "obsidian-livesync",
"version": "0.19.7",
"version": "0.19.12",
"description": "Reflect your vault changes to some other devices immediately. Please make sure to disable other synchronize solutions to avoid content corruption or duplication.",
"main": "main.js",
"type": "module",
@@ -47,6 +47,7 @@
"dependencies": {
"diff-match-patch": "^1.0.5",
"idb": "^7.1.1",
"xxhash-wasm": "^0.4.2"
"xxhash-wasm": "^0.4.2",
"xxhash-wasm-102": "npm:xxhash-wasm@^1.0.2"
}
}

100
src/CmdConfigSync.ts
View File

@@ -7,14 +7,14 @@ import { ICXHeader, PERIODIC_PLUGIN_SWEEP, } from "./types";
import { Parallels, delay, getDocData } from "./lib/src/utils";
import { Logger } from "./lib/src/logger";
import { WrappedNotice } from "./lib/src/wrapper";
import { base64ToArrayBuffer, arrayBufferToBase64, readString, uint8ArrayToHexString } from "./lib/src/strbin";
import { base64ToArrayBuffer, arrayBufferToBase64, readString, crc32CKHash } from "./lib/src/strbin";
import { runWithLock } from "./lib/src/lock";
import { LiveSyncCommands } from "./LiveSyncCommands";
import { stripAllPrefixes } from "./lib/src/path";
import { PeriodicProcessor, askYesNo, disposeMemoObject, memoIfNotExist, memoObject, retrieveMemoObject, scheduleTask } from "./utils";
import { Semaphore } from "./lib/src/semaphore";
import { PluginDialogModal } from "./dialogs";
import { JsonResolveModal } from "./JsonResolveModal";
import { pipeGeneratorToGenerator, processAllGeneratorTasksWithConcurrencyLimit } from './lib/src/task';
function serialize<T>(obj: T): string {
@@ -36,14 +36,6 @@ function deserialize<T>(str: string, def: T) {
export const pluginList = writable([] as PluginDataExDisplay[]);
export const pluginIsEnumerating = writable(false);
const encoder = new TextEncoder();
const hashString = (async (key: string) => {
// const buff = writeString(key);
const buff = encoder.encode(key);
const digest = await crypto.subtle.digest('SHA-256', buff);
return uint8ArrayToHexString(new Uint8Array(digest));
})
export type PluginDataExFile = {
filename: string,
data?: string[],
@@ -180,34 +172,25 @@ export class ConfigSync extends LiveSyncCommands {
pluginList.set(this.pluginList)
return;
}
await runWithLock("update-plugin-list", false, async () => {
// if (updatedDocumentPath != "") pluginList.update(e => e.filter(ee => ee.documentPath != updatedDocumentPath));
// const work: Record<string, Record<string, Record<string, Record<string, PluginDataEntryEx>>>> = {};
const entries = [] as PluginDataExDisplay[]
const plugins = this.localDatabase.findEntries(ICXHeader + "", `${ICXHeader}\u{10ffff}`, { include_docs: true });
const semaphore = Semaphore(4);
const para = Parallels();
let count = 0;
pluginIsEnumerating.set(true);
let processed = false;
try {
for await (const plugin of plugins) {
const path = plugin.path || this.getPath(plugin);
if (updatedDocumentPath && updatedDocumentPath != path) {
continue;
}
processed = true;
const oldEntry = (this.pluginList.find(e => e.documentPath == path));
if (oldEntry && oldEntry.mtime == plugin.mtime) continue;
await para.wait(5);
para.add((async (v) => {
const release = await semaphore.acquire(1);
scheduleTask("update-plugin-list-task", 200, async () => {
await runWithLock("update-plugin-list", false, async () => {
try {
const updatedDocumentId = updatedDocumentPath ? await this.path2id(updatedDocumentPath) : "";
const plugins = updatedDocumentPath ?
this.localDatabase.findEntries(updatedDocumentId, updatedDocumentId + "\u{10ffff}", { include_docs: true, key: updatedDocumentId, limit: 1 }) :
this.localDatabase.findEntries(ICXHeader + "", `${ICXHeader}\u{10ffff}`, { include_docs: true });
let count = 0;
pluginIsEnumerating.set(true);
for await (const v of processAllGeneratorTasksWithConcurrencyLimit(20, pipeGeneratorToGenerator(plugins, async plugin => {
const path = plugin.path || this.getPath(plugin);
if (updatedDocumentPath && updatedDocumentPath != path) {
return false;
}
const oldEntry = (this.pluginList.find(e => e.documentPath == path));
if (oldEntry && oldEntry.mtime == plugin.mtime) return false;
try {
count++;
if (count % 10 == 0) Logger(`Enumerating files... ${count}`, logLevel, "get-plugins");
Logger(`plugin-${path}`, LOG_LEVEL.VERBOSE);
const wx = await this.localDatabase.getDBEntry(path, null, false, false);
if (wx) {
@@ -216,41 +199,40 @@ export class ConfigSync extends LiveSyncCommands {
for (const file of data.files) {
const work = { ...file };
const tempStr = getDocData(work.data);
work.data = [await hashString(tempStr)];
work.data = [crc32CKHash(tempStr)];
xFiles.push(work);
}
entries.push({
return ({
...data,
documentPath: this.getPath(wx),
files: xFiles
});
}
// return entries;
} catch (ex) {
//TODO
Logger(`Something happened at enumerating customization :${v.path}`, LOG_LEVEL.NOTICE);
Logger(`Something happened at enumerating customization :${path}`, LOG_LEVEL.NOTICE);
console.warn(ex);
} finally {
release();
}
return false;
}))) {
if ("ok" in v) {
if (v.ok != false) {
let newList = [...this.pluginList];
const item = v.ok;
newList = newList.filter(x => x.documentPath != item.documentPath);
newList.push(item)
if (updatedDocumentPath != "") newList = newList.filter(e => e.documentPath != updatedDocumentPath);
this.pluginList = newList;
pluginList.set(newList);
}
}
}
)(plugin));
Logger(`All files enumerated`, logLevel, "get-plugins");
} finally {
pluginIsEnumerating.set(false);
}
await para.all();
let newList = [...this.pluginList];
for (const item of entries) {
newList = newList.filter(x => x.documentPath != item.documentPath);
newList.push(item)
}
if (updatedDocumentPath != "" && !processed) newList = newList.filter(e => e.documentPath != updatedDocumentPath);
this.pluginList = newList;
pluginList.set(newList);
Logger(`All files enumerated`, logLevel, "get-plugins");
} finally {
pluginIsEnumerating.set(false);
}
});
});
// return entries;
}
@@ -482,6 +464,10 @@ export class ConfigSync extends LiveSyncCommands {
}
async storeCustomizationFiles(path: FilePath, termOverRide?: string) {
const term = termOverRide || this.plugin.deviceAndVaultName;
if (term == "") {
Logger("We have to configure the device name", LOG_LEVEL.NOTICE);
return;
}
const vf = this.filenameToUnifiedKey(path, term);
return await runWithLock(`plugin-${vf}`, false, async () => {
const category = this.getFileCategory(path);

6
src/CmdHiddenFileSync.ts
View File

@@ -1,6 +1,6 @@
import { Notice, normalizePath, PluginManifest } from "./deps";
import { EntryDoc, LoadedEntry, LOG_LEVEL, InternalFileEntry, FilePathWithPrefix, FilePath } from "./lib/src/types";
import { InternalFileInfo, ICHeader, ICHeaderEnd } from "./types";
import { Notice, normalizePath, type PluginManifest } from "./deps";
import { type EntryDoc, type LoadedEntry, LOG_LEVEL, type InternalFileEntry, type FilePathWithPrefix, type FilePath } from "./lib/src/types";
import { type InternalFileInfo, ICHeader, ICHeaderEnd } from "./types";
import { Parallels, delay, isDocContentSame } from "./lib/src/utils";
import { Logger } from "./lib/src/logger";
import { PouchDB } from "./lib/src/pouchdb-browser.js";

6
src/CmdSetupLiveSync.ts
View File

@@ -41,7 +41,7 @@ export class SetupLiveSync extends LiveSyncCommands {
async realizeSettingSyncMode() { }
async command_copySetupURI() {
const encryptingPassphrase = await askString(this.app, "Encrypt your settings", "The passphrase to encrypt the setup URI", "");
const encryptingPassphrase = await askString(this.app, "Encrypt your settings", "The passphrase to encrypt the setup URI", "", true);
if (encryptingPassphrase === false)
return;
const setting = { ...this.settings, configPassphraseStore: "", encryptedCouchDBConnection: "", encryptedPassphrase: "" };
@@ -57,7 +57,7 @@ export class SetupLiveSync extends LiveSyncCommands {
Logger("Setup URI copied to clipboard", LOG_LEVEL.NOTICE);
}
async command_copySetupURIFull() {
const encryptingPassphrase = await askString(this.app, "Encrypt your settings", "The passphrase to encrypt the setup URI", "");
const encryptingPassphrase = await askString(this.app, "Encrypt your settings", "The passphrase to encrypt the setup URI", "", true);
if (encryptingPassphrase === false)
return;
const setting = { ...this.settings, configPassphraseStore: "", encryptedCouchDBConnection: "", encryptedPassphrase: "" };
@@ -81,7 +81,7 @@ export class SetupLiveSync extends LiveSyncCommands {
async setupWizard(confString: string) {
try {
const oldConf = JSON.parse(JSON.stringify(this.settings));
const encryptingPassphrase = await askString(this.app, "Passphrase", "The passphrase to decrypt your setup URI", "");
const encryptingPassphrase = await askString(this.app, "Passphrase", "The passphrase to decrypt your setup URI", "", true);
if (encryptingPassphrase === false)
return;
const newConf = await JSON.parse(await decrypt(confString, encryptingPassphrase, false));

32
src/DocumentHistoryModal.ts
View File

@@ -1,9 +1,8 @@
import { TFile, Modal, App } from "./deps";
import { TFile, Modal, App, DIFF_DELETE, DIFF_EQUAL, DIFF_INSERT, diff_match_patch } from "./deps";
import { getPathFromTFile, isValidPath } from "./utils";
import { base64ToArrayBuffer, base64ToString, escapeStringToHTML } from "./lib/src/strbin";
import ObsidianLiveSyncPlugin from "./main";
import { DIFF_DELETE, DIFF_EQUAL, DIFF_INSERT, diff_match_patch } from "diff-match-patch";
import { DocumentID, FilePathWithPrefix, LoadedEntry, LOG_LEVEL } from "./lib/src/types";
import { type DocumentID, type FilePathWithPrefix, type LoadedEntry, LOG_LEVEL } from "./lib/src/types";
import { Logger } from "./lib/src/logger";
import { isErrorOfMissingDoc } from "./lib/src/utils_couchdb";
import { getDocData } from "./lib/src/utils";
@@ -24,12 +23,14 @@ export class DocumentHistoryModal extends Modal {
currentDoc: LoadedEntry;
currentText = "";
currentDeleted = false;
initialRev: string;
constructor(app: App, plugin: ObsidianLiveSyncPlugin, file: TFile | FilePathWithPrefix, id: DocumentID) {
constructor(app: App, plugin: ObsidianLiveSyncPlugin, file: TFile | FilePathWithPrefix, id: DocumentID, revision?: string) {
super(app);
this.plugin = plugin;
this.file = (file instanceof TFile) ? getPathFromTFile(file) : file;
this.id = id;
this.initialRev = revision;
if (!file) {
this.file = this.plugin.id2path(id, null);
}
@@ -38,7 +39,7 @@ export class DocumentHistoryModal extends Modal {
}
}
async loadFile() {
async loadFile(initialRev: string) {
if (!this.id) {
this.id = await this.plugin.path2id(this.file);
}
@@ -49,7 +50,7 @@ export class DocumentHistoryModal extends Modal {
this.range.max = `${this.revs_info.length - 1}`;
this.range.value = this.range.max;
this.fileInfo.setText(`${this.file} / ${this.revs_info.length} revisions`);
await this.loadRevs();
await this.loadRevs(initialRev);
} catch (ex) {
if (isErrorOfMissingDoc(ex)) {
this.range.max = "0";
@@ -63,18 +64,27 @@ export class DocumentHistoryModal extends Modal {
}
}
}
async loadRevs() {
async loadRevs(initialRev?: string) {
if (this.revs_info.length == 0) return;
const db = this.plugin.localDatabase;
if (initialRev) {
const rIndex = this.revs_info.findIndex(e => e.rev == initialRev);
if (rIndex >= 0) {
this.range.value = `${this.revs_info.length - 1 - rIndex}`;
}
}
const index = this.revs_info.length - 1 - (this.range.value as any) / 1;
const rev = this.revs_info[index];
const w = await db.getDBEntry(this.file, { rev: rev.rev }, false, false, true);
await this.showExactRev(rev.rev);
}
async showExactRev(rev: string) {
const db = this.plugin.localDatabase;
const w = await db.getDBEntry(this.file, { rev: rev }, false, false, true);
this.currentText = "";
this.currentDeleted = false;
if (w === false) {
this.currentDeleted = true;
this.info.innerHTML = "";
this.contentView.innerHTML = `Could not read this revision<br>(${rev.rev})`;
this.contentView.innerHTML = `Could not read this revision<br>(${rev})`;
} else {
this.currentDoc = w;
this.info.innerHTML = `Modified:${new Date(w.mtime).toLocaleString()}`;
@@ -158,7 +168,7 @@ export class DocumentHistoryModal extends Modal {
.addClass("op-info");
this.info = contentEl.createDiv("");
this.info.addClass("op-info");
this.loadFile();
this.loadFile(this.initialRev);
const div = contentEl.createDiv({ text: "Loading old revisions..." });
this.contentView = div;
div.addClass("op-scrollable");

328
src/GlobalHistory.svelte Normal file
View File

@@ -0,0 +1,328 @@
<script lang="ts">
import ObsidianLiveSyncPlugin from "./main";
import { onDestroy, onMount } from "svelte";
import type { AnyEntry, FilePathWithPrefix } from "./lib/src/types";
import { getDocData, isDocContentSame } from "./lib/src/utils";
import { diff_match_patch } from "./deps";
import { DocumentHistoryModal } from "./DocumentHistoryModal";
import { isPlainText, stripAllPrefixes } from "./lib/src/path";
import { TFile } from "./deps";
import { arrayBufferToBase64 } from "./lib/src/strbin";
export let plugin: ObsidianLiveSyncPlugin;
let showDiffInfo = false;
let showChunkCorrected = false;
let checkStorageDiff = false;
let range_from_epoch = Date.now() - 3600000 * 24 * 7;
let range_to_epoch = Date.now() + 3600000 * 24 * 2;
const timezoneOffset = new Date().getTimezoneOffset();
let dispDateFrom = new Date(range_from_epoch - timezoneOffset).toISOString().split("T")[0];
let dispDateTo = new Date(range_to_epoch - timezoneOffset).toISOString().split("T")[0];
$: {
range_from_epoch = new Date(dispDateFrom).getTime() + timezoneOffset;
range_to_epoch = new Date(dispDateTo).getTime() + timezoneOffset;
getHistory(showDiffInfo, showChunkCorrected, checkStorageDiff);
}
function mtimeToDate(mtime: number) {
return new Date(mtime).toLocaleString();
}
type HistoryData = {
id: string;
rev: string;
path: string;
dirname: string;
filename: string;
mtime: number;
mtimeDisp: string;
isDeleted: boolean;
size: number;
changes: string;
chunks: string;
isPlain: boolean;
};
let history = [] as HistoryData[];
let loading = false;
async function fetchChanges(): Promise<HistoryData[]> {
try {
const db = plugin.localDatabase;
let result = [] as typeof history;
for await (const docA of db.findAllNormalDocs()) {
if (docA.mtime < range_from_epoch) {
continue;
}
if (docA.type != "newnote" && docA.type != "plain") continue;
const path = plugin.getPath(docA as AnyEntry);
const isPlain = isPlainText(docA.path);
const revs = await db.getRaw(docA._id, { revs_info: true });
let p: string = undefined;
const reversedRevs = revs._revs_info.reverse();
const DIFF_DELETE = -1;
const DIFF_EQUAL = 0;
const DIFF_INSERT = 1;
for (const revInfo of reversedRevs) {
if (revInfo.status == "available") {
const doc =
(!isPlain && showDiffInfo) || (checkStorageDiff && revInfo.rev == docA._rev)
? await db.getDBEntry(path, { rev: revInfo.rev }, false, false, true)
: await db.getDBEntryMeta(path, { rev: revInfo.rev }, true);
if (doc === false) continue;
const rev = revInfo.rev;
const mtime = "mtime" in doc ? doc.mtime : 0;
if (range_from_epoch > mtime) {
continue;
}
if (range_to_epoch < mtime) {
continue;
}
let diffDetail = "";
if (showDiffInfo && !isPlain) {
const data = getDocData(doc.data);
if (p === undefined) {
p = data;
}
if (p != data) {
const dmp = new diff_match_patch();
const diff = dmp.diff_main(p, data);
dmp.diff_cleanupSemantic(diff);
p = data;
const pxinit = {
[DIFF_DELETE]: 0,
[DIFF_EQUAL]: 0,
[DIFF_INSERT]: 0,
} as { [key: number]: number };
const px = diff.reduce((p, c) => ({ ...p, [c[0]]: (p[c[0]] ?? 0) + c[1].length }), pxinit);
diffDetail = `-${px[DIFF_DELETE]}, +${px[DIFF_INSERT]}`;
}
}
const isDeleted = doc._deleted || (doc as any)?.deleted || false;
if (isDeleted) {
diffDetail += " 🗑️";
}
if (rev == docA._rev) {
if (checkStorageDiff) {
const abs = plugin.app.vault.getAbstractFileByPath(stripAllPrefixes(plugin.getPath(docA)));
if (abs instanceof TFile) {
let result = false;
if (isPlainText(docA.path)) {
const data = await plugin.app.vault.read(abs);
result = isDocContentSame(data, doc.data);
} else {
const data = await plugin.app.vault.readBinary(abs);
const dataEEncoded = await arrayBufferToBase64(data);
result = isDocContentSame(dataEEncoded, doc.data);
}
if (result) {
diffDetail += " ⚖️";
} else {
diffDetail += " ⚠️";
}
}
}
}
const docPath = plugin.getPath(doc as AnyEntry);
const [filename, ...pathItems] = docPath.split("/").reverse();
let chunksStatus = "";
if (showChunkCorrected) {
const chunks = (doc as any)?.children ?? [];
const loadedChunks = await db.allDocsRaw({ keys: [...chunks] });
const totalCount = loadedChunks.rows.length;
const errorCount = loadedChunks.rows.filter((e) => "error" in e).length;
if (errorCount == 0) {
chunksStatus = `✅ ${totalCount}`;
} else {
chunksStatus = `🔎 ${errorCount} ✅ ${totalCount}`;
}
}
result.push({
id: doc._id,
rev: doc._rev,
path: docPath,
dirname: pathItems.reverse().join("/"),
filename: filename,
mtime: mtime,
mtimeDisp: mtimeToDate(mtime),
size: (doc as any)?.size ?? 0,
isDeleted: isDeleted,
changes: diffDetail,
chunks: chunksStatus,
isPlain: isPlain,
});
}
}
}
return [...result].sort((a, b) => b.mtime - a.mtime);
} finally {
loading = false;
}
}
async function getHistory(showDiffInfo: boolean, showChunkCorrected: boolean, checkStorageDiff: boolean) {
loading = true;
const newDisplay = [];
const page = await fetchChanges();
newDisplay.push(...page);
history = [...newDisplay];
}
function nextWeek() {
dispDateTo = new Date(range_to_epoch - timezoneOffset + 3600 * 1000 * 24 * 7).toISOString().split("T")[0];
}
function prevWeek() {
dispDateFrom = new Date(range_from_epoch - timezoneOffset - 3600 * 1000 * 24 * 7).toISOString().split("T")[0];
}
onMount(async () => {
await getHistory(showDiffInfo, showChunkCorrected, checkStorageDiff);
});
onDestroy(() => {});
function showHistory(file: string, rev: string) {
new DocumentHistoryModal(plugin.app, plugin, file as unknown as FilePathWithPrefix, null, rev).open();
}
function openFile(file: string) {
plugin.app.workspace.openLinkText(file, file);
}
</script>
<div class="globalhistory">
<h1>Vault history</h1>
<div class="control">
<div class="row"><label for="">From:</label><input type="date" bind:value={dispDateFrom} disabled={loading} /></div>
<div class="row"><label for="">To:</label><input type="date" bind:value={dispDateTo} disabled={loading} /></div>
<div class="row">
<label for="">Info:</label>
<label><input type="checkbox" bind:checked={showDiffInfo} disabled={loading} /><span>Diff</span></label>
<label><input type="checkbox" bind:checked={showChunkCorrected} disabled={loading} /><span>Chunks</span></label>
<label><input type="checkbox" bind:checked={checkStorageDiff} disabled={loading} /><span>File integrity</span></label>
</div>
</div>
{#if loading}
<div class="">Gathering information...</div>
{/if}
<table>
<tr>
<th> Date </th>
<th> Path </th>
<th> Rev </th>
<th> Stat </th>
{#if showChunkCorrected}
<th> Chunks </th>
{/if}
</tr>
<tr>
<td colspan="5" class="more">
{#if loading}
<div class="" />
{:else}
<div><button on:click={() => nextWeek()}>+1 week</button></div>
{/if}
</td>
</tr>
{#each history as entry}
<tr>
<td class="mtime">
{entry.mtimeDisp}
</td>
<td class="path">
<div class="filenames">
<span class="path">/{entry.dirname.split("/").join(`​/`)}</span>
<span class="filename"><a on:click={() => openFile(entry.path)}>{entry.filename}</a></span>
</div>
</td>
<td>
<span class="rev">
{#if entry.isPlain}
<a on:click={() => showHistory(entry.path, entry.rev)}>{entry.rev}</a>
{:else}
{entry.rev}
{/if}
</span>
</td>
<td>
{entry.changes}
</td>
{#if showChunkCorrected}
<td>
{entry.chunks}
</td>
{/if}
</tr>
{/each}
<tr>
<td colspan="5" class="more">
{#if loading}
<div class="" />
{:else}
<div><button on:click={() => prevWeek()}>+1 week</button></div>
{/if}
</td>
</tr>
</table>
</div>
<style>
* {
box-sizing: border-box;
}
.globalhistory {
margin-bottom: 2em;
}
table {
width: 100%;
}
.more > div {
display: flex;
}
.more > div > button {
flex-grow: 1;
}
th {
position: sticky;
top: 0;
backdrop-filter: blur(10px);
}
td.mtime {
white-space: break-spaces;
}
td.path {
word-break: break-word;
}
.row {
display: flex;
flex-direction: row;
flex-wrap: wrap;
}
.row > label {
display: flex;
align-items: center;
min-width: 5em;
}
.row > input {
flex-grow: 1;
}
.filenames {
display: flex;
flex-direction: column;
}
.filenames > .path {
font-size: 70%;
}
.rev {
text-overflow: ellipsis;
max-width: 3em;
display: inline-block;
overflow: hidden;
white-space: nowrap;
}
</style>

47
src/GlobalHistoryView.ts Normal file
View File

@@ -0,0 +1,47 @@
import {
ItemView,
WorkspaceLeaf
} from "./deps";
import GlobalHistoryComponent from "./GlobalHistory.svelte";
import type ObsidianLiveSyncPlugin from "./main";
export const VIEW_TYPE_GLOBAL_HISTORY = "global-history";
export class GlobalHistoryView extends ItemView {
component: GlobalHistoryComponent;
plugin: ObsidianLiveSyncPlugin;
icon: "clock";
title: string;
navigation: true;
getIcon(): string {
return "clock";
}
constructor(leaf: WorkspaceLeaf, plugin: ObsidianLiveSyncPlugin) {
super(leaf);
this.plugin = plugin;
}
getViewType() {
return VIEW_TYPE_GLOBAL_HISTORY;
}
getDisplayText() {
return "Vault history";
}
async onOpen() {
this.component = new GlobalHistoryComponent({
target: this.contentEl,
props: {
plugin: this.plugin,
},
});
}
async onClose() {
this.component.$destroy();
}
}

2
src/JsonResolvePane.svelte
View File

@@ -1,5 +1,5 @@
<script lang="ts">
import { type Diff, DIFF_DELETE, DIFF_INSERT, diff_match_patch } from "diff-match-patch";
import { type Diff, DIFF_DELETE, DIFF_INSERT, diff_match_patch } from "./deps";
import type { FilePath, LoadedEntry } from "./lib/src/types";
import { base64ToString } from "./lib/src/strbin";
import { getDocData } from "./lib/src/utils";

81
src/LogPane.svelte Normal file
View File

@@ -0,0 +1,81 @@
<script lang="ts">
import { onDestroy, onMount } from "svelte";
import { logMessageStore } from "./lib/src/stores";
let unsubscribe: () => void;
let messages = [] as string[];
let wrapRight = false;
let autoScroll = true;
let suspended = false;
onMount(async () => {
unsubscribe = logMessageStore.observe((e) => {
if (!suspended) {
messages = [...e];
if (autoScroll) {
if (scroll) scroll.scrollTop = scroll.scrollHeight;
}
}
});
logMessageStore.invalidate();
setTimeout(() => {
if (scroll) scroll.scrollTop = scroll.scrollHeight;
}, 100);
});
onDestroy(() => {
if (unsubscribe) unsubscribe();
});
let scroll: HTMLDivElement;
</script>
<div class="logpane">
<!-- <h1>Self-hosted LiveSync Log</h1> -->
<div class="control">
<div class="row">
<label><input type="checkbox" bind:checked={wrapRight} /><span>Wrap</span></label>
<label><input type="checkbox" bind:checked={autoScroll} /><span>Auto scroll</span></label>
<label><input type="checkbox" bind:checked={suspended} /><span>Pause</span></label>
</div>
</div>
<div class="log" bind:this={scroll}>
{#each messages as line}
<pre class:wrap-right={wrapRight}>{line}</pre>
{/each}
</div>
</div>
<style>
* {
box-sizing: border-box;
}
.logpane {
display: flex;
height: 100%;
flex-direction: column;
}
.log {
overflow-y: scroll;
user-select: text;
padding-bottom: 2em;
}
.log > pre {
margin: 0;
}
.log > pre.wrap-right {
word-break: break-all;
max-width: 100%;
width: 100%;
white-space: normal;
}
.row {
display: flex;
flex-direction: row;
justify-content: flex-end;
}
.row > label {
display: flex;
align-items: center;
min-width: 5em;
margin-right: 1em;
}
</style>

46
src/LogPaneView.ts Normal file
View File

@@ -0,0 +1,46 @@
import {
ItemView,
WorkspaceLeaf
} from "obsidian";
import LogPaneComponent from "./LogPane.svelte";
import type ObsidianLiveSyncPlugin from "./main";
export const VIEW_TYPE_LOG = "log-log";
// Show notes as like scroll.
export class LogPaneView extends ItemView {
component: LogPaneComponent;
plugin: ObsidianLiveSyncPlugin;
icon: "view-log";
title: string;
navigation: true;
getIcon(): string {
return "view-log";
}
constructor(leaf: WorkspaceLeaf, plugin: ObsidianLiveSyncPlugin) {
super(leaf);
this.plugin = plugin;
}
getViewType() {
return VIEW_TYPE_LOG;
}
getDisplayText() {
return "Self-hosted LiveSync Log";
}
async onOpen() {
this.component = new LogPaneComponent({
target: this.contentEl,
props: {
},
});
}
async onClose() {
this.component.$destroy();
}
}

88
src/ObsidianLiveSyncSettingTab.ts
View File

@@ -1,5 +1,5 @@
import { App, PluginSettingTab, Setting, sanitizeHTMLToDom, TextAreaComponent, MarkdownRenderer, stringifyYaml } from "./deps";
import { DEFAULT_SETTINGS, LOG_LEVEL, type ObsidianLiveSyncSettings, type ConfigPassphraseStore, type RemoteDBSettings } from "./lib/src/types";
import { DEFAULT_SETTINGS, LOG_LEVEL, type ObsidianLiveSyncSettings, type ConfigPassphraseStore, type RemoteDBSettings, type FilePathWithPrefix, type HashAlgorithm, type DocumentID } from "./lib/src/types";
import { delay } from "./lib/src/utils";
import { Semaphore } from "./lib/src/semaphore";
import { versionNumberString2Number } from "./lib/src/strbin";
@@ -111,7 +111,7 @@ export class ObsidianLiveSyncSettingTab extends PluginSettingTab {
}
MarkdownRenderer.renderMarkdown(updateInformation, informationDivEl, "/", null);
MarkdownRenderer.renderMarkdown(updateInformation, informationDivEl, "/", this.plugin);
addScreenElement("100", containerInformationEl);
@@ -1536,7 +1536,73 @@ ${stringifyYaml(pluginConfig)}`;
Logger("done", LOG_LEVEL.NOTICE, "verify");
})
);
new Setting(containerHatchEl)
.setName("Check and convert non-path-obfuscated files")
.setDesc("")
.addButton((button) =>
button
.setButtonText("Perform")
.setDisabled(false)
.setWarning()
.onClick(async () => {
for await (const docName of this.plugin.localDatabase.findAllDocNames()) {
if (!docName.startsWith("f:")) {
const idEncoded = await this.plugin.path2id(docName as FilePathWithPrefix);
const doc = await this.plugin.localDatabase.getRaw(docName as DocumentID);
if (!doc) continue;
if (doc.type != "newnote" && doc.type != "plain") {
continue;
}
if (doc?.deleted ?? false) continue;
const newDoc = { ...doc };
//Prepare converted data
newDoc._id = idEncoded;
newDoc.path = docName as FilePathWithPrefix;
delete newDoc._rev;
try {
const obfuscatedDoc = await this.plugin.localDatabase.getRaw(idEncoded, { revs_info: true });
// Unfortunately we have to delete one of them.
// Just now, save it as a conflicted document.
obfuscatedDoc._revs_info?.shift(); // Drop latest revision.
const previousRev = obfuscatedDoc._revs_info?.shift(); // Use second revision.
if (previousRev) {
newDoc._rev = previousRev.rev;
} else {
//If there are no revisions, set the possibly unique one
newDoc._rev = "1-" + (`00000000000000000000000000000000${~~(Math.random() * 1e9)}${~~(Math.random() * 1e9)}${~~(Math.random() * 1e9)}${~~(Math.random() * 1e9)}`.slice(-32));
}
const ret = await this.plugin.localDatabase.putRaw(newDoc, { force: true });
if (ret.ok) {
Logger(`${docName} has been converted as conflicted document`, LOG_LEVEL.NOTICE);
doc._deleted = true;
if ((await this.plugin.localDatabase.putRaw(doc)).ok) {
Logger(`Old ${docName} has been deleted`, LOG_LEVEL.NOTICE);
}
await this.plugin.showIfConflicted(docName as FilePathWithPrefix);
} else {
Logger(`Converting ${docName} Failed!`, LOG_LEVEL.NOTICE);
Logger(ret, LOG_LEVEL.VERBOSE);
}
} catch (ex) {
if (ex?.status == 404) {
// We can perform this safely
if ((await this.plugin.localDatabase.putRaw(newDoc)).ok) {
Logger(`${docName} has been converted`, LOG_LEVEL.NOTICE);
doc._deleted = true;
if ((await this.plugin.localDatabase.putRaw(doc)).ok) {
Logger(`Old ${docName} has been deleted`, LOG_LEVEL.NOTICE);
}
}
} else {
Logger(`Something went wrong on converting ${docName}`, LOG_LEVEL.NOTICE);
Logger(ex, LOG_LEVEL.VERBOSE);
// Something wrong.
}
}
}
}
Logger(`Converting finished`, LOG_LEVEL.NOTICE);
}));
new Setting(containerHatchEl)
.setName("Suspend file watching")
.setDesc("Stop watching for file change.")
@@ -1649,6 +1715,22 @@ ${stringifyYaml(pluginConfig)}`;
await this.plugin.initializeDatabase();
})
})
new Setting(containerHatchEl)
.setName("The Hash algorithm for chunk IDs")
.setDesc("xxhash64 is the current default.")
.setClass("wizardHidden")
.addDropdown((dropdown) =>
dropdown
.addOptions({ "": "Old Algorithm", "xxhash32": "xxhash32 (Fast)", "xxhash64": "xxhash64 (Fastest)" } as Record<HashAlgorithm, string>)
.setValue(this.plugin.settings.hashAlg)
.onChange(async (value: HashAlgorithm) => {
this.plugin.settings.hashAlg = value;
await this.plugin.saveSettings();
})
)
.setClass("wizardHidden");
addScreenElement("50", containerHatchEl);

9
src/deps.ts
View File

@@ -1,12 +1,13 @@
import { FilePath } from "./lib/src/types";
import { type FilePath } from "./lib/src/types";
export {
addIcon, App, DataWriteOptions, debounce, Editor, FuzzySuggestModal, MarkdownRenderer, MarkdownView, Modal, Notice, Platform, Plugin, PluginManifest,
PluginSettingTab, Plugin_2, requestUrl, RequestUrlParam, RequestUrlResponse, sanitizeHTMLToDom, Setting, stringifyYaml, TAbstractFile, TextAreaComponent, TFile, TFolder,
parseYaml
addIcon, App, debounce, Editor, FuzzySuggestModal, MarkdownRenderer, MarkdownView, Modal, Notice, Platform, Plugin, PluginSettingTab, Plugin_2, requestUrl, sanitizeHTMLToDom, Setting, stringifyYaml, TAbstractFile, TextAreaComponent, TFile, TFolder,
parseYaml, ItemView, WorkspaceLeaf
} from "obsidian";
export type { DataWriteOptions, PluginManifest, RequestUrlParam, RequestUrlResponse } from "obsidian";
import {
normalizePath as normalizePath_
} from "obsidian";
const normalizePath = normalizePath_ as <T extends string | FilePath>(from: T) => T;
export { normalizePath }
export { type Diff, DIFF_DELETE, DIFF_EQUAL, DIFF_INSERT, diff_match_patch } from "diff-match-patch";

6
src/dialogs.ts
View File

@@ -43,13 +43,15 @@ export class InputStringDialog extends Modal {
key: string;
placeholder: string;
isManuallyClosed = false;
isPassword = false;
constructor(app: App, title: string, key: string, placeholder: string, onSubmit: (result: string | false) => void) {
constructor(app: App, title: string, key: string, placeholder: string, isPassword: boolean, onSubmit: (result: string | false) => void) {
super(app);
this.onSubmit = onSubmit;
this.title = title;
this.placeholder = placeholder;
this.key = key;
this.isPassword = isPassword;
}
onOpen() {
@@ -58,7 +60,7 @@ export class InputStringDialog extends Modal {
contentEl.createEl("h1", { text: this.title });
// For enter to submit
const formEl = contentEl.createEl("form");
new Setting(formEl).setName(this.key).addText((text) =>
new Setting(formEl).setName(this.key).setClass(this.isPassword ? "password-input" : "normal-input").addText((text) =>
text.onChange((value) => {
this.result = value;
})

2
src/lib

Submodule src/lib updated: 63fa0074fe...aacf942453

120
src/main.ts
View File

@@ -1,13 +1,12 @@
const isDebug = false;
import { type Diff, DIFF_DELETE, DIFF_EQUAL, DIFF_INSERT, diff_match_patch } from "diff-match-patch";
import { type Diff, DIFF_DELETE, DIFF_EQUAL, DIFF_INSERT, diff_match_patch } from "./deps";
import { debounce, Notice, Plugin, TFile, addIcon, TFolder, normalizePath, TAbstractFile, Editor, MarkdownView, type RequestUrlParam, type RequestUrlResponse, requestUrl } from "./deps";
import { type EntryDoc, type LoadedEntry, type ObsidianLiveSyncSettings, type diff_check_result, type diff_result_leaf, type EntryBody, LOG_LEVEL, VER, DEFAULT_SETTINGS, type diff_result, FLAGMD_REDFLAG, SYNCINFO_ID, SALT_OF_PASSPHRASE, type ConfigPassphraseStore, type CouchDBConnection, FLAGMD_REDFLAG2, FLAGMD_REDFLAG3, PREFIXMD_LOGFILE, type DatabaseConnectingStatus, type EntryHasPath, type DocumentID, type FilePathWithPrefix, type FilePath, type AnyEntry } from "./lib/src/types";
import { type InternalFileInfo, type queueItem, type CacheData, type FileEventItem, FileWatchEventQueueMax } from "./types";
import { getDocData, isDocContentSame, Parallels } from "./lib/src/utils";
import { Logger } from "./lib/src/logger";
import { arrayToChunkedArray, getDocData, isDocContentSame } from "./lib/src/utils";
import { Logger, setGlobalLogFunction } from "./lib/src/logger";
import { PouchDB } from "./lib/src/pouchdb-browser.js";
import { LogDisplayModal } from "./LogDisplayModal";
import { ConflictResolveModal } from "./ConflictResolveModal";
import { ObsidianLiveSyncSettingTab } from "./ObsidianLiveSyncSettingTab";
import { DocumentHistoryModal } from "./DocumentHistoryModal";
@@ -15,7 +14,7 @@ import { applyPatch, cancelAllPeriodicTask, cancelAllTasks, cancelTask, generate
import { encrypt, tryDecrypt } from "./lib/src/e2ee_v2";
import { enableEncryption, isCloudantURI, isErrorOfMissingDoc, isValidRemoteCouchDBURI } from "./lib/src/utils_couchdb";
import { getGlobalStore, ObservableStore, observeStores } from "./lib/src/store";
import { lockStore, logMessageStore, logStore } from "./lib/src/stores";
import { lockStore, logMessageStore, logStore, type LogEntry } from "./lib/src/stores";
import { setNoticeClass } from "./lib/src/wrapper";
import { base64ToString, versionNumberString2Number, base64ToArrayBuffer, arrayBufferToBase64 } from "./lib/src/strbin";
import { addPrefix, isPlainText, shouldBeIgnored, stripAllPrefixes } from "./lib/src/path";
@@ -30,9 +29,19 @@ import { HiddenFileSync } from "./CmdHiddenFileSync";
import { SetupLiveSync } from "./CmdSetupLiveSync";
import { ConfigSync } from "./CmdConfigSync";
import { confirmWithMessage } from "./dialogs";
import { GlobalHistoryView, VIEW_TYPE_GLOBAL_HISTORY } from "./GlobalHistoryView";
import { LogPaneView, VIEW_TYPE_LOG } from "./LogPaneView";
import { mapAllTasksWithConcurrencyLimit, processAllTasksWithConcurrencyLimit } from "./lib/src/task";
setNoticeClass(Notice);
// DI the log again.
setGlobalLogFunction((message: any, level?: LOG_LEVEL, key?: string) => {
const entry = { message, level, key } as LogEntry;
logStore.push(entry);
});
logStore.intercept(e => e.slice(Math.min(e.length - 200, 0)));
export default class ObsidianLiveSyncPlugin extends Plugin
implements LiveSyncLocalDBEnv, LiveSyncReplicatorEnv {
@@ -539,7 +548,7 @@ export default class ObsidianLiveSyncPlugin extends Plugin
});
this.addRibbonIcon("view-log", "Show log", () => {
new LogDisplayModal(this.app, this).open();
this.showView(VIEW_TYPE_LOG);
});
this.addSettingTab(new ObsidianLiveSyncSettingTab(this.app, this));
@@ -650,8 +659,44 @@ export default class ObsidianLiveSyncPlugin extends Plugin
},
})
this.registerView(
VIEW_TYPE_GLOBAL_HISTORY,
(leaf) => new GlobalHistoryView(leaf, this)
);
this.registerView(
VIEW_TYPE_LOG,
(leaf) => new LogPaneView(leaf, this)
);
this.addCommand({
id: "livesync-global-history",
name: "Show vault history",
callback: () => {
this.showGlobalHistory()
}
})
}
async showView(viewType: string) {
const leaves = this.app.workspace.getLeavesOfType(viewType);
if (leaves.length == 0) {
await this.app.workspace.getLeaf(true).setViewState({
type: viewType,
active: true,
});
} else {
leaves[0].setViewState({
type: viewType,
active: true,
})
}
if (leaves.length > 0) {
this.app.workspace.revealLeaf(
leaves[0]
);
}
}
showGlobalHistory() {
this.showView(VIEW_TYPE_GLOBAL_HISTORY);
}
onunload() {
for (const addOn of this.addOns) {
@@ -1003,7 +1048,7 @@ export default class ObsidianLiveSyncPlugin extends Plugin
}
//--> Basic document Functions
notifies: { [key: string]: { notice: Notice; timer: NodeJS.Timeout; count: number } } = {};
notifies: { [key: string]: { notice: Notice; timer: ReturnType<typeof setTimeout>; count: number } } = {};
lastLog = "";
// eslint-disable-next-line require-await
@@ -1382,7 +1427,7 @@ export default class ObsidianLiveSyncPlugin extends Plugin
//---> Sync
async parseReplicationResult(docs: Array<PouchDB.Core.ExistingDocument<EntryDoc>>): Promise<void> {
const docsSorted = docs.sort((a, b) => b.mtime - a.mtime);
const docsSorted = docs.sort((a: any, b: any) => b?.mtime ?? 0 - a?.mtime ?? 0);
L1:
for (const change of docsSorted) {
if (isChunk(change._id)) {
@@ -1471,7 +1516,7 @@ export default class ObsidianLiveSyncPlugin extends Plugin
this.statusBar.title = e.syncStatus;
let waiting = "";
if (this.settings.batchSave && !this.settings.liveSync) {
const len = this.vaultManager.getQueueLength();
const len = this.vaultManager?.getQueueLength();
if (len != 0) {
waiting = ` 🛫${len}`;
}
@@ -1673,21 +1718,27 @@ Or if you are sure know what had been happened, we can unlock the database from
const runAll = async<T>(procedureName: string, objects: T[], callback: (arg: T) => Promise<void>) => {
Logger(procedureName);
if (!this.localDatabase.isReady) throw Error("Database is not ready!");
const para = Parallels();
for (const v of objects) {
await para.wait(10);
para.add((async (v) => {
try {
await callback(v);
} catch (ex) {
Logger(`Error while ${procedureName}`, LOG_LEVEL.NOTICE);
Logger(ex);
}
})(v));
const procs = objects.map(e => async () => {
try {
await callback(e);
return true;
} catch (ex) {
Logger(`Error while ${procedureName}`, LOG_LEVEL.NOTICE);
Logger(ex, LOG_LEVEL.VERBOSE);
return false;
}
});
let success = 0;
let failed = 0;
for await (const v of processAllTasksWithConcurrencyLimit(10, procs)) {
if ("ok" in v && v.ok) {
success++;
} else {
failed++;
}
}
await para.all();
Logger(`${procedureName} done.`);
Logger(`${procedureName}: PASS:${success}, FAILED:${failed}`);
}
await runAll("UPDATE DATABASE", onlyInStorage, async (e) => {
@@ -1711,22 +1762,19 @@ Or if you are sure know what had been happened, we can unlock the database from
if (!initialScan) {
let caches: { [key: string]: { storageMtime: number; docMtime: number } } = {};
caches = await this.kvDB.get<{ [key: string]: { storageMtime: number; docMtime: number } }>("diff-caches") || {};
const docsCount = syncFiles.length;
do {
const syncFilesXSrc = syncFiles.splice(0, 100);
const syncFilesX = [] as { file: TFile, id: DocumentID }[];
for (const file of syncFilesXSrc) {
const id = await this.path2id(getPathFromTFile(file));
syncFilesX.push({ file: file, id: id });
}
const docs = await this.localDatabase.allDocsRaw<EntryDoc>({ keys: syncFilesX.map(e => e.id), include_docs: true })
const docsMap = docs.rows.reduce((p, c) => ({ ...p, [c.id]: c.doc }), {} as Record<DocumentID, EntryDoc>)
const syncFilesToSync = syncFilesX.map((e) => ({ file: e.file, doc: docsMap[e.id] as LoadedEntry }));
await runAll(`CHECK FILE STATUS:${syncFiles.length}/${docsCount}`, syncFilesToSync, async (e) => {
const syncFilesBatch = [...arrayToChunkedArray(syncFiles, 100)];
const processes = syncFilesBatch.map((files, idx, total) => async () => {
const dbEntries = await mapAllTasksWithConcurrencyLimit(10, files.map(file => async () => ({ file: file, id: await this.path2id(getPathFromTFile(file)) })));
const dbEntriesOk = dbEntries.map(e => "ok" in e ? e.ok : undefined).filter(e => e);
const docs = await this.localDatabase.allDocsRaw<EntryDoc>({ keys: dbEntriesOk.map(e => e.id), include_docs: true });
const docsMap = docs.rows.reduce((p, c) => ({ ...p, [c.id]: c.doc }), {} as Record<DocumentID, EntryDoc>);
const syncFilesToSync = dbEntriesOk.map((e) => ({ file: e.file, doc: docsMap[e.id] as LoadedEntry }));
await runAll(`CHECK FILE STATUS:${idx + 1}/${total.length}`, syncFilesToSync, async (e) => {
caches = await this.syncFileBetweenDBandStorage(e.file, e.doc, initialScan, caches);
});
} while (syncFiles.length > 0);
})
await mapAllTasksWithConcurrencyLimit(2, processes);
await this.kvDB.set("diff-caches", caches);
}

6
src/utils.ts
View File

@@ -405,9 +405,9 @@ export const askSelectString = (app: App, message: string, items: string[]): Pro
};
export const askString = (app: App, title: string, key: string, placeholder: string): Promise<string | false> => {
export const askString = (app: App, title: string, key: string, placeholder: string, isPassword?: boolean): Promise<string | false> => {
return new Promise((res) => {
const dialog = new InputStringDialog(app, title, key, placeholder, (result) => res(result));
const dialog = new InputStringDialog(app, title, key, placeholder, isPassword, (result) => res(result));
dialog.open();
});
};
@@ -737,4 +737,4 @@ export const remoteDatabaseCleanup = async (plugin: ObsidianLiveSyncPlugin, dryR
Logger(ex, LOG_LEVEL.VERBOSE);
}
});
}
}

6
styles.css
View File

@@ -255,4 +255,8 @@ div.sls-setting-menu-btn {
.sls-setting-hidden {
display: none;
}
}
.password-input > .setting-item-control >input {
-webkit-text-security: disc;
}

3
tsconfig.json
View File

@@ -21,7 +21,8 @@
"ES5",
"ES6",
"ES7",
"es2019.array"
"es2019.array",
"ES2020.BigInt",
]
},
"include": [

67
updates.md
View File

@@ -14,53 +14,32 @@ I hope you will give it a try.
#### Minors
- 0.19.1
- Fixed: Fixed hidden file handling on Linux
- Improved: Now customization sync works more smoothly.
- 0.19.2
- Fixed:
- Fixed garbage collection error while unreferenced chunks exist many.
- Fixed filename validation on Linux.
- Improved:
- Showing status is now thinned for performance.
- Enhance caching while collecting chunks.
- 0.19.3
- Improved:
- Now replication will be paced by collecting chunks. If synchronisation has been deadlocked, please enable `Do not pace synchronization` once.
- 0.19.4
- Improved:
- Reduced remote database checking to improve speed and reduce bandwidth.
- Fixed:
- Chunks which previously misinterpreted are now interpreted correctly.
- No more missing chunks which not be found forever, except if it has been actually missing.
- Deleted file detection on hidden file synchronising now works fine.
- Now the Customisation sync is surely quiet while it has been disabled.
- 0.19.5
- Fixed:
- Now hidden file synchronisation would not be hanged, even if so many files exist.
- Improved:
- Customisation sync works more smoothly.
- Note: Concurrent processing has been rollbacked into the original implementation. As a result, the total number of processes is no longer shown next to the hourglass icon. However, only the processes that are running concurrently are shown.
- 0.19.6
- Fixed:
- Logging has been tweaked.
- No more too many planes and rockets.
- The batch database update now surely only works in non-live mode.
- Internal things:
- Some frameworks has been upgraded.
- Import declaration has been fixed.
- Improved:
- The plug-in now asks to enable a new adaptor, when rebuilding, if it is not enabled yet.
- The setting dialogue refined.
- Configurations for compatibilities have been moved under the hatch.
- Made it clear that disabled is the default.
- Ambiguous names configuration have been renamed.
- Items that have no meaning in the settings are no longer displayed.
- Some items have been reordered for clarity.
- Each configuration has been grouped.
- 0.19.1 to 0.19.6 has been moved into the updates_old.md
- 0.19.7
- Fixed:
- The initial pane of Setting dialogue is now changed to General Settings.
- The Setup Wizard is now able to flush existing settings and get into the mode again.
- 0.19.8
- New feature:
- Vault history: A tab has been implemented to give a birds-eye view of the changes that have occurred in the vault.
- Improved:
- Now the passphrases on the dialogue masked out. Thank you @antoKeinanen!
- Log dialogue is now shown as one of tabs.
- Fixed:
- Some minor issues has been fixed.
- 0.19.9
- New feature (For fixing a problem):
- We can fix the database obfuscated and plain paths that have been mixed up.
- Improvements
- Customisation Sync performance has been improved.
- 0.19.10
- Fixed
- Fixed the issue about fixing the database.
- 0.19.11
- Improvements:
- Hashing ChunkID has been improved.
- Logging keeps 400 lines now.
- Refactored:
- Import statement has been fixed about types.
... To continue on to `updates_old.md`.

58
updates_old.md
View File

@@ -1,3 +1,61 @@
### 0.19.0
#### Customization sync
Since `Plugin and their settings` have been broken, so I tried to fix it, not just fix it, but fix it the way it should be.
Now, we have `Customization sync`.
It is a real shame that the compatibility between these features has been broken. However, this new feature is surely useful and I believe that worth getting over the pain.
We can use the new feature with the same configuration. Only the menu on the command palette has been changed. The dialog can be opened by `Show customization sync dialog`.
I hope you will give it a try.
#### Minors
- 0.19.1
- Fixed: Fixed hidden file handling on Linux
- Improved: Now customization sync works more smoothly.
- 0.19.2
- Fixed:
- Fixed garbage collection error while unreferenced chunks exist many.
- Fixed filename validation on Linux.
- Improved:
- Showing status is now thinned for performance.
- Enhance caching while collecting chunks.
- 0.19.3
- Improved:
- Now replication will be paced by collecting chunks. If synchronisation has been deadlocked, please enable `Do not pace synchronization` once.
- 0.19.4
- Improved:
- Reduced remote database checking to improve speed and reduce bandwidth.
- Fixed:
- Chunks which previously misinterpreted are now interpreted correctly.
- No more missing chunks which not be found forever, except if it has been actually missing.
- Deleted file detection on hidden file synchronising now works fine.
- Now the Customisation sync is surely quiet while it has been disabled.
- 0.19.5
- Fixed:
- Now hidden file synchronisation would not be hanged, even if so many files exist.
- Improved:
- Customisation sync works more smoothly.
- Note: Concurrent processing has been rollbacked into the original implementation. As a result, the total number of processes is no longer shown next to the hourglass icon. However, only the processes that are running concurrently are shown.
- 0.19.6
- Fixed:
- Logging has been tweaked.
- No more too many planes and rockets.
- The batch database update now surely only works in non-live mode.
- Internal things:
- Some frameworks has been upgraded.
- Import declaration has been fixed.
- Improved:
- The plug-in now asks to enable a new adaptor, when rebuilding, if it is not enabled yet.
- The setting dialogue refined.
- Configurations for compatibilities have been moved under the hatch.
- Made it clear that disabled is the default.
- Ambiguous names configuration have been renamed.
- Items that have no meaning in the settings are no longer displayed.
- Some items have been reordered for clarity.
- Each configuration has been grouped.
### 0.18.0