public/obsidian-livesync
1
0
Fork 0
mirror of https://github.com/vrtmrz/obsidian-livesync.git synced 2026-03-13 05:18:49 +00:00
Code Issues Packages Projects Releases Wiki Activity

Compare commits

base: public:0.24.19
Branches Tags
public:main public:beta public:alpha public:snyk-fix-34b8981b68f7111ada8b22377113a98c public:snyk-upgrade-7fd31e6f474568014502e7205847f257 public:snyk-fix-90f3b71450f7543a1c61724d8afa0e71 public:refactor_services_2 public:refactor_services public:dev public:snyk-upgrade-fe9625385aa4be849c7157033fd2abb0 public:snyk-upgrade-99a31e70c822894595544b0c9813b11a public:svelteui public:disenchant public:snyk-fix-61f76ef0bce2a276df9552c2fc80a14f public:snyk-upgrade-2baa7938ba318464c3bad5c3fd5cbda5 public:snyk-upgrade-7af62a575eab20e8fb4e9565bf1adcfc public:snyk-upgrade-b32f281b31a81483f126b8fb6c54a957 public:snyk-upgrade-bd61b1103442d3faecc9fc726c34e9a4 public:snyk-upgrade-5b2d6584430c061b5187611a5d7fb64a public:snyk-fix-0e9fbbcb374fcb8d5e8cb027d8414fb7 public:snyk-upgrade-5242daf91e706d1ae500c9ea6545636e public:snyk-upgrade-70434a6822fbad83424135dfad9df909 public:snyk-upgrade-8fa01ccad0352125f1866d41e57d35e1 public:snyk-upgrade-39eb97b4655e70091f718a88fccd57c1 public:snyk-upgrade-2b99d54d19e207887e36c3071b796925 public:forprivacy public:snyk-upgrade-dad9088d0a333677e0157e3deef7fc38 public:snyk-upgrade-b9a76d74b3e5723759af5b3319aabb80 public:snyk-upgrade-7da6459d8ff5d293d80de9d80061d912 public:snyk-upgrade-6a3786fc4cbe394542e9d1b6058c46e7 public:snyk-upgrade-8fc9e62771dc41b0f23dc0efefb5c949
public:0.25.52-patched-1 public:0.25.52 public:0.25.51 public:0.25.50 public:0.25.49 public:0.25.48 public:0.25.47 public:0.25.46 public:0.25.45 public:0.25.44 public:0.25.43-patched-9 public:0.25.43-patched-8 public:0.25.43-patched-7 public:0.25.43-patched-6 public:0.25.43-patched-5 public:0.25.43-patched-4 public:0.25.43-patched-3 public:0.25.43-patched-2 public:0.25.43-patched-1 public:0.25.43 public:0.25.42 public:0.25.41-patched-3 public:0.25.41-patched-2 public:0.25.41-patched-1 public:0.25.41 public:0.25.40 public:0.25.39 public:0.25.38 public:0.25.37 public:0.25.36 public:0.25.35 public:0.25.34 public:0.25.33 public:0.25.32 public:0.25.31 public:0.25.30 public:0.25.30-dev public:0.25.29 public:0.25.28 public:0.25.27 public:0.25.26 public:0.25.25 public:0.25.24 public:0.25.24.beta3 public:0.25.24.beta2 public:0.25.24.beta1 public:0.25.23 public:0.25.23.beta1 public:0.25.22 public:0.25.21 public:0.25.21.beta2 public:0.25.21.beta1 public:0.25.20 public:0.25.19 public:0.25.18 public:0.25.17 public:0.25.16 public:0.25.15 public:0.25.14 public:0.25.13 public:0.25.12 public:0.25.11 public:0.25.10 public:0.25.9 public:0.25.8 public:0.25.7 public:0.25.6 public:0.25.5 public:0.25.4 public:0.25.3 public:0.25.2 public:0.25.1 public:0.25.0 public:0.25.0-beta1 public:0.24.31 public:0.24.30 public:0.24.29 public:0.24.28 public:0.24.27 public:0.24.26 public:0.24.25 public:0.24.24 public:0.24.23 public:0.24.22 public:0.24.21 public:0.24.20 public:0.24.19 public:0.24.18 public:0.24.17 public:0.24.16 public:0.24.15 public:0.24.14 public:0.24.13 public:0.24.12 public:0.24.11 public:0.24.10 public:0.24.9 public:0.24.8 public:0.24.7 public:0.24.6 public:0.24.5 public:0.24.4 public:0.24.3 public:0.24.2 public:0.24.1 public:0.24.0 public:0.24.0.dev-rc8 public:0.24.0.dev-rc7 public:0.24.0.dev-rc6 public:0.24.0.dev-rc5 public:0.24.0.dev-rc4 public:0.24.0.dev-rc3 public:0.24.0.dev-rc2 public:0.23.23 public:0.23.22 public:0.23.21 public:0.23.20 public:0.23.19 public:0.23.18 public:0.23.17 public:0.23.16 public:0.23.15 public:0.23.14 public:0.23.13 public:0.23.12 public:0.23.11 public:0.23.10 public:0.23.9 public:0.23.8 public:0.23.7 public:0.23.6 public:0.23.5 public:0.23.4 public:0.23.3 public:0.23.2 public:0.23.1 public:0.23.0 public:0.22.19 public:0.22.18 public:0.22.17 public:0.22.16 public:0.22.15 public:0.22.14 public:0.22.13 public:0.22.12 public:0.22.11 public:0.22.10 public:0.22.9 public:0.22.8 public:0.22.7 public:0.22.6 public:0.22.5 public:0.22.4 public:0.22.3 public:0.22.2 public:0.22.1 public:0.22.0 public:0.21.5 public:0.21.4 public:0.21.3 public:0.21.2 public:0.21.1 public:0.21.0 public:0.20.7 public:0.20.6 public:0.20.5 public:0.20.4 public:0.20.3 public:0.20.2 public:0.20.1 public:0.20.0 public:0.19.23 public:0.19.22 public:0.19.21 public:0.19.20 public:0.19.19 public:0.19.18 public:0.19.17 public:0.19.16 public:0.19.15 public:0.19.14 public:0.19.13 public:0.19.12 public:0.19.11 public:0.19.10 public:0.19.9 public:0.19.8 public:0.19.7 public:0.19.6 public:0.19.5 public:0.19.4 public:0.19.3 public:0.19.2 public:0.19.1 public:0.19.0 public:0.18.6 public:0.18.5 public:0.18.4 public:0.18.3 public:0.18.2 public:0.18.1 public:0.18.0 public:0.17.34 public:0.17.33 public:0.17.32 public:0.17.31 public:0.17.30 public:0.17.29 public:0.17.28 public:0.17.27 public:0.17.26 public:0.17.25 public:0.17.24 public:0.17.23 public:0.17.22 public:0.17.21 public:0.17.20 public:0.17.19 public:0.17.18 public:0.17.17 public:0.17.16 public:0.17.15 public:0.17.14 public:0.17.13 public:0.17.12 public:0.17.11 public:0.17.10 public:0.17.9 public:0.17.8 public:0.17.7 public:0.17.6 public:0.17.5 public:0.17.4 public:0.17.3 public:0.17.2 public:0.17.1 public:0.17.0 public:0.16.8 public:0.16.7 public:0.16.6 public:0.16.5 public:0.16.4 public:0.16.3 public:0.16.2 public:0.16.1 public:0.16.0 public:0.15.10 public:0.15.9 public:0.15.8 public:0.15.7 public:0.15.6 public:0.15.5 public:0.15.4 public:0.15.3 public:0.15.2 public:0.15.1 public:0.15.0 public:0.14.8 public:0.14.7 public:0.14.6 public:0.14.5 public:0.14.4 public:0.14.3 public:0.14.2 public:0.14.1 public:0.14.0 public:0.13.4 public:0.13.3 public:0.13.2 public:0.13.1 public:0.13.0 public:0.12.5 public:0.12.4 public:0.12.3 public:0.12.2 public:0.12.1 public:0.12.0 public:0.11.10 public:0.11.9 public:0.11.8 public:0.11.7 public:0.11.6 public:0.11.5 public:0.11.4 public:0.11.3 public:0.11.2 public:0.11.1 public:0.11.0 public:0.10.1 public:0.10.0 public:0.9.0 public:0.8.8 public:0.8.7 public:0.8.6 public:0.8.5 public:0.8.3 public:0.8.2 public:0.8.1 public:0.8.0 public:0.7.2 public:0.7.1 public:0.7.0 public:0.6.1 public:0.6.0 public:0.5.0 public:0.4.1 public:0.4.0 public:0.3.1 public:0.3.0 public:0.2.1 public:0.2.0 public:0.1.26 public:0.1.25 public:0.1.24 public:0.1.23 public:0.1.22 public:0.1.21 public:0.1.20 public:0.1.19 public:0.1.18 public:0.1.17 public:0.1.16 public:0.1.15 public:0.1.14 public:0.1.13 public:0.1.12 public:0.1.11 public:0.1.10 public:0.1.9 public:0.1.8 public:0.1.7 public:0.1.6 public:0.1.5 public:0.1.4 public:0.1.3 public:0.1.2 public:0.1.1 public:0.1.0 public:0.0.9 public:0.0.8 public:0.0.7 public:0.0.6
...
compare: public:0.24.25
Branches Tags
public:beta public:alpha public:main public:snyk-fix-34b8981b68f7111ada8b22377113a98c public:snyk-upgrade-7fd31e6f474568014502e7205847f257 public:snyk-fix-90f3b71450f7543a1c61724d8afa0e71 public:refactor_services_2 public:refactor_services public:dev public:snyk-upgrade-fe9625385aa4be849c7157033fd2abb0 public:snyk-upgrade-99a31e70c822894595544b0c9813b11a public:svelteui public:disenchant public:snyk-fix-61f76ef0bce2a276df9552c2fc80a14f public:snyk-upgrade-2baa7938ba318464c3bad5c3fd5cbda5 public:snyk-upgrade-7af62a575eab20e8fb4e9565bf1adcfc public:snyk-upgrade-b32f281b31a81483f126b8fb6c54a957 public:snyk-upgrade-bd61b1103442d3faecc9fc726c34e9a4 public:snyk-upgrade-5b2d6584430c061b5187611a5d7fb64a public:snyk-fix-0e9fbbcb374fcb8d5e8cb027d8414fb7 public:snyk-upgrade-5242daf91e706d1ae500c9ea6545636e public:snyk-upgrade-70434a6822fbad83424135dfad9df909 public:snyk-upgrade-8fa01ccad0352125f1866d41e57d35e1 public:snyk-upgrade-39eb97b4655e70091f718a88fccd57c1 public:snyk-upgrade-2b99d54d19e207887e36c3071b796925 public:forprivacy public:snyk-upgrade-dad9088d0a333677e0157e3deef7fc38 public:snyk-upgrade-b9a76d74b3e5723759af5b3319aabb80 public:snyk-upgrade-7da6459d8ff5d293d80de9d80061d912 public:snyk-upgrade-6a3786fc4cbe394542e9d1b6058c46e7 public:snyk-upgrade-8fc9e62771dc41b0f23dc0efefb5c949
public:0.25.52-patched-1 public:0.25.52 public:0.25.51 public:0.25.50 public:0.25.49 public:0.25.48 public:0.25.47 public:0.25.46 public:0.25.45 public:0.25.44 public:0.25.43-patched-9 public:0.25.43-patched-8 public:0.25.43-patched-7 public:0.25.43-patched-6 public:0.25.43-patched-5 public:0.25.43-patched-4 public:0.25.43-patched-3 public:0.25.43-patched-2 public:0.25.43-patched-1 public:0.25.43 public:0.25.42 public:0.25.41-patched-3 public:0.25.41-patched-2 public:0.25.41-patched-1 public:0.25.41 public:0.25.40 public:0.25.39 public:0.25.38 public:0.25.37 public:0.25.36 public:0.25.35 public:0.25.34 public:0.25.33 public:0.25.32 public:0.25.31 public:0.25.30 public:0.25.30-dev public:0.25.29 public:0.25.28 public:0.25.27 public:0.25.26 public:0.25.25 public:0.25.24 public:0.25.24.beta3 public:0.25.24.beta2 public:0.25.24.beta1 public:0.25.23 public:0.25.23.beta1 public:0.25.22 public:0.25.21 public:0.25.21.beta2 public:0.25.21.beta1 public:0.25.20 public:0.25.19 public:0.25.18 public:0.25.17 public:0.25.16 public:0.25.15 public:0.25.14 public:0.25.13 public:0.25.12 public:0.25.11 public:0.25.10 public:0.25.9 public:0.25.8 public:0.25.7 public:0.25.6 public:0.25.5 public:0.25.4 public:0.25.3 public:0.25.2 public:0.25.1 public:0.25.0 public:0.25.0-beta1 public:0.24.31 public:0.24.30 public:0.24.29 public:0.24.28 public:0.24.27 public:0.24.26 public:0.24.25 public:0.24.24 public:0.24.23 public:0.24.22 public:0.24.21 public:0.24.20 public:0.24.19 public:0.24.18 public:0.24.17 public:0.24.16 public:0.24.15 public:0.24.14 public:0.24.13 public:0.24.12 public:0.24.11 public:0.24.10 public:0.24.9 public:0.24.8 public:0.24.7 public:0.24.6 public:0.24.5 public:0.24.4 public:0.24.3 public:0.24.2 public:0.24.1 public:0.24.0 public:0.24.0.dev-rc8 public:0.24.0.dev-rc7 public:0.24.0.dev-rc6 public:0.24.0.dev-rc5 public:0.24.0.dev-rc4 public:0.24.0.dev-rc3 public:0.24.0.dev-rc2 public:0.23.23 public:0.23.22 public:0.23.21 public:0.23.20 public:0.23.19 public:0.23.18 public:0.23.17 public:0.23.16 public:0.23.15 public:0.23.14 public:0.23.13 public:0.23.12 public:0.23.11 public:0.23.10 public:0.23.9 public:0.23.8 public:0.23.7 public:0.23.6 public:0.23.5 public:0.23.4 public:0.23.3 public:0.23.2 public:0.23.1 public:0.23.0 public:0.22.19 public:0.22.18 public:0.22.17 public:0.22.16 public:0.22.15 public:0.22.14 public:0.22.13 public:0.22.12 public:0.22.11 public:0.22.10 public:0.22.9 public:0.22.8 public:0.22.7 public:0.22.6 public:0.22.5 public:0.22.4 public:0.22.3 public:0.22.2 public:0.22.1 public:0.22.0 public:0.21.5 public:0.21.4 public:0.21.3 public:0.21.2 public:0.21.1 public:0.21.0 public:0.20.7 public:0.20.6 public:0.20.5 public:0.20.4 public:0.20.3 public:0.20.2 public:0.20.1 public:0.20.0 public:0.19.23 public:0.19.22 public:0.19.21 public:0.19.20 public:0.19.19 public:0.19.18 public:0.19.17 public:0.19.16 public:0.19.15 public:0.19.14 public:0.19.13 public:0.19.12 public:0.19.11 public:0.19.10 public:0.19.9 public:0.19.8 public:0.19.7 public:0.19.6 public:0.19.5 public:0.19.4 public:0.19.3 public:0.19.2 public:0.19.1 public:0.19.0 public:0.18.6 public:0.18.5 public:0.18.4 public:0.18.3 public:0.18.2 public:0.18.1 public:0.18.0 public:0.17.34 public:0.17.33 public:0.17.32 public:0.17.31 public:0.17.30 public:0.17.29 public:0.17.28 public:0.17.27 public:0.17.26 public:0.17.25 public:0.17.24 public:0.17.23 public:0.17.22 public:0.17.21 public:0.17.20 public:0.17.19 public:0.17.18 public:0.17.17 public:0.17.16 public:0.17.15 public:0.17.14 public:0.17.13 public:0.17.12 public:0.17.11 public:0.17.10 public:0.17.9 public:0.17.8 public:0.17.7 public:0.17.6 public:0.17.5 public:0.17.4 public:0.17.3 public:0.17.2 public:0.17.1 public:0.17.0 public:0.16.8 public:0.16.7 public:0.16.6 public:0.16.5 public:0.16.4 public:0.16.3 public:0.16.2 public:0.16.1 public:0.16.0 public:0.15.10 public:0.15.9 public:0.15.8 public:0.15.7 public:0.15.6 public:0.15.5 public:0.15.4 public:0.15.3 public:0.15.2 public:0.15.1 public:0.15.0 public:0.14.8 public:0.14.7 public:0.14.6 public:0.14.5 public:0.14.4 public:0.14.3 public:0.14.2 public:0.14.1 public:0.14.0 public:0.13.4 public:0.13.3 public:0.13.2 public:0.13.1 public:0.13.0 public:0.12.5 public:0.12.4 public:0.12.3 public:0.12.2 public:0.12.1 public:0.12.0 public:0.11.10 public:0.11.9 public:0.11.8 public:0.11.7 public:0.11.6 public:0.11.5 public:0.11.4 public:0.11.3 public:0.11.2 public:0.11.1 public:0.11.0 public:0.10.1 public:0.10.0 public:0.9.0 public:0.8.8 public:0.8.7 public:0.8.6 public:0.8.5 public:0.8.3 public:0.8.2 public:0.8.1 public:0.8.0 public:0.7.2 public:0.7.1 public:0.7.0 public:0.6.1 public:0.6.0 public:0.5.0 public:0.4.1 public:0.4.0 public:0.3.1 public:0.3.0 public:0.2.1 public:0.2.0 public:0.1.26 public:0.1.25 public:0.1.24 public:0.1.23 public:0.1.22 public:0.1.21 public:0.1.20 public:0.1.19 public:0.1.18 public:0.1.17 public:0.1.16 public:0.1.15 public:0.1.14 public:0.1.13 public:0.1.12 public:0.1.11 public:0.1.10 public:0.1.9 public:0.1.8 public:0.1.7 public:0.1.6 public:0.1.5 public:0.1.4 public:0.1.3 public:0.1.2 public:0.1.1 public:0.1.0 public:0.0.9 public:0.0.8 public:0.0.7 public:0.0.6

22 Commits
0.24.19 ... 0.24.25

Author SHA1 Message Date
vorotamoroz
381055fc93 bump 2025-04-22 11:29:42 +01:00
vorotamoroz
37d12916fc ## 0.24.25 
### Improved

- Peer-to-peer synchronisation has been got more robust.

### Fixed

- No longer broken falsy values in settings during set-up by the QR code generation.

### Refactored

- Some `window` references now have pointed to `globalThis`.
- Some sloppy-import has been fixed.
- A server side implementation `Synchromesh` has been suffixed with `deno` instead of `server` now.
 2025-04-22 11:28:55 +01:00
vorotamoroz
944aa846c4 bump 2025-04-15 11:10:37 +01:00
vorotamoroz
abca808e29 ### Fixed 
- No longer broken JSON files including `\n`, during the bucket synchronisation. (#623)
- Custom headers and JWT tokens are now correctly sent to the server during configuration checking. (#624)

### Improved

- Bucket synchronisation has been enhanced for better performance and reliability.
    - Now less duplicated chunks are sent to the server.
    - Fetching conflicted files from the server is now more reliable.
    - Dependent libraries have been updated to the latest version.
 2025-04-15 11:09:49 +01:00
vorotamoroz
90bb610133 Update submodule 2025-04-14 03:23:24 +01:00
vorotamoroz
9c5e9fe63b Conclusion stated. 2025-04-11 14:13:22 +01:00
vorotamoroz
00dfae24d7 bump 2025-04-10 14:28:03 +01:00
vorotamoroz
d8a41fe45d ### New Feature 
- Now, we can send custom headers to the server.
- Authentication with JWT in CouchDB is now supported.

### Improved

- The QR Code for set-up can be shown also from the setting dialogue now.
- Conflict checking for preventing unexpected overwriting on the boot-up process has been quite faster.

### Fixed

- Some bugs on Dev and Testing modules have been fixed.
 2025-04-10 14:24:33 +01:00
vorotamoroz
30467d1c25 Add link to P2P pseudo client. 2025-04-04 12:30:21 +01:00
vorotamoroz
f8351f1d45 Merge branch 'main' of https://github.com/vrtmrz/obsidian-livesync 2025-04-04 11:52:12 +01:00
vorotamoroz
5924af98ab Update Library (Probably no effect) 2025-04-04 11:52:05 +01:00
vorotamoroz
2769b61da4 Update setup_own_server.md 
Add note; #609
 2025-04-04 18:24:13 +09:00
vorotamoroz
bb4409221d Update README.md 2025-04-03 20:57:15 +09:00
vorotamoroz
f398c14200 Bump for release-mistake. 2025-04-01 10:42:29 +01:00
vorotamoroz
27d58508dc Missed 2025-04-01 10:38:12 +01:00
vorotamoroz
d4dea5b226 bump 2025-04-01 10:21:38 +01:00
vorotamoroz
c79dc30cba ## 0.24.21 
### Fixed

- No longer conflicted files are handled in the boot-up process. No more unexpected overwriting.
    - It ignores `Always overwrite with a newer file`, and always be prevented for the safety. Please pick it manually or open the file.
- Some log messages on conflict resolution has been corrected.
- Automatic merge notifications, displayed on the grounds of `same`, have been degraded to logs.

### Improved

- Now we can fetch the remote database with keeping local files completely intact.
    - In new option, all files are stored into the local database before the fetching, and will be merged automatically or detected as conflicts.
- The dialogue presenting options when performing `Fetch` are now more informative.

### Refactored

- Some class methods have been fixed its arguments to be more consistent.
- Types have been defined for some conditional results.
 2025-04-01 10:20:21 +01:00
vorotamoroz
b3119ee8a9 bump and update dependencies 2025-03-24 18:55:02 +09:00
vorotamoroz
2a1d71da5c Improved: show details of TypeError using Obsidian API. 2025-03-24 12:04:58 +09:00
vorotamoroz
24f31ed19e Merge branch 'main' of https://github.com/vrtmrz/obsidian-livesync 2025-03-19 04:43:06 +01:00
vorotamoroz
a982629ae6 Update draft : possibly I should share cSpell dictionary. 2025-03-19 04:42:50 +01:00
vorotamoroz
85140aecab Update README.md 2025-03-05 20:40:04 +09:00
30 changed files with 2855 additions and 3025 deletions
Expand all files Collapse all files

7
README.md
View File

@@ -18,10 +18,11 @@ Note: This plugin cannot synchronise with the official "Obsidian Sync".
- Supporting End-to-end encryption.
- Synchronisation of settings, snippets, themes, and plug-ins, via [Customization sync(Beta)](#customization-sync) or [Hidden File Sync](#hiddenfilesync)
- WebClip from [obsidian-livesync-webclip](https://chrome.google.com/webstore/detail/obsidian-livesync-webclip/jfpaflmpckblieefkegjncjoceapakdf)
- WebRTC peer-to-peer synchronisation without the need any `host` is now possible. (Experimental)
- WebRTC peer-to-peer synchronisation without the need for any `host` is now possible. (Experimental)
- This feature is still in the experimental stage. Please be careful when using it.
- Instead of using server, you can use [webpeer](https://github.com/vrtmrz/livesync-commonlib/tree/main/apps/webpeer) the pseudo client for receiving and sending between devices.
- Instead of using public servers, you can use [webpeer](https://github.com/vrtmrz/livesync-commonlib/tree/main/apps/webpeer) the pseudo client for receiving and sending between devices.
- A pre-built instance is served at [fancy-syncing.vrtmrz.net/webpeer](https://fancy-syncing.vrtmrz.net/webpeer/) (in the vrtmrz blog site). This is of course also peer-to-peer. Feel free to use it.
- There is an [English explanatory article](https://fancy-syncing.vrtmrz.net/blog/0034-p2p-sync-en.html), and [Japanese explanatory article](https://fancy-syncing.vrtmrz.net/blog/0034-p2p-sync).
This plug-in might be useful for researchers, engineers, and developers with a need to keep their notes fully self-hosted for security reasons. Or just anyone who would like the peace of mind of knowing that their notes are fully private.

43
docs/design_docs_of_keep_newborn_chunks.md
View File

@@ -1,6 +1,6 @@
# Keep newborn chunks in Eden.
# Keep newborn chunks in Eden
NOTE: This is the planned feature design document. This is planned, but not be implemented now (v0.23.3). This has not reached the design freeze and will be added to from time to time.
Notice: deprecated. please refer to the result section of this document.
## Goal
@@ -19,15 +19,18 @@ Reduce the number of chunks which in volatile, and reduce the usage of storage o
- The problem is that this unnecessary chunking slows down both local and remote operations.
## Prerequisite
- The implementation must be able to control the size of the document appropriately so that it does not become non-transferable (1).
- The implementation must be such that data corruption can be avoided even if forward compatibility is not maintained; due to the nature of Self-hosted LiveSync, backward version connexions are expected.
- The implementation must be such that data corruption can be avoided even if forward compatibility is not maintained; due to the nature of Self-hosted LiveSync, backward version connexions are expected.
- Viewed as a feature:
- This feature should be disabled for migration users.
- This feature should be enabled for new users and after rebuilds of migrated users.
- Therefore, back into the implementation view, Ideally, the implementation should be such that data recovery can be achieved by immediately upgrading after replication.
## Outlined methods and implementation plans
### Abstract
To store and transfer only stable chunks independently and share them from multiple documents after stabilisation, new chunks, i.e. chunks that are considered non-stable, are modified to be stored in the document and transferred with the document. In this case, care should be taken not to exceed prerequisite (1).
If this is achieved, the non-leaf document will not be transferred, and even if it is, the chunk will be stored in the document, so that the size can be reduced by the compaction.
@@ -40,11 +43,11 @@ Details are given below.
type EntryWithEden = {
eden: {
[key: DocumentID]: {
data: string,
epoch: number, // The document revision which this chunk has been born.
}
}
}
data: string;
epoch: number; // The document revision which this chunk has been born.
};
};
};
```
2. The following configuration items are added:
Note: These configurations should be shared as `Tweaks value` between each client.
@@ -63,6 +66,7 @@ Details are given below.
5. In End-to-End Encryption, property `eden` of documents will also be encrypted.
### Note
- When this feature has been enabled, forward compatibility is temporarily lost. However, it is detected as missing chunks, and this data is not reflected in the storage in the old version. Therefore, no data loss will occur.
## Test strategy
@@ -77,5 +81,26 @@ Details are given below.
- Indeed, we lack a fulfilled configuration table. Efforts will be made and, if they can be produced, this document will then be referenced. But not required while in the experimental or beta feature.
- However, this might be an essential feature. Further efforts are desired.
## Results from actual operation
After implementing this feature, we have been using it for a while. The following results were obtained.
- Drawbacks were thought not to be a problem, but they were actually a problem:
- A document with `Eden` has a quite larger history compared to a document without `Eden`.
- Self-hosted LiveSync does not perform compaction aggressively, which results in the remote database becoming partially bloated.
- Compaction of the Remote Database (CouchDB) requires the same amount of free space as the size of the database. Therefore, it is not possible to perform compaction on a remote database if we reached to the maximum size of the database. It means that when we detect it, it is too late.
- We have mentioned that `We need compaction` in previous sections. However, but it was so hard to be determined whether the compaction is required or not, until the database is bloated. (Of course, it requires some time to compact the database, and, literally, some document loses its history. It is not a good idea to perform frequently and meaninglessly. We need manual decision, but indeed difficult to normal users).
### Consideration and Conclusion
To be described after implemented, tested, and, released.
This feature results in two aspects:
- For the users who are familiar with the CouchDB, this feature is a bit useful. They can watch and handle the database by themselves.
- For the users who are not familiar with the CouchDB, i.e., normal users, this feature is not so useful, either. They are not familiar with the database, and they do not know how to handle it. Therefore, they cannot decide whether the compaction is required or not.
Hence, this feature would be kept as an experimental feature, but it is not enabled by default. In addition to that, it is marked as deprecated. Detailed notice will be noisy for the users who are not familiar with the CouchDB. Details would be kept in this document, for the future.
It is not recommended to use this feature, unless the person who is familiar with the CouchDB and the database management.
Vorotamoroz has written this document. Bias: I am the first author of this plug-in, familiar with the CouchDB.
Research and development has been frozen on 2025-04-11. But, bugs will be fixed if they are found. Please feel free to report them.

21
docs/setup_own_server.md
View File

@@ -31,7 +31,7 @@ export hostname=localhost:5984
export username=goojdasjdas #Please change as you like.
export password=kpkdasdosakpdsa #Please change as you like
# Prepare directories which saving data and configurations.
# Prepare directories which save data and configurations.
mkdir couchdb-data
mkdir couchdb-etc
```
@@ -45,19 +45,19 @@ $ docker run --name couchdb-for-ols --rm -it -e COUCHDB_USER=${username} -e COUC
If your container has been exited, please check the permission of couchdb-data, and couchdb-etc.
Once CouchDB run, these directories will be owned by uid:`5984`. Please chown it for you again.
2. Enable it in background
2. Enable it in the background
```
$ docker run --name couchdb-for-ols -d --restart always -e COUCHDB_USER=${username} -e COUCHDB_PASSWORD=${password} -v ${PWD}/couchdb-data:/opt/couchdb/data -v ${PWD}/couchdb-etc:/opt/couchdb/etc/local.d -p 5984:5984 couchdb
```
### B. Install CouchDB directly
Please refer the [official document](https://docs.couchdb.org/en/stable/install/index.html). However, we do not have to configure it fully. Just administrator needs to be configured.
Please refer to the [official document](https://docs.couchdb.org/en/stable/install/index.html). However, we do not have to configure it fully. Just the administrator needs to be configured.
## 2. Run couchdb-init.sh for initialise
```
curl -s https://raw.githubusercontent.com/vrtmrz/obsidian-livesync/main/utils/couchdb/couchdb-init.sh | bash
```
If it results like following:
If it results like the following:
```
-- Configuring CouchDB by REST APIs... -->
{"ok":true}
@@ -80,7 +80,7 @@ Your CouchDB has been initialised successfully. If you want this manually, pleas
- You can skip this instruction if you using only in intranet and only with desktop devices.
- For mobile devices, Obsidian requires a valid SSL certificate. Usually, it needs exposing the internet.
Whatever solutions we can use. For the simplicity, following sample uses Cloudflare Zero Trust for testing.
Whatever solutions we can use. For simplicity, the following sample uses Cloudflare Zero Trust for testing.
```
cloudflared tunnel --url http://localhost:5984
@@ -99,12 +99,12 @@ You will then get the following output:
:
:
```
Now `https://tiles-photograph-routine-groundwater.trycloudflare.com` is our server. Make it into background once please.
Now `https://tiles-photograph-routine-groundwater.trycloudflare.com` is our server. Make it into the background once, please.
## 4. Client Setup
> [!TIP]
> Now manually configuration is not recommended for some reasons. However, if you want to do so, please use `Setup wizard`. The recommended extra configurations will be also set.
> Now manual configuration is not recommended for some reasons. However, if you want to do so, please use `Setup wizard`. The recommended extra configurations will be also set.
### 1. Generate the setup URI on a desktop device or server
```bash
@@ -116,6 +116,13 @@ export password=abc123
deno run -A https://raw.githubusercontent.com/vrtmrz/obsidian-livesync/main/utils/flyio/generate_setupuri.ts
```
> [!TIP]
> What is the `passphrase`? Is it different from `uri_passphrase`?
> Yes, the `passphrase` we have exported now is for an End-to-End Encryption passphrase.
> And, `uri_passphrase` that used in the `generate_setupuri.ts` is a different one; for decrypting Set-up URI at using that.
> Why: I (vorotamoroz) think that the passphrase of the Setup-URI should be different from the E2EE passphrase to prevent exposure caused by operational errors or the possibility of evil in our environment. On top of that, I believe that it is desirable for the Setup-URI to be random. Setup-URI is inevitably long, so it goes through the clipboard. I think that its passphrase should not go through the same path, so it should essentially be typed manually.
> Hence, if we keep empty for uri_passphrase, generate_setupuri.ts generates an adjective-noun-randomnumber passphrase so that we can remember it without going through the clipboard.
You will then get the following output:
```bash

26
docs/terms.md
View File

@@ -1,10 +1,24 @@
# Terms used in this project
# Notes on Terminology, Spelling, Vocabulary Conventions
## Terms
## Spelling and Vocabulary conventions
### Chunks
<!-- TBW, sorry for the draft! -->
1. Almost all of the english words are written in British English. For example, "organisation" instead of "organization", "synchronisation" instead of "synchronization", etc. This convention originated from the author's personal preference but is now maintained for consistency.
2. Idiomatic terms, such as used in HTML, CSS, and JavaScript, are usually be aligned with the language used in the technology. For example, "color" instead of "colour", "program" instead of "programme", etc. Especially, terms which are used for attributes, properties, and methods are notable.
<!-- Please feel free to write any terms that should be mentioned. And please make pull request. I would love to fill the rest. -->
<!-- ### Chunks -->
3. We use `dialogue` in documentation for consistency. While `dialog` may appear in source code, particularly in class names, method names, and attributes (following technical conventions in No. 2), we consistently use `dialogue` for user-facing messages and general documentation text. This approach balances No. 1 with No. 2.
4. Contractions are not used. For example, "do not" instead of "don't", "cannot" instead of "can't", etc. especially `'d`.
- We may encounter difficulties with tenses.
5. However, try using affirmative forms, `Discard` instead of `Do not keep`, `Continue` instead of `Do not stop`, etc.
- Some languages, such as Japanese, have a different meaning for `yes` and `no` between affirmative and negative questions.
## Terminology
- Self-hosted LiveSync
- This plug-in name. `Self-hosted` is one word.
- LiveSync
- Very confusing term.
- As shorten-form of `Self-hosted LiveSync`.
- As a name of synchronisation mode. This should be changed to `Continuos`, in contrast to `Periodic`.

2
manifest.json
View File

@@ -1,7 +1,7 @@
{
"id": "obsidian-livesync",
"name": "Self-hosted LiveSync",
"version": "0.24.19",
"version": "0.24.25",
"minAppVersion": "0.9.12",
"description": "Community implementation of self-hosted livesync. Reflect your vault changes to some other devices immediately. Please make sure to disable other synchronize solutions to avoid content corruption or duplication.",
"author": "vorotamoroz",

4780
package-lock.json generated
View File

File diff suppressed because it is too large Load Diff

14
package.json
View File

@@ -1,6 +1,6 @@
{
"name": "obsidian-livesync",
"version": "0.24.19",
"version": "0.24.25",
"description": "Reflect your vault changes to some other devices immediately. Please make sure to disable other synchronize solutions to avoid content corruption or duplication.",
"main": "main.js",
"type": "module",
@@ -68,19 +68,19 @@
"typescript": "^5.7.3"
},
"dependencies": {
"@aws-sdk/client-s3": "^3.645.0",
"@smithy/fetch-http-handler": "^3.2.4",
"@smithy/protocol-http": "^4.1.0",
"@smithy/querystring-builder": "^3.0.3",
"@aws-sdk/client-s3": "^3.787.0",
"@smithy/fetch-http-handler": "^5.0.2",
"@smithy/protocol-http": "^5.1.0",
"@smithy/querystring-builder": "^4.0.2",
"diff-match-patch": "^1.0.5",
"esbuild-plugin-inline-worker": "^0.1.1",
"fflate": "^0.8.2",
"idb": "^8.0.2",
"minimatch": "^10.0.1",
"octagonal-wheels": "^0.1.24",
"octagonal-wheels": "^0.1.25",
"qrcode-generator": "^1.4.4",
"svelte-check": "^4.1.4",
"trystero": "^0.20.1",
"trystero": "^0.21.3",
"xxhash-wasm-102": "npm:xxhash-wasm@^1.0.2"
}
}

2
src/common/events.ts
View File

@@ -10,6 +10,7 @@ export const EVENT_REQUEST_OPEN_SETTINGS = "request-open-settings";
export const EVENT_REQUEST_OPEN_SETTING_WIZARD = "request-open-setting-wizard";
export const EVENT_REQUEST_OPEN_SETUP_URI = "request-open-setup-uri";
export const EVENT_REQUEST_COPY_SETUP_URI = "request-copy-setup-uri";
export const EVENT_REQUEST_SHOW_SETUP_QR = "request-show-setup-qr";
export const EVENT_REQUEST_RELOAD_SETTING_TAB = "reload-setting-tab";
@@ -35,6 +36,7 @@ declare global {
[EVENT_REQUEST_OPEN_P2P]: undefined;
[EVENT_REQUEST_OPEN_SETUP_URI]: undefined;
[EVENT_REQUEST_COPY_SETUP_URI]: undefined;
[EVENT_REQUEST_SHOW_SETUP_QR]: undefined;
[EVENT_REQUEST_RUN_DOCTOR]: string;
}
}

53
src/common/utils.ts
View File

@@ -15,6 +15,7 @@ import {
LOG_LEVEL_NOTICE,
LOG_LEVEL_VERBOSE,
type AnyEntry,
type CouchDBCredentials,
type DocumentID,
type EntryHasPath,
type FilePath,
@@ -31,6 +32,7 @@ import type { KeyValueDatabase } from "./KeyValueDB.ts";
import { scheduleTask } from "octagonal-wheels/concurrency/task";
import { EVENT_PLUGIN_UNLOADED, eventHub } from "./events.ts";
import { promiseWithResolver, type PromiseWithResolvers } from "octagonal-wheels/promises";
import { AuthorizationHeaderGenerator } from "../lib/src/replication/httplib.ts";
export { scheduleTask, cancelTask, cancelAllTasks } from "../lib/src/concurrency/task.ts";
@@ -230,17 +232,17 @@ export const _requestToCouchDBFetch = async (
export const _requestToCouchDB = async (
baseUri: string,
username: string,
password: string,
credentials: CouchDBCredentials,
origin: string,
path?: string,
body?: any,
method?: string
method?: string,
customHeaders?: Record<string, string>
) => {
const utf8str = String.fromCharCode.apply(null, [...writeString(`${username}:${password}`)]);
const encoded = window.btoa(utf8str);
const authHeader = "Basic " + encoded;
const transformedHeaders: Record<string, string> = { authorization: authHeader, origin: origin };
// Create each time to avoid caching.
const authHeaderGen = new AuthorizationHeaderGenerator();
const authHeader = await authHeaderGen.getAuthorizationHeader(credentials);
const transformedHeaders: Record<string, string> = { authorization: authHeader, origin: origin, ...customHeaders };
const uri = `${baseUri}/${path}`;
const requestParam: RequestUrlParam = {
url: uri,
@@ -251,6 +253,9 @@ export const _requestToCouchDB = async (
};
return await requestUrl(requestParam);
};
/**
* @deprecated Use requestToCouchDBWithCredentials instead.
*/
export const requestToCouchDB = async (
baseUri: string,
username: string,
@@ -258,12 +263,34 @@ export const requestToCouchDB = async (
origin: string = "",
key?: string,
body?: string,
method?: string
method?: string,
customHeaders?: Record<string, string>
) => {
const uri = `_node/_local/_config${key ? "/" + key : ""}`;
return await _requestToCouchDB(baseUri, username, password, origin, uri, body, method);
return await _requestToCouchDB(
baseUri,
{ username, password, type: "basic" },
origin,
uri,
body,
method,
customHeaders
);
};
export function requestToCouchDBWithCredentials(
baseUri: string,
credentials: CouchDBCredentials,
origin: string = "",
key?: string,
body?: string,
method?: string,
customHeaders?: Record<string, string>
) {
const uri = `_node/_local/_config${key ? "/" + key : ""}`;
return _requestToCouchDB(baseUri, credentials, origin, uri, body, method, customHeaders);
}
export const BASE_IS_NEW = Symbol("base");
export const TARGET_IS_NEW = Symbol("target");
export const EVEN = Symbol("even");
@@ -586,10 +613,10 @@ const decodePrefixMapNumber = Object.fromEntries(
);
export function encodeAnyArray(obj: any[]): string {
const tempArray = obj.map((v) => {
if (v == null) return "n";
if (v == false) return "f";
if (v == true) return "t";
if (v == undefined) return "u";
if (v === null) return "n";
if (v === false) return "f";
if (v === true) return "t";
if (v === undefined) return "u";
if (typeof v == "number") {
const b36 = v.toString(36);
const strNum = v.toString();

4
src/features/ConfigSync/CmdConfigSync.ts
View File

@@ -1306,7 +1306,7 @@ export class ConfigSync extends LiveSyncCommands implements IObsidianModule {
);
return;
}
const docXDoc = await this.localDatabase.getDBEntryFromMeta(old, {}, false, false);
const docXDoc = await this.localDatabase.getDBEntryFromMeta(old, false, false);
if (docXDoc == false) {
throw "Could not load the document";
}
@@ -1440,7 +1440,7 @@ export class ConfigSync extends LiveSyncCommands implements IObsidianModule {
// this._log(`STORAGE --> DB:${prefixedFileName}: (config) Skipped (Same time)`, LOG_LEVEL_VERBOSE);
return true;
}
const oldC = await this.localDatabase.getDBEntryFromMeta(old, {}, false, false);
const oldC = await this.localDatabase.getDBEntryFromMeta(old, false, false);
if (oldC) {
const d = (await deserialize(getDocDataAsArray(oldC.data), {})) as PluginDataEx;
if (d.files.length == dt.files.length) {

2
src/features/HiddenFileSync/CmdHiddenFileSync.ts
View File

@@ -1490,7 +1490,7 @@ Offline Changed files: ${files.length}`;
}
return false;
} else {
const fileOnDB = await this.localDatabase.getDBEntryFromMeta(metaOnDB, {}, false, true, true);
const fileOnDB = await this.localDatabase.getDBEntryFromMeta(metaOnDB, false, true);
if (fileOnDB === false) {
throw new Error(`Failed to read file from database:${storageFilePath}`);
}

2
src/lib

Submodule src/lib updated: a5d21afb61...c8bb4fedbb

9
src/main.ts
View File

@@ -18,6 +18,7 @@ import {
type AUTO_MERGED,
type RemoteDBSettings,
type TweakValues,
type CouchDBCredentials,
} from "./lib/src/common/types.ts";
import { type FileEventItem } from "./common/types.ts";
import { type SimpleStore } from "./lib/src/common/utils.ts";
@@ -283,16 +284,14 @@ export default class ObsidianLiveSyncPlugin
$$connectRemoteCouchDB(
uri: string,
auth: {
username: string;
password: string;
},
auth: CouchDBCredentials,
disableRequestURI: boolean,
passphrase: string | false,
useDynamicIterationCount: boolean,
performSetup: boolean,
skipInfo: boolean,
compression: boolean
compression: boolean,
customHeaders: Record<string, string>
): Promise<
| string
| {

8
src/modules/core/ModuleDatabaseFileAccess.ts
View File

@@ -313,13 +313,7 @@ export class ModuleDatabaseFileAccess extends AbstractModule implements IObsidia
if (skipCheck && !(await this.checkIsTargetFile(meta.path))) {
return false;
}
const doc = await this.localDatabase.getDBEntryFromMeta(
meta as LoadedEntry,
undefined,
false,
waitForReady,
true
);
const doc = await this.localDatabase.getDBEntryFromMeta(meta as LoadedEntry, false, waitForReady);
if (doc === false) {
return false;
}

6
src/modules/core/ModuleRebuilder.ts
View File

@@ -176,7 +176,7 @@ export class ModuleRebuilder extends AbstractModule implements ICoreModule, Rebu
}
}
}
async fetchLocal(makeLocalChunkBeforeSync?: boolean) {
async fetchLocal(makeLocalChunkBeforeSync?: boolean, preventMakeLocalFilesBeforeSync?: boolean) {
await this.core.$allSuspendExtraSync();
await this.askUseNewAdapter();
this.core.settings.isConfigured = true;
@@ -189,6 +189,10 @@ export class ModuleRebuilder extends AbstractModule implements ICoreModule, Rebu
this.core.$$markIsReady();
if (makeLocalChunkBeforeSync) {
await this.core.fileHandler.createAllChunks(true);
} else if (!preventMakeLocalFilesBeforeSync) {
await this.core.$$initializeDatabase(true);
} else {
// Do not create local file entries before sync (Means use remote information)
}
await this.core.$$markRemoteResolved();
await delay(500);

2
src/modules/core/ModuleReplicator.ts
View File

@@ -338,7 +338,7 @@ Even if you choose to clean up, you will see this option again if you exit Obsid
// If `Read chunks online` is disabled, chunks should be transferred before here.
// However, in some cases, chunks are after that. So, if missing chunks exist, we have to wait for them.
const doc = await this.localDatabase.getDBEntryFromMeta({ ...dbDoc }, {}, false, true, true);
const doc = await this.localDatabase.getDBEntryFromMeta({ ...dbDoc }, false, true);
if (!doc) {
Logger(
`Something went wrong while gathering content of ${path} (${dbDoc._id.substring(0, 8)}, ${dbDoc._rev?.substring(0, 10)}) `,

14
src/modules/coreFeatures/ModuleConflictResolver.ts
View File

@@ -44,13 +44,16 @@ export class ModuleConflictResolver extends AbstractModule implements ICoreModul
return MISSING_OR_ERROR;
}
eventHub.emitEvent("conflict-cancelled", path);
this._log(`${title} Conflicted revision deleted ${displayRev(deleteRevision)} ${path}`, LOG_LEVEL_INFO);
this._log(
`${title} Conflicted revision has been deleted ${displayRev(deleteRevision)} ${path}`,
LOG_LEVEL_INFO
);
if ((await this.core.databaseFileAccess.getConflictedRevs(path)).length != 0) {
this._log(`${title} some conflicts are left in ${path}`, LOG_LEVEL_INFO);
return AUTO_MERGED;
}
this._log(`${title} ${path} is a plugin metadata file, no need to write to storage`, LOG_LEVEL_INFO);
if (isPluginMetadata(path) || isCustomisationSyncMetadata(path)) {
this._log(`${title} ${path} is a plugin metadata file, no need to write to storage`, LOG_LEVEL_INFO);
return AUTO_MERGED;
}
// If no conflicts were found, write the resolved content to the storage.
@@ -58,7 +61,8 @@ export class ModuleConflictResolver extends AbstractModule implements ICoreModul
this._log(`Could not write the resolved content to the storage: ${path}`, LOG_LEVEL_NOTICE);
return MISSING_OR_ERROR;
}
this._log(`${path} Has been merged automatically`, LOG_LEVEL_NOTICE);
const level = subTitle.indexOf("same") !== -1 ? LOG_LEVEL_INFO : LOG_LEVEL_NOTICE;
this._log(`${path} has been merged automatically`, level);
return AUTO_MERGED;
}
@@ -108,7 +112,9 @@ export class ModuleConflictResolver extends AbstractModule implements ICoreModul
`${isSame ? "same" : ""}`,
`${isBinary ? "binary" : ""}`,
`${alwaysNewer ? "alwaysNewer" : ""}`,
].join(",");
]
.filter((e) => e.trim())
.join(",");
return await this.core.$$resolveConflictByDeletingRev(path, loser.rev, subTitle);
}
// make diff.

38
src/modules/coreFeatures/ModuleRedFlag.ts
View File

@@ -9,6 +9,7 @@ import {
} from "../../lib/src/common/types.ts";
import { AbstractModule } from "../AbstractModule.ts";
import type { ICoreModule } from "../ModuleTypes.ts";
import { $msg } from "../../lib/src/common/i18n.ts";
export class ModuleRedFlag extends AbstractModule implements ICoreModule {
async isFlagFileExist(path: string) {
@@ -105,16 +106,35 @@ export class ModuleRedFlag extends AbstractModule implements ICoreModule {
`${FLAGMD_REDFLAG3} or ${FLAGMD_REDFLAG3_HR} has been detected! Self-hosted LiveSync will discard the local database and fetch everything from the remote once again.`,
LOG_LEVEL_NOTICE
);
const makeLocalChunkBeforeSync =
(await this.core.confirm.askYesNoDialog(
`Do you want to create local chunks before fetching?
> [!MORE]-
> If creating local chunks before fetching, only the difference between the local and remote will be fetched.
const method1 = $msg("RedFlag.Fetch.Method.FetchSafer");
const method2 = $msg("RedFlag.Fetch.Method.FetchSmoother");
const method3 = $msg("RedFlag.Fetch.Method.FetchTraditional");
const methods = [method1, method2, method3] as const;
const chunkMode = await this.core.confirm.askSelectStringDialogue(
$msg("RedFlag.Fetch.Method.Desc"),
methods,
{
defaultAction: method1,
timeout: 0,
title: $msg("RedFlag.Fetch.Method.Title"),
}
);
let makeLocalChunkBeforeSync = false;
let preventMakeLocalFilesBeforeSync = false;
if (chunkMode === method1) {
preventMakeLocalFilesBeforeSync = true;
} else if (chunkMode === method2) {
makeLocalChunkBeforeSync = true;
} else if (chunkMode === method3) {
// Do nothing.
} else {
this._log("Cancelled the fetch operation", LOG_LEVEL_NOTICE);
return false;
}
await this.core.rebuilder.$fetchLocal(makeLocalChunkBeforeSync, preventMakeLocalFilesBeforeSync);
`,
{ defaultOption: "Yes", title: "Trick to transfer efficiently" }
)) == "yes";
await this.core.rebuilder.$fetchLocal(makeLocalChunkBeforeSync);
await this.deleteRedFlag3();
if (this.settings.suspendFileWatching) {
if (

17
src/modules/essential/ModuleInitializerFile.ts
View File

@@ -81,9 +81,9 @@ export class ModuleInitializerFile extends AbstractModule implements ICoreModule
this._log("Collecting local files on the DB", LOG_LEVEL_VERBOSE);
const _DBEntries = [] as MetaEntry[];
// const _DBEntriesTask = [] as (() => Promise<MetaEntry | false>)[];
let count = 0;
for await (const doc of this.localDatabase.findAllNormalDocs()) {
// Fetch all documents from the database (including conflicts to prevent overwriting).
for await (const doc of this.localDatabase.findAllNormalDocs({ conflicts: true })) {
count++;
if (count % 25 == 0)
this._log(
@@ -180,7 +180,7 @@ export class ModuleInitializerFile extends AbstractModule implements ICoreModule
};
initProcess.push(
runAll("UPDATE DATABASE", filesExistOnlyInStorage, async (e) => {
// console.warn("UPDATE DATABASE", e);
// Exists in storage but not in database.
const file = storageFileNameMap[storageFileNameCI2CS[e]];
if (!this.core.$$isFileSizeExceeded(file.stat.size)) {
const path = file.path;
@@ -195,9 +195,15 @@ export class ModuleInitializerFile extends AbstractModule implements ICoreModule
initProcess.push(
runAll("UPDATE STORAGE", filesExistOnlyInDatabase, async (e) => {
const w = databaseFileNameMap[databaseFileNameCI2CS[e]];
// Exists in database but not in storage.
const path = getPath(w) ?? e;
if (w && !(w.deleted || w._deleted)) {
if (!this.core.$$isFileSizeExceeded(w.size)) {
// Prevent applying the conflicted state to the storage.
if (w._conflicts?.length ?? 0 > 0) {
this._log(`UPDATE STORAGE: ${path} has conflicts. skipped (x)`, LOG_LEVEL_INFO);
return;
}
// await this.pullFile(path, undefined, false, undefined, false);
// Memo: No need to force
await this.core.fileHandler.dbToStorage(path, null, true);
@@ -229,6 +235,11 @@ export class ModuleInitializerFile extends AbstractModule implements ICoreModule
initProcess.push(
runAll("SYNC DATABASE AND STORAGE", fileMap, async (e) => {
const { file, doc } = e;
// Prevent applying the conflicted state to the storage.
if (doc._conflicts?.length ?? 0 > 0) {
this._log(`SYNC DATABASE AND STORAGE: ${file.path} has conflicts. skipped`, LOG_LEVEL_INFO);
return;
}
if (!this.core.$$isFileSizeExceeded(file.stat.size) && !this.core.$$isFileSizeExceeded(doc.size)) {
await this.syncFileBetweenDBandStorage(file, doc);
} else {

224
src/modules/essentialObsidian/ModuleObsidianAPI.ts
View File

@@ -1,7 +1,7 @@
import { AbstractObsidianModule, type IObsidianModule } from "../AbstractObsidianModule.ts";
import { LOG_LEVEL_DEBUG, LOG_LEVEL_VERBOSE } from "octagonal-wheels/common/logger";
import { LOG_LEVEL_DEBUG, LOG_LEVEL_NOTICE, LOG_LEVEL_VERBOSE } from "octagonal-wheels/common/logger";
import { Notice, requestUrl, type RequestUrlParam, type RequestUrlResponse } from "../../deps.ts";
import { type EntryDoc, type FilePathWithPrefix } from "../../lib/src/common/types.ts";
import { type CouchDBCredentials, type EntryDoc, type FilePathWithPrefix } from "../../lib/src/common/types.ts";
import { getPathFromTFile } from "../../common/utils.ts";
import {
disableEncryption,
@@ -13,7 +13,7 @@ import {
import { setNoticeClass } from "../../lib/src/mock_and_interop/wrapper.ts";
import { ObsHttpHandler } from "./APILib/ObsHttpHandler.ts";
import { PouchDB } from "../../lib/src/pouchdb/pouchdb-browser.ts";
import { reactive, reactiveSource } from "octagonal-wheels/dataobject/reactive";
import { AuthorizationHeaderGenerator } from "../../lib/src/replication/httplib.ts";
setNoticeClass(Notice);
@@ -33,10 +33,8 @@ async function fetchByAPI(request: RequestUrlParam): Promise<RequestUrlResponse>
export class ModuleObsidianAPI extends AbstractObsidianModule implements IObsidianModule {
_customHandler!: ObsHttpHandler;
authHeaderSource = reactiveSource<string>("");
authHeader = reactive(() =>
this.authHeaderSource.value == "" ? "" : "Basic " + window.btoa(this.authHeaderSource.value)
);
_authHeader = new AuthorizationHeaderGenerator();
last_successful_post = false;
$$customFetchHandler(): ObsHttpHandler {
@@ -47,31 +45,80 @@ export class ModuleObsidianAPI extends AbstractObsidianModule implements IObsidi
return !this.last_successful_post;
}
async fetchByAPI(
url: string,
localURL: string,
method: string,
authHeader: string,
opts?: RequestInit
): Promise<Response> {
const body = opts?.body as string;
const transformedHeaders = { ...(opts?.headers as Record<string, string>) };
if (authHeader != "") transformedHeaders["authorization"] = authHeader;
delete transformedHeaders["host"];
delete transformedHeaders["Host"];
delete transformedHeaders["content-length"];
delete transformedHeaders["Content-Length"];
const requestParam: RequestUrlParam = {
url,
method: opts?.method,
body: body,
headers: transformedHeaders,
contentType: "application/json",
// contentType: opts.headers,
};
const size = body ? ` (${body.length})` : "";
try {
this.plugin.requestCount.value = this.plugin.requestCount.value + 1;
const r = await fetchByAPI(requestParam);
if (method == "POST" || method == "PUT") {
this.last_successful_post = r.status - (r.status % 100) == 200;
} else {
this.last_successful_post = true;
}
this._log(`HTTP:${method}${size} to:${localURL} -> ${r.status}`, LOG_LEVEL_DEBUG);
return new Response(r.arrayBuffer, {
headers: r.headers,
status: r.status,
statusText: `${r.status}`,
});
} catch (ex) {
this._log(`HTTP:${method}${size} to:${localURL} -> failed`, LOG_LEVEL_VERBOSE);
// limit only in bulk_docs.
if (url.toString().indexOf("_bulk_docs") !== -1) {
this.last_successful_post = false;
}
this._log(ex);
throw ex;
} finally {
this.plugin.responseCount.value = this.plugin.responseCount.value + 1;
}
}
async $$connectRemoteCouchDB(
uri: string,
auth: { username: string; password: string },
auth: CouchDBCredentials,
disableRequestURI: boolean,
passphrase: string | false,
useDynamicIterationCount: boolean,
performSetup: boolean,
skipInfo: boolean,
compression: boolean
compression: boolean,
customHeaders: Record<string, string>
): Promise<string | { db: PouchDB.Database<EntryDoc>; info: PouchDB.Core.DatabaseInfo }> {
if (!isValidRemoteCouchDBURI(uri)) return "Remote URI is not valid";
if (uri.toLowerCase() != uri) return "Remote URI and database name could not contain capital letters.";
if (uri.indexOf(" ") !== -1) return "Remote URI and database name could not contain spaces.";
const userNameAndPassword = auth.username && auth.password ? `${auth.username}:${auth.password}` : "";
if (this.authHeaderSource.value != userNameAndPassword) {
this.authHeaderSource.value = userNameAndPassword;
}
const authHeader = this.authHeader.value;
// const _this = this;
// let authHeader = await this._authHeader.getAuthorizationHeader(auth);
const conf: PouchDB.HttpAdapter.HttpAdapterConfiguration = {
adapter: "http",
auth,
auth: "username" in auth ? auth : undefined,
skip_setup: !performSetup,
fetch: async (url: string | Request, opts?: RequestInit) => {
const authHeader = await this._authHeader.getAuthorizationHeader(auth);
let size = "";
const localURL = url.toString().substring(uri.length);
const method = opts?.method ?? "GET";
@@ -87,88 +134,86 @@ export class ModuleObsidianAPI extends AbstractObsidianModule implements IObsidi
}
size = ` (${opts_length})`;
}
if (!disableRequestURI && typeof url == "string" && typeof (opts?.body ?? "") == "string") {
const body = opts?.body as string;
try {
const headers = new Headers(opts?.headers);
if (customHeaders) {
for (const [key, value] of Object.entries(customHeaders)) {
if (key && value) {
headers.append(key, value);
}
}
}
if (!("username" in auth)) {
headers.append("authorization", authHeader);
}
const transformedHeaders = { ...(opts?.headers as Record<string, string>) };
if (authHeader != "") transformedHeaders["authorization"] = authHeader;
delete transformedHeaders["host"];
delete transformedHeaders["Host"];
delete transformedHeaders["content-length"];
delete transformedHeaders["Content-Length"];
const requestParam: RequestUrlParam = {
url,
method: opts?.method,
body: body,
headers: transformedHeaders,
contentType: "application/json",
// contentType: opts.headers,
};
if (!disableRequestURI && typeof url == "string" && typeof (opts?.body ?? "") == "string") {
// Deprecated configuration, only for backward compatibility.
return await this.fetchByAPI(url, localURL, method, authHeader, { ...opts, headers });
}
// --> native Fetch API.
try {
this.plugin.requestCount.value = this.plugin.requestCount.value + 1;
const r = await fetchByAPI(requestParam);
const response: Response = await fetch(url, { ...opts, headers });
if (method == "POST" || method == "PUT") {
this.last_successful_post = r.status - (r.status % 100) == 200;
this.last_successful_post = response.ok;
} else {
this.last_successful_post = true;
}
this._log(`HTTP:${method}${size} to:${localURL} -> ${r.status}`, LOG_LEVEL_DEBUG);
return new Response(r.arrayBuffer, {
headers: r.headers,
status: r.status,
statusText: `${r.status}`,
});
} catch (ex) {
this._log(`HTTP:${method}${size} to:${localURL} -> failed`, LOG_LEVEL_VERBOSE);
// limit only in bulk_docs.
if (url.toString().indexOf("_bulk_docs") !== -1) {
this.last_successful_post = false;
}
this._log(ex);
throw ex;
} finally {
this.plugin.responseCount.value = this.plugin.responseCount.value + 1;
}
}
try {
if (this.settings.enableDebugTools) {
// Issue #407
(opts!.headers as Headers).append("ngrok-skip-browser-warning", "123");
}
this.plugin.requestCount.value = this.plugin.requestCount.value + 1;
const response: Response = await fetch(url, opts);
if (method == "POST" || method == "PUT") {
this.last_successful_post = response.ok;
} else {
this.last_successful_post = true;
}
this._log(`HTTP:${method}${size} to:${localURL} -> ${response.status}`, LOG_LEVEL_DEBUG);
if (Math.floor(response.status / 100) !== 2) {
if (method != "GET" && localURL.indexOf("/_local/") === -1 && !localURL.endsWith("/")) {
const r = response.clone();
this._log(
`The request may have failed. The reason sent by the server: ${r.status}: ${r.statusText}`
);
try {
this._log(await (await r.blob()).text(), LOG_LEVEL_VERBOSE);
} catch (_) {
this._log("Cloud not parse response", LOG_LEVEL_VERBOSE);
this._log(_, LOG_LEVEL_VERBOSE);
this._log(`HTTP:${method}${size} to:${localURL} -> ${response.status}`, LOG_LEVEL_DEBUG);
if (Math.floor(response.status / 100) !== 2) {
if (response.status == 404) {
if (method === "GET" && localURL.indexOf("/_local/") === -1) {
this._log(
`Just checkpoint or some server information has been missing. The 404 error shown above is not an error.`,
LOG_LEVEL_VERBOSE
);
}
} else {
const r = response.clone();
this._log(
`The request may have failed. The reason sent by the server: ${r.status}: ${r.statusText}`,
LOG_LEVEL_NOTICE
);
try {
const result = await r.text();
this._log(result, LOG_LEVEL_VERBOSE);
} catch (_) {
this._log("Cloud not fetch response body", LOG_LEVEL_VERBOSE);
this._log(_, LOG_LEVEL_VERBOSE);
}
}
} else {
this._log(
`Just checkpoint or some server information has been missing. The 404 error shown above is not an error.`,
LOG_LEVEL_VERBOSE
);
}
return response;
} catch (ex) {
if (ex instanceof TypeError) {
this._log(
"Failed to fetch by native fetch API. Trying to fetch by API to get more information."
);
const resp2 = await this.fetchByAPI(url.toString(), localURL, method, authHeader, {
...opts,
headers,
});
if (resp2.status / 100 == 2) {
this._log(
"The request was successful by API. But the native fetch API failed! Please check CORS settings on the remote database!. While this condition, you cannot enable LiveSync",
LOG_LEVEL_NOTICE
);
return resp2;
}
const r2 = resp2.clone();
const msg = await r2.text();
this._log(`Failed to fetch by API. ${resp2.status}: ${msg}`, LOG_LEVEL_NOTICE);
return resp2;
}
throw ex;
}
return response;
} catch (ex) {
} catch (ex: any) {
this._log(`HTTP:${method}${size} to:${localURL} -> failed`, LOG_LEVEL_VERBOSE);
const msg = ex instanceof Error ? `${ex?.name}:${ex?.message}` : ex?.toString();
this._log(`Failed to fetch: ${msg}`, LOG_LEVEL_NOTICE);
this._log(ex, LOG_LEVEL_VERBOSE);
// limit only in bulk_docs.
if (url.toString().indexOf("_bulk_docs") !== -1) {
this.last_successful_post = false;
@@ -178,6 +223,7 @@ export class ModuleObsidianAPI extends AbstractObsidianModule implements IObsidi
} finally {
this.plugin.responseCount.value = this.plugin.responseCount.value + 1;
}
// return await fetch(url, opts);
},
};
@@ -195,11 +241,7 @@ export class ModuleObsidianAPI extends AbstractObsidianModule implements IObsidi
const info = await db.info();
return { db: db, info: info };
} catch (ex: any) {
let msg = `${ex?.name}:${ex?.message}`;
if (ex?.name == "TypeError" && ex?.message == "Failed to fetch") {
msg +=
"\n**Note** This error caused by many reasons. The only sure thing is you didn't touch the server.\nTo check details, open inspector.";
}
const msg = `${ex?.name}:${ex?.message}`;
this._log(ex, LOG_LEVEL_VERBOSE);
return msg;
}

41
src/modules/extras/ModuleDev.ts
View File

@@ -1,4 +1,4 @@
import { fireAndForget } from "octagonal-wheels/promises";
import { delay, fireAndForget } from "octagonal-wheels/promises";
import { __onMissingTranslation } from "../../lib/src/common/i18n";
import { AbstractObsidianModule, type IObsidianModule } from "../AbstractObsidianModule.ts";
import { LOG_LEVEL_VERBOSE } from "octagonal-wheels/common/logger";
@@ -6,6 +6,7 @@ import { eventHub } from "../../common/events";
import { enableTestFunction } from "./devUtil/testUtils.ts";
import { TestPaneView, VIEW_TYPE_TEST } from "./devUtil/TestPaneView.ts";
import { writable } from "svelte/store";
import type { FilePathWithPrefix } from "../../lib/src/common/types.ts";
export class ModuleDev extends AbstractObsidianModule implements IObsidianModule {
$everyOnloadStart(): Promise<boolean> {
@@ -98,9 +99,41 @@ export class ModuleDev extends AbstractObsidianModule implements IObsidianModule
}
async $everyOnLayoutReady(): Promise<boolean> {
if (!this.settings.enableDebugTools) return Promise.resolve(true);
if (await this.core.storageAccess.isExistsIncludeHidden("_SHOWDIALOGAUTO.md")) {
void this.core.$$showView(VIEW_TYPE_TEST);
}
// if (await this.core.storageAccess.isExistsIncludeHidden("_SHOWDIALOGAUTO.md")) {
// void this.core.$$showView(VIEW_TYPE_TEST);
// }
this.addCommand({
id: "test-create-conflict",
name: "Create conflict",
callback: async () => {
const filename = "test-create-conflict.md";
const content = `# Test create conflict\n\n`;
const w = await this.core.databaseFileAccess.store({
name: filename as FilePathWithPrefix,
path: filename as FilePathWithPrefix,
body: new Blob([content], { type: "text/markdown" }),
stat: {
ctime: new Date().getTime(),
mtime: new Date().getTime(),
size: content.length,
type: "file",
},
});
if (w) {
const id = await this.core.$$path2id(filename as FilePathWithPrefix);
const f = await this.core.localDatabase.getRaw(id);
console.log(f);
console.log(f._rev);
const revConflict = f._rev.split("-")[0] + "-" + (parseInt(f._rev.split("-")[1]) + 1).toString();
console.log(await this.core.localDatabase.bulkDocsRaw([f], { new_edits: false }));
console.log(
await this.core.localDatabase.bulkDocsRaw([{ ...f, _rev: revConflict }], { new_edits: false })
);
}
},
});
await delay(1);
return true;
}
testResults = writable<[boolean, string, string][]>([]);

4
src/modules/extras/ModuleReplicateTest.ts
View File

@@ -160,6 +160,10 @@ export class ModuleReplicateTest extends AbstractObsidianModule implements IObsi
}
async _dumpFileList(outFile?: string) {
if (!this.core || !this.core.storageAccess) {
this._log("No storage access", LOG_LEVEL_INFO);
return;
}
const files = this.core.storageAccess.getFiles();
const out = [] as any[];
const webcrypto = await getWebCrypto();

8
src/modules/features/ModuleObsidianSetting.ts
View File

@@ -94,6 +94,14 @@ export class ModuleObsidianSettings extends AbstractObsidianModule implements IO
region: settings.region,
secretKey: settings.secretKey,
useCustomRequestHandler: settings.useCustomRequestHandler,
bucketCustomHeaders: settings.bucketCustomHeaders,
couchDB_CustomHeaders: settings.couchDB_CustomHeaders,
useJWT: settings.useJWT,
jwtKey: settings.jwtKey,
jwtAlgorithm: settings.jwtAlgorithm,
jwtKid: settings.jwtKid,
jwtExpDuration: settings.jwtExpDuration,
jwtSub: settings.jwtSub,
};
settings.encryptedCouchDBConnection = await this.encryptConfigurationItem(
JSON.stringify(connectionSetting),

5
src/modules/features/ModuleObsidianSettingAsMarkdown.ts
View File

@@ -191,6 +191,11 @@ export class ModuleObsidianSettingsAsMarkdown extends AbstractObsidianModule imp
delete saveData.couchDB_USER;
delete saveData.couchDB_PASSWORD;
delete saveData.passphrase;
delete saveData.jwtKey;
delete saveData.jwtKid;
delete saveData.jwtSub;
delete saveData.couchDB_CustomHeaders;
delete saveData.bucketCustomHeaders;
}
return saveData;
}

21
src/modules/features/ModuleSetupObsidian.ts
View File

@@ -9,7 +9,12 @@ import { configURIBase, configURIBaseQR } from "../../common/types.ts";
// import { PouchDB } from "../../lib/src/pouchdb/pouchdb-browser.js";
import { decrypt, encrypt } from "../../lib/src/encryption/e2ee_v2.ts";
import { fireAndForget } from "../../lib/src/common/utils.ts";
import { EVENT_REQUEST_COPY_SETUP_URI, EVENT_REQUEST_OPEN_SETUP_URI, eventHub } from "../../common/events.ts";
import {
EVENT_REQUEST_COPY_SETUP_URI,
EVENT_REQUEST_OPEN_SETUP_URI,
EVENT_REQUEST_SHOW_SETUP_QR,
eventHub,
} from "../../common/events.ts";
import { AbstractObsidianModule, type IObsidianModule } from "../AbstractObsidianModule.ts";
import { decodeAnyArray, encodeAnyArray } from "../../common/utils.ts";
import qrcode from "qrcode-generator";
@@ -54,6 +59,7 @@ export class ModuleSetupObsidian extends AbstractObsidianModule implements IObsi
});
eventHub.onEvent(EVENT_REQUEST_OPEN_SETUP_URI, () => fireAndForget(() => this.command_openSetupURI()));
eventHub.onEvent(EVENT_REQUEST_COPY_SETUP_URI, () => fireAndForget(() => this.command_copySetupURI()));
eventHub.onEvent(EVENT_REQUEST_SHOW_SETUP_QR, () => fireAndForget(() => this.encodeQR()));
return Promise.resolve(true);
}
async encodeQR() {
@@ -61,14 +67,13 @@ export class ModuleSetupObsidian extends AbstractObsidianModule implements IObsi
const fullIndexes = Object.entries(KeyIndexOfSettings) as [keyof ObsidianLiveSyncSettings, number][];
for (const [settingKey, index] of fullIndexes) {
const settingValue = this.settings[settingKey];
if (index < 0) {
// This setting should be ignored.
continue;
}
settingArr[index] = settingValue;
}
const w = encodeAnyArray(settingArr);
// console.warn(w.length)
// console.warn(w);
// const j = decodeAnyArray(w);
// console.warn(j);
// console.warn(`is equal: ${isObjectDifferent(settingArr, j)}`);
const qr = qrcode(0, "L");
const uri = `${configURIBaseQR}${encodeURIComponent(w)}`;
qr.addData(uri);
@@ -84,6 +89,10 @@ export class ModuleSetupObsidian extends AbstractObsidianModule implements IObsi
const fullIndexes = Object.entries(KeyIndexOfSettings) as [keyof ObsidianLiveSyncSettings, number][];
const newSettings = { ...DEFAULT_SETTINGS } as ObsidianLiveSyncSettings;
for (const [settingKey, index] of fullIndexes) {
if (index < 0) {
// This setting should be ignored.
continue;
}
if (index >= settingArr.length) {
// Possibly a new setting added.
continue;

196
src/modules/features/SettingDialogue/ObsidianLiveSyncSettingTab.ts
View File

@@ -32,10 +32,11 @@ import {
delay,
isDocContentSame,
isObjectDifferent,
parseHeaderValues,
readAsBlob,
sizeToHumanReadable,
} from "../../../lib/src/common/utils.ts";
import { versionNumberString2Number } from "../../../lib/src/string_and_binary/convert.ts";
import { arrayBufferToBase64Single, versionNumberString2Number } from "../../../lib/src/string_and_binary/convert.ts";
import { Logger } from "../../../lib/src/common/logger.ts";
import {
balanceChunkPurgedDBs,
@@ -45,7 +46,7 @@ import {
} from "../../../lib/src/pouchdb/utils_couchdb.ts";
import { testCrypt } from "../../../lib/src/encryption/e2ee_v2.ts";
import ObsidianLiveSyncPlugin from "../../../main.ts";
import { getPath, requestToCouchDB, scheduleTask } from "../../../common/utils.ts";
import { getPath, requestToCouchDBWithCredentials, scheduleTask } from "../../../common/utils.ts";
import { request } from "obsidian";
import { addPrefix, shouldBeIgnored, stripAllPrefixes } from "../../../lib/src/string_and_binary/path.ts";
import MultipleRegExpControl from "./MultipleRegExpControl.svelte";
@@ -72,6 +73,7 @@ import {
EVENT_REQUEST_OPEN_SETUP_URI,
EVENT_REQUEST_RELOAD_SETTING_TAB,
EVENT_REQUEST_RUN_DOCTOR,
EVENT_REQUEST_SHOW_SETUP_QR,
eventHub,
} from "../../../common/events.ts";
import { skipIfDuplicated } from "octagonal-wheels/concurrency/lock";
@@ -81,6 +83,8 @@ import { HiddenFileSync } from "../../../features/HiddenFileSync/CmdHiddenFileSy
import { EVENT_REQUEST_SHOW_HISTORY } from "../../../common/obsidianEvents.ts";
import { LocalDatabaseMaintenance } from "../../../features/LocalDatabaseMainte/CmdLocalDatabaseMainte.ts";
import { mount } from "svelte";
import { getWebCrypto } from "../../../lib/src/mods.ts";
import { generateCredentialObject } from "../../../lib/src/replication/httplib.ts";
export type OnUpdateResult = {
visibility?: boolean;
@@ -814,6 +818,12 @@ export class ObsidianLiveSyncSettingTab extends PluginSettingTab {
return false;
};
const enableOnlySyncDisabled = enableOnly(() => !isAnySyncEnabled());
const combineOnUpdate = (func1: OnUpdateFunc, func2: OnUpdateFunc): OnUpdateFunc => {
return () => ({
...func1(),
...func2(),
});
};
const onlyOnP2POrCouchDB = () =>
({
visibility:
@@ -979,7 +989,16 @@ export class ObsidianLiveSyncSettingTab extends PluginSettingTab {
eventHub.emitEvent(EVENT_REQUEST_COPY_SETUP_URI);
});
});
new Setting(paneEl)
.setName($msg("Setup.ShowQRCode"))
.setDesc($msg("Setup.ShowQRCode.Desc"))
.addButton((text) => {
text.setButtonText($msg("Setup.ShowQRCode")).onClick(() => {
eventHub.emitEvent(EVENT_REQUEST_SHOW_SETUP_QR);
});
});
});
void addPanel(paneEl, $msg("obsidianLiveSyncSettingTab.titleReset")).then((paneEl) => {
new Setting(paneEl)
.setName($msg("obsidianLiveSyncSettingTab.nameDiscardSettings"))
@@ -1167,11 +1186,16 @@ export class ObsidianLiveSyncSettingTab extends PluginSettingTab {
return;
}
// Tip: Add log for cloudant as Logger($msg("obsidianLiveSyncSettingTab.logServerConfigurationCheck"));
const r = await requestToCouchDB(
const customHeaders = parseHeaderValues(this.editingSettings.couchDB_CustomHeaders);
const credential = generateCredentialObject(this.editingSettings);
const r = await requestToCouchDBWithCredentials(
this.editingSettings.couchDB_URI,
this.editingSettings.couchDB_USER,
this.editingSettings.couchDB_PASSWORD,
window.origin
credential,
window.origin,
undefined,
undefined,
undefined,
customHeaders
);
const responseConfig = r.json;
@@ -1184,13 +1208,14 @@ export class ObsidianLiveSyncSettingTab extends PluginSettingTab {
x.querySelector("button")?.addEventListener("click", () => {
fireAndForget(async () => {
Logger($msg("obsidianLiveSyncSettingTab.logCouchDbConfigSet", { title, key, value }));
const res = await requestToCouchDB(
const res = await requestToCouchDBWithCredentials(
this.editingSettings.couchDB_URI,
this.editingSettings.couchDB_USER,
this.editingSettings.couchDB_PASSWORD,
credential,
undefined,
key,
value
value,
undefined,
customHeaders
);
if (res.status == 200) {
Logger(
@@ -1325,11 +1350,14 @@ export class ObsidianLiveSyncSettingTab extends PluginSettingTab {
// Request header check
const origins = ["app://obsidian.md", "capacitor://localhost", "http://localhost"];
for (const org of origins) {
const rr = await requestToCouchDB(
const rr = await requestToCouchDBWithCredentials(
this.editingSettings.couchDB_URI,
this.editingSettings.couchDB_USER,
this.editingSettings.couchDB_PASSWORD,
org
credential,
org,
undefined,
undefined,
undefined,
customHeaders
);
const responseHeaders = Object.fromEntries(
Object.entries(rr.headers).map((e) => {
@@ -1444,6 +1472,10 @@ The pane also can be launched by \`P2P Replicator\` command from the Command Pal
new Setting(paneEl).autoWireText("bucket", { holdValue: true });
new Setting(paneEl).autoWireToggle("useCustomRequestHandler", { holdValue: true });
new Setting(paneEl).autoWireTextArea("bucketCustomHeaders", {
holdValue: true,
placeHolder: "x-custom-header: value\n x-custom-header2: value2",
});
new Setting(paneEl)
.setName($msg("obsidianLiveSyncSettingTab.nameTestConnection"))
.addButton((button) =>
@@ -1465,6 +1497,7 @@ The pane also can be launched by \`P2P Replicator\` command from the Command Pal
"secretKey",
"bucket",
"useCustomRequestHandler",
"bucketCustomHeaders",
])
.addOnUpdate(onlyOnMinIO);
});
@@ -1511,20 +1544,119 @@ The pane also can be launched by \`P2P Replicator\` command from the Command Pal
holdValue: true,
onUpdate: enableOnlySyncDisabled,
});
new Setting(paneEl).autoWireText("couchDB_USER", {
new Setting(paneEl).autoWireToggle("useJWT", {
holdValue: true,
onUpdate: enableOnlySyncDisabled,
});
new Setting(paneEl).autoWireText("couchDB_USER", {
holdValue: true,
onUpdate: combineOnUpdate(
enableOnlySyncDisabled,
visibleOnly(() => !this.editingSettings.useJWT)
),
});
new Setting(paneEl).autoWireText("couchDB_PASSWORD", {
holdValue: true,
isPassword: true,
onUpdate: enableOnlySyncDisabled,
onUpdate: combineOnUpdate(
enableOnlySyncDisabled,
visibleOnly(() => !this.editingSettings.useJWT)
),
});
const algorithms = {
["HS256"]: "HS256",
["HS512"]: "HS512",
["ES256"]: "ES256",
["ES512"]: "ES512",
} as const;
new Setting(paneEl).autoWireDropDown("jwtAlgorithm", {
options: algorithms,
onUpdate: combineOnUpdate(
enableOnlySyncDisabled,
visibleOnly(() => this.editingSettings.useJWT)
),
});
new Setting(paneEl).autoWireTextArea("jwtKey", {
holdValue: true,
onUpdate: combineOnUpdate(
enableOnlySyncDisabled,
visibleOnly(() => this.editingSettings.useJWT)
),
});
// eslint-disable-next-line prefer-const
let generatedKeyDivEl: HTMLDivElement;
new Setting(paneEl)
.setDesc("Generate ES256 Keypair for testing")
.addButton((button) =>
button.setButtonText("Generate").onClick(async () => {
const crypto = await getWebCrypto();
const keyPair = await crypto.subtle.generateKey(
{ name: "ECDSA", namedCurve: "P-256" },
true,
["sign", "verify"]
);
const pubKey = await crypto.subtle.exportKey("spki", keyPair.publicKey);
const privateKey = await crypto.subtle.exportKey("pkcs8", keyPair.privateKey);
const encodedPublicKey = await arrayBufferToBase64Single(pubKey);
const encodedPrivateKey = await arrayBufferToBase64Single(privateKey);
const privateKeyPem = `> -----BEGIN PRIVATE KEY-----\n> ${encodedPrivateKey}\n> -----END PRIVATE KEY-----`;
const publicKeyPem = `> -----BEGIN PUBLIC KEY-----\\n${encodedPublicKey}\\n-----END PUBLIC KEY-----`;
const title = $msg("Setting.GenerateKeyPair.Title");
const msg = $msg("Setting.GenerateKeyPair.Desc", {
public_key: publicKeyPem,
private_key: privateKeyPem,
});
await MarkdownRenderer.render(
this.plugin.app,
"## " + title + "\n\n" + msg,
generatedKeyDivEl,
"/",
this.plugin
);
})
)
.addOnUpdate(
combineOnUpdate(
enableOnlySyncDisabled,
visibleOnly(() => this.editingSettings.useJWT)
)
);
generatedKeyDivEl = this.createEl(
paneEl,
"div",
{ text: "" },
(el) => {},
visibleOnly(() => this.editingSettings.useJWT)
);
new Setting(paneEl).autoWireText("jwtKid", {
holdValue: true,
onUpdate: combineOnUpdate(
enableOnlySyncDisabled,
visibleOnly(() => this.editingSettings.useJWT)
),
});
new Setting(paneEl).autoWireText("jwtSub", {
holdValue: true,
onUpdate: combineOnUpdate(
enableOnlySyncDisabled,
visibleOnly(() => this.editingSettings.useJWT)
),
});
new Setting(paneEl).autoWireNumeric("jwtExpDuration", {
holdValue: true,
onUpdate: combineOnUpdate(
enableOnlySyncDisabled,
visibleOnly(() => this.editingSettings.useJWT)
),
});
new Setting(paneEl).autoWireText("couchDB_DBNAME", {
holdValue: true,
onUpdate: enableOnlySyncDisabled,
});
new Setting(paneEl).autoWireTextArea("couchDB_CustomHeaders", { holdValue: true });
new Setting(paneEl)
.setName($msg("obsidianLiveSyncSettingTab.nameTestDatabaseConnection"))
.setClass("wizardHidden")
@@ -1562,6 +1694,13 @@ The pane also can be launched by \`P2P Replicator\` command from the Command Pal
"couchDB_USER",
"couchDB_PASSWORD",
"couchDB_DBNAME",
"jwtAlgorithm",
"jwtExpDuration",
"jwtKey",
"jwtSub",
"jwtKid",
"useJWT",
"couchDB_CustomHeaders",
])
.addOnUpdate(onlyOnCouchDB);
});
@@ -1985,7 +2124,6 @@ The pane also can be launched by \`P2P Replicator\` command from the Command Pal
LEVEL_ADVANCED
).then((paneEl) => {
paneEl.addClass("wizardHidden");
new Setting(paneEl)
.autoWireText("settingSyncFile", { holdValue: true })
.addApplyButton(["settingSyncFile"]);
@@ -2279,11 +2417,16 @@ The pane also can be launched by \`P2P Replicator\` command from the Command Pal
const REDACTED = "𝑅𝐸𝐷𝐴𝐶𝑇𝐸𝐷";
if (this.editingSettings.remoteType == REMOTE_COUCHDB) {
try {
const r = await requestToCouchDB(
const credential = generateCredentialObject(this.editingSettings);
const customHeaders = parseHeaderValues(this.editingSettings.couchDB_CustomHeaders);
const r = await requestToCouchDBWithCredentials(
this.editingSettings.couchDB_URI,
this.editingSettings.couchDB_USER,
this.editingSettings.couchDB_PASSWORD,
window.origin
credential,
window.origin,
undefined,
undefined,
undefined,
customHeaders
);
Logger(JSON.stringify(r.json, null, 2));
@@ -2330,6 +2473,11 @@ The pane also can be launched by \`P2P Replicator\` command from the Command Pal
pluginConfig.P2P_passphrase = redact(pluginConfig.P2P_passphrase);
pluginConfig.P2P_roomID = redact(pluginConfig.P2P_roomID);
pluginConfig.P2P_relays = redact(pluginConfig.P2P_relays);
pluginConfig.jwtKey = redact(pluginConfig.jwtKey);
pluginConfig.jwtSub = redact(pluginConfig.jwtSub);
pluginConfig.jwtKid = redact(pluginConfig.jwtKid);
pluginConfig.bucketCustomHeaders = redact(pluginConfig.bucketCustomHeaders);
pluginConfig.couchDB_CustomHeaders = redact(pluginConfig.couchDB_CustomHeaders);
const endpoint = pluginConfig.endpoint;
if (endpoint == "") {
pluginConfig.endpoint = "Not configured or AWS";
@@ -3383,6 +3531,7 @@ ${stringifyYaml(pluginConfig)}`;
const region = this.plugin.settings.region;
const endpoint = this.plugin.settings.endpoint;
const useCustomRequestHandler = this.plugin.settings.useCustomRequestHandler;
const customHeaders = this.plugin.settings.bucketCustomHeaders;
return new JournalSyncMinio(
id,
key,
@@ -3391,7 +3540,8 @@ ${stringifyYaml(pluginConfig)}`;
this.plugin.simpleStore,
this.plugin,
useCustomRequestHandler,
region
region,
customHeaders
);
}
async resetRemoteBucket() {

2
src/modules/interfaces/DatabaseRebuilder.ts
View File

@@ -4,7 +4,7 @@ export interface Rebuilder {
): Promise<void>;
$rebuildRemote(): Promise<void>;
$rebuildEverything(): Promise<void>;
$fetchLocal(makeLocalChunkBeforeSync?: boolean): Promise<void>;
$fetchLocal(makeLocalChunkBeforeSync?: boolean, preventMakeLocalFilesBeforeSync?: boolean): Promise<void>;
scheduleRebuild(): Promise<void>;
scheduleFetch(): Promise<void>;

8
styles.css
View File

@@ -443,4 +443,12 @@ span.ls-mark-cr::after {
justify-content: center;
align-items: center;
max-width: max-content;
}
.sls-keypair pre {
max-width: 100%;
overflow-x: auto;
white-space: pre-wrap;
word-break: break-all;
}

192
updates.md
View File

@@ -10,138 +10,88 @@ Nevertheless, that being said, to be more honest, I still have not decided what
Note: Already you have noticed this, but let me mention it again, this is a significantly large update. If you have noticed anything, please let me know. I will try to fix it as soon as possible (Some address is on my [profile](https://github.com/vrtmrz)).
## 0.24.19
### New Feature
- Now we can generate a QR Code for transferring the configuration to another device.
- This QR Code can be scanned by the camera app or something QR Code Reader of another device, and via Obsidian URL, the configuration will be transferred.
- Note: This QR Code is not encrypted. So, please be careful when transferring the configuration.
## 0.24.18
### Fixed
- Now no chunk creation errors will be raised after switching `Compute revisions for chunks`.
- Some invisible file can be handled correctly (e.g., `writing-goals-history.csv`).
- Fetching configuration from the server is now saves the configuration immediately (if we are not in the wizard).
## 0.24.25
### Improved
- Mismatched configuration dialogue is now more informative, and rewritten to more user-friendly.
- Applying configuration mismatch is now without rebuilding (at our own risks).
- Now, rebuilding is decided more fine grained.
- Peer-to-peer synchronisation has been got more robust.
### Improved internally
### Fixed
- Translations can be nested. i.e., task:`Some procedure`, check: `%{task} checking`, checkfailed: `%{check} failed` produces `Some procedure checking failed`.
- Max to 10 levels of nesting
- No longer broken falsy values in settings during set-up by the QR code generation.
## 0.24.17
### Refactored
Confession. I got the default values wrong. So scary and sorry.
- Some `window` references now have pointed to `globalThis`.
- Some sloppy-import has been fixed.
- A server side implementation `Synchromesh` has been suffixed with `deno` instead of `server` now.
## 0.24.24
### Fixed
- No longer broken JSON files including `\n`, during the bucket synchronisation. (#623)
- Custom headers and JWT tokens are now correctly sent to the server during configuration checking. (#624)
### Improved
- Bucket synchronisation has been enhanced for better performance and reliability.
- Now less duplicated chunks are sent to the server.
Note: If you have encountered about too less chunks, please let me know. However, you can send it to the server by `Overwrite remote`.
- Fetching conflicted files from the server is now more reliable.
- Dependent libraries have been updated to the latest version.
- Also, let me know if you have encountered any issues with this update. Especially you are using a device that has been in use for a little longer.
## 0.24.23
### New Feature
- Now, we can send custom headers to the server.
- They can be sent to either CouchDB or Object Storage.
- Authentication with JWT in CouchDB is now supported.
- I will describe steps later, but please refer to the [CouchDB document](https://docs.couchdb.org/en/stable/config/auth.html#authentication-configuration).
- A JWT keypair for testing can be generated in the setting dialogue.
### Improved
- The QR Code for set-up can be shown also from the setting dialogue now.
- Conflict checking for preventing unexpected overwriting on the boot-up process has been quite faster.
### Fixed
- Some bugs on Dev and Testing modules have been fixed.
## 0.24.22 ~~0.24.21~~
(Really sorry for the confusion. I have got a miss at releasing...).
### Fixed
- No longer conflicted files are handled in the boot-up process. No more unexpected overwriting.
- It ignores `Always overwrite with a newer file`, and always be prevented for the safety. Please pick it manually or open the file.
- Some log messages on conflict resolution has been corrected.
- Automatic merge notifications, displayed on the grounds of `same`, have been degraded to logs.
### Improved
- Now we can fetch the remote database with keeping local files completely intact.
- In new option, all files are stored into the local database before the fetching, and will be merged automatically or detected as conflicts.
- The dialogue presenting options when performing `Fetch` are now more informative.
### Refactored
- Some class methods have been fixed its arguments to be more consistent.
- Types have been defined for some conditional results.
## 0.24.20
### Improved
- Now we can see the detail of `TypeError` using Obsidian API during remote database access.
### Behaviour and default changed
- **NOW INDEED AND ACTUALLY** `Compute revisions for chunks` are backed into enabled again. it is necessary for garbage collection of chunks.
- As far as existing users are concerned, this will not automatically change, but the Doctor will inform us.
## 0.24.16
### Improved
#### Peer-to-Peer
- Now peer-to-peer synchronisation checks the settings are compatible with each other.
- No longer unexpected database broken, phew.
- Peer-to-peer synchronisation now handles the platform and detects pseudo-clients.
- Pseudo clients will not decrypt/encrypt anything, just relay the data. Hence, always settings are not compatible. Therefore, we have to accept the incompatibility for pseudo clients.
#### General
- New migration method has been implemented, that called `Doctor`.
- `Doctor` checks the difference between the ideal and actual values and encourages corrective action. To facilitate our decision, the reasons for this and the recommendations are also presented.
- This can be used not only during migration. We can invoke the doctor from the settings for trouble-shooting.
- The minimum interval for replication to be caused when an event occurs can now be configurable.
- Some detail note has been added and change nuance about the `Report` in the setting dialogue, which had less informative.
### Behaviour and default changed
- `Compute revisions for chunks` are backed into enabled again. it is necessary for garbage collection of chunks.
- As far as existing users are concerned, this will not automatically change, but the Doctor will inform us.
### Refactored
- Platform specific codes are more separated. No longer `node` modules were used in the browser and Obsidian.
## 0.24.15
### Fixed
- Now, even without WeakRef, Polyfill is used and the whole thing works without error. However, if you can switch WebView Engine, it is recommended to switch to a WebView Engine that supports WeakRef.
## 0.24.14
### Fixed
- Resolving conflicts of JSON files (and sensibly merging them) is now working fine, again!
- And, failure logs are more informative.
- More robust to release the event listeners on unwatching the local database.
### Refactored
- JSON file conflict resolution dialogue has been rewritten into svelte v5.
- Upgrade eslint.
- Remove unnecessary pragma comments for eslint.
## 0.24.13
Sorry for the lack of replies. The ones that were not good are popping up, so I am just going to go ahead and get this one... However, they realised that refactoring and restructuring is about clarifying the problem. Your patience and understanding is much appreciated.
### Fixed
#### General Replication
- No longer unexpected errors occur when the replication is stopped during for some reason (e.g., network disconnection).
#### Peer-to-Peer Synchronisation
- Set-up process will not receive data from unexpected sources.
- No longer resource leaks while enabling the `broadcasting changes`
- Logs are less verbose.
- Received data is now correctly dispatched to other devices.
- `Timeout` error now more informative.
- No longer timeout error occurs for reporting the progress to other devices.
- Decision dialogues for the same thing are not shown multiply at the same time anymore.
- Disconnection of the peer-to-peer synchronisation is now more robust and less error-prone.
#### Webpeer
- Now we can toggle Peers' configuration.
### Refactored
- Cross-platform compatibility layer has been improved.
- Common events are moved to the common library.
- Displaying replication status of the peer-to-peer synchronisation is separated from the main-log-logic.
- Some file names have been changed to be more consistent.
## 0.24.12
I created a SPA called [webpeer](https://github.com/vrtmrz/livesync-commonlib/tree/main/apps/webpeer) (well, right... I will think of a name again), which replaces the server when using Peer-to-Peer synchronisation. This is a pseudo-client that appears to other devices as if it were one of the clients. . As with the client, it receives and sends data without storing it as a file.
And, this is just a single web page, without any server-side code. It is a static web page that can be hosted on any static web server, such as GitHub Pages, Netlify, or Vercel. All you have to do is to open the page and enter several items, and leave it open.
### Fixed
- No longer unnecessary acknowledgements are sent when starting peer-to-peer synchronisation.
### Refactored
- Platform impedance-matching-layer has been improved.
- And you can see the actual usage of this on [webpeer](https://github.com/vrtmrz/livesync-commonlib/tree/main/apps/webpeer) that a pseudo client for peer-to-peer synchronisation.
- Some UIs have been got isomorphic among Obsidian and web applications (for `webpeer`).
Older notes are in [updates_old.md](https://github.com/vrtmrz/obsidian-livesync/blob/main/updates_old.md).

129
updates_old.md
View File

@@ -13,6 +13,135 @@ Finally, I would like to once again express my respect and gratitude to all of y
Thank you, and I hope your troubles will be resolved!
---
## 0.24.19
### New Feature
- Now we can generate a QR Code for transferring the configuration to another device.
- This QR Code can be scanned by the camera app or something QR Code Reader of another device, and via Obsidian URL, the configuration will be transferred.
- Note: This QR Code is not encrypted. So, please be careful when transferring the configuration.
## 0.24.18
### Fixed
- Now no chunk creation errors will be raised after switching `Compute revisions for chunks`.
- Some invisible file can be handled correctly (e.g., `writing-goals-history.csv`).
- Fetching configuration from the server is now saves the configuration immediately (if we are not in the wizard).
### Improved
- Mismatched configuration dialogue is now more informative, and rewritten to more user-friendly.
- Applying configuration mismatch is now without rebuilding (at our own risks).
- Now, rebuilding is decided more fine grained.
### Improved internally
- Translations can be nested. i.e., task:`Some procedure`, check: `%{task} checking`, checkfailed: `%{check} failed` produces `Some procedure checking failed`.
- Max to 10 levels of nesting
## 0.24.17
Confession. I got the default values wrong. So scary and sorry.
## 0.24.16
### Improved
#### Peer-to-Peer
- Now peer-to-peer synchronisation checks the settings are compatible with each other.
- No longer unexpected database broken, phew.
- Peer-to-peer synchronisation now handles the platform and detects pseudo-clients.
- Pseudo clients will not decrypt/encrypt anything, just relay the data. Hence, always settings are not compatible. Therefore, we have to accept the incompatibility for pseudo clients.
#### General
- New migration method has been implemented, that called `Doctor`.
- `Doctor` checks the difference between the ideal and actual values and encourages corrective action. To facilitate our decision, the reasons for this and the recommendations are also presented.
- This can be used not only during migration. We can invoke the doctor from the settings for trouble-shooting.
- The minimum interval for replication to be caused when an event occurs can now be configurable.
- Some detail note has been added and change nuance about the `Report` in the setting dialogue, which had less informative.
### Behaviour and default changed
- `Compute revisions for chunks` are backed into enabled again. it is necessary for garbage collection of chunks.
- As far as existing users are concerned, this will not automatically change, but the Doctor will inform us.
### Refactored
- Platform specific codes are more separated. No longer `node` modules were used in the browser and Obsidian.
## 0.24.15
### Fixed
- Now, even without WeakRef, Polyfill is used and the whole thing works without error. However, if you can switch WebView Engine, it is recommended to switch to a WebView Engine that supports WeakRef.
## 0.24.14
### Fixed
- Resolving conflicts of JSON files (and sensibly merging them) is now working fine, again!
- And, failure logs are more informative.
- More robust to release the event listeners on unwatching the local database.
### Refactored
- JSON file conflict resolution dialogue has been rewritten into svelte v5.
- Upgrade eslint.
- Remove unnecessary pragma comments for eslint.
## 0.24.13
Sorry for the lack of replies. The ones that were not good are popping up, so I am just going to go ahead and get this one... However, they realised that refactoring and restructuring is about clarifying the problem. Your patience and understanding is much appreciated.
### Fixed
#### General Replication
- No longer unexpected errors occur when the replication is stopped during for some reason (e.g., network disconnection).
#### Peer-to-Peer Synchronisation
- Set-up process will not receive data from unexpected sources.
- No longer resource leaks while enabling the `broadcasting changes`
- Logs are less verbose.
- Received data is now correctly dispatched to other devices.
- `Timeout` error now more informative.
- No longer timeout error occurs for reporting the progress to other devices.
- Decision dialogues for the same thing are not shown multiply at the same time anymore.
- Disconnection of the peer-to-peer synchronisation is now more robust and less error-prone.
#### Webpeer
- Now we can toggle Peers' configuration.
### Refactored
- Cross-platform compatibility layer has been improved.
- Common events are moved to the common library.
- Displaying replication status of the peer-to-peer synchronisation is separated from the main-log-logic.
- Some file names have been changed to be more consistent.
## 0.24.12
I created a SPA called [webpeer](https://github.com/vrtmrz/livesync-commonlib/tree/main/apps/webpeer) (well, right... I will think of a name again), which replaces the server when using Peer-to-Peer synchronisation. This is a pseudo-client that appears to other devices as if it were one of the clients. . As with the client, it receives and sends data without storing it as a file.
And, this is just a single web page, without any server-side code. It is a static web page that can be hosted on any static web server, such as GitHub Pages, Netlify, or Vercel. All you have to do is to open the page and enter several items, and leave it open.
### Fixed
- No longer unnecessary acknowledgements are sent when starting peer-to-peer synchronisation.
### Refactored
- Platform impedance-matching-layer has been improved.
- And you can see the actual usage of this on [webpeer](https://github.com/vrtmrz/livesync-commonlib/tree/main/apps/webpeer) that a pseudo client for peer-to-peer synchronisation.
- Some UIs have been got isomorphic among Obsidian and web applications (for `webpeer`).
## 0.24.11