public/obsidian-livesync
1
0
Fork 0
mirror of https://github.com/vrtmrz/obsidian-livesync.git synced 2026-05-21 14:51:34 +00:00
Code Issues Packages Projects Releases Wiki Activity

Compare commits

base: public:0.24.12
Branches Tags
public:main public:feat_tweak_auto_adjust public:test_real_obsidian public:beta public:enhance_fix_p2p public:v0_25_62 public:update_lib public:feat-userhashsalt public:auto_conf_2 public:auto_conf public:alpha public:refactor_services_2 public:refactor_services public:dev public:svelteui public:forprivacy
public:0.25.67 public:0.25.66 public:0.25.65 public:0.25.64-patch1 public:0.25.64 public:0.25.63 public:0.25.62 public:0.25.61 public:0.25.60 public:0.25.59 public:0.25.58 public:0.25.57 public:0.25.56+patched5 public:0.25.56+patched4 public:0.25.56+patched3 public:0.25.56+patched2 public:0.25.56+patched1 public:0.25.56 public:0.25.55 public:0.25.54 public:0.25.53 public:0.25.52-patched-3 public:0.25.52-patched-2 public:0.25.52-patched-1 public:0.25.52 public:0.25.51 public:0.25.50 public:0.25.49 public:0.25.48 public:0.25.47 public:0.25.46 public:0.25.45 public:0.25.44 public:0.25.43-patched-9 public:0.25.43-patched-8 public:0.25.43-patched-7 public:0.25.43-patched-6 public:0.25.43-patched-5 public:0.25.43-patched-4 public:0.25.43-patched-3 public:0.25.43-patched-2 public:0.25.43-patched-1 public:0.25.43 public:0.25.42 public:0.25.41-patched-3 public:0.25.41-patched-2 public:0.25.41-patched-1 public:0.25.41 public:0.25.40 public:0.25.39 public:0.25.38 public:0.25.37 public:0.25.36 public:0.25.35 public:0.25.34 public:0.25.33 public:0.25.32 public:0.25.31 public:0.25.30 public:0.25.30-dev public:0.25.29 public:0.25.28 public:0.25.27 public:0.25.26 public:0.25.25 public:0.25.24 public:0.25.24.beta3 public:0.25.24.beta2 public:0.25.24.beta1 public:0.25.23 public:0.25.23.beta1 public:0.25.22 public:0.25.21 public:0.25.21.beta2 public:0.25.21.beta1 public:0.25.20 public:0.25.19 public:0.25.18 public:0.25.17 public:0.25.16 public:0.25.15 public:0.25.14 public:0.25.13 public:0.25.12 public:0.25.11 public:0.25.10 public:0.25.9 public:0.25.8 public:0.25.7 public:0.25.6 public:0.25.5 public:0.25.4 public:0.25.3 public:0.25.2 public:0.25.1 public:0.25.0 public:0.25.0-beta1 public:0.24.31 public:0.24.30 public:0.24.29 public:0.24.28 public:0.24.27 public:0.24.26 public:0.24.25 public:0.24.24 public:0.24.23 public:0.24.22 public:0.24.21 public:0.24.20 public:0.24.19 public:0.24.18 public:0.24.17 public:0.24.16 public:0.24.15 public:0.24.14 public:0.24.13 public:0.24.12 public:0.24.11 public:0.24.10 public:0.24.9 public:0.24.8 public:0.24.7 public:0.24.6 public:0.24.5 public:0.24.4 public:0.24.3 public:0.24.2 public:0.24.1 public:0.24.0 public:0.24.0.dev-rc8 public:0.24.0.dev-rc7 public:0.24.0.dev-rc6 public:0.24.0.dev-rc5 public:0.24.0.dev-rc4 public:0.24.0.dev-rc3 public:0.24.0.dev-rc2 public:0.23.23 public:0.23.22 public:0.23.21 public:0.23.20 public:0.23.19 public:0.23.18 public:0.23.17 public:0.23.16 public:0.23.15 public:0.23.14 public:0.23.13 public:0.23.12 public:0.23.11 public:0.23.10 public:0.23.9 public:0.23.8 public:0.23.7 public:0.23.6 public:0.23.5 public:0.23.4 public:0.23.3 public:0.23.2 public:0.23.1 public:0.23.0 public:0.22.19 public:0.22.18 public:0.22.17 public:0.22.16 public:0.22.15 public:0.22.14 public:0.22.13 public:0.22.12 public:0.22.11 public:0.22.10 public:0.22.9 public:0.22.8 public:0.22.7 public:0.22.6 public:0.22.5 public:0.22.4 public:0.22.3 public:0.22.2 public:0.22.1 public:0.22.0 public:0.21.5 public:0.21.4 public:0.21.3 public:0.21.2 public:0.21.1 public:0.21.0 public:0.20.7 public:0.20.6 public:0.20.5 public:0.20.4 public:0.20.3 public:0.20.2 public:0.20.1 public:0.20.0 public:0.19.23 public:0.19.22 public:0.19.21 public:0.19.20 public:0.19.19 public:0.19.18 public:0.19.17 public:0.19.16 public:0.19.15 public:0.19.14 public:0.19.13 public:0.19.12 public:0.19.11 public:0.19.10 public:0.19.9 public:0.19.8 public:0.19.7 public:0.19.6 public:0.19.5 public:0.19.4 public:0.19.3 public:0.19.2 public:0.19.1 public:0.19.0 public:0.18.6 public:0.18.5 public:0.18.4 public:0.18.3 public:0.18.2 public:0.18.1 public:0.18.0 public:0.17.34 public:0.17.33 public:0.17.32 public:0.17.31 public:0.17.30 public:0.17.29 public:0.17.28 public:0.17.27 public:0.17.26 public:0.17.25 public:0.17.24 public:0.17.23 public:0.17.22 public:0.17.21 public:0.17.20 public:0.17.19 public:0.17.18 public:0.17.17 public:0.17.16 public:0.17.15 public:0.17.14 public:0.17.13 public:0.17.12 public:0.17.11 public:0.17.10 public:0.17.9 public:0.17.8 public:0.17.7 public:0.17.6 public:0.17.5 public:0.17.4 public:0.17.3 public:0.17.2 public:0.17.1 public:0.17.0 public:0.16.8 public:0.16.7 public:0.16.6 public:0.16.5 public:0.16.4 public:0.16.3 public:0.16.2 public:0.16.1 public:0.16.0 public:0.15.10 public:0.15.9 public:0.15.8 public:0.15.7 public:0.15.6 public:0.15.5 public:0.15.4 public:0.15.3 public:0.15.2 public:0.15.1 public:0.15.0 public:0.14.8 public:0.14.7 public:0.14.6 public:0.14.5 public:0.14.4 public:0.14.3 public:0.14.2 public:0.14.1 public:0.14.0 public:0.13.4 public:0.13.3 public:0.13.2 public:0.13.1 public:0.13.0 public:0.12.5 public:0.12.4 public:0.12.3 public:0.12.2 public:0.12.1 public:0.12.0 public:0.11.10 public:0.11.9 public:0.11.8 public:0.11.7 public:0.11.6 public:0.11.5 public:0.11.4 public:0.11.3 public:0.11.2 public:0.11.1 public:0.11.0 public:0.10.1 public:0.10.0 public:0.9.0 public:0.8.8 public:0.8.7 public:0.8.6 public:0.8.5 public:0.8.3 public:0.8.2 public:0.8.1 public:0.8.0 public:0.7.2 public:0.7.1 public:0.7.0 public:0.6.1 public:0.6.0 public:0.5.0 public:0.4.1 public:0.4.0 public:0.3.1 public:0.3.0 public:0.2.1 public:0.2.0 public:0.1.26 public:0.1.25 public:0.1.24 public:0.1.23 public:0.1.22 public:0.1.21 public:0.1.20 public:0.1.19 public:0.1.18 public:0.1.17 public:0.1.16 public:0.1.15 public:0.1.14 public:0.1.13 public:0.1.12 public:0.1.11 public:0.1.10 public:0.1.9 public:0.1.8 public:0.1.7 public:0.1.6 public:0.1.5 public:0.1.4 public:0.1.3 public:0.1.2 public:0.1.1 public:0.1.0 public:0.0.9 public:0.0.8 public:0.0.7 public:0.0.6
...
compare: public:0.24.24
Branches Tags
public:main public:feat_tweak_auto_adjust public:test_real_obsidian public:beta public:enhance_fix_p2p public:v0_25_62 public:update_lib public:feat-userhashsalt public:auto_conf_2 public:auto_conf public:alpha public:refactor_services_2 public:refactor_services public:dev public:svelteui public:forprivacy
public:0.25.67 public:0.25.66 public:0.25.65 public:0.25.64-patch1 public:0.25.64 public:0.25.63 public:0.25.62 public:0.25.61 public:0.25.60 public:0.25.59 public:0.25.58 public:0.25.57 public:0.25.56+patched5 public:0.25.56+patched4 public:0.25.56+patched3 public:0.25.56+patched2 public:0.25.56+patched1 public:0.25.56 public:0.25.55 public:0.25.54 public:0.25.53 public:0.25.52-patched-3 public:0.25.52-patched-2 public:0.25.52-patched-1 public:0.25.52 public:0.25.51 public:0.25.50 public:0.25.49 public:0.25.48 public:0.25.47 public:0.25.46 public:0.25.45 public:0.25.44 public:0.25.43-patched-9 public:0.25.43-patched-8 public:0.25.43-patched-7 public:0.25.43-patched-6 public:0.25.43-patched-5 public:0.25.43-patched-4 public:0.25.43-patched-3 public:0.25.43-patched-2 public:0.25.43-patched-1 public:0.25.43 public:0.25.42 public:0.25.41-patched-3 public:0.25.41-patched-2 public:0.25.41-patched-1 public:0.25.41 public:0.25.40 public:0.25.39 public:0.25.38 public:0.25.37 public:0.25.36 public:0.25.35 public:0.25.34 public:0.25.33 public:0.25.32 public:0.25.31 public:0.25.30 public:0.25.30-dev public:0.25.29 public:0.25.28 public:0.25.27 public:0.25.26 public:0.25.25 public:0.25.24 public:0.25.24.beta3 public:0.25.24.beta2 public:0.25.24.beta1 public:0.25.23 public:0.25.23.beta1 public:0.25.22 public:0.25.21 public:0.25.21.beta2 public:0.25.21.beta1 public:0.25.20 public:0.25.19 public:0.25.18 public:0.25.17 public:0.25.16 public:0.25.15 public:0.25.14 public:0.25.13 public:0.25.12 public:0.25.11 public:0.25.10 public:0.25.9 public:0.25.8 public:0.25.7 public:0.25.6 public:0.25.5 public:0.25.4 public:0.25.3 public:0.25.2 public:0.25.1 public:0.25.0 public:0.25.0-beta1 public:0.24.31 public:0.24.30 public:0.24.29 public:0.24.28 public:0.24.27 public:0.24.26 public:0.24.25 public:0.24.24 public:0.24.23 public:0.24.22 public:0.24.21 public:0.24.20 public:0.24.19 public:0.24.18 public:0.24.17 public:0.24.16 public:0.24.15 public:0.24.14 public:0.24.13 public:0.24.12 public:0.24.11 public:0.24.10 public:0.24.9 public:0.24.8 public:0.24.7 public:0.24.6 public:0.24.5 public:0.24.4 public:0.24.3 public:0.24.2 public:0.24.1 public:0.24.0 public:0.24.0.dev-rc8 public:0.24.0.dev-rc7 public:0.24.0.dev-rc6 public:0.24.0.dev-rc5 public:0.24.0.dev-rc4 public:0.24.0.dev-rc3 public:0.24.0.dev-rc2 public:0.23.23 public:0.23.22 public:0.23.21 public:0.23.20 public:0.23.19 public:0.23.18 public:0.23.17 public:0.23.16 public:0.23.15 public:0.23.14 public:0.23.13 public:0.23.12 public:0.23.11 public:0.23.10 public:0.23.9 public:0.23.8 public:0.23.7 public:0.23.6 public:0.23.5 public:0.23.4 public:0.23.3 public:0.23.2 public:0.23.1 public:0.23.0 public:0.22.19 public:0.22.18 public:0.22.17 public:0.22.16 public:0.22.15 public:0.22.14 public:0.22.13 public:0.22.12 public:0.22.11 public:0.22.10 public:0.22.9 public:0.22.8 public:0.22.7 public:0.22.6 public:0.22.5 public:0.22.4 public:0.22.3 public:0.22.2 public:0.22.1 public:0.22.0 public:0.21.5 public:0.21.4 public:0.21.3 public:0.21.2 public:0.21.1 public:0.21.0 public:0.20.7 public:0.20.6 public:0.20.5 public:0.20.4 public:0.20.3 public:0.20.2 public:0.20.1 public:0.20.0 public:0.19.23 public:0.19.22 public:0.19.21 public:0.19.20 public:0.19.19 public:0.19.18 public:0.19.17 public:0.19.16 public:0.19.15 public:0.19.14 public:0.19.13 public:0.19.12 public:0.19.11 public:0.19.10 public:0.19.9 public:0.19.8 public:0.19.7 public:0.19.6 public:0.19.5 public:0.19.4 public:0.19.3 public:0.19.2 public:0.19.1 public:0.19.0 public:0.18.6 public:0.18.5 public:0.18.4 public:0.18.3 public:0.18.2 public:0.18.1 public:0.18.0 public:0.17.34 public:0.17.33 public:0.17.32 public:0.17.31 public:0.17.30 public:0.17.29 public:0.17.28 public:0.17.27 public:0.17.26 public:0.17.25 public:0.17.24 public:0.17.23 public:0.17.22 public:0.17.21 public:0.17.20 public:0.17.19 public:0.17.18 public:0.17.17 public:0.17.16 public:0.17.15 public:0.17.14 public:0.17.13 public:0.17.12 public:0.17.11 public:0.17.10 public:0.17.9 public:0.17.8 public:0.17.7 public:0.17.6 public:0.17.5 public:0.17.4 public:0.17.3 public:0.17.2 public:0.17.1 public:0.17.0 public:0.16.8 public:0.16.7 public:0.16.6 public:0.16.5 public:0.16.4 public:0.16.3 public:0.16.2 public:0.16.1 public:0.16.0 public:0.15.10 public:0.15.9 public:0.15.8 public:0.15.7 public:0.15.6 public:0.15.5 public:0.15.4 public:0.15.3 public:0.15.2 public:0.15.1 public:0.15.0 public:0.14.8 public:0.14.7 public:0.14.6 public:0.14.5 public:0.14.4 public:0.14.3 public:0.14.2 public:0.14.1 public:0.14.0 public:0.13.4 public:0.13.3 public:0.13.2 public:0.13.1 public:0.13.0 public:0.12.5 public:0.12.4 public:0.12.3 public:0.12.2 public:0.12.1 public:0.12.0 public:0.11.10 public:0.11.9 public:0.11.8 public:0.11.7 public:0.11.6 public:0.11.5 public:0.11.4 public:0.11.3 public:0.11.2 public:0.11.1 public:0.11.0 public:0.10.1 public:0.10.0 public:0.9.0 public:0.8.8 public:0.8.7 public:0.8.6 public:0.8.5 public:0.8.3 public:0.8.2 public:0.8.1 public:0.8.0 public:0.7.2 public:0.7.1 public:0.7.0 public:0.6.1 public:0.6.0 public:0.5.0 public:0.4.1 public:0.4.0 public:0.3.1 public:0.3.0 public:0.2.1 public:0.2.0 public:0.1.26 public:0.1.25 public:0.1.24 public:0.1.23 public:0.1.22 public:0.1.21 public:0.1.20 public:0.1.19 public:0.1.18 public:0.1.17 public:0.1.16 public:0.1.15 public:0.1.14 public:0.1.13 public:0.1.12 public:0.1.11 public:0.1.10 public:0.1.9 public:0.1.8 public:0.1.7 public:0.1.6 public:0.1.5 public:0.1.4 public:0.1.3 public:0.1.2 public:0.1.1 public:0.1.0 public:0.0.9 public:0.0.8 public:0.0.7 public:0.0.6

35 Commits
0.24.12 ... 0.24.24

Author SHA1 Message Date
vorotamoroz
944aa846c4 bump 2025-04-15 11:10:37 +01:00
vorotamoroz
abca808e29 ### Fixed 
- No longer broken JSON files including `\n`, during the bucket synchronisation. (#623)
- Custom headers and JWT tokens are now correctly sent to the server during configuration checking. (#624)

### Improved

- Bucket synchronisation has been enhanced for better performance and reliability.
    - Now less duplicated chunks are sent to the server.
    - Fetching conflicted files from the server is now more reliable.
    - Dependent libraries have been updated to the latest version.
 2025-04-15 11:09:49 +01:00
vorotamoroz
90bb610133 Update submodule 2025-04-14 03:23:24 +01:00
vorotamoroz
9c5e9fe63b Conclusion stated. 2025-04-11 14:13:22 +01:00
vorotamoroz
00dfae24d7 bump 2025-04-10 14:28:03 +01:00
vorotamoroz
d8a41fe45d ### New Feature 
- Now, we can send custom headers to the server.
- Authentication with JWT in CouchDB is now supported.

### Improved

- The QR Code for set-up can be shown also from the setting dialogue now.
- Conflict checking for preventing unexpected overwriting on the boot-up process has been quite faster.

### Fixed

- Some bugs on Dev and Testing modules have been fixed.
 2025-04-10 14:24:33 +01:00
vorotamoroz
30467d1c25 Add link to P2P pseudo client. 2025-04-04 12:30:21 +01:00
vorotamoroz
f8351f1d45 Merge branch 'main' of https://github.com/vrtmrz/obsidian-livesync 2025-04-04 11:52:12 +01:00
vorotamoroz
5924af98ab Update Library (Probably no effect) 2025-04-04 11:52:05 +01:00
vorotamoroz
2769b61da4 Update setup_own_server.md 
Add note; #609
 2025-04-04 18:24:13 +09:00
vorotamoroz
bb4409221d Update README.md 2025-04-03 20:57:15 +09:00
vorotamoroz
f398c14200 Bump for release-mistake. 2025-04-01 10:42:29 +01:00
vorotamoroz
27d58508dc Missed 2025-04-01 10:38:12 +01:00
vorotamoroz
d4dea5b226 bump 2025-04-01 10:21:38 +01:00
vorotamoroz
c79dc30cba ## 0.24.21 
### Fixed

- No longer conflicted files are handled in the boot-up process. No more unexpected overwriting.
    - It ignores `Always overwrite with a newer file`, and always be prevented for the safety. Please pick it manually or open the file.
- Some log messages on conflict resolution has been corrected.
- Automatic merge notifications, displayed on the grounds of `same`, have been degraded to logs.

### Improved

- Now we can fetch the remote database with keeping local files completely intact.
    - In new option, all files are stored into the local database before the fetching, and will be merged automatically or detected as conflicts.
- The dialogue presenting options when performing `Fetch` are now more informative.

### Refactored

- Some class methods have been fixed its arguments to be more consistent.
- Types have been defined for some conditional results.
 2025-04-01 10:20:21 +01:00
vorotamoroz
b3119ee8a9 bump and update dependencies 2025-03-24 18:55:02 +09:00
vorotamoroz
2a1d71da5c Improved: show details of TypeError using Obsidian API. 2025-03-24 12:04:58 +09:00
vorotamoroz
24f31ed19e Merge branch 'main' of https://github.com/vrtmrz/obsidian-livesync 2025-03-19 04:43:06 +01:00
vorotamoroz
a982629ae6 Update draft : possibly I should share cSpell dictionary. 2025-03-19 04:42:50 +01:00
vorotamoroz
85140aecab Update README.md 2025-03-05 20:40:04 +09:00
vorotamoroz
3f2e23ee88 bump 2025-03-05 11:13:58 +00:00
vorotamoroz
6049c19e8a ## 0.24.19 
### New Feature

- Now we can generate a QR Code for transferring the configuration to another device.
 2025-03-05 11:12:00 +00:00
vorotamoroz
65648683a3 bump 2025-02-28 12:02:45 +00:00
vorotamoroz
5d70f2c1e9 ## 0.24.18 
### Fixed

- Now no chunk creation errors will be raised after switching `Compute revisions for chunks`.
- Some invisible file can be handled correctly (e.g., `writing-goals-history.csv`).
- Fetching configuration from the server is now saves the configuration immediately (if we are not in the wizard).

### Improved

- Mismatched configuration dialogue is now more informative, and rewritten to more user-friendly.
- Applying configuration mismatch is now without rebuilding (at our own risks).
- Now, rebuilding is decided more fine grained.

### Improved internally

- Translations can be nested. i.e., task:`Some procedure`, check: `%{task} checking`, checkfailed: `%{check} failed` produces `Some procedure checking failed`.
 2025-02-28 11:58:15 +00:00
vorotamoroz
cbcfdc453e update default vault and bump for release 2025-02-27 13:39:28 +00:00
vorotamoroz
a4eb21593c bump 2025-02-27 13:24:51 +00:00
vorotamoroz
05eb2c8262 ## 0.24.16 
### Improved

#### Peer-to-Peer

- Now peer-to-peer synchronisation checks the settings are compatible with each other.
- Peer-to-peer synchronisation now handles the platform and detects pseudo-clients.

#### General

- New migration method has been implemented, that called `Doctor`.

- The minimum interval for replication to be caused when an event occurs can now be configurable.
- Some detail note has been added and change nuance about the `Report` in the setting dialogue, which had less informative.

### Behaviour and default changed

- `Compute revisions for chunks` are backed into enabled again. it is necessary for garbage collection of chunks.

### Refactored

- Platform specific codes are more separated. No longer `node` modules were used in the browser and Obsidian.
 2025-02-27 13:23:11 +00:00
vorotamoroz
fecefa3631 ### Fixed 
- Now, even without WeakRef, Polyfill is used and the whole thing works without error. However, if you can switch WebView Engine, it is recommended to switch to a WebView Engine that supports WeakRef.

And bumped.
 2025-02-20 10:40:18 +00:00
vorotamoroz
f8c4d5ccb0 Add a bit more. 2025-02-18 13:48:41 +00:00
vorotamoroz
e63e79bc8e Add note of flag files. 2025-02-18 13:36:15 +00:00
vorotamoroz
ed76125f3d bump 2025-02-18 13:02:54 +00:00
vorotamoroz
70f4e23474 ## 0.24.14 
### Fixed

- Resolving conflicts of JSON files (and sensibly merging them) is now working fine, again!
    - And, failure logs are more informative.
- More robust to release the event listeners on unwatching the local database.

### Refactored

- JSON file conflict resolution dialogue has been rewritten into svelte v5.
- Upgrade eslint.
- Remove unnecessary pragma comments for eslint.
 2025-02-18 12:59:18 +00:00
vorotamoroz
f6d5b78cc8 bump 2025-02-17 11:35:34 +00:00
vorotamoroz
405624b51b ## 0.24.13 
### Fixed
#### General Replication
- No longer unexpected errors occur when the replication is stopped during for some reason (e.g., network disconnection).
#### Peer-to-Peer Synchronisation
- Set-up process will not receive data from unexpected sources.
- No longer resource leaks while enabling the `broadcasting changes`
- Logs are less verbose.
- Received data is now correctly dispatched to other devices.
- `Timeout` error now more informative.
- No longer timeout error occurs for reporting the progress to other devices.
- Decision dialogues for the same thing are not shown multiply at the same time anymore.
- Disconnection of the peer-to-peer synchronisation is now more robust and less error-prone.
#### Webpeer
- Now we can toggle Peers' configuration.
### Refactored
- Cross-platform compatibility layer has been improved.
- Common events are moved to the common library.
- Displaying replication status of the peer-to-peer synchronisation is separated from the main-log-logic.
- Some file names have been changed to be more consistent.
 2025-02-17 11:33:35 +00:00
vorotamoroz
90c0ff22b9 Add paths for future maintenance. 2025-02-17 11:30:42 +00:00
57 changed files with 7195 additions and 5825 deletions
Expand all files Collapse all files

11
.eslintignore
View File

@@ -1,11 +0,0 @@
node_modules
build
.eslintrc.js.bak
src/lib/src/patches/pouchdb-utils
esbuild.config.mjs
rollup.config.js
src/lib/test
src/lib/src/cli
main.js
src/lib/apps/webpeer/dist
src/lib/apps/webpeer/svelte.config.js

29
.eslintrc
View File

@@ -1,13 +1,34 @@
{ {
"root": true, "root": true,
"parser": "@typescript-eslint/parser", "parser": "@typescript-eslint/parser",
"plugins": ["@typescript-eslint"], "plugins": [
"extends": ["eslint:recommended", "plugin:@typescript-eslint/eslint-recommended", "plugin:@typescript-eslint/recommended"], "@typescript-eslint",
"eslint-plugin-svelte",
"eslint-plugin-import"
],
"extends": [
"eslint:recommended",
"plugin:@typescript-eslint/eslint-recommended",
"plugin:@typescript-eslint/recommended"
],
"parserOptions": { "parserOptions": {
"sourceType": "module", "sourceType": "module",
"project": ["tsconfig.json"] "project": [
"tsconfig.json"
]
}, },
"ignorePatterns": [], "ignorePatterns": [
"**/node_modules/*",
"**/jest.config.js",
"src/lib/coverage",
"src/lib/browsertest",
"**/test.ts",
"**/tests.ts",
"**/**test.ts",
"**/**.test.ts",
"esbuild.*.mjs",
"terser.*.mjs"
],
"rules": { "rules": {
"no-unused-vars": "off", "no-unused-vars": "off",
"@typescript-eslint/no-unused-vars": [ "@typescript-eslint/no-unused-vars": [

7
README.md
View File

@@ -18,10 +18,11 @@ Note: This plugin cannot synchronise with the official "Obsidian Sync".
- Supporting End-to-end encryption. - Supporting End-to-end encryption.
- Synchronisation of settings, snippets, themes, and plug-ins, via [Customization sync(Beta)](#customization-sync) or [Hidden File Sync](#hiddenfilesync) - Synchronisation of settings, snippets, themes, and plug-ins, via [Customization sync(Beta)](#customization-sync) or [Hidden File Sync](#hiddenfilesync)
- WebClip from [obsidian-livesync-webclip](https://chrome.google.com/webstore/detail/obsidian-livesync-webclip/jfpaflmpckblieefkegjncjoceapakdf) - WebClip from [obsidian-livesync-webclip](https://chrome.google.com/webstore/detail/obsidian-livesync-webclip/jfpaflmpckblieefkegjncjoceapakdf)
- WebRTC peer-to-peer synchronisation without the need any `host` is now possible. (Experimental) - WebRTC peer-to-peer synchronisation without the need for any `host` is now possible. (Experimental)
- This feature is still in the experimental stage. Please be careful when using it. - This feature is still in the experimental stage. Please be careful when using it.
- Instead of using server, you can use [webpeer](https://github.com/vrtmrz/livesync-commonlib/tree/main/apps/webpeer) the pseudo client for receiving and sending between devices. - Instead of using public servers, you can use [webpeer](https://github.com/vrtmrz/livesync-commonlib/tree/main/apps/webpeer) the pseudo client for receiving and sending between devices.
- A pre-built instance is served at [fancy-syncing.vrtmrz.net/webpeer](https://fancy-syncing.vrtmrz.net/webpeer/) (in the vrtmrz blog site). This is of course also peer-to-peer. Feel free to use it.
- There is an [English explanatory article](https://fancy-syncing.vrtmrz.net/blog/0034-p2p-sync-en.html), and [Japanese explanatory article](https://fancy-syncing.vrtmrz.net/blog/0034-p2p-sync).
This plug-in might be useful for researchers, engineers, and developers with a need to keep their notes fully self-hosted for security reasons. Or just anyone who would like the peace of mind of knowing that their notes are fully private. This plug-in might be useful for researchers, engineers, and developers with a need to keep their notes fully self-hosted for security reasons. Or just anyone who would like the peace of mind of knowing that their notes are fully private.

43
docs/design_docs_of_keep_newborn_chunks.md
View File

@@ -1,6 +1,6 @@
# Keep newborn chunks in Eden. # Keep newborn chunks in Eden
NOTE: This is the planned feature design document. This is planned, but not be implemented now (v0.23.3). This has not reached the design freeze and will be added to from time to time. Notice: deprecated. please refer to the result section of this document.
## Goal ## Goal
@@ -19,15 +19,18 @@ Reduce the number of chunks which in volatile, and reduce the usage of storage o
- The problem is that this unnecessary chunking slows down both local and remote operations. - The problem is that this unnecessary chunking slows down both local and remote operations.
## Prerequisite ## Prerequisite
- The implementation must be able to control the size of the document appropriately so that it does not become non-transferable (1). - The implementation must be able to control the size of the document appropriately so that it does not become non-transferable (1).
- The implementation must be such that data corruption can be avoided even if forward compatibility is not maintained; due to the nature of Self-hosted LiveSync, backward version connexions are expected. - The implementation must be such that data corruption can be avoided even if forward compatibility is not maintained; due to the nature of Self-hosted LiveSync, backward version connexions are expected.
- Viewed as a feature: - Viewed as a feature:
- This feature should be disabled for migration users. - This feature should be disabled for migration users.
- This feature should be enabled for new users and after rebuilds of migrated users. - This feature should be enabled for new users and after rebuilds of migrated users.
- Therefore, back into the implementation view, Ideally, the implementation should be such that data recovery can be achieved by immediately upgrading after replication. - Therefore, back into the implementation view, Ideally, the implementation should be such that data recovery can be achieved by immediately upgrading after replication.
## Outlined methods and implementation plans ## Outlined methods and implementation plans
### Abstract ### Abstract
To store and transfer only stable chunks independently and share them from multiple documents after stabilisation, new chunks, i.e. chunks that are considered non-stable, are modified to be stored in the document and transferred with the document. In this case, care should be taken not to exceed prerequisite (1). To store and transfer only stable chunks independently and share them from multiple documents after stabilisation, new chunks, i.e. chunks that are considered non-stable, are modified to be stored in the document and transferred with the document. In this case, care should be taken not to exceed prerequisite (1).
If this is achieved, the non-leaf document will not be transferred, and even if it is, the chunk will be stored in the document, so that the size can be reduced by the compaction. If this is achieved, the non-leaf document will not be transferred, and even if it is, the chunk will be stored in the document, so that the size can be reduced by the compaction.
@@ -40,11 +43,11 @@ Details are given below.
type EntryWithEden = { type EntryWithEden = {
eden: { eden: {
[key: DocumentID]: { [key: DocumentID]: {
data: string, data: string;
epoch: number, // The document revision which this chunk has been born. epoch: number; // The document revision which this chunk has been born.
} };
} };
} };
``` ```
2. The following configuration items are added: 2. The following configuration items are added:
Note: These configurations should be shared as `Tweaks value` between each client. Note: These configurations should be shared as `Tweaks value` between each client.
@@ -63,6 +66,7 @@ Details are given below.
5. In End-to-End Encryption, property `eden` of documents will also be encrypted. 5. In End-to-End Encryption, property `eden` of documents will also be encrypted.
### Note ### Note
- When this feature has been enabled, forward compatibility is temporarily lost. However, it is detected as missing chunks, and this data is not reflected in the storage in the old version. Therefore, no data loss will occur. - When this feature has been enabled, forward compatibility is temporarily lost. However, it is detected as missing chunks, and this data is not reflected in the storage in the old version. Therefore, no data loss will occur.
## Test strategy ## Test strategy
@@ -77,5 +81,26 @@ Details are given below.
- Indeed, we lack a fulfilled configuration table. Efforts will be made and, if they can be produced, this document will then be referenced. But not required while in the experimental or beta feature. - Indeed, we lack a fulfilled configuration table. Efforts will be made and, if they can be produced, this document will then be referenced. But not required while in the experimental or beta feature.
- However, this might be an essential feature. Further efforts are desired. - However, this might be an essential feature. Further efforts are desired.
## Results from actual operation
After implementing this feature, we have been using it for a while. The following results were obtained.
- Drawbacks were thought not to be a problem, but they were actually a problem:
- A document with `Eden` has a quite larger history compared to a document without `Eden`.
- Self-hosted LiveSync does not perform compaction aggressively, which results in the remote database becoming partially bloated.
- Compaction of the Remote Database (CouchDB) requires the same amount of free space as the size of the database. Therefore, it is not possible to perform compaction on a remote database if we reached to the maximum size of the database. It means that when we detect it, it is too late.
- We have mentioned that `We need compaction` in previous sections. However, but it was so hard to be determined whether the compaction is required or not, until the database is bloated. (Of course, it requires some time to compact the database, and, literally, some document loses its history. It is not a good idea to perform frequently and meaninglessly. We need manual decision, but indeed difficult to normal users).
### Consideration and Conclusion ### Consideration and Conclusion
To be described after implemented, tested, and, released.
This feature results in two aspects:
- For the users who are familiar with the CouchDB, this feature is a bit useful. They can watch and handle the database by themselves.
- For the users who are not familiar with the CouchDB, i.e., normal users, this feature is not so useful, either. They are not familiar with the database, and they do not know how to handle it. Therefore, they cannot decide whether the compaction is required or not.
Hence, this feature would be kept as an experimental feature, but it is not enabled by default. In addition to that, it is marked as deprecated. Detailed notice will be noisy for the users who are not familiar with the CouchDB. Details would be kept in this document, for the future.
It is not recommended to use this feature, unless the person who is familiar with the CouchDB and the database management.
Vorotamoroz has written this document. Bias: I am the first author of this plug-in, familiar with the CouchDB.
Research and development has been frozen on 2025-04-11. But, bugs will be fixed if they are found. Please feel free to report them.

21
docs/setup_own_server.md
View File

@@ -31,7 +31,7 @@ export hostname=localhost:5984
export username=goojdasjdas #Please change as you like. export username=goojdasjdas #Please change as you like.
export password=kpkdasdosakpdsa #Please change as you like export password=kpkdasdosakpdsa #Please change as you like
# Prepare directories which saving data and configurations. # Prepare directories which save data and configurations.
mkdir couchdb-data mkdir couchdb-data
mkdir couchdb-etc mkdir couchdb-etc
``` ```
@@ -45,19 +45,19 @@ $ docker run --name couchdb-for-ols --rm -it -e COUCHDB_USER=${username} -e COUC
If your container has been exited, please check the permission of couchdb-data, and couchdb-etc. If your container has been exited, please check the permission of couchdb-data, and couchdb-etc.
Once CouchDB run, these directories will be owned by uid:`5984`. Please chown it for you again. Once CouchDB run, these directories will be owned by uid:`5984`. Please chown it for you again.
2. Enable it in background 2. Enable it in the background
``` ```
$ docker run --name couchdb-for-ols -d --restart always -e COUCHDB_USER=${username} -e COUCHDB_PASSWORD=${password} -v ${PWD}/couchdb-data:/opt/couchdb/data -v ${PWD}/couchdb-etc:/opt/couchdb/etc/local.d -p 5984:5984 couchdb $ docker run --name couchdb-for-ols -d --restart always -e COUCHDB_USER=${username} -e COUCHDB_PASSWORD=${password} -v ${PWD}/couchdb-data:/opt/couchdb/data -v ${PWD}/couchdb-etc:/opt/couchdb/etc/local.d -p 5984:5984 couchdb
``` ```
### B. Install CouchDB directly ### B. Install CouchDB directly
Please refer the [official document](https://docs.couchdb.org/en/stable/install/index.html). However, we do not have to configure it fully. Just administrator needs to be configured. Please refer to the [official document](https://docs.couchdb.org/en/stable/install/index.html). However, we do not have to configure it fully. Just the administrator needs to be configured.
## 2. Run couchdb-init.sh for initialise ## 2. Run couchdb-init.sh for initialise
``` ```
curl -s https://raw.githubusercontent.com/vrtmrz/obsidian-livesync/main/utils/couchdb/couchdb-init.sh | bash curl -s https://raw.githubusercontent.com/vrtmrz/obsidian-livesync/main/utils/couchdb/couchdb-init.sh | bash
``` ```
If it results like following: If it results like the following:
``` ```
-- Configuring CouchDB by REST APIs... --> -- Configuring CouchDB by REST APIs... -->
{"ok":true} {"ok":true}
@@ -80,7 +80,7 @@ Your CouchDB has been initialised successfully. If you want this manually, pleas
- You can skip this instruction if you using only in intranet and only with desktop devices. - You can skip this instruction if you using only in intranet and only with desktop devices.
- For mobile devices, Obsidian requires a valid SSL certificate. Usually, it needs exposing the internet. - For mobile devices, Obsidian requires a valid SSL certificate. Usually, it needs exposing the internet.
Whatever solutions we can use. For the simplicity, following sample uses Cloudflare Zero Trust for testing. Whatever solutions we can use. For simplicity, the following sample uses Cloudflare Zero Trust for testing.
``` ```
cloudflared tunnel --url http://localhost:5984 cloudflared tunnel --url http://localhost:5984
@@ -99,12 +99,12 @@ You will then get the following output:
: :
: :
``` ```
Now `https://tiles-photograph-routine-groundwater.trycloudflare.com` is our server. Make it into background once please. Now `https://tiles-photograph-routine-groundwater.trycloudflare.com` is our server. Make it into the background once, please.
## 4. Client Setup ## 4. Client Setup
> [!TIP] > [!TIP]
> Now manually configuration is not recommended for some reasons. However, if you want to do so, please use `Setup wizard`. The recommended extra configurations will be also set. > Now manual configuration is not recommended for some reasons. However, if you want to do so, please use `Setup wizard`. The recommended extra configurations will be also set.
### 1. Generate the setup URI on a desktop device or server ### 1. Generate the setup URI on a desktop device or server
```bash ```bash
@@ -116,6 +116,13 @@ export password=abc123
deno run -A https://raw.githubusercontent.com/vrtmrz/obsidian-livesync/main/utils/flyio/generate_setupuri.ts deno run -A https://raw.githubusercontent.com/vrtmrz/obsidian-livesync/main/utils/flyio/generate_setupuri.ts
``` ```
> [!TIP]
> What is the `passphrase`? Is it different from `uri_passphrase`?
> Yes, the `passphrase` we have exported now is for an End-to-End Encryption passphrase.
> And, `uri_passphrase` that used in the `generate_setupuri.ts` is a different one; for decrypting Set-up URI at using that.
> Why: I (vorotamoroz) think that the passphrase of the Setup-URI should be different from the E2EE passphrase to prevent exposure caused by operational errors or the possibility of evil in our environment. On top of that, I believe that it is desirable for the Setup-URI to be random. Setup-URI is inevitably long, so it goes through the clipboard. I think that its passphrase should not go through the same path, so it should essentially be typed manually.
> Hence, if we keep empty for uri_passphrase, generate_setupuri.ts generates an adjective-noun-randomnumber passphrase so that we can remember it without going through the clipboard.
You will then get the following output: You will then get the following output:
```bash ```bash

26
docs/terms.md
View File

@@ -1,10 +1,24 @@
# Terms used in this project # Notes on Terminology, Spelling, Vocabulary Conventions
## Terms ## Spelling and Vocabulary conventions
### Chunks 1. Almost all of the english words are written in British English. For example, "organisation" instead of "organization", "synchronisation" instead of "synchronization", etc. This convention originated from the author's personal preference but is now maintained for consistency.
<!-- TBW, sorry for the draft! -->
2. Idiomatic terms, such as used in HTML, CSS, and JavaScript, are usually be aligned with the language used in the technology. For example, "color" instead of "colour", "program" instead of "programme", etc. Especially, terms which are used for attributes, properties, and methods are notable.
<!-- Please feel free to write any terms that should be mentioned. And please make pull request. I would love to fill the rest. --> 3. We use `dialogue` in documentation for consistency. While `dialog` may appear in source code, particularly in class names, method names, and attributes (following technical conventions in No. 2), we consistently use `dialogue` for user-facing messages and general documentation text. This approach balances No. 1 with No. 2.
<!-- ### Chunks -->
4. Contractions are not used. For example, "do not" instead of "don't", "cannot" instead of "can't", etc. especially `'d`.
- We may encounter difficulties with tenses.
5. However, try using affirmative forms, `Discard` instead of `Do not keep`, `Continue` instead of `Do not stop`, etc.
- Some languages, such as Japanese, have a different meaning for `yes` and `no` between affirmative and negative questions.
## Terminology
- Self-hosted LiveSync
- This plug-in name. `Self-hosted` is one word.
- LiveSync
- Very confusing term.
- As shorten-form of `Self-hosted LiveSync`.
- As a name of synchronisation mode. This should be changed to `Continuos`, in contrast to `Periodic`.

59
docs/troubleshooting.md
View File

@@ -1,6 +1,6 @@
<!-- 2024-02-15 --> <!-- 2025-02-18 -->
# Tips and Troubleshooting
# Tips and Troubleshooting
- [Tips and Troubleshooting](#tips-and-troubleshooting) - [Tips and Troubleshooting](#tips-and-troubleshooting)
- [Notable bugs and fixes](#notable-bugs-and-fixes) - [Notable bugs and fixes](#notable-bugs-and-fixes)
@@ -25,14 +25,16 @@
<!-- - --> <!-- - -->
## Notable bugs and fixes ## Notable bugs and fixes
### Binary files get bigger on iOS ### Binary files get bigger on iOS
- Reported at: v0.20.x - Reported at: v0.20.x
- Fixed at: v0.21.2 (Fixed but not reviewed) - Fixed at: v0.21.2 (Fixed but not reviewed)
- Required action: larger files will not be fixed automatically, please perform `Verify and repair all files`. If our local database and storage are not matched, we will be asked to apply which one. - Required action: larger files will not be fixed automatically, please perform `Verify and repair all files`. If our local database and storage are not matched, we will be asked to apply which one.
### Some setting name has been changed ### Some setting name has been changed
- Fixed at: v0.22.6 - Fixed at: v0.22.6
| Previous name | New name | | Previous name | New name |
@@ -58,13 +60,16 @@ Therefore, experienced users (especially those stable enough not to have to rebu
Please disable it when you have enough time. Please disable it when you have enough time.
### ZIP (or any extensions) files were not synchronised. Why? ### ZIP (or any extensions) files were not synchronised. Why?
It depends on Obsidian detects. May toggling `Detect all extensions` of `File and links` (setting of Obsidian) will help us. It depends on Obsidian detects. May toggling `Detect all extensions` of `File and links` (setting of Obsidian) will help us.
### I hope to report the issue, but you said you needs `Report`. How to make it? ### I hope to report the issue, but you said you needs `Report`. How to make it?
We can copy the report to the clipboard, by pressing the `Make report` button on the `Hatch` pane. We can copy the report to the clipboard, by pressing the `Make report` button on the `Hatch` pane.
![Screenshot](../images/hatch.png) ![Screenshot](../images/hatch.png)
### Where can I check the log? ### Where can I check the log?
We can launch the log pane by `Show log` on the command palette. We can launch the log pane by `Show log` on the command palette.
And if you have troubled something, please enable the `Verbose Log` on the `General Setting` pane. And if you have troubled something, please enable the `Verbose Log` on the `General Setting` pane.
@@ -73,16 +78,20 @@ However, the logs would not be kept so long and cleared when restarted. If you w
![ScreenShot](../images/write_logs_into_the_file.png) ![ScreenShot](../images/write_logs_into_the_file.png)
> [!IMPORTANT] > [!IMPORTANT]
>
> - Writing logs into the file will impact the performance. > - Writing logs into the file will impact the performance.
> - Please make sure that you have erased all your confidential information before reporting issue. > - Please make sure that you have erased all your confidential information before reporting issue.
### Why are the logs volatile and ephemeral? ### Why are the logs volatile and ephemeral?
To avoid unexpected exposure to our confidential things. To avoid unexpected exposure to our confidential things.
### Some network logs are not written into the file. ### Some network logs are not written into the file.
Especially the CORS error will be reported as a general error to the plug-in for security reasons. So we cannot detect and log it. We are only able to investigate them by [Checking the network log](#checking-the-network-log). Especially the CORS error will be reported as a general error to the plug-in for security reasons. So we cannot detect and log it. We are only able to investigate them by [Checking the network log](#checking-the-network-log).
### If a file were deleted or trimmed, the capacity of the database should be reduced, right? ### If a file were deleted or trimmed, the capacity of the database should be reduced, right?
No, even though if files were deleted, chunks were not deleted. No, even though if files were deleted, chunks were not deleted.
Self-hosted LiveSync splits the files into multiple chunks and transfers only newly created. This behaviour enables us to less traffic. And, the chunks will be shared between the files to reduce the total usage of the database. Self-hosted LiveSync splits the files into multiple chunks and transfers only newly created. This behaviour enables us to less traffic. And, the chunks will be shared between the files to reduce the total usage of the database.
@@ -93,24 +102,42 @@ To shrink the database size, `Rebuild everything` only reliably and effectively.
### How can I use the DevTools? ### How can I use the DevTools?
#### Checking the network log #### Checking the network log
1. Open the network pane. 1. Open the network pane.
2. Find the requests marked in red. 2. Find the requests marked in red.
![Errored](../images/devtools1.png) ![Errored](../images/devtools1.png)
3. Capture the `Headers`, `Payload`, and, `Response`. **Please be sure to keep important information confidential**. If the `Response` contains secrets, you can omitted that. 3. Capture the `Headers`, `Payload`, and, `Response`. **Please be sure to keep important information confidential**. If the `Response` contains secrets, you can omitted that.
Note: Headers contains a some credentials. **The path of the request URL, Remote Address, authority, and authorization must be concealed.** Note: Headers contains a some credentials. **The path of the request URL, Remote Address, authority, and authorization must be concealed.**
![Concealed sample](../images/devtools2.png) ![Concealed sample](../images/devtools2.png)
## Troubleshooting ## Troubleshooting
<!-- Add here --> <!-- Add here -->
### On the mobile device, cannot synchronise on the local network! ### On the mobile device, cannot synchronise on the local network!
Obsidian mobile is not able to connect to the non-secure end-point, such as starting with `http://`. Make sure your URI of CouchDB. Also not able to use a self-signed certificate. Obsidian mobile is not able to connect to the non-secure end-point, such as starting with `http://`. Make sure your URI of CouchDB. Also not able to use a self-signed certificate.
### I think that something bad happening on the vault... ### I think that something bad happening on the vault...
Place `redflag.md` on top of the vault, and restart Obsidian. The most simple way is to create a new note and rename it to `redflag`. Of course, we can put it without Obsidian. Place `redflag.md` on top of the vault, and restart Obsidian. The most simple way is to create a new note and rename it to `redflag`. Of course, we can put it without Obsidian.
If there is `redflag.md`, Self-hosted LiveSync suspends all database and storage processes. If there is `redflag.md`, Self-hosted LiveSync suspends all database and storage processes.
There are some options to use `redflag.md`.
| Filename | Human-Friendly Name | Description |
| ------------- | ------------------- | ------------------------------------------------------------------------------------ |
| `redflag.md` | - | Suspends all processes. |
| `redflag2.md` | `flag_rebuild.md` | Suspends all processes, and rebuild both local and remote databases by local files. |
| `redflag3.md` | `flag_fetch.md` | Suspends all processes, discard the local database, and fetch from the remote again. |
When fetching everything remotely or performing a rebuild, restarting Obsidian is performed once for safety reasons. At that time, Self-hosted LiveSync uses these files to determine whether the process should be carried out.
(The use of normal markdown files is a trick to externally force cancellation in the event of faults in the rebuild or fetch function itself, especially on mobile devices).
This mechanism is also used for set-up. And just for information, these files are also not subject to synchronisation.
However, occasionally the deletion of files may fail. This should generally work normally after restarting Obsidian. (As far as I can observe).
## Tips ## Tips
### How to resolve `Tweaks Mismatched of Changed` ### How to resolve `Tweaks Mismatched of Changed`
@@ -133,16 +160,16 @@ If we see it for the first time, it reflects the settings of the device that has
<!-- Add here --> <!-- Add here -->
### Old tips ### Old tips
- Rarely, a file in the database could be corrupted. The plugin will not write to local storage when a file looks corrupted. If a local version of the file is on your device, the corruption could be fixed by editing the local file and synchronizing it. But if the file does not exist on any of your devices, then it can not be rescued. In this case, you can delete these items from the settings dialog.
- To stop the boot-up sequence (eg. for fixing problems on databases), you can put a `redflag.md` file (or directory) at the root of your vault. - Rarely, a file in the database could be corrupted. The plugin will not write to local storage when a file looks corrupted. If a local version of the file is on your device, the corruption could be fixed by editing the local file and synchronizing it. But if the file does not exist on any of your devices, then it can not be rescued. In this case, you can delete these items from the settings dialog.
Tip for iOS: a redflag directory can be created at the root of the vault using the File application. - To stop the boot-up sequence (eg. for fixing problems on databases), you can put a `redflag.md` file (or directory) at the root of your vault.
- Also, with `redflag2.md` placed, we can automatically rebuild both the local and the remote databases during the boot-up sequence. With `redflag3.md`, we can discard only the local database and fetch from the remote again. Tip for iOS: a redflag directory can be created at the root of the vault using the File application.
- Q: The database is growing, how can I shrink it down? - Also, with `redflag2.md` placed, we can automatically rebuild both the local and the remote databases during the boot-up sequence. With `redflag3.md`, we can discard only the local database and fetch from the remote again.
A: each of the docs is saved with their past 100 revisions for detecting and resolving conflicts. Picturing that one device has been offline for a while, and comes online again. The device has to compare its notes with the remotely saved ones. If there exists a historic revision in which the note used to be identical, it could be updated safely (like git fast-forward). Even if that is not in revision histories, we only have to check the differences after the revision that both devices commonly have. This is like git's conflict-resolving method. So, We have to make the database again like an enlarged git repo if you want to solve the root of the problem. - Q: The database is growing, how can I shrink it down?
- And more technical Information is in the [Technical Information](tech_info.md) A: each of the docs is saved with their past 100 revisions for detecting and resolving conflicts. Picturing that one device has been offline for a while, and comes online again. The device has to compare its notes with the remotely saved ones. If there exists a historic revision in which the note used to be identical, it could be updated safely (like git fast-forward). Even if that is not in revision histories, we only have to check the differences after the revision that both devices commonly have. This is like git's conflict-resolving method. So, We have to make the database again like an enlarged git repo if you want to solve the root of the problem.
- If you want to synchronize files without obsidian, you can use [filesystem-livesync](https://github.com/vrtmrz/filesystem-livesync). - And more technical Information is in the [Technical Information](tech_info.md)
- WebClipper is also available on Chrome Web Store:[obsidian-livesync-webclip](https://chrome.google.com/webstore/detail/obsidian-livesync-webclip/jfpaflmpckblieefkegjncjoceapakdf) - If you want to synchronize files without obsidian, you can use [filesystem-livesync](https://github.com/vrtmrz/filesystem-livesync).
- WebClipper is also available on Chrome Web Store:[obsidian-livesync-webclip](https://chrome.google.com/webstore/detail/obsidian-livesync-webclip/jfpaflmpckblieefkegjncjoceapakdf)
Repo is here: [obsidian-livesync-webclip](https://github.com/vrtmrz/obsidian-livesync-webclip). (Docs are a work in progress.) Repo is here: [obsidian-livesync-webclip](https://github.com/vrtmrz/obsidian-livesync-webclip). (Docs are a work in progress.)

2
esbuild.config.mjs
View File

@@ -4,7 +4,7 @@ import esbuild from "esbuild";
import process from "process"; import process from "process";
import builtins from "builtin-modules"; import builtins from "builtin-modules";
import sveltePlugin from "esbuild-svelte"; import sveltePlugin from "esbuild-svelte";
import sveltePreprocess from "svelte-preprocess"; import { sveltePreprocess } from "svelte-preprocess";
import fs from "node:fs"; import fs from "node:fs";
// import terser from "terser"; // import terser from "terser";
import { minify } from "terser"; import { minify } from "terser";

100
eslint.config.mjs Normal file
View File

@@ -0,0 +1,100 @@
import typescriptEslint from "@typescript-eslint/eslint-plugin";
import svelte from "eslint-plugin-svelte";
import _import from "eslint-plugin-import";
import { fixupPluginRules } from "@eslint/compat";
import tsParser from "@typescript-eslint/parser";
import path from "node:path";
import { fileURLToPath } from "node:url";
import js from "@eslint/js";
import { FlatCompat } from "@eslint/eslintrc";
const __filename = fileURLToPath(import.meta.url);
const __dirname = path.dirname(__filename);
const compat = new FlatCompat({
baseDirectory: __dirname,
recommendedConfig: js.configs.recommended,
allConfig: js.configs.all,
});
export default [
{
ignores: [
"**/node_modules/*",
"**/jest.config.js",
"src/lib/coverage",
"src/lib/browsertest",
"**/test.ts",
"**/tests.ts",
"**/**test.ts",
"**/**.test.ts",
"**/esbuild.*.mjs",
"**/terser.*.mjs",
"**/node_modules",
"**/build",
"**/.eslintrc.js.bak",
"src/lib/src/patches/pouchdb-utils",
"**/esbuild.config.mjs",
"**/rollup.config.js",
"modules/octagonal-wheels/rollup.config.js",
"modules/octagonal-wheels/dist/**/*",
"src/lib/test",
"src/lib/src/cli",
"**/main.js",
"src/lib/apps/webpeer/dist",
"src/lib/apps/webpeer/svelte.config.js",
],
},
...compat.extends(
"eslint:recommended",
"plugin:@typescript-eslint/eslint-recommended",
"plugin:@typescript-eslint/recommended"
),
{
plugins: {
"@typescript-eslint": typescriptEslint,
svelte,
import: fixupPluginRules(_import),
},
languageOptions: {
parser: tsParser,
ecmaVersion: 5,
sourceType: "module",
parserOptions: {
project: ["tsconfig.json"],
},
},
rules: {
"no-unused-vars": "off",
"@typescript-eslint/no-unused-vars": [
"error",
{
args: "none",
},
],
"no-unused-labels": "off",
"@typescript-eslint/ban-ts-comment": "off",
"no-prototype-builtins": "off",
"@typescript-eslint/no-empty-function": "off",
"require-await": "error",
"@typescript-eslint/require-await": "warn",
"@typescript-eslint/no-misused-promises": "warn",
"@typescript-eslint/no-floating-promises": "warn",
"no-async-promise-executor": "warn",
"@typescript-eslint/no-explicit-any": "off",
"@typescript-eslint/no-unnecessary-type-assertion": "error",
"no-constant-condition": [
"error",
{
checkLoops: false,
},
],
},
},
];

2
manifest.json
View File

@@ -1,7 +1,7 @@
{ {
"id": "obsidian-livesync", "id": "obsidian-livesync",
"name": "Self-hosted LiveSync", "name": "Self-hosted LiveSync",
"version": "0.24.12", "version": "0.24.24",
"minAppVersion": "0.9.12", "minAppVersion": "0.9.12",
"description": "Community implementation of self-hosted livesync. Reflect your vault changes to some other devices immediately. Please make sure to disable other synchronize solutions to avoid content corruption or duplication.", "description": "Community implementation of self-hosted livesync. Reflect your vault changes to some other devices immediately. Please make sure to disable other synchronize solutions to avoid content corruption or duplication.",
"author": "vorotamoroz", "author": "vorotamoroz",

9395
package-lock.json generated
View File

File diff suppressed because it is too large Load Diff

34
package.json
View File

@@ -1,6 +1,6 @@
{ {
"name": "obsidian-livesync", "name": "obsidian-livesync",
"version": "0.24.12", "version": "0.24.24",
"description": "Reflect your vault changes to some other devices immediately. Please make sure to disable other synchronize solutions to avoid content corruption or duplication.", "description": "Reflect your vault changes to some other devices immediately. Please make sure to disable other synchronize solutions to avoid content corruption or duplication.",
"main": "main.js", "main": "main.js",
"type": "module", "type": "module",
@@ -22,6 +22,9 @@
"license": "MIT", "license": "MIT",
"devDependencies": { "devDependencies": {
"@chialab/esbuild-plugin-worker": "^0.18.1", "@chialab/esbuild-plugin-worker": "^0.18.1",
"@eslint/compat": "^1.2.6",
"@eslint/eslintrc": "^3.2.0",
"@eslint/js": "^9.20.0",
"@tsconfig/svelte": "^5.0.4", "@tsconfig/svelte": "^5.0.4",
"@types/diff-match-patch": "^1.0.36", "@types/diff-match-patch": "^1.0.36",
"@types/node": "^22.5.4", "@types/node": "^22.5.4",
@@ -33,17 +36,17 @@
"@types/pouchdb-mapreduce": "^6.1.10", "@types/pouchdb-mapreduce": "^6.1.10",
"@types/pouchdb-replication": "^6.4.7", "@types/pouchdb-replication": "^6.4.7",
"@types/transform-pouch": "^1.0.6", "@types/transform-pouch": "^1.0.6",
"@typescript-eslint/eslint-plugin": "^8.23.0", "@typescript-eslint/eslint-plugin": "^8.24.1",
"@typescript-eslint/parser": "^8.23.0", "@typescript-eslint/parser": "^8.24.1",
"builtin-modules": "^4.0.0", "builtin-modules": "^4.0.0",
"esbuild": "0.24.2", "esbuild": "0.24.2",
"esbuild-svelte": "^0.9.0", "esbuild-svelte": "^0.9.0",
"eslint": "^8.57.0", "eslint": "^9.20.1",
"eslint-config-airbnb-base": "^15.0.0",
"eslint-plugin-import": "^2.31.0", "eslint-plugin-import": "^2.31.0",
"eslint-plugin-svelte": "^2.46.1",
"events": "^3.3.0", "events": "^3.3.0",
"obsidian": "^1.7.2", "obsidian": "^1.7.2",
"postcss": "^8.5.1", "postcss": "^8.5.2",
"postcss-load-config": "^6.0.1", "postcss-load-config": "^6.0.1",
"pouchdb-adapter-http": "^9.0.0", "pouchdb-adapter-http": "^9.0.0",
"pouchdb-adapter-idb": "^9.0.0", "pouchdb-adapter-idb": "^9.0.0",
@@ -55,28 +58,29 @@
"pouchdb-merge": "^9.0.0", "pouchdb-merge": "^9.0.0",
"pouchdb-replication": "^9.0.0", "pouchdb-replication": "^9.0.0",
"pouchdb-utils": "^9.0.0", "pouchdb-utils": "^9.0.0",
"prettier": "^3.4.2", "prettier": "^3.5.1",
"svelte": "^5.19.7", "svelte": "^5.20.1",
"svelte-preprocess": "^6.0.3", "svelte-preprocess": "^6.0.3",
"terser": "^5.37.0", "terser": "^5.39.0",
"transform-pouch": "^2.0.0", "transform-pouch": "^2.0.0",
"tslib": "^2.8.1", "tslib": "^2.8.1",
"tsx": "^4.19.2", "tsx": "^4.19.2",
"typescript": "^5.7.3" "typescript": "^5.7.3"
}, },
"dependencies": { "dependencies": {
"@aws-sdk/client-s3": "^3.645.0", "@aws-sdk/client-s3": "^3.787.0",
"@smithy/fetch-http-handler": "^3.2.4", "@smithy/fetch-http-handler": "^5.0.2",
"@smithy/protocol-http": "^4.1.0", "@smithy/protocol-http": "^5.1.0",
"@smithy/querystring-builder": "^3.0.3", "@smithy/querystring-builder": "^4.0.2",
"diff-match-patch": "^1.0.5", "diff-match-patch": "^1.0.5",
"esbuild-plugin-inline-worker": "^0.1.1", "esbuild-plugin-inline-worker": "^0.1.1",
"fflate": "^0.8.2", "fflate": "^0.8.2",
"idb": "^8.0.2", "idb": "^8.0.2",
"minimatch": "^10.0.1", "minimatch": "^10.0.1",
"octagonal-wheels": "^0.1.23", "octagonal-wheels": "^0.1.25",
"qrcode-generator": "^1.4.4",
"svelte-check": "^4.1.4", "svelte-check": "^4.1.4",
"trystero": "^0.20.0", "trystero": "^0.21.1",
"xxhash-wasm-102": "npm:xxhash-wasm@^1.0.2" "xxhash-wasm-102": "npm:xxhash-wasm@^1.0.2"
} }
} }

36
src/common/events.ts
View File

@@ -1,23 +1,16 @@
import type { FilePathWithPrefix, ObsidianLiveSyncSettings } from "../lib/src/common/types";
import { eventHub } from "../lib/src/hub/hub"; import { eventHub } from "../lib/src/hub/hub";
import type ObsidianLiveSyncPlugin from "../main"; import type ObsidianLiveSyncPlugin from "../main";
export const EVENT_LAYOUT_READY = "layout-ready";
export const EVENT_PLUGIN_LOADED = "plugin-loaded"; export const EVENT_PLUGIN_LOADED = "plugin-loaded";
export const EVENT_PLUGIN_UNLOADED = "plugin-unloaded"; export const EVENT_PLUGIN_UNLOADED = "plugin-unloaded";
export const EVENT_SETTING_SAVED = "setting-saved";
export const EVENT_FILE_RENAMED = "file-renamed";
export const EVENT_FILE_SAVED = "file-saved"; export const EVENT_FILE_SAVED = "file-saved";
export const EVENT_LEAF_ACTIVE_CHANGED = "leaf-active-changed"; export const EVENT_LEAF_ACTIVE_CHANGED = "leaf-active-changed";
export const EVENT_DATABASE_REBUILT = "database-rebuilt";
export const EVENT_LOG_ADDED = "log-added";
export const EVENT_REQUEST_OPEN_SETTINGS = "request-open-settings"; export const EVENT_REQUEST_OPEN_SETTINGS = "request-open-settings";
export const EVENT_REQUEST_OPEN_SETTING_WIZARD = "request-open-setting-wizard"; export const EVENT_REQUEST_OPEN_SETTING_WIZARD = "request-open-setting-wizard";
export const EVENT_REQUEST_OPEN_SETUP_URI = "request-open-setup-uri"; export const EVENT_REQUEST_OPEN_SETUP_URI = "request-open-setup-uri";
export const EVENT_REQUEST_COPY_SETUP_URI = "request-copy-setup-uri"; export const EVENT_REQUEST_COPY_SETUP_URI = "request-copy-setup-uri";
export const EVENT_REQUEST_SHOW_SETUP_QR = "request-show-setup-qr";
export const EVENT_REQUEST_RELOAD_SETTING_TAB = "reload-setting-tab"; export const EVENT_REQUEST_RELOAD_SETTING_TAB = "reload-setting-tab";
@@ -26,32 +19,27 @@ export const EVENT_REQUEST_OPEN_PLUGIN_SYNC_DIALOG = "request-open-plugin-sync-d
export const EVENT_REQUEST_OPEN_P2P = "request-open-p2p"; export const EVENT_REQUEST_OPEN_P2P = "request-open-p2p";
export const EVENT_REQUEST_CLOSE_P2P = "request-close-p2p"; export const EVENT_REQUEST_CLOSE_P2P = "request-close-p2p";
export const EVENT_REQUEST_RUN_DOCTOR = "request-run-doctor";
// export const EVENT_FILE_CHANGED = "file-changed"; // export const EVENT_FILE_CHANGED = "file-changed";
declare global { declare global {
interface LSEvents { interface LSEvents {
[EVENT_REQUEST_OPEN_PLUGIN_SYNC_DIALOG]: undefined;
[EVENT_FILE_SAVED]: undefined;
[EVENT_REQUEST_OPEN_SETUP_URI]: undefined;
[EVENT_REQUEST_COPY_SETUP_URI]: undefined;
[EVENT_REQUEST_RELOAD_SETTING_TAB]: undefined;
[EVENT_PLUGIN_UNLOADED]: undefined;
[EVENT_SETTING_SAVED]: ObsidianLiveSyncSettings;
[EVENT_PLUGIN_LOADED]: ObsidianLiveSyncPlugin; [EVENT_PLUGIN_LOADED]: ObsidianLiveSyncPlugin;
[EVENT_LAYOUT_READY]: undefined; [EVENT_PLUGIN_UNLOADED]: undefined;
"event-file-changed": { file: FilePathWithPrefix; automated: boolean }; [EVENT_REQUEST_OPEN_PLUGIN_SYNC_DIALOG]: undefined;
"document-stub-created": {
toc: Set<string>;
stub: { [key: string]: { [key: string]: Map<string, Record<string, string>> } };
};
[EVENT_REQUEST_OPEN_SETTINGS]: undefined; [EVENT_REQUEST_OPEN_SETTINGS]: undefined;
[EVENT_REQUEST_OPEN_SETTING_WIZARD]: undefined; [EVENT_REQUEST_OPEN_SETTING_WIZARD]: undefined;
[EVENT_FILE_RENAMED]: { newPath: FilePathWithPrefix; old: FilePathWithPrefix }; [EVENT_REQUEST_RELOAD_SETTING_TAB]: undefined;
[EVENT_LEAF_ACTIVE_CHANGED]: undefined; [EVENT_LEAF_ACTIVE_CHANGED]: undefined;
[EVENT_REQUEST_OPEN_P2P]: undefined;
[EVENT_REQUEST_CLOSE_P2P]: undefined; [EVENT_REQUEST_CLOSE_P2P]: undefined;
[EVENT_DATABASE_REBUILT]: undefined; [EVENT_REQUEST_OPEN_P2P]: undefined;
[EVENT_REQUEST_OPEN_SETUP_URI]: undefined;
[EVENT_REQUEST_COPY_SETUP_URI]: undefined;
[EVENT_REQUEST_SHOW_SETUP_QR]: undefined;
[EVENT_REQUEST_RUN_DOCTOR]: string;
} }
} }
export * from "../lib/src/events/coreEvents.ts";
export { eventHub }; export { eventHub };

1
src/common/types.ts
View File

@@ -88,3 +88,4 @@ export const ICXHeader = "ix:";
export const FileWatchEventQueueMax = 10; export const FileWatchEventQueueMax = 10;
export const configURIBase = "obsidian://setuplivesync?settings="; export const configURIBase = "obsidian://setuplivesync?settings=";
export const configURIBaseQR = "obsidian://setuplivesync?settingsQR=";

206
src/common/utils.ts
View File

@@ -15,6 +15,7 @@ import {
LOG_LEVEL_NOTICE, LOG_LEVEL_NOTICE,
LOG_LEVEL_VERBOSE, LOG_LEVEL_VERBOSE,
type AnyEntry, type AnyEntry,
type CouchDBCredentials,
type DocumentID, type DocumentID,
type EntryHasPath, type EntryHasPath,
type FilePath, type FilePath,
@@ -30,6 +31,8 @@ import { sameChangePairs } from "./stores.ts";
import type { KeyValueDatabase } from "./KeyValueDB.ts"; import type { KeyValueDatabase } from "./KeyValueDB.ts";
import { scheduleTask } from "octagonal-wheels/concurrency/task"; import { scheduleTask } from "octagonal-wheels/concurrency/task";
import { EVENT_PLUGIN_UNLOADED, eventHub } from "./events.ts"; import { EVENT_PLUGIN_UNLOADED, eventHub } from "./events.ts";
import { promiseWithResolver, type PromiseWithResolvers } from "octagonal-wheels/promises";
import { AuthorizationHeaderGenerator } from "../lib/src/replication/httplib.ts";
export { scheduleTask, cancelTask, cancelAllTasks } from "../lib/src/concurrency/task.ts"; export { scheduleTask, cancelTask, cancelAllTasks } from "../lib/src/concurrency/task.ts";
@@ -229,17 +232,17 @@ export const _requestToCouchDBFetch = async (
export const _requestToCouchDB = async ( export const _requestToCouchDB = async (
baseUri: string, baseUri: string,
username: string, credentials: CouchDBCredentials,
password: string,
origin: string, origin: string,
path?: string, path?: string,
body?: any, body?: any,
method?: string method?: string,
customHeaders?: Record<string, string>
) => { ) => {
const utf8str = String.fromCharCode.apply(null, [...writeString(`${username}:${password}`)]); // Create each time to avoid caching.
const encoded = window.btoa(utf8str); const authHeaderGen = new AuthorizationHeaderGenerator();
const authHeader = "Basic " + encoded; const authHeader = await authHeaderGen.getAuthorizationHeader(credentials);
const transformedHeaders: Record<string, string> = { authorization: authHeader, origin: origin }; const transformedHeaders: Record<string, string> = { authorization: authHeader, origin: origin, ...customHeaders };
const uri = `${baseUri}/${path}`; const uri = `${baseUri}/${path}`;
const requestParam: RequestUrlParam = { const requestParam: RequestUrlParam = {
url: uri, url: uri,
@@ -250,6 +253,9 @@ export const _requestToCouchDB = async (
}; };
return await requestUrl(requestParam); return await requestUrl(requestParam);
}; };
/**
* @deprecated Use requestToCouchDBWithCredentials instead.
*/
export const requestToCouchDB = async ( export const requestToCouchDB = async (
baseUri: string, baseUri: string,
username: string, username: string,
@@ -257,12 +263,34 @@ export const requestToCouchDB = async (
origin: string = "", origin: string = "",
key?: string, key?: string,
body?: string, body?: string,
method?: string method?: string,
customHeaders?: Record<string, string>
) => { ) => {
const uri = `_node/_local/_config${key ? "/" + key : ""}`; const uri = `_node/_local/_config${key ? "/" + key : ""}`;
return await _requestToCouchDB(baseUri, username, password, origin, uri, body, method); return await _requestToCouchDB(
baseUri,
{ username, password, type: "basic" },
origin,
uri,
body,
method,
customHeaders
);
}; };
export function requestToCouchDBWithCredentials(
baseUri: string,
credentials: CouchDBCredentials,
origin: string = "",
key?: string,
body?: string,
method?: string,
customHeaders?: Record<string, string>
) {
const uri = `_node/_local/_config${key ? "/" + key : ""}`;
return _requestToCouchDB(baseUri, credentials, origin, uri, body, method, customHeaders);
}
export const BASE_IS_NEW = Symbol("base"); export const BASE_IS_NEW = Symbol("base");
export const TARGET_IS_NEW = Symbol("target"); export const TARGET_IS_NEW = Symbol("target");
export const EVEN = Symbol("even"); export const EVEN = Symbol("even");
@@ -493,3 +521,163 @@ export function onlyInNTimes(n: number, proc: (progress: number) => any) {
} }
}; };
} }
const waitingTasks = {} as Record<string, { task?: PromiseWithResolvers<any>; previous: number; leastNext: number }>;
export function rateLimitedSharedExecution<T>(key: string, interval: number, proc: () => Promise<T>): Promise<T> {
if (!(key in waitingTasks)) {
waitingTasks[key] = { task: undefined, previous: 0, leastNext: 0 };
}
if (waitingTasks[key].task) {
// Extend the previous execution time.
waitingTasks[key].leastNext = Date.now() + interval;
return waitingTasks[key].task.promise;
}
const previous = waitingTasks[key].previous;
const delay = previous == 0 ? 0 : Math.max(interval - (Date.now() - previous), 0);
const task = promiseWithResolver<T>();
void task.promise.finally(() => {
if (waitingTasks[key].task === task) {
waitingTasks[key].task = undefined;
waitingTasks[key].previous = Math.max(Date.now(), waitingTasks[key].leastNext);
}
});
waitingTasks[key] = {
task,
previous: Date.now(),
leastNext: Date.now() + interval,
};
void scheduleTask("thin-out-" + key, delay, async () => {
try {
task.resolve(await proc());
} catch (ex) {
task.reject(ex);
}
});
return task.promise;
}
export function updatePreviousExecutionTime(key: string, timeDelta: number = 0) {
if (!(key in waitingTasks)) {
waitingTasks[key] = { task: undefined, previous: 0, leastNext: 0 };
}
waitingTasks[key].leastNext = Math.max(Date.now() + timeDelta, waitingTasks[key].leastNext);
}
const prefixMapObject = {
s: {
1: "V",
2: "W",
3: "X",
4: "Y",
5: "Z",
},
o: {
1: "v",
2: "w",
3: "x",
4: "y",
5: "z",
},
} as Record<string, Record<number, string>>;
const decodePrefixMapObject = Object.fromEntries(
Object.entries(prefixMapObject).flatMap(([prefix, map]) =>
Object.entries(map).map(([len, char]) => [char, { prefix, len: parseInt(len) }])
)
);
const prefixMapNumber = {
n: {
1: "a",
2: "b",
3: "c",
4: "d",
5: "e",
},
N: {
1: "A",
2: "B",
3: "C",
4: "D",
5: "E",
},
} as Record<string, Record<number, string>>;
const decodePrefixMapNumber = Object.fromEntries(
Object.entries(prefixMapNumber).flatMap(([prefix, map]) =>
Object.entries(map).map(([len, char]) => [char, { prefix, len: parseInt(len) }])
)
);
export function encodeAnyArray(obj: any[]): string {
const tempArray = obj.map((v) => {
if (v == null) return "n";
if (v == false) return "f";
if (v == true) return "t";
if (v == undefined) return "u";
if (typeof v == "number") {
const b36 = v.toString(36);
const strNum = v.toString();
const expression = b36.length < strNum.length ? "N" : "n";
const encodedStr = expression == "N" ? b36 : strNum;
const len = encodedStr.length.toString(36);
const lenLen = len.length;
const prefix2 = prefixMapNumber[expression][lenLen];
return prefix2 + len + encodedStr;
}
const str = typeof v == "string" ? v : JSON.stringify(v);
const prefix = typeof v == "string" ? "s" : "o";
const length = str.length.toString(36);
const lenLen = length.length;
const prefix2 = prefixMapObject[prefix][lenLen];
return prefix2 + length + str;
});
const w = tempArray.join("");
return w;
}
const decodeMapConstant = {
u: undefined,
n: null,
f: false,
t: true,
} as Record<string, any>;
export function decodeAnyArray(str: string): any[] {
const result = [];
let i = 0;
while (i < str.length) {
const char = str[i];
i++;
if (char in decodeMapConstant) {
result.push(decodeMapConstant[char]);
continue;
}
if (char in decodePrefixMapNumber) {
const { prefix, len } = decodePrefixMapNumber[char];
const lenStr = str.substring(i, i + len);
i += len;
const radix = prefix == "N" ? 36 : 10;
const lenNum = parseInt(lenStr, 36);
const value = str.substring(i, i + lenNum);
i += lenNum;
result.push(parseInt(value, radix));
continue;
}
const { prefix, len } = decodePrefixMapObject[char];
const lenStr = str.substring(i, i + len);
i += len;
const lenNum = parseInt(lenStr, 36);
const value = str.substring(i, i + lenNum);
i += lenNum;
if (prefix == "s") {
result.push(value);
} else {
result.push(JSON.parse(value));
}
}
return result;
}

4
src/features/ConfigSync/CmdConfigSync.ts
View File

@@ -1306,7 +1306,7 @@ export class ConfigSync extends LiveSyncCommands implements IObsidianModule {
); );
return; return;
} }
const docXDoc = await this.localDatabase.getDBEntryFromMeta(old, {}, false, false); const docXDoc = await this.localDatabase.getDBEntryFromMeta(old, false, false);
if (docXDoc == false) { if (docXDoc == false) {
throw "Could not load the document"; throw "Could not load the document";
} }
@@ -1440,7 +1440,7 @@ export class ConfigSync extends LiveSyncCommands implements IObsidianModule {
// this._log(`STORAGE --> DB:${prefixedFileName}: (config) Skipped (Same time)`, LOG_LEVEL_VERBOSE); // this._log(`STORAGE --> DB:${prefixedFileName}: (config) Skipped (Same time)`, LOG_LEVEL_VERBOSE);
return true; return true;
} }
const oldC = await this.localDatabase.getDBEntryFromMeta(old, {}, false, false); const oldC = await this.localDatabase.getDBEntryFromMeta(old, false, false);
if (oldC) { if (oldC) {
const d = (await deserialize(getDocDataAsArray(oldC.data), {})) as PluginDataEx; const d = (await deserialize(getDocDataAsArray(oldC.data), {})) as PluginDataEx;
if (d.files.length == dt.files.length) { if (d.files.length == dt.files.length) {

7
src/features/HiddenFileCommon/JsonResolveModal.ts
View File

@@ -2,13 +2,14 @@ import { App, Modal } from "../../deps.ts";
import { type FilePath, type LoadedEntry } from "../../lib/src/common/types.ts"; import { type FilePath, type LoadedEntry } from "../../lib/src/common/types.ts";
import JsonResolvePane from "./JsonResolvePane.svelte"; import JsonResolvePane from "./JsonResolvePane.svelte";
import { waitForSignal } from "../../lib/src/common/utils.ts"; import { waitForSignal } from "../../lib/src/common/utils.ts";
import { mount, unmount } from "svelte";
export class JsonResolveModal extends Modal { export class JsonResolveModal extends Modal {
// result: Array<[number, string]>; // result: Array<[number, string]>;
filename: FilePath; filename: FilePath;
callback?: (keepRev?: string, mergedStr?: string) => Promise<void>; callback?: (keepRev?: string, mergedStr?: string) => Promise<void>;
docs: LoadedEntry[]; docs: LoadedEntry[];
component?: JsonResolvePane; component?: ReturnType<typeof mount>;
nameA: string; nameA: string;
nameB: string; nameB: string;
defaultSelect: string; defaultSelect: string;
@@ -55,7 +56,7 @@ export class JsonResolveModal extends Modal {
contentEl.empty(); contentEl.empty();
if (this.component == undefined) { if (this.component == undefined) {
this.component = new JsonResolvePane({ this.component = mount(JsonResolvePane, {
target: contentEl, target: contentEl,
props: { props: {
docs: this.docs, docs: this.docs,
@@ -81,7 +82,7 @@ export class JsonResolveModal extends Modal {
void this.callback(undefined); void this.callback(undefined);
} }
if (this.component != undefined) { if (this.component != undefined) {
this.component.$destroy(); void unmount(this.component);
this.component = undefined; this.component = undefined;
} }
} }

191
src/features/HiddenFileCommon/JsonResolvePane.svelte
View File

@@ -2,29 +2,64 @@
import { type Diff, DIFF_DELETE, DIFF_INSERT, diff_match_patch } from "../../deps.ts"; import { type Diff, DIFF_DELETE, DIFF_INSERT, diff_match_patch } from "../../deps.ts";
import type { FilePath, LoadedEntry } from "../../lib/src/common/types.ts"; import type { FilePath, LoadedEntry } from "../../lib/src/common/types.ts";
import { decodeBinary, readString } from "../../lib/src/string_and_binary/convert.ts"; import { decodeBinary, readString } from "../../lib/src/string_and_binary/convert.ts";
import { getDocData, mergeObject } from "../../lib/src/common/utils.ts"; import { getDocData, isObjectDifferent, mergeObject } from "../../lib/src/common/utils.ts";
export let docs: LoadedEntry[] = []; interface Props {
export let callback: (keepRev?: string, mergedStr?: string) => Promise<void> = async (_, __) => { docs?: LoadedEntry[];
Promise.resolve(); callback?: (keepRev?: string, mergedStr?: string) => Promise<void>;
}; filename?: FilePath;
export let filename: FilePath = "" as FilePath; nameA?: string;
export let nameA: string = "A"; nameB?: string;
export let nameB: string = "B"; defaultSelect?: string;
export let defaultSelect: string = ""; keepOrder?: boolean;
export let keepOrder = false; hideLocal?: boolean;
export let hideLocal: boolean = false; }
let docA: LoadedEntry;
let docB: LoadedEntry; let {
let docAContent = ""; docs = $bindable([]),
let docBContent = ""; callback = $bindable((async (_, __) => {
let objA: any = {}; Promise.resolve();
let objB: any = {}; }) as (keepRev?: string, mergedStr?: string) => Promise<void>),
let objAB: any = {}; filename = $bindable("" as FilePath),
let objBA: any = {}; nameA = $bindable("A"),
let diffs: Diff[]; nameB = $bindable("B"),
defaultSelect = $bindable("" as string),
keepOrder = $bindable(false),
hideLocal = $bindable(false),
}: Props = $props();
type JSONData = Record<string | number | symbol, any> | [any];
const docsArray = $derived.by(() => {
if (docs && docs.length >= 1) {
if (keepOrder || docs[0].mtime < docs[1].mtime) {
return { a: docs[0], b: docs[1] } as const;
} else {
return { a: docs[1], b: docs[0] } as const;
}
}
return { a: false, b: false } as const;
});
const docA = $derived(docsArray.a);
const docB = $derived(docsArray.b);
const docAContent = $derived(docA && docToString(docA));
const docBContent = $derived(docB && docToString(docB));
function parseJson(json: string | false) {
if (json === false) return false;
try {
return JSON.parse(json) as JSONData;
} catch (ex) {
return false;
}
}
const objA = $derived(parseJson(docAContent) || {});
const objB = $derived(parseJson(docBContent) || {});
const objAB = $derived(mergeObject(objA, objB));
const objBAw = $derived(mergeObject(objB, objA));
const objBA = $derived(isObjectDifferent(objBAw, objAB) ? objBAw : false);
let diffs: Diff[] = $derived.by(() => (objA && selectedObj ? getJsonDiff(objA, selectedObj) : []));
type SelectModes = "" | "A" | "B" | "AB" | "BA"; type SelectModes = "" | "A" | "B" | "AB" | "BA";
let mode: SelectModes = defaultSelect as SelectModes; let mode: SelectModes = $state(defaultSelect as SelectModes);
function docToString(doc: LoadedEntry) { function docToString(doc: LoadedEntry) {
return doc.datatype == "plain" ? getDocData(doc.data) : readString(new Uint8Array(decodeBinary(doc.data))); return doc.datatype == "plain" ? getDocData(doc.data) : readString(new Uint8Array(decodeBinary(doc.data)));
@@ -45,6 +80,7 @@
return getDiff(JSON.stringify(a, null, 2), JSON.stringify(b, null, 2)); return getDiff(JSON.stringify(a, null, 2), JSON.stringify(b, null, 2));
} }
function apply() { function apply() {
if (!docA || !docB) return;
if (docA._id == docB._id) { if (docA._id == docB._id) {
if (mode == "A") return callback(docA._rev!, undefined); if (mode == "A") return callback(docA._rev!, undefined);
if (mode == "B") return callback(docB._rev!, undefined); if (mode == "B") return callback(docB._rev!, undefined);
@@ -59,50 +95,23 @@
function cancel() { function cancel() {
callback(undefined, undefined); callback(undefined, undefined);
} }
$: { const mergedObjs = $derived.by(
if (docs && docs.length >= 1) { () =>
if (keepOrder || docs[0].mtime < docs[1].mtime) { ({
docA = docs[0]; "": false,
docB = docs[1]; A: objA,
} else { B: objB,
docA = docs[1]; AB: objAB,
docB = docs[0]; BA: objBA,
} }) as Record<SelectModes, JSONData | false>
docAContent = docToString(docA); );
docBContent = docToString(docB);
try { let selectedObj = $derived(mode in mergedObjs ? mergedObjs[mode] : {});
objA = false;
objB = false;
objA = JSON.parse(docAContent);
objB = JSON.parse(docBContent);
objAB = mergeObject(objA, objB);
objBA = mergeObject(objB, objA);
if (JSON.stringify(objAB) == JSON.stringify(objBA)) {
objBA = false;
}
} catch (ex) {
objBA = false;
objAB = false;
}
}
}
$: mergedObjs = {
"": false,
A: objA,
B: objB,
AB: objAB,
BA: objBA,
};
$: selectedObj = mode in mergedObjs ? mergedObjs[mode] : {}; let modesSrc = $state([] as ["" | "A" | "B" | "AB" | "BA", string][]);
$: {
diffs = getJsonDiff(objA, selectedObj);
}
let modes = [] as ["" | "A" | "B" | "AB" | "BA", string][]; const modes = $derived.by(() => {
$: { let newModes = [] as typeof modesSrc;
let newModes = [] as typeof modes;
if (!hideLocal) { if (!hideLocal) {
newModes.push(["", "Not now"]); newModes.push(["", "Not now"]);
@@ -111,15 +120,15 @@
newModes.push(["B", nameB || "B"]); newModes.push(["B", nameB || "B"]);
newModes.push(["AB", `${nameA || "A"} + ${nameB || "B"}`]); newModes.push(["AB", `${nameA || "A"} + ${nameB || "B"}`]);
newModes.push(["BA", `${nameB || "B"} + ${nameA || "A"}`]); newModes.push(["BA", `${nameB || "B"} + ${nameA || "A"}`]);
modes = newModes; return newModes;
} });
</script> </script>
<h2>{filename}</h2> <h2>{filename}</h2>
{#if !docA || !docB} {#if !docA || !docB}
<div class="message">Just for a minute, please!</div> <div class="message">Just for a minute, please!</div>
<div class="buttons"> <div class="buttons">
<button on:click={apply}>Dismiss</button> <button onclick={apply}>Dismiss</button>
</div> </div>
{:else} {:else}
<div class="options"> <div class="options">
@@ -148,39 +157,39 @@
<div class="infos"> <div class="infos">
<table> <table>
<tbody> <tbody>
<tr> <tr>
<th>{nameA}</th> <th>{nameA}</th>
<td <td
>{#if docA._id == docB._id} >{#if docA._id == docB._id}
Rev:{revStringToRevNumber(docA._rev)} Rev:{revStringToRevNumber(docA._rev)}
{/if} {/if}
{new Date(docA.mtime).toLocaleString()}</td {new Date(docA.mtime).toLocaleString()}</td
> >
<td> <td>
{docAContent.length} letters {docAContent && docAContent.length} letters
</td> </td>
</tr> </tr>
<tr> <tr>
<th>{nameB}</th> <th>{nameB}</th>
<td <td
>{#if docA._id == docB._id} >{#if docA._id == docB._id}
Rev:{revStringToRevNumber(docB._rev)} Rev:{revStringToRevNumber(docB._rev)}
{/if} {/if}
{new Date(docB.mtime).toLocaleString()}</td {new Date(docB.mtime).toLocaleString()}</td
> >
<td> <td>
{docBContent.length} letters {docBContent && docBContent.length} letters
</td> </td>
</tr> </tr>
</tbody> </tbody>
</table> </table>
</div> </div>
<div class="buttons"> <div class="buttons">
{#if hideLocal} {#if hideLocal}
<button on:click={cancel}>Cancel</button> <button onclick={cancel}>Cancel</button>
{/if} {/if}
<button on:click={apply}>Apply</button> <button onclick={apply}>Apply</button>
</div> </div>
{/if} {/if}

4
src/features/HiddenFileSync/CmdHiddenFileSync.ts
View File

@@ -701,7 +701,7 @@ Offline Changed files: ${processFiles.length}`;
?.filter((e) => e.status == "available" && Number(e.rev.split("-")[0]) < conflictedRevNo) ?.filter((e) => e.status == "available" && Number(e.rev.split("-")[0]) < conflictedRevNo)
.first()?.rev ?? ""; .first()?.rev ?? "";
const result = await this.plugin.localDatabase.mergeObject( const result = await this.plugin.localDatabase.mergeObject(
path, doc.path,
commonBase, commonBase,
doc._rev, doc._rev,
conflictedRev conflictedRev
@@ -1490,7 +1490,7 @@ Offline Changed files: ${files.length}`;
} }
return false; return false;
} else { } else {
const fileOnDB = await this.localDatabase.getDBEntryFromMeta(metaOnDB, {}, false, true, true); const fileOnDB = await this.localDatabase.getDBEntryFromMeta(metaOnDB, false, true);
if (fileOnDB === false) { if (fileOnDB === false) {
throw new Error(`Failed to read file from database:${storageFilePath}`); throw new Error(`Failed to read file from database:${storageFilePath}`);
} }

179
src/features/P2PSync/CmdP2PReplicator.ts Normal file
View File

@@ -0,0 +1,179 @@
import type { IObsidianModule } from "../../modules/AbstractObsidianModule";
import { P2PReplicatorPaneView, VIEW_TYPE_P2P } from "./P2PReplicator/P2PReplicatorPaneView.ts";
import {
AutoAccepting,
LOG_LEVEL_NOTICE,
REMOTE_P2P,
type EntryDoc,
type P2PSyncSetting,
type RemoteDBSettings,
} from "../../lib/src/common/types.ts";
import { LiveSyncCommands } from "../LiveSyncCommands.ts";
import { LiveSyncTrysteroReplicator } from "../../lib/src/replication/trystero/LiveSyncTrysteroReplicator.ts";
import { EVENT_REQUEST_OPEN_P2P, eventHub } from "../../common/events.ts";
import type { LiveSyncAbstractReplicator } from "../../lib/src/replication/LiveSyncAbstractReplicator.ts";
import { Logger } from "octagonal-wheels/common/logger";
import type { CommandShim } from "../../lib/src/replication/trystero/P2PReplicatorPaneCommon.ts";
import {
P2PReplicatorMixIn,
removeP2PReplicatorInstance,
type P2PReplicatorBase,
} from "../../lib/src/replication/trystero/P2PReplicatorCore.ts";
import { reactiveSource } from "octagonal-wheels/dataobject/reactive_v2";
import type { Confirm } from "../../lib/src/interfaces/Confirm.ts";
import type ObsidianLiveSyncPlugin from "../../main.ts";
import type { SimpleStore } from "octagonal-wheels/databases/SimpleStoreBase";
import { getPlatformName } from "../../lib/src/PlatformAPIs/obsidian/Environment.ts";
class P2PReplicatorCommandBase extends LiveSyncCommands implements P2PReplicatorBase {
storeP2PStatusLine = reactiveSource("");
getSettings(): P2PSyncSetting {
return this.plugin.settings;
}
get settings() {
return this.plugin.settings;
}
getDB() {
return this.plugin.localDatabase.localDatabase;
}
get confirm(): Confirm {
return this.plugin.confirm;
}
_simpleStore!: SimpleStore<any>;
simpleStore(): SimpleStore<any> {
return this._simpleStore;
}
constructor(plugin: ObsidianLiveSyncPlugin) {
super(plugin);
}
async handleReplicatedDocuments(docs: EntryDoc[]): Promise<void> {
// console.log("Processing Replicated Docs", docs);
return await this.plugin.$$parseReplicationResult(docs as PouchDB.Core.ExistingDocument<EntryDoc>[]);
}
onunload(): void {
throw new Error("Method not implemented.");
}
onload(): void | Promise<void> {
throw new Error("Method not implemented.");
}
init() {
this._simpleStore = this.plugin.$$getSimpleStore("p2p-sync");
return Promise.resolve(this);
}
}
export class P2PReplicator
extends P2PReplicatorMixIn(P2PReplicatorCommandBase)
implements IObsidianModule, CommandShim
{
storeP2PStatusLine = reactiveSource("");
$anyNewReplicator(settingOverride: Partial<RemoteDBSettings> = {}): Promise<LiveSyncAbstractReplicator> {
const settings = { ...this.settings, ...settingOverride };
if (settings.remoteType == REMOTE_P2P) {
return Promise.resolve(new LiveSyncTrysteroReplicator(this.plugin));
}
return undefined!;
}
override getPlatform(): string {
return getPlatformName();
}
override onunload(): void {
removeP2PReplicatorInstance();
void this.close();
}
override onload(): void | Promise<void> {
eventHub.onEvent(EVENT_REQUEST_OPEN_P2P, () => {
void this.openPane();
});
this.p2pLogCollector.p2pReplicationLine.onChanged((line) => {
this.storeP2PStatusLine.value = line.value;
});
}
async $everyOnInitializeDatabase(): Promise<boolean> {
await this.initialiseP2PReplicator();
return Promise.resolve(true);
}
async $allSuspendExtraSync() {
this.plugin.settings.P2P_Enabled = false;
this.plugin.settings.P2P_AutoAccepting = AutoAccepting.NONE;
this.plugin.settings.P2P_AutoBroadcast = false;
this.plugin.settings.P2P_AutoStart = false;
this.plugin.settings.P2P_AutoSyncPeers = "";
this.plugin.settings.P2P_AutoWatchPeers = "";
return await Promise.resolve(true);
}
async $everyOnLoadStart() {
return await Promise.resolve();
}
async openPane() {
await this.plugin.$$showView(VIEW_TYPE_P2P);
}
async $everyOnloadStart(): Promise<boolean> {
this.plugin.registerView(VIEW_TYPE_P2P, (leaf) => new P2PReplicatorPaneView(leaf, this.plugin));
this.plugin.addCommand({
id: "open-p2p-replicator",
name: "P2P Sync : Open P2P Replicator",
callback: async () => {
await this.openPane();
},
});
this.plugin.addCommand({
id: "p2p-establish-connection",
name: "P2P Sync : Connect to the Signalling Server",
checkCallback: (isChecking) => {
if (isChecking) {
return !(this._replicatorInstance?.server?.isServing ?? false);
}
void this.open();
},
});
this.plugin.addCommand({
id: "p2p-close-connection",
name: "P2P Sync : Disconnect from the Signalling Server",
checkCallback: (isChecking) => {
if (isChecking) {
return this._replicatorInstance?.server?.isServing ?? false;
}
Logger(`Closing P2P Connection`, LOG_LEVEL_NOTICE);
void this.close();
},
});
this.plugin.addCommand({
id: "replicate-now-by-p2p",
name: "Replicate now by P2P",
checkCallback: (isChecking) => {
if (isChecking) {
if (this.settings.remoteType == REMOTE_P2P) return false;
if (!this._replicatorInstance?.server?.isServing) return false;
return true;
}
void this._replicatorInstance?.replicateFromCommand(false);
},
});
this.plugin
.addRibbonIcon("waypoints", "P2P Replicator", async () => {
await this.openPane();
})
.addClass("livesync-ribbon-replicate-p2p");
return await Promise.resolve(true);
}
$everyAfterResumeProcess(): Promise<boolean> {
if (this.settings.P2P_Enabled && this.settings.P2P_AutoStart) {
setTimeout(() => void this.open(), 100);
}
return Promise.resolve(true);
}
}

240
src/features/P2PSync/CmdP2PSync.ts
View File

@@ -1,240 +0,0 @@
import type { IObsidianModule } from "../../modules/AbstractObsidianModule";
import { P2PReplicatorPaneView, VIEW_TYPE_P2P } from "./P2PReplicator/P2PReplicatorPaneView.ts";
import { TrysteroReplicator } from "../../lib/src/replication/trystero/TrysteroReplicator.ts";
import {
AutoAccepting,
DEFAULT_SETTINGS,
LOG_LEVEL_INFO,
LOG_LEVEL_NOTICE,
LOG_LEVEL_VERBOSE,
REMOTE_P2P,
type EntryDoc,
type RemoteDBSettings,
} from "../../lib/src/common/types.ts";
import { LiveSyncCommands } from "../LiveSyncCommands.ts";
import {
LiveSyncTrysteroReplicator,
setReplicatorFunc,
} from "../../lib/src/replication/trystero/LiveSyncTrysteroReplicator.ts";
import {
EVENT_DATABASE_REBUILT,
EVENT_PLUGIN_UNLOADED,
EVENT_REQUEST_OPEN_P2P,
EVENT_SETTING_SAVED,
eventHub,
} from "../../common/events.ts";
import {
EVENT_ADVERTISEMENT_RECEIVED,
EVENT_DEVICE_LEAVED,
EVENT_P2P_REQUEST_FORCE_OPEN,
EVENT_REQUEST_STATUS,
} from "../../lib/src/replication/trystero/TrysteroReplicatorP2PServer.ts";
import type { LiveSyncAbstractReplicator } from "../../lib/src/replication/LiveSyncAbstractReplicator.ts";
import { Logger } from "octagonal-wheels/common/logger";
import { $msg } from "../../lib/src/common/i18n.ts";
import type { CommandShim } from "./P2PReplicator/P2PReplicatorPaneCommon.ts";
export class P2PReplicator extends LiveSyncCommands implements IObsidianModule, CommandShim {
$anyNewReplicator(settingOverride: Partial<RemoteDBSettings> = {}): Promise<LiveSyncAbstractReplicator> {
const settings = { ...this.settings, ...settingOverride };
if (settings.remoteType == REMOTE_P2P) {
return Promise.resolve(new LiveSyncTrysteroReplicator(this.plugin));
}
return undefined!;
}
_replicatorInstance?: TrysteroReplicator;
onunload(): void {
setReplicatorFunc(() => undefined);
void this.close();
}
onload(): void | Promise<void> {
setReplicatorFunc(() => this._replicatorInstance);
eventHub.onEvent(EVENT_ADVERTISEMENT_RECEIVED, (peerId) => this._replicatorInstance?.onNewPeer(peerId));
eventHub.onEvent(EVENT_DEVICE_LEAVED, (info) => this._replicatorInstance?.onPeerLeaved(info));
eventHub.onEvent(EVENT_REQUEST_STATUS, () => {
this._replicatorInstance?.requestStatus();
});
eventHub.onEvent(EVENT_P2P_REQUEST_FORCE_OPEN, () => {
void this.open();
});
eventHub.onEvent(EVENT_REQUEST_OPEN_P2P, () => {
void this.openPane();
});
eventHub.onEvent(EVENT_DATABASE_REBUILT, async () => {
await this.initialiseP2PReplicator();
});
eventHub.onEvent(EVENT_PLUGIN_UNLOADED, () => {
void this.close();
});
eventHub.onEvent(EVENT_SETTING_SAVED, async () => {
await this.initialiseP2PReplicator();
});
// throw new Error("Method not implemented.");
}
async $everyOnInitializeDatabase(): Promise<boolean> {
await this.initialiseP2PReplicator();
return Promise.resolve(true);
}
async $allSuspendExtraSync() {
this.plugin.settings.P2P_Enabled = false;
this.plugin.settings.P2P_AutoAccepting = AutoAccepting.NONE;
this.plugin.settings.P2P_AutoBroadcast = false;
this.plugin.settings.P2P_AutoStart = false;
this.plugin.settings.P2P_AutoSyncPeers = "";
this.plugin.settings.P2P_AutoWatchPeers = "";
return await Promise.resolve(true);
}
async $everyOnLoadStart() {
return await Promise.resolve();
}
async openPane() {
await this.plugin.$$showView(VIEW_TYPE_P2P);
}
async $everyOnloadStart(): Promise<boolean> {
this.plugin.registerView(VIEW_TYPE_P2P, (leaf) => new P2PReplicatorPaneView(leaf, this.plugin));
this.plugin.addCommand({
id: "open-p2p-replicator",
name: "P2P Sync : Open P2P Replicator",
callback: async () => {
await this.openPane();
},
});
this.plugin.addCommand({
id: "p2p-establish-connection",
name: "P2P Sync : Connect to the Signalling Server",
checkCallback: (isChecking) => {
if (isChecking) {
return !(this._replicatorInstance?.server?.isServing ?? false);
}
void this.open();
},
});
this.plugin.addCommand({
id: "p2p-close-connection",
name: "P2P Sync : Disconnect from the Signalling Server",
checkCallback: (isChecking) => {
if (isChecking) {
return this._replicatorInstance?.server?.isServing ?? false;
}
Logger(`Closing P2P Connection`, LOG_LEVEL_NOTICE);
void this.close();
},
});
this.plugin.addCommand({
id: "replicate-now-by-p2p",
name: "Replicate now by P2P",
checkCallback: (isChecking) => {
if (isChecking) {
if (this.settings.remoteType == REMOTE_P2P) return false;
if (!this._replicatorInstance?.server?.isServing) return false;
return true;
}
void this._replicatorInstance?.replicateFromCommand(false);
},
});
this.plugin
.addRibbonIcon("waypoints", "P2P Replicator", async () => {
await this.openPane();
})
.addClass("livesync-ribbon-replicate-p2p");
return await Promise.resolve(true);
}
$everyAfterResumeProcess(): Promise<boolean> {
if (this.settings.P2P_Enabled && this.settings.P2P_AutoStart) {
setTimeout(() => void this.open(), 100);
}
return Promise.resolve(true);
}
async open() {
if (!this.settings.P2P_Enabled) {
this._notice($msg("P2P.NotEnabled"));
return;
}
if (!this._replicatorInstance) {
await this.initialiseP2PReplicator();
if (!this.settings.P2P_AutoStart) {
// While auto start is enabled, we don't need to open the connection (Literally, it's already opened automatically)
await this._replicatorInstance!.open();
}
} else {
await this._replicatorInstance?.open();
}
}
async close() {
await this._replicatorInstance?.close();
this._replicatorInstance = undefined;
}
getConfig(key: string) {
const vaultName = this.plugin.$$getVaultName();
const dbKey = `${vaultName}-${key}`;
return localStorage.getItem(dbKey);
}
setConfig(key: string, value: string) {
const vaultName = this.plugin.$$getVaultName();
const dbKey = `${vaultName}-${key}`;
localStorage.setItem(dbKey, value);
}
async initialiseP2PReplicator(): Promise<TrysteroReplicator> {
const getPlugin = () => this.plugin;
try {
// const plugin = this.plugin;
if (this._replicatorInstance) {
await this._replicatorInstance.close();
this._replicatorInstance = undefined;
}
if (!this.settings.P2P_AppID) {
this.settings.P2P_AppID = DEFAULT_SETTINGS.P2P_AppID;
}
const initialDeviceName = this.getConfig("p2p_device_name") || this.plugin.$$getDeviceAndVaultName();
const env = {
get db() {
return getPlugin().localDatabase.localDatabase;
},
get confirm() {
return getPlugin().confirm;
},
get deviceName() {
return initialDeviceName;
},
platform: "wip",
get settings() {
return getPlugin().settings;
},
async processReplicatedDocs(docs: EntryDoc[]): Promise<void> {
return await getPlugin().$$parseReplicationResult(
docs as PouchDB.Core.ExistingDocument<EntryDoc>[]
);
},
simpleStore: getPlugin().$$getSimpleStore("p2p-sync"),
};
this._replicatorInstance = new TrysteroReplicator(env);
if (this.settings.P2P_AutoStart && this.settings.P2P_Enabled) {
await this.open();
}
return this._replicatorInstance;
} catch (e) {
this._log(
e instanceof Error ? e.message : "Something occurred on Initialising P2P Replicator",
LOG_LEVEL_INFO
);
this._log(e, LOG_LEVEL_VERBOSE);
throw e;
}
}
enableBroadcastCastings() {
return this?._replicatorInstance?.enableBroadcastChanges();
}
disableBroadcastCastings() {
return this?._replicatorInstance?.disableBroadcastChanges();
}
}

20
src/features/P2PSync/P2PReplicator/P2PReplicatorPane.svelte
View File

@@ -7,7 +7,7 @@
type CommandShim, type CommandShim,
type PeerStatus, type PeerStatus,
type PluginShim, type PluginShim,
} from "./P2PReplicatorPaneCommon"; } from "../../../lib/src/replication/trystero/P2PReplicatorPaneCommon";
import PeerStatusRow from "../P2PReplicator/PeerStatusRow.svelte"; import PeerStatusRow from "../P2PReplicator/PeerStatusRow.svelte";
import { EVENT_LAYOUT_READY, eventHub } from "../../../common/events"; import { EVENT_LAYOUT_READY, eventHub } from "../../../common/events";
import { import {
@@ -294,7 +294,14 @@
<th> Room ID </th> <th> Room ID </th>
<td> <td>
<label class={{ "is-dirty": isRoomIdModified }}> <label class={{ "is-dirty": isRoomIdModified }}>
<input type="text" placeholder="anything-you-like" bind:value={eRoomId} autocomplete="off"/> <input
type="text"
placeholder="anything-you-like"
bind:value={eRoomId}
autocomplete="off"
spellcheck="false"
autocorrect="off"
/>
<button onclick={() => chooseRandom()}> Use Random Number </button> <button onclick={() => chooseRandom()}> Use Random Number </button>
</label> </label>
<span> <span>
@@ -320,9 +327,14 @@
<th> This device name </th> <th> This device name </th>
<td> <td>
<label class={{ "is-dirty": isDeviceNameModified }}> <label class={{ "is-dirty": isDeviceNameModified }}>
<input type="text" placeholder="iphone-16" bind:value={eDeviceName} <input type="text" placeholder="iphone-16" bind:value={eDeviceName} autocomplete="off" />
autocomplete="off" />
</label> </label>
<span>
<small>
Device name to identify the device. Please use shorter one for the stable peer
detection, i.e., "iphone-16" or "macbook-2021".
</small>
</span>
</td> </td>
</tr> </tr>
<tr> <tr>

58
src/features/P2PSync/P2PReplicator/P2PReplicatorPaneCommon.ts
View File

@@ -1,58 +0,0 @@
import type { P2PSyncSetting } from "../../../lib/src/common/types";
export const EVENT_P2P_PEER_SHOW_EXTRA_MENU = "p2p-peer-show-extra-menu";
export enum AcceptedStatus {
UNKNOWN = "Unknown",
ACCEPTED = "Accepted",
DENIED = "Denied",
ACCEPTED_IN_SESSION = "Accepted in session",
DENIED_IN_SESSION = "Denied in session",
}
export type PeerExtraMenuEvent = {
peer: PeerStatus;
event: MouseEvent;
};
export enum ConnectionStatus {
CONNECTED = "Connected",
CONNECTED_LIVE = "Connected(live)",
DISCONNECTED = "Disconnected",
}
export type PeerStatus = {
name: string;
peerId: string;
syncOnConnect: boolean;
watchOnConnect: boolean;
syncOnReplicationCommand: boolean;
accepted: AcceptedStatus;
status: ConnectionStatus;
isFetching: boolean;
isSending: boolean;
isWatching: boolean;
};
declare global {
interface LSEvents {
[EVENT_P2P_PEER_SHOW_EXTRA_MENU]: PeerExtraMenuEvent;
// [EVENT_P2P_REPLICATOR_PROGRESS]: P2PReplicationReport;
}
}
export interface PluginShim {
saveSettings: () => Promise<void>;
settings: P2PSyncSetting;
rebuilder: any;
$$scheduleAppReload: () => void;
$$getVaultName: () => string;
// confirm: any;
}
export interface CommandShim {
getConfig(key: string): string | null;
setConfig(key: string, value: string): void;
open(): Promise<void>;
close(): Promise<void>;
enableBroadcastCastings(): void; // cmdSync._replicatorInstance?.enableBroadcastChanges();
disableBroadcastCastings(): void; ///cmdSync._replicatorInstance?.disableBroadcastChanges();
}

8
src/features/P2PSync/P2PReplicator/P2PReplicatorPaneView.ts
View File

@@ -4,11 +4,15 @@ import type ObsidianLiveSyncPlugin from "../../../main.ts";
import { mount } from "svelte"; import { mount } from "svelte";
import { SvelteItemView } from "../../../common/SvelteItemView.ts"; import { SvelteItemView } from "../../../common/SvelteItemView.ts";
import { eventHub } from "../../../common/events.ts"; import { eventHub } from "../../../common/events.ts";
import { EVENT_P2P_PEER_SHOW_EXTRA_MENU, type PeerStatus } from "./P2PReplicatorPaneCommon.ts";
import { unique } from "octagonal-wheels/collection"; import { unique } from "octagonal-wheels/collection";
import { LOG_LEVEL_NOTICE, REMOTE_P2P } from "../../../lib/src/common/types.ts"; import { LOG_LEVEL_NOTICE, REMOTE_P2P } from "../../../lib/src/common/types.ts";
import { Logger } from "../../../lib/src/common/logger.ts"; import { Logger } from "../../../lib/src/common/logger.ts";
import { P2PReplicator } from "../CmdP2PSync.ts"; import { P2PReplicator } from "../CmdP2PReplicator.ts";
import {
EVENT_P2P_PEER_SHOW_EXTRA_MENU,
type PeerStatus,
} from "../../../lib/src/replication/trystero/P2PReplicatorPaneCommon.ts";
export const VIEW_TYPE_P2P = "p2p-replicator"; export const VIEW_TYPE_P2P = "p2p-replicator";
function addToList(item: string, list: string) { function addToList(item: string, list: string) {

6
src/features/P2PSync/P2PReplicator/PeerStatusRow.svelte
View File

@@ -1,9 +1,9 @@
<script lang="ts"> <script lang="ts">
import { getContext } from "svelte"; import { getContext } from "svelte";
import { AcceptedStatus, type PeerStatus } from "./P2PReplicatorPaneCommon"; import { AcceptedStatus, type PeerStatus } from "../../../lib/src/replication/trystero/P2PReplicatorPaneCommon";
import type { P2PReplicator } from "../CmdP2PSync"; import type { P2PReplicator } from "../CmdP2PReplicator";
import { eventHub } from "../../../common/events"; import { eventHub } from "../../../common/events";
import { EVENT_P2P_PEER_SHOW_EXTRA_MENU } from "./P2PReplicatorPaneCommon"; import { EVENT_P2P_PEER_SHOW_EXTRA_MENU } from "../../../lib/src/replication/trystero/P2PReplicatorPaneCommon";
interface Props { interface Props {
peerStatus: PeerStatus; peerStatus: PeerStatus;

2
src/lib

Submodule src/lib updated: 9f71ed12ad...be13c18ec1

18
src/main.ts
View File

@@ -18,6 +18,7 @@ import {
type AUTO_MERGED, type AUTO_MERGED,
type RemoteDBSettings, type RemoteDBSettings,
type TweakValues, type TweakValues,
type CouchDBCredentials,
} from "./lib/src/common/types.ts"; } from "./lib/src/common/types.ts";
import { type FileEventItem } from "./common/types.ts"; import { type FileEventItem } from "./common/types.ts";
import { type SimpleStore } from "./lib/src/common/utils.ts"; import { type SimpleStore } from "./lib/src/common/utils.ts";
@@ -82,7 +83,7 @@ import { ModuleReplicateTest } from "./modules/extras/ModuleReplicateTest.ts";
import { ModuleLiveSyncMain } from "./modules/main/ModuleLiveSyncMain.ts"; import { ModuleLiveSyncMain } from "./modules/main/ModuleLiveSyncMain.ts";
import { ModuleExtraSyncObsidian } from "./modules/extraFeaturesObsidian/ModuleExtraSyncObsidian.ts"; import { ModuleExtraSyncObsidian } from "./modules/extraFeaturesObsidian/ModuleExtraSyncObsidian.ts";
import { LocalDatabaseMaintenance } from "./features/LocalDatabaseMainte/CmdLocalDatabaseMainte.ts"; import { LocalDatabaseMaintenance } from "./features/LocalDatabaseMainte/CmdLocalDatabaseMainte.ts";
import { P2PReplicator } from "./features/P2PSync/CmdP2PSync.ts"; import { P2PReplicator } from "./features/P2PSync/CmdP2PReplicator.ts";
function throwShouldBeOverridden(): never { function throwShouldBeOverridden(): never {
throw new Error("This function should be overridden by the module."); throw new Error("This function should be overridden by the module.");
@@ -283,16 +284,14 @@ export default class ObsidianLiveSyncPlugin
$$connectRemoteCouchDB( $$connectRemoteCouchDB(
uri: string, uri: string,
auth: { auth: CouchDBCredentials,
username: string;
password: string;
},
disableRequestURI: boolean, disableRequestURI: boolean,
passphrase: string | false, passphrase: string | false,
useDynamicIterationCount: boolean, useDynamicIterationCount: boolean,
performSetup: boolean, performSetup: boolean,
skipInfo: boolean, skipInfo: boolean,
compression: boolean compression: boolean,
customHeaders: Record<string, string>
): Promise< ): Promise<
| string | string
| { | {
@@ -570,6 +569,9 @@ export default class ObsidianLiveSyncPlugin
$$replicate(showMessage: boolean = false): Promise<boolean | void> { $$replicate(showMessage: boolean = false): Promise<boolean | void> {
throwShouldBeOverridden(); throwShouldBeOverridden();
} }
$$replicateByEvent(showMessage: boolean = false): Promise<boolean | void> {
throwShouldBeOverridden();
}
$everyOnDatabaseInitialized(showingNotice: boolean): Promise<boolean> { $everyOnDatabaseInitialized(showingNotice: boolean): Promise<boolean> {
throwShouldBeOverridden(); throwShouldBeOverridden();
@@ -636,10 +638,6 @@ export default class ObsidianLiveSyncPlugin
throwShouldBeOverridden(); throwShouldBeOverridden();
} }
$$waitForReplicationOnce(): Promise<boolean | void> {
throwShouldBeOverridden();
}
$$resetLocalDatabase(): Promise<void> { $$resetLocalDatabase(): Promise<void> {
throwShouldBeOverridden(); throwShouldBeOverridden();
} }

8
src/modules/core/ModuleDatabaseFileAccess.ts
View File

@@ -313,13 +313,7 @@ export class ModuleDatabaseFileAccess extends AbstractModule implements IObsidia
if (skipCheck && !(await this.checkIsTargetFile(meta.path))) { if (skipCheck && !(await this.checkIsTargetFile(meta.path))) {
return false; return false;
} }
const doc = await this.localDatabase.getDBEntryFromMeta( const doc = await this.localDatabase.getDBEntryFromMeta(meta as LoadedEntry, false, waitForReady);
meta as LoadedEntry,
undefined,
false,
waitForReady,
true
);
if (doc === false) { if (doc === false) {
return false; return false;
} }

2
src/modules/core/ModuleFileHandler.ts
View File

@@ -297,7 +297,7 @@ export class ModuleFileHandler extends AbstractModule implements ICoreModule {
} }
await this.storage.ensureDir(path); await this.storage.ensureDir(path);
const ret = await this.storage.writeFileAuto(path, docData, { ctime: docRead.ctime, mtime: docRead.mtime }); const ret = await this.storage.writeFileAuto(path, docData, { ctime: docRead.ctime, mtime: docRead.mtime });
this.storage.touched(path); await this.storage.touched(path);
this.storage.triggerFileEvent(mode, path); this.storage.triggerFileEvent(mode, path);
return ret; return ret;
} }

6
src/modules/core/ModuleRebuilder.ts
View File

@@ -176,7 +176,7 @@ export class ModuleRebuilder extends AbstractModule implements ICoreModule, Rebu
} }
} }
} }
async fetchLocal(makeLocalChunkBeforeSync?: boolean) { async fetchLocal(makeLocalChunkBeforeSync?: boolean, preventMakeLocalFilesBeforeSync?: boolean) {
await this.core.$allSuspendExtraSync(); await this.core.$allSuspendExtraSync();
await this.askUseNewAdapter(); await this.askUseNewAdapter();
this.core.settings.isConfigured = true; this.core.settings.isConfigured = true;
@@ -189,6 +189,10 @@ export class ModuleRebuilder extends AbstractModule implements ICoreModule, Rebu
this.core.$$markIsReady(); this.core.$$markIsReady();
if (makeLocalChunkBeforeSync) { if (makeLocalChunkBeforeSync) {
await this.core.fileHandler.createAllChunks(true); await this.core.fileHandler.createAllChunks(true);
} else if (!preventMakeLocalFilesBeforeSync) {
await this.core.$$initializeDatabase(true);
} else {
// Do not create local file entries before sync (Means use remote information)
} }
await this.core.$$markRemoteResolved(); await this.core.$$markRemoteResolved();
await delay(500); await delay(500);

220
src/modules/core/ModuleReplicator.ts
View File

@@ -18,17 +18,27 @@ import {
type MetaEntry, type MetaEntry,
} from "../../lib/src/common/types"; } from "../../lib/src/common/types";
import { QueueProcessor } from "octagonal-wheels/concurrency/processor"; import { QueueProcessor } from "octagonal-wheels/concurrency/processor";
import { getPath, isChunk, isValidPath, scheduleTask } from "../../common/utils"; import {
getPath,
isChunk,
isValidPath,
rateLimitedSharedExecution,
scheduleTask,
updatePreviousExecutionTime,
} from "../../common/utils";
import { isAnyNote } from "../../lib/src/common/utils"; import { isAnyNote } from "../../lib/src/common/utils";
import { EVENT_FILE_SAVED, eventHub } from "../../common/events"; import { EVENT_FILE_SAVED, eventHub } from "../../common/events";
import type { LiveSyncAbstractReplicator } from "../../lib/src/replication/LiveSyncAbstractReplicator"; import type { LiveSyncAbstractReplicator } from "../../lib/src/replication/LiveSyncAbstractReplicator";
import { globalSlipBoard } from "../../lib/src/bureau/bureau"; import { globalSlipBoard } from "../../lib/src/bureau/bureau";
import { $msg } from "../../lib/src/common/i18n";
const KEY_REPLICATION_ON_EVENT = "replicationOnEvent";
const REPLICATION_ON_EVENT_FORECASTED_TIME = 5000;
export class ModuleReplicator extends AbstractModule implements ICoreModule { export class ModuleReplicator extends AbstractModule implements ICoreModule {
$everyOnloadAfterLoadSettings(): Promise<boolean> { $everyOnloadAfterLoadSettings(): Promise<boolean> {
eventHub.onEvent(EVENT_FILE_SAVED, () => { eventHub.onEvent(EVENT_FILE_SAVED, () => {
if (this.settings.syncOnSave && !this.core.$$isSuspended()) { if (this.settings.syncOnSave && !this.core.$$isSuspended()) {
scheduleTask("perform-replicate-after-save", 250, () => this.core.$$waitForReplicationOnce()); scheduleTask("perform-replicate-after-save", 250, () => this.core.$$replicateByEvent());
} }
}); });
return Promise.resolve(true); return Promise.resolve(true);
@@ -37,7 +47,7 @@ export class ModuleReplicator extends AbstractModule implements ICoreModule {
async setReplicator() { async setReplicator() {
const replicator = await this.core.$anyNewReplicator(); const replicator = await this.core.$anyNewReplicator();
if (!replicator) { if (!replicator) {
this._log("No replicator is available, this is the fatal error.", LOG_LEVEL_NOTICE); this._log($msg("Replicator.Message.InitialiseFatalError"), LOG_LEVEL_NOTICE);
return false; return false;
} }
this.core.replicator = replicator; this.core.replicator = replicator;
@@ -61,23 +71,91 @@ export class ModuleReplicator extends AbstractModule implements ICoreModule {
await this.loadQueuedFiles(); await this.loadQueuedFiles();
return true; return true;
} }
async $$replicate(showMessage: boolean = false): Promise<boolean | void> { async $$replicate(showMessage: boolean = false): Promise<boolean | void> {
try {
updatePreviousExecutionTime(KEY_REPLICATION_ON_EVENT, REPLICATION_ON_EVENT_FORECASTED_TIME);
return await this.$$_replicate(showMessage);
} finally {
updatePreviousExecutionTime(KEY_REPLICATION_ON_EVENT);
}
}
/**
* obsolete method. No longer maintained and will be removed in the future.
* @deprecated v0.24.17
* @param showMessage If true, show message to the user.
*/
async cleaned(showMessage: boolean) {
Logger(`The remote database has been cleaned.`, showMessage ? LOG_LEVEL_NOTICE : LOG_LEVEL_INFO);
await skipIfDuplicated("cleanup", async () => {
const count = await purgeUnreferencedChunks(this.localDatabase.localDatabase, true);
const message = `The remote database has been cleaned up.
To synchronize, this device must be also cleaned up. ${count} chunk(s) will be erased from this device.
However, If there are many chunks to be deleted, maybe fetching again is faster.
We will lose the history of this device if we fetch the remote database again.
Even if you choose to clean up, you will see this option again if you exit Obsidian and then synchronise again.`;
const CHOICE_FETCH = "Fetch again";
const CHOICE_CLEAN = "Cleanup";
const CHOICE_DISMISS = "Dismiss";
const ret = await this.core.confirm.confirmWithMessage(
"Cleaned",
message,
[CHOICE_FETCH, CHOICE_CLEAN, CHOICE_DISMISS],
CHOICE_DISMISS,
30
);
if (ret == CHOICE_FETCH) {
await this.core.rebuilder.$performRebuildDB("localOnly");
}
if (ret == CHOICE_CLEAN) {
const replicator = this.core.$$getReplicator();
if (!(replicator instanceof LiveSyncCouchDBReplicator)) return;
const remoteDB = await replicator.connectRemoteCouchDBWithSetting(
this.settings,
this.core.$$isMobile(),
true
);
if (typeof remoteDB == "string") {
Logger(remoteDB, LOG_LEVEL_NOTICE);
return false;
}
await purgeUnreferencedChunks(this.localDatabase.localDatabase, false);
this.localDatabase.hashCaches.clear();
// Perform the synchronisation once.
if (await this.core.replicator.openReplication(this.settings, false, showMessage, true)) {
await balanceChunkPurgedDBs(this.localDatabase.localDatabase, remoteDB.db);
await purgeUnreferencedChunks(this.localDatabase.localDatabase, false);
this.localDatabase.hashCaches.clear();
await this.core.$$getReplicator().markRemoteResolved(this.settings);
Logger("The local database has been cleaned up.", showMessage ? LOG_LEVEL_NOTICE : LOG_LEVEL_INFO);
} else {
Logger(
"Replication has been cancelled. Please try it again.",
showMessage ? LOG_LEVEL_NOTICE : LOG_LEVEL_INFO
);
}
}
});
}
async $$_replicate(showMessage: boolean = false): Promise<boolean | void> {
//--? //--?
if (!this.core.$$isReady()) return; if (!this.core.$$isReady()) return;
if (isLockAcquired("cleanup")) { if (isLockAcquired("cleanup")) {
Logger("Database cleaning up is in process. replication has been cancelled", LOG_LEVEL_NOTICE); Logger($msg("Replicator.Message.Cleaned"), LOG_LEVEL_NOTICE);
return; return;
} }
if (this.settings.versionUpFlash != "") { if (this.settings.versionUpFlash != "") {
Logger("Open settings and check message, please. replication has been cancelled.", LOG_LEVEL_NOTICE); Logger($msg("Replicator.Message.VersionUpFlash"), LOG_LEVEL_NOTICE);
return; return;
} }
if (!(await this.core.$everyCommitPendingFileEvent())) { if (!(await this.core.$everyCommitPendingFileEvent())) {
Logger("Some file events are pending. Replication has been cancelled.", LOG_LEVEL_NOTICE); Logger($msg("Replicator.Message.Pending"), LOG_LEVEL_NOTICE);
return false; return false;
} }
if (!(await this.core.$everyBeforeReplicate(showMessage))) { if (!(await this.core.$everyBeforeReplicate(showMessage))) {
Logger(`Replication has been cancelled by some module failure`, LOG_LEVEL_NOTICE); Logger($msg("Replicator.Message.SomeModuleFailed"), LOG_LEVEL_NOTICE);
return false; return false;
} }
@@ -89,109 +167,47 @@ export class ModuleReplicator extends AbstractModule implements ICoreModule {
} else { } else {
if (this.core.replicator?.remoteLockedAndDeviceNotAccepted) { if (this.core.replicator?.remoteLockedAndDeviceNotAccepted) {
if (this.core.replicator.remoteCleaned && this.settings.useIndexedDBAdapter) { if (this.core.replicator.remoteCleaned && this.settings.useIndexedDBAdapter) {
Logger( await this.cleaned(showMessage);
`The remote database has been cleaned.`,
showMessage ? LOG_LEVEL_NOTICE : LOG_LEVEL_INFO
);
await skipIfDuplicated("cleanup", async () => {
const count = await purgeUnreferencedChunks(this.localDatabase.localDatabase, true);
const message = `The remote database has been cleaned up.
To synchronize, this device must be also cleaned up. ${count} chunk(s) will be erased from this device.
However, If there are many chunks to be deleted, maybe fetching again is faster.
We will lose the history of this device if we fetch the remote database again.
Even if you choose to clean up, you will see this option again if you exit Obsidian and then synchronise again.`;
const CHOICE_FETCH = "Fetch again";
const CHOICE_CLEAN = "Cleanup";
const CHOICE_DISMISS = "Dismiss";
const ret = await this.core.confirm.confirmWithMessage(
"Cleaned",
message,
[CHOICE_FETCH, CHOICE_CLEAN, CHOICE_DISMISS],
CHOICE_DISMISS,
30
);
if (ret == CHOICE_FETCH) {
await this.core.rebuilder.$performRebuildDB("localOnly");
}
if (ret == CHOICE_CLEAN) {
const replicator = this.core.$$getReplicator();
if (!(replicator instanceof LiveSyncCouchDBReplicator)) return;
const remoteDB = await replicator.connectRemoteCouchDBWithSetting(
this.settings,
this.core.$$isMobile(),
true
);
if (typeof remoteDB == "string") {
Logger(remoteDB, LOG_LEVEL_NOTICE);
return false;
}
await purgeUnreferencedChunks(this.localDatabase.localDatabase, false);
this.localDatabase.hashCaches.clear();
// Perform the synchronisation once.
if (
await this.core.replicator.openReplication(this.settings, false, showMessage, true)
) {
await balanceChunkPurgedDBs(this.localDatabase.localDatabase, remoteDB.db);
await purgeUnreferencedChunks(this.localDatabase.localDatabase, false);
this.localDatabase.hashCaches.clear();
await this.core.$$getReplicator().markRemoteResolved(this.settings);
Logger(
"The local database has been cleaned up.",
showMessage ? LOG_LEVEL_NOTICE : LOG_LEVEL_INFO
);
} else {
Logger(
"Replication has been cancelled. Please try it again.",
showMessage ? LOG_LEVEL_NOTICE : LOG_LEVEL_INFO
);
}
}
});
} else { } else {
const message = ` const message = $msg("Replicator.Dialogue.Locked.Message");
The remote database has been rebuilt. const CHOICE_FETCH = $msg("Replicator.Dialogue.Locked.Action.Fetch");
To synchronize, this device must fetch everything again once. const CHOICE_DISMISS = $msg("Replicator.Dialogue.Locked.Action.Dismiss");
Or if you are sure know what had been happened, we can unlock the database from the setting dialog. const CHOICE_UNLOCK = $msg("Replicator.Dialogue.Locked.Action.Unlock");
`; const ret = await this.core.confirm.askSelectStringDialogue(
const CHOICE_FETCH = "Fetch again";
const CHOICE_DISMISS = "Dismiss";
const ret = await this.core.confirm.confirmWithMessage(
"Locked",
message, message,
[CHOICE_FETCH, CHOICE_DISMISS], [CHOICE_FETCH, CHOICE_UNLOCK, CHOICE_DISMISS],
CHOICE_DISMISS, {
10 title: $msg("Replicator.Dialogue.Locked.Title"),
defaultAction: CHOICE_DISMISS,
timeout: 60,
}
); );
if (ret == CHOICE_FETCH) { if (ret == CHOICE_FETCH) {
const CHOICE_RESTART = "Restart"; this._log($msg("Replicator.Dialogue.Locked.Message.Fetch"), LOG_LEVEL_NOTICE);
const CHOICE_WITHOUT_RESTART = "Without restart"; await this.core.rebuilder.scheduleFetch();
if ( this.core.$$scheduleAppReload();
(await this.core.confirm.askSelectStringDialogue( return;
"Self-hosted LiveSync restarts Obsidian to fetch everything safely. However, you can do it without restarting. Please choose one.", } else if (ret == CHOICE_UNLOCK) {
[CHOICE_RESTART, CHOICE_WITHOUT_RESTART], await this.core.replicator.markRemoteResolved(this.settings);
{ this._log($msg("Replicator.Dialogue.Locked.Message.Unlocked"), LOG_LEVEL_NOTICE);
title: "Fetch again", return;
defaultAction: CHOICE_RESTART,
timeout: 30,
}
)) == CHOICE_RESTART
) {
await this.core.rebuilder.scheduleFetch();
// await this.core.$$scheduleAppReload();
return;
} else {
await this.core.rebuilder.$performRebuildDB("localOnly");
}
} }
} }
} }
} }
} }
return ret; return ret;
} }
async $$replicateByEvent(): Promise<boolean | void> {
const least = this.settings.syncMinimumInterval;
if (least > 0) {
return rateLimitedSharedExecution(KEY_REPLICATION_ON_EVENT, least, async () => {
return await this.$$replicate();
});
}
return await shareRunningResult(`replication`, () => this.core.$$replicate());
}
$$parseReplicationResult(docs: Array<PouchDB.Core.ExistingDocument<EntryDoc>>): void { $$parseReplicationResult(docs: Array<PouchDB.Core.ExistingDocument<EntryDoc>>): void {
if (this.settings.suspendParseReplicationResult && !this.replicationResultProcessor.isSuspended) { if (this.settings.suspendParseReplicationResult && !this.replicationResultProcessor.isSuspended) {
this.replicationResultProcessor.suspend(); this.replicationResultProcessor.suspend();
@@ -322,7 +338,7 @@ Or if you are sure know what had been happened, we can unlock the database from
// If `Read chunks online` is disabled, chunks should be transferred before here. // If `Read chunks online` is disabled, chunks should be transferred before here.
// However, in some cases, chunks are after that. So, if missing chunks exist, we have to wait for them. // However, in some cases, chunks are after that. So, if missing chunks exist, we have to wait for them.
const doc = await this.localDatabase.getDBEntryFromMeta({ ...dbDoc }, {}, false, true, true); const doc = await this.localDatabase.getDBEntryFromMeta({ ...dbDoc }, false, true);
if (!doc) { if (!doc) {
Logger( Logger(
`Something went wrong while gathering content of ${path} (${dbDoc._id.substring(0, 8)}, ${dbDoc._rev?.substring(0, 10)}) `, `Something went wrong while gathering content of ${path} (${dbDoc._id.substring(0, 8)}, ${dbDoc._rev?.substring(0, 10)}) `,
@@ -387,7 +403,7 @@ Or if you are sure know what had been happened, we can unlock the database from
): Promise<boolean> { ): Promise<boolean> {
if (!this.core.$$isReady()) return false; if (!this.core.$$isReady()) return false;
if (!(await this.core.$everyBeforeReplicate(showingNotice))) { if (!(await this.core.$everyBeforeReplicate(showingNotice))) {
Logger(`Replication has been cancelled by some module failure`, LOG_LEVEL_NOTICE); Logger($msg("Replicator.Message.SomeModuleFailed"), LOG_LEVEL_NOTICE);
return false; return false;
} }
if (!sendChunksInBulkDisabled) { if (!sendChunksInBulkDisabled) {
@@ -416,8 +432,4 @@ Or if you are sure know what had been happened, we can unlock the database from
if (checkResult == "CHECKAGAIN") return await this.core.$$replicateAllFromServer(showingNotice); if (checkResult == "CHECKAGAIN") return await this.core.$$replicateAllFromServer(showingNotice);
return !checkResult; return !checkResult;
} }
async $$waitForReplicationOnce(): Promise<boolean | void> {
return await shareRunningResult(`replication`, () => this.core.$$replicate());
}
} }

16
src/modules/coreFeatures/ModuleConflictResolver.ts
View File

@@ -44,13 +44,16 @@ export class ModuleConflictResolver extends AbstractModule implements ICoreModul
return MISSING_OR_ERROR; return MISSING_OR_ERROR;
} }
eventHub.emitEvent("conflict-cancelled", path); eventHub.emitEvent("conflict-cancelled", path);
this._log(`${title} Conflicted revision deleted ${displayRev(deleteRevision)} ${path}`, LOG_LEVEL_INFO); this._log(
`${title} Conflicted revision has been deleted ${displayRev(deleteRevision)} ${path}`,
LOG_LEVEL_INFO
);
if ((await this.core.databaseFileAccess.getConflictedRevs(path)).length != 0) { if ((await this.core.databaseFileAccess.getConflictedRevs(path)).length != 0) {
this._log(`${title} some conflicts are left in ${path}`, LOG_LEVEL_INFO); this._log(`${title} some conflicts are left in ${path}`, LOG_LEVEL_INFO);
return AUTO_MERGED; return AUTO_MERGED;
} }
this._log(`${title} ${path} is a plugin metadata file, no need to write to storage`, LOG_LEVEL_INFO);
if (isPluginMetadata(path) || isCustomisationSyncMetadata(path)) { if (isPluginMetadata(path) || isCustomisationSyncMetadata(path)) {
this._log(`${title} ${path} is a plugin metadata file, no need to write to storage`, LOG_LEVEL_INFO);
return AUTO_MERGED; return AUTO_MERGED;
} }
// If no conflicts were found, write the resolved content to the storage. // If no conflicts were found, write the resolved content to the storage.
@@ -58,7 +61,8 @@ export class ModuleConflictResolver extends AbstractModule implements ICoreModul
this._log(`Could not write the resolved content to the storage: ${path}`, LOG_LEVEL_NOTICE); this._log(`Could not write the resolved content to the storage: ${path}`, LOG_LEVEL_NOTICE);
return MISSING_OR_ERROR; return MISSING_OR_ERROR;
} }
this._log(`${path} Has been merged automatically`, LOG_LEVEL_NOTICE); const level = subTitle.indexOf("same") !== -1 ? LOG_LEVEL_INFO : LOG_LEVEL_NOTICE;
this._log(`${path} has been merged automatically`, level);
return AUTO_MERGED; return AUTO_MERGED;
} }
@@ -108,7 +112,9 @@ export class ModuleConflictResolver extends AbstractModule implements ICoreModul
`${isSame ? "same" : ""}`, `${isSame ? "same" : ""}`,
`${isBinary ? "binary" : ""}`, `${isBinary ? "binary" : ""}`,
`${alwaysNewer ? "alwaysNewer" : ""}`, `${alwaysNewer ? "alwaysNewer" : ""}`,
].join(","); ]
.filter((e) => e.trim())
.join(",");
return await this.core.$$resolveConflictByDeletingRev(path, loser.rev, subTitle); return await this.core.$$resolveConflictByDeletingRev(path, loser.rev, subTitle);
} }
// make diff. // make diff.
@@ -140,7 +146,7 @@ export class ModuleConflictResolver extends AbstractModule implements ICoreModul
//auto resolved, but need check again; //auto resolved, but need check again;
if (this.settings.syncAfterMerge && !this.core.$$isSuspended()) { if (this.settings.syncAfterMerge && !this.core.$$isSuspended()) {
//Wait for the running replication, if not running replication, run it once. //Wait for the running replication, if not running replication, run it once.
await this.core.$$waitForReplicationOnce(); await this.core.$$replicateByEvent();
} }
this._log("[conflict] Automatically merged, but we have to check it again"); this._log("[conflict] Automatically merged, but we have to check it again");
await this.core.$$queueConflictCheck(filename); await this.core.$$queueConflictCheck(filename);

38
src/modules/coreFeatures/ModuleRedFlag.ts
View File

@@ -9,6 +9,7 @@ import {
} from "../../lib/src/common/types.ts"; } from "../../lib/src/common/types.ts";
import { AbstractModule } from "../AbstractModule.ts"; import { AbstractModule } from "../AbstractModule.ts";
import type { ICoreModule } from "../ModuleTypes.ts"; import type { ICoreModule } from "../ModuleTypes.ts";
import { $msg } from "../../lib/src/common/i18n.ts";
export class ModuleRedFlag extends AbstractModule implements ICoreModule { export class ModuleRedFlag extends AbstractModule implements ICoreModule {
async isFlagFileExist(path: string) { async isFlagFileExist(path: string) {
@@ -105,16 +106,35 @@ export class ModuleRedFlag extends AbstractModule implements ICoreModule {
`${FLAGMD_REDFLAG3} or ${FLAGMD_REDFLAG3_HR} has been detected! Self-hosted LiveSync will discard the local database and fetch everything from the remote once again.`, `${FLAGMD_REDFLAG3} or ${FLAGMD_REDFLAG3_HR} has been detected! Self-hosted LiveSync will discard the local database and fetch everything from the remote once again.`,
LOG_LEVEL_NOTICE LOG_LEVEL_NOTICE
); );
const makeLocalChunkBeforeSync = const method1 = $msg("RedFlag.Fetch.Method.FetchSafer");
(await this.core.confirm.askYesNoDialog( const method2 = $msg("RedFlag.Fetch.Method.FetchSmoother");
`Do you want to create local chunks before fetching? const method3 = $msg("RedFlag.Fetch.Method.FetchTraditional");
> [!MORE]-
> If creating local chunks before fetching, only the difference between the local and remote will be fetched. const methods = [method1, method2, method3] as const;
const chunkMode = await this.core.confirm.askSelectStringDialogue(
$msg("RedFlag.Fetch.Method.Desc"),
methods,
{
defaultAction: method1,
timeout: 0,
title: $msg("RedFlag.Fetch.Method.Title"),
}
);
let makeLocalChunkBeforeSync = false;
let preventMakeLocalFilesBeforeSync = false;
if (chunkMode === method1) {
preventMakeLocalFilesBeforeSync = true;
} else if (chunkMode === method2) {
makeLocalChunkBeforeSync = true;
} else if (chunkMode === method3) {
// Do nothing.
} else {
this._log("Cancelled the fetch operation", LOG_LEVEL_NOTICE);
return false;
}
await this.core.rebuilder.$fetchLocal(makeLocalChunkBeforeSync, preventMakeLocalFilesBeforeSync);
`,
{ defaultOption: "Yes", title: "Trick to transfer efficiently" }
)) == "yes";
await this.core.rebuilder.$fetchLocal(makeLocalChunkBeforeSync);
await this.deleteRedFlag3(); await this.deleteRedFlag3();
if (this.settings.suspendFileWatching) { if (this.settings.suspendFileWatching) {
if ( if (

193
src/modules/coreFeatures/ModuleResolveMismatchedTweaks.ts
View File

@@ -2,14 +2,17 @@ import { Logger, LOG_LEVEL_NOTICE } from "octagonal-wheels/common/logger";
import { extractObject } from "octagonal-wheels/object"; import { extractObject } from "octagonal-wheels/object";
import { import {
TweakValuesShouldMatchedTemplate, TweakValuesShouldMatchedTemplate,
CompatibilityBreakingTweakValues, IncompatibleChanges,
confName, confName,
type TweakValues, type TweakValues,
type RemoteDBSettings, type RemoteDBSettings,
IncompatibleChangesInSpecificPattern,
CompatibleButLossyChanges,
} from "../../lib/src/common/types.ts"; } from "../../lib/src/common/types.ts";
import { escapeMarkdownValue } from "../../lib/src/common/utils.ts"; import { escapeMarkdownValue } from "../../lib/src/common/utils.ts";
import { AbstractModule } from "../AbstractModule.ts"; import { AbstractModule } from "../AbstractModule.ts";
import type { ICoreModule } from "../ModuleTypes.ts"; import type { ICoreModule } from "../ModuleTypes.ts";
import { $msg } from "../../lib/src/common/i18n.ts";
export class ModuleResolvingMismatchedTweaks extends AbstractModule implements ICoreModule { export class ModuleResolvingMismatchedTweaks extends AbstractModule implements ICoreModule {
async $anyAfterConnectCheckFailed(): Promise<boolean | "CHECKAGAIN" | undefined> { async $anyAfterConnectCheckFailed(): Promise<boolean | "CHECKAGAIN" | undefined> {
@@ -28,65 +31,100 @@ export class ModuleResolvingMismatchedTweaks extends AbstractModule implements I
const mine = extractObject(TweakValuesShouldMatchedTemplate, this.settings); const mine = extractObject(TweakValuesShouldMatchedTemplate, this.settings);
const items = Object.entries(TweakValuesShouldMatchedTemplate); const items = Object.entries(TweakValuesShouldMatchedTemplate);
let rebuildRequired = false; let rebuildRequired = false;
let rebuildRecommended = false;
// Making tables: // Making tables:
let table = `| Value name | This device | Configured | \n` + `|: --- |: --- :|: ---- :| \n`; // let table = `| Value name | This device | Configured | \n` + `|: --- |: --- :|: ---- :| \n`;
const tableRows = [];
// const items = [mine,preferred] // const items = [mine,preferred]
for (const v of items) { for (const v of items) {
const key = v[0] as keyof typeof TweakValuesShouldMatchedTemplate; const key = v[0] as keyof typeof TweakValuesShouldMatchedTemplate;
const valueMine = escapeMarkdownValue(mine[key]); const valueMine = escapeMarkdownValue(mine[key]);
const valuePreferred = escapeMarkdownValue(preferred[key]); const valuePreferred = escapeMarkdownValue(preferred[key]);
if (valueMine == valuePreferred) continue; if (valueMine == valuePreferred) continue;
if (CompatibilityBreakingTweakValues.indexOf(key) !== -1) { if (IncompatibleChanges.indexOf(key) !== -1) {
rebuildRequired = true; rebuildRequired = true;
} }
table += `| ${confName(key)} | ${valueMine} | ${valuePreferred} | \n`; for (const pattern of IncompatibleChangesInSpecificPattern) {
if (pattern.key !== key) continue;
// if from value supplied, check if current value have been violated : in other words, if the current value is the same as the from value, it should require a rebuild.
const isFromConditionMet = "from" in pattern ? pattern.from === mine[key] : false;
// and, if to value supplied, same as above.
const isToConditionMet = "to" in pattern ? pattern.to === preferred[key] : false;
// if either of them is true, it should require a rebuild, if the pattern is not a recommendation.
if (isFromConditionMet || isToConditionMet) {
if (pattern.isRecommendation) {
rebuildRecommended = true;
} else {
rebuildRequired = true;
}
}
}
if (CompatibleButLossyChanges.indexOf(key) !== -1) {
rebuildRecommended = true;
}
// table += `| ${confName(key)} | ${valueMine} | ${valuePreferred} | \n`;
tableRows.push(
$msg("TweakMismatchResolve.Table.Row", {
name: confName(key),
self: valueMine,
remote: valuePreferred,
})
);
} }
const additionalMessage = rebuildRequired const additionalMessage =
? ` rebuildRequired && this.core.settings.isConfigured
? $msg("TweakMismatchResolve.Message.WarningIncompatibleRebuildRequired")
: "";
const additionalMessage2 =
rebuildRecommended && this.core.settings.isConfigured
? $msg("TweakMismatchResolve.Message.WarningIncompatibleRebuildRecommended")
: "";
**Note**: We have detected that some of the values are different to make incompatible the local database with the remote database. const table = $msg("TweakMismatchResolve.Table", { rows: tableRows.join("\n") });
If you choose to use the configured values, the local database will be rebuilt, and if you choose to use the values of this device, the remote database will be rebuilt.
Both of them takes a few minutes. Please choose after considering the situation.`
: "";
const message = ` const message = $msg("TweakMismatchResolve.Message.MainTweakResolving", {
Your configuration has not been matched with the one on the remote server. table: table,
(Which you had decided once before, or set by initially synchronised device). additionalMessage: [additionalMessage, additionalMessage2].filter((v) => v).join("\n"),
});
Configured values: const CHOICE_USE_REMOTE = $msg("TweakMismatchResolve.Action.UseRemote");
const CHOICE_USE_REMOTE_WITH_REBUILD = $msg("TweakMismatchResolve.Action.UseRemoteWithRebuild");
const CHOICE_USE_REMOTE_PREVENT_REBUILD = $msg("TweakMismatchResolve.Action.UseRemoteAcceptIncompatible");
const CHOICE_USE_MINE = $msg("TweakMismatchResolve.Action.UseMine");
const CHOICE_USE_MINE_WITH_REBUILD = $msg("TweakMismatchResolve.Action.UseMineWithRebuild");
const CHOICE_USE_MINE_PREVENT_REBUILD = $msg("TweakMismatchResolve.Action.UseMineAcceptIncompatible");
const CHOICE_DISMISS = $msg("TweakMismatchResolve.Action.Dismiss");
${table} const CHOICE_AND_VALUES = [] as [string, [result: TweakValues | boolean, rebuild: boolean]][];
Please select which one you want to use. if (rebuildRequired) {
CHOICE_AND_VALUES.push([CHOICE_USE_REMOTE_WITH_REBUILD, [preferred, true]]);
- Use configured: Update settings of this device by configured one on the remote server. CHOICE_AND_VALUES.push([CHOICE_USE_MINE_WITH_REBUILD, [true, true]]);
You should select this if you have changed the settings on ** another device **. CHOICE_AND_VALUES.push([CHOICE_USE_REMOTE_PREVENT_REBUILD, [preferred, false]]);
- Update with mine: Update settings on the remote server by the settings of this device. CHOICE_AND_VALUES.push([CHOICE_USE_MINE_PREVENT_REBUILD, [true, false]]);
You should select this if you have changed the settings on ** this device **. } else if (rebuildRecommended) {
- Dismiss: Ignore this message and keep the current settings. CHOICE_AND_VALUES.push([CHOICE_USE_REMOTE, [preferred, false]]);
You cannot synchronise until you resolve this issue without enabling \`Do not check configuration mismatch before replication\`.${additionalMessage}`; CHOICE_AND_VALUES.push([CHOICE_USE_MINE, [true, false]]);
CHOICE_AND_VALUES.push([CHOICE_USE_REMOTE_WITH_REBUILD, [true, true]]);
const CHOICE_USE_REMOTE = "Use configured"; CHOICE_AND_VALUES.push([CHOICE_USE_MINE_WITH_REBUILD, [true, true]]);
const CHOICE_USR_MINE = "Update with mine"; } else {
const CHOICE_DISMISS = "Dismiss"; CHOICE_AND_VALUES.push([CHOICE_USE_REMOTE, [preferred, false]]);
const CHOICE_AND_VALUES = [ CHOICE_AND_VALUES.push([CHOICE_USE_MINE, [true, false]]);
[CHOICE_USE_REMOTE, preferred], }
[CHOICE_USR_MINE, true], CHOICE_AND_VALUES.push([CHOICE_DISMISS, [false, false]]);
[CHOICE_DISMISS, false], const CHOICES = Object.fromEntries(CHOICE_AND_VALUES) as Record<
]; string,
const CHOICES = Object.fromEntries(CHOICE_AND_VALUES) as Record<string, TweakValues | boolean>; [TweakValues | boolean, performRebuild: boolean]
const retKey = await this.core.confirm.confirmWithMessage( >;
"Tweaks Mismatched or Changed", const retKey = await this.core.confirm.askSelectStringDialogue(message, Object.keys(CHOICES), {
message, title: $msg("TweakMismatchResolve.Title.TweakResolving"),
Object.keys(CHOICES), timeout: 60,
CHOICE_DISMISS, defaultAction: CHOICE_DISMISS,
60 });
);
if (!retKey) return [false, false]; if (!retKey) return [false, false];
return [CHOICES[retKey], rebuildRequired]; return CHOICES[retKey];
} }
async $$askResolvingMismatchedTweaks(): Promise<"OK" | "CHECKAGAIN" | "IGNORE"> { async $$askResolvingMismatchedTweaks(): Promise<"OK" | "CHECKAGAIN" | "IGNORE"> {
@@ -143,28 +181,56 @@ Please select which one you want to use.
return { result: false, requireFetch: false }; return { result: false, requireFetch: false };
} }
} }
async $$askUseRemoteConfiguration( async $$askUseRemoteConfiguration(
trialSetting: RemoteDBSettings, trialSetting: RemoteDBSettings,
preferred: TweakValues preferred: TweakValues
): Promise<{ result: false | TweakValues; requireFetch: boolean }> { ): Promise<{ result: false | TweakValues; requireFetch: boolean }> {
const items = Object.entries(TweakValuesShouldMatchedTemplate); const items = Object.entries(TweakValuesShouldMatchedTemplate);
let rebuildRequired = false; let rebuildRequired = false;
let rebuildRecommended = false;
// Making tables: // Making tables:
let table = `| Value name | This device | Stored | \n` + `|: --- |: ---- :|: ---- :| \n`; // let table = `| Value name | This device | On Remote | \n` + `|: --- |: ---- :|: ---- :| \n`;
let differenceCount = 0; let differenceCount = 0;
const tableRows = [] as string[];
// const items = [mine,preferred] // const items = [mine,preferred]
for (const v of items) { for (const v of items) {
const key = v[0] as keyof typeof TweakValuesShouldMatchedTemplate; const key = v[0] as keyof typeof TweakValuesShouldMatchedTemplate;
const valuePreferred = escapeMarkdownValue(preferred[key]); const remoteValueForDisplay = escapeMarkdownValue(preferred[key]);
const currentDisp = `${escapeMarkdownValue((trialSetting as TweakValues)?.[key])} |`; const currentValueForDisplay = `${escapeMarkdownValue((trialSetting as TweakValues)?.[key])}`;
if ((trialSetting as TweakValues)?.[key] !== preferred[key]) { if ((trialSetting as TweakValues)?.[key] !== preferred[key]) {
if (CompatibilityBreakingTweakValues.indexOf(key) !== -1) { if (IncompatibleChanges.indexOf(key) !== -1) {
rebuildRequired = true; rebuildRequired = true;
} }
for (const pattern of IncompatibleChangesInSpecificPattern) {
if (pattern.key !== key) continue;
// if from value supplied, check if current value have been violated : in other words, if the current value is the same as the from value, it should require a rebuild.
const isFromConditionMet =
"from" in pattern ? pattern.from === (trialSetting as TweakValues)?.[key] : false;
// and, if to value supplied, same as above.
const isToConditionMet = "to" in pattern ? pattern.to === preferred[key] : false;
// if either of them is true, it should require a rebuild, if the pattern is not a recommendation.
if (isFromConditionMet || isToConditionMet) {
if (pattern.isRecommendation) {
rebuildRecommended = true;
} else {
rebuildRequired = true;
}
}
}
if (CompatibleButLossyChanges.indexOf(key) !== -1) {
rebuildRecommended = true;
}
} else { } else {
continue; continue;
} }
table += `| ${confName(key)} | ${currentDisp} ${valuePreferred} | \n`; tableRows.push(
$msg("TweakMismatchResolve.Table.Row", {
name: confName(key),
self: currentValueForDisplay,
remote: remoteValueForDisplay,
})
);
differenceCount++; differenceCount++;
} }
@@ -174,33 +240,28 @@ Please select which one you want to use.
} }
const additionalMessage = const additionalMessage =
rebuildRequired && this.core.settings.isConfigured rebuildRequired && this.core.settings.isConfigured
? ` ? $msg("TweakMismatchResolve.Message.UseRemote.WarningRebuildRequired")
: "";
>[!WARNING] const additionalMessage2 =
> Some remote configurations are not compatible with the local database of this device. Rebuilding the local database will be required. rebuildRecommended && this.core.settings.isConfigured
***Please ensure that you have time and are connected to a stable network to apply!***` ? $msg("TweakMismatchResolve.Message.UseRemote.WarningRebuildRecommended")
: ""; : "";
const message = ` const table = $msg("TweakMismatchResolve.Table", { rows: tableRows.join("\n") });
The settings in the remote database are as follows.
If you want to use these settings, please select "Use configured".
If you want to keep the settings of this device, please select "Dismiss".
${table} const message = $msg("TweakMismatchResolve.Message.Main", {
table: table,
additionalMessage: [additionalMessage, additionalMessage2].filter((v) => v).join("\n"),
});
>[!TIP] const CHOICE_USE_REMOTE = $msg("TweakMismatchResolve.Action.UseConfigured");
> If you want to synchronise all settings, please use \`Sync settings via markdown\` after applying minimal configuration with this feature. const CHOICE_DISMISS = $msg("TweakMismatchResolve.Action.Dismiss");
${additionalMessage}`;
const CHOICE_USE_REMOTE = "Use configured";
const CHOICE_DISMISS = "Dismiss";
// const CHOICE_AND_VALUES = [ // const CHOICE_AND_VALUES = [
// [CHOICE_USE_REMOTE, preferred], // [CHOICE_USE_REMOTE, preferred],
// [CHOICE_DISMISS, false]] // [CHOICE_DISMISS, false]]
const CHOICES = [CHOICE_USE_REMOTE, CHOICE_DISMISS]; const CHOICES = [CHOICE_USE_REMOTE, CHOICE_DISMISS];
const retKey = await this.core.confirm.askSelectStringDialogue(message, CHOICES, { const retKey = await this.core.confirm.askSelectStringDialogue(message, CHOICES, {
title: "Use Remote Configuration", title: $msg("TweakMismatchResolve.Title.UseRemoteConfig"),
timeout: 0, timeout: 0,
defaultAction: CHOICE_DISMISS, defaultAction: CHOICE_DISMISS,
}); });

29
src/modules/coreObsidian/ModuleFileAccessObsidian.ts
View File

@@ -1,4 +1,4 @@
import { normalizePath, TFile, TFolder, type ListedFiles } from "obsidian"; import { TFile, TFolder, type ListedFiles } from "obsidian";
import { SerializedFileAccess } from "./storageLib/SerializedFileAccess"; import { SerializedFileAccess } from "./storageLib/SerializedFileAccess";
import { AbstractObsidianModule, type IObsidianModule } from "../AbstractObsidianModule.ts"; import { AbstractObsidianModule, type IObsidianModule } from "../AbstractObsidianModule.ts";
import { LOG_LEVEL_INFO, LOG_LEVEL_VERBOSE } from "octagonal-wheels/common/logger"; import { LOG_LEVEL_INFO, LOG_LEVEL_VERBOSE } from "octagonal-wheels/common/logger";
@@ -60,7 +60,23 @@ export class ModuleFileAccessObsidian extends AbstractObsidianModule implements
if (file instanceof TFile) { if (file instanceof TFile) {
return this.vaultAccess.vaultModify(file, data, opt); return this.vaultAccess.vaultModify(file, data, opt);
} else if (file === null) { } else if (file === null) {
return (await this.vaultAccess.vaultCreate(path, data, opt)) instanceof TFile; if (!path.endsWith(".md")) {
// Very rare case, we encountered this case with `writing-goals-history.csv` file.
// Indeed, that file not appears in the File Explorer, but it exists in the vault.
// Hence, we cannot retrieve the file from the vault by getAbstractFileByPath, and we cannot write it via vaultModify.
// It makes `File already exists` error.
// Therefore, we need to write it via adapterWrite.
// Maybe there are others like this, so I will write it via adapterWrite.
// This is a workaround for the issue, but I don't know if this is the right solution.
// (So limits to non-md files).
// Has Obsidian been patched?, anyway, writing directly might be a safer approach.
// However, does changes of that file trigger file-change event?
await this.vaultAccess.adapterWrite(path, data, opt);
// For safety, check existence
return await this.vaultAccess.adapterExists(path);
} else {
return (await this.vaultAccess.vaultCreate(path, data, opt)) instanceof TFile;
}
} else { } else {
this._log(`Could not write file (Possibly already exists as a folder): ${path}`, LOG_LEVEL_VERBOSE); this._log(`Could not write file (Possibly already exists as a folder): ${path}`, LOG_LEVEL_VERBOSE);
return false; return false;
@@ -158,8 +174,9 @@ export class ModuleFileAccessObsidian extends AbstractObsidianModule implements
} }
} }
triggerFileEvent(event: string, path: string): void { triggerFileEvent(event: string, path: string): void {
// this.app.vault.trigger("file-change", path); const file = this.vaultAccess.getAbstractFileByPath(path);
this.vaultAccess.trigger(event, this.vaultAccess.getAbstractFileByPath(normalizePath(path))); if (file === null) return;
this.vaultAccess.trigger(event, file);
} }
async triggerHiddenFile(path: string): Promise<void> { async triggerHiddenFile(path: string): Promise<void> {
//@ts-ignore internal function //@ts-ignore internal function
@@ -258,9 +275,9 @@ export class ModuleFileAccessObsidian extends AbstractObsidianModule implements
} }
return files as FilePath[]; return files as FilePath[];
} }
touched(file: UXFileInfoStub | FilePathWithPrefix): void { async touched(file: UXFileInfoStub | FilePathWithPrefix): Promise<void> {
const path = typeof file === "string" ? file : file.path; const path = typeof file === "string" ? file : file.path;
this.vaultAccess.touch(path as FilePath); await this.vaultAccess.touch(path as FilePath);
} }
recentlyTouched(file: UXFileInfoStub | FilePathWithPrefix): boolean { recentlyTouched(file: UXFileInfoStub | FilePathWithPrefix): boolean {
const xFile = typeof file === "string" ? (this.vaultAccess.getAbstractFileByPath(file) as TFile) : file; const xFile = typeof file === "string" ? (this.vaultAccess.getAbstractFileByPath(file) as TFile) : file;

12
src/modules/coreObsidian/storageLib/SerializedFileAccess.ts
View File

@@ -199,9 +199,15 @@ export class SerializedFileAccess {
touchedFiles: string[] = []; touchedFiles: string[] = [];
touch(file: TFile | FilePath) { _statInternal(file: FilePath) {
const f = file instanceof TFile ? file : (this.getAbstractFileByPath(file) as TFile); return this.app.vault.adapter.stat(file);
const key = `${f.path}-${f.stat.mtime}-${f.stat.size}`; }
async touch(file: TFile | FilePath) {
const path = file instanceof TFile ? (file.path as FilePath) : file;
const statOrg = file instanceof TFile ? file.stat : await this._statInternal(path);
const stat = statOrg || { mtime: 0, size: 0 };
const key = `${path}-${stat.mtime}-${stat.size}`;
this.touchedFiles.unshift(key); this.touchedFiles.unshift(key);
this.touchedFiles = this.touchedFiles.slice(0, 100); this.touchedFiles = this.touchedFiles.slice(0, 100);
} }

17
src/modules/essential/ModuleInitializerFile.ts
View File

@@ -81,9 +81,9 @@ export class ModuleInitializerFile extends AbstractModule implements ICoreModule
this._log("Collecting local files on the DB", LOG_LEVEL_VERBOSE); this._log("Collecting local files on the DB", LOG_LEVEL_VERBOSE);
const _DBEntries = [] as MetaEntry[]; const _DBEntries = [] as MetaEntry[];
// const _DBEntriesTask = [] as (() => Promise<MetaEntry | false>)[];
let count = 0; let count = 0;
for await (const doc of this.localDatabase.findAllNormalDocs()) { // Fetch all documents from the database (including conflicts to prevent overwriting).
for await (const doc of this.localDatabase.findAllNormalDocs({ conflicts: true })) {
count++; count++;
if (count % 25 == 0) if (count % 25 == 0)
this._log( this._log(
@@ -180,7 +180,7 @@ export class ModuleInitializerFile extends AbstractModule implements ICoreModule
}; };
initProcess.push( initProcess.push(
runAll("UPDATE DATABASE", filesExistOnlyInStorage, async (e) => { runAll("UPDATE DATABASE", filesExistOnlyInStorage, async (e) => {
// console.warn("UPDATE DATABASE", e); // Exists in storage but not in database.
const file = storageFileNameMap[storageFileNameCI2CS[e]]; const file = storageFileNameMap[storageFileNameCI2CS[e]];
if (!this.core.$$isFileSizeExceeded(file.stat.size)) { if (!this.core.$$isFileSizeExceeded(file.stat.size)) {
const path = file.path; const path = file.path;
@@ -195,9 +195,15 @@ export class ModuleInitializerFile extends AbstractModule implements ICoreModule
initProcess.push( initProcess.push(
runAll("UPDATE STORAGE", filesExistOnlyInDatabase, async (e) => { runAll("UPDATE STORAGE", filesExistOnlyInDatabase, async (e) => {
const w = databaseFileNameMap[databaseFileNameCI2CS[e]]; const w = databaseFileNameMap[databaseFileNameCI2CS[e]];
// Exists in database but not in storage.
const path = getPath(w) ?? e; const path = getPath(w) ?? e;
if (w && !(w.deleted || w._deleted)) { if (w && !(w.deleted || w._deleted)) {
if (!this.core.$$isFileSizeExceeded(w.size)) { if (!this.core.$$isFileSizeExceeded(w.size)) {
// Prevent applying the conflicted state to the storage.
if (w._conflicts?.length ?? 0 > 0) {
this._log(`UPDATE STORAGE: ${path} has conflicts. skipped (x)`, LOG_LEVEL_INFO);
return;
}
// await this.pullFile(path, undefined, false, undefined, false); // await this.pullFile(path, undefined, false, undefined, false);
// Memo: No need to force // Memo: No need to force
await this.core.fileHandler.dbToStorage(path, null, true); await this.core.fileHandler.dbToStorage(path, null, true);
@@ -229,6 +235,11 @@ export class ModuleInitializerFile extends AbstractModule implements ICoreModule
initProcess.push( initProcess.push(
runAll("SYNC DATABASE AND STORAGE", fileMap, async (e) => { runAll("SYNC DATABASE AND STORAGE", fileMap, async (e) => {
const { file, doc } = e; const { file, doc } = e;
// Prevent applying the conflicted state to the storage.
if (doc._conflicts?.length ?? 0 > 0) {
this._log(`SYNC DATABASE AND STORAGE: ${file.path} has conflicts. skipped`, LOG_LEVEL_INFO);
return;
}
if (!this.core.$$isFileSizeExceeded(file.stat.size) && !this.core.$$isFileSizeExceeded(doc.size)) { if (!this.core.$$isFileSizeExceeded(file.stat.size) && !this.core.$$isFileSizeExceeded(doc.size)) {
await this.syncFileBetweenDBandStorage(file, doc); await this.syncFileBetweenDBandStorage(file, doc);
} else { } else {

435
src/modules/essential/ModuleMigration.ts
View File

@@ -1,17 +1,144 @@
import { LOG_LEVEL_INFO, LOG_LEVEL_NOTICE, LOG_LEVEL_VERBOSE } from "octagonal-wheels/common/logger.js"; import { LOG_LEVEL_INFO, LOG_LEVEL_NOTICE, LOG_LEVEL_VERBOSE } from "octagonal-wheels/common/logger.js";
import { SETTING_VERSION_SUPPORT_CASE_INSENSITIVE } from "../../lib/src/common/types.js"; import { type ObsidianLiveSyncSettings } from "../../lib/src/common/types.js";
import { import {
EVENT_REQUEST_OPEN_P2P, EVENT_REQUEST_OPEN_P2P,
EVENT_REQUEST_OPEN_SETTING_WIZARD, EVENT_REQUEST_OPEN_SETTING_WIZARD,
EVENT_REQUEST_OPEN_SETTINGS, EVENT_REQUEST_OPEN_SETTINGS,
EVENT_REQUEST_OPEN_SETUP_URI, EVENT_REQUEST_OPEN_SETUP_URI,
EVENT_REQUEST_RUN_DOCTOR,
eventHub, eventHub,
} from "../../common/events.ts"; } from "../../common/events.ts";
import { AbstractModule } from "../AbstractModule.ts"; import { AbstractModule } from "../AbstractModule.ts";
import type { ICoreModule } from "../ModuleTypes.ts"; import type { ICoreModule } from "../ModuleTypes.ts";
import { $msg } from "src/lib/src/common/i18n.ts"; import { $msg } from "src/lib/src/common/i18n.ts";
import { checkUnsuitableValues, RuleLevel, type RuleForType } from "../../lib/src/common/configForDoc.ts";
import { getConfName, type AllSettingItemKey } from "../features/SettingDialogue/settingConstants.ts";
export class ModuleMigration extends AbstractModule implements ICoreModule { export class ModuleMigration extends AbstractModule implements ICoreModule {
async migrateUsingDoctor(skipRebuild: boolean = false, activateReason = "updated", forceRescan = false) {
const r = checkUnsuitableValues(this.core.settings);
if (!forceRescan && r.version == this.settings.doctorProcessedVersion) {
const isIssueFound = Object.keys(r.rules).length > 0;
const msg = isIssueFound ? "Issues found" : "No issues found";
this._log(`${msg} but marked as to be silent`, LOG_LEVEL_VERBOSE);
return;
}
const issues = Object.entries(r.rules);
if (issues.length == 0) {
this._log(
$msg("Doctor.Message.NoIssues"),
activateReason !== "updated" ? LOG_LEVEL_NOTICE : LOG_LEVEL_INFO
);
return;
} else {
const OPT_YES = `${$msg("Doctor.Button.Yes")}` as const;
const OPT_NO = `${$msg("Doctor.Button.No")}` as const;
const OPT_DISMISS = `${$msg("Doctor.Button.DismissThisVersion")}` as const;
// this._log(`Issues found in ${key}`, LOG_LEVEL_VERBOSE);
const issues = Object.keys(r.rules)
.map((key) => `- ${getConfName(key as AllSettingItemKey)}`)
.join("\n");
const msg = await this.core.confirm.askSelectStringDialogue(
$msg("Doctor.Dialogue.Main", { activateReason, issues }),
[OPT_YES, OPT_NO, OPT_DISMISS],
{
title: $msg("Doctor.Dialogue.Title"),
defaultAction: OPT_YES,
}
);
if (msg == OPT_DISMISS) {
this.settings.doctorProcessedVersion = r.version;
await this.core.saveSettings();
this._log("Marked as to be silent", LOG_LEVEL_VERBOSE);
return;
}
if (msg != OPT_YES) return;
let shouldRebuild = false;
let shouldRebuildLocal = false;
const issueItems = Object.entries(r.rules) as [keyof ObsidianLiveSyncSettings, RuleForType<any>][];
this._log(`${issueItems.length} Issue(s) found `, LOG_LEVEL_VERBOSE);
let idx = 0;
const applySettings = {} as Partial<ObsidianLiveSyncSettings>;
const OPT_FIX = `${$msg("Doctor.Button.Fix")}` as const;
const OPT_SKIP = `${$msg("Doctor.Button.Skip")}` as const;
const OPT_FIXBUTNOREBUILD = `${$msg("Doctor.Button.FixButNoRebuild")}` as const;
let skipped = 0;
for (const [key, value] of issueItems) {
const levelMap = {
[RuleLevel.Necessary]: $msg("Doctor.Level.Necessary"),
[RuleLevel.Recommended]: $msg("Doctor.Level.Recommended"),
[RuleLevel.Optional]: $msg("Doctor.Level.Optional"),
[RuleLevel.Must]: $msg("Doctor.Level.Must"),
};
const level = value.level ? levelMap[value.level] : "Unknown";
const options = [OPT_FIX] as [typeof OPT_FIX | typeof OPT_SKIP | typeof OPT_FIXBUTNOREBUILD];
if ((!skipRebuild && value.requireRebuild) || value.requireRebuildLocal) {
options.push(OPT_FIXBUTNOREBUILD);
}
options.push(OPT_SKIP);
const note = skipRebuild
? ""
: `${value.requireRebuild ? $msg("Doctor.Message.RebuildRequired") : ""}${value.requireRebuildLocal ? $msg("Doctor.Message.RebuildLocalRequired") : ""}`;
const ret = await this.core.confirm.askSelectStringDialogue(
$msg("Doctor.Dialogue.MainFix", {
name: getConfName(key as AllSettingItemKey),
current: `${this.settings[key]}`,
reason: value.reason ?? " N/A ",
ideal: `${value.value}`,
level: `${level}`,
note: note,
}),
options,
{
title: $msg("Doctor.Dialogue.TitleFix", { current: `${++idx}`, total: `${issueItems.length}` }),
defaultAction: OPT_FIX,
}
);
if (ret == OPT_FIX || ret == OPT_FIXBUTNOREBUILD) {
//@ts-ignore
applySettings[key] = value.value;
if (ret == OPT_FIX) {
shouldRebuild = shouldRebuild || value.requireRebuild || false;
shouldRebuildLocal = shouldRebuildLocal || value.requireRebuildLocal || false;
}
} else {
skipped++;
}
}
if (Object.keys(applySettings).length > 0) {
this.settings = {
...this.settings,
...applySettings,
};
}
if (skipped == 0) {
this.settings.doctorProcessedVersion = r.version;
} else {
if (
(await this.core.confirm.askYesNoDialog($msg("Doctor.Message.SomeSkipped"), {
title: $msg("Doctor.Dialogue.TitleAlmostDone"),
defaultOption: "No",
})) == "no"
) {
// Some skipped, and user wants
this.settings.doctorProcessedVersion = r.version;
}
}
await this.core.saveSettings();
if (!skipRebuild) {
if (shouldRebuild) {
await this.core.rebuilder.scheduleRebuild();
await this.core.$$performRestart();
} else if (shouldRebuildLocal) {
await this.core.rebuilder.scheduleFetch();
await this.core.$$performRestart();
}
}
}
}
async migrateDisableBulkSend() { async migrateDisableBulkSend() {
if (this.settings.sendChunksBulk) { if (this.settings.sendChunksBulk) {
this._log($msg("moduleMigration.logBulkSendCorrupted"), LOG_LEVEL_NOTICE); this._log($msg("moduleMigration.logBulkSendCorrupted"), LOG_LEVEL_NOTICE);
@@ -20,157 +147,157 @@ export class ModuleMigration extends AbstractModule implements ICoreModule {
await this.saveSettings(); await this.saveSettings();
} }
} }
async migrationCheck() { // async migrationCheck() {
const old = this.settings.settingVersion; // const old = this.settings.settingVersion;
const current = SETTING_VERSION_SUPPORT_CASE_INSENSITIVE; // const current = SETTING_VERSION_SUPPORT_CASE_INSENSITIVE;
// Check each migrations(old -> current) // // Check each migrations(old -> current)
if (!(await this.migrateToCaseInsensitive(old, current))) { // if (!(await this.migrateToCaseInsensitive(old, current))) {
this._log( // this._log(
$msg("moduleMigration.logMigrationFailed", { // $msg("moduleMigration.logMigrationFailed", {
old: old.toString(), // old: old.toString(),
current: current.toString(), // current: current.toString(),
}), // }),
LOG_LEVEL_NOTICE // LOG_LEVEL_NOTICE
); // );
return; // return;
} // }
} // }
async migrateToCaseInsensitive(old: number, current: number) { // async migrateToCaseInsensitive(old: number, current: number) {
if ( // if (
this.settings.handleFilenameCaseSensitive !== undefined && // this.settings.handleFilenameCaseSensitive !== undefined &&
this.settings.doNotUseFixedRevisionForChunks !== undefined // this.settings.doNotUseFixedRevisionForChunks !== undefined
) { // ) {
if (current < SETTING_VERSION_SUPPORT_CASE_INSENSITIVE) { // if (current < SETTING_VERSION_SUPPORT_CASE_INSENSITIVE) {
this.settings.settingVersion = SETTING_VERSION_SUPPORT_CASE_INSENSITIVE; // this.settings.settingVersion = SETTING_VERSION_SUPPORT_CASE_INSENSITIVE;
await this.saveSettings(); // await this.saveSettings();
} // }
return true; // return true;
} // }
if ( // if (
old >= SETTING_VERSION_SUPPORT_CASE_INSENSITIVE && // old >= SETTING_VERSION_SUPPORT_CASE_INSENSITIVE &&
this.settings.handleFilenameCaseSensitive !== undefined && // this.settings.handleFilenameCaseSensitive !== undefined &&
this.settings.doNotUseFixedRevisionForChunks !== undefined // this.settings.doNotUseFixedRevisionForChunks !== undefined
) { // ) {
return true; // return true;
} // }
let remoteHandleFilenameCaseSensitive: undefined | boolean = undefined; // let remoteHandleFilenameCaseSensitive: undefined | boolean = undefined;
let remoteDoNotUseFixedRevisionForChunks: undefined | boolean = undefined; // let remoteDoNotUseFixedRevisionForChunks: undefined | boolean = undefined;
let remoteChecked = false; // let remoteChecked = false;
try { // try {
const remoteInfo = await this.core.replicator.getRemotePreferredTweakValues(this.settings); // const remoteInfo = await this.core.replicator.getRemotePreferredTweakValues(this.settings);
if (remoteInfo) { // if (remoteInfo) {
remoteHandleFilenameCaseSensitive = // remoteHandleFilenameCaseSensitive =
"handleFilenameCaseSensitive" in remoteInfo ? remoteInfo.handleFilenameCaseSensitive : false; // "handleFilenameCaseSensitive" in remoteInfo ? remoteInfo.handleFilenameCaseSensitive : false;
remoteDoNotUseFixedRevisionForChunks = // remoteDoNotUseFixedRevisionForChunks =
"doNotUseFixedRevisionForChunks" in remoteInfo ? remoteInfo.doNotUseFixedRevisionForChunks : false; // "doNotUseFixedRevisionForChunks" in remoteInfo ? remoteInfo.doNotUseFixedRevisionForChunks : false;
if ( // if (
remoteHandleFilenameCaseSensitive !== undefined || // remoteHandleFilenameCaseSensitive !== undefined ||
remoteDoNotUseFixedRevisionForChunks !== undefined // remoteDoNotUseFixedRevisionForChunks !== undefined
) { // ) {
remoteChecked = true; // remoteChecked = true;
} // }
} else { // } else {
this._log($msg("moduleMigration.logFetchRemoteTweakFailed"), LOG_LEVEL_INFO); // this._log($msg("moduleMigration.logFetchRemoteTweakFailed"), LOG_LEVEL_INFO);
} // }
} catch (ex) { // } catch (ex) {
this._log($msg("moduleMigration.logRemoteTweakUnavailable"), LOG_LEVEL_INFO); // this._log($msg("moduleMigration.logRemoteTweakUnavailable"), LOG_LEVEL_INFO);
this._log(ex, LOG_LEVEL_VERBOSE); // this._log(ex, LOG_LEVEL_VERBOSE);
} // }
if (remoteChecked) { // if (remoteChecked) {
// The case that the remote could be checked. // // The case that the remote could be checked.
if (remoteHandleFilenameCaseSensitive && remoteDoNotUseFixedRevisionForChunks) { // if (remoteHandleFilenameCaseSensitive && remoteDoNotUseFixedRevisionForChunks) {
// Migrated, but configured as same as old behaviour. // // Migrated, but configured as same as old behaviour.
this.settings.handleFilenameCaseSensitive = true; // this.settings.handleFilenameCaseSensitive = true;
this.settings.doNotUseFixedRevisionForChunks = true; // this.settings.doNotUseFixedRevisionForChunks = true;
this.settings.settingVersion = SETTING_VERSION_SUPPORT_CASE_INSENSITIVE; // this.settings.settingVersion = SETTING_VERSION_SUPPORT_CASE_INSENSITIVE;
this._log( // this._log(
$msg("moduleMigration.logMigratedSameBehaviour", { // $msg("moduleMigration.logMigratedSameBehaviour", {
current: current.toString(), // current: current.toString(),
}), // }),
LOG_LEVEL_INFO // LOG_LEVEL_INFO
); // );
await this.saveSettings(); // await this.saveSettings();
return true; // return true;
} // }
const message = $msg("moduleMigration.msgFetchRemoteAgain"); // const message = $msg("moduleMigration.msgFetchRemoteAgain");
const OPTION_FETCH = $msg("moduleMigration.optionYesFetchAgain"); // const OPTION_FETCH = $msg("moduleMigration.optionYesFetchAgain");
const DISMISS = $msg("moduleMigration.optionNoAskAgain"); // const DISMISS = $msg("moduleMigration.optionNoAskAgain");
const options = [OPTION_FETCH, DISMISS]; // const options = [OPTION_FETCH, DISMISS];
const ret = await this.core.confirm.confirmWithMessage( // const ret = await this.core.confirm.confirmWithMessage(
$msg("moduleMigration.titleCaseSensitivity"), // $msg("moduleMigration.titleCaseSensitivity"),
message, // message,
options, // options,
DISMISS, // DISMISS,
40 // 40
); // );
if (ret == OPTION_FETCH) { // if (ret == OPTION_FETCH) {
this.settings.handleFilenameCaseSensitive = remoteHandleFilenameCaseSensitive || false; // this.settings.handleFilenameCaseSensitive = remoteHandleFilenameCaseSensitive || false;
this.settings.doNotUseFixedRevisionForChunks = remoteDoNotUseFixedRevisionForChunks || false; // this.settings.doNotUseFixedRevisionForChunks = remoteDoNotUseFixedRevisionForChunks || false;
this.settings.settingVersion = SETTING_VERSION_SUPPORT_CASE_INSENSITIVE; // this.settings.settingVersion = SETTING_VERSION_SUPPORT_CASE_INSENSITIVE;
await this.saveSettings(); // await this.saveSettings();
try { // try {
await this.core.rebuilder.scheduleFetch(); // await this.core.rebuilder.scheduleFetch();
return; // return;
} catch (ex) { // } catch (ex) {
this._log($msg("moduleMigration.logRedflag2CreationFail"), LOG_LEVEL_VERBOSE); // this._log($msg("moduleMigration.logRedflag2CreationFail"), LOG_LEVEL_VERBOSE);
this._log(ex, LOG_LEVEL_VERBOSE); // this._log(ex, LOG_LEVEL_VERBOSE);
} // }
return false; // return false;
} else { // } else {
return false; // return false;
} // }
} // }
const ENABLE_BOTH = $msg("moduleMigration.optionEnableBoth"); // const ENABLE_BOTH = $msg("moduleMigration.optionEnableBoth");
const ENABLE_FILENAME_CASE_INSENSITIVE = $msg("moduleMigration.optionEnableFilenameCaseInsensitive"); // const ENABLE_FILENAME_CASE_INSENSITIVE = $msg("moduleMigration.optionEnableFilenameCaseInsensitive");
const ENABLE_FIXED_REVISION_FOR_CHUNKS = $msg("moduleMigration.optionEnableFixedRevisionForChunks"); // const ENABLE_FIXED_REVISION_FOR_CHUNKS = $msg("moduleMigration.optionEnableFixedRevisionForChunks");
const ADJUST_TO_REMOTE = $msg("moduleMigration.optionAdjustRemote"); // const ADJUST_TO_REMOTE = $msg("moduleMigration.optionAdjustRemote");
const KEEP = $msg("moduleMigration.optionKeepPreviousBehaviour"); // const KEEP = $msg("moduleMigration.optionKeepPreviousBehaviour");
const DISMISS = $msg("moduleMigration.optionDecideLater"); // const DISMISS = $msg("moduleMigration.optionDecideLater");
const message = $msg("moduleMigration.msgSinceV02321"); // const message = $msg("moduleMigration.msgSinceV02321");
const options = [ENABLE_BOTH, ENABLE_FILENAME_CASE_INSENSITIVE, ENABLE_FIXED_REVISION_FOR_CHUNKS]; // const options = [ENABLE_BOTH, ENABLE_FILENAME_CASE_INSENSITIVE, ENABLE_FIXED_REVISION_FOR_CHUNKS];
if (remoteChecked) { // if (remoteChecked) {
options.push(ADJUST_TO_REMOTE); // options.push(ADJUST_TO_REMOTE);
} // }
options.push(KEEP, DISMISS); // options.push(KEEP, DISMISS);
const ret = await this.core.confirm.confirmWithMessage( // const ret = await this.core.confirm.confirmWithMessage(
$msg("moduleMigration.titleCaseSensitivity"), // $msg("moduleMigration.titleCaseSensitivity"),
message, // message,
options, // options,
DISMISS, // DISMISS,
40 // 40
); // );
console.dir(ret); // console.dir(ret);
switch (ret) { // switch (ret) {
case ENABLE_BOTH: // case ENABLE_BOTH:
this.settings.handleFilenameCaseSensitive = false; // this.settings.handleFilenameCaseSensitive = false;
this.settings.doNotUseFixedRevisionForChunks = false; // this.settings.doNotUseFixedRevisionForChunks = false;
break; // break;
case ENABLE_FILENAME_CASE_INSENSITIVE: // case ENABLE_FILENAME_CASE_INSENSITIVE:
this.settings.handleFilenameCaseSensitive = false; // this.settings.handleFilenameCaseSensitive = false;
this.settings.doNotUseFixedRevisionForChunks = true; // this.settings.doNotUseFixedRevisionForChunks = true;
break; // break;
case ENABLE_FIXED_REVISION_FOR_CHUNKS: // case ENABLE_FIXED_REVISION_FOR_CHUNKS:
this.settings.doNotUseFixedRevisionForChunks = false; // this.settings.doNotUseFixedRevisionForChunks = false;
this.settings.handleFilenameCaseSensitive = true; // this.settings.handleFilenameCaseSensitive = true;
break; // break;
case KEEP: // case KEEP:
this.settings.handleFilenameCaseSensitive = true; // this.settings.handleFilenameCaseSensitive = true;
this.settings.doNotUseFixedRevisionForChunks = true; // this.settings.doNotUseFixedRevisionForChunks = true;
this.settings.settingVersion = SETTING_VERSION_SUPPORT_CASE_INSENSITIVE; // this.settings.settingVersion = SETTING_VERSION_SUPPORT_CASE_INSENSITIVE;
await this.saveSettings(); // await this.saveSettings();
return true; // return true;
case DISMISS: // case DISMISS:
default: // default:
return false; // return false;
} // }
this.settings.settingVersion = SETTING_VERSION_SUPPORT_CASE_INSENSITIVE; // this.settings.settingVersion = SETTING_VERSION_SUPPORT_CASE_INSENSITIVE;
await this.saveSettings(); // await this.saveSettings();
await this.core.rebuilder.scheduleRebuild(); // await this.core.rebuilder.scheduleRebuild();
await this.core.$$performRestart(); // await this.core.$$performRestart();
} // }
async initialMessage() { async initialMessage() {
const message = $msg("moduleMigration.msgInitialSetup", { const message = $msg("moduleMigration.msgInitialSetup", {
@@ -226,7 +353,8 @@ export class ModuleMigration extends AbstractModule implements ICoreModule {
return false; return false;
} }
if (this.settings.isConfigured) { if (this.settings.isConfigured) {
await this.migrationCheck(); await this.migrateUsingDoctor(false);
// await this.migrationCheck();
await this.migrateDisableBulkSend(); await this.migrateDisableBulkSend();
} }
if (!this.settings.isConfigured) { if (!this.settings.isConfigured) {
@@ -235,7 +363,14 @@ export class ModuleMigration extends AbstractModule implements ICoreModule {
this._log($msg("moduleMigration.logSetupCancelled"), LOG_LEVEL_NOTICE); this._log($msg("moduleMigration.logSetupCancelled"), LOG_LEVEL_NOTICE);
return false; return false;
} }
await this.migrateUsingDoctor(true);
} }
return true; return true;
} }
$everyOnLayoutReady(): Promise<boolean> {
eventHub.onEvent(EVENT_REQUEST_RUN_DOCTOR, async (reason) => {
await this.migrateUsingDoctor(false, reason, true);
});
return Promise.resolve(true);
}
} }

224
src/modules/essentialObsidian/ModuleObsidianAPI.ts
View File

@@ -1,7 +1,7 @@
import { AbstractObsidianModule, type IObsidianModule } from "../AbstractObsidianModule.ts"; import { AbstractObsidianModule, type IObsidianModule } from "../AbstractObsidianModule.ts";
import { LOG_LEVEL_DEBUG, LOG_LEVEL_VERBOSE } from "octagonal-wheels/common/logger"; import { LOG_LEVEL_DEBUG, LOG_LEVEL_NOTICE, LOG_LEVEL_VERBOSE } from "octagonal-wheels/common/logger";
import { Notice, requestUrl, type RequestUrlParam, type RequestUrlResponse } from "../../deps.ts"; import { Notice, requestUrl, type RequestUrlParam, type RequestUrlResponse } from "../../deps.ts";
import { type EntryDoc, type FilePathWithPrefix } from "../../lib/src/common/types.ts"; import { type CouchDBCredentials, type EntryDoc, type FilePathWithPrefix } from "../../lib/src/common/types.ts";
import { getPathFromTFile } from "../../common/utils.ts"; import { getPathFromTFile } from "../../common/utils.ts";
import { import {
disableEncryption, disableEncryption,
@@ -13,7 +13,7 @@ import {
import { setNoticeClass } from "../../lib/src/mock_and_interop/wrapper.ts"; import { setNoticeClass } from "../../lib/src/mock_and_interop/wrapper.ts";
import { ObsHttpHandler } from "./APILib/ObsHttpHandler.ts"; import { ObsHttpHandler } from "./APILib/ObsHttpHandler.ts";
import { PouchDB } from "../../lib/src/pouchdb/pouchdb-browser.ts"; import { PouchDB } from "../../lib/src/pouchdb/pouchdb-browser.ts";
import { reactive, reactiveSource } from "octagonal-wheels/dataobject/reactive"; import { AuthorizationHeaderGenerator } from "../../lib/src/replication/httplib.ts";
setNoticeClass(Notice); setNoticeClass(Notice);
@@ -33,10 +33,8 @@ async function fetchByAPI(request: RequestUrlParam): Promise<RequestUrlResponse>
export class ModuleObsidianAPI extends AbstractObsidianModule implements IObsidianModule { export class ModuleObsidianAPI extends AbstractObsidianModule implements IObsidianModule {
_customHandler!: ObsHttpHandler; _customHandler!: ObsHttpHandler;
authHeaderSource = reactiveSource<string>("");
authHeader = reactive(() => _authHeader = new AuthorizationHeaderGenerator();
this.authHeaderSource.value == "" ? "" : "Basic " + window.btoa(this.authHeaderSource.value)
);
last_successful_post = false; last_successful_post = false;
$$customFetchHandler(): ObsHttpHandler { $$customFetchHandler(): ObsHttpHandler {
@@ -47,31 +45,80 @@ export class ModuleObsidianAPI extends AbstractObsidianModule implements IObsidi
return !this.last_successful_post; return !this.last_successful_post;
} }
async fetchByAPI(
url: string,
localURL: string,
method: string,
authHeader: string,
opts?: RequestInit
): Promise<Response> {
const body = opts?.body as string;
const transformedHeaders = { ...(opts?.headers as Record<string, string>) };
if (authHeader != "") transformedHeaders["authorization"] = authHeader;
delete transformedHeaders["host"];
delete transformedHeaders["Host"];
delete transformedHeaders["content-length"];
delete transformedHeaders["Content-Length"];
const requestParam: RequestUrlParam = {
url,
method: opts?.method,
body: body,
headers: transformedHeaders,
contentType: "application/json",
// contentType: opts.headers,
};
const size = body ? ` (${body.length})` : "";
try {
this.plugin.requestCount.value = this.plugin.requestCount.value + 1;
const r = await fetchByAPI(requestParam);
if (method == "POST" || method == "PUT") {
this.last_successful_post = r.status - (r.status % 100) == 200;
} else {
this.last_successful_post = true;
}
this._log(`HTTP:${method}${size} to:${localURL} -> ${r.status}`, LOG_LEVEL_DEBUG);
return new Response(r.arrayBuffer, {
headers: r.headers,
status: r.status,
statusText: `${r.status}`,
});
} catch (ex) {
this._log(`HTTP:${method}${size} to:${localURL} -> failed`, LOG_LEVEL_VERBOSE);
// limit only in bulk_docs.
if (url.toString().indexOf("_bulk_docs") !== -1) {
this.last_successful_post = false;
}
this._log(ex);
throw ex;
} finally {
this.plugin.responseCount.value = this.plugin.responseCount.value + 1;
}
}
async $$connectRemoteCouchDB( async $$connectRemoteCouchDB(
uri: string, uri: string,
auth: { username: string; password: string }, auth: CouchDBCredentials,
disableRequestURI: boolean, disableRequestURI: boolean,
passphrase: string | false, passphrase: string | false,
useDynamicIterationCount: boolean, useDynamicIterationCount: boolean,
performSetup: boolean, performSetup: boolean,
skipInfo: boolean, skipInfo: boolean,
compression: boolean compression: boolean,
customHeaders: Record<string, string>
): Promise<string | { db: PouchDB.Database<EntryDoc>; info: PouchDB.Core.DatabaseInfo }> { ): Promise<string | { db: PouchDB.Database<EntryDoc>; info: PouchDB.Core.DatabaseInfo }> {
if (!isValidRemoteCouchDBURI(uri)) return "Remote URI is not valid"; if (!isValidRemoteCouchDBURI(uri)) return "Remote URI is not valid";
if (uri.toLowerCase() != uri) return "Remote URI and database name could not contain capital letters."; if (uri.toLowerCase() != uri) return "Remote URI and database name could not contain capital letters.";
if (uri.indexOf(" ") !== -1) return "Remote URI and database name could not contain spaces."; if (uri.indexOf(" ") !== -1) return "Remote URI and database name could not contain spaces.";
const userNameAndPassword = auth.username && auth.password ? `${auth.username}:${auth.password}` : ""; // let authHeader = await this._authHeader.getAuthorizationHeader(auth);
if (this.authHeaderSource.value != userNameAndPassword) {
this.authHeaderSource.value = userNameAndPassword;
}
const authHeader = this.authHeader.value;
// const _this = this;
const conf: PouchDB.HttpAdapter.HttpAdapterConfiguration = { const conf: PouchDB.HttpAdapter.HttpAdapterConfiguration = {
adapter: "http", adapter: "http",
auth, auth: "username" in auth ? auth : undefined,
skip_setup: !performSetup, skip_setup: !performSetup,
fetch: async (url: string | Request, opts?: RequestInit) => { fetch: async (url: string | Request, opts?: RequestInit) => {
const authHeader = await this._authHeader.getAuthorizationHeader(auth);
let size = ""; let size = "";
const localURL = url.toString().substring(uri.length); const localURL = url.toString().substring(uri.length);
const method = opts?.method ?? "GET"; const method = opts?.method ?? "GET";
@@ -87,88 +134,86 @@ export class ModuleObsidianAPI extends AbstractObsidianModule implements IObsidi
} }
size = ` (${opts_length})`; size = ` (${opts_length})`;
} }
if (!disableRequestURI && typeof url == "string" && typeof (opts?.body ?? "") == "string") { try {
const body = opts?.body as string; const headers = new Headers(opts?.headers);
if (customHeaders) {
for (const [key, value] of Object.entries(customHeaders)) {
if (key && value) {
headers.append(key, value);
}
}
}
if (!("username" in auth)) {
headers.append("authorization", authHeader);
}
const transformedHeaders = { ...(opts?.headers as Record<string, string>) }; if (!disableRequestURI && typeof url == "string" && typeof (opts?.body ?? "") == "string") {
if (authHeader != "") transformedHeaders["authorization"] = authHeader; // Deprecated configuration, only for backward compatibility.
delete transformedHeaders["host"]; return await this.fetchByAPI(url, localURL, method, authHeader, { ...opts, headers });
delete transformedHeaders["Host"]; }
delete transformedHeaders["content-length"]; // --> native Fetch API.
delete transformedHeaders["Content-Length"];
const requestParam: RequestUrlParam = {
url,
method: opts?.method,
body: body,
headers: transformedHeaders,
contentType: "application/json",
// contentType: opts.headers,
};
try { try {
this.plugin.requestCount.value = this.plugin.requestCount.value + 1; this.plugin.requestCount.value = this.plugin.requestCount.value + 1;
const r = await fetchByAPI(requestParam); const response: Response = await fetch(url, { ...opts, headers });
if (method == "POST" || method == "PUT") { if (method == "POST" || method == "PUT") {
this.last_successful_post = r.status - (r.status % 100) == 200; this.last_successful_post = response.ok;
} else { } else {
this.last_successful_post = true; this.last_successful_post = true;
} }
this._log(`HTTP:${method}${size} to:${localURL} -> ${r.status}`, LOG_LEVEL_DEBUG); this._log(`HTTP:${method}${size} to:${localURL} -> ${response.status}`, LOG_LEVEL_DEBUG);
if (Math.floor(response.status / 100) !== 2) {
return new Response(r.arrayBuffer, { if (response.status == 404) {
headers: r.headers, if (method === "GET" && localURL.indexOf("/_local/") === -1) {
status: r.status, this._log(
statusText: `${r.status}`, `Just checkpoint or some server information has been missing. The 404 error shown above is not an error.`,
}); LOG_LEVEL_VERBOSE
} catch (ex) { );
this._log(`HTTP:${method}${size} to:${localURL} -> failed`, LOG_LEVEL_VERBOSE); }
// limit only in bulk_docs. } else {
if (url.toString().indexOf("_bulk_docs") !== -1) { const r = response.clone();
this.last_successful_post = false; this._log(
} `The request may have failed. The reason sent by the server: ${r.status}: ${r.statusText}`,
this._log(ex); LOG_LEVEL_NOTICE
throw ex; );
} finally { try {
this.plugin.responseCount.value = this.plugin.responseCount.value + 1; const result = await r.text();
} this._log(result, LOG_LEVEL_VERBOSE);
} } catch (_) {
this._log("Cloud not fetch response body", LOG_LEVEL_VERBOSE);
try { this._log(_, LOG_LEVEL_VERBOSE);
if (this.settings.enableDebugTools) { }
// Issue #407
(opts!.headers as Headers).append("ngrok-skip-browser-warning", "123");
}
this.plugin.requestCount.value = this.plugin.requestCount.value + 1;
const response: Response = await fetch(url, opts);
if (method == "POST" || method == "PUT") {
this.last_successful_post = response.ok;
} else {
this.last_successful_post = true;
}
this._log(`HTTP:${method}${size} to:${localURL} -> ${response.status}`, LOG_LEVEL_DEBUG);
if (Math.floor(response.status / 100) !== 2) {
if (method != "GET" && localURL.indexOf("/_local/") === -1 && !localURL.endsWith("/")) {
const r = response.clone();
this._log(
`The request may have failed. The reason sent by the server: ${r.status}: ${r.statusText}`
);
try {
this._log(await (await r.blob()).text(), LOG_LEVEL_VERBOSE);
} catch (_) {
this._log("Cloud not parse response", LOG_LEVEL_VERBOSE);
this._log(_, LOG_LEVEL_VERBOSE);
} }
} else {
this._log(
`Just checkpoint or some server information has been missing. The 404 error shown above is not an error.`,
LOG_LEVEL_VERBOSE
);
} }
return response;
} catch (ex) {
if (ex instanceof TypeError) {
this._log(
"Failed to fetch by native fetch API. Trying to fetch by API to get more information."
);
const resp2 = await this.fetchByAPI(url.toString(), localURL, method, authHeader, {
...opts,
headers,
});
if (resp2.status / 100 == 2) {
this._log(
"The request was successful by API. But the native fetch API failed! Please check CORS settings on the remote database!. While this condition, you cannot enable LiveSync",
LOG_LEVEL_NOTICE
);
return resp2;
}
const r2 = resp2.clone();
const msg = await r2.text();
this._log(`Failed to fetch by API. ${resp2.status}: ${msg}`, LOG_LEVEL_NOTICE);
return resp2;
}
throw ex;
} }
return response; } catch (ex: any) {
} catch (ex) {
this._log(`HTTP:${method}${size} to:${localURL} -> failed`, LOG_LEVEL_VERBOSE); this._log(`HTTP:${method}${size} to:${localURL} -> failed`, LOG_LEVEL_VERBOSE);
const msg = ex instanceof Error ? `${ex?.name}:${ex?.message}` : ex?.toString();
this._log(`Failed to fetch: ${msg}`, LOG_LEVEL_NOTICE);
this._log(ex, LOG_LEVEL_VERBOSE);
// limit only in bulk_docs. // limit only in bulk_docs.
if (url.toString().indexOf("_bulk_docs") !== -1) { if (url.toString().indexOf("_bulk_docs") !== -1) {
this.last_successful_post = false; this.last_successful_post = false;
@@ -178,6 +223,7 @@ export class ModuleObsidianAPI extends AbstractObsidianModule implements IObsidi
} finally { } finally {
this.plugin.responseCount.value = this.plugin.responseCount.value + 1; this.plugin.responseCount.value = this.plugin.responseCount.value + 1;
} }
// return await fetch(url, opts); // return await fetch(url, opts);
}, },
}; };
@@ -195,11 +241,7 @@ export class ModuleObsidianAPI extends AbstractObsidianModule implements IObsidi
const info = await db.info(); const info = await db.info();
return { db: db, info: info }; return { db: db, info: info };
} catch (ex: any) { } catch (ex: any) {
let msg = `${ex?.name}:${ex?.message}`; const msg = `${ex?.name}:${ex?.message}`;
if (ex?.name == "TypeError" && ex?.message == "Failed to fetch") {
msg +=
"\n**Note** This error caused by many reasons. The only sure thing is you didn't touch the server.\nTo check details, open inspector.";
}
this._log(ex, LOG_LEVEL_VERBOSE); this._log(ex, LOG_LEVEL_VERBOSE);
return msg; return msg;
} }

4
src/modules/essentialObsidian/ModuleObsidianEvents.ts
View File

@@ -56,7 +56,7 @@ export class ModuleObsidianEvents extends AbstractObsidianModule implements IObs
} else { } else {
if (this.settings.syncOnEditorSave) { if (this.settings.syncOnEditorSave) {
this._log("Sync on Editor Save.", LOG_LEVEL_VERBOSE); this._log("Sync on Editor Save.", LOG_LEVEL_VERBOSE);
fireAndForget(() => this.core.$$replicate()); fireAndForget(() => this.core.$$replicateByEvent());
} }
} }
}); });
@@ -155,7 +155,7 @@ export class ModuleObsidianEvents extends AbstractObsidianModule implements IObs
return; return;
} }
if (this.settings.syncOnFileOpen && !this.core.$$isSuspended()) { if (this.settings.syncOnFileOpen && !this.core.$$isSuspended()) {
await this.core.$$replicate(); await this.core.$$replicateByEvent();
} }
await this.core.$$queueConflictCheckIfOpen(file.path as FilePathWithPrefix); await this.core.$$queueConflictCheckIfOpen(file.path as FilePathWithPrefix);
} }

1
src/modules/essentialObsidian/ModuleObsidianMenu.ts
View File

@@ -106,7 +106,6 @@ export class ModuleObsidianMenu extends AbstractObsidianModule implements IObsid
} }
$everyOnload(): Promise<boolean> { $everyOnload(): Promise<boolean> {
this.app.workspace.onLayoutReady(this.core.$$onLiveSyncReady.bind(this.core)); this.app.workspace.onLayoutReady(this.core.$$onLiveSyncReady.bind(this.core));
// eslint-disable-next-line no-unused-labels
return Promise.resolve(true); return Promise.resolve(true);
} }

42
src/modules/extras/ModuleDev.ts
View File

@@ -1,4 +1,4 @@
import { fireAndForget } from "octagonal-wheels/promises"; import { delay, fireAndForget } from "octagonal-wheels/promises";
import { __onMissingTranslation } from "../../lib/src/common/i18n"; import { __onMissingTranslation } from "../../lib/src/common/i18n";
import { AbstractObsidianModule, type IObsidianModule } from "../AbstractObsidianModule.ts"; import { AbstractObsidianModule, type IObsidianModule } from "../AbstractObsidianModule.ts";
import { LOG_LEVEL_VERBOSE } from "octagonal-wheels/common/logger"; import { LOG_LEVEL_VERBOSE } from "octagonal-wheels/common/logger";
@@ -6,6 +6,7 @@ import { eventHub } from "../../common/events";
import { enableTestFunction } from "./devUtil/testUtils.ts"; import { enableTestFunction } from "./devUtil/testUtils.ts";
import { TestPaneView, VIEW_TYPE_TEST } from "./devUtil/TestPaneView.ts"; import { TestPaneView, VIEW_TYPE_TEST } from "./devUtil/TestPaneView.ts";
import { writable } from "svelte/store"; import { writable } from "svelte/store";
import type { FilePathWithPrefix } from "../../lib/src/common/types.ts";
export class ModuleDev extends AbstractObsidianModule implements IObsidianModule { export class ModuleDev extends AbstractObsidianModule implements IObsidianModule {
$everyOnloadStart(): Promise<boolean> { $everyOnloadStart(): Promise<boolean> {
@@ -36,7 +37,6 @@ export class ModuleDev extends AbstractObsidianModule implements IObsidianModule
$everyOnloadAfterLoadSettings(): Promise<boolean> { $everyOnloadAfterLoadSettings(): Promise<boolean> {
if (!this.settings.enableDebugTools) return Promise.resolve(true); if (!this.settings.enableDebugTools) return Promise.resolve(true);
// eslint-disable-next-line no-unused-labels
this.onMissingTranslation = this.onMissingTranslation.bind(this); this.onMissingTranslation = this.onMissingTranslation.bind(this);
__onMissingTranslation((key) => { __onMissingTranslation((key) => {
void this.onMissingTranslation(key); void this.onMissingTranslation(key);
@@ -99,9 +99,41 @@ export class ModuleDev extends AbstractObsidianModule implements IObsidianModule
} }
async $everyOnLayoutReady(): Promise<boolean> { async $everyOnLayoutReady(): Promise<boolean> {
if (!this.settings.enableDebugTools) return Promise.resolve(true); if (!this.settings.enableDebugTools) return Promise.resolve(true);
if (await this.core.storageAccess.isExistsIncludeHidden("_SHOWDIALOGAUTO.md")) { // if (await this.core.storageAccess.isExistsIncludeHidden("_SHOWDIALOGAUTO.md")) {
void this.core.$$showView(VIEW_TYPE_TEST); // void this.core.$$showView(VIEW_TYPE_TEST);
} // }
this.addCommand({
id: "test-create-conflict",
name: "Create conflict",
callback: async () => {
const filename = "test-create-conflict.md";
const content = `# Test create conflict\n\n`;
const w = await this.core.databaseFileAccess.store({
name: filename as FilePathWithPrefix,
path: filename as FilePathWithPrefix,
body: new Blob([content], { type: "text/markdown" }),
stat: {
ctime: new Date().getTime(),
mtime: new Date().getTime(),
size: content.length,
type: "file",
},
});
if (w) {
const id = await this.core.$$path2id(filename as FilePathWithPrefix);
const f = await this.core.localDatabase.getRaw(id);
console.log(f);
console.log(f._rev);
const revConflict = f._rev.split("-")[0] + "-" + (parseInt(f._rev.split("-")[1]) + 1).toString();
console.log(await this.core.localDatabase.bulkDocsRaw([f], { new_edits: false }));
console.log(
await this.core.localDatabase.bulkDocsRaw([{ ...f, _rev: revConflict }], { new_edits: false })
);
}
},
});
await delay(1);
return true; return true;
} }
testResults = writable<[boolean, string, string][]>([]); testResults = writable<[boolean, string, string][]>([]);

4
src/modules/extras/ModuleReplicateTest.ts
View File

@@ -160,6 +160,10 @@ export class ModuleReplicateTest extends AbstractObsidianModule implements IObsi
} }
async _dumpFileList(outFile?: string) { async _dumpFileList(outFile?: string) {
if (!this.core || !this.core.storageAccess) {
this._log("No storage access", LOG_LEVEL_INFO);
return;
}
const files = this.core.storageAccess.getFiles(); const files = this.core.storageAccess.getFiles();
const out = [] as any[]; const out = [] as any[];
const webcrypto = await getWebCrypto(); const webcrypto = await getWebCrypto();

2
src/modules/extras/devUtil/TestPaneView.ts
View File

@@ -30,7 +30,6 @@ export class TestPaneView extends ItemView {
return "Self-hosted LiveSync Test and Results"; return "Self-hosted LiveSync Test and Results";
} }
// eslint-disable-next-line require-await
async onOpen() { async onOpen() {
this.component = new TestPaneComponent({ this.component = new TestPaneComponent({
target: this.contentEl, target: this.contentEl,
@@ -42,7 +41,6 @@ export class TestPaneView extends ItemView {
await Promise.resolve(); await Promise.resolve();
} }
// eslint-disable-next-line require-await
async onClose() { async onClose() {
this.component?.$destroy(); this.component?.$destroy();
await Promise.resolve(); await Promise.resolve();

2
src/modules/features/ModuleInteractiveConflictResolver.ts
View File

@@ -90,7 +90,7 @@ export class ModuleInteractiveConflictResolver extends AbstractObsidianModule im
// So we have to run replication if configured. // So we have to run replication if configured.
// TODO: Make this is as a event request // TODO: Make this is as a event request
if (this.settings.syncAfterMerge && !this.core.$$isSuspended()) { if (this.settings.syncAfterMerge && !this.core.$$isSuspended()) {
await this.core.$$waitForReplicationOnce(); await this.core.$$replicateByEvent();
} }
// And, check it again. // And, check it again.
await this.core.$$queueConflictCheck(filename); await this.core.$$queueConflictCheck(filename);

94
src/modules/features/ModuleLog.ts
View File

@@ -27,14 +27,7 @@ import { QueueProcessor } from "octagonal-wheels/concurrency/processor";
import { LogPaneView, VIEW_TYPE_LOG } from "./Log/LogPaneView.ts"; import { LogPaneView, VIEW_TYPE_LOG } from "./Log/LogPaneView.ts";
import { serialized } from "octagonal-wheels/concurrency/lock"; import { serialized } from "octagonal-wheels/concurrency/lock";
import { $msg } from "src/lib/src/common/i18n.ts"; import { $msg } from "src/lib/src/common/i18n.ts";
import type { P2PReplicationProgress } from "../../lib/src/replication/trystero/TrysteroReplicator.ts"; import { P2PLogCollector } from "../../lib/src/replication/trystero/P2PReplicatorCore.ts";
import {
EVENT_ADVERTISEMENT_RECEIVED,
EVENT_DEVICE_LEAVED,
EVENT_P2P_CONNECTED,
EVENT_P2P_DISCONNECTED,
EVENT_P2P_REPLICATOR_PROGRESS,
} from "src/lib/src/replication/trystero/TrysteroReplicatorP2PServer.ts";
// This module cannot be a core module because it depends on the Obsidian UI. // This module cannot be a core module because it depends on the Obsidian UI.
@@ -71,6 +64,7 @@ export class ModuleLog extends AbstractObsidianModule implements IObsidianModule
statusBarLabels!: ReactiveValue<{ message: string; status: string }>; statusBarLabels!: ReactiveValue<{ message: string; status: string }>;
statusLog = reactiveSource(""); statusLog = reactiveSource("");
notifies: { [key: string]: { notice: Notice; count: number } } = {}; notifies: { [key: string]: { notice: Notice; count: number } } = {};
p2pLogCollector = new P2PLogCollector();
observeForLogs() { observeForLogs() {
const padSpaces = `\u{2007}`.repeat(10); const padSpaces = `\u{2007}`.repeat(10);
@@ -176,7 +170,7 @@ export class ModuleLog extends AbstractObsidianModule implements IObsidianModule
const queued = queueCountLabel(); const queued = queueCountLabel();
const waiting = waitingLabel(); const waiting = waitingLabel();
const networkActivity = requestingStatLabel(); const networkActivity = requestingStatLabel();
const p2p = this.p2pReplicationLine.value; const p2p = this.p2pLogCollector.p2pReplicationLine.value;
return { return {
message: `${networkActivity}Sync: ${w} ↑ ${sent}${pushLast} ↓ ${arrived}${pullLast}${waiting}${queued}${p2p == "" ? "" : "\n" + p2p}`, message: `${networkActivity}Sync: ${w} ↑ ${sent}${pushLast} ↓ ${arrived}${pullLast}${waiting}${queued}${p2p == "" ? "" : "\n" + p2p}`,
}; };
@@ -203,90 +197,10 @@ export class ModuleLog extends AbstractObsidianModule implements IObsidianModule
statusBarLabels.onChanged((label) => applyToDisplay(label.value)); statusBarLabels.onChanged((label) => applyToDisplay(label.value));
} }
p2pReplicationResult = new Map<string, P2PReplicationProgress>();
updateP2PReplicationLine() {
const p2pReplicationResultX = [...this.p2pReplicationResult.values()].sort((a, b) =>
a.peerId.localeCompare(b.peerId)
);
const renderProgress = (current: number, max: number) => {
if (current == max) return `${current}`;
return `${current} (${max})`;
};
const line = p2pReplicationResultX
.map(
(e) =>
`${e.fetching.isActive || e.sending.isActive ? "⚡" : "💤"} ${e.peerName} ↑ ${renderProgress(e.sending.current, e.sending.max)} ↓ ${renderProgress(e.fetching.current, e.fetching.max)} `
)
.join("\n");
this.p2pReplicationLine.value = line;
}
// p2pReplicationResultX = reactiveSource([] as P2PReplicationProgress[]);
p2pReplicationLine = reactiveSource("");
$everyOnload(): Promise<boolean> { $everyOnload(): Promise<boolean> {
eventHub.onEvent(EVENT_LEAF_ACTIVE_CHANGED, () => this.onActiveLeafChange()); eventHub.onEvent(EVENT_LEAF_ACTIVE_CHANGED, () => this.onActiveLeafChange());
eventHub.onceEvent(EVENT_LAYOUT_READY, () => this.onActiveLeafChange()); eventHub.onceEvent(EVENT_LAYOUT_READY, () => this.onActiveLeafChange());
eventHub.onEvent(EVENT_ADVERTISEMENT_RECEIVED, (data) => {
this.p2pReplicationResult.set(data.peerId, {
peerId: data.peerId,
peerName: data.name,
fetching: {
current: 0,
max: 0,
isActive: false,
},
sending: {
current: 0,
max: 0,
isActive: false,
},
});
this.updateP2PReplicationLine();
});
eventHub.onEvent(EVENT_P2P_CONNECTED, () => {
this.p2pReplicationResult.clear();
this.updateP2PReplicationLine();
});
eventHub.onEvent(EVENT_P2P_DISCONNECTED, () => {
this.p2pReplicationResult.clear();
this.updateP2PReplicationLine();
});
eventHub.onEvent(EVENT_DEVICE_LEAVED, (peerId) => {
this.p2pReplicationResult.delete(peerId);
this.updateP2PReplicationLine();
});
eventHub.onEvent(EVENT_P2P_REPLICATOR_PROGRESS, (data) => {
const prev = this.p2pReplicationResult.get(data.peerId) || {
peerId: data.peerId,
peerName: data.peerName,
fetching: {
current: 0,
max: 0,
isActive: false,
},
sending: {
current: 0,
max: 0,
isActive: false,
},
};
if ("fetching" in data) {
if (data.fetching.isActive) {
prev.fetching = data.fetching;
} else {
prev.fetching.isActive = false;
}
}
if ("sending" in data) {
if (data.sending.isActive) {
prev.sending = data.sending;
} else {
prev.sending.isActive = false;
}
}
this.p2pReplicationResult.set(data.peerId, prev);
this.updateP2PReplicationLine();
});
return Promise.resolve(true); return Promise.resolve(true);
} }
adjustStatusDivPosition() { adjustStatusDivPosition() {

8
src/modules/features/ModuleObsidianSetting.ts
View File

@@ -94,6 +94,14 @@ export class ModuleObsidianSettings extends AbstractObsidianModule implements IO
region: settings.region, region: settings.region,
secretKey: settings.secretKey, secretKey: settings.secretKey,
useCustomRequestHandler: settings.useCustomRequestHandler, useCustomRequestHandler: settings.useCustomRequestHandler,
bucketCustomHeaders: settings.bucketCustomHeaders,
couchDB_CustomHeaders: settings.couchDB_CustomHeaders,
useJWT: settings.useJWT,
jwtKey: settings.jwtKey,
jwtAlgorithm: settings.jwtAlgorithm,
jwtKid: settings.jwtKid,
jwtExpDuration: settings.jwtExpDuration,
jwtSub: settings.jwtSub,
}; };
settings.encryptedCouchDBConnection = await this.encryptConfigurationItem( settings.encryptedCouchDBConnection = await this.encryptConfigurationItem(
JSON.stringify(connectionSetting), JSON.stringify(connectionSetting),

5
src/modules/features/ModuleObsidianSettingAsMarkdown.ts
View File

@@ -191,6 +191,11 @@ export class ModuleObsidianSettingsAsMarkdown extends AbstractObsidianModule imp
delete saveData.couchDB_USER; delete saveData.couchDB_USER;
delete saveData.couchDB_PASSWORD; delete saveData.couchDB_PASSWORD;
delete saveData.passphrase; delete saveData.passphrase;
delete saveData.jwtKey;
delete saveData.jwtKid;
delete saveData.jwtSub;
delete saveData.couchDB_CustomHeaders;
delete saveData.bucketCustomHeaders;
} }
return saveData; return saveData;
} }

345
src/modules/features/ModuleSetupObsidian.ts
View File

@@ -1,22 +1,39 @@
import { import {
type ObsidianLiveSyncSettings, type ObsidianLiveSyncSettings,
DEFAULT_SETTINGS, DEFAULT_SETTINGS,
KeyIndexOfSettings,
LOG_LEVEL_NOTICE, LOG_LEVEL_NOTICE,
LOG_LEVEL_VERBOSE, LOG_LEVEL_VERBOSE,
} from "../../lib/src/common/types.ts"; } from "../../lib/src/common/types.ts";
import { configURIBase } from "../../common/types.ts"; import { configURIBase, configURIBaseQR } from "../../common/types.ts";
// import { PouchDB } from "../../lib/src/pouchdb/pouchdb-browser.js"; // import { PouchDB } from "../../lib/src/pouchdb/pouchdb-browser.js";
import { decrypt, encrypt } from "../../lib/src/encryption/e2ee_v2.ts"; import { decrypt, encrypt } from "../../lib/src/encryption/e2ee_v2.ts";
import { fireAndForget } from "../../lib/src/common/utils.ts"; import { fireAndForget } from "../../lib/src/common/utils.ts";
import { EVENT_REQUEST_COPY_SETUP_URI, EVENT_REQUEST_OPEN_SETUP_URI, eventHub } from "../../common/events.ts"; import {
EVENT_REQUEST_COPY_SETUP_URI,
EVENT_REQUEST_OPEN_SETUP_URI,
EVENT_REQUEST_SHOW_SETUP_QR,
eventHub,
} from "../../common/events.ts";
import { AbstractObsidianModule, type IObsidianModule } from "../AbstractObsidianModule.ts"; import { AbstractObsidianModule, type IObsidianModule } from "../AbstractObsidianModule.ts";
import { decodeAnyArray, encodeAnyArray } from "../../common/utils.ts";
import qrcode from "qrcode-generator";
import { $msg } from "../../lib/src/common/i18n.ts";
export class ModuleSetupObsidian extends AbstractObsidianModule implements IObsidianModule { export class ModuleSetupObsidian extends AbstractObsidianModule implements IObsidianModule {
$everyOnload(): Promise<boolean> { $everyOnload(): Promise<boolean> {
this.registerObsidianProtocolHandler( this.registerObsidianProtocolHandler("setuplivesync", async (conf: any) => {
"setuplivesync", if (conf.settings) {
async (conf: any) => await this.setupWizard(conf.settings) await this.setupWizard(conf.settings);
); } else if (conf.settingsQR) {
await this.decodeQR(conf.settingsQR);
}
});
this.addCommand({
id: "livesync-setting-qr",
name: "Show settings as a QR code",
callback: () => fireAndForget(this.encodeQR()),
});
this.addCommand({ this.addCommand({
id: "livesync-copysetupuri", id: "livesync-copysetupuri",
@@ -42,9 +59,48 @@ export class ModuleSetupObsidian extends AbstractObsidianModule implements IObsi
}); });
eventHub.onEvent(EVENT_REQUEST_OPEN_SETUP_URI, () => fireAndForget(() => this.command_openSetupURI())); eventHub.onEvent(EVENT_REQUEST_OPEN_SETUP_URI, () => fireAndForget(() => this.command_openSetupURI()));
eventHub.onEvent(EVENT_REQUEST_COPY_SETUP_URI, () => fireAndForget(() => this.command_copySetupURI())); eventHub.onEvent(EVENT_REQUEST_COPY_SETUP_URI, () => fireAndForget(() => this.command_copySetupURI()));
eventHub.onEvent(EVENT_REQUEST_SHOW_SETUP_QR, () => fireAndForget(() => this.encodeQR()));
return Promise.resolve(true); return Promise.resolve(true);
} }
async encodeQR() {
const settingArr = [];
const fullIndexes = Object.entries(KeyIndexOfSettings) as [keyof ObsidianLiveSyncSettings, number][];
for (const [settingKey, index] of fullIndexes) {
const settingValue = this.settings[settingKey];
settingArr[index] = settingValue;
}
const w = encodeAnyArray(settingArr);
// console.warn(w.length)
// console.warn(w);
// const j = decodeAnyArray(w);
// console.warn(j);
// console.warn(`is equal: ${isObjectDifferent(settingArr, j)}`);
const qr = qrcode(0, "L");
const uri = `${configURIBaseQR}${encodeURIComponent(w)}`;
qr.addData(uri);
qr.make();
const img = qr.createSvgTag(3);
const msg = $msg("Setup.QRCode", { qr_image: img });
await this.core.confirm.confirmWithMessage("Settings QR Code", msg, ["OK"], "OK");
return await Promise.resolve(w);
}
async decodeQR(qr: string) {
const settingArr = decodeAnyArray(qr);
// console.warn(settingArr);
const fullIndexes = Object.entries(KeyIndexOfSettings) as [keyof ObsidianLiveSyncSettings, number][];
const newSettings = { ...DEFAULT_SETTINGS } as ObsidianLiveSyncSettings;
for (const [settingKey, index] of fullIndexes) {
if (index >= settingArr.length) {
// Possibly a new setting added.
continue;
}
const settingValue = settingArr[index];
//@ts-ignore
newSettings[settingKey] = settingValue;
}
console.warn(newSettings);
await this.applySettingWizard(this.settings, newSettings, "QR Code");
}
async command_copySetupURI(stripExtra = true) { async command_copySetupURI(stripExtra = true) {
const encryptingPassphrase = await this.core.confirm.askString( const encryptingPassphrase = await this.core.confirm.askString(
"Encrypt your settings", "Encrypt your settings",
@@ -74,7 +130,7 @@ export class ModuleSetupObsidian extends AbstractObsidianModule implements IObsi
const encryptedSetting = encodeURIComponent( const encryptedSetting = encodeURIComponent(
await encrypt(JSON.stringify(setting), encryptingPassphrase, false) await encrypt(JSON.stringify(setting), encryptingPassphrase, false)
); );
const uri = `${configURIBase}${encryptedSetting}`; const uri = `${configURIBase}${encryptedSetting} `;
await navigator.clipboard.writeText(uri); await navigator.clipboard.writeText(uri);
this._log("Setup URI copied to clipboard", LOG_LEVEL_NOTICE); this._log("Setup URI copied to clipboard", LOG_LEVEL_NOTICE);
} }
@@ -95,7 +151,7 @@ export class ModuleSetupObsidian extends AbstractObsidianModule implements IObsi
const encryptedSetting = encodeURIComponent( const encryptedSetting = encodeURIComponent(
await encrypt(JSON.stringify(setting), encryptingPassphrase, false) await encrypt(JSON.stringify(setting), encryptingPassphrase, false)
); );
const uri = `${configURIBase}${encryptedSetting}`; const uri = `${configURIBase}${encryptedSetting} `;
await navigator.clipboard.writeText(uri); await navigator.clipboard.writeText(uri);
this._log("Setup URI copied to clipboard", LOG_LEVEL_NOTICE); this._log("Setup URI copied to clipboard", LOG_LEVEL_NOTICE);
} }
@@ -103,16 +159,155 @@ export class ModuleSetupObsidian extends AbstractObsidianModule implements IObsi
await this.command_copySetupURI(false); await this.command_copySetupURI(false);
} }
async command_openSetupURI() { async command_openSetupURI() {
const setupURI = await this.core.confirm.askString("Easy setup", "Set up URI", `${configURIBase}aaaaa`); const setupURI = await this.core.confirm.askString("Easy setup", "Set up URI", `${configURIBase} aaaaa`);
if (setupURI === false) return; if (setupURI === false) return;
if (!setupURI.startsWith(`${configURIBase}`)) { if (!setupURI.startsWith(`${configURIBase}`)) {
this._log("Set up URI looks wrong.", LOG_LEVEL_NOTICE); this._log("Set up URI looks wrong.", LOG_LEVEL_NOTICE);
return; return;
} }
const config = decodeURIComponent(setupURI.substring(configURIBase.length)); const config = decodeURIComponent(setupURI.substring(configURIBase.length));
console.dir(config);
await this.setupWizard(config); await this.setupWizard(config);
} }
async applySettingWizard(
oldConf: ObsidianLiveSyncSettings,
newConf: ObsidianLiveSyncSettings,
method = "Setup URI"
) {
const result = await this.core.confirm.askYesNoDialog(
"Importing Configuration from the " + method + ". Are you sure to proceed ? ",
{}
);
if (result == "yes") {
const newSettingW = Object.assign({}, DEFAULT_SETTINGS, newConf) as ObsidianLiveSyncSettings;
this.core.replicator.closeReplication();
this.settings.suspendFileWatching = true;
console.dir(newSettingW);
// Back into the default method once.
newSettingW.configPassphraseStore = "";
newSettingW.encryptedPassphrase = "";
newSettingW.encryptedCouchDBConnection = "";
newSettingW.additionalSuffixOfDatabaseName = `${"appId" in this.app ? this.app.appId : ""} `;
const setupJustImport = "Don't sync anything, just apply the settings.";
const setupAsNew = "This is a new client - sync everything from the remote server.";
const setupAsMerge = "This is an existing client - merge existing files with the server.";
const setupAgain = "Initialise new server data - ideal for new or broken servers.";
const setupManually = "Continue and configure manually.";
newSettingW.syncInternalFiles = false;
newSettingW.usePluginSync = false;
newSettingW.isConfigured = true;
// Migrate completely obsoleted configuration.
if (!newSettingW.useIndexedDBAdapter) {
newSettingW.useIndexedDBAdapter = true;
}
const setupType = await this.core.confirm.askSelectStringDialogue(
"How would you like to set it up?",
[setupAsNew, setupAgain, setupAsMerge, setupJustImport, setupManually],
{ defaultAction: setupAsNew }
);
if (setupType == setupJustImport) {
this.core.settings = newSettingW;
this.core.$$clearUsedPassphrase();
await this.core.saveSettings();
} else if (setupType == setupAsNew) {
this.core.settings = newSettingW;
this.core.$$clearUsedPassphrase();
await this.core.saveSettings();
await this.core.rebuilder.$fetchLocal();
} else if (setupType == setupAsMerge) {
this.core.settings = newSettingW;
this.core.$$clearUsedPassphrase();
await this.core.saveSettings();
await this.core.rebuilder.$fetchLocal(true);
} else if (setupType == setupAgain) {
const confirm =
"This operation will rebuild all databases with files on this device. Any files on the remote database not synced here will be lost.";
if (
(await this.core.confirm.askSelectStringDialogue(
"Are you sure you want to do this?",
["Cancel", confirm],
{ defaultAction: "Cancel" }
)) != confirm
) {
return;
}
this.core.settings = newSettingW;
await this.core.saveSettings();
this.core.$$clearUsedPassphrase();
await this.core.rebuilder.$rebuildEverything();
} else if (setupType == setupManually) {
const keepLocalDB = await this.core.confirm.askYesNoDialog("Keep local DB?", {
defaultOption: "No",
});
const keepRemoteDB = await this.core.confirm.askYesNoDialog("Keep remote DB?", {
defaultOption: "No",
});
if (keepLocalDB == "yes" && keepRemoteDB == "yes") {
// nothing to do. so peaceful.
this.core.settings = newSettingW;
this.core.$$clearUsedPassphrase();
await this.core.$allSuspendAllSync();
await this.core.$allSuspendExtraSync();
await this.core.saveSettings();
const replicate = await this.core.confirm.askYesNoDialog("Unlock and replicate?", {
defaultOption: "Yes",
});
if (replicate == "yes") {
await this.core.$$replicate(true);
await this.core.$$markRemoteUnlocked();
}
this._log("Configuration loaded.", LOG_LEVEL_NOTICE);
return;
}
if (keepLocalDB == "no" && keepRemoteDB == "no") {
const reset = await this.core.confirm.askYesNoDialog("Drop everything?", {
defaultOption: "No",
});
if (reset != "yes") {
this._log("Cancelled", LOG_LEVEL_NOTICE);
this.core.settings = oldConf;
return;
}
}
let initDB;
this.core.settings = newSettingW;
this.core.$$clearUsedPassphrase();
await this.core.saveSettings();
if (keepLocalDB == "no") {
await this.core.$$resetLocalDatabase();
await this.core.localDatabase.initializeDatabase();
const rebuild = await this.core.confirm.askYesNoDialog("Rebuild the database?", {
defaultOption: "Yes",
});
if (rebuild == "yes") {
initDB = this.core.$$initializeDatabase(true);
} else {
await this.core.$$markRemoteResolved();
}
}
if (keepRemoteDB == "no") {
await this.core.$$tryResetRemoteDatabase();
await this.core.$$markRemoteLocked();
}
if (keepLocalDB == "no" || keepRemoteDB == "no") {
const replicate = await this.core.confirm.askYesNoDialog("Replicate once?", {
defaultOption: "Yes",
});
if (replicate == "yes") {
if (initDB != null) {
await initDB;
}
await this.core.$$replicate(true);
}
}
}
this._log("Configuration loaded.", LOG_LEVEL_NOTICE);
} else {
this._log("Cancelled", LOG_LEVEL_NOTICE);
this.core.settings = oldConf;
return;
}
}
async setupWizard(confString: string) { async setupWizard(confString: string) {
try { try {
const oldConf = JSON.parse(JSON.stringify(this.settings)); const oldConf = JSON.parse(JSON.stringify(this.settings));
@@ -125,133 +320,7 @@ export class ModuleSetupObsidian extends AbstractObsidianModule implements IObsi
if (encryptingPassphrase === false) return; if (encryptingPassphrase === false) return;
const newConf = await JSON.parse(await decrypt(confString, encryptingPassphrase, false)); const newConf = await JSON.parse(await decrypt(confString, encryptingPassphrase, false));
if (newConf) { if (newConf) {
const result = await this.core.confirm.askYesNoDialog( await this.applySettingWizard(oldConf, newConf);
"Importing Configuration from the Setup-URI. Are you sure to proceed?",
{}
);
if (result == "yes") {
const newSettingW = Object.assign({}, DEFAULT_SETTINGS, newConf) as ObsidianLiveSyncSettings;
this.core.replicator.closeReplication();
this.settings.suspendFileWatching = true;
console.dir(newSettingW);
// Back into the default method once.
newSettingW.configPassphraseStore = "";
newSettingW.encryptedPassphrase = "";
newSettingW.encryptedCouchDBConnection = "";
newSettingW.additionalSuffixOfDatabaseName = `${"appId" in this.app ? this.app.appId : ""}`;
const setupJustImport = "Don't sync anything, just apply the settings.";
const setupAsNew = "This is a new client - sync everything from the remote server.";
const setupAsMerge = "This is an existing client - merge existing files with the server.";
const setupAgain = "Initialise new server data - ideal for new or broken servers.";
const setupManually = "Continue and configure manually.";
newSettingW.syncInternalFiles = false;
newSettingW.usePluginSync = false;
newSettingW.isConfigured = true;
// Migrate completely obsoleted configuration.
if (!newSettingW.useIndexedDBAdapter) {
newSettingW.useIndexedDBAdapter = true;
}
const setupType = await this.core.confirm.askSelectStringDialogue(
"How would you like to set it up?",
[setupAsNew, setupAgain, setupAsMerge, setupJustImport, setupManually],
{ defaultAction: setupAsNew }
);
if (setupType == setupJustImport) {
this.core.settings = newSettingW;
this.core.$$clearUsedPassphrase();
await this.core.saveSettings();
} else if (setupType == setupAsNew) {
this.core.settings = newSettingW;
this.core.$$clearUsedPassphrase();
await this.core.rebuilder.$fetchLocal();
} else if (setupType == setupAsMerge) {
this.core.settings = newSettingW;
this.core.$$clearUsedPassphrase();
await this.core.rebuilder.$fetchLocal(true);
} else if (setupType == setupAgain) {
const confirm =
"This operation will rebuild all databases with files on this device. Any files on the remote database not synced here will be lost.";
if (
(await this.core.confirm.askSelectStringDialogue(
"Are you sure you want to do this?",
["Cancel", confirm],
{ defaultAction: "Cancel" }
)) != confirm
) {
return;
}
this.core.settings = newSettingW;
this.core.$$clearUsedPassphrase();
await this.core.rebuilder.$rebuildEverything();
} else if (setupType == setupManually) {
const keepLocalDB = await this.core.confirm.askYesNoDialog("Keep local DB?", {
defaultOption: "No",
});
const keepRemoteDB = await this.core.confirm.askYesNoDialog("Keep remote DB?", {
defaultOption: "No",
});
if (keepLocalDB == "yes" && keepRemoteDB == "yes") {
// nothing to do. so peaceful.
this.core.settings = newSettingW;
this.core.$$clearUsedPassphrase();
await this.core.$allSuspendAllSync();
await this.core.$allSuspendExtraSync();
await this.core.saveSettings();
const replicate = await this.core.confirm.askYesNoDialog("Unlock and replicate?", {
defaultOption: "Yes",
});
if (replicate == "yes") {
await this.core.$$replicate(true);
await this.core.$$markRemoteUnlocked();
}
this._log("Configuration loaded.", LOG_LEVEL_NOTICE);
return;
}
if (keepLocalDB == "no" && keepRemoteDB == "no") {
const reset = await this.core.confirm.askYesNoDialog("Drop everything?", {
defaultOption: "No",
});
if (reset != "yes") {
this._log("Cancelled", LOG_LEVEL_NOTICE);
this.core.settings = oldConf;
return;
}
}
let initDB;
this.core.settings = newSettingW;
this.core.$$clearUsedPassphrase();
await this.core.saveSettings();
if (keepLocalDB == "no") {
await this.core.$$resetLocalDatabase();
await this.core.localDatabase.initializeDatabase();
const rebuild = await this.core.confirm.askYesNoDialog("Rebuild the database?", {
defaultOption: "Yes",
});
if (rebuild == "yes") {
initDB = this.core.$$initializeDatabase(true);
} else {
await this.core.$$markRemoteResolved();
}
}
if (keepRemoteDB == "no") {
await this.core.$$tryResetRemoteDatabase();
await this.core.$$markRemoteLocked();
}
if (keepLocalDB == "no" || keepRemoteDB == "no") {
const replicate = await this.core.confirm.askYesNoDialog("Replicate once?", {
defaultOption: "Yes",
});
if (replicate == "yes") {
if (initDB != null) {
await initDB;
}
await this.core.$$replicate(true);
}
}
}
}
this._log("Configuration loaded.", LOG_LEVEL_NOTICE); this._log("Configuration loaded.", LOG_LEVEL_NOTICE);
} else { } else {
this._log("Cancelled.", LOG_LEVEL_NOTICE); this._log("Cancelled.", LOG_LEVEL_NOTICE);

253
src/modules/features/SettingDialogue/ObsidianLiveSyncSettingTab.ts
View File

@@ -32,10 +32,11 @@ import {
delay, delay,
isDocContentSame, isDocContentSame,
isObjectDifferent, isObjectDifferent,
parseHeaderValues,
readAsBlob, readAsBlob,
sizeToHumanReadable, sizeToHumanReadable,
} from "../../../lib/src/common/utils.ts"; } from "../../../lib/src/common/utils.ts";
import { versionNumberString2Number } from "../../../lib/src/string_and_binary/convert.ts"; import { arrayBufferToBase64Single, versionNumberString2Number } from "../../../lib/src/string_and_binary/convert.ts";
import { Logger } from "../../../lib/src/common/logger.ts"; import { Logger } from "../../../lib/src/common/logger.ts";
import { import {
balanceChunkPurgedDBs, balanceChunkPurgedDBs,
@@ -45,7 +46,7 @@ import {
} from "../../../lib/src/pouchdb/utils_couchdb.ts"; } from "../../../lib/src/pouchdb/utils_couchdb.ts";
import { testCrypt } from "../../../lib/src/encryption/e2ee_v2.ts"; import { testCrypt } from "../../../lib/src/encryption/e2ee_v2.ts";
import ObsidianLiveSyncPlugin from "../../../main.ts"; import ObsidianLiveSyncPlugin from "../../../main.ts";
import { getPath, requestToCouchDB, scheduleTask } from "../../../common/utils.ts"; import { getPath, requestToCouchDBWithCredentials, scheduleTask } from "../../../common/utils.ts";
import { request } from "obsidian"; import { request } from "obsidian";
import { addPrefix, shouldBeIgnored, stripAllPrefixes } from "../../../lib/src/string_and_binary/path.ts"; import { addPrefix, shouldBeIgnored, stripAllPrefixes } from "../../../lib/src/string_and_binary/path.ts";
import MultipleRegExpControl from "./MultipleRegExpControl.svelte"; import MultipleRegExpControl from "./MultipleRegExpControl.svelte";
@@ -71,6 +72,8 @@ import {
EVENT_REQUEST_OPEN_PLUGIN_SYNC_DIALOG, EVENT_REQUEST_OPEN_PLUGIN_SYNC_DIALOG,
EVENT_REQUEST_OPEN_SETUP_URI, EVENT_REQUEST_OPEN_SETUP_URI,
EVENT_REQUEST_RELOAD_SETTING_TAB, EVENT_REQUEST_RELOAD_SETTING_TAB,
EVENT_REQUEST_RUN_DOCTOR,
EVENT_REQUEST_SHOW_SETUP_QR,
eventHub, eventHub,
} from "../../../common/events.ts"; } from "../../../common/events.ts";
import { skipIfDuplicated } from "octagonal-wheels/concurrency/lock"; import { skipIfDuplicated } from "octagonal-wheels/concurrency/lock";
@@ -80,6 +83,8 @@ import { HiddenFileSync } from "../../../features/HiddenFileSync/CmdHiddenFileSy
import { EVENT_REQUEST_SHOW_HISTORY } from "../../../common/obsidianEvents.ts"; import { EVENT_REQUEST_SHOW_HISTORY } from "../../../common/obsidianEvents.ts";
import { LocalDatabaseMaintenance } from "../../../features/LocalDatabaseMainte/CmdLocalDatabaseMainte.ts"; import { LocalDatabaseMaintenance } from "../../../features/LocalDatabaseMainte/CmdLocalDatabaseMainte.ts";
import { mount } from "svelte"; import { mount } from "svelte";
import { getWebCrypto } from "../../../lib/src/mods.ts";
import { generateCredentialObject } from "../../../lib/src/replication/httplib.ts";
export type OnUpdateResult = { export type OnUpdateResult = {
visibility?: boolean; visibility?: boolean;
@@ -813,6 +818,12 @@ export class ObsidianLiveSyncSettingTab extends PluginSettingTab {
return false; return false;
}; };
const enableOnlySyncDisabled = enableOnly(() => !isAnySyncEnabled()); const enableOnlySyncDisabled = enableOnly(() => !isAnySyncEnabled());
const combineOnUpdate = (func1: OnUpdateFunc, func2: OnUpdateFunc): OnUpdateFunc => {
return () => ({
...func1(),
...func2(),
});
};
const onlyOnP2POrCouchDB = () => const onlyOnP2POrCouchDB = () =>
({ ({
visibility: visibility:
@@ -978,7 +989,16 @@ export class ObsidianLiveSyncSettingTab extends PluginSettingTab {
eventHub.emitEvent(EVENT_REQUEST_COPY_SETUP_URI); eventHub.emitEvent(EVENT_REQUEST_COPY_SETUP_URI);
}); });
}); });
new Setting(paneEl)
.setName($msg("Setup.ShowQRCode"))
.setDesc($msg("Setup.ShowQRCode.Desc"))
.addButton((text) => {
text.setButtonText($msg("Setup.ShowQRCode")).onClick(() => {
eventHub.emitEvent(EVENT_REQUEST_SHOW_SETUP_QR);
});
});
}); });
void addPanel(paneEl, $msg("obsidianLiveSyncSettingTab.titleReset")).then((paneEl) => { void addPanel(paneEl, $msg("obsidianLiveSyncSettingTab.titleReset")).then((paneEl) => {
new Setting(paneEl) new Setting(paneEl)
.setName($msg("obsidianLiveSyncSettingTab.nameDiscardSettings")) .setName($msg("obsidianLiveSyncSettingTab.nameDiscardSettings"))
@@ -1166,11 +1186,16 @@ export class ObsidianLiveSyncSettingTab extends PluginSettingTab {
return; return;
} }
// Tip: Add log for cloudant as Logger($msg("obsidianLiveSyncSettingTab.logServerConfigurationCheck")); // Tip: Add log for cloudant as Logger($msg("obsidianLiveSyncSettingTab.logServerConfigurationCheck"));
const r = await requestToCouchDB( const customHeaders = parseHeaderValues(this.editingSettings.couchDB_CustomHeaders);
const credential = generateCredentialObject(this.editingSettings);
const r = await requestToCouchDBWithCredentials(
this.editingSettings.couchDB_URI, this.editingSettings.couchDB_URI,
this.editingSettings.couchDB_USER, credential,
this.editingSettings.couchDB_PASSWORD, window.origin,
window.origin undefined,
undefined,
undefined,
customHeaders
); );
const responseConfig = r.json; const responseConfig = r.json;
@@ -1183,13 +1208,14 @@ export class ObsidianLiveSyncSettingTab extends PluginSettingTab {
x.querySelector("button")?.addEventListener("click", () => { x.querySelector("button")?.addEventListener("click", () => {
fireAndForget(async () => { fireAndForget(async () => {
Logger($msg("obsidianLiveSyncSettingTab.logCouchDbConfigSet", { title, key, value })); Logger($msg("obsidianLiveSyncSettingTab.logCouchDbConfigSet", { title, key, value }));
const res = await requestToCouchDB( const res = await requestToCouchDBWithCredentials(
this.editingSettings.couchDB_URI, this.editingSettings.couchDB_URI,
this.editingSettings.couchDB_USER, credential,
this.editingSettings.couchDB_PASSWORD,
undefined, undefined,
key, key,
value value,
undefined,
customHeaders
); );
if (res.status == 200) { if (res.status == 200) {
Logger( Logger(
@@ -1324,11 +1350,14 @@ export class ObsidianLiveSyncSettingTab extends PluginSettingTab {
// Request header check // Request header check
const origins = ["app://obsidian.md", "capacitor://localhost", "http://localhost"]; const origins = ["app://obsidian.md", "capacitor://localhost", "http://localhost"];
for (const org of origins) { for (const org of origins) {
const rr = await requestToCouchDB( const rr = await requestToCouchDBWithCredentials(
this.editingSettings.couchDB_URI, this.editingSettings.couchDB_URI,
this.editingSettings.couchDB_USER, credential,
this.editingSettings.couchDB_PASSWORD, org,
org undefined,
undefined,
undefined,
customHeaders
); );
const responseHeaders = Object.fromEntries( const responseHeaders = Object.fromEntries(
Object.entries(rr.headers).map((e) => { Object.entries(rr.headers).map((e) => {
@@ -1443,6 +1472,10 @@ The pane also can be launched by \`P2P Replicator\` command from the Command Pal
new Setting(paneEl).autoWireText("bucket", { holdValue: true }); new Setting(paneEl).autoWireText("bucket", { holdValue: true });
new Setting(paneEl).autoWireToggle("useCustomRequestHandler", { holdValue: true }); new Setting(paneEl).autoWireToggle("useCustomRequestHandler", { holdValue: true });
new Setting(paneEl).autoWireTextArea("bucketCustomHeaders", {
holdValue: true,
placeHolder: "x-custom-header: value\n x-custom-header2: value2",
});
new Setting(paneEl) new Setting(paneEl)
.setName($msg("obsidianLiveSyncSettingTab.nameTestConnection")) .setName($msg("obsidianLiveSyncSettingTab.nameTestConnection"))
.addButton((button) => .addButton((button) =>
@@ -1464,6 +1497,7 @@ The pane also can be launched by \`P2P Replicator\` command from the Command Pal
"secretKey", "secretKey",
"bucket", "bucket",
"useCustomRequestHandler", "useCustomRequestHandler",
"bucketCustomHeaders",
]) ])
.addOnUpdate(onlyOnMinIO); .addOnUpdate(onlyOnMinIO);
}); });
@@ -1510,20 +1544,119 @@ The pane also can be launched by \`P2P Replicator\` command from the Command Pal
holdValue: true, holdValue: true,
onUpdate: enableOnlySyncDisabled, onUpdate: enableOnlySyncDisabled,
}); });
new Setting(paneEl).autoWireText("couchDB_USER", { new Setting(paneEl).autoWireToggle("useJWT", {
holdValue: true, holdValue: true,
onUpdate: enableOnlySyncDisabled, onUpdate: enableOnlySyncDisabled,
}); });
new Setting(paneEl).autoWireText("couchDB_USER", {
holdValue: true,
onUpdate: combineOnUpdate(
enableOnlySyncDisabled,
visibleOnly(() => !this.editingSettings.useJWT)
),
});
new Setting(paneEl).autoWireText("couchDB_PASSWORD", { new Setting(paneEl).autoWireText("couchDB_PASSWORD", {
holdValue: true, holdValue: true,
isPassword: true, isPassword: true,
onUpdate: enableOnlySyncDisabled, onUpdate: combineOnUpdate(
enableOnlySyncDisabled,
visibleOnly(() => !this.editingSettings.useJWT)
),
});
const algorithms = {
["HS256"]: "HS256",
["HS512"]: "HS512",
["ES256"]: "ES256",
["ES512"]: "ES512",
} as const;
new Setting(paneEl).autoWireDropDown("jwtAlgorithm", {
options: algorithms,
onUpdate: combineOnUpdate(
enableOnlySyncDisabled,
visibleOnly(() => this.editingSettings.useJWT)
),
});
new Setting(paneEl).autoWireTextArea("jwtKey", {
holdValue: true,
onUpdate: combineOnUpdate(
enableOnlySyncDisabled,
visibleOnly(() => this.editingSettings.useJWT)
),
});
// eslint-disable-next-line prefer-const
let generatedKeyDivEl: HTMLDivElement;
new Setting(paneEl)
.setDesc("Generate ES256 Keypair for testing")
.addButton((button) =>
button.setButtonText("Generate").onClick(async () => {
const crypto = await getWebCrypto();
const keyPair = await crypto.subtle.generateKey(
{ name: "ECDSA", namedCurve: "P-256" },
true,
["sign", "verify"]
);
const pubKey = await crypto.subtle.exportKey("spki", keyPair.publicKey);
const privateKey = await crypto.subtle.exportKey("pkcs8", keyPair.privateKey);
const encodedPublicKey = await arrayBufferToBase64Single(pubKey);
const encodedPrivateKey = await arrayBufferToBase64Single(privateKey);
const privateKeyPem = `> -----BEGIN PRIVATE KEY-----\n> ${encodedPrivateKey}\n> -----END PRIVATE KEY-----`;
const publicKeyPem = `> -----BEGIN PUBLIC KEY-----\\n${encodedPublicKey}\\n-----END PUBLIC KEY-----`;
const title = $msg("Setting.GenerateKeyPair.Title");
const msg = $msg("Setting.GenerateKeyPair.Desc", {
public_key: publicKeyPem,
private_key: privateKeyPem,
});
await MarkdownRenderer.render(
this.plugin.app,
"## " + title + "\n\n" + msg,
generatedKeyDivEl,
"/",
this.plugin
);
})
)
.addOnUpdate(
combineOnUpdate(
enableOnlySyncDisabled,
visibleOnly(() => this.editingSettings.useJWT)
)
);
generatedKeyDivEl = this.createEl(
paneEl,
"div",
{ text: "" },
(el) => {},
visibleOnly(() => this.editingSettings.useJWT)
);
new Setting(paneEl).autoWireText("jwtKid", {
holdValue: true,
onUpdate: combineOnUpdate(
enableOnlySyncDisabled,
visibleOnly(() => this.editingSettings.useJWT)
),
});
new Setting(paneEl).autoWireText("jwtSub", {
holdValue: true,
onUpdate: combineOnUpdate(
enableOnlySyncDisabled,
visibleOnly(() => this.editingSettings.useJWT)
),
});
new Setting(paneEl).autoWireNumeric("jwtExpDuration", {
holdValue: true,
onUpdate: combineOnUpdate(
enableOnlySyncDisabled,
visibleOnly(() => this.editingSettings.useJWT)
),
}); });
new Setting(paneEl).autoWireText("couchDB_DBNAME", { new Setting(paneEl).autoWireText("couchDB_DBNAME", {
holdValue: true, holdValue: true,
onUpdate: enableOnlySyncDisabled, onUpdate: enableOnlySyncDisabled,
}); });
new Setting(paneEl).autoWireTextArea("couchDB_CustomHeaders", { holdValue: true });
new Setting(paneEl) new Setting(paneEl)
.setName($msg("obsidianLiveSyncSettingTab.nameTestDatabaseConnection")) .setName($msg("obsidianLiveSyncSettingTab.nameTestDatabaseConnection"))
.setClass("wizardHidden") .setClass("wizardHidden")
@@ -1561,6 +1694,13 @@ The pane also can be launched by \`P2P Replicator\` command from the Command Pal
"couchDB_USER", "couchDB_USER",
"couchDB_PASSWORD", "couchDB_PASSWORD",
"couchDB_DBNAME", "couchDB_DBNAME",
"jwtAlgorithm",
"jwtExpDuration",
"jwtKey",
"jwtSub",
"jwtKid",
"useJWT",
"couchDB_CustomHeaders",
]) ])
.addOnUpdate(onlyOnCouchDB); .addOnUpdate(onlyOnCouchDB);
}); });
@@ -1613,8 +1753,33 @@ The pane also can be launched by \`P2P Replicator\` command from the Command Pal
const newTweaks = const newTweaks =
await this.plugin.$$checkAndAskUseRemoteConfiguration(trialSetting); await this.plugin.$$checkAndAskUseRemoteConfiguration(trialSetting);
if (newTweaks.result !== false) { if (newTweaks.result !== false) {
this.editingSettings = { ...this.editingSettings, ...newTweaks.result }; if (this.inWizard) {
this.requestUpdate(); this.editingSettings = { ...this.editingSettings, ...newTweaks.result };
this.requestUpdate();
return;
} else {
this.closeSetting();
this.plugin.settings = { ...this.plugin.settings, ...newTweaks.result };
if (newTweaks.requireFetch) {
if (
(await this.plugin.confirm.askYesNoDialog(
$msg("SettingTab.Message.AskRebuild"),
{
defaultOption: "Yes",
}
)) == "no"
) {
await this.plugin.$$saveSettingData();
return;
}
await this.plugin.$$saveSettingData();
await this.plugin.rebuilder.scheduleFetch();
await this.plugin.$$scheduleAppReload();
return;
} else {
await this.plugin.$$saveSettingData();
}
}
} }
}) })
); );
@@ -1890,6 +2055,9 @@ The pane also can be launched by \`P2P Replicator\` command from the Command Pal
}) })
.setClass("wizardHidden"); .setClass("wizardHidden");
new Setting(paneEl).autoWireNumeric("syncMinimumInterval", {
onUpdate: onlyOnNonLiveSync,
});
new Setting(paneEl) new Setting(paneEl)
.setClass("wizardHidden") .setClass("wizardHidden")
.autoWireToggle("syncOnSave", { onUpdate: onlyOnNonLiveSync }); .autoWireToggle("syncOnSave", { onUpdate: onlyOnNonLiveSync });
@@ -1956,7 +2124,6 @@ The pane also can be launched by \`P2P Replicator\` command from the Command Pal
LEVEL_ADVANCED LEVEL_ADVANCED
).then((paneEl) => { ).then((paneEl) => {
paneEl.addClass("wizardHidden"); paneEl.addClass("wizardHidden");
new Setting(paneEl) new Setting(paneEl)
.autoWireText("settingSyncFile", { holdValue: true }) .autoWireText("settingSyncFile", { holdValue: true })
.addApplyButton(["settingSyncFile"]); .addApplyButton(["settingSyncFile"]);
@@ -2226,10 +2393,23 @@ The pane also can be launched by \`P2P Replicator\` command from the Command Pal
void addPane(containerEl, "Hatch", "🧰", 50, true).then((paneEl) => { void addPane(containerEl, "Hatch", "🧰", 50, true).then((paneEl) => {
// const hatchWarn = this.createEl(paneEl, "div", { text: `To stop the boot up sequence for fixing problems on databases, you can put redflag.md on top of your vault (Rebooting obsidian is required).` }); // const hatchWarn = this.createEl(paneEl, "div", { text: `To stop the boot up sequence for fixing problems on databases, you can put redflag.md on top of your vault (Rebooting obsidian is required).` });
// hatchWarn.addClass("op-warn-info"); // hatchWarn.addClass("op-warn-info");
void addPanel(paneEl, "Reporting Issue").then((paneEl) => { void addPanel(paneEl, $msg("Setting.TroubleShooting")).then((paneEl) => {
new Setting(paneEl).setName("Make report to inform the issue").addButton((button) => new Setting(paneEl)
.setName($msg("Setting.TroubleShooting.Doctor"))
.setDesc($msg("Setting.TroubleShooting.Doctor.Desc"))
.addButton((button) =>
button
.setButtonText("Run Doctor")
.setCta()
.setDisabled(false)
.onClick(() => {
this.closeSetting();
eventHub.emitEvent(EVENT_REQUEST_RUN_DOCTOR, "you wanted(Thank you)!");
})
);
new Setting(paneEl).setName("Prepare the 'report' to create an issue").addButton((button) =>
button button
.setButtonText("Make report") .setButtonText("Copy Report to clipboard")
.setCta() .setCta()
.setDisabled(false) .setDisabled(false)
.onClick(async () => { .onClick(async () => {
@@ -2237,11 +2417,16 @@ The pane also can be launched by \`P2P Replicator\` command from the Command Pal
const REDACTED = "𝑅𝐸𝐷𝐴𝐶𝑇𝐸𝐷"; const REDACTED = "𝑅𝐸𝐷𝐴𝐶𝑇𝐸𝐷";
if (this.editingSettings.remoteType == REMOTE_COUCHDB) { if (this.editingSettings.remoteType == REMOTE_COUCHDB) {
try { try {
const r = await requestToCouchDB( const credential = generateCredentialObject(this.editingSettings);
const customHeaders = parseHeaderValues(this.editingSettings.couchDB_CustomHeaders);
const r = await requestToCouchDBWithCredentials(
this.editingSettings.couchDB_URI, this.editingSettings.couchDB_URI,
this.editingSettings.couchDB_USER, credential,
this.editingSettings.couchDB_PASSWORD, window.origin,
window.origin undefined,
undefined,
undefined,
customHeaders
); );
Logger(JSON.stringify(r.json, null, 2)); Logger(JSON.stringify(r.json, null, 2));
@@ -2288,6 +2473,11 @@ The pane also can be launched by \`P2P Replicator\` command from the Command Pal
pluginConfig.P2P_passphrase = redact(pluginConfig.P2P_passphrase); pluginConfig.P2P_passphrase = redact(pluginConfig.P2P_passphrase);
pluginConfig.P2P_roomID = redact(pluginConfig.P2P_roomID); pluginConfig.P2P_roomID = redact(pluginConfig.P2P_roomID);
pluginConfig.P2P_relays = redact(pluginConfig.P2P_relays); pluginConfig.P2P_relays = redact(pluginConfig.P2P_relays);
pluginConfig.jwtKey = redact(pluginConfig.jwtKey);
pluginConfig.jwtSub = redact(pluginConfig.jwtSub);
pluginConfig.jwtKid = redact(pluginConfig.jwtKid);
pluginConfig.bucketCustomHeaders = redact(pluginConfig.bucketCustomHeaders);
pluginConfig.couchDB_CustomHeaders = redact(pluginConfig.couchDB_CustomHeaders);
const endpoint = pluginConfig.endpoint; const endpoint = pluginConfig.endpoint;
if (endpoint == "") { if (endpoint == "") {
pluginConfig.endpoint = "Not configured or AWS"; pluginConfig.endpoint = "Not configured or AWS";
@@ -2310,7 +2500,10 @@ version:${manifestVersion}
${stringifyYaml(pluginConfig)}`; ${stringifyYaml(pluginConfig)}`;
console.log(msgConfig); console.log(msgConfig);
await navigator.clipboard.writeText(msgConfig); await navigator.clipboard.writeText(msgConfig);
Logger(`Information has been copied to clipboard`, LOG_LEVEL_NOTICE); Logger(
`Generated report has been copied to clipboard. Please report the issue with this! Thank you for your cooperation!`,
LOG_LEVEL_NOTICE
);
}) })
); );
new Setting(paneEl).autoWireToggle("writeLogToTheFile"); new Setting(paneEl).autoWireToggle("writeLogToTheFile");
@@ -3338,6 +3531,7 @@ ${stringifyYaml(pluginConfig)}`;
const region = this.plugin.settings.region; const region = this.plugin.settings.region;
const endpoint = this.plugin.settings.endpoint; const endpoint = this.plugin.settings.endpoint;
const useCustomRequestHandler = this.plugin.settings.useCustomRequestHandler; const useCustomRequestHandler = this.plugin.settings.useCustomRequestHandler;
const customHeaders = this.plugin.settings.bucketCustomHeaders;
return new JournalSyncMinio( return new JournalSyncMinio(
id, id,
key, key,
@@ -3346,7 +3540,8 @@ ${stringifyYaml(pluginConfig)}`;
this.plugin.simpleStore, this.plugin.simpleStore,
this.plugin, this.plugin,
useCustomRequestHandler, useCustomRequestHandler,
region region,
customHeaders
); );
} }
async resetRemoteBucket() { async resetRemoteBucket() {

8
src/modules/features/SettingDialogue/settingConstants.ts
View File

@@ -346,8 +346,8 @@ export const SettingInformation: Partial<Record<keyof AllSettings, Configuration
desc: "If this enabled, All files are handled as case-Sensitive (Previous behaviour).", desc: "If this enabled, All files are handled as case-Sensitive (Previous behaviour).",
}, },
doNotUseFixedRevisionForChunks: { doNotUseFixedRevisionForChunks: {
name: "Compute revisions for chunks (Previous behaviour)", name: "Compute revisions for chunks",
desc: "If this enabled, all chunks will be stored with the revision made from its content. (Previous behaviour)", desc: "If this enabled, all chunks will be stored with the revision made from its content.",
}, },
sendChunksBulkMaxSize: { sendChunksBulkMaxSize: {
name: "Maximum size of chunks to send in one request", name: "Maximum size of chunks to send in one request",
@@ -373,6 +373,10 @@ export const SettingInformation: Partial<Record<keyof AllSettings, Configuration
name: "Suppress notification of hidden files change", name: "Suppress notification of hidden files change",
desc: "If enabled, the notification of hidden files change will be suppressed.", desc: "If enabled, the notification of hidden files change will be suppressed.",
}, },
syncMinimumInterval: {
name: "Minimum interval for syncing",
desc: "The minimum interval for automatic synchronisation on event.",
},
}; };
function translateInfo(infoSrc: ConfigurationItem | undefined | false) { function translateInfo(infoSrc: ConfigurationItem | undefined | false) {
if (!infoSrc) return false; if (!infoSrc) return false;

2
src/modules/interfaces/DatabaseRebuilder.ts
View File

@@ -4,7 +4,7 @@ export interface Rebuilder {
): Promise<void>; ): Promise<void>;
$rebuildRemote(): Promise<void>; $rebuildRemote(): Promise<void>;
$rebuildEverything(): Promise<void>; $rebuildEverything(): Promise<void>;
$fetchLocal(makeLocalChunkBeforeSync?: boolean): Promise<void>; $fetchLocal(makeLocalChunkBeforeSync?: boolean, preventMakeLocalFilesBeforeSync?: boolean): Promise<void>;
scheduleRebuild(): Promise<void>; scheduleRebuild(): Promise<void>;
scheduleFetch(): Promise<void>; scheduleFetch(): Promise<void>;

2
src/modules/interfaces/StorageAccess.ts
View File

@@ -38,7 +38,7 @@ export interface StorageAccess {
getFiles(): UXFileInfoStub[]; getFiles(): UXFileInfoStub[];
getFileNames(): FilePathWithPrefix[]; getFileNames(): FilePathWithPrefix[];
touched(file: UXFileInfoStub | FilePathWithPrefix): void; touched(file: UXFileInfoStub | FilePathWithPrefix): Promise<void>;
recentlyTouched(file: UXFileInfoStub | FilePathWithPrefix): boolean; recentlyTouched(file: UXFileInfoStub | FilePathWithPrefix): boolean;
clearTouched(): void; clearTouched(): void;

15
styles.css
View File

@@ -437,3 +437,18 @@ span.ls-mark-cr::after {
.sls-dialogue-note-countdown { .sls-dialogue-note-countdown {
font-size: 0.8em; font-size: 0.8em;
} }
.sls-qr {
display: flex;
justify-content: center;
align-items: center;
max-width: max-content;
}
.sls-keypair pre {
max-width: 100%;
overflow-x: auto;
white-space: pre-wrap;
word-break: break-all;
}

6
tsconfig.json
View File

@@ -16,7 +16,11 @@
"noEmit": true, "noEmit": true,
"lib": ["es2018", "DOM", "ES5", "ES6", "ES7", "es2019.array", "ES2021.WeakRef", "ES2020.BigInt", "ESNext.Intl"], "lib": ["es2018", "DOM", "ES5", "ES6", "ES7", "es2019.array", "ES2021.WeakRef", "ES2020.BigInt", "ESNext.Intl"],
"strictBindCallApply": true, "strictBindCallApply": true,
"strictFunctionTypes": true "strictFunctionTypes": true,
"paths": {
"@/*": ["src/*"],
"@lib/*": ["src/lib/src/*"]
}
}, },
"include": ["**/*.ts"], "include": ["**/*.ts"],
"exclude": ["pouchdb-browser-webpack", "utils"] "exclude": ["pouchdb-browser-webpack", "utils"]

138
updates.md
View File

@@ -10,95 +10,103 @@ Nevertheless, that being said, to be more honest, I still have not decided what
Note: Already you have noticed this, but let me mention it again, this is a significantly large update. If you have noticed anything, please let me know. I will try to fix it as soon as possible (Some address is on my [profile](https://github.com/vrtmrz)). Note: Already you have noticed this, but let me mention it again, this is a significantly large update. If you have noticed anything, please let me know. I will try to fix it as soon as possible (Some address is on my [profile](https://github.com/vrtmrz)).
## 0.24.12 ## 0.24.24
I created a SPA called [webpeer](https://github.com/vrtmrz/livesync-commonlib/tree/main/apps/webpeer) (well, right... I will think of a name again), which replaces the server when using Peer-to-Peer synchronisation. This is a pseudo-client that appears to other devices as if it were one of the clients. . As with the client, it receives and sends data without storing it as a file.
And, this is just a single web page, without any server-side code. It is a static web page that can be hosted on any static web server, such as GitHub Pages, Netlify, or Vercel. All you have to do is to open the page and enter several items, and leave it open.
### Fixed ### Fixed
- No longer unnecessary acknowledgements are sent when starting peer-to-peer synchronisation. - No longer broken JSON files including `\n`, during the bucket synchronisation. (#623)
- Custom headers and JWT tokens are now correctly sent to the server during configuration checking. (#624)
### Improved
- Bucket synchronisation has been enhanced for better performance and reliability.
- Now less duplicated chunks are sent to the server.
Note: If you have encountered about too less chunks, please let me know. However, you can send it to the server by `Overwrite remote`.
- Fetching conflicted files from the server is now more reliable.
- Dependent libraries have been updated to the latest version.
- Also, let me know if you have encountered any issues with this update. Especially you are using a device that has been in use for a little longer.
## 0.24.23
### New Feature
- Now, we can send custom headers to the server.
- They can be sent to either CouchDB or Object Storage.
- Authentication with JWT in CouchDB is now supported.
- I will describe steps later, but please refer to the [CouchDB document](https://docs.couchdb.org/en/stable/config/auth.html#authentication-configuration).
- A JWT keypair for testing can be generated in the setting dialogue.
### Improved
- The QR Code for set-up can be shown also from the setting dialogue now.
- Conflict checking for preventing unexpected overwriting on the boot-up process has been quite faster.
### Fixed
- Some bugs on Dev and Testing modules have been fixed.
## 0.24.22 ~~0.24.21~~
(Really sorry for the confusion. I have got a miss at releasing...).
### Fixed
- No longer conflicted files are handled in the boot-up process. No more unexpected overwriting.
- It ignores `Always overwrite with a newer file`, and always be prevented for the safety. Please pick it manually or open the file.
- Some log messages on conflict resolution has been corrected.
- Automatic merge notifications, displayed on the grounds of `same`, have been degraded to logs.
### Improved
- Now we can fetch the remote database with keeping local files completely intact.
- In new option, all files are stored into the local database before the fetching, and will be merged automatically or detected as conflicts.
- The dialogue presenting options when performing `Fetch` are now more informative.
### Refactored ### Refactored
- Platform impedance-matching-layer has been improved. - Some class methods have been fixed its arguments to be more consistent.
- And you can see the actual usage of this on [webpeer](https://github.com/vrtmrz/livesync-commonlib/tree/main/apps/webpeer) that a pseudo client for peer-to-peer synchronisation. - Types have been defined for some conditional results.
- Some UIs have been got isomorphic among Obsidian and web applications (for `webpeer`).
## 0.24.11 ## 0.24.20
### Improved ### Improved
- New Translation: `es` (Spanish) by @zeedif (Thank you so much)! - Now we can see the detail of `TypeError` using Obsidian API during remote database access.
- Now all of messages can be selectable and copyable, also on the iPhone, iPad, and Android devices. Now we can copy or share the messages easily.
## 0.24.19
### New Feature ### New Feature
- Peer-to-Peer Synchronisation has been implemented! - Now we can generate a QR Code for transferring the configuration to another device.
- This feature is still in early beta, and it is recommended to use it with caution. - This QR Code can be scanned by the camera app or something QR Code Reader of another device, and via Obsidian URL, the configuration will be transferred.
- However, it is a significant step towards the self-hosting concept. It is now possible to synchronise your data without using any remote database or storage. It is a direct connection between your devices. - Note: This QR Code is not encrypted. So, please be careful when transferring the configuration.
- Note: We should keep the device online to synchronise the data. It is not a background synchronisation. Also it needs a signalling server to establish the connection. But, the signalling server is used only for establishing the connection, and it does not store any data.
## 0.24.18
### Fixed ### Fixed
- No longer memory or resource leaks when the plug-in is disabled. - Now no chunk creation errors will be raised after switching `Compute revisions for chunks`.
- Now deleted chunks are correctly detected on conflict resolution, and we are guided to resurrect them. - Some invisible file can be handled correctly (e.g., `writing-goals-history.csv`).
- Hanging issue during the initial synchronisation has been fixed. - Fetching configuration from the server is now saves the configuration immediately (if we are not in the wizard).
- Some unnecessary logs have been removed.
- Now all modal dialogues are correctly closed when the plug-in is disabled.
### Refactor
- Several interfaces have been moved to the separated library.
- Translations have been moved to each language file, and during the build, they are merged into one file.
- Non-mobile friendly code has been removed and replaced with the safer code.
- (Now a days, mostly server-side engine can use webcrypto, so it will be rewritten in the future more).
- Started writing Platform impedance-matching-layer.
- Svelte has been updated to v5.
- Some function have got more robust type definitions.
- Terser optimisation has slightly improved.
- During the build, analysis meta-file of the bundled codes will be generated.
## 0.24.10
### Fixed
- Fixed the issue which the filename is shown as `undefined`.
- Fixed the issue where files transferred at short intervals were not reflected.
### Improved ### Improved
- Add more translations: `ja-JP` (Japanese) by @kohki-shikata (Thank you so much)! - Mismatched configuration dialogue is now more informative, and rewritten to more user-friendly.
- Applying configuration mismatch is now without rebuilding (at our own risks).
- Now, rebuilding is decided more fine grained.
### Internal ### Improved internally
- Some files have been prettified. - Translations can be nested. i.e., task:`Some procedure`, check: `%{task} checking`, checkfailed: `%{check} failed` produces `Some procedure checking failed`.
- Max to 10 levels of nesting
## 0.24.9 ## 0.24.17
Skipped. Confession. I got the default values wrong. So scary and sorry.
## 0.24.8 ### Behaviour and default changed
### Fixed - **NOW INDEED AND ACTUALLY** `Compute revisions for chunks` are backed into enabled again. it is necessary for garbage collection of chunks.
- As far as existing users are concerned, this will not automatically change, but the Doctor will inform us.
- Some parallel-processing tasks are now performed more safely.
- Some error messages has been fixed.
### Improved
- Synchronisation is now more efficient and faster.
- Saving chunks is a bit more robust.
### New Feature
- We can remove orphaned chunks again, now!
- Without rebuilding the database!
- Note: Please synchronise devices completely before removing orphaned chunks.
- Note2: Deleted files are using chunks, if you want to remove them, please commit the deletion first. (`Commit File Deletion`)
- Note3: If you lost some chunks, do not worry. They will be resurrected if not so much time has passed. Try `Resurrect deleted chunks`.
- Note4: This feature is still beta. Please report any issues you encounter.
- Note5: Please disable `On demand chunk fetching`, and enable `Compute revisions for each chunk` before using this feature.
- These settings is going to be default in the future.
Older notes are in [updates_old.md](https://github.com/vrtmrz/obsidian-livesync/blob/main/updates_old.md). Older notes are in [updates_old.md](https://github.com/vrtmrz/obsidian-livesync/blob/main/updates_old.md).

176
updates_old.md
View File

@@ -14,6 +14,182 @@ Thank you, and I hope your troubles will be resolved!
--- ---
## 0.24.16
### Improved
#### Peer-to-Peer
- Now peer-to-peer synchronisation checks the settings are compatible with each other.
- No longer unexpected database broken, phew.
- Peer-to-peer synchronisation now handles the platform and detects pseudo-clients.
- Pseudo clients will not decrypt/encrypt anything, just relay the data. Hence, always settings are not compatible. Therefore, we have to accept the incompatibility for pseudo clients.
#### General
- New migration method has been implemented, that called `Doctor`.
- `Doctor` checks the difference between the ideal and actual values and encourages corrective action. To facilitate our decision, the reasons for this and the recommendations are also presented.
- This can be used not only during migration. We can invoke the doctor from the settings for trouble-shooting.
- The minimum interval for replication to be caused when an event occurs can now be configurable.
- Some detail note has been added and change nuance about the `Report` in the setting dialogue, which had less informative.
### Behaviour and default changed
- `Compute revisions for chunks` are backed into enabled again. it is necessary for garbage collection of chunks.
- As far as existing users are concerned, this will not automatically change, but the Doctor will inform us.
### Refactored
- Platform specific codes are more separated. No longer `node` modules were used in the browser and Obsidian.
## 0.24.15
### Fixed
- Now, even without WeakRef, Polyfill is used and the whole thing works without error. However, if you can switch WebView Engine, it is recommended to switch to a WebView Engine that supports WeakRef.
## 0.24.14
### Fixed
- Resolving conflicts of JSON files (and sensibly merging them) is now working fine, again!
- And, failure logs are more informative.
- More robust to release the event listeners on unwatching the local database.
### Refactored
- JSON file conflict resolution dialogue has been rewritten into svelte v5.
- Upgrade eslint.
- Remove unnecessary pragma comments for eslint.
## 0.24.13
Sorry for the lack of replies. The ones that were not good are popping up, so I am just going to go ahead and get this one... However, they realised that refactoring and restructuring is about clarifying the problem. Your patience and understanding is much appreciated.
### Fixed
#### General Replication
- No longer unexpected errors occur when the replication is stopped during for some reason (e.g., network disconnection).
#### Peer-to-Peer Synchronisation
- Set-up process will not receive data from unexpected sources.
- No longer resource leaks while enabling the `broadcasting changes`
- Logs are less verbose.
- Received data is now correctly dispatched to other devices.
- `Timeout` error now more informative.
- No longer timeout error occurs for reporting the progress to other devices.
- Decision dialogues for the same thing are not shown multiply at the same time anymore.
- Disconnection of the peer-to-peer synchronisation is now more robust and less error-prone.
#### Webpeer
- Now we can toggle Peers' configuration.
### Refactored
- Cross-platform compatibility layer has been improved.
- Common events are moved to the common library.
- Displaying replication status of the peer-to-peer synchronisation is separated from the main-log-logic.
- Some file names have been changed to be more consistent.
## 0.24.12
I created a SPA called [webpeer](https://github.com/vrtmrz/livesync-commonlib/tree/main/apps/webpeer) (well, right... I will think of a name again), which replaces the server when using Peer-to-Peer synchronisation. This is a pseudo-client that appears to other devices as if it were one of the clients. . As with the client, it receives and sends data without storing it as a file.
And, this is just a single web page, without any server-side code. It is a static web page that can be hosted on any static web server, such as GitHub Pages, Netlify, or Vercel. All you have to do is to open the page and enter several items, and leave it open.
### Fixed
- No longer unnecessary acknowledgements are sent when starting peer-to-peer synchronisation.
### Refactored
- Platform impedance-matching-layer has been improved.
- And you can see the actual usage of this on [webpeer](https://github.com/vrtmrz/livesync-commonlib/tree/main/apps/webpeer) that a pseudo client for peer-to-peer synchronisation.
- Some UIs have been got isomorphic among Obsidian and web applications (for `webpeer`).
## 0.24.11
### Improved
- New Translation: `es` (Spanish) by @zeedif (Thank you so much)!
- Now all of messages can be selectable and copyable, also on the iPhone, iPad, and Android devices. Now we can copy or share the messages easily.
### New Feature
- Peer-to-Peer Synchronisation has been implemented!
- This feature is still in early beta, and it is recommended to use it with caution.
- However, it is a significant step towards the self-hosting concept. It is now possible to synchronise your data without using any remote database or storage. It is a direct connection between your devices.
- Note: We should keep the device online to synchronise the data. It is not a background synchronisation. Also it needs a signalling server to establish the connection. But, the signalling server is used only for establishing the connection, and it does not store any data.
### Fixed
- No longer memory or resource leaks when the plug-in is disabled.
- Now deleted chunks are correctly detected on conflict resolution, and we are guided to resurrect them.
- Hanging issue during the initial synchronisation has been fixed.
- Some unnecessary logs have been removed.
- Now all modal dialogues are correctly closed when the plug-in is disabled.
### Refactor
- Several interfaces have been moved to the separated library.
- Translations have been moved to each language file, and during the build, they are merged into one file.
- Non-mobile friendly code has been removed and replaced with the safer code.
- (Now a days, mostly server-side engine can use webcrypto, so it will be rewritten in the future more).
- Started writing Platform impedance-matching-layer.
- Svelte has been updated to v5.
- Some function have got more robust type definitions.
- Terser optimisation has slightly improved.
- During the build, analysis meta-file of the bundled codes will be generated.
## 0.24.10
### Fixed
- Fixed the issue which the filename is shown as `undefined`.
- Fixed the issue where files transferred at short intervals were not reflected.
### Improved
- Add more translations: `ja-JP` (Japanese) by @kohki-shikata (Thank you so much)!
### Internal
- Some files have been prettified.
## 0.24.9
Skipped.
## 0.24.8
### Fixed
- Some parallel-processing tasks are now performed more safely.
- Some error messages has been fixed.
### Improved
- Synchronisation is now more efficient and faster.
- Saving chunks is a bit more robust.
### New Feature
- We can remove orphaned chunks again, now!
- Without rebuilding the database!
- Note: Please synchronise devices completely before removing orphaned chunks.
- Note2: Deleted files are using chunks, if you want to remove them, please commit the deletion first. (`Commit File Deletion`)
- Note3: If you lost some chunks, do not worry. They will be resurrected if not so much time has passed. Try `Resurrect deleted chunks`.
- Note4: This feature is still beta. Please report any issues you encounter.
- Note5: Please disable `On demand chunk fetching`, and enable `Compute revisions for each chunk` before using this feature.
- These settings is going to be default in the future.
## 0.24.7 ## 0.24.7
### Fixed (Security) ### Fixed (Security)