public/obsidian-livesync
1
0
Fork 0
mirror of https://github.com/vrtmrz/obsidian-livesync.git synced 2026-03-15 06:18:49 +00:00
Code Issues Packages Projects Releases Wiki Activity

Compare commits

base: public:0.24.23
Branches Tags
public:main public:beta public:alpha public:snyk-fix-34b8981b68f7111ada8b22377113a98c public:snyk-upgrade-7fd31e6f474568014502e7205847f257 public:snyk-fix-90f3b71450f7543a1c61724d8afa0e71 public:refactor_services_2 public:refactor_services public:dev public:snyk-upgrade-fe9625385aa4be849c7157033fd2abb0 public:snyk-upgrade-99a31e70c822894595544b0c9813b11a public:svelteui public:disenchant public:snyk-fix-61f76ef0bce2a276df9552c2fc80a14f public:snyk-upgrade-2baa7938ba318464c3bad5c3fd5cbda5 public:snyk-upgrade-7af62a575eab20e8fb4e9565bf1adcfc public:snyk-upgrade-b32f281b31a81483f126b8fb6c54a957 public:snyk-upgrade-bd61b1103442d3faecc9fc726c34e9a4 public:snyk-upgrade-5b2d6584430c061b5187611a5d7fb64a public:snyk-fix-0e9fbbcb374fcb8d5e8cb027d8414fb7 public:snyk-upgrade-5242daf91e706d1ae500c9ea6545636e public:snyk-upgrade-70434a6822fbad83424135dfad9df909 public:snyk-upgrade-8fa01ccad0352125f1866d41e57d35e1 public:snyk-upgrade-39eb97b4655e70091f718a88fccd57c1 public:snyk-upgrade-2b99d54d19e207887e36c3071b796925 public:forprivacy public:snyk-upgrade-dad9088d0a333677e0157e3deef7fc38 public:snyk-upgrade-b9a76d74b3e5723759af5b3319aabb80 public:snyk-upgrade-7da6459d8ff5d293d80de9d80061d912 public:snyk-upgrade-6a3786fc4cbe394542e9d1b6058c46e7 public:snyk-upgrade-8fc9e62771dc41b0f23dc0efefb5c949
public:0.25.52-patched-2 public:0.25.52-patched-1 public:0.25.52 public:0.25.51 public:0.25.50 public:0.25.49 public:0.25.48 public:0.25.47 public:0.25.46 public:0.25.45 public:0.25.44 public:0.25.43-patched-9 public:0.25.43-patched-8 public:0.25.43-patched-7 public:0.25.43-patched-6 public:0.25.43-patched-5 public:0.25.43-patched-4 public:0.25.43-patched-3 public:0.25.43-patched-2 public:0.25.43-patched-1 public:0.25.43 public:0.25.42 public:0.25.41-patched-3 public:0.25.41-patched-2 public:0.25.41-patched-1 public:0.25.41 public:0.25.40 public:0.25.39 public:0.25.38 public:0.25.37 public:0.25.36 public:0.25.35 public:0.25.34 public:0.25.33 public:0.25.32 public:0.25.31 public:0.25.30 public:0.25.30-dev public:0.25.29 public:0.25.28 public:0.25.27 public:0.25.26 public:0.25.25 public:0.25.24 public:0.25.24.beta3 public:0.25.24.beta2 public:0.25.24.beta1 public:0.25.23 public:0.25.23.beta1 public:0.25.22 public:0.25.21 public:0.25.21.beta2 public:0.25.21.beta1 public:0.25.20 public:0.25.19 public:0.25.18 public:0.25.17 public:0.25.16 public:0.25.15 public:0.25.14 public:0.25.13 public:0.25.12 public:0.25.11 public:0.25.10 public:0.25.9 public:0.25.8 public:0.25.7 public:0.25.6 public:0.25.5 public:0.25.4 public:0.25.3 public:0.25.2 public:0.25.1 public:0.25.0 public:0.25.0-beta1 public:0.24.31 public:0.24.30 public:0.24.29 public:0.24.28 public:0.24.27 public:0.24.26 public:0.24.25 public:0.24.24 public:0.24.23 public:0.24.22 public:0.24.21 public:0.24.20 public:0.24.19 public:0.24.18 public:0.24.17 public:0.24.16 public:0.24.15 public:0.24.14 public:0.24.13 public:0.24.12 public:0.24.11 public:0.24.10 public:0.24.9 public:0.24.8 public:0.24.7 public:0.24.6 public:0.24.5 public:0.24.4 public:0.24.3 public:0.24.2 public:0.24.1 public:0.24.0 public:0.24.0.dev-rc8 public:0.24.0.dev-rc7 public:0.24.0.dev-rc6 public:0.24.0.dev-rc5 public:0.24.0.dev-rc4 public:0.24.0.dev-rc3 public:0.24.0.dev-rc2 public:0.23.23 public:0.23.22 public:0.23.21 public:0.23.20 public:0.23.19 public:0.23.18 public:0.23.17 public:0.23.16 public:0.23.15 public:0.23.14 public:0.23.13 public:0.23.12 public:0.23.11 public:0.23.10 public:0.23.9 public:0.23.8 public:0.23.7 public:0.23.6 public:0.23.5 public:0.23.4 public:0.23.3 public:0.23.2 public:0.23.1 public:0.23.0 public:0.22.19 public:0.22.18 public:0.22.17 public:0.22.16 public:0.22.15 public:0.22.14 public:0.22.13 public:0.22.12 public:0.22.11 public:0.22.10 public:0.22.9 public:0.22.8 public:0.22.7 public:0.22.6 public:0.22.5 public:0.22.4 public:0.22.3 public:0.22.2 public:0.22.1 public:0.22.0 public:0.21.5 public:0.21.4 public:0.21.3 public:0.21.2 public:0.21.1 public:0.21.0 public:0.20.7 public:0.20.6 public:0.20.5 public:0.20.4 public:0.20.3 public:0.20.2 public:0.20.1 public:0.20.0 public:0.19.23 public:0.19.22 public:0.19.21 public:0.19.20 public:0.19.19 public:0.19.18 public:0.19.17 public:0.19.16 public:0.19.15 public:0.19.14 public:0.19.13 public:0.19.12 public:0.19.11 public:0.19.10 public:0.19.9 public:0.19.8 public:0.19.7 public:0.19.6 public:0.19.5 public:0.19.4 public:0.19.3 public:0.19.2 public:0.19.1 public:0.19.0 public:0.18.6 public:0.18.5 public:0.18.4 public:0.18.3 public:0.18.2 public:0.18.1 public:0.18.0 public:0.17.34 public:0.17.33 public:0.17.32 public:0.17.31 public:0.17.30 public:0.17.29 public:0.17.28 public:0.17.27 public:0.17.26 public:0.17.25 public:0.17.24 public:0.17.23 public:0.17.22 public:0.17.21 public:0.17.20 public:0.17.19 public:0.17.18 public:0.17.17 public:0.17.16 public:0.17.15 public:0.17.14 public:0.17.13 public:0.17.12 public:0.17.11 public:0.17.10 public:0.17.9 public:0.17.8 public:0.17.7 public:0.17.6 public:0.17.5 public:0.17.4 public:0.17.3 public:0.17.2 public:0.17.1 public:0.17.0 public:0.16.8 public:0.16.7 public:0.16.6 public:0.16.5 public:0.16.4 public:0.16.3 public:0.16.2 public:0.16.1 public:0.16.0 public:0.15.10 public:0.15.9 public:0.15.8 public:0.15.7 public:0.15.6 public:0.15.5 public:0.15.4 public:0.15.3 public:0.15.2 public:0.15.1 public:0.15.0 public:0.14.8 public:0.14.7 public:0.14.6 public:0.14.5 public:0.14.4 public:0.14.3 public:0.14.2 public:0.14.1 public:0.14.0 public:0.13.4 public:0.13.3 public:0.13.2 public:0.13.1 public:0.13.0 public:0.12.5 public:0.12.4 public:0.12.3 public:0.12.2 public:0.12.1 public:0.12.0 public:0.11.10 public:0.11.9 public:0.11.8 public:0.11.7 public:0.11.6 public:0.11.5 public:0.11.4 public:0.11.3 public:0.11.2 public:0.11.1 public:0.11.0 public:0.10.1 public:0.10.0 public:0.9.0 public:0.8.8 public:0.8.7 public:0.8.6 public:0.8.5 public:0.8.3 public:0.8.2 public:0.8.1 public:0.8.0 public:0.7.2 public:0.7.1 public:0.7.0 public:0.6.1 public:0.6.0 public:0.5.0 public:0.4.1 public:0.4.0 public:0.3.1 public:0.3.0 public:0.2.1 public:0.2.0 public:0.1.26 public:0.1.25 public:0.1.24 public:0.1.23 public:0.1.22 public:0.1.21 public:0.1.20 public:0.1.19 public:0.1.18 public:0.1.17 public:0.1.16 public:0.1.15 public:0.1.14 public:0.1.13 public:0.1.12 public:0.1.11 public:0.1.10 public:0.1.9 public:0.1.8 public:0.1.7 public:0.1.6 public:0.1.5 public:0.1.4 public:0.1.3 public:0.1.2 public:0.1.1 public:0.1.0 public:0.0.9 public:0.0.8 public:0.0.7 public:0.0.6
...
compare: public:0.24.25
Branches Tags
public:beta public:alpha public:main public:snyk-fix-34b8981b68f7111ada8b22377113a98c public:snyk-upgrade-7fd31e6f474568014502e7205847f257 public:snyk-fix-90f3b71450f7543a1c61724d8afa0e71 public:refactor_services_2 public:refactor_services public:dev public:snyk-upgrade-fe9625385aa4be849c7157033fd2abb0 public:snyk-upgrade-99a31e70c822894595544b0c9813b11a public:svelteui public:disenchant public:snyk-fix-61f76ef0bce2a276df9552c2fc80a14f public:snyk-upgrade-2baa7938ba318464c3bad5c3fd5cbda5 public:snyk-upgrade-7af62a575eab20e8fb4e9565bf1adcfc public:snyk-upgrade-b32f281b31a81483f126b8fb6c54a957 public:snyk-upgrade-bd61b1103442d3faecc9fc726c34e9a4 public:snyk-upgrade-5b2d6584430c061b5187611a5d7fb64a public:snyk-fix-0e9fbbcb374fcb8d5e8cb027d8414fb7 public:snyk-upgrade-5242daf91e706d1ae500c9ea6545636e public:snyk-upgrade-70434a6822fbad83424135dfad9df909 public:snyk-upgrade-8fa01ccad0352125f1866d41e57d35e1 public:snyk-upgrade-39eb97b4655e70091f718a88fccd57c1 public:snyk-upgrade-2b99d54d19e207887e36c3071b796925 public:forprivacy public:snyk-upgrade-dad9088d0a333677e0157e3deef7fc38 public:snyk-upgrade-b9a76d74b3e5723759af5b3319aabb80 public:snyk-upgrade-7da6459d8ff5d293d80de9d80061d912 public:snyk-upgrade-6a3786fc4cbe394542e9d1b6058c46e7 public:snyk-upgrade-8fc9e62771dc41b0f23dc0efefb5c949
public:0.25.52-patched-2 public:0.25.52-patched-1 public:0.25.52 public:0.25.51 public:0.25.50 public:0.25.49 public:0.25.48 public:0.25.47 public:0.25.46 public:0.25.45 public:0.25.44 public:0.25.43-patched-9 public:0.25.43-patched-8 public:0.25.43-patched-7 public:0.25.43-patched-6 public:0.25.43-patched-5 public:0.25.43-patched-4 public:0.25.43-patched-3 public:0.25.43-patched-2 public:0.25.43-patched-1 public:0.25.43 public:0.25.42 public:0.25.41-patched-3 public:0.25.41-patched-2 public:0.25.41-patched-1 public:0.25.41 public:0.25.40 public:0.25.39 public:0.25.38 public:0.25.37 public:0.25.36 public:0.25.35 public:0.25.34 public:0.25.33 public:0.25.32 public:0.25.31 public:0.25.30 public:0.25.30-dev public:0.25.29 public:0.25.28 public:0.25.27 public:0.25.26 public:0.25.25 public:0.25.24 public:0.25.24.beta3 public:0.25.24.beta2 public:0.25.24.beta1 public:0.25.23 public:0.25.23.beta1 public:0.25.22 public:0.25.21 public:0.25.21.beta2 public:0.25.21.beta1 public:0.25.20 public:0.25.19 public:0.25.18 public:0.25.17 public:0.25.16 public:0.25.15 public:0.25.14 public:0.25.13 public:0.25.12 public:0.25.11 public:0.25.10 public:0.25.9 public:0.25.8 public:0.25.7 public:0.25.6 public:0.25.5 public:0.25.4 public:0.25.3 public:0.25.2 public:0.25.1 public:0.25.0 public:0.25.0-beta1 public:0.24.31 public:0.24.30 public:0.24.29 public:0.24.28 public:0.24.27 public:0.24.26 public:0.24.25 public:0.24.24 public:0.24.23 public:0.24.22 public:0.24.21 public:0.24.20 public:0.24.19 public:0.24.18 public:0.24.17 public:0.24.16 public:0.24.15 public:0.24.14 public:0.24.13 public:0.24.12 public:0.24.11 public:0.24.10 public:0.24.9 public:0.24.8 public:0.24.7 public:0.24.6 public:0.24.5 public:0.24.4 public:0.24.3 public:0.24.2 public:0.24.1 public:0.24.0 public:0.24.0.dev-rc8 public:0.24.0.dev-rc7 public:0.24.0.dev-rc6 public:0.24.0.dev-rc5 public:0.24.0.dev-rc4 public:0.24.0.dev-rc3 public:0.24.0.dev-rc2 public:0.23.23 public:0.23.22 public:0.23.21 public:0.23.20 public:0.23.19 public:0.23.18 public:0.23.17 public:0.23.16 public:0.23.15 public:0.23.14 public:0.23.13 public:0.23.12 public:0.23.11 public:0.23.10 public:0.23.9 public:0.23.8 public:0.23.7 public:0.23.6 public:0.23.5 public:0.23.4 public:0.23.3 public:0.23.2 public:0.23.1 public:0.23.0 public:0.22.19 public:0.22.18 public:0.22.17 public:0.22.16 public:0.22.15 public:0.22.14 public:0.22.13 public:0.22.12 public:0.22.11 public:0.22.10 public:0.22.9 public:0.22.8 public:0.22.7 public:0.22.6 public:0.22.5 public:0.22.4 public:0.22.3 public:0.22.2 public:0.22.1 public:0.22.0 public:0.21.5 public:0.21.4 public:0.21.3 public:0.21.2 public:0.21.1 public:0.21.0 public:0.20.7 public:0.20.6 public:0.20.5 public:0.20.4 public:0.20.3 public:0.20.2 public:0.20.1 public:0.20.0 public:0.19.23 public:0.19.22 public:0.19.21 public:0.19.20 public:0.19.19 public:0.19.18 public:0.19.17 public:0.19.16 public:0.19.15 public:0.19.14 public:0.19.13 public:0.19.12 public:0.19.11 public:0.19.10 public:0.19.9 public:0.19.8 public:0.19.7 public:0.19.6 public:0.19.5 public:0.19.4 public:0.19.3 public:0.19.2 public:0.19.1 public:0.19.0 public:0.18.6 public:0.18.5 public:0.18.4 public:0.18.3 public:0.18.2 public:0.18.1 public:0.18.0 public:0.17.34 public:0.17.33 public:0.17.32 public:0.17.31 public:0.17.30 public:0.17.29 public:0.17.28 public:0.17.27 public:0.17.26 public:0.17.25 public:0.17.24 public:0.17.23 public:0.17.22 public:0.17.21 public:0.17.20 public:0.17.19 public:0.17.18 public:0.17.17 public:0.17.16 public:0.17.15 public:0.17.14 public:0.17.13 public:0.17.12 public:0.17.11 public:0.17.10 public:0.17.9 public:0.17.8 public:0.17.7 public:0.17.6 public:0.17.5 public:0.17.4 public:0.17.3 public:0.17.2 public:0.17.1 public:0.17.0 public:0.16.8 public:0.16.7 public:0.16.6 public:0.16.5 public:0.16.4 public:0.16.3 public:0.16.2 public:0.16.1 public:0.16.0 public:0.15.10 public:0.15.9 public:0.15.8 public:0.15.7 public:0.15.6 public:0.15.5 public:0.15.4 public:0.15.3 public:0.15.2 public:0.15.1 public:0.15.0 public:0.14.8 public:0.14.7 public:0.14.6 public:0.14.5 public:0.14.4 public:0.14.3 public:0.14.2 public:0.14.1 public:0.14.0 public:0.13.4 public:0.13.3 public:0.13.2 public:0.13.1 public:0.13.0 public:0.12.5 public:0.12.4 public:0.12.3 public:0.12.2 public:0.12.1 public:0.12.0 public:0.11.10 public:0.11.9 public:0.11.8 public:0.11.7 public:0.11.6 public:0.11.5 public:0.11.4 public:0.11.3 public:0.11.2 public:0.11.1 public:0.11.0 public:0.10.1 public:0.10.0 public:0.9.0 public:0.8.8 public:0.8.7 public:0.8.6 public:0.8.5 public:0.8.3 public:0.8.2 public:0.8.1 public:0.8.0 public:0.7.2 public:0.7.1 public:0.7.0 public:0.6.1 public:0.6.0 public:0.5.0 public:0.4.1 public:0.4.0 public:0.3.1 public:0.3.0 public:0.2.1 public:0.2.0 public:0.1.26 public:0.1.25 public:0.1.24 public:0.1.23 public:0.1.22 public:0.1.21 public:0.1.20 public:0.1.19 public:0.1.18 public:0.1.17 public:0.1.16 public:0.1.15 public:0.1.14 public:0.1.13 public:0.1.12 public:0.1.11 public:0.1.10 public:0.1.9 public:0.1.8 public:0.1.7 public:0.1.6 public:0.1.5 public:0.1.4 public:0.1.3 public:0.1.2 public:0.1.1 public:0.1.0 public:0.0.9 public:0.0.8 public:0.0.7 public:0.0.6

6 Commits
0.24.23 ... 0.24.25

Author SHA1 Message Date
vorotamoroz
381055fc93 bump 2025-04-22 11:29:42 +01:00
vorotamoroz
37d12916fc ## 0.24.25 
### Improved

- Peer-to-peer synchronisation has been got more robust.

### Fixed

- No longer broken falsy values in settings during set-up by the QR code generation.

### Refactored

- Some `window` references now have pointed to `globalThis`.
- Some sloppy-import has been fixed.
- A server side implementation `Synchromesh` has been suffixed with `deno` instead of `server` now.
 2025-04-22 11:28:55 +01:00
vorotamoroz
944aa846c4 bump 2025-04-15 11:10:37 +01:00
vorotamoroz
abca808e29 ### Fixed 
- No longer broken JSON files including `\n`, during the bucket synchronisation. (#623)
- Custom headers and JWT tokens are now correctly sent to the server during configuration checking. (#624)

### Improved

- Bucket synchronisation has been enhanced for better performance and reliability.
    - Now less duplicated chunks are sent to the server.
    - Fetching conflicted files from the server is now more reliable.
    - Dependent libraries have been updated to the latest version.
 2025-04-15 11:09:49 +01:00
vorotamoroz
90bb610133 Update submodule 2025-04-14 03:23:24 +01:00
vorotamoroz
9c5e9fe63b Conclusion stated. 2025-04-11 14:13:22 +01:00
11 changed files with 2245 additions and 2939 deletions
Expand all files Collapse all files

43
docs/design_docs_of_keep_newborn_chunks.md
View File

@@ -1,6 +1,6 @@
# Keep newborn chunks in Eden.
# Keep newborn chunks in Eden
NOTE: This is the planned feature design document. This is planned, but not be implemented now (v0.23.3). This has not reached the design freeze and will be added to from time to time.
Notice: deprecated. please refer to the result section of this document.
## Goal
@@ -19,15 +19,18 @@ Reduce the number of chunks which in volatile, and reduce the usage of storage o
- The problem is that this unnecessary chunking slows down both local and remote operations.
## Prerequisite
- The implementation must be able to control the size of the document appropriately so that it does not become non-transferable (1).
- The implementation must be such that data corruption can be avoided even if forward compatibility is not maintained; due to the nature of Self-hosted LiveSync, backward version connexions are expected.
- The implementation must be such that data corruption can be avoided even if forward compatibility is not maintained; due to the nature of Self-hosted LiveSync, backward version connexions are expected.
- Viewed as a feature:
- This feature should be disabled for migration users.
- This feature should be enabled for new users and after rebuilds of migrated users.
- Therefore, back into the implementation view, Ideally, the implementation should be such that data recovery can be achieved by immediately upgrading after replication.
## Outlined methods and implementation plans
### Abstract
To store and transfer only stable chunks independently and share them from multiple documents after stabilisation, new chunks, i.e. chunks that are considered non-stable, are modified to be stored in the document and transferred with the document. In this case, care should be taken not to exceed prerequisite (1).
If this is achieved, the non-leaf document will not be transferred, and even if it is, the chunk will be stored in the document, so that the size can be reduced by the compaction.
@@ -40,11 +43,11 @@ Details are given below.
type EntryWithEden = {
eden: {
[key: DocumentID]: {
data: string,
epoch: number, // The document revision which this chunk has been born.
}
}
}
data: string;
epoch: number; // The document revision which this chunk has been born.
};
};
};
```
2. The following configuration items are added:
Note: These configurations should be shared as `Tweaks value` between each client.
@@ -63,6 +66,7 @@ Details are given below.
5. In End-to-End Encryption, property `eden` of documents will also be encrypted.
### Note
- When this feature has been enabled, forward compatibility is temporarily lost. However, it is detected as missing chunks, and this data is not reflected in the storage in the old version. Therefore, no data loss will occur.
## Test strategy
@@ -77,5 +81,26 @@ Details are given below.
- Indeed, we lack a fulfilled configuration table. Efforts will be made and, if they can be produced, this document will then be referenced. But not required while in the experimental or beta feature.
- However, this might be an essential feature. Further efforts are desired.
## Results from actual operation
After implementing this feature, we have been using it for a while. The following results were obtained.
- Drawbacks were thought not to be a problem, but they were actually a problem:
- A document with `Eden` has a quite larger history compared to a document without `Eden`.
- Self-hosted LiveSync does not perform compaction aggressively, which results in the remote database becoming partially bloated.
- Compaction of the Remote Database (CouchDB) requires the same amount of free space as the size of the database. Therefore, it is not possible to perform compaction on a remote database if we reached to the maximum size of the database. It means that when we detect it, it is too late.
- We have mentioned that `We need compaction` in previous sections. However, but it was so hard to be determined whether the compaction is required or not, until the database is bloated. (Of course, it requires some time to compact the database, and, literally, some document loses its history. It is not a good idea to perform frequently and meaninglessly. We need manual decision, but indeed difficult to normal users).
### Consideration and Conclusion
To be described after implemented, tested, and, released.
This feature results in two aspects:
- For the users who are familiar with the CouchDB, this feature is a bit useful. They can watch and handle the database by themselves.
- For the users who are not familiar with the CouchDB, i.e., normal users, this feature is not so useful, either. They are not familiar with the database, and they do not know how to handle it. Therefore, they cannot decide whether the compaction is required or not.
Hence, this feature would be kept as an experimental feature, but it is not enabled by default. In addition to that, it is marked as deprecated. Detailed notice will be noisy for the users who are not familiar with the CouchDB. Details would be kept in this document, for the future.
It is not recommended to use this feature, unless the person who is familiar with the CouchDB and the database management.
Vorotamoroz has written this document. Bias: I am the first author of this plug-in, familiar with the CouchDB.
Research and development has been frozen on 2025-04-11. But, bugs will be fixed if they are found. Please feel free to report them.

2
manifest.json
View File

@@ -1,7 +1,7 @@
{
"id": "obsidian-livesync",
"name": "Self-hosted LiveSync",
"version": "0.24.23",
"version": "0.24.25",
"minAppVersion": "0.9.12",
"description": "Community implementation of self-hosted livesync. Reflect your vault changes to some other devices immediately. Please make sure to disable other synchronize solutions to avoid content corruption or duplication.",
"author": "vorotamoroz",

4716
package-lock.json generated
View File

File diff suppressed because it is too large Load Diff

12
package.json
View File

@@ -1,6 +1,6 @@
{
"name": "obsidian-livesync",
"version": "0.24.23",
"version": "0.24.25",
"description": "Reflect your vault changes to some other devices immediately. Please make sure to disable other synchronize solutions to avoid content corruption or duplication.",
"main": "main.js",
"type": "module",
@@ -68,10 +68,10 @@
"typescript": "^5.7.3"
},
"dependencies": {
"@aws-sdk/client-s3": "^3.645.0",
"@smithy/fetch-http-handler": "^3.2.4",
"@smithy/protocol-http": "^4.1.0",
"@smithy/querystring-builder": "^3.0.3",
"@aws-sdk/client-s3": "^3.787.0",
"@smithy/fetch-http-handler": "^5.0.2",
"@smithy/protocol-http": "^5.1.0",
"@smithy/querystring-builder": "^4.0.2",
"diff-match-patch": "^1.0.5",
"esbuild-plugin-inline-worker": "^0.1.1",
"fflate": "^0.8.2",
@@ -80,7 +80,7 @@
"octagonal-wheels": "^0.1.25",
"qrcode-generator": "^1.4.4",
"svelte-check": "^4.1.4",
"trystero": "^0.20.1",
"trystero": "^0.21.3",
"xxhash-wasm-102": "npm:xxhash-wasm@^1.0.2"
}
}

53
src/common/utils.ts
View File

@@ -15,6 +15,7 @@ import {
LOG_LEVEL_NOTICE,
LOG_LEVEL_VERBOSE,
type AnyEntry,
type CouchDBCredentials,
type DocumentID,
type EntryHasPath,
type FilePath,
@@ -31,6 +32,7 @@ import type { KeyValueDatabase } from "./KeyValueDB.ts";
import { scheduleTask } from "octagonal-wheels/concurrency/task";
import { EVENT_PLUGIN_UNLOADED, eventHub } from "./events.ts";
import { promiseWithResolver, type PromiseWithResolvers } from "octagonal-wheels/promises";
import { AuthorizationHeaderGenerator } from "../lib/src/replication/httplib.ts";
export { scheduleTask, cancelTask, cancelAllTasks } from "../lib/src/concurrency/task.ts";
@@ -230,17 +232,17 @@ export const _requestToCouchDBFetch = async (
export const _requestToCouchDB = async (
baseUri: string,
username: string,
password: string,
credentials: CouchDBCredentials,
origin: string,
path?: string,
body?: any,
method?: string
method?: string,
customHeaders?: Record<string, string>
) => {
const utf8str = String.fromCharCode.apply(null, [...writeString(`${username}:${password}`)]);
const encoded = window.btoa(utf8str);
const authHeader = "Basic " + encoded;
const transformedHeaders: Record<string, string> = { authorization: authHeader, origin: origin };
// Create each time to avoid caching.
const authHeaderGen = new AuthorizationHeaderGenerator();
const authHeader = await authHeaderGen.getAuthorizationHeader(credentials);
const transformedHeaders: Record<string, string> = { authorization: authHeader, origin: origin, ...customHeaders };
const uri = `${baseUri}/${path}`;
const requestParam: RequestUrlParam = {
url: uri,
@@ -251,6 +253,9 @@ export const _requestToCouchDB = async (
};
return await requestUrl(requestParam);
};
/**
* @deprecated Use requestToCouchDBWithCredentials instead.
*/
export const requestToCouchDB = async (
baseUri: string,
username: string,
@@ -258,12 +263,34 @@ export const requestToCouchDB = async (
origin: string = "",
key?: string,
body?: string,
method?: string
method?: string,
customHeaders?: Record<string, string>
) => {
const uri = `_node/_local/_config${key ? "/" + key : ""}`;
return await _requestToCouchDB(baseUri, username, password, origin, uri, body, method);
return await _requestToCouchDB(
baseUri,
{ username, password, type: "basic" },
origin,
uri,
body,
method,
customHeaders
);
};
export function requestToCouchDBWithCredentials(
baseUri: string,
credentials: CouchDBCredentials,
origin: string = "",
key?: string,
body?: string,
method?: string,
customHeaders?: Record<string, string>
) {
const uri = `_node/_local/_config${key ? "/" + key : ""}`;
return _requestToCouchDB(baseUri, credentials, origin, uri, body, method, customHeaders);
}
export const BASE_IS_NEW = Symbol("base");
export const TARGET_IS_NEW = Symbol("target");
export const EVEN = Symbol("even");
@@ -586,10 +613,10 @@ const decodePrefixMapNumber = Object.fromEntries(
);
export function encodeAnyArray(obj: any[]): string {
const tempArray = obj.map((v) => {
if (v == null) return "n";
if (v == false) return "f";
if (v == true) return "t";
if (v == undefined) return "u";
if (v === null) return "n";
if (v === false) return "f";
if (v === true) return "t";
if (v === undefined) return "u";
if (typeof v == "number") {
const b36 = v.toString(36);
const strNum = v.toString();

2
src/lib

Submodule src/lib updated: ad3f7ee995...c8bb4fedbb

200
src/modules/essentialObsidian/ModuleObsidianAPI.ts
View File

@@ -1,16 +1,7 @@
import { AbstractObsidianModule, type IObsidianModule } from "../AbstractObsidianModule.ts";
import { LOG_LEVEL_DEBUG, LOG_LEVEL_NOTICE, LOG_LEVEL_VERBOSE } from "octagonal-wheels/common/logger";
import { Notice, requestUrl, type RequestUrlParam, type RequestUrlResponse } from "../../deps.ts";
import {
type CouchDBCredentials,
type EntryDoc,
type FilePathWithPrefix,
type JWTCredentials,
type JWTHeader,
type JWTParams,
type JWTPayload,
type PreparedJWT,
} from "../../lib/src/common/types.ts";
import { type CouchDBCredentials, type EntryDoc, type FilePathWithPrefix } from "../../lib/src/common/types.ts";
import { getPathFromTFile } from "../../common/utils.ts";
import {
disableEncryption,
@@ -22,9 +13,7 @@ import {
import { setNoticeClass } from "../../lib/src/mock_and_interop/wrapper.ts";
import { ObsHttpHandler } from "./APILib/ObsHttpHandler.ts";
import { PouchDB } from "../../lib/src/pouchdb/pouchdb-browser.ts";
import { reactive, reactiveSource } from "octagonal-wheels/dataobject/reactive.js";
import { arrayBufferToBase64Single, writeString } from "../../lib/src/string_and_binary/convert.ts";
import { Refiner } from "octagonal-wheels/dataobject/Refiner";
import { AuthorizationHeaderGenerator } from "../../lib/src/replication/httplib.ts";
setNoticeClass(Notice);
@@ -44,10 +33,8 @@ async function fetchByAPI(request: RequestUrlParam): Promise<RequestUrlResponse>
export class ModuleObsidianAPI extends AbstractObsidianModule implements IObsidianModule {
_customHandler!: ObsHttpHandler;
authHeaderSource = reactiveSource<string>("");
authHeader = reactive(() =>
this.authHeaderSource.value == "" ? "" : "Basic " + window.btoa(this.authHeaderSource.value)
);
_authHeader = new AuthorizationHeaderGenerator();
last_successful_post = false;
$$customFetchHandler(): ObsHttpHandler {
@@ -110,135 +97,6 @@ export class ModuleObsidianAPI extends AbstractObsidianModule implements IObsidi
}
}
_importKey(auth: JWTCredentials) {
if (auth.jwtAlgorithm == "HS256" || auth.jwtAlgorithm == "HS512") {
const key = (auth.jwtKey || "").trim();
if (key == "") {
throw new Error("JWT key is empty");
}
const binaryDerString = window.atob(key);
const binaryDer = new Uint8Array(binaryDerString.length);
for (let i = 0; i < binaryDerString.length; i++) {
binaryDer[i] = binaryDerString.charCodeAt(i);
}
const hashName = auth.jwtAlgorithm == "HS256" ? "SHA-256" : "SHA-512";
return crypto.subtle.importKey("raw", binaryDer, { name: "HMAC", hash: { name: hashName } }, true, [
"sign",
]);
} else if (auth.jwtAlgorithm == "ES256" || auth.jwtAlgorithm == "ES512") {
const pem = auth.jwtKey
.replace(/-----BEGIN [^-]+-----/, "")
.replace(/-----END [^-]+-----/, "")
.replace(/\s+/g, "");
// const pem = key.replace(/\s/g, "");
const binaryDerString = window.atob(pem);
const binaryDer = new Uint8Array(binaryDerString.length);
for (let i = 0; i < binaryDerString.length; i++) {
binaryDer[i] = binaryDerString.charCodeAt(i);
}
// const binaryDer = base64ToArrayBuffer(pem);
const namedCurve = auth.jwtAlgorithm == "ES256" ? "P-256" : "P-521";
const param = { name: "ECDSA", namedCurve };
return crypto.subtle.importKey("pkcs8", binaryDer, param, true, ["sign"]);
} else {
throw new Error("Supplied JWT algorithm is not supported.");
}
}
_currentCryptoKey = new Refiner<JWTCredentials, CryptoKey>({
evaluation: async (auth, previous) => {
return await this._importKey(auth);
},
});
_jwt = new Refiner<JWTParams, PreparedJWT>({
evaluation: async (params, previous) => {
const encodedHeader = btoa(JSON.stringify(params.header));
const encodedPayload = btoa(JSON.stringify(params.payload));
const buff = `${encodedHeader}.${encodedPayload}`.replace(/\+/g, "-").replace(/\//g, "_").replace(/=/g, "");
const key = await this._currentCryptoKey.update(params.credentials).value;
let token = "";
if (params.header.alg == "ES256" || params.header.alg == "ES512") {
const jwt = await crypto.subtle.sign(
{ name: "ECDSA", hash: { name: "SHA-256" } },
key,
writeString(buff)
);
token = (await arrayBufferToBase64Single(jwt))
.replace(/\+/g, "-")
.replace(/\//g, "_")
.replace(/=/g, "");
} else if (params.header.alg == "HS256" || params.header.alg == "HS512") {
const jwt = await crypto.subtle.sign(
{ name: "HMAC", hash: { name: params.header.alg } },
key,
writeString(buff)
);
token = (await arrayBufferToBase64Single(jwt))
.replace(/\+/g, "-")
.replace(/\//g, "_")
.replace(/=/g, "");
} else {
throw new Error("JWT algorithm is not supported.");
}
return {
...params,
token: `${buff}.${token}`,
} as PreparedJWT;
},
});
_jwtParams = new Refiner<JWTCredentials, JWTParams>({
evaluation(source, previous) {
const kid = source.jwtKid || undefined;
const sub = (source.jwtSub || "").trim();
if (sub == "") {
throw new Error("JWT sub is empty");
}
const algorithm = source.jwtAlgorithm || "";
if (!algorithm) {
throw new Error("JWT algorithm is not configured.");
}
if (algorithm != "HS256" && algorithm != "HS512" && algorithm != "ES256" && algorithm != "ES512") {
throw new Error("JWT algorithm is not supported.");
}
const header: JWTHeader = {
alg: source.jwtAlgorithm || "HS256",
typ: "JWT",
kid,
};
const iat = ~~(new Date().getTime() / 1000);
const exp = iat + (source.jwtExpDuration || 5) * 60; // 5 minutes
const payload = {
exp,
iat,
sub: source.jwtSub || "",
"_couchdb.roles": ["_admin"],
} satisfies JWTPayload;
return {
header,
payload,
credentials: source,
};
},
shouldUpdate(isDifferent, source, previous) {
if (isDifferent) {
return true;
}
if (!previous) {
return true;
}
// if expired.
const d = ~~(new Date().getTime() / 1000);
if (previous.payload.exp < d) {
// console.warn(`jwt expired ${previous.payload.exp} < ${d}`);
return true;
}
return false;
},
});
async $$connectRemoteCouchDB(
uri: string,
auth: CouchDBCredentials,
@@ -253,25 +111,14 @@ export class ModuleObsidianAPI extends AbstractObsidianModule implements IObsidi
if (!isValidRemoteCouchDBURI(uri)) return "Remote URI is not valid";
if (uri.toLowerCase() != uri) return "Remote URI and database name could not contain capital letters.";
if (uri.indexOf(" ") !== -1) return "Remote URI and database name could not contain spaces.";
let authHeader = "";
if ("username" in auth) {
const userNameAndPassword = auth.username && auth.password ? `${auth.username}:${auth.password}` : "";
if (this.authHeaderSource.value != userNameAndPassword) {
this.authHeaderSource.value = userNameAndPassword;
}
authHeader = this.authHeader.value;
} else if ("jwtAlgorithm" in auth) {
const params = await this._jwtParams.update(auth).value;
const jwt = await this._jwt.update(params).value;
const token = jwt.token;
authHeader = `Bearer ${token}`;
}
// let authHeader = await this._authHeader.getAuthorizationHeader(auth);
const conf: PouchDB.HttpAdapter.HttpAdapterConfiguration = {
adapter: "http",
auth: "username" in auth ? auth : undefined,
skip_setup: !performSetup,
fetch: async (url: string | Request, opts?: RequestInit) => {
const authHeader = await this._authHeader.getAuthorizationHeader(auth);
let size = "";
const localURL = url.toString().substring(uri.length);
const method = opts?.method ?? "GET";
@@ -288,27 +135,27 @@ export class ModuleObsidianAPI extends AbstractObsidianModule implements IObsidi
size = ` (${opts_length})`;
}
try {
const headers = new Headers(opts?.headers);
if (customHeaders) {
for (const [key, value] of Object.entries(customHeaders)) {
if (key && value) {
headers.append(key, value);
}
}
}
if (!("username" in auth)) {
headers.append("authorization", authHeader);
}
if (!disableRequestURI && typeof url == "string" && typeof (opts?.body ?? "") == "string") {
return await this.fetchByAPI(url, localURL, method, authHeader, opts);
// Deprecated configuration, only for backward compatibility.
return await this.fetchByAPI(url, localURL, method, authHeader, { ...opts, headers });
}
// --> native Fetch API.
try {
if (customHeaders) {
for (const [key, value] of Object.entries(customHeaders)) {
if (key && value) {
(opts!.headers as Headers).append(key, value);
}
}
// // Issue #407
// (opts!.headers as Headers).append("ngrok-skip-browser-warning", "123");
}
// debugger;
if (!("username" in auth)) {
(opts!.headers as Headers).append("authorization", authHeader);
}
this.plugin.requestCount.value = this.plugin.requestCount.value + 1;
const response: Response = await fetch(url, opts);
const response: Response = await fetch(url, { ...opts, headers });
if (method == "POST" || method == "PUT") {
this.last_successful_post = response.ok;
} else {
@@ -344,7 +191,10 @@ export class ModuleObsidianAPI extends AbstractObsidianModule implements IObsidi
this._log(
"Failed to fetch by native fetch API. Trying to fetch by API to get more information."
);
const resp2 = await this.fetchByAPI(url.toString(), localURL, method, authHeader, opts);
const resp2 = await this.fetchByAPI(url.toString(), localURL, method, authHeader, {
...opts,
headers,
});
if (resp2.status / 100 == 2) {
this._log(
"The request was successful by API. But the native fetch API failed! Please check CORS settings on the remote database!. While this condition, you cannot enable LiveSync",

13
src/modules/features/ModuleSetupObsidian.ts
View File

@@ -67,14 +67,13 @@ export class ModuleSetupObsidian extends AbstractObsidianModule implements IObsi
const fullIndexes = Object.entries(KeyIndexOfSettings) as [keyof ObsidianLiveSyncSettings, number][];
for (const [settingKey, index] of fullIndexes) {
const settingValue = this.settings[settingKey];
if (index < 0) {
// This setting should be ignored.
continue;
}
settingArr[index] = settingValue;
}
const w = encodeAnyArray(settingArr);
// console.warn(w.length)
// console.warn(w);
// const j = decodeAnyArray(w);
// console.warn(j);
// console.warn(`is equal: ${isObjectDifferent(settingArr, j)}`);
const qr = qrcode(0, "L");
const uri = `${configURIBaseQR}${encodeURIComponent(w)}`;
qr.addData(uri);
@@ -90,6 +89,10 @@ export class ModuleSetupObsidian extends AbstractObsidianModule implements IObsi
const fullIndexes = Object.entries(KeyIndexOfSettings) as [keyof ObsidianLiveSyncSettings, number][];
const newSettings = { ...DEFAULT_SETTINGS } as ObsidianLiveSyncSettings;
for (const [settingKey, index] of fullIndexes) {
if (index < 0) {
// This setting should be ignored.
continue;
}
if (index >= settingArr.length) {
// Possibly a new setting added.
continue;

50
src/modules/features/SettingDialogue/ObsidianLiveSyncSettingTab.ts
View File

@@ -32,6 +32,7 @@ import {
delay,
isDocContentSame,
isObjectDifferent,
parseHeaderValues,
readAsBlob,
sizeToHumanReadable,
} from "../../../lib/src/common/utils.ts";
@@ -45,7 +46,7 @@ import {
} from "../../../lib/src/pouchdb/utils_couchdb.ts";
import { testCrypt } from "../../../lib/src/encryption/e2ee_v2.ts";
import ObsidianLiveSyncPlugin from "../../../main.ts";
import { getPath, requestToCouchDB, scheduleTask } from "../../../common/utils.ts";
import { getPath, requestToCouchDBWithCredentials, scheduleTask } from "../../../common/utils.ts";
import { request } from "obsidian";
import { addPrefix, shouldBeIgnored, stripAllPrefixes } from "../../../lib/src/string_and_binary/path.ts";
import MultipleRegExpControl from "./MultipleRegExpControl.svelte";
@@ -83,6 +84,7 @@ import { EVENT_REQUEST_SHOW_HISTORY } from "../../../common/obsidianEvents.ts";
import { LocalDatabaseMaintenance } from "../../../features/LocalDatabaseMainte/CmdLocalDatabaseMainte.ts";
import { mount } from "svelte";
import { getWebCrypto } from "../../../lib/src/mods.ts";
import { generateCredentialObject } from "../../../lib/src/replication/httplib.ts";
export type OnUpdateResult = {
visibility?: boolean;
@@ -1184,11 +1186,16 @@ export class ObsidianLiveSyncSettingTab extends PluginSettingTab {
return;
}
// Tip: Add log for cloudant as Logger($msg("obsidianLiveSyncSettingTab.logServerConfigurationCheck"));
const r = await requestToCouchDB(
const customHeaders = parseHeaderValues(this.editingSettings.couchDB_CustomHeaders);
const credential = generateCredentialObject(this.editingSettings);
const r = await requestToCouchDBWithCredentials(
this.editingSettings.couchDB_URI,
this.editingSettings.couchDB_USER,
this.editingSettings.couchDB_PASSWORD,
window.origin
credential,
window.origin,
undefined,
undefined,
undefined,
customHeaders
);
const responseConfig = r.json;
@@ -1201,13 +1208,14 @@ export class ObsidianLiveSyncSettingTab extends PluginSettingTab {
x.querySelector("button")?.addEventListener("click", () => {
fireAndForget(async () => {
Logger($msg("obsidianLiveSyncSettingTab.logCouchDbConfigSet", { title, key, value }));
const res = await requestToCouchDB(
const res = await requestToCouchDBWithCredentials(
this.editingSettings.couchDB_URI,
this.editingSettings.couchDB_USER,
this.editingSettings.couchDB_PASSWORD,
credential,
undefined,
key,
value
value,
undefined,
customHeaders
);
if (res.status == 200) {
Logger(
@@ -1342,11 +1350,14 @@ export class ObsidianLiveSyncSettingTab extends PluginSettingTab {
// Request header check
const origins = ["app://obsidian.md", "capacitor://localhost", "http://localhost"];
for (const org of origins) {
const rr = await requestToCouchDB(
const rr = await requestToCouchDBWithCredentials(
this.editingSettings.couchDB_URI,
this.editingSettings.couchDB_USER,
this.editingSettings.couchDB_PASSWORD,
org
credential,
org,
undefined,
undefined,
undefined,
customHeaders
);
const responseHeaders = Object.fromEntries(
Object.entries(rr.headers).map((e) => {
@@ -2406,11 +2417,16 @@ The pane also can be launched by \`P2P Replicator\` command from the Command Pal
const REDACTED = "𝑅𝐸𝐷𝐴𝐶𝑇𝐸𝐷";
if (this.editingSettings.remoteType == REMOTE_COUCHDB) {
try {
const r = await requestToCouchDB(
const credential = generateCredentialObject(this.editingSettings);
const customHeaders = parseHeaderValues(this.editingSettings.couchDB_CustomHeaders);
const r = await requestToCouchDBWithCredentials(
this.editingSettings.couchDB_URI,
this.editingSettings.couchDB_USER,
this.editingSettings.couchDB_PASSWORD,
window.origin
credential,
window.origin,
undefined,
undefined,
undefined,
customHeaders
);
Logger(JSON.stringify(r.json, null, 2));

64
updates.md
View File

@@ -10,6 +10,38 @@ Nevertheless, that being said, to be more honest, I still have not decided what
Note: Already you have noticed this, but let me mention it again, this is a significantly large update. If you have noticed anything, please let me know. I will try to fix it as soon as possible (Some address is on my [profile](https://github.com/vrtmrz)).
## 0.24.25
### Improved
- Peer-to-peer synchronisation has been got more robust.
### Fixed
- No longer broken falsy values in settings during set-up by the QR code generation.
### Refactored
- Some `window` references now have pointed to `globalThis`.
- Some sloppy-import has been fixed.
- A server side implementation `Synchromesh` has been suffixed with `deno` instead of `server` now.
## 0.24.24
### Fixed
- No longer broken JSON files including `\n`, during the bucket synchronisation. (#623)
- Custom headers and JWT tokens are now correctly sent to the server during configuration checking. (#624)
### Improved
- Bucket synchronisation has been enhanced for better performance and reliability.
- Now less duplicated chunks are sent to the server.
Note: If you have encountered about too less chunks, please let me know. However, you can send it to the server by `Overwrite remote`.
- Fetching conflicted files from the server is now more reliable.
- Dependent libraries have been updated to the latest version.
- Also, let me know if you have encountered any issues with this update. Especially you are using a device that has been in use for a little longer.
## 0.24.23
### New Feature
@@ -57,41 +89,9 @@ Note: Already you have noticed this, but let me mention it again, this is a sign
- Now we can see the detail of `TypeError` using Obsidian API during remote database access.
## 0.24.19
### New Feature
- Now we can generate a QR Code for transferring the configuration to another device.
- This QR Code can be scanned by the camera app or something QR Code Reader of another device, and via Obsidian URL, the configuration will be transferred.
- Note: This QR Code is not encrypted. So, please be careful when transferring the configuration.
## 0.24.18
### Fixed
- Now no chunk creation errors will be raised after switching `Compute revisions for chunks`.
- Some invisible file can be handled correctly (e.g., `writing-goals-history.csv`).
- Fetching configuration from the server is now saves the configuration immediately (if we are not in the wizard).
### Improved
- Mismatched configuration dialogue is now more informative, and rewritten to more user-friendly.
- Applying configuration mismatch is now without rebuilding (at our own risks).
- Now, rebuilding is decided more fine grained.
### Improved internally
- Translations can be nested. i.e., task:`Some procedure`, check: `%{task} checking`, checkfailed: `%{check} failed` produces `Some procedure checking failed`.
- Max to 10 levels of nesting
## 0.24.17
Confession. I got the default values wrong. So scary and sorry.
### Behaviour and default changed
- **NOW INDEED AND ACTUALLY** `Compute revisions for chunks` are backed into enabled again. it is necessary for garbage collection of chunks.
- As far as existing users are concerned, this will not automatically change, but the Doctor will inform us.
Older notes are in [updates_old.md](https://github.com/vrtmrz/obsidian-livesync/blob/main/updates_old.md).

29
updates_old.md
View File

@@ -13,7 +13,36 @@ Finally, I would like to once again express my respect and gratitude to all of y
Thank you, and I hope your troubles will be resolved!
---
## 0.24.19
### New Feature
- Now we can generate a QR Code for transferring the configuration to another device.
- This QR Code can be scanned by the camera app or something QR Code Reader of another device, and via Obsidian URL, the configuration will be transferred.
- Note: This QR Code is not encrypted. So, please be careful when transferring the configuration.
## 0.24.18
### Fixed
- Now no chunk creation errors will be raised after switching `Compute revisions for chunks`.
- Some invisible file can be handled correctly (e.g., `writing-goals-history.csv`).
- Fetching configuration from the server is now saves the configuration immediately (if we are not in the wizard).
### Improved
- Mismatched configuration dialogue is now more informative, and rewritten to more user-friendly.
- Applying configuration mismatch is now without rebuilding (at our own risks).
- Now, rebuilding is decided more fine grained.
### Improved internally
- Translations can be nested. i.e., task:`Some procedure`, check: `%{task} checking`, checkfailed: `%{check} failed` produces `Some procedure checking failed`.
- Max to 10 levels of nesting
## 0.24.17
Confession. I got the default values wrong. So scary and sorry.
## 0.24.16