public/obsidian-livesync
1
0
Fork 0
mirror of https://github.com/vrtmrz/obsidian-livesync.git synced 2026-02-22 20:18:48 +00:00
Code Issues Packages Projects Releases Wiki Activity

Compare commits

base: public:0.16.3
Branches Tags
public:main public:beta public:snyk-upgrade-7fd31e6f474568014502e7205847f257 public:snyk-fix-90f3b71450f7543a1c61724d8afa0e71 public:refactor_services_2 public:refactor_services public:dev public:snyk-upgrade-fe9625385aa4be849c7157033fd2abb0 public:snyk-upgrade-99a31e70c822894595544b0c9813b11a public:svelteui public:disenchant public:snyk-fix-61f76ef0bce2a276df9552c2fc80a14f public:snyk-upgrade-2baa7938ba318464c3bad5c3fd5cbda5 public:snyk-upgrade-7af62a575eab20e8fb4e9565bf1adcfc public:snyk-upgrade-b32f281b31a81483f126b8fb6c54a957 public:snyk-upgrade-bd61b1103442d3faecc9fc726c34e9a4 public:snyk-upgrade-5b2d6584430c061b5187611a5d7fb64a public:snyk-fix-0e9fbbcb374fcb8d5e8cb027d8414fb7 public:snyk-upgrade-5242daf91e706d1ae500c9ea6545636e public:snyk-upgrade-70434a6822fbad83424135dfad9df909 public:snyk-upgrade-8fa01ccad0352125f1866d41e57d35e1 public:snyk-upgrade-39eb97b4655e70091f718a88fccd57c1 public:snyk-upgrade-2b99d54d19e207887e36c3071b796925 public:forprivacy public:snyk-upgrade-dad9088d0a333677e0157e3deef7fc38 public:snyk-upgrade-b9a76d74b3e5723759af5b3319aabb80 public:snyk-upgrade-7da6459d8ff5d293d80de9d80061d912 public:snyk-upgrade-6a3786fc4cbe394542e9d1b6058c46e7 public:snyk-upgrade-8fc9e62771dc41b0f23dc0efefb5c949
public:0.25.43-patched-9 public:0.25.43-patched-8 public:0.25.43-patched-7 public:0.25.43-patched-6 public:0.25.43-patched-5 public:0.25.43-patched-4 public:0.25.43-patched-3 public:0.25.43-patched-2 public:0.25.43-patched-1 public:0.25.43 public:0.25.42 public:0.25.41-patched-3 public:0.25.41-patched-2 public:0.25.41-patched-1 public:0.25.41 public:0.25.40 public:0.25.39 public:0.25.38 public:0.25.37 public:0.25.36 public:0.25.35 public:0.25.34 public:0.25.33 public:0.25.32 public:0.25.31 public:0.25.30 public:0.25.30-dev public:0.25.29 public:0.25.28 public:0.25.27 public:0.25.26 public:0.25.25 public:0.25.24 public:0.25.24.beta3 public:0.25.24.beta2 public:0.25.24.beta1 public:0.25.23 public:0.25.23.beta1 public:0.25.22 public:0.25.21 public:0.25.21.beta2 public:0.25.21.beta1 public:0.25.20 public:0.25.19 public:0.25.18 public:0.25.17 public:0.25.16 public:0.25.15 public:0.25.14 public:0.25.13 public:0.25.12 public:0.25.11 public:0.25.10 public:0.25.9 public:0.25.8 public:0.25.7 public:0.25.6 public:0.25.5 public:0.25.4 public:0.25.3 public:0.25.2 public:0.25.1 public:0.25.0 public:0.25.0-beta1 public:0.24.31 public:0.24.30 public:0.24.29 public:0.24.28 public:0.24.27 public:0.24.26 public:0.24.25 public:0.24.24 public:0.24.23 public:0.24.22 public:0.24.21 public:0.24.20 public:0.24.19 public:0.24.18 public:0.24.17 public:0.24.16 public:0.24.15 public:0.24.14 public:0.24.13 public:0.24.12 public:0.24.11 public:0.24.10 public:0.24.9 public:0.24.8 public:0.24.7 public:0.24.6 public:0.24.5 public:0.24.4 public:0.24.3 public:0.24.2 public:0.24.1 public:0.24.0 public:0.24.0.dev-rc8 public:0.24.0.dev-rc7 public:0.24.0.dev-rc6 public:0.24.0.dev-rc5 public:0.24.0.dev-rc4 public:0.24.0.dev-rc3 public:0.24.0.dev-rc2 public:0.23.23 public:0.23.22 public:0.23.21 public:0.23.20 public:0.23.19 public:0.23.18 public:0.23.17 public:0.23.16 public:0.23.15 public:0.23.14 public:0.23.13 public:0.23.12 public:0.23.11 public:0.23.10 public:0.23.9 public:0.23.8 public:0.23.7 public:0.23.6 public:0.23.5 public:0.23.4 public:0.23.3 public:0.23.2 public:0.23.1 public:0.23.0 public:0.22.19 public:0.22.18 public:0.22.17 public:0.22.16 public:0.22.15 public:0.22.14 public:0.22.13 public:0.22.12 public:0.22.11 public:0.22.10 public:0.22.9 public:0.22.8 public:0.22.7 public:0.22.6 public:0.22.5 public:0.22.4 public:0.22.3 public:0.22.2 public:0.22.1 public:0.22.0 public:0.21.5 public:0.21.4 public:0.21.3 public:0.21.2 public:0.21.1 public:0.21.0 public:0.20.7 public:0.20.6 public:0.20.5 public:0.20.4 public:0.20.3 public:0.20.2 public:0.20.1 public:0.20.0 public:0.19.23 public:0.19.22 public:0.19.21 public:0.19.20 public:0.19.19 public:0.19.18 public:0.19.17 public:0.19.16 public:0.19.15 public:0.19.14 public:0.19.13 public:0.19.12 public:0.19.11 public:0.19.10 public:0.19.9 public:0.19.8 public:0.19.7 public:0.19.6 public:0.19.5 public:0.19.4 public:0.19.3 public:0.19.2 public:0.19.1 public:0.19.0 public:0.18.6 public:0.18.5 public:0.18.4 public:0.18.3 public:0.18.2 public:0.18.1 public:0.18.0 public:0.17.34 public:0.17.33 public:0.17.32 public:0.17.31 public:0.17.30 public:0.17.29 public:0.17.28 public:0.17.27 public:0.17.26 public:0.17.25 public:0.17.24 public:0.17.23 public:0.17.22 public:0.17.21 public:0.17.20 public:0.17.19 public:0.17.18 public:0.17.17 public:0.17.16 public:0.17.15 public:0.17.14 public:0.17.13 public:0.17.12 public:0.17.11 public:0.17.10 public:0.17.9 public:0.17.8 public:0.17.7 public:0.17.6 public:0.17.5 public:0.17.4 public:0.17.3 public:0.17.2 public:0.17.1 public:0.17.0 public:0.16.8 public:0.16.7 public:0.16.6 public:0.16.5 public:0.16.4 public:0.16.3 public:0.16.2 public:0.16.1 public:0.16.0 public:0.15.10 public:0.15.9 public:0.15.8 public:0.15.7 public:0.15.6 public:0.15.5 public:0.15.4 public:0.15.3 public:0.15.2 public:0.15.1 public:0.15.0 public:0.14.8 public:0.14.7 public:0.14.6 public:0.14.5 public:0.14.4 public:0.14.3 public:0.14.2 public:0.14.1 public:0.14.0 public:0.13.4 public:0.13.3 public:0.13.2 public:0.13.1 public:0.13.0 public:0.12.5 public:0.12.4 public:0.12.3 public:0.12.2 public:0.12.1 public:0.12.0 public:0.11.10 public:0.11.9 public:0.11.8 public:0.11.7 public:0.11.6 public:0.11.5 public:0.11.4 public:0.11.3 public:0.11.2 public:0.11.1 public:0.11.0 public:0.10.1 public:0.10.0 public:0.9.0 public:0.8.8 public:0.8.7 public:0.8.6 public:0.8.5 public:0.8.3 public:0.8.2 public:0.8.1 public:0.8.0 public:0.7.2 public:0.7.1 public:0.7.0 public:0.6.1 public:0.6.0 public:0.5.0 public:0.4.1 public:0.4.0 public:0.3.1 public:0.3.0 public:0.2.1 public:0.2.0 public:0.1.26 public:0.1.25 public:0.1.24 public:0.1.23 public:0.1.22 public:0.1.21 public:0.1.20 public:0.1.19 public:0.1.18 public:0.1.17 public:0.1.16 public:0.1.15 public:0.1.14 public:0.1.13 public:0.1.12 public:0.1.11 public:0.1.10 public:0.1.9 public:0.1.8 public:0.1.7 public:0.1.6 public:0.1.5 public:0.1.4 public:0.1.3 public:0.1.2 public:0.1.1 public:0.1.0 public:0.0.9 public:0.0.8 public:0.0.7 public:0.0.6
...
compare: public:0.17.8
Branches Tags
public:beta public:main public:snyk-upgrade-7fd31e6f474568014502e7205847f257 public:snyk-fix-90f3b71450f7543a1c61724d8afa0e71 public:refactor_services_2 public:refactor_services public:dev public:snyk-upgrade-fe9625385aa4be849c7157033fd2abb0 public:snyk-upgrade-99a31e70c822894595544b0c9813b11a public:svelteui public:disenchant public:snyk-fix-61f76ef0bce2a276df9552c2fc80a14f public:snyk-upgrade-2baa7938ba318464c3bad5c3fd5cbda5 public:snyk-upgrade-7af62a575eab20e8fb4e9565bf1adcfc public:snyk-upgrade-b32f281b31a81483f126b8fb6c54a957 public:snyk-upgrade-bd61b1103442d3faecc9fc726c34e9a4 public:snyk-upgrade-5b2d6584430c061b5187611a5d7fb64a public:snyk-fix-0e9fbbcb374fcb8d5e8cb027d8414fb7 public:snyk-upgrade-5242daf91e706d1ae500c9ea6545636e public:snyk-upgrade-70434a6822fbad83424135dfad9df909 public:snyk-upgrade-8fa01ccad0352125f1866d41e57d35e1 public:snyk-upgrade-39eb97b4655e70091f718a88fccd57c1 public:snyk-upgrade-2b99d54d19e207887e36c3071b796925 public:forprivacy public:snyk-upgrade-dad9088d0a333677e0157e3deef7fc38 public:snyk-upgrade-b9a76d74b3e5723759af5b3319aabb80 public:snyk-upgrade-7da6459d8ff5d293d80de9d80061d912 public:snyk-upgrade-6a3786fc4cbe394542e9d1b6058c46e7 public:snyk-upgrade-8fc9e62771dc41b0f23dc0efefb5c949
public:0.25.43-patched-9 public:0.25.43-patched-8 public:0.25.43-patched-7 public:0.25.43-patched-6 public:0.25.43-patched-5 public:0.25.43-patched-4 public:0.25.43-patched-3 public:0.25.43-patched-2 public:0.25.43-patched-1 public:0.25.43 public:0.25.42 public:0.25.41-patched-3 public:0.25.41-patched-2 public:0.25.41-patched-1 public:0.25.41 public:0.25.40 public:0.25.39 public:0.25.38 public:0.25.37 public:0.25.36 public:0.25.35 public:0.25.34 public:0.25.33 public:0.25.32 public:0.25.31 public:0.25.30 public:0.25.30-dev public:0.25.29 public:0.25.28 public:0.25.27 public:0.25.26 public:0.25.25 public:0.25.24 public:0.25.24.beta3 public:0.25.24.beta2 public:0.25.24.beta1 public:0.25.23 public:0.25.23.beta1 public:0.25.22 public:0.25.21 public:0.25.21.beta2 public:0.25.21.beta1 public:0.25.20 public:0.25.19 public:0.25.18 public:0.25.17 public:0.25.16 public:0.25.15 public:0.25.14 public:0.25.13 public:0.25.12 public:0.25.11 public:0.25.10 public:0.25.9 public:0.25.8 public:0.25.7 public:0.25.6 public:0.25.5 public:0.25.4 public:0.25.3 public:0.25.2 public:0.25.1 public:0.25.0 public:0.25.0-beta1 public:0.24.31 public:0.24.30 public:0.24.29 public:0.24.28 public:0.24.27 public:0.24.26 public:0.24.25 public:0.24.24 public:0.24.23 public:0.24.22 public:0.24.21 public:0.24.20 public:0.24.19 public:0.24.18 public:0.24.17 public:0.24.16 public:0.24.15 public:0.24.14 public:0.24.13 public:0.24.12 public:0.24.11 public:0.24.10 public:0.24.9 public:0.24.8 public:0.24.7 public:0.24.6 public:0.24.5 public:0.24.4 public:0.24.3 public:0.24.2 public:0.24.1 public:0.24.0 public:0.24.0.dev-rc8 public:0.24.0.dev-rc7 public:0.24.0.dev-rc6 public:0.24.0.dev-rc5 public:0.24.0.dev-rc4 public:0.24.0.dev-rc3 public:0.24.0.dev-rc2 public:0.23.23 public:0.23.22 public:0.23.21 public:0.23.20 public:0.23.19 public:0.23.18 public:0.23.17 public:0.23.16 public:0.23.15 public:0.23.14 public:0.23.13 public:0.23.12 public:0.23.11 public:0.23.10 public:0.23.9 public:0.23.8 public:0.23.7 public:0.23.6 public:0.23.5 public:0.23.4 public:0.23.3 public:0.23.2 public:0.23.1 public:0.23.0 public:0.22.19 public:0.22.18 public:0.22.17 public:0.22.16 public:0.22.15 public:0.22.14 public:0.22.13 public:0.22.12 public:0.22.11 public:0.22.10 public:0.22.9 public:0.22.8 public:0.22.7 public:0.22.6 public:0.22.5 public:0.22.4 public:0.22.3 public:0.22.2 public:0.22.1 public:0.22.0 public:0.21.5 public:0.21.4 public:0.21.3 public:0.21.2 public:0.21.1 public:0.21.0 public:0.20.7 public:0.20.6 public:0.20.5 public:0.20.4 public:0.20.3 public:0.20.2 public:0.20.1 public:0.20.0 public:0.19.23 public:0.19.22 public:0.19.21 public:0.19.20 public:0.19.19 public:0.19.18 public:0.19.17 public:0.19.16 public:0.19.15 public:0.19.14 public:0.19.13 public:0.19.12 public:0.19.11 public:0.19.10 public:0.19.9 public:0.19.8 public:0.19.7 public:0.19.6 public:0.19.5 public:0.19.4 public:0.19.3 public:0.19.2 public:0.19.1 public:0.19.0 public:0.18.6 public:0.18.5 public:0.18.4 public:0.18.3 public:0.18.2 public:0.18.1 public:0.18.0 public:0.17.34 public:0.17.33 public:0.17.32 public:0.17.31 public:0.17.30 public:0.17.29 public:0.17.28 public:0.17.27 public:0.17.26 public:0.17.25 public:0.17.24 public:0.17.23 public:0.17.22 public:0.17.21 public:0.17.20 public:0.17.19 public:0.17.18 public:0.17.17 public:0.17.16 public:0.17.15 public:0.17.14 public:0.17.13 public:0.17.12 public:0.17.11 public:0.17.10 public:0.17.9 public:0.17.8 public:0.17.7 public:0.17.6 public:0.17.5 public:0.17.4 public:0.17.3 public:0.17.2 public:0.17.1 public:0.17.0 public:0.16.8 public:0.16.7 public:0.16.6 public:0.16.5 public:0.16.4 public:0.16.3 public:0.16.2 public:0.16.1 public:0.16.0 public:0.15.10 public:0.15.9 public:0.15.8 public:0.15.7 public:0.15.6 public:0.15.5 public:0.15.4 public:0.15.3 public:0.15.2 public:0.15.1 public:0.15.0 public:0.14.8 public:0.14.7 public:0.14.6 public:0.14.5 public:0.14.4 public:0.14.3 public:0.14.2 public:0.14.1 public:0.14.0 public:0.13.4 public:0.13.3 public:0.13.2 public:0.13.1 public:0.13.0 public:0.12.5 public:0.12.4 public:0.12.3 public:0.12.2 public:0.12.1 public:0.12.0 public:0.11.10 public:0.11.9 public:0.11.8 public:0.11.7 public:0.11.6 public:0.11.5 public:0.11.4 public:0.11.3 public:0.11.2 public:0.11.1 public:0.11.0 public:0.10.1 public:0.10.0 public:0.9.0 public:0.8.8 public:0.8.7 public:0.8.6 public:0.8.5 public:0.8.3 public:0.8.2 public:0.8.1 public:0.8.0 public:0.7.2 public:0.7.1 public:0.7.0 public:0.6.1 public:0.6.0 public:0.5.0 public:0.4.1 public:0.4.0 public:0.3.1 public:0.3.0 public:0.2.1 public:0.2.0 public:0.1.26 public:0.1.25 public:0.1.24 public:0.1.23 public:0.1.22 public:0.1.21 public:0.1.20 public:0.1.19 public:0.1.18 public:0.1.17 public:0.1.16 public:0.1.15 public:0.1.14 public:0.1.13 public:0.1.12 public:0.1.11 public:0.1.10 public:0.1.9 public:0.1.8 public:0.1.7 public:0.1.6 public:0.1.5 public:0.1.4 public:0.1.3 public:0.1.2 public:0.1.1 public:0.1.0 public:0.0.9 public:0.0.8 public:0.0.7 public:0.0.6

34 Commits
0.16.3 ... 0.17.8

Author SHA1 Message Date
vorotamoroz
2100e22276 bump 2022-12-27 18:12:00 +09:00
vorotamoroz
ec08dc5fe8 Improved: 
- Performance improved
  Prebuilt PouchDB is no longer used.

Fixed:
- Merging hidden files is also fixed.

New Feature:
- Now we can synchronise automatically after merging conflicts.
 2022-12-27 18:09:51 +09:00
vorotamoroz
c92e94e552 fix eslint configuration 2022-12-27 17:58:42 +09:00
vorotamoroz
c7db8592c6 bump 2022-12-26 16:16:46 +09:00
vorotamoroz
fc3617d9f9 Fixed: 
- Fixed merging issues.
- Fixed button styling.

Changed:
- Default behaviour of conflict checking on synchronising.
 2022-12-26 16:12:57 +09:00
vorotamoroz
34c1b040db bump 2022-12-24 21:05:32 +09:00
vorotamoroz
6b85aecafe Fixed: 
- Now our renamed/deleted files have been surely deleted again.
 2022-12-24 21:05:09 +09:00
vorotamoroz
4dabadd5ea A little better. 2022-12-22 18:02:12 +09:00
vorotamoroz
0619c96c48 bump 2022-12-22 17:59:30 +09:00
vorotamoroz
b0f612b61c New feature: 
- Use dynamic iteration count
Fixed:
- Read chunks online will fetch the remote chunks correctly.
- Read chunks online will save fetched chunks to the local database.
 2022-12-22 17:58:52 +09:00
vorotamoroz
81caad8602 Canvases are now treated as a sort of plain text file. 2022-12-21 15:09:10 +09:00
vorotamoroz
f5e28b5e1c bump 2022-12-21 14:51:59 +09:00
vorotamoroz
0c206226b1 New feature 
- JSON merging for data and canvas.
 2022-12-21 14:51:39 +09:00
vorotamoroz
1ad5dcc1cc bump 2022-12-16 18:56:28 +09:00
vorotamoroz
a512566e5b New feature 
- We can merge conflicted documents automatically if sensible.

Fixed:
- Writing to the storage will be pended while they have conflicts after replication.

Minor changes included.
 2022-12-16 18:55:04 +09:00
vorotamoroz
02de82af46 bump 2022-12-06 18:03:31 +09:00
vorotamoroz
840e03a2d3 Fixed: 
- Now we can verify and repair database again.
 2022-12-06 17:59:48 +09:00
vorotamoroz
96b676caf3 bump 2022-12-05 19:53:24 +09:00
vorotamoroz
a8219de375 Improved: 
- Splitting markdown
- Saving chunks

Changed:
- Chunk ID numbering rules

Fixed:
- Just weed.
 2022-12-05 19:37:24 +09:00
vorotamoroz
db3eb7e1a0 bump 2022-11-24 14:14:27 +09:00
vorotamoroz
50f51393fc upgrade lib. 2022-11-24 14:14:17 +09:00
vorotamoroz
8a04e332d6 Fix check warning for max_document_size, max_http_request_size as like as #145 2022-11-23 15:41:09 +09:00
vorotamoroz
12ae17aa2f Merge pull request #145 from Bpazy/patch-1 
Fix check warning for max_document_size, max_http_request_size
 2022-11-23 15:37:32 +09:00
Ziyuan Han
657f12f966 Fix check warning 2022-11-23 14:12:50 +08:00
Ziyuan Han
15a7bed448 Fix check warning 2022-11-23 14:11:44 +08:00
vorotamoroz
420c3b94df bump 2022-11-23 10:34:04 +09:00
vorotamoroz
239c087132 framework and dependency upgraded. 2022-11-23 10:27:12 +09:00
vorotamoroz
d1a633c799 bump 2022-11-07 17:26:40 +09:00
vorotamoroz
1c07cd92fc - Fixed 
- Automatic (temporary) batch size adjustment has been restored to work correctly.
  - Chunk splitting has been backed to the previous behaviour for saving them correctly.
- Improved
  - Corrupted chunks will be detected automatically.
  - Now on the case-insensitive system, `aaa.md` and `AAA.md` will be treated as the same file or path at applying changesets.
 2022-11-07 17:26:33 +09:00
vorotamoroz
adc84d53b1 bump again 2022-10-27 17:43:00 +09:00
vorotamoroz
c3a762ceed bump 2022-10-27 17:42:15 +09:00
vorotamoroz
5945638633 Fixed: 
- Conflict detection and merging of deleted files.
- Fixed wrong logs.
- Fix redundant logs.

Implemented
- Automatically deletion of old metadata.
 2022-10-27 17:41:26 +09:00
vorotamoroz
331acd463d bump 2022-10-25 11:48:21 +09:00
vorotamoroz
9d4f41bbf9 Fixed failure of detection 2022-10-25 11:47:02 +09:00
20 changed files with 2779 additions and 1831 deletions
Expand all files Collapse all files

5
.eslintignore
View File

@@ -1,3 +1,4 @@
npm node_modules
node_modules
build
.eslintrc.js.bak
.eslintrc.js.bak
src/lib/src/patches/pouchdb-utils

32
README.md
View File

@@ -2,7 +2,7 @@
[Japanese docs](./README_ja.md) [Chinese docs](./README_cn.md).
Self-hosted LiveSync is a community implemented synchronization plugin.
Self-hosted LiveSync is a community-implemented synchronization plugin.
A self-hosted or purchased CouchDB acts as the intermediate server. Available on every obsidian-compatible platform.
Note: It has no compatibility with the official "Obsidian Sync".
@@ -16,19 +16,19 @@ Before installing or upgrading LiveSync, please back your vault up.
- Visual conflict resolver included.
- Bidirectional synchronization between devices nearly in real-time
- You can use CouchDB or its compatibles like IBM Cloudant.
- End-to-End encryption supported.
- End-to-End encryption is supported.
- Plugin synchronization(Beta)
- Receive WebClip from [obsidian-livesync-webclip](https://chrome.google.com/webstore/detail/obsidian-livesync-webclip/jfpaflmpckblieefkegjncjoceapakdf) (End-to-End encryption will not be applicable.)
Useful for researchers, engineers and developers with a need to keep their notes fully self-hosted for security reasons. Or just anyone who would like the peace of mind knowing that their notes are fully private.
Useful for researchers, engineers and developers with a need to keep their notes fully self-hosted for security reasons. Or just anyone who would like the peace of mind of knowing that their notes are fully private.
## IMPORTANT NOTICE
- Do not use in conjunction with another synchronization solution (including iCloud, Obsidian Sync). Before enabling this plugin, make sure to disable all the other synchronization methods to avoid content corruption or duplication. If you want to synchronize to two or more services, do them one by one and never enable two synchronization methods at the same time.
- Do not enable this plugin with another synchronization solution at the same time (including iCloud and Obsidian Sync). Before enabling this plugin, make sure to disable all the other synchronization methods to avoid content corruption or duplication. If you want to synchronize to two or more services, do them one by one and never enable two synchronization methods at the same time.
This includes not putting your vault inside a cloud-synchronized folder (eg. an iCloud folder or Dropbox folder)
- This is a synchronization plugin. Not a backup solutions. Do not rely on this for backup.
- This is a synchronization plugin. Not a backup solution. Do not rely on this for backup.
- If the device's storage runs out, database corruption may happen.
- Hidden files or any other invisible files wouldn't be kept in the database, thus won't be synchronized. (**and may also get deleted**)
- Hidden files or any other invisible files wouldn't be kept in the database, and thus won't be synchronized. (**and may also get deleted**)
## How to use
@@ -38,7 +38,7 @@ First, get your database ready. IBM Cloudant is preferred for testing. Or you ca
1. [Setup IBM Cloudant](docs/setup_cloudant.md)
2. [Setup your CouchDB](docs/setup_own_server.md)
Note: More information about alternative hosting methods needed! Currently, [using fly.io](https://github.com/vrtmrz/obsidian-livesync/discussions/85) is being discussed.
Note: More information about alternative hosting methods is needed! Currently, [using fly.io](https://github.com/vrtmrz/obsidian-livesync/discussions/85) is being discussed.
### Configure the plugin
@@ -46,13 +46,13 @@ See [Quick setup guide](doccs/../docs/quick_setup.md)
## Something looks corrupted...
Please open the configuration link again and Answer as below:
Please open the configuration link again and Answer below:
- If your local database looks corrupted (in other words, when your Obsidian getting weird even standalone.)
- Answer `No` to `Keep local DB?`
- If your remote database looks corrupted (in other words, when something happens while replicating)
- Answer `No` to `Keep remote DB?`
If you answered `No` to both, your databases will be rebuilt by the content on your device. And the remote database will lock out other devices. You have to synchronize all your devices again. (When this time, almost all your files should be synchronized with a timestamp. So you can use a existed vault).
If you answered `No` to both, your databases will be rebuilt by the content on your device. And the remote database will lock out other devices. You have to synchronize all your devices again. (When this time, almost all your files should be synchronized with a timestamp. So you can use an existing vault).
## Test Server
@@ -78,18 +78,18 @@ If you have deleted or renamed files, please wait until ⏳ icon disappeared.
## Hints
- If a folder becomes empty after a replication, it will be deleted by default. But you can toggle this behaviour. Check the [Settings](docs/settings.md).
- LiveSync mode drains more batteries in mobile devices. Periodic sync with some automatic sync is recommended.
- Mobile Obsidian can not connect to a non-secure (HTTP) or a locally-signed servers, even if the root certificate is installed on the device.
- Mobile Obsidian can not connect to non-secure (HTTP) or locally-signed servers, even if the root certificate is installed on the device.
- There are no 'exclude_folders' like configurations.
- While synchronizing, files are compared by their modification time and the older ones will be overwritten by the newer ones. Then plugin checks for conflicts and if a merge is needed, a dialog will open.
- Rarely, a file in the database could be corrupted. The plugin will not write to local storage when a file looks corrupted. If a local version of the file is on your device, the corruption could be fixed by editing the local file and synchronizing it. But if the file does not exist on any of your devices, then it can not be rescued. In this case you can delete these items from the settings dialog.
- To stop the boot up sequence (eg. for fixing problems on databases), you can put a `redflag.md` file at the root of your vault.
- Q: Database is growing, how can I shrink it down?
A: each of the docs is saved with their past 100 revisions for detecting and resolving conflicts. Picturing that one device has been offline for a while, and comes online again. The device has to compare its notes with the remotely saved ones. If there exists a historic revision in which the note used to be identical, it could be updated safely (like git fast-forward). Even if that is not in revision histories, we only have to check the differences after the revision that both devices commonly have. This is like git's conflict resolving method. So, We have to make the database again like an enlarged git repo if you want to solve the root of the problem.
- And more technical Information are in the [Technical Information](docs/tech_info.md)
- Rarely, a file in the database could be corrupted. The plugin will not write to local storage when a file looks corrupted. If a local version of the file is on your device, the corruption could be fixed by editing the local file and synchronizing it. But if the file does not exist on any of your devices, then it can not be rescued. In this case, you can delete these items from the settings dialog.
- To stop the boot-up sequence (eg. for fixing problems on databases), you can put a `redflag.md` file at the root of your vault.
- Q: The database is growing, how can I shrink it down?
A: each of the docs is saved with their past 100 revisions for detecting and resolving conflicts. Picturing that one device has been offline for a while, and comes online again. The device has to compare its notes with the remotely saved ones. If there exists a historic revision in which the note used to be identical, it could be updated safely (like git fast-forward). Even if that is not in revision histories, we only have to check the differences after the revision that both devices commonly have. This is like git's conflict-resolving method. So, We have to make the database again like an enlarged git repo if you want to solve the root of the problem.
- And more technical Information is in the [Technical Information](docs/tech_info.md)
- If you want to synchronize files without obsidian, you can use [filesystem-livesync](https://github.com/vrtmrz/filesystem-livesync).
- WebClipper is also available on Chrome Web Store:[obsidian-livesync-webclip](https://chrome.google.com/webstore/detail/obsidian-livesync-webclip/jfpaflmpckblieefkegjncjoceapakdf)
Repo is here: [obsidian-livesync-webclip](https://github.com/vrtmrz/obsidian-livesync-webclip). (Docs are work in progress.)
Repo is here: [obsidian-livesync-webclip](https://github.com/vrtmrz/obsidian-livesync-webclip). (Docs are a work in progress.)
## License

12
docs/quick_setup.md
View File

@@ -1,5 +1,5 @@
# Quick setup
The Setup wizard has been implemented since v0.15.0. This simplifies the initial set-up.
The Setup wizard has been implemented since v0.15.0. This simplifies the initial setup.
Note: The subsequent devices should be set up using the `Copy setup URI` and `Open setup URI`.
@@ -34,18 +34,18 @@ Enter the information in the database we have set up.
![](../images/quick_setup_4.png)
If End to End encryption is enabled, the possibility of a third party who does not know the Passphrase being able to read the contents of the Remote database in the event that they are leaked is reduced. So we strongly recommend to enable it.
If End to End encryption is enabled, the possibility of a third party who does not know the Passphrase being able to read the contents of the Remote database if they are leaked is reduced. So we strongly recommend enabling it.
Encryption is based on 256-bit AES-GCM.
This setting can be disabled if you are inside a closed network and it is clear that you will not be accessed by third parties.
### Test database connectionとCheck database configuraion
### Test database connection and Check database configuration
Here we can check the status of the connection to the database and the database settings.
![](../images/quick_setup_5.png)
#### Test Database Connection
Check whether we can connect to the database. If it fails, there are a number of reasons, but once you have done the `Check database configuration`, check if it fails there too.
Check whether we can connect to the database. If it fails, there are several reasons, but once you have done the `Check database configuration`, check if it fails there too.
#### Check database configuration
@@ -64,11 +64,11 @@ Go to the Local Database configuration.
### Discard exist database and proceed
Discard the contents of the Remote database and go to the Local Database configuration.
## Local Database confiuration
## Local Database configuration
![](../images/quick_setup_8.png)
Configure the local database. If we already have a Vaults with Self-hosted LiveSync installed and having same directory name as currently we are setting up, please specify a different suffix than the Vault you have already set up here.
Configure the local database. If we already have a Vaults with Self-hosted LiveSync installed and having the same directory name as currently we are setting up, please specify a different suffix than the Vault you have already set up here.
## Miscellaneous
Finally, finish the miscellaneous configurations and select a preset for synchronisation.

30
docs/settings.md
View File

@@ -17,14 +17,14 @@ If you feel something, please feel free to inform me.
| 🚑 | [Corrupted data](#corrupted-data) |
## Remote Database Configurations
Configure settings of synchronize server. If any synchronization is enabled, you can't edit this section. Please disable all synchronization to change.
Configure the settings of synchronize server. If any synchronization is enabled, you can't edit this section. Please disable all synchronization to change.
### URI
URI of CouchDB. In the case of Cloudant, It's "External Endpoint(preferred)".
**Do not end it up with a slash** when it doesn't contain the database name.
### Username
Your CouchDB's Username. With administrator's privilege is preferred.
Your CouchDB's Username. Administrator's privilege is preferred.
### Password
Your CouchDB's Password.
@@ -47,11 +47,11 @@ The passphrase to used as the key of encryption. Please use the long text.
### Apply
Set the End to End encryption enabled and its passphrase for use in replication.
If you change the passphrase of a existing database, overwriting the remote database is strongly recommended.
If you change the passphrase of an existing database, overwriting the remote database is strongly recommended.
### Overwrite remote database
Overwrite the remote database by the local database using the passphrase you applied.
Overwrite the remote database with the local database using the passphrase you applied.
### Rebuild
@@ -61,17 +61,17 @@ Rebuild remote and local databases with local files. It will delete all document
You can check the connection by clicking this button.
### Check database configuration
You can check and modify your CouchDB's configuration from here directly.
You can check and modify your CouchDB configuration from here directly.
### Lock remote database.
Other devices are banned from the database when you have locked the database.
If you have something troubled with other devices, you can protect the vault and remote database by your device.
If you have something troubled with other devices, you can protect the vault and remote database with your device.
## Local Database Configurations
"Local Database" is created inside your obsidian.
### Batch database update
Delay database update until raise replication, open another file, window visibility changed, or file events except for file modification.
Delay database update until raise replication, open another file, window visibility changes, or file events except for file modification.
This option can not be used with LiveSync at the same time.
@@ -81,23 +81,23 @@ If one device rebuilds or locks the remote database, every other device will be
### minimum chunk size and LongLine threshold
The configuration of chunk splitting.
Self-hosted LiveSync splits the note into chunks for efficient synchronization. This chunk should be longer than "Minimum chunk size".
Self-hosted LiveSync splits the note into chunks for efficient synchronization. This chunk should be longer than the "Minimum chunk size".
Specifically, the length of the chunk is determined by the following orders.
1. Find the nearest newline character, and if it is farther than LongLineThreshold, this piece becomes an independent chunk.
2. If not, find nearest to these items.
1. Newline character
2. Empty line (Windows style)
3. Empty line (non-Windows style)
3. Compare the farther in these 3 positions and next "\[newline\]#" position, pick a shorter piece to as chunk.
2. If not, find the nearest to these items.
1. A newline character
2. An empty line (Windows style)
3. An empty line (non-Windows style)
3. Compare the farther in these 3 positions and the next "newline\]#" position, and pick a shorter piece as a chunk.
This rule was made empirically from my dataset. If this rule acts as badly on your data. Please give me the information.
You can dump saved note structure to `Dump informations of this doc`. Replace every character to x except newline and "#" when sending information to me.
You can dump saved note structure to `Dump informations of this doc`. Replace every character with x except newline and "#" when sending information to me.
Default values are 20 letters and 250 letters.
The default values are 20 letters and 250 letters.
## General Settings

14
docs/setup_own_server.md
View File

@@ -1,7 +1,7 @@
# Setup CouchDB to your server
## Install CouchDB and access from PC or Mac
## Install CouchDB and access from a PC or Mac
The easiest way to set up the CouchDB is using the [docker image]((https://hub.docker.com/_/couchdb)).
@@ -10,9 +10,11 @@ But some additional configurations are required in `local.ini` to use from Self-
```
[couchdb]
single_node=true
max_document_size = 50000000
[chttpd]
require_valid_user = true
max_http_request_size = 4294967296
[chttpd_auth]
require_valid_user = true
@@ -37,7 +39,7 @@ $ docker run --rm -it -e COUCHDB_USER=admin -e COUCHDB_PASSWORD=password -v /pat
*Remember to replace the path with the path to your local.ini*
Note: At this time, the file owner of local.ini became 5984:5984. It's the limitation docker image. please change the owner before editing local.ini again.
If you could confirm that Self-hosted LiveSync can sync with the server, launch docker image as background as you like.
If you could confirm that Self-hosted LiveSync can sync with the server, launch the docker image as a background as you like.
Example to run docker in detached mode:
```
@@ -45,10 +47,10 @@ $ docker run -d --restart always -e COUCHDB_USER=admin -e COUCHDB_PASSWORD=passw
```
*Remember to replace the path with the path to your local.ini*
## Access from mobile device
## Access from a mobile device
If you want to access Self-hosted LiveSync from mobile devices, you need a valid SSL certificate.
### Testing from mobile
### Testing from a mobile
In the testing phase, [localhost.run](http://localhost.run/) or something like services is very useful.
example on using localhost.run)
@@ -92,6 +94,6 @@ Set the A record of your domain to point to your server, and host reverse proxy
Note: Mounting CouchDB on the top directory is not recommended.
Using Caddy is a handy way to serve the server with SSL automatically.
I have published [docker-compose.yml and ini files](https://github.com/vrtmrz/self-hosted-livesync-server) that launches Caddy and CouchDB at once. Please try it out.
I have published [docker-compose.yml and ini files](https://github.com/vrtmrz/self-hosted-livesync-server) that launch Caddy and CouchDB at once. Please try it out.
And, be sure to check the server log and be careful of malicious access.
And, be sure to check the server log and be careful of malicious access.

4
docs/setup_own_server_cn.md
View File

@@ -11,9 +11,11 @@
```
[couchdb]
single_node=true
max_document_size = 50000000
[chttpd]
require_valid_user = true
max_http_request_size = 4294967296
[chttpd_auth]
require_valid_user = true
@@ -92,4 +94,4 @@ Note: 不推荐将 CouchDB 挂载到根目录
提供了 [docker-compose.yml 和 ini 文件](https://github.com/vrtmrz/self-hosted-livesync-server) 可以同时启动 Caddy 和 CouchDB。
注意检查服务器日志，当心恶意访问。
注意检查服务器日志，当心恶意访问。

2
docs/setup_own_server_ja.md
View File

@@ -8,12 +8,14 @@ CouchDBを構築するには、[Dockerのイメージ](https://hub.docker.com/_/
```
[couchdb]
single_node=true
max_document_size = 50000000
[chttpd]
require_valid_user = true
[chttpd_auth]
require_valid_user = true
max_http_request_size = 4294967296
authentication_redirect = /_utils/session.html
[httpd]

4
esbuild.config.mjs
View File

@@ -25,14 +25,16 @@ esbuild
"MANIFEST_VERSION": `"${manifestJson.version}"`,
"PACKAGE_VERSION": `"${packageJson.version}"`,
"UPDATE_INFO": `${updateInfo}`,
"global":"window",
},
external: ["obsidian", "electron", ...builtins],
external: ["obsidian", "electron", "crypto"],
format: "cjs",
watch: !prod,
target: "es2018",
logLevel: "info",
sourcemap: prod ? false : "inline",
treeShaking: true,
platform: "browser",
plugins: [
sveltePlugin({
preprocess: sveltePreprocess(),

2
manifest.json
View File

@@ -1,7 +1,7 @@
{
"id": "obsidian-livesync",
"name": "Self-hosted LiveSync",
"version": "0.16.3",
"version": "0.17.8",
"minAppVersion": "0.9.12",
"description": "Community implementation of self-hosted livesync. Reflect your vault changes to some other devices immediately. Please make sure to disable other synchronize solutions to avoid content corruption or duplication.",
"author": "vorotamoroz",

3539
package-lock.json generated
View File

File diff suppressed because it is too large Load Diff

47
package.json
View File

@@ -1,6 +1,6 @@
{
"name": "obsidian-livesync",
"version": "0.16.3",
"version": "0.17.8",
"description": "Reflect your vault changes to some other devices immediately. Please make sure to disable other synchronize solutions to avoid content corruption or duplication.",
"main": "main.js",
"type": "module",
@@ -13,34 +13,39 @@
"author": "vorotamoroz",
"license": "MIT",
"devDependencies": {
"@rollup/plugin-commonjs": "^18.0.0",
"@rollup/plugin-node-resolve": "^11.2.1",
"@rollup/plugin-typescript": "^8.2.1",
"@types/diff-match-patch": "^1.0.32",
"@types/pouchdb": "^6.4.0",
"@types/pouchdb-browser": "^6.1.3",
"@typescript-eslint/eslint-plugin": "^5.7.0",
"@typescript-eslint/parser": "^5.0.0",
"builtin-modules": "^3.2.0",
"esbuild": "0.13.12",
"esbuild-svelte": "^0.7.0",
"eslint": "^7.32.0",
"eslint-config-airbnb-base": "^14.2.1",
"eslint-plugin-import": "^2.25.2",
"@typescript-eslint/eslint-plugin": "^5.44.0",
"@typescript-eslint/parser": "^5.44.0",
"builtin-modules": "^3.3.0",
"esbuild": "0.15.15",
"esbuild-svelte": "^0.7.3",
"eslint": "^8.28.0",
"eslint-config-airbnb-base": "^15.0.0",
"eslint-plugin-import": "^2.26.0",
"events": "^3.3.0",
"obsidian": "^0.16.3",
"postcss": "^8.4.14",
"postcss-load-config": "^3.1.4",
"rollup": "^2.32.1",
"svelte": "^3.49.0",
"postcss": "^8.4.19",
"postcss-load-config": "^4.0.1",
"pouchdb-adapter-http": "^8.0.0",
"pouchdb-adapter-idb": "^8.0.0",
"pouchdb-core": "^8.0.0",
"pouchdb-find": "^8.0.0",
"pouchdb-mapreduce": "^8.0.0",
"pouchdb-replication": "^8.0.0",
"pouchdb-utils": "file:src/lib/src/patches/pouchdb-utils",
"svelte": "^3.53.1",
"svelte-preprocess": "^4.10.7",
"tslib": "^2.2.0",
"typescript": "^4.2.4"
"transform-pouch": "^2.0.0",
"tslib": "^2.4.1",
"typescript": "^4.9.3"
},
"dependencies": {
"diff-match-patch": "^1.0.5",
"esbuild": "0.13.12",
"esbuild-svelte": "^0.7.0",
"idb": "^7.0.2",
"esbuild": "0.15.15",
"esbuild-svelte": "^0.7.3",
"idb": "^7.1.1",
"xxhash-wasm": "^0.4.2"
}
}

31
rollup.config.js
View File

@@ -1,31 +0,0 @@
import typescript from "@rollup/plugin-typescript";
import { nodeResolve } from "@rollup/plugin-node-resolve";
import commonjs from "@rollup/plugin-commonjs";
const isProd = process.env.BUILD === "production";
const banner = `/*
THIS IS A GENERATED/BUNDLED FILE BY ROLLUP
if you want to view the source visit the plugins github repository
*/
`;
export default {
input: "./src/main.ts",
output: {
dir: ".",
sourcemap: "inline",
sourcemapExcludeSources: isProd,
format: "cjs",
exports: "default",
banner,
},
external: ["obsidian"],
plugins: [
typescript({ exclude: ["pouchdb-browser.js", "pouchdb-browser-webpack"] }),
nodeResolve({
browser: true,
}),
commonjs(),
],
};

4
src/ConflictResolveModal.ts
View File

@@ -38,8 +38,8 @@ export class ConflictResolveModal extends Modal {
diff = diff.replace(/\n/g, "<br>");
div.innerHTML = diff;
const div2 = contentEl.createDiv("");
const date1 = new Date(this.result.left.mtime).toLocaleString();
const date2 = new Date(this.result.right.mtime).toLocaleString();
const date1 = new Date(this.result.left.mtime).toLocaleString() + (this.result.left.deleted ? " (Deleted)" : "");
const date2 = new Date(this.result.right.mtime).toLocaleString() + (this.result.right.deleted ? " (Deleted)" : "");
div2.innerHTML = `
<span class='deleted'>A:${date1}</span><br /><span class='added'>B:${date2}</span><br>
`;

8
src/LocalPouchDB.ts
View File

@@ -35,7 +35,7 @@ export class LocalPouchDB extends LocalPouchDBBase {
last_successful_post = false;
getLastPostFailedBySize() {
return this.last_successful_post;
return !this.last_successful_post;
}
async fetchByAPI(request: RequestUrlParam): Promise<RequestUrlResponse> {
const ret = await requestUrl(request);
@@ -52,7 +52,7 @@ export class LocalPouchDB extends LocalPouchDBBase {
}
async connectRemoteCouchDB(uri: string, auth: { username: string; password: string }, disableRequestURI: boolean, passphrase: string | boolean): Promise<string | { db: PouchDB.Database<EntryDoc>; info: PouchDB.Core.DatabaseInfo }> {
async connectRemoteCouchDB(uri: string, auth: { username: string; password: string }, disableRequestURI: boolean, passphrase: string | boolean, useDynamicIterationCount: boolean): Promise<string | { db: PouchDB.Database<EntryDoc>; info: PouchDB.Core.DatabaseInfo }> {
if (!isValidRemoteCouchDBURI(uri)) return "Remote URI is not valid";
if (uri.toLowerCase() != uri) return "Remote URI and database name could not contain capital letters.";
if (uri.indexOf(" ") !== -1) return "Remote URI and database name could not contain spaces.";
@@ -75,7 +75,7 @@ export class LocalPouchDB extends LocalPouchDBBase {
const method = opts.method ?? "GET";
if (opts.body) {
const opts_length = opts.body.toString().length;
if (opts_length > 1024 * 1024 * 10) {
if (opts_length > 1000 * 1000 * 10) {
// over 10MB
if (isCloudantURI(uri)) {
this.last_successful_post = false;
@@ -155,7 +155,7 @@ export class LocalPouchDB extends LocalPouchDBBase {
const db: PouchDB.Database<EntryDoc> = new PouchDB<EntryDoc>(uri, conf);
if (passphrase && typeof passphrase === "string") {
enableEncryption(db, passphrase);
enableEncryption(db, passphrase, useDynamicIterationCount);
}
try {
const info = await db.info();

121
src/ObsidianLiveSyncSettingTab.ts
View File

@@ -1,7 +1,7 @@
import { App, PluginSettingTab, Setting, sanitizeHTMLToDom, RequestUrlParam, requestUrl, TextAreaComponent, MarkdownRenderer, stringifyYaml } from "obsidian";
import { DEFAULT_SETTINGS, LOG_LEVEL, ObsidianLiveSyncSettings, RemoteDBSettings } from "./lib/src/types";
import { path2id, id2path } from "./utils";
import { delay, versionNumberString2Number } from "./lib/src/utils";
import { delay, Semaphore, versionNumberString2Number } from "./lib/src/utils";
import { Logger } from "./lib/src/logger";
import { checkSyncInfo, isCloudantURI } from "./lib/src/utils_couchdb.js";
import { testCrypt } from "./lib/src/e2ee_v2";
@@ -133,6 +133,7 @@ export class ObsidianLiveSyncSettingTab extends PluginSettingTab {
if (this.plugin.settings.syncOnFileOpen) return true;
if (this.plugin.settings.syncOnSave) return true;
if (this.plugin.settings.syncOnStart) return true;
if (this.plugin.settings.syncAfterMerge) return true;
if (this.plugin.localDatabase.syncStatus == "CONNECTED") return true;
if (this.plugin.localDatabase.syncStatus == "PAUSED") return true;
return false;
@@ -144,7 +145,7 @@ export class ObsidianLiveSyncSettingTab extends PluginSettingTab {
new Setting(setupWizardEl)
.setName("Discard the existing configuration and set up")
.addButton((text) => {
text.setButtonText("Next").onClick(async () => {
text.setButtonText("Next").onClick(() => {
if (JSON.stringify(this.plugin.settings) != JSON.stringify(DEFAULT_SETTINGS)) {
this.plugin.localDatabase.closeReplication();
this.plugin.settings = { ...DEFAULT_SETTINGS };
@@ -170,6 +171,7 @@ export class ObsidianLiveSyncSettingTab extends PluginSettingTab {
this.plugin.settings.syncOnSave = false;
this.plugin.settings.syncOnStart = false;
this.plugin.settings.syncOnFileOpen = false;
this.plugin.settings.syncAfterMerge = false;
this.plugin.localDatabase.closeReplication();
await this.plugin.saveSettings();
containerEl.addClass("isWizard");
@@ -226,7 +228,7 @@ export class ObsidianLiveSyncSettingTab extends PluginSettingTab {
syncLive.forEach((e) => {
e.setDisabled(false).setTooltip("");
});
} else if (this.plugin.settings.syncOnFileOpen || this.plugin.settings.syncOnSave || this.plugin.settings.syncOnStart || this.plugin.settings.periodicReplication) {
} else if (this.plugin.settings.syncOnFileOpen || this.plugin.settings.syncOnSave || this.plugin.settings.syncOnStart || this.plugin.settings.periodicReplication || this.plugin.settings.syncAfterMerge) {
syncNonLive.forEach((e) => {
e.setDisabled(false).setTooltip("");
});
@@ -297,10 +299,12 @@ export class ObsidianLiveSyncSettingTab extends PluginSettingTab {
if (inWizard) {
this.plugin.settings.encrypt = value;
passphrase.setDisabled(!value);
dynamicIteration.setDisabled(!value);
await this.plugin.saveSettings();
} else {
this.plugin.settings.workingEncrypt = value;
passphrase.setDisabled(!value);
dynamicIteration.setDisabled(!value);
await this.plugin.saveSettings();
}
})
@@ -325,11 +329,30 @@ export class ObsidianLiveSyncSettingTab extends PluginSettingTab {
});
passphrase.setDisabled(!this.plugin.settings.workingEncrypt);
const dynamicIteration = new Setting(containerRemoteDatabaseEl)
.setName("Use dynamic iteration count (experimental)")
.setDesc("Balancing the encryption/decryption load against the length of the passphrase if toggled. (v0.17.5 or higher required)")
.addToggle((toggle) => {
toggle.setValue(this.plugin.settings.workingUseDynamicIterationCount)
.onChange(async (value) => {
if (inWizard) {
this.plugin.settings.useDynamicIterationCount = value;
await this.plugin.saveSettings();
} else {
this.plugin.settings.workingUseDynamicIterationCount = value;
await this.plugin.saveSettings();
}
});
})
.setClass("wizardHidden");
dynamicIteration.setDisabled(!this.plugin.settings.workingEncrypt);
const checkWorkingPassphrase = async (): Promise<boolean> => {
const settingForCheck: RemoteDBSettings = {
...this.plugin.settings,
encrypt: this.plugin.settings.workingEncrypt,
passphrase: this.plugin.settings.workingPassphrase,
useDynamicIterationCount: this.plugin.settings.workingUseDynamicIterationCount,
};
console.dir(settingForCheck);
const db = await this.plugin.localDatabase.connectRemoteCouchDBWithSetting(settingForCheck, this.plugin.localDatabase.isMobile);
@@ -355,7 +378,7 @@ export class ObsidianLiveSyncSettingTab extends PluginSettingTab {
Logger("WARNING! Your device would not support encryption.", LOG_LEVEL.NOTICE);
return;
}
if (!(await checkWorkingPassphrase())) {
if (!(await checkWorkingPassphrase()) && !sendToServer) {
return;
}
if (!this.plugin.settings.workingEncrypt) {
@@ -366,8 +389,10 @@ export class ObsidianLiveSyncSettingTab extends PluginSettingTab {
this.plugin.settings.syncOnSave = false;
this.plugin.settings.syncOnStart = false;
this.plugin.settings.syncOnFileOpen = false;
this.plugin.settings.syncAfterMerge = false;
this.plugin.settings.encrypt = this.plugin.settings.workingEncrypt;
this.plugin.settings.passphrase = this.plugin.settings.workingPassphrase;
this.plugin.settings.useDynamicIterationCount = this.plugin.settings.workingUseDynamicIterationCount;
await this.plugin.saveSettings();
if (sendToServer) {
@@ -390,7 +415,6 @@ export class ObsidianLiveSyncSettingTab extends PluginSettingTab {
.setButtonText("Apply")
.setWarning()
.setDisabled(false)
.setClass("sls-btn-right")
.onClick(async () => {
await applyEncryption(true);
})
@@ -400,7 +424,6 @@ export class ObsidianLiveSyncSettingTab extends PluginSettingTab {
.setButtonText("Apply w/o rebuilding")
.setWarning()
.setDisabled(false)
.setClass("sls-btn-right")
.onClick(async () => {
await applyEncryption(false);
})
@@ -413,7 +436,7 @@ export class ObsidianLiveSyncSettingTab extends PluginSettingTab {
this.plugin.settings.syncOnSave = false;
this.plugin.settings.syncOnStart = false;
this.plugin.settings.syncOnFileOpen = false;
this.plugin.settings.syncAfterMerge = false;
await this.plugin.saveSettings();
applyDisplayEnabled();
@@ -448,7 +471,6 @@ export class ObsidianLiveSyncSettingTab extends PluginSettingTab {
.setButtonText("Send")
.setWarning()
.setDisabled(false)
.setClass("sls-btn-left")
.onClick(async () => {
await rebuildDB("remoteOnly");
})
@@ -463,7 +485,6 @@ export class ObsidianLiveSyncSettingTab extends PluginSettingTab {
.setButtonText("Rebuild")
.setWarning()
.setDisabled(false)
.setClass("sls-btn-left")
.onClick(async () => {
await rebuildDB("rebuildBothByThisDevice");
})
@@ -675,7 +696,7 @@ export class ObsidianLiveSyncSettingTab extends PluginSettingTab {
.setButtonText("Next")
.setClass("mod-cta")
.setDisabled(false)
.onClick(async () => {
.onClick(() => {
if (!this.plugin.settings.encrypt) {
this.plugin.settings.passphrase = "";
}
@@ -696,7 +717,7 @@ export class ObsidianLiveSyncSettingTab extends PluginSettingTab {
.setButtonText("Discard exist database and proceed")
.setDisabled(false)
.setWarning()
.onClick(async () => {
.onClick(() => {
if (!this.plugin.settings.encrypt) {
this.plugin.settings.passphrase = "";
}
@@ -738,7 +759,6 @@ export class ObsidianLiveSyncSettingTab extends PluginSettingTab {
.setButtonText("Fetch")
.setWarning()
.setDisabled(false)
.setClass("sls-btn-left")
.onClick(async () => {
await rebuildDB("localOnly");
})
@@ -776,7 +796,7 @@ export class ObsidianLiveSyncSettingTab extends PluginSettingTab {
button
.setButtonText("Next")
.setDisabled(false)
.onClick(async () => {
.onClick(() => {
changeDisplay("40");
})
);
@@ -813,6 +833,24 @@ export class ObsidianLiveSyncSettingTab extends PluginSettingTab {
})
}
);
new Setting(containerGeneralSettingsEl)
.setName("Delete old metadata of deleted files on start-up")
.setClass("wizardHidden")
.setDesc("(Days passed, 0 to disable automatic-deletion)")
.addText((text) => {
text.setPlaceholder("")
.setValue(this.plugin.settings.automaticallyDeleteMetadataOfDeletedFiles + "")
.onChange(async (value) => {
let v = Number(value);
if (isNaN(v)) {
v = 0;
}
this.plugin.settings.automaticallyDeleteMetadataOfDeletedFiles = v;
await this.plugin.saveSettings();
});
text.inputEl.setAttribute("type", "number");
});
addScreenElement("20", containerGeneralSettingsEl);
const containerSyncSettingEl = containerEl.createDiv();
@@ -914,7 +952,17 @@ export class ObsidianLiveSyncSettingTab extends PluginSettingTab {
await this.plugin.saveSettings();
applyDisplayEnabled();
})
)
),
new Setting(containerSyncSettingEl)
.setName("Sync after merging file")
.setDesc("Sync automatically after merging files")
.addToggle((toggle) =>
toggle.setValue(this.plugin.settings.syncAfterMerge).onChange(async (value) => {
this.plugin.settings.syncAfterMerge = value;
await this.plugin.saveSettings();
applyDisplayEnabled();
})
),
);
new Setting(containerSyncSettingEl)
@@ -956,6 +1004,24 @@ export class ObsidianLiveSyncSettingTab extends PluginSettingTab {
await this.plugin.saveSettings();
})
);
new Setting(containerSyncSettingEl)
.setName("Disable sensible auto merging on markdown files")
.setDesc("If this switch is turned on, a merge dialog will be displayed, even if the sensible-merge is possible automatically. (Turn on to previous behavior)")
.addToggle((toggle) =>
toggle.setValue(this.plugin.settings.disableMarkdownAutoMerge).onChange(async (value) => {
this.plugin.settings.disableMarkdownAutoMerge = value;
await this.plugin.saveSettings();
})
);
new Setting(containerSyncSettingEl)
.setName("Write documents after synchronization even if they have conflict")
.setDesc("Turn on to previous behavior")
.addToggle((toggle) =>
toggle.setValue(this.plugin.settings.writeDocumentsIfConflicted).onChange(async (value) => {
this.plugin.settings.writeDocumentsIfConflicted = value;
await this.plugin.saveSettings();
})
);
new Setting(containerSyncSettingEl)
@@ -1045,7 +1111,6 @@ export class ObsidianLiveSyncSettingTab extends PluginSettingTab {
.setButtonText("Touch")
.setWarning()
.setDisabled(false)
.setClass("sls-btn-left")
.onClick(async () => {
const filesAll = await this.plugin.scanInternalFiles();
const targetFiles = await this.plugin.filterTargetFiles(filesAll);
@@ -1220,6 +1285,7 @@ export class ObsidianLiveSyncSettingTab extends PluginSettingTab {
this.plugin.settings.syncOnSave = false;
this.plugin.settings.syncOnStart = false;
this.plugin.settings.syncOnFileOpen = false;
this.plugin.settings.syncAfterMerge = false;
if (currentPreset == "LIVESYNC") {
this.plugin.settings.liveSync = true;
Logger("Synchronization setting configured as LiveSync.", LOG_LEVEL.NOTICE);
@@ -1229,6 +1295,7 @@ export class ObsidianLiveSyncSettingTab extends PluginSettingTab {
this.plugin.settings.syncOnSave = false;
this.plugin.settings.syncOnStart = true;
this.plugin.settings.syncOnFileOpen = true;
this.plugin.settings.syncAfterMerge = true;
Logger("Synchronization setting configured as Periodic sync with batch database update.", LOG_LEVEL.NOTICE);
} else {
Logger("All synchronization disabled.", LOG_LEVEL.NOTICE);
@@ -1270,7 +1337,6 @@ export class ObsidianLiveSyncSettingTab extends PluginSettingTab {
new Setting(containerHatchEl)
.setName("Make report to inform the issue")
.setDesc("Verify and repair all files and update database without restoring")
.addButton((button) =>
button
.setButtonText("Make report")
@@ -1350,21 +1416,28 @@ ${stringifyYaml(pluginConfig)}`;
.setDisabled(false)
.setWarning()
.onClick(async () => {
const semaphore = Semaphore(10);
const files = this.app.vault.getFiles();
Logger("Verify and repair all files started", LOG_LEVEL.NOTICE, "verify");
// const notice = NewNotice("", 0);
let i = 0;
for (const file of files) {
i++;
Logger(`Update into ${file.path}`);
Logger(`${i}/${files.length}\n${file.path}`, LOG_LEVEL.NOTICE, "verify");
const processes = files.map(e => (async (file) => {
const releaser = await semaphore.acquire(1, "verifyAndRepair");
try {
await this.plugin.updateIntoDB(file);
Logger(`Update into ${file.path}`);
await this.plugin.updateIntoDB(file, false, null, true);
i++;
Logger(`${i}/${files.length}\n${file.path}`, LOG_LEVEL.NOTICE, "verify");
} catch (ex) {
Logger("could not update:");
i++;
Logger(`Error while verifyAndRepair`, LOG_LEVEL.NOTICE);
Logger(ex);
} finally {
releaser();
}
}
)(e));
await Promise.all(processes);
Logger("done", LOG_LEVEL.NOTICE, "verify");
})
);

2
src/lib

Submodule src/lib updated: d73b96ba2a...4ef5986b4d

528
src/main.ts
View File

@@ -1,5 +1,5 @@
import { debounce, Notice, Plugin, TFile, addIcon, TFolder, normalizePath, TAbstractFile, Editor, MarkdownView, PluginManifest, App, } from "obsidian";
import { diff_match_patch } from "diff-match-patch";
import { Diff, DIFF_DELETE, DIFF_EQUAL, DIFF_INSERT, diff_match_patch } from "diff-match-patch";
import { EntryDoc, LoadedEntry, ObsidianLiveSyncSettings, diff_check_result, diff_result_leaf, EntryBody, LOG_LEVEL, VER, DEFAULT_SETTINGS, diff_result, FLAGMD_REDFLAG, SYNCINFO_ID, InternalFileEntry } from "./lib/src/types";
import { PluginDataEntry, PERIODIC_PLUGIN_SWEEP, PluginList, DevicePluginList, InternalFileInfo } from "./types";
@@ -29,7 +29,7 @@ import { DocumentHistoryModal } from "./DocumentHistoryModal";
import { clearAllPeriodic, clearAllTriggers, clearTrigger, disposeMemoObject, id2path, memoIfNotExist, memoObject, path2id, retrieveMemoObject, setTrigger } from "./utils";
import { applyPatch, clearAllPeriodic, clearAllTriggers, clearTrigger, disposeMemoObject, generatePatchObj, id2path, isObjectMargeApplicable, isSensibleMargeApplicable, memoIfNotExist, memoObject, path2id, retrieveMemoObject, setTrigger, tryParseJSON } from "./utils";
import { decrypt, encrypt } from "./lib/src/e2ee_v2";
const isDebug = false;
@@ -44,6 +44,16 @@ const ICHeaderEnd = "i;";
const ICHeaderLength = ICHeader.length;
const FileWatchEventQueueMax = 10;
function getAbstractFileByPath(path: string): TAbstractFile | null {
// Hidden API but so useful.
if ("getAbstractFileByPathInsensitive" in app.vault && (app.vault.adapter?.insensitive ?? false)) {
// @ts-ignore
return app.vault.getAbstractFileByPathInsensitive(path);
} else {
return app.vault.getAbstractFileByPath(path);
}
}
/**
* returns is internal chunk of file
* @param str ID
@@ -97,7 +107,7 @@ const askString = (app: App, title: string, key: string, placeholder: string): P
};
let touchedFiles: string[] = [];
function touch(file: TFile | string) {
const f = file instanceof TFile ? file : app.vault.getAbstractFileByPath(file) as TFile;
const f = file instanceof TFile ? file : getAbstractFileByPath(file) as TFile;
const key = `${f.path}-${f.stat.mtime}-${f.stat.size}`;
touchedFiles.unshift(key);
touchedFiles = touchedFiles.slice(0, 100);
@@ -147,7 +157,7 @@ export default class ObsidianLiveSyncPlugin extends Plugin {
}
isRedFlagRaised(): boolean {
const redflag = this.app.vault.getAbstractFileByPath(normalizePath(FLAGMD_REDFLAG));
const redflag = getAbstractFileByPath(normalizePath(FLAGMD_REDFLAG));
if (redflag != null) {
return true;
}
@@ -174,7 +184,6 @@ export default class ObsidianLiveSyncPlugin extends Plugin {
nextKey = `${row.id}\u{10ffff}`;
if (!("type" in doc)) continue;
if (doc.type == "newnote" || doc.type == "plain") {
// const docId = doc._id.startsWith("i:") ? doc._id.substring("i:".length) : doc._id;
notes.push({ path: id2path(doc._id), mtime: doc.mtime });
}
if (isChunk(nextKey)) {
@@ -203,8 +212,9 @@ export default class ObsidianLiveSyncPlugin extends Plugin {
nextKey = `${row.id}\u{10ffff}`;
if (!("_conflicts" in doc)) continue;
if (isInternalChunk(row.id)) continue;
if (doc._deleted) continue;
if ("deleted" in doc && doc.deleted) continue;
// We have to check also deleted files.
// if (doc._deleted) continue;
// if ("deleted" in doc && doc.deleted) continue;
if (doc.type == "newnote" || doc.type == "plain") {
// const docId = doc._id.startsWith("i:") ? doc._id.substring("i:".length) : doc._id;
notes.push({ path: id2path(doc._id), mtime: doc.mtime });
@@ -226,11 +236,52 @@ export default class ObsidianLiveSyncPlugin extends Plugin {
if (isInternalChunk(target)) {
//NOP
} else {
await this.showIfConflicted(this.app.vault.getAbstractFileByPath(target) as TFile);
await this.showIfConflicted(target);
}
}
}
async collectDeletedFiles() {
const pageLimit = 1000;
let nextKey = "";
const limitDays = this.settings.automaticallyDeleteMetadataOfDeletedFiles;
if (limitDays <= 0) return;
Logger(`Checking expired file history`);
const limit = Date.now() - (86400 * 1000 * limitDays);
const notes: { path: string, mtime: number, ttl: number, doc: PouchDB.Core.ExistingDocument<EntryDoc & PouchDB.Core.AllDocsMeta> }[] = [];
do {
const docs = await this.localDatabase.localDatabase.allDocs({ limit: pageLimit, startkey: nextKey, conflicts: true, include_docs: true });
nextKey = "";
for (const row of docs.rows) {
const doc = row.doc;
nextKey = `${row.id}\u{10ffff}`;
if (doc.type == "newnote" || doc.type == "plain") {
if (doc.deleted && (doc.mtime - limit) < 0) {
notes.push({ path: id2path(doc._id), mtime: doc.mtime, ttl: (doc.mtime - limit) / 1000 / 86400, doc: doc });
}
}
if (isChunk(nextKey)) {
// skip the chunk zone.
nextKey = CHeaderEnd;
}
}
} while (nextKey != "");
if (notes.length == 0) {
Logger("There are no old documents");
Logger(`Checking expired file history done`);
return;
}
for (const v of notes) {
Logger(`Deletion history expired: ${v.path}`);
const delDoc = v.doc;
delDoc._deleted = true;
// console.dir(delDoc);
await this.localDatabase.localDatabase.put(delDoc);
}
Logger(`Checking expired file history done`);
}
async onload() {
setLogger(this.addLog.bind(this)); // Logger moved to global.
Logger("loading plugin");
@@ -258,6 +309,7 @@ export default class ObsidianLiveSyncPlugin extends Plugin {
this.settings.syncOnSave = false;
this.settings.syncOnStart = false;
this.settings.syncOnFileOpen = false;
this.settings.syncAfterMerge = false;
this.settings.periodicReplication = false;
this.settings.versionUpFlash = "Self-hosted LiveSync has been upgraded and some behaviors have changed incompatibly. All automatic synchronization is now disabled temporary. Ensure that other devices are also upgraded, and enable synchronization again.";
this.saveSettings();
@@ -324,6 +376,7 @@ export default class ObsidianLiveSyncPlugin extends Plugin {
this.settings.syncOnSave = false;
this.settings.syncOnStart = false;
this.settings.syncOnFileOpen = false;
this.settings.syncAfterMerge = false;
this.settings.autoSweepPlugins = false;
this.settings.usePluginSync = false;
this.settings.suspendFileWatching = true;
@@ -367,7 +420,7 @@ export default class ObsidianLiveSyncPlugin extends Plugin {
delete setting[k];
}
}
const encryptedSetting = encodeURIComponent(await encrypt(JSON.stringify(setting), encryptingPassphrase));
const encryptedSetting = encodeURIComponent(await encrypt(JSON.stringify(setting), encryptingPassphrase, false));
const uri = `${configURIBase}${encryptedSetting}`;
await navigator.clipboard.writeText(uri);
Logger("Setup URI copied to clipboard", LOG_LEVEL.NOTICE);
@@ -380,7 +433,7 @@ export default class ObsidianLiveSyncPlugin extends Plugin {
const encryptingPassphrase = await askString(this.app, "Encrypt your settings", "Passphrase", "");
if (encryptingPassphrase === false) return;
const setting = { ...this.settings };
const encryptedSetting = encodeURIComponent(await encrypt(JSON.stringify(setting), encryptingPassphrase));
const encryptedSetting = encodeURIComponent(await encrypt(JSON.stringify(setting), encryptingPassphrase, false));
const uri = `${configURIBase}${encryptedSetting}`;
await navigator.clipboard.writeText(uri);
Logger("Setup URI copied to clipboard", LOG_LEVEL.NOTICE);
@@ -406,7 +459,7 @@ export default class ObsidianLiveSyncPlugin extends Plugin {
const oldConf = JSON.parse(JSON.stringify(this.settings));
const encryptingPassphrase = await askString(this.app, "Passphrase", "Passphrase for your settings", "");
if (encryptingPassphrase === false) return;
const newConf = await JSON.parse(await decrypt(confString, encryptingPassphrase));
const newConf = await JSON.parse(await decrypt(confString, encryptingPassphrase, false));
if (newConf) {
const result = await askYesNo(this.app, "Importing LiveSync's conf, OK?");
if (result == "yes") {
@@ -528,7 +581,7 @@ export default class ObsidianLiveSyncPlugin extends Plugin {
id: "livesync-checkdoc-conflicted",
name: "Resolve if conflicted.",
editorCallback: async (editor: Editor, view: MarkdownView) => {
await this.showIfConflicted(view.file);
await this.showIfConflicted(view.file.path);
},
});
@@ -658,7 +711,7 @@ export default class ObsidianLiveSyncPlugin extends Plugin {
clearAllPeriodic();
clearAllTriggers();
window.removeEventListener("visibilitychange", this.watchWindowVisibility);
window.removeEventListener("online", this.watchOnline)
window.removeEventListener("online", this.watchOnline);
Logger("unloading plugin");
}
@@ -921,7 +974,7 @@ export default class ObsidianLiveSyncPlugin extends Plugin {
if (this.settings.syncOnFileOpen && !this.suspended) {
await this.replicate();
}
await this.showIfConflicted(file);
await this.showIfConflicted(file.path);
}
async applyBatchChange() {
@@ -1009,7 +1062,7 @@ export default class ObsidianLiveSyncPlugin extends Plugin {
for (const i of newFiles) {
try {
const newFilePath = normalizePath(this.getFilePath(i));
const newFile = this.app.vault.getAbstractFileByPath(newFilePath);
const newFile = getAbstractFileByPath(newFilePath);
if (newFile instanceof TFile) {
Logger(`save ${newFile.path} into db`);
await this.updateIntoDB(newFile);
@@ -1170,7 +1223,7 @@ export default class ObsidianLiveSyncPlugin extends Plugin {
touch(newFile);
this.app.vault.trigger("create", newFile);
} catch (ex) {
Logger(msg + "ERROR, Could not parse: " + path + "(" + doc.datatype + ")", LOG_LEVEL.NOTICE);
Logger(msg + "ERROR, Could not create: " + path + "(" + doc.datatype + ")", LOG_LEVEL.NOTICE);
Logger(ex, LOG_LEVEL.VERBOSE);
}
} else {
@@ -1237,7 +1290,7 @@ export default class ObsidianLiveSyncPlugin extends Plugin {
await this.app.vault.modifyBinary(file, bin, { ctime: doc.ctime, mtime: doc.mtime });
// this.batchFileChange = this.batchFileChange.filter((e) => e != file.path);
Logger(msg + path);
const xf = this.app.vault.getAbstractFileByPath(file.path) as TFile;
const xf = getAbstractFileByPath(file.path) as TFile;
touch(xf);
this.app.vault.trigger("modify", xf);
} catch (ex) {
@@ -1254,7 +1307,7 @@ export default class ObsidianLiveSyncPlugin extends Plugin {
await this.app.vault.modify(file, doc.data, { ctime: doc.ctime, mtime: doc.mtime });
Logger(msg + path);
// this.batchFileChange = this.batchFileChange.filter((e) => e != file.path);
const xf = this.app.vault.getAbstractFileByPath(file.path) as TFile;
const xf = getAbstractFileByPath(file.path) as TFile;
touch(xf);
this.app.vault.trigger("modify", xf);
} catch (ex) {
@@ -1304,7 +1357,7 @@ export default class ObsidianLiveSyncPlugin extends Plugin {
}
async handleDBChangedAsync(change: EntryBody) {
const targetFile = this.app.vault.getAbstractFileByPath(id2path(change._id));
const targetFile = getAbstractFileByPath(id2path(change._id));
if (targetFile == null) {
if (change._deleted || change.deleted) {
return;
@@ -1314,13 +1367,30 @@ export default class ObsidianLiveSyncPlugin extends Plugin {
} else if (targetFile instanceof TFile) {
const doc = change;
const file = targetFile;
await this.doc2storage_modify(doc, file);
if (!this.settings.checkConflictOnlyOnOpen) {
this.queueConflictedCheck(file);
} else {
const af = app.workspace.getActiveFile();
if (af && af.path == file.path) {
const queueConflictCheck = () => {
if (!this.settings.checkConflictOnlyOnOpen) {
this.queueConflictedCheck(file);
return true;
} else {
const af = app.workspace.getActiveFile();
if (af && af.path == file.path) {
this.queueConflictedCheck(file);
return true;
}
}
return false;
}
if (this.settings.writeDocumentsIfConflicted) {
await this.doc2storage_modify(doc, file);
queueConflictCheck();
} else {
const d = await this.localDatabase.getDBEntryMeta(id2path(change._id), { conflicts: true }, true);
if (d && !d._conflicts) {
await this.doc2storage_modify(doc, file);
} else {
if (!queueConflictCheck()) {
Logger(`${id2path(change._id)} is conflicted, write to the storage has been pended.`, LOG_LEVEL.NOTICE);
}
}
}
} else {
@@ -1420,7 +1490,7 @@ export default class ObsidianLiveSyncPlugin extends Plugin {
if (!this.isTargetFile(id2path(doc._id))) return;
const skipOldFile = this.settings.skipOlderFilesOnSync && false; //patched temporary.
if ((!isInternalChunk(doc._id)) && skipOldFile) {
const info = this.app.vault.getAbstractFileByPath(id2path(doc._id));
const info = getAbstractFileByPath(id2path(doc._id));
if (info && info instanceof TFile) {
const localMtime = ~~((info as TFile).stat.mtime / 1000);
@@ -1727,10 +1797,18 @@ export default class ObsidianLiveSyncPlugin extends Plugin {
Logger("Initializing", LOG_LEVEL.NOTICE, "syncAll");
}
await this.collectDeletedFiles();
const filesStorage = this.app.vault.getFiles().filter(e => this.isTargetFile(e));
const filesStorageName = filesStorage.map((e) => e.path);
const wf = await this.localDatabase.localDatabase.allDocs();
const filesDatabase = wf.rows.filter((e) => !isChunk(e.id) && !isPluginChunk(e.id) && e.id != "obsydian_livesync_version").filter(e => isValidPath(e.id)).map((e) => id2path(e.id)).filter(e => this.isTargetFile(e));
const filesDatabase = wf.rows.filter((e) =>
!isChunk(e.id) &&
!isPluginChunk(e.id) &&
e.id != "obsydian_livesync_version" &&
e.id != "_design/replicate"
)
.filter(e => isValidPath(e.id)).map((e) => id2path(e.id)).filter(e => this.isTargetFile(e));
const isInitialized = await (this.localDatabase.kvDB.get<boolean>("initialized")) || false;
// Make chunk bigger if it is the initial scan. There must be non-active docs.
if (filesDatabase.length == 0 && !isInitialized) {
@@ -1748,28 +1826,28 @@ export default class ObsidianLiveSyncPlugin extends Plugin {
this.setStatusBarText(`UPDATE DATABASE`);
const runAll = async<T>(procedureName: string, objects: T[], callback: (arg: T) => Promise<void>) => {
const count = objects.length;
// const count = objects.length;
Logger(procedureName);
let i = 0;
// let i = 0;
const semaphore = Semaphore(10);
Logger(`${procedureName} exec.`);
// Logger(`${procedureName} exec.`);
if (!this.localDatabase.isReady) throw Error("Database is not ready!");
const processes = objects.map(e => (async (v) => {
const releaser = await semaphore.acquire(1, procedureName);
try {
await callback(v);
i++;
if (i % 50 == 0) {
const notify = `${procedureName} : ${i}/${count}`;
if (showingNotice) {
Logger(notify, LOG_LEVEL.NOTICE, "syncAll");
} else {
Logger(notify);
}
this.setStatusBarText(notify);
}
// i++;
// if (i % 50 == 0) {
// const notify = `${procedureName} : ${i}/${count}`;
// if (showingNotice) {
// Logger(notify, LOG_LEVEL.NOTICE, "syncAll");
// } else {
// Logger(notify);
// }
// this.setStatusBarText(notify);
// }
} catch (ex) {
Logger(`Error while ${procedureName}`, LOG_LEVEL.NOTICE);
Logger(ex);
@@ -1785,18 +1863,19 @@ export default class ObsidianLiveSyncPlugin extends Plugin {
await runAll("UPDATE DATABASE", onlyInStorage, async (e) => {
Logger(`Update into ${e.path}`);
await this.updateIntoDB(e, initialScan);
});
if (!initialScan) {
await runAll("UPDATE STORAGE", onlyInDatabase, async (e) => {
const w = await this.localDatabase.getDBEntryMeta(e);
if (w) {
const w = await this.localDatabase.getDBEntryMeta(e, {}, true);
if (w && !(w.deleted || w._deleted)) {
Logger(`Check or pull from db:${e}`);
await this.pullFile(e, filesStorage, false, null, false);
Logger(`Check or pull from db:${e} OK`);
} else if (w) {
Logger(`Deletion history skipped: ${e}`, LOG_LEVEL.VERBOSE);
} else {
Logger(`entry not found, maybe deleted (it is normal behavior):${e}`);
Logger(`entry not found: ${e}`);
}
});
}
@@ -1872,7 +1951,7 @@ export default class ObsidianLiveSyncPlugin extends Plugin {
// --> conflict resolving
async getConflictedDoc(path: string, rev: string): Promise<false | diff_result_leaf> {
try {
const doc = await this.localDatabase.getDBEntry(path, { rev: rev }, false, false);
const doc = await this.localDatabase.getDBEntry(path, { rev: rev }, false, false, true);
if (doc === false) return false;
let data = doc.data;
if (doc.datatype == "newnote") {
@@ -1881,6 +1960,7 @@ export default class ObsidianLiveSyncPlugin extends Plugin {
data = doc.data;
}
return {
deleted: doc.deleted || doc._deleted,
ctime: doc.ctime,
mtime: doc.mtime,
rev: rev,
@@ -1893,6 +1973,193 @@ export default class ObsidianLiveSyncPlugin extends Plugin {
}
return false;
}
//TODO: TIDY UP
async mergeSensibly(path: string, baseRev: string, currentRev: string, conflictedRev: string): Promise<Diff[] | false> {
const baseLeaf = await this.getConflictedDoc(path, baseRev);
const leftLeaf = await this.getConflictedDoc(path, currentRev);
const rightLeaf = await this.getConflictedDoc(path, conflictedRev);
let autoMerge = false;
if (baseLeaf == false || leftLeaf == false || rightLeaf == false) {
return false;
}
// diff between base and each revision
const dmp = new diff_match_patch();
const mapLeft = dmp.diff_linesToChars_(baseLeaf.data, leftLeaf.data);
const diffLeftSrc = dmp.diff_main(mapLeft.chars1, mapLeft.chars2, false);
dmp.diff_charsToLines_(diffLeftSrc, mapLeft.lineArray);
const mapRight = dmp.diff_linesToChars_(baseLeaf.data, rightLeaf.data);
const diffRightSrc = dmp.diff_main(mapRight.chars1, mapRight.chars2, false);
dmp.diff_charsToLines_(diffRightSrc, mapRight.lineArray);
function splitDiffPiece(src: Diff[]): Diff[] {
const ret = [] as Diff[];
do {
const d = src.shift();
const pieces = d[1].split(/([^\n]*\n)/).filter(f => f != "");
if (typeof (d) == "undefined") {
break;
}
if (d[0] != DIFF_DELETE) {
ret.push(...(pieces.map(e => [d[0], e] as Diff)));
}
if (d[0] == DIFF_DELETE) {
const nd = src.shift();
if (typeof (nd) != "undefined") {
const piecesPair = nd[1].split(/([^\n]*\n)/).filter(f => f != "");
if (nd[0] == DIFF_INSERT) {
// it might be pair
for (const pt of pieces) {
ret.push([d[0], pt]);
const pairP = piecesPair.shift();
if (typeof (pairP) != "undefined") ret.push([DIFF_INSERT, pairP]);
}
ret.push(...(piecesPair.map(e => [nd[0], e] as Diff)));
} else {
ret.push(...(pieces.map(e => [d[0], e] as Diff)));
ret.push(...(piecesPair.map(e => [nd[0], e] as Diff)));
}
} else {
ret.push(...(pieces.map(e => [0, e] as Diff)));
}
}
} while (src.length > 0);
return ret;
}
const diffLeft = splitDiffPiece(diffLeftSrc);
const diffRight = splitDiffPiece(diffRightSrc);
let rightIdx = 0;
let leftIdx = 0;
const merged = [] as Diff[];
autoMerge = true;
LOOP_MERGE:
do {
if (leftIdx >= diffLeft.length && rightIdx >= diffRight.length) {
break LOOP_MERGE;
}
const leftItem = diffLeft[leftIdx] ?? [0, ""];
const rightItem = diffRight[rightIdx] ?? [0, ""];
leftIdx++;
rightIdx++;
// when completely same, leave it .
if (leftItem[0] == DIFF_EQUAL && rightItem[0] == DIFF_EQUAL && leftItem[1] == rightItem[1]) {
merged.push(leftItem);
continue;
}
if (leftItem[0] == DIFF_DELETE && rightItem[0] == DIFF_DELETE && leftItem[1] == rightItem[1]) {
// when deleted evenly,
const nextLeftIdx = leftIdx;
const nextRightIdx = rightIdx;
const [nextLeftItem, nextRightItem] = [diffLeft[nextLeftIdx] ?? [0, ""], diffRight[nextRightIdx] ?? [0, ""]];
if ((nextLeftItem[0] == DIFF_INSERT && nextRightItem[0] == DIFF_INSERT) && nextLeftItem[1] != nextRightItem[1]) {
//but next line looks like different
autoMerge = false;
break;
} else {
merged.push(leftItem);
continue;
}
}
// when inserted evenly
if (leftItem[0] == DIFF_INSERT && rightItem[0] == DIFF_INSERT) {
if (leftItem[1] == rightItem[1]) {
merged.push(leftItem);
continue;
} else {
// sort by file date.
if (leftLeaf.mtime <= rightLeaf.mtime) {
merged.push(leftItem);
merged.push(rightItem);
continue;
} else {
merged.push(rightItem);
merged.push(leftItem);
continue;
}
}
}
// when on inserting, index should be fixed again.
if (leftItem[0] == DIFF_INSERT) {
rightIdx--;
merged.push(leftItem);
continue;
}
if (rightItem[0] == DIFF_INSERT) {
leftIdx--;
merged.push(rightItem);
continue;
}
// except insertion, the line should not be different.
if (rightItem[1] != leftItem[1]) {
//TODO: SHOULD BE PANIC.
Logger(`MERGING PANIC:${leftItem[0]},${leftItem[1]} == ${rightItem[0]},${rightItem[1]}`, LOG_LEVEL.VERBOSE);
autoMerge = false;
break LOOP_MERGE;
}
if (leftItem[0] == DIFF_DELETE) {
if (rightItem[0] == DIFF_EQUAL) {
merged.push(leftItem);
continue;
} else {
//we cannot perform auto merge.
autoMerge = false;
break LOOP_MERGE;
}
}
if (rightItem[0] == DIFF_DELETE) {
if (leftItem[0] == DIFF_EQUAL) {
merged.push(rightItem);
continue;
} else {
//we cannot perform auto merge.
autoMerge = false;
break LOOP_MERGE;
}
}
Logger(`Weird condition:${leftItem[0]},${leftItem[1]} == ${rightItem[0]},${rightItem[1]}`, LOG_LEVEL.VERBOSE);
// here is the exception
break LOOP_MERGE;
} while (leftIdx < diffLeft.length || rightIdx < diffRight.length);
if (autoMerge) {
Logger(`Sensibly merge available`, LOG_LEVEL.VERBOSE);
return merged;
} else {
return false;
}
}
async mergeObject(path: string, baseRev: string, currentRev: string, conflictedRev: string): Promise<string | false> {
const baseLeaf = await this.getConflictedDoc(path, baseRev);
const leftLeaf = await this.getConflictedDoc(path, currentRev);
const rightLeaf = await this.getConflictedDoc(path, conflictedRev);
if (baseLeaf == false || leftLeaf == false || rightLeaf == false) {
return false;
}
const baseObj = { data: tryParseJSON(baseLeaf.data, {}) } as Record<string | number | symbol, any>;
const leftObj = { data: tryParseJSON(leftLeaf.data, {}) } as Record<string | number | symbol, any>;
const rightObj = { data: tryParseJSON(rightLeaf.data, {}) } as Record<string | number | symbol, any>;
const diffLeft = generatePatchObj(baseObj, leftObj);
const diffRight = generatePatchObj(baseObj, rightObj);
const patches = [
{ mtime: leftLeaf.mtime, patch: diffLeft },
{ mtime: rightLeaf.mtime, patch: diffRight }
].sort((a, b) => a.mtime - b.mtime);
let newObj = { ...baseObj };
try {
for (const patch of patches) {
newObj = applyPatch(newObj, patch.patch);
}
return JSON.stringify(newObj.data);
} catch (ex) {
Logger("Could not merge object");
Logger(ex, LOG_LEVEL.VERBOSE)
return false;
}
}
/**
* Getting file conflicted status.
@@ -1900,14 +2167,61 @@ export default class ObsidianLiveSyncPlugin extends Plugin {
* @returns true -> resolved, false -> nothing to do, or check result.
*/
async getConflictedStatus(path: string): Promise<diff_check_result> {
const test = await this.localDatabase.getDBEntry(path, { conflicts: true }, false, false);
const test = await this.localDatabase.getDBEntry(path, { conflicts: true }, false, false, true);
if (test === false) return false;
if (test == null) return false;
if (!test._conflicts) return false;
if (test._conflicts.length == 0) return false;
const conflicts = test._conflicts.sort((a, b) => Number(a.split("-")[0]) - Number(b.split("-")[0]));
if ((isSensibleMargeApplicable(path) || isObjectMargeApplicable(path)) && !this.settings.disableMarkdownAutoMerge) {
const conflictedRev = conflicts[0];
const conflictedRevNo = Number(conflictedRev.split("-")[0]);
//Search
const revFrom = (await this.localDatabase.localDatabase.get(id2path(path), { revs_info: true })) as unknown as LoadedEntry & PouchDB.Core.GetMeta;
const commonBase = revFrom._revs_info.filter(e => e.status == "available" && Number(e.rev.split("-")[0]) < conflictedRevNo).first()?.rev ?? "";
let p = undefined;
if (commonBase) {
if (isSensibleMargeApplicable(path)) {
const result = await this.mergeSensibly(path, commonBase, test._rev, conflictedRev);
if (result) {
p = result.filter(e => e[0] != DIFF_DELETE).map((e) => e[1]).join("");
// can be merged.
Logger(`Sensible merge:${path}`, LOG_LEVEL.INFO);
} else {
Logger(`Sensible merge is not applicable.`, LOG_LEVEL.VERBOSE);
}
} else if (isObjectMargeApplicable(path)) {
// can be merged.
const result = await this.mergeObject(path, commonBase, test._rev, conflictedRev);
if (result) {
Logger(`Object merge:${path}`, LOG_LEVEL.INFO);
p = result;
} else {
Logger(`Object merge is not applicable.`, LOG_LEVEL.VERBOSE);
}
}
if (p != undefined) {
// remove conflicted revision.
await this.localDatabase.deleteDBEntry(path, { rev: conflictedRev });
const file = getAbstractFileByPath(path) as TFile;
if (file) {
await this.app.vault.modify(file, p);
await this.updateIntoDB(file);
} else {
const newFile = await this.app.vault.create(path, p);
await this.updateIntoDB(newFile);
}
await this.pullFile(path);
Logger(`Automatically merged (sensible) :${path}`, LOG_LEVEL.INFO);
return true;
}
}
}
// should be one or more conflicts;
const leftLeaf = await this.getConflictedDoc(path, test._rev);
const rightLeaf = await this.getConflictedDoc(path, test._conflicts[0]);
const rightLeaf = await this.getConflictedDoc(path, conflicts[0]);
if (leftLeaf == false) {
// what's going on..
Logger(`could not get current revisions:${path}`, LOG_LEVEL.NOTICE);
@@ -1915,13 +2229,13 @@ export default class ObsidianLiveSyncPlugin extends Plugin {
}
if (rightLeaf == false) {
// Conflicted item could not load, delete this.
await this.localDatabase.deleteDBEntry(path, { rev: test._conflicts[0] });
await this.localDatabase.deleteDBEntry(path, { rev: conflicts[0] });
await this.pullFile(path, null, true);
Logger(`could not get old revisions, automatically used newer one:${path}`, LOG_LEVEL.NOTICE);
return true;
}
// first,check for same contents
if (leftLeaf.data == rightLeaf.data) {
// first, check for same contents and deletion status.
if (leftLeaf.data == rightLeaf.data && leftLeaf.deleted == rightLeaf.deleted) {
let leaf = leftLeaf;
if (leftLeaf.mtime > rightLeaf.mtime) {
leaf = rightLeaf;
@@ -1955,11 +2269,11 @@ export default class ObsidianLiveSyncPlugin extends Plugin {
};
}
showMergeDialog(file: TFile, conflictCheckResult: diff_result): Promise<boolean> {
showMergeDialog(filename: string, conflictCheckResult: diff_result): Promise<boolean> {
return new Promise((res, rej) => {
Logger("open conflict dialog", LOG_LEVEL.VERBOSE);
new ConflictResolveModal(this.app, conflictCheckResult, async (selected) => {
const testDoc = await this.localDatabase.getDBEntry(file.path, { conflicts: true });
const testDoc = await this.localDatabase.getDBEntry(filename, { conflicts: true }, false, false, true);
if (testDoc === false) {
Logger("Missing file..", LOG_LEVEL.VERBOSE);
return res(true);
@@ -1971,28 +2285,39 @@ export default class ObsidianLiveSyncPlugin extends Plugin {
const toDelete = selected;
const toKeep = conflictCheckResult.left.rev != toDelete ? conflictCheckResult.left.rev : conflictCheckResult.right.rev;
if (toDelete == "") {
//concat both,
// write data,and delete both old rev.
// concat both,
// delete conflicted revision and write a new file, store it again.
const p = conflictCheckResult.diff.map((e) => e[1]).join("");
await this.localDatabase.deleteDBEntry(file.path, { rev: conflictCheckResult.left.rev });
await this.localDatabase.deleteDBEntry(file.path, { rev: conflictCheckResult.right.rev });
await this.app.vault.modify(file, p);
await this.updateIntoDB(file);
await this.pullFile(file.path);
await this.localDatabase.deleteDBEntry(filename, { rev: testDoc._conflicts[0] });
const file = getAbstractFileByPath(filename) as TFile;
if (file) {
await this.app.vault.modify(file, p);
await this.updateIntoDB(file);
} else {
const newFile = await this.app.vault.create(filename, p);
await this.updateIntoDB(newFile);
}
await this.pullFile(filename);
Logger("concat both file");
if (this.settings.syncAfterMerge && !this.suspended) {
await this.replicate();
}
setTimeout(() => {
//resolved, check again.
this.showIfConflicted(file);
this.showIfConflicted(filename);
}, 500);
} else if (toDelete == null) {
Logger("Leave it still conflicted");
} else {
Logger(`Conflict resolved:${file.path}`);
await this.localDatabase.deleteDBEntry(file.path, { rev: toDelete });
await this.pullFile(file.path, null, true, toKeep);
await this.localDatabase.deleteDBEntry(filename, { rev: toDelete });
await this.pullFile(filename, null, true, toKeep);
Logger(`Conflict resolved:${filename}`);
if (this.settings.syncAfterMerge && !this.suspended) {
await this.replicate();
}
setTimeout(() => {
//resolved, check again.
this.showIfConflicted(file);
this.showIfConflicted(filename);
}, 500);
}
@@ -2017,39 +2342,42 @@ export default class ObsidianLiveSyncPlugin extends Plugin {
const checkFiles = JSON.parse(JSON.stringify(this.conflictedCheckFiles)) as string[];
for (const filename of checkFiles) {
try {
const file = this.app.vault.getAbstractFileByPath(filename);
const file = getAbstractFileByPath(filename);
if (file != null && file instanceof TFile) {
await this.showIfConflicted(file);
await this.showIfConflicted(file.path);
}
} catch (ex) {
Logger(ex);
}
}
}, 1000);
}, 100);
}
async showIfConflicted(file: TFile) {
async showIfConflicted(filename: string) {
await runWithLock("conflicted", false, async () => {
const conflictCheckResult = await this.getConflictedStatus(file.path);
const conflictCheckResult = await this.getConflictedStatus(filename);
if (conflictCheckResult === false) {
//nothing to do.
return;
}
if (conflictCheckResult === true) {
//auto resolved, but need check again;
if (this.settings.syncAfterMerge && !this.suspended) {
await this.replicate();
}
Logger("conflict:Automatically merged, but we have to check it again");
setTimeout(() => {
this.showIfConflicted(file);
this.showIfConflicted(filename);
}, 500);
return;
}
//there conflicts, and have to resolve ;
await this.showMergeDialog(file, conflictCheckResult);
await this.showMergeDialog(filename, conflictCheckResult);
});
}
async pullFile(filename: string, fileList?: TFile[], force?: boolean, rev?: string, waitForReady = true) {
const targetFile = this.app.vault.getAbstractFileByPath(id2path(filename));
const targetFile = getAbstractFileByPath(id2path(filename));
if (!this.isTargetFile(id2path(filename))) return;
if (targetFile == null) {
//have to create;
@@ -2080,7 +2408,7 @@ export default class ObsidianLiveSyncPlugin extends Plugin {
throw new Error(`Missing doc:${(file as any).path}`)
}
if (!(file instanceof TFile) && "path" in file) {
const w = this.app.vault.getAbstractFileByPath((file as any).path);
const w = getAbstractFileByPath((file as any).path);
if (w instanceof TFile) {
file = w;
} else {
@@ -2125,7 +2453,7 @@ export default class ObsidianLiveSyncPlugin extends Plugin {
}
async updateIntoDB(file: TFile, initialScan?: boolean, cache?: CacheData) {
async updateIntoDB(file: TFile, initialScan?: boolean, cache?: CacheData, force?: boolean) {
if (!this.isTargetFile(file)) return;
if (shouldBeIgnored(file.path)) {
return;
@@ -2167,15 +2495,24 @@ export default class ObsidianLiveSyncPlugin extends Plugin {
if (recentlyTouched(file)) {
return true;
}
const old = await this.localDatabase.getDBEntry(fullPath, null, false, false);
if (old !== false) {
const oldData = { data: old.data, deleted: old._deleted || old.deleted, };
const newData = { data: d.data, deleted: d._deleted || d.deleted };
if (JSON.stringify(oldData) == JSON.stringify(newData)) {
Logger(msg + "Skipped (not changed) " + fullPath + ((d._deleted || d.deleted) ? " (deleted)" : ""), LOG_LEVEL.VERBOSE);
return true;
try {
const old = await this.localDatabase.getDBEntry(fullPath, null, false, false);
if (old !== false) {
const oldData = { data: old.data, deleted: old._deleted || old.deleted, };
const newData = { data: d.data, deleted: d._deleted || d.deleted };
if (JSON.stringify(oldData) == JSON.stringify(newData)) {
Logger(msg + "Skipped (not changed) " + fullPath + ((d._deleted || d.deleted) ? " (deleted)" : ""), LOG_LEVEL.VERBOSE);
return true;
}
// d._rev = old._rev;
}
// d._rev = old._rev;
} catch (ex) {
if (force) {
Logger(msg + "Error, Could not check the diff for the old one." + (force ? "force writing." : "") + fullPath + ((d._deleted || d.deleted) ? " (deleted)" : ""), LOG_LEVEL.VERBOSE);
} else {
Logger(msg + "Error, Could not check the diff for the old one." + fullPath + ((d._deleted || d.deleted) ? " (deleted)" : ""), LOG_LEVEL.VERBOSE);
}
return !force;
}
return false;
});
@@ -2635,8 +2972,33 @@ export default class ObsidianLiveSyncPlugin extends Plugin {
if (!("_conflicts" in doc)) return false;
if (doc._conflicts.length == 0) return false;
Logger(`Hidden file conflicted:${id2filenameInternalChunk(id)}`);
const conflicts = doc._conflicts.sort((a, b) => Number(a.split("-")[0]) - Number(b.split("-")[0]));
const revA = doc._rev;
const revB = doc._conflicts[0];
const revB = conflicts[0];
const conflictedRev = conflicts[0];
const conflictedRevNo = Number(conflictedRev.split("-")[0]);
//Search
const revFrom = (await this.localDatabase.localDatabase.get(id, { revs_info: true })) as unknown as LoadedEntry & PouchDB.Core.GetMeta;
const commonBase = revFrom._revs_info.filter(e => e.status == "available" && Number(e.rev.split("-")[0]) < conflictedRevNo).first()?.rev ?? "";
const result = await this.mergeObject(id, commonBase, doc._rev, conflictedRev);
if (result) {
Logger(`Object merge:${id}`, LOG_LEVEL.INFO);
const filename = id2filenameInternalChunk(id);
const isExists = await this.app.vault.adapter.exists(filename);
if (!isExists) {
await this.ensureDirectoryEx(filename);
}
await this.app.vault.adapter.write(filename, result);
const stat = await this.app.vault.adapter.stat(filename);
await this.storeInternalFileToDatabase({ path: filename, ...stat });
await this.extractInternalFileFromDatabase(filename);
await this.localDatabase.localDatabase.remove(id, revB);
return this.resolveConflictOnInternalFile(id);
} else {
Logger(`Object merge is not applicable.`, LOG_LEVEL.VERBOSE);
}
const revBDoc = await this.localDatabase.localDatabase.get(id, { rev: revB });
// determine which revision should been deleted.

127
src/utils.ts
View File

@@ -72,4 +72,129 @@ export function retrieveMemoObject<T>(key: string): T | false {
}
export function disposeMemoObject(key: string) {
delete memos[key];
}
}
export function isSensibleMargeApplicable(path: string) {
if (path.endsWith(".md")) return true;
return false;
}
export function isObjectMargeApplicable(path: string) {
if (path.endsWith(".canvas")) return true;
if (path.endsWith(".json")) return true;
return false;
}
export function tryParseJSON(str: string, fallbackValue?: any) {
try {
return JSON.parse(str);
} catch (ex) {
return fallbackValue;
}
}
const MARK_OPERATOR = `\u{0001}`;
const MARK_DELETED = `${MARK_OPERATOR}__DELETED`;
const MARK_ISARRAY = `${MARK_OPERATOR}__ARRAY`;
const MARK_SWAPPED = `${MARK_OPERATOR}__SWAP`;
function unorderedArrayToObject(obj: Array<any>) {
return obj.map(e => ({ [e.id as string]: e })).reduce((p, c) => ({ ...p, ...c }), {})
}
function objectToUnorderedArray(obj: object) {
const entries = Object.entries(obj);
if (entries.some(e => e[0] != e[1]?.id)) throw new Error("Item looks like not unordered array")
return entries.map(e => e[1]);
}
function generatePatchUnorderedArray(from: Array<any>, to: Array<any>) {
if (from.every(e => typeof (e) == "object" && ("id" in e)) && to.every(e => typeof (e) == "object" && ("id" in e))) {
const fObj = unorderedArrayToObject(from);
const tObj = unorderedArrayToObject(to);
const diff = generatePatchObj(fObj, tObj);
if (Object.keys(diff).length > 0) {
return { [MARK_ISARRAY]: diff };
} else {
return {};
}
}
return { [MARK_SWAPPED]: to };
}
export function generatePatchObj(from: Record<string | number | symbol, any>, to: Record<string | number | symbol, any>) {
const entries = Object.entries(from);
const tempMap = new Map<string | number | symbol, any>(entries);
const ret = {} as Record<string | number | symbol, any>;
const newEntries = Object.entries(to);
for (const [key, value] of newEntries) {
if (!tempMap.has(key)) {
//New
ret[key] = value;
tempMap.delete(key);
} else {
//Exists
const v = tempMap.get(key);
if (typeof (v) !== typeof (value) || (Array.isArray(v) !== Array.isArray(value))) {
//if type is not match, replace completely.
ret[key] = { [MARK_SWAPPED]: value };
} else {
if (typeof (v) == "object" && typeof (value) == "object" && !Array.isArray(v) && !Array.isArray(value)) {
const wk = generatePatchObj(v, value);
if (Object.keys(wk).length > 0) ret[key] = wk;
} else if (typeof (v) == "object" && typeof (value) == "object" && Array.isArray(v) && Array.isArray(value)) {
const wk = generatePatchUnorderedArray(v, value);
if (Object.keys(wk).length > 0) ret[key] = wk;
} else if (typeof (v) != "object" && typeof (value) != "object") {
if (JSON.stringify(tempMap.get(key)) !== JSON.stringify(value)) {
ret[key] = value;
}
} else {
if (JSON.stringify(tempMap.get(key)) !== JSON.stringify(value)) {
ret[key] = { [MARK_SWAPPED]: value };
}
}
}
tempMap.delete(key);
}
}
//Not used item, means deleted one
for (const [key,] of tempMap) {
ret[key] = MARK_DELETED
}
return ret;
}
export function applyPatch(from: Record<string | number | symbol, any>, patch: Record<string | number | symbol, any>) {
const ret = from;
const patches = Object.entries(patch);
for (const [key, value] of patches) {
if (value == MARK_DELETED) {
delete ret[key];
continue;
}
if (typeof (value) == "object") {
if (MARK_SWAPPED in value) {
ret[key] = value[MARK_SWAPPED];
continue;
}
if (MARK_ISARRAY in value) {
if (!(key in ret)) ret[key] = [];
if (!Array.isArray(ret[key])) {
throw new Error("Patch target type is mismatched (array to something)");
}
const orgArrayObject = unorderedArrayToObject(ret[key]);
const appliedObject = applyPatch(orgArrayObject, value[MARK_ISARRAY]);
const appliedArray = objectToUnorderedArray(appliedObject);
ret[key] = [...appliedArray];
} else {
if (!(key in ret)) {
ret[key] = value;
continue;
}
ret[key] = applyPatch(ret[key], value);
}
} else {
ret[key] = value;
}
}
return ret;
}

14
styles.css
View File

@@ -85,13 +85,6 @@
} */
.sls-btn-left {
padding-right: 4px;
}
.sls-btn-right {
padding-left: 4px;
}
.sls-header-button {
margin-left: 2em;
@@ -106,7 +99,8 @@
}
.CodeMirror-wrap::before,
.cm-s-obsidian>.cm-editor::before {
.cm-s-obsidian>.cm-editor::before,
.canvas-wrapper::before {
content: var(--slsmessage);
text-align: right;
white-space: pre-wrap;
@@ -122,6 +116,10 @@
filter: grayscale(100%);
}
.canvas-wrapper::before {
right: 48px;
}
.CodeMirror-wrap::before {
right: 0px;
}

84
updates.md
View File

@@ -1,3 +1,46 @@
### 0.17.0
- 0.17.0 has no surfaced changes but the design of saving chunks has been changed. They have compatibility but changing files after upgrading makes different chunks than before 0.16.x.
Please rebuild databases once if you have been worried about storage usage.
- Improved:
- Splitting markdown
- Saving chunks
- Changed:
- Chunk ID numbering rules
#### Minors
- 0.17.1
- Fixed: Now we can verify and repair the database.
- Refactored inside.
- 0.17.2
- New feature
- We can merge conflicted documents automatically if sensible.
- Fixed
- Writing to the storage will be pended while they have conflicts after replication.
- 0.17.3
- Now we supported canvas! And conflicted JSON files are also synchronised with merging its content if they are obvious.
- 0.17.4
- Canvases are now treated as a sort of plain text file. now we transfer only the metadata and chunks that have differences.
- 0.17.5 Now `read chunks online` had been fixed, and a new feature: `Use dynamic iteration count` to reduce the load on encryption/decryption.
Note: `Use dynamic iteration count` is not compatible with earlier versions.
- 0.17.6 Now our renamed/deleted files have been surely deleted again.
- 0.17.7
- Fixed:
- Fixed merging issues.
- Fixed button styling.
- Changed:
- Conflict checking on synchronising has been enabled for every note in default.
- 0.17.8
- Improved: Performance improved. Prebuilt PouchDB is no longer used.
- Fixed: Merging hidden files is also fixed.
- New Feature: Now we can synchronise automatically after merging conflicts.
### 0.16.0
- Now hidden files need not be scanned. Changes will be detected automatically.
- If you want it to back to its previous behaviour, please disable `Monitor changes to internal files`.
@@ -10,25 +53,24 @@
- 0.16.3
- Fixed detection of IBM Cloudant (And if there are some issues, be fixed automatically).
- A configuration information reporting tool has been implemented.
- 0.16.4 Fixed detection failure. Please set the `Chunk size` again when using a self-hosted database.
- 0.16.5
- Fixed
- Conflict detection and merging now be able to treat deleted files.
- Logs while the boot-up sequence has been tidied up.
- Fixed incorrect log entries.
- New Feature
- The feature of automatically deleting old expired metadata has been implemented.
We can configure it in `Delete old metadata of deleted files on start-up` in the `General Settings` pane.
- 0.16.6
- Fixed
- Automatic (temporary) batch size adjustment has been restored to work correctly.
- Chunk splitting has been backed to the previous behaviour for saving them correctly.
- Improved
- Corrupted chunks will be detected automatically.
- Now on the case-insensitive system, `aaa.md` and `AAA.md` will be treated as the same file or path at applying changesets.
- 0.16.7 Nothing has been changed except toolsets, framework library, and as like them. Please inform me if something had been getting strange!
- 0.16.8 Now we can synchronise without `bad_request:invalid UTF-8 JSON` even while end-to-end encryption has been disabled.
### 0.15.0
- Outdated configuration items have been removed.
- Setup wizard has been implemented!
I appreciate for reviewing and giving me advice @Pouhon158!
#### Minors
- 0.15.1 Missed the stylesheet.
- 0.15.2 The wizard has been improved and documented!
- 0.15.3 Fixed the issue about locking/unlocking remote database while rebuilding in the wizard.
- 0.15.4 Fixed issues about asynchronous processing (e.g., Conflict check or hidden file detection)
- 0.15.5 Add new features for setting Self-hosted LiveSync up more easier.
- 0.15.6 File tracking logic has been refined.
- 0.15.7 Fixed bug about renaming file.
- 0.15.8 Fixed bug about deleting empty directory, weird behaviour on boot-sequence on mobile devices.
- 0.15.9 Improved chunk retrieving, now chunks are retrieved in batch on continuous requests.
- 0.15.10 Fixed:
- The boot sequence has been corrected and now boots smoothly.
- Auto applying of batch save will be processed earlier than before.
... To continue on to `updates_old.md`.
Note:
Before 0.16.5, LiveSync had some issues making chunks. In this case, synchronisation had became been always failing after a corrupted one should be made. After 0.16.6, the corrupted chunk is automatically detected. Sorry for troubling you but please do `rebuild everything` when this plug-in notified so.