public/obsidian-livesync
1
0
Fork 0
mirror of https://github.com/vrtmrz/obsidian-livesync.git synced 2026-05-18 05:21:22 +00:00
Code Issues Packages Projects Releases Wiki Activity

Compare commits

base: public:0.1.10
Branches Tags
public:main public:enhance_fix_p2p public:test_real_obsidian public:v0_25_62 public:update_lib public:feat-userhashsalt public:auto_conf_2 public:auto_conf public:beta public:alpha public:refactor_services_2 public:refactor_services public:dev public:svelteui public:forprivacy
public:0.25.64 public:0.25.63 public:0.25.62 public:0.25.61 public:0.25.60 public:0.25.59 public:0.25.58 public:0.25.57 public:0.25.56+patched5 public:0.25.56+patched4 public:0.25.56+patched3 public:0.25.56+patched2 public:0.25.56+patched1 public:0.25.56 public:0.25.55 public:0.25.54 public:0.25.53 public:0.25.52-patched-3 public:0.25.52-patched-2 public:0.25.52-patched-1 public:0.25.52 public:0.25.51 public:0.25.50 public:0.25.49 public:0.25.48 public:0.25.47 public:0.25.46 public:0.25.45 public:0.25.44 public:0.25.43-patched-9 public:0.25.43-patched-8 public:0.25.43-patched-7 public:0.25.43-patched-6 public:0.25.43-patched-5 public:0.25.43-patched-4 public:0.25.43-patched-3 public:0.25.43-patched-2 public:0.25.43-patched-1 public:0.25.43 public:0.25.42 public:0.25.41-patched-3 public:0.25.41-patched-2 public:0.25.41-patched-1 public:0.25.41 public:0.25.40 public:0.25.39 public:0.25.38 public:0.25.37 public:0.25.36 public:0.25.35 public:0.25.34 public:0.25.33 public:0.25.32 public:0.25.31 public:0.25.30 public:0.25.30-dev public:0.25.29 public:0.25.28 public:0.25.27 public:0.25.26 public:0.25.25 public:0.25.24 public:0.25.24.beta3 public:0.25.24.beta2 public:0.25.24.beta1 public:0.25.23 public:0.25.23.beta1 public:0.25.22 public:0.25.21 public:0.25.21.beta2 public:0.25.21.beta1 public:0.25.20 public:0.25.19 public:0.25.18 public:0.25.17 public:0.25.16 public:0.25.15 public:0.25.14 public:0.25.13 public:0.25.12 public:0.25.11 public:0.25.10 public:0.25.9 public:0.25.8 public:0.25.7 public:0.25.6 public:0.25.5 public:0.25.4 public:0.25.3 public:0.25.2 public:0.25.1 public:0.25.0 public:0.25.0-beta1 public:0.24.31 public:0.24.30 public:0.24.29 public:0.24.28 public:0.24.27 public:0.24.26 public:0.24.25 public:0.24.24 public:0.24.23 public:0.24.22 public:0.24.21 public:0.24.20 public:0.24.19 public:0.24.18 public:0.24.17 public:0.24.16 public:0.24.15 public:0.24.14 public:0.24.13 public:0.24.12 public:0.24.11 public:0.24.10 public:0.24.9 public:0.24.8 public:0.24.7 public:0.24.6 public:0.24.5 public:0.24.4 public:0.24.3 public:0.24.2 public:0.24.1 public:0.24.0 public:0.24.0.dev-rc8 public:0.24.0.dev-rc7 public:0.24.0.dev-rc6 public:0.24.0.dev-rc5 public:0.24.0.dev-rc4 public:0.24.0.dev-rc3 public:0.24.0.dev-rc2 public:0.23.23 public:0.23.22 public:0.23.21 public:0.23.20 public:0.23.19 public:0.23.18 public:0.23.17 public:0.23.16 public:0.23.15 public:0.23.14 public:0.23.13 public:0.23.12 public:0.23.11 public:0.23.10 public:0.23.9 public:0.23.8 public:0.23.7 public:0.23.6 public:0.23.5 public:0.23.4 public:0.23.3 public:0.23.2 public:0.23.1 public:0.23.0 public:0.22.19 public:0.22.18 public:0.22.17 public:0.22.16 public:0.22.15 public:0.22.14 public:0.22.13 public:0.22.12 public:0.22.11 public:0.22.10 public:0.22.9 public:0.22.8 public:0.22.7 public:0.22.6 public:0.22.5 public:0.22.4 public:0.22.3 public:0.22.2 public:0.22.1 public:0.22.0 public:0.21.5 public:0.21.4 public:0.21.3 public:0.21.2 public:0.21.1 public:0.21.0 public:0.20.7 public:0.20.6 public:0.20.5 public:0.20.4 public:0.20.3 public:0.20.2 public:0.20.1 public:0.20.0 public:0.19.23 public:0.19.22 public:0.19.21 public:0.19.20 public:0.19.19 public:0.19.18 public:0.19.17 public:0.19.16 public:0.19.15 public:0.19.14 public:0.19.13 public:0.19.12 public:0.19.11 public:0.19.10 public:0.19.9 public:0.19.8 public:0.19.7 public:0.19.6 public:0.19.5 public:0.19.4 public:0.19.3 public:0.19.2 public:0.19.1 public:0.19.0 public:0.18.6 public:0.18.5 public:0.18.4 public:0.18.3 public:0.18.2 public:0.18.1 public:0.18.0 public:0.17.34 public:0.17.33 public:0.17.32 public:0.17.31 public:0.17.30 public:0.17.29 public:0.17.28 public:0.17.27 public:0.17.26 public:0.17.25 public:0.17.24 public:0.17.23 public:0.17.22 public:0.17.21 public:0.17.20 public:0.17.19 public:0.17.18 public:0.17.17 public:0.17.16 public:0.17.15 public:0.17.14 public:0.17.13 public:0.17.12 public:0.17.11 public:0.17.10 public:0.17.9 public:0.17.8 public:0.17.7 public:0.17.6 public:0.17.5 public:0.17.4 public:0.17.3 public:0.17.2 public:0.17.1 public:0.17.0 public:0.16.8 public:0.16.7 public:0.16.6 public:0.16.5 public:0.16.4 public:0.16.3 public:0.16.2 public:0.16.1 public:0.16.0 public:0.15.10 public:0.15.9 public:0.15.8 public:0.15.7 public:0.15.6 public:0.15.5 public:0.15.4 public:0.15.3 public:0.15.2 public:0.15.1 public:0.15.0 public:0.14.8 public:0.14.7 public:0.14.6 public:0.14.5 public:0.14.4 public:0.14.3 public:0.14.2 public:0.14.1 public:0.14.0 public:0.13.4 public:0.13.3 public:0.13.2 public:0.13.1 public:0.13.0 public:0.12.5 public:0.12.4 public:0.12.3 public:0.12.2 public:0.12.1 public:0.12.0 public:0.11.10 public:0.11.9 public:0.11.8 public:0.11.7 public:0.11.6 public:0.11.5 public:0.11.4 public:0.11.3 public:0.11.2 public:0.11.1 public:0.11.0 public:0.10.1 public:0.10.0 public:0.9.0 public:0.8.8 public:0.8.7 public:0.8.6 public:0.8.5 public:0.8.3 public:0.8.2 public:0.8.1 public:0.8.0 public:0.7.2 public:0.7.1 public:0.7.0 public:0.6.1 public:0.6.0 public:0.5.0 public:0.4.1 public:0.4.0 public:0.3.1 public:0.3.0 public:0.2.1 public:0.2.0 public:0.1.26 public:0.1.25 public:0.1.24 public:0.1.23 public:0.1.22 public:0.1.21 public:0.1.20 public:0.1.19 public:0.1.18 public:0.1.17 public:0.1.16 public:0.1.15 public:0.1.14 public:0.1.13 public:0.1.12 public:0.1.11 public:0.1.10 public:0.1.9 public:0.1.8 public:0.1.7 public:0.1.6 public:0.1.5 public:0.1.4 public:0.1.3 public:0.1.2 public:0.1.1 public:0.1.0 public:0.0.9 public:0.0.8 public:0.0.7 public:0.0.6
...
compare: public:0.1.14
Branches Tags
public:main public:enhance_fix_p2p public:test_real_obsidian public:v0_25_62 public:update_lib public:feat-userhashsalt public:auto_conf_2 public:auto_conf public:beta public:alpha public:refactor_services_2 public:refactor_services public:dev public:svelteui public:forprivacy
public:0.25.64 public:0.25.63 public:0.25.62 public:0.25.61 public:0.25.60 public:0.25.59 public:0.25.58 public:0.25.57 public:0.25.56+patched5 public:0.25.56+patched4 public:0.25.56+patched3 public:0.25.56+patched2 public:0.25.56+patched1 public:0.25.56 public:0.25.55 public:0.25.54 public:0.25.53 public:0.25.52-patched-3 public:0.25.52-patched-2 public:0.25.52-patched-1 public:0.25.52 public:0.25.51 public:0.25.50 public:0.25.49 public:0.25.48 public:0.25.47 public:0.25.46 public:0.25.45 public:0.25.44 public:0.25.43-patched-9 public:0.25.43-patched-8 public:0.25.43-patched-7 public:0.25.43-patched-6 public:0.25.43-patched-5 public:0.25.43-patched-4 public:0.25.43-patched-3 public:0.25.43-patched-2 public:0.25.43-patched-1 public:0.25.43 public:0.25.42 public:0.25.41-patched-3 public:0.25.41-patched-2 public:0.25.41-patched-1 public:0.25.41 public:0.25.40 public:0.25.39 public:0.25.38 public:0.25.37 public:0.25.36 public:0.25.35 public:0.25.34 public:0.25.33 public:0.25.32 public:0.25.31 public:0.25.30 public:0.25.30-dev public:0.25.29 public:0.25.28 public:0.25.27 public:0.25.26 public:0.25.25 public:0.25.24 public:0.25.24.beta3 public:0.25.24.beta2 public:0.25.24.beta1 public:0.25.23 public:0.25.23.beta1 public:0.25.22 public:0.25.21 public:0.25.21.beta2 public:0.25.21.beta1 public:0.25.20 public:0.25.19 public:0.25.18 public:0.25.17 public:0.25.16 public:0.25.15 public:0.25.14 public:0.25.13 public:0.25.12 public:0.25.11 public:0.25.10 public:0.25.9 public:0.25.8 public:0.25.7 public:0.25.6 public:0.25.5 public:0.25.4 public:0.25.3 public:0.25.2 public:0.25.1 public:0.25.0 public:0.25.0-beta1 public:0.24.31 public:0.24.30 public:0.24.29 public:0.24.28 public:0.24.27 public:0.24.26 public:0.24.25 public:0.24.24 public:0.24.23 public:0.24.22 public:0.24.21 public:0.24.20 public:0.24.19 public:0.24.18 public:0.24.17 public:0.24.16 public:0.24.15 public:0.24.14 public:0.24.13 public:0.24.12 public:0.24.11 public:0.24.10 public:0.24.9 public:0.24.8 public:0.24.7 public:0.24.6 public:0.24.5 public:0.24.4 public:0.24.3 public:0.24.2 public:0.24.1 public:0.24.0 public:0.24.0.dev-rc8 public:0.24.0.dev-rc7 public:0.24.0.dev-rc6 public:0.24.0.dev-rc5 public:0.24.0.dev-rc4 public:0.24.0.dev-rc3 public:0.24.0.dev-rc2 public:0.23.23 public:0.23.22 public:0.23.21 public:0.23.20 public:0.23.19 public:0.23.18 public:0.23.17 public:0.23.16 public:0.23.15 public:0.23.14 public:0.23.13 public:0.23.12 public:0.23.11 public:0.23.10 public:0.23.9 public:0.23.8 public:0.23.7 public:0.23.6 public:0.23.5 public:0.23.4 public:0.23.3 public:0.23.2 public:0.23.1 public:0.23.0 public:0.22.19 public:0.22.18 public:0.22.17 public:0.22.16 public:0.22.15 public:0.22.14 public:0.22.13 public:0.22.12 public:0.22.11 public:0.22.10 public:0.22.9 public:0.22.8 public:0.22.7 public:0.22.6 public:0.22.5 public:0.22.4 public:0.22.3 public:0.22.2 public:0.22.1 public:0.22.0 public:0.21.5 public:0.21.4 public:0.21.3 public:0.21.2 public:0.21.1 public:0.21.0 public:0.20.7 public:0.20.6 public:0.20.5 public:0.20.4 public:0.20.3 public:0.20.2 public:0.20.1 public:0.20.0 public:0.19.23 public:0.19.22 public:0.19.21 public:0.19.20 public:0.19.19 public:0.19.18 public:0.19.17 public:0.19.16 public:0.19.15 public:0.19.14 public:0.19.13 public:0.19.12 public:0.19.11 public:0.19.10 public:0.19.9 public:0.19.8 public:0.19.7 public:0.19.6 public:0.19.5 public:0.19.4 public:0.19.3 public:0.19.2 public:0.19.1 public:0.19.0 public:0.18.6 public:0.18.5 public:0.18.4 public:0.18.3 public:0.18.2 public:0.18.1 public:0.18.0 public:0.17.34 public:0.17.33 public:0.17.32 public:0.17.31 public:0.17.30 public:0.17.29 public:0.17.28 public:0.17.27 public:0.17.26 public:0.17.25 public:0.17.24 public:0.17.23 public:0.17.22 public:0.17.21 public:0.17.20 public:0.17.19 public:0.17.18 public:0.17.17 public:0.17.16 public:0.17.15 public:0.17.14 public:0.17.13 public:0.17.12 public:0.17.11 public:0.17.10 public:0.17.9 public:0.17.8 public:0.17.7 public:0.17.6 public:0.17.5 public:0.17.4 public:0.17.3 public:0.17.2 public:0.17.1 public:0.17.0 public:0.16.8 public:0.16.7 public:0.16.6 public:0.16.5 public:0.16.4 public:0.16.3 public:0.16.2 public:0.16.1 public:0.16.0 public:0.15.10 public:0.15.9 public:0.15.8 public:0.15.7 public:0.15.6 public:0.15.5 public:0.15.4 public:0.15.3 public:0.15.2 public:0.15.1 public:0.15.0 public:0.14.8 public:0.14.7 public:0.14.6 public:0.14.5 public:0.14.4 public:0.14.3 public:0.14.2 public:0.14.1 public:0.14.0 public:0.13.4 public:0.13.3 public:0.13.2 public:0.13.1 public:0.13.0 public:0.12.5 public:0.12.4 public:0.12.3 public:0.12.2 public:0.12.1 public:0.12.0 public:0.11.10 public:0.11.9 public:0.11.8 public:0.11.7 public:0.11.6 public:0.11.5 public:0.11.4 public:0.11.3 public:0.11.2 public:0.11.1 public:0.11.0 public:0.10.1 public:0.10.0 public:0.9.0 public:0.8.8 public:0.8.7 public:0.8.6 public:0.8.5 public:0.8.3 public:0.8.2 public:0.8.1 public:0.8.0 public:0.7.2 public:0.7.1 public:0.7.0 public:0.6.1 public:0.6.0 public:0.5.0 public:0.4.1 public:0.4.0 public:0.3.1 public:0.3.0 public:0.2.1 public:0.2.0 public:0.1.26 public:0.1.25 public:0.1.24 public:0.1.23 public:0.1.22 public:0.1.21 public:0.1.20 public:0.1.19 public:0.1.18 public:0.1.17 public:0.1.16 public:0.1.15 public:0.1.14 public:0.1.13 public:0.1.12 public:0.1.11 public:0.1.10 public:0.1.9 public:0.1.8 public:0.1.7 public:0.1.6 public:0.1.5 public:0.1.4 public:0.1.3 public:0.1.2 public:0.1.1 public:0.1.0 public:0.0.9 public:0.0.8 public:0.0.7 public:0.0.6

4 Commits
0.1.10 ... 0.1.14

Author SHA1 Message Date
vorotamoroz
9facb57760 Bug fixed and new feature implemented 
- Synchronization Timing problem fixed
- Performance improvement of handling large files
- Timeout for collecting leaves extended
- Periodic synchronization implemented
- Dumping document information implemented.
- Folder watching problem fixed.
- Delay vault watching for database ready.
 2021-11-10 18:07:09 +09:00
vorotamoroz
155439ed56 URG, attachment doesn't captured. 2021-11-10 10:06:36 +09:00
vorotamoroz
04e3004aca Improved: 
Folder deletion and renaming are now tracked.
Database update fixed up. But be a little heavier.
Touched up the readme.
 2021-11-09 15:05:12 +09:00
vorotamoroz
53b4d4cd20 fixes below: 
Make text selectable in log dialog.
Dumping errors when couldn't connect
Invalid uri could not detected.
Add tooltips to statusbar
 2021-11-08 17:52:07 +09:00
6 changed files with 381 additions and 122 deletions
Expand all files Collapse all files

26
README.md
View File

@@ -1,4 +1,5 @@
# Self-hosted LiveSync # Self-hosted LiveSync
**Renamed from: obsidian-livesync** **Renamed from: obsidian-livesync**
This is the obsidian plugin that enables livesync between multi-devices with self-hosted database. This is the obsidian plugin that enables livesync between multi-devices with self-hosted database.
@@ -11,7 +12,7 @@ Community implementation, not compatible with official "Sync".
**It's getting almost stable now, But Please make sure to back your vault up!** **It's getting almost stable now, But Please make sure to back your vault up!**
Limitations: Folder deletion handling is not completed. Limitations: ~~Folder deletion handling is not completed.~~ **It would work now.**
## This plugin enables.. ## This plugin enables..
@@ -77,7 +78,6 @@ Note: The figure is drawn as single-directional, between two devices. But everyt
![dedupe](images/2.png) ![dedupe](images/2.png)
## Cloudant Setup ## Cloudant Setup
### Creating an Instance ### Creating an Instance
@@ -105,7 +105,7 @@ Select Multitenant(it's the default) and the region as you like.
![step 8](instruction_images/cloudant_8.png) ![step 8](instruction_images/cloudant_8.png)
7. In resource details, there's information to connect from self-hosted-livesync. 7. In resource details, there's information to connect from self-hosted-livesync.
Copy the "External Endpoint(preferred)" address. <sup>(\*1)</sup> Copy the "External Endpoint(preferred)" address. <sup>(\*1)</sup>. We use this address later, with the database name.
![step 9](instruction_images/cloudant_9.png) ![step 9](instruction_images/cloudant_9.png)
### CouchDB setup ### CouchDB setup
@@ -120,11 +120,13 @@ Select Multitenant(it's the default) and the region as you like.
_NOTE: of course We want to set "app://obsidian.md" but it's not acceptable on Cloudant._ _NOTE: of course We want to set "app://obsidian.md" but it's not acceptable on Cloudant._
![step 2](instruction_images/couchdb_2.png) ![step 2](instruction_images/couchdb_2.png)
1. And open the "Databases" tab and hit the "Create Database" button. 1. And open the "Databases" tab and hit the "Create Database" button.
Enter the name as you like <sup>(\*2)</sup> and Hit the "Create" button below. Enter the name as you like <sup>(\*2)</sup> and Hit the "Create" button below.
![step 3](instruction_images/couchdb_3.png) ![step 3](instruction_images/couchdb_3.png)
1. If the database was shown with joyful messages, then you can close this browser tab now. 1. If the database was shown with joyful messages, setup is almost done.
And, once you have confirmed that you can create a database, usullay there is no need to open this screen.
You can create a database from Self-hosted LiveSync.
![step 4](instruction_images/couchdb_4.png) ![step 4](instruction_images/couchdb_4.png)
### Credentials Setup ### Credentials Setup
@@ -135,7 +137,7 @@ Select Multitenant(it's the default) and the region as you like.
1. The dialog to create a credential will be shown. 1. The dialog to create a credential will be shown.
type any name or leave it default, hit the "Add" button. type any name or leave it default, hit the "Add" button.
![step 2](instruction_images/credentials_2.png) ![step 2](instruction_images/credentials_2.png)
_NOTE: This "name" is not related to your username that uses in self-hosted-livesync._ _NOTE: This "name" is not related to your username that uses in Self-hosted LiveSync._
1. Back to "Service credentials", the new credential should be created. 1. Back to "Service credentials", the new credential should be created.
open details. open details.
@@ -145,16 +147,16 @@ Select Multitenant(it's the default) and the region as you like.
follow the figure, it's follow the figure, it's
"apikey-v2-2unu15184f7o8emr90xlqgkm2ncwhbltml6tgnjl9sd5"<sup>(\*3)</sup> and "c2c11651d75497fa3d3c486e4c8bdf27"<sup>(\*4)</sup> "apikey-v2-2unu15184f7o8emr90xlqgkm2ncwhbltml6tgnjl9sd5"<sup>(\*3)</sup> and "c2c11651d75497fa3d3c486e4c8bdf27"<sup>(\*4)</sup>
### self-hosted-livesync setting ### Self-hosted LiveSync setting
![xx](instruction_images/obsidian_sync_1.png) ![xx](instruction_images/obsidian_sync_1.png)
example values. example values.
| Items | Value | example | | Items | Value | example |
| ------------------- | ----------- | --------------------------------------------------------------------------- | | ------------------- | -------------------------------- | --------------------------------------------------------------------------- |
| CouchDB Remote URI: | (\*1)/(\*2) | https://xxxxxxxxxxxxxxxxx-bluemix.cloudantnosqldb.appdomain.cloud/sync-test | | CouchDB Remote URI: | (\*1)/(\*2) or any favorite name | https://xxxxxxxxxxxxxxxxx-bluemix.cloudantnosqldb.appdomain.cloud/sync-test |
| CouchDB Username | (\*3) | apikey-v2-2unu15184f7o8emr90xlqgkm2ncwhbltml6tgnjl9sd5 | | CouchDB Username | (\*3) | apikey-v2-2unu15184f7o8emr90xlqgkm2ncwhbltml6tgnjl9sd5 |
| CouchDB Password | (\*4) | c2c11651d75497fa3d3c486e4c8bdf27 | | CouchDB Password | (\*4) | c2c11651d75497fa3d3c486e4c8bdf27 |
# License # License

468
main.ts
View File

@@ -1,4 +1,4 @@
import { App, debounce, Modal, Notice, Plugin, PluginSettingTab, Setting, TFile, addIcon, TFolder, normalizePath } from "obsidian"; import { App, debounce, Modal, Notice, Plugin, PluginSettingTab, Setting, TFile, addIcon, TFolder, normalizePath, TAbstractFile, Editor, MarkdownView } from "obsidian";
import { PouchDB } from "./pouchdb-browser-webpack/dist/pouchdb-browser"; import { PouchDB } from "./pouchdb-browser-webpack/dist/pouchdb-browser";
import { DIFF_DELETE, DIFF_EQUAL, DIFF_INSERT, diff_match_patch } from "diff-match-patch"; import { DIFF_DELETE, DIFF_EQUAL, DIFF_INSERT, diff_match_patch } from "diff-match-patch";
import xxhash from "xxhash-wasm"; import xxhash from "xxhash-wasm";
@@ -11,7 +11,7 @@ const MAX_DOC_SIZE_BIN = 102400; // 100kb
const VER = 10; const VER = 10;
const RECENT_MOFIDIED_DOCS_QTY = 30; const RECENT_MOFIDIED_DOCS_QTY = 30;
const LEAF_WAIT_TIMEOUT = 30000; // in synchronization, waiting missing leaf time out. const LEAF_WAIT_TIMEOUT = 90000; // in synchronization, waiting missing leaf time out.
const LOG_LEVEL = { const LOG_LEVEL = {
VERBOSE: 1, VERBOSE: 1,
INFO: 10, INFO: 10,
@@ -31,6 +31,7 @@ interface ObsidianLiveSyncSettings {
liveSync: boolean; liveSync: boolean;
syncOnSave: boolean; syncOnSave: boolean;
syncOnStart: boolean; syncOnStart: boolean;
syncOnFileOpen: boolean;
savingDelay: number; savingDelay: number;
lessInformationInLog: boolean; lessInformationInLog: boolean;
gcDelay: number; gcDelay: number;
@@ -40,6 +41,8 @@ interface ObsidianLiveSyncSettings {
showVerboseLog: boolean; showVerboseLog: boolean;
suspendFileWatching: boolean; suspendFileWatching: boolean;
trashInsteadDelete: boolean; trashInsteadDelete: boolean;
periodicReplication: boolean;
periodicReplicationInterval: number;
} }
const DEFAULT_SETTINGS: ObsidianLiveSyncSettings = { const DEFAULT_SETTINGS: ObsidianLiveSyncSettings = {
@@ -58,6 +61,9 @@ const DEFAULT_SETTINGS: ObsidianLiveSyncSettings = {
showVerboseLog: false, showVerboseLog: false,
suspendFileWatching: false, suspendFileWatching: false,
trashInsteadDelete: false, trashInsteadDelete: false,
periodicReplication: false,
periodicReplicationInterval: 60,
syncOnFileOpen: false,
}; };
interface Entry { interface Entry {
_id: string; _id: string;
@@ -159,7 +165,7 @@ type Credential = {
type EntryDocResponse = EntryDoc & PouchDB.Core.IdMeta & PouchDB.Core.GetMeta; type EntryDocResponse = EntryDoc & PouchDB.Core.IdMeta & PouchDB.Core.GetMeta;
//-->Functions. //-->Functions.
function arrayBufferToBase64(buffer: ArrayBuffer) { function arrayBufferToBase64Old(buffer: ArrayBuffer) {
var binary = ""; var binary = "";
var bytes = new Uint8Array(buffer); var bytes = new Uint8Array(buffer);
var len = bytes.byteLength; var len = bytes.byteLength;
@@ -168,6 +174,18 @@ function arrayBufferToBase64(buffer: ArrayBuffer) {
} }
return window.btoa(binary); return window.btoa(binary);
} }
// Ten times faster.
function arrayBufferToBase64(buffer: ArrayBuffer): Promise<string> {
return new Promise((res) => {
var blob = new Blob([buffer], { type: "application/octet-binary" });
var reader = new FileReader();
reader.onload = function (evt) {
var dataurl = evt.target.result.toString();
res(dataurl.substr(dataurl.indexOf(",") + 1));
};
reader.readAsDataURL(blob);
});
}
function base64ToArrayBuffer(base64: string): ArrayBuffer { function base64ToArrayBuffer(base64: string): ArrayBuffer {
try { try {
@@ -231,7 +249,7 @@ const isValidRemoteCouchDBURI = (uri: string): boolean => {
return false; return false;
}; };
const connectRemoteCouchDB = async (uri: string, auth: { username: string; password: string }): Promise<false | { db: PouchDB.Database; info: any }> => { const connectRemoteCouchDB = async (uri: string, auth: { username: string; password: string }): Promise<false | { db: PouchDB.Database; info: any }> => {
if (!isValidRemoteCouchDBURI(uri)) false; if (!isValidRemoteCouchDBURI(uri)) return false;
let db = new PouchDB(uri, { let db = new PouchDB(uri, {
auth, auth,
}); });
@@ -239,6 +257,7 @@ const connectRemoteCouchDB = async (uri: string, auth: { username: string; passw
let info = await db.info(); let info = await db.info();
return { db: db, info: info }; return { db: db, info: info };
} catch (ex) { } catch (ex) {
Logger(ex, LOG_LEVEL.VERBOSE);
return false; return false;
} }
}; };
@@ -314,6 +333,7 @@ class LocalPouchDB {
settings: ObsidianLiveSyncSettings; settings: ObsidianLiveSyncSettings;
localDatabase: PouchDB.Database<EntryDoc>; localDatabase: PouchDB.Database<EntryDoc>;
nodeid: string = ""; nodeid: string = "";
isReady: boolean = false;
recentModifiedDocs: string[] = []; recentModifiedDocs: string[] = [];
h32: (input: string, seed?: number) => string; h32: (input: string, seed?: number) => string;
@@ -337,10 +357,17 @@ class LocalPouchDB {
this.dbname = dbname; this.dbname = dbname;
this.settings = settings; this.settings = settings;
this.initializeDatabase(); // this.initializeDatabase();
} }
close() { close() {
this.localDatabase.close(); this.isReady = false;
if (this.changeHandler != null) {
this.changeHandler.cancel();
this.changeHandler.removeAllListeners();
}
if (this.localDatabase != null) {
this.localDatabase.close();
}
} }
status() { status() {
if (this.syncHandler == null) { if (this.syncHandler == null) {
@@ -369,6 +396,7 @@ class LocalPouchDB {
changeHandler: PouchDB.Core.Changes<{}> = null; changeHandler: PouchDB.Core.Changes<{}> = null;
async initializeDatabase() { async initializeDatabase() {
await this.prepareHashFunctions();
if (this.localDatabase != null) this.localDatabase.close(); if (this.localDatabase != null) this.localDatabase.close();
if (this.changeHandler != null) { if (this.changeHandler != null) {
this.changeHandler.cancel(); this.changeHandler.cancel();
@@ -379,6 +407,9 @@ class LocalPouchDB {
revs_limit: 100, revs_limit: 100,
deterministic_revs: true, deterministic_revs: true,
}); });
Logger("Database Info");
Logger(await this.localDatabase.info(), LOG_LEVEL.VERBOSE);
// initialize local node information. // initialize local node information.
let nodeinfo: EntryNodeInfo = await resolveWithIgnoreKnownError<EntryNodeInfo>(this.localDatabase.get(NODEINFO_DOCID), { let nodeinfo: EntryNodeInfo = await resolveWithIgnoreKnownError<EntryNodeInfo>(this.localDatabase.get(NODEINFO_DOCID), {
_id: NODEINFO_DOCID, _id: NODEINFO_DOCID,
@@ -389,6 +420,9 @@ class LocalPouchDB {
nodeinfo.nodeid = Math.random().toString(36).slice(-10); nodeinfo.nodeid = Math.random().toString(36).slice(-10);
await this.localDatabase.put(nodeinfo); await this.localDatabase.put(nodeinfo);
} }
this.localDatabase.on("close", () => {
this.isReady = false;
});
this.nodeid = nodeinfo.nodeid; this.nodeid = nodeinfo.nodeid;
// Traceing the leaf id // Traceing the leaf id
@@ -404,7 +438,7 @@ class LocalPouchDB {
this.docSeq = `${e.seq}`; this.docSeq = `${e.seq}`;
}); });
this.changeHandler = changes; this.changeHandler = changes;
await this.prepareHashFunctions(); this.isReady = true;
} }
async prepareHashFunctions() { async prepareHashFunctions() {
@@ -513,6 +547,7 @@ class LocalPouchDB {
children: [], children: [],
datatype: "newnote", datatype: "newnote",
}; };
return doc;
} }
} catch (ex) { } catch (ex) {
if (ex.status && ex.status == 404) { if (ex.status && ex.status == 404) {
@@ -522,7 +557,7 @@ class LocalPouchDB {
} }
return false; return false;
} }
async getDBEntry(id: string, opt?: PouchDB.Core.GetOptions, retryCount = 5): Promise<false | LoadedEntry> { async getDBEntry(id: string, opt?: PouchDB.Core.GetOptions, dump = false): Promise<false | LoadedEntry> {
try { try {
let obj: EntryDocResponse = null; let obj: EntryDocResponse = null;
if (opt) { if (opt) {
@@ -554,15 +589,28 @@ class LocalPouchDB {
if (typeof this.corruptedEntries[doc._id] != "undefined") { if (typeof this.corruptedEntries[doc._id] != "undefined") {
delete this.corruptedEntries[doc._id]; delete this.corruptedEntries[doc._id];
} }
if (dump) {
Logger(`Simple doc`);
Logger(doc);
}
return doc; return doc;
// simple note // simple note
} }
if (obj.type == "newnote" || obj.type == "plain") { if (obj.type == "newnote" || obj.type == "plain") {
// search childrens // search childrens
try { try {
if (dump) {
Logger(`Enhanced doc`);
Logger(obj);
}
let childrens; let childrens;
try { try {
childrens = await Promise.all(obj.children.map((e) => this.getDBLeaf(e))); childrens = await Promise.all(obj.children.map((e) => this.getDBLeaf(e)));
if (dump) {
Logger(`childrens:`);
Logger(childrens);
}
} catch (ex) { } catch (ex) {
Logger(`Something went wrong on reading elements of ${obj._id} from database.`, LOG_LEVEL.NOTICE); Logger(`Something went wrong on reading elements of ${obj._id} from database.`, LOG_LEVEL.NOTICE);
this.corruptedEntries[obj._id] = obj; this.corruptedEntries[obj._id] = obj;
@@ -582,6 +630,10 @@ class LocalPouchDB {
datatype: obj.type, datatype: obj.type,
_conflicts: obj._conflicts, _conflicts: obj._conflicts,
}; };
if (dump) {
Logger(`therefore:`);
Logger(doc);
}
if (typeof this.corruptedEntries[doc._id] != "undefined") { if (typeof this.corruptedEntries[doc._id] != "undefined") {
delete this.corruptedEntries[doc._id]; delete this.corruptedEntries[doc._id];
} }
@@ -662,7 +714,7 @@ class LocalPouchDB {
console.log("!" + v.id); console.log("!" + v.id);
} else { } else {
if (!v.id.startsWith("h:")) { if (!v.id.startsWith("h:")) {
console.log("?" + v.id); // console.log("?" + v.id);
} }
} }
} }
@@ -692,6 +744,18 @@ class LocalPouchDB {
Logger(`deleteDBEntryPrefix:deleted ${deleteCount} items, skipped ${notfound}`); Logger(`deleteDBEntryPrefix:deleted ${deleteCount} items, skipped ${notfound}`);
return true; return true;
} }
isPlainText(filename: string): boolean {
if (filename.endsWith(".md")) return true;
if (filename.endsWith(".txt")) return true;
if (filename.endsWith(".svg")) return true;
if (filename.endsWith(".html")) return true;
if (filename.endsWith(".csv")) return true;
if (filename.endsWith(".css")) return true;
if (filename.endsWith(".js")) return true;
if (filename.endsWith(".xml")) return true;
return false;
}
async putDBEntry(note: LoadedEntry) { async putDBEntry(note: LoadedEntry) {
let leftData = note.data; let leftData = note.data;
let savenNotes = []; let savenNotes = [];
@@ -701,10 +765,11 @@ class LocalPouchDB {
let pieceSize = MAX_DOC_SIZE_BIN; let pieceSize = MAX_DOC_SIZE_BIN;
let plainSplit = false; let plainSplit = false;
let cacheUsed = 0; let cacheUsed = 0;
if (note._id.endsWith(".md")) { if (this.isPlainText(note._id)) {
pieceSize = MAX_DOC_SIZE; pieceSize = MAX_DOC_SIZE;
plainSplit = true; plainSplit = true;
} }
let newLeafs: EntryLeaf[] = [];
do { do {
// To keep low bandwith and database size, // To keep low bandwith and database size,
// Dedup pieces on database. // Dedup pieces on database.
@@ -714,11 +779,11 @@ class LocalPouchDB {
// 3. \r\n\r\n should break // 3. \r\n\r\n should break
// 4. \n# should break. // 4. \n# should break.
let cPieceSize = pieceSize; let cPieceSize = pieceSize;
let minimumChunkSize = this.settings.minimumChunkSize;
if (minimumChunkSize < 10) minimumChunkSize = 10;
let longLineThreshold = this.settings.longLineThreshold;
if (longLineThreshold < 100) longLineThreshold = 100;
if (plainSplit) { if (plainSplit) {
let minimumChunkSize = this.settings.minimumChunkSize;
if (minimumChunkSize < 10) minimumChunkSize = 10;
let longLineThreshold = this.settings.longLineThreshold;
if (longLineThreshold < 100) longLineThreshold = 100;
cPieceSize = 0; cPieceSize = 0;
// lookup for next splittion . // lookup for next splittion .
// we're standing on "\n" // we're standing on "\n"
@@ -747,11 +812,13 @@ class LocalPouchDB {
} while (cPieceSize < minimumChunkSize); } while (cPieceSize < minimumChunkSize);
} }
// piece size determined.
let piece = leftData.substring(0, cPieceSize); let piece = leftData.substring(0, cPieceSize);
leftData = leftData.substring(cPieceSize); leftData = leftData.substring(cPieceSize);
processed++; processed++;
let leafid = ""; let leafid = "";
// Get has of piece. // Get hash of piece.
let hashedPiece: string = ""; let hashedPiece: string = "";
let hashQ: number = 0; // if hash collided, **IF**, count it up. let hashQ: number = 0; // if hash collided, **IF**, count it up.
let tryNextHash = false; let tryNextHash = false;
@@ -802,53 +869,72 @@ class LocalPouchDB {
data: piece, data: piece,
type: "leaf", type: "leaf",
}; };
let result = await this.localDatabase.put(d); newLeafs.push(d);
this.updateRecentModifiedDocs(result.id, result.rev, d._deleted); this.hashCache[piece] = leafid;
if (result.ok) { this.hashCacheRev[leafid] = piece;
Logger(`save ok:id:${result.id} rev:${result.rev}`, LOG_LEVEL.VERBOSE); made++;
this.hashCache[piece] = leafid;
this.hashCacheRev[leafid] = piece;
made++;
} else {
Logger("save faild");
}
} else { } else {
skiped++; skiped++;
} }
} }
savenNotes.push(leafid); savenNotes.push(leafid);
} while (leftData != ""); } while (leftData != "");
Logger(`note content saven, pieces:${processed} new:${made}, skip:${skiped}, cache:${cacheUsed}`); let saved = true;
let newDoc: PlainEntry | NewEntry = { if (newLeafs.length > 0) {
NewNote: true, try {
children: savenNotes, let result = await this.localDatabase.bulkDocs(newLeafs);
_id: note._id, for (let item of result) {
ctime: note.ctime, if ((item as any).ok) {
mtime: note.mtime, this.updateRecentModifiedDocs(item.id, item.rev, false);
size: note.size,
type: plainSplit ? "plain" : "newnote", Logger(`save ok:id:${item.id} rev:${item.rev}`, LOG_LEVEL.VERBOSE);
}; } else {
// Here for upsert logic, Logger(`save failed:id:${item.id} rev:${item.rev}`, LOG_LEVEL.NOTICE);
try { Logger(item);
let old = await this.localDatabase.get(newDoc._id); this.disposeHashCache();
if (!old.type || old.type == "notes" || old.type == "newnote" || old.type == "plain") { saved = false;
// simple use rev for new doc }
newDoc._rev = old._rev; }
} } catch (ex) {
} catch (ex) { Logger("ERROR ON SAVING LEAVES ");
if (ex.status && ex.status == 404) { Logger(ex);
// NO OP/ saved = false;
} else {
throw ex;
} }
} }
let r = await this.localDatabase.put(newDoc); if (saved) {
this.updateRecentModifiedDocs(r.id, r.rev, newDoc._deleted); Logger(`note content saven, pieces:${processed} new:${made}, skip:${skiped}, cache:${cacheUsed}`);
if (typeof this.corruptedEntries[note._id] != "undefined") { let newDoc: PlainEntry | NewEntry = {
delete this.corruptedEntries[note._id]; NewNote: true,
children: savenNotes,
_id: note._id,
ctime: note.ctime,
mtime: note.mtime,
size: note.size,
type: plainSplit ? "plain" : "newnote",
};
// Here for upsert logic,
try {
let old = await this.localDatabase.get(newDoc._id);
if (!old.type || old.type == "notes" || old.type == "newnote" || old.type == "plain") {
// simple use rev for new doc
newDoc._rev = old._rev;
}
} catch (ex) {
if (ex.status && ex.status == 404) {
// NO OP/
} else {
throw ex;
}
}
let r = await this.localDatabase.put(newDoc);
this.updateRecentModifiedDocs(r.id, r.rev, newDoc._deleted);
if (typeof this.corruptedEntries[note._id] != "undefined") {
delete this.corruptedEntries[note._id];
}
Logger(`note saven:${newDoc._id}:${r.rev}`);
} else {
Logger(`note coud not saved:${note._id}`);
} }
Logger(`note saven:${newDoc._id}:${r.rev}`);
} }
syncHandler: PouchDB.Replication.Sync<{}> = null; syncHandler: PouchDB.Replication.Sync<{}> = null;
@@ -899,12 +985,12 @@ class LocalPouchDB {
this.docSent += e.docs_written; this.docSent += e.docs_written;
this.docArrived += e.docs_read; this.docArrived += e.docs_read;
this.updateInfo(); this.updateInfo();
Logger(`sending..:${e.docs.length}`); Logger(`replicateAllToServer: sending..:${e.docs.length}`);
}) })
.on("complete", async (info) => { .on("complete", async (info) => {
this.syncStatus = "COMPLETED"; this.syncStatus = "COMPLETED";
this.updateInfo(); this.updateInfo();
Logger("Completed", LOG_LEVEL.NOTICE); Logger("replicateAllToServer: Completed", LOG_LEVEL.NOTICE);
replicate.cancel(); replicate.cancel();
replicate.removeAllListeners(); replicate.removeAllListeners();
res(true); res(true);
@@ -912,13 +998,21 @@ class LocalPouchDB {
.on("error", (e) => { .on("error", (e) => {
this.syncStatus = "ERRORED"; this.syncStatus = "ERRORED";
this.updateInfo(); this.updateInfo();
Logger("Pulling Replication error", LOG_LEVEL.NOTICE); Logger("replicateAllToServer: Pulling Replication error", LOG_LEVEL.INFO);
Logger(e); Logger(e);
replicate.cancel();
replicate.removeAllListeners();
rej(e); rej(e);
}); });
}); });
} }
async openReplication(setting: ObsidianLiveSyncSettings, keepAlive: boolean, showResult: boolean, callback: (e: PouchDB.Core.ExistingDocument<{}>[]) => Promise<void>) { async openReplication(setting: ObsidianLiveSyncSettings, keepAlive: boolean, showResult: boolean, callback: (e: PouchDB.Core.ExistingDocument<{}>[]) => Promise<void>) {
if (!this.isReady) {
Logger("Database is not ready.");
return false;
}
if (setting.versionUpFlash != "") { if (setting.versionUpFlash != "") {
new Notice("Open settings and check message, please."); new Notice("Open settings and check message, please.");
return; return;
@@ -972,11 +1066,15 @@ class LocalPouchDB {
let db = dbret.db; let db = dbret.db;
//replicate once //replicate once
this.syncStatus = "CONNECTED"; this.syncStatus = "CONNECTED";
Logger("Pull before replicate.");
Logger(await this.localDatabase.info(), LOG_LEVEL.VERBOSE);
Logger(await db.info(), LOG_LEVEL.VERBOSE);
let replicate = this.localDatabase.replicate.from(db, syncOptionBase); let replicate = this.localDatabase.replicate.from(db, syncOptionBase);
replicate replicate
.on("active", () => { .on("active", () => {
this.syncStatus = "CONNECTED"; this.syncStatus = "CONNECTED";
this.updateInfo(); this.updateInfo();
Logger("Replication pull activated.");
}) })
.on("change", async (e) => { .on("change", async (e) => {
// when in first run, replication will send us tombstone data // when in first run, replication will send us tombstone data
@@ -1003,6 +1101,7 @@ class LocalPouchDB {
this.syncHandler.cancel(); this.syncHandler.cancel();
this.syncHandler.removeAllListeners(); this.syncHandler.removeAllListeners();
} }
Logger("Replication pull completed.");
this.syncHandler = this.localDatabase.sync(db, syncOption); this.syncHandler = this.localDatabase.sync(db, syncOption);
this.syncHandler this.syncHandler
.on("active", () => { .on("active", () => {
@@ -1050,7 +1149,13 @@ class LocalPouchDB {
.on("error", (e) => { .on("error", (e) => {
this.syncStatus = "ERRORED"; this.syncStatus = "ERRORED";
this.updateInfo(); this.updateInfo();
Logger("Pulling Replication error", LOG_LEVEL.NOTICE); Logger("Pulling Replication error", LOG_LEVEL.INFO);
replicate.cancel();
replicate.removeAllListeners();
this.syncHandler.cancel();
this.syncHandler.removeAllListeners();
this.syncHandler = null;
// debugger;
Logger(e); Logger(e);
}); });
} }
@@ -1069,9 +1174,11 @@ class LocalPouchDB {
async resetDatabase() { async resetDatabase() {
if (this.changeHandler != null) { if (this.changeHandler != null) {
this.changeHandler.removeAllListeners();
this.changeHandler.cancel(); this.changeHandler.cancel();
} }
await this.closeReplication(); await this.closeReplication();
this.isReady = false;
await this.localDatabase.destroy(); await this.localDatabase.destroy();
this.localDatabase = null; this.localDatabase = null;
await this.initializeDatabase(); await this.initializeDatabase();
@@ -1079,7 +1186,6 @@ class LocalPouchDB {
Logger("Local Database Reset", LOG_LEVEL.NOTICE); Logger("Local Database Reset", LOG_LEVEL.NOTICE);
} }
async tryResetRemoteDatabase(setting: ObsidianLiveSyncSettings) { async tryResetRemoteDatabase(setting: ObsidianLiveSyncSettings) {
await this.closeReplication();
await this.closeReplication(); await this.closeReplication();
let uri = setting.couchDB_URI; let uri = setting.couchDB_URI;
let auth: Credential = { let auth: Credential = {
@@ -1180,9 +1286,11 @@ class LocalPouchDB {
let readCount = 0; let readCount = 0;
let hashPieces: string[] = []; let hashPieces: string[] = [];
let usedPieces: string[] = []; let usedPieces: string[] = [];
Logger("Collecting Garbage");
do { do {
let result = await this.localDatabase.allDocs({ include_docs: true, skip: c, limit: 100, conflicts: true }); let result = await this.localDatabase.allDocs({ include_docs: true, skip: c, limit: 500, conflicts: true });
readCount = result.rows.length; readCount = result.rows.length;
Logger("checked:" + readCount);
if (readCount > 0) { if (readCount > 0) {
//there are some result //there are some result
for (let v of result.rows) { for (let v of result.rows) {
@@ -1208,13 +1316,21 @@ class LocalPouchDB {
c += readCount; c += readCount;
} while (readCount != 0); } while (readCount != 0);
// items collected. // items collected.
Logger("Finding unused pieces");
const garbages = hashPieces.filter((e) => usedPieces.indexOf(e) == -1); const garbages = hashPieces.filter((e) => usedPieces.indexOf(e) == -1);
let deleteCount = 0; let deleteCount = 0;
Logger("we have to delete:" + garbages.length);
let deleteDoc: EntryDoc[] = [];
for (let v of garbages) { for (let v of garbages) {
try { try {
let item = await this.localDatabase.get(v); let item = await this.localDatabase.get(v);
item._deleted = true; item._deleted = true;
await this.localDatabase.put(item); deleteDoc.push(item);
if (deleteDoc.length > 50) {
await this.localDatabase.bulkDocs(deleteDoc);
deleteDoc = [];
Logger("delete:" + deleteCount);
}
deleteCount++; deleteCount++;
} catch (ex) { } catch (ex) {
if (ex.status && ex.status == 404) { if (ex.status && ex.status == 404) {
@@ -1224,6 +1340,9 @@ class LocalPouchDB {
} }
} }
} }
if (deleteDoc.length > 0) {
await this.localDatabase.bulkDocs(deleteDoc);
}
Logger(`GC:deleted ${deleteCount} items.`); Logger(`GC:deleted ${deleteCount} items.`);
} }
} }
@@ -1245,6 +1364,7 @@ export default class ObsidianLiveSyncPlugin extends Plugin {
this.settings.liveSync = false; this.settings.liveSync = false;
this.settings.syncOnSave = false; this.settings.syncOnSave = false;
this.settings.syncOnStart = false; this.settings.syncOnStart = false;
this.settings.periodicReplication = false;
this.settings.versionUpFlash = "I changed specifications incompatiblly, so when you enable sync again, be sure to made version up all nother devides."; this.settings.versionUpFlash = "I changed specifications incompatiblly, so when you enable sync again, be sure to made version up all nother devides.";
this.saveSettings(); this.saveSettings();
} }
@@ -1283,35 +1403,30 @@ export default class ObsidianLiveSyncPlugin extends Plugin {
let delay = this.settings.savingDelay; let delay = this.settings.savingDelay;
if (delay < 200) delay = 200; if (delay < 200) delay = 200;
if (delay > 5000) delay = 5000; if (delay > 5000) delay = 5000;
this.watchVaultChange = debounce(this.watchVaultChange.bind(this), delay, false); // this.watchVaultChange = debounce(this.watchVaultChange.bind(this), delay, false);
this.watchVaultDelete = debounce(this.watchVaultDelete.bind(this), delay, false); // this.watchVaultDelete = debounce(this.watchVaultDelete.bind(this), delay, false);
this.watchVaultRename = debounce(this.watchVaultRename.bind(this), delay, false); // this.watchVaultRename = debounce(this.watchVaultRename.bind(this), delay, false);
this.watchWorkspaceOpen = debounce(this.watchWorkspaceOpen.bind(this), delay, false);
this.watchVaultChange = this.watchVaultChange.bind(this);
this.watchVaultCreate = this.watchVaultCreate.bind(this);
this.watchVaultDelete = this.watchVaultDelete.bind(this);
this.watchVaultRename = this.watchVaultRename.bind(this);
this.watchWorkspaceOpen = debounce(this.watchWorkspaceOpen.bind(this), delay, false);
this.watchWindowVisiblity = debounce(this.watchWindowVisiblity.bind(this), delay, false);
this.registerWatchEvents();
this.parseReplicationResult = this.parseReplicationResult.bind(this); this.parseReplicationResult = this.parseReplicationResult.bind(this);
this.periodicSync = this.periodicSync.bind(this);
this.setPeriodicSync = this.setPeriodicSync.bind(this);
// this.registerWatchEvents();
this.addSettingTab(new ObsidianLiveSyncSettingTab(this.app, this)); this.addSettingTab(new ObsidianLiveSyncSettingTab(this.app, this));
this.app.workspace.onLayoutReady(async () => { this.app.workspace.onLayoutReady(async () => {
await this.initializeDatabase(); await this.initializeDatabase();
this.realizeSettingSyncMode(); this.realizeSettingSyncMode();
if (this.settings.syncOnStart) { this.registerWatchEvents();
await this.replicate(false);
}
}); });
// when in mobile, too long suspended , connection won't back if setting retry:true
this.registerInterval(
window.setInterval(async () => {
if (this.settings.liveSync) {
await this.localDatabase.closeReplication();
if (this.settings.liveSync) {
this.localDatabase.openReplication(this.settings, true, false, this.parseReplicationResult);
}
}
}, 60 * 1000)
);
this.addCommand({ this.addCommand({
id: "livesync-replicate", id: "livesync-replicate",
name: "Replicate now", name: "Replicate now",
@@ -1319,6 +1434,14 @@ export default class ObsidianLiveSyncPlugin extends Plugin {
this.replicate(); this.replicate();
}, },
}); });
this.addCommand({
id: "livesync-dump",
name: "Dump informations of this doc ",
editorCallback: (editor: Editor, view: MarkdownView) => {
//this.replicate();
this.localDatabase.getDBEntry(view.file.path, {}, true);
},
});
// this.addCommand({ // this.addCommand({
// id: "livesync-test", // id: "livesync-test",
// name: "test reset db and replicate", // name: "test reset db and replicate",
@@ -1349,14 +1472,13 @@ export default class ObsidianLiveSyncPlugin extends Plugin {
this.saveSettings(); this.saveSettings();
}, },
}); });
this.watchWindowVisiblity = this.watchWindowVisiblity.bind(this);
window.addEventListener("visibilitychange", this.watchWindowVisiblity);
} }
onunload() { onunload() {
if (this.gcTimerHandler != null) { if (this.gcTimerHandler != null) {
clearTimeout(this.gcTimerHandler); clearTimeout(this.gcTimerHandler);
this.gcTimerHandler = null; this.gcTimerHandler = null;
} }
this.clearPeriodicSync();
this.localDatabase.closeReplication(); this.localDatabase.closeReplication();
this.localDatabase.close(); this.localDatabase.close();
window.removeEventListener("visibilitychange", this.watchWindowVisiblity); window.removeEventListener("visibilitychange", this.watchWindowVisiblity);
@@ -1368,6 +1490,7 @@ export default class ObsidianLiveSyncPlugin extends Plugin {
this.localDatabase.close(); this.localDatabase.close();
} }
let vaultName = this.app.vault.getName(); let vaultName = this.app.vault.getName();
Logger("Open Database...");
this.localDatabase = new LocalPouchDB(this.settings, vaultName); this.localDatabase = new LocalPouchDB(this.settings, vaultName);
this.localDatabase.updateInfo = () => { this.localDatabase.updateInfo = () => {
this.refreshStatusText(); this.refreshStatusText();
@@ -1385,6 +1508,7 @@ export default class ObsidianLiveSyncPlugin extends Plugin {
async saveSettings() { async saveSettings() {
await this.saveData(this.settings); await this.saveData(this.settings);
this.localDatabase.settings = this.settings; this.localDatabase.settings = this.settings;
this.realizeSettingSyncMode();
} }
gcTimerHandler: any = null; gcTimerHandler: any = null;
gcHook() { gcHook() {
@@ -1405,19 +1529,27 @@ export default class ObsidianLiveSyncPlugin extends Plugin {
this.registerEvent(this.app.vault.on("rename", this.watchVaultRename)); this.registerEvent(this.app.vault.on("rename", this.watchVaultRename));
this.registerEvent(this.app.vault.on("create", this.watchVaultChange)); this.registerEvent(this.app.vault.on("create", this.watchVaultChange));
this.registerEvent(this.app.workspace.on("file-open", this.watchWorkspaceOpen)); this.registerEvent(this.app.workspace.on("file-open", this.watchWorkspaceOpen));
window.addEventListener("visibilitychange", this.watchWindowVisiblity);
} }
watchWindowVisiblity() { watchWindowVisiblity() {
this.watchWindowVisiblityAsync();
}
async watchWindowVisiblityAsync() {
if (this.settings.suspendFileWatching) return; if (this.settings.suspendFileWatching) return;
let isHidden = document.hidden; let isHidden = document.hidden;
if (isHidden) { if (isHidden) {
this.localDatabase.closeReplication(); this.localDatabase.closeReplication();
this.clearPeriodicSync();
} else { } else {
if (this.settings.liveSync) { if (this.settings.liveSync) {
this.localDatabase.openReplication(this.settings, true, false, this.parseReplicationResult); await this.localDatabase.openReplication(this.settings, true, false, this.parseReplicationResult);
} }
if (this.settings.syncOnStart) { if (this.settings.syncOnStart) {
this.localDatabase.openReplication(this.settings, false, false, this.parseReplicationResult); await this.localDatabase.openReplication(this.settings, false, false, this.parseReplicationResult);
}
if (this.settings.periodicReplication) {
this.setPeriodicSync();
} }
} }
this.gcHook(); this.gcHook();
@@ -1425,39 +1557,96 @@ export default class ObsidianLiveSyncPlugin extends Plugin {
watchWorkspaceOpen(file: TFile) { watchWorkspaceOpen(file: TFile) {
if (this.settings.suspendFileWatching) return; if (this.settings.suspendFileWatching) return;
this.watchWorkspaceOpenAsync(file);
}
async watchWorkspaceOpenAsync(file: TFile) {
if (file == null) return; if (file == null) return;
if (this.settings.syncOnFileOpen) {
await this.replicate();
}
this.localDatabase.disposeHashCache(); this.localDatabase.disposeHashCache();
this.showIfConflicted(file); await this.showIfConflicted(file);
this.gcHook(); this.gcHook();
} }
watchVaultCreate(file: TFile, ...args: any[]) {
if (this.settings.suspendFileWatching) return;
this.watchVaultChangeAsync(file, ...args);
}
watchVaultChange(file: TFile, ...args: any[]) { watchVaultChange(file: TFile, ...args: any[]) {
if (this.settings.suspendFileWatching) return; if (this.settings.suspendFileWatching) return;
this.updateIntoDB(file); this.watchVaultChangeAsync(file, ...args);
this.gcHook();
} }
watchVaultDelete(file: TFile & TFolder) { batchFileChange: string[] = [];
async watchVaultChangeAsync(file: TFile, ...args: any[]) {
if (file instanceof TFile) {
await this.updateIntoDB(file);
this.gcHook();
}
}
watchVaultDelete(file: TFile | TFolder) {
console.log(`${file.path} delete`);
if (this.settings.suspendFileWatching) return; if (this.settings.suspendFileWatching) return;
if (file.children) { this.watchVaultDeleteAsync(file);
//folder }
this.deleteFolderOnDB(file); async watchVaultDeleteAsync(file: TFile | TFolder) {
// this.app.vault.delete(file); if (file instanceof TFile) {
} else { await this.deleteFromDB(file);
this.deleteFromDB(file); } else if (file instanceof TFolder) {
await this.deleteFolderOnDB(file);
} }
this.gcHook(); this.gcHook();
} }
watchVaultRename(file: TFile & TFolder, oldFile: any) { GetAllFilesRecursively(file: TAbstractFile): TFile[] {
if (this.settings.suspendFileWatching) return; if (file instanceof TFile) {
if (file.children) { return [file];
// this.renameFolder(file,oldFile); } else if (file instanceof TFolder) {
Logger(`folder name changed:(this operation is not supported) ${file.path}`, LOG_LEVEL.NOTICE); let result: TFile[] = [];
for (var v of file.children) {
result.push(...this.GetAllFilesRecursively(v));
}
return result;
} else { } else {
this.updateIntoDB(file); Logger(`Filetype error:${file.path}`, LOG_LEVEL.NOTICE);
this.deleteFromDBbyPath(oldFile); throw new Error(`Filetype error:${file.path}`);
}
}
watchVaultRename(file: TFile | TFolder, oldFile: any) {
if (this.settings.suspendFileWatching) return;
this.watchVaultRenameAsync(file, oldFile);
}
getFilePath(file: TAbstractFile): string {
if (file instanceof TFolder) {
if (file.isRoot()) return "";
return this.getFilePath(file.parent) + "/" + file.name;
}
if (file instanceof TFile) {
return this.getFilePath(file.parent) + "/" + file.name;
}
}
async watchVaultRenameAsync(file: TFile | TFolder, oldFile: any) {
Logger(`${oldFile} renamed to ${file.path}`, LOG_LEVEL.VERBOSE);
if (file instanceof TFolder) {
const newFiles = this.GetAllFilesRecursively(file);
// for guard edge cases. this won't happen and each file's event will be raise.
for (const i of newFiles) {
let newFilePath = normalizePath(this.getFilePath(i));
let newFile = this.app.vault.getAbstractFileByPath(newFilePath);
if (newFile instanceof TFile) {
Logger(`save ${newFile.path} into db`);
await this.updateIntoDB(newFile);
}
}
Logger(`delete below ${oldFile} from db`);
await this.deleteFromDBbyPath(oldFile);
} else if (file instanceof TFile) {
Logger(`file save ${file.path} into db`);
await this.updateIntoDB(file);
Logger(`deleted ${oldFile} into db`);
await this.deleteFromDBbyPath(oldFile);
} }
this.gcHook(); this.gcHook();
} }
addLogHook: () => void = null;
//--> Basic document Functions //--> Basic document Functions
async addLog(message: any, level: LOG_LEVEL = LOG_LEVEL.INFO) { async addLog(message: any, level: LOG_LEVEL = LOG_LEVEL.INFO) {
if (level < LOG_LEVEL.INFO && this.settings && this.settings.lessInformationInLog) { if (level < LOG_LEVEL.INFO && this.settings && this.settings.lessInformationInLog) {
@@ -1479,6 +1668,7 @@ export default class ObsidianLiveSyncPlugin extends Plugin {
if (level >= LOG_LEVEL.NOTICE) { if (level >= LOG_LEVEL.NOTICE) {
new Notice(messagecontent); new Notice(messagecontent);
} }
if (this.addLogHook != null) this.addLogHook();
} }
async ensureDirectory(fullpath: string) { async ensureDirectory(fullpath: string) {
@@ -1633,6 +1823,7 @@ export default class ObsidianLiveSyncPlugin extends Plugin {
} }
} }
periodicSyncHandler: NodeJS.Timer = null;
//---> Sync //---> Sync
async parseReplicationResult(docs: Array<PouchDB.Core.ExistingDocument<EntryDoc>>): Promise<void> { async parseReplicationResult(docs: Array<PouchDB.Core.ExistingDocument<EntryDoc>>): Promise<void> {
this.refreshStatusText(); this.refreshStatusText();
@@ -1653,12 +1844,29 @@ export default class ObsidianLiveSyncPlugin extends Plugin {
this.gcHook(); this.gcHook();
} }
} }
clearPeriodicSync() {
if (this.periodicSyncHandler != null) {
clearInterval(this.periodicSyncHandler);
this.periodicSyncHandler = null;
}
}
setPeriodicSync() {
if (this.settings.periodicReplication && this.settings.periodicReplicationInterval > 0) {
this.clearPeriodicSync();
this.periodicSyncHandler = setInterval(() => this.periodicSync, Math.max(this.settings.periodicReplicationInterval, 30) * 1000);
}
}
async periodicSync() {
await this.replicate();
}
realizeSettingSyncMode() { realizeSettingSyncMode() {
this.localDatabase.closeReplication(); this.localDatabase.closeReplication();
if (this.settings.liveSync) { if (this.settings.liveSync) {
this.localDatabase.openReplication(this.settings, true, false, this.parseReplicationResult); this.localDatabase.openReplication(this.settings, true, false, this.parseReplicationResult);
this.refreshStatusText(); this.refreshStatusText();
} }
this.clearPeriodicSync();
this.setPeriodicSync();
} }
refreshStatusText() { refreshStatusText() {
let sent = this.localDatabase.docSent; let sent = this.localDatabase.docSent;
@@ -1683,6 +1891,7 @@ export default class ObsidianLiveSyncPlugin extends Plugin {
default: default:
w = "?"; w = "?";
} }
this.statusBar.title = this.localDatabase.syncStatus;
this.statusBar.setText(`Sync:${w} ↑${sent} ↓${arrived}`); this.statusBar.setText(`Sync:${w} ↑${sent} ↓${arrived}`);
} }
async replicate(showMessage?: boolean) { async replicate(showMessage?: boolean) {
@@ -1715,7 +1924,7 @@ export default class ObsidianLiveSyncPlugin extends Plugin {
const filesStorage = this.app.vault.getFiles(); const filesStorage = this.app.vault.getFiles();
const filesStorageName = filesStorage.map((e) => e.path); const filesStorageName = filesStorage.map((e) => e.path);
const wf = await this.localDatabase.localDatabase.allDocs(); const wf = await this.localDatabase.localDatabase.allDocs();
const filesDatabase = wf.rows.filter((e) => !e.id.startsWith("h:")).map((e) => normalizePath(e.id)); const filesDatabase = wf.rows.filter((e) => !e.id.startsWith("h:") && e.id != "obsydian_livesync_version").map((e) => normalizePath(e.id));
const onlyInStorage = filesStorage.filter((e) => filesDatabase.indexOf(e.path) == -1); const onlyInStorage = filesStorage.filter((e) => filesDatabase.indexOf(e.path) == -1);
const onlyInDatabase = filesDatabase.filter((e) => filesStorageName.indexOf(e) == -1); const onlyInDatabase = filesDatabase.filter((e) => filesStorageName.indexOf(e) == -1);
@@ -1728,12 +1937,14 @@ export default class ObsidianLiveSyncPlugin extends Plugin {
this.statusBar.setText(`UPDATE DATABASE`); this.statusBar.setText(`UPDATE DATABASE`);
// just write to DB from storage. // just write to DB from storage.
for (let v of onlyInStorage) { for (let v of onlyInStorage) {
Logger(`Update into ${v.path}`);
await this.updateIntoDB(v); await this.updateIntoDB(v);
} }
// simply realize it // simply realize it
this.statusBar.setText(`UPDATE STORAGE`); this.statusBar.setText(`UPDATE STORAGE`);
Logger("Writing files that only in database"); Logger("Writing files that only in database");
for (let v of onlyInDatabase) { for (let v of onlyInDatabase) {
Logger(`Pull from db:${v}`);
await this.pullFile(v, filesStorage); await this.pullFile(v, filesStorage);
} }
// have to sync below.. // have to sync below..
@@ -1741,6 +1952,7 @@ export default class ObsidianLiveSyncPlugin extends Plugin {
for (let v of syncFiles) { for (let v of syncFiles) {
await this.syncFileBetweenDBandStorage(v, filesStorage); await this.syncFileBetweenDBandStorage(v, filesStorage);
} }
this.statusBar.setText(`NOW TRACKING!`);
Logger("Initialized"); Logger("Initialized");
} }
async deleteFolderOnDB(folder: TFolder) { async deleteFolderOnDB(folder: TFolder) {
@@ -1966,7 +2178,7 @@ export default class ObsidianLiveSyncPlugin extends Plugin {
let datatype: "plain" | "newnote" = "newnote"; let datatype: "plain" | "newnote" = "newnote";
if (file.extension != "md") { if (file.extension != "md") {
let contentBin = await this.app.vault.readBinary(file); let contentBin = await this.app.vault.readBinary(file);
content = arrayBufferToBase64(contentBin); content = await arrayBufferToBase64(contentBin);
datatype = "newnote"; datatype = "newnote";
} else { } else {
content = await this.app.vault.read(file); content = await this.app.vault.read(file);
@@ -2049,13 +2261,13 @@ class LogDisplayModal extends Modal {
div.addClass("op-pre"); div.addClass("op-pre");
this.logEl = div; this.logEl = div;
this.updateLog = this.updateLog.bind(this); this.updateLog = this.updateLog.bind(this);
// this.plugin.onLogChanged = this.updateLog; this.plugin.addLogHook = this.updateLog;
this.updateLog(); this.updateLog();
} }
onClose() { onClose() {
let { contentEl } = this; let { contentEl } = this;
contentEl.empty(); contentEl.empty();
// this.plugin.onLogChanged = null; this.plugin.addLogHook = null;
} }
} }
class ConflictResolveModal extends Modal { class ConflictResolveModal extends Modal {
@@ -2280,6 +2492,33 @@ class ObsidianLiveSyncSettingTab extends PluginSettingTab {
this.plugin.realizeSettingSyncMode(); this.plugin.realizeSettingSyncMode();
}) })
); );
new Setting(containerEl)
.setName("Periodic Sync")
.setDesc("Sync periodically")
.addToggle((toggle) =>
toggle.setValue(this.plugin.settings.periodicReplication).onChange(async (value) => {
this.plugin.settings.periodicReplication = value;
await this.plugin.saveSettings();
})
);
new Setting(containerEl)
.setName("Periodic sync intreval")
.setDesc("Interval (sec)")
.addText((text) => {
text.setPlaceholder("")
.setValue(this.plugin.settings.periodicReplicationInterval + "")
.onChange(async (value) => {
let v = Number(value);
if (isNaN(v) || v > 5000) {
return 0;
}
this.plugin.settings.periodicReplicationInterval = v;
await this.plugin.saveSettings();
});
text.inputEl.setAttribute("type", "number");
});
new Setting(containerEl) new Setting(containerEl)
.setName("Sync on Save") .setName("Sync on Save")
.setDesc("When you save file, sync automatically") .setDesc("When you save file, sync automatically")
@@ -2289,6 +2528,15 @@ class ObsidianLiveSyncSettingTab extends PluginSettingTab {
await this.plugin.saveSettings(); await this.plugin.saveSettings();
}) })
); );
new Setting(containerEl)
.setName("Sync on File Open")
.setDesc("When you open file, sync automatically")
.addToggle((toggle) =>
toggle.setValue(this.plugin.settings.syncOnFileOpen).onChange(async (value) => {
this.plugin.settings.syncOnFileOpen = value;
await this.plugin.saveSettings();
})
);
new Setting(containerEl) new Setting(containerEl)
.setName("Sync on Start") .setName("Sync on Start")
.setDesc("Start synchronization on Obsidian started.") .setDesc("Start synchronization on Obsidian started.")
@@ -2389,6 +2637,12 @@ class ObsidianLiveSyncSettingTab extends PluginSettingTab {
.setWarning() .setWarning()
.setDisabled(false) .setDisabled(false)
.onClick(async () => { .onClick(async () => {
this.plugin.settings.liveSync = false;
this.plugin.settings.periodicReplication = false;
this.plugin.settings.syncOnSave = false;
this.plugin.settings.syncOnStart = false;
await this.plugin.saveSettings();
await this.plugin.saveSettings();
await this.plugin.resetLocalDatabase(); await this.plugin.resetLocalDatabase();
await this.plugin.initializeDatabase(); await this.plugin.initializeDatabase();
await this.plugin.tryResetRemoteDatabase(); await this.plugin.tryResetRemoteDatabase();
@@ -2467,6 +2721,8 @@ class ObsidianLiveSyncSettingTab extends PluginSettingTab {
}); });
}); });
} }
} else {
let cx = containerEl.createEl("div", { text: "There's no collupted data." });
} }
} }
} }

2
manifest.json
View File

@@ -1,7 +1,7 @@
{ {
"id": "obsidian-livesync", "id": "obsidian-livesync",
"name": "Self-hosted LiveSync", "name": "Self-hosted LiveSync",
"version": "0.1.10", "version": "0.1.14",
"minAppVersion": "0.9.12", "minAppVersion": "0.9.12",
"description": "Community implementation of self-hosted livesync. Reflect your vault changes to some other devices immediately. Please make sure to disable other synchronize solutions to avoid content corruption or duplication.", "description": "Community implementation of self-hosted livesync. Reflect your vault changes to some other devices immediately. Please make sure to disable other synchronize solutions to avoid content corruption or duplication.",
"author": "vorotamoroz", "author": "vorotamoroz",

4
package-lock.json generated
View File

@@ -1,12 +1,12 @@
{ {
"name": "obsidian-livesync", "name": "obsidian-livesync",
"version": "0.1.10", "version": "0.1.14",
"lockfileVersion": 2, "lockfileVersion": 2,
"requires": true, "requires": true,
"packages": { "packages": {
"": { "": {
"name": "obsidian-livesync", "name": "obsidian-livesync",
"version": "0.1.10", "version": "0.1.14",
"license": "MIT", "license": "MIT",
"dependencies": { "dependencies": {
"diff-match-patch": "^1.0.5", "diff-match-patch": "^1.0.5",

2
package.json
View File

@@ -1,6 +1,6 @@
{ {
"name": "obsidian-livesync", "name": "obsidian-livesync",
"version": "0.1.10", "version": "0.1.14",
"description": "Reflect your vault changes to some other devices immediately. Please make sure to disable other synchronize solutions to avoid content corruption or duplication.", "description": "Reflect your vault changes to some other devices immediately. Please make sure to disable other synchronize solutions to avoid content corruption or duplication.",
"main": "main.js", "main": "main.js",
"scripts": { "scripts": {

1
styles.css
View File

@@ -14,6 +14,7 @@
overflow-y: scroll; overflow-y: scroll;
/* min-height: 280px; */ /* min-height: 280px; */
max-height: 280px; max-height: 280px;
user-select: text;
} }
.op-pre { .op-pre {
white-space: pre-wrap; white-space: pre-wrap;