mirror of
https://github.com/vrtmrz/obsidian-livesync.git
synced 2025-12-13 09:45:56 +00:00
Fixed and implemented:
- New configuration to solve synchronization failure on large vault. - Preset misconfigurations - Sometimes hanged on replication. - Wrote documents
This commit is contained in:
vorotamoroz
@@ -23,7 +23,7 @@ The Database name to synchronize.
|
||||
## Local Database Configurations
|
||||
"Local Database" is created inside your obsidian.
|
||||
|
||||
### Batch database update (beta)
|
||||
### Batch database update
|
||||
Delay database update until raise replication, open another file, window visibility changed, or file events except for file modification.
|
||||
This option can not be used with LiveSync at the same time.
|
||||
|
||||
@@ -78,6 +78,27 @@ When running these operations, every synchronization settings is disabled.
|
||||
|
||||
**And even your passphrase is wrong, It doesn't be checked before the plugin really decrypts. So If you set the wrong passphrase and run "Apply and Receive", you will get an amount of decryption error. But, this is the specification.**
|
||||
|
||||
### minimum chunk size and LongLine threshold
|
||||
The configuration of chunk splitting.
|
||||
|
||||
Self-hosted LiveSync splits the note into chunks for efficient synchronization. This chunk should be longer than "Minimum chunk size".
|
||||
|
||||
Specifically, the length of the chunk is determined by the following orders.
|
||||
|
||||
1. Find the nearest newline character, and if it is farther than LongLineThreshold, this piece becomes an independent chunk.
|
||||
|
||||
2. If not, find nearest to these items.
|
||||
1. Newline character
|
||||
2. Empty line (Windows style)
|
||||
3. Empty line (non-Windows style)
|
||||
3. Compare the farther in these 3 positions and next "\[newline\]#" position, pick a shorter piece to as chunk.
|
||||
|
||||
This rule was made empirically from my dataset. If this rule acts as badly on your data. Please give me the information.
|
||||
|
||||
You can dump saved note structure to `Dump informations of this doc`. Replace every character to x except newline and "#" when sending information to me.
|
||||
|
||||
Default values are 20 letters and 250 letters.
|
||||
|
||||
## General Settings
|
||||
|
||||
### Do not show low-priority log
|
||||
@@ -122,32 +143,58 @@ Self-hosted LiveSync will delete the folder when the folder becomes empty. If th
|
||||
### Use newer file if conflicted (beta)
|
||||
Always use the newer file to resolve and overwrite when conflict has occurred.
|
||||
|
||||
### minimum chunk size and LongLine threshold
|
||||
The configuration of chunk splitting.
|
||||
### Advanced settings
|
||||
Self-hosted LiveSync using PouchDB and synchronizes with the remote by [this protocol](https://docs.couchdb.org/en/stable/replication/protocol.html).
|
||||
So, it splits every entry into chunks to be acceptable by the database with limited payload size and document size.
|
||||
|
||||
Self-hosted LiveSync splits the note into chunks for efficient synchronization. This chunk should be longer than "Minimum chunk size".
|
||||
However, it was not enough.
|
||||
According to [2.4.2.5.2. Upload Batch of Changed Documents](https://docs.couchdb.org/en/stable/replication/protocol.html#upload-batch-of-changed-documents) in [Replicate Changes](https://docs.couchdb.org/en/stable/replication/protocol.html#replicate-changes), it might become a bigger request.
|
||||
|
||||
Specifically, the length of the chunk is determined by the following orders.
|
||||
Unfortunately, there is no way to deal with this automatically by size for every request.
|
||||
Therefore, I made it possible to configure this.
|
||||
|
||||
1. Find the nearest newline character, and if it is farther than LongLineThreshold, this piece becomes an independent chunk.
|
||||
Note: If you set these values lower number, the number of requests will increase.
|
||||
Therefore, if you are far from the server, the total throughput will be low, and the traffic will increase.
|
||||
|
||||
2. If not, find nearest to these items.
|
||||
1. Newline character
|
||||
2. Empty line (Windows style)
|
||||
3. Empty line (non-Windows style)
|
||||
3. Compare the farther in these 3 positions and next "\[newline\]#" position, pick a shorter piece to as chunk.
|
||||
### Batch size
|
||||
Number of change feed items to process at a time. Defaults to 250.
|
||||
|
||||
This rule was made empirically from my dataset. If this rule acts as badly on your data. Please give me the information.
|
||||
|
||||
You can dump saved note structure to `Dump informations of this doc`. Replace every character to x except newline and "#" when sending information to me.
|
||||
|
||||
Default values are 20 letters and 250 letters.
|
||||
### Batch limit
|
||||
Number of batches to process at a time. Defaults to 40. This along with batch size controls how many docs are kept in memory at a time.
|
||||
|
||||
## Miscellaneous
|
||||
|
||||
### Show status inside editor
|
||||
Show information inside the editor pane.
|
||||
It would be useful for mobile.
|
||||
|
||||
### Check integrity on saving
|
||||
Check all chunks are correctly saved on saving.
|
||||
|
||||
### Presets
|
||||
You can set synchronization method at once as these pattern:
|
||||
- LiveSync
|
||||
- LiveSync : enabled
|
||||
- Batch database update : disabled
|
||||
- Periodic Sync : disabled
|
||||
- Sync on Save : disabled
|
||||
- Sync on File Open : disabled
|
||||
- Sync on Start : disabled
|
||||
- Periodic w/ batch
|
||||
- LiveSync : disabled
|
||||
- Batch database update : enabled
|
||||
- Periodic Sync : enabled
|
||||
- Sync on Save : disabled
|
||||
- Sync on File Open : enabled
|
||||
- Sync on Start : enabled
|
||||
- Disable all sync
|
||||
- LiveSync : disabled
|
||||
- Batch database update : disabled
|
||||
- Periodic Sync : disabled
|
||||
- Sync on Save : disabled
|
||||
- Sync on File Open : disabled
|
||||
- Sync on Start : disabled
|
||||
|
||||
## Hatch
|
||||
From here, everything is under the hood. Please handle it with care.
|
||||
|
||||
|
||||
@@ -25,7 +25,7 @@ CouchDBのURIを入力します。Cloudantの場合は「External Endpoint(prefe
|
||||
## Local Database Configurations
|
||||
端末内に作成されるデータベースの設定です。
|
||||
|
||||
### Batch database update (beta)
|
||||
### Batch database update
|
||||
データベースの更新を以下の事象が発生するまで遅延させます。
|
||||
- レプリケーションが発生する
|
||||
- 他のファイルを開く
|
||||
@@ -78,6 +78,24 @@ End to End 暗号化を行うに当たって、異なるパスフレーズで暗
|
||||
どちらのオペレーションも、実行するとすべての同期設定が無効化されます。
|
||||
**また、パスフレーズのチェックは、実際に復号するまで行いません。そのため、パスフレーズを間違えて設定し、Apply and receiveで同期を行うと、大量のエラーが発生します。これは仕様です。**
|
||||
|
||||
### minimum chunk size と LongLine threshold
|
||||
チャンクの分割についての設定です。
|
||||
Self-hosted LiveSyncは一つのチャンクのサイズを最低minimum chunk size文字確保した上で、できるだけ効率的に同期できるよう、ノートを分割してチャンクを作成します。
|
||||
これは、同期を行う際に、一定の文字数で分割した場合、先頭の方を編集すると、その後の分割位置がすべてずれ、結果としてほぼまるごとのファイルのファイル送受信を行うことになっていた問題を避けるために実装されました。
|
||||
具体的には、先頭から順に直近の下記の箇所を検索し、一番長く切れたものを一つのチャンクとします。
|
||||
|
||||
1. 次の改行を探し、それがLongLine Thresholdより先であれば、一つのチャンクとして確定します。
|
||||
|
||||
2. そうではない場合は、下記を順に探します。
|
||||
1. 改行
|
||||
2. windowsでの空行がある所
|
||||
3. 非Windowsでの空行がある所
|
||||
3. この三つのうち一番遠い場所と、 「改行後、#から始まる所」を比べ、短い方をチャンクとします。
|
||||
|
||||
このルールは経験則的に作りました。実データが偏っているため。もし思わぬ挙動をしている場合は、是非コマンドから`Dump informations of this doc`を選択し、情報をください。
|
||||
改行文字と#を除き、すべて●に置換しても、アルゴリズムは有効に働きます。
|
||||
デフォルトは20文字と、250文字です。
|
||||
|
||||
## General Settings
|
||||
一般的な設定です。
|
||||
|
||||
@@ -124,24 +142,6 @@ Self-hosted LiveSyncは通常、フォルダ内のファイルがすべて削除
|
||||
### Use newer file if conflicted (beta)
|
||||
競合が発生したとき、常に新しいファイルを使用して競合を自動的に解決します。
|
||||
|
||||
### minimum chunk size と LongLine threshold
|
||||
チャンクの分割についての設定です。
|
||||
Self-hosted LiveSyncは一つのチャンクのサイズを最低minimum chunk size文字確保した上で、できるだけ効率的に同期できるよう、ノートを分割してチャンクを作成します。
|
||||
これは、同期を行う際に、一定の文字数で分割した場合、先頭の方を編集すると、その後の分割位置がすべてずれ、結果としてほぼまるごとのファイルのファイル送受信を行うことになっていた問題を避けるために実装されました。
|
||||
具体的には、先頭から順に直近の下記の箇所を検索し、一番長く切れたものを一つのチャンクとします。
|
||||
|
||||
1. 次の改行を探し、それがLongLine Thresholdより先であれば、一つのチャンクとして確定します。
|
||||
|
||||
2. そうではない場合は、下記を順に探します。
|
||||
1. 改行
|
||||
2. windowsでの空行がある所
|
||||
3. 非Windowsでの空行がある所
|
||||
3. この三つのうち一番遠い場所と、 「改行後、#から始まる所」を比べ、短い方をチャンクとします。
|
||||
|
||||
このルールは経験則的に作りました。実データが偏っているため。もし思わぬ挙動をしている場合は、是非コマンドから`Dump informations of this doc`を選択し、情報をください。
|
||||
改行文字と#を除き、すべて●に置換しても、アルゴリズムは有効に働きます。
|
||||
デフォルトは20文字と、250文字です。
|
||||
|
||||
## Miscellaneous
|
||||
その他の設定です
|
||||
### Show status inside editor
|
||||
|
||||
@@ -1,7 +1,7 @@
|
||||
{
|
||||
"id": "obsidian-livesync",
|
||||
"name": "Self-hosted LiveSync",
|
||||
"version": "0.2.1",
|
||||
"version": "0.3.0",
|
||||
"minAppVersion": "0.9.12",
|
||||
"description": "Community implementation of self-hosted livesync. Reflect your vault changes to some other devices immediately. Please make sure to disable other synchronize solutions to avoid content corruption or duplication.",
|
||||
"author": "vorotamoroz",
|
||||
|
||||
120
package-lock.json
generated
120
package-lock.json
generated
@@ -1,12 +1,12 @@
|
||||
{
|
||||
"name": "obsidian-livesync",
|
||||
"version": "0.2.1",
|
||||
"version": "0.3.0",
|
||||
"lockfileVersion": 2,
|
||||
"requires": true,
|
||||
"packages": {
|
||||
"": {
|
||||
"name": "obsidian-livesync",
|
||||
"version": "0.2.1",
|
||||
"version": "0.3.0",
|
||||
"license": "MIT",
|
||||
"dependencies": {
|
||||
"diff-match-patch": "^1.0.5",
|
||||
@@ -23,7 +23,7 @@
|
||||
"eslint": "^7.32.0",
|
||||
"eslint-config-airbnb-base": "^14.2.1",
|
||||
"eslint-plugin-import": "^2.25.2",
|
||||
"obsidian": "^0.12.0",
|
||||
"obsidian": "^0.13.11",
|
||||
"rollup": "^2.32.1",
|
||||
"tslib": "^2.2.0",
|
||||
"typescript": "^4.2.4"
|
||||
@@ -132,6 +132,43 @@
|
||||
"node": ">=4"
|
||||
}
|
||||
},
|
||||
"node_modules/@codemirror/rangeset": {
|
||||
"version": "0.19.5",
|
||||
"resolved": "https://registry.npmjs.org/@codemirror/rangeset/-/rangeset-0.19.5.tgz",
|
||||
"integrity": "sha512-L3b+RIwIRKOJ3pJLOtpkxCUjGnxZKFyPb0CjYWKnVLuzEIaEExWWK7sp6rsejxOy8RjYzfCHlFhYB4UdQN7brw==",
|
||||
"dev": true,
|
||||
"dependencies": {
|
||||
"@codemirror/state": "^0.19.0"
|
||||
}
|
||||
},
|
||||
"node_modules/@codemirror/state": {
|
||||
"version": "0.19.6",
|
||||
"resolved": "https://registry.npmjs.org/@codemirror/state/-/state-0.19.6.tgz",
|
||||
"integrity": "sha512-sqIQZE9VqwQj7D4c2oz9mfLhlT1ElAzGB5lO1lE33BPyrdNy1cJyCIOecT4cn4VeJOFrnjOeu+IftZ3zqdFETw==",
|
||||
"dev": true,
|
||||
"dependencies": {
|
||||
"@codemirror/text": "^0.19.0"
|
||||
}
|
||||
},
|
||||
"node_modules/@codemirror/text": {
|
||||
"version": "0.19.5",
|
||||
"resolved": "https://registry.npmjs.org/@codemirror/text/-/text-0.19.5.tgz",
|
||||
"integrity": "sha512-Syu5Xc7tZzeUAM/y4fETkT0zgGr48rDG+w4U38bPwSIUr+L9S/7w2wDE1WGNzjaZPz12F6gb1gxWiSTg9ocLow==",
|
||||
"dev": true
|
||||
},
|
||||
"node_modules/@codemirror/view": {
|
||||
"version": "0.19.37",
|
||||
"resolved": "https://registry.npmjs.org/@codemirror/view/-/view-0.19.37.tgz",
|
||||
"integrity": "sha512-SLuLx9p0O1ZHXLehvl5MwSvUrQRcsNGemzTgJ0zRajmc3BBsNigI1PXxdo7tvBhO5DcAzRRBXoke9DZFUR6Qqg==",
|
||||
"dev": true,
|
||||
"dependencies": {
|
||||
"@codemirror/rangeset": "^0.19.5",
|
||||
"@codemirror/state": "^0.19.3",
|
||||
"@codemirror/text": "^0.19.0",
|
||||
"style-mod": "^4.0.0",
|
||||
"w3c-keyname": "^2.2.4"
|
||||
}
|
||||
},
|
||||
"node_modules/@eslint/eslintrc": {
|
||||
"version": "0.4.3",
|
||||
"resolved": "https://registry.npmjs.org/@eslint/eslintrc/-/eslintrc-0.4.3.tgz",
|
||||
@@ -2185,11 +2222,13 @@
|
||||
}
|
||||
},
|
||||
"node_modules/obsidian": {
|
||||
"version": "0.12.17",
|
||||
"resolved": "https://registry.npmjs.org/obsidian/-/obsidian-0.12.17.tgz",
|
||||
"integrity": "sha512-YvCAlRym9D8zNPXt6Ez8QubSTVGoChx6lb58zqI13Dcrz3l1lgUO+pcOGDiD5Qa67nzDZLXo3aV2rqkCCpTvGQ==",
|
||||
"version": "0.13.11",
|
||||
"resolved": "https://registry.npmjs.org/obsidian/-/obsidian-0.13.11.tgz",
|
||||
"integrity": "sha512-KxOvAh4CG5vzcukmHvyuK9hUIr6ZFlM9FQfGZEwrrEV8VG2/W2Tk5cWrg0VM7EkGE3QBmjX6owjIDIO8QDXVUQ==",
|
||||
"dev": true,
|
||||
"dependencies": {
|
||||
"@codemirror/state": "^0.19.6",
|
||||
"@codemirror/view": "^0.19.31",
|
||||
"@types/codemirror": "0.0.108",
|
||||
"moment": "2.29.1"
|
||||
}
|
||||
@@ -2645,6 +2684,12 @@
|
||||
"url": "https://github.com/sponsors/sindresorhus"
|
||||
}
|
||||
},
|
||||
"node_modules/style-mod": {
|
||||
"version": "4.0.0",
|
||||
"resolved": "https://registry.npmjs.org/style-mod/-/style-mod-4.0.0.tgz",
|
||||
"integrity": "sha512-OPhtyEjyyN9x3nhPsu76f52yUGXiZcgvsrFVtvTkyGRQJ0XK+GPc6ov1z+lRpbeabka+MYEQxOYRnt5nF30aMw==",
|
||||
"dev": true
|
||||
},
|
||||
"node_modules/supports-color": {
|
||||
"version": "7.2.0",
|
||||
"resolved": "https://registry.npmjs.org/supports-color/-/supports-color-7.2.0.tgz",
|
||||
@@ -2819,6 +2864,12 @@
|
||||
"integrity": "sha512-l8lCEmLcLYZh4nbunNZvQCJc5pv7+RCwa8q/LdUx8u7lsWvPDKmpodJAJNwkAhJC//dFY48KuIEmjtd4RViDrA==",
|
||||
"dev": true
|
||||
},
|
||||
"node_modules/w3c-keyname": {
|
||||
"version": "2.2.4",
|
||||
"resolved": "https://registry.npmjs.org/w3c-keyname/-/w3c-keyname-2.2.4.tgz",
|
||||
"integrity": "sha512-tOhfEwEzFLJzf6d1ZPkYfGj+FWhIpBux9ppoP3rlclw3Z0BZv3N7b7030Z1kYth+6rDuAsXUFr+d0VE6Ed1ikw==",
|
||||
"dev": true
|
||||
},
|
||||
"node_modules/which": {
|
||||
"version": "2.0.2",
|
||||
"resolved": "https://registry.npmjs.org/which/-/which-2.0.2.tgz",
|
||||
@@ -2962,6 +3013,43 @@
|
||||
}
|
||||
}
|
||||
},
|
||||
"@codemirror/rangeset": {
|
||||
"version": "0.19.5",
|
||||
"resolved": "https://registry.npmjs.org/@codemirror/rangeset/-/rangeset-0.19.5.tgz",
|
||||
"integrity": "sha512-L3b+RIwIRKOJ3pJLOtpkxCUjGnxZKFyPb0CjYWKnVLuzEIaEExWWK7sp6rsejxOy8RjYzfCHlFhYB4UdQN7brw==",
|
||||
"dev": true,
|
||||
"requires": {
|
||||
"@codemirror/state": "^0.19.0"
|
||||
}
|
||||
},
|
||||
"@codemirror/state": {
|
||||
"version": "0.19.6",
|
||||
"resolved": "https://registry.npmjs.org/@codemirror/state/-/state-0.19.6.tgz",
|
||||
"integrity": "sha512-sqIQZE9VqwQj7D4c2oz9mfLhlT1ElAzGB5lO1lE33BPyrdNy1cJyCIOecT4cn4VeJOFrnjOeu+IftZ3zqdFETw==",
|
||||
"dev": true,
|
||||
"requires": {
|
||||
"@codemirror/text": "^0.19.0"
|
||||
}
|
||||
},
|
||||
"@codemirror/text": {
|
||||
"version": "0.19.5",
|
||||
"resolved": "https://registry.npmjs.org/@codemirror/text/-/text-0.19.5.tgz",
|
||||
"integrity": "sha512-Syu5Xc7tZzeUAM/y4fETkT0zgGr48rDG+w4U38bPwSIUr+L9S/7w2wDE1WGNzjaZPz12F6gb1gxWiSTg9ocLow==",
|
||||
"dev": true
|
||||
},
|
||||
"@codemirror/view": {
|
||||
"version": "0.19.37",
|
||||
"resolved": "https://registry.npmjs.org/@codemirror/view/-/view-0.19.37.tgz",
|
||||
"integrity": "sha512-SLuLx9p0O1ZHXLehvl5MwSvUrQRcsNGemzTgJ0zRajmc3BBsNigI1PXxdo7tvBhO5DcAzRRBXoke9DZFUR6Qqg==",
|
||||
"dev": true,
|
||||
"requires": {
|
||||
"@codemirror/rangeset": "^0.19.5",
|
||||
"@codemirror/state": "^0.19.3",
|
||||
"@codemirror/text": "^0.19.0",
|
||||
"style-mod": "^4.0.0",
|
||||
"w3c-keyname": "^2.2.4"
|
||||
}
|
||||
},
|
||||
"@eslint/eslintrc": {
|
||||
"version": "0.4.3",
|
||||
"resolved": "https://registry.npmjs.org/@eslint/eslintrc/-/eslintrc-0.4.3.tgz",
|
||||
@@ -4511,11 +4599,13 @@
|
||||
}
|
||||
},
|
||||
"obsidian": {
|
||||
"version": "0.12.17",
|
||||
"resolved": "https://registry.npmjs.org/obsidian/-/obsidian-0.12.17.tgz",
|
||||
"integrity": "sha512-YvCAlRym9D8zNPXt6Ez8QubSTVGoChx6lb58zqI13Dcrz3l1lgUO+pcOGDiD5Qa67nzDZLXo3aV2rqkCCpTvGQ==",
|
||||
"version": "0.13.11",
|
||||
"resolved": "https://registry.npmjs.org/obsidian/-/obsidian-0.13.11.tgz",
|
||||
"integrity": "sha512-KxOvAh4CG5vzcukmHvyuK9hUIr6ZFlM9FQfGZEwrrEV8VG2/W2Tk5cWrg0VM7EkGE3QBmjX6owjIDIO8QDXVUQ==",
|
||||
"dev": true,
|
||||
"requires": {
|
||||
"@codemirror/state": "^0.19.6",
|
||||
"@codemirror/view": "^0.19.31",
|
||||
"@types/codemirror": "0.0.108",
|
||||
"moment": "2.29.1"
|
||||
}
|
||||
@@ -4822,6 +4912,12 @@
|
||||
"integrity": "sha512-6fPc+R4ihwqP6N/aIv2f1gMH8lOVtWQHoqC4yK6oSDVVocumAsfCqjkXnqiYMhmMwS/mEHLp7Vehlt3ql6lEig==",
|
||||
"dev": true
|
||||
},
|
||||
"style-mod": {
|
||||
"version": "4.0.0",
|
||||
"resolved": "https://registry.npmjs.org/style-mod/-/style-mod-4.0.0.tgz",
|
||||
"integrity": "sha512-OPhtyEjyyN9x3nhPsu76f52yUGXiZcgvsrFVtvTkyGRQJ0XK+GPc6ov1z+lRpbeabka+MYEQxOYRnt5nF30aMw==",
|
||||
"dev": true
|
||||
},
|
||||
"supports-color": {
|
||||
"version": "7.2.0",
|
||||
"resolved": "https://registry.npmjs.org/supports-color/-/supports-color-7.2.0.tgz",
|
||||
@@ -4962,6 +5058,12 @@
|
||||
"integrity": "sha512-l8lCEmLcLYZh4nbunNZvQCJc5pv7+RCwa8q/LdUx8u7lsWvPDKmpodJAJNwkAhJC//dFY48KuIEmjtd4RViDrA==",
|
||||
"dev": true
|
||||
},
|
||||
"w3c-keyname": {
|
||||
"version": "2.2.4",
|
||||
"resolved": "https://registry.npmjs.org/w3c-keyname/-/w3c-keyname-2.2.4.tgz",
|
||||
"integrity": "sha512-tOhfEwEzFLJzf6d1ZPkYfGj+FWhIpBux9ppoP3rlclw3Z0BZv3N7b7030Z1kYth+6rDuAsXUFr+d0VE6Ed1ikw==",
|
||||
"dev": true
|
||||
},
|
||||
"which": {
|
||||
"version": "2.0.2",
|
||||
"resolved": "https://registry.npmjs.org/which/-/which-2.0.2.tgz",
|
||||
|
||||
@@ -1,6 +1,6 @@
|
||||
{
|
||||
"name": "obsidian-livesync",
|
||||
"version": "0.2.1",
|
||||
"version": "0.3.0",
|
||||
"description": "Reflect your vault changes to some other devices immediately. Please make sure to disable other synchronize solutions to avoid content corruption or duplication.",
|
||||
"main": "main.js",
|
||||
"scripts": {
|
||||
@@ -22,7 +22,7 @@
|
||||
"eslint": "^7.32.0",
|
||||
"eslint-config-airbnb-base": "^14.2.1",
|
||||
"eslint-plugin-import": "^2.25.2",
|
||||
"obsidian": "^0.12.0",
|
||||
"obsidian": "^0.13.11",
|
||||
"rollup": "^2.32.1",
|
||||
"tslib": "^2.2.0",
|
||||
"typescript": "^4.2.4"
|
||||
|
||||
@@ -756,8 +756,8 @@ export class LocalPouchDB {
|
||||
}
|
||||
|
||||
const syncOptionBase: PouchDB.Replication.SyncOptions = {
|
||||
batch_size: 250,
|
||||
batches_limit: 40,
|
||||
batches_limit: setting.batches_limit,
|
||||
batch_size: setting.batch_size,
|
||||
};
|
||||
|
||||
const db = dbret.db;
|
||||
@@ -851,8 +851,8 @@ export class LocalPouchDB {
|
||||
}
|
||||
|
||||
const syncOptionBase: PouchDB.Replication.SyncOptions = {
|
||||
batch_size: 250,
|
||||
batches_limit: 40,
|
||||
batches_limit: setting.batches_limit,
|
||||
batch_size: setting.batch_size,
|
||||
};
|
||||
const syncOption: PouchDB.Replication.SyncOptions = keepAlive ? { live: true, retry: true, heartbeat: 30000, ...syncOptionBase } : { ...syncOptionBase };
|
||||
|
||||
@@ -876,9 +876,11 @@ export class LocalPouchDB {
|
||||
//replicate once
|
||||
this.syncStatus = "STARTED";
|
||||
|
||||
return new Promise<boolean>(async (res, rej) => {
|
||||
let resolved = false;
|
||||
const docArrivedOnStart = this.docArrived;
|
||||
const docSentOnStart = this.docSent;
|
||||
const _openReplicationSync = () => {
|
||||
Logger("Sync Main Started");
|
||||
this.syncHandler = this.cancelHandler(this.syncHandler);
|
||||
this.syncHandler = this.localDatabase.sync<EntryDoc>(db, syncOption);
|
||||
this.syncHandler
|
||||
@@ -899,7 +901,7 @@ export class LocalPouchDB {
|
||||
this.docSent += e.change.docs.length;
|
||||
}
|
||||
if (notice != null) {
|
||||
notice.setMessage(`↑${e.change.docs_written} ↓${e.change.docs_read}`);
|
||||
notice.setMessage(`↑${this.docSent - docSentOnStart} ↓${this.docArrived - docArrivedOnStart}`);
|
||||
}
|
||||
this.updateInfo();
|
||||
} catch (ex) {
|
||||
@@ -915,7 +917,6 @@ export class LocalPouchDB {
|
||||
if (!keepAlive) {
|
||||
this.syncHandler = this.cancelHandler(this.syncHandler);
|
||||
// if keep alive runnning, resolve here,
|
||||
res(true);
|
||||
}
|
||||
})
|
||||
.on("denied", (e) => {
|
||||
@@ -924,8 +925,7 @@ export class LocalPouchDB {
|
||||
this.syncHandler = this.cancelHandler(this.syncHandler);
|
||||
if (notice != null) notice.hide();
|
||||
Logger("Replication denied", LOG_LEVEL.NOTICE);
|
||||
// Logger(e);
|
||||
rej(e);
|
||||
Logger(e);
|
||||
})
|
||||
.on("error", (e) => {
|
||||
this.syncStatus = "ERRORED";
|
||||
@@ -933,8 +933,7 @@ export class LocalPouchDB {
|
||||
this.updateInfo();
|
||||
if (notice != null) notice.hide();
|
||||
Logger("Replication error", LOG_LEVEL.NOTICE);
|
||||
// Logger(e);
|
||||
rej(e);
|
||||
Logger(e);
|
||||
})
|
||||
.on("paused", (e) => {
|
||||
this.syncStatus = "PAUSED";
|
||||
@@ -944,19 +943,22 @@ export class LocalPouchDB {
|
||||
if (keepAlive && !resolved) {
|
||||
// if keep alive runnning, resolve here,
|
||||
resolved = true;
|
||||
res(true);
|
||||
}
|
||||
// Logger(e);
|
||||
});
|
||||
return this.syncHandler;
|
||||
};
|
||||
if (!keepAlive) {
|
||||
return await _openReplicationSync();
|
||||
await _openReplicationSync();
|
||||
return true;
|
||||
}
|
||||
this.syncHandler = this.cancelHandler(this.syncHandler);
|
||||
Logger("Pull before replicate.");
|
||||
Logger(await this.localDatabase.info(), LOG_LEVEL.VERBOSE);
|
||||
Logger(await db.info(), LOG_LEVEL.VERBOSE);
|
||||
const replicate = this.localDatabase.replicate.from(db, syncOptionBase);
|
||||
let replicate: PouchDB.Replication.Replication<EntryDoc>;
|
||||
try {
|
||||
replicate = this.localDatabase.replicate.from(db, syncOptionBase);
|
||||
replicate
|
||||
.on("active", () => {
|
||||
this.syncStatus = "CONNECTED";
|
||||
@@ -979,36 +981,24 @@ export class LocalPouchDB {
|
||||
Logger("Replication callback error");
|
||||
Logger(ex);
|
||||
}
|
||||
})
|
||||
.on("complete", async (info) => {
|
||||
});
|
||||
this.syncStatus = "COMPLETED";
|
||||
this.updateInfo();
|
||||
this.cancelHandler(replicate);
|
||||
this.syncHandler = this.cancelHandler(this.syncHandler);
|
||||
Logger("Replication pull completed.");
|
||||
await _openReplicationSync();
|
||||
})
|
||||
.on("denied", (e) => {
|
||||
_openReplicationSync();
|
||||
return true;
|
||||
} catch (ex) {
|
||||
this.syncStatus = "ERRORED";
|
||||
this.updateInfo();
|
||||
if (notice != null) notice.hide();
|
||||
Logger("Pulling Replication denied", LOG_LEVEL.NOTICE);
|
||||
this.cancelHandler(replicate);
|
||||
this.syncHandler = this.cancelHandler(this.syncHandler);
|
||||
rej(e);
|
||||
})
|
||||
.on("error", (e) => {
|
||||
this.syncStatus = "ERRORED";
|
||||
this.updateInfo();
|
||||
Logger("Pulling Replication error", LOG_LEVEL.INFO);
|
||||
Logger("Pulling Replication error", LOG_LEVEL.NOTICE);
|
||||
this.cancelHandler(replicate);
|
||||
this.syncHandler = this.cancelHandler(this.syncHandler);
|
||||
if (notice != null) notice.hide();
|
||||
// debugger;
|
||||
Logger(e);
|
||||
rej(e);
|
||||
});
|
||||
});
|
||||
throw ex;
|
||||
}
|
||||
}
|
||||
|
||||
closeReplication() {
|
||||
|
||||
@@ -1,4 +1,4 @@
|
||||
import { App, Notice, PluginSettingTab, Setting } from "obsidian";
|
||||
import { App, Notice, PluginSettingTab, Setting, sanitizeHTMLToDom } from "obsidian";
|
||||
import { EntryDoc, LOG_LEVEL } from "./types";
|
||||
import { escapeStringToHTML, versionNumberString2Number, path2id, id2path, runWithLock } from "./utils";
|
||||
import { Logger } from "./logger";
|
||||
@@ -82,6 +82,8 @@ export class ObsidianLiveSyncSettingTab extends PluginSettingTab {
|
||||
if (this.plugin.settings.syncOnFileOpen) return true;
|
||||
if (this.plugin.settings.syncOnSave) return true;
|
||||
if (this.plugin.settings.syncOnStart) return true;
|
||||
if (this.plugin.localDatabase.syncStatus == "CONNECTED") return true;
|
||||
if (this.plugin.localDatabase.syncStatus == "PAUSED") return true;
|
||||
return false;
|
||||
};
|
||||
const applyDisplayEnabled = () => {
|
||||
@@ -304,6 +306,44 @@ export class ObsidianLiveSyncSettingTab extends PluginSettingTab {
|
||||
})
|
||||
);
|
||||
|
||||
containerLocalDatabaseEl.createEl("div", {
|
||||
text: sanitizeHTMLToDom(`Advanced settings<br>
|
||||
Configuration of how LiveSync makes chunks from the file.`),
|
||||
});
|
||||
new Setting(containerLocalDatabaseEl)
|
||||
.setName("Minimum chunk size")
|
||||
.setDesc("(letters), minimum chunk size.")
|
||||
.addText((text) => {
|
||||
text.setPlaceholder("")
|
||||
.setValue(this.plugin.settings.minimumChunkSize + "")
|
||||
.onChange(async (value) => {
|
||||
let v = Number(value);
|
||||
if (isNaN(v) || v < 10 || v > 1000) {
|
||||
v = 10;
|
||||
}
|
||||
this.plugin.settings.minimumChunkSize = v;
|
||||
await this.plugin.saveSettings();
|
||||
});
|
||||
text.inputEl.setAttribute("type", "number");
|
||||
});
|
||||
|
||||
new Setting(containerLocalDatabaseEl)
|
||||
.setName("LongLine Threshold")
|
||||
.setDesc("(letters), If the line is longer than this, make the line to chunk")
|
||||
.addText((text) => {
|
||||
text.setPlaceholder("")
|
||||
.setValue(this.plugin.settings.longLineThreshold + "")
|
||||
.onChange(async (value) => {
|
||||
let v = Number(value);
|
||||
if (isNaN(v) || v < 10 || v > 1000) {
|
||||
v = 10;
|
||||
}
|
||||
this.plugin.settings.longLineThreshold = v;
|
||||
await this.plugin.saveSettings();
|
||||
});
|
||||
text.inputEl.setAttribute("type", "number");
|
||||
});
|
||||
|
||||
addScreenElement("10", containerLocalDatabaseEl);
|
||||
const containerGeneralSettingsEl = containerEl.createDiv();
|
||||
containerGeneralSettingsEl.createEl("h3", { text: "General Settings" });
|
||||
@@ -458,35 +498,40 @@ export class ObsidianLiveSyncSettingTab extends PluginSettingTab {
|
||||
await this.plugin.saveSettings();
|
||||
})
|
||||
);
|
||||
|
||||
containerSyncSettingEl.createEl("div", {
|
||||
text: sanitizeHTMLToDom(`Advanced settings<br>
|
||||
If you reached the payload size limit when using IBM Cloudant, please set batch size and batch limit to a lower value.`),
|
||||
});
|
||||
new Setting(containerSyncSettingEl)
|
||||
.setName("Minimum chunk size")
|
||||
.setDesc("(letters), minimum chunk size.")
|
||||
.setName("Batch size")
|
||||
.setDesc("Number of change feed items to process at a time. Defaults to 250.")
|
||||
.addText((text) => {
|
||||
text.setPlaceholder("")
|
||||
.setValue(this.plugin.settings.minimumChunkSize + "")
|
||||
.setValue(this.plugin.settings.batch_size + "")
|
||||
.onChange(async (value) => {
|
||||
let v = Number(value);
|
||||
if (isNaN(v) || v < 10 || v > 1000) {
|
||||
if (isNaN(v) || v < 10) {
|
||||
v = 10;
|
||||
}
|
||||
this.plugin.settings.minimumChunkSize = v;
|
||||
this.plugin.settings.batch_size = v;
|
||||
await this.plugin.saveSettings();
|
||||
});
|
||||
text.inputEl.setAttribute("type", "number");
|
||||
});
|
||||
|
||||
new Setting(containerSyncSettingEl)
|
||||
.setName("LongLine Threshold")
|
||||
.setDesc("(letters), If the line is longer than this, make the line to chunk")
|
||||
.setName("Batch limit")
|
||||
.setDesc("Number of batches to process at a time. Defaults to 40. This along with batch size controls how many docs are kept in memory at a time.")
|
||||
.addText((text) => {
|
||||
text.setPlaceholder("")
|
||||
.setValue(this.plugin.settings.longLineThreshold + "")
|
||||
.setValue(this.plugin.settings.batches_limit + "")
|
||||
.onChange(async (value) => {
|
||||
let v = Number(value);
|
||||
if (isNaN(v) || v < 10 || v > 1000) {
|
||||
if (isNaN(v) || v < 10) {
|
||||
v = 10;
|
||||
}
|
||||
this.plugin.settings.longLineThreshold = v;
|
||||
this.plugin.settings.batches_limit = v;
|
||||
await this.plugin.saveSettings();
|
||||
});
|
||||
text.inputEl.setAttribute("type", "number");
|
||||
@@ -544,10 +589,10 @@ export class ObsidianLiveSyncSettingTab extends PluginSettingTab {
|
||||
Logger("Synchronization setting configured as LiveSync.", LOG_LEVEL.NOTICE);
|
||||
} else if (currentPrest == "PERIODIC") {
|
||||
this.plugin.settings.batchSave = true;
|
||||
this.plugin.settings.periodicReplication = false;
|
||||
this.plugin.settings.periodicReplication = true;
|
||||
this.plugin.settings.syncOnSave = false;
|
||||
this.plugin.settings.syncOnStart = false;
|
||||
this.plugin.settings.syncOnFileOpen = false;
|
||||
this.plugin.settings.syncOnStart = true;
|
||||
this.plugin.settings.syncOnFileOpen = true;
|
||||
Logger("Synchronization setting configured as Periodic sync with batch database update.", LOG_LEVEL.NOTICE);
|
||||
} else {
|
||||
Logger("All synchronization disabled.", LOG_LEVEL.NOTICE);
|
||||
|
||||
@@ -56,6 +56,8 @@ export interface ObsidianLiveSyncSettings {
|
||||
autoSweepPluginsPeriodic: boolean;
|
||||
notifyPluginOrSettingUpdated: boolean;
|
||||
checkIntegrityOnSave: boolean;
|
||||
batch_size: number;
|
||||
batches_limit: number;
|
||||
}
|
||||
|
||||
export const DEFAULT_SETTINGS: ObsidianLiveSyncSettings = {
|
||||
@@ -94,6 +96,8 @@ export const DEFAULT_SETTINGS: ObsidianLiveSyncSettings = {
|
||||
autoSweepPluginsPeriodic: false,
|
||||
notifyPluginOrSettingUpdated: false,
|
||||
checkIntegrityOnSave: false,
|
||||
batch_size: 250,
|
||||
batches_limit: 40,
|
||||
};
|
||||
|
||||
export const PERIODIC_PLUGIN_SWEEP = 60;
|
||||
|
||||
Reference in New Issue
Block a user