bumped

Print exception on failure in certain cases

.github/FUNDING.yml

@@ -0,0 +1,3 @@
+# These are supported funding model platforms
+
+github: vrtmrz

README.md

@@ -75,12 +75,15 @@ Synchronization status is shown in statusbar.
     -   ⚠ Error occurred.
 -   ↑ Uploaded pieces
 -   ↓ Downloaded pieces
+-   ⏳ Count of the pending process  
+If you have deleted or renamed files, please wait until this disappears.
 
 # More supplements
 
 -   When synchronized, files are compared by their modified times and overwritten by the newer ones once. Then plugin checks the conflicts and if a merge is needed, the dialog will open.
 -   Rarely, the file in the database would be broken. The plugin will not write storage when it looks broken, so some old files must be on your device. If you edit the file, it will be cured. But if the file does not exist on any device, can not rescue it. So you can delete these items from the setting dialog.
 -   If your database looks corrupted, try "Drop History". Usually, It is the easiest way.
+-   To stop the bootup sequence for fixing problems on databases, you can put `redflag.md` on top of your vault.
 -   Q: Database is growing, how can I shrink it up?
     A: each of the docs is saved with their old 100 revisions to detect and resolve confliction. Picture yourself that one device has been off the line for a while, and joined again. The device has to check his note and remote saved note. If exists in revision histories of remote notes even though the device's note is a little different from the latest one, it could be merged safely. Even if that is not in revision histories, we only have to check differences after the revision that both devices commonly have. This is like The git's conflict resolving method. So, We have to make the database again like an enlarged git repo if you want to solve the root of the problem.
 -   And more technical Information are in the [Technical Information](docs/tech_info.md)

README_ja.md

@@ -78,11 +78,14 @@ Self-hosted LiveSync用にWebClipperも作りました。Chrome Web Storeから
   - ⚠ エラーが発生しています
 - ↑ 送信したデータ数
 - ↓ 受信したデータ数
+-   ⏳ 保留している処理の数です  
+ファイルを削除したりリネームした場合、この表示が消えるまでお待ちください。
 
 # さらなる補足
 - ファイルは同期された後、タイムスタンプを比較して新しければいったん新しい方で上書きされます。その後、衝突が発生したかによって、マージが行われます。
 - まれにファイルが破損することがあります。破損したファイルに関してはディスクへの反映を試みないため、実際には使用しているデバイスには少し古いファイルが残っていることが多いです。そのファイルを再度更新してもらうと、データベースが更新されて問題なくなるケースがあります。ファイルがどの端末にも存在しない場合は、設定画面から、削除できます。
 - データベースが変。そういうときは、いったんデータベースをDrop Historyのapply and sendで再初期化してみてください。だいたい直ります。
+- データベースの復旧中に再起動した場合など、うまくローカルデータベースを修正できない際には、Vaultのトップに`redflag.md`というファイルを置いてください。起動時のシーケンスがスキップされます。
 - データベースが大きくなってきてるんだけど、小さくできる？→各ノートは、それぞれの古い100リビジョンとともに保存されています。例えば、しばらくオフラインだったあるデバイスが、久しぶりに同期したと想定してみてください。そのとき、そのデバイスは最新とは少し異なるリビジョンを持ってるはずです。その場合でも、リモートのリビジョン履歴にリモートのものが存在した場合、安全にマージできます。もしリビジョン履歴に存在しなかった場合、確認しなければいけない差分も、対象を存在して持っている共通のリビジョン以降のみに絞れます。ちょうどGitのような方法で、衝突を解決している形になるのです。そのため、肥大化したリポジトリの解消と同様に、本質的にデータベースを小さくしたい場合は、データベースの作り直しが必要です。
 - その他の技術的なお話は、[技術的な内容](docs/tech_info_ja.md)に書いてあります。
 

docs/settings.md

@@ -212,8 +212,11 @@ The remote database indicates that has been unlocked Pattern 1.
 When you mark all devices as resolved, you can unlock the database.
 But, there's no problem even if you leave it as it is.
 
-### Reread all files
-Reread all files in the vault, and update them into the database if there's diff or could not read from the database.
+### Verify and repair all files
+read all files in the vault, and update them into the database if there's diff or could not read from the database.
+
+### Sanity check
+Make sure that all the files on the local database have all chunks.
 
 ### Drop history
 Drop all histories on the local database and the remote database, and initialize When synchronization time has been prolonged to the new device or new vault, or database size became to be much larger. Try this.

docs/settings_ja.md

@@ -142,12 +142,34 @@ Self-hosted LiveSyncは通常、フォルダ内のファイルがすべて削除
 ### Use newer file if conflicted (beta)
 競合が発生したとき、常に新しいファイルを使用して競合を自動的に解決します。
 
+### Advanced settings
+Self-hosted LiveSyncはPouchDBを使用し、リモートと[このプロトコル](https://docs.couchdb.org/en/stable/replication/protocol.html)で同期しています。  
+そのため、全てのノートなどはデータベースが許容するペイロードサイズやドキュメントサイズに併せてチャンクに分割されています。
+
+しかしながら、それだけでは不十分なケースがあり、[Replicate Changes](https://docs.couchdb.org/en/stable/replication/protocol.html#replicate-changes)の[2.4.2.5.2. Upload Batch of Changed Documents](https://docs.couchdb.org/en/stable/replication/protocol.html#upload-batch-of-changed-documents)を参照すると、このリクエストは巨大になる可能性がありました。
+
+残念ながら、このサイズを呼び出しごとに自動的に調整する方法はありません。  
+そのため、設定を変更できるように機能追加いたしました。
+
+備考：もし小さな値を設定した場合、リクエスト数は増えます。  
+もしサーバから遠い場合、トータルのスループットは遅くなり、転送量は増えます。
+
+### Batch size
+一度に処理するChange feedの数です。デフォルトは250です。
+
+### Batch limit
+一度に処理するBatchの数です。デフォルトは40です。
+
 ## Miscellaneous
 その他の設定です
 ### Show status inside editor
-同期の情報をエディター内に表示します。
+同期の情報をエディター内に表示します。  
 モバイルで便利です。
 
+### Check integrity on saving
+保存時にデータが全て保存できたかチェックを行います。
+
+
 ## Hatch
 ここから先は、困ったときに開ける蓋の中身です。注意して使用してください。
 
@@ -166,9 +188,12 @@ Self-hosted LiveSyncは通常、フォルダ内のファイルがすべて削除
 ご使用のすべてのデバイスでロックを解除した場合は、データベースのロックを解除することができます。  
 ただし、このまま放置しても問題はありません。
 
-### Reread all files
+### Verify and repair all files
 Vault内のファイルを全て読み込み直し、もし差分があったり、データベースから正常に読み込めなかったものに関して、データベースに反映します。
 
+### Sanity check
+ローカルデータベースに保存されている全てのファイルが正しくチャンクを持っていることを確認します。
+
 ### Drop history
 データベースに記録されている履歴を削除し、データベースを初期化します。  
 新しい端末や新しいVaultへの同期にやたらと時間がかかったり、データベースサイズが肥大化したりしてきた際に使用してください。  

esbuild.config.mjs

@@ -0,0 +1,37 @@
+import esbuild from "esbuild";
+import process from "process";
+import builtins from "builtin-modules";
+import sveltePlugin from "esbuild-svelte";
+import sveltePreprocess from "svelte-preprocess";
+
+const banner = `/*
+THIS IS A GENERATED/BUNDLED FILE BY ESBUILD
+if you want to view the source, please visit the github repository of this plugin
+*/
+`;
+
+const prod = process.argv[2] === "production";
+
+esbuild
+    .build({
+        banner: {
+            js: banner,
+        },
+        entryPoints: ["src/main.ts"],
+        bundle: true,
+        external: ["obsidian", "electron", ...builtins],
+        format: "cjs",
+        watch: !prod,
+        target: "es2015",
+        logLevel: "info",
+        sourcemap: prod ? false : "inline",
+        treeShaking: true,
+        plugins: [
+            sveltePlugin({
+                preprocess: sveltePreprocess(),
+                compilerOptions: { css: true },
+            }),
+        ],
+        outfile: "main.js",
+    })
+    .catch(() => process.exit(1));

manifest.json

@@ -1,7 +1,7 @@
 {
     "id": "obsidian-livesync",
     "name": "Self-hosted LiveSync",
-    "version": "0.3.0",
+    "version": "0.7.2",
     "minAppVersion": "0.9.12",
     "description": "Community implementation of self-hosted livesync. Reflect your vault changes to some other devices immediately. Please make sure to disable other synchronize solutions to avoid content corruption or duplication.",
     "author": "vorotamoroz",

package.json

@@ -1,11 +1,12 @@
 {
     "name": "obsidian-livesync",
-    "version": "0.3.0",
+    "version": "0.7.2",
     "description": "Reflect your vault changes to some other devices immediately. Please make sure to disable other synchronize solutions to avoid content corruption or duplication.",
     "main": "main.js",
+    "type": "module",
     "scripts": {
-        "dev": "rollup --config rollup.config.js -w",
-        "build": "rollup --config rollup.config.js --environment BUILD:production",
+        "dev": "node esbuild.config.mjs",
+        "build": "node esbuild.config.mjs production",
         "lint": "eslint src"
     },
     "keywords": [],
@@ -16,6 +17,7 @@
         "@rollup/plugin-node-resolve": "^11.2.1",
         "@rollup/plugin-typescript": "^8.2.1",
         "@types/diff-match-patch": "^1.0.32",
+        "@types/pouchdb": "^6.4.0",
         "@types/pouchdb-browser": "^6.1.3",
         "@typescript-eslint/eslint-plugin": "^5.7.0",
         "@typescript-eslint/parser": "^5.0.0",
@@ -25,7 +27,11 @@
         "obsidian": "^0.13.11",
         "rollup": "^2.32.1",
         "tslib": "^2.2.0",
-        "typescript": "^4.2.4"
+        "typescript": "^4.2.4",
+        "builtin-modules": "^3.2.0",
+        "esbuild": "0.13.12",
+        "esbuild-svelte": "^0.6.0",
+        "svelte-preprocess": "^4.10.2"
     },
     "dependencies": {
         "diff-match-patch": "^1.0.5",

src/ConflictResolveModal.ts

@@ -7,6 +7,7 @@ export class ConflictResolveModal extends Modal {
     // result: Array<[number, string]>;
     result: diff_result;
     callback: (remove_rev: string) => Promise<void>;
+
     constructor(app: App, diff: diff_result, callback: (remove_rev: string) => Promise<void>) {
         super(app);
         this.result = diff;
@@ -45,18 +46,21 @@ export class ConflictResolveModal extends Modal {
         contentEl.createEl("button", { text: "Keep A" }, (e) => {
             e.addEventListener("click", async () => {
                 await this.callback(this.result.right.rev);
+                this.callback = null;
                 this.close();
             });
         });
         contentEl.createEl("button", { text: "Keep B" }, (e) => {
             e.addEventListener("click", async () => {
                 await this.callback(this.result.left.rev);
+                this.callback = null;
                 this.close();
             });
         });
         contentEl.createEl("button", { text: "Concat both" }, (e) => {
             e.addEventListener("click", async () => {
-                await this.callback(null);
+                await this.callback("");
+                this.callback = null;
                 this.close();
             });
         });
@@ -70,5 +74,8 @@ export class ConflictResolveModal extends Modal {
     onClose() {
         const { contentEl } = this;
         contentEl.empty();
+        if (this.callback != null) {
+            this.callback(null);
+        }
     }
 }

src/DocumentHistoryModal.ts

@@ -0,0 +1,132 @@
+import { TFile, Modal, App } from "obsidian";
+import { path2id, escapeStringToHTML } from "./utils";
+import ObsidianLiveSyncPlugin from "./main";
+import { DIFF_DELETE, DIFF_EQUAL, DIFF_INSERT, diff_match_patch } from "diff-match-patch";
+
+export class DocumentHistoryModal extends Modal {
+    plugin: ObsidianLiveSyncPlugin;
+    range: HTMLInputElement;
+    contentView: HTMLDivElement;
+    info: HTMLDivElement;
+    fileInfo: HTMLDivElement;
+    showDiff = false;
+
+    file: string;
+
+    revs_info: PouchDB.Core.RevisionInfo[] = [];
+
+    constructor(app: App, plugin: ObsidianLiveSyncPlugin, file: TFile) {
+        super(app);
+        this.plugin = plugin;
+        this.file = file.path;
+        if (localStorage.getItem("ols-history-highlightdiff") == "1") {
+            this.showDiff = true;
+        }
+    }
+    async loadFile() {
+        const db = this.plugin.localDatabase;
+        const w = await db.localDatabase.get(path2id(this.file), { revs_info: true });
+        this.revs_info = w._revs_info.filter((e) => e.status == "available");
+        this.range.max = `${this.revs_info.length - 1}`;
+        this.range.value = this.range.max;
+        this.fileInfo.setText(`${this.file} / ${this.revs_info.length} revisions`);
+        await this.loadRevs();
+    }
+    async loadRevs() {
+        const db = this.plugin.localDatabase;
+        const index = this.revs_info.length - 1 - (this.range.value as any) / 1;
+        const rev = this.revs_info[index];
+        const w = await db.getDBEntry(path2id(this.file), { rev: rev.rev }, false, false);
+
+        if (w === false) {
+            this.info.innerHTML = "";
+            this.contentView.innerHTML = `Could not read this revision<br>(${rev.rev})`;
+        } else {
+            this.info.innerHTML = `Modified:${new Date(w.mtime).toLocaleString()}`;
+            let result = "";
+            if (this.showDiff) {
+                const prevRevIdx = this.revs_info.length - 1 - ((this.range.value as any) / 1 - 1);
+                if (prevRevIdx >= 0 && prevRevIdx < this.revs_info.length) {
+                    const oldRev = this.revs_info[prevRevIdx].rev;
+                    const w2 = await db.getDBEntry(path2id(this.file), { rev: oldRev }, false, false);
+                    if (w2 != false) {
+                        const dmp = new diff_match_patch();
+                        const diff = dmp.diff_main(w2.data, w.data);
+                        dmp.diff_cleanupSemantic(diff);
+                        for (const v of diff) {
+                            const x1 = v[0];
+                            const x2 = v[1];
+                            if (x1 == DIFF_DELETE) {
+                                result += "<span class='history-deleted'>" + escapeStringToHTML(x2) + "</span>";
+                            } else if (x1 == DIFF_EQUAL) {
+                                result += "<span class='history-normal'>" + escapeStringToHTML(x2) + "</span>";
+                            } else if (x1 == DIFF_INSERT) {
+                                result += "<span class='history-added'>" + escapeStringToHTML(x2) + "</span>";
+                            }
+                        }
+
+                        result = result.replace(/\n/g, "<br>");
+                    } else {
+                        result = escapeStringToHTML(w.data);
+                    }
+                } else {
+                    result = escapeStringToHTML(w.data);
+                }
+            } else {
+                result = escapeStringToHTML(w.data);
+            }
+            this.contentView.innerHTML = result;
+        }
+    }
+
+    onOpen() {
+        const { contentEl } = this;
+
+        contentEl.empty();
+        contentEl.createEl("h2", { text: "Document History" });
+        this.fileInfo = contentEl.createDiv("");
+        this.fileInfo.addClass("op-info");
+        const divView = contentEl.createDiv("");
+        divView.addClass("op-flex");
+
+        divView.createEl("input", { type: "range" }, (e) => {
+            this.range = e;
+            e.addEventListener("change", (e) => {
+                this.loadRevs();
+            });
+            e.addEventListener("input", (e) => {
+                this.loadRevs();
+            });
+        });
+        contentEl
+            .createDiv("", (e) => {
+                e.createEl("label", {}, (label) => {
+                    label.appendChild(
+                        createEl("input", { type: "checkbox" }, (checkbox) => {
+                            if (this.showDiff) {
+                                checkbox.checked = true;
+                            }
+                            checkbox.addEventListener("input", (evt: any) => {
+                                this.showDiff = checkbox.checked;
+                                localStorage.setItem("ols-history-highlightdiff", this.showDiff == true ? "1" : "");
+                                this.loadRevs();
+                            });
+                        })
+                    );
+                    label.appendText("Highlight diff");
+                });
+            })
+            .addClass("op-info");
+        this.info = contentEl.createDiv("");
+        this.info.addClass("op-info");
+        this.loadFile();
+        const div = contentEl.createDiv({ text: "Loading old revisions..." });
+        this.contentView = div;
+        div.addClass("op-scrollable");
+        div.addClass("op-pre");
+    }
+    onClose() {
+        const { contentEl } = this;
+        contentEl.empty();
+    }
+}

src/LocalPouchDB.ts

@@ -23,9 +23,9 @@ import {
     MILSTONE_DOCID,
     DatabaseConnectingStatus,
 } from "./types";
-import { resolveWithIgnoreKnownError, delay, path2id, runWithLock } from "./utils";
+import { resolveWithIgnoreKnownError, delay, path2id, runWithLock, isPlainText } from "./utils";
 import { Logger } from "./logger";
-import { checkRemoteVersion, connectRemoteCouchDB } from "./utils_couchdb";
+import { checkRemoteVersion, connectRemoteCouchDB, getLastPostFailedBySize } from "./utils_couchdb";
 import { decrypt, encrypt } from "./e2ee";
 
 export class LocalPouchDB {
@@ -121,7 +121,7 @@ export class LocalPouchDB {
         this.changeHandler = this.cancelHandler(this.changeHandler);
         this.localDatabase = null;
         this.localDatabase = new PouchDB<EntryDoc>(this.dbname + "-livesync", {
-            auto_compaction: true,
+            auto_compaction: this.settings.useHistory ? false : true,
             revs_limit: 100,
             deterministic_revs: true,
         });
@@ -357,7 +357,7 @@ export class LocalPouchDB {
                             Logger(childrens);
                         }
                     } catch (ex) {
-                        Logger(`Something went wrong on reading elements of ${obj._id} from database.`, LOG_LEVEL.NOTICE);
+                        Logger(`Something went wrong on reading elements of ${obj._id} from database:`, LOG_LEVEL.NOTICE);
                         Logger(ex, LOG_LEVEL.VERBOSE);
                         this.corruptedEntries[obj._id] = obj;
                         return false;
@@ -388,7 +388,7 @@ export class LocalPouchDB {
                         Logger(`Missing document content!, could not read ${obj._id} from database.`, LOG_LEVEL.NOTICE);
                         return false;
                     }
-                    Logger(`Something went wrong on reading ${obj._id} from database.`, LOG_LEVEL.NOTICE);
+                    Logger(`Something went wrong on reading ${obj._id} from database:`, LOG_LEVEL.NOTICE);
                     Logger(ex);
                 }
             }
@@ -403,41 +403,44 @@ export class LocalPouchDB {
     async deleteDBEntry(path: string, opt?: PouchDB.Core.GetOptions): Promise<boolean> {
         await this.waitForGCComplete();
         const id = path2id(path);
+
         try {
             let obj: EntryDocResponse = null;
-            if (opt) {
-                obj = await this.localDatabase.get(id, opt);
-            } else {
-                obj = await this.localDatabase.get(id);
-            }
+            return await runWithLock("file:" + id, false, async () => {
+                if (opt) {
+                    obj = await this.localDatabase.get(id, opt);
+                } else {
+                    obj = await this.localDatabase.get(id);
+                }
 
-            if (obj.type && obj.type == "leaf") {
-                //do nothing for leaf;
-                return false;
-            }
-            //Check it out and fix docs to regular case
-            if (!obj.type || (obj.type && obj.type == "notes")) {
-                obj._deleted = true;
-                const r = await this.localDatabase.put(obj);
-                this.updateRecentModifiedDocs(r.id, r.rev, true);
-                if (typeof this.corruptedEntries[obj._id] != "undefined") {
-                    delete this.corruptedEntries[obj._id];
+                if (obj.type && obj.type == "leaf") {
+                    //do nothing for leaf;
+                    return false;
                 }
-                return true;
-                // simple note
-            }
-            if (obj.type == "newnote" || obj.type == "plain") {
-                obj._deleted = true;
-                const r = await this.localDatabase.put(obj);
-                Logger(`entry removed:${obj._id}-${r.rev}`);
-                this.updateRecentModifiedDocs(r.id, r.rev, true);
-                if (typeof this.corruptedEntries[obj._id] != "undefined") {
-                    delete this.corruptedEntries[obj._id];
+                //Check it out and fix docs to regular case
+                if (!obj.type || (obj.type && obj.type == "notes")) {
+                    obj._deleted = true;
+                    const r = await this.localDatabase.put(obj);
+                    this.updateRecentModifiedDocs(r.id, r.rev, true);
+                    if (typeof this.corruptedEntries[obj._id] != "undefined") {
+                        delete this.corruptedEntries[obj._id];
+                    }
+                    return true;
+                    // simple note
                 }
-                return true;
-            } else {
-                return false;
-            }
+                if (obj.type == "newnote" || obj.type == "plain") {
+                    obj._deleted = true;
+                    const r = await this.localDatabase.put(obj);
+                    Logger(`entry removed:${obj._id}-${r.rev}`);
+                    this.updateRecentModifiedDocs(r.id, r.rev, true);
+                    if (typeof this.corruptedEntries[obj._id] != "undefined") {
+                        delete this.corruptedEntries[obj._id];
+                    }
+                    return true;
+                } else {
+                    return false;
+                }
+            });
         } catch (ex) {
             if (ex.status && ex.status == 404) {
                 return false;
@@ -478,10 +481,13 @@ export class LocalPouchDB {
         let notfound = 0;
         for (const v of delDocs) {
             try {
-                const item = await this.localDatabase.get(v);
-                item._deleted = true;
-                await this.localDatabase.put(item);
-                this.updateRecentModifiedDocs(item._id, item._rev, true);
+                await runWithLock("file:" + v, false, async () => {
+                    const item = await this.localDatabase.get(v);
+                    item._deleted = true;
+                    await this.localDatabase.put(item);
+                    this.updateRecentModifiedDocs(item._id, item._rev, true);
+                });
+
                 deleteCount++;
             } catch (ex) {
                 if (ex.status && ex.status == 404) {
@@ -495,18 +501,6 @@ export class LocalPouchDB {
         Logger(`deleteDBEntryPrefix:deleted ${deleteCount} items, skipped ${notfound}`);
         return true;
     }
-    isPlainText(filename: string): boolean {
-        if (filename.endsWith(".md")) return true;
-        if (filename.endsWith(".txt")) return true;
-        if (filename.endsWith(".svg")) return true;
-        if (filename.endsWith(".html")) return true;
-        if (filename.endsWith(".csv")) return true;
-        if (filename.endsWith(".css")) return true;
-        if (filename.endsWith(".js")) return true;
-        if (filename.endsWith(".xml")) return true;
-
-        return false;
-    }
     async putDBEntry(note: LoadedEntry) {
         await this.waitForGCComplete();
         let leftData = note.data;
@@ -518,7 +512,7 @@ export class LocalPouchDB {
         let plainSplit = false;
         let cacheUsed = 0;
         const userpasswordHash = this.h32Raw(new TextEncoder().encode(this.settings.passphrase));
-        if (this.isPlainText(note._id)) {
+        if (isPlainText(note._id)) {
             pieceSize = MAX_DOC_SIZE;
             plainSplit = true;
         }
@@ -540,7 +534,6 @@ export class LocalPouchDB {
                 cPieceSize = 0;
                 // lookup for next splittion .
                 // we're standing on "\n"
-                // debugger
                 do {
                     const n1 = leftData.indexOf("\n", cPieceSize + 1);
                     const n2 = leftData.indexOf("\n\n", cPieceSize + 1);
@@ -601,7 +594,7 @@ export class LocalPouchDB {
                             try {
                                 pieceData.data = await decrypt(pieceData.data, this.settings.passphrase);
                             } catch (e) {
-                                Logger("Decode failed !");
+                                Logger("Decode failed!");
                                 throw e;
                             }
                         }
@@ -674,8 +667,8 @@ export class LocalPouchDB {
                     }
                 }
             } catch (ex) {
-                Logger("ERROR ON SAVING LEAVES ");
-                Logger(ex);
+                Logger("ERROR ON SAVING LEAVES:", LOG_LEVEL.NOTICE);
+                Logger(ex, LOG_LEVEL.NOTICE);
                 saved = false;
             }
         }
@@ -691,33 +684,35 @@ export class LocalPouchDB {
                 type: plainSplit ? "plain" : "newnote",
             };
             // Here for upsert logic,
-            try {
-                const old = await this.localDatabase.get(newDoc._id);
-                if (!old.type || old.type == "notes" || old.type == "newnote" || old.type == "plain") {
-                    // simple use rev for new doc
-                    newDoc._rev = old._rev;
+            await runWithLock("file:" + newDoc._id, false, async () => {
+                try {
+                    const old = await this.localDatabase.get(newDoc._id);
+                    if (!old.type || old.type == "notes" || old.type == "newnote" || old.type == "plain") {
+                        // simple use rev for new doc
+                        newDoc._rev = old._rev;
+                    }
+                } catch (ex) {
+                    if (ex.status && ex.status == 404) {
+                        // NO OP/
+                    } else {
+                        throw ex;
+                    }
                 }
-            } catch (ex) {
-                if (ex.status && ex.status == 404) {
-                    // NO OP/
+                const r = await this.localDatabase.put(newDoc, { force: true });
+                this.updateRecentModifiedDocs(r.id, r.rev, newDoc._deleted);
+                if (typeof this.corruptedEntries[note._id] != "undefined") {
+                    delete this.corruptedEntries[note._id];
+                }
+                if (this.settings.checkIntegrityOnSave) {
+                    if (!this.sanCheck(await this.localDatabase.get(r.id))) {
+                        Logger("note save failed!", LOG_LEVEL.NOTICE);
+                    } else {
+                        Logger(`note has been surely saved:${newDoc._id}:${r.rev}`);
+                    }
                 } else {
-                    throw ex;
+                    Logger(`note saved:${newDoc._id}:${r.rev}`);
                 }
-            }
-            const r = await this.localDatabase.put(newDoc, { force: true });
-            this.updateRecentModifiedDocs(r.id, r.rev, newDoc._deleted);
-            if (typeof this.corruptedEntries[note._id] != "undefined") {
-                delete this.corruptedEntries[note._id];
-            }
-            if (this.settings.checkIntegrityOnSave) {
-                if (!this.sanCheck(await this.localDatabase.get(r.id))) {
-                    Logger("note save failed!", LOG_LEVEL.NOTICE);
-                } else {
-                    Logger(`note has been surely saved:${newDoc._id}:${r.rev}`);
-                }
-            } else {
-                Logger(`note saved:${newDoc._id}:${r.rev}`);
-            }
+            });
         } else {
             Logger(`note coud not saved:${note._id}`);
         }
@@ -756,6 +751,12 @@ export class LocalPouchDB {
             }
 
             const syncOptionBase: PouchDB.Replication.SyncOptions = {
+                pull: {
+                    checkpoint: "target",
+                },
+                push: {
+                    checkpoint: "source",
+                },
                 batches_limit: setting.batches_limit,
                 batch_size: setting.batch_size,
             };
@@ -763,7 +764,7 @@ export class LocalPouchDB {
             const db = dbret.db;
             const totalCount = (await this.localDatabase.info()).doc_count;
             //replicate once
-            const replicate = this.localDatabase.replicate.to(db, syncOptionBase);
+            const replicate = this.localDatabase.replicate.to(db, { checkpoint: "source", ...syncOptionBase });
             replicate
                 .on("active", () => {
                     this.syncStatus = "CONNECTED";
@@ -799,7 +800,7 @@ export class LocalPouchDB {
         });
     }
 
-    async checkReplicationConnectivity(setting: ObsidianLiveSyncSettings, keepAlive: boolean) {
+    async checkReplicationConnectivity(setting: ObsidianLiveSyncSettings, keepAlive: boolean, skipCheck: boolean) {
         if (!this.isReady) {
             Logger("Database is not ready.");
             return false;
@@ -821,73 +822,99 @@ export class LocalPouchDB {
         }
         const dbret = await connectRemoteCouchDB(uri, auth);
         if (typeof dbret === "string") {
-            Logger(`could not connect to ${uri}:${dbret}`, LOG_LEVEL.NOTICE);
+            Logger(`could not connect to ${uri}: ${dbret}`, LOG_LEVEL.NOTICE);
             return false;
         }
 
-        if (!(await checkRemoteVersion(dbret.db, this.migrate.bind(this), VER))) {
-            Logger("Remote database is newer or corrupted, make sure to latest version of self-hosted-livesync installed", LOG_LEVEL.NOTICE);
-            return false;
+        if (!skipCheck) {
+            if (!(await checkRemoteVersion(dbret.db, this.migrate.bind(this), VER))) {
+                Logger("Remote database is newer or corrupted, make sure to latest version of self-hosted-livesync installed", LOG_LEVEL.NOTICE);
+                return false;
+            }
+
+            const defMilestonePoint: EntryMilestoneInfo = {
+                _id: MILSTONE_DOCID,
+                type: "milestoneinfo",
+                created: (new Date() as any) / 1,
+                locked: false,
+                accepted_nodes: [this.nodeid],
+            };
+            // const remoteInfo = dbret.info;
+            // const localInfo = await this.localDatabase.info();
+            // const remoteDocsCount = remoteInfo.doc_count;
+            // const localDocsCount = localInfo.doc_count;
+            // const remoteUpdSeq = typeof remoteInfo.update_seq == "string" ? Number(remoteInfo.update_seq.split("-")[0]) : remoteInfo.update_seq;
+            // const localUpdSeq = typeof localInfo.update_seq == "string" ? Number(localInfo.update_seq.split("-")[0]) : localInfo.update_seq;
+
+            // Logger(`Database diffences: remote:${remoteDocsCount} docs / last update ${remoteUpdSeq}`);
+            // Logger(`Database diffences: local :${localDocsCount} docs / last update ${localUpdSeq}`);
+
+            const remoteMilestone: EntryMilestoneInfo = await resolveWithIgnoreKnownError(dbret.db.get(MILSTONE_DOCID), defMilestonePoint);
+            this.remoteLocked = remoteMilestone.locked;
+            this.remoteLockedAndDeviceNotAccepted = remoteMilestone.locked && remoteMilestone.accepted_nodes.indexOf(this.nodeid) == -1;
+
+            if (remoteMilestone.locked && remoteMilestone.accepted_nodes.indexOf(this.nodeid) == -1) {
+                Logger("Remote database marked as 'Auto Sync Locked'. And this devide does not marked as resolved device. see settings dialog.", LOG_LEVEL.NOTICE);
+                return false;
+            }
+            if (typeof remoteMilestone._rev == "undefined") {
+                await dbret.db.put(remoteMilestone);
+            }
         }
-
-        const defMilestonePoint: EntryMilestoneInfo = {
-            _id: MILSTONE_DOCID,
-            type: "milestoneinfo",
-            created: (new Date() as any) / 1,
-            locked: false,
-            accepted_nodes: [this.nodeid],
-        };
-
-        const remoteMilestone: EntryMilestoneInfo = await resolveWithIgnoreKnownError(dbret.db.get(MILSTONE_DOCID), defMilestonePoint);
-        this.remoteLocked = remoteMilestone.locked;
-        this.remoteLockedAndDeviceNotAccepted = remoteMilestone.locked && remoteMilestone.accepted_nodes.indexOf(this.nodeid) == -1;
-
-        if (remoteMilestone.locked && remoteMilestone.accepted_nodes.indexOf(this.nodeid) == -1) {
-            Logger("Remote database marked as 'Auto Sync Locked'. And this devide does not marked as resolved device. see settings dialog.", LOG_LEVEL.NOTICE);
-            return false;
-        }
-        if (typeof remoteMilestone._rev == "undefined") {
-            await dbret.db.put(remoteMilestone);
-        }
-
         const syncOptionBase: PouchDB.Replication.SyncOptions = {
             batches_limit: setting.batches_limit,
             batch_size: setting.batch_size,
         };
         const syncOption: PouchDB.Replication.SyncOptions = keepAlive ? { live: true, retry: true, heartbeat: 30000, ...syncOptionBase } : { ...syncOptionBase };
 
-        return { db: dbret.db, syncOptionBase, syncOption };
+        return { db: dbret.db, info: dbret.info, syncOptionBase, syncOption };
     }
 
     async openReplication(setting: ObsidianLiveSyncSettings, keepAlive: boolean, showResult: boolean, callback: (e: PouchDB.Core.ExistingDocument<EntryDoc>[]) => Promise<void>): Promise<boolean> {
         return await runWithLock("replicate", false, () => {
-            return this._openReplication(setting, keepAlive, showResult, callback);
+            return this._openReplication(setting, keepAlive, showResult, callback, false);
         });
     }
 
-    async _openReplication(setting: ObsidianLiveSyncSettings, keepAlive: boolean, showResult: boolean, callback: (e: PouchDB.Core.ExistingDocument<EntryDoc>[]) => Promise<void>): Promise<boolean> {
-        const ret = await this.checkReplicationConnectivity(setting, keepAlive);
+    originalSetting: ObsidianLiveSyncSettings = null;
+    // last_seq: number = 200;
+    async _openReplication(setting: ObsidianLiveSyncSettings, keepAlive: boolean, showResult: boolean, callback: (e: PouchDB.Core.ExistingDocument<EntryDoc>[]) => Promise<void>, retrying: boolean): Promise<boolean> {
+        const ret = await this.checkReplicationConnectivity(setting, keepAlive, retrying);
         if (ret === false) return false;
         let notice: Notice = null;
         if (showResult) {
-            notice = new Notice("Replicating", 0);
+            notice = new Notice("Looking for the point last synchronized point.", 0);
         }
         const { db, syncOptionBase, syncOption } = ret;
         //replicate once
         this.syncStatus = "STARTED";
+        this.updateInfo();
 
         let resolved = false;
         const docArrivedOnStart = this.docArrived;
         const docSentOnStart = this.docSent;
+
         const _openReplicationSync = () => {
             Logger("Sync Main Started");
+            if (!retrying) {
+                this.originalSetting = setting;
+            }
             this.syncHandler = this.cancelHandler(this.syncHandler);
-            this.syncHandler = this.localDatabase.sync<EntryDoc>(db, syncOption);
+            this.syncHandler = this.localDatabase.sync<EntryDoc>(db, {
+                ...syncOption,
+                pull: {
+                    checkpoint: "target",
+                },
+                push: {
+                    checkpoint: "source",
+                },
+            });
             this.syncHandler
                 .on("active", () => {
                     this.syncStatus = "CONNECTED";
                     this.updateInfo();
                     Logger("Replication activated");
+                    if (notice != null) notice.setMessage(`Activated..`);
                 })
                 .on("change", async (e) => {
                     try {
@@ -905,8 +932,18 @@ export class LocalPouchDB {
                         }
                         this.updateInfo();
                     } catch (ex) {
-                        Logger("Replication callback error");
-                        Logger(ex);
+                        Logger("Replication callback error", LOG_LEVEL.NOTICE);
+                        Logger(ex, LOG_LEVEL.NOTICE);
+                    }
+                    // re-connect to retry with original setting
+                    if (retrying) {
+                        if (this.docSent - docSentOnStart + (this.docArrived - docArrivedOnStart) > this.originalSetting.batch_size * 2) {
+                            // restore sync values
+                            Logger("Back into original settings once.");
+                            if (notice != null) notice.hide();
+                            this.syncHandler = this.cancelHandler(this.syncHandler);
+                            this._openReplication(this.originalSetting, keepAlive, showResult, callback, false);
+                        }
                     }
                 })
                 .on("complete", (e) => {
@@ -932,8 +969,25 @@ export class LocalPouchDB {
                     this.syncHandler = this.cancelHandler(this.syncHandler);
                     this.updateInfo();
                     if (notice != null) notice.hide();
-                    Logger("Replication error", LOG_LEVEL.NOTICE);
-                    Logger(e);
+                    if (getLastPostFailedBySize()) {
+                        if (keepAlive) {
+                            Logger("Replication stopped.", LOG_LEVEL.NOTICE);
+                        } else {
+                            // Duplicate settings for smaller batch.
+                            const xsetting: ObsidianLiveSyncSettings = JSON.parse(JSON.stringify(setting));
+                            xsetting.batch_size = Math.ceil(xsetting.batch_size / 2);
+                            xsetting.batches_limit = Math.ceil(xsetting.batches_limit / 2);
+                            if (xsetting.batch_size <= 3 || xsetting.batches_limit <= 3) {
+                                Logger("We can't replicate more lower value.", showResult ? LOG_LEVEL.NOTICE : LOG_LEVEL.INFO);
+                            } else {
+                                Logger(`Retry with lower batch size:${xsetting.batch_size}/${xsetting.batches_limit}`, showResult ? LOG_LEVEL.NOTICE : LOG_LEVEL.INFO);
+                                this._openReplication(xsetting, keepAlive, showResult, callback, true);
+                            }
+                        }
+                    } else {
+                        Logger("Replication error", LOG_LEVEL.NOTICE);
+                        Logger(e);
+                    }
                 })
                 .on("paused", (e) => {
                     this.syncStatus = "PAUSED";
@@ -958,7 +1012,7 @@ export class LocalPouchDB {
         Logger(await db.info(), LOG_LEVEL.VERBOSE);
         let replicate: PouchDB.Replication.Replication<EntryDoc>;
         try {
-            replicate = this.localDatabase.replicate.from(db, syncOptionBase);
+            replicate = this.localDatabase.replicate.from(db, { checkpoint: "target", ...syncOptionBase });
             replicate
                 .on("active", () => {
                     this.syncStatus = "CONNECTED";
@@ -978,8 +1032,8 @@ export class LocalPouchDB {
                             notice.setMessage(`Replication pulled:${e.docs_read}`);
                         }
                     } catch (ex) {
-                        Logger("Replication callback error");
-                        Logger(ex);
+                        Logger("Replication callback error", LOG_LEVEL.NOTICE);
+                        Logger(ex, LOG_LEVEL.NOTICE);
                     }
                 });
             this.syncStatus = "COMPLETED";
@@ -992,11 +1046,11 @@ export class LocalPouchDB {
         } catch (ex) {
             this.syncStatus = "ERRORED";
             this.updateInfo();
-            Logger("Pulling Replication error", LOG_LEVEL.NOTICE);
+            Logger("Pulling Replication error:", LOG_LEVEL.NOTICE);
+            Logger(ex, LOG_LEVEL.NOTICE);
             this.cancelHandler(replicate);
             this.syncHandler = this.cancelHandler(this.syncHandler);
             if (notice != null) notice.hide();
-            // debugger;
             throw ex;
         }
     }
@@ -1034,7 +1088,8 @@ export class LocalPouchDB {
             Logger("Remote Database Destroyed", LOG_LEVEL.NOTICE);
             await this.tryCreateRemoteDatabase(setting);
         } catch (ex) {
-            Logger("something happend on Remote Database Destory", LOG_LEVEL.NOTICE);
+            Logger("Something happened on Remote Database Destory:", LOG_LEVEL.NOTICE);
+            Logger(ex, LOG_LEVEL.NOTICE);
         }
     }
     async tryCreateRemoteDatabase(setting: ObsidianLiveSyncSettings) {
@@ -1139,7 +1194,13 @@ export class LocalPouchDB {
         }
         return false;
     }
+
     async garbageCollect() {
+        // if (this.settings.useHistory) {
+        //     Logger("GC skipped for using history", LOG_LEVEL.VERBOSE);
+        //     return;
+        // }
+        // NOTE:Garbage collection could break old revisions.
         await runWithLock("replicate", true, async () => {
             if (this.gcRunning) return;
             this.gcRunning = true;
@@ -1153,29 +1214,36 @@ export class LocalPouchDB {
                 let usedPieces: string[] = [];
                 Logger("Collecting Garbage");
                 do {
-                    const result = await this.localDatabase.allDocs({ include_docs: true, skip: c, limit: 500, conflicts: true });
+                    const result = await this.localDatabase.allDocs({ include_docs: false, skip: c, limit: 2000, conflicts: true });
                     readCount = result.rows.length;
                     Logger("checked:" + readCount);
                     if (readCount > 0) {
                         //there are some result
                         for (const v of result.rows) {
-                            const doc = v.doc;
-                            if (doc.type == "newnote" || doc.type == "plain") {
-                                // used pieces memo.
-                                usedPieces = Array.from(new Set([...usedPieces, ...doc.children]));
-                                if (doc._conflicts) {
-                                    for (const cid of doc._conflicts) {
-                                        const p = await this.localDatabase.get<EntryDoc>(doc._id, { rev: cid });
-                                        if (p.type == "newnote" || p.type == "plain") {
-                                            usedPieces = Array.from(new Set([...usedPieces, ...p.children]));
+                            if (v.id.startsWith("h:")) {
+                                hashPieces = Array.from(new Set([...hashPieces, v.id]));
+                            } else {
+                                const docT = await this.localDatabase.get(v.id, { revs_info: true });
+                                const revs = docT._revs_info;
+                                // console.log(`revs:${revs.length}`)
+                                for (const rev of revs) {
+                                    if (rev.status != "available") continue;
+                                    // console.log(`id:${docT._id},rev:${rev.rev}`);
+                                    const doc = await this.localDatabase.get(v.id, { rev: rev.rev });
+                                    if ("children" in doc) {
+                                        // used pieces memo.
+                                        usedPieces = Array.from(new Set([...usedPieces, ...doc.children]));
+                                        if (doc._conflicts) {
+                                            for (const cid of doc._conflicts) {
+                                                const p = await this.localDatabase.get<EntryDoc>(doc._id, { rev: cid });
+                                                if (p.type == "newnote" || p.type == "plain") {
+                                                    usedPieces = Array.from(new Set([...usedPieces, ...p.children]));
+                                                }
+                                            }
                                         }
                                     }
                                 }
                             }
-                            if (doc.type == "leaf") {
-                                // all pieces.
-                                hashPieces = Array.from(new Set([...hashPieces, doc._id]));
-                            }
                         }
                     }
                     c += readCount;

src/ObsidianLiveSyncSettingTab.ts

@@ -1,6 +1,6 @@
 import { App, Notice, PluginSettingTab, Setting, sanitizeHTMLToDom } from "obsidian";
 import { EntryDoc, LOG_LEVEL } from "./types";
-import { escapeStringToHTML, versionNumberString2Number, path2id, id2path, runWithLock } from "./utils";
+import { path2id, id2path, runWithLock } from "./utils";
 import { Logger } from "./logger";
 import { connectRemoteCouchDB } from "./utils_couchdb";
 import { testCrypt } from "./e2ee";
@@ -72,7 +72,7 @@ export class ObsidianLiveSyncSettingTab extends PluginSettingTab {
 
         const containerRemoteDatabaseEl = containerEl.createDiv();
         containerRemoteDatabaseEl.createEl("h3", { text: "Remote Database configuration" });
-        const syncWarn = containerRemoteDatabaseEl.createEl("div", { text: "The remote configuration is locked while any synchronization is enabled." });
+        const syncWarn = containerRemoteDatabaseEl.createEl("div", { text: `These settings are kept locked while automatic synchronization options are enabled. Disable these options in the "Sync Settings" tab to unlock.` });
         syncWarn.addClass("op-warn");
         syncWarn.addClass("sls-hidden");
 
@@ -602,6 +602,15 @@ export class ObsidianLiveSyncSettingTab extends PluginSettingTab {
                     })
             );
 
+        new Setting(containerMiscellaneousEl)
+            .setName("Use history (beta)")
+            .setDesc("Use history dialog (Restart required, auto compaction would be disabled, and more storage will be consumed)")
+            .addToggle((toggle) =>
+                toggle.setValue(this.plugin.settings.useHistory).onChange(async (value) => {
+                    this.plugin.settings.useHistory = value;
+                    await this.plugin.saveSettings();
+                })
+            );
         addScreenElement("40", containerMiscellaneousEl);
 
         const containerHatchEl = containerEl.createDiv();
@@ -635,6 +644,8 @@ export class ObsidianLiveSyncSettingTab extends PluginSettingTab {
                 c.addClass("op-warn");
             }
         }
+        const hatchWarn = containerHatchEl.createEl("div", { text: `To stop the bootup sequence for fixing problems on databases, you can put redflag.md on top of your vault (Rebooting obsidian is required).` });
+        hatchWarn.addClass("op-warn");
         const dropHistory = async (sendToServer: boolean) => {
             this.plugin.settings.liveSync = false;
             this.plugin.settings.periodicReplication = false;
@@ -763,6 +774,10 @@ export class ObsidianLiveSyncSettingTab extends PluginSettingTab {
                 })
             );
 
+        containerHatchEl.createEl("div", {
+            text: sanitizeHTMLToDom(`Advanced buttons<br>
+                These buttons could break your database easily.`),
+        });
         new Setting(containerHatchEl)
             .setName("Reset remote database")
             .setDesc("Reset remote database, this affects only database. If you replicate again, remote database will restored by local database.")
@@ -805,7 +820,7 @@ export class ObsidianLiveSyncSettingTab extends PluginSettingTab {
         // With great respect, thank you TfTHacker!
         // refered: https://github.com/TfTHacker/obsidian42-brat/blob/main/src/features/BetaPlugins.ts
         const containerPluginSettings = containerEl.createDiv();
-        containerPluginSettings.createEl("h3", { text: "Plugins and settings (bleeding edge)" });
+        containerPluginSettings.createEl("h3", { text: "Plugins and settings (beta)" });
 
         const updateDisabledOfDeviceAndVaultName = () => {
             vaultName.setDisabled(this.plugin.settings.autoSweepPlugins || this.plugin.settings.autoSweepPluginsPeriodic);
@@ -815,14 +830,12 @@ export class ObsidianLiveSyncSettingTab extends PluginSettingTab {
             toggle.setValue(this.plugin.settings.usePluginSync).onChange(async (value) => {
                 this.plugin.settings.usePluginSync = value;
                 await this.plugin.saveSettings();
-                updatePluginPane();
             })
         );
         new Setting(containerPluginSettings).setName("Show own plugins and settings").addToggle((toggle) =>
             toggle.setValue(this.plugin.settings.showOwnPlugins).onChange(async (value) => {
                 this.plugin.settings.showOwnPlugins = value;
                 await this.plugin.saveSettings();
-                updatePluginPane();
             })
         );
 
@@ -862,240 +875,26 @@ export class ObsidianLiveSyncSettingTab extends PluginSettingTab {
             .setDesc("")
             .addText((text) => {
                 text.setPlaceholder("desktop-main")
-                    .setValue(this.plugin.settings.deviceAndVaultName)
+                    .setValue(this.plugin.deviceAndVaultName)
                     .onChange(async (value) => {
-                        this.plugin.settings.deviceAndVaultName = value;
+                        this.plugin.deviceAndVaultName = value;
                         await this.plugin.saveSettings();
                     });
                 // text.inputEl.setAttribute("type", "password");
             });
+        new Setting(containerPluginSettings)
+            .setName("Open")
+            .setDesc("Open the plugin dialog")
+            .addButton((button) => {
+                button
+                    .setButtonText("Open")
+                    .setDisabled(false)
+                    .onClick(() => {
+                        this.plugin.showPluginSyncModal();
+                    });
+            });
 
         updateDisabledOfDeviceAndVaultName();
-        const sweepPlugin = async (showMessage: boolean) => {
-            if (!this.plugin.settings.usePluginSync) {
-                return;
-            }
-            await this.plugin.sweepPlugin(showMessage);
-            updatePluginPane();
-        };
-        const updatePluginPane = async () => {
-            pluginConfig.innerHTML = "<div class='sls-plugins-wrap'>Retrieving...</div>";
-            const { plugins, allPlugins, thisDevicePlugins } = await this.plugin.getPluginList();
-            let html = `
-            <div class='sls-plugins-wrap'>
-            <table class='sls-plugins-tbl'>
-            `;
-            for (const vaults in plugins) {
-                if (!this.plugin.settings.showOwnPlugins && vaults == this.plugin.settings.deviceAndVaultName) continue;
-                html += `
-                <tr>
-                    <th colspan=1 class='sls-plugins-tbl-device-head'>${escapeStringToHTML(vaults)}</th>
-                    <td class='sls-plugins-tbl-device-head sls-plugins-tbl-buttons'>
-                        <button class='sls-plugin-apply-all-newer-plugin mod-cta' data-key="${vaults}" aria-label="Apply all newer (without setting)">⚡</button>
-                        <button class='sls-plugin-apply-all-newer-setting mod-cta' data-key="${vaults}" aria-label="Apply all newer settings">📚</button>
-                        <button class='sls-plugin-delete mod-warning' data-key="${vaults}" aria-label="Delete">❌</button>
-                    </td>
-                </tr>`;
-                for (const v of plugins[vaults]) {
-                    const mtime = v.mtime == 0 ? "-" : new Date(v.mtime).toLocaleString();
-                    let settingApplyable: boolean | string = "-";
-                    let settingFleshness = "";
-                    let isSameVersion = false;
-                    let isSameContents = false;
-                    if (thisDevicePlugins[v.manifest.id]) {
-                        if (thisDevicePlugins[v.manifest.id].manifest.version == v.manifest.version) {
-                            isSameVersion = true;
-                        }
-                        if (thisDevicePlugins[v.manifest.id].styleCss == v.styleCss && thisDevicePlugins[v.manifest.id].mainJs == v.mainJs && thisDevicePlugins[v.manifest.id].manifestJson == v.manifestJson) {
-                            isSameContents = true;
-                        }
-                    }
-                    if (thisDevicePlugins[v.manifest.id] && v.dataJson) {
-                        // have this plugin.
-                        const localSetting = thisDevicePlugins[v.manifest.id].dataJson || null;
-
-                        try {
-                            const remoteSetting = v.dataJson;
-                            if (!localSetting) {
-                                settingFleshness = "newer";
-                                settingApplyable = true;
-                            } else if (localSetting == remoteSetting) {
-                                settingApplyable = "even";
-                            } else {
-                                if (v.mtime > thisDevicePlugins[v.manifest.id].mtime) {
-                                    settingFleshness = "newer";
-                                } else {
-                                    settingFleshness = "older";
-                                }
-                                settingApplyable = true;
-                            }
-                        } catch (ex) {
-                            settingApplyable = "could not decrypt";
-                        }
-                    } else if (!v.dataJson) {
-                        settingApplyable = "N/A";
-                    }
-                    // very ugly way.
-                    const piece = `
-                    <tr class='divider'>
-                        <th colspan=2></th>
-                    </tr>
-                    <tr>
-                        <th class='sls-table-head'>${escapeStringToHTML(v.manifest.name)}</th>
-                        <td class="sls-table-tail tcenter">${isSameContents ? "even" : `<button data-key='${v._id}' class='apply-plugin-version mod-cta'>Use (${isSameVersion ? "=" : ""}${v.manifest.version}) </button>`}</td>
-                    </tr>
-                    <tr>
-                        <td class="sls-table-head tcenter">${escapeStringToHTML(mtime)}</td>
-                        <td class="sls-table-tail tcenter">${settingApplyable === true ? "<button data-key='" + v._id + "' class='apply-plugin-data mod-cta'>Apply (" + settingFleshness + ")</button>" : settingApplyable}</td>
-                    </tr>
-                    `;
-                    html += piece;
-                }
-                html += `
-                <tr class='divider'>
-                    <th colspan=2></th>
-                </tr>
-`;
-            }
-            html += "</table></div>";
-            pluginConfig.innerHTML = html;
-            pluginConfig.querySelectorAll(".apply-plugin-data").forEach((e) =>
-                e.addEventListener("click", async (evt) => {
-                    const plugin = allPlugins[e.attributes.getNamedItem("data-key").value];
-                    Logger(`Updating plugin:${plugin.manifest.name}`, LOG_LEVEL.NOTICE);
-                    await this.plugin.applyPluginData(plugin);
-                    Logger(`Setting done:${plugin.manifest.name}`, LOG_LEVEL.NOTICE);
-                    await sweepPlugin(true);
-                })
-            );
-            pluginConfig.querySelectorAll(".apply-plugin-version").forEach((e) =>
-                e.addEventListener("click", async (evt) => {
-                    const plugin = allPlugins[e.attributes.getNamedItem("data-key").value];
-                    Logger(`Setting plugin:${plugin.manifest.name}`, LOG_LEVEL.NOTICE);
-                    await this.plugin.applyPlugin(plugin);
-                    Logger(`Updated plugin:${plugin.manifest.name}`, LOG_LEVEL.NOTICE);
-                    await sweepPlugin(true);
-                })
-            );
-            pluginConfig.querySelectorAll(".sls-plugin-apply-all-newer-plugin").forEach((e) =>
-                e.addEventListener("click", async (evt) => {
-                    Logger("Apply all newer plugins.", LOG_LEVEL.NOTICE);
-                    const vaultname = e.attributes.getNamedItem("data-key").value;
-                    const plugins = Object.values(allPlugins).filter((e) => e.deviceVaultName == vaultname && e.manifest.id != "obsidian-livesync");
-                    for (const plugin of plugins) {
-                        const currentPlugin = thisDevicePlugins[plugin.manifest.id];
-                        if (currentPlugin) {
-                            const thisVersion = versionNumberString2Number(plugin.manifest.version);
-                            const currentVersion = versionNumberString2Number(currentPlugin.manifest.version);
-                            if (thisVersion > currentVersion) {
-                                Logger(`Updating plugin:${plugin.manifest.name}`, LOG_LEVEL.NOTICE);
-                                await this.plugin.applyPlugin(plugin);
-                                Logger(`Updated plugin:${plugin.manifest.name}`, LOG_LEVEL.NOTICE);
-                            } else {
-                                Logger(`Plugin ${plugin.manifest.name} is not new`);
-                            }
-                        } else {
-                            Logger(`Updating plugin:${plugin.manifest.name}`, LOG_LEVEL.NOTICE);
-                            await this.plugin.applyPlugin(plugin);
-                            Logger(`Updated plugin:${plugin.manifest.name}`, LOG_LEVEL.NOTICE);
-                        }
-                    }
-                    await sweepPlugin(true);
-                    Logger("Done", LOG_LEVEL.NOTICE);
-                })
-            );
-            pluginConfig.querySelectorAll(".sls-plugin-apply-all-newer-setting").forEach((e) =>
-                e.addEventListener("click", async (evt) => {
-                    Logger("Apply all newer settings.", LOG_LEVEL.NOTICE);
-                    const vaultname = e.attributes.getNamedItem("data-key").value;
-                    const plugins = Object.values(allPlugins).filter((e) => e.deviceVaultName == vaultname && e.manifest.id != "obsidian-livesync");
-                    for (const plugin of plugins) {
-                        const currentPlugin = thisDevicePlugins[plugin.manifest.id];
-                        if (currentPlugin) {
-                            const thisVersion = plugin.mtime;
-                            const currentVersion = currentPlugin.mtime;
-                            if (thisVersion > currentVersion) {
-                                Logger(`Setting plugin:${plugin.manifest.name}`, LOG_LEVEL.NOTICE);
-                                await this.plugin.applyPluginData(plugin);
-                                Logger(`Setting done:${plugin.manifest.name}`, LOG_LEVEL.NOTICE);
-                            } else {
-                                Logger(`Setting ${plugin.manifest.name} is not new`);
-                            }
-                        } else {
-                            Logger(`Setting plugin:${plugin.manifest.name}`, LOG_LEVEL.NOTICE);
-                            await this.plugin.applyPluginData(plugin);
-                            Logger(`Setting done:${plugin.manifest.name}`, LOG_LEVEL.NOTICE);
-                        }
-                    }
-                    await sweepPlugin(true);
-                    Logger("Done", LOG_LEVEL.NOTICE);
-                })
-            );
-            pluginConfig.querySelectorAll(".sls-plugin-delete").forEach((e) =>
-                e.addEventListener("click", async (evt) => {
-                    const db = this.plugin.localDatabase.localDatabase;
-                    const vaultname = e.attributes.getNamedItem("data-key").value;
-                    const oldDocs = await db.allDocs({ startkey: `ps:${vaultname}-`, endkey: `ps:${vaultname}.`, include_docs: true });
-                    Logger(`Deleting ${vaultname}`, LOG_LEVEL.NOTICE);
-                    const delDocs = oldDocs.rows.map((e) => {
-                        e.doc._deleted = true;
-                        return e.doc;
-                    });
-                    await db.bulkDocs(delDocs);
-                    Logger(`Deleted ${vaultname}`, LOG_LEVEL.NOTICE);
-                    await this.plugin.replicate(true);
-                    await updatePluginPane();
-                })
-            );
-        };
-
-        const pluginConfig = containerPluginSettings.createEl("div");
-
-        new Setting(containerPluginSettings)
-            .setName("Reload")
-            .setDesc("Replicate once and reload the list")
-            .addButton((button) =>
-                button
-                    .setButtonText("Reload")
-                    .setDisabled(false)
-                    .onClick(async () => {
-                        if (!this.plugin.settings.usePluginSync) {
-                            return;
-                        }
-                        await this.plugin.replicate(true);
-                        await updatePluginPane();
-                    })
-            );
-        new Setting(containerPluginSettings)
-            .setName("Save plugins into the database")
-            .setDesc("")
-            .addButton((button) =>
-                button
-                    .setButtonText("Save plugins")
-                    .setDisabled(false)
-                    .onClick(async () => {
-                        if (!this.plugin.settings.usePluginSync) {
-                            return;
-                        }
-                        Logger("Save plugins.", LOG_LEVEL.NOTICE);
-                        await sweepPlugin(true);
-                        Logger("All plugins have been saved.", LOG_LEVEL.NOTICE);
-                        await this.plugin.replicate(true);
-                    })
-            );
-        new Setting(containerPluginSettings)
-            .setName("Check updates")
-            .setDesc("")
-            .addButton((button) =>
-                button
-                    .setButtonText("Check")
-                    .setDisabled(false)
-                    .onClick(async () => {
-                        Logger("Checking plugins.", LOG_LEVEL.NOTICE);
-                        await this.plugin.checkPluginUpdate();
-                    })
-            );
-        updatePluginPane();
 
         addScreenElement("60", containerPluginSettings);
 

src/PluginPane.svelte

@@ -0,0 +1,290 @@
+<script lang="ts">
+    import ObsidianLiveSyncPlugin from "./main";
+    import { onMount } from "svelte";
+    import { DevicePluginList, PluginDataEntry } from "./types";
+    import { versionNumberString2Number } from "./utils";
+
+    type JudgeResult = "" | "NEWER" | "EVEN" | "EVEN_BUT_DIFFERENT" | "OLDER" | "REMOTE_ONLY";
+
+    interface PluginDataEntryDisp extends PluginDataEntry {
+        versionInfo: string;
+        mtimeInfo: string;
+        mtimeFlag: JudgeResult;
+        versionFlag: JudgeResult;
+    }
+
+    export let plugin: ObsidianLiveSyncPlugin;
+    let plugins: PluginDataEntry[] = [];
+    let deviceAndPlugins: { [key: string]: PluginDataEntryDisp[] } = {};
+    let devicePluginList: [string, PluginDataEntryDisp[]][] = [];
+    let ownPlugins: DevicePluginList = null;
+    let showOwnPlugins = false;
+    let targetList: { [key: string]: boolean } = {};
+
+    function saveTargetList() {
+        window.localStorage.setItem("ols-plugin-targetlist", JSON.stringify(targetList));
+    }
+
+    function loadTargetList() {
+        let e = window.localStorage.getItem("ols-plugin-targetlist") || "{}";
+        try {
+            targetList = JSON.parse(e);
+        } catch (_) {
+            // NO OP.
+        }
+    }
+
+    function clearSelection() {
+        targetList = {};
+    }
+
+    async function updateList() {
+        let x = await plugin.getPluginList();
+        ownPlugins = x.thisDevicePlugins;
+        plugins = Object.values(x.allPlugins);
+        let targetListItems = Array.from(new Set(plugins.map((e) => e.deviceVaultName + "---" + e.manifest.id)));
+        let newTargetList: { [key: string]: boolean } = {};
+        for (const id of targetListItems) {
+            for (const tag of ["---plugin", "---setting"]) {
+                newTargetList[id + tag] = id + tag in targetList && targetList[id + tag];
+            }
+        }
+        targetList = newTargetList;
+        saveTargetList();
+    }
+
+    $: {
+        deviceAndPlugins = {};
+        for (const p of plugins) {
+            if (p.deviceVaultName == plugin.deviceAndVaultName && !showOwnPlugins) {
+                continue;
+            }
+            if (!(p.deviceVaultName in deviceAndPlugins)) {
+                deviceAndPlugins[p.deviceVaultName] = [];
+            }
+            let dispInfo: PluginDataEntryDisp = { ...p, versionInfo: "", mtimeInfo: "", versionFlag: "", mtimeFlag: "" };
+            dispInfo.versionInfo = p.manifest.version;
+            let x = new Date().getTime() / 1000;
+            let mtime = p.mtime / 1000;
+            let diff = (x - mtime) / 60;
+            if (p.mtime == 0) {
+                dispInfo.mtimeInfo = `-`;
+            } else if (diff < 60) {
+                dispInfo.mtimeInfo = `${diff | 0} Mins ago`;
+            } else if (diff < 60 * 24) {
+                dispInfo.mtimeInfo = `${(diff / 60) | 0} Hours ago`;
+            } else if (diff < 60 * 24 * 10) {
+                dispInfo.mtimeInfo = `${(diff / (60 * 24)) | 0} Days ago`;
+            } else {
+                dispInfo.mtimeInfo = new Date(dispInfo.mtime).toLocaleString();
+            }
+            // compare with own plugin
+            let id = p.manifest.id;
+
+            if (id in ownPlugins) {
+                // Which we have.
+                const ownPlugin = ownPlugins[id];
+                let localVer = versionNumberString2Number(ownPlugin.manifest.version);
+                let pluginVer = versionNumberString2Number(p.manifest.version);
+                if (localVer > pluginVer) {
+                    dispInfo.versionFlag = "OLDER";
+                } else if (localVer == pluginVer) {
+                    if (ownPlugin.manifestJson + (ownPlugin.styleCss ?? "") + ownPlugin.mainJs != p.manifestJson + (p.styleCss ?? "") + p.mainJs) {
+                        dispInfo.versionFlag = "EVEN_BUT_DIFFERENT";
+                    } else {
+                        dispInfo.versionFlag = "EVEN";
+                    }
+                } else if (localVer < pluginVer) {
+                    dispInfo.versionFlag = "NEWER";
+                }
+                if ((ownPlugin.dataJson ?? "") == (p.dataJson ?? "")) {
+                    if (ownPlugin.mtime == 0 && p.mtime == 0) {
+                        dispInfo.mtimeFlag = "";
+                    } else {
+                        dispInfo.mtimeFlag = "EVEN";
+                    }
+                } else {
+                    if (((ownPlugin.mtime / 1000) | 0) > ((p.mtime / 1000) | 0)) {
+                        dispInfo.mtimeFlag = "OLDER";
+                    } else if (((ownPlugin.mtime / 1000) | 0) == ((p.mtime / 1000) | 0)) {
+                        dispInfo.mtimeFlag = "EVEN_BUT_DIFFERENT";
+                    } else if (((ownPlugin.mtime / 1000) | 0) < ((p.mtime / 1000) | 0)) {
+                        dispInfo.mtimeFlag = "NEWER";
+                    }
+                }
+            } else {
+                dispInfo.versionFlag = "REMOTE_ONLY";
+                dispInfo.mtimeFlag = "REMOTE_ONLY";
+            }
+
+            deviceAndPlugins[p.deviceVaultName].push(dispInfo);
+        }
+        devicePluginList = Object.entries(deviceAndPlugins);
+    }
+
+    function getDispString(stat: JudgeResult): string {
+        if (stat == "") return "";
+        if (stat == "NEWER") return " (Newer)";
+        if (stat == "OLDER") return " (Older)";
+        if (stat == "EVEN") return " (Even)";
+        if (stat == "EVEN_BUT_DIFFERENT") return " (Even but different)";
+        if (stat == "REMOTE_ONLY") return " (Remote Only)";
+        return "";
+    }
+
+    onMount(async () => {
+        loadTargetList();
+        await updateList();
+    });
+
+    function toggleShowOwnPlugins() {
+        showOwnPlugins = !showOwnPlugins;
+    }
+
+    function toggleTarget(key: string) {
+        targetList[key] = !targetList[key];
+        saveTargetList();
+    }
+
+    function toggleAll(devicename: string) {
+        for (const c in targetList) {
+            if (c.startsWith(devicename)) {
+                targetList[c] = true;
+            }
+        }
+    }
+
+    async function sweepPlugins() {
+        //@ts-ignore
+        await plugin.app.plugins.loadManifests();
+        await plugin.sweepPlugin(true);
+        updateList();
+    }
+
+    async function applyPlugins() {
+        for (const c in targetList) {
+            if (targetList[c] == true) {
+                const [deviceAndVault, id, opt] = c.split("---");
+                if (deviceAndVault in deviceAndPlugins) {
+                    const entry = deviceAndPlugins[deviceAndVault].find((e) => e.manifest.id == id);
+                    if (entry) {
+                        if (opt == "plugin") {
+                            if (entry.versionFlag != "EVEN") await plugin.applyPlugin(entry);
+                        } else if (opt == "setting") {
+                            if (entry.mtimeFlag != "EVEN") await plugin.applyPluginData(entry);
+                        }
+                    }
+                }
+            }
+        }
+        //@ts-ignore
+        await plugin.app.plugins.loadManifests();
+        await plugin.sweepPlugin(true);
+        updateList();
+    }
+
+    async function checkUpdates() {
+        await plugin.checkPluginUpdate();
+    }
+    async function replicateAndRefresh() {
+        await plugin.replicate(true);
+        updateList();
+    }
+</script>
+
+<div>
+    <h1>Plugins and their settings</h1>
+    <div class="ols-plugins-div-buttons">
+        Show own items
+        <div class="checkbox-container" class:is-enabled={showOwnPlugins} on:click={toggleShowOwnPlugins} />
+    </div>
+    <div class="sls-plugins-wrap">
+        <table class="sls-plugins-tbl">
+            <tr style="position:sticky">
+                <th class="sls-plugins-tbl-device-head">Name</th>
+                <th class="sls-plugins-tbl-device-head">Info</th>
+                <th class="sls-plugins-tbl-device-head">Target</th>
+            </tr>
+            {#if devicePluginList.length == 0}
+                <tr>
+                    <td colspan="3" class="sls-table-tail tcenter"> Retrieving... </td>
+                </tr>
+            {/if}
+            {#each devicePluginList as [deviceName, devicePlugins]}
+                <tr>
+                    <th colspan="2" class="sls-plugins-tbl-device-head">{deviceName}</th>
+                    <th class="sls-plugins-tbl-device-head">
+                        <button class="mod-cta" on:click={() => toggleAll(deviceName)}>✔</button>
+                    </th>
+                </tr>
+                {#each devicePlugins as plugin}
+                    <tr>
+                        <td class="sls-table-head">{plugin.manifest.name}</td>
+                        <td class="sls-table-tail tcenter">{plugin.versionInfo}{getDispString(plugin.versionFlag)}</td>
+                        <td class="sls-table-tail tcenter">
+                            {#if plugin.versionFlag === "EVEN" || plugin.versionFlag === ""}
+                                -
+                            {:else}
+                                <div class="wrapToggle">
+                                    <div
+                                        class="checkbox-container"
+                                        class:is-enabled={targetList[plugin.deviceVaultName + "---" + plugin.manifest.id + "---plugin"]}
+                                        on:click={() => toggleTarget(plugin.deviceVaultName + "---" + plugin.manifest.id + "---plugin")}
+                                    />
+                                </div>
+                            {/if}
+                        </td>
+                    </tr>
+                    <tr>
+                        <td class="sls-table-head">Settings</td>
+                        <td class="sls-table-tail tcenter">{plugin.mtimeInfo}{getDispString(plugin.mtimeFlag)}</td>
+                        <td class="sls-table-tail tcenter">
+                            {#if plugin.mtimeFlag === "EVEN" || plugin.mtimeFlag === ""}
+                                -
+                            {:else}
+                                <div class="wrapToggle">
+                                    <div
+                                        class="checkbox-container"
+                                        class:is-enabled={targetList[plugin.deviceVaultName + "---" + plugin.manifest.id + "---setting"]}
+                                        on:click={() => toggleTarget(plugin.deviceVaultName + "---" + plugin.manifest.id + "---setting")}
+                                    />
+                                </div>
+                            {/if}
+                        </td>
+                    </tr>
+                    <tr class="divider">
+                        <th colspan="3" />
+                    </tr>
+                {/each}
+            {/each}
+        </table>
+    </div>
+    <div class="ols-plugins-div-buttons">
+        <button class="" on:click={replicateAndRefresh}>Replicate and refresh</button>
+        <button class="" on:click={clearSelection}>Clear Selection</button>
+    </div>
+
+    <div class="ols-plugins-div-buttons">
+        <button class="mod-cta" on:click={checkUpdates}>Check Updates</button>
+        <button class="mod-cta" on:click={sweepPlugins}>Sweep installed</button>
+        <button class="mod-cta" on:click={applyPlugins}>Apply all</button>
+    </div>
+    <!--    <div class="ols-plugins-div-buttons">-->
+    <!--        <button class="mod-warning" on:click={applyPlugins}>Delete all selected</button>-->
+    <!--    </div>-->
+</div>
+
+<style>
+    .ols-plugins-div-buttons {
+        display: flex;
+        flex-direction: row;
+        justify-content: flex-end;
+        margin-top: 8px;
+    }
+
+    .wrapToggle {
+        display: flex;
+        justify-content: center;
+        align-content: center;
+    }
+</style>

src/types.ts

@@ -2,6 +2,7 @@
 // and cloudant limitation is 1MB , we use 900kb;
 
 import { PluginManifest } from "obsidian";
+import * as PouchDB from "pouchdb";
 
 export const MAX_DOC_SIZE = 1000; // for .md file, but if delimiters exists. use that before.
 export const MAX_DOC_SIZE_BIN = 102400; // 100kb
@@ -58,6 +59,7 @@ export interface ObsidianLiveSyncSettings {
     checkIntegrityOnSave: boolean;
     batch_size: number;
     batches_limit: number;
+    useHistory: boolean;
 }
 
 export const DEFAULT_SETTINGS: ObsidianLiveSyncSettings = {
@@ -98,6 +100,7 @@ export const DEFAULT_SETTINGS: ObsidianLiveSyncSettings = {
     checkIntegrityOnSave: false,
     batch_size: 250,
     batches_limit: 40,
+    useHistory: false,
 };
 
 export const PERIODIC_PLUGIN_SWEEP = 60;
@@ -226,3 +229,5 @@ export interface PluginList {
 export interface DevicePluginList {
     [key: string]: PluginDataEntry;
 }
+
+export const FLAGMD_REDFLAG = "redflag.md";

src/utils.ts

@@ -1,6 +1,6 @@
 import { normalizePath } from "obsidian";
 import { Logger } from "./logger";
-import { LOG_LEVEL } from "./types";
+import { FLAGMD_REDFLAG, LOG_LEVEL } from "./types";
 
 export function arrayBufferToBase64(buffer: ArrayBuffer): Promise<string> {
     return new Promise((res) => {
@@ -79,6 +79,13 @@ export function isValidPath(filename: string): boolean {
     return sx == filename;
 }
 
+export function shouldBeIgnored(filename: string): boolean {
+    if (filename == FLAGMD_REDFLAG) {
+        return true;
+    }
+    return false;
+}
+
 export function versionNumberString2Number(version: string): number {
     return version // "1.23.45"
         .split(".") // 1  23  45
@@ -114,20 +121,43 @@ function objectToKey(key: any): string {
     const keys = Object.keys(key).sort((a, b) => a.localeCompare(b));
     return keys.map((e) => e + objectToKey(key[e])).join(":");
 }
-// Just run some async/await as like transacion SERIALIZABLE
+export function getProcessingCounts() {
+    let count = 0;
+    for (const v in pendingProcs) {
+        count += pendingProcs[v].length;
+    }
+    count += runningProcs.length;
+    return count;
+}
 
+let externalNotifier: () => void = () => {};
+let notifyTimer: number = null;
+export function setLockNotifier(fn: () => void) {
+    externalNotifier = fn;
+}
+function notifyLock() {
+    if (notifyTimer != null) {
+        window.clearTimeout(notifyTimer);
+    }
+    notifyTimer = window.setTimeout(() => {
+        externalNotifier();
+    }, 100);
+}
+// Just run async/await as like transacion ISOLATION SERIALIZABLE
 export function runWithLock<T>(key: unknown, ignoreWhenRunning: boolean, proc: () => Promise<T>): Promise<T> {
-    Logger(`Lock:${key}:enter`, LOG_LEVEL.VERBOSE);
+    // Logger(`Lock:${key}:enter`, LOG_LEVEL.VERBOSE);
     const lockKey = typeof key === "string" ? key : objectToKey(key);
     const handleNextProcs = () => {
         if (typeof pendingProcs[lockKey] === "undefined") {
             //simply unlock
             runningProcs.remove(lockKey);
-            Logger(`Lock:${lockKey}:released`, LOG_LEVEL.VERBOSE);
+            notifyLock();
+            // Logger(`Lock:${lockKey}:released`, LOG_LEVEL.VERBOSE);
         } else {
             Logger(`Lock:${lockKey}:left ${pendingProcs[lockKey].length}`, LOG_LEVEL.VERBOSE);
             let nextProc = null;
             nextProc = pendingProcs[lockKey].shift();
+            notifyLock();
             if (nextProc) {
                 // left some
                 nextProc()
@@ -138,11 +168,17 @@ export function runWithLock<T>(key: unknown, ignoreWhenRunning: boolean, proc: (
                     .finally(() => {
                         if (pendingProcs && lockKey in pendingProcs && pendingProcs[lockKey].length == 0) {
                             delete pendingProcs[lockKey];
+                            notifyLock();
                         }
                         queueMicrotask(() => {
                             handleNextProcs();
                         });
                     });
+            } else {
+                if (pendingProcs && lockKey in pendingProcs && pendingProcs[lockKey].length == 0) {
+                    delete pendingProcs[lockKey];
+                    notifyLock();
+                }
             }
         }
     };
@@ -164,7 +200,7 @@ export function runWithLock<T>(key: unknown, ignoreWhenRunning: boolean, proc: (
             new Promise<void>((res, rej) => {
                 proc()
                     .then((v) => {
-                        Logger(`Lock:${key}:processed`, LOG_LEVEL.VERBOSE);
+                        // Logger(`Lock:${key}:processed`, LOG_LEVEL.VERBOSE);
                         handleNextProcs();
                         responderRes(v);
                         res();
@@ -178,10 +214,13 @@ export function runWithLock<T>(key: unknown, ignoreWhenRunning: boolean, proc: (
             });
 
         pendingProcs[lockKey].push(subproc);
+        notifyLock();
+        // Logger(`Lock:${lockKey}:queud:left${pendingProcs[lockKey].length}`, LOG_LEVEL.VERBOSE);
         return responder;
     } else {
         runningProcs.push(lockKey);
-        Logger(`Lock:${lockKey}:aqquired`, LOG_LEVEL.VERBOSE);
+        notifyLock();
+        // Logger(`Lock:${lockKey}:aqquired`, LOG_LEVEL.VERBOSE);
         return new Promise((res, rej) => {
             proc()
                 .then((v) => {
@@ -195,3 +234,16 @@ export function runWithLock<T>(key: unknown, ignoreWhenRunning: boolean, proc: (
         });
     }
 }
+
+export function isPlainText(filename: string): boolean {
+    if (filename.endsWith(".md")) return true;
+    if (filename.endsWith(".txt")) return true;
+    if (filename.endsWith(".svg")) return true;
+    if (filename.endsWith(".html")) return true;
+    if (filename.endsWith(".csv")) return true;
+    if (filename.endsWith(".css")) return true;
+    if (filename.endsWith(".js")) return true;
+    if (filename.endsWith(".xml")) return true;
+
+    return false;
+}

src/utils_couchdb.ts

@@ -8,11 +8,54 @@ export const isValidRemoteCouchDBURI = (uri: string): boolean => {
     if (uri.startsWith("http://")) return true;
     return false;
 };
+let last_post_successed = false;
+export const getLastPostFailedBySize = () => {
+    return !last_post_successed;
+};
 export const connectRemoteCouchDB = async (uri: string, auth: { username: string; password: string }): Promise<string | { db: PouchDB.Database<EntryDoc>; info: PouchDB.Core.DatabaseInfo }> => {
     if (!isValidRemoteCouchDBURI(uri)) return "Remote URI is not valid";
-    const db: PouchDB.Database<EntryDoc> = new PouchDB<EntryDoc>(uri, {
+    const conf: PouchDB.HttpAdapter.HttpAdapterConfiguration = {
+        adapter: "http",
         auth,
-    });
+        fetch: async function (url: string | Request, opts: RequestInit) {
+            let size_ok = true;
+            let size = "";
+            const localURL = url.toString().substring(uri.length);
+            const method = opts.method ?? "GET";
+            if (opts.body) {
+                const opts_length = opts.body.toString().length;
+                if (opts_length > 1024 * 1024 * 10) {
+                    // over 10MB
+                    size_ok = false;
+                    if (uri.contains(".cloudantnosqldb.")) {
+                        last_post_successed = false;
+                        Logger("This request should fail on IBM Cloudant.", LOG_LEVEL.VERBOSE);
+                        throw new Error("This request should fail on IBM Cloudant.");
+                    }
+                }
+                size = ` (${opts_length})`;
+            }
+            try {
+                const responce: Response = await fetch(url, opts);
+                if (method == "POST" || method == "PUT") {
+                    last_post_successed = responce.ok;
+                } else {
+                    last_post_successed = true;
+                }
+                Logger(`HTTP:${method}${size} to:${localURL} -> ${responce.status}`, LOG_LEVEL.VERBOSE);
+                return responce;
+            } catch (ex) {
+                Logger(`HTTP:${method}${size} to:${localURL} -> failed`, LOG_LEVEL.VERBOSE);
+                if (!size_ok && (method == "POST" || method == "PUT")) {
+                    last_post_successed = false;
+                }
+                Logger(ex);
+                throw ex;
+            }
+            // return await fetch(url, opts);
+        },
+    };
+    const db: PouchDB.Database<EntryDoc> = new PouchDB<EntryDoc>(uri, conf);
     try {
         const info = await db.info();
         return { db: db, info: info };

styles.css

@@ -34,11 +34,13 @@
 .sls-plugins-wrap {
     display: flex;
     flex-grow: 1;
-    /* overflow: scroll; */
+    max-height: 50vh;
+    overflow-y: scroll;
 }
 .sls-plugins-tbl {
     border: 1px solid var(--background-modifier-border);
     width: 100%;
+    max-height: 80%;
 }
 .divider th {
     border-top: 1px solid var(--background-modifier-border);
@@ -140,3 +142,33 @@ div.sls-setting-menu-btn {
     background-color: var(--background-secondary-alt);
     color: var(--text-accent);
 }
+.op-flex {
+    display: flex;
+}
+.op-flex input {
+    display: inline-flex;
+    flex-grow: 1;
+    margin-bottom: 8px;
+}
+
+.op-info {
+    display: inline-flex;
+    flex-grow: 1;
+    border-bottom: 1px solid var(--background-modifier-border);
+    width: 100%;
+    margin-bottom: 4px;
+    padding-bottom: 4px;
+}
+
+.history-added {
+    color: var(--text-on-accent);
+    background-color: var(--text-accent);
+}
+.history-normal {
+    color: var(--text-normal);
+}
+.history-deleted {
+    color: var(--text-on-accent);
+    background-color: var(--text-muted);
+    text-decoration: line-through;
+}

tsconfig.json

@@ -1,21 +1,17 @@
 {
     "compilerOptions": {
         "baseUrl": ".",
-        "inlineSourceMap": true,
-        "inlineSources": true,
         "module": "ESNext",
-        "target": "es6",
+        "target": "ES6",
         "allowJs": true,
         "noImplicitAny": true,
         "moduleResolution": "node",
+        "types": ["svelte", "node"],
+        // "importsNotUsedAsValues": "error",
         "importHelpers": true,
-        "noImplicitReturns": true,
-        "noImplicitThis": true,
-        "strictFunctionTypes": true,
         "alwaysStrict": true,
-        "lib": ["dom", "es5", "ES6", "ES7", "es2020"]
+        "lib": ["es2018", "DOM", "ES5", "ES6", "ES7"]
     },
-    "include": ["./src/*.ts"],
-    // "files": ["./src/main.ts"],
+    "include": ["**/*.ts"],
     "exclude": ["pouchdb-browser-webpack"]
 }