bump

- If any conflicted files have been left, they will be reported.
- Fixed:
  - Now the name of the conflicting file is shown on the conflict-resolving dialogue.
  - Hidden files are now able to be merged again.
  - No longer error caused at plug-in being loaded.
- Improved:
  - Caching chunks are now limited in total size of cached chunks.

README.md

@@ -82,7 +82,9 @@ If you have deleted or renamed files, please wait until ⏳ icon disappeared.
 -   There are no 'exclude_folders' like configurations.
 -   While synchronizing, files are compared by their modification time and the older ones will be overwritten by the newer ones. Then plugin checks for conflicts and if a merge is needed, a dialog will open.
 -   Rarely, a file in the database could be corrupted. The plugin will not write to local storage when a file looks corrupted. If a local version of the file is on your device, the corruption could be fixed by editing the local file and synchronizing it. But if the file does not exist on any of your devices, then it can not be rescued. In this case, you can delete these items from the settings dialog.
--   To stop the boot-up sequence (eg. for fixing problems on databases), you can put a `redflag.md` file at the root of your vault.
+-   To stop the boot-up sequence (eg. for fixing problems on databases), you can put a `redflag.md` file (or directory) at the root of your vault.
+    Tip for iOS: a redflag directory can be created at the root of the vault using the File application.
+-   Also, with `redflag2.md` placed, we can automatically rebuild both the local and the remote databases during the boot-up sequence. With `redflag3.md`, we can discard only the local database and fetch from the remote again.
 -   Q: The database is growing, how can I shrink it down?
     A: each of the docs is saved with their past 100 revisions for detecting and resolving conflicts. Picturing that one device has been offline for a while, and comes online again. The device has to compare its notes with the remotely saved ones. If there exists a historic revision in which the note used to be identical, it could be updated safely (like git fast-forward). Even if that is not in revision histories, we only have to check the differences after the revision that both devices commonly have. This is like git's conflict-resolving method. So, We have to make the database again like an enlarged git repo if you want to solve the root of the problem.
 -   And more technical Information is in the [Technical Information](docs/tech_info.md)

manifest.json

@@ -1,7 +1,7 @@
 {
     "id": "obsidian-livesync",
     "name": "Self-hosted LiveSync",
-    "version": "0.17.21",
+    "version": "0.17.24",
     "minAppVersion": "0.9.12",
     "description": "Community implementation of self-hosted livesync. Reflect your vault changes to some other devices immediately. Please make sure to disable other synchronize solutions to avoid content corruption or duplication.",
     "author": "vorotamoroz",

package-lock.json

@@ -1,12 +1,12 @@
 {
     "name": "obsidian-livesync",
-    "version": "0.17.21",
+    "version": "0.17.24",
     "lockfileVersion": 2,
     "requires": true,
     "packages": {
         "": {
             "name": "obsidian-livesync",
-            "version": "0.17.21",
+            "version": "0.17.24",
             "license": "MIT",
             "dependencies": {
                 "diff-match-patch": "^1.0.5",

package.json

@@ -1,6 +1,6 @@
 {
     "name": "obsidian-livesync",
-    "version": "0.17.21",
+    "version": "0.17.24",
     "description": "Reflect your vault changes to some other devices immediately. Please make sure to disable other synchronize solutions to avoid content corruption or duplication.",
     "main": "main.js",
     "type": "module",

src/ConflictResolveModal.ts

@@ -22,7 +22,7 @@ export class ConflictResolveModal extends Modal {
         contentEl.empty();
 
         contentEl.createEl("h2", { text: "This document has conflicted changes." });
-        contentEl.createEl("span", this.filename);
+        contentEl.createEl("span", { text: this.filename });
         const div = contentEl.createDiv("");
         div.addClass("op-scrollable");
         let diff = "";

src/DocumentHistoryModal.ts

@@ -1,7 +1,6 @@
 import { TFile, Modal, App } from "obsidian";
-import { path2id } from "./utils";
+import { isValidPath, path2id } from "./utils";
 import { base64ToArrayBuffer, base64ToString, escapeStringToHTML } from "./lib/src/strbin";
-import { isValidPath } from "./lib/src/path";
 import ObsidianLiveSyncPlugin from "./main";
 import { DIFF_DELETE, DIFF_EQUAL, DIFF_INSERT, diff_match_patch } from "diff-match-patch";
 import { LoadedEntry, LOG_LEVEL } from "./lib/src/types";

src/ObsidianLiveSyncSettingTab.ts

@@ -1291,7 +1291,39 @@ export class ObsidianLiveSyncSettingTab extends PluginSettingTab {
                     })
                 return toggle;
             }
-        );
+            );
+        new Setting(containerSyncSettingEl)
+            .setName("A number of hashes to be cached")
+            .setDesc("")
+            .addText((text) => {
+                text.setPlaceholder("")
+                    .setValue(this.plugin.settings.hashCacheMaxCount + "")
+                    .onChange(async (value) => {
+                        let v = Number(value);
+                        if (isNaN(v) || v < 10) {
+                            v = 10;
+                        }
+                        this.plugin.settings.hashCacheMaxCount = v;
+                        await this.plugin.saveSettings();
+                    });
+                text.inputEl.setAttribute("type", "number");
+            });
+        new Setting(containerSyncSettingEl)
+            .setName("The total length of hashes to be cached")
+            .setDesc("(Mega chars)")
+            .addText((text) => {
+                text.setPlaceholder("")
+                    .setValue(this.plugin.settings.hashCacheMaxAmount + "")
+                    .onChange(async (value) => {
+                        let v = Number(value);
+                        if (isNaN(v) || v < 1) {
+                            v = 1;
+                        }
+                        this.plugin.settings.hashCacheMaxAmount = v;
+                        await this.plugin.saveSettings();
+                    });
+                text.inputEl.setAttribute("type", "number");
+            });
 
         addScreenElement("30", containerSyncSettingEl);
         const containerMiscellaneousEl = containerEl.createDiv();
@@ -1538,6 +1570,15 @@ ${stringifyYaml(pluginConfig)}`;
                     await this.plugin.saveSettings();
                 })
             );
+        new Setting(containerHatchEl)
+            .setName("Write logs into the file")
+            .setDesc("Warning! This will have a serious impact on performance. And the logs will not be synchronised under the default name. Please be careful with logs; they often contain your confidential information.")
+            .addToggle((toggle) =>
+                toggle.setValue(this.plugin.settings.writeLogToTheFile).onChange(async (value) => {
+                    this.plugin.settings.writeLogToTheFile = value;
+                    await this.plugin.saveSettings();
+                })
+            );
 
         new Setting(containerHatchEl)
             .setName("Discard local database to reset or uninstall Self-hosted LiveSync")

src/main.ts

@@ -1,6 +1,6 @@
 import { debounce, Notice, Plugin, TFile, addIcon, TFolder, normalizePath, TAbstractFile, Editor, MarkdownView, PluginManifest, App } from "obsidian";
 import { Diff, DIFF_DELETE, DIFF_EQUAL, DIFF_INSERT, diff_match_patch } from "diff-match-patch";
-import { EntryDoc, LoadedEntry, ObsidianLiveSyncSettings, diff_check_result, diff_result_leaf, EntryBody, LOG_LEVEL, VER, DEFAULT_SETTINGS, diff_result, FLAGMD_REDFLAG, SYNCINFO_ID, InternalFileEntry, SALT_OF_PASSPHRASE, ConfigPassphraseStore, CouchDBConnection, FLAGMD_REDFLAG2 } from "./lib/src/types";
+import { EntryDoc, LoadedEntry, ObsidianLiveSyncSettings, diff_check_result, diff_result_leaf, EntryBody, LOG_LEVEL, VER, DEFAULT_SETTINGS, diff_result, FLAGMD_REDFLAG, SYNCINFO_ID, InternalFileEntry, SALT_OF_PASSPHRASE, ConfigPassphraseStore, CouchDBConnection, FLAGMD_REDFLAG2, FLAGMD_REDFLAG3, PREFIXMD_LOGFILE } from "./lib/src/types";
 import { PluginDataEntry, PERIODIC_PLUGIN_SWEEP, PluginList, DevicePluginList, InternalFileInfo, queueItem, FileInfo } from "./types";
 import { delay, getDocData, isDocContentSame } from "./lib/src/utils";
 import { Logger } from "./lib/src/logger";
@@ -9,7 +9,7 @@ import { LogDisplayModal } from "./LogDisplayModal";
 import { ConflictResolveModal } from "./ConflictResolveModal";
 import { ObsidianLiveSyncSettingTab } from "./ObsidianLiveSyncSettingTab";
 import { DocumentHistoryModal } from "./DocumentHistoryModal";
-import { applyPatch, clearAllPeriodic, clearAllTriggers, clearTrigger, disposeMemoObject, generatePatchObj, id2path, isObjectMargeApplicable, isSensibleMargeApplicable, memoIfNotExist, memoObject, flattenObject, path2id, retrieveMemoObject, setTrigger, tryParseJSON } from "./utils";
+import { applyPatch, clearAllPeriodic, clearAllTriggers, clearTrigger, disposeMemoObject, generatePatchObj, id2path, isObjectMargeApplicable, isSensibleMargeApplicable, memoIfNotExist, memoObject, flattenObject, path2id, retrieveMemoObject, setTrigger, tryParseJSON, createFile, modifyFile, isValidPath } from "./utils";
 import { decrypt, encrypt, tryDecrypt } from "./lib/src/e2ee_v2";
 
 const isDebug = false;
@@ -20,7 +20,7 @@ import { getGlobalStore, observeStores } from "./lib/src/store";
 import { lockStore, logMessageStore, logStore } from "./lib/src/stores";
 import { NewNotice, setNoticeClass, WrappedNotice } from "./lib/src/wrapper";
 import { base64ToString, versionNumberString2Number, base64ToArrayBuffer, arrayBufferToBase64 } from "./lib/src/strbin";
-import { isPlainText, isValidPath, shouldBeIgnored } from "./lib/src/path";
+import { isPlainText, shouldBeIgnored } from "./lib/src/path";
 import { runWithLock } from "./lib/src/lock";
 import { Semaphore } from "./lib/src/semaphore";
 import { JsonResolveModal } from "./JsonResolveModal";
@@ -32,6 +32,8 @@ const ICHeaderEnd = "i;";
 const ICHeaderLength = ICHeader.length;
 const FileWatchEventQueueMax = 10;
 
+const configURIBase = "obsidian://setuplivesync?settings=";
+
 function getAbstractFileByPath(path: string): TAbstractFile | null {
     // Hidden API but so useful.
     // @ts-ignore
@@ -136,6 +138,8 @@ export default class ObsidianLiveSyncPlugin extends Plugin {
     deviceAndVaultName: string;
     isMobile = false;
     isReady = false;
+    packageVersion = "";
+    manifestVersion = "";
 
     watchedFileEventQueue = [] as FileEventItem[];
 
@@ -169,6 +173,19 @@ export default class ObsidianLiveSyncPlugin extends Plugin {
             await app.vault.delete(redflag, true);
         }
     }
+    isRedFlag3Raised(): boolean {
+        const redflag = getAbstractFileByPath(normalizePath(FLAGMD_REDFLAG3));
+        if (redflag != null) {
+            return true;
+        }
+        return false;
+    }
+    async deleteRedFlag3() {
+        const redflag = getAbstractFileByPath(normalizePath(FLAGMD_REDFLAG3));
+        if (redflag != null) {
+            await app.vault.delete(redflag, true);
+        }
+    }
 
     showHistory(file: TFile | string) {
         if (!this.settings.useHistory) {
@@ -179,26 +196,10 @@ export default class ObsidianLiveSyncPlugin extends Plugin {
     }
 
     async fileHistory() {
-        const pageLimit = 1000;
-        let nextKey = "";
         const notes: { path: string, mtime: number }[] = [];
-        do {
-            const docs = await this.localDatabase.localDatabase.allDocs({ limit: pageLimit, startkey: nextKey, include_docs: true });
-            nextKey = "";
-            for (const row of docs.rows) {
-                const doc = row.doc;
-                nextKey = `${row.id}\u{10ffff}`;
-                if (!("type" in doc)) continue;
-                if (doc.type == "newnote" || doc.type == "plain") {
-                    notes.push({ path: id2path(doc._id), mtime: doc.mtime });
-                }
-                if (isChunk(nextKey)) {
-                    // skip the chunk zone.
-                    nextKey = CHeaderEnd;
-                }
-            }
-        } while (nextKey != "");
-
+        for await (const doc of this.localDatabase.findAllDocs()) {
+            notes.push({ path: id2path(doc._id), mtime: doc.mtime });
+        }
         notes.sort((a, b) => b.mtime - a.mtime);
         const notesList = notes.map(e => e.path);
         const target = await askSelectString(this.app, "File to view History", notesList);
@@ -207,31 +208,11 @@ export default class ObsidianLiveSyncPlugin extends Plugin {
         }
     }
     async pickFileForResolve() {
-        const pageLimit = 1000;
-        let nextKey = "";
         const notes: { path: string, mtime: number }[] = [];
-        do {
-            const docs = await this.localDatabase.localDatabase.allDocs({ limit: pageLimit, startkey: nextKey, conflicts: true, include_docs: true });
-            nextKey = "";
-            for (const row of docs.rows) {
-                const doc = row.doc;
-                nextKey = `${row.id}\u{10ffff}`;
-                if (isChunk(nextKey)) {
-                    // skip the chunk zone.
-                    nextKey = CHeaderEnd;
-                }
-                if (!("_conflicts" in doc)) continue;
-                if (isInternalMetadata(row.id)) continue;
-                // We have to check also deleted files.
-                // if (doc._deleted) continue;
-                // if ("deleted" in doc && doc.deleted) continue;
-                if (doc.type == "newnote" || doc.type == "plain") {
-                    // const docId = doc._id.startsWith("i:") ? doc._id.substring("i:".length) : doc._id;
-                    notes.push({ path: id2path(doc._id), mtime: doc.mtime });
-                }
-
-            }
-        } while (nextKey != "");
+        for await (const doc of this.localDatabase.findAllDocs({ conflicts: true })) {
+            if (!("_conflicts" in doc)) continue;
+            notes.push({ path: id2path(doc._id), mtime: doc.mtime });
+        }
         notes.sort((a, b) => b.mtime - a.mtime);
         const notesList = notes.map(e => e.path);
         if (notesList.length == 0) {
@@ -242,6 +223,7 @@ export default class ObsidianLiveSyncPlugin extends Plugin {
         if (target) {
             if (isInternalMetadata(target)) {
                 //NOP
+                await this.resolveConflictOnInternalFile(target);
             } else {
                 await this.showIfConflicted(target);
             }
@@ -288,7 +270,252 @@ export default class ObsidianLiveSyncPlugin extends Plugin {
         }
         Logger(`Checking expired file history done`);
     }
+    async onLayoutReady() {
+        this.registerFileWatchEvents();
+        if (this.localDatabase.isReady)
+            try {
+                if (this.isRedFlagRaised() || this.isRedFlag2Raised() || this.isRedFlag3Raised()) {
+                    this.settings.batchSave = false;
+                    this.settings.liveSync = false;
+                    this.settings.periodicReplication = false;
+                    this.settings.syncOnSave = false;
+                    this.settings.syncOnStart = false;
+                    this.settings.syncOnFileOpen = false;
+                    this.settings.syncAfterMerge = false;
+                    this.settings.autoSweepPlugins = false;
+                    this.settings.usePluginSync = false;
+                    this.settings.suspendFileWatching = true;
+                    this.settings.syncInternalFiles = false;
+                    await this.saveSettings();
+                    if (this.isRedFlag2Raised()) {
+                        Logger(`${FLAGMD_REDFLAG2} has been detected! Self-hosted LiveSync suspends all sync and rebuild everything.`, LOG_LEVEL.NOTICE);
+                        await this.resetLocalDatabase();
+                        await this.initializeDatabase(true);
+                        await this.markRemoteLocked();
+                        await this.tryResetRemoteDatabase();
+                        await this.markRemoteLocked();
+                        await this.replicateAllToServer(true);
+                        await this.deleteRedFlag2();
+                        if (await askYesNo(this.app, "Do you want to disable Suspend file watching and restart obsidian now?") == "yes") {
+                            this.settings.suspendFileWatching = false;
+                            await this.saveSettings();
+                            // @ts-ignore
+                            this.app.commands.executeCommandById("app:reload")
+                        }
+                    } else if (this.isRedFlag3Raised()) {
+                        Logger(`${FLAGMD_REDFLAG3} has been detected! Self-hosted LiveSync will discard the local database and fetch everything from the remote once again.`, LOG_LEVEL.NOTICE);
+                        await this.resetLocalDatabase();
+                        await this.markRemoteResolved();
+                        await this.openDatabase();
+                        this.isReady = true;
+                        await this.replicate(true);
+                        await this.deleteRedFlag3();
+                        if (await askYesNo(this.app, "Do you want to disable Suspend file watching and restart obsidian now?") == "yes") {
+                            this.settings.suspendFileWatching = false;
+                            await this.saveSettings();
+                            // @ts-ignore
+                            this.app.commands.executeCommandById("app:reload")
+                        }
+                    } else {
+                        this.settings.writeLogToTheFile = true;
+                        await this.openDatabase();
+                        const warningMessage = "The red flag is raised! The whole initialize steps are skipped, and any file changes are not captured.";
+                        Logger(warningMessage, LOG_LEVEL.NOTICE);
+                        this.setStatusBarText(warningMessage);
+                    }
+                } else {
+                    if (this.settings.suspendFileWatching) {
+                        Logger("'Suspend file watching' turned on. Are you sure this is what you intended? Every modification on the vault will be ignored.", LOG_LEVEL.NOTICE);
+                    }
+                    const isInitialized = await this.initializeDatabase();
+                    if (!isInitialized) {
+                        //TODO:stop all sync.
+                        return false;
+                    }
+                }
+                await this.realizeSettingSyncMode();
+                this.registerWatchEvents();
+                if (this.settings.syncOnStart) {
+                    this.localDatabase.openReplication(this.settings, false, false, this.parseReplicationResult);
+                }
+                this.scanStat();
+            } catch (ex) {
+                Logger("Error while loading Self-hosted LiveSync", LOG_LEVEL.NOTICE);
+                Logger(ex, LOG_LEVEL.VERBOSE);
+            }
+    }
 
+    /**
+     * Scan status 
+     */
+    async scanStat() {
+        const notes: { path: string, mtime: number }[] = [];
+        Logger(`Additional safety scan..`, LOG_LEVEL.VERBOSE);
+        for await (const doc of this.localDatabase.findAllDocs({ conflicts: true })) {
+            if (!("_conflicts" in doc)) continue;
+            notes.push({ path: id2path(doc._id), mtime: doc.mtime });
+        }
+        if (notes.length > 0) {
+            Logger(`Some files have been left conflicted! Please resolve them by "Pick a file to resolve conflict". The list is written in the log.`, LOG_LEVEL.NOTICE);
+            for (const note of notes) {
+                Logger(`Conflicted: ${note.path}`);
+            }
+        } else {
+            Logger(`There are no conflicted files`, LOG_LEVEL.VERBOSE);
+        }
+        Logger(`Additional safety scan done`, LOG_LEVEL.VERBOSE);
+    }
+    async command_copySetupURI() {
+        const encryptingPassphrase = await askString(this.app, "Encrypt your settings", "The passphrase to encrypt the setup URI", "");
+        if (encryptingPassphrase === false) return;
+        const setting = { ...this.settings, configPassphraseStore: "", encryptedCouchDBConnection: "", encryptedPassphrase: "" };
+        const keys = Object.keys(setting) as (keyof ObsidianLiveSyncSettings)[];
+        for (const k of keys) {
+            if (JSON.stringify(k in setting ? setting[k] : "") == JSON.stringify(k in DEFAULT_SETTINGS ? DEFAULT_SETTINGS[k] : "*")) {
+                delete setting[k];
+            }
+        }
+        const encryptedSetting = encodeURIComponent(await encrypt(JSON.stringify(setting), encryptingPassphrase, false));
+        const uri = `${configURIBase}${encryptedSetting}`;
+        await navigator.clipboard.writeText(uri);
+        Logger("Setup URI copied to clipboard", LOG_LEVEL.NOTICE);
+    }
+    async command_copySetupURIFull() {
+        const encryptingPassphrase = await askString(this.app, "Encrypt your settings", "The passphrase to encrypt the setup URI", "");
+        if (encryptingPassphrase === false) return;
+        const setting = { ...this.settings, configPassphraseStore: "", encryptedCouchDBConnection: "", encryptedPassphrase: "" };
+        const encryptedSetting = encodeURIComponent(await encrypt(JSON.stringify(setting), encryptingPassphrase, false));
+        const uri = `${configURIBase}${encryptedSetting}`;
+        await navigator.clipboard.writeText(uri);
+        Logger("Setup URI copied to clipboard", LOG_LEVEL.NOTICE);
+    }
+    async command_openSetupURI() {
+        const setupURI = await askString(this.app, "Easy setup", "Set up URI", `${configURIBase}aaaaa`);
+        if (setupURI === false) return;
+        if (!setupURI.startsWith(`${configURIBase}`)) {
+            Logger("Set up URI looks wrong.", LOG_LEVEL.NOTICE);
+            return;
+        }
+        const config = decodeURIComponent(setupURI.substring(configURIBase.length));
+        console.dir(config)
+        await this.setupWizard(config);
+    }
+    async setupWizard(confString: string) {
+        try {
+            const oldConf = JSON.parse(JSON.stringify(this.settings));
+            const encryptingPassphrase = await askString(this.app, "Passphrase", "The passphrase to decrypt your setup URI", "");
+            if (encryptingPassphrase === false) return;
+            const newConf = await JSON.parse(await decrypt(confString, encryptingPassphrase, false));
+            if (newConf) {
+                const result = await askYesNo(this.app, "Importing LiveSync's conf, OK?");
+                if (result == "yes") {
+                    const newSettingW = Object.assign({}, DEFAULT_SETTINGS, newConf) as ObsidianLiveSyncSettings;
+                    this.localDatabase.closeReplication();
+                    this.settings.suspendFileWatching = true;
+                    console.dir(newSettingW);
+                    // Back into the default method once.
+                    newSettingW.configPassphraseStore = "";
+                    newSettingW.encryptedPassphrase = "";
+                    newSettingW.encryptedCouchDBConnection = "";
+                    const setupJustImport = "Just import setting";
+                    const setupAsNew = "Set it up as secondary or subsequent device";
+                    const setupAgain = "Reconfigure and reconstitute the data";
+                    const setupManually = "Leave everything to me";
+
+                    const setupType = await askSelectString(this.app, "How would you like to set it up?", [setupAsNew, setupAgain, setupJustImport, setupManually]);
+                    if (setupType == setupJustImport) {
+                        this.settings = newSettingW;
+                        this.usedPassphrase = "";
+                        await this.saveSettings();
+                    } else if (setupType == setupAsNew) {
+                        this.settings = newSettingW;
+                        this.usedPassphrase = "";
+                        await this.saveSettings();
+                        await this.resetLocalOldDatabase();
+                        await this.resetLocalDatabase();
+                        await this.localDatabase.initializeDatabase();
+                        await this.markRemoteResolved();
+                        await this.replicate(true);
+                    } else if (setupType == setupAgain) {
+                        const confirm = "I know this operation will rebuild all my databases with files on this device, and files that are on the remote database and I didn't synchronize to any other devices will be lost and want to proceed indeed.";
+                        if (await askSelectString(this.app, "Do you really want to do this?", ["Cancel", confirm]) != confirm) {
+                            return;
+                        }
+                        this.settings = newSettingW;
+                        this.usedPassphrase = "";
+                        await this.saveSettings();
+                        await this.resetLocalOldDatabase();
+                        await this.resetLocalDatabase();
+                        await this.localDatabase.initializeDatabase();
+                        await this.initializeDatabase(true);
+                        await this.tryResetRemoteDatabase();
+                        await this.markRemoteLocked();
+                        await this.markRemoteResolved();
+                        await this.replicate(true);
+
+                    } else if (setupType == setupManually) {
+                        const keepLocalDB = await askYesNo(this.app, "Keep local DB?");
+                        const keepRemoteDB = await askYesNo(this.app, "Keep remote DB?");
+                        if (keepLocalDB == "yes" && keepRemoteDB == "yes") {
+                            // nothing to do. so peaceful.
+                            this.settings = newSettingW;
+                            this.usedPassphrase = "";
+                            await this.saveSettings();
+                            const replicate = await askYesNo(this.app, "Unlock and replicate?");
+                            if (replicate == "yes") {
+                                await this.replicate(true);
+                                await this.markRemoteUnlocked();
+                            }
+                            Logger("Configuration loaded.", LOG_LEVEL.NOTICE);
+                            return;
+                        }
+                        if (keepLocalDB == "no" && keepRemoteDB == "no") {
+                            const reset = await askYesNo(this.app, "Drop everything?");
+                            if (reset != "yes") {
+                                Logger("Cancelled", LOG_LEVEL.NOTICE);
+                                this.settings = oldConf;
+                                return;
+                            }
+                        }
+                        let initDB;
+                        this.settings = newSettingW;
+                        this.usedPassphrase = "";
+                        await this.saveSettings();
+                        if (keepLocalDB == "no") {
+                            this.resetLocalOldDatabase();
+                            this.resetLocalDatabase();
+                            this.localDatabase.initializeDatabase();
+                            const rebuild = await askYesNo(this.app, "Rebuild the database?");
+                            if (rebuild == "yes") {
+                                initDB = this.initializeDatabase(true);
+                            } else {
+                                this.markRemoteResolved();
+                            }
+                        }
+                        if (keepRemoteDB == "no") {
+                            await this.tryResetRemoteDatabase();
+                            await this.markRemoteLocked();
+                        }
+                        if (keepLocalDB == "no" || keepRemoteDB == "no") {
+                            const replicate = await askYesNo(this.app, "Replicate once?");
+                            if (replicate == "yes") {
+                                if (initDB != null) {
+                                    await initDB;
+                                }
+                                await this.replicate(true);
+                            }
+                        }
+                    }
+                }
+
+                Logger("Configuration loaded.", LOG_LEVEL.NOTICE);
+            } else {
+                Logger("Cancelled.", LOG_LEVEL.NOTICE);
+            }
+        } catch (ex) {
+            Logger("Couldn't parse or decrypt configuration uri.", LOG_LEVEL.NOTICE);
+        }
+    }
     async onload() {
         logStore.subscribe(e => this.addLog(e.message, e.level, e.key));
         Logger("loading plugin");
@@ -297,6 +524,8 @@ export default class ObsidianLiveSyncPlugin extends Plugin {
         //@ts-ignore
         const packageVersion: string = PACKAGE_VERSION || "0.0.0";
 
+        this.manifestVersion = manifestVersion;
+        this.packageVersion = packageVersion;
 
         Logger(`Self-hosted LiveSync v${manifestVersion} ${packageVersion} `);
         const lsKey = "obsidian-live-sync-ver" + this.getVaultName();
@@ -324,6 +553,34 @@ export default class ObsidianLiveSyncPlugin extends Plugin {
         localStorage.setItem(lsKey, `${VER}`);
         await this.openDatabase();
 
+        this.watchVaultChange = this.watchVaultChange.bind(this);
+        this.watchVaultCreate = this.watchVaultCreate.bind(this);
+        this.watchVaultDelete = this.watchVaultDelete.bind(this);
+        this.watchVaultRename = this.watchVaultRename.bind(this);
+        this.watchVaultRawEvents = this.watchVaultRawEvents.bind(this);
+        this.watchWorkspaceOpen = debounce(this.watchWorkspaceOpen.bind(this), 1000, false);
+        this.watchWindowVisibility = debounce(this.watchWindowVisibility.bind(this), 1000, false);
+        this.watchOnline = debounce(this.watchOnline.bind(this), 500, false);
+
+        this.parseReplicationResult = this.parseReplicationResult.bind(this);
+
+        this.setPeriodicSync = this.setPeriodicSync.bind(this);
+        this.clearPeriodicSync = this.clearPeriodicSync.bind(this);
+        this.periodicSync = this.periodicSync.bind(this);
+        this.loadQueuedFiles = this.loadQueuedFiles.bind(this);
+
+        this.getPluginList = this.getPluginList.bind(this);
+
+        this.triggerRealizeSettingSyncMode = debounce(this.triggerRealizeSettingSyncMode.bind(this), 1000);
+        this.triggerCheckPluginUpdate = debounce(this.triggerCheckPluginUpdate.bind(this), 3000);
+        this.setupWizard = this.setupWizard.bind(this);
+
+        this.statusBar = this.addStatusBarItem();
+        this.statusBar.addClass("syncstatusbar");
+
+        this.statusBar2 = this.addStatusBarItem();
+
+
         addIcon(
             "replicate",
             `<g transform="matrix(1.15 0 0 1.15 -8.31 -9.52)" fill="currentColor" fill-rule="evenodd">
@@ -348,248 +605,27 @@ export default class ObsidianLiveSyncPlugin extends Plugin {
             new LogDisplayModal(this.app, this).open();
         });
 
-        this.statusBar = this.addStatusBarItem();
-        this.statusBar.addClass("syncstatusbar");
-
-        this.statusBar2 = this.addStatusBarItem();
-        this.watchVaultChange = this.watchVaultChange.bind(this);
-        this.watchVaultCreate = this.watchVaultCreate.bind(this);
-        this.watchVaultDelete = this.watchVaultDelete.bind(this);
-        this.watchVaultRename = this.watchVaultRename.bind(this);
-        this.watchVaultRawEvents = this.watchVaultRawEvents.bind(this);
-        this.watchWorkspaceOpen = debounce(this.watchWorkspaceOpen.bind(this), 1000, false);
-        this.watchWindowVisibility = debounce(this.watchWindowVisibility.bind(this), 1000, false);
-        this.watchOnline = debounce(this.watchOnline.bind(this), 500, false);
-
-        this.parseReplicationResult = this.parseReplicationResult.bind(this);
-
-        this.setPeriodicSync = this.setPeriodicSync.bind(this);
-        this.clearPeriodicSync = this.clearPeriodicSync.bind(this);
-        this.periodicSync = this.periodicSync.bind(this);
-        this.loadQueuedFiles = this.loadQueuedFiles.bind(this);
-
-        this.getPluginList = this.getPluginList.bind(this);
-        // this.registerWatchEvents();
         this.addSettingTab(new ObsidianLiveSyncSettingTab(this.app, this));
 
-        this.app.workspace.onLayoutReady(async () => {
-            this.registerFileWatchEvents();
-            if (this.localDatabase.isReady)
-                try {
-                    if (this.isRedFlagRaised() || this.isRedFlag2Raised()) {
-                        this.settings.batchSave = false;
-                        this.settings.liveSync = false;
-                        this.settings.periodicReplication = false;
-                        this.settings.syncOnSave = false;
-                        this.settings.syncOnStart = false;
-                        this.settings.syncOnFileOpen = false;
-                        this.settings.syncAfterMerge = false;
-                        this.settings.autoSweepPlugins = false;
-                        this.settings.usePluginSync = false;
-                        this.settings.suspendFileWatching = true;
-                        this.settings.syncInternalFiles = false;
-                        await this.saveSettings();
-                        if (this.isRedFlag2Raised()) {
-                            Logger(`${FLAGMD_REDFLAG2} has been detected! Self-hosted LiveSync suspends all sync and rebuild everything.`, LOG_LEVEL.NOTICE);
-                            await this.resetLocalDatabase();
-                            await this.initializeDatabase(true);
-                            await this.markRemoteLocked();
-                            await this.tryResetRemoteDatabase();
-                            await this.markRemoteLocked();
-                            await this.replicateAllToServer(true);
-                            await this.deleteRedFlag2();
-                        } else {
-                            await this.openDatabase();
-                            const warningMessage = "The red flag is raised! The whole initialize steps are skipped, and any file changes are not captured.";
-                            Logger(warningMessage, LOG_LEVEL.NOTICE);
-                            this.setStatusBarText(warningMessage);
-                        }
-                    } else {
-                        if (this.settings.suspendFileWatching) {
-                            Logger("'Suspend file watching' turned on. Are you sure this is what you intended? Every modification on the vault will be ignored.", LOG_LEVEL.NOTICE);
-                        }
-                        const isInitialized = await this.initializeDatabase();
-                        if (!isInitialized) {
-                            //TODO:stop all sync.
-                            return false;
-                        }
-                    }
-                    await this.realizeSettingSyncMode();
-                    this.registerWatchEvents();
-                    if (this.settings.syncOnStart) {
-                        this.localDatabase.openReplication(this.settings, false, false, this.parseReplicationResult);
-                    }
-                } catch (ex) {
-                    Logger("Error while loading Self-hosted LiveSync", LOG_LEVEL.NOTICE);
-                    Logger(ex, LOG_LEVEL.VERBOSE);
-                }
-        });
-        const configURIBase = "obsidian://setuplivesync?settings=";
+        this.app.workspace.onLayoutReady(this.onLayoutReady.bind(this));
+        this.registerObsidianProtocolHandler("setuplivesync", async (conf: any) => await this.setupWizard(conf.settings));
+
         this.addCommand({
             id: "livesync-copysetupuri",
             name: "Copy the setup URI",
-            callback: async () => {
-                const encryptingPassphrase = await askString(this.app, "Encrypt your settings", "The passphrase to encrypt the setup URI", "");
-                if (encryptingPassphrase === false) return;
-                const setting = { ...this.settings, configPassphraseStore: "", encryptedCouchDBConnection: "", encryptedPassphrase: "" };
-                const keys = Object.keys(setting) as (keyof ObsidianLiveSyncSettings)[];
-                for (const k of keys) {
-                    if (JSON.stringify(k in setting ? setting[k] : "") == JSON.stringify(k in DEFAULT_SETTINGS ? DEFAULT_SETTINGS[k] : "*")) {
-                        delete setting[k];
-                    }
-                }
-                const encryptedSetting = encodeURIComponent(await encrypt(JSON.stringify(setting), encryptingPassphrase, false));
-                const uri = `${configURIBase}${encryptedSetting}`;
-                await navigator.clipboard.writeText(uri);
-                Logger("Setup URI copied to clipboard", LOG_LEVEL.NOTICE);
-            },
+            callback: this.command_copySetupURI.bind(this),
         });
+
         this.addCommand({
             id: "livesync-copysetupurifull",
             name: "Copy the setup URI (Full)",
-            callback: async () => {
-                const encryptingPassphrase = await askString(this.app, "Encrypt your settings", "The passphrase to encrypt the setup URI", "");
-                if (encryptingPassphrase === false) return;
-                const setting = { ...this.settings, configPassphraseStore: "", encryptedCouchDBConnection: "", encryptedPassphrase: "" };
-                const encryptedSetting = encodeURIComponent(await encrypt(JSON.stringify(setting), encryptingPassphrase, false));
-                const uri = `${configURIBase}${encryptedSetting}`;
-                await navigator.clipboard.writeText(uri);
-                Logger("Setup URI copied to clipboard", LOG_LEVEL.NOTICE);
-            },
+            callback: this.command_copySetupURIFull.bind(this),
         });
+
         this.addCommand({
             id: "livesync-opensetupuri",
             name: "Open the setup URI",
-            callback: async () => {
-                const setupURI = await askString(this.app, "Easy setup", "Set up URI", `${configURIBase}aaaaa`);
-                if (setupURI === false) return;
-                if (!setupURI.startsWith(`${configURIBase}`)) {
-                    Logger("Set up URI looks wrong.", LOG_LEVEL.NOTICE);
-                    return;
-                }
-                const config = decodeURIComponent(setupURI.substring(configURIBase.length));
-                console.dir(config)
-                await setupWizard(config);
-            },
-        });
-        const setupWizard = async (confString: string) => {
-            try {
-                const oldConf = JSON.parse(JSON.stringify(this.settings));
-                const encryptingPassphrase = await askString(this.app, "Passphrase", "The passphrase to decrypt your setup URI", "");
-                if (encryptingPassphrase === false) return;
-                const newConf = await JSON.parse(await decrypt(confString, encryptingPassphrase, false));
-                if (newConf) {
-                    const result = await askYesNo(this.app, "Importing LiveSync's conf, OK?");
-                    if (result == "yes") {
-                        const newSettingW = Object.assign({}, DEFAULT_SETTINGS, newConf) as ObsidianLiveSyncSettings;
-                        this.localDatabase.closeReplication();
-                        this.settings.suspendFileWatching = true;
-                        console.dir(newSettingW);
-                        // Back into the default method once.
-                        newSettingW.configPassphraseStore = "";
-                        newSettingW.encryptedPassphrase = "";
-                        newSettingW.encryptedCouchDBConnection = "";
-                        const setupJustImport = "Just import setting";
-                        const setupAsNew = "Set it up as secondary or subsequent device";
-                        const setupAgain = "Reconfigure and reconstitute the data";
-                        const setupManually = "Leave everything to me";
-
-                        const setupType = await askSelectString(this.app, "How would you like to set it up?", [setupAsNew, setupAgain, setupJustImport, setupManually]);
-                        if (setupType == setupJustImport) {
-                            this.settings = newSettingW;
-                            this.usedPassphrase = "";
-                            await this.saveSettings();
-                        } else if (setupType == setupAsNew) {
-                            this.settings = newSettingW;
-                            this.usedPassphrase = "";
-                            await this.saveSettings();
-                            await this.resetLocalOldDatabase();
-                            await this.resetLocalDatabase();
-                            await this.localDatabase.initializeDatabase();
-                            await this.markRemoteResolved();
-                            await this.replicate(true);
-                        } else if (setupType == setupAgain) {
-                            const confirm = "I know this operation will rebuild all my databases with files on this device, and files that are on the remote database and I didn't synchronize to any other devices will be lost and want to proceed indeed.";
-                            if (await askSelectString(this.app, "Do you really want to do this?", ["Cancel", confirm]) != confirm) {
-                                return;
-                            }
-                            this.settings = newSettingW;
-                            this.usedPassphrase = "";
-                            await this.saveSettings();
-                            await this.resetLocalOldDatabase();
-                            await this.resetLocalDatabase();
-                            await this.localDatabase.initializeDatabase();
-                            await this.initializeDatabase(true);
-                            await this.tryResetRemoteDatabase();
-                            await this.markRemoteLocked();
-                            await this.markRemoteResolved();
-                            await this.replicate(true);
-
-                        } else if (setupType == setupManually) {
-                            const keepLocalDB = await askYesNo(this.app, "Keep local DB?");
-                            const keepRemoteDB = await askYesNo(this.app, "Keep remote DB?");
-                            if (keepLocalDB == "yes" && keepRemoteDB == "yes") {
-                                // nothing to do. so peaceful.
-                                this.settings = newSettingW;
-                                this.usedPassphrase = "";
-                                await this.saveSettings();
-                                const replicate = await askYesNo(this.app, "Unlock and replicate?");
-                                if (replicate == "yes") {
-                                    await this.replicate(true);
-                                    await this.markRemoteUnlocked();
-                                }
-                                Logger("Configuration loaded.", LOG_LEVEL.NOTICE);
-                                return;
-                            }
-                            if (keepLocalDB == "no" && keepRemoteDB == "no") {
-                                const reset = await askYesNo(this.app, "Drop everything?");
-                                if (reset != "yes") {
-                                    Logger("Cancelled", LOG_LEVEL.NOTICE);
-                                    this.settings = oldConf;
-                                    return;
-                                }
-                            }
-                            let initDB;
-                            this.settings = newSettingW;
-                            this.usedPassphrase = "";
-                            await this.saveSettings();
-                            if (keepLocalDB == "no") {
-                                this.resetLocalOldDatabase();
-                                this.resetLocalDatabase();
-                                this.localDatabase.initializeDatabase();
-                                const rebuild = await askYesNo(this.app, "Rebuild the database?");
-                                if (rebuild == "yes") {
-                                    initDB = this.initializeDatabase(true);
-                                } else {
-                                    this.markRemoteResolved();
-                                }
-                            }
-                            if (keepRemoteDB == "no") {
-                                await this.tryResetRemoteDatabase();
-                                await this.markRemoteLocked();
-                            }
-                            if (keepLocalDB == "no" || keepRemoteDB == "no") {
-                                const replicate = await askYesNo(this.app, "Replicate once?");
-                                if (replicate == "yes") {
-                                    if (initDB != null) {
-                                        await initDB;
-                                    }
-                                    await this.replicate(true);
-                                }
-                            }
-                        }
-                    }
-
-                    Logger("Configuration loaded.", LOG_LEVEL.NOTICE);
-                } else {
-                    Logger("Cancelled.", LOG_LEVEL.NOTICE);
-                }
-            } catch (ex) {
-                Logger("Couldn't parse or decrypt configuration uri.", LOG_LEVEL.NOTICE);
-            }
-        };
-        this.registerObsidianProtocolHandler("setuplivesync", async (conf: any) => {
-            await setupWizard(conf.settings);
+            callback: this.command_openSetupURI.bind(this),
         });
         this.addCommand({
             id: "livesync-replicate",
@@ -962,6 +998,9 @@ export default class ObsidianLiveSyncPlugin extends Plugin {
     async appendWatchEvent(params: { type: FileEventType, file: TAbstractFile | InternalFileInfo, oldPath?: string }[], ctx?: any) {
         let forcePerform = false;
         for (const param of params) {
+            if (shouldBeIgnored(param.file.path)) {
+                continue;
+            }
             const atomicKey = [0, 0, 0, 0, 0, 0].map(e => `${Math.floor(Math.random() * 100000)}`).join("-");
             const type = param.type;
             const file = param.file;
@@ -1257,11 +1296,24 @@ export default class ObsidianLiveSyncPlugin extends Plugin {
             return;
         }
         const vaultName = this.getVaultName();
-        const timestamp = new Date().toLocaleString();
+        const now = new Date();
+        const timestamp = now.toLocaleString();
         const messageContent = typeof message == "string" ? message : message instanceof Error ? `${message.name}:${message.message}` : JSON.stringify(message, null, 2);
+        if (message instanceof Error) {
+            // debugger;
+        }
         const newMessage = timestamp + "->" + messageContent;
 
         console.log(vaultName + ":" + newMessage);
+        if (this.settings?.writeLogToTheFile) {
+            const time = now.toISOString().split("T")[0];
+            const logDate = `${PREFIXMD_LOGFILE}${time}.md`;
+            const file = this.app.vault.getAbstractFileByPath(normalizePath(logDate));
+            if (!file) {
+                this.app.vault.adapter.append(normalizePath(logDate), "```\n");
+            }
+            this.app.vault.adapter.append(normalizePath(logDate), vaultName + ":" + newMessage + "\n");
+        }
         logMessageStore.apply(e => [...e, newMessage].slice(-100));
         this.setStatusBarText(null, messageContent.substring(0, 30));
 
@@ -1325,58 +1377,61 @@ export default class ObsidianLiveSyncPlugin extends Plugin {
         }
     }
 
-    async doc2storage_create(docEntry: EntryBody, force?: boolean) {
+    async doc2storage(docEntry: EntryBody, file?: TFile, force?: boolean) {
+        const mode = file == undefined ? "create" : "modify";
+
         const pathSrc = id2path(docEntry._id);
         if (shouldBeIgnored(pathSrc)) {
             return;
         }
         if (!this.isTargetFile(pathSrc)) return;
+        if (docEntry._deleted || docEntry.deleted) {
+            // This occurs not only when files are deleted, but also when conflicts are resolved.
+            // We have to check no other revisions are left.
+            const lastDocs = await this.localDatabase.getDBEntry(pathSrc);
+            if (lastDocs === false) {
+                await this.deleteVaultItem(file);
+            } else {
+                // it perhaps delete some revisions.
+                // may be we have to reload this
+                await this.pullFile(pathSrc, null, true);
+                Logger(`delete skipped:${lastDocs._id}`, LOG_LEVEL.VERBOSE);
+            }
+            return;
+        }
+        const localMtime = ~~(file?.stat?.mtime || 0 / 1000);
+        const docMtime = ~~(docEntry.mtime / 1000);
 
         const doc = await this.localDatabase.getDBEntry(pathSrc, { rev: docEntry._rev });
         if (doc === false) return;
-        const msg = `DB -> STORAGE (create${force ? ",force" : ""},${doc.datatype}) `;
         const path = id2path(doc._id);
-        if (doc.datatype == "newnote") {
-            const bin = base64ToArrayBuffer(doc.data);
-            if (bin != null) {
-                if (!isValidPath(path)) {
-                    Logger(msg + "ERROR, invalid path: " + path, LOG_LEVEL.NOTICE);
-                    return;
-                }
-                await this.ensureDirectory(path);
-                try {
-                    const newFile = await this.app.vault.createBinary(normalizePath(path), bin, {
-                        ctime: doc.ctime,
-                        mtime: doc.mtime,
-                    });
-                    Logger(msg + path);
-                    touch(newFile);
-                    this.app.vault.trigger("create", newFile);
-                } catch (ex) {
-                    Logger(msg + "ERROR, Could not write: " + path, LOG_LEVEL.NOTICE);
-                    Logger(ex, LOG_LEVEL.VERBOSE);
-                }
+        const msg = `DB -> STORAGE (${mode}${force ? ",force" : ""},${doc.datatype}) `;
+        if (doc.datatype != "newnote" && doc.datatype != "plain") {
+            Logger(msg + "ERROR, Invalid datatype: " + path + "(" + doc.datatype + ")", LOG_LEVEL.NOTICE);
+            return;
+        }
+        if (!force && localMtime >= docMtime) return;
+        if (!isValidPath(path)) {
+            Logger(msg + "ERROR, invalid path: " + path, LOG_LEVEL.NOTICE);
+            return;
+        }
+        const writeData = doc.datatype == "newnote" ? base64ToArrayBuffer(doc.data) : getDocData(doc.data);
+        await this.ensureDirectoryEx(path);
+        try {
+            let outFile;
+            if (mode == "create") {
+                outFile = await createFile(normalizePath(path), writeData, { ctime: doc.ctime, mtime: doc.mtime, });
+            } else {
+                await modifyFile(file, writeData, { ctime: doc.ctime, mtime: doc.mtime });
+                outFile = getAbstractFileByPath(file.path) as TFile;
             }
-        } else if (doc.datatype == "plain") {
-            if (!isValidPath(path)) {
-                Logger(msg + "ERROR, invalid path: " + path, LOG_LEVEL.NOTICE);
-                return;
-            }
-            await this.ensureDirectory(path);
-            try {
-                const newFile = await this.app.vault.create(normalizePath(path), getDocData(doc.data), {
-                    ctime: doc.ctime,
-                    mtime: doc.mtime,
-                });
-                Logger(msg + path);
-                touch(newFile);
-                this.app.vault.trigger("create", newFile);
-            } catch (ex) {
-                Logger(msg + "ERROR, Could not create: " + path + "(" + doc.datatype + ")", LOG_LEVEL.NOTICE);
-                Logger(ex, LOG_LEVEL.VERBOSE);
-            }
-        } else {
-            Logger(msg + "ERROR, Could not parse: " + path + "(" + doc.datatype + ")", LOG_LEVEL.NOTICE);
+            Logger(msg + path);
+            touch(outFile);
+            this.app.vault.trigger(mode, outFile);
+
+        } catch (ex) {
+            Logger(msg + "ERROR, Could not write: " + path, LOG_LEVEL.NOTICE);
+            Logger(ex, LOG_LEVEL.VERBOSE);
         }
     }
 
@@ -1400,79 +1455,6 @@ export default class ObsidianLiveSyncPlugin extends Plugin {
         }
     }
 
-    async doc2storage_modify(docEntry: EntryBody, file: TFile, force?: boolean) {
-        const pathSrc = id2path(docEntry._id);
-        if (shouldBeIgnored(pathSrc)) {
-            return;
-        }
-        if (!this.isTargetFile(pathSrc)) return;
-        if (docEntry._deleted || docEntry.deleted) {
-            // This occurs not only when files are deleted, but also when conflicts are resolved.
-            // We have to check no other revisions are left.
-            const lastDocs = await this.localDatabase.getDBEntry(pathSrc);
-            if (lastDocs === false) {
-                await this.deleteVaultItem(file);
-            } else {
-                // it perhaps delete some revisions.
-                // may be we have to reload this
-                await this.pullFile(pathSrc, null, true);
-                Logger(`delete skipped:${lastDocs._id}`, LOG_LEVEL.VERBOSE);
-            }
-            return;
-        }
-        const localMtime = ~~(file.stat.mtime / 1000);
-        const docMtime = ~~(docEntry.mtime / 1000);
-        if (localMtime < docMtime || force) {
-            const doc = await this.localDatabase.getDBEntry(pathSrc);
-            if (doc === false) return;
-            const msg = `DB -> STORAGE (modify${force ? ",force" : ""},${doc.datatype}) `;
-            const path = id2path(doc._id);
-            if (doc.datatype == "newnote") {
-                const bin = base64ToArrayBuffer(doc.data);
-                if (bin != null) {
-                    if (!isValidPath(path)) {
-                        Logger(msg + "ERROR, invalid path: " + path, LOG_LEVEL.NOTICE);
-                        return;
-                    }
-                    await this.ensureDirectory(path);
-                    try {
-                        await this.app.vault.modifyBinary(file, bin, { ctime: doc.ctime, mtime: doc.mtime });
-                        // this.batchFileChange = this.batchFileChange.filter((e) => e != file.path);
-                        Logger(msg + path);
-                        const xf = getAbstractFileByPath(file.path) as TFile;
-                        touch(xf);
-                        this.app.vault.trigger("modify", xf);
-                    } catch (ex) {
-                        Logger(msg + "ERROR, Could not write: " + path, LOG_LEVEL.NOTICE);
-                    }
-                }
-            } else if (doc.datatype == "plain") {
-                if (!isValidPath(path)) {
-                    Logger(msg + "ERROR, invalid path: " + path, LOG_LEVEL.NOTICE);
-                    return;
-                }
-                await this.ensureDirectory(path);
-                try {
-                    await this.app.vault.modify(file, getDocData(doc.data), { ctime: doc.ctime, mtime: doc.mtime });
-                    Logger(msg + path);
-                    // this.batchFileChange = this.batchFileChange.filter((e) => e != file.path);
-                    const xf = getAbstractFileByPath(file.path) as TFile;
-                    touch(xf);
-                    this.app.vault.trigger("modify", xf);
-                } catch (ex) {
-                    Logger(msg + "ERROR, Could not write: " + path, LOG_LEVEL.NOTICE);
-                }
-            } else {
-                Logger(msg + "ERROR, Could not parse: " + path + "(" + doc.datatype + ")", LOG_LEVEL.NOTICE);
-            }
-        } else if (localMtime > docMtime) {
-            // newer local file.
-            // ?
-        } else {
-            //Nothing have to op.
-            //eq.case
-        }
-    }
 
     queuedEntries: EntryBody[] = [];
     handleDBChanged(change: EntryBody) {
@@ -1511,7 +1493,7 @@ export default class ObsidianLiveSyncPlugin extends Plugin {
                 return;
             }
             const doc = change;
-            await this.doc2storage_create(doc);
+            await this.doc2storage(doc);
         } else if (targetFile instanceof TFile) {
             const doc = change;
             const file = targetFile;
@@ -1529,12 +1511,12 @@ export default class ObsidianLiveSyncPlugin extends Plugin {
                 return false;
             }
             if (this.settings.writeDocumentsIfConflicted) {
-                await this.doc2storage_modify(doc, file);
+                await this.doc2storage(doc, file);
                 queueConflictCheck();
             } else {
                 const d = await this.localDatabase.getDBEntryMeta(id2path(change._id), { conflicts: true }, true);
                 if (d && !d._conflicts) {
-                    await this.doc2storage_modify(doc, file);
+                    await this.doc2storage(doc, file);
                 } else {
                     if (!queueConflictCheck()) {
                         Logger(`${id2path(change._id)} is conflicted, write to the storage has been pended.`, LOG_LEVEL.NOTICE);
@@ -1632,6 +1614,19 @@ export default class ObsidianLiveSyncPlugin extends Plugin {
     async parseIncomingDoc(doc: PouchDB.Core.ExistingDocument<EntryBody>) {
         if (!this.isTargetFile(id2path(doc._id))) return;
         const skipOldFile = this.settings.skipOlderFilesOnSync && false; //patched temporary.
+        // Do not handle internal files if the feature has not been enabled.
+        if (isInternalMetadata(doc._id) && !this.settings.syncInternalFiles) return;
+        // It is better for your own safety, not to handle the following files
+        const ignoreFiles = [
+            "_design/replicate",
+            FLAGMD_REDFLAG,
+            FLAGMD_REDFLAG2,
+            FLAGMD_REDFLAG3
+        ];
+        if (!isInternalMetadata(doc._id) && ignoreFiles.contains(id2path(doc._id))) {
+            return;
+
+        }
         if ((!isInternalMetadata(doc._id)) && skipOldFile) {
             const info = getAbstractFileByPath(id2path(doc._id));
 
@@ -1911,7 +1906,7 @@ export default class ObsidianLiveSyncPlugin extends Plugin {
         if (this.settings.syncInternalFiles && this.settings.syncInternalFilesBeforeReplication && !this.settings.watchInternalFileChanges) {
             await this.syncInternalFilesAndDatabase("push", showMessage);
         }
-        this.localDatabase.openReplication(this.settings, false, showMessage, this.parseReplicationResult);
+        await this.localDatabase.openReplication(this.settings, false, showMessage, this.parseReplicationResult);
     }
 
     async initializeDatabase(showingNotice?: boolean) {
@@ -2640,7 +2635,7 @@ export default class ObsidianLiveSyncPlugin extends Plugin {
                 Logger(`${filename} Skipped`);
                 return;
             }
-            await this.doc2storage_create(doc, force);
+            await this.doc2storage(doc, undefined, force);
         } else if (targetFile instanceof TFile) {
             //normal case
             const file = targetFile;
@@ -2649,7 +2644,7 @@ export default class ObsidianLiveSyncPlugin extends Plugin {
                 Logger(`${filename} Skipped`);
                 return;
             }
-            await this.doc2storage_modify(doc, file, force);
+            await this.doc2storage(doc, file, force);
         } else {
             Logger(`target files:${filename} is exists as the folder`);
             //something went wrong..
@@ -2691,7 +2686,7 @@ export default class ObsidianLiveSyncPlugin extends Plugin {
             Logger(`${storageMtime} < ${docMtime}`);
             const docx = await this.localDatabase.getDBEntry(file.path, null, false, false);
             if (docx != false) {
-                await this.doc2storage_modify(docx, file);
+                await this.doc2storage(docx, file);
             } else {
                 Logger("STORAGE <- DB :" + file.path + " Skipped");
             }

src/utils.ts

@@ -1,6 +1,8 @@
-import { normalizePath } from "obsidian";
+import { DataWriteOptions, normalizePath, TFile, Platform } from "obsidian";
+import { path2id_base, id2path_base, isValidFilenameInLinux, isValidFilenameInDarwin, isValidFilenameInWidows, isValidFilenameInAndroid } from "./lib/src/path";
 
-import { path2id_base, id2path_base } from "./lib/src/path";
+import { Logger } from "./lib/src/logger";
+import { LOG_LEVEL } from "./lib/src/types";
 
 // For backward compatibility, using the path for determining id.
 // Only CouchDB unacceptable ID (that starts with an underscore) has been prefixed with "/".
@@ -259,4 +261,33 @@ export function flattenObject(obj: Record<string | number | symbol, any>, path:
         ret.push(...p);
     }
     return ret;
+}
+
+export function modifyFile(file: TFile, data: string | ArrayBuffer, options?: DataWriteOptions) {
+    if (typeof (data) === "string") {
+        return app.vault.modify(file, data, options);
+    } else {
+        return app.vault.modifyBinary(file, data, options);
+    }
+}
+export function createFile(path: string, data: string | ArrayBuffer, options?: DataWriteOptions): Promise<TFile> {
+    if (typeof (data) === "string") {
+        return app.vault.create(path, data, options);
+    } else {
+        return app.vault.createBinary(path, data, options);
+    }
+}
+
+export function isValidPath(filename: string) {
+    if (Platform.isDesktop) {
+        // if(Platform.isMacOS) return isValidFilenameInDarwin(filename);
+        if (process.platform == "darwin") return isValidFilenameInDarwin(filename);
+        if (process.platform == "linux") return isValidFilenameInLinux(filename);
+        return isValidFilenameInWidows(filename);
+    }
+    if (Platform.isAndroidApp) return isValidFilenameInAndroid(filename);
+    if (Platform.isIosApp) return isValidFilenameInDarwin(filename);
+    //Fallback
+    Logger("Could not determine platform for checking filename", LOG_LEVEL.VERBOSE);
+    return isValidFilenameInWidows(filename);
 }

updates.md

@@ -10,63 +10,7 @@
     - Chunk ID numbering rules
 
 #### Minors
-- 0.17.1
-  - Fixed: Now we can verify and repair the database.
-  - Refactored inside.
-
-- 0.17.2
-  - New feature
-    - We can merge conflicted documents automatically if sensible.
-  - Fixed
-    - Writing to the storage will be pended while they have conflicts after replication.
-
-- 0.17.3
-  - Now we supported canvas! And conflicted JSON files are also synchronised with merging its content if they are obvious.
-
-- 0.17.4
-  - Canvases are now treated as a sort of plain text file. now we transfer only the metadata and chunks that have differences.
-
-- 0.17.5 Now `read chunks online` had been fixed, and a new feature: `Use dynamic iteration count` to reduce the load on encryption/decryption.
-  Note: `Use dynamic iteration count` is not compatible with earlier versions.
-- 0.17.6 Now our renamed/deleted files have been surely deleted again.
-- 0.17.7
-  - Fixed:
-    - Fixed merging issues.
-    - Fixed button styling.
-  - Changed:
-    - Conflict checking on synchronising has been enabled for every note in default.
-- 0.17.8
-  - Improved: Performance improved. Prebuilt PouchDB is no longer used.
-  - Fixed: Merging hidden files is also fixed.
-  - New Feature: Now we can synchronise automatically after merging conflicts.
-- 0.17.9
-  - Fixed: Conflict merge of internal files is no longer broken.
-  - Improved: Smoother status display inside the editor.
-- 0.17.10
-  - Fixed: Large file synchronising has been now addressed!
-  Note: When synchronising large files, we have to set `Chunk size` to lower than 50, disable `Read chunks online`, `Batch size` should be set 50-100, and `Batch limit` could be around 20.
-- 0.17.11
-  - Fixed:
-    - Performance improvement
-    - Now `Chunk size` can be set to under one hundred.
-  - New feature:
-    - The number of transfers required before replication stabilises is now displayed.
-- 0.17.12: Skipped.
-- 0.17.13
-  - Fixed: Document history is now displayed again.
-  - Reorganised: Many files have been refactored.
-- 0.17.14: Skipped.
-- 0.17.15
-  - Improved:
-    - Confidential information has no longer stored in data.json as is.
-    - Synchronising progress has been shown in the notification.
-    - We can commit passphrases with a keyboard.
-    - Configuration which had not been saved yet is marked now.
-    - Now the filename is shown on the Conflict resolving dialog
-  - Fixed:
-    - Hidden files have been synchronised again.
-    - Rename of files has been fixed again.
-  And, minor changes have been included.
+- __0.17.1 to 0.17.15 has been moved into `update_old.md`__
 
 - 0.17.16:
   - Improved:
@@ -96,5 +40,28 @@
       - We have been grateful to @karasevm!
     - The conflicted `data.json` is no longer merged automatically.
       - This behaviour is not configurable, unlike the `Use newer file if conflicted` of normal files.
+- 0.17.22
+  - Fixed:
+    - Now hidden files will not be synchronised while we are not configured.
+    - Some processes could start without waiting for synchronisation to complete, but now they will wait for.
+  - Improved
+    - Now, by placing `redflag3.md`, we can discard the local database and fetch again.
+    - The document has been updated! Thanks to @hilsonp!
+- 0.17.23
+  - Improved:
+    - Now we can preserve the logs into the file.
+      - Note: This option will be enabled automatically also when we flagging a red flag.
+    - File names can now be made platform-appropriate.
+  - Refactored:
+    - Some redundant implementations have been sorted out.
+- 0.17.24
+  - New feature:
+    - If any conflicted files have been left, they will be reported.
+  - Fixed:
+    - Now the name of the conflicting file is shown on the conflict-resolving dialogue.
+    - Hidden files are now able to be merged again.
+    - No longer error caused at plug-in being loaded.
+  - Improved:
+    - Caching chunks are now limited in total size of cached chunks.
 
 ... To continue on to `updates_old.md`.

updates_old.md

@@ -1,3 +1,73 @@
+### 0.17.0
+- 0.17.0 has no surfaced changes but the design of saving chunks has been changed. They have compatibility but changing files after upgrading makes different chunks than before 0.16.x.
+  Please rebuild databases once if you have been worried about storage usage.
+
+  - Improved:
+    - Splitting markdown
+    - Saving chunks
+
+  - Changed:
+    - Chunk ID numbering rules
+
+#### Minors
+
+- 0.17.1
+  - Fixed: Now we can verify and repair the database.
+  - Refactored inside.
+
+- 0.17.2
+  - New feature
+    - We can merge conflicted documents automatically if sensible.
+  - Fixed
+    - Writing to the storage will be pended while they have conflicts after replication.
+
+- 0.17.3
+  - Now we supported canvas! And conflicted JSON files are also synchronised with merging its content if they are obvious.
+
+- 0.17.4
+  - Canvases are now treated as a sort of plain text file. now we transfer only the metadata and chunks that have differences.
+
+- 0.17.5 Now `read chunks online` had been fixed, and a new feature: `Use dynamic iteration count` to reduce the load on encryption/decryption.
+  Note: `Use dynamic iteration count` is not compatible with earlier versions.
+- 0.17.6 Now our renamed/deleted files have been surely deleted again.
+- 0.17.7
+  - Fixed:
+    - Fixed merging issues.
+    - Fixed button styling.
+  - Changed:
+    - Conflict checking on synchronising has been enabled for every note in default.
+- 0.17.8
+  - Improved: Performance improved. Prebuilt PouchDB is no longer used.
+  - Fixed: Merging hidden files is also fixed.
+  - New Feature: Now we can synchronise automatically after merging conflicts.
+- 0.17.9
+  - Fixed: Conflict merge of internal files is no longer broken.
+  - Improved: Smoother status display inside the editor.
+- 0.17.10
+  - Fixed: Large file synchronising has been now addressed!
+  Note: When synchronising large files, we have to set `Chunk size` to lower than 50, disable `Read chunks online`, `Batch size` should be set 50-100, and `Batch limit` could be around 20.
+- 0.17.11
+  - Fixed:
+    - Performance improvement
+    - Now `Chunk size` can be set to under one hundred.
+  - New feature:
+    - The number of transfers required before replication stabilises is now displayed.
+- 0.17.12: Skipped.
+- 0.17.13
+  - Fixed: Document history is now displayed again.
+  - Reorganised: Many files have been refactored.
+- 0.17.14: Skipped.
+- 0.17.15
+  - Improved:
+    - Confidential information has no longer stored in data.json as is.
+    - Synchronising progress has been shown in the notification.
+    - We can commit passphrases with a keyboard.
+    - Configuration which had not been saved yet is marked now.
+    - Now the filename is shown on the Conflict resolving dialog
+  - Fixed:
+    - Hidden files have been synchronised again.
+    - Rename of files has been fixed again.
+  And, minor changes have been included.
 
 ### 0.16.0
 - Now hidden files need not be scanned. Changes will be detected automatically.