Compare commits

...

14 Commits

Author SHA1 Message Date
vorotamoroz
b00b0cc5e5 bump 2024-03-15 10:37:15 +01:00
vorotamoroz
d7985a6b41 Improved:
- Now using HTTP for the remote database URI warns of an error (on mobile) or notice (on desktop).
2024-03-15 10:36:00 +01:00
vorotamoroz
486e816902 Update dependencies 2024-03-15 10:35:41 +01:00
vorotamoroz
ef9b19c24b bump 2024-03-04 04:07:51 +00:00
vorotamoroz
4ed9494176 Changed:
- The default settings has been changed.
Improved:
- Default and preferred settings are applied on completion of the wizard.
Fixed:
- Now Initialisation `Fetch` will be performed smoothly and there will be fewer conflicts.
- No longer stuck while Handling transferred or initialised documents.
2024-03-04 04:07:11 +00:00
vorotamoroz
fcd56d59d5 bump 2024-03-01 08:33:37 +00:00
vorotamoroz
1cabfcfd19 Fixed:
- `Verify and repair all files` is no longer broken.
New feature::
- Now `Verify and repair all files` can restore or show history
Improved:
- Performance improved
2024-03-01 08:32:48 +00:00
vorotamoroz
37a18dbfef bump 2024-03-01 03:28:46 +00:00
vorotamoroz
e7edf88713 Fixed
- No longer unchanged hidden files and customisations are saved and transferred now.
- File integrity of vault history indicates the integrity correctly.
Improved
- In the report, the schema of the remote database URI is now printed.
2024-03-01 03:28:06 +00:00
vorotamoroz
90ff75ab35 add notes. 2024-02-29 00:30:07 +00:00
vorotamoroz
bff1d661f5 Update troubleshooting.md
Fix grammar
2024-02-29 00:42:28 +09:00
vorotamoroz
6b59c14774 Update doc 2024-02-28 08:29:06 +00:00
vorotamoroz
8249274eac bump 2024-02-28 08:28:07 +00:00
vorotamoroz
3c6dae7814 - Fixed:
- Fixed a bug on `fetch chunks on demand` that could not fetch the chunks on demand.
- Improved:
  - `fetch chunks on demand` works more smoothly.
  - Initialisation `Fetch` is now more efficient.
- Tidied:
  - Removed some meaningless codes.
2024-02-28 08:27:17 +00:00
17 changed files with 1950 additions and 1509 deletions

View File

@@ -1,10 +1,25 @@
<!-- 2024-02-15 -->
# Tips and Troubleshooting
- [Notable bugs and fixes](#notable-bugs-and-fixes)
- [FAQ](#faq)
- [Troubleshooting](#troubleshooting)
- [Tips](#tips)
- [Tips and Troubleshooting](#tips-and-troubleshooting)
- [Notable bugs and fixes](#notable-bugs-and-fixes)
- [Binary files get bigger on iOS](#binary-files-get-bigger-on-ios)
- [Some setting name has been changed](#some-setting-name-has-been-changed)
- [FAQ](#faq)
- [Why `Use an old adapter for compatibility` is somehow enabled in my vault?](#why-use-an-old-adapter-for-compatibility-is-somehow-enabled-in-my-vault)
- [ZIP (or any extensions) files were not synchronised. Why?](#zip-or-any-extensions-files-were-not-synchronised-why)
- [I hope to report the issue, but you said you needs `Report`. How to make it?](#i-hope-to-report-the-issue-but-you-said-you-needs-report-how-to-make-it)
- [Where can I check the log?](#where-can-i-check-the-log)
- [Why are the logs volatile and ephemeral?](#why-are-the-logs-volatile-and-ephemeral)
- [Some network logs are not written into the file.](#some-network-logs-are-not-written-into-the-file)
- [If a file were deleted or trimmed, the capacity of the database should be reduced, right?](#if-a-file-were-deleted-or-trimmed-the-capacity-of-the-database-should-be-reduced-right)
- [Troubleshooting](#troubleshooting)
- [On the mobile device, cannot synchronise on the local network!](#on-the-mobile-device-cannot-synchronise-on-the-local-network)
- [I think that something bad happening on the vault...](#i-think-that-something-bad-happening-on-the-vault)
- [Tips](#tips)
- [Old tips](#old-tips)
<!-- - -->
@@ -39,20 +54,56 @@ When you rebuild everything or fetch from the remote again, you will be asked to
Therefore, experienced users (especially those stable enough not to have to rebuild the database) may have this toggle enabled in their Vault.
Please disable it when you have enough time.
### ZIP (or any extensions) files were not synchronised. Why?
It depends on Obsidian detects. May toggling `Detect all extensions` of `File and links` (setting of Obsidian) will help us.
### I hope to report the issue, but you said you needs `Report`. How to make it?
We can copy the report to the clipboard, by pressing the `Make report` button on the `Hatch` pane.
![Screenshot](../images/hatch.png)
### Where can I check the log?
We can launch the log pane by `Show log` on the command palette.
And if you have troubled something, please enable the `Verbose Log` on the `General Setting` pane.
However, the logs would not be kept so long and cleared when restarted. If you want to check the logs, please enable `Write logs into the file` temporarily.
![ScreenShot](../images/write_logs_into_the_file.png)
> [!IMPORTANT]
> - Writing logs into the file will impact the performance.
> - Please make sure that you have erased all your confidential information before reporting issue.
### Why are the logs volatile and ephemeral?
To avoid unexpected exposure to our confidential things.
### Some network logs are not written into the file.
Especially the CORS error will be reported as a general error to the plug-in for security reasons. So we cannot detect and log it.
### If a file were deleted or trimmed, the capacity of the database should be reduced, right?
No, even though if files were deleted, chunks were not deleted.
Self-hosted LiveSync splits the files into multiple chunks and transfers only newly created. This behaviour enables us to less traffic. And, the chunks will be shared between the files to reduce the total usage of the database.
And one more thing, we can handle the conflicts on any device even though it has happened on other devices. This means that conflicts will happen in the past, after the time we have synchronised. Hence we cannot collect and delete the unused chunks even though if we are not currently referenced.
To shrink the database size, `Rebuild everything` only reliably and effectively. But do not worry, if we have synchronised well. We have the actual and real files. Only it takes a bit of time and traffics.
<!-- Add here -->
## Troubleshooting
<!-- Add here -->
### On the mobile device, cannot synchronise on the local network!
Obsidian mobile is not able to connect to the non-secure end-point, such as starting with `http://`. Make sure your URI of CouchDB. Also not able to use a self-signed certificate.
### I think that something bad happening on the vault...
Place `redflag.md` on top of the vault, and restart Obsidian. The most simple way is to create a new note and rename it to `redflag`. Of course, we can put it without Obsidian.
If there is `redflag.md`, Self-hosted LiveSync suspends all database and storage processes.
## Tips
<!-- Add here -->
### Old tips
- If a folder becomes empty after a replication, it will be deleted by default. But you can toggle this behaviour. Check the [Settings](settings.md).
- LiveSync mode drains more batteries in mobile devices. Periodic sync with some automatic sync is recommended.
- Mobile Obsidian can not connect to non-secure (HTTP) or locally-signed servers, even if the root certificate is installed on the device.
- There are no 'exclude_folders' like configurations.
- While synchronizing, files are compared by their modification time and the older ones will be overwritten by the newer ones. Then plugin checks for conflicts and if a merge is needed, a dialog will open.
- Rarely, a file in the database could be corrupted. The plugin will not write to local storage when a file looks corrupted. If a local version of the file is on your device, the corruption could be fixed by editing the local file and synchronizing it. But if the file does not exist on any of your devices, then it can not be rescued. In this case, you can delete these items from the settings dialog.
- To stop the boot-up sequence (eg. for fixing problems on databases), you can put a `redflag.md` file (or directory) at the root of your vault.
Tip for iOS: a redflag directory can be created at the root of the vault using the File application.

BIN
images/hatch.png Normal file

Binary file not shown.

After

Width:  |  Height:  |  Size: 13 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 17 KiB

View File

@@ -1,7 +1,7 @@
{
"id": "obsidian-livesync",
"name": "Self-hosted LiveSync",
"version": "0.22.8",
"version": "0.22.13",
"minAppVersion": "0.9.12",
"description": "Community implementation of self-hosted livesync. Reflect your vault changes to some other devices immediately. Please make sure to disable other synchronize solutions to avoid content corruption or duplication.",
"author": "vorotamoroz",

2646
package-lock.json generated

File diff suppressed because it is too large Load Diff

View File

@@ -1,6 +1,6 @@
{
"name": "obsidian-livesync",
"version": "0.22.8",
"version": "0.22.13",
"description": "Reflect your vault changes to some other devices immediately. Please make sure to disable other synchronize solutions to avoid content corruption or duplication.",
"main": "main.js",
"type": "module",
@@ -13,29 +13,29 @@
"author": "vorotamoroz",
"license": "MIT",
"devDependencies": {
"@tsconfig/svelte": "^5.0.0",
"@types/diff-match-patch": "^1.0.32",
"@types/node": "^20.2.5",
"@types/pouchdb": "^6.4.0",
"@types/pouchdb-adapter-http": "^6.1.3",
"@types/pouchdb-adapter-idb": "^6.1.4",
"@types/pouchdb-browser": "^6.1.3",
"@types/pouchdb-core": "^7.0.11",
"@types/pouchdb-mapreduce": "^6.1.7",
"@types/pouchdb-replication": "^6.4.4",
"@types/transform-pouch": "^1.0.2",
"@typescript-eslint/eslint-plugin": "^6.2.1",
"@typescript-eslint/parser": "^6.2.1",
"@tsconfig/svelte": "^5.0.2",
"@types/diff-match-patch": "^1.0.36",
"@types/node": "^20.11.28",
"@types/pouchdb": "^6.4.2",
"@types/pouchdb-adapter-http": "^6.1.6",
"@types/pouchdb-adapter-idb": "^6.1.7",
"@types/pouchdb-browser": "^6.1.5",
"@types/pouchdb-core": "^7.0.14",
"@types/pouchdb-mapreduce": "^6.1.10",
"@types/pouchdb-replication": "^6.4.7",
"@types/transform-pouch": "^1.0.6",
"@typescript-eslint/eslint-plugin": "^7.2.0",
"@typescript-eslint/parser": "^7.2.0",
"builtin-modules": "^3.3.0",
"esbuild": "0.18.17",
"esbuild-svelte": "^0.7.4",
"eslint": "^8.46.0",
"esbuild": "0.20.2",
"esbuild-svelte": "^0.8.0",
"eslint": "^8.57.0",
"eslint-config-airbnb-base": "^15.0.0",
"eslint-plugin-import": "^2.28.0",
"eslint-plugin-import": "^2.29.1",
"events": "^3.3.0",
"obsidian": "^1.4.11",
"postcss": "^8.4.27",
"postcss-load-config": "^4.0.1",
"obsidian": "^1.5.7",
"postcss": "^8.4.35",
"postcss-load-config": "^5.0.3",
"pouchdb-adapter-http": "^8.0.1",
"pouchdb-adapter-idb": "^8.0.1",
"pouchdb-adapter-indexeddb": "^8.0.1",
@@ -46,16 +46,16 @@
"pouchdb-merge": "^8.0.1",
"pouchdb-replication": "^8.0.1",
"pouchdb-utils": "^8.0.1",
"svelte": "^4.1.2",
"svelte-preprocess": "^5.0.4",
"terser": "^5.19.2",
"svelte": "^4.2.12",
"svelte-preprocess": "^5.1.3",
"terser": "^5.29.2",
"transform-pouch": "^2.0.0",
"tslib": "^2.6.1",
"typescript": "^5.1.6"
"tslib": "^2.6.2",
"typescript": "^5.4.2"
},
"dependencies": {
"diff-match-patch": "^1.0.5",
"idb": "^7.1.1",
"idb": "^8.0.0",
"minimatch": "^9.0.3",
"xxhash-wasm": "0.4.2",
"xxhash-wasm-102": "npm:xxhash-wasm@^1.0.2"

View File

@@ -4,7 +4,7 @@ import { Notice, type PluginManifest, parseYaml, normalizePath, type ListedFiles
import type { EntryDoc, LoadedEntry, InternalFileEntry, FilePathWithPrefix, FilePath, DocumentID, AnyEntry, SavingEntry } from "./lib/src/types";
import { LOG_LEVEL_INFO, LOG_LEVEL_NOTICE, LOG_LEVEL_VERBOSE, MODE_SELECTIVE } from "./lib/src/types";
import { ICXHeader, PERIODIC_PLUGIN_SWEEP, } from "./types";
import { createTextBlob, delay, getDocData, sendSignal, waitForSignal } from "./lib/src/utils";
import { createTextBlob, delay, getDocData, isDocContentSame, sendSignal, waitForSignal } from "./lib/src/utils";
import { Logger } from "./lib/src/logger";
import { WrappedNotice } from "./lib/src/wrapper";
import { readString, decodeBinary, arrayBufferToBase64, sha1 } from "./lib/src/strbin";
@@ -178,9 +178,7 @@ export class ConfigSync extends LiveSyncCommands {
get kvDB() {
return this.plugin.kvDB;
}
ensureDirectoryEx(fullPath: string) {
return this.plugin.ensureDirectoryEx(fullPath);
}
pluginDialog: PluginDialogModal = null;
periodicPluginSweepProcessor = new PeriodicProcessor(this.plugin, async () => await this.scanAllConfigFiles(false));
@@ -433,7 +431,7 @@ export class ConfigSync extends LiveSyncCommands {
try {
// console.dir(f);
const path = `${baseDir}/${f.filename}`;
await this.ensureDirectoryEx(path);
await this.vaultAccess.ensureDirectory(path);
if (!content) {
const dt = decodeBinary(f.data);
await this.vaultAccess.adapterWrite(path, dt);
@@ -689,9 +687,21 @@ export class ConfigSync extends LiveSyncCommands {
};
} else {
if (old.mtime == mtime) {
// Logger(`STORAGE --> DB:${file.path}: (hidden) Not changed`, LOG_LEVEL_VERBOSE);
// Logger(`STORAGE --> DB:${prefixedFileName}: (config) Skipped (Same time)`, LOG_LEVEL_VERBOSE);
return true;
}
const oldC = await this.localDatabase.getDBEntryFromMeta(old, {}, false, false);
if (oldC) {
const d = await deserialize(getDocData(oldC.data), {}) as PluginDataEx;
const diffs = (d.files.map(previous => ({ prev: previous, curr: dt.files.find(e => e.filename == previous.filename) })).map(async e => {
try { return await isDocContentSame(e.curr.data, e.prev.data) } catch (_) { return false }
}))
const isSame = (await Promise.all(diffs)).every(e => e == true);
if (isSame) {
Logger(`STORAGE --> DB:${prefixedFileName}: (config) Skipped (Same content)`, LOG_LEVEL_VERBOSE);
return true;
}
}
saveData =
{
...old,

View File

@@ -20,9 +20,6 @@ export class HiddenFileSync extends LiveSyncCommands {
get kvDB() {
return this.plugin.kvDB;
}
ensureDirectoryEx(fullPath: string) {
return this.plugin.ensureDirectoryEx(fullPath);
}
getConflictedDoc(path: FilePathWithPrefix, rev: string) {
return this.plugin.getConflictedDoc(path, rev);
}
@@ -202,7 +199,7 @@ export class HiddenFileSync extends LiveSyncCommands {
const filename = stripAllPrefixes(path);
const isExists = await this.plugin.vaultAccess.adapterExists(filename);
if (!isExists) {
await this.ensureDirectoryEx(filename);
await this.vaultAccess.ensureDirectory(filename);
}
await this.plugin.vaultAccess.adapterWrite(filename, result);
const stat = await this.vaultAccess.adapterStat(filename);
@@ -482,7 +479,7 @@ export class HiddenFileSync extends LiveSyncCommands {
type: "newnote",
};
} else {
if (await isDocContentSame(old.data, content) && !forceWrite) {
if (await isDocContentSame(createBinaryBlob(decodeBinary(old.data)), content) && !forceWrite) {
// Logger(`STORAGE --> DB:${file.path}: (hidden) Not changed`, LOG_LEVEL_VERBOSE);
return;
}
@@ -498,7 +495,7 @@ export class HiddenFileSync extends LiveSyncCommands {
type: "newnote",
};
}
const ret = await this.localDatabase.putDBEntry(saveData, true);
const ret = await this.localDatabase.putDBEntry(saveData);
Logger(`STORAGE --> DB:${file.path}: (hidden) Done`);
return ret;
} catch (ex) {
@@ -599,7 +596,7 @@ export class HiddenFileSync extends LiveSyncCommands {
return true;
}
if (!isExists) {
await this.ensureDirectoryEx(filename);
await this.vaultAccess.ensureDirectory(filename);
await this.plugin.vaultAccess.adapterWrite(filename, decodeBinary(fileOnDB.data), { mtime: fileOnDB.mtime, ctime: fileOnDB.ctime });
try {
//@ts-ignore internalAPI
@@ -668,7 +665,7 @@ export class HiddenFileSync extends LiveSyncCommands {
if (!keep && result) {
const isExists = await this.plugin.vaultAccess.adapterExists(filename);
if (!isExists) {
await this.ensureDirectoryEx(filename);
await this.vaultAccess.ensureDirectory(filename);
}
await this.plugin.vaultAccess.adapterWrite(filename, result);
const stat = await this.plugin.vaultAccess.adapterStat(filename);

View File

@@ -134,7 +134,7 @@ export class SetupLiveSync extends LiveSyncCommands {
} else if (setupType == setupAsMerge) {
this.plugin.settings = newSettingW;
this.plugin.usedPassphrase = "";
await this.fetchLocalWithKeepLocal();
await this.fetchLocalWithRebuild();
} else if (setupType == setupAgain) {
const confirm = "I know this operation will rebuild all my databases with files on this device, and files that are on the remote database and I didn't synchronize to any other devices will be lost and want to proceed indeed.";
if (await askSelectString(this.app, "Do you really want to do this?", ["Cancel", confirm]) != confirm) {
@@ -359,7 +359,7 @@ Of course, we are able to disable these features.`
Logger(`Fetching chunks done`, LOG_LEVEL_NOTICE);
}
}
async fetchLocal() {
async fetchLocal(makeLocalChunkBeforeSync?: boolean) {
this.suspendExtraSync();
await this.askUseNewAdapter();
this.plugin.settings.isConfigured = true;
@@ -368,35 +368,23 @@ Of course, we are able to disable these features.`
await this.resetLocalDatabase();
await delay(1000);
await this.plugin.openDatabase();
await this.plugin.markRemoteResolved();
this.plugin.isReady = true;
if (makeLocalChunkBeforeSync) {
await this.plugin.initializeDatabase(true);
}
await this.plugin.markRemoteResolved();
await delay(500);
await this.plugin.replicateAllFromServer(true);
await delay(1000);
await this.plugin.replicateAllFromServer(true);
await this.fetchRemoteChunks();
// if (!tryLessFetching) {
// await this.fetchRemoteChunks();
// }
await this.resumeReflectingDatabase();
await this.askHiddenFileConfiguration({ enableFetch: true });
}
async fetchLocalWithKeepLocal() {
this.suspendExtraSync();
await this.askUseNewAdapter();
this.plugin.settings.isConfigured = true;
await this.suspendReflectingDatabase();
await this.plugin.realizeSettingSyncMode();
await this.resetLocalDatabase();
await delay(1000);
await this.plugin.initializeDatabase(true);
await this.plugin.openDatabase();
await this.plugin.markRemoteResolved();
this.plugin.isReady = true;
await delay(500);
await this.plugin.replicateAllFromServer(true);
await delay(1000);
await this.plugin.replicateAllFromServer(true);
await this.fetchRemoteChunks();
await this.resumeReflectingDatabase();
await this.askHiddenFileConfiguration({ enableFetch: true });
async fetchLocalWithRebuild() {
return await this.fetchLocal(true);
}
async rebuildRemote() {
this.suspendExtraSync();

View File

@@ -7,6 +7,7 @@
import { DocumentHistoryModal } from "./DocumentHistoryModal";
import { isPlainText, stripAllPrefixes } from "./lib/src/path";
import { TFile } from "./deps";
import { decodeBinary } from "./lib/src/strbin";
export let plugin: ObsidianLiveSyncPlugin;
let showDiffInfo = false;
@@ -113,7 +114,7 @@
} else {
const data = await plugin.vaultAccess.adapterReadBinary(abs);
const dataEEncoded = createBinaryBlob(data);
result = await isDocContentSame(dataEEncoded, doc.data);
result = await isDocContentSame(dataEEncoded, createBinaryBlob(decodeBinary(doc.data)));
}
if (result) {
diffDetail += " ⚖️";

View File

@@ -1,13 +1,14 @@
import { App, PluginSettingTab, Setting, sanitizeHTMLToDom, TextAreaComponent, MarkdownRenderer, stringifyYaml } from "./deps";
import { DEFAULT_SETTINGS, type ObsidianLiveSyncSettings, type ConfigPassphraseStore, type RemoteDBSettings, type FilePathWithPrefix, type HashAlgorithm, type DocumentID, LOG_LEVEL_NOTICE, LOG_LEVEL_VERBOSE, LOG_LEVEL_INFO } from "./lib/src/types";
import { DEFAULT_SETTINGS, type ObsidianLiveSyncSettings, type ConfigPassphraseStore, type RemoteDBSettings, type FilePathWithPrefix, type HashAlgorithm, type DocumentID, LOG_LEVEL_NOTICE, LOG_LEVEL_VERBOSE, LOG_LEVEL_INFO, type LoadedEntry, PREFERRED_SETTING_CLOUDANT, PREFERRED_SETTING_SELF_HOSTED } from "./lib/src/types";
import { createBinaryBlob, createTextBlob, delay, isDocContentSame } from "./lib/src/utils";
import { versionNumberString2Number } from "./lib/src/strbin";
import { decodeBinary, versionNumberString2Number } from "./lib/src/strbin";
import { Logger } from "./lib/src/logger";
import { checkSyncInfo, isCloudantURI } from "./lib/src/utils_couchdb";
import { testCrypt } from "./lib/src/e2ee_v2";
import ObsidianLiveSyncPlugin from "./main";
import { askYesNo, performRebuildDB, requestToCouchDB, scheduleTask } from "./utils";
import { request, type ButtonComponent } from "obsidian";
import { request, type ButtonComponent, TFile } from "obsidian";
import { shouldBeIgnored } from "./lib/src/path";
export class ObsidianLiveSyncSettingTab extends PluginSettingTab {
@@ -266,6 +267,16 @@ export class ObsidianLiveSyncSettingTab extends PluginSettingTab {
const containerRemoteDatabaseEl = containerEl.createDiv();
containerRemoteDatabaseEl.createEl("h3", { text: "Remote Database configuration" });
const syncWarn = containerRemoteDatabaseEl.createEl("div", { text: `These settings are kept locked while any synchronization options are enabled. Disable these options in the "Sync Settings" tab to unlock.` });
if (this.plugin.settings.couchDB_URI.startsWith("http://")) {
if (this.plugin.isMobile) {
containerRemoteDatabaseEl.createEl("div", { text: `Configured as using plain HTTP. We cannot connect to the remote. Please set up the credentials and use HTTPS for the remote URI.` })
.addClass("op-warn");
} else {
containerRemoteDatabaseEl.createEl("div", { text: `Configured as using plain HTTP. We might fail on mobile devices.` })
.addClass("op-warn-info");
}
}
syncWarn.addClass("op-warn-info");
syncWarn.addClass("sls-hidden");
@@ -732,7 +743,7 @@ export class ObsidianLiveSyncSettingTab extends PluginSettingTab {
}
};
const rebuildDB = async (method: "localOnly" | "remoteOnly" | "rebuildBothByThisDevice") => {
const rebuildDB = async (method: "localOnly" | "remoteOnly" | "rebuildBothByThisDevice" | "localOnlyWithChunks") => {
if (encrypt && passphrase == "") {
Logger("If you enable encryption, you have to set the passphrase", LOG_LEVEL_NOTICE);
return;
@@ -773,9 +784,10 @@ export class ObsidianLiveSyncSettingTab extends PluginSettingTab {
this.plugin.settings.passphrase = "";
}
if (isCloudantURI(this.plugin.settings.couchDB_URI)) {
this.plugin.settings.customChunkSize = 0;
// this.plugin.settings.customChunkSize = 0;
this.plugin.settings = { ...this.plugin.settings, ...PREFERRED_SETTING_CLOUDANT };
} else {
this.plugin.settings.customChunkSize = 50;
this.plugin.settings = { ...this.plugin.settings, ...PREFERRED_SETTING_SELF_HOSTED };
}
changeDisplay("30")
})
@@ -1055,7 +1067,7 @@ export class ObsidianLiveSyncSettingTab extends PluginSettingTab {
await this.plugin.saveSettings();
await rebuildDB("localOnly");
Logger("All done! Please set up subsequent devices with 'Copy current settings as a new setup URI' and 'Use the copied setup URI'.", LOG_LEVEL_NOTICE);
await this.plugin.addOnSetup.command_openSetupURI();
await this.plugin.addOnSetup.command_copySetupURI();
} else {
this.askReload();
}
@@ -1635,7 +1647,8 @@ export class ObsidianLiveSyncSettingTab extends PluginSettingTab {
const pluginConfig = JSON.parse(JSON.stringify(this.plugin.settings)) as ObsidianLiveSyncSettings;
pluginConfig.couchDB_DBNAME = REDACTED;
pluginConfig.couchDB_PASSWORD = REDACTED;
pluginConfig.couchDB_URI = isCloudantURI(pluginConfig.couchDB_URI) ? "cloudant" : "self-hosted";
const scheme = pluginConfig.couchDB_URI.startsWith("http:") ? "(HTTP)" : (pluginConfig.couchDB_URI.startsWith("https:")) ? "(HTTPS)" : ""
pluginConfig.couchDB_URI = isCloudantURI(pluginConfig.couchDB_URI) ? "cloudant" : `self-hosted${scheme}`;
pluginConfig.couchDB_USER = REDACTED;
pluginConfig.passphrase = REDACTED;
pluginConfig.encryptedPassphrase = REDACTED;
@@ -1697,6 +1710,59 @@ ${stringifyYaml(pluginConfig)}`;
const hatchWarn = containerHatchEl.createEl("div", { text: `To stop the boot up sequence for fixing problems on databases, you can put redflag.md on top of your vault (Rebooting obsidian is required).` });
hatchWarn.addClass("op-warn-info");
const addResult = (path: string, file: TFile | false, fileOnDB: LoadedEntry | false) => {
resultArea.appendChild(resultArea.createEl("div", {}, el => {
el.appendChild(el.createEl("h6", { text: path }));
el.appendChild(el.createEl("div", {}, infoGroupEl => {
infoGroupEl.appendChild(infoGroupEl.createEl("div", { text: `Storage : Modified: ${!file ? `Missing:` : `${new Date(file.stat.mtime).toLocaleString()}, Size:${file.stat.size}`}` }))
infoGroupEl.appendChild(infoGroupEl.createEl("div", { text: `Database: Modified: ${!fileOnDB ? `Missing:` : `${new Date(fileOnDB.mtime).toLocaleString()}, Size:${fileOnDB.size}`}` }))
}));
if (fileOnDB && file) {
el.appendChild(el.createEl("button", { text: "Show history" }, buttonEl => {
buttonEl.onClickEvent(() => {
this.plugin.showHistory(file, fileOnDB._id);
})
}))
}
if (file) {
el.appendChild(el.createEl("button", { text: "Storage -> Database" }, buttonEl => {
buttonEl.onClickEvent(() => {
this.plugin.updateIntoDB(file, undefined, true);
el.remove();
})
}))
}
if (fileOnDB) {
el.appendChild(el.createEl("button", { text: "Database -> Storage" }, buttonEl => {
buttonEl.onClickEvent(() => {
this.plugin.pullFile(this.plugin.getPath(fileOnDB), [], true, undefined, false);
el.remove();
})
}))
}
return el;
}))
}
const checkBetweenStorageAndDatabase = async (file: TFile, fileOnDB: LoadedEntry) => {
let content: Blob;
let dataContent: Blob;
if (fileOnDB.type == "newnote") {
dataContent = createBinaryBlob(decodeBinary(fileOnDB.data));
content = createBinaryBlob(await this.plugin.vaultAccess.vaultReadBinary(file));
} else {
dataContent = createTextBlob(fileOnDB.data);
content = createTextBlob(await this.plugin.vaultAccess.vaultRead(file));
}
if (await isDocContentSame(content, dataContent)) {
Logger(`Compare: SAME: ${file.path}`)
} else {
Logger(`Compare: CONTENT IS NOT MATCHED! ${file.path}`, LOG_LEVEL_NOTICE);
addResult(file.path, file, fileOnDB)
}
}
new Setting(containerHatchEl)
.setName("Verify and repair all files")
.setDesc("Compare the content of files between on local database and storage. If not matched, you will asked which one want to keep.")
@@ -1707,47 +1773,36 @@ ${stringifyYaml(pluginConfig)}`;
.setWarning()
.onClick(async () => {
const files = this.app.vault.getFiles();
const documents = [] as FilePathWithPrefix[];
const adn = this.plugin.localDatabase.findAllNormalDocs()
for await (const i of adn) documents.push(this.plugin.getPath(i));
const allPaths = [...new Set([...documents, ...files.map(e => e.path as FilePathWithPrefix)])];
let i = 0;
for (const file of files) {
for (const path of allPaths) {
i++;
Logger(`${i}/${files.length}\n${file.path}`, LOG_LEVEL_NOTICE, "verify");
if (!await this.plugin.isTargetFile(file)) continue;
const fileOnDB = await this.plugin.localDatabase.getDBEntry(file.path as FilePathWithPrefix);
if (!fileOnDB) {
Logger(`Compare: Not found on local database: ${file.path}`, LOG_LEVEL_NOTICE);
Logger(`${i}/${files.length}\n${path}`, LOG_LEVEL_NOTICE, "verify");
if (shouldBeIgnored(path)) continue;
const abstractFile = this.plugin.vaultAccess.getAbstractFileByPath(path);
const fileOnStorage = abstractFile instanceof TFile ? abstractFile : false;
if (!await this.plugin.isTargetFile(path)) continue;
if (fileOnStorage && this.plugin.isFileSizeExceeded(fileOnStorage.stat.size)) continue;
const fileOnDB = await this.plugin.localDatabase.getDBEntry(path);
if (fileOnDB && this.plugin.isFileSizeExceeded(fileOnDB.size)) continue;
if (!fileOnDB && fileOnStorage) {
Logger(`Compare: Not found on the local database: ${path}`, LOG_LEVEL_NOTICE);
addResult(path, fileOnStorage, false)
continue;
}
let content: Blob;
if (fileOnDB.type == "newnote") {
content = createBinaryBlob(await this.plugin.vaultAccess.vaultReadBinary(file));
} else {
content = createTextBlob(await this.plugin.vaultAccess.vaultRead(file));
if (fileOnDB && !fileOnStorage) {
Logger(`Compare: Not found on the storage: ${path}`, LOG_LEVEL_NOTICE);
addResult(path, false, fileOnDB)
continue;
}
if (isDocContentSame(content, fileOnDB.data)) {
Logger(`Compare: SAME: ${file.path}`)
} else {
Logger(`Compare: CONTENT IS NOT MATCHED! ${file.path}`, LOG_LEVEL_NOTICE);
resultArea.appendChild(resultArea.createEl("div", {}, el => {
el.appendChild(el.createEl("h6", { text: file.path }));
el.appendChild(el.createEl("div", {}, infoGroupEl => {
infoGroupEl.appendChild(infoGroupEl.createEl("div", { text: `Storage : Modified: ${new Date(file.stat.mtime).toLocaleString()}, Size:${file.stat.size}` }))
infoGroupEl.appendChild(infoGroupEl.createEl("div", { text: `Database: Modified: ${new Date(fileOnDB.mtime).toLocaleString()}, Size:${content.size}` }))
}));
el.appendChild(el.createEl("button", { text: "Storage -> Database" }, buttonEl => {
buttonEl.onClickEvent(() => {
this.plugin.updateIntoDB(file, undefined, true);
el.remove();
})
}))
el.appendChild(el.createEl("button", { text: "Database -> Storage" }, buttonEl => {
buttonEl.onClickEvent(() => {
this.plugin.pullFile(file.path as FilePathWithPrefix, [], true, undefined, false);
el.remove();
})
}))
return el;
}))
if (fileOnStorage && fileOnDB) {
await checkBetweenStorageAndDatabase(fileOnStorage, fileOnDB)
}
}
Logger("done", LOG_LEVEL_NOTICE, "verify");
@@ -2124,6 +2179,19 @@ ${stringifyYaml(pluginConfig)}`;
})
)
new Setting(containerMaintenanceEl)
.setName("Fetch rebuilt DB (Save local documents before)")
.setDesc("Restore or reconstruct local database from remote database but use local chunks.")
.addButton((button) =>
button
.setButtonText("Save and Fetch")
.setWarning()
.setDisabled(false)
.onClick(async () => {
await rebuildDB("localOnlyWithChunks");
})
)
new Setting(containerMaintenanceEl)
.setName("Discard local database to reset or uninstall Self-hosted LiveSync")
.addButton((button) =>

View File

@@ -1,5 +1,6 @@
import { type App, TFile, type DataWriteOptions, TFolder, TAbstractFile } from "./deps";
import { serialized } from "./lib/src/lock";
import { Logger } from "./lib/src/logger";
import type { FilePath } from "./lib/src/types";
import { createBinaryBlob, isDocContentSame } from "./lib/src/utils";
import type { InternalFileInfo } from "./types";
@@ -107,6 +108,15 @@ export class SerializedFileAccess {
return await processWriteFile(path, () => this.app.vault.createBinary(path, toArrayBuffer(data), options));
}
}
trigger(name: string, ...data: any[]) {
return this.app.vault.trigger(name, ...data);
}
async adapterAppend(normalizedPath: string, data: string, options?: DataWriteOptions) {
return await this.app.vault.adapter.append(normalizedPath, data, options)
}
async delete(file: TFile | TFolder, force = false) {
return await processWriteFile(file, () => this.app.vault.delete(file, force));
}
@@ -127,6 +137,31 @@ export class SerializedFileAccess {
// }
}
getFiles() {
return this.app.vault.getFiles();
}
async ensureDirectory(fullPath: string) {
const pathElements = fullPath.split("/");
pathElements.pop();
let c = "";
for (const v of pathElements) {
c += v;
try {
await this.app.vault.adapter.mkdir(c);
} catch (ex) {
// basically skip exceptions.
if (ex.message && ex.message == "Folder already exists.") {
// especially this message is.
} else {
Logger("Folder Create Error");
Logger(ex);
}
}
c += "/";
}
}
touchedFiles: string[] = [];

Submodule src/lib updated: 1a3488339e...b9b70535ed

View File

@@ -84,7 +84,6 @@ export default class ObsidianLiveSyncPlugin extends Plugin
this._suspended = value;
}
deviceAndVaultName = "";
isMobile = false;
isReady = false;
packageVersion = "";
manifestVersion = "";
@@ -248,6 +247,22 @@ export default class ObsidianLiveSyncPlugin extends Plugin
}
}
get isMobile() {
// @ts-ignore: internal API
return this.app.isMobile
}
get vaultName() {
return this.app.vault.getName()
}
getActiveFile() {
return this.app.workspace.getActiveFile();
}
get appId() {
return `${("appId" in this.app ? this.app.appId : "")}`;
}
id2path(id: DocumentID, entry?: EntryHasPath, stripPrefix?: boolean): FilePathWithPrefix {
const tempId = id2path(id, entry);
if (stripPrefix && isInternalMetadata(tempId)) {
@@ -309,13 +324,7 @@ export default class ObsidianLiveSyncPlugin extends Plugin
// end interfaces
getVaultName(): string {
return this.app.vault.getName() + (this.settings?.additionalSuffixOfDatabaseName ? ("-" + this.settings.additionalSuffixOfDatabaseName) : "");
}
setInterval(handler: () => any, timeout?: number): number {
const timer = window.setInterval(handler, timeout);
this.registerInterval(timer);
return timer;
return this.vaultName + (this.settings?.additionalSuffixOfDatabaseName ? ("-" + this.settings.additionalSuffixOfDatabaseName) : "");
}
isFlagFileExist(path: string) {
@@ -361,7 +370,7 @@ export default class ObsidianLiveSyncPlugin extends Plugin
}
notes.sort((a, b) => b.mtime - a.mtime);
const notesList = notes.map(e => e.dispPath);
const target = await askSelectString(this.app, "File to view History", notesList);
const target = await this.askSelectString("File to view History", notesList);
if (target) {
const targetId = notes.find(e => e.dispPath == target);
this.showHistory(targetId.path, targetId.id);
@@ -379,7 +388,7 @@ export default class ObsidianLiveSyncPlugin extends Plugin
Logger("There are no conflicted documents", LOG_LEVEL_NOTICE);
return false;
}
const target = await askSelectString(this.app, "File to resolve conflict", notesList);
const target = await this.askSelectString("File to resolve conflict", notesList);
if (target) {
const targetItem = notes.find(e => e.dispPath == target);
this.resolveConflicted(targetItem.path);
@@ -458,10 +467,7 @@ Click anywhere to stop counting down.
const ret = await confirmWithMessage(this, "Welcome to Self-hosted LiveSync", message, [USE_SETUP, OPEN_SETUP, DISMISS], DISMISS, 40);
if (ret === OPEN_SETUP) {
try {
//@ts-ignore: undocumented api
this.app.setting.open();
//@ts-ignore: undocumented api
this.app.setting.openTabById("obsidian-livesync");
this.openSetting();
} catch (ex) {
Logger("Something went wrong on opening setting dialog, please open it manually", LOG_LEVEL_NOTICE);
}
@@ -482,22 +488,20 @@ Click anywhere to stop counting down.
Logger(`${FLAGMD_REDFLAG2} or ${FLAGMD_REDFLAG2_HR} has been detected! Self-hosted LiveSync suspends all sync and rebuild everything.`, LOG_LEVEL_NOTICE);
await this.addOnSetup.rebuildEverything();
await this.deleteRedFlag2();
if (await askYesNo(this.app, "Do you want to disable Suspend file watching and restart obsidian now?") == "yes") {
if (await this.askYesNo("Do you want to disable Suspend file watching and restart obsidian now?") == "yes") {
this.settings.suspendFileWatching = false;
await this.saveSettings();
// @ts-ignore
this.app.commands.executeCommandById("app:reload")
this.performAppReload();
}
} else if (this.isRedFlag3Raised()) {
Logger(`${FLAGMD_REDFLAG3} or ${FLAGMD_REDFLAG3_HR} has been detected! Self-hosted LiveSync will discard the local database and fetch everything from the remote once again.`, LOG_LEVEL_NOTICE);
await this.addOnSetup.fetchLocal();
await this.deleteRedFlag3();
if (this.settings.suspendFileWatching) {
if (await askYesNo(this.app, "Do you want to disable Suspend file watching and restart obsidian now?") == "yes") {
if (await this.askYesNo("Do you want to disable Suspend file watching and restart obsidian now?") == "yes") {
this.settings.suspendFileWatching = false;
await this.saveSettings();
// @ts-ignore
this.app.commands.executeCommandById("app:reload")
this.performAppReload();
}
}
} else {
@@ -546,8 +550,7 @@ Click anywhere to stop counting down.
this.askInPopup(`conflicting-detected-on-safety`, `Some files have been left conflicted! Press {HERE} to resolve them, or you can do it later by "Pick a file to resolve conflict`, (anchor) => {
anchor.text = "HERE";
anchor.addEventListener("click", () => {
// @ts-ignore
this.app.commands.executeCommandById("obsidian-livesync:livesync-all-conflictcheck");
this.performCommand("obsidian-livesync:livesync-all-conflictcheck");
});
}
);
@@ -629,7 +632,7 @@ Note: We can always able to read V1 format. It will be progressively converted.
// this.localDatabase.getDBEntry(getPathFromTFile(file), {}, true, false);
// },
callback: () => {
const file = this.app.workspace.getActiveFile();
const file = this.getActiveFile();
if (!file) return;
this.localDatabase.getDBEntry(getPathFromTFile(file), {}, true, false);
},
@@ -678,7 +681,7 @@ Note: We can always able to read V1 format. It will be progressively converted.
id: "livesync-history",
name: "Show history",
callback: () => {
const file = this.app.workspace.getActiveFile();
const file = this.getActiveFile();
if (file) this.showHistory(file, null);
}
});
@@ -791,8 +794,7 @@ Note: We can always able to read V1 format. It will be progressively converted.
}
//@ts-ignore
if (this.app.isMobile) {
this.isMobile = true;
if (this.isMobile) {
this.settings.disableRequestURI = true;
}
if (last_version && Number(last_version) < VER) {
@@ -869,8 +871,6 @@ Note: We can always able to read V1 format. It will be progressively converted.
}
const vaultName = this.getVaultName();
Logger("Waiting for ready...");
//@ts-ignore
this.isMobile = this.app.isMobile;
this.localDatabase = new LiveSyncLocalDB(vaultName, this);
initializeStores(vaultName);
return await this.localDatabase.initializeDatabase();
@@ -933,7 +933,7 @@ Note: We can always able to read V1 format. It will be progressively converted.
if (JSON.stringify(settings) !== JSON.stringify(DEFAULT_SETTINGS)) {
settings.isConfigured = true;
} else {
settings.additionalSuffixOfDatabaseName = `${("appId" in this.app ? this.app.appId : "")}`
settings.additionalSuffixOfDatabaseName = this.appId;
settings.isConfigured = false;
}
}
@@ -1057,7 +1057,7 @@ Note: We can always able to read V1 format. It will be progressively converted.
}
async parseSettingFromMarkdown(filename: string, data?: string) {
const file = this.app.vault.getAbstractFileByPath(filename);
const file = this.vaultAccess.getAbstractFileByPath(filename);
if (!(file instanceof TFile)) return {
preamble: "",
body: "",
@@ -1066,7 +1066,7 @@ Note: We can always able to read V1 format. It will be progressively converted.
if (data) {
return this.extractSettingFromWholeText(data);
}
const parseData = data ?? await this.app.vault.read(file);
const parseData = data ?? await this.vaultAccess.vaultRead(file);
return this.extractSettingFromWholeText(parseData);
}
@@ -1115,7 +1115,7 @@ Note: We can always able to read V1 format. It will be progressively converted.
const APPLY_AND_REBUILD = "Apply settings and restart obsidian with red_flag_rebuild.md";
const APPLY_AND_FETCH = "Apply settings and restart obsidian with red_flag_fetch.md";
const CANCEL = "Cancel";
const result = await askSelectString(this.app, "Ready for apply the setting.", [APPLY_AND_RESTART, APPLY_ONLY, APPLY_AND_FETCH, APPLY_AND_REBUILD, CANCEL]);
const result = await this.askSelectString("Ready for apply the setting.", [APPLY_AND_RESTART, APPLY_ONLY, APPLY_AND_FETCH, APPLY_AND_REBUILD, CANCEL]);
if (result == APPLY_ONLY || result == APPLY_AND_RESTART || result == APPLY_AND_REBUILD || result == APPLY_AND_FETCH) {
this.settings = settingToApply;
await this.saveSettingData();
@@ -1124,13 +1124,12 @@ Note: We can always able to read V1 format. It will be progressively converted.
return;
}
if (result == APPLY_AND_REBUILD) {
await this.app.vault.create(FLAGMD_REDFLAG2_HR, "");
await this.vaultAccess.vaultCreate(FLAGMD_REDFLAG2_HR, "");
}
if (result == APPLY_AND_FETCH) {
await this.app.vault.create(FLAGMD_REDFLAG3_HR, "");
await this.vaultAccess.vaultCreate(FLAGMD_REDFLAG3_HR, "");
}
// @ts-ignore
this.app.commands.executeCommandById("app:reload");
this.performAppReload();
}
}
)
@@ -1150,11 +1149,11 @@ Note: We can always able to read V1 format. It will be progressively converted.
async saveSettingToMarkdown(filename: string) {
const saveData = this.generateSettingForMarkdown();
let file = this.app.vault.getAbstractFileByPath(filename);
let file = this.vaultAccess.getAbstractFileByPath(filename);
if (!file) {
await this.ensureDirectoryEx(filename);
await this.vaultAccess.ensureDirectory(filename);
const initialContent = `This file contains Self-hosted LiveSync settings as YAML.
Except for the \`livesync-setting\` code block, we can add a note for free.
@@ -1167,21 +1166,21 @@ We can perform a command in this file.
`
file = await this.app.vault.create(filename, initialContent + SETTING_HEADER + "\n" + SETTING_FOOTER);
file = await this.vaultAccess.vaultCreate(filename, initialContent + SETTING_HEADER + "\n" + SETTING_FOOTER);
}
if (!(file instanceof TFile)) {
Logger(`Markdown Setting: ${filename} already exists as a folder`, LOG_LEVEL_NOTICE);
return;
}
const data = await this.app.vault.read(file);
const data = await this.vaultAccess.vaultRead(file);
const { preamble, body, postscript } = this.extractSettingFromWholeText(data);
const newBody = stringifyYaml(saveData);
if (newBody == body) {
Logger("Markdown setting: Nothing had been changed", LOG_LEVEL_VERBOSE);
} else {
await this.app.vault.modify(file, preamble + SETTING_HEADER + newBody + SETTING_FOOTER + postscript);
await this.vaultAccess.vaultModify(file, preamble + SETTING_HEADER + newBody + SETTING_FOOTER + postscript);
Logger(`Markdown setting: ${filename} has been updated!`, LOG_LEVEL_VERBOSE);
}
}
@@ -1224,8 +1223,7 @@ We can perform a command in this file.
const _this = this;
//@ts-ignore
window.CodeMirrorAdapter.commands.save = () => {
//@ts-ignore
_this.app.commands.executeCommandById('editor:save-file');
_this.performCommand('editor:save-file');
};
}
registerWatchEvents() {
@@ -1235,7 +1233,6 @@ We can perform a command in this file.
this.registerDomEvent(window, "offline", this.watchOnline);
}
watchOnline() {
scheduleTask("watch-online", 500, () => fireAndForget(() => this.watchOnlineAsync()));
}
@@ -1456,9 +1453,9 @@ We can perform a command in this file.
const logDate = `${PREFIXMD_LOGFILE}${time}.md`;
const file = this.vaultAccess.getAbstractFileByPath(normalizePath(logDate));
if (!file) {
this.app.vault.adapter.append(normalizePath(logDate), "```\n");
this.vaultAccess.adapterAppend(normalizePath(logDate), "```\n");
}
this.app.vault.adapter.append(normalizePath(logDate), vaultName + ":" + newMessage + "\n");
this.vaultAccess.adapterAppend(normalizePath(logDate), vaultName + ":" + newMessage + "\n");
}
recentLogProcessor.enqueue(newMessage);
@@ -1496,28 +1493,6 @@ We can perform a command in this file.
})
}
}
async ensureDirectory(fullPath: string) {
const pathElements = fullPath.split("/");
pathElements.pop();
let c = "";
for (const v of pathElements) {
c += v;
try {
await this.app.vault.createFolder(c);
} catch (ex) {
// basically skip exceptions.
if (ex.message && ex.message == "Folder already exists.") {
// especially this message is.
} else {
Logger("Folder Create Error");
Logger(ex);
}
}
c += "/";
}
}
async processEntryDoc(docEntry: EntryBody, file: TFile | undefined, force?: boolean) {
const mode = file == undefined ? "create" : "modify";
@@ -1577,7 +1552,7 @@ We can perform a command in this file.
return;
}
const writeData = doc.datatype == "newnote" ? decodeBinary(doc.data) : getDocData(doc.data);
await this.ensureDirectoryEx(path);
await this.vaultAccess.ensureDirectory(path);
try {
let outFile;
let isChanged = true;
@@ -1592,7 +1567,7 @@ We can perform a command in this file.
if (isChanged) {
Logger(msg + path);
this.vaultAccess.touch(outFile);
this.app.vault.trigger(mode, outFile);
this.vaultAccess.trigger(mode, outFile);
} else {
Logger(msg + "Skipped, the file is the same: " + path, LOG_LEVEL_VERBOSE);
}
@@ -1626,7 +1601,7 @@ We can perform a command in this file.
queueConflictCheck(file: FilePathWithPrefix | TFile) {
const path = file instanceof TFile ? getPathFromTFile(file) : file;
if (this.settings.checkConflictOnlyOnOpen) {
const af = this.app.workspace.getActiveFile();
const af = this.getActiveFile();
if (af && af.path != path) {
Logger(`${file} is conflicted, merging process has been postponed.`, LOG_LEVEL_NOTICE);
return;
@@ -1656,7 +1631,7 @@ We can perform a command in this file.
}
databaseQueueCount = reactiveSource(0);
databaseQueuedProcessor = new KeyedQueueProcessor(async (docs: EntryBody[]) => {
databaseQueuedProcessor = new QueueProcessor(async (docs: EntryBody[]) => {
const dbDoc = docs[0];
const path = this.getPath(dbDoc);
// If `Read chunks online` is disabled, chunks should be transferred before here.
@@ -1733,7 +1708,7 @@ We can perform a command in this file.
Logger(`Processing ${change.path} has been skipped due to file size exceeding the limit`, LOG_LEVEL_NOTICE);
return;
}
this.databaseQueuedProcessor.enqueueWithKey(change.path, change);
this.databaseQueuedProcessor.enqueue(change);
}
return;
}, { batchSize: 1, suspended: true, concurrentLimit: 100, delay: 0, totalRemainingReactiveSource: this.replicationResultCount }).startPipeline().onUpdateProgress(() => {
@@ -2036,7 +2011,7 @@ Or if you are sure know what had been happened, we can unlock the database from
await this.collectDeletedFiles();
Logger("Collecting local files on the storage", LOG_LEVEL_VERBOSE);
const filesStorageSrc = this.app.vault.getFiles();
const filesStorageSrc = this.vaultAccess.getFiles();
const filesStorage = [] as typeof filesStorageSrc;
for (const f of filesStorageSrc) {
@@ -2534,7 +2509,7 @@ Or if you are sure know what had been happened, we can unlock the database from
return;
}
if (this.settings.showMergeDialogOnlyOnActive) {
const af = this.app.workspace.getActiveFile();
const af = this.getActiveFile();
if (af && af.path != filename) {
Logger(`${filename} is conflicted. Merging process has been postponed to the file have got opened.`, LOG_LEVEL_NOTICE);
return;
@@ -2820,27 +2795,6 @@ Or if you are sure know what had been happened, we can unlock the database from
await this.replicator.tryCreateRemoteDatabase(this.settings);
}
async ensureDirectoryEx(fullPath: string) {
const pathElements = fullPath.split("/");
pathElements.pop();
let c = "";
for (const v of pathElements) {
c += v;
try {
await this.app.vault.adapter.mkdir(c);
} catch (ex) {
// basically skip exceptions.
if (ex.message && ex.message == "Folder already exists.") {
// especially this message is.
} else {
Logger("Folder Create Error");
Logger(ex);
}
}
c += "/";
}
}
filterTargetFiles(files: InternalFileInfo[], targetFiles: string[] | false = false) {
const ignorePatterns = this.settings.syncInternalFilesIgnorePatterns
.replace(/\n| /g, "")
@@ -2949,6 +2903,12 @@ Or if you are sure know what had been happened, we can unlock the database from
});
}
askYesNo(message: string): Promise<"yes" | "no"> {
return askYesNo(this.app, message);
}
askSelectString(message: string, items: string[]): Promise<string> {
return askSelectString(this.app, message, items);
}
askInPopup(key: string, dialogText: string, anchorCallback: (anchor: HTMLAnchorElement) => void) {
@@ -2967,7 +2927,6 @@ Or if you are sure know what had been happened, we can unlock the database from
const popupKey = "popup-" + key;
scheduleTask(popupKey, 1000, async () => {
const popup = await memoIfNotExist(popupKey, () => new Notice(fragment, 0));
//@ts-ignore
const isShown = popup?.noticeEl?.isShown();
if (!isShown) {
memoObject(popupKey, new Notice(fragment, 0));
@@ -2976,7 +2935,6 @@ Or if you are sure know what had been happened, we can unlock the database from
const popup = retrieveMemoObject<Notice>(popupKey);
if (!popup)
return;
//@ts-ignore
if (popup?.noticeEl?.isShown()) {
popup.hide();
}
@@ -2984,5 +2942,19 @@ Or if you are sure know what had been happened, we can unlock the database from
});
});
}
openSetting() {
//@ts-ignore: undocumented api
this.app.setting.open();
//@ts-ignore: undocumented api
this.app.setting.openTabById("obsidian-livesync");
}
performAppReload() {
this.performCommand("app:reload");
}
performCommand(id: string) {
// @ts-ignore
this.app.commands.executeCommandById(id)
}
}

View File

@@ -405,10 +405,13 @@ export const requestToCouchDB = async (baseUri: string, username: string, passwo
return await _requestToCouchDB(baseUri, username, password, origin, uri, body, method);
};
export async function performRebuildDB(plugin: ObsidianLiveSyncPlugin, method: "localOnly" | "remoteOnly" | "rebuildBothByThisDevice") {
export async function performRebuildDB(plugin: ObsidianLiveSyncPlugin, method: "localOnly" | "remoteOnly" | "rebuildBothByThisDevice" | "localOnlyWithChunks") {
if (method == "localOnly") {
await plugin.addOnSetup.fetchLocal();
}
if (method == "localOnlyWithChunks") {
await plugin.addOnSetup.fetchLocal(true);
}
if (method == "remoteOnly") {
await plugin.addOnSetup.rebuildRemote();
}

View File

@@ -10,6 +10,42 @@ Note: we got a very performance improvement.
Note at 0.22.2: **Now, to rescue mobile devices, Maximum file size is set to 50 by default**. Please configure the limit as you need. If you do not want to limit the sizes, set zero manually, please.
#### Version history
- 0.22.13:
- Improved:
- Now using HTTP for the remote database URI warns of an error (on mobile) or notice (on desktop).
- Refactored:
- Dependencies have been polished.
- 0.22.12:
- Changed:
- The default settings has been changed.
- Improved:
- Default and preferred settings are applied on completion of the wizard.
- Fixed:
- Now Initialisation `Fetch` will be performed smoothly and there will be fewer conflicts.
- No longer stuck while Handling transferred or initialised documents.
- 0.22.11:
- Fixed:
- `Verify and repair all files` is no longer broken.
- New feature:
- Now `Verify and repair all files` is able to...
- Restore if the file only in the local database.
- Show the history.
- Improved:
- Performance improved.
- 0.22.10
- Fixed:
- No longer unchanged hidden files and customisations are saved and transferred now.
- File integrity of vault history indicates the integrity correctly.
- Improved:
- In the report, the schema of the remote database URI is now printed.
- 0.22.9
- Fixed:
- Fixed a bug on `fetch chunks on demand` that could not fetch the chunks on demand.
- Improved:
- `fetch chunks on demand` works more smoothly.
- Initialisation `Fetch` is now more efficient.
- Tidied:
- Removed some meaningless codes.
- 0.22.8
- Fixed:
- Now fetch and unlock the locked remote database works well again.
@@ -26,104 +62,4 @@ Note at 0.22.2: **Now, to rescue mobile devices, Maximum file size is set to 50
- Changed:
- Now no longer `fetch chunks on demand` needs `Pacing replication`
- The setting `Do not pace synchronization` has been deleted.
- 0.22.7
- Fixed:
- No longer deleted hidden files were ignored.
- The document history dialogue is now able to process the deleted revisions.
- Deletion of a hidden file is now surely performed even if the file is already conflicted.
- 0.22.6
- Fixed:
- Fixed a problem with synchronisation taking a long time to start in some cases.
- The first synchronisation after update might take a bit longer.
- Now we can disable E2EE encryption.
- Improved:
- `Setup Wizard` is now more clear.
- `Minimal Setup` is now more simple.
- Self-hosted LiveSync now be able to use even if there are vaults with the same name.
- Database suffix will automatically added.
- Now Self-hosted LiveSync waits until set-up is complete.
- Show reload prompts when possibly recommended while settings.
- New feature:
- A guidance dialogue prompting for settings will be shown after the installation.
- Changed
- `Open setup URI` is now `Use the copied setup URI`
- `Copy setup URI` is now `Copy current settings as a new setup URI`
- `Setup Wizard` is now `Minimal Setup`
- `Check database configuration` is now `Check and Fix database configuration`
- 0.22.5
- Fixed:
- Some description of settings have been refined
- New feature:
- TroubleShooting is now shown in the setting dialogue.
- 0.22.4
- Fixed:
- Now the result of conflict resolution could be surely written into the storage.
- Deleted files can be handled correctly again in the history dialogue and conflict dialogue.
- Some wrong log messages were fixed.
- Change handling now has become more stable.
- Some event handling became to be safer.
- Improved:
- Dumping document information shows conflicts and revisions.
- The timestamp-only differences can be surely cached.
- Timestamp difference detection can be rounded by two seconds.
- Refactored:
- A bit of organisation to write the test.
- 0.22.3
- Fixed:
- No longer detects storage changes which have been caused by Self-hosted LiveSync itself.
- Setting sync file will be detected only if it has been configured now.
- And its log will be shown only while the verbose log is enabled.
- Customisation file enumeration has got less blingy.
- Deletion of files is now reliably synchronised.
- Fixed and improved:
- In-editor-status is now shown in the following areas:
- Note editing pane (Source mode and live-preview mode).
- New tab pane.
- Canvas pane.
- 0.22.2
- Fixed:
- Now the results of resolving conflicts are surely synchronised.
- Modified:
- Some setting items got new clear names. (`Sync Settings` -> `Targets`).
- New feature:
- We can limit the synchronising files by their size. (`Sync Settings` -> `Targets` -> `Maximum file size`).
- It depends on the size of the newer one.
- At Obsidian 1.5.3 on mobile, we should set this to around 50MB to avoid restarting Obsidian.
- Now the settings could be stored in a specific markdown file to synchronise or switch it (`General Setting` -> `Share settings via markdown`).
- [Screwdriver](https://github.com/vrtmrz/obsidian-screwdriver) is quite good, but mostly we only need this.
- Customisation of the obsoleted device is now able to be deleted at once.
- We have to put the maintenance mode in at the Customisation sync dialogue.
- 0.22.1
- New feature:
- We can perform automatic conflict resolution for inactive files, and postpone only manual ones by `Postpone manual resolution of inactive files`.
- Now we can see the image in the document history dialogue.
- We can see the difference of the image, in the document history dialogue.
- And also we can highlight differences.
- Improved:
- Hidden file sync has been stabilised.
- Now automatically reloads the conflict-resolution dialogue when new conflicted revisions have arrived.
- Fixed:
- No longer periodic process runs after unloading the plug-in.
- Now the modification of binary files is surely stored in the storage.
- 0.22.0
- Refined:
- Task scheduling logics has been rewritten.
- Screen updates are also now efficient.
- Possibly many bugs and fragile behaviour has been fixed.
- Status updates and logging have been thinned out to display.
- Fixed:
- Remote-chunk-fetching now works with keeping request intervals
- New feature:
- We can show only the icons in the editor.
- Progress indicators have been more meaningful:
- 📥 Unprocessed transferred items
- 📄 Working database operation
- 💾 Working write storage processes
- ⏳ Working read storage processes
- 🛫 Pending read storage processes
- ⚙️ Working or pending storage processes of hidden files
- 🧩 Waiting chunks
- 🔌 Working Customisation items (Configuration, snippets and plug-ins)
... To continue on to `updates_old.md`.

View File

@@ -1,3 +1,117 @@
### 0.22.0
A few years passed since Self-hosted LiveSync was born, and our codebase had been very complicated. This could be patient now, but it should be a tremendous hurt.
Therefore at v0.22.0, for future maintainability, I refined task scheduling logic totally.
Of course, I think this would be our suffering in some cases. However, I would love to ask you for your cooperation and contribution.
Sorry for being absent so much long. And thank you for your patience!
Note: we got a very performance improvement.
Note at 0.22.2: **Now, to rescue mobile devices, Maximum file size is set to 50 by default**. Please configure the limit as you need. If you do not want to limit the sizes, set zero manually, please.
#### Version history
- 0.22.7
- Fixed:
- No longer deleted hidden files were ignored.
- The document history dialogue is now able to process the deleted revisions.
- Deletion of a hidden file is now surely performed even if the file is already conflicted.
- 0.22.6
- Fixed:
- Fixed a problem with synchronisation taking a long time to start in some cases.
- The first synchronisation after update might take a bit longer.
- Now we can disable E2EE encryption.
- Improved:
- `Setup Wizard` is now more clear.
- `Minimal Setup` is now more simple.
- Self-hosted LiveSync now be able to use even if there are vaults with the same name.
- Database suffix will automatically added.
- Now Self-hosted LiveSync waits until set-up is complete.
- Show reload prompts when possibly recommended while settings.
- New feature:
- A guidance dialogue prompting for settings will be shown after the installation.
- Changed
- `Open setup URI` is now `Use the copied setup URI`
- `Copy setup URI` is now `Copy current settings as a new setup URI`
- `Setup Wizard` is now `Minimal Setup`
- `Check database configuration` is now `Check and Fix database configuration`
- 0.22.5
- Fixed:
- Some description of settings have been refined
- New feature:
- TroubleShooting is now shown in the setting dialogue.
- 0.22.4
- Fixed:
- Now the result of conflict resolution could be surely written into the storage.
- Deleted files can be handled correctly again in the history dialogue and conflict dialogue.
- Some wrong log messages were fixed.
- Change handling now has become more stable.
- Some event handling became to be safer.
- Improved:
- Dumping document information shows conflicts and revisions.
- The timestamp-only differences can be surely cached.
- Timestamp difference detection can be rounded by two seconds.
- Refactored:
- A bit of organisation to write the test.
- 0.22.3
- Fixed:
- No longer detects storage changes which have been caused by Self-hosted LiveSync itself.
- Setting sync file will be detected only if it has been configured now.
- And its log will be shown only while the verbose log is enabled.
- Customisation file enumeration has got less blingy.
- Deletion of files is now reliably synchronised.
- Fixed and improved:
- In-editor-status is now shown in the following areas:
- Note editing pane (Source mode and live-preview mode).
- New tab pane.
- Canvas pane.
- 0.22.2
- Fixed:
- Now the results of resolving conflicts are surely synchronised.
- Modified:
- Some setting items got new clear names. (`Sync Settings` -> `Targets`).
- New feature:
- We can limit the synchronising files by their size. (`Sync Settings` -> `Targets` -> `Maximum file size`).
- It depends on the size of the newer one.
- At Obsidian 1.5.3 on mobile, we should set this to around 50MB to avoid restarting Obsidian.
- Now the settings could be stored in a specific markdown file to synchronise or switch it (`General Setting` -> `Share settings via markdown`).
- [Screwdriver](https://github.com/vrtmrz/obsidian-screwdriver) is quite good, but mostly we only need this.
- Customisation of the obsoleted device is now able to be deleted at once.
- We have to put the maintenance mode in at the Customisation sync dialogue.
- 0.22.1
- New feature:
- We can perform automatic conflict resolution for inactive files, and postpone only manual ones by `Postpone manual resolution of inactive files`.
- Now we can see the image in the document history dialogue.
- We can see the difference of the image, in the document history dialogue.
- And also we can highlight differences.
- Improved:
- Hidden file sync has been stabilised.
- Now automatically reloads the conflict-resolution dialogue when new conflicted revisions have arrived.
- Fixed:
- No longer periodic process runs after unloading the plug-in.
- Now the modification of binary files is surely stored in the storage.
- 0.22.0
- Refined:
- Task scheduling logics has been rewritten.
- Screen updates are also now efficient.
- Possibly many bugs and fragile behaviour has been fixed.
- Status updates and logging have been thinned out to display.
- Fixed:
- Remote-chunk-fetching now works with keeping request intervals
- New feature:
- We can show only the icons in the editor.
- Progress indicators have been more meaningful:
- 📥 Unprocessed transferred items
- 📄 Working database operation
- 💾 Working write storage processes
- ⏳ Working read storage processes
- 🛫 Pending read storage processes
- ⚙️ Working or pending storage processes of hidden files
- 🧩 Waiting chunks
- 🔌 Working Customisation items (Configuration, snippets and plug-ins)
... To continue on to `updates_old.md`.
### 0.21.0
The E2EE encryption V2 format has been reverted. That was probably the cause of the glitch.
Instead, to maintain efficiency, files are treated with Blob until just before saving. Along with this, the old-fashioned encryption format has also been discontinued.