Compare commits

...

14 Commits

Author SHA1 Message Date
vorotamoroz
b9527ccab0 bump 2024-01-30 17:31:52 +00:00
vorotamoroz
fa3aa2702c Fixed:
- Now the result of conflict resolution could be surely written into the storage.
- Deleted files can be handled correctly again in the history dialogue and conflict dialogue.
- Some wrong log messages were fixed.
- Change handling now has become more stable.
- Some event handling became to be safer.
Improved:
- Dumping document information shows conflicts and revisions.
- The timestamp-only differences can be surely cached.
- Timestamp difference detection can be rounded by two seconds.
Refactored:
- A bit of organisation to write the test.
2024-01-30 17:31:02 +00:00
vorotamoroz
93e7cbb133 bump. 2024-01-29 08:41:03 +00:00
vorotamoroz
716ae32e02 Fixed:
- Deletion of files is now reliably synchronised.
2024-01-29 08:40:41 +00:00
vorotamoroz
d6d8cbcf5a bump 2024-01-29 07:57:02 +00:00
vorotamoroz
efd348b266 Fixed:
- No longer detects storage changes which have been caused by Self-hosted LiveSync itself.
- Setting sync file will be detected only if it has been configured now.
  - And its log will be shown only while the verbose log is enabled.
- Customisation file enumeration has got less blingy.
Fixed and improved:
- In-editor-status is now shown in the following areas:
  - Note editing pane (Source mode and live-preview mode).
  - New tab pane.
  - Canvas pane.
2024-01-29 07:56:02 +00:00
vorotamoroz
8969b1800a bump 2024-01-24 08:53:00 +00:00
vorotamoroz
2c8e026e29 Fixed:
- Now the results of resolving conflicts are surely synchronised.
Modified:
- Some setting items got new clear names.
New feature:
- We can limit the synchronising files by their size.
- Now the settings could be stored in a specific markdown file to synchronise or switch it
- Customisation of the obsoleted device is now able to be deleted at once.
2024-01-24 08:52:47 +00:00
vorotamoroz
a6c27eab3d Merge pull request #367 from calvinbui/patch-1
Skip workspace-mobile.json for cross-platform sync
2024-01-24 15:55:50 +09:00
vorotamoroz
9b5c57d540 Merge pull request #336 from toon159/patch-1
Lower payload size limit and batch limit from 10 to 2
2024-01-22 13:03:29 +09:00
Calvin Bui
c251c596e8 Skip workspace-mobile.json for cross-platform sync 2024-01-18 13:23:48 +11:00
vorotamoroz
61188cfaef bump 2024-01-16 08:36:37 +00:00
vorotamoroz
97d944fd75 New feature:
- We can perform automatic conflict resolution for inactive files, and postpone only manual ones by `Postpone manual resolution of inactive files`.
- Now we can see the image in the document history dialogue.
  - We can see the difference of the image, in the document history dialogue.
	- And also we can highlight differences.

Improved:
- Hidden file sync has been stabilised.
- Now automatically reloads the conflict-resolution dialogue when new conflicted revisions have arrived.

Fixed:
- No longer periodic process runs after unloading the plug-in.
- Now the modification of binary files is surely stored in the storage.
2024-01-16 08:32:43 +00:00
Vichaya Raksakunpanich
5802ed31be Update ObsidianLiveSyncSettingTab.ts
Lower payload size limit and batch limit to 2 due to IBM Cloudant read/write limitation. Hope it will fix the "Replication error".
2023-11-24 16:25:16 +07:00
18 changed files with 1078 additions and 450 deletions

View File

@@ -1,7 +1,7 @@
{ {
"id": "obsidian-livesync", "id": "obsidian-livesync",
"name": "Self-hosted LiveSync", "name": "Self-hosted LiveSync",
"version": "0.22.0", "version": "0.22.4",
"minAppVersion": "0.9.12", "minAppVersion": "0.9.12",
"description": "Community implementation of self-hosted livesync. Reflect your vault changes to some other devices immediately. Please make sure to disable other synchronize solutions to avoid content corruption or duplication.", "description": "Community implementation of self-hosted livesync. Reflect your vault changes to some other devices immediately. Please make sure to disable other synchronize solutions to avoid content corruption or duplication.",
"author": "vorotamoroz", "author": "vorotamoroz",

4
package-lock.json generated
View File

@@ -1,12 +1,12 @@
{ {
"name": "obsidian-livesync", "name": "obsidian-livesync",
"version": "0.22.0", "version": "0.22.4",
"lockfileVersion": 2, "lockfileVersion": 2,
"requires": true, "requires": true,
"packages": { "packages": {
"": { "": {
"name": "obsidian-livesync", "name": "obsidian-livesync",
"version": "0.22.0", "version": "0.22.4",
"license": "MIT", "license": "MIT",
"dependencies": { "dependencies": {
"diff-match-patch": "^1.0.5", "diff-match-patch": "^1.0.5",

View File

@@ -1,6 +1,6 @@
{ {
"name": "obsidian-livesync", "name": "obsidian-livesync",
"version": "0.22.0", "version": "0.22.4",
"description": "Reflect your vault changes to some other devices immediately. Please make sure to disable other synchronize solutions to avoid content corruption or duplication.", "description": "Reflect your vault changes to some other devices immediately. Please make sure to disable other synchronize solutions to avoid content corruption or duplication.",
"main": "main.js", "main": "main.js",
"type": "module", "type": "module",

View File

@@ -4,7 +4,7 @@ import { Notice, type PluginManifest, parseYaml, normalizePath } from "./deps";
import type { EntryDoc, LoadedEntry, InternalFileEntry, FilePathWithPrefix, FilePath, DocumentID, AnyEntry, SavingEntry } from "./lib/src/types"; import type { EntryDoc, LoadedEntry, InternalFileEntry, FilePathWithPrefix, FilePath, DocumentID, AnyEntry, SavingEntry } from "./lib/src/types";
import { LOG_LEVEL_INFO, LOG_LEVEL_NOTICE, LOG_LEVEL_VERBOSE, MODE_SELECTIVE } from "./lib/src/types"; import { LOG_LEVEL_INFO, LOG_LEVEL_NOTICE, LOG_LEVEL_VERBOSE, MODE_SELECTIVE } from "./lib/src/types";
import { ICXHeader, PERIODIC_PLUGIN_SWEEP, } from "./types"; import { ICXHeader, PERIODIC_PLUGIN_SWEEP, } from "./types";
import { createTextBlob, delay, getDocData } from "./lib/src/utils"; import { createTextBlob, delay, getDocData, sendSignal, waitForSignal } from "./lib/src/utils";
import { Logger } from "./lib/src/logger"; import { Logger } from "./lib/src/logger";
import { WrappedNotice } from "./lib/src/wrapper"; import { WrappedNotice } from "./lib/src/wrapper";
import { readString, decodeBinary, arrayBufferToBase64, sha1 } from "./lib/src/strbin"; import { readString, decodeBinary, arrayBufferToBase64, sha1 } from "./lib/src/strbin";
@@ -335,7 +335,9 @@ export class ConfigSync extends LiveSyncCommands {
return; return;
}, { suspended: true, batchSize: 1, concurrentLimit: 5, delay: 300, yieldThreshold: 10 }).pipeTo( }, { suspended: true, batchSize: 1, concurrentLimit: 5, delay: 300, yieldThreshold: 10 }).pipeTo(
new QueueProcessor( new QueueProcessor(
(pluginDataList) => { async (pluginDataList) => {
// Concurrency is two, therefore, we can unlock the previous awaiting.
sendSignal("plugin-next-load");
let newList = [...this.pluginList]; let newList = [...this.pluginList];
for (const item of pluginDataList) { for (const item of pluginDataList) {
newList = newList.filter(x => x.documentPath != item.documentPath); newList = newList.filter(x => x.documentPath != item.documentPath);
@@ -343,9 +345,13 @@ export class ConfigSync extends LiveSyncCommands {
} }
this.pluginList = newList; this.pluginList = newList;
pluginList.set(newList); pluginList.set(newList);
if (pluginDataList.length != 10) {
// If the queue is going to be empty, await subsequent for a while.
await waitForSignal("plugin-next-load", 1000);
}
return; return;
} }
, { suspended: true, batchSize: 1000, concurrentLimit: 10, delay: 200, yieldThreshold: 25, totalRemainingReactiveSource: pluginScanningCount })).startPipeline().root.onIdle(() => { , { suspended: true, batchSize: 10, concurrentLimit: 2, delay: 250, yieldThreshold: 25, totalRemainingReactiveSource: pluginScanningCount })).startPipeline().root.onIdle(() => {
Logger(`All files enumerated`, LOG_LEVEL_INFO, "get-plugins"); Logger(`All files enumerated`, LOG_LEVEL_INFO, "get-plugins");
this.createMissingConfigurationEntry(); this.createMissingConfigurationEntry();
}); });

View File

@@ -1,7 +1,7 @@
import { normalizePath, type PluginManifest } from "./deps"; import { normalizePath, type PluginManifest } from "./deps";
import { type EntryDoc, type LoadedEntry, type InternalFileEntry, type FilePathWithPrefix, type FilePath, LOG_LEVEL_INFO, LOG_LEVEL_NOTICE, LOG_LEVEL_VERBOSE, MODE_SELECTIVE, MODE_PAUSED, type SavingEntry } from "./lib/src/types"; import { type EntryDoc, type LoadedEntry, type InternalFileEntry, type FilePathWithPrefix, type FilePath, LOG_LEVEL_INFO, LOG_LEVEL_NOTICE, LOG_LEVEL_VERBOSE, MODE_SELECTIVE, MODE_PAUSED, type SavingEntry } from "./lib/src/types";
import { type InternalFileInfo, ICHeader, ICHeaderEnd } from "./types"; import { type InternalFileInfo, ICHeader, ICHeaderEnd } from "./types";
import { createBinaryBlob, delay, isDocContentSame } from "./lib/src/utils"; import { createBinaryBlob, isDocContentSame, sendSignal } from "./lib/src/utils";
import { Logger } from "./lib/src/logger"; import { Logger } from "./lib/src/logger";
import { PouchDB } from "./lib/src/pouchdb-browser.js"; import { PouchDB } from "./lib/src/pouchdb-browser.js";
import { isInternalMetadata, PeriodicProcessor } from "./utils"; import { isInternalMetadata, PeriodicProcessor } from "./utils";
@@ -86,7 +86,7 @@ export class HiddenFileSync extends LiveSyncCommands {
await this.syncInternalFilesAndDatabase("pull", false, false, filenames); await this.syncInternalFilesAndDatabase("pull", false, false, filenames);
Logger(`DONE :Applying hidden ${filenames.length} files change`, LOG_LEVEL_VERBOSE); Logger(`DONE :Applying hidden ${filenames.length} files change`, LOG_LEVEL_VERBOSE);
return; return;
}, { batchSize: 100, concurrentLimit: 1, delay: 100, yieldThreshold: 10, suspended: false, totalRemainingReactiveSource: hiddenFilesEventCount } }, { batchSize: 100, concurrentLimit: 1, delay: 10, yieldThreshold: 10, suspended: false, totalRemainingReactiveSource: hiddenFilesEventCount }
); );
recentProcessedInternalFiles = [] as string[]; recentProcessedInternalFiles = [] as string[];
@@ -134,25 +134,34 @@ export class HiddenFileSync extends LiveSyncCommands {
async resolveConflictOnInternalFiles() { async resolveConflictOnInternalFiles() {
// Scan all conflicted internal files // Scan all conflicted internal files
const conflicted = this.localDatabase.findEntries(ICHeader, ICHeaderEnd, { conflicts: true }); const conflicted = this.localDatabase.findEntries(ICHeader, ICHeaderEnd, { conflicts: true });
for await (const doc of conflicted) { this.conflictResolutionProcessor.suspend();
if (!("_conflicts" in doc)) try {
continue; for await (const doc of conflicted) {
if (isInternalMetadata(doc._id)) { if (!("_conflicts" in doc))
await this.resolveConflictOnInternalFile(doc.path); continue;
if (isInternalMetadata(doc._id)) {
this.conflictResolutionProcessor.enqueue(doc.path);
}
} }
} catch (ex) {
Logger("something went wrong on resolving all conflicted internal files");
Logger(ex, LOG_LEVEL_VERBOSE);
} }
await this.conflictResolutionProcessor.startPipeline().waitForPipeline();
} }
async resolveConflictOnInternalFile(path: FilePathWithPrefix): Promise<boolean> { conflictResolutionProcessor = new QueueProcessor(async (paths: FilePathWithPrefix[]) => {
const path = paths[0];
sendSignal(`cancel-internal-conflict:${path}`);
try { try {
// Retrieve data // Retrieve data
const id = await this.path2id(path, ICHeader); const id = await this.path2id(path, ICHeader);
const doc = await this.localDatabase.getRaw(id, { conflicts: true }); const doc = await this.localDatabase.getRaw(id, { conflicts: true });
// If there is no conflict, return with false. // If there is no conflict, return with false.
if (!("_conflicts" in doc)) if (!("_conflicts" in doc))
return false; return;
if (doc._conflicts.length == 0) if (doc._conflicts.length == 0)
return false; return;
Logger(`Hidden file conflicted:${path}`); Logger(`Hidden file conflicted:${path}`);
const conflicts = doc._conflicts.sort((a, b) => Number(a.split("-")[0]) - Number(b.split("-")[0])); const conflicts = doc._conflicts.sort((a, b) => Number(a.split("-")[0]) - Number(b.split("-")[0]));
const revA = doc._rev; const revA = doc._rev;
@@ -177,21 +186,12 @@ export class HiddenFileSync extends LiveSyncCommands {
await this.storeInternalFileToDatabase({ path: filename, ...stat }); await this.storeInternalFileToDatabase({ path: filename, ...stat });
await this.extractInternalFileFromDatabase(filename); await this.extractInternalFileFromDatabase(filename);
await this.localDatabase.removeRaw(id, revB); await this.localDatabase.removeRaw(id, revB);
return this.resolveConflictOnInternalFile(path); this.conflictResolutionProcessor.enqueue(path);
return;
} else { } else {
Logger(`Object merge is not applicable.`, LOG_LEVEL_VERBOSE); Logger(`Object merge is not applicable.`, LOG_LEVEL_VERBOSE);
} }
return [{ path, revA, revB }];
const docAMerge = await this.localDatabase.getDBEntry(path, { rev: revA });
const docBMerge = await this.localDatabase.getDBEntry(path, { rev: revB });
if (docAMerge != false && docBMerge != false) {
if (await this.showJSONMergeDialogAndMerge(docAMerge, docBMerge)) {
await delay(200);
// Again for other conflicted revisions.
return this.resolveConflictOnInternalFile(path);
}
return false;
}
} }
const revBDoc = await this.localDatabase.getRaw(id, { rev: revB }); const revBDoc = await this.localDatabase.getRaw(id, { rev: revB });
// determine which revision should been deleted. // determine which revision should been deleted.
@@ -205,12 +205,31 @@ export class HiddenFileSync extends LiveSyncCommands {
await this.localDatabase.removeRaw(id, delRev); await this.localDatabase.removeRaw(id, delRev);
Logger(`Older one has been deleted:${path}`); Logger(`Older one has been deleted:${path}`);
// check the file again // check the file again
return this.resolveConflictOnInternalFile(path); this.conflictResolutionProcessor.enqueue(path);
return;
} catch (ex) { } catch (ex) {
Logger(`Failed to resolve conflict (Hidden): ${path}`); Logger(`Failed to resolve conflict (Hidden): ${path}`);
Logger(ex, LOG_LEVEL_VERBOSE); Logger(ex, LOG_LEVEL_VERBOSE);
return false; return;
} }
}, {
suspended: false, batchSize: 1, concurrentLimit: 5, delay: 10, keepResultUntilDownstreamConnected: true, yieldThreshold: 10,
pipeTo: new QueueProcessor(async (results) => {
const { path, revA, revB } = results[0]
const docAMerge = await this.localDatabase.getDBEntry(path, { rev: revA });
const docBMerge = await this.localDatabase.getDBEntry(path, { rev: revB });
if (docAMerge != false && docBMerge != false) {
if (await this.showJSONMergeDialogAndMerge(docAMerge, docBMerge)) {
// Again for other conflicted revisions.
this.conflictResolutionProcessor.enqueue(path);
}
return;
}
}, { suspended: false, batchSize: 1, concurrentLimit: 1, delay: 10, keepResultUntilDownstreamConnected: false, yieldThreshold: 10 })
})
queueConflictCheck(path: FilePathWithPrefix) {
this.conflictResolutionProcessor.enqueue(path);
} }
//TODO: Tidy up. Even though it is experimental feature, So dirty... //TODO: Tidy up. Even though it is experimental feature, So dirty...
@@ -595,7 +614,7 @@ export class HiddenFileSync extends LiveSyncCommands {
showJSONMergeDialogAndMerge(docA: LoadedEntry, docB: LoadedEntry): Promise<boolean> { showJSONMergeDialogAndMerge(docA: LoadedEntry, docB: LoadedEntry): Promise<boolean> {
return serialized("conflict:merge-data", () => new Promise((res) => { return new Promise((res) => {
Logger("Opening data-merging dialog", LOG_LEVEL_VERBOSE); Logger("Opening data-merging dialog", LOG_LEVEL_VERBOSE);
const docs = [docA, docB]; const docs = [docA, docB];
const path = stripAllPrefixes(docA.path); const path = stripAllPrefixes(docA.path);
@@ -649,7 +668,7 @@ export class HiddenFileSync extends LiveSyncCommands {
} }
}); });
modal.open(); modal.open();
})); });
} }
async scanInternalFiles(): Promise<InternalFileInfo[]> { async scanInternalFiles(): Promise<InternalFileInfo[]> {

View File

@@ -1,23 +1,41 @@
import { App, Modal } from "./deps"; import { App, Modal } from "./deps";
import { DIFF_DELETE, DIFF_EQUAL, DIFF_INSERT } from "diff-match-patch"; import { DIFF_DELETE, DIFF_EQUAL, DIFF_INSERT } from "diff-match-patch";
import { type diff_result } from "./lib/src/types"; import { CANCELLED, LEAVE_TO_SUBSEQUENT, RESULT_TIMED_OUT, type diff_result } from "./lib/src/types";
import { escapeStringToHTML } from "./lib/src/strbin"; import { escapeStringToHTML } from "./lib/src/strbin";
import { delay, sendValue, waitForValue } from "./lib/src/utils";
export type MergeDialogResult = typeof LEAVE_TO_SUBSEQUENT | typeof CANCELLED | string;
export class ConflictResolveModal extends Modal { export class ConflictResolveModal extends Modal {
// result: Array<[number, string]>;
result: diff_result; result: diff_result;
filename: string; filename: string;
callback: (remove_rev: string) => Promise<void>;
constructor(app: App, filename: string, diff: diff_result, callback: (remove_rev: string) => Promise<void>) { response: MergeDialogResult = CANCELLED;
isClosed = false;
consumed = false;
constructor(app: App, filename: string, diff: diff_result) {
super(app); super(app);
this.result = diff; this.result = diff;
this.callback = callback;
this.filename = filename; this.filename = filename;
// Send cancel signal for the previous merge dialogue
// if not there, simply be ignored.
// sendValue("close-resolve-conflict:" + this.filename, false);
sendValue("cancel-resolve-conflict:" + this.filename, true);
} }
onOpen() { onOpen() {
const { contentEl } = this; const { contentEl } = this;
// Send cancel signal for the previous merge dialogue
// if not there, simply be ignored.
sendValue("cancel-resolve-conflict:" + this.filename, true);
setTimeout(async () => {
const forceClose = await waitForValue("cancel-resolve-conflict:" + this.filename);
// debugger;
if (forceClose) {
this.sendResponse(CANCELLED);
}
}, 10)
// sendValue("close-resolve-conflict:" + this.filename, false);
this.titleEl.setText("Conflicting changes"); this.titleEl.setText("Conflicting changes");
contentEl.empty(); contentEl.empty();
contentEl.createEl("span", { text: this.filename }); contentEl.createEl("span", { text: this.filename });
@@ -44,42 +62,32 @@ export class ConflictResolveModal extends Modal {
div2.innerHTML = ` div2.innerHTML = `
<span class='deleted'>A:${date1}</span><br /><span class='added'>B:${date2}</span><br> <span class='deleted'>A:${date1}</span><br /><span class='added'>B:${date2}</span><br>
`; `;
contentEl.createEl("button", { text: "Keep A" }, (e) => { contentEl.createEl("button", { text: "Keep A" }, (e) => e.addEventListener("click", () => this.sendResponse(this.result.right.rev)));
e.addEventListener("click", async () => { contentEl.createEl("button", { text: "Keep B" }, (e) => e.addEventListener("click", () => this.sendResponse(this.result.left.rev)));
const callback = this.callback; contentEl.createEl("button", { text: "Concat both" }, (e) => e.addEventListener("click", () => this.sendResponse(LEAVE_TO_SUBSEQUENT)));
this.callback = null; contentEl.createEl("button", { text: "Not now" }, (e) => e.addEventListener("click", () => this.sendResponse(CANCELLED)));
this.close(); }
await callback(this.result.right.rev);
}); sendResponse(result: MergeDialogResult) {
}); this.response = result;
contentEl.createEl("button", { text: "Keep B" }, (e) => { this.close();
e.addEventListener("click", async () => {
const callback = this.callback;
this.callback = null;
this.close();
await callback(this.result.left.rev);
});
});
contentEl.createEl("button", { text: "Concat both" }, (e) => {
e.addEventListener("click", async () => {
const callback = this.callback;
this.callback = null;
this.close();
await callback("");
});
});
contentEl.createEl("button", { text: "Not now" }, (e) => {
e.addEventListener("click", () => {
this.close();
});
});
} }
onClose() { onClose() {
const { contentEl } = this; const { contentEl } = this;
contentEl.empty(); contentEl.empty();
if (this.callback != null) { if (this.consumed) {
this.callback(null); return;
} }
this.consumed = true;
sendValue("close-resolve-conflict:" + this.filename, this.response);
sendValue("cancel-resolve-conflict:" + this.filename, false);
}
async waitForResult(): Promise<MergeDialogResult> {
await delay(100);
const r = await waitForValue<MergeDialogResult>("close-resolve-conflict:" + this.filename);
if (r === RESULT_TIMED_OUT) return CANCELLED;
return r;
} }
} }

View File

@@ -6,8 +6,34 @@ import { type DocumentID, type FilePathWithPrefix, type LoadedEntry, LOG_LEVEL_I
import { Logger } from "./lib/src/logger"; import { Logger } from "./lib/src/logger";
import { isErrorOfMissingDoc } from "./lib/src/utils_couchdb"; import { isErrorOfMissingDoc } from "./lib/src/utils_couchdb";
import { getDocData } from "./lib/src/utils"; import { getDocData } from "./lib/src/utils";
import { stripPrefix } from "./lib/src/path"; import { isPlainText, stripPrefix } from "./lib/src/path";
function isImage(path: string) {
const ext = path.split(".").splice(-1)[0].toLowerCase();
return ["png", "jpg", "jpeg", "gif", "bmp", "webp"].includes(ext);
}
function isComparableText(path: string) {
const ext = path.split(".").splice(-1)[0].toLowerCase();
return isPlainText(path) || ["md", "mdx", "txt", "json"].includes(ext);
}
function isComparableTextDecode(path: string) {
const ext = path.split(".").splice(-1)[0].toLowerCase();
return ["json"].includes(ext)
}
function readDocument(w: LoadedEntry) {
if (isImage(w.path)) {
return new Uint8Array(decodeBinary(w.data));
}
if (w.data == "plain") return getDocData(w.data);
if (isComparableTextDecode(w.path)) return readString(new Uint8Array(decodeBinary(w.data)));
if (isComparableText(w.path)) return getDocData(w.data);
try {
return readString(new Uint8Array(decodeBinary(w.data)));
} catch (ex) {
// NO OP.
}
return getDocData(w.data);
}
export class DocumentHistoryModal extends Modal { export class DocumentHistoryModal extends Modal {
plugin: ObsidianLiveSyncPlugin; plugin: ObsidianLiveSyncPlugin;
range!: HTMLInputElement; range!: HTMLInputElement;
@@ -56,7 +82,6 @@ export class DocumentHistoryModal extends Modal {
this.range.max = "0"; this.range.max = "0";
this.range.value = ""; this.range.value = "";
this.range.disabled = true; this.range.disabled = true;
this.showDiff
this.contentView.setText(`History of this file was not recorded.`); this.contentView.setText(`History of this file was not recorded.`);
} else { } else {
this.contentView.setText(`Error occurred.`); this.contentView.setText(`Error occurred.`);
@@ -76,6 +101,22 @@ export class DocumentHistoryModal extends Modal {
const rev = this.revs_info[index]; const rev = this.revs_info[index];
await this.showExactRev(rev.rev); await this.showExactRev(rev.rev);
} }
BlobURLs = new Map<string, string>();
revokeURL(key: string) {
const v = this.BlobURLs.get(key);
if (v) {
URL.revokeObjectURL(v);
}
this.BlobURLs.set(key, undefined);
}
generateBlobURL(key: string, data: Uint8Array) {
this.revokeURL(key);
const v = URL.createObjectURL(new Blob([data], { endings: "transparent", type: "application/octet-stream" }));
this.BlobURLs.set(key, v);
return v;
}
async showExactRev(rev: string) { async showExactRev(rev: string) {
const db = this.plugin.localDatabase; const db = this.plugin.localDatabase;
const w = await db.getDBEntry(this.file, { rev: rev }, false, false, true); const w = await db.getDBEntry(this.file, { rev: rev }, false, false, true);
@@ -88,42 +129,67 @@ export class DocumentHistoryModal extends Modal {
} else { } else {
this.currentDoc = w; this.currentDoc = w;
this.info.innerHTML = `Modified:${new Date(w.mtime).toLocaleString()}`; this.info.innerHTML = `Modified:${new Date(w.mtime).toLocaleString()}`;
let result = ""; let result = undefined;
const w1data = w.datatype == "plain" ? getDocData(w.data) : readString(new Uint8Array(decodeBinary(w.data))); const w1data = readDocument(w);
this.currentDeleted = !!w.deleted; this.currentDeleted = !!w.deleted;
this.currentText = w1data; // this.currentText = w1data;
if (this.showDiff) { if (this.showDiff) {
const prevRevIdx = this.revs_info.length - 1 - ((this.range.value as any) / 1 - 1); const prevRevIdx = this.revs_info.length - 1 - ((this.range.value as any) / 1 - 1);
if (prevRevIdx >= 0 && prevRevIdx < this.revs_info.length) { if (prevRevIdx >= 0 && prevRevIdx < this.revs_info.length) {
const oldRev = this.revs_info[prevRevIdx].rev; const oldRev = this.revs_info[prevRevIdx].rev;
const w2 = await db.getDBEntry(this.file, { rev: oldRev }, false, false, true); const w2 = await db.getDBEntry(this.file, { rev: oldRev }, false, false, true);
if (w2 != false) { if (w2 != false) {
const dmp = new diff_match_patch(); if (typeof w1data == "string") {
const w2data = w2.datatype == "plain" ? getDocData(w2.data) : readString(new Uint8Array(decodeBinary(w2.data))); result = "";
const diff = dmp.diff_main(w2data, w1data); const dmp = new diff_match_patch();
dmp.diff_cleanupSemantic(diff); const w2data = readDocument(w2) as string;
for (const v of diff) { const diff = dmp.diff_main(w2data, w1data);
const x1 = v[0]; dmp.diff_cleanupSemantic(diff);
const x2 = v[1]; for (const v of diff) {
if (x1 == DIFF_DELETE) { const x1 = v[0];
result += "<span class='history-deleted'>" + escapeStringToHTML(x2) + "</span>"; const x2 = v[1];
} else if (x1 == DIFF_EQUAL) { if (x1 == DIFF_DELETE) {
result += "<span class='history-normal'>" + escapeStringToHTML(x2) + "</span>"; result += "<span class='history-deleted'>" + escapeStringToHTML(x2) + "</span>";
} else if (x1 == DIFF_INSERT) { } else if (x1 == DIFF_EQUAL) {
result += "<span class='history-added'>" + escapeStringToHTML(x2) + "</span>"; result += "<span class='history-normal'>" + escapeStringToHTML(x2) + "</span>";
} else if (x1 == DIFF_INSERT) {
result += "<span class='history-added'>" + escapeStringToHTML(x2) + "</span>";
}
} }
result = result.replace(/\n/g, "<br>");
} else if (isImage(this.file)) {
const src = this.generateBlobURL("base", w1data);
const overlay = this.generateBlobURL("overlay", readDocument(w2) as Uint8Array);
result =
`<div class='ls-imgdiff-wrap'>
<div class='overlay'>
<img class='img-base' src="${src}">
<img class='img-overlay' src='${overlay}'>
</div>
</div>`;
this.contentView.removeClass("op-pre");
} }
}
}
result = result.replace(/\n/g, "<br>"); }
} else { if (result == undefined) {
result = escapeStringToHTML(w1data); if (typeof w1data != "string") {
if (isImage(this.file)) {
const src = this.generateBlobURL("base", w1data);
result =
`<div class='ls-imgdiff-wrap'>
<div class='overlay'>
<img class='img-base' src="${src}">
</div>
</div>`;
this.contentView.removeClass("op-pre");
} }
} else { } else {
result = escapeStringToHTML(w1data); result = escapeStringToHTML(w1data);
} }
} else {
result = escapeStringToHTML(w1data);
} }
if (result == undefined) result = typeof w1data == "string" ? escapeStringToHTML(w1data) : "Binary file";
this.contentView.innerHTML = (this.currentDeleted ? "(At this revision, the file has been deleted)\n" : "") + result; this.contentView.innerHTML = (this.currentDeleted ? "(At this revision, the file has been deleted)\n" : "") + result;
} }
} }
@@ -217,5 +283,9 @@ export class DocumentHistoryModal extends Modal {
onClose() { onClose() {
const { contentEl } = this; const { contentEl } = this;
contentEl.empty(); contentEl.empty();
this.BlobURLs.forEach(value => {
console.log(value);
if (value) URL.revokeObjectURL(value);
})
} }
} }

View File

@@ -1,6 +1,7 @@
import { App, Modal } from "./deps"; import { App, Modal } from "./deps";
import { type FilePath, type LoadedEntry } from "./lib/src/types"; import { type FilePath, type LoadedEntry } from "./lib/src/types";
import JsonResolvePane from "./JsonResolvePane.svelte"; import JsonResolvePane from "./JsonResolvePane.svelte";
import { waitForSignal } from "./lib/src/utils";
export class JsonResolveModal extends Modal { export class JsonResolveModal extends Modal {
// result: Array<[number, string]>; // result: Array<[number, string]>;
@@ -20,6 +21,7 @@ export class JsonResolveModal extends Modal {
this.nameA = nameA; this.nameA = nameA;
this.nameB = nameB; this.nameB = nameB;
this.defaultSelect = defaultSelect; this.defaultSelect = defaultSelect;
waitForSignal(`cancel-internal-conflict:${filename}`).then(() => this.close());
} }
async UICallback(keepRev: string, mergedStr?: string) { async UICallback(keepRev: string, mergedStr?: string) {
this.close(); this.close();

View File

@@ -8,6 +8,7 @@ import { checkSyncInfo, isCloudantURI } from "./lib/src/utils_couchdb";
import { testCrypt } from "./lib/src/e2ee_v2"; import { testCrypt } from "./lib/src/e2ee_v2";
import ObsidianLiveSyncPlugin from "./main"; import ObsidianLiveSyncPlugin from "./main";
import { askYesNo, performRebuildDB, requestToCouchDB, scheduleTask } from "./utils"; import { askYesNo, performRebuildDB, requestToCouchDB, scheduleTask } from "./utils";
import type { ButtonComponent } from "obsidian";
export class ObsidianLiveSyncSettingTab extends PluginSettingTab { export class ObsidianLiveSyncSettingTab extends PluginSettingTab {
@@ -34,6 +35,10 @@ export class ObsidianLiveSyncSettingTab extends PluginSettingTab {
containerEl.empty(); containerEl.empty();
// const preferred_setting = isCloudantURI(this.plugin.settings.couchDB_URI) ? PREFERRED_SETTING_CLOUDANT : PREFERRED_SETTING_SELF_HOSTED;
// const default_setting = { ...DEFAULT_SETTINGS };
containerEl.createEl("h2", { text: "Settings for Self-hosted LiveSync." }); containerEl.createEl("h2", { text: "Settings for Self-hosted LiveSync." });
containerEl.addClass("sls-setting"); containerEl.addClass("sls-setting");
containerEl.removeClass("isWizard"); containerEl.removeClass("isWizard");
@@ -819,6 +824,53 @@ export class ObsidianLiveSyncSettingTab extends PluginSettingTab {
text.inputEl.setAttribute("type", "number"); text.inputEl.setAttribute("type", "number");
}); });
containerGeneralSettingsEl.createEl("h4", { text: "Share settings via markdown" });
let settingSyncFile = this.plugin.settings.settingSyncFile;
let buttonApplyFilename: ButtonComponent;
new Setting(containerGeneralSettingsEl)
.setName("Filename")
.setDesc("If you set this, all settings are saved in a markdown file. You will also be notified when new settings were arrived. You can set different files by the platform.")
.addText((text) => {
text.setPlaceholder("livesync/setting.md")
.setValue(settingSyncFile)
.onChange((value) => {
settingSyncFile = value;
if (settingSyncFile == this.plugin.settings.settingSyncFile) {
buttonApplyFilename.removeCta()
buttonApplyFilename.setDisabled(true);
} else {
buttonApplyFilename.setCta()
buttonApplyFilename.setDisabled(false);
}
})
}).addButton(button => {
button.setButtonText("Apply")
.onClick(async () => {
this.plugin.settings.settingSyncFile = settingSyncFile;
await this.plugin.saveSettings();
this.display();
})
buttonApplyFilename = button;
})
new Setting(containerGeneralSettingsEl)
.setName("Write credentials in the file")
.setDesc("(Not recommended) If set, credentials will be stored in the file.")
.addToggle(toggle => {
toggle.setValue(this.plugin.settings.writeCredentialsForSettingSync)
.onChange(async (value) => {
this.plugin.settings.writeCredentialsForSettingSync = value;
await this.plugin.saveSettings();
})
});
new Setting(containerGeneralSettingsEl)
.setName("Notify all setting files")
.addToggle(toggle => {
toggle.setValue(this.plugin.settings.notifyAllSettingSyncFile)
.onChange(async (value) => {
this.plugin.settings.notifyAllSettingSyncFile = value;
await this.plugin.saveSettings();
})
});
containerGeneralSettingsEl.createEl("h4", { text: "Advanced Confidentiality" }); containerGeneralSettingsEl.createEl("h4", { text: "Advanced Confidentiality" });
@@ -1116,7 +1168,7 @@ export class ObsidianLiveSyncSettingTab extends PluginSettingTab {
); );
new Setting(containerSyncSettingEl) new Setting(containerSyncSettingEl)
.setName("Postpone resolution of unopened files") .setName("Postpone resolution of inactive files")
.setClass("wizardHidden") .setClass("wizardHidden")
.addToggle((toggle) => .addToggle((toggle) =>
toggle.setValue(this.plugin.settings.checkConflictOnlyOnOpen).onChange(async (value) => { toggle.setValue(this.plugin.settings.checkConflictOnlyOnOpen).onChange(async (value) => {
@@ -1124,6 +1176,15 @@ export class ObsidianLiveSyncSettingTab extends PluginSettingTab {
await this.plugin.saveSettings(); await this.plugin.saveSettings();
}) })
); );
new Setting(containerSyncSettingEl)
.setName("Postpone manual resolution of inactive files")
.setClass("wizardHidden")
.addToggle((toggle) =>
toggle.setValue(this.plugin.settings.showMergeDialogOnlyOnActive).onChange(async (value) => {
this.plugin.settings.showMergeDialogOnlyOnActive = value;
await this.plugin.saveSettings();
})
);
containerSyncSettingEl.createEl("h4", { text: "Compatibility" }).addClass("wizardHidden"); containerSyncSettingEl.createEl("h4", { text: "Compatibility" }).addClass("wizardHidden");
new Setting(containerSyncSettingEl) new Setting(containerSyncSettingEl)
.setName("Always resolve conflict manually") .setName("Always resolve conflict manually")
@@ -1229,7 +1290,7 @@ export class ObsidianLiveSyncSettingTab extends PluginSettingTab {
}); });
let skipPatternTextArea: TextAreaComponent = null; let skipPatternTextArea: TextAreaComponent = null;
const defaultSkipPattern = "\\/node_modules\\/, \\/\\.git\\/, \\/obsidian-livesync\\/"; const defaultSkipPattern = "\\/node_modules\\/, \\/\\.git\\/, \\/obsidian-livesync\\/";
const defaultSkipPatternXPlat = defaultSkipPattern + ",\\/workspace$ ,\\/workspace.json$"; const defaultSkipPatternXPlat = defaultSkipPattern + ",\\/workspace$ ,\\/workspace.json$,\\/workspace-mobile.json$";
new Setting(containerSyncSettingEl) new Setting(containerSyncSettingEl)
.setName("Folders and files to ignore") .setName("Folders and files to ignore")
.setDesc( .setDesc(
@@ -1313,11 +1374,35 @@ export class ObsidianLiveSyncSettingTab extends PluginSettingTab {
}); });
containerSyncSettingEl.createEl("h4", { containerSyncSettingEl.createEl("h4", {
text: sanitizeHTMLToDom(`Synchronization target filters`), text: sanitizeHTMLToDom(`Targets`),
}).addClass("wizardHidden"); }).addClass("wizardHidden");
new Setting(containerSyncSettingEl) new Setting(containerSyncSettingEl)
.setName("Regular expression to ignore files") .setName("Synchronising files")
.setDesc("If this is set, any changes to local and remote files that match this will be skipped.") .setDesc("(RegExp) Empty to sync all files. set filter as a regular expression to limit synchronising files.")
.setClass("wizardHidden")
.addTextArea((text) => {
text
.setValue(this.plugin.settings.syncOnlyRegEx)
.setPlaceholder("\\.md$|\\.txt")
.onChange(async (value) => {
let isValidRegExp = false;
try {
new RegExp(value);
isValidRegExp = true;
} catch (_) {
// NO OP.
}
if (isValidRegExp || value.trim() == "") {
this.plugin.settings.syncOnlyRegEx = value;
await this.plugin.saveSettings();
}
})
return text;
}
);
new Setting(containerSyncSettingEl)
.setName("Non-Synchronising files")
.setDesc("(RegExp) If this is set, any changes to local and remote files that match this will be skipped.")
.setClass("wizardHidden") .setClass("wizardHidden")
.addTextArea((text) => { .addTextArea((text) => {
text text
@@ -1340,29 +1425,22 @@ export class ObsidianLiveSyncSettingTab extends PluginSettingTab {
} }
); );
new Setting(containerSyncSettingEl) new Setting(containerSyncSettingEl)
.setName("Regular expression for restricting synchronization targets") .setName("Maximum file size")
.setDesc("If this is set, changes to local and remote files that only match this will be processed.") .setDesc("(MB) If this is set, changes to local and remote files that are larger than this will be skipped. If the file becomes smaller again, a newer one will be used.")
.setClass("wizardHidden") .setClass("wizardHidden")
.addTextArea((text) => { .addText((text) => {
text text.setPlaceholder("")
.setValue(this.plugin.settings.syncOnlyRegEx) .setValue(this.plugin.settings.syncMaxSizeInMB + "")
.setPlaceholder("\\.md$|\\.txt")
.onChange(async (value) => { .onChange(async (value) => {
let isValidRegExp = false; let v = Number(value);
try { if (isNaN(v) || v < 1) {
new RegExp(value); v = 0;
isValidRegExp = true;
} catch (_) {
// NO OP.
} }
if (isValidRegExp || value.trim() == "") { this.plugin.settings.syncMaxSizeInMB = v;
this.plugin.settings.syncOnlyRegEx = value; await this.plugin.saveSettings();
await this.plugin.saveSettings(); });
} text.inputEl.setAttribute("type", "number");
}) });
return text;
}
);
new Setting(containerSyncSettingEl) new Setting(containerSyncSettingEl)
.setName("(Beta) Use ignore files") .setName("(Beta) Use ignore files")
.setDesc("If this is set, changes to local files which are matched by the ignore files will be skipped. Remote changes are determined using local ignore files.") .setDesc("If this is set, changes to local files which are matched by the ignore files will be skipped. Remote changes are determined using local ignore files.")
@@ -1403,15 +1481,15 @@ export class ObsidianLiveSyncSettingTab extends PluginSettingTab {
}).addClass("wizardHidden"); }).addClass("wizardHidden");
new Setting(containerSyncSettingEl) new Setting(containerSyncSettingEl)
.setName("Batch size") .setName("Batch size")
.setDesc("Number of change feed items to process at a time. Defaults to 50.") .setDesc("Number of change feed items to process at a time. Defaults to 50. Minimum is 2.")
.setClass("wizardHidden") .setClass("wizardHidden")
.addText((text) => { .addText((text) => {
text.setPlaceholder("") text.setPlaceholder("")
.setValue(this.plugin.settings.batch_size + "") .setValue(this.plugin.settings.batch_size + "")
.onChange(async (value) => { .onChange(async (value) => {
let v = Number(value); let v = Number(value);
if (isNaN(v) || v < 10) { if (isNaN(v) || v < 2) {
v = 10; v = 2;
} }
this.plugin.settings.batch_size = v; this.plugin.settings.batch_size = v;
await this.plugin.saveSettings(); await this.plugin.saveSettings();
@@ -1421,15 +1499,15 @@ export class ObsidianLiveSyncSettingTab extends PluginSettingTab {
new Setting(containerSyncSettingEl) new Setting(containerSyncSettingEl)
.setName("Batch limit") .setName("Batch limit")
.setDesc("Number of batches to process at a time. Defaults to 40. This along with batch size controls how many docs are kept in memory at a time.") .setDesc("Number of batches to process at a time. Defaults to 40. Minimum is 2. This along with batch size controls how many docs are kept in memory at a time.")
.setClass("wizardHidden") .setClass("wizardHidden")
.addText((text) => { .addText((text) => {
text.setPlaceholder("") text.setPlaceholder("")
.setValue(this.plugin.settings.batches_limit + "") .setValue(this.plugin.settings.batches_limit + "")
.onChange(async (value) => { .onChange(async (value) => {
let v = Number(value); let v = Number(value);
if (isNaN(v) || v < 10) { if (isNaN(v) || v < 2) {
v = 10; v = 2;
} }
this.plugin.settings.batches_limit = v; this.plugin.settings.batches_limit = v;
await this.plugin.saveSettings(); await this.plugin.saveSettings();
@@ -1526,8 +1604,10 @@ export class ObsidianLiveSyncSettingTab extends PluginSettingTab {
pluginConfig.encryptedPassphrase = REDACTED; pluginConfig.encryptedPassphrase = REDACTED;
pluginConfig.encryptedCouchDBConnection = REDACTED; pluginConfig.encryptedCouchDBConnection = REDACTED;
pluginConfig.pluginSyncExtendedSetting = {}; pluginConfig.pluginSyncExtendedSetting = {};
const obsidianInfo = navigator.userAgent;
const msgConfig = `----remote config---- const msgConfig = `---- Obsidian info ----
${obsidianInfo}
---- remote config ----
${stringifyYaml(responseConfig)} ${stringifyYaml(responseConfig)}
---- Plug-in config --- ---- Plug-in config ---
version:${manifestVersion} version:${manifestVersion}
@@ -1644,7 +1724,7 @@ ${stringifyYaml(pluginConfig)}`;
if ((await this.plugin.localDatabase.putRaw(doc)).ok) { if ((await this.plugin.localDatabase.putRaw(doc)).ok) {
Logger(`Old ${docName} has been deleted`, LOG_LEVEL_NOTICE); Logger(`Old ${docName} has been deleted`, LOG_LEVEL_NOTICE);
} }
await this.plugin.showIfConflicted(docName as FilePathWithPrefix); await this.plugin.queueConflictCheck(docName as FilePathWithPrefix);
} else { } else {
Logger(`Converting ${docName} Failed!`, LOG_LEVEL_NOTICE); Logger(`Converting ${docName} Failed!`, LOG_LEVEL_NOTICE);
Logger(ret, LOG_LEVEL_VERBOSE); Logger(ret, LOG_LEVEL_VERBOSE);

View File

@@ -27,8 +27,10 @@
async function requestReload() { async function requestReload() {
await addOn.reloadPluginList(true); await addOn.reloadPluginList(true);
} }
let allTerms = [] as string[];
pluginList.subscribe((e) => { pluginList.subscribe((e) => {
list = e; list = e;
allTerms = unique(list.map((e) => e.term));
}); });
pluginIsEnumerating.subscribe((e) => { pluginIsEnumerating.subscribe((e) => {
loading = e; loading = e;
@@ -172,7 +174,7 @@
.filter((e) => `${e.category}/${e.name}` == key) .filter((e) => `${e.category}/${e.name}` == key)
.map((e) => e.files) .map((e) => e.files)
.flat() .flat()
.map((e) => e.filename) .map((e) => e.filename),
); );
automaticList.set(key, mode); automaticList.set(key, mode);
automaticListDisp = automaticList; automaticListDisp = automaticList;
@@ -218,6 +220,16 @@
.sort((a, b) => (a.displayName ?? a.name).localeCompare(b.displayName ?? b.name)) .sort((a, b) => (a.displayName ?? a.name).localeCompare(b.displayName ?? b.name))
.reduce((p, c) => ({ ...p, [c.category]: unique(c.category in p ? [...p[c.category], c.displayName ?? c.name] : [c.displayName ?? c.name]) }), {} as Record<string, string[]>); .reduce((p, c) => ({ ...p, [c.category]: unique(c.category in p ? [...p[c.category], c.displayName ?? c.name] : [c.displayName ?? c.name]) }), {} as Record<string, string[]>);
} }
let deleteTerm = "";
async function deleteAllItems(term: string) {
const deleteItems = list.filter((e) => e.term == term);
for (const item of deleteItems) {
await deleteData(item);
}
addOn.reloadPluginList(true);
}
</script> </script>
<div> <div>
@@ -322,6 +334,29 @@
</div> </div>
{/if} {/if}
</div> </div>
{#if isMaintenanceMode}
<div class="list">
<div>
<h3>Maintenance Commands</h3>
<div class="maintenancerow">
<label for="">Delete All of </label>
<select bind:value={deleteTerm}>
{#each allTerms as term}
<option value={term}>{term}</option>
{/each}
</select>
<button
class="status"
on:click={(evt) => {
deleteAllItems(deleteTerm);
}}
>
🗑️
</button>
</div>
</div>
</div>
{/if}
<div class="buttons"> <div class="buttons">
<label><span>Hide not applicable items</span><input type="checkbox" bind:checked={hideEven} /></label> <label><span>Hide not applicable items</span><input type="checkbox" bind:checked={hideEven} /></label>
</div> </div>
@@ -423,4 +458,13 @@
align-items: center; align-items: center;
min-height: 3em; min-height: 3em;
} }
.maintenancerow {
display: flex;
justify-content: flex-end;
align-items: center;
}
.maintenancerow label {
margin-right: 0.5em;
margin-left: 0.5em;
}
</style> </style>

View File

@@ -2,6 +2,9 @@ import { type App, TFile, type DataWriteOptions, TFolder, TAbstractFile } from "
import { serialized } from "./lib/src/lock"; import { serialized } from "./lib/src/lock";
import type { FilePath } from "./lib/src/types"; import type { FilePath } from "./lib/src/types";
import { createBinaryBlob, isDocContentSame } from "./lib/src/utils"; import { createBinaryBlob, isDocContentSame } from "./lib/src/utils";
import type { InternalFileInfo } from "./types";
import { markChangesAreSame } from "./utils";
function getFileLockKey(file: TFile | TFolder | string) { function getFileLockKey(file: TFile | TFolder | string) {
return `fl:${typeof (file) == "string" ? file : file.path}`; return `fl:${typeof (file) == "string" ? file : file.path}`;
} }
@@ -15,6 +18,15 @@ function toArrayBuffer(arr: Uint8Array | ArrayBuffer | DataView): ArrayBufferLik
return arr; return arr;
} }
async function processReadFile<T>(file: TFile | TFolder | string, proc: () => Promise<T>) {
const ret = await serialized(getFileLockKey(file), () => proc());
return ret;
}
async function processWriteFile<T>(file: TFile | TFolder | string, proc: () => Promise<T>) {
const ret = await serialized(getFileLockKey(file), () => proc());
return ret;
}
export class SerializedFileAccess { export class SerializedFileAccess {
app: App app: App
constructor(app: App) { constructor(app: App) {
@@ -23,60 +35,64 @@ export class SerializedFileAccess {
async adapterStat(file: TFile | string) { async adapterStat(file: TFile | string) {
const path = file instanceof TFile ? file.path : file; const path = file instanceof TFile ? file.path : file;
return await serialized(getFileLockKey(path), () => this.app.vault.adapter.stat(path)); return await processReadFile(file, () => this.app.vault.adapter.stat(path));
} }
async adapterExists(file: TFile | string) { async adapterExists(file: TFile | string) {
const path = file instanceof TFile ? file.path : file; const path = file instanceof TFile ? file.path : file;
return await serialized(getFileLockKey(path), () => this.app.vault.adapter.exists(path)); return await processReadFile(file, () => this.app.vault.adapter.exists(path));
} }
async adapterRemove(file: TFile | string) { async adapterRemove(file: TFile | string) {
const path = file instanceof TFile ? file.path : file; const path = file instanceof TFile ? file.path : file;
return await serialized(getFileLockKey(path), () => this.app.vault.adapter.remove(path)); return await processReadFile(file, () => this.app.vault.adapter.remove(path));
} }
async adapterRead(file: TFile | string) { async adapterRead(file: TFile | string) {
const path = file instanceof TFile ? file.path : file; const path = file instanceof TFile ? file.path : file;
return await serialized(getFileLockKey(path), () => this.app.vault.adapter.read(path)); return await processReadFile(file, () => this.app.vault.adapter.read(path));
} }
async adapterReadBinary(file: TFile | string) { async adapterReadBinary(file: TFile | string) {
const path = file instanceof TFile ? file.path : file; const path = file instanceof TFile ? file.path : file;
return await serialized(getFileLockKey(path), () => this.app.vault.adapter.readBinary(path)); return await processReadFile(file, () => this.app.vault.adapter.readBinary(path));
} }
async adapterWrite(file: TFile | string, data: string | ArrayBuffer | Uint8Array, options?: DataWriteOptions) { async adapterWrite(file: TFile | string, data: string | ArrayBuffer | Uint8Array, options?: DataWriteOptions) {
const path = file instanceof TFile ? file.path : file; const path = file instanceof TFile ? file.path : file;
if (typeof (data) === "string") { if (typeof (data) === "string") {
return await serialized(getFileLockKey(path), () => this.app.vault.adapter.write(path, data, options)); return await processWriteFile(file, () => this.app.vault.adapter.write(path, data, options));
} else { } else {
return await serialized(getFileLockKey(path), () => this.app.vault.adapter.writeBinary(path, toArrayBuffer(data), options)); return await processWriteFile(file, () => this.app.vault.adapter.writeBinary(path, toArrayBuffer(data), options));
} }
} }
async vaultCacheRead(file: TFile) { async vaultCacheRead(file: TFile) {
return await serialized(getFileLockKey(file), () => this.app.vault.cachedRead(file)); return await processReadFile(file, () => this.app.vault.cachedRead(file));
} }
async vaultRead(file: TFile) { async vaultRead(file: TFile) {
return await serialized(getFileLockKey(file), () => this.app.vault.read(file)); return await processReadFile(file, () => this.app.vault.read(file));
} }
async vaultReadBinary(file: TFile) { async vaultReadBinary(file: TFile) {
return await serialized(getFileLockKey(file), () => this.app.vault.readBinary(file)); return await processReadFile(file, () => this.app.vault.readBinary(file));
} }
async vaultModify(file: TFile, data: string | ArrayBuffer | Uint8Array, options?: DataWriteOptions) { async vaultModify(file: TFile, data: string | ArrayBuffer | Uint8Array, options?: DataWriteOptions) {
if (typeof (data) === "string") { if (typeof (data) === "string") {
return await serialized(getFileLockKey(file), async () => { return await processWriteFile(file, async () => {
const oldData = await this.app.vault.read(file); const oldData = await this.app.vault.read(file);
if (data === oldData) return false if (data === oldData) {
markChangesAreSame(file, file.stat.mtime, options.mtime);
return false
}
await this.app.vault.modify(file, data, options) await this.app.vault.modify(file, data, options)
return true; return true;
} }
); );
} else { } else {
return await serialized(getFileLockKey(file), async () => { return await processWriteFile(file, async () => {
const oldData = await this.app.vault.readBinary(file); const oldData = await this.app.vault.readBinary(file);
if (isDocContentSame(createBinaryBlob(oldData), createBinaryBlob(data))) { if (await isDocContentSame(createBinaryBlob(oldData), createBinaryBlob(data))) {
markChangesAreSame(file, file.stat.mtime, options.mtime);
return false; return false;
} }
await this.app.vault.modifyBinary(file, toArrayBuffer(data), options) await this.app.vault.modifyBinary(file, toArrayBuffer(data), options)
@@ -86,16 +102,16 @@ export class SerializedFileAccess {
} }
async vaultCreate(path: string, data: string | ArrayBuffer | Uint8Array, options?: DataWriteOptions): Promise<TFile> { async vaultCreate(path: string, data: string | ArrayBuffer | Uint8Array, options?: DataWriteOptions): Promise<TFile> {
if (typeof (data) === "string") { if (typeof (data) === "string") {
return await serialized(getFileLockKey(path), () => this.app.vault.create(path, data, options)); return await processWriteFile(path, () => this.app.vault.create(path, data, options));
} else { } else {
return await serialized(getFileLockKey(path), () => this.app.vault.createBinary(path, toArrayBuffer(data), options)); return await processWriteFile(path, () => this.app.vault.createBinary(path, toArrayBuffer(data), options));
} }
} }
async delete(file: TFile | TFolder, force = false) { async delete(file: TFile | TFolder, force = false) {
return await serialized(getFileLockKey(file), () => this.app.vault.delete(file, force)); return await processWriteFile(file, () => this.app.vault.delete(file, force));
} }
async trash(file: TFile | TFolder, force = false) { async trash(file: TFile | TFolder, force = false) {
return await serialized(getFileLockKey(file), () => this.app.vault.trash(file, force)); return await processWriteFile(file, () => this.app.vault.trash(file, force));
} }
getAbstractFileByPath(path: FilePath | string): TAbstractFile | null { getAbstractFileByPath(path: FilePath | string): TAbstractFile | null {
@@ -121,8 +137,8 @@ export class SerializedFileAccess {
this.touchedFiles.unshift(key); this.touchedFiles.unshift(key);
this.touchedFiles = this.touchedFiles.slice(0, 100); this.touchedFiles = this.touchedFiles.slice(0, 100);
} }
recentlyTouched(file: TFile) { recentlyTouched(file: TFile | InternalFileInfo) {
const key = `${file.path}-${file.stat.mtime}-${file.stat.size}`; const key = file instanceof TFile ? `${file.path}-${file.stat.mtime}-${file.stat.size}` : `${file.path}-${file.mtime}-${file.size}`;
if (this.touchedFiles.indexOf(key) == -1) return false; if (this.touchedFiles.indexOf(key) == -1) return false;
return true; return true;
} }

View File

@@ -1,8 +1,10 @@
import type { SerializedFileAccess } from "./SerializedFileAccess"; import type { SerializedFileAccess } from "./SerializedFileAccess";
import { Plugin, TAbstractFile, TFile, TFolder } from "./deps"; import { Plugin, TAbstractFile, TFile, TFolder } from "./deps";
import { Logger } from "./lib/src/logger";
import { isPlainText, shouldBeIgnored } from "./lib/src/path"; import { isPlainText, shouldBeIgnored } from "./lib/src/path";
import type { KeyedQueueProcessor } from "./lib/src/processor"; import type { KeyedQueueProcessor } from "./lib/src/processor";
import { type FilePath, type ObsidianLiveSyncSettings } from "./lib/src/types"; import { LOG_LEVEL_NOTICE, type FilePath, type ObsidianLiveSyncSettings } from "./lib/src/types";
import { delay } from "./lib/src/utils";
import { type FileEventItem, type FileEventType, type FileInfo, type InternalFileInfo } from "./types"; import { type FileEventItem, type FileEventType, type FileInfo, type InternalFileInfo } from "./types";
@@ -17,7 +19,8 @@ type LiveSyncForStorageEventManager = Plugin &
vaultAccess: SerializedFileAccess vaultAccess: SerializedFileAccess
} & { } & {
isTargetFile: (file: string | TAbstractFile) => Promise<boolean>, isTargetFile: (file: string | TAbstractFile) => Promise<boolean>,
fileEventQueue: KeyedQueueProcessor<FileEventItem, any> fileEventQueue: KeyedQueueProcessor<FileEventItem, any>,
isFileSizeExceeded: (size: number) => boolean;
}; };
@@ -96,6 +99,11 @@ export class StorageEventManagerObsidian extends StorageEventManager {
const type = param.type; const type = param.type;
const file = param.file; const file = param.file;
const oldPath = param.oldPath; const oldPath = param.oldPath;
const size = file instanceof TFile ? file.stat.size : (file as InternalFileInfo)?.size ?? 0;
if (this.plugin.isFileSizeExceeded(size) && (type == "CREATE" || type == "CHANGED")) {
Logger(`The storage file has been changed but exceeds the maximum size. Skipping: ${param.file.path}`, LOG_LEVEL_NOTICE);
continue;
}
if (file instanceof TFolder) continue; if (file instanceof TFolder) continue;
if (!await this.plugin.isTargetFile(file.path)) continue; if (!await this.plugin.isTargetFile(file.path)) continue;
if (this.plugin.settings.suspendFileWatching) continue; if (this.plugin.settings.suspendFileWatching) continue;
@@ -103,6 +111,8 @@ export class StorageEventManagerObsidian extends StorageEventManager {
let cache: null | string | ArrayBuffer; let cache: null | string | ArrayBuffer;
// new file or something changed, cache the changes. // new file or something changed, cache the changes.
if (file instanceof TFile && (type == "CREATE" || type == "CHANGED")) { if (file instanceof TFile && (type == "CREATE" || type == "CHANGED")) {
// Wait for a bit while to let the writer has marked `touched` at the file.
await delay(10);
if (this.plugin.vaultAccess.recentlyTouched(file)) { if (this.plugin.vaultAccess.recentlyTouched(file)) {
continue; continue;
} }

Submodule src/lib updated: 33f7e69433...1d6c0cc6aa

File diff suppressed because it is too large Load Diff

7
src/stores.ts Normal file
View File

@@ -0,0 +1,7 @@
import { PersistentMap } from "./lib/src/PersistentMap";
export let sameChangePairs: PersistentMap<number[]>;
export function initializeStores(vaultName: string) {
sameChangePairs = new PersistentMap<number[]>(`ls-persist-same-changes-${vaultName}`);
}

View File

@@ -1,12 +1,14 @@
import { normalizePath, Platform, TAbstractFile, App, Plugin, type RequestUrlParam, requestUrl } from "./deps"; import { normalizePath, Platform, TAbstractFile, App, type RequestUrlParam, requestUrl, TFile } from "./deps";
import { path2id_base, id2path_base, isValidFilenameInLinux, isValidFilenameInDarwin, isValidFilenameInWidows, isValidFilenameInAndroid, stripAllPrefixes } from "./lib/src/path"; import { path2id_base, id2path_base, isValidFilenameInLinux, isValidFilenameInDarwin, isValidFilenameInWidows, isValidFilenameInAndroid, stripAllPrefixes } from "./lib/src/path";
import { Logger } from "./lib/src/logger"; import { Logger } from "./lib/src/logger";
import { LOG_LEVEL_VERBOSE, type AnyEntry, type DocumentID, type EntryHasPath, type FilePath, type FilePathWithPrefix } from "./lib/src/types"; import { LOG_LEVEL_VERBOSE, type AnyEntry, type DocumentID, type EntryHasPath, type FilePath, type FilePathWithPrefix } from "./lib/src/types";
import { CHeader, ICHeader, ICHeaderLength, ICXHeader, PSCHeader } from "./types"; import { CHeader, ICHeader, ICHeaderLength, ICXHeader, PSCHeader } from "./types";
import { InputStringDialog, PopoverSelectString } from "./dialogs"; import { InputStringDialog, PopoverSelectString } from "./dialogs";
import ObsidianLiveSyncPlugin from "./main"; import type ObsidianLiveSyncPlugin from "./main";
import { writeString } from "./lib/src/strbin"; import { writeString } from "./lib/src/strbin";
import { fireAndForget } from "./lib/src/utils";
import { sameChangePairs } from "./stores";
export { scheduleTask, setPeriodicTask, cancelTask, cancelAllTasks, cancelPeriodicTask, cancelAllPeriodicTask, } from "./lib/src/task"; export { scheduleTask, setPeriodicTask, cancelTask, cancelAllTasks, cancelPeriodicTask, cancelAllPeriodicTask, } from "./lib/src/task";
@@ -336,8 +338,8 @@ export const askString = (app: App, title: string, key: string, placeholder: str
export class PeriodicProcessor { export class PeriodicProcessor {
_process: () => Promise<any>; _process: () => Promise<any>;
_timer?: number; _timer?: number;
_plugin: Plugin; _plugin: ObsidianLiveSyncPlugin;
constructor(plugin: Plugin, process: () => Promise<any>) { constructor(plugin: ObsidianLiveSyncPlugin, process: () => Promise<any>) {
this._plugin = plugin; this._plugin = plugin;
this._process = process; this._process = process;
} }
@@ -351,12 +353,19 @@ export class PeriodicProcessor {
enable(interval: number) { enable(interval: number) {
this.disable(); this.disable();
if (interval == 0) return; if (interval == 0) return;
this._timer = window.setInterval(() => this.process().then(() => { }), interval); this._timer = window.setInterval(() => fireAndForget(async () => {
await this.process();
if (this._plugin._unloaded) {
this.disable();
}
}), interval);
this._plugin.registerInterval(this._timer); this._plugin.registerInterval(this._timer);
} }
disable() { disable() {
if (this._timer !== undefined) window.clearInterval(this._timer); if (this._timer !== undefined) {
this._timer = undefined; window.clearInterval(this._timer);
this._timer = undefined;
}
} }
} }
@@ -407,3 +416,48 @@ export async function performRebuildDB(plugin: ObsidianLiveSyncPlugin, method: "
await plugin.addOnSetup.rebuildEverything(); await plugin.addOnSetup.rebuildEverything();
} }
} }
export const BASE_IS_NEW = Symbol("base");
export const TARGET_IS_NEW = Symbol("target");
export const EVEN = Symbol("even");
// Why 2000? : ZIP FILE Does not have enough resolution.
const resolution = 2000;
export function compareMTime(baseMTime: number, targetMTime: number): typeof BASE_IS_NEW | typeof TARGET_IS_NEW | typeof EVEN {
const truncatedBaseMTime = (~~(baseMTime / resolution)) * resolution;
const truncatedTargetMTime = (~~(targetMTime / resolution)) * resolution;
// Logger(`Resolution MTime ${truncatedBaseMTime} and ${truncatedTargetMTime} `, LOG_LEVEL_VERBOSE);
if (truncatedBaseMTime == truncatedTargetMTime) return EVEN;
if (truncatedBaseMTime > truncatedTargetMTime) return BASE_IS_NEW;
if (truncatedBaseMTime < truncatedTargetMTime) return TARGET_IS_NEW;
throw new Error("Unexpected error");
}
export function markChangesAreSame(file: TFile | AnyEntry | string, mtime1: number, mtime2: number) {
if (mtime1 === mtime2) return true;
const key = typeof file == "string" ? file : file instanceof TFile ? file.path : file.path ?? file._id;
const pairs = sameChangePairs.get(key, []);
if (pairs.some(e => e == mtime1 || e == mtime2)) {
sameChangePairs.set(key, [...new Set([...pairs, mtime1, mtime2])]);
} else {
sameChangePairs.set(key, [mtime1, mtime2]);
}
}
export function isMarkedAsSameChanges(file: TFile | AnyEntry | string, mtimes: number[]) {
const key = typeof file == "string" ? file : file instanceof TFile ? file.path : file.path ?? file._id;
const pairs = sameChangePairs.get(key, []);
if (mtimes.every(e => pairs.indexOf(e) !== -1)) {
return EVEN;
}
}
export function compareFileFreshness(baseFile: TFile | AnyEntry, checkTarget: TFile | AnyEntry): typeof BASE_IS_NEW | typeof TARGET_IS_NEW | typeof EVEN {
const modifiedBase = baseFile instanceof TFile ? baseFile?.stat?.mtime ?? 0 : baseFile?.mtime ?? 0;
const modifiedTarget = checkTarget instanceof TFile ? checkTarget?.stat?.mtime ?? 0 : checkTarget?.mtime ?? 0;
if (modifiedBase && modifiedTarget && isMarkedAsSameChanges(baseFile, [modifiedBase, modifiedTarget])) {
return EVEN;
}
return compareMTime(modifiedBase, modifiedTarget);
}

View File

@@ -85,7 +85,6 @@
} */ } */
.sls-header-button { .sls-header-button {
margin-left: 2em; margin-left: 2em;
} }
@@ -99,8 +98,10 @@
} }
.CodeMirror-wrap::before, .CodeMirror-wrap::before,
.cm-s-obsidian>.cm-editor::before, .markdown-preview-view.cm-s-obsidian::before,
.canvas-wrapper::before { .markdown-source-view.cm-s-obsidian::before,
.canvas-wrapper::before,
.empty-state::before {
content: attr(data-log); content: attr(data-log);
text-align: right; text-align: right;
white-space: pre-wrap; white-space: pre-wrap;
@@ -116,6 +117,19 @@
filter: grayscale(100%); filter: grayscale(100%);
} }
.empty-state::before,
.markdown-preview-view.cm-s-obsidian::before,
.markdown-source-view.cm-s-obsidian::before {
top: var(--header-height);
right: 1em;
}
.is-mobile .empty-state::before,
.is-mobile .markdown-preview-view.cm-s-obsidian::before,
.is-mobile .markdown-source-view.cm-s-obsidian::before {
top: var(--view-header-height);
right: 1em;
}
.canvas-wrapper::before { .canvas-wrapper::before {
right: 48px; right: 48px;
} }
@@ -124,7 +138,7 @@
right: 0px; right: 0px;
} }
.cm-s-obsidian>.cm-editor::before { .cm-s-obsidian > .cm-editor::before {
right: 16px; right: 16px;
} }
@@ -153,8 +167,8 @@ div.sls-setting-menu-btn {
/* width: 100%; */ /* width: 100%; */
} }
.sls-setting-tab:hover~div.sls-setting-menu-btn, .sls-setting-tab:hover ~ div.sls-setting-menu-btn,
.sls-setting-label.selected .sls-setting-tab:checked~div.sls-setting-menu-btn { .sls-setting-label.selected .sls-setting-tab:checked ~ div.sls-setting-menu-btn {
background-color: var(--interactive-accent); background-color: var(--interactive-accent);
color: var(--text-on-accent); color: var(--text-on-accent);
} }
@@ -257,8 +271,8 @@ div.sls-setting-menu-btn {
display: none; display: none;
} }
.password-input > .setting-item-control >input { .password-input > .setting-item-control > input {
-webkit-text-security: disc; -webkit-text-security: disc;
} }
span.ls-mark-cr::after { span.ls-mark-cr::after {
@@ -271,3 +285,38 @@ span.ls-mark-cr::after {
.deleted span.ls-mark-cr::after { .deleted span.ls-mark-cr::after {
color: var(--text-on-accent); color: var(--text-on-accent);
} }
.ls-imgdiff-wrap {
display: flex;
justify-content: center;
align-items: center;
}
.ls-imgdiff-wrap .overlay {
position: relative;
}
.ls-imgdiff-wrap .overlay .img-base {
position: relative;
top: 0;
left: 0;
}
.ls-imgdiff-wrap .overlay .img-overlay {
-webkit-filter: invert(100%) opacity(50%);
filter: invert(100%) opacity(50%);
position: absolute;
top: 0;
left: 0;
animation: ls-blink-diff 0.5s cubic-bezier(0.4, 0, 1, 1) infinite alternate;
}
@keyframes ls-blink-diff {
0% {
opacity: 0;
}
50% {
opacity: 0;
}
100% {
opacity: 1;
}
}

View File

@@ -7,8 +7,59 @@ Of course, I think this would be our suffering in some cases. However, I would l
Sorry for being absent so much long. And thank you for your patience! Sorry for being absent so much long. And thank you for your patience!
Note: we got a very performance improvement. Note: we got a very performance improvement.
Note at 0.22.2: **Now, to rescue mobile devices, Maximum file size is set to 50 by default**. Please configure the limit as you need. If you do not want to limit the sizes, set zero manually, please.
#### Version history #### Version history
- 0.23.4
- Fixed:
- Now the result of conflict resolution could be surely written into the storage.
- Deleted files can be handled correctly again in the history dialogue and conflict dialogue.
- Some wrong log messages were fixed.
- Change handling now has become more stable.
- Some event handling became to be safer.
- Improved:
- Dumping document information shows conflicts and revisions.
- The timestamp-only differences can be surely cached.
- Timestamp difference detection can be rounded by two seconds.
- Refactored:
- A bit of organisation to write the test.
- 0.22.3
- Fixed:
- No longer detects storage changes which have been caused by Self-hosted LiveSync itself.
- Setting sync file will be detected only if it has been configured now.
- And its log will be shown only while the verbose log is enabled.
- Customisation file enumeration has got less blingy.
- Deletion of files is now reliably synchronised.
- Fixed and improved:
- In-editor-status is now shown in the following areas:
- Note editing pane (Source mode and live-preview mode).
- New tab pane.
- Canvas pane.
- 0.22.2
- Fixed:
- Now the results of resolving conflicts are surely synchronised.
- Modified:
- Some setting items got new clear names. (`Sync Settings` -> `Targets`).
- New feature:
- We can limit the synchronising files by their size. (`Sync Settings` -> `Targets` -> `Maximum file size`).
- It depends on the size of the newer one.
- At Obsidian 1.5.3 on mobile, we should set this to around 50MB to avoid restarting Obsidian.
- Now the settings could be stored in a specific markdown file to synchronise or switch it (`General Setting` -> `Share settings via markdown`).
- [Screwdriver](https://github.com/vrtmrz/obsidian-screwdriver) is quite good, but mostly we only need this.
- Customisation of the obsoleted device is now able to be deleted at once.
- We have to put the maintenance mode in at the Customisation sync dialogue.
- 0.22.1
- New feature:
- We can perform automatic conflict resolution for inactive files, and postpone only manual ones by `Postpone manual resolution of inactive files`.
- Now we can see the image in the document history dialogue.
- We can see the difference of the image, in the document history dialogue.
- And also we can highlight differences.
- Improved:
- Hidden file sync has been stabilised.
- Now automatically reloads the conflict-resolution dialogue when new conflicted revisions have arrived.
- Fixed:
- No longer periodic process runs after unloading the plug-in.
- Now the modification of binary files is surely stored in the storage.
- 0.22.0 - 0.22.0
- Refined: - Refined:
- Task scheduling logics has been rewritten. - Task scheduling logics has been rewritten.