Compare commits

...

24 Commits

Author SHA1 Message Date
vorotamoroz
29b89efc47 ## 0.25.19
### Improved
- Now encoding/decoding for chunk data and encryption/decryption are performed in native functions (if they were available).
2025-09-18 12:29:09 +01:00
vorotamoroz
ef3eef2d08 Bump 2025-09-17 09:07:49 +01:00
vorotamoroz
ffbbe32e36 ### Fixed
- Property encryption detection now works correctly (On Self-hosted LiveSync, it was not broken, but as a library, it was not working correctly).
- Initialising the chunk splitter is now surely performed.
- DirectFileManipulator now works fine (as a library)
    - Old `DirectFileManipulatorV1` is now removed.

### Refactored

- Removed some unnecessary intermediate files.
2025-09-17 09:05:16 +01:00
vorotamoroz
0a5371cdee bump 2025-09-16 10:47:00 +01:00
vorotamoroz
466bb142e2 Refactored: removed some unnecessary intermediate file 2025-09-16 10:45:14 +01:00
vorotamoroz
d394a4ce7f ### Fixed
- No longer information-level logs have produced during toggling `Show only notifications` in the settings (#708).
- Ignoring filters for Hidden file sync now works correctly (#709).
2025-09-16 10:30:48 +01:00
vorotamoroz
71ce76e502 Merge pull request #704 from Gron-HD/main
Add Docker Compose troubleshooting for own server setup
2025-09-16 15:33:14 +09:00
vorotamoroz
ae7a7dd456 Merge pull request #706 from abhith/patch-1
docs(README): update links for Customisation Sync and Hidden File Sync
2025-09-16 15:31:00 +09:00
vorotamoroz
4048186bb5 bump 2025-09-04 11:46:50 +01:00
vorotamoroz
2b94fd9139 ## Improved
- Improved connectivity for P2P connections
- The connection to the signalling server can now be disconnected while in the background or when explicitly disconnected.
  - These features use a patch that has not been incorporated upstream.
2025-09-04 11:44:49 +01:00
vorotamoroz
ec72ece86d bump 2025-09-03 10:12:51 +01:00
vorotamoroz
e394a994c5 Fix typo 2025-09-03 10:11:46 +01:00
vorotamoroz
aa23b6a39a ### Improved
- Now we can configure `forcePathStyle` for bucket synchronisation (#707).
2025-09-03 10:08:49 +01:00
vorotamoroz
58e328a591 bump 2025-09-02 10:27:23 +01:00
vorotamoroz
1730c39d70 ### Fixed
- Opening IndexedDB handling has been ensured.
- Migration check of corrupted files detection has been fixed.
    - Now informs us about conflicted files as non-recoverable, but noted so.
    - No longer errors on not-found files.
2025-09-02 10:24:13 +01:00
Abhith Rajan
dfeac201a2 docs(README): update links for Customisation Sync and Hidden File Sync sections 2025-09-01 21:54:11 +04:00
vorotamoroz
b42152db5e bump 2025-09-01 12:28:01 +09:00
vorotamoroz
171cfc0a38 ### Fixed
- Conflict resolving dialogue now properly displays the changeset name instead of A or B (#691).
2025-09-01 12:23:38 +09:00
vorotamoroz
d2787bdb6a Update older dependencies 2025-09-01 12:21:12 +09:00
Ron Gerber
44b022f003 Add Docker Compose troubleshooting for own server setup
Added another option for the init command that passes the variables directly. When i tried the command above i got the mentioned error.
2025-08-30 01:00:58 +02:00
vorotamoroz
58845276e7 bump 2025-08-29 11:48:33 +01:00
vorotamoroz
a2cc093a9e ### Fixed
- Fixed an issue with automatic synchronisation starting (#702).
2025-08-29 11:46:11 +01:00
vorotamoroz
fec203a751 bump 2025-08-28 10:27:31 +01:00
vorotamoroz
1a06837769 ### Fixed
- Automatic translation detection on the first launch now works correctly (#630).
- No errors are shown during synchronisations in offline (if not explicitly requested) (#699).
- Missing some checking during automatic-synchronisation now works correctly.
2025-08-28 10:26:17 +01:00
32 changed files with 853 additions and 662 deletions

View File

@@ -18,7 +18,7 @@ Additionally, it supports peer-to-peer synchronisation using WebRTC now (experim
- Use open-source solutions for the server.
- Compatible solutions are supported.
- Support end-to-end encryption.
- Synchronise settings, snippets, themes, and plug-ins via [Customisation Sync (Beta)](#customization-sync) or [Hidden File Sync](#hiddenfilesync).
- Synchronise settings, snippets, themes, and plug-ins via [Customisation Sync (Beta)](docs/settings.md#6-customization-sync-advanced) or [Hidden File Sync](docs/settings.md#7-hidden-files-advanced).
- Enable WebRTC peer-to-peer synchronisation without requiring a `host` (Experimental).
- This feature is still in the experimental stage. Please exercise caution when using it.
- WebRTC is a peer-to-peer synchronisation method, so **at least one device must be online to synchronise**.

View File

@@ -132,6 +132,11 @@ If it results like the following:
Your CouchDB has been initialised successfully. If you want this manually, please read the script.
If you are using Docker Compose and the above command does not work or displays `ERROR: Hostname missing`, you can try running the following command, replacing the placeholders with your own values:
```
curl -s https://raw.githubusercontent.com/vrtmrz/obsidian-livesync/main/utils/couchdb/couchdb-init.sh | hostname=http://<YOUR SERVER IP>:5984 username=<INSERT USERNAME HERE> password=<INSERT PASSWORD HERE> bash
```
## 3. Expose CouchDB to the Internet
- You can skip this instruction if you using only in intranet and only with desktop devices.

View File

@@ -1,7 +1,7 @@
{
"id": "obsidian-livesync",
"name": "Self-hosted LiveSync",
"version": "0.25.10",
"version": "0.25.19",
"minAppVersion": "0.9.12",
"description": "Community implementation of self-hosted livesync. Reflect your vault changes to some other devices immediately. Please make sure to disable other synchronize solutions to avoid content corruption or duplication.",
"author": "vorotamoroz",

968
package-lock.json generated

File diff suppressed because it is too large Load Diff

View File

@@ -1,6 +1,6 @@
{
"name": "obsidian-livesync",
"version": "0.25.10",
"version": "0.25.19",
"description": "Reflect your vault changes to some other devices immediately. Please make sure to disable other synchronize solutions to avoid content corruption or duplication.",
"main": "main.js",
"type": "module",
@@ -92,10 +92,10 @@
"fflate": "^0.8.2",
"idb": "^8.0.3",
"minimatch": "^10.0.1",
"octagonal-wheels": "^0.1.37",
"octagonal-wheels": "^0.1.40",
"qrcode-generator": "^1.4.4",
"svelte-check": "^4.1.7",
"trystero": "^0.21.5",
"trystero": "github:vrtmrz/trystero#9e892a93ec14eeb57ce806d272fbb7c3935256d8",
"xxhash-wasm-102": "npm:xxhash-wasm@^1.0.2"
}
}

View File

@@ -8,8 +8,8 @@ export const OpenKeyValueDatabase = async (dbKey: string): Promise<KeyValueDatab
}
const storeKey = dbKey;
const dbPromise = openDB(dbKey, 1, {
upgrade(db) {
db.createObjectStore(storeKey);
upgrade(db, _oldVersion, _newVersion, _transaction, _event) {
return db.createObjectStore(storeKey);
},
});
const db = await dbPromise;

View File

@@ -1,4 +1,4 @@
import { PersistentMap } from "../lib/src/dataobject/PersistentMap.ts";
import { PersistentMap } from "octagonal-wheels/dataobject/PersistentMap";
export let sameChangePairs: PersistentMap<number[]>;

View File

@@ -35,7 +35,7 @@ import { promiseWithResolver, type PromiseWithResolvers } from "octagonal-wheels
import { AuthorizationHeaderGenerator } from "../lib/src/replication/httplib.ts";
import type { KeyValueDatabase } from "../lib/src/interfaces/KeyValueDatabase.ts";
export { scheduleTask, cancelTask, cancelAllTasks } from "../lib/src/concurrency/task.ts";
export { scheduleTask, cancelTask, cancelAllTasks } from "octagonal-wheels/concurrency/task";
// For backward compatibility, using the path for determining id.
// Only CouchDB unacceptable ID (that starts with an underscore) has been prefixed with "/".

View File

@@ -45,7 +45,7 @@ import {
} from "../../lib/src/common/utils.ts";
import { digestHash } from "../../lib/src/string_and_binary/hash.ts";
import { arrayBufferToBase64, decodeBinary, readString } from "../../lib/src/string_and_binary/convert.ts";
import { serialized, shareRunningResult } from "../../lib/src/concurrency/lock.ts";
import { serialized, shareRunningResult } from "octagonal-wheels/concurrency/lock";
import { LiveSyncCommands } from "../LiveSyncCommands.ts";
import { stripAllPrefixes } from "../../lib/src/string_and_binary/path.ts";
import {
@@ -62,7 +62,7 @@ import {
scheduleTask,
} from "../../common/utils.ts";
import { JsonResolveModal } from "../HiddenFileCommon/JsonResolveModal.ts";
import { QueueProcessor } from "../../lib/src/concurrency/processor.ts";
import { QueueProcessor } from "octagonal-wheels/concurrency/processor";
import { pluginScanningCount } from "../../lib/src/mock_and_interop/stores.ts";
import type ObsidianLiveSyncPlugin from "../../main.ts";
import { base64ToArrayBuffer, base64ToString } from "octagonal-wheels/binary/base64";

View File

@@ -45,11 +45,11 @@ import {
BASE_IS_NEW,
EVEN,
} from "../../common/utils.ts";
import { serialized, skipIfDuplicated } from "../../lib/src/concurrency/lock.ts";
import { serialized, skipIfDuplicated } from "octagonal-wheels/concurrency/lock";
import { JsonResolveModal } from "../HiddenFileCommon/JsonResolveModal.ts";
import { LiveSyncCommands } from "../LiveSyncCommands.ts";
import { addPrefix, stripAllPrefixes } from "../../lib/src/string_and_binary/path.ts";
import { QueueProcessor } from "../../lib/src/concurrency/processor.ts";
import { QueueProcessor } from "octagonal-wheels/concurrency/processor";
import { hiddenFilesEventCount, hiddenFilesProcessingCount } from "../../lib/src/mock_and_interop/stores.ts";
import type { IObsidianModule } from "../../modules/AbstractObsidianModule.ts";
import { EVENT_SETTING_SAVED, eventHub } from "../../common/events.ts";
@@ -1765,10 +1765,7 @@ ${messageFetch}${messageOverwrite}${messageMerge}
continue L1;
}
}
if (
ignoreFilter &&
ignoreFilter.some((e) => (e.pattern.startsWith("/") || e.pattern.startsWith("\\/")) && e.test(v))
) {
if (ignoreFilter && ignoreFilter.some((e) => e.test(v))) {
continue L1;
}
if (await this.plugin.$$isIgnoredByIgnoreFiles(v)) {

View File

@@ -174,6 +174,13 @@ export class P2PReplicator
if (this.settings.P2P_Enabled && this.settings.P2P_AutoStart) {
setTimeout(() => void this.open(), 100);
}
const rep = this._replicatorInstance;
rep?.allowReconnection();
return Promise.resolve(true);
}
$everyBeforeSuspendProcess(): Promise<boolean> {
const rep = this._replicatorInstance;
rep?.disconnectFromServer();
return Promise.resolve(true);
}
}

Submodule src/lib updated: e488bca9fc...f021a9a6f4

View File

@@ -31,7 +31,7 @@ import { type KeyValueDatabase } from "./lib/src/interfaces/KeyValueDatabase.ts"
import { LiveSyncCommands } from "./features/LiveSyncCommands.ts";
import { HiddenFileSync } from "./features/HiddenFileSync/CmdHiddenFileSync.ts";
import { ConfigSync } from "./features/ConfigSync/CmdConfigSync.ts";
import { reactiveSource, type ReactiveValue } from "./lib/src/dataobject/reactive.js";
import { reactiveSource, type ReactiveValue } from "octagonal-wheels/dataobject/reactive";
import { type LiveSyncJournalReplicatorEnv } from "./lib/src/replication/journal/LiveSyncJournalReplicator.js";
import { type LiveSyncCouchDBReplicatorEnv } from "./lib/src/replication/couchdb/LiveSyncReplicator.js";
import type { CheckPointInfo } from "./lib/src/replication/journal/JournalSyncTypes.js";
@@ -582,6 +582,11 @@ export default class ObsidianLiveSyncPlugin
$everyBeforeReplicate(showMessage: boolean): Promise<boolean> {
return InterceptiveEvery;
}
$$canReplicate(showMessage: boolean = false): Promise<boolean> {
throwShouldBeOverridden();
}
$$replicate(showMessage: boolean = false): Promise<boolean | void> {
throwShouldBeOverridden();
}

View File

@@ -17,6 +17,7 @@ import {
type EntryLeaf,
type LoadedEntry,
type MetaEntry,
type RemoteType,
} from "../../lib/src/common/types";
import { QueueProcessor } from "octagonal-wheels/concurrency/processor";
import {
@@ -38,7 +39,8 @@ const KEY_REPLICATION_ON_EVENT = "replicationOnEvent";
const REPLICATION_ON_EVENT_FORECASTED_TIME = 5000;
export class ModuleReplicator extends AbstractModule implements ICoreModule {
_replicatorType?: string;
_replicatorType?: RemoteType;
$everyOnloadAfterLoadSettings(): Promise<boolean> {
eventHub.onEvent(EVENT_FILE_SAVED, () => {
if (this.settings.syncOnSave && !this.core.$$isSuspended()) {
@@ -91,6 +93,10 @@ export class ModuleReplicator extends AbstractModule implements ICoreModule {
async $everyBeforeReplicate(showMessage: boolean): Promise<boolean> {
// Checking salt
if (!this.core.managers.networkManager.isOnline) {
this._log("Network is offline", showMessage ? LOG_LEVEL_NOTICE : LOG_LEVEL_INFO);
return false;
}
// Showing message is false: that because be shown here. (And it is a fatal error, no way to hide it).
if (!(await this.ensureReplicatorPBKDF2Salt(false))) {
Logger("Failed to initialise the encryption key, preventing replication.", LOG_LEVEL_NOTICE);
@@ -167,25 +173,42 @@ Even if you choose to clean up, you will see this option again if you exit Obsid
}
});
}
async $$_replicate(showMessage: boolean = false): Promise<boolean | void> {
//--?
if (!this.core.$$isReady()) return;
async $$canReplicate(showMessage: boolean = false): Promise<boolean> {
if (!this.core.$$isReady()) {
Logger(`Not ready`);
return false;
}
if (isLockAcquired("cleanup")) {
Logger($msg("Replicator.Message.Cleaned"), LOG_LEVEL_NOTICE);
return;
return false;
}
if (this.settings.versionUpFlash != "") {
Logger($msg("Replicator.Message.VersionUpFlash"), LOG_LEVEL_NOTICE);
return;
return false;
}
if (!(await this.core.$everyCommitPendingFileEvent())) {
Logger($msg("Replicator.Message.Pending"), LOG_LEVEL_NOTICE);
return false;
}
if (!this.core.managers.networkManager.isOnline) {
this._log("Network is offline", showMessage ? LOG_LEVEL_NOTICE : LOG_LEVEL_INFO);
return false;
}
if (!(await this.core.$everyBeforeReplicate(showMessage))) {
Logger($msg("Replicator.Message.SomeModuleFailed"), LOG_LEVEL_NOTICE);
return false;
}
return true;
}
async $$_replicate(showMessage: boolean = false): Promise<boolean | void> {
const checkBeforeReplicate = await this.$$canReplicate(showMessage);
if (!checkBeforeReplicate) return false;
//<-- Here could be an module.
const ret = await this.core.replicator.openReplication(this.settings, false, showMessage, false);

View File

@@ -15,15 +15,22 @@ export class ModuleReplicatorCouchDB extends AbstractModule implements ICoreModu
return Promise.resolve(new LiveSyncCouchDBReplicator(this.core));
}
$everyAfterResumeProcess(): Promise<boolean> {
if (!this.core.$$isSuspended) return Promise.resolve(true);
if (!this.core.$$isReady) return Promise.resolve(true);
if (this.settings.remoteType != REMOTE_MINIO && this.settings.remoteType != REMOTE_P2P) {
// If LiveSync enabled, open replication
if (this.settings.liveSync) {
fireAndForget(() => this.core.replicator.openReplication(this.settings, true, false, false));
}
// If sync on start enabled, open replication
if (!this.settings.liveSync && this.settings.syncOnStart) {
// Possibly ok as if only share the result
fireAndForget(() => this.core.replicator.openReplication(this.settings, false, false, false));
const LiveSyncEnabled = this.settings.liveSync;
const continuous = LiveSyncEnabled;
const eventualOnStart = !LiveSyncEnabled && this.settings.syncOnStart;
// If enabled LiveSync or on start, open replication
if (LiveSyncEnabled || eventualOnStart) {
// And note that we do not open the conflict detection dialogue directly during this process.
// This should be raised explicitly if needed.
fireAndForget(async () => {
const canReplicate = await this.core.$$canReplicate(false);
if (!canReplicate) return;
void this.core.replicator.openReplication(this.settings, continuous, false, false);
});
}
}

View File

@@ -6,6 +6,10 @@ import { $msg } from "src/lib/src/common/i18n.ts";
export class ModuleCheckRemoteSize extends AbstractModule implements ICoreModule {
async $allScanStat(): Promise<boolean> {
if (this.core.managers.networkManager.isOnline === false) {
this._log("Network is offline, skipping remote size check.", LOG_LEVEL_INFO);
return true;
}
this._log($msg("moduleCheckRemoteSize.logCheckingStorageSizes"), LOG_LEVEL_VERBOSE);
if (this.settings.notifyThresholdOfRemoteStorageSize < 0) {
const message = $msg("moduleCheckRemoteSize.msgSetDBCapacity");

View File

@@ -1,5 +1,5 @@
import { type App, TFile, type DataWriteOptions, TFolder, TAbstractFile } from "../../../deps.ts";
import { serialized } from "../../../lib/src/concurrency/lock.ts";
import { serialized } from "octagonal-wheels/concurrency/lock";
import { Logger } from "../../../lib/src/common/logger.ts";
import { isPlainText } from "../../../lib/src/string_and_binary/path.ts";
import type { FilePath, HasSettings, UXFileInfoStub } from "../../../lib/src/common/types.ts";
@@ -11,7 +11,7 @@ import { type UXFileInfo } from "../../../lib/src/common/types.ts";
function getFileLockKey(file: TFile | TFolder | string | UXFileInfo) {
return `fl:${typeof file == "string" ? file : file.path}`;
}
function toArrayBuffer(arr: Uint8Array | ArrayBuffer | DataView): ArrayBufferLike {
function toArrayBuffer(arr: Uint8Array<ArrayBuffer> | ArrayBuffer | DataView<ArrayBuffer>): ArrayBuffer {
if (arr instanceof Uint8Array) {
return arr.buffer;
}
@@ -77,7 +77,11 @@ export class SerializedFileAccess {
return await processReadFile(file, () => this.app.vault.adapter.readBinary(path));
}
async adapterWrite(file: TFile | string, data: string | ArrayBuffer | Uint8Array, options?: DataWriteOptions) {
async adapterWrite(
file: TFile | string,
data: string | ArrayBuffer | Uint8Array<ArrayBuffer>,
options?: DataWriteOptions
) {
const path = file instanceof TFile ? file.path : file;
if (typeof data === "string") {
return await processWriteFile(file, () => this.app.vault.adapter.write(path, data, options));
@@ -106,7 +110,7 @@ export class SerializedFileAccess {
return await processReadFile(file, () => this.app.vault.readBinary(file));
}
async vaultModify(file: TFile, data: string | ArrayBuffer | Uint8Array, options?: DataWriteOptions) {
async vaultModify(file: TFile, data: string | ArrayBuffer | Uint8Array<ArrayBuffer>, options?: DataWriteOptions) {
if (typeof data === "string") {
return await processWriteFile(file, async () => {
const oldData = await this.app.vault.read(file);
@@ -131,7 +135,7 @@ export class SerializedFileAccess {
}
async vaultCreate(
path: string,
data: string | ArrayBuffer | Uint8Array,
data: string | ArrayBuffer | Uint8Array<ArrayBuffer>,
options?: DataWriteOptions
): Promise<TFile> {
if (typeof data === "string") {

View File

@@ -14,14 +14,14 @@ import {
} from "../../../lib/src/common/types.ts";
import { delay, fireAndForget, getFileRegExp } from "../../../lib/src/common/utils.ts";
import { type FileEventItem, type FileEventType } from "../../../common/types.ts";
import { serialized, skipIfDuplicated } from "../../../lib/src/concurrency/lock.ts";
import { serialized, skipIfDuplicated } from "octagonal-wheels/concurrency/lock";
import {
finishAllWaitingForTimeout,
finishWaitingForTimeout,
isWaitingForTimeout,
waitForTimeout,
} from "../../../lib/src/concurrency/task.ts";
import { Semaphore } from "../../../lib/src/concurrency/semaphore.ts";
} from "octagonal-wheels/concurrency/task";
import { Semaphore } from "octagonal-wheels/concurrency/semaphore";
import type { LiveSyncCore } from "../../../main.ts";
import { InternalFileToUXFileInfoStub, TFileToUXFileInfoStub } from "./utilObsidian.ts";
import ObsidianLiveSyncPlugin from "../../../main.ts";
@@ -207,6 +207,9 @@ export class StorageEventManagerObsidian extends StorageEventManager {
}
}
if (file instanceof TFolder) continue;
// TODO: Confirm why only the TFolder skipping
// Possibly following line is needed...
// if (file?.isFolder) continue;
if (!(await this.core.$$isTargetFile(file.path))) continue;
// Stop cache using to prevent the corruption;

View File

@@ -17,6 +17,15 @@ import { isMetaEntry } from "../../lib/src/common/types.ts";
import { isDeletedEntry, isDocContentSame, isLoadedEntry, readAsBlob } from "../../lib/src/common/utils.ts";
import { countCompromisedChunks } from "../../lib/src/pouchdb/negotiation.ts";
type ErrorInfo = {
path: string;
recordedSize: number;
actualSize: number;
storageSize: number;
contentMatched: boolean;
isConflicted?: boolean;
};
export class ModuleMigration extends AbstractModule implements ICoreModule {
async migrateUsingDoctor(skipRebuild: boolean = false, activateReason = "updated", forceRescan = false) {
const { shouldRebuild, shouldRebuildLocal, isModified, settings } = await performDoctorConsultation(
@@ -112,7 +121,8 @@ export class ModuleMigration extends AbstractModule implements ICoreModule {
}
this._log("Checking for incomplete documents...", LOG_LEVEL_NOTICE, "check-incomplete");
const errorFiles = [];
const errorFiles = [] as ErrorInfo[];
for await (const metaDoc of this.localDatabase.findAllNormalDocs({ conflicts: true })) {
const path = getPath(metaDoc);
@@ -133,17 +143,38 @@ export class ModuleMigration extends AbstractModule implements ICoreModule {
if (isDeletedEntry(doc)) {
continue;
}
const storageFileContent = await this.core.storageAccess.readHiddenFileBinary(path);
const isConflicted = metaDoc?._conflicts && metaDoc._conflicts.length > 0;
let storageFileContent;
try {
storageFileContent = await this.core.storageAccess.readHiddenFileBinary(path);
} catch (e) {
Logger(`Failed to read file ${path}: Possibly unprocessed or missing`);
Logger(e, LOG_LEVEL_VERBOSE);
continue;
}
// const storageFileBlob = createBlob(storageFileContent);
const sizeOnStorage = storageFileContent.byteLength;
const recordedSize = doc.size;
const docBlob = readAsBlob(doc);
const actualSize = docBlob.size;
if (recordedSize !== actualSize || sizeOnStorage !== actualSize || sizeOnStorage !== recordedSize) {
if (
recordedSize !== actualSize ||
sizeOnStorage !== actualSize ||
sizeOnStorage !== recordedSize ||
isConflicted
) {
const contentMatched = await isDocContentSame(doc.data, storageFileContent);
errorFiles.push({ path, recordedSize, actualSize, storageSize: sizeOnStorage, contentMatched });
errorFiles.push({
path,
recordedSize,
actualSize,
storageSize: sizeOnStorage,
contentMatched,
isConflicted,
});
Logger(
`Size mismatch for ${path}: ${recordedSize} (DB Recorded) , ${actualSize} (DB Stored) , ${sizeOnStorage} (Storage Stored), ${contentMatched ? "Content Matched" : "Content Mismatched"}`
`Size mismatch for ${path}: ${recordedSize} (DB Recorded) , ${actualSize} (DB Stored) , ${sizeOnStorage} (Storage Stored), ${contentMatched ? "Content Matched" : "Content Mismatched"} ${isConflicted ? "Conflicted" : "Not Conflicted"}`
);
}
}
@@ -167,24 +198,23 @@ export class ModuleMigration extends AbstractModule implements ICoreModule {
// Probably restored by the user by resolving A or B on other device, We should overwrite the storage
// Also do not fix it automatically. It should be overwritten by replication.
const recoverable = errorFiles.filter((e) => {
return e.recordedSize === e.storageSize;
return e.recordedSize === e.storageSize && !e.isConflicted;
});
const unrecoverable = errorFiles.filter((e) => {
return e.recordedSize !== e.storageSize;
return e.recordedSize !== e.storageSize || e.isConflicted;
});
const fileInfo = (e: (typeof errorFiles)[0]) => {
return `${e.path} (M: ${e.recordedSize}, A: ${e.actualSize}, S: ${e.storageSize}) ${e.isConflicted ? "(Conflicted)" : ""}`;
};
const messageUnrecoverable =
unrecoverable.length > 0
? $msg("moduleMigration.fix0256.messageUnrecoverable", {
filesNotRecoverable: unrecoverable
.map((e) => `- ${e.path} (M: ${e.recordedSize}, A: ${e.actualSize}, S: ${e.storageSize})`)
.join("\n"),
filesNotRecoverable: unrecoverable.map((e) => `- ${fileInfo(e)}`).join("\n"),
})
: "";
const message = $msg("moduleMigration.fix0256.message", {
files: recoverable
.map((e) => `- ${e.path} (M: ${e.recordedSize}, A: ${e.actualSize}, S: ${e.storageSize})`)
.join("\n"),
files: recoverable.map((e) => `- ${fileInfo(e)}`).join("\n"),
messageUnrecoverable,
});
const CHECK_IT_LATER = $msg("moduleMigration.fix0256.buttons.checkItLater");
@@ -228,7 +258,9 @@ export class ModuleMigration extends AbstractModule implements ICoreModule {
// Check local database for compromised chunks
const localCompromised = await countCompromisedChunks(this.localDatabase.localDatabase);
const remote = this.core.$$getReplicator();
const remoteCompromised = await remote.countCompromisedChunks();
const remoteCompromised = this.core.managers.networkManager.isOnline
? await remote.countCompromisedChunks()
: 0;
if (localCompromised === false) {
Logger(`Failed to count compromised chunks in local database`, LOG_LEVEL_NOTICE);
return false;

View File

@@ -106,6 +106,9 @@ export class ModuleObsidianAPI extends AbstractObsidianModule implements IObsidi
if (!isValidRemoteCouchDBURI(uri)) return "Remote URI is not valid";
if (uri.toLowerCase() != uri) return "Remote URI and database name could not contain capital letters.";
if (uri.indexOf(" ") !== -1) return "Remote URI and database name could not contain spaces.";
if (!this.core.managers.networkManager.isOnline) {
return "Network is offline";
}
// let authHeader = await this._authHeader.getAuthorizationHeader(auth);
const conf: PouchDB.HttpAdapter.HttpAdapterConfiguration = {

View File

@@ -1,5 +1,5 @@
import { fireAndForget } from "../../../lib/src/common/utils.ts";
import { serialized } from "../../../lib/src/concurrency/lock.ts";
import { serialized } from "octagonal-wheels/concurrency/lock";
import type ObsidianLiveSyncPlugin from "../../../main.ts";
let plugin: ObsidianLiveSyncPlugin;

View File

@@ -1,4 +1,4 @@
import { Trench } from "../../../lib/src/memory/memutil.ts";
import { Trench } from "octagonal-wheels/memory/memutil";
import type ObsidianLiveSyncPlugin from "../../../main.ts";
type MeasureResult = [times: number, spent: number];
type NamedMeasureResult = [name: string, result: MeasureResult];

View File

@@ -25,8 +25,8 @@ export class ConflictResolveModal extends Modal {
title: string = "Conflicting changes";
pluginPickMode: boolean = false;
localName: string = "Use Base";
remoteName: string = "Use Conflicted";
localName: string = "Base";
remoteName: string = "Conflicted";
offEvent?: ReturnType<typeof eventHub.onEvent>;
constructor(app: App, filename: string, diff: diff_result, pluginPickMode?: boolean, remoteName?: string) {
@@ -36,8 +36,8 @@ export class ConflictResolveModal extends Modal {
this.pluginPickMode = pluginPickMode || false;
if (this.pluginPickMode) {
this.title = "Pick a version";
this.remoteName = `Use ${remoteName || "Remote"}`;
this.localName = "Use Local";
this.remoteName = `${remoteName || "Remote"}`;
this.localName = "Local";
}
// Send cancel signal for the previous merge dialogue
// if not there, simply be ignored.
@@ -93,12 +93,13 @@ export class ConflictResolveModal extends Modal {
const date2 =
new Date(this.result.right.mtime).toLocaleString() + (this.result.right.deleted ? " (Deleted)" : "");
div2.innerHTML = `
<span class='deleted'>A:${date1}</span><br /><span class='added'>B:${date2}</span><br>
<span class='deleted'><span class='conflict-dev-name'>${this.localName}</span>: ${date1}</span><br>
<span class='added'><span class='conflict-dev-name'>${this.remoteName}</span>: ${date2}</span><br>
`;
contentEl.createEl("button", { text: this.localName }, (e) =>
contentEl.createEl("button", { text: `Use ${this.localName}` }, (e) =>
e.addEventListener("click", () => this.sendResponse(this.result.right.rev))
).style.marginRight = "4px";
contentEl.createEl("button", { text: this.remoteName }, (e) =>
contentEl.createEl("button", { text: `Use ${this.remoteName}` }, (e) =>
e.addEventListener("click", () => this.sendResponse(this.result.left.rev))
).style.marginRight = "4px";
if (!this.pluginPickMode) {

View File

@@ -1,7 +1,7 @@
<script lang="ts">
import { onDestroy, onMount } from "svelte";
import { logMessages } from "../../../lib/src/mock_and_interop/stores";
import { reactive, type ReactiveInstance } from "../../../lib/src/dataobject/reactive";
import { reactive, type ReactiveInstance } from "octagonal-wheels/dataobject/reactive";
import { Logger } from "../../../lib/src/common/logger";
import { $msg as msg, currentLang as lang } from "../../../lib/src/common/i18n.ts";

View File

@@ -13,7 +13,7 @@ import { ConflictResolveModal } from "./InteractiveConflictResolving/ConflictRes
import { AbstractObsidianModule, type IObsidianModule } from "../AbstractObsidianModule.ts";
import { displayRev, getPath, getPathWithoutPrefix } from "../../common/utils.ts";
import { fireAndForget } from "octagonal-wheels/promises";
import { serialized } from "../../lib/src/concurrency/lock.ts";
import { serialized } from "octagonal-wheels/concurrency/lock";
export class ModuleInteractiveConflictResolver extends AbstractObsidianModule implements IObsidianModule {
$everyOnloadStart(): Promise<boolean> {

View File

@@ -370,7 +370,7 @@ export class ModuleLog extends AbstractObsidianModule implements IObsidianModule
if (level == LOG_LEVEL_DEBUG && !showDebugLog) {
return;
}
if (level < LOG_LEVEL_INFO && this.settings && this.settings.lessInformationInLog) {
if (level <= LOG_LEVEL_INFO && this.settings && this.settings.lessInformationInLog) {
return;
}
if (this.settings && !this.settings.showVerboseLog && level == LOG_LEVEL_VERBOSE) {

View File

@@ -1,6 +1,6 @@
import { type IObsidianModule, AbstractObsidianModule } from "../AbstractObsidianModule.ts";
// import { PouchDB } from "../../lib/src/pouchdb/pouchdb-browser";
import { EVENT_REQUEST_RELOAD_SETTING_TAB, EVENT_SETTING_SAVED, eventHub } from "../../common/events";
import { EVENT_REQUEST_RELOAD_SETTING_TAB, EVENT_SETTING_SAVED, eventHub } from "../../common/events.ts";
import {
type BucketSyncSetting,
ChunkAlgorithmNames,
@@ -11,8 +11,8 @@ import {
SALT_OF_PASSPHRASE,
} from "../../lib/src/common/types";
import { LOG_LEVEL_NOTICE, LOG_LEVEL_URGENT } from "octagonal-wheels/common/logger";
import { $msg, setLang } from "../../lib/src/common/i18n";
import { isCloudantURI } from "../../lib/src/pouchdb/utils_couchdb";
import { $msg, setLang } from "../../lib/src/common/i18n.ts";
import { isCloudantURI } from "../../lib/src/pouchdb/utils_couchdb.ts";
import { getLanguage } from "obsidian";
import { SUPPORTED_I18N_LANGS, type I18N_LANGS } from "../../lib/src/common/rosetta.ts";
import { decryptString, encryptString } from "@/lib/src/encryption/stringEncryption.ts";
@@ -23,8 +23,7 @@ export class ModuleObsidianSettings extends AbstractObsidianModule implements IO
const obsidianLanguage = getLanguage();
if (
SUPPORTED_I18N_LANGS.indexOf(obsidianLanguage) !== -1 && // Check if the language is supported
obsidianLanguage != this.settings.displayLanguage && // Check if the language is different from the current setting
this.settings.displayLanguage != ""
obsidianLanguage != this.settings.displayLanguage // Check if the language is different from the current setting
) {
// Check if the current setting is not empty (Means migrated or installed).
this.settings.displayLanguage = obsidianLanguage as I18N_LANGS;
@@ -141,6 +140,7 @@ export class ModuleObsidianSettings extends AbstractObsidianModule implements IO
jwtSub: settings.jwtSub,
useRequestAPI: settings.useRequestAPI,
bucketPrefix: settings.bucketPrefix,
forcePathStyle: settings.forcePathStyle,
};
settings.encryptedCouchDBConnection = await this.encryptConfigurationItem(
JSON.stringify(connectionSetting),

View File

@@ -17,7 +17,7 @@ import { delay, isObjectDifferent, sizeToHumanReadable } from "../../../lib/src/
import { versionNumberString2Number } from "../../../lib/src/string_and_binary/convert.ts";
import { Logger } from "../../../lib/src/common/logger.ts";
import { checkSyncInfo } from "@/lib/src/pouchdb/negotiation.ts";
import { testCrypt } from "../../../lib/src/encryption/e2ee_v2.ts";
import { testCrypt } from "octagonal-wheels/encryption/encryption";
import ObsidianLiveSyncPlugin from "../../../main.ts";
import { scheduleTask } from "../../../common/utils.ts";
import { LiveSyncCouchDBReplicator } from "../../../lib/src/replication/couchdb/LiveSyncReplicator.ts";
@@ -859,26 +859,7 @@ export class ObsidianLiveSyncSettingTab extends PluginSettingTab {
}
getMinioJournalSyncClient() {
const id = this.plugin.settings.accessKey;
const key = this.plugin.settings.secretKey;
const bucket = this.plugin.settings.bucket;
const prefix = this.plugin.settings.bucketPrefix;
const region = this.plugin.settings.region;
const endpoint = this.plugin.settings.endpoint;
const useCustomRequestHandler = this.plugin.settings.useCustomRequestHandler;
const customHeaders = this.plugin.settings.bucketCustomHeaders;
return new JournalSyncMinio(
id,
key,
endpoint,
bucket,
prefix,
this.plugin.simpleStore,
this.plugin,
useCustomRequestHandler,
region,
customHeaders
);
return new JournalSyncMinio(this.plugin.settings, this.plugin.simpleStore, this.plugin);
}
async resetRemoteBucket() {
const minioJournal = this.getMinioJournalSyncClient();

View File

@@ -320,6 +320,7 @@ The pane also can be launched by \`P2P Replicator\` command from the Command Pal
syncWarnMinio.addClass("op-warn-info");
new Setting(paneEl).autoWireText("endpoint", { holdValue: true });
new Setting(paneEl).autoWireToggle("forcePathStyle", { holdValue: true });
new Setting(paneEl).autoWireText("accessKey", { holdValue: true });
new Setting(paneEl).autoWireText("secretKey", {

View File

@@ -12,6 +12,11 @@
background-color: var(--text-muted);
}
.conflict-dev-name {
display: inline-block;
min-width: 5em;
}
.op-scrollable {
overflow-y: scroll;
/* min-height: 280px; */

View File

@@ -1,94 +1,6 @@
## 0.25.10
## 0.25
26th August, 2025
### New experimental feature
- We can perform Garbage Collection (Beta2) without rebuilding the entire database, and also fetch the database.
- Note that this feature is very experimental and should be used with caution.
- This feature requires disabling `Fetch chunks on demand`.
### Fixed
- Resetting the bucket now properly clears all uploaded files.
### Refactored
- Some files have been moved to better reflect their purpose and improve maintainability.
- The extensive LiveSyncLocalDB has been split into separate files for each role.
## 0.25.9
20th August, 2025
### Fixed
- CORS Checking messages now use replacements.
- Configuring CORS setting via the UI now respects the existing rules.
- Now startup-checking works correctly again, performs migration check serially and then it will also fix starting LiveSync or start-up sync. (#696)
- Statusline in editor now supported 'Bases'.
## 0.25.8
18th August, 2025
### New feature
- Insecure chunk detection has been implemented.
- A notification dialogue will be shown if any insecure chunks are detected; these may have been created by v0.25.6 due to its issue. If this dialogue appears, please ensure you rebuild the database after backing it up.
### Fixed
- Unexpected `Failed to obtain PBKDF2 salt` or similar errors during bucket-synchronisation no longer occur.
- Unexpected long delays for chunk-missing documents when using bucket-synchronisation have been resolved.
- Fetched remote chunks are now properly stored in the local database if `Fetch chunks on demand` is enabled.
- The 'fetch' dialogue's message has been refined.
- No longer overwriting any corrupted documents to the storage on boot-sequence.
### Refactored
- Type errors have been corrected.
## 0.25.7
15th August, 2025
**Since the release of 0.25.6, there are two large problem. Please update immediately.**
- We may have corrupted some documents during the migration process. **Please check your documents on the wizard.**
- Due to a chunk ID assignment issue, some data has not been encrypted. **Please rebuild the database using Rebuild Everything** if you have enabled E2EE.
**_So, If you have enabled E2EE, please perform `Rebuild everything`. If not, please check your documents on the wizard._**
In next version, insecure chunk detection will be implemented.
### Fixed
- Off-loaded chunking have been fixed to ensure proper functionality (#693).
- Chunk document ID assignment has been fixed.
- Replication prevention message during version up detection has been improved (#686).
- `Keep A` and `Keep B` on Conflict resolving dialogue has been renamed to `Use Base` and `Use Conflicted` (#691).
### Improved
- Metadata and content-size unmatched documents are now detected and reported, prevented to be applied to the storage.
- This behaviour can be configured in `Patch` -> `Edge case addressing (Behaviour)` -> `Process files even if seems to be corrupted`
- Note: this toggle is for the direct-database-manipulation users.
### New Features
- `Scan for Broken files` has been implemented on `Hatch` -> `TroubleShooting`.
### Refactored
- Off-loaded processes have been refactored for the better maintainability.
- Files prefixed `bg.worker` are now work on the worker threads.
- Files prefixed `bgWorker.` are now also controls these worker threads. (I know what you want to say... I will rename them).
- Removed unused code.
## 0.25.0
19th July, 2025 (beta1 in 0.25.0-beta1, 13th July, 2025)
Since 19th July, 2025 (beta1 in 0.25.0-beta1, 13th July, 2025)
After reading Issue #668, I conducted another self-review of the E2EE-related code. In retrospect, it was clearly written by someone inexperienced, which is understandable, but it is still rather embarrassing. Three years is certainly enough time for growth.
@@ -96,5 +8,91 @@ I have now rewritten the E2EE code to be more robust and easier to understand. I
As a result, this is the first time in a while that forward compatibility has been broken. We have also taken the opportunity to change all metadata to use encryption rather than obfuscation. Furthermore, the `Dynamic Iteration Count` setting is now redundant and has been moved to the `Patches` pane in the settings. Thanks to Rabin-Karp, the eden setting is also no longer necessary and has been relocated accordingly. Therefore, v0.25.0 represents a legitimate and correct evolution.
## 0.25.19
18th September, 2025
### Improved
- Now encoding/decoding for chunk data and encryption/decryption are performed in native functions (if they were available).
- This uses Uint8Array.fromBase64 and Uint8Array.toBase64, which are natively available in iOS 18.2+ and Android with Chrome 140+.
- In Android, WebView is by default updated with Chrome, so it should be available in most cases.
- Note that this is not available in Desktop yet (due to being based on Electron). We are staying tuned for future updates.
- This realised by an external(?) package [octagonal-wheels](https://github.com/vrtmrz/octagonal-wheels). Therefore, this update only updates the dependency.
## 0.25.18
17th September, 2025
### Fixed
- Property encryption detection now works correctly (On Self-hosted LiveSync, it was not broken, but as a library, it was not working correctly).
- Initialising the chunk splitter is now surely performed.
- DirectFileManipulator now works fine (as a library)
- Old `DirectFileManipulatorV1` is now removed.
### Refactored
- Removed some unnecessary intermediate files.
## 0.25.17
16th September, 2025
### Fixed
- No longer information-level logs have produced during toggling `Show only notifications` in the settings (#708).
- Ignoring filters for Hidden file sync now works correctly (#709).
### Refactored
- Removed some unnecessary intermediate files.
## 0.25.16
4th September, 2025
### Improved
- Improved connectivity for P2P connections
- The connection to the signalling server can now be disconnected while in the background or when explicitly disconnected.
- These features use a patch that has not been incorporated upstream.
- This patch is available at [vrtmrz/trystero](https://github.com/vrtmrz/trystero).
## 0.25.15
3rd September, 2025
### Improved
- Now we can configure `forcePathStyle` for bucket synchronisation (#707).
## 0.25.14
2nd September, 2025
### Fixed
- Opening IndexedDB handling has been ensured.
- Migration check of corrupted files detection has been fixed.
- Now informs us about conflicted files as non-recoverable, but noted so.
- No longer errors on not-found files.
## 0.25.13
1st September, 2025
### Fixed
- Conflict resolving dialogue now properly displays the changeset name instead of A or B (#691).
## 0.25.12
29th August, 2025
### Fixed
- Fixed an issue with automatic synchronisation starting (#702).
Older notes are in
[updates_old.md](https://github.com/vrtmrz/obsidian-livesync/blob/main/updates_old.md).

View File

@@ -1,7 +1,6 @@
# 0.25
## 0.25.0
19th July, 2025 (beta1 in 0.25.0-beta1, 13th July, 2025)
Since 19th July, 2025 (beta1 in 0.25.0-beta1, 13th July, 2025)
After reading Issue #668, I conducted another self-review of the E2EE-related code. In retrospect, it was clearly written by someone inexperienced, which is understandable, but it is still rather embarrassing. Three years is certainly enough time for growth.
@@ -11,6 +10,106 @@ As a result, this is the first time in a while that forward compatibility has be
---
## 0.25.11
28th August, 2025
### Fixed
- Automatic translation detection on the first launch now works correctly (#630).
- No errors are shown during synchronisations in offline (if not explicitly requested) (#699).
- Missing some checking during automatic-synchronisation now works correctly.
## 0.25.10
26th August, 2025
### New experimental feature
- We can perform Garbage Collection (Beta2) without rebuilding the entire database, and also fetch the database.
- Note that this feature is very experimental and should be used with caution.
- This feature requires disabling `Fetch chunks on demand`.
### Fixed
- Resetting the bucket now properly clears all uploaded files.
### Refactored
- Some files have been moved to better reflect their purpose and improve maintainability.
- The extensive LiveSyncLocalDB has been split into separate files for each role.
### Fixed
- Unexpected `Failed to obtain PBKDF2 salt` or similar errors during bucket-synchronisation no longer occur.
- Unexpected long delays for chunk-missing documents when using bucket-synchronisation have been resolved.
- Fetched remote chunks are now properly stored in the local database if `Fetch chunks on demand` is enabled.
- The 'fetch' dialogue's message has been refined.
- No longer overwriting any corrupted documents to the storage on boot-sequence.
### Refactored
- Type errors have been corrected.
## 0.25.9
20th August, 2025
### Fixed
- CORS Checking messages now use replacements.
- Configuring CORS setting via the UI now respects the existing rules.
- Now startup-checking works correctly again, performs migration check serially and then it will also fix starting LiveSync or start-up sync. (#696)
- Statusline in editor now supported 'Bases'.
## 0.25.8
18th August, 2025
### New feature
- Insecure chunk detection has been implemented.
- A notification dialogue will be shown if any insecure chunks are detected; these may have been created by v0.25.6 due to its issue. If this dialogue appears, please ensure you rebuild the database after backing it up.
## 0.25.7
15th August, 2025
**Since the release of 0.25.6, there are two large problem. Please update immediately.**
- We may have corrupted some documents during the migration process. **Please check your documents on the wizard.**
- Due to a chunk ID assignment issue, some data has not been encrypted. **Please rebuild the database using Rebuild Everything** if you have enabled E2EE.
**_So, If you have enabled E2EE, please perform `Rebuild everything`. If not, please check your documents on the wizard._**
In next version, insecure chunk detection will be implemented.
### Fixed
- Off-loaded chunking have been fixed to ensure proper functionality (#693).
- Chunk document ID assignment has been fixed.
- Replication prevention message during version up detection has been improved (#686).
- `Keep A` and `Keep B` on Conflict resolving dialogue has been renamed to `Use Base` and `Use Conflicted` (#691).
### Improved
- Metadata and content-size unmatched documents are now detected and reported, prevented to be applied to the storage.
- This behaviour can be configured in `Patch` -> `Edge case addressing (Behaviour)` -> `Process files even if seems to be corrupted`
- Note: this toggle is for the direct-database-manipulation users.
### New Features
- `Scan for Broken files` has been implemented on `Hatch` -> `TroubleShooting`.
### Refactored
- Off-loaded processes have been refactored for the better maintainability.
- Files prefixed `bg.worker` are now work on the worker threads.
- Files prefixed `bgWorker.` are now also controls these worker threads. (I know what you want to say... I will rename them).
- Removed unused code.
## ~~0.25.5~~ 0.25.6
(0.25.5 has been withdrawn due to a bug in the `Fetch chunks on demand` feature).