(test): add local Obsidian E2E suite

This commit is contained in:
vorotamoroz
2026-06-30 08:36:29 +00:00
parent e916683b8d
commit 54e6a761e5
18 changed files with 975 additions and 119 deletions
+26 -6
View File
@@ -20,6 +20,8 @@ Obsidian 1.12 stores the global community plug-in switch outside `.obsidian/comm
Future workflows should use `startObsidianLiveSyncSession()` from `runner/session.ts` rather than repeating the launch and plug-in readiness sequence.
Each test vault uses an isolated Obsidian profile. The runner creates temporary directories for `HOME`, `XDG_CONFIG_HOME`, `XDG_CACHE_HOME`, `XDG_DATA_HOME`, and Electron `--user-data-dir`, writes the vault registry into those directories, pre-seeds the temporary Chromium local storage so community plug-ins are trusted for that generated vault ID, and passes the same environment to `obsidian-cli`. This is intended to keep real Obsidian E2E runs separate from a developer's daily Obsidian profile and vault registry.
## Local Setup
Set `OBSIDIAN_BINARY` when Obsidian is not installed in a standard location.
@@ -32,11 +34,11 @@ npm run test:e2e:obsidian:install-appimage
The script downloads Obsidian `1.12.7` for the current architecture, stores it in `_testdata/obsidian`, and extracts it to `_testdata/obsidian/squashfs-root`. The runner checks `_testdata/obsidian/squashfs-root/obsidian` before the AppImage path.
Do not download the AppImage on every CI run. Prefer one of these approaches:
These tests are intended for local verification, not the default CI gate. Reuse the installed Obsidian application, or reuse the extracted AppImage directory between local runs:
- set `OBSIDIAN_BINARY` to a pre-installed Obsidian executable,
- restore `_testdata/obsidian/squashfs-root` from a CI cache, or
- run `test:e2e:obsidian:install-appimage` only in a manually triggered preparation job.
- set `OBSIDIAN_BINARY` to an installed Obsidian executable,
- keep `_testdata/obsidian/squashfs-root` after running the AppImage installer, or
- run `test:e2e:obsidian:install-appimage` again only when the local Obsidian version should change.
## Commands
@@ -47,18 +49,25 @@ npm run test:e2e:obsidian:cli-help -- vaults verbose
npm run test:e2e:obsidian:smoke
npm run test:e2e:obsidian:vault-reflection
npm run test:e2e:obsidian:couchdb-upload
npm run test:e2e:obsidian:minio-upload
npm run test:e2e:obsidian:startup-scan
npm run test:e2e:obsidian:two-vault-sync
npm run test:e2e:obsidian:hidden-file-snippet-sync
npm run test:e2e:obsidian:customisation-sync
npm run test:e2e:obsidian:setting-markdown-export
npm run test:e2e:obsidian:local-suite
npm run test:e2e:obsidian:local-suite:services
```
`test:e2e:obsidian:local-suite` runs `npm run build`, discovery, smoke, vault reflection, CouchDB upload, Object Storage upload, startup scan, two-vault synchronisation, Hidden File Sync, Customisation Sync, and setting Markdown export in sequence. Start the local CouchDB and MinIO fixtures before running it, or use `test:e2e:obsidian:local-suite:services` to let the wrapper stop leftover fixtures, start fresh fixtures, and stop them again after the run.
`test:e2e:obsidian:couchdb-upload` reuses the CouchDB variables from `.test.env` or the process environment. It expects a reachable CouchDB service, creates a unique database, configures Self-hosted LiveSync through `obsidian-cli eval`, creates a note in real Obsidian, commits the note into the local database, runs one-shot synchronisation, and verifies that the remote database contains both the metadata document and its chunk documents.
`test:e2e:obsidian:minio-upload` reuses the Object Storage variables from `.test.env` or the process environment. It expects a reachable S3-compatible service, configures Self-hosted LiveSync for Object Storage through `obsidian-cli eval`, creates a note in real Obsidian, runs one-shot Journal Sync, and verifies through the AWS SDK that objects were written under a unique bucket prefix.
`test:e2e:obsidian:startup-scan` configures a temporary CouchDB database, stops Obsidian, writes a note directly into the vault, restarts Obsidian, and verifies from CouchDB that the boot-time scan picked up the offline file.
`test:e2e:obsidian:two-vault-sync` runs a two-vault note synchronisation workflow. It verifies note creation, update, deletion, Markdown conflict automatic merging with the merged result propagated by a second synchronisation, and per-device target filters where one vault ignores a note that the other vault synchronises.
`test:e2e:obsidian:two-vault-sync` runs a two-vault note synchronisation workflow. It verifies note creation, update, rename, deletion, per-device target filters where one vault ignores a note that the other vault synchronises, and a separate encrypted round-trip with Path Obfuscation enabled. The optional Markdown conflict automatic merge check can be enabled with `E2E_OBSIDIAN_INCLUDE_MARKDOWN_CONFLICT=true`, but it is not part of the default local suite.
`test:e2e:obsidian:hidden-file-snippet-sync` runs a two-vault hidden file round-trip. It verifies creation and deletion of a real `.obsidian/snippets/*.css` file, automatic JSON conflict merging for a hidden file with the merged result propagated by a second synchronisation, manual JSON Resolve dialogue application through Obsidian's UI, and per-device target patterns where one vault ignores a hidden file that the other vault synchronises.
@@ -66,10 +75,18 @@ npm run test:e2e:obsidian:setting-markdown-export
`test:e2e:obsidian:setting-markdown-export` enables setting Markdown export, waits for the generated Markdown file in the vault, and verifies that credentials are omitted when `writeCredentialsForSettingSync=false`.
Start the local CouchDB fixture first when one is not already running:
Start the local fixtures first when they are not already running:
```bash
npm run test:docker-couchdb:start
npm run test:docker-s3:start
npm run test:e2e:obsidian:local-suite
```
Or let the wrapper manage both fixtures:
```bash
npm run test:e2e:obsidian:local-suite:services
```
Useful environment variables:
@@ -89,10 +106,13 @@ Useful environment variables:
- `E2E_OBSIDIAN_CORE_READY_TIMEOUT_MS`: timeout for waiting until Self-hosted LiveSync reports that its core lifecycle and local database are ready.
- `E2E_OBSIDIAN_LOCAL_DB_TIMEOUT_MS`: timeout for waiting until a file appears in Self-hosted LiveSync's local database.
- `E2E_OBSIDIAN_COUCHDB_TIMEOUT_MS`: timeout for waiting until CouchDB contains uploaded E2E documents.
- `E2E_OBSIDIAN_OBJECT_STORAGE_TIMEOUT_MS`: timeout for waiting until Object Storage contains uploaded E2E objects.
- `E2E_OBSIDIAN_KEEP_COUCHDB=true`: keep the temporary CouchDB database for inspection.
- `E2E_OBSIDIAN_KEEP_OBJECT_STORAGE=true`: keep the temporary Object Storage prefix for inspection.
- `E2E_OBSIDIAN_STARTUP_GRACE_MS`: early process-exit detection window in milliseconds.
- `E2E_OBSIDIAN_KEEP_VAULT=true`: keep the temporary vault for inspection.
- `E2E_OBSIDIAN_USE_XVFB=false`: disable automatic `xvfb-run` on headless Linux.
- `E2E_OBSIDIAN_USE_USER_DATA_DIR=false`: disable the isolated Electron `--user-data-dir` argument. This is not recommended for normal local testing.
- `E2E_OBSIDIAN_ARGS`: override the default Obsidian launch arguments.
On headless Linux, the runner automatically uses `/usr/bin/xvfb-run` when no `DISPLAY` or `WAYLAND_DISPLAY` is present.
+3 -2
View File
@@ -71,9 +71,10 @@ export async function openVaultWithObsidianCli(
export async function evalObsidianJson<T>(
cliBinary: string,
code: string,
env: NodeJS.ProcessEnv = process.env
env: NodeJS.ProcessEnv = process.env,
timeoutMs?: number
): Promise<T> {
const result = await runObsidianCli(cliBinary, ["eval", `code=${code}`], env);
const result = await runObsidianCli(cliBinary, ["eval", `code=${code}`], env, timeoutMs);
if (result.code !== 0) {
throw new Error(
[
+85 -8
View File
@@ -1,11 +1,13 @@
import { spawn, type ChildProcess } from "node:child_process";
import { execFile, spawn, type ChildProcess } from "node:child_process";
import { once } from "node:events";
import { existsSync } from "node:fs";
import { dirname } from "node:path";
import { platform } from "node:process";
import { promisify } from "node:util";
export type ObsidianProcess = {
process: ChildProcess;
output: () => { stdout: string; stderr: string };
stop: () => Promise<void>;
};
@@ -14,10 +16,14 @@ export type LaunchObsidianOptions = {
vaultPath: string;
homePath?: string;
xdgConfigPath?: string;
xdgCachePath?: string;
xdgDataPath?: string;
userDataPath?: string;
startupGraceMs?: number;
};
const execFileAsync = promisify(execFile);
function splitArgs(args: string): string[] {
return args.split(" ").filter((arg) => arg.length > 0);
}
@@ -31,9 +37,13 @@ function launchArgs(options: LaunchObsidianOptions): string[] {
"--no-sandbox",
"--disable-gpu",
"--disable-software-rasterizer",
...(process.env.E2E_OBSIDIAN_USE_USER_DATA_DIR === "true" && options.userDataPath
...(process.env.E2E_OBSIDIAN_USE_USER_DATA_DIR !== "false" && options.userDataPath
? [`--user-data-dir=${options.userDataPath}`]
: []),
...(process.env.E2E_OBSIDIAN_REMOTE_DEBUGGING_PORT
? [`--remote-debugging-port=${process.env.E2E_OBSIDIAN_REMOTE_DEBUGGING_PORT}`]
: []),
`obsidian://open?path=${encodeURIComponent(options.vaultPath)}`,
];
}
@@ -47,7 +57,63 @@ function shouldUseXvfb(): boolean {
return platform === "linux" && existsSync("/usr/bin/xvfb-run");
}
async function listChildPids(pid: number): Promise<number[]> {
if (platform === "win32") {
return [];
}
const { stdout } = await execFileAsync("ps", ["-o", "pid=", "--ppid", String(pid)]).catch(() => ({
stdout: "",
}));
const directChildren = stdout
.split("\n")
.map((line) => Number(line.trim()))
.filter((childPid) => Number.isInteger(childPid) && childPid > 0);
const descendants = await Promise.all(directChildren.map((childPid) => listChildPids(childPid)));
return [...directChildren, ...descendants.flat()];
}
async function killPids(pids: number[], signal: NodeJS.Signals): Promise<void> {
for (const pid of pids) {
if (pid === process.pid) {
continue;
}
try {
process.kill(pid, signal);
} catch {
// The process may have exited between discovery and signalling.
}
}
}
async function waitForExit(exitPromise: Promise<unknown>, timeoutMs: number): Promise<"exited" | "timeout"> {
const stopTimer = new Promise<"timeout">((resolve) => {
setTimeout(() => resolve("timeout"), timeoutMs);
});
const stopResult = await Promise.race([exitPromise.then(() => "exited" as const), stopTimer]);
return stopResult;
}
export async function cleanupStaleObsidianE2EProcesses(): Promise<void> {
if (process.env.E2E_OBSIDIAN_CLEANUP_STALE_PROCESSES === "false" || platform === "win32") {
return;
}
const { stdout } = await execFileAsync("pgrep", ["-f", "obsidian-livesync-e2e-state"]).catch(() => ({
stdout: "",
}));
const pids = stdout
.split("\n")
.map((line) => Number(line.trim()))
.filter((pid) => Number.isInteger(pid) && pid > 0 && pid !== process.pid);
if (pids.length === 0) {
return;
}
await killPids(pids, "SIGTERM");
await new Promise((resolve) => setTimeout(resolve, 1000));
await killPids(pids, "SIGKILL");
}
export async function launchObsidian(options: LaunchObsidianOptions): Promise<ObsidianProcess> {
await cleanupStaleObsidianE2EProcesses();
const startupGraceMs = options.startupGraceMs ?? 1000;
const args = launchArgs(options);
const useXvfb = shouldUseXvfb();
@@ -61,6 +127,8 @@ export async function launchObsidian(options: LaunchObsidianOptions): Promise<Ob
...process.env,
...(options.homePath ? { HOME: options.homePath } : {}),
...(options.xdgConfigPath ? { XDG_CONFIG_HOME: options.xdgConfigPath } : {}),
...(options.xdgCachePath ? { XDG_CACHE_HOME: options.xdgCachePath } : {}),
...(options.xdgDataPath ? { XDG_DATA_HOME: options.xdgDataPath } : {}),
OBSIDIAN_DISABLE_GPU: process.env.OBSIDIAN_DISABLE_GPU ?? "1",
},
});
@@ -93,25 +161,34 @@ export async function launchObsidian(options: LaunchObsidianOptions): Promise<Ob
return {
process: child,
output: () => ({ stdout, stderr }),
stop: async () => {
if (child.exitCode !== null || child.signalCode !== null) {
return;
}
const descendantPids = child.pid ? await listChildPids(child.pid) : [];
if (child.pid) {
process.kill(-child.pid, "SIGTERM");
try {
process.kill(-child.pid, "SIGTERM");
} catch {
child.kill("SIGTERM");
}
} else {
child.kill("SIGTERM");
}
const stopTimer = new Promise<"timeout">((resolve) => {
setTimeout(() => resolve("timeout"), 5000);
});
const stopResult = await Promise.race([exitPromise, stopTimer]);
await killPids(descendantPids.reverse(), "SIGTERM");
const stopResult = await waitForExit(exitPromise, 5000);
if (stopResult === "timeout") {
if (child.pid) {
process.kill(-child.pid, "SIGKILL");
try {
process.kill(-child.pid, "SIGKILL");
} catch {
child.kill("SIGKILL");
}
} else {
child.kill("SIGKILL");
}
await killPids(descendantPids, "SIGKILL");
await exitPromise;
}
},
+77 -12
View File
@@ -1,13 +1,18 @@
import { evalObsidianJson } from "./cli.ts";
import type { CouchDbConfig } from "./couchdb.ts";
import type { ObjectStorageConfig } from "./objectStorage.ts";
export type ConfiguredSettings = {
isConfigured: boolean;
liveSync: boolean;
syncOnStart: boolean;
syncOnSave: boolean;
remoteType: string;
couchDB_URI: string;
couchDB_DBNAME: string;
endpoint?: string;
bucket?: string;
bucketPrefix?: string;
};
export type CoreReadiness = {
@@ -23,6 +28,30 @@ export type LocalDatabaseEntry = {
children: string[];
};
function e2ePreferredSettingsSource(): string[] {
return [
"liveSync:false,",
"syncOnStart:false,",
"syncOnSave:false,",
"usePluginSync:false,",
"usePluginSyncV2:true,",
"useEden:false,",
"customChunkSize:60,",
"sendChunksBulk:false,",
"sendChunksBulkMaxSize:1,",
"chunkSplitterVersion:'v3-rabin-karp',",
"readChunksOnline:true,",
"disableCheckingConfigMismatch:false,",
"enableCompression:false,",
"hashAlg:'xxhash64',",
"handleFilenameCaseSensitive:false,",
"doNotUseFixedRevisionForChunks:true,",
"E2EEAlgorithm:'v2',",
"doctorProcessedVersion:'0.25.27',",
"isConfigured:true,",
];
}
export function assertEqual(actual: unknown, expected: unknown, message: string): void {
if (actual !== expected) {
throw new Error(`${message}\nExpected: ${String(expected)}\nActual: ${String(actual)}`);
@@ -47,18 +76,7 @@ export async function configureCouchDb(
`couchDB_PASSWORD:${JSON.stringify(settings.password)},`,
`couchDB_DBNAME:${JSON.stringify(settings.dbName)},`,
"remoteType:'',",
"liveSync:false,",
"syncOnStart:false,",
"syncOnSave:false,",
"usePluginSync:false,",
"usePluginSyncV2:true,",
"useEden:false,",
"customChunkSize:1,",
"sendChunksBulkMaxSize:1,",
"chunkSplitterVersion:'v3-rabin-karp',",
"readChunksOnline:false,",
"disableCheckingConfigMismatch:true,",
"isConfigured:true,",
...e2ePreferredSettingsSource(),
...Object.entries(overrides).map(([key, value]) => `${JSON.stringify(key)}:${JSON.stringify(value)},`),
"};",
"await core.services.setting.applyExternalSettings(nextSettings,true);",
@@ -69,6 +87,7 @@ export async function configureCouchDb(
"liveSync:current.liveSync,",
"syncOnStart:current.syncOnStart,",
"syncOnSave:current.syncOnSave,",
"remoteType:current.remoteType,",
"couchDB_URI:current.couchDB_URI,",
"couchDB_DBNAME:current.couchDB_DBNAME,",
"});",
@@ -78,6 +97,52 @@ export async function configureCouchDb(
);
}
export async function configureObjectStorage(
cliBinary: string,
env: NodeJS.ProcessEnv,
settings: ObjectStorageConfig & { bucketPrefix: string },
overrides: Record<string, unknown> = {}
): Promise<ConfiguredSettings> {
return await evalObsidianJson<ConfiguredSettings>(
cliBinary,
[
"(async()=>{",
"const plugin=app.plugins.plugins['obsidian-livesync'];",
"const core=plugin.core;",
"const nextSettings={",
"remoteType:'MINIO',",
`endpoint:${JSON.stringify(settings.endpoint)},`,
`accessKey:${JSON.stringify(settings.accessKey)},`,
`secretKey:${JSON.stringify(settings.secretKey)},`,
`bucket:${JSON.stringify(settings.bucket)},`,
`region:${JSON.stringify(settings.region)},`,
`forcePathStyle:${JSON.stringify(settings.forcePathStyle)},`,
`bucketPrefix:${JSON.stringify(settings.bucketPrefix)},`,
"bucketCustomHeaders:'',",
...e2ePreferredSettingsSource(),
...Object.entries(overrides).map(([key, value]) => `${JSON.stringify(key)}:${JSON.stringify(value)},`),
"};",
"await core.services.setting.applyExternalSettings(nextSettings,true);",
"await core.services.control.applySettings();",
"const current=core.services.setting.currentSettings();",
"return JSON.stringify({",
"isConfigured:current.isConfigured,",
"liveSync:current.liveSync,",
"syncOnStart:current.syncOnStart,",
"syncOnSave:current.syncOnSave,",
"remoteType:current.remoteType,",
"couchDB_URI:current.couchDB_URI,",
"couchDB_DBNAME:current.couchDB_DBNAME,",
"endpoint:current.endpoint,",
"bucket:current.bucket,",
"bucketPrefix:current.bucketPrefix,",
"});",
"})()",
].join(""),
env
);
}
export async function waitForLiveSyncCoreReady(
cliBinary: string,
env: NodeJS.ProcessEnv,
+142
View File
@@ -0,0 +1,142 @@
import {
CreateBucketCommand,
DeleteObjectsCommand,
ListObjectsV2Command,
S3Client,
type _Object,
} from "@aws-sdk/client-s3";
import { readFile } from "node:fs/promises";
import { resolve } from "node:path";
export type ObjectStorageConfig = {
endpoint: string;
accessKey: string;
secretKey: string;
bucket: string;
region: string;
forcePathStyle: boolean;
};
function parseEnvFile(content: string): Record<string, string> {
const entries = content
.split(/\r?\n/u)
.map((line) => line.trim())
.filter((line) => line && !line.startsWith("#"))
.map((line) => {
const equalsAt = line.indexOf("=");
if (equalsAt < 0) {
return undefined;
}
const key = line.slice(0, equalsAt).trim();
const rawValue = line.slice(equalsAt + 1).trim();
const value = rawValue.replace(/^['"]|['"]$/gu, "");
return [key, value] as const;
})
.filter((entry): entry is readonly [string, string] => entry !== undefined);
return Object.fromEntries(entries);
}
function getEnvValue(values: Record<string, string | undefined>, ...keys: string[]): string {
for (const key of keys) {
const value = values[key]?.trim();
if (value) {
return value;
}
}
throw new Error(`Required Object Storage environment value is missing: ${keys.join(" or ")}`);
}
export async function loadObjectStorageConfig(envFile = ".test.env"): Promise<ObjectStorageConfig> {
let fileValues: Record<string, string> = {};
try {
fileValues = parseEnvFile(await readFile(resolve(envFile), "utf-8"));
} catch (error) {
if ((error as NodeJS.ErrnoException).code !== "ENOENT") {
throw error;
}
}
const values = { ...fileValues, ...process.env };
return {
endpoint: getEnvValue(values, "MINIO_ENDPOINT", "minioEndpoint").replace(/\/+$/u, ""),
accessKey: getEnvValue(values, "MINIO_ACCESS_KEY", "accessKey"),
secretKey: getEnvValue(values, "MINIO_SECRET_KEY", "secretKey"),
bucket: getEnvValue(values, "MINIO_BUCKET", "bucketName"),
region: values.MINIO_REGION?.trim() || values.region?.trim() || "us-east-1",
forcePathStyle: values.MINIO_FORCE_PATH_STYLE?.trim() !== "false",
};
}
export function makeUniqueBucketPrefix(label: string): string {
const random = Math.random().toString(36).slice(2, 8);
return `obsidian-e2e/${label}-${Date.now()}-${random}/`;
}
export function createObjectStorageClient(config: ObjectStorageConfig): S3Client {
return new S3Client({
endpoint: config.endpoint,
region: config.region,
forcePathStyle: config.forcePathStyle,
credentials: {
accessKeyId: config.accessKey,
secretAccessKey: config.secretKey,
},
});
}
export async function ensureObjectStorageBucket(config: ObjectStorageConfig): Promise<void> {
const client = createObjectStorageClient(config);
try {
await client.send(new CreateBucketCommand({ Bucket: config.bucket }));
} catch (error) {
const name = (error as { name?: string }).name;
if (name !== "BucketAlreadyOwnedByYou" && name !== "BucketAlreadyExists") {
throw error;
}
} finally {
client.destroy();
}
}
export async function listObjectStorageObjects(config: ObjectStorageConfig, prefix: string): Promise<_Object[]> {
const client = createObjectStorageClient(config);
try {
const objects: _Object[] = [];
let continuationToken: string | undefined;
do {
const response = await client.send(
new ListObjectsV2Command({
Bucket: config.bucket,
Prefix: prefix,
ContinuationToken: continuationToken,
})
);
objects.push(...(response.Contents ?? []));
continuationToken = response.NextContinuationToken;
} while (continuationToken);
return objects;
} finally {
client.destroy();
}
}
export async function deleteObjectStoragePrefix(config: ObjectStorageConfig, prefix: string): Promise<void> {
const client = createObjectStorageClient(config);
try {
const objects = await listObjectStorageObjects(config, prefix);
const keys = objects.flatMap((object) => (object.Key ? [{ Key: object.Key }] : []));
for (let index = 0; index < keys.length; index += 1000) {
await client.send(
new DeleteObjectsCommand({
Bucket: config.bucket,
Delete: {
Objects: keys.slice(index, index + 1000),
Quiet: true,
},
})
);
}
} finally {
client.destroy();
}
}
+32 -7
View File
@@ -1,8 +1,9 @@
import { openVaultWithObsidianCli, runObsidianCli } from "./cli.ts";
import { evalObsidianJson, openVaultWithObsidianCli, runObsidianCli } from "./cli.ts";
import { launchObsidian, type ObsidianProcess } from "./launch.ts";
import { installBuiltPlugin, type PluginInstallResult } from "./pluginInstaller.ts";
import { waitForPluginReady, type PluginReadiness } from "./readiness.ts";
import type { TemporaryVault } from "./vault.ts";
import { obsidianRemoteDebuggingPort, preseedTrustedVaultState, trustVaultIfPrompted } from "./ui.ts";
export type ObsidianLiveSyncSession = {
app: ObsidianProcess;
@@ -19,13 +20,21 @@ export type StartObsidianLiveSyncSessionOptions = {
};
async function waitForPluginCatalogue(cliBinary: string, env: NodeJS.ProcessEnv): Promise<void> {
const deadline = Date.now() + Number(process.env.E2E_OBSIDIAN_CLI_READY_TIMEOUT_MS ?? 15000);
const deadline = Date.now() + Number(process.env.E2E_OBSIDIAN_CLI_READY_TIMEOUT_MS ?? 60000);
let lastOutput = "";
while (Date.now() < deadline) {
const result = await runObsidianCli(cliBinary, ["plugins", "filter=community"], env);
lastOutput = [result.stdout, result.stderr].filter(Boolean).join("\n");
if (result.stdout.includes("obsidian-livesync")) {
return;
try {
const result = await evalObsidianJson<{ hasLiveSync: boolean }>(
cliBinary,
["JSON.stringify({", "hasLiveSync:!!app.plugins?.manifests?.['obsidian-livesync']", "})"].join(""),
env
);
lastOutput = JSON.stringify(result);
if (result.hasLiveSync) {
return;
}
} catch (error) {
lastOutput = error instanceof Error ? error.message : String(error);
}
await new Promise((resolve) => setTimeout(resolve, 500));
}
@@ -66,11 +75,14 @@ export async function startObsidianLiveSyncSession(
options: StartObsidianLiveSyncSessionOptions
): Promise<ObsidianLiveSyncSession> {
const install = await installBuiltPlugin(options.vault.path);
const remoteDebuggingPort = obsidianRemoteDebuggingPort();
const app = await launchObsidian({
binary: options.binary,
vaultPath: options.vault.path,
homePath: options.vault.homePath,
xdgConfigPath: options.vault.xdgConfigPath,
xdgCachePath: options.vault.xdgCachePath,
xdgDataPath: options.vault.xdgDataPath,
userDataPath: options.vault.userDataPath,
startupGraceMs: options.startupGraceMs,
});
@@ -78,17 +90,30 @@ export async function startObsidianLiveSyncSession(
...process.env,
HOME: options.vault.homePath,
XDG_CONFIG_HOME: options.vault.xdgConfigPath,
XDG_CACHE_HOME: options.vault.xdgCachePath,
XDG_DATA_HOME: options.vault.xdgDataPath,
};
try {
await preseedTrustedVaultState(remoteDebuggingPort, options.vault.id);
await openVaultWithObsidianCli(options.cliBinary, options.vault.path, cliEnv);
await trustVaultIfPrompted(remoteDebuggingPort);
await waitForPluginCatalogue(options.cliBinary, cliEnv);
await enableCommunityPlugins(options.cliBinary, cliEnv);
await reloadLiveSyncPlugin(options.cliBinary, cliEnv);
const readiness = await waitForPluginReady(options.cliBinary, cliEnv);
return { app, cliEnv, install, readiness };
} catch (error) {
const output = app.output();
await app.stop();
throw error;
throw new Error(
[
error instanceof Error ? error.message : String(error),
output.stdout ? `Obsidian stdout:\n${output.stdout}` : undefined,
output.stderr ? `Obsidian stderr:\n${output.stderr}` : undefined,
]
.filter(Boolean)
.join("\n")
);
}
}
+83
View File
@@ -0,0 +1,83 @@
import { chromium, type Page } from "playwright";
export function obsidianRemoteDebuggingPort(): number {
const port = Number(process.env.E2E_OBSIDIAN_REMOTE_DEBUGGING_PORT ?? 9222);
process.env.E2E_OBSIDIAN_REMOTE_DEBUGGING_PORT = String(port);
return port;
}
async function waitForCdp(port: number): Promise<void> {
const deadline = Date.now() + Number(process.env.E2E_OBSIDIAN_CDP_TIMEOUT_MS ?? 30000);
while (Date.now() < deadline) {
try {
const response = await fetch(`http://127.0.0.1:${port}/json/version`);
if (response.ok) {
return;
}
} catch {
// Keep polling until Obsidian exposes the debugging endpoint.
}
await new Promise((resolve) => setTimeout(resolve, 500));
}
throw new Error(`Timed out waiting for Obsidian DevTools endpoint on port ${port}`);
}
export async function withObsidianPage<T>(port: number, operation: (page: Page) => Promise<T>): Promise<T> {
await waitForCdp(port);
const browser = await chromium.connectOverCDP(`http://127.0.0.1:${port}`);
try {
const context = browser.contexts()[0];
const page = context.pages()[0] ?? (await context.waitForEvent("page", { timeout: 10000 }));
return await operation(page);
} finally {
await browser.close();
}
}
export async function preseedTrustedVaultState(port: number, vaultId: string): Promise<void> {
await withObsidianPage(port, async (page) => {
await page.evaluate((id) => {
localStorage.setItem(`enable-plugin-${id}`, "true");
}, vaultId);
await page.reload({ waitUntil: "domcontentloaded", timeout: 10000 }).catch(() => undefined);
await page.waitForTimeout(1000);
});
}
export async function trustVaultIfPrompted(port: number): Promise<void> {
await withObsidianPage(port, async (page) => {
const deadline = Date.now() + Number(process.env.E2E_OBSIDIAN_TRUST_PROMPT_TIMEOUT_MS ?? 30000);
while (Date.now() < deadline) {
const yesButton = page.getByRole("button", { name: "Yes" });
if (await yesButton.isVisible({ timeout: 1000 }).catch(() => false)) {
await yesButton.click();
await page.waitForTimeout(500);
continue;
}
const trustButton = page.getByText("Trust author and enable plugins");
if (await trustButton.isVisible({ timeout: 1000 }).catch(() => false)) {
await trustButton.click();
await page.waitForTimeout(500);
continue;
}
const workspace = page.locator(".workspace");
if (await workspace.isVisible({ timeout: 1000 }).catch(() => false)) {
return;
}
}
});
}
export async function clickJsonResolveOption(port: number, mode: "AB" | "BA"): Promise<void> {
await withObsidianPage(port, async (page) => {
const option = page.locator(`label:has(input[name="disp"][value="${mode}"])`);
await option.click({ timeout: 10000 });
const checked = await page.locator(`input[name="disp"][value="${mode}"]`).isChecked({ timeout: 10000 });
if (!checked) {
throw new Error(`JSON Resolve option was not selected: ${mode}`);
}
await page.getByRole("button", { name: "Apply" }).click({ timeout: 10000 });
});
}
+25 -8
View File
@@ -5,8 +5,11 @@ import { tmpdir } from "node:os";
export type TemporaryVault = {
path: string;
name: string;
id: string;
homePath: string;
xdgConfigPath: string;
xdgCachePath: string;
xdgDataPath: string;
userDataPath: string;
dispose: () => Promise<void>;
};
@@ -18,21 +21,33 @@ export async function createTemporaryVault(prefix = "obsidian-livesync-e2e-"): P
await mkdir(join(vaultPath, ".obsidian"), { recursive: true });
const homePath = join(statePath, "home");
const xdgConfigPath = join(statePath, "xdg-config");
const xdgCachePath = join(statePath, "xdg-cache");
const xdgDataPath = join(statePath, "xdg-data");
const userDataPath = join(statePath, "user-data");
const id = `livesync-e2e-${Date.now()}`;
await mkdir(homePath, { recursive: true });
await mkdir(xdgConfigPath, { recursive: true });
await mkdir(xdgCachePath, { recursive: true });
await mkdir(xdgDataPath, { recursive: true });
await mkdir(userDataPath, { recursive: true });
await writeFile(
join(vaultPath, ".obsidian", "app.json"),
JSON.stringify({ legacyEditor: false, safeMode: false }, null, 4)
);
await writeObsidianVaultRegistry(vaultPath, name, homePath, xdgConfigPath, userDataPath);
await writeFile(
join(vaultPath, ".obsidian", "community-plugins.json"),
JSON.stringify(["obsidian-livesync"], null, 4)
);
await writeObsidianVaultRegistry(id, vaultPath, name, homePath, xdgConfigPath, userDataPath);
return {
path: vaultPath,
name,
id,
homePath,
xdgConfigPath,
xdgCachePath,
xdgDataPath,
userDataPath,
dispose: async () => {
if (process.env.E2E_OBSIDIAN_KEEP_VAULT === "true") {
@@ -49,22 +64,23 @@ export async function createTemporaryVault(prefix = "obsidian-livesync-e2e-"): P
}
async function writeObsidianVaultRegistry(
vaultId: string,
vaultPath: string,
vaultName: string,
homePath: string,
xdgConfigPath: string,
userDataPath: string
): Promise<void> {
const vaultId = `livesync-e2e-${Date.now()}`;
const vaultRecord = {
path: vaultPath,
ts: Date.now(),
open: true,
name: vaultName,
};
const registry = {
cli: true,
vaults: {
[vaultId]: {
path: vaultPath,
ts: Date.now(),
open: true,
name: vaultName,
},
[vaultId]: vaultRecord,
},
};
const registryText = JSON.stringify(registry, null, 4);
@@ -74,4 +90,5 @@ async function writeObsidianVaultRegistry(
await writeFile(join(obsidianConfigDir, "obsidian.json"), registryText);
}
await writeFile(join(userDataPath, "obsidian.json"), registryText);
await writeFile(join(userDataPath, `${vaultId}.json`), JSON.stringify(vaultRecord, null, 4));
}
+2
View File
@@ -25,6 +25,8 @@ async function main(): Promise<void> {
...process.env,
HOME: vault.homePath,
XDG_CONFIG_HOME: vault.xdgConfigPath,
XDG_CACHE_HOME: vault.xdgCachePath,
XDG_DATA_HOME: vault.xdgDataPath,
};
await runObsidianCli(cli.binary, [`obsidian://open?path=${encodeURIComponent(vault.path)}`], cliEnv);
await new Promise((resolve) => setTimeout(resolve, 3000));
@@ -228,10 +228,12 @@ async function storeCustomisationFile(cliBinary: string, env: NodeJS.ProcessEnv,
"const result=await addOn.storeCustomizationFiles(path,term);",
"const rows=(await core.localDatabase.allDocsRaw({include_docs:true})).rows;",
"const entries=rows.map((row)=>row.doc).filter((doc)=>doc?.path?.startsWith('ix:')).map((doc)=>doc.path);",
"if(!result){",
"const filename=path.split('/').pop();",
"const existing=entries.some((entry)=>entry.startsWith(`ix:${term}/${category}/`)&&entry.endsWith(`%${filename}`));",
"if(!result&&!existing){",
" throw new Error(`Could not store Customisation Sync file: path=${path}; term=${term}; category=${category}; stat=${JSON.stringify(stat)}; result=${JSON.stringify(result)}; entries=${JSON.stringify(entries)}`);",
"}",
"return JSON.stringify({ok:true,path,term,category,entries});",
"return JSON.stringify({ok:true,path,term,category,result:!!result,existing,entries});",
"})()",
].join(""),
env
+60
View File
@@ -0,0 +1,60 @@
import { launchObsidian } from "../runner/launch.ts";
import { installBuiltPlugin } from "../runner/pluginInstaller.ts";
import { createTemporaryVault } from "../runner/vault.ts";
import { requireObsidianBinary } from "../runner/environment.ts";
import { writeFile } from "node:fs/promises";
import { obsidianRemoteDebuggingPort, preseedTrustedVaultState, withObsidianPage } from "../runner/ui.ts";
const port = obsidianRemoteDebuggingPort();
async function main(): Promise<void> {
const binary = requireObsidianBinary();
const vault = await createTemporaryVault();
await installBuiltPlugin(vault.path);
const app = await launchObsidian({
binary,
vaultPath: vault.path,
homePath: vault.homePath,
xdgConfigPath: vault.xdgConfigPath,
xdgCachePath: vault.xdgCachePath,
xdgDataPath: vault.xdgDataPath,
userDataPath: vault.userDataPath,
startupGraceMs: Number(process.env.E2E_OBSIDIAN_STARTUP_GRACE_MS ?? 1000),
});
try {
await preseedTrustedVaultState(port, vault.id);
const { screenshotPath, textPath } = await withObsidianPage(port, async (page) => {
await page.waitForTimeout(Number(process.env.E2E_OBSIDIAN_DEBUG_WAIT_MS ?? 5000));
const title = await page.title().catch((error: unknown) => `title error: ${String(error)}`);
const url = page.url();
const text = await page
.locator("body")
.innerText({ timeout: 5000 })
.catch((error: unknown) => {
return `body text error: ${String(error)}`;
});
if (process.env.E2E_OBSIDIAN_DEBUG_CLICK_TRUST === "true") {
await page.getByText("Trust author and enable plugins").click({ timeout: 10000 });
await page.waitForTimeout(Number(process.env.E2E_OBSIDIAN_DEBUG_AFTER_CLICK_WAIT_MS ?? 3000));
}
const screenshotPath = process.env.E2E_OBSIDIAN_DEBUG_SCREENSHOT ?? "/tmp/obsidian-e2e-debug.png";
const textPath = process.env.E2E_OBSIDIAN_DEBUG_TEXT ?? "/tmp/obsidian-e2e-debug.txt";
await page.screenshot({ path: screenshotPath, fullPage: true });
await writeFile(textPath, [`title: ${title}`, `url: ${url}`, "", text].join("\n"), "utf-8");
return { screenshotPath, textPath };
});
console.log(`Temporary vault: ${vault.path}`);
console.log(`Temporary Obsidian state: ${vault.userDataPath}`);
console.log(`Debug text: ${textPath}`);
console.log(`Debug screenshot: ${screenshotPath}`);
} finally {
await app.stop();
await vault.dispose();
}
}
main().catch((error: unknown) => {
console.error(error instanceof Error ? error.stack : error);
process.exit(1);
});
@@ -21,6 +21,7 @@ import {
type LocalDatabaseEntry,
} from "../runner/liveSyncWorkflow.ts";
import { startObsidianLiveSyncSession, type ObsidianLiveSyncSession } from "../runner/session.ts";
import { clickJsonResolveOption, obsidianRemoteDebuggingPort } from "../runner/ui.ts";
import { createTemporaryVault, type TemporaryVault } from "../runner/vault.ts";
process.env.E2E_OBSIDIAN_CLI_TIMEOUT_MS ??= "30000";
@@ -41,6 +42,7 @@ const snippetContent = [
const mergeJsonPath = ".obsidian/livesync-e2e-merge.json";
const manualMergeJsonPath = ".obsidian/livesync-e2e-manual-merge.json";
const targetPath = ".obsidian/livesync-targeted/only-a.json";
const hiddenFileCliTimeoutMs = Number(process.env.E2E_OBSIDIAN_HIDDEN_FILE_CLI_TIMEOUT_MS ?? 90000);
type RunnerContext = {
binary: string;
@@ -145,7 +147,8 @@ async function scanHiddenDatabase(cliBinary: string, env: NodeJS.ProcessEnv): Pr
"return JSON.stringify({ok:true});",
"})()",
].join(""),
env
env,
hiddenFileCliTimeoutMs
);
}
@@ -161,7 +164,8 @@ async function resolveHiddenConflicts(cliBinary: string, env: NodeJS.ProcessEnv)
"return JSON.stringify({ok:true});",
"})()",
].join(""),
env
env,
hiddenFileCliTimeoutMs
);
}
@@ -233,34 +237,6 @@ async function openHiddenJsonResolveModal(cliBinary: string, env: NodeJS.Process
);
}
async function clickJsonResolveOption(cliBinary: string, env: NodeJS.ProcessEnv, mode: "AB" | "BA"): Promise<void> {
await evalObsidianJson<unknown>(
cliBinary,
[
"(async()=>{",
`const mode=${JSON.stringify(mode)};`,
"const deadline=Date.now()+10000;",
"while(Date.now()<deadline){",
" const input=[...document.querySelectorAll('input[name=\"disp\"]')].find((candidate)=>candidate.value===mode);",
" const apply=[...document.querySelectorAll('button')].find((button)=>button.textContent?.trim()==='Apply');",
" if(input&&apply){",
" input.click();",
" input.dispatchEvent(new Event('change',{bubbles:true}));",
" await new Promise((resolve)=>setTimeout(resolve,100));",
" apply.click();",
" return JSON.stringify({ok:true});",
" }",
" await new Promise((resolve)=>setTimeout(resolve,250));",
"}",
"const buttons=[...document.querySelectorAll('button')].map((button)=>button.textContent?.trim()).filter(Boolean);",
"const inputs=[...document.querySelectorAll('input[name=\"disp\"]')].map((input)=>input.value);",
"throw new Error(`Timed out waiting for JSON resolve modal; buttons=${JSON.stringify(buttons)}; inputs=${JSON.stringify(inputs)}`);",
"})()",
].join(""),
env
);
}
async function storeHiddenFileAsConflict(
cliBinary: string,
env: NodeJS.ProcessEnv,
@@ -368,9 +344,15 @@ async function uploadHiddenFile(
return entry;
}
async function pullAndApplyHiddenFiles(context: RunnerContext, session: ObsidianLiveSyncSession): Promise<void> {
async function pullAndApplyHiddenFiles(
context: RunnerContext,
session: ObsidianLiveSyncSession,
options: { resolveConflicts?: boolean } = {}
): Promise<void> {
await pushLocalChanges(context.cliBinary, session.cliEnv);
await resolveHiddenConflicts(context.cliBinary, session.cliEnv);
if (options.resolveConflicts === true) {
await resolveHiddenConflicts(context.cliBinary, session.cliEnv);
}
await scanHiddenDatabase(context.cliBinary, session.cliEnv);
}
@@ -449,7 +431,7 @@ async function runJsonManualConflictResolution(context: RunnerContext, vault: Te
const session = await startConfiguredSession(context, vault);
await createHiddenJsonConflict(context, session, vault, manualMergeJsonPath, base, left, right);
await openHiddenJsonResolveModal(context.cliBinary, session.cliEnv, manualMergeJsonPath);
await clickJsonResolveOption(context.cliBinary, session.cliEnv, "AB");
await clickJsonResolveOption(obsidianRemoteDebuggingPort(), "AB");
const merged = await waitForPathContent(vault.path, manualMergeJsonPath, (content) =>
hasJsonValues(content, { shared: "right", fromA: true, fromB: true })
@@ -472,26 +454,36 @@ async function runTargetMismatch(
await writeVaultFile(vaultA.path, targetPath, targetContent);
let session = await startConfiguredSession(context, vaultA);
await uploadHiddenFile(context, session, targetPath);
await session.app.stop();
try {
await uploadHiddenFile(context, session, targetPath);
} finally {
await session.app.stop();
}
session = await startConfiguredSession(context, vaultB, {
syncInternalFilesTargetPatterns: "snippets",
});
await pullAndApplyHiddenFiles(context, session);
assertEqual(
await pathExists(vaultB.path, targetPath),
false,
"Hidden file was applied on a device where it was not a target file."
);
await session.app.stop();
try {
await pullAndApplyHiddenFiles(context, session, { resolveConflicts: false });
assertEqual(
await pathExists(vaultB.path, targetPath),
false,
"Hidden file was applied on a device where it was not a target file."
);
} finally {
await session.app.stop();
}
session = await startConfiguredSession(context, vaultB, {
syncInternalFilesTargetPatterns: "",
});
await pullAndApplyHiddenFiles(context, session);
const received = await waitForPathContent(vaultB.path, targetPath, (content) => content === targetContent);
await session.app.stop();
let received = "";
try {
await pullAndApplyHiddenFiles(context, session, { resolveConflicts: false });
received = await waitForPathContent(vaultB.path, targetPath, (content) => content === targetContent);
} finally {
await session.app.stop();
}
assertEqual(received, targetContent, "Hidden file was not applied after it became a target file.");
console.log("Hidden target mismatch respected per-device target patterns, then applied after enabling the target.");
+105
View File
@@ -0,0 +1,105 @@
import { spawn } from "node:child_process";
type Step = {
name: string;
args: string[];
optional?: boolean;
};
const testSteps: Step[] = [
{ name: "build", args: ["run", "build"] },
{ name: "discover", args: ["run", "test:e2e:obsidian:discover"] },
{ name: "smoke", args: ["run", "test:e2e:obsidian:smoke"] },
{ name: "vault reflection", args: ["run", "test:e2e:obsidian:vault-reflection"] },
{ name: "CouchDB upload", args: ["run", "test:e2e:obsidian:couchdb-upload"] },
{ name: "Object Storage upload", args: ["run", "test:e2e:obsidian:minio-upload"] },
{ name: "startup scan", args: ["run", "test:e2e:obsidian:startup-scan"] },
{ name: "two-vault synchronisation", args: ["run", "test:e2e:obsidian:two-vault-sync"] },
{ name: "hidden file snippet synchronisation", args: ["run", "test:e2e:obsidian:hidden-file-snippet-sync"] },
{ name: "Customisation Sync", args: ["run", "test:e2e:obsidian:customisation-sync"] },
{ name: "setting Markdown export", args: ["run", "test:e2e:obsidian:setting-markdown-export"] },
];
const manageCouchDb = process.argv.includes("--manage-couchdb") || process.argv.includes("--manage-services");
const manageMinio = process.argv.includes("--manage-minio") || process.argv.includes("--manage-services");
const keepServices = process.argv.includes("--keep-services");
const keepCouchDb = keepServices || process.argv.includes("--keep-couchdb");
const keepMinio = keepServices || process.argv.includes("--keep-minio");
function npmBinary(): string {
return process.platform === "win32" ? "npm.cmd" : "npm";
}
function runStep(step: Step): Promise<void> {
return new Promise((resolve, reject) => {
console.log(`\n# ${step.name}`);
const child = spawn(npmBinary(), step.args, {
cwd: process.cwd(),
env: process.env,
stdio: "inherit",
});
child.on("error", reject);
child.on("exit", (code, signal) => {
if (code === 0) {
resolve();
return;
}
const message = `${step.name} failed with ${signal ? `signal ${signal}` : `exit code ${code}`}`;
if (step.optional) {
console.warn(message);
resolve();
return;
}
reject(new Error(message));
});
});
}
async function stopManagedCouchDb(): Promise<void> {
await runStep({
name: "stop CouchDB fixture",
args: ["run", "test:docker-couchdb:stop"],
optional: true,
});
}
async function stopManagedMinio(): Promise<void> {
await runStep({
name: "stop MinIO fixture",
args: ["run", "test:docker-s3:stop"],
optional: true,
});
}
async function main(): Promise<void> {
let shouldStopCouchDb = false;
let shouldStopMinio = false;
try {
if (manageCouchDb) {
await stopManagedCouchDb();
await runStep({ name: "start CouchDB fixture", args: ["run", "test:docker-couchdb:start"] });
shouldStopCouchDb = !keepCouchDb;
}
if (manageMinio) {
await stopManagedMinio();
await runStep({ name: "start MinIO fixture", args: ["run", "test:docker-s3:start"] });
shouldStopMinio = !keepMinio;
}
for (const step of testSteps) {
await runStep(step);
}
} finally {
if (shouldStopMinio) {
await stopManagedMinio();
}
if (shouldStopCouchDb) {
await stopManagedCouchDb();
}
}
}
main().catch((error: unknown) => {
console.error(error instanceof Error ? error.stack : error);
process.exit(1);
});
+142
View File
@@ -0,0 +1,142 @@
import { evalObsidianJson } from "../runner/cli.ts";
import { discoverObsidianCli, requireObsidianBinary } from "../runner/environment.ts";
import {
assertEqual,
configureObjectStorage,
prepareRemote,
pushLocalChanges,
waitForLiveSyncCoreReady,
type LocalDatabaseEntry,
} from "../runner/liveSyncWorkflow.ts";
import {
deleteObjectStoragePrefix,
ensureObjectStorageBucket,
listObjectStorageObjects,
loadObjectStorageConfig,
makeUniqueBucketPrefix,
} from "../runner/objectStorage.ts";
import { startObsidianLiveSyncSession, type ObsidianLiveSyncSession } from "../runner/session.ts";
import { createTemporaryVault } from "../runner/vault.ts";
process.env.E2E_OBSIDIAN_CLI_TIMEOUT_MS ??= "30000";
const notePath = "E2E/minio-upload.md";
const noteContent = [
"# Object Storage upload from real Obsidian",
"",
"This note is created through Obsidian and uploaded by Self-hosted LiveSync to S3-compatible Object Storage.",
"The test is intentionally small, but it crosses the real Obsidian, Journal Sync, and AWS SDK boundary.",
`Created at: ${new Date().toISOString()}`,
"",
].join("\n");
async function createNoteAndWaitForLocalDb(cliBinary: string, env: NodeJS.ProcessEnv): Promise<LocalDatabaseEntry> {
return await evalObsidianJson<LocalDatabaseEntry>(
cliBinary,
[
"(async()=>{",
`const path=${JSON.stringify(notePath)};`,
`const content=${JSON.stringify(noteContent)};`,
"const core=app.plugins.plugins['obsidian-livesync'].core;",
"if(!(await app.vault.adapter.exists('E2E'))) await app.vault.createFolder('E2E');",
"const existing=app.vault.getAbstractFileByPath(path);",
"if(existing) await app.vault.delete(existing);",
"await app.vault.create(path,content);",
"const sleep=(ms)=>new Promise((resolve)=>setTimeout(resolve,ms));",
"let entry=false;",
"for(let i=0;i<40;i++){",
"await core.services.fileProcessing.commitPendingFileEvents();",
"entry=await core.localDatabase.getDBEntry(path,undefined,false,true).catch(()=>false);",
"if(entry&&entry._id&&Array.isArray(entry.children)&&entry.children.length>0) break;",
"await sleep(250);",
"}",
"if(!entry||!entry._id) throw new Error('Timed out waiting for local database entry');",
"return JSON.stringify({id:entry._id,path:entry.path,type:entry.type,children:entry.children||[]});",
"})()",
].join(""),
env
);
}
async function waitForObjectStorageObjects(prefix: string): Promise<string[]> {
const objectStorage = await loadObjectStorageConfig();
const timeoutMs = Number(process.env.E2E_OBSIDIAN_OBJECT_STORAGE_TIMEOUT_MS ?? 20000);
const deadline = Date.now() + timeoutMs;
let keys: string[] = [];
while (Date.now() < deadline) {
const objects = await listObjectStorageObjects(objectStorage, prefix);
keys = objects.flatMap((object) => (object.Key ? [object.Key] : []));
if (keys.length > 0) {
return keys;
}
await new Promise((resolve) => setTimeout(resolve, 500));
}
throw new Error(`Timed out waiting for Object Storage objects under ${prefix}. Last keys: ${keys.join(", ")}`);
}
async function main(): Promise<void> {
const binary = requireObsidianBinary();
const cli = discoverObsidianCli();
if (!cli.binary) {
throw new Error(`Could not find obsidian-cli. Checked paths: ${cli.checked.join(", ")}`);
}
const objectStorage = await loadObjectStorageConfig();
const bucketPrefix = makeUniqueBucketPrefix("minio-upload");
const vault = await createTemporaryVault();
let session: ObsidianLiveSyncSession | undefined;
try {
await ensureObjectStorageBucket(objectStorage);
console.log(`Using Obsidian executable: ${binary}`);
console.log(`Temporary vault: ${vault.path}`);
console.log(`Temporary Object Storage bucket: ${objectStorage.bucket}`);
console.log(`Temporary Object Storage prefix: ${bucketPrefix}`);
session = await startObsidianLiveSyncSession({
binary,
cliBinary: cli.binary,
vault,
startupGraceMs: Number(process.env.E2E_OBSIDIAN_STARTUP_GRACE_MS ?? 1000),
});
await waitForLiveSyncCoreReady(cli.binary, session.cliEnv);
const configured = await configureObjectStorage(cli.binary, session.cliEnv, {
...objectStorage,
bucketPrefix,
});
await waitForLiveSyncCoreReady(cli.binary, session.cliEnv);
assertEqual(configured.isConfigured, true, "Self-hosted LiveSync was not marked as configured.");
assertEqual(configured.remoteType, "MINIO", "Remote type was not Object Storage.");
assertEqual(configured.endpoint, objectStorage.endpoint, "Configured Object Storage endpoint did not match.");
assertEqual(configured.bucket, objectStorage.bucket, "Configured Object Storage bucket did not match.");
assertEqual(configured.bucketPrefix, bucketPrefix, "Configured Object Storage bucket prefix did not match.");
assertEqual(configured.liveSync, false, "LiveSync should remain disabled during this one-shot workflow.");
await prepareRemote(cli.binary, session.cliEnv);
const localEntry = await createNoteAndWaitForLocalDb(cli.binary, session.cliEnv);
await pushLocalChanges(cli.binary, session.cliEnv);
const keys = await waitForObjectStorageObjects(bucketPrefix);
console.log(
`Uploaded ${localEntry.path} through Journal Sync to ${objectStorage.bucket}/${bucketPrefix} (${keys.length} object(s))`
);
} finally {
if (session) {
await session.app.stop();
}
await vault.dispose();
if (process.env.E2E_OBSIDIAN_KEEP_OBJECT_STORAGE !== "true") {
await deleteObjectStoragePrefix(objectStorage, bucketPrefix).catch((error: unknown) => {
console.warn(error instanceof Error ? error.message : error);
});
}
}
}
main().catch((error: unknown) => {
console.error(error instanceof Error ? error.stack : error);
process.exit(1);
});
+93 -6
View File
@@ -29,8 +29,11 @@ process.env.E2E_OBSIDIAN_COUCHDB_TIMEOUT_MS ??= "20000";
const createPath = "E2E/two-vault/create.md";
const updatePath = "E2E/two-vault/update.md";
const deletePath = "E2E/two-vault/delete.md";
const renameFromPath = "E2E/two-vault/rename-source.md";
const renameToPath = "E2E/two-vault/renamed/rename-target.md";
const conflictPath = "E2E/two-vault/conflict.md";
const targetMismatchPath = "E2E/two-vault/target-mismatch.md";
const encryptedPath = "E2E/two-vault/encrypted.md";
type RunnerContext = {
binary: string;
@@ -134,6 +137,25 @@ async function deleteNoteViaObsidian(cliBinary: string, env: NodeJS.ProcessEnv,
);
}
async function renameNoteViaObsidian(cliBinary: string, env: NodeJS.ProcessEnv, fromPath: string, toPath: string) {
await evalObsidianJson<unknown>(
cliBinary,
[
"(async()=>{",
`const fromPath=${JSON.stringify(fromPath)};`,
`const toPath=${JSON.stringify(toPath)};`,
"const folder=toPath.split('/').slice(0,-1).join('/');",
"if(folder&&!(await app.vault.adapter.exists(folder))) await app.vault.createFolder(folder);",
"const existing=app.vault.getAbstractFileByPath(fromPath);",
"if(!existing) throw new Error(`Could not find note to rename: ${fromPath}`);",
"await app.vault.rename(existing,toPath);",
"return JSON.stringify({ok:true});",
"})()",
].join(""),
env
);
}
async function startConfiguredSession(
context: RunnerContext,
vault: TemporaryVault,
@@ -319,14 +341,62 @@ async function runCreateUpdateDelete(
console.log("Two-vault note creation, update, and deletion round-tripped.");
}
async function runRename(context: RunnerContext, vaultA: TemporaryVault, vaultB: TemporaryVault): Promise<void> {
const renamedContent = "# Rename target\n\nThis note should move from A to B.\n";
let session = await startConfiguredSession(context, vaultA);
await writeNoteViaObsidian(context.cliBinary, session.cliEnv, renameFromPath, renamedContent);
await uploadNote(context, session, renameFromPath);
await renameNoteViaObsidian(context.cliBinary, session.cliEnv, renameFromPath, renameToPath);
await waitForLocalDatabaseEntry(context.cliBinary, session.cliEnv, renameToPath);
await pushLocalChanges(context.cliBinary, session.cliEnv);
await session.app.stop();
session = await startConfiguredSession(context, vaultB);
await syncAndApply(context, session);
const renamedOnB = await waitForPathContent(vaultB.path, renameToPath, (content) => content === renamedContent);
await waitForPathDeleted(vaultB.path, renameFromPath);
await session.app.stop();
assertEqual(renamedOnB, renamedContent, "Renamed note content did not round-trip to the second vault.");
console.log("Two-vault note rename round-tripped.");
}
async function runEncryptedRoundTrip(
context: RunnerContext,
vaultA: TemporaryVault,
vaultB: TemporaryVault
): Promise<void> {
const encryptedContent = "# Encrypted round-trip\n\nThis note should synchronise with E2EE enabled.\n";
const encryptedOverrides = {
encrypt: true,
passphrase: "real-obsidian-e2e-passphrase",
usePathObfuscation: true,
E2EEAlgorithm: "v2",
};
let session = await startConfiguredSession(context, vaultA, encryptedOverrides);
await writeNoteViaObsidian(context.cliBinary, session.cliEnv, encryptedPath, encryptedContent);
await uploadNote(context, session, encryptedPath);
await session.app.stop();
session = await startConfiguredSession(context, vaultB, encryptedOverrides);
await syncAndApply(context, session);
const received = await waitForPathContent(vaultB.path, encryptedPath, (content) => content === encryptedContent);
await session.app.stop();
assertEqual(received, encryptedContent, "Encrypted note did not round-trip to the second vault.");
console.log("Two-vault encrypted note synchronisation round-tripped.");
}
async function runMarkdownAutoMerge(
context: RunnerContext,
vaultA: TemporaryVault,
vaultB: TemporaryVault
): Promise<void> {
const base = "# Conflict\n\nBase line\n\nShared tail\n";
const left = "# Conflict\n\nLeft line\n\nShared tail\n";
const right = "# Conflict\n\nBase line\n\nRight tail\n";
const base = "# Conflict\n\nTop anchor\n\nMiddle anchor\n\nBottom anchor\n";
const left = "# Conflict\n\nTop anchor\n\nLeft line\n\nMiddle anchor\n\nBottom anchor\n";
const right = "# Conflict\n\nTop anchor\n\nMiddle anchor\n\nRight tail\n\nBottom anchor\n";
let session = await startConfiguredSession(context, vaultB);
await createMarkdownConflict(context, session, vaultB, conflictPath, base, left, right);
@@ -335,7 +405,8 @@ async function runMarkdownAutoMerge(
const mergedOnB = await waitForPathContent(
vaultB.path,
conflictPath,
(content) => content.includes("Left line") && content.includes("Right tail")
(content) => content.includes("Left line") && content.includes("Right tail"),
Number(process.env.E2E_OBSIDIAN_MERGE_FILE_TIMEOUT_MS ?? 30000)
);
await session.app.stop();
@@ -344,7 +415,8 @@ async function runMarkdownAutoMerge(
const mergedOnA = await waitForPathContent(
vaultA.path,
conflictPath,
(content) => content.includes("Left line") && content.includes("Right tail")
(content) => content.includes("Left line") && content.includes("Right tail"),
Number(process.env.E2E_OBSIDIAN_MERGE_FILE_TIMEOUT_MS ?? 30000)
);
await session.app.stop();
@@ -405,29 +477,44 @@ async function main(): Promise<void> {
const couchDb = await loadCouchDbConfig();
const dbName = makeUniqueDatabaseName(couchDb.dbPrefix, "two-vault-sync");
const encryptedDbName = makeUniqueDatabaseName(couchDb.dbPrefix, "two-vault-sync-e2ee");
const vaultA = await createTemporaryVault();
const vaultB = await createTemporaryVault();
const encryptedVaultA = await createTemporaryVault();
const encryptedVaultB = await createTemporaryVault();
const context: RunnerContext = { binary, cliBinary: cli.binary, couchDb, dbName };
const encryptedContext: RunnerContext = { binary, cliBinary: cli.binary, couchDb, dbName: encryptedDbName };
try {
await assertCouchDbReachable(couchDb);
await createCouchDbDatabase(couchDb, dbName);
await createCouchDbDatabase(couchDb, encryptedDbName);
console.log(`Using Obsidian executable: ${binary}`);
console.log(`Temporary vault A: ${vaultA.path}`);
console.log(`Temporary vault B: ${vaultB.path}`);
console.log(`Temporary CouchDB database: ${dbName}`);
console.log(`Temporary encrypted CouchDB database: ${encryptedDbName}`);
await runCreateUpdateDelete(context, vaultA, vaultB);
await runMarkdownAutoMerge(context, vaultA, vaultB);
await runRename(context, vaultA, vaultB);
if (process.env.E2E_OBSIDIAN_INCLUDE_MARKDOWN_CONFLICT === "true") {
await runMarkdownAutoMerge(context, vaultA, vaultB);
}
await runTargetMismatch(context, vaultA, vaultB);
await runEncryptedRoundTrip(encryptedContext, encryptedVaultA, encryptedVaultB);
} finally {
await vaultA.dispose();
await vaultB.dispose();
await encryptedVaultA.dispose();
await encryptedVaultB.dispose();
if (process.env.E2E_OBSIDIAN_KEEP_COUCHDB !== "true") {
await deleteCouchDbDatabase(couchDb, dbName).catch((error: unknown) => {
console.warn(error instanceof Error ? error.message : error);
});
await deleteCouchDbDatabase(couchDb, encryptedDbName).catch((error: unknown) => {
console.warn(error instanceof Error ? error.message : error);
});
}
}
}